AI Security Engineering · Module 00
Instructor Guide
Teaching intent: Build a precise security-practitioner mental model. Learners do not need to train a large model or derive Transformer mathematics. They must accurately trace data, parameters, context, authority, versions, and metrics.
Mandatory terminology rules
- A model is one component of an AI system.
- Inference and ordinary RAG do not update model parameters.
- Prompt engineering is not authorization.
- Attention is not a factuality mechanism.
- Model-selected is not application-authorized.
- Safety, security, privacy, robustness, fairness, and reliability are not synonyms.
- Every metric requires a task, dataset, threshold or aggregation, and limitation.
- Qualify overloaded terms using the canonical glossary.
Run of show
| Block | Teaching move | Exit evidence |
|---|---|---|
| AI map and tasks | Build the hierarchy and classify familiar systems | Model versus system distinction |
| Learning and data | Trace a phishing classifier through splits and metrics | No leakage; task and costs defined |
| Neural networks | Walk forward pass, loss, gradient, optimizer | Parameters versus hyperparameters |
| Transformer and LLM | Tokenize examples and trace next-token generation | Tokens, attention, logits, decoding understood |
| LLM lifecycle | Compare prompting, RAG, PEFT, SFT, DPO, quantization | Learner selects correct adaptation method |
| RAG | Draw ingestion and query paths separately | Authorization before prompting; lineage preserved |
| Agents and MCP | Separate host, client, server, tool, resource, prompt | Authority enforced outside model |
| Evaluation and MLOps | Challenge a context-free “95% accuracy” claim | Decision-relevant metrics and versions |
| Practical | Trace baseline, RAG, and MCP-agent architectures | Complete data/identity/telemetry trace |
| Check | Use the knowledge questions | Pass or remediation |
Common misconceptions and corrections
| Misconception | Correction |
|---|---|
| “The model learned the retrieved document.” | It received retrieved chunks in inference context; weights were not updated. |
| “Embeddings are encrypted text.” | They are learned vectors; they are not encryption and may leak sensitive information. |
| “Temperature zero is deterministic.” | It narrows decoding randomness, but runtimes and APIs may still vary. |
| “The system prompt has highest security privilege.” | It is model context, not a deterministic enforcement boundary. |
| “MCP authorizes tools.” | MCP defines interaction; applications and deployments enforce authentication, authorization, consent, and isolation. |
| “A high accuracy score means the agent is safe.” | Accuracy covers a defined prediction task and says nothing by itself about agent authority, security, or harm. |
Assessment
| Criterion | Weight |
|---|---|
| Accurate concept relationships and terminology | 25% |
| ML and LLM lifecycle trace | 20% |
| RAG architecture and retrieval controls | 20% |
| Agent/MCP identity and authority trace | 20% |
| Metric selection and limitations | 15% |
Pass standard: 70% overall and at least 8/10 on the knowledge check. Require revision if the learner conflates training with inference, RAG with fine-tuning, model with system, or tool selection with authorization.
Chapter 3 evidence-sheet rubric
| Criterion | Points |
|---|---|
| System graph distinguishes model, parser or feature step, configuration, policy, and downstream action | 2 |
| Artifact and event evidence is versioned and sufficient to reproduce the comparison | 2 |
| Hypothesis names competing explanations and one observable that distinguishes them | 2 |
| Access level, perturbation budget, query budget, and evidence-ladder labels are explicit | 2 |
| Control, telemetry, bypass condition, and residual unknown are documented | 2 |
Chapter 3 pass standard: 8/10 with 2/2 on access and evidence. Require revision if a score is treated as authorization, a research demonstration is labeled as a local incident, or the affected artifact or route version is missing.
Chapter 4 LLM request-trace rubric
| Criterion | Points | Full-credit standard |
|---|---|---|
| Scope and safe execution | 10 | Names the authorized system, uses a harmless marker, and excludes secrets and third-party data. |
| Request-path accuracy | 20 | Separates identity, context, template, tokenizer, model, decoding, parser, policy, approval, and action. |
| Reproducibility record | 20 | Preserves sufficient versions, digests, settings, provenance, and outputs to repeat the test. |
| Evidence reasoning | 20 | Separates observed facts, reproduced behavior, inference, alternatives, and unknowns. |
| Control and detection design | 20 | Proposes an independently enforced control and telemetry-backed detection hypothesis with limitations. |
| Communication | 10 | Uses bounded language and does not equate model output with authorization, execution, or verified impact. |
Chapter 4 pass standard: 70/100 with at least half credit in every criterion. Require revision for unauthorized data, unsupported impact claims, missing effective template or generation settings, or an ATLAS label presented as incident evidence.
Preparation
- Review the glossary and current MCP architecture documentation.
- Prepare one tokenizer demonstration without sensitive data.
- Choose a simple classifier example relevant to the audience.
- Prepare baseline LLM, RAG, and agent architecture diagrams.
- Do not require external model accounts; conceptual traces are sufficient.
- Record terminology changes between course releases.