AI Security Engineering · Module 01
Learner Workbook
Module: AI Security Threat Landscape
Learner: __________________________________ Date: __________________
A. Decision and scope
Organizational context:
Consumer and decision:
System, time horizon, and key assumptions:
B. Six-layer attack-surface map
| Layer | Assets | Trust boundary / identity | Attacker goal | Required evidence |
|---|---|---|---|---|
| Data | ||||
| Model | ||||
| Application / RAG | ||||
| Agent / tools / MCP | ||||
| Identity / tenancy | ||||
| Platform / MLOps |
Component with greatest authority:
C. Real-case classification
| Case | On / through / around | Layer | Evidence status | Impact | Evidence does not prove | |
|---|---|---|---|---|---|---|
| ShadowRay | ||||||
| Malicious models | ||||||
| EchoLeak | ||||||
| MCP tool poisoning | ||||||
| Provider misuse reports | ||||||
| DeepSeek exposure |
C.1 Intelligence requirement and source record
Consumer:
Decision:
System boundary and evidence threshold:
| Source | First-hand / reproduced / secondary | Version or configuration | Direct facts | Visibility limit | Corroboration |
|---|---|---|---|---|---|
C.2 Claim ledger
Label each statement as a fact, judgment, assumption, gap, or recommendation.
| Claim | Type | Source / reasoning | Confidence | What would change it? | Action |
|---|---|---|---|---|---|
D. Case reconstruction — repeat for three cases
Case and primary source:
Evidence status:
Directly established facts:
Attacker prerequisite or exposure condition:
Entry point:
Affected asset:
Trust-boundary failure:
Observed or demonstrated action:
Impact:
Known mitigation:
Known exploitation status:
Important limitation:
Organizational relevance:
Information that would change confidence:
E. Provider CTI comparison
| Question | OpenAI | Anthropic | |
|---|---|---|---|
| What could the provider observe? | |||
| Which actor tasks used AI? | |||
| Was capability uplift measured? | |||
| What conventional behavior remained? | |||
| What was disrupted? | |||
| What can the report not establish? |
F. Procedure-first framework mapping
| Factual procedure | Source and status | NIST AI 100-2 | ATLAS / ATT&CK | OWASP |
|---|---|---|---|---|
G. Controls and telemetry
| Threat path | Prevent | Constrain | Detect | Preserve | Owner |
|---|---|---|---|---|---|
G.1 Collection plan
Turn an information gap into a collection requirement that answers a decision question.
| Question | Observable source | Required fields | Owner | Decision informed |
|---|---|---|---|---|
| Can an untrusted actor reach the AI control plane? | ||||
| Which content entered model context? | ||||
| Which tool definition and authority were used? | ||||
| Which model artifact entered the environment? |
H. Technical assessment — maximum 700 words
TITLE:
ORGANIZATIONAL CONTEXT AND DECISION:
KEY JUDGMENTS:
1.
2.
3.
EVIDENCE BASE AND CONFIDENCE:
AI-SYSTEM EXPOSURE:
PRIORITY CONTROLS:
1.
2.
3.
TELEMETRY REQUIREMENTS:
1.
2.
3.
IMPORTANT EVIDENCE LIMITATIONS:
LATER MODULES REQUIRED FOR VALIDATION:
I. Technical briefing
- Which case is most relevant and why?
- What occurred or was demonstrated?
- Which trust boundary failed?
- Which immediate control reduces risk?
- What telemetry is missing?
- Which tempting claim is unsupported?
J. Submission checklist
□ System map □ Six-case matrix □ Three reconstructions □ Framework mapping
□ Control/telemetry matrix □ 700-word assessment □ Briefing □ Sources and limitations