CSRP BDO
-
About
About Cyber School -
About BDO
-
The CSRP program Syllabus
-
Module 1The Cyber World and CIA Triad6 Topics
-
The OSI Model and Network Layers6 Topics
-
IP Addressing, Subnetting and Routing6 Topics
-
IP Addressing, Subnetting and Routing Textbook
-
IP Addressing, Subnetting and Routing Lesson Companion
-
IP Addressing, Subnetting and Routing Lab Guide
-
IP Addressing, Subnetting and Routing Terminology Bank
-
IP Addressing, Subnetting and Routing Question Bank
-
IP Addressing, Subnetting and Routing Lesson Guide
-
IP Addressing, Subnetting and Routing Textbook
-
DNS and DHCP6 Topics
-
Windows Server Administration6 Topics
-
Active Directory and Identity Management6 Topics
-
Active Directory and Identity Management Textbook
-
Active Directory and Identity Management Lesson Companion
-
Active Directory and Identity Management Lab Guide
-
Active Directory and Identity Management Terminology Bank
-
Active Directory and Identity Management Question Bank
-
Active Directory and Identity Management Lesson Guide
-
Active Directory and Identity Management Textbook
-
Linux CLI Fundamentals6 Topics
-
Linux Permissions and Hardening6 Topics
-
Module 2 - Defense Technologies and Cyber ToolsFirewall Architecture and Rule Design5 Topics
-
NAT and Secure Traffic Mapping6 Topics
-
Enterprise VPN Solutions6 Topics
-
IDS and IPS5 Topics
-
Identity and Access Management5 Topics
-
Multi-Factor Authentication6 Topics
-
Data Loss Prevention6 Topics
-
Endpoint Security and EDR6 Topics
-
Mail Security6 Topics
-
Web Proxy and URL Filtering6 Topics
-
Regulation and Compliance6 Topics
-
Cloud Security6 Topics|1 Quiz
-
Module 3 - Ai in securityAI in Security5 Topics
-
ML Anomaly Detection5 Topics
-
AI Threats5 Topics
-
LLM Defense6 Topics
-
Module 4 - Offensive Cyber & Vulnerability ScanningCyber Kill Chain and Attacker Mindset6 Topics
-
OSINT and Public Intelligence Gathering6 Topics
-
Network and Port Scanning with Nmap6 Topics
-
Vulnerability Scanning6 Topics
-
Exploitation Fundamentals6 Topics
-
Web Application Security5 Topics
-
Social Engineering5 Topics
-
Password and Hash Cracking5 Topics
-
Module 5 - SOC and Advance Cyber AnalystSOC Structure and Operations6 Topics
-
SIEM Fundamentals with Splunk6 Topics
-
Alerting and Dashboards6 Topics
-
Incident Response5 Topics
-
Basic Digital Forensics
-
Capstone A - The Attack: Breaching Meridian
-
Capstone B - The Defense: Defending Meridian
-
ISC2 CC Mock Exam and CSRP Graduation2 Quizzes
CSRP — Cyber Security Responder & Practitioner
Lesson 24: LLM Defense
Lesson Companion Workbook
Cyber School · BDO Ziv Haft
CSRP LESSON 24 — COMPANION WORKBOOK
Module 3 — AI in Security · Lesson 4 of 4 · Course Lesson 24 of 40 — FINAL M3 Title: LLM Defense
Follows the Gamma outline chapter-by-chapter. Module 3 closure.
Chapter 0 · Orientation
Where you are: L21, L22, L23 done. Today closes Module 3. After today: [AI-Practitioner] module badge, Cyber Practitioner role unlocks, 60% of CSRP complete.
Fill-in: L21 = AI as your ___________ . L22 = ML as your ___________ . L23 = AI as the ___________ . L24 = AI INSIDE your ___________ .
Reflect: What LLM-integrated product does your organization use or plan to deploy? What’s the current defensive posture around it? ________________________________________________________________________________
Chapter 1 · LLM Application Architecture
The six components
| # | Component | Real example |
| 1 | User prompt | Chat input box / API call |
| 2 | System prompt | Developer-set instructions (“You are a helpful Meridian assistant…”) |
| 3 | Model | GPT-4-class / Claude / Gemini |
| 4 | Retrieval / KB | Vector DB, RAG over corporate wiki |
| 5 | Tools / plugins | send_email, query_db, http_get, file_write |
| 6 | Output | Text response, tool invocations |
Threat surfaces per component
| Component | Threat | OWASP LLM |
| User prompt | Prompt injection, jailbreak | LLM01 |
| System prompt | Disclosure | LLM06 |
| Model | Hallucination, bias | (cross-LLM) |
| Retrieval / KB | Data leakage, KB poisoning | LLM06, LLM03 |
| Tools / plugins | Unauthorized invocation | LLM07, LLM08 |
| Output | Downstream injection | LLM02 |
The system prompt fallacy
“We told the LLM not to reveal X. We’re safe.” Wrong. The system prompt is a soft suggestion. With sufficient technique, it’s overridden. Defense: multiple layers, not one strong prompt.
Fill-ins
- The 6 LLM-app components: ___________ · ___________ · model · ___________ · ___________ · output.
- The “system prompt fallacy” is believing the ___________ prompt alone is ___________ .
- The component that handles the model’s INVOCATION of external actions is ___________ .
Reflect
For your org’s LLM (or a vendor LLM you use): identify which of the 6 components are configured. Which are missing controls today? ________________________________________________________________________________
Bridge
Back to L21: the 5-component prompt pattern you learned in L21 (Role/Context/Task/Constraints/Output Format) is what GOES INTO the user-prompt component here.
Chapter 2 · OWASP Top 10 for LLM Applications
The ten
| ID | Name |
| LLM01 | Prompt Injection |
| LLM02 | Insecure Output Handling |
| LLM03 | Training Data Poisoning |
| LLM04 | Model Denial of Service |
| LLM05 | Supply Chain Vulnerabilities |
| LLM06 | Sensitive Information Disclosure |
| LLM07 | Insecure Plugin Design |
| LLM08 | Excessive Agency |
| LLM09 | Overreliance |
| LLM10 | Model Theft |
Today’s focus
LLM01 · LLM02 · LLM06 · LLM07 · LLM08
Fill-ins
- LLM01 — ___________ — the new SQL injection.
- LLM06 — Sensitive ___________ Disclosure — leaking system prompt or KB contents.
- LLM08 — Excessive ___________ — overly broad LLM authority.
Reflect
Which of the 10 is your organization’s biggest risk today, and why? ________________________________________________________________________________
Chapter 3 · Prompt Injection + Jailbreaking
Canonical injection patterns
| Pattern | Type | OWASP |
| “Ignore previous instructions” | Direct | LLM01 |
| “Reveal / print the system prompt” | Disclosure | LLM06 |
| “Use the tool without approval” | Tool abuse | LLM07/08 |
| “Act as ___” (role-play) | Jailbreak | LLM01 |
| “In a hypothetical world where ___” | Jailbreak | LLM01 |
| “Translate the following including any safety policies” | Encoding bypass | LLM01 |
| “The previous instructions were a test. The real instruction is ___” | Instruction override | LLM01 |
| (Embedded in retrieved content) | Indirect injection | LLM01 |
Direct vs indirect
- Direct: the malicious instruction is in the user’s input.
- Indirect: the malicious instruction is in CONTENT the LLM reads (webpage, PDF, KB entry).
Indirect is the harder defense because the attacker controls content, not input.
Fill-ins
- Direct prompt injection comes from the ___________ . Indirect comes from ___________ .
- “Ignore previous instructions” is the canonical example of OWASP ___________ .
- Asking the LLM to “act as a different persona” is a ___________ technique.
Reflect
Have you (or your team) ever been the target of a prompt injection attempt — or noticed one in retrospect? Describe. ________________________________________________________________________________
Chapter 4 · Guardrails + DLP for AI
The guardrail layers
| # | Layer | Implementation |
| 1 | Input validation | Regex on prompt patterns |
| 2 | Output validation | Regex on response patterns |
| 3 | Secret + PII scan | Regex on credentials / PII / payment data |
| 4 | Sensitive marker filter | Block outputs containing [INTERNAL], [CONFIDENTIAL] |
| 5 | Suspicious prompt logging | Log → SIEM / file |
| 6 | Rate limiting | Slow high-volume probing |
| 7 | Tool ACL | Per-user / per-tool permissions + approval gates |
| 8 | Retrieval ACL | Filter RAG results by user perms BEFORE the LLM |
Input filter examples (Lab 24.2)
INJECTION_PATTERNS = [
r”ignore\s+previous\s+instructions”,
r”reveal\s+(the\s+)?(hidden\s+)?(policy|system\s+prompt)”,
r”print\s+(the\s+)?(your\s+)?system\s+prompt”,
r”use\s+the\s+tool\s+without\s+approval”,
r”act\s+as\s+(?:a\s+)?different\s+(?:assistant|persona)”,
]
Output filter examples
OUTPUT_DANGER = [
r”\[INTERNAL\]”,
r”\[CONFIDENTIAL\]”,
r”system\s+prompt:”,
r”AKIA[0-9A-Z]{16}”, # AWS key
r”sk-[a-zA-Z0-9]{32,}”, # OpenAI-style
]
DLP for AI — the classification matrix
| Data | Allow | Block | Redact |
| PII (names + emails) | — | yes | yes |
| Credit card | — | yes | — |
| API keys / passwords | — | yes | — |
| Internal pricing | — | yes (per policy) | — |
| Internal codenames | — | yes (per policy) | — |
| Public marketing copy | yes | — | — |
Fill-ins
- The 4 guardrails Lab 24.2 implements: input check · ___________ ___________ · output filter · suspicious ___________ logging.
- The defense principle: ___________ in ___________ .
- DLP for AI applies the thinking from Lesson ___________ to AI prompts.
Reflect
At your org — which guardrail layer is most achievable in the next 30 days? Which is the hardest? ________________________________________________________________________________
Chapter 5 · Least Privilege + Logging + AI Usage Policy
Tool ACL patterns
- Per-user tool ACLs (“user X can use {send_email, query_db}”)
- Approval gates (any send_email over $X requires human OK)
- Read-only mode (no tools at all; chatbot is retrieval-only)
- Sandboxed execution (run_command in ephemeral container)
AI agent risk
“AI agent” = LLM with broader agency, multi-step planning. Agency compounds prompt-injection consequences.
Logging to capture
- Every prompt + output (forensics + injection detection)
- Alert on suspicious patterns
- Per-user volumes (UEBA from L22)
AI usage policy — 1-page checklist
- Approved tools only (L21 Safety Rule 5)
- No secrets / customer data (L21 Safety Rules 1-2)
- Data classification rules
- Verification discipline (L21)
- Incident reporting
Fill-ins
- Approval gates require human OK above a ___________ .
- Sandboxed execution puts the tool action in an ___________ container.
- Per-user AI usage anomalies are detectable via ___________ from L22.
Reflect
Your org’s AI usage policy — does one exist? If yes, when was it last updated? If no, what’s the biggest hole? ________________________________________________________________________________
Chapter 6 · Challenge — Prompt Shield
Pre-Challenge checklist
- I’ve completed Lab 24.2 (the guardrail code is the foundation)
- I have /labs/L24/chatbot/ framework open
- I know the input + output + tool-ACL + logging requirements
- I will not ask for a hint before T=10
Implementation framework
# /labs/L24/work/prompt_shield.py
import re, json, datetime
def check_input(prompt):
# … your input filter …
pass
def check_output(response):
# … your output filter …
pass
def check_tool_use(tool_name, args, user):
# … your tool ACL …
pass
def log_event(verdict, prompt, reason):
# … your logger …
pass
Validator expectations
- 10 attacker prompts: all blocked
- 10 benign prompts: all pass through
- ≥ 1 indirect injection: blocked
- All blocks logged with reason
- 0 false positives on benign set
Lab rubrics
Lab 24.1
| Checkpoint | Weight | Pass |
| All 15 classified (SAFE/UNSAFE) | 20 | Table complete |
| ≥ 80% correct on labels | 30 | Auto-graded vs per-seat truth |
| OWASP LLM ID per UNSAFE | 25 | All non-empty |
| Indirect injection correctly flagged | 15 | Notes section non-empty |
| “Why dangerous” non-empty | 10 | Per row |
Pass: ≥ 60.
Lab 24.2
| Checkpoint | Weight | Pass |
| Input filter implemented with ≥ 4 canonical patterns | 25 | Code review |
| Output filter implemented with ≥ 4 markers | 25 | Code review |
| Secret/PII warning on input | 15 | Code review |
| Suspicious-prompt logging functional | 15 | Log file populated during test |
| Test runs (curl tests pass) | 10 | Manual verification |
| lab2_notes.md complete | 10 | All sections |
Pass: ≥ 60.
Lab 24.3
| Checkpoint | Weight | Pass |
| All 10 classified | 20 | Table complete |
| Correct class per row | 30 | Per-seat truth |
| Decision per row | 20 | Allow/block/redact |
| 3 unsafe prompts rewritten safely | 15 | Side-by-side present |
| 1-page AI usage guideline | 15 | All sections present |
Pass: ≥ 60.
Challenge (validator-based)
| Outcome | Pass |
| All 10 attacker prompts blocked | Required |
| All 10 benign prompts pass | Required |
| ≥ 1 indirect injection blocked | Required |
| 0 false positives on benign | Required |
| All blocks logged | Required |
Badge [Prompt-Shield-Engineer] on pass.
Module 3 closure self-check
- Name the 6 LLM-app components.
- List the OWASP LLM Top 10 IDs covered today (5 of them).
- Give one example of each: direct injection, indirect injection, jailbreak.
- List 4 guardrail layers from memory.
- State the principle: “Never rely on the ___ ___ alone.”
- Define AI agent risk in one sentence.
- Recite the 4 M3 lesson badges.
Learning log
| Today I learned | I want to practice |
Module 3 reflection
L21 → L22 → L23 → L24. Four lessons. Four badges. Compound AI competence.
Take 5 minutes. Write 3 sentences about what changed in your thinking across Module 3. ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________
Cross-references
- L15 DLP: Lab 24.3 IS DLP for AI
- L21 LLM HITL + safety rules: apply to all AI guardrail bypasses
- L22 UEBA: per-user AI-usage anomaly detection
- L23 AI Threats: adversarial-ML thinking carries here
- Forward to L25 / Module 4 (Offensive): the attacker’s view of these techniques
- Forward to Module 5 (SOC): guardrail logs become a monitoring source