The LLM & RAG Threat Model: OWASP LLM Top 10

Why RAG Expands the Attack Surface

A plain LLM call has a small attack surface: a prompt goes in, text comes out. A RAG system is a different animal. It stitches together your private data store, a retrieval layer, an LLM, and — increasingly — tools that take actions. Each seam where one component hands data to another is a new trust boundary, and security lives or dies at those boundaries.

The Four New Trust Boundaries

1. Ingestion — you pull documents from wikis, tickets, PDFs, uploads. Some of that content was written by people you don't control.
2. Retrieval — at query time the system pulls chunks back out and places them into the prompt. The model now reads text it didn't author and can't distinguish from your instructions.
3. Generation — the model's output may be rendered as HTML, run as a tool call, or stored — so an unsafe output becomes an unsafe action.
4. Tools / actions — an agentic RAG system can send email, run queries, or call APIs. The blast radius of a bad decision is no longer "a wrong sentence."

The Regulated-Industries Stake

For a consumer chatbot, a bad answer is embarrassing. For a RAG system over patient records, financial data, or legal files, the failure modes are compliance incidents:

• A retrieved chunk leaks one patient's PHI into another patient's answer → a HIPAA breach.
• The model invents a confident, wrong figure in a financial summary → a disclosure problem.
• A user with no entitlement retrieves a document they shouldn't see → an access-control violation.

The mission of this track is the site's mission narrowed to one lens: production AI infrastructure for engineers shipping RAG into regulated industries. Security isn't a feature you bolt on at the end — it's the property that makes the system shippable at all.

Security Is a Property of the Whole Pipeline

You can't secure a RAG system by hardening one component. A perfectly locked-down vector store doesn't help if the prompt-assembly step trusts a malicious retrieved chunk. This week works boundary by boundary: the threat model (today), input defenses (Day 2), data minimization (Day 3), the data layer (Day 4), and provenance/audit (Day 5).