LLM Security — Prompt Injection, Jailbreaks and Defenses

Deploying an LLM to production? You’ve just opened a new attack surface. Prompt injection is the SQL injection of the AI era. And most companies aren’t prepared for it.

Prompt Injection¶

An attacker embeds instructions in the input that override the system prompt. Indirect prompt injection is more insidious: malicious instructions hidden in documents that the model processes via RAG.

Jailbreaking¶

DAN, roleplay attacks, encoding tricks — attackers are creative. The model starts generating content it would normally refuse.

Defense Strategies¶

• Input sanitization: Filter known attack patterns
• Privilege separation: The LLM must not have access to everything — least privilege
• Output validation: Check what the model returns — PII, system prompt leaks
• Guardrails: NVIDIA NeMo Guardrails, Guardrails AI frameworks
• Red teaming: Regularly test your own system

OWASP Top 10 for LLMs¶

OWASP released a Top 10 for LLM applications. Number one: prompt injection. We recommend studying it as the foundation for your security review.

LLM Security Is Day Zero¶

Defense against prompt injection is not a solved problem. Layered defense, monitoring, and rapid incident response are critical.