OpenAI Proposes Instruction Hierarchy to Defend Against Prompt Injection
Original title: The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
68AI Score
Addressing LLM vulnerabilities to prompt injection and jailbreaks, this research introduces the 'Instruction Hierarchy' concept. By training models to prioritize privileged instructions (e.g., system prompts) over user inputs, it effectively defends against malicious attempts to overwrite original directives.
Related Entities
Full Report
Today's LLMs are susceptible to prompt injections, jailbreaks, and other attacks that allow adversaries to overwrite a model's original instructions with their own malicious prompts.