Termsslug: jailbreaking

Jailbreaking

越狱攻击

Techniques using specific prompts to bypass model safety restrictions and generate prohibited content.

Curated Intelligence

5 items · Page 1/1
Rohan Paul (@rohanpaul_ai)T2·Open Source
68 pts
White Circle Open-Sources RISE: Automated Jailbreak Search for Image Models Using Qwen and Gemini Flash

White Circle has open-sourced RISE, an automated red-teaming tool that searches for attack strategies to bypass safeguards in AI image generators. By leveraging off-the-shelf models like Qwen and Gemini Flash as attacker agents without additional training, it generates prompts to test model robustness, offering defenders an immediate evaluation capability.

MIT Technology ReviewT2·Models
68 pts
The Fragility of AI Refusal: Over-Reliance on Model 'No' Capabilities Raises Safety and Alignment Risks

The article argues that current AI safety strategies over-rely on Large Language Models' ability to refuse harmful requests, a capability that is neither innate nor robust. By reviewing Anthropic's HHH principles and early OpenAI model behaviors, the author demonstrates that training models simply to say 'no' is highly vulnerable to adversarial attacks, potentially leading to critical safety misjudgments.

Hugging Face BlogT1·Models
68 pts
ServiceNow Releases AprielGuard: An 8B LLM Safety Guardrail

ServiceNow AI has released and open-sourced AprielGuard, an 8B parameter safety-security safeguard model designed for modern LLM systems. It detects 16 categories of safety risks (e.g., toxicity, hate speech) and a wide range of adversarial attacks, including prompt injections and jailbreaks, providing a unified risk assessment framework for agentic systems.

OpenAI 官方博客T1·Industry
68 pts
OpenAI and Anthropic Release Joint Safety Evaluation Findings

OpenAI and Anthropic completed the first cross-lab joint safety evaluation, testing each other's models for misalignment, hallucinations, and jailbreaking. This collaboration aims to validate model safety through third-party perspectives, highlighting both the progress and challenges of industry cooperation in AI alignment research.