← Home
Cognitive Revolution · January 25, 2025 · 75m

Red-Teaming AI: Inside Model Evaluation

Nathan Labenz
SpotifyApple Podcasts

Inside the process of evaluating AI model safety through red-teaming. How labs test for dangerous capabilities, what the evaluations reveal, and why the process is inherently incomplete.

Canon

•
Red-teaming reveals the true self of AI systems
Red-teaming strips away the fine-tuned politeness of AI models to reveal their underlying capabilities — the true self beneath the safety training.
ClaudeChatGPTGemini
•
The Stoic evaluator — testing what you can control in an uncertain domain
AI safety evaluators can control the rigor of their testing methodology but not the completeness of their coverage. The Stoic approach focuses on process quality rather than guaranteeing safety.
ClaudeChatGPTGemini