What is AI red teaming and how does it work?

Оригинальный автор：Red Team Models

Первоисточник：https://www.youtube.com/embed/pUuzkubNSCw

#AIsecurity #redteam #LLM AI red teaming attacks a deployed model, its prompts, tools and data the way an adversary would. Microsoft has run 80+ such operations since 2021. We walk one real attack class end to end: an indirect prompt injection hidden in an email that makes an inbox agent forward ten confidential threads on a single 'summarize my inbox' request. Then the numbers from two 2025 competitions in which every frontier model tested broke, and what a first red-team operation actually needs. In this video you will learn: - Why red teaming is not safety benchmarking (Microsoft's lesson 3 from 80+ operations) - The five-part threat model: actor, attack, weakness, impact, system - How an indirect prompt injection hijacks an email agent without any model access - Why one token stream makes prompt injection (OWASP LLM01) land on every model - UK AISI x Gray Swan and NIST CAISI results: 22 of 22 and 13 of 13 models broken - Break-fix layers, PyRIT automation, and the four things a first operation needs For security engineers, AI product teams and anyone shipping an LLM agent with tools. Sources: Microsoft AI Red Team, Lessons from red teaming 100 generative AI products (2025); UK AI Security Institute and Gray Swan agent red-teaming challenge (2025); NIST CAISI agent hijacking evaluation (2025); OWASP Top 10 for LLM Applications; NIST AI 600-1 Generative AI Profile; EU AI Act Article 55. Capsule: We attack the model so you don't ship the hole. Subscribe to Red Team Models, and send the one prompt injection you think no model can catch. #AIsecurity #redteam #LLM #promptinjection #adversarialML #AIredteaming #LLMsecurity #agentsecurity Chapters: 00:00 - What AI red teaming is 00:53 - Red teaming vs benchmarks 01:54 - The five-part threat model 02:58 - One prompt injection, one break 04:07 - Why it works on every model 05:25 - Every model broke: the numbers 06:33 - The fix is a break-fix loop 07:50 - Your first red-team operation