CD ai-pm-workbench-international

| 🛡️ AI safety and compliance risks are significant | **EU AI Act · China Deep Synthesis Regulations · Layered Safety Guardrails · Red Teaming** |

| 9️⃣ Safety Guardrails & Red Team | Layered guardrails · Red teaming · Jailbreak prevention | AI Safety Plan |

| AI Safety | Input Guardrails · Output Guardrails · PII Redaction · Jailbreak Detection · Red Teaming |

| **Safety** | Harmful content output rate | Red team testing + automated scanning | <0.1% |

### Phase 10: Safety Guardrails & Red Team Testing

└── System Layer: Agent tool abuse, privilege escalation, resource exhaustion

#### 10.3 Red Team Testing

**Lessons from Microsoft's Red Teaming of 100+ Products:**

- Injection Detection: LLM classifier detects patterns like "ignore previous instructions"

| Red Team Testing | Quarterly | Security team triggers |

| Scale | +LLM injection detection, Hallucination detection | +Red Team testing | +Model safety assessment |

| Adversarial Test Set | | Red team + security experts | Monthly |

| Red Team Test | Quarterly | Red Team Report |

| Adversarial Tasks | | Abnormal inputs | Red team constructed |

| Safety | Harmful Output Rate | Red Team Testing | <0.1% |

| Role-play Jailbreak | Detect "You are now..." type instructions | "DAN mode", "Ignore previous instructions" |

| Privilege Escalation | Agent gains permissions beyond design | Least privilege principle + permission audit |

## 7. Red Team Testing

| Agent Attack | Tool Abuse/Privilege Escalation | 50+ | Each Agent update |

### 7.3 Red Team Testing Process

□ Red Team Testing: Completed with no P0/P1 issues

| Security/Compliance | | | Guardrails + Red Team Testing + Compliance Review |

| Implement safety guardrails / Red teaming | [Phase 10: Safety Guardrails & Red Teaming](#phas…ing) |

| **Fairness & Safety** | Bias detection, harmful content filtering rate | Bias audit, Red teaming |

### Phase 10: Safety Guardrails & Red Teaming

□ Red team testing completed and high-risk items fixed

□ AI Security Basics (Injection / Jailbreak / Guardrails / Red Team Testing)