Automated Security Auditing & Vulnerability Discovery

During evaluation, Opus 4.6 autonomously discovered over 500 zero-day vulnerabilities in real open source projects. These were not toy examples — they were genuine, previously unknown security flaws that required coordinated disclosure. This lesson teaches you to apply that same capability to your own codebase.

Why LLM-Based Security Auditing Works

Traditional static analysis tools (SAST) match patterns. They find known vulnerability signatures — SQL injection via string concatenation, XSS via unsanitized output. They miss vulnerabilities that require understanding program semantics.

Opus 4.6 understands:

• Data flow across function boundaries, modules, and async operations
• Temporal ordering — what happens before and after a security check
• Business logic — whether an authorization check is correct for the domain
• Implicit assumptions — what the developer assumed but did not enforce

graph TD
    A[Traditional SAST] --> B[Pattern Matching]
    B --> C[Known vulnerability signatures]
    C --> D[High false positive rate]
    
    E[Opus 4.6 Audit] --> F[Semantic Understanding]
    F --> G[Cross-function data flow analysis]
    F --> H[Temporal reasoning]
    F --> I[Business logic validation]
    G --> J[Novel vulnerability classes]
    H --> J
    I --> J

Vulnerability Classes Opus 4.6 Excels At

TOCTOU (Time-of-Check to Time-of-Use)

TOCTOU bugs occur when a security check and the operation it guards are not atomic. The state can change between the check and the use:

# TOCTOU vulnerability — Opus 4.6 catches this
def delete_file(user, file_path):
    # CHECK: Does user own this file?
    if file_belongs_to_user(user, file_path):  # Time of Check
        time.sleep(0.01)  # Simulating real-world delay
        os.remove(file_path)  # Time of Use — file_path could be different now!

A race condition between the ownership check and the deletion allows an attacker to swap the file via a symlink after the check passes.

claude "Scan the codebase for TOCTOU vulnerabilities. Look for patterns where:
  1. A permission/ownership check is followed by a file system operation
  2. A database read for authorization is followed by a write operation
  3. Any check-then-act pattern that is not atomic
  
  For each finding, explain the race window and provide a fix."

Confused Deputy Attacks

A confused deputy vulnerability occurs when a privileged service performs an action on behalf of a less-privileged caller without properly validating that the caller has the right to request that action:

// Confused deputy — the API endpoint has admin DB access
// but does not verify the caller's authorization for THIS resource
app.get('/api/documents/:id', async (req, res) => {
  const doc = await db.documents.findById(req.params.id);
  // Missing: does req.user have access to THIS document?
  res.json(doc);
});

claude "Audit all API endpoints for confused deputy vulnerabilities.
  
  For each endpoint, verify:
  1. Authentication is enforced (who is the caller?)
  2. Authorization is checked for the SPECIFIC resource (can they access THIS item?)
  3. The authorization check cannot be bypassed via parameter manipulation
  
  Flag any endpoint where resource-level authorization is missing."

Injection Points

Beyond SQL injection, Opus 4.6 identifies injection in templates, shell commands, LDAP queries, NoSQL queries, and header injection:

claude "Perform a comprehensive injection audit:
  
  1. SQL injection — including ORM bypass patterns
  2. NoSQL injection — especially MongoDB query operator injection
  3. Command injection — any user input reaching shell execution
  4. Template injection — server-side template engines
  5. Header injection — CRLF injection in HTTP headers
  6. Path traversal — user input in file system paths
  
  For each category, trace user input from entry point (request handler) 
  to dangerous sink (query execution, file operation, shell command).
  Show the complete data flow path."

Race Conditions

Race conditions in web applications are more common than most teams realize:

// Race condition in balance check
async function processPayment(userId: string, amount: number) {
  const user = await db.users.findById(userId);
  
  if (user.balance >= amount) {        // Read balance
    // Another request can modify balance here!
    await db.users.update(userId, {    // Write balance
      balance: user.balance - amount
    });
    return { success: true };
  }
  return { success: false, reason: 'insufficient_funds' };
}

claude "Find race conditions in the codebase. Focus on:
  1. Read-then-write patterns without database-level locking
  2. Check-then-act patterns in concurrent request handlers
  3. Shared mutable state accessed from async operations
  4. Counter/sequence generation without atomic operations
  
  For each race condition, provide:
  - The exact race window (what can happen between which operations)
  - The attack scenario (how an attacker exploits it)
  - The fix (pessimistic locking, optimistic locking, or atomic operations)"

Building a Reusable Security Audit Pipeline

Instead of running ad-hoc security scans, build a systematic pipeline that runs on every code change.

The Audit Script

#!/usr/bin/env python3
"""Automated security audit pipeline using Opus 4.6."""
import json
import os
import subprocess
from anthropic import Anthropic
from pathlib import Path

VULNERABILITY_CATEGORIES = [
    {
        "name": "injection",
        "prompt": """Audit for injection vulnerabilities. Trace user input from 
        HTTP request handlers to dangerous sinks: SQL queries, shell commands, 
        file paths, template engines, LDAP queries. Report complete data flow.""",
    },
    {
        "name": "authentication",
        "prompt": """Audit authentication implementation. Check for: 
        timing-safe comparison on secrets, proper session invalidation, 
        JWT validation completeness (alg, exp, iss, aud), password hashing 
        algorithm strength, brute force protection.""",
    },
    {
        "name": "authorization",
        "prompt": """Audit authorization logic. Check for: 
        missing resource-level access checks (confused deputy), 
        privilege escalation paths, IDOR vulnerabilities, 
        role hierarchy bypass, horizontal access control gaps.""",
    },
    {
        "name": "race_conditions",
        "prompt": """Audit for race conditions. Check for: 
        TOCTOU bugs, non-atomic read-modify-write on shared state, 
        double-spend vulnerabilities, concurrent request handling 
        without proper locking.""",
    },
    {
        "name": "data_exposure",
        "prompt": """Audit for data exposure. Check for: 
        sensitive data in logs, error messages leaking internals, 
        API responses including fields the caller should not see, 
        credentials in configuration files, PII in URLs.""",
    },
]

def load_codebase_context(src_dir: str, max_tokens: int = 400000) -> str:
    """Load source files into a single context string."""
    context_parts = []
    extensions = {'.ts', '.tsx', '.js', '.py', '.go', '.java', '.rs'}
    
    for path in sorted(Path(src_dir).rglob('*')):
        if path.suffix in extensions and 'node_modules' not in str(path):
            try:
                content = path.read_text(encoding='utf-8')
                context_parts.append(f"--- FILE: {path} ---\n{content}\n")
            except (UnicodeDecodeError, PermissionError):
                continue
    
    return '\n'.join(context_parts)

def run_audit(client: Anthropic, codebase: str, category: dict) -> dict:
    """Run a single audit category."""
    response = client.messages.create(
        model="claude-opus-4-6-20260205",
        max_tokens=8192,
        thinking={"type": "adaptive", "budget_tokens": 15000},
        system="""You are a senior security researcher performing a code audit.
        Report ONLY confirmed vulnerabilities with complete evidence.
        Do not report style issues, theoretical risks, or best practice violations.
        Each finding must include: file, line range, vulnerability type, 
        severity (critical/high/medium/low), exploit scenario, and fix.""",
        messages=[{
            "role": "user",
            "content": f"{category['prompt']}\n\n"
                       f"Respond in JSON: {{\"findings\": [{{\"file\": str, "
                       f"\"lines\": str, \"type\": str, \"severity\": str, "
                       f"\"description\": str, \"exploit\": str, \"fix\": str}}]}}\n\n"
                       f"CODEBASE:\n{codebase}"
        }]
    )
    
    text = response.content[-1].text
    return json.loads(text)

def main():
    client = Anthropic()
    codebase = load_codebase_context("src/")
    
    all_findings = []
    for category in VULNERABILITY_CATEGORIES:
        print(f"Auditing: {category['name']}...")
        result = run_audit(client, codebase, category)
        for finding in result.get("findings", []):
            finding["category"] = category["name"]
            all_findings.append(finding)
    
    # Sort by severity
    severity_order = {"critical": 0, "high": 1, "medium": 2, "low": 3}
    all_findings.sort(key=lambda f: severity_order.get(f["severity"], 4))
    
    # Output report
    report = {"total": len(all_findings), "findings": all_findings}
    Path("security-audit-report.json").write_text(
        json.dumps(report, indent=2)
    )
    
    critical = sum(1 for f in all_findings if f["severity"] == "critical")
    high = sum(1 for f in all_findings if f["severity"] == "high")
    print(f"\nAudit complete: {len(all_findings)} findings")
    print(f"  🔴 Critical: {critical}")
    print(f"  🟠 High: {high}")
    
    if critical > 0:
        print("\n⛔ CRITICAL vulnerabilities found. Do not deploy.")
        exit(1)

if __name__ == "__main__":
    main()

Pipeline Architecture

graph TD
    A[Code Push / PR] --> B[CI Trigger]
    B --> C[Load Changed Files + Dependencies]
    C --> D[Injection Audit]
    C --> E[Auth Audit]
    C --> F[Race Condition Audit]
    C --> G[Data Exposure Audit]
    D --> H[Aggregate Findings]
    E --> H
    F --> H
    G --> H
    H --> I{Critical Findings?}
    I -->|Yes| J[Block Merge + Alert Security Team]
    I -->|No| K{High Findings?}
    K -->|Yes| L[Request Review + Add PR Comment]
    K -->|No| M[Pass — Merge Allowed]

Optimizing Audit Accuracy

Reducing False Positives

The most important metric for a security audit tool is signal-to-noise ratio. A tool that produces 50 findings with 45 false positives is worse than useless.

system_prompt = """You are a senior security researcher. Your reputation 
depends on accuracy.

Rules:
1. Only report vulnerabilities you can demonstrate with a concrete exploit
2. If a framework or library already mitigates a risk, do not report it
3. Check for existing sanitization/validation before reporting injection
4. Verify that authentication middleware is NOT already applied via 
   route-level or global configuration before reporting missing auth
5. When uncertain, state your confidence level — do not omit the finding, 
   but mark it as "needs manual verification"
"""

Differential Auditing

For CI/CD integration, audit only changed code plus its dependency context:

def get_changed_files() -> list[str]:
    """Get files changed in the current PR."""
    result = subprocess.run(
        ["git", "diff", "--name-only", "origin/main...HEAD"],
        capture_output=True, text=True
    )
    return [f for f in result.stdout.strip().split('\n') if f]

def get_dependency_context(changed_files: list[str]) -> list[str]:
    """Find files that import or are imported by changed files."""
    all_files = set(changed_files)
    
    for filepath in changed_files:
        # Find files that import this file
        result = subprocess.run(
            ["grep", "-rl", f"from.*{Path(filepath).stem}", "src/"],
            capture_output=True, text=True
        )
        importers = result.stdout.strip().split('\n')
        all_files.update(f for f in importers if f)
    
    return list(all_files)

Real-World Vulnerability Examples

These patterns are based on the types of vulnerabilities Opus 4.6 has been demonstrated to find:

Example 1: Prototype Pollution via Deep Merge

// Vulnerable deep merge — found by Opus 4.6
function deepMerge(target, source) {
  for (const key in source) {
    if (typeof source[key] === 'object' && source[key] !== null) {
      target[key] = target[key] || {};
      deepMerge(target[key], source[key]);  // No __proto__ check!
    } else {
      target[key] = source[key];
    }
  }
  return target;
}

// Exploit: POST {"__proto__": {"isAdmin": true}}
// Now every object in the application has isAdmin === true

Example 2: JWT Algorithm Confusion

# Vulnerable JWT validation — found by Opus 4.6
def verify_token(token):
    # Accepts whatever algorithm the token specifies!
    payload = jwt.decode(token, PUBLIC_KEY, algorithms=["RS256", "HS256"])
    return payload

# Exploit: Attacker creates token with alg=HS256, 
# signs with the PUBLIC key (which is public!).
# Server verifies HS256 signature using PUBLIC_KEY as the HMAC secret.

Example 3: Mass Assignment

// Vulnerable user update — found by Opus 4.6  
app.put('/api/users/:id', auth, async (req, res) => {
  await db.users.update(req.params.id, req.body);  // Passes ALL fields!
  res.json({ success: true });
});

// Exploit: PUT /api/users/123 {"role": "admin", "name": "hacker"}
// Attacker escalates to admin by including 'role' in request body

Integrating with Existing Security Tools

Opus 4.6 does not replace your existing security toolchain — it complements it:

Layer	Tool	What It Catches
SAST	Semgrep, CodeQL	Known patterns, fast, low false positives
SCA	Snyk, Dependabot	Vulnerable dependencies
DAST	OWASP ZAP, Burp Suite	Runtime vulnerabilities in deployed apps
LLM Audit	Opus 4.6	Novel logic bugs, business logic flaws, complex race conditions
Pentest	Human security researchers	Everything above + social engineering, physical

The LLM audit layer catches what falls between the cracks of pattern-matching tools and full penetration tests.

In the next lesson, you will build a complete CI/CD pipeline that runs these security audits automatically on every pull request.