Model Card — AegisGate Threat Detector v11b

This model card follows the ML Model Card framework (Mitchell et al., 2019) and provides transparency into the AegisGate threat detection model.

Model Details

Intended Use

• Primary use: Detect adversarial AI threats mapped to the MITRE ATLAS framework in HTTP API requests, MCP tool calls, A2A inter-agent communication, and AI responses
• Primary users: Security engineers deploying AI systems in production
• Out of scope: Not a general-purpose text classifier. Not for content moderation outside the AI threat domain. Not a replacement for human security review.

Training Data

• Split: 80% train (56,457) / 10% val (7,057) / 10% test (7,058)
• Data collection: All synthetic or publicly available datasets. No real user data. Seeds derived from publicly documented ATLAS techniques.
• Augmentation: 52 ATLAS payload seeds × 50 transforms (character substitution, encoding, linguistic, whitespace, fragmentation)
• Preprocessing: Character-level normalization to Latin-1 (PAD=0, UNK=1)

Evaluation Metrics

Test Set Results (held-out 10%)

Confusion Matrix: TP=1,492, FP=35, FN=87, TN=5,444

Evasion Suite Results

Model Integrity

The platform verifies the model hash at load time. If the hash does not match, the model is rejected and the system falls back to heuristic detection.

Limitations

1. Character-level model: Cannot detect semantic-level attacks that don’t manifest as character patterns
2. Latin-1 vocabulary: Non-Latin scripts (CJK, Arabic, Cyrillic) map to UNK — handled by the Unicode homoglyph detector, not the model
3. Synthetic training data: Real-world performance may differ; shadow mode is recommended for the first 7 days
4. Heuristic fallback: Covers obfuscated/l33tspeak variants the model may miss — the layered architecture (L1 regex + L2 ATLAS/compliance + L3 ML) provides defense in depth

Ethical Considerations

• No user profiling: The model detects attack patterns, not attackers. It should not be used for user profiling or discrimination
• False positive management: 0.64% FPR on test set. CalibrationManager provides zero-FPR threshold tuning. Shadow mode provides a safety net
• No real user data: Training uses only synthetic and public datasets
• Transparency: This model card, the architecture, training methodology, and evaluation metrics are published openly

Deployment Configuration

# aegisgate-platform.yaml
ml_threat_detection_enabled: true
ml_shadow_mode: true        # Log-only for first 7 days
ml_threshold: 0.50          # Calibrated for 0% FPR on benign corpus
ml_model_path: "pkg/ml/models/threat_cnn_bilstm.onnx"
ml_model_hash: "8e13c793c32816aa0f6e2af13ffadd4f38f707b4ac8906b56ddfa77da51ea8e5"

Graceful Degradation

When ML is disabled or ONNX runtime is unavailable:

• Heuristic detection maintains coverage with 0% FPR
• Regex patterns (L1) and ATLAS/compliance checks (L2) remain active
• The system never fails open — if the ML model cannot load, the system continues protecting with L1+L2

A/B Testing

AegisGate includes a built-in A/B testing framework for comparing model variants:

# Create test
curl -X POST -H "X-API-Key: your-key" -d '{
  "name": "v11b-vs-v12",
  "variants": [
    {"name": "champion", "weight": 90, "model_ref": "model-v11b"},
    {"name": "challenger", "weight": 10, "model_ref": "model-v12"}
  ]
}' https://aegisgate.yourcompany.com/api/v1/abtest/tests

# Start, monitor metrics, promote winner

Version History

Citation

AegisGate Platform v4.4.1
Char CNN-BiLSTM with Attention — Threat Detection Model v11b
Apache License 2.0
https://github.com/aegisgatesecurity/aegisgate-platform