Security Integration Tests Agent

Overview

pkg/security/

Key Files

• pkg/security/testdata/samples.json

  - Test data with benign and malicious samples

• pkg/security/hf/integration_test.go

  - HF client integration tests

• pkg/security/integration_test.go

  - Full analyzer integration tests with event bus

• pkg/security/hf/client.go

  - HuggingFace API client implementation

• pkg/security/analyzer.go

  - Main security analyzer that subscribes to MCP events

Running Tests

# Run integration tests (requires HF_TOKEN environment variable)
HF_TOKEN=hf_xxx make test-integration

# Run with custom model
HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration

# Run unit tests only (no API calls, uses mock server)
go test -v ./pkg/security/...

Test Structure

Test Samples (

samples.json

)

{
  "benign_samples": [
    {
      "description": "Description of the sample",
      "text": "The text to analyze",
      "expected_detected": false
    }
  ],
  "malicious_samples": [
    {
      "description": "Direct instruction override",
      "text": "Ignore all previous instructions...",
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ],
  "mcp_tool_calls": [
    {
      "description": "Malicious tool call",
      "method": "tools/call",
      "params": { "name": "run_command", "arguments": {...} },
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ]
}

Integration Test Tags

//go:build integration

go test ./...

Adding New Test Samples

1. Edit

   pkg/security/testdata/samples.json

2. Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)
3. Set

   expected_detected

   and optionally

   min_risk_score

4. Run integration tests to verify

Common Issues

"Forbidden" Error

• Ensure HF_TOKEN is set and valid
• Note:

  meta-llama/Llama-Prompt-Guard-2-86M

  is deprecated on HF Inference API
• Default test model is

  protectai/deberta-v3-base-prompt-injection-v2

  (publicly accessible)

Model Loading

• HuggingFace warms up models on demand
• Tests may skip with "Model loading" message on first run
• Re-run tests after model is warm

Network Issues

• Integration tests require network access to HuggingFace API
• Tests will fail in sandboxed environments without network access

Risk Levels

• none

  : score < 0.3

• low

  : score 0.3-0.5

• medium

  : score 0.5-0.7

• high

  : score 0.7-0.9

• critical

  : score >= 0.9

Categories

• benign

  : Normal, safe content

• injection

  : Prompt injection attempt

• jailbreak

  : Jailbreak attempt

• malicious

  : Malicious content (Prompt Guard v2)