Back to Details

security-integration-tests

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Security Integration Tests Agent

安全集成测试Agent

Overview

概述

The security package (
pkg/security/
) provides prompt injection detection using the HuggingFace Inference API. Integration tests verify detection accuracy with real API calls.
安全包（
pkg/security/
）通过HuggingFace Inference API提供提示注入检测功能。集成测试通过真实API调用验证检测的准确性。

Key Files

关键文件

  • pkg/security/testdata/samples.json
    - Test data with benign and malicious samples
  • pkg/security/hf/integration_test.go
    - HF client integration tests
  • pkg/security/integration_test.go
    - Full analyzer integration tests with event bus
  • pkg/security/hf/client.go
    - HuggingFace API client implementation
  • pkg/security/analyzer.go
    - Main security analyzer that subscribes to MCP events
  • pkg/security/testdata/samples.json
    - 包含良性和恶意样本的测试数据
  • pkg/security/hf/integration_test.go
    - HF客户端集成测试
  • pkg/security/integration_test.go
    - 带事件总线的完整分析器集成测试
  • pkg/security/hf/client.go
    - HuggingFace API客户端实现
  • pkg/security/analyzer.go
    - 订阅MCP事件的主安全分析器

Running Tests

运行测试

bash
undefined
bash
undefined

Run integration tests (requires HF_TOKEN environment variable)

Run integration tests (requires HF_TOKEN environment variable)

HF_TOKEN=hf_xxx make test-integration
HF_TOKEN=hf_xxx make test-integration

Run with custom model

Run with custom model

HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration
HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration

Run unit tests only (no API calls, uses mock server)

Run unit tests only (no API calls, uses mock server)

go test -v ./pkg/security/...
undefined
go test -v ./pkg/security/...
undefined

Test Structure

测试结构

Test Samples (
samples.json
)

测试样本（
samples.json
）

json
{
  "benign_samples": [
    {
      "description": "Description of the sample",
      "text": "The text to analyze",
      "expected_detected": false
    }
  ],
  "malicious_samples": [
    {
      "description": "Direct instruction override",
      "text": "Ignore all previous instructions...",
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ],
  "mcp_tool_calls": [
    {
      "description": "Malicious tool call",
      "method": "tools/call",
      "params": { "name": "run_command", "arguments": {...} },
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ]
}
json
{
  "benign_samples": [
    {
      "description": "Description of the sample",
      "text": "The text to analyze",
      "expected_detected": false
    }
  ],
  "malicious_samples": [
    {
      "description": "Direct instruction override",
      "text": "Ignore all previous instructions...",
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ],
  "mcp_tool_calls": [
    {
      "description": "Malicious tool call",
      "method": "tools/call",
      "params": { "name": "run_command", "arguments": {...} },
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ]
}

Integration Test Tags

集成测试标签

Integration tests use the build tag 
//go:build integration
and are excluded from regular 
go test ./...
runs.
集成测试使用构建标签
//go:build integration
，会被排除在常规
go test ./...
运行之外。

Adding New Test Samples

添加新测试样本

  1. Edit 
    pkg/security/testdata/samples.json
  2. Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)
  3. Set 
    expected_detected
    and optionally 
    min_risk_score
  4. Run integration tests to verify
  1. 编辑
    pkg/security/testdata/samples.json
  2. 将样本添加到对应的类别（benign_samples、malicious_samples或mcp_tool_calls）
  3. 设置
    expected_detected
    （可选设置
    min_risk_score
    ）
  4. 运行集成测试进行验证

Common Issues

常见问题

"Forbidden" Error

"Forbidden"错误

  • Ensure HF_TOKEN is set and valid
  • Note: 
    meta-llama/Llama-Prompt-Guard-2-86M
    is deprecated on HF Inference API
  • Default test model is 
    protectai/deberta-v3-base-prompt-injection-v2
    (publicly accessible)
  • 确保已设置有效的HF_TOKEN
  • 注意：
    meta-llama/Llama-Prompt-Guard-2-86M
    在HF Inference API中已废弃
  • 默认测试模型为
    protectai/deberta-v3-base-prompt-injection-v2
    （可公开访问）

Model Loading

模型加载

  • HuggingFace warms up models on demand
  • Tests may skip with "Model loading" message on first run
  • Re-run tests after model is warm
  • HuggingFace会按需预热模型
  • 首次运行时测试可能会因"Model loading"提示而跳过
  • 模型预热完成后重新运行测试

Network Issues

网络问题

  • Integration tests require network access to HuggingFace API
  • Tests will fail in sandboxed environments without network access
  • 集成测试需要访问HuggingFace API的网络权限
  • 在无网络访问的沙箱环境中测试会失败

Risk Levels

风险等级

  • none
    : score < 0.3
  • low
    : score 0.3-0.5
  • medium
    : score 0.5-0.7
  • high
    : score 0.7-0.9
  • critical
    : score >= 0.9
  • none
    ：分数 < 0.3
  • low
    ：分数 0.3-0.5
  • medium
    ：分数 0.5-0.7
  • high
    ：分数 0.7-0.9
  • critical
    ：分数 >= 0.9

Categories

类别

  • benign
    : Normal, safe content
  • injection
    : Prompt injection attempt
  • jailbreak
    : Jailbreak attempt
  • malicious
    : Malicious content (Prompt Guard v2)
  • benign
    ：正常、安全的内容
  • injection
    ：提示注入尝试
  • jailbreak
    ：越狱尝试
  • malicious
    ：恶意内容（Prompt Guard v2）