ai-scanner-garak

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

AI Scanner (0din-ai/ai-scanner)

AI Scanner（0din-ai/ai-scanner）

Skill by ara.so — Daily 2026 Skills collection.

AI Scanner is an open-source Ruby on Rails web application for AI model security assessments, wrapping NVIDIA garak with a multi-tenant UI, scheduling, PDF reports, and SIEM integration. It runs 179 community probes across 35 vulnerability families aligned with the OWASP LLM Top 10.

由ara.so提供的Skill——2026每日Skill合集。

AI Scanner是一款开源的Ruby on Rails Web应用，用于AI模型安全评估，它为NVIDIA garak封装了多租户UI、任务调度、PDF报告生成以及SIEM集成功能。它可运行179个社区探针，覆盖与OWASP LLM Top 10对齐的35个漏洞家族。

Installation

安装

Quick Install (Docker)

快速安装（Docker方式）

bash

curl -sL https://raw.githubusercontent.com/0din-ai/ai-scanner/main/scripts/install.sh | bash

bash

curl -sL https://raw.githubusercontent.com/0din-ai/ai-scanner/main/scripts/install.sh | bash

Manual Install

手动安装

bash

curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/dist/docker-compose.yml
curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/.env.example
cp .env.example .env

Edit

.env

with required values:

bash

undefined

bash

curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/dist/docker-compose.yml
curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/.env.example
cp .env.example .env

编辑

.env

文件并填入必要值：

bash

undefined

Generate a secure key

生成安全密钥

openssl rand -hex 64

.env minimum required values

.env文件必填项

SECRET_KEY_BASE=<output_of_above_command> POSTGRES_PASSWORD=<your_secure_db_password>


```bash
docker compose up -d

Access at

http://localhost

— default credentials:

admin@example.com

password

.
Change the default password immediately after first login.

SECRET_KEY_BASE=<上述命令的输出> POSTGRES_PASSWORD=<你的安全数据库密码>


```bash
docker compose up -d

访问

http://localhost

——默认凭据：

admin@example.com

password

。
首次登录后请立即修改默认密码。

Configuration (.env)

配置（.env文件）

bash

undefined

bash

undefined

Required

必填项

SECRET_KEY_BASE=<64-byte-hex> POSTGRES_PASSWORD=<strong-password>

SECRET_KEY_BASE=<64字节十六进制字符串> POSTGRES_PASSWORD=<强密码>

Optional: custom port

可选：自定义端口

PORT=8080

Optional: SIEM integration

可选：SIEM集成

SPLUNK_HEC_URL=https://splunk.example.com:8088/services/collector SPLUNK_HEC_TOKEN=$SPLUNK_HEC_TOKEN RSYSLOG_HOST=syslog.example.com RSYSLOG_PORT=514

Optional: email

可选：邮件配置

SMTP_HOST=smtp.example.com SMTP_PORT=587 SMTP_USERNAME=$SMTP_USERNAME SMTP_PASSWORD=$SMTP_PASSWORD

undefined

SMTP_HOST=smtp.example.com SMTP_PORT=587 SMTP_USERNAME=$SMTP_USERNAME SMTP_PASSWORD=$SMTP_PASSWORD

undefined

Core Concepts

核心概念

Concept	Description
Target	An AI system to test — API-based LLM or browser-based chat UI
Probe	A single attack test (e.g., prompt injection, data leakage)
Scan	A run of selected probes against a target
ASR	Attack Success Rate — percentage of probes that succeeded
Organization	Tenant boundary; users and scans are scoped per org

概念	描述
Target（目标）	要测试的AI系统——基于API的LLM或基于浏览器的聊天UI
Probe（探针）	单个攻击测试（如提示注入、数据泄露）
Scan（扫描任务）	针对某个目标运行选定探针的过程
ASR（攻击成功率）	攻击成功率——成功的探针占比
Organization（组织）	租户边界；用户和扫描任务按组织划分范围

Setting Up a Target

设置目标

Targets define what you're scanning. Two types:

API-based LLM Target (e.g., OpenAI-compatible endpoint):

ruby

undefined

目标定义了你要扫描的对象，分为两种类型：

基于API的LLM目标（如兼容OpenAI的端点）：

ruby

undefined

In Rails console or via UI — representative model

在Rails控制台或通过UI操作——示例代码

target = Target.create!( name: "Production GPT-4", target_type: "api", api_endpoint: "https://api.openai.com/v1/chat/completions", api_key: ENV["OPENAI_API_KEY"], model_name: "gpt-4", organization: current_organization )


**Browser-based Chat UI Target**:

```ruby
target = Target.create!(
  name: "Internal Chatbot UI",
  target_type: "browser",
  url: "https://chatbot.internal.example.com",
  organization: current_organization
)


**基于浏览器的聊天UI目标**：

```ruby
target = Target.create!(
  name: "Internal Chatbot UI",
  target_type: "browser",
  url: "https://chatbot.internal.example.com",
  organization: current_organization
)

Running a Scan

运行扫描任务

Via UI

通过UI操作

Navigate to Targets → select your target
Click New Scan
Select probe families or individual probes
Click Run Scan

导航至**Targets（目标）**页面 → 选择你的目标
点击New Scan（新建扫描任务）
选择探针家族或单个探针
点击Run Scan（运行扫描）

Via Rails Console

通过Rails控制台

ruby

undefined

ruby

undefined

On-demand scan with specific probe families

按需运行扫描，指定特定探针家族

scan = Scan.create!( target: target, probe_families: ["prompt_injection", "data_leakage", "insecure_output"], organization: current_organization ) ScanJob.perform_later(scan.id)

undefined

scan = Scan.create!( target: target, probe_families: ["prompt_injection", "data_leakage", "insecure_output"], organization: current_organization ) ScanJob.perform_later(scan.id)

undefined

Scheduled Recurring Scan

定期重复扫描任务

ruby

undefined

ruby

undefined

Weekly scan every Monday at 2am

每周一凌晨2点执行扫描任务

scheduled_scan = ScheduledScan.create!( target: target, probe_families: ["prompt_injection", "jailbreak"], cron_expression: "0 2 * * 1", organization: current_organization )

undefined

scheduled_scan = ScheduledScan.create!( target: target, probe_families: ["prompt_injection", "jailbreak"], cron_expression: "0 2 * * 1", organization: current_organization )

undefined

Probe Families (35 total, aligned to OWASP LLM Top 10)

探针家族（共35个，与OWASP LLM Top 10对齐）

Key probe families available:

ruby

undefined

可用的核心探针家族：

ruby

undefined

List all available probe families

列出所有可用的探针家族

Garak::ProbeRegistry.families

=> ["prompt_injection", "jailbreak", "data_leakage", "insecure_output",

"supply_chain", "sensitive_info", "excessive_agency", "overreliance",

"model_theft", "malicious_plugins", ...]

Get probes within a family

获取某个家族下的探针

Garak::ProbeRegistry.probes_for("prompt_injection")

=> 179 total probes across all families

=> 所有家族共包含179个探针

undefined

undefined

Viewing Results

查看结果

Attack Success Rate (ASR)

攻击成功率（ASR）

ruby

scan = Scan.find(scan_id)

puts scan.asr_score          # => 0.23 (23% attack success rate)
puts scan.status             # => "completed"
puts scan.probe_results.count # => 47

ruby

scan = Scan.find(scan_id)

puts scan.asr_score          # => 0.23（23%的攻击成功率）
puts scan.status             # => "completed（已完成）"
puts scan.probe_results.count # => 47

Per-probe breakdown

单个探针的结果明细

scan.probe_results.each do |result| puts "#{result.probe_name}: #{result.passed? ? 'SAFE' : 'VULNERABLE'}" puts " Attempts: #{result.attempt_count}" puts " ASR: #{result.asr_score}" end

undefined

scan.probe_results.each do |result| puts "#{result.probe_name}: #{result.passed? ? 'SAFE（安全）' : 'VULNERABLE（存在漏洞）'}" puts " 尝试次数: #{result.attempt_count}" puts " 攻击成功率: #{result.asr_score}" end

undefined

Trend Tracking

趋势跟踪

ruby

undefined

ruby

undefined

Compare ASR across scan runs for a target

对比同一目标多次扫描的攻击成功率

target.scans.completed.order(:created_at).map do |scan| { date: scan.created_at, asr: scan.asr_score } end

undefined

target.scans.completed.order(:created_at).map do |scan| { date: scan.created_at, asr: scan.asr_score } end

undefined

PDF Report Export

PDF报告导出

ruby

undefined

ruby

undefined

Generate PDF report for a scan

为某扫描任务生成PDF报告

scan = Scan.find(scan_id) pdf_path = ReportExporter.export_pdf(scan)

Includes: executive summary, per-probe results, per-attempt drill-down

报告包含：执行摘要、单个探针结果、每一次尝试的详细信息


Via UI: Navigate to a completed scan → **Export PDF**.


通过UI操作：导航至已完成的扫描任务页面 → **Export PDF（导出PDF）**。

SIEM Integration

SIEM集成

Splunk

ruby

undefined

ruby

undefined

config/initializers/siem.rb

SiemIntegration.configure do |config| config.provider = :splunk config.splunk_hec_url = ENV["SPLUNK_HEC_URL"] config.splunk_hec_token = ENV["SPLUNK_HEC_TOKEN"] config.forward_on_completion = true end

undefined

SiemIntegration.configure do |config| config.provider = :splunk config.splunk_hec_url = ENV["SPLUNK_HEC_URL"] config.splunk_hec_token = ENV["SPLUNK_HEC_TOKEN"] config.forward_on_completion = true end

undefined

Rsyslog

ruby

SiemIntegration.configure do |config|
  config.provider = :rsyslog
  config.rsyslog_host = ENV["RSYSLOG_HOST"]
  config.rsyslog_port = ENV["RSYSLOG_PORT"].to_i
end

ruby

SiemIntegration.configure do |config|
  config.provider = :rsyslog
  config.rsyslog_host = ENV["RSYSLOG_HOST"]
  config.rsyslog_port = ENV["RSYSLOG_PORT"].to_i
end

Multi-Tenant Organization Management

多租户组织管理

ruby

undefined

ruby

undefined

Create a new organization

创建新组织

org = Organization.create!(name: "Security Team Alpha")

Invite a user

邀请用户

user = User.invite!( email: "analyst@example.com", organization: org, role: "analyst" # roles: "admin", "analyst", "viewer" )

user = User.invite!( email: "analyst@example.com", organization: org, role: "analyst" # 角色："admin（管理员）", "analyst（分析师）", "viewer（查看者）" )

Data is encrypted at rest per organization

数据按组织进行静态加密

org.encryption_key # => managed automatically

undefined

org.encryption_key # => 自动管理

undefined

Development Setup

开发环境搭建

bash

git clone https://github.com/0din-ai/ai-scanner.git
cd ai-scanner
cp .env.example .env.development

bash

git clone https://github.com/0din-ai/ai-scanner.git
cd ai-scanner
cp .env.example .env.development

Install dependencies

安装依赖

bundle install

Database setup

数据库设置

rails db:create db:migrate db:seed

Install garak (Python dependency)

安装garak（Python依赖）

pip install garak

Start development server

启动开发服务器

bin/dev

undefined

bin/dev

undefined

Running Tests

运行测试

bash

undefined

bash

undefined

Full test suite

完整测试套件

bundle exec rspec

Specific area

特定模块测试

bundle exec rspec spec/models/scan_spec.rb bundle exec rspec spec/jobs/scan_job_spec.rb

Lint

代码检查

bundle exec rubocop

undefined

bundle exec rubocop

undefined

Common Patterns

常见使用场景

Testing a New LLM Before Deployment

部署前测试新LLM

ruby

undefined

ruby

undefined

Comprehensive pre-deployment scan

全面的部署前扫描

target = Target.create!( name: "New Model v2 - Pre-deploy", target_type: "api", api_endpoint: ENV["NEW_MODEL_ENDPOINT"], api_key: ENV["NEW_MODEL_API_KEY"], model_name: "new-model-v2", organization: current_organization )

Run all 35 probe families

运行所有35个探针家族의扫描

scan = Scan.create!( target: target, probe_families: Garak::ProbeRegistry.families, organization: current_organization ) ScanJob.perform_now(scan.id)

if scan.reload.asr_score > 0.15 puts "WARNING: ASR #{scan.asr_score} exceeds threshold. Review before deploying." else puts "PASS: Model meets security threshold." end

undefined

scan = Scan.create!( target: target, probe_families: Garak::ProbeRegistry.families, organization: current_organization ) ScanJob.perform_now(scan.id)

if scan.reload.asr_score > 0.15 puts "警告：攻击成功率#{scan.asr_score}超过阈值。部署前请进行审核。" else puts "通过：模型符合安全阈值要求。" end

undefined

Using the Mock LLM for Testing Scanner Setup

使用Mock LLM测试扫描器设置

The built-in Mock LLM lets you validate your scanner configuration without hitting real APIs:

ruby

target = Target.create!(
  name: "Mock LLM",
  target_type: "mock",
  organization: current_organization
)

内置的Mock LLM可让你无需调用真实API即可验证扫描器配置：

ruby

target = Target.create!(
  name: "Mock LLM",
  target_type: "mock",
  organization: current_organization
)

Run a quick scan to verify everything works end-to-end

运行快速扫描以端到端验证所有功能是否正常

undefined

undefined

Webhook on Scan Completion

扫描完成时触发Webhook

ruby

undefined

ruby

undefined

config/initializers/scan_hooks.rb

ActiveSupport::Notifications.subscribe("scan.completed") do |_, _, _, _, payload| scan = Scan.find(payload[:scan_id]) if scan.asr_score > 0.20 SlackNotifier.alert( channel: "#security-alerts", message: "High ASR detected: #{scan.asr_score} on #{scan.target.name}" ) end end

undefined

ActiveSupport::Notifications.subscribe("scan.completed") do |_, _, _, _, payload| scan = Scan.find(payload[:scan_id]) if scan.asr_score > 0.20 SlackNotifier.alert( channel: "#security-alerts", message: "检测到高攻击成功率：#{scan.target.name}的ASR为#{scan.asr_score}" ) end end

undefined

Docker Compose Production Tips

Docker Compose生产环境小贴士

yaml

undefined

yaml

undefined

docker-compose.override.yml — production additions

docker-compose.override.yml —— 生产环境附加配置

services: web: environment: RAILS_ENV: production FORCE_SSL: "true" labels: - "traefik.enable=true" - "traefik.http.routers.scanner.rule=Host(

scanner.example.com

)" - "traefik.http.routers.scanner.tls.certresolver=letsencrypt"


```bash

services: web: environment: RAILS_ENV: production FORCE_SSL: "true" labels: - "traefik.enable=true" - "traefik.http.routers.scanner.rule=Host(

scanner.example.com

)" - "traefik.http.routers.scanner.tls.certresolver=letsencrypt"


```bash

Upgrade

升级

docker compose pull docker compose up -d docker compose exec web rails db:migrate

undefined

docker compose pull docker compose up -d docker compose exec web rails db:migrate

undefined

Troubleshooting

故障排查

Problem	Solution
Scan stuck in "running"	Check `docker compose logs worker` — garak Python process may have crashed
`SECRET_KEY_BASE` error on start	Run `openssl rand -hex 64` and set in `.env`
Can't connect to target API	Verify API key env var is set; check firewall allows outbound from container
Browser target scan fails	Ensure Playwright/Chrome is available in the worker container
PDF export blank	Check `wkhtmltopdf` is installed in the web container
SIEM not receiving events	Verify `SPLUNK_HEC_URL` includes full path `/services/collector`

bash

undefined

问题	解决方案
扫描任务一直处于“running（运行中）”状态	查看 `docker compose logs worker` 日志——garak Python进程可能已崩溃
启动时出现 `SECRET_KEY_BASE` 错误	运行 `openssl rand -hex 64` 生成密钥并在 `.env` 中设置
无法连接到目标API	确认API密钥的环境变量已设置；检查防火墙是否允许容器出站请求
浏览器类型目标扫描失败	确保worker容器中已安装Playwright/Chrome
PDF导出为空	检查web容器中是否已安装 `wkhtmltopdf`
SIEM未接收到事件	确认 `SPLUNK_HEC_URL` 包含完整路径 `/services/collector`

bash

undefined

View all service logs

查看所有服务日志

docker compose logs -f

Check worker specifically (runs garak)

专门查看worker日志（运行garak的服务）

docker compose logs -f worker

Rails console for debugging

进入Rails控制台进行调试

docker compose exec web rails console

Check garak is working

检查garak是否正常工作

docker compose exec worker python -c "import garak; print(garak.version)"

undefined

docker compose exec worker python -c "import garak; print(garak.version)"

undefined

Key Files (for Contributors)

核心文件（面向贡献者）

app/
  models/
    scan.rb          # Core scan model, ASR calculation
    target.rb        # Target types and validation
    probe_result.rb  # Per-probe result storage
  jobs/
    scan_job.rb      # Async job that invokes garak
  services/
    garak_runner.rb  # Ruby wrapper around garak CLI
    report_exporter.rb
    siem_integration.rb
lib/
  garak/
    probe_registry.rb  # 179 probes, 35 families
dist/
  docker-compose.yml   # Production compose file
scripts/
  install.sh           # One-line installer

app/
  models/
    scan.rb          # 核心扫描模型，ASR计算逻辑
    target.rb        # 目标类型与验证逻辑
    probe_result.rb  # 单个探针结果存储模型
  jobs/
    scan_job.rb      # 调用garak的异步任务
  services/
    garak_runner.rb  # 封装garak CLI的Ruby工具
    report_exporter.rb
    siem_integration.rb
lib/
  garak/
    probe_registry.rb  # 179个探针、35个家族的注册管理
dist/
  docker-compose.yml   # 生产环境Compose配置文件
scripts/
  install.sh           # 一键安装脚本