Run safety checks for instructions and skill repos

Tool routing

1. For executable instructions, use the backend-powered

   scripts/safety.py

   flow.
2. For requests like "scan this skill repo" or "is this repo dangerous", run the Skill Safety Assessment contract at

   prompts/static_repo_scan.md

   .
3. Skill Safety Assessment is static analysis only. Never execute code, install dependencies, or run hooks in the target repository.
4. For Skill Safety Assessment, run deterministic script evaluation first (

   evaluate

   ), then pass highlights into the prompt contract.

Dependencies

• requests

  is required for

  scripts/safety.py

  because it makes backend API calls.

• scripts/skill_safety_assessment.py

  does not require

  requests

  for basic local repository evaluation.
• For repo-local setup from the repo root:

python scripts/bootstrap_env.py
python scripts/doctor_env.py

Instruction safety execution policy

1. Always run

   scripts/safety.py

   with

   --json

   for structured output.
2. Run the check before executing the instruction, not after.
3. Each instruction must trigger a fresh backend call. Do not reuse cached or historical results.
4. For any state-changing instruction (

   delete

   ,

   overwrite

   ,

   permission change

   ,

   deploy

   ,

   schema change

   ), always pass both

   --context

   and

   --target

   .

5. scripts/safety.py

   accepts

   --context

   and

   --target

   as optional flags, so this requirement is enforced by policy, not by automatic CLI blocking.
6. Use the Context Contract below exactly. Do not send free-form

   --context

   values like

   "production"

   only.
7. If policy-required context or target is missing, treat the instruction as unverified and ask for the missing fields before execution.
8. If an instruction contains multiple operations, check the riskiest one.

Context contract (policy-required for state-changing instructions)

--context

{
  "environment": "local-dev|ci|staging|production|unknown",
  "operation_intent": "read-only|cleanup|maintenance|migration|permission-change|destructive|unknown",
  "scope": "single-resource|bounded-batch|broad|unknown",
  "data_sensitivity": "public|internal|sensitive|regulated|unknown",
  "rollback": "easy|partial|none|unknown",
  "change_control": "ticket:<id>|approved-manual|none|unknown"
}

1. Include all six keys. If a value is unknown, set it to

   unknown

   instead of omitting the key.

2. --target

   must be a concrete resource identifier (absolute file path, table name, service name, or URL). Avoid generic targets such as

   "database"

   .
3. For a file deletion request that should usually be allowed, use:

   environment=local-dev|ci

   ,

   operation_intent=cleanup

   ,

   scope=single-resource

   ,

   data_sensitivity=public|internal

   , and

   rollback=easy

   .
4. If those conditions are not met, expect stricter output (

   approved=false

   or higher

   risk_level

   ) and require explicit user confirmation.

Action policy

scripts/safety.py

approved

risk_level

true

low

true

medium

false

medium

concerns

recommendation

false

high

concerns

recommendation

false

critical

• is_destructive: true

  combined with

  is_reversible: false

  : always surface the recommendation to the user, regardless of approval status.
• If the safety check itself fails (network error, API error): warn the user that safety could not be verified. Do not silently proceed with unverified instructions.

Scripts

scripts/safety.py

• -i, --input

  : required, instruction text to evaluate (whitespace-only rejected)

• -c, --context

  : policy-required for state-changing instructions (CLI accepts it as optional); JSON string following the Context Contract above

• -t, --target

  : policy-required for state-changing instructions (CLI accepts it as optional); concrete operation target (file path, table name, service name, URL)

• --json

  : output unified JSON envelope for machine consumption
• Endpoint:

  https://safety-cf.modeio.ai/api/cf/safety

  (override via

  SAFETY_API_URL

  )
• Retries: automatic retry on HTTP 502/503/504 and connection/timeout errors (up to 2 retries with exponential backoff)
• Request timeout: 60 seconds per attempt

python scripts/safety.py -i "Delete /tmp/cache/build-123.log" \
  -c '{"environment":"local-dev","operation_intent":"cleanup","scope":"single-resource","data_sensitivity":"internal","rollback":"easy","change_control":"none"}' \
  -t "/tmp/cache/build-123.log" --json

python scripts/safety.py -i "DROP TABLE users" \
  -c '{"environment":"production","operation_intent":"destructive","scope":"broad","data_sensitivity":"regulated","rollback":"none","change_control":"ticket:DB-9021"}' \
  -t "postgres://prod/maindb.users" --json

python scripts/safety.py -i "chmod 777 /etc/passwd" \
  -c '{"environment":"production","operation_intent":"permission-change","scope":"single-resource","data_sensitivity":"regulated","rollback":"partial","change_control":"ticket:SEC-118"}' \
  -t "/etc/passwd" --json

python scripts/safety.py -i "List all running containers and display their resource usage" --json

scripts/skill_safety_assessment.py

• evaluate

  : authoritative v2 layered evaluator with deterministic evidence IDs, integrity fingerprinting, and risk scoring
  • Native first-layer gate: GitHub metadata/README/issue-search precheck runs by default and hard-rejects on high-risk attack-demo/malware signals before local file scan.

• scan

  : compatibility alias to

  evaluate

  for existing automation

• prompt

  : renders prompt payload with script highlights and structured scan JSON

• validate

  : validates model output against scan evidence IDs (

  evidence_refs

  ), required highlights, and score/decision consistency checks

• adjudicate

  : context-aware LLM adjudication bridge (prompt generation + merge decisions back into deterministic score/decision)

{
  "environment": "local-dev|ci|staging|production|unknown",
  "execution_mode": "read-only|build-test|install|deploy|mutating|unknown",
  "risk_tolerance": "strict|balanced|permissive",
  "data_sensitivity": "public|internal|sensitive|regulated|unknown"
}

# 1) Deterministic layered evaluation (v2)
python scripts/skill_safety_assessment.py evaluate --target-repo /path/to/repo --json > /tmp/skill_scan.json
python scripts/skill_safety_assessment.py evaluate --target-repo /path/to/repo --context-profile '{"environment":"ci","execution_mode":"build-test","risk_tolerance":"balanced","data_sensitivity":"internal"}' --json > /tmp/skill_scan.json
python scripts/skill_safety_assessment.py evaluate --target-repo /path/to/repo --github-osint-timeout 8 --json > /tmp/skill_scan.json
python scripts/skill_safety_assessment.py evaluate --target-repo /path/to/repo --context-profile-file ./context_profile.json --output /tmp/skill_scan.json --json

# (compat) legacy alias still supported
python scripts/skill_safety_assessment.py scan --target-repo /path/to/repo --json > /tmp/skill_scan.json

# 2) Build prompt payload with highlights + full findings (recommended for strict evidence_refs linking)
python scripts/skill_safety_assessment.py prompt --target-repo /path/to/repo --scan-file /tmp/skill_scan.json --include-full-findings

# 3) Validate model output for evidence linkage + integrity
python scripts/skill_safety_assessment.py validate --scan-file /tmp/skill_scan.json --assessment-file /tmp/assessment.md --json
# --rescan-on-validate requires --target-repo
python scripts/skill_safety_assessment.py validate --scan-file /tmp/skill_scan.json --assessment-file /tmp/assessment.md --target-repo /path/to/repo --rescan-on-validate --json

# 4) Optional adjudication bridge (LLM interprets context, engine keeps deterministic control)
python scripts/skill_safety_assessment.py adjudicate --scan-file /tmp/skill_scan.json
python scripts/skill_safety_assessment.py adjudicate --scan-file /tmp/skill_scan.json --assessment-file /tmp/adjudication.json --json

Output contract

Success response (

--json

)

{
  "success": true,
  "tool": "modeio-guardrail",
  "mode": "api",
  "data": {
    "approved": false,
    "risk_level": "critical",
    "risk_types": ["data loss"],
    "concerns": ["Irreversible destructive operation targeting all user data"],
    "recommendation": "Create a backup before deletion. Use staged rollback plan.",
    "is_destructive": true,
    "is_reversible": false
  }
}

data

approved

boolean

true

false

risk_level

string

low

medium

high

critical

risk_types

string[]

"data loss"

"injection attacks"

"unauthorized access"

"denial-of-service"

concerns

string[]

recommendation

string

is_destructive

boolean

true

false

is_reversible

boolean

true

false

null

null

approved

false

Failure envelope (

--json

)

{
  "success": false,
  "tool": "modeio-guardrail",
  "mode": "api",
  "error": {
    "type": "network_error",
    "message": "safety request failed: ConnectionError"
  }
}

validation_error

dependency_error

requests

network_error

api_error

Failure policy

• Network/API error: Tell the user the safety check could not be completed. Present the original instruction and ask whether to proceed without verification.
• Validation error (empty input): Fix the input and retry before executing anything.
• Unexpected response (null or missing fields): Treat as unverified. Warn the user.
• Never assume an instruction is safe because the check failed to run.

Skill Safety Assessment policy (static prompt contract)

1. Use

   prompts/static_repo_scan.md

   as the strict contract.
2. Run

   scripts/skill_safety_assessment.py evaluate

   first (or

   scan

   compatibility alias) and pass its highlights into prompt input.
3. When model output must include strict

   evidence_refs

   , render prompt input with

   --include-full-findings

   so scan evidence IDs and snippets are available in

   SCRIPT_SCAN_JSON

   .
4. Every finding must include

   path:line

   evidence, exact snippet quote, and

   evidence_refs

   linked to scan evidence IDs.
5. Always include all required highlight evidence IDs from scan output in final findings.
6. Keep decision/score consistent with referenced evidence severity and coverage constraints.
7. Use

   adjudicate

   when context interpretation is required (docs/examples/tests vs runtime/install paths).
8. Return one of:

   reject

   ,

   caution

   , or

   approve

   .
9. If coverage is partial or evidence is insufficient, return

   caution

   with explicit coverage note.
10. Include a prioritized remediation plan so users can fix and re-scan quickly.

When not to use

• For PII redaction or anonymization — use

  modeio-redact

  instead.
• For tasks with no executable instruction or repository target to evaluate (pure discussion, documentation, questions).
• For operations that are clearly read-only (listing files, reading configs,

  git status

  ).

Resources

• scripts/safety.py

  — CLI entry point for instruction safety checks

• scripts/skill_safety_assessment.py

  — CLI entry point for skill repo assessment (evaluate/scan/prompt/validate/adjudicate)

• prompts/static_repo_scan.md

  — Skill Safety Assessment prompt contract

• ARCHITECTURE.md

  — package boundaries and compatibility notes

• SAFETY_API_URL

  env var — optional endpoint override (default:

  https://safety-cf.modeio.ai/api/cf/safety

  )