Loading...
Loading...
Use Slopwatch to detect LLM reward hacking in .NET code changes. Run after every code modification to catch disabled tests, suppressed warnings, empty catch blocks, and other shortcuts that mask real problems.
npx skill4agent add aaronontheweb/dotnet-skills dotnet-slopwatch| Pattern | Example | Why It's Bad |
|---|---|---|
| Disabled tests | | Hides failures instead of fixing them |
| Warning suppression | | Silences compiler without fixing issue |
| Empty catch blocks | | Swallows errors, hides bugs |
| Arbitrary delays | | Masks race conditions, makes tests slow |
| Project-level suppression | | Disables warnings project-wide |
| CPM bypass | | Undermines central package management |
.config/dotnet-tools.json{
"version": 1,
"isRoot": true,
"tools": {
"slopwatch.cmd": {
"version": "0.2.0",
"commands": ["slopwatch"],
"rollForward": false
}
}
}dotnet tool restoredotnet tool install --global Slopwatch.Cmd# Initialize baseline from existing code
slopwatch init
# This creates .slopwatch/baseline.json
git add .slopwatch/baseline.json
git commit -m "Add slopwatch baseline"# Analyze for new issues (uses baseline)
slopwatch analyze
# Use strict mode - fail on warnings too
slopwatch analyze --fail-on warning# Example: LLM disabled a test
❌ SW001 [Error]: Disabled test detected
File: tests/MyApp.Tests/OrderTests.cs:45
Pattern: [Fact(Skip="Test is flaky")]
# Correct response: Ask for actual fix
"This test was disabled instead of fixed. Please investigate why
it's flaky and fix the underlying timing/race condition issue."# Add current detections to baseline (use sparingly!)
slopwatch analyze --update-baseline.claude/settings.json{
"hooks": {
"PostToolUse": [
{
"matcher": "Write|Edit|MultiEdit",
"hooks": [
{
"type": "command",
"command": "slopwatch analyze -d . --hook",
"timeout": 60000
}
]
}
]
}
}--hookjobs:
slopwatch:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Install Slopwatch
run: dotnet tool install --global Slopwatch.Cmd
- name: Run Slopwatch
run: slopwatch analyze -d . --fail-on warning- task: DotNetCoreCLI@2
displayName: 'Install Slopwatch'
inputs:
command: 'custom'
custom: 'tool'
arguments: 'install --global Slopwatch.Cmd'
- script: slopwatch analyze -d . --fail-on warning
displayName: 'Slopwatch Analysis'| Rule | Severity | What It Catches |
|---|---|---|
| SW001 | Error | Disabled tests ( |
| SW002 | Warning | Warning suppression ( |
| SW003 | Error | Empty catch blocks that swallow exceptions |
| SW004 | Warning | Arbitrary delays in tests ( |
| SW005 | Warning | Project file slop ( |
| SW006 | Warning | CPM bypass ( |
.slopwatch/slopwatch.json{
"minSeverity": "warning",
"rules": {
"SW001": { "enabled": true, "severity": "error" },
"SW002": { "enabled": true, "severity": "warning" },
"SW003": { "enabled": true, "severity": "error" },
"SW004": { "enabled": true, "severity": "warning" },
"SW005": { "enabled": true, "severity": "warning" },
"SW006": { "enabled": true, "severity": "warning" }
},
"exclude": [
"**/Generated/**",
"**/obj/**",
"**/bin/**"
]
}{
"minSeverity": "warning",
"rules": {
"SW001": { "enabled": true, "severity": "error" },
"SW002": { "enabled": true, "severity": "error" },
"SW003": { "enabled": true, "severity": "error" },
"SW004": { "enabled": true, "severity": "error" },
"SW005": { "enabled": true, "severity": "error" },
"SW006": { "enabled": true, "severity": "error" }
}
}# First time setup
slopwatch init
git add .slopwatch/baseline.json
# After every LLM code change
slopwatch analyze
# Strict mode (recommended)
slopwatch analyze --fail-on warning
# With stats (performance debugging)
slopwatch analyze --stats
# Update baseline (rare, document why)
slopwatch analyze --update-baseline
# JSON output for tooling
slopwatch analyze --output json| Scenario | Action | Required |
|---|---|---|
| Third-party forces pattern | Update baseline | Code comment explaining why |
| Generated code (not editable) | Add to exclude list | Document in config |
| Intentional rate limiting delay | Update baseline | Code comment, not in test |
| Legacy code cleanup | One-time baseline update | PR description |