Loading...
Loading...
Fix broken data scrapers and pipelines. Use when data acquisition fails, a scraper breaks, an API returns errors, or data format has changed. Also handles submitting upstream issues or PRs when the problem is in a dependency like soccerdata or kloppy.
npx skill4agent add withqwerty/nutmeg nutmeg-healdocs/accuracy-guardrail.mdsearch_docs.nutmeg.user.md/nutmeg| Symptom | Likely cause |
|---|---|
| HTTP 403/429 | Rate limited or blocked. Wait and retry with backoff |
| HTTP 404 | URL/endpoint changed. Check if site restructured |
| Parse error (HTML) | Website redesigned. Scraper selectors need updating |
| Parse error (JSON) | API response schema changed. Check for versioning |
| Empty response | Data not available for this competition/season |
| Import error | Library version changed. Check changelog |
| Authentication error | Key expired, rotated, or wrong format |
/nutmeg:acquire# Retry with exponential backoff
import time
def fetch_with_retry(url, max_retries=3):
for attempt in range(max_retries):
try:
resp = requests.get(url, timeout=30)
resp.raise_for_status()
return resp.json()
except requests.RequestException as e:
if attempt == max_retries - 1:
raise
wait = 2 ** attempt
print(f"Attempt {attempt + 1} failed, retrying in {wait}s: {e}")
time.sleep(wait)| Source | Common issue | Fix |
|---|---|---|
| FBref | 429 rate limit | Add 6s delay between requests |
| WhoScored | Cloudflare blocks | Use headed browser (Playwright) |
| Understat | JSON parse error | Response is JSONP, strip callback wrapper |
| SportMonks | 401 | Token expired or plan limit hit |
| StatsBomb open data | 404 | Match/competition not in open dataset |