Fix OpenAPI Spec
Use this skill when the task is to diagnose an OpenAPI specification that is preventing it from running as mock or test.
Canonical Paths
Use these exact paths and file names everywhere in this workflow.
- Editable spec copy:
<spec-name>-updated.<ext>
- Actions log:
fix-log-<spec-name>-<current-date>.md
Workflow State Machine
Phase 1 -> Phase 2 -> Phase 3 -> Phase 4 validation -> STOP for user approval -> Phase 4 fixes only if explicitly approved
Docker Execution Rule
- Assume Docker is available and the Docker engine is running.
- Do not ask the user about Docker availability before attempting ,
scripts/run_loop_test.ps1
, , or scripts/validate_spec.ps1
.
- Run the relevant script first.
- If the script output indicates a Docker-specific failure such as Docker not being installed, Docker not being on , Docker Desktop not being available, or the Docker daemon / engine not running, stop and ask the user exactly:
Please confirm if docker engine is running
- Do not claim validation or loop testing is unavailable until after a script run fails for a Docker-specific reason.
- If it appears to be a permissions issue, try resolving it using your environment’s built-in privilege escalation mechanisms available to you.
Working area
Ignore any old test files or log files that existed before this conversation. You'll work with the output of the relevant commands as per the workflow state machine.
Phase 1: Setup
- Create a copy of the input spec path for edits named
<spec-name>-updated.<ext>
in the same directory as the source spec.
- Note: Always edit the copy, not the original spec.
- Create a single append-only actions log file named
fix-log-<spec-name>-<current-date>.md
in the spec directory, with the following initial content:
# Actions log for <spec-file>
- Update docker using this command:
docker pull specmatic/specmatic:latest
Phase 2: Auto-fix obscure errors
- Run the loop test script against the editable copy.
- Immediately after the loop test ends, append a structured entry to
fix-log-<spec-name>-<current-date>.md
with this format:
## <title>
**Timestamp**: <timestamp>
**Action**: Loop test run
**Result**: [pass|fail] - <brief description of the result>
- If the loop test passes, go to step 5, the final check. All failed tests are considered "must-fix" errors.
- If the loop test fails, attempt to fix one of the errors following the guidelines for addressing errors, and then go back to step 1.
- If any must-fix errors remain to be addressed, go back to step 1 and repeat the process until all must-fix errors have been addressed.
- Final check: once all the tests are passing, go back to step one and run the loop test again to make sure that there are no errors left.
Guidelines for addressing errors:
-
Classify every failure before moving on:
- : the run points to a concrete schema, example, enum, , , or constraint defect in the spec.
- : the error is unclear (does not identify a concrete spec defect or provide enough information to point directly to where the issue is), or if it appears that Specmatic cannot handle a value / feature in the spec that is perfectly legimiate (e.g. valid regex syntax, valid OpenAPI syntax, etc)
- Treat any opaque error as a . "Opaque" means the message does not identify a concrete spec defect or does not provide enough information to fix the spec confidently.
- If relaxing or removing a valid constraint is used to make the workflow pass, log it as a .
- A status mismatch is not opaque merely because the final summary only shows . Before classifying it as , inspect the raw temp test log around the full request/response transcript. Look both above and below the failed scenario summary. Only classify as opaque if no response body or adjacent transcript contains a concrete contract path or rule violation.
- If Specmatic-generated traffic fails against the same specification, classify it as unless the failing constraint is itself invalid OpenAPI, invalid syntax, or demonstrably conflicts with another schema rule such as enum/example values.
-
If the issue is classified as a
, log it immediately when first observed in
fix-log-<spec-name>-<current-date>.md
, using this format:
## <title>
**Timestamp**: <timestamp>
**Action**: Issue Classification
**Error**: <error message that prompted this bug classification>
**Classification**: Specmatic Bug
**Reason**: <why this is considered a Specmatic Bug instead of a concrete spec defect>
**Logs**: <log snippet showing the issue that prompted the fix>
-
If the issue is classified as a
, log it immediately when first observed in
fix-log-<spec-name>-<current-date>.md
, using this format:
## <title>
**Timestamp**: <timestamp>
**Action**: Issue Classification
**Error**: <error message that prompted this bug classification>
**Classification**: Spec Issue
**Reason**: <why this is considered a Spec Issue instead of a Specmatic Bug>
**Logs**: <log snippet showing the issue that prompted the fix>
-
Do not wait for the bug to be resolved before logging it.
-
Form a hypothesis about the cause of the error, and whether there is an allowed fix which you have not yet tried that will resolve it. Even if the error seems unclear or opaque, form a hypothesis anyway, make your best guess, based on the error message, field names, and schema context.
-
If allowed fixes are available to be tried as per the "Allowed fixes" section,
- attempt the fixes in order of preference as outlined,
- Immediately after applying a fix, append a structured entry to
fix-log-<spec-name>-<current-date>.md
with the format:
## Fix: <summary of the fix>
**Timestamp**: <timestamp>
**Action**: Fix applied
**Fix**: <description>
**Reason**: <why you think this fix should resolve the issue>
**Logs**: <log snippet showing the issue that prompted the fix>
- Then go back to Step 1 of this phase and run the loop test again to see if the fix resolved the issue.
-
However if there are no more allowed fixes left to try and the issue remains unresolved, consider the issue addressed and log a "Specmatic Bug" entry in the following format:
## Unfixable: <summary of the bug>
**Timestamp**: <timestamp>
**Action**: Dropping issue as unfixable
**Classification**: Specmatic Bug
**Reason**: <Why you concluded that this was unfixable>
**Logs**: <log snippet showing the issue that prompted this conclusion>
-
When logging a failed loop run, include the most specific inner contract error found in the response body. Do not log only the outer status mismatch if the response body contains a deeper rule violation.
For example, log:
REQUEST.QUERY.path.to.something <error>
or
RESPONSE.BODY.path.to.something <error>
or
instead of:
confirm-payment-source returned 400 instead of 200
.
Phase 3: Report on auto-fix phase
Prepare a console report for the user. Do not mention mechanics such as test execution details, mock startup details, or pass/fail runtime chatter.
- Read fix log at
fix-log-<spec-name>-<current-date>.md
in order to identify Specmatic bugs and applied fixes. Do not depend on memory.
- If it contains
Classification**: Specmatic Bug
, the final response MUST include a section.
- If it contains , the final response MUST include a section.
- Specmatic bugs section should be followed by the Fixes applied section if both exist, else just the one that exists should be printed.
- If any Specmatic bugs exist, the final response MUST say to send:
- original spec
- updated spec
- fix log
- Do not substitute validation/test summaries for these sections.
The format for the Specmatic bugs section is:
Specmatic bugs:
- <bug title>: <brief description of the bug>
- <bug title>: <brief description of the bug>
Please send the following files to the Specmatic team for analysis:
- <original spec file>
- <updated spec file>
- <fix log file>
The format for the list of fixes applied is:
Fixes applied:
- <item fixed>: <brief description of the fix>
- <item fixed>: <brief description of the fix>
Phase 4: Address load time errors and warnings
Note: Even if the user asks to fix the spec, they really want control over what gets fixed, and how the fix is done. So Phase 4 validation may be run automatically, but Phase 4 fixes must never be applied automatically.
- Run the validate command on the
<spec-name>-updated.<ext>
spec.
- If there are no errors or warnings, report to the user that the spec is now valid and can be used for testing and mocking with Specmatic, and end the process.
- If there are errors or warnings, do NOT fix any errors or warnings. Do not infer approval from the original request, or from prior sessions. Only proceed if the next user message explicitly approves fixing the listed Phase 4 issues or requests specific fixes. Instead, list them out (not just the ids, include the descriptions if Specmatic provided it), and ask the user if they want to address spec defects on the copy.
- For each load-time error or warning that the users wants to address:
- Classify and log the error following the same classification and logging contract as in Phase 2.
- Apply allowed fixes to the copy based on the error messages and classification, following the same guidelines for allowed fixes as in Phase 2.
- Repeat this phase until all actionable load-time errors are addressed.
- If you conclude that an issue cannot be fixed using allowed fixes, log it as a "Specmatic Bug" following the same contract as in Phase 2, consider it addressed.
Disallowed fixes
The following changes are not allowed:
- Widening a JSON schema from
additionalProperties: false
to additionalProperties: true
- Dropping a property from a JSON schema completely. At most you may make properties optional or mandatory
- Changing descriptions, titles, summaries, and other metadata (these are never parsed by Specmatic)
- Avoid changing the request or response datatype
- Remove a security scheme from an API
Allowed fixes
Guidelines for fixing constraints and examples:
- If you feel that an obscure must-fix error is related to a constraint (datatype is still correct but constraint such as regex, minLength, maximum, etc is broken), attempt the following fixes in order of preference:
- If the constraint syntax is invalid, fix the syntax and re-attempt the loop test.
- If the loop test fails, then based on the field name and schema context, make an educated guess, update the constraint, and re-attempt the loop test.
- Final option, remove the constraint and re-attempt the loop test.
- When an enum does not match the schema constraint, remove the constraint.
- When an example value does not match schema, update the example value to something meaningful based on the datatype and field name.
- If you get an error saying expected X status but got 400, it means the mock didn't like the request. You will find details adjacent to that message saying why the 400 was returned by the mock.
- For regexes, Specmatic uses the dk.brics.automaton.Regexp library, which supports a specific regex syntax found in
references/briks-automaton-regex-rules.md
.
When a scenario fails with HTTP status mismatch
If a loop test reports
R0002: HTTP status mismatch
, do not classify it as opaque until you inspect the request/response transcript for
that same failed scenario.
Specmatic often prints the detailed contract failure inside the mock response body, before the final
summary.
Search upward from the failed scenario or from
Specification expected status ... but response contained status 400
for the nearest
preceding:
X-Specmatic-Result: failure
Error from contract <spec-file>
- , , , or any schema path
If the response body contains a concrete
or
path, treat that as the primary failure, not the outer
status mismatch.
Example:
- Outer error:
Specification expected status 200 but response contained status 400
- Response body detail:
>> REQUEST.BODY.payment_source.card.network_token.cryptogram
- Classification: Spec Issue if the path points to a concrete schema constraint that can be fixed.
Logging Contract (Mandatory)
- After every loop test run: append one loop-run entry to
fix-log-<spec-name>-<current-date>.md
.
- After every applied fix: append one fix entry to
fix-log-<spec-name>-<current-date>.md
.
- For every observed or : append one matching classification entry to
fix-log-<spec-name>-<current-date>.md
.
Available scripts
Note:
- Scripts for MacOS / Linux require bash
- Scripts for Windows require powershell
Scripts for MacOS / Linux (requires bash)
- - runs the loop test
- - validates a spec
Scripts for Windows (requires powershell)
scripts/run_loop_test.ps1
- runs the loop test
scripts/validate_spec.ps1
- validates a spec
When running these scripts, resolve the path from the fix-openapi-spec skill bundle directory.