Stop AI from Treating Skills as Background Reading
The real challenge is to make every AI model and AI agent work correctly and consistently. The hardest fixes are mechanical: an executable skill contract, a public status checklist, and early gates.
AI agents today finish tasks through a workflow — a composition of multiple skills. The hidden assumption is that if the model can read each skill, it will execute it. In practice the failure is rarely “no workflow”; it is “some skill inside the workflow was never executed correctly.”
Background / Problem
While evaluating XiaoMi MiMo V2.6 on an app-release task (itself a workflow of several skills), the model never invoked the skill that generates release notes. It read the skill, treated it as background context rather than a plan to run, and shipped a broken release.
When asked why, the model’s own analysis was candid: it had treated the skill as background reading, not a test plan. It checked the wrong artifacts, re-confirmed its own blind spots through three review rounds, and delivered a release that never actually passed the release-page gate.
Approaches considered
| Approach | Pros | Cons | Best for |
|---|---|---|---|
| Prompt “execute the skill” | Zero infra, one-line change | Not model-agnostic; still optional in the model’s mind | One-off tasks on a known model |
| Post-task verification | Catches real defects | High overhead; rework after the fact | Requirements easy to verify at the end |
| Sub-agent review | Catches issues during execution | Reviewer inherits the same limited context | When a well-scoped second pair of eyes exists |
| Executable guardrails in the workflow | Mechanical; model-agnostic; early | Needs up-front gate design | Multi-step, high-stakes, no-skip tasks |
No single fix covers every failure mode.
Three guardrails
Guardrail 1 — Make the skill contract executable
The skill said “prove the note in the same stub run” and “check the store release page itself” — instructions a human can ignore. Make them mechanical:
- Read-level gate: wire a publish dry-run assertion into
build.py testor CI. Any asset command missing--notesfails the build. The contract becomes something the machine enforces. - Review-scoping rule: reviewers receive the skill’s checklist as the source of truth. The model’s own findings are appended, never replace it.
Guardrail 2 — A public status checklist for multi-step flows
In a 9-step implementor skill (Load Context → Draft Plan → Create Worktree → Self-Verify → Independent Review → Summarize → Push PR → Update Status → Cleanup), DeepSeek, Qwen, HY, and GLM — across CodeBuddy, Trae, and OpenCode — all randomly skipped steps. The fix: list every step and track its status in a checklist. Once the checklist is in the prompt, skipping a step requires the model to explicitly mark it done or not-applicable, which is hard to do silently.
Guardrail 3 — Gate early, not after the fact
Post-task checks catch problems after the work is done. The robust guardrails sit at the decision point: a CI check before the release goes green, a dry-run assertion before the publish command runs, a checklist update before the next step unlocks.
Results / signals to track
No hard metrics yet, but three signals are worth collecting:
- Release failure rate before and after adding explicit execution language and read-level gates.
- Step-skip rate across the 9-step flow, measured by checklist completion.
- Cross-model consistency of the same prompt on MiMo / DeepSeek / Qwen / HY / GLM.
What I’d do differently
- Never assume “the model read the skill” equals “the model ran the skill.”
- Put the checklist inside the prompt, not in a separate document.
- Make gates mechanical, not disciplinary.
Reusable checklist
- Is every skill explicitly framed as an action to execute, not a document to reference?
- Does the prompt contain a status checklist listing every required step?
- Is there an early gate (CI, dry-run, assertion) before the final deliverable?
- Are reviewers scoped by the skill’s own checklist, not by prior findings?
- Are post-task checks reserved only for requirements that cannot be gated earlier?
Conclusion
An AI model can read a skill inside a workflow and still not execute it; the root cause is treating it as background context. The strongest fixes are mechanical: an executable skill contract, a public status checklist, and early gates. The smallest first step is to add one line to your workflow prompt telling the model it must call the skill — and making that call checkable.