#190 — Agent skills are context management, not magic
July 9, 2026·10 min read
Contents
The bottom line: Skills don't make your AI agent smarter. They control what it knows, when it knows it, and whether it can prove the job is done. The model is not the problem. Your context strategy is.
How it actually works
A skill is a folder — a SKILL.md plus optional scripts and reference files — not a prompt hack. At startup, the model loads only a name and one-line description per skill. Full instructions load only when a task matches. Supporting files and scripts only load if the instructions explicitly call for them.
This three-stage loading is called progressive disclosure. Without it, installing dozens of skills would blow out the context window before you even ask a question. With it, the model carries a lightweight catalog and pulls in detail on demand.
In practice: You ask your agent to review a migration PR → it scans every skill description → matches db-migration-review → loads the full SKILL.md → runs scripts/verify.sh against your actual migration files → returns a structured risk assessment with a pass/fail verdict. If the script fails, the model interprets the error and decides whether to retry, escalate, or report. That failure-handling loop is where most of the real complexity lives.
The thing most founders get wrong
CLAUDE.md / AGENTS.md / Copilot custom instructions are for always-on context: stable repo-wide rules like project layout, build commands, coding conventions, and what "done" means. Skills are for on-demand context: specialized workflows that should only appear when relevant.
These are not interchangeable. Putting specialized workflows into your always-on file drowns the model in instructions that have nothing to do with the current task — and routing accuracy degrades quietly. Keep your always-on file under ~200 lines. If you're above that, you have skills waiting to be extracted.
The other extension layers solve different problems:
- MCP — external tools and live systems
- Hooks — deterministic enforcement (formatting on save, blocking dangerous writes); not suggestions to the model, guarantees
- Subagents / worktrees — isolated or parallel work so exploration noise doesn't pollute main context
None of these replace a well-written skill. They're complementary.
When to build one
If you're rewriting the same prompt, correcting the same workflow, or running the same checklist more than twice — that's a skill. Common candidates:
- PR review and migration planning
- Log triage and release notes
- Deployment pipelines
- Recurring ops: standup posts, dependency audits, ticket creation, orphan resource cleanup
Don't extract too early. Get one task working cleanly first, then formalize it. Anthropic's own guidance: build evaluations first, baseline the model without the skill, then add only the minimal instructions needed to close real gaps. Start narrow, then expand — otherwise you're just productizing your confusion.
Four skill types — pick one per skill
The best skills fit cleanly into one category. When a skill straddles two or more, that's usually a sign it should be split.
| Type | What it does | Risk level | Notes |
|---|---|---|---|
| Knowledge | Overrides model defaults with your patterns, gotchas, internal library quirks | Low | Safe to auto-fire |
| Execution | Orchestrates tools/scripts into a repeatable workflow (deploy, scaffold, migrate) | High | Should require explicit invocation |
| Verification | Proves output is correct — Playwright, test harness, assertions | Low | Highest-leverage investment per hour |
| Automation | Handles recurring business ops (standups, recaps, audits) | High | Store previous run logs so the model can track what changed |
Risk level drives invocation policy: a knowledge skill can auto-fire safely; a deployment skill should never auto-fire.
Most real workflows combine two or three types. A migration review might load a knowledge skill for your DB conventions, an execution skill for the review workflow, and a verification skill to run the checks.
Writing a skill that actually works
The description is the routing key. The model scans every skill description at startup to decide what matches. Vague descriptions trigger on everything and select nothing.
❌ "Helps with code review, deployment, and documentation"
✅ "Review PostgreSQL schema migrations and rollback safety. Use when creating, editing, or validating SQL migrations or rollback plans."
The five rules:
- One skill = one job. Split anything that straddles categories — a skill that tries to do six things does all of them poorly
- Skip the obvious. The model already knows how to write code; focus on what it will get wrong in your specific codebase
- Build a gotchas section and keep adding to it. Every time the model fails using your skill, add the failure to the list. This is the part that compounds
- Start instruction-only, then add scripts only when they earn it. A script earns its place when it replaces fragile model-generated steps with reliable execution, saves tokens, or removes ambiguity
- Write procedures, not prose. Decision points, verification steps, escalation paths — the more you encode, the less the model improvises
Multi-skill composition
This is where things get interesting — and fragile.
Routing competition is the core problem. When api-reviewer, code-quality-checker, and pr-review-standards are all installed, "review this PR" could plausibly match all three. If descriptions overlap, the model guesses. Fix this by making descriptions mutually exclusive: one skill reviews migrations, another reviews API contracts, a third enforces style.
Ordering isn't guaranteed. When multiple skills should activate for one task, the model decides sequence. You can influence this by making dependencies explicit in your skill's workflow section ("Before running this, check whether schema-conventions applies") — but you cannot guarantee it.
Composition is not deterministic. Two runs of the same prompt can trigger different skill combinations depending on phrasing, context window state, or how many other skills are loaded. You're shaping probabilities, not designing a deterministic system. This is why active skill count should stay low — use role-based bundles when you need more. More skills ≠ more capability.
Evaluating if your skill actually works
"Test 3–5 queries" is technically correct and practically insufficient. A real evaluation needs three things:
1. Before/after measurement. Run the model on the same task without the skill and record what happens. Then deploy and compare. Measure: success rate, retries, manual corrections, time-to-completion. A skill that produces correct output but takes 3x as long because it loaded unnecessary context is still a bad skill.
2. Trigger testing. Separately from output quality — does the skill fire when it should? Stay quiet when it shouldn't? Handle ambiguous prompts gracefully? Output quality metrics will never catch a misfiring skill.
3. Observability. Log skill invocations. Track which skills activate, how often, and whether users override or correct the result. Use a PreToolUse hook to log skill usage. A spike in corrections is the first sign something is misfiring.
Why skills fail in production
The model isn't getting dumber. Your context strategy is broken. Four root causes:
- Routing ambiguity — overlapping descriptions cause the wrong skill to fire; output looks plausible but answers the wrong question; hardest failure to detect without trigger testing
- Context overload — too many active skills or a bloated always-on file; the model's outputs degrade quietly as it competes for attention across too much material
- Hidden dependencies — the skill assumes packages are installed, services are running, or credentials are configured — none of it declared; script fails at runtime with an error the model can't interpret; self-contained scripts with explicit dependencies aren't pedantry, they're portability
- Missing verification — skill tells the model what to do but not how to prove it worked; model says "done," you find the bug hours later
Security: the thing nobody mentions
Skills are a real attack surface. A malicious skill executes with the same permissions as your agent — hidden instructions, embedded prompt injection, scripts with full shell access. This is documented, not theoretical.
Treat third-party skills the way you treat third-party code: read them before you install them, and don't give them more access than they need.
Tool comparison (Claude Code vs. Codex vs. Copilot)
| Claude Code | Codex | GitHub Copilot | |
|---|---|---|---|
| Config file | CLAUDE.md + auto memory | AGENTS.md | Custom instructions |
| Skill directory | .claude/skills | .agents/skills | .github/skills, .claude/skills, .agents/skills |
| Standout features | context: fork, !command shell injection, Chrome integration, hooks | Cloud tasks, worktrees, codex exec for CI, /plan /review slash commands | Cleanest cross-agent portability; treats skills as open-standard infrastructure |
| Parallel agents | Built-in subagents | Explicit — you ask for them | N/A |
| Complexity | Richest — most rope to hang yourself | Lean, orchestration-first | Simplest onramp |
Advanced patterns (for when you're ready)
!commandsubstitution (Claude Code) — injects live shell output into the prompt before the model sees it; useful for diagnostics, dangerous if the command produces huge outputcontext: fork(Claude Code) — runs the skill in an isolated subagent; excellent for PR summarization and research-heavy workflowsagents/openai.yaml(Codex) — sets invocation policy and tool dependencies; worktrees let you schedule recurring skill-backed tasks in the background- Persistent state across runs — skills can store data in append-only logs, JSON files, or SQLite; in Claude Code, write to the
CLAUDE_PLUGIN_DATApath, not the skill directory itself, which gets wiped on upgrade
The practical template
my-skill/
├── SKILL.md
├── references/
│ └── checks.md
└── scripts/
└── verify.sh
SKILL.md skeleton:
---
name: db-migration-review
description: >-
Review PostgreSQL schema migrations and rollback safety.
Use when creating, editing, or validating SQL migrations
or rollback plans.
---
# When to use
Schema changes, rollback planning, or migration reviews.
# Inputs
- Migration files or PR diff
- Target service and database version
# Workflow
Open changed migration files and related application code.
Classify: additive, destructive, backfill, index, constraint, rename.
Check forward path, rollback path, and operational risk.
Run `scripts/verify.sh`. Summarize risks, required fixes, approval status.
# Verification
- Run `scripts/verify.sh`
- Confirm tests pass
- Confirm rollback path exists or explicitly state why not
# Escalate when
- Data loss is possible
- Rollback is irreversible
- Runtime and migration ordering is unsafe
Keep edge-case lore in references/. Add a script only if it materially improves reliability.
The real bottom line
If your agent is unreliable, the problem is almost never the model. It's what the model knows, when it knows it, and whether it has a way to prove it succeeded. Skills are how you fix that — not by giving the agent more power, but by giving it less noise and more structure.
Skills earn their overhead when specialized workflows pile up and your always-on config bloats past ~200 lines. Until then, stay lean. If you only have two or three workflows worth formalizing, keep them in CLAUDE.md and skip the abstraction entirely.
Everything else is vendor paint.
Frequently asked questions
What's the difference between CLAUDE.md and a skill file?
Think of CLAUDE.md as your team handbook — always loaded, repo-wide rules, build commands, coding conventions, and what 'done' means. Skills are specialist contractors: they sit idle until the right task comes in, then load their full context. Mixing them is the #1 mistake. Once founders split the two, routing accuracy and output quality both improve measurably because the model stops competing for attention across instructions irrelevant to the task at hand.
How many agent skills should I build before it becomes counterproductive?
There's no magic number, but a reliable heuristic: if your CLAUDE.md or AGENTS.md exceeds ~200 lines, you have skills waiting to be extracted. Below that threshold, keep workflows in your always-on config and skip the abstraction entirely. Beyond it, the routing surface area compounds faster than the capability gains. Anthropic's own guidance recommends keeping active skill count low and using role-based bundles rather than growing an unbounded catalog.
Can I use the same skill file across Claude Code, Codex, and GitHub Copilot?
Yes — the shared format (name, description, SKILL.md) is intentionally cross-agent. VS Code explicitly treats skills as open-standard infrastructure, recognizing project skills from .github/skills, .claude/skills, and .agents/skills. Core instructions are portable; client-specific metadata like invocation policy, tool restrictions, and deployment configs are where portability ends. Plan for a thin adapter layer per platform, not a full rewrite.
My AI agent keeps picking the wrong skill — how do I fix routing failures?
The problem is almost always overlapping trigger phrases in your descriptions. The model scans every skill's description at startup and guesses when multiple descriptions compete for the same words. The fix is mutual exclusivity: one skill owns 'migrations,' another owns 'API contracts,' a third owns 'style review.' Never share trigger language across skills. If three skills all contain 'review,' the model is guessing — and it'll guess wrong roughly as often as right.
How do you write a skill description that actually triggers correctly?
Treat the description as a routing key, not a marketing tagline. It needs to answer two questions: what task does this skill handle, and what should it NOT handle. A description like 'Helps with backend code' triggers on everything and selects nothing. Compare: 'Review PostgreSQL schema migrations and rollback safety. Use when creating, editing, or validating SQL migrations or rollback plans.' One is a category; the other is a contract. Longer, specific descriptions outperform short, clever ones every time.
What's the ROI of building a verification skill vs. just trusting the model output?
A verification skill is the highest-leverage hour you'll spend on your agent stack. Without one, the model says 'done' and you find the bug hours later in production. With one, the skill runs assertions, test suites, or Playwright checks programmatically before reporting success. One engineering week spent building solid verification skills eliminates a recurring class of silent failures that compound across every future run — making it one of the few infrastructure investments with immediate, measurable payback.
Are agent skills a security risk I should worry about?
Yes — and most teams don't. A malicious skill executes with the same permissions as your agent: full shell access, file writes, API calls. This isn't theoretical; prompt injection via untrusted skill files is a documented attack vector. The rule is simple: treat third-party skills exactly like third-party code. Read them before installing. Scope permissions to the minimum required. Never auto-fire execution or automation skills. An unreviewed skill from a public library is a supply chain risk, not a productivity shortcut.
How do agent skills compare to just writing better system prompts?
A great system prompt and a skill can produce identical output — until they can't. The difference is consistency at scale, not capability. A prompt depends on you remembering to include the right context every time. A skill loads it automatically when the task matches, routes to the right workflow, and enforces verification without you thinking about it. For one-off tasks, a good prompt wins on simplicity. For repeated workflows with 5+ runs per week, the compounding discipline of a skill outperforms even your best prompt.
Can agent skills run background tasks or scheduled automation?
In Codex, yes — cloud tasks and worktrees let you schedule recurring skill-backed workflows in the background without an open session. Examples include dependency audits, orphan resource cleanup, weekly standup summaries, and ticket triage. For these to work reliably across runs, the skill should append results to a persistent log (e.g., standups.log) so the model can read its own history and track what's changed. Claude Code supports claude -p for non-interactive execution in CI/CD pipelines with the same pattern.
What's the right way to handle a skill that fails mid-workflow?
The skill should have an explicit escalation policy — not just a verification step. There's a meaningful difference between 'the script returned an error' and 'data loss is possible.' Good skills encode these branches: retry if the failure is environmental, report-and-stop if the rollback path is unsafe, escalate to a human if the risk is irreversible. Models are bad at improvising escalation decisions. When you leave it undefined, you get a confident 'I completed the task' followed by a silent failure. Define the failure tree in the skill, not in your head.
How do I test whether a new skill actually improves agent performance?
Run a proper before/after benchmark: same task, same inputs, no skill — record success rate, retry count, and time-to-completion as your baseline. Then deploy the skill and run the same set. Separately, test trigger accuracy: does the skill fire on the right prompts and stay quiet on irrelevant ones? Output quality metrics alone will never catch a skill that triggers on the wrong task. Log invocations with a PreToolUse hook and watch for correction spikes — they're the first signal something is misfiring.
What's the difference between agent skills and MCP (Model Context Protocol)?
Skills and MCP solve different problems and are meant to be used together. MCP connects your agent to live external systems — databases, APIs, Slack, GitHub. Skills encode how to use those tools in your specific workflows: in what order, under what conditions, and how to prove the result is correct. MCP is the pipe; the skill is the procedure. A deployment skill might invoke an MCP-connected cloud tool as one of its steps. Replacing one with the other is like replacing your plumbing with a plumber.
When should I use a Claude Code hook instead of a skill?
Use a hook when you need a guarantee, not a suggestion. Hooks enforce deterministic behavior — formatting on every save, blocking writes to protected files, logging every tool call — regardless of what the model decides. Skills route context to the model and guide its reasoning, but the model can still deviate. If the behavior must happen 100% of the time with no exceptions (e.g., 'never write to production config'), that's a hook. If the behavior is a workflow the agent should follow when the context matches, that's a skill.
What's the difference between a subagent and a skill in Claude Code?
A skill is a set of instructions; a subagent is an isolated execution context. Claude Code's context: fork directive runs a skill inside a sandboxed subagent — separate context window, separate tool permissions, separate conversation history. This is critical for research-heavy or exploratory work where you don't want noise contaminating your main session. Think of the skill as the job description and the subagent as the contractor hired to do it in a clean room. Most skills don't need forking; PR summarization and multi-step research tasks usually do.
How do agent skills persist data across multiple runs?
Skills can read and write persistent state using append-only logs, JSON files, or SQLite databases. In Claude Code, store this data in ${CLAUDE_PLUGIN_DATA} — not in the skill directory itself, which will be wiped on upgrade. In Codex, worktree environments persist between scheduled runs. A recurring standup skill, for example, appends each session's output to standups.log; on the next run, the model reads that file to understand what's changed and avoid repeating itself. Without persistent state, automation skills are stateless and can't track progress over time.
Do agent skills work in CI/CD pipelines?
Yes — and this is one of the most underused applications. Claude Code's claude -p flag enables fully non-interactive execution, which makes skills compatible with GitHub Actions, GitLab CI, and any other pipeline runner. A PR review skill can fire on every pull request, run your verification script, and post a structured risk summary as a review comment — with no human in the loop. Codex's codex exec supports the same pattern. The key requirement: your skill's scripts must be fully self-contained, with no prompts for user input and explicit dependency declarations.
What's the difference between knowledge skills, execution skills, and verification skills?
The three most useful categories map to what the model knows, what it does, and how it proves it. Knowledge skills override defaults — your internal library's gotchas, your design system's patterns, the edge cases the model consistently gets wrong in your codebase. Execution skills orchestrate tools into repeatable procedures — deploy pipelines, migration generators, scaffold workflows. Verification skills run programmatic checks to prove the output is correct before reporting success. Most real workflows layer all three: a knowledge skill for context, an execution skill for the procedure, and a verification skill as the gate.
Why does my AI agent produce inconsistent results even when I'm giving it the same prompt?
Inconsistency is almost always a context management problem, not a model problem. Three common causes: routing ambiguity (the wrong skill fires depending on subtle phrasing variations), context overload (too many active skills competing for attention), and non-deterministic composition (two runs of the same prompt can trigger different skill combinations depending on context window state). The fix is tighter skill descriptions, lower active skill count, and a PreToolUse hook that logs exactly which skills fired. Once you can see the routing, you can fix the inconsistency.
Keep reading
#191 — Canada's AI strategy: The founder's playbook
Ottawa recently dropped its national AI playbook, and if you're building an AI-native company, there's real money and market signal buried in the policy-speak.
#192 — Cold Take: Canada doesn't have a capital problem, it has a mindset problem
The US bets on grants and matched equity to keep early failure cheap; Canada bets on loans that make early failure personally expensive.
#193 — Cold Take: Canada's Deep Tech "valley of death" is still a problem
BDC's Deep Tech Fund collapse highlights Canada's misaligned investment structures and the worsening gap between research and commercialization.