Here's the failure nobody warns you about when you build your first agent skill: you write it, it's good, you ship it — and the agent never uses it. No error. No sign. It just quietly falls back to its base behavior and your skill sits in the folder collecting dust. You paid to write something the model decided not to load.

The instinct is to rewrite the body, add more detail, make the instructions better. That's fixing the wrong half. Claude decides whether to load a skill from its description — one line — and the body isn't even in context when that decision happens. So the thing to measure isn't whether the skill's instructions are good. It's whether the skill triggers. That's a trigger eval, and it's the cheapest high-leverage test in the whole skills workflow.

The 30-second version: Build a labelled corpus of 30–50 prompts, each tagged with the skill that should fire (including hard negatives that should fire nothing). Run each in a clean session, record which skill actually loaded, and compute recall (did it fire when it should?) and precision (did it stay quiet when it shouldn't?). Fix the description, not the body, until both are high — then gate it in CI so a future description edit can't silently regress the trigger.

New to skills? Start with the SKILL.md build guide. Already shipping them? This pairs with how to version and roll back a skill — evals catch the regression, version control lets you undo it.

Two different evals, and most people skip the first#

There are two questions you can ask about a skill, and they need two different tests:

  1. Does it trigger? Of the prompts where this skill should fire, does it? Of the prompts where it shouldn't, does it stay quiet?
  2. Once triggered, is the output good? Did the skill's instructions produce a better result than base behavior?

Almost everyone jumps straight to (2) — grading outputs — and never runs (1). But a trigger failure makes an output eval meaningless: you can't grade the output of a skill that never loaded. Worse, output evals are expensive (a model has to grade each result) while trigger evals are nearly free (you just check which skill woke up). Run the cheap, foundational one first.

Step 1 — Build a labelled corpus, hard negatives included#

Write 30 to 50 prompts a real user might send, and label each with the skill you expect to fire — or none. The count matters: enough to show a real signal, few enough to hand-label in one sitting and re-run in seconds.

The part people get wrong is only writing positive examples. A corpus of prompts that should all trigger the skill will cheerfully report 100% recall while the skill misfires on half your actual traffic. You need hard negatives — prompts that look adjacent but should trigger nothing, or a different skill.

{"prompt": "summarize what changed in my working tree", "expect": "summarize-changes"}
{"prompt": "write me a commit message for these edits", "expect": "summarize-changes"}
{"prompt": "what does this repo do?", "expect": "none"}
{"prompt": "deploy the app to production", "expect": "deploy"}
{"prompt": "explain the difference between git merge and rebase", "expect": "none"}

That third and fifth line are the ones that earn their keep — an overbroad description like "helps with git" will grab them, and only a hard negative will tell you.

Step 2 — Run each prompt in a clean session#

Reproducibility is the whole game here. If session A has your last conversation still in context and session B doesn't, the same prompt can trigger differently, and your eval measures noise. Start each prompt fresh, capture which skill activated, and record it next to the label. You can observe the active skill from the skill listing / /status, or capture it programmatically if you're driving the agent through the SDK.

The result you're building is a table of (prompt, expected, actual) rows. That's all a trigger eval is.

Step 3 — Score precision and recall (and read the confusion matrix)#

Two numbers turn that table into a decision:

The scoring is deterministic — no model in the loop:

// rows: [{ prompt, expect, actual }]
function scoreSkill(rows, skill) {
  const tp = rows.filter(r => r.expect === skill && r.actual === skill).length;
  const fn = rows.filter(r => r.expect === skill && r.actual !== skill).length;
  const fp = rows.filter(r => r.expect !== skill && r.actual === skill).length;
  const recall    = tp / (tp + fn || 1);
  const precision = tp / (tp + fp || 1);
  return { recall, precision, tp, fn, fp };
}

When precision is low, don't stop at the number — look at which skill fired instead. That's the confusion matrix, and it tells you whether the problem is one overbroad description or two skills fighting over the same prompts (in which case you fix both descriptions so their scopes stop overlapping). The compare table above maps each symptom to its fix.

Step 4 — Fix the description, not the body#

Every trigger fix is a description edit:

This is precisely the lever Anthropic's own skill-creator tooling pulls: it optimizes the description and reported improved triggering on 5 of 6 public skills. Rewrite, re-run the corpus, watch the two numbers move. Because the eval is cheap, you can do this loop five times in the time one output eval takes.

Step 5 — Gate it in CI so the trigger can't regress#

A trigger assertion is deterministic — "did skill X load for prompt Y" needs no model call and never flakes — so it belongs in CI like any other test. Set a floor (say, recall ≥ 0.9 and precision ≥ 0.9 on the corpus) and fail the build below it.

This is where trigger evals and skill versioning lock together. Editing a description is a breaking change to when a skill fires — but it's invisible, because nothing errors. A CI trigger eval turns that invisible risk into a red check: the eval catches the regression before it ships, and version control lets you revert if one slips through. Write the skill once; let the eval keep it honest every time you touch the one line that decides whether it runs at all.