Prompting & Structured Output

1 · From asking to specifying: the three levels

▶ Watch: the same request at three levels · 3 min

Consultants already know this problem from delegation. Tell a junior colleague "write up the vendor meeting" and you get their guess at what you wanted: maybe a narrative, maybe bullets, maybe two pages when you needed five lines. Hand the same person a template with five labeled boxes and you get the five boxes, filled in, every time. Nothing about their talent changed. The request changed.

Claude works the same way, and the same fix applies. Every request you make sits at one of three levels:

Each level costs more to set up and buys more consistency. The skill this module teaches is picking the level the work deserves, then writing it. You have been asking all course. The other two levels are the difference between work you fix and work you hand off, and the exam gives them a whole domain: 20 percent of the questions, six task statements, walking this exact ladder.

2 · Explicit criteria: the rubric the model can hit (TS 4.1)

An ask leaves quality undefined. A specification defines it, as criteria: statements about the output that are either met or not met. "Make it good" is not a criterion, because two people could disagree about whether it was hit. "Every bullet names an owner and a date" is a criterion, because nobody can disagree.

The same meeting note, at both levels:

> summarize the pbm renewal meeting note

The summary comes back readable and organized however Claude organized it. Some runs lead with the discussion, some with the decisions. Nothing is wrong, and nothing is dependable either.

> summarize the pbm renewal meeting note. criteria: five bullets or fewer.
  every bullet names who committed to what, with the date if one was said.
  anything unresolved goes under a separate "Open" heading. no other headings.

Now the output has a definition of done, and you can check it in ten seconds: count the bullets, scan for names and dates, look for exactly one extra heading. When a criterion is missed, you say which one, and the fix is one turn.

Is it a criterion? Judge each, then flip.

The discipline, in four rules:

• Each criterion is checkable. Counts, names, formats, and orderings qualify. "Concise" and "professional" do not; replace them with the count or the format you actually mean.
• State the shape as well as the content. How many, in what order, under which headings. Shape is where two runs of the same prompt drift most.
• Say what to do with the gaps. "If no date was said, write none stated" gives Claude a legal move. Without it, a plausible date is sometimes the model's best guess at being helpful.
• The colleague test. Anthropic's own prompting guidance calls this its golden rule: show your prompt to a colleague with minimal context and ask them to follow it. If they would be confused, Claude will be too. The same docs give criteria a memorable frame: good success criteria are specific, measurable, achievable, and relevant, and "the model should classify sentiments well" is their own example of a bad one.

Criteria name categories, never confidence levels. The exam grounds this in a scenario worth knowing on its own merits: automated review, where the output is a list of findings and the failure mode is false alarms. "Check that the comments are accurate" is the vague version. "Flag a comment only when the behavior it claims contradicts what the code actually does" is the criterion, and the difference is that the second names the exact category of finding to report. What does not work is turning adjectives into dials: "be conservative" and "only report high-confidence findings" fail to improve precision, because the model's confidence is not a setting you can adjust with a word. And false alarms are not cosmetic: when one category of finding cries wolf, readers stop trusting the accurate categories too. The working move when a category runs hot is to turn it off while its criteria get rewritten, which keeps the trustworthy categories trusted.

Why this earns its place in your week: criteria are reusable. Written once, they belong in a skill file, and every future summary comes back in the same shape without retyping a word. The specification is an asset, and the ask never is.

3 · Show it examples: few-shot prompting (TS 4.2)

Few-shot prompting is the practice of including worked examples in the prompt. The name is jargon with a plain meaning: a "shot" is one example, so few-shot means showing a few. Instructions tell the model about the output. Examples show the output, and they carry a hundred small decisions (punctuation, tone, ordering, how dates get written, what gets left out) that instructions never think to state.

The everyday version, in your own repo:

> write this week's status update from my notes in 30 Meetings. match the
  format of these two exactly:

  WEEK OF AUG 11 · PBM RENEWAL
  Done: rate tables reconciled (owner: consulting team)
  Next: legal review of clause 7, due Aug 22
  Risk: vendor slow on data feed. escalated Aug 13.

  WEEK OF AUG 4 · PBM RENEWAL
  Done: kickoff complete, access granted
  Next: rate table reconciliation
  Risk: none this week

The description of that format would run a paragraph and still miss the middle dot in the header and the lowercase "due." The two examples carry all of it. On how many: the official prompting docs suggest three to five varied examples, and the exam guide's own numbers are two to four targeted ones. The honest rule underneath both: a few examples chosen to cover the cases that actually vary beat a pile of examples that all look alike.

Examples have a second job beyond format, and the exam leans on it: showing how to handle the ambiguous case. An example that demonstrates the borderline call, with a line of reasoning for why it went the way it went ("this is discussion, not a commitment, because nobody is named"), teaches judgment the model can generalize to cases you never showed it. And in extraction work, examples drawn from differently shaped source documents measurably cut both invented fields and empty ones, because the model has seen what each shape of input should produce.

Three rules keep examples working for you:

• Examples must agree with your instructions. When an instruction says five bullets and your pasted example has seven, the model has to pick which to believe, and either choice is a coin flip you built into your own prompt. Check for disagreement before blaming the model.
• Vary the examples where reality varies. If some weeks have no risk, include an example of a no-risk week ("Risk: none this week" above is doing that job), so the model sees what that case looks like instead of inventing a risk to fill the slot.
• Examples belong in skills. Pasting examples every time works. Putting them in a skill file once works better: the skill's instructions carry the criteria, the skill's examples carry the format, and every session after that gets both for free. This is most of what separates a skill that produces consistent output from one that produces suggestions.
• Examples teach only what the document shows. A worked example that includes knowledge from outside its source (an owner's full name that a person happened to know, a date confirmed in a hallway) teaches the model that importing outside knowledge is the norm, and field teams have learned that the expensive way: their corrected examples taught the model to invent. Keep every example strictly derivable from the source it claims to come from.

4 · The contract: structured output (TS 4.3)

First, the words. JSON is a plain text format for structured data: labeled fields and their values, wrapped in braces. It is what software uses to hand data to other software, and one commitment from a meeting note looks like this in it:

{
  "commitment": "legal review of clause 7",
  "owner": "vendor counsel",
  "due": "2026-08-22",
  "source_note": "2026-08-11 pbm-renewal.md"
}

A schema is the written contract for that shape: which fields must exist, what type each holds, and whether extras are allowed. For the record above:

A schema is checkable by software, which is the whole point: a program can look at a piece of JSON and say yes, this conforms, or no, and say exactly where it fails. Why a consultant cares: the moment output stops being read by you and starts being consumed by something else (a tracker, a spreadsheet, a dashboard, a next step in an automation), format failures stop being cosmetic. A missing field does not look wrong; it breaks whatever reads it, at whatever hour it reads it. The contract level exists for exactly that handoff.

In your own chair, the contract move is asking for output to land as a structured file with named columns, and stating the schema in plain words:

> extract every commitment from the notes in 30 Meetings into
  commitments.csv. columns: commitment, owner, due, source_note.
  every row must fill all four. if the note names no due date, write
  "none stated". no columns other than these four.

At the API level, which is what the exam asks about, the enforcement is mechanical. This block is exam material: your teammates will never touch it, and the exam tests it directly. Two routes exist, and the exam centers on the first:

• The tool route, which the exam tests. Define a tool whose input schema is the output shape you want, and the model must call that tool, which means filling in the schema. The tool does not have to do anything; the schema is the product. Marking the tool definition strict: true makes the guarantee hard: the platform validates the model's output against the schema, so a nonconforming answer cannot come back as a tool call.
• The direct route, which the platform now ships as its own feature. The API's structured outputs feature takes your JSON schema in the request itself and constrains the response as it is generated, so the output is schema-valid by construction. The docs put the guarantee flatly: always valid, type safe, no retries needed for schema violations.

Two boundaries the exam draws around this, both worth having straight:

• A strict schema eliminates syntax errors, never semantic ones. With enforcement on, the JSON always parses and the fields are always the right types. Line items that fail to add up to the stated total, a value landed in the wrong field, a date that appears in no source: all of that still passes, because the shape is right and the shape is all a schema checks. Truth checking is a separate gate, and it gets its own section next.
• Schema design is where invention gets prevented. A field marked required forces the model to put something in it, and when the source document has nothing to offer, "something" is how fabrication happens. Fields the source may not contain get marked optional (allowed to be empty), which is the schema-level version of the "none stated" rule. The same thinking gives choice fields an "unclear" option for ambiguous cases and an "other" option with a detail field, so the model always has a legal move that is not a lie.

One more control, exam-labeled: tool_choice sets how hard the platform pushes. "auto" lets the model answer in text or call a tool, "any" requires it to call some tool (which is how structured output gets guaranteed), and naming a specific tool forces exactly that one. And headless Claude Code runs speak the same language: claude -p with --output-format json returns the run's result as machine-readable JSON, and adding --json-schema with your schema puts a conforming answer in the result's structured_output field, ready for the next step in a pipeline.

The one sentence version, and it maps this to something you already know: a schema does for output what a tool description does for input. It replaces "the model usually gets it right" with "the shape is checked."

5 · Trust, then verify: validation, retry, feedback (TS 4.4)

A schema catches malformed output. It cannot catch fluent, well formed, wrong output: the plausible date that was never said, the owner who was in the meeting but made no commitment. Extraction quality needs a second kind of check, and a loop around both. Four stations:

• Extract. The prompt with criteria, examples, and the schema.
• Validate, twice. Shape: does every row conform to the schema? Grounding: does every value trace back to a source? The grounding gate is a prompt of its own: "for each row in commitments.csv, find the sentence in the source note that supports it. list any row you cannot support."
• Retry with feedback. A failed row goes back with the failure attached: not "try again" but "row 12 cites a due date of Sep 15; the note contains no date. correct it or mark it none stated." The error message is the model's next input, so a specific error buys a real correction and a vague one buys a guess. The same rule that governs tool errors governs your feedback.
• Cap and flag. Retries get a limit, two or three, and whatever still fails lands in the flagged pile for a person. A flagged row costs you thirty seconds. An invented date that ships costs a client conversation.

One judgment call sits inside station 3, and the exam tests it directly: knowing when a retry cannot help. A retry fixes what the model got wrong about information it has: format mistakes, structure mistakes, a value it misread. A retry cannot conjure information the source never contained. When the note states no date, no number of retries produces the date, and every attempt is a fresh invitation to invent one. Rows that fail because the information is absent skip the retry entirely and go straight to the flagged pile, marked as gaps rather than errors.

The strong version of the validation station asks the model to check itself in ways software can then verify: extract the stated total and compute the sum of the line items as separate fields, and a mismatch between them flags itself. A "conflict_detected" field does the same for sources that disagree. The pattern: design the output so that inconsistency has to show up as two fields that fail to match, because two fields that fail to match is something a check can catch mechanically.

What this changes about your asks: you stop requesting extractions and start requesting extractions with their checks attached. One prompt shape covers it:

> extract the commitments per the tracker skill. then check every row
  against its source note and list any row you cannot support, with the
  reason. fix what you can, cap the fixes at two passes, and put anything
  still unsupported in a "needs review" section instead of the tracker.

6 · Scale: batch processing and the second pass (TS 4.5 · 4.6)

Batch processing means running the same operation over many items. Two decisions matter: how items are grouped, and which lane they run in.

Grouping. Independent items get independent runs. Forty notes through one extraction each beats one session holding all forty, because each run starts with a clean context window and one document's worth of attention, and one bad note fails alone instead of contaminating a shared session. You have seen this shape before: it is the same reason a subagent gets its own window. In your chair, the ask is plain:

> process each note in 30 Meetings separately: same tracker skill, one
  note at a time, appending each result to commitments.csv

The lane, which is exam material. The API has a dedicated batch lane, the Message Batches API: submit up to 100,000 requests as one batch, get results asynchronously within 24 hours (most far sooner), at half the price of the standard API. Each request in the batch carries a custom_id you choose, and results come back keyed by that id, in no particular order, so the id is how you match answers to inputs. The tradeoff is latency for cost: nothing about a batch is interactive.

The second pass (TS 4.6). Volume work adds a quality tool that single documents rarely need: another instance. Two patterns, both built on a fact you already know, that separate sessions share nothing:

• Multi-pass review: one session extracts, a second session with a fresh window checks the output against the sources. The checker session did not write the rows, so it has no stake in them, the same reason consulting firms put a second reader on deliverables. The author of a mistake is the least likely reader to catch it, and that is as true of sessions as of people.
• Multi-instance comparison: the same hard item goes to two or three separate runs, and disagreement between them is the signal. Where independent runs agree, confidence is high. Where they disagree, a person looks. This costs multiples of the single run price, so it is for decisions that are expensive to get wrong, and the batch discount is what makes it affordable at volume.

7 · Case study: the commitments tracker

The five consultant team at the regional health plan had forty meeting notes and a recurring embarrassment: commitments made in meetings kept resurfacing late, because they lived in prose nobody reread. The fix they wanted was a tracker: every commitment, owner, and due date, in one file the whole team checked on Mondays.

The first attempt was an ask. "Go through the meeting notes and pull out the commitments into a table." The table came back, and it was different every week: columns appeared and disappeared, owners were sometimes people and sometimes departments, dates arrived in three formats, and one week the table politely included a "sentiment" column nobody asked for. Usable, with twenty minutes of cleanup, every single week.

The fix took one afternoon, and nobody touched the notes. The team changed two things: the request became a skill with criteria and two worked examples, and the output got a contract.

Two worked example rows went into the skill under the criteria, one with a date and one with "none stated", so both cases had a model to match.

Replay one note through both worlds. The August 11 note contains "vendor counsel will complete the legal review of clause 7 the week after next." The old world produced Legal review | Vendor | soon in whichever columns that week's table had. The new world produces one row: legal review of clause 7 | vendor counsel | none stated | 2026-08-11 pbm-renewal.md, and "none stated" is there because the note says "week after next," and the criteria forbid converting a relative phrase into an invented date.

The near miss

The first full run, before the validation pass was added, did exactly that conversion: one row carried due: 2026-08-25, a real looking date computed from "week after next." Play it forward a week to see what that row costs. The tracker goes to the client sponsor on Monday. The sponsor's team plans the clause 7 review around August 25. The vendor's counsel, who said "week after next" loosely and committed to nothing, hears about a deadline they never gave, in a document with the consulting firm's name on it. The conversation that follows is about whether the tracker can be trusted, which is the conversation that ends trackers.

What actually happened: the grounding pass ("find the sentence that supports each row") failed that row, it landed in the needs review pile, and a person changed it to "none stated" in thirty seconds. The team then wrote the rule into the skill: relative dates are never converted; they are quoted or marked.

What the working pipeline looks like now, start to finish: each Monday, one prompt runs the tracker skill over the week's notes, one note at a time. Every row carries its source. A grounding pass checks each row and routes failures to needs review, capped at two fix attempts. A second session, fresh window, reads the final tracker against the notes before it leaves the building. Twenty minutes of weekly cleanup became four, and the four are spent on flagged rows, which is where a person's judgment actually earns something.

8 · Watch it built, then run it yourself

▶ Watch the demo: the extraction pipeline · 4 min

The prompt pack: every demo prompt, cleaned and explained

The demo builds the commitments tracker in the second brain repo, from the user's chair, in three parts. Part one writes the contract: the tracker skill gets its criteria, its two worked examples, and the four column schema, all by one ask. Part two runs and validates: the extraction over the notes one at a time, the grounding pass catching one row whose date appears in no source, the specific retry message, and the row landing in needs review instead of the tracker. Part three proves it: a fresh session runs the whole pipeline on a new note, the tracker gains one source backed row, and the deliberately planted trap (a note with a vague date) lands in needs review instead of shipping.

Then run the pack in your own repo. The prompt pack linked above holds every prompt from the demo, cleaned up and explained. Copied as is or rephrased in your own words, check what the demo checked as you go:

• The tracker skill exists, carrying criteria, two examples, and the four column shape.
• A run over your notes produces rows that all fill all four columns.
• The grounding pass names the source sentence for each row, or flags the row.
• A planted vague date lands in needs review as "none stated" material, never as an invented date.

9 · Wrap-up: what you should be able to do cold

• Name the three levels (ask, specification, contract) and pick the right one for a described piece of work
• Turn a vague request into checkable criteria, including the missing-data rule, and say why categorical criteria beat "be conservative" style confidence dials
• Say when examples beat instructions, how many to include (a few, varied, targeted), that ambiguous-case examples with reasoning generalize, and where examples live long term (in the skill)
• Explain what JSON and a schema are in plain words, and why "please return JSON" is not enforcement
• Describe the exam-level enforcement pattern: a tool whose input schema is the output shape, with strict validation, the three tool_choice settings, and the headless flags (--output-format json, --json-schema)
• State what a strict schema does and does not catch: syntax errors eliminated, semantic errors untouched, and fabrication prevented by optional fields, never by required ones
• Draw the four-station loop: extract, validate (shape and grounding), retry with specific feedback, cap and flag, and say when a retry cannot help (the information is absent from the source)
• Choose between an interactive loop and the Message Batches API for a described workload, and cite the batch facts: asynchronous, within 24 hours, no guaranteed turnaround, half price, keyed by custom_id, no tool calls mid-request, sample before you submit
• Explain why a fresh instance catches what same-session self-review misses, when to split a big review into per-document passes plus an integration pass, and when multi-instance comparison is worth its cost

Take the Module 07 quiz →  then hit the Study App for Domain 4 reps.