▶ Demo: the exam coach

Module 10 demo · ~5 minutes · a coach that drills from your own notes, a log of misses, a drill that chases them, and a gate that reads every domain

Repo, files & checks

client-project/
CLAUDE.md
.claude/skills/ (meeting-brief, and the rest of the course)
study/
exam-guide-v0.2.pdf
notes/ (your notes from ten modules)
Part 1 · build the coach
study/task-statements.md · 30 sections from the guide
.claude/skills/exam-coach/SKILL.md
study/mistakes.md · one line per miss
drill me on domain 5: five scenario questions, one at a time ✓
Part 2 · learn from the misses
answered 5.2 with the confidence score; logged
.claude/skills/weak-spots/SKILL.md
weak spots: only 5.2 and 1.5 asked ✓
Part 3 · the gate
.claude/skills/readiness/SKILL.md
study/readiness.md: NO GO, Domain 1 at 78 percent
after two drill rounds: GO, every domain 85 or better ✓
fresh session: the coach still drills from the log ✓
study/ · what the coach will read
study/
  exam-guide-v0.2.pdf     the official guide
  notes/                  your notes, ten modules

The coach drills you from your own material
and the guide's task statements. Nothing it
asks comes from anywhere else, so every
question is one the exam could ask.
study/task-statements.md · excerpt
# Task statements (Exam Guide v0.2)

## 5.2 Design effective escalation and
   ambiguity resolution patterns
Knowledge
- Escalation triggers: policy gaps, explicit
  request for a human, no meaningful progress
- Sentiment and self-reported confidence are
  unreliable proxies for case complexity
Skills
- Honor an explicit request for a human
  immediately, before investigating
- Ask for another identifier on multiple
  customer matches; never pick heuristically
Anti-patterns
- Confidence score as an escalation trigger

(29 more sections, same shape)
.claude/skills/exam-coach/SKILL.md
---
name: exam-coach
description: Drill the user for the Claude
  Certified Architect exam. Use when they say
  drill me, quiz me, or name a domain or task
  statement to practice.
---
1. Read study/task-statements.md and
   study/mistakes.md.
2. Pick 5 task statements: the ones named, or
   the domain named, weighted toward statements
   with misses in the log.
3. For each: write one scenario question in the
   guide's style (production symptom, four
   options, one correct, three plausible).
   Ask it. Wait for the answer. Do not reveal.
4. After the answer: say correct or wrong, give
   the reason in three sentences, and cite the
   task statement bullet it tests.
5. On a wrong answer, append one line to
   study/mistakes.md: date | TS | what was
   chosen | why it is wrong | the bullet.
6. End with a score out of 5 and the task
   statements to revisit.
the session · "drill me on domain 5"
> drill me on domain 5

Read study/task-statements.md, study/mistakes.md
(empty). Five questions from 5.1 to 5.6.

Q1 of 5 (TS 5.1). A support agent has handled
a long case. After summarization, the refund
amount the customer stated ($412.50) has become
"a partial refund." What design prevents this?
  A) A longer summary
  B) A case facts block kept outside the
     summarized history
  C) Asking the model to be careful with numbers
  D) Turning off compaction (the automatic
     summarizing that trims old context)

Your answer?
the session · after the answer
> B

Correct. Numbers, dates, and stated expectations
are the first casualties of summarization, so
they live in a structure that never gets
summarized. The guide's bullet for 5.1:
transactional facts in a persistent block
outside summarized history.

Q2 of 5 (TS 5.2). First-contact resolution is
55 percent. A colleague proposes escalating
whenever the model's self-reported confidence
is under 70 percent. Is that the right lever?
  A) Yes, with the threshold tuned
  B) No; define explicit escalation criteria
  C) Yes, if a second model scores confidence
  D) No; remove escalation entirely
the session · a miss, logged
> A

Wrong. Self-reported confidence is text the
model generated, checked against nothing. The
guide names it, with sentiment, as an unreliable
proxy for case complexity. The triggers that
count: a policy gap, an explicit request for a
human, no meaningful progress.

Appended to study/mistakes.md:
2026-09-08 | 5.2 | chose confidence threshold |
  confidence is not a complexity signal |
  "sentiment and self-reported confidence are
  unreliable proxies"

Q3 of 5 (TS 5.3) ...
.claude/skills/weak-spots/SKILL.md
---
name: weak-spots
description: Drill only the task statements the
  user has missed. Use when they say weak spots,
  drill my misses, or what am I getting wrong.
---
1. Read study/mistakes.md. Count misses per
   task statement.
2. Rank them. Take the top three (ties by most
   recent).
3. Run the exam-coach procedure on those three
   only, two questions each, with fresh
   scenarios (never reuse a logged question).
4. If a statement is answered correctly twice
   in a row across sessions, note it as
   "recovered" in the log. It leaves the top
   three; it does not leave the file.
the session · "drill my weak spots"
> drill my weak spots

study/mistakes.md: 3 entries.
  5.2  x2   (Sep 8, Sep 9)
  1.5  x1   (Sep 9)
Top three: 5.2, 1.5. Two questions each.

Q1 of 4 (TS 5.2). A customer writes "this is
ridiculous, I want a human." The agent can see
the fix is a one-step address correction. What
does it do?
  A) Fix the address, then offer a human
  B) Escalate immediately, before investigating
  C) Ask why they want a human
  D) Apologize and fix it

Your answer?
.claude/skills/readiness/SKILL.md
---
name: readiness
description: Decide whether the user is ready
  to book the exam. Use when they say am I
  ready, readiness check, or paste domain scores.
---
1. Take the five domain percentages the user
   pastes from the study app and the mock.
2. Read study/mistakes.md; list statements
   missed twice or more without a "recovered"
   note.
3. Write study/readiness.md:
   - each domain, its score, and GO or NO GO
     against 85 percent
   - the overall verdict: GO only if every
     domain is GO and no statement is open
   - the checklist: booked, ID ready, room
     tested, sleep, two minutes a question,
     flag and move on
4. If NO GO: name the domains and statements
   to drill and stop there.
study/readiness.md · first run
# Readiness  2026-09-09

| Domain | Score | Gate |
|---|---|---|
| 1 Agentic Architecture | 78% | NO GO |
| 2 Tool Design and MCP | 91% | GO |
| 3 Claude Code Config | 88% | GO |
| 4 Prompt Engineering | 86% | GO |
| 5 Context and Reliability | 90% | GO |

Open statements: 5.2 (2 misses), 1.5 (1 miss)

Verdict: NO GO. The average is 86.6 and the
exam does not grade on the average. Domain 1
is 27 percent of the paper. Drill 1.5, 1.6,
1.7, then rerun.
study/readiness.md · two days later
# Readiness  2026-09-11

| Domain | Score | Gate |
|---|---|---|
| 1 Agentic Architecture | 87% | GO |
| 2 Tool Design and MCP | 91% | GO |
| 3 Claude Code Config | 88% | GO |
| 4 Prompt Engineering | 89% | GO |
| 5 Context and Reliability | 92% | GO |

Open statements: none (5.2 recovered Sep 10,
1.5 recovered Sep 11)

Verdict: GO. Book it.
Checklist: [ ] booked  [ ] ID  [ ] room tested
[ ] sleep  [ ] 2 min a question  [ ] flag and
move on
fresh session · the coach remembers
(fresh session)
> drill me on domain 1

Read study/task-statements.md, study/mistakes.md
(5 entries, 2 recovered).
Weighting toward 1.5 and 1.7, your history.

Q1 of 5 (TS 1.7). A long refactoring session
was interrupted. Resuming it replays yesterday's
stale tool results. What is the better pattern?
  ...

Nothing from earlier sessions carries into
this window. The log does, and the log is
enough to keep drilling the right statements.
Module 10 · Demo
The exam coach
Ten modules of notes and the official guide become a coach that drills you in the exam's own style, logs every miss against its task statement, chases the misses until they recover, and tells you when to book. Three parts:
1 · Build the coach 2 · Learn from the misses 3 · The gate
Ask before you tell, log every miss, gate every domain
Three skills and one log file, built from the guide and your own notes. The prompt pack builds them in your second brain; the readiness file is what you read the morning of the exam.

Open the prompt pack →   Sit the mock →

Part 1 · Build the coach Part 2 · Learn from the misses Part 3 · The gate