Module 10 demo · ~5 minutes · a coach that drills from your own notes, a log of misses, a drill that chases them, and a gate that reads every domain
study/
exam-guide-v0.2.pdf the official guide
notes/ your notes, ten modules
The coach drills you from your own material
and the guide's task statements. Nothing it
asks comes from anywhere else, so every
question is one the exam could ask.
# Task statements (Exam Guide v0.2)
## 5.2 Design effective escalation and
ambiguity resolution patterns
Knowledge
- Escalation triggers: policy gaps, explicit
request for a human, no meaningful progress
- Sentiment and self-reported confidence are
unreliable proxies for case complexity
Skills
- Honor an explicit request for a human
immediately, before investigating
- Ask for another identifier on multiple
customer matches; never pick heuristically
Anti-patterns
- Confidence score as an escalation trigger
(29 more sections, same shape)
---
name: exam-coach
description: Drill the user for the Claude
Certified Architect exam. Use when they say
drill me, quiz me, or name a domain or task
statement to practice.
---
1. Read study/task-statements.md and
study/mistakes.md.
2. Pick 5 task statements: the ones named, or
the domain named, weighted toward statements
with misses in the log.
3. For each: write one scenario question in the
guide's style (production symptom, four
options, one correct, three plausible).
Ask it. Wait for the answer. Do not reveal.
4. After the answer: say correct or wrong, give
the reason in three sentences, and cite the
task statement bullet it tests.
5. On a wrong answer, append one line to
study/mistakes.md: date | TS | what was
chosen | why it is wrong | the bullet.
6. End with a score out of 5 and the task
statements to revisit.
> drill me on domain 5
Read study/task-statements.md, study/mistakes.md
(empty). Five questions from 5.1 to 5.6.
Q1 of 5 (TS 5.1). A support agent has handled
a long case. After summarization, the refund
amount the customer stated ($412.50) has become
"a partial refund." What design prevents this?
A) A longer summary
B) A case facts block kept outside the
summarized history
C) Asking the model to be careful with numbers
D) Turning off compaction (the automatic
summarizing that trims old context)
Your answer?
> B
Correct. Numbers, dates, and stated expectations
are the first casualties of summarization, so
they live in a structure that never gets
summarized. The guide's bullet for 5.1:
transactional facts in a persistent block
outside summarized history.
Q2 of 5 (TS 5.2). First-contact resolution is
55 percent. A colleague proposes escalating
whenever the model's self-reported confidence
is under 70 percent. Is that the right lever?
A) Yes, with the threshold tuned
B) No; define explicit escalation criteria
C) Yes, if a second model scores confidence
D) No; remove escalation entirely
> A
Wrong. Self-reported confidence is text the
model generated, checked against nothing. The
guide names it, with sentiment, as an unreliable
proxy for case complexity. The triggers that
count: a policy gap, an explicit request for a
human, no meaningful progress.
Appended to study/mistakes.md:
2026-09-08 | 5.2 | chose confidence threshold |
confidence is not a complexity signal |
"sentiment and self-reported confidence are
unreliable proxies"
Q3 of 5 (TS 5.3) ...
---
name: weak-spots
description: Drill only the task statements the
user has missed. Use when they say weak spots,
drill my misses, or what am I getting wrong.
---
1. Read study/mistakes.md. Count misses per
task statement.
2. Rank them. Take the top three (ties by most
recent).
3. Run the exam-coach procedure on those three
only, two questions each, with fresh
scenarios (never reuse a logged question).
4. If a statement is answered correctly twice
in a row across sessions, note it as
"recovered" in the log. It leaves the top
three; it does not leave the file.
> drill my weak spots
study/mistakes.md: 3 entries.
5.2 x2 (Sep 8, Sep 9)
1.5 x1 (Sep 9)
Top three: 5.2, 1.5. Two questions each.
Q1 of 4 (TS 5.2). A customer writes "this is
ridiculous, I want a human." The agent can see
the fix is a one-step address correction. What
does it do?
A) Fix the address, then offer a human
B) Escalate immediately, before investigating
C) Ask why they want a human
D) Apologize and fix it
Your answer?
---
name: readiness
description: Decide whether the user is ready
to book the exam. Use when they say am I
ready, readiness check, or paste domain scores.
---
1. Take the five domain percentages the user
pastes from the study app and the mock.
2. Read study/mistakes.md; list statements
missed twice or more without a "recovered"
note.
3. Write study/readiness.md:
- each domain, its score, and GO or NO GO
against 85 percent
- the overall verdict: GO only if every
domain is GO and no statement is open
- the checklist: booked, ID ready, room
tested, sleep, two minutes a question,
flag and move on
4. If NO GO: name the domains and statements
to drill and stop there.
# Readiness 2026-09-09
| Domain | Score | Gate |
|---|---|---|
| 1 Agentic Architecture | 78% | NO GO |
| 2 Tool Design and MCP | 91% | GO |
| 3 Claude Code Config | 88% | GO |
| 4 Prompt Engineering | 86% | GO |
| 5 Context and Reliability | 90% | GO |
Open statements: 5.2 (2 misses), 1.5 (1 miss)
Verdict: NO GO. The average is 86.6 and the
exam does not grade on the average. Domain 1
is 27 percent of the paper. Drill 1.5, 1.6,
1.7, then rerun.
# Readiness 2026-09-11
| Domain | Score | Gate |
|---|---|---|
| 1 Agentic Architecture | 87% | GO |
| 2 Tool Design and MCP | 91% | GO |
| 3 Claude Code Config | 88% | GO |
| 4 Prompt Engineering | 89% | GO |
| 5 Context and Reliability | 92% | GO |
Open statements: none (5.2 recovered Sep 10,
1.5 recovered Sep 11)
Verdict: GO. Book it.
Checklist: [ ] booked [ ] ID [ ] room tested
[ ] sleep [ ] 2 min a question [ ] flag and
move on
(fresh session)
> drill me on domain 1
Read study/task-statements.md, study/mistakes.md
(5 entries, 2 recovered).
Weighting toward 1.5 and 1.7, your history.
Q1 of 5 (TS 1.7). A long refactoring session
was interrupted. Resuming it replays yesterday's
stale tool results. What is the better pattern?
...
Nothing from earlier sessions carries into
this window. The log does, and the log is
enough to keep drilling the right statements.