What happened
During a tilth prep-feature interview, the seeder asked two questions in a single ask_user call but the option list only enumerated answers for the first one:
? What should happen on success — silent exit (0), or a confirmation message like "Added: remember the milk"? And should empty
todo strings be rejected with an error?
1) Silent success (exit 0, no output)
2) Confirmation message to stdout
3) Confirmation to stderr
0) Other (I'll specify)
>
The three options answer "what should happen on success" — none of them speaks to "should empty strings be rejected." The user is forced to either:
- Pick an option and silently drop the second question (the seeder then bakes in whatever default it imagined for empty-string handling).
- Hit
0) Other and free-form an answer to both questions, which defeats the purpose of the multi-choice anchoring.
Either way, the load-bearing decision about empty-string handling either gets buried as an open_questions entry or quietly assumed.
Why this is happening
tilth/seed/prompts.md currently has:
One question per turn; follow up freely when an answer surfaces a new question or contradicts the codebase.
That's correct but easy for the model to interpret loosely — "one topic per turn" rather than "one answer-slot per turn." Compound questions joined by "and" / "or" still feel like one topic to the model, especially when the second clause is a short clarifier.
The §3 ("Anchored interview") guidance for ask_user with options says options "must be concrete and grounded in actual code" but doesn't address the compound-question failure mode.
Suggested fix
Sharpen the rule in tilth/seed/prompts.md §3. Two candidate phrasings:
- Strict: "One question, one answer slot. If the user's answer can't be expressed by picking exactly one option (or one free-form sentence), the question is two questions — split it across turns."
- Mechanical: "Before sending
ask_user, re-read your question text and count how many distinct decisions the user has to make to fully answer it. If the count is greater than 1, split into separate ask_user calls. Never ask 'X and Y?' with options that only enumerate answers to X."
Probably worth adding a worked counter-example (the snippet above is a good one) to anchor what the failure looks like.
Adjacent thought (not in scope for this fix)
The frontend has no enforcement for this — it accepts whatever the model passes. We could harden it by lint-checking the question text (e.g. reject if it contains " and " plus a question mark in the second clause), but prompt-side guidance is the right starting point because the harder rule (don't bury load-bearing decisions in open_questions) is also prompt-side.
Related
- The
open_questions discipline in §5 — load-bearing decisions must be asked, not logged. A compound question that drops half its content into the seeder's mental model is the quiet version of the same bug open_questions exists to prevent.
What happened
During a
tilth prep-featureinterview, the seeder asked two questions in a singleask_usercall but the option list only enumerated answers for the first one:The three options answer "what should happen on success" — none of them speaks to "should empty strings be rejected." The user is forced to either:
0) Otherand free-form an answer to both questions, which defeats the purpose of the multi-choice anchoring.Either way, the load-bearing decision about empty-string handling either gets buried as an
open_questionsentry or quietly assumed.Why this is happening
tilth/seed/prompts.mdcurrently has:That's correct but easy for the model to interpret loosely — "one topic per turn" rather than "one answer-slot per turn." Compound questions joined by "and" / "or" still feel like one topic to the model, especially when the second clause is a short clarifier.
The §3 ("Anchored interview") guidance for
ask_user with optionssays options "must be concrete and grounded in actual code" but doesn't address the compound-question failure mode.Suggested fix
Sharpen the rule in
tilth/seed/prompts.md§3. Two candidate phrasings:ask_user, re-read your question text and count how many distinct decisions the user has to make to fully answer it. If the count is greater than 1, split into separateask_usercalls. Never ask 'X and Y?' with options that only enumerate answers to X."Probably worth adding a worked counter-example (the snippet above is a good one) to anchor what the failure looks like.
Adjacent thought (not in scope for this fix)
The frontend has no enforcement for this — it accepts whatever the model passes. We could harden it by lint-checking the question text (e.g. reject if it contains " and " plus a question mark in the second clause), but prompt-side guidance is the right starting point because the harder rule (don't bury load-bearing decisions in
open_questions) is also prompt-side.Related
open_questionsdiscipline in §5 — load-bearing decisions must be asked, not logged. A compound question that drops half its content into the seeder's mental model is the quiet version of the same bugopen_questionsexists to prevent.