Add test command (closes #3)#45
Open
rikuson wants to merge 6 commits into
Open
Conversation
Rename cmd_super to cmd_test so the dispatched `quiz test` subcommand actually resolves. Add `--non-interactive`/`-n` to suppress ANSI colors and exit non-zero on any wrong answer (for CI use), and `--filter`/`-F` to restrict the quiz set via a script under `.filters/<name>.bash`. A default filter can be set via `QUIZ_FILTER` in `.quizrc` at the store root. Replace the placeholder test_expect_failure stubs with real assertions and add coverage for the new flags and config path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The multiline insert flow now opens $EDITOR twice instead of reading YAML from stdin, so piping malformed YAML through `quiz add -m` no longer produces the file these tests need. Write the malformed YAML straight into the store instead — the tests are about cmd_test's schema validation, not cmd_insert. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Open
When a quiz's stored answer spans multiple lines (YAML block scalar), the single-line read in cmd_test made it impossible to enter the full answer. End a line with `\` to continue the answer on the next line. Also quote the here-string operands of `tr` so newlines in answer/input survive the comparison instead of collapsing to spaces. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Count the trailing backslashes on each input line: 2n become n literal backslashes (no continuation), 2n+1 become n literal backslashes plus a line continuation. This lets users enter answers that legitimately end in a backslash by typing two of them, while preserving the existing single-backslash continuation behavior. Also quote the equality test — without it, bash treats the right side as a glob pattern, and a backslash followed by newline in the expected value silently collapses, producing spurious mismatches on multi-line answers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new quiz test subcommand (and makes bare quiz default to the same) to walk quizzes, compare answers case-insensitively, support non-interactive/CI mode, and optionally restrict quizzes via filter scripts. Updates docs/manpage and introduces a dedicated sharness test suite for the new behavior.
Changes:
- Implement
testsubcommand with--non-interactive/-nand--filter/-Fsupport, plus default-command routing updates. - Add sharness coverage for interactive/non-interactive runs, multiline/backslash behavior, filters, and error paths.
- Update README/man page usage examples and dependency list.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/t0010-test-command.sh | Adds coverage for the new test command, flags, filters, and schema/empty-store failures. |
| tests/sharness.sh | Ensures test stderr is redirected consistently (matches verbose vs non-verbose behavior). |
| src/quiz-store.sh | Implements cmd_test, adds filter support and default routing to test, updates usage output. |
| README.md | Documents quiz test, CI mode, filters, and updates examples/dependencies. |
| man/quiz.1 | Updates CLI documentation and examples to reflect test as the default command. |
Comments suppressed due to low confidence (1)
src/quiz-store.sh:137
- The usage text still advertises
$PROGRAM [show] quiz-name, but the dispatcher no longer defaults toshowwhen a quiz name is provided (unknown args fall through tocmd_test). Update the usage line to requireshowexplicitly (or restore the previous default-show behavior).
echo " $PROGRAM ls [subfolder]"
echo " List quizzes."
echo " $PROGRAM find quiz-names..."
echo " List quizzes that match quiz-names."
echo " $PROGRAM [show] quiz-name"
echo " Show existing quiz."
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+185
to
+186
| [[ -f "$PREFIX/.quizrc" ]] && source "$PREFIX/.quizrc" | ||
| [[ -z $filter && -n $QUIZ_FILTER ]] && filter="$QUIZ_FILTER" |
| fi | ||
| done | ||
| [[ -z $filter_file ]] && die "Error: filter '$filter' not found." | ||
| quizfiles="$(echo "$quizfiles" | bash "$filter_file")" |
| [[ -z $quizfiles ]] && die "Error: there is no matched quiz found." | ||
|
|
||
| local fail=0 | ||
| while read quizfile <&3; do |
| printf "\e[94m%s\e[1m%s\e[0m:\n" "$quizfile_dir" "$quizfile" | ||
| echo "$grepresults" | ||
| done < <(find -L "$PREFIX" -path '*/.git' -prune -o -path '*/.extensions' -prune -o -iname '*.yml' -print0) | ||
| done < <(find -L "$PREFIX" -path '*/.git' -prune -o -path '*/.extensions' -prune -o -iname '*.yml' -print0 | sort) |
| "$QUIZ" init && | ||
| printf "question1\nanswer1" | "$QUIZ" add quiz1 && | ||
| out="$(printf "answer1\n" | "$QUIZ" test --non-interactive)" && | ||
| ! printf "%s" "$out" | grep -q $"\x1B" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
quiz testsubcommand walks every quiz interactively, comparing your answer to the stored one (case-insensitive). Barequizfalls through to the same.--non-interactive/-nsuppresses ANSI colors and exits non-zero on any wrong answer, suitable for GitHub Actions / CI use.--filter/-F <name>restricts the quiz set via a script at$QUIZ_STORE_DIR/.filters/<name>.bashthat reads quiz file paths on stdin and emits the filtered subset on stdout. A default filter can be set in$QUIZ_STORE_DIR/.quizrcviaQUIZ_FILTER=<name>.t0010-test-command.shcovers all flag combinations and the schema/empty-store error paths.Closes #3.
Test plan
make testpasses (all suites green, including 14 cases int0010-test-command.sh)quiz testwalks all quizzes interactivelyquiz test --non-interactiveexits 0 on all-correct, exits 1 on any wrong, no ANSI codes in outputquiz test --filter <name>runs only the filtered subset; missing filter errors clearly.quizrcdefault filter is honored when no--filteris passedquiz(no args) still works as the default test entry pointeditorconfig-checkerpasses🤖 Generated with Claude Code