Add LLM-generated questions to create-quiz + backfill media-recording README#85
Merged
Conversation
Per question, the instructor now picks [p]laceholder, [g]enerate w/ LLM, or [e]nd quiz. The 'g' path sends a natural-language seed to the cloud text model (default gemini-3-flash-preview) and returns a Canvas-shaped question of one of seven auto-gradable types (multiple_choice, multiple_answers, true_false, fill_in_multiple_blanks, multiple_dropdowns, matching, calculated). The proposed question is previewed and the instructor accepts/regenerates/skips before it lands on Canvas. points_possible is forced to 1; the instructor refines the rest in the Canvas UI. The canvigator_llm module is lazy-imported only on the first 'g', so pure-placeholder runs still work without OLLAMA_API_KEY set. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds README sections for create-media-recording-assignment, get-media-recordings, and analyze-media-recordings (the workflow shipped in #84 but never documented in the README). Also updates the available-tasks list, the --dry-run task list, and adds an --auto-grade example. Backfills coverage that was previously only in CLAUDE.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7b97577196
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The "scale": 0" had a stray trailing quote, making the JSON template shown to the LLM syntactically invalid. Weaker models can mirror the malformed pattern, which then fails _parse_quiz_question and the generation gets skipped instead of producing a usable draft. Caught in PR review. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously [r]egenerate called the model with the same seed prompt and
temperature=0.4, so successive drafts often looked nearly identical.
Now _reviewGeneratedQuestion accumulates each rejected draft and passes
the list to generate_quiz_question, which:
- appends a "REJECTED draft(s); produce a SUBSTANTIVELY DIFFERENT
new draft" instruction to the user prompt with a compact summary
of each prior draft (type, question text, marked answers / match
pairs / variables-formulas), and
- bumps temperature 0.4 -> 0.8 when prior_drafts is non-empty, so
the sampler has room to actually diverge.
The first generation behavior is unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
create-quizLLM generation. Per question, the instructor now picks[p]laceholder,[g]enerate w/ LLM, or[e]nd quiz. Thegpath sends a natural-language seed (e.g. "a question on Big-O for binary search") to the cloud text model (defaultgemini-3-flash-preview, requiresOLLAMA_API_KEY) and returns a Canvas-shaped question of one of seven auto-gradable types:multiple_choice_question,multiple_answers_question,true_false_question,fill_in_multiple_blanks_question,multiple_dropdowns_question,matching_question,calculated_question. The proposed question is previewed (with*marking correct answers, plus per-type extras like match pairs / formula variables) and the instructor accepts/regenerates/skips before it's pushed to Canvas.points_possibleis forced to1; the instructor refines wording and points later in the Canvas UI. Thecanvigator_llmmodule is lazy-imported only on the firstg, so pure-placeholder runs still work withoutOLLAMA_API_KEYset.create-media-recording-assignment,get-media-recordings,analyze-media-recordings; updates the available-tasks list, the--dry-runflag note, and adds an--auto-gradeexample.New pure helpers in
canvigator_llm.py:_build_quiz_question_prompt,_parse_quiz_question(markdown-fence-tolerant JSON parse, stripspoints_possible, rejects unknown types), and the cloud-boundgenerate_quiz_question(user_seed). New helper incanvigator_course.py:_reviewGeneratedQuestionfor the[y]/[r]/[s]accept/regenerate/skip flow.Test plan
python -m pytest test_canvigator.py— 230/230 passing (9 new tests for the quiz-question helpers).flake8(both passes fromCLAUDE.md) — clean.OLLAMA_API_KEYset:[p]→ typed description → placeholder created (today's behavior).[g]→ seed for a multiple-choice question → preview shows 4 answers with one starred →yaccepts.[g]→ seed for a true/false question → type comes back astrue_false_question.[g]→rregenerates with the same seed.[g]→sskips and doesn't write.[e](and Enter) → ends the quiz with whatever's been added.OLLAMA_API_KEYunset, a pure-placeholder run still works (no lazy-import triggered).OLLAMA_API_KEYset butOLLAMA_TEXT_MODELpointing at a bogus model,gfails gracefully and re-prompts.🤖 Generated with Claude Code