Skip to content

Add prep-class-digest task for pre-class cohort gap synthesis #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sgeinitz merged 3 commits into from
May 9, 2026
Merged

Add prep-class-digest task for pre-class cohort gap synthesis #86

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8480bbb
Add prep-class-digest task for pre-class cohort gap synthesis
sgeinitz May 7, 2026
3381552
Document --all support for send-quiz-reminder in --help and README
sgeinitz May 8, 2026
f417cda
Add per-task --help to canvigator.py CLI
sgeinitz May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,13 +117,15 @@ python canvigator.py --tag get-quiz-questions # add LLM-generated topic tags
python canvigator.py --reply-window-days N <task> # set the days-after-send window for assess-replies (default: 5)
python canvigator.py --months N delete-old-conversations # cutoff age in months for delete-old-conversations (default: 6)
python canvigator.py --auto-grade get-media-recordings # skip per-student review and auto-grade every submission at full credit
python canvigator.py --days N prep-class-digest # synthesize a 1-page brief on cohort gaps from the last N days (default 7)
python canvigator.py --cloud-questions prep-class-digest # opt the discussion-question step into cloud Gemini 3 with a redacted prompt (default: local Gemma 4)
```

The `--crn` option selects a course by its CRN (the last 5 digits of the Canvas
course code), bypassing the interactive course selection prompt. This is useful
for automated/scheduled runs, e.g. `python canvigator.py --crn 12345 get-activity`.

Available tasks: `analyze-media-recordings`, `assess-replies`, `award-bonus`, `award-bonus-partner-only`, `award-bonus-retake-only`, `create-media-recording-assignment`, `create-pairs`, `create-quiz`, `delete-old-conversations`, `export-anon-data`, `generate-follow-up-questions`, `get-activity`, `get-conversations`, `get-gradebook`, `get-media-recordings`, `get-quiz-questions`, `get-quiz-submission-events`, `get-roster`, `send-follow-up-assessments`, `send-follow-up-question`, `send-quiz-reminder`
Available tasks: `analyze-media-recordings`, `assess-replies`, `award-bonus`, `award-bonus-partner-only`, `award-bonus-retake-only`, `create-media-recording-assignment`, `create-pairs`, `create-quiz`, `delete-old-conversations`, `export-anon-data`, `generate-follow-up-questions`, `get-activity`, `get-conversations`, `get-gradebook`, `get-media-recordings`, `get-quiz-questions`, `get-quiz-submission-events`, `get-roster`, `prep-class-digest`, `send-follow-up-assessments`, `send-follow-up-question`, `send-quiz-reminder`

All tasks begin by prompting you to select a course. Output files are written to
`data/<course>/` and `figures/<course>/`, where `<course>` is derived from the
Expand Down Expand Up @@ -712,6 +714,16 @@ every send is reviewed before going to Canvas.
Use `--dry-run` to preview all messages (recipient, subject, body, and reason)
without sending anything to Canvas and without the interactive prompt.

Pass `--all` (or `-a`) to skip the interactive quiz picker and instead
iterate every published quiz with a future `due_at`. Each student's state is
aggregated across those quizzes and the task sends **one consolidated Canvas
message per student** listing every eligible quiz they still have work to do
on (no attempt, imperfect score, page-blur, or perfect-clean), with a single
course-level manifest `course_reminder_sent[_dryrun]_YYYYMMDD.csv` instead of
the per-quiz manifest. Quizzes without a `*_questions_w_tags_*.csv` are
skipped with a warning. Combine with `--dry-run` to preview the consolidated
messages without sending.

---

#### Media-recording check-in workflow
Expand Down Expand Up @@ -813,3 +825,53 @@ and `get-quiz-questions --tag` (to produce the tags CSV) first.
| **Input** | `data/<course>/assignment<id>_recordings_YYYYMMDD.csv` (from `get-media-recordings`) |
| | `data/<course>/<quiz>_<id>_questions_w_tags_YYYYMMDD.csv` (from `get-quiz-questions --tag`) |
| **Output** | `data/<course>/assignment<id>_analysis_YYYYMMDD.md` — Markdown report |

---

#### `prep-class-digest` — Pre-class "where the cohort actually is" Markdown brief

Synthesizes a 1-page brief from three signal sources collected over a
configurable lookback window (default 7 days, override with `--days N`):

- **Recent quiz misses** — joined to the topic tags from
`*_questions_w_tags_*.csv` so misses are aggregated by concept rather than
by individual question.
- **Follow-up reply themes** — failing/borderline rows in
`*_followup_assessments.csv` (within the window) are summarized by a local
Gemma 4 call into "where students went wrong" bullets, one block per
quiz/question.
- **Media-recording transcripts** — every
`assignment<id>_recordings_*.csv` in the window is run through the existing
tag-classification + theme-extraction Gemma 4 pipeline (the same one used
by `analyze-media-recordings`).

The brief ends with **2–3 suggested in-class discussion questions** targeted
at the highest-priority gaps. By default these are drafted by the local
Gemma 4 model with a full-fidelity prompt that includes the derived theme
bullets and evidence snippets. With `--cloud-questions` (`-q`) the step
routes to cloud Gemini 3 instead, but only with a *redacted* prompt that
contains tag names + integer miss counts + integer theme-cluster counts —
no transcripts, no `criteria_evaluations`, no theme text. Student-derived
content stays local in either configuration.

Empty windows (no quiz/follow-up/recording activity in the lookback
period) short-circuit with a friendly message and write no file. Same-day
re-runs overwrite.

| | Files |
|---|---|
| **Input** | `data/<course>/<quiz>_<id>_all_subs_by_question_YYYYMMDD.csv` files in window (from `get-quiz-submission-events`) |
| | `data/<course>/<quiz>_<id>_questions_w_tags_YYYYMMDD.csv` per quiz (from `get-quiz-questions --tag`) |
| | `data/<course>/<quiz>_<id>_followup_assessments.csv` (from `assess-replies`, persistent) |
| | `data/<course>/assignment<id>_recordings_YYYYMMDD.csv` files in window (from `get-media-recordings`) |
| **Output** | `data/<course>/class_digest_YYYYMMDD.md` — Markdown report (5 sections: header, quiz performance table, follow-up reply themes, media-recording themes, suggested discussion questions) |

**Typical workflow:**
1. Mid-week, after a quiz has been returned and any follow-ups assessed:
```bash
python canvigator.py prep-class-digest # default 7-day window, Gemma-only
python canvigator.py --days 14 prep-class-digest # widen the window
python canvigator.py --cloud-questions prep-class-digest # opt the question step into Gemini
```
2. Open the generated `class_digest_*.md`, scan the priority gaps, and use
the suggested questions to seed a 5–10 minute in-class discussion.
108 changes: 71 additions & 37 deletions canvigator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@
('delete-old-conversations', 'Delete Canvas conversations older than N months (account-wide; default 6; override with --months/-m)'),
('get-gradebook', 'Export course gradebook'),
('get-roster', 'Export the full course roster (name, id, sis_id, enrollment_type)'),
('send-quiz-reminder', 'Send quiz reminder messages to students'),
('send-quiz-reminder', 'Send quiz reminder messages to students (use --all for one consolidated message across every published, future-due quiz)'),
('prep-class-digest', 'Synthesize a 1-page Markdown brief on cohort gaps from the last N days (default 7; override with --days/-n)'),
]),
]
task_descriptions = {name: desc for _, items in task_groups for name, desc in items}
Expand All @@ -42,26 +43,8 @@

def print_help():
"""Print usage information with grouped task descriptions."""
print("Usage: canvigator.py [OPTIONS] <task>\n")
print("Options (short and long forms are interchangeable):")
print(" -d, --dry-run Preview changes without modifying Canvas (applies to bonus, reminder, follow-up,")
print(" feedback, delete-old-conversations, get-media-recordings — CSV is still written)")
print(" -t, --tag Use a cloud LLM via Ollama to tag questions (get-quiz-questions only)")
print(" -a, --all Run across every quiz in the course (get-quiz-questions, get-quiz-submission-events, send-quiz-reminder)")
print(" -c, --crn <CRN> Select course by CRN (last 5 digits of course code)")
print(" -m, --months <N> Age threshold in months for delete-old-conversations (default: 6)")
print(" -w, --reply-window-days <N> Days to accept replies after follow-up sent (default: 5, assess-replies only)")
print(" -g, --auto-grade Skip per-student review prompt and auto-award points_possible (get-media-recordings only)")
max_name = max(len(t) for t in tasks)
for header, items in task_groups:
print(f"\n{header}")
for name, desc in items:
print(f" {name:<{max_name}} {desc}")
print("\nNotes:")
print(" generate-follow-up-questions uses a cloud LLM (via Ollama) in two steps:")
print(" 1. Classifies each question as 'explain' (oral) or 'draw' (visual)")
print(" 2. Generates a mode-appropriate open-ended question for instructor review")
print(" Output CSV includes a question_mode column so the instructor can override choices.")
import canvigator_help as ch
ch.print_global_help(task_groups)


def _run_assignment_task(task, course, canvas, canv_config, dry_run, auto_grade_flag):
Expand All @@ -79,6 +62,31 @@ def _run_assignment_task(task, course, canvas, canv_config, dry_run, auto_grade_
cassign.getMediaRecordings(auto_grade=auto_grade_flag, dry_run=dry_run)


# Course-level tasks that need only `course` + a few primitives. Kept in a
# helper to keep the main if/elif chain under flake8's max-complexity limit.
COURSE_TASKS = frozenset({
'get-activity', 'create-quiz', 'get-gradebook', 'get-roster',
'get-conversations', 'prep-class-digest',
})


def _run_course_task(task, course, canv_config, n_days, cloud_questions):
"""Dispatch a course-level task that doesn't need quiz/assignment selection."""
if task == 'get-activity':
course.saveStudentActivity(canv_config.data_path)
elif task == 'create-quiz':
course.createQuiz()
elif task == 'get-gradebook':
course.exportGradebook(canv_config.data_path)
elif task == 'get-roster':
course.exportRoster(canv_config.data_path)
elif task == 'get-conversations':
course.exportConversations(canv_config.data_path)
elif task == 'prep-class-digest':
import canvigator_digest as cd
cd.prepClassDigest(course, days=n_days, cloud_questions=cloud_questions)


def _run_quiz_task(task, quiz, dry_run, tag, reply_window_days):
"""Dispatch a quiz-level task to the appropriate method."""
if task == 'get-quiz-questions':
Expand Down Expand Up @@ -122,9 +130,20 @@ def _run_quiz_task(task, quiz, dry_run, tag, reply_window_days):
'-m': '--months',
'-w': '--reply-window-days',
'-g': '--auto-grade',
'-n': '--days',
'-q': '--cloud-questions',
'-h': '--help',
}
args = [_SHORT_TO_LONG.get(a, a) for a in sys.argv[1:]]

# --help is intentionally stripped before the rest of the parser so that
# `canvigator.py <task> --help` and `canvigator.py --help <task>` behave the
# same. The bare `canvigator.py --help` case falls through to the empty-args
# branch below, which prints global help.
help_requested = '--help' in args
while '--help' in args:
args.remove('--help')

dry_run = '--dry-run' in args
if dry_run:
args.remove('--dry-run')
Expand All @@ -141,6 +160,10 @@ def _run_quiz_task(task, quiz, dry_run, tag, reply_window_days):
if auto_grade_flag:
args.remove('--auto-grade')

cloud_questions_flag = '--cloud-questions' in args
if cloud_questions_flag:
args.remove('--cloud-questions')

crn = None
if '--crn' in args:
crn_idx = args.index('--crn')
Expand Down Expand Up @@ -186,20 +209,43 @@ def _run_quiz_task(task, quiz, dry_run, tag, reply_window_days):
args.pop(rw_idx) # remove '--reply-window-days'
args.pop(rw_idx) # remove the value

n_days = 7
if '--days' in args:
nd_idx = args.index('--days')
if nd_idx + 1 >= len(args):
print("Error: --days/-n requires a numeric value (number of days)")
sys.exit(1)
try:
n_days = int(args[nd_idx + 1])
if n_days < 1:
raise ValueError
except ValueError:
print(f"Error: --days/-n must be a positive integer, got '{args[nd_idx + 1]}'")
sys.exit(1)
args.pop(nd_idx) # remove '--days'
args.pop(nd_idx) # remove the value

if len(args) < 1:
print_help()
sys.exit(1)
# Distinguish bare `canvigator.py --help` (success) from `canvigator.py`
# with no args at all (usage error).
sys.exit(0 if help_requested else 1)

task = args[0]

if task in ("help", "--help"):
if task == "help":
print_help()
sys.exit(0)

if task not in tasks:
print(f"Invalid task: '{task}'. Run with --help to see available tasks.")
sys.exit(1)

if help_requested:
import canvigator_help as ch
ch.print_task_help(task)
sys.exit(0)

# export-anon-data works with local files only — no Canvas API needed
if task == 'export-anon-data':
import logging
Expand Down Expand Up @@ -293,30 +339,18 @@ def _run_quiz_task(task, quiz, dry_run, tag, reply_window_days):

course = cc.CanvigatorCourse(canvas, course_choice, canv_config, verbose=False)

if task == 'get-activity':
course.saveStudentActivity(canv_config.data_path)
if task in COURSE_TASKS:
_run_course_task(task, course, canv_config, n_days, cloud_questions_flag)

elif task == 'get-quiz-submission-events' and all_quizzes_flag:
course.getAllQuizzesAndSubmissions()

elif task == 'send-quiz-reminder' and all_quizzes_flag:
course.sendAllQuizReminders(dry_run=dry_run)

elif task == 'create-quiz':
course.createQuiz()

elif task in ('create-media-recording-assignment', 'get-media-recordings', 'analyze-media-recordings'):
_run_assignment_task(task, course, canvas, canv_config, dry_run, auto_grade_flag)

elif task == 'get-gradebook':
course.exportGradebook(canv_config.data_path)

elif task == 'get-roster':
course.exportRoster(canv_config.data_path)

elif task == 'get-conversations':
course.exportConversations(canv_config.data_path)

elif task == 'generate-follow-up-questions':
# Driven by a pre-selected tagged-questions CSV, not a Canvas quiz
tagged_csv = cu.selectCSVFromList(
Expand Down
Loading
Loading