diff --git a/CHANGELOG.md b/CHANGELOG.md index 59afa7d..cb536f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,23 @@ All notable changes to nightshift are documented in this file. +## [v0.3.4] - 2026-02-28 + +### Features +- **Configurable agent timeouts** — add `--timeout` to `nightshift run` and `nightshift daemon`, with daemon re-exec forwarding the flag (#27) +- **Expanded PII scanner guidance** — add detailed instructions for detecting hardcoded PII, leaked env files, unsafe storage, and related exposure patterns in the built-in task (#34) + +### Fixes +- **Timeout handling and diagnostics** — preserve partial output on timeout, terminate full process groups, and surface partial logs from failed plan/implement/review steps (#33) +- **Copilot CLI integration** — improve binary resolution, permission gating, and CLI flag handling for Copilot runs (#39) +- **Provider config YAML serialization** — write provider settings with the correct YAML keys during setup (#43) +- **Configured run limits and budget fallback** — honor `schedule.max_projects` and `schedule.max_tasks`, improve budget calibration at day and week boundaries, and preserve Codex fallback permissions in headless runs (#42) + +### Other +- **Task reference docs** — add a comprehensive reference page for all 59 built-in tasks and refresh related task documentation (#30) +- **Docs cleanup** — remove auto-generated implementation docs from the repository (#40) +- **Low-risk cleanup** — resolve Copilot helper lint warnings and replace `WriteString(fmt.Sprintf(...))` patterns with `fmt.Fprintf` in reporting and setup code (#38, #41) + ## [v0.3.3] - 2026-02-19 ### Features diff --git a/docs/HISTORY.md b/docs/HISTORY.md new file mode 100644 index 0000000..47118bb --- /dev/null +++ b/docs/HISTORY.md @@ -0,0 +1,293 @@ +# Nightshift: Historical Context + +This document is a curated summary of the historical context of the Nightshift +codebase: how the project came to exist, which components were built (and when), +what major decisions shaped the architecture, and which recent fixes have +stabilized the system. It complements `README.md` (user-facing intro) and +`docs/permissions-auth-surface-map.md` (current security/auth surface). + +Commits are referenced by short SHA. To inspect any one in detail: + +```bash +git show +``` + +--- + +## 1. Project Origin + +Nightshift began in the first quarter of the project's life as a Go CLI that +runs coding agents overnight against your own repositories — turning otherwise +idle weekly token allotments into background dead-code/doc-drift/test-gap/etc. +work. The earliest commits scaffold the project and lay the design contract: + +- [`628f01d`] chore: initialize git repo with Go gitignore +- [`be0865f`] docs: add README and MIT license +- [`a84934f`] docs: add technical specification (SPEC.md) +- [`3233791`] chore: scaffold Go project structure +- [`8dffb29`] docs: incorporate review notes into SPEC.md + +Two design choices set the tone for everything that followed: + +1. **Branch/PR-only output.** Nightshift never writes directly to a primary + branch; every run lands in a feature branch / PR so users can close what + they don't like. This is the "whole rollback plan" mentioned in + `README.md`. +2. **Budget-aware execution.** From day one the agent was meant to live inside + weekly token allotments, deferring work rather than blowing past quota. + +--- + +## 2. Component-by-Component Timeline + +### 2.1 Configuration (`internal/config`) + +- [`91de755`] feat(config): implement YAML config loading and validation + — initial schema, validation, defaults. +- [`ba73293`] chore(config): change default `max_percent` from 10 → 75. + Codified the user expectation that idle quota should mostly be used. +- [`99bcd82`] feat: add `CustomTaskConfig` struct — first-class user-defined + tasks alongside built-ins. +- [`ee98bd8`] feat: add custom task config validation. +- [`615e2b9`] fix(#19): config `max_projects`, budget day-boundary, codex + fallback flags — stabilization pass on long-standing UX issues. +- [`519821c`] fix: serialize provider config with correct YAML key names + (fixes #20) — round-trip bug surfaced once the setup wizard started writing + config back to disk. + +### 2.2 Logging & State (`internal/logging`, `internal/state`) + +- [`81f95f1`] feat(logging): structured logging with rotation. +- [`b6f3a80`] feat(state): run history and task tracking. +- [`8220d43`] feat: task cooldown / interval system — prevents the same task + from being picked every night. + +### 2.3 CLI Surface (`cmd/nightshift`, `internal/agents`, `internal/orchestrator`) + +The CLI is structured around cobra. Commands landed roughly in the order a +user would touch them: + +- [`4e3e9b1`] feat(cli): cobra command structure +- [`ae85ac7`] feat(cli): `init` +- [`bfe2ca5`] feat(cli): `run` +- [`20f7479`] feat(cli): `daemon` +- [`274afae`] feat(cli): `status` / `logs` +- [`7486bd9`] feat(cli): `budget` +- [`1da2620`] feat(cli): `config` +- [`ce58dc8`] feat(cli): `install` / `uninstall` +- [`ccef144`] feat(commands): `task list/show/run` subcommands +- [`9c56de6`] feat: `nightshift stats` (with JSON output) + +Run-UX work over several iterations: + +- [`7eaba8e`] `--max-projects` / `--max-tasks` flags +- [`d6ab73a`] `--ignore-budget` flag +- [`bcbb941`] preflight summary display +- [`bd8cd82`] confirmation prompt + `--yes` flag +- [`43cb05d`] `--dry-run` shows preflight and exits +- [`85480af`] `--random-task` flag +- [`0b33782`] live colored output in interactive mode +- [`4c0a915`] `--timeout` flag for `run` and `daemon` + +### 2.4 Providers (`internal/providers`) + +Provider support grew from Claude → Codex → Copilot, with parsing fidelity +improving in passes: + +- [`7ceda39`] / [`cbfe759`] feat(providers): Claude data parser +- [`9b4e553`] feat(providers): Codex data parser +- [`d6c21dd`] feat(providers): parse Codex session JSONL for daily usage +- [`63463f4`] feat(providers): parse and store reset times from scraped output +- [`63a297d`] fix(providers): compute **billable** tokens excluding cached input +- [`0b2fec0`] / [`d88b989`] / [`3430191`] direct JSONL scanning for Claude, + then revert to stats-cache as primary with JSONL as fallback (a deliberate + walk-back after stability issues). +- [`caa932d`] Add GitHub Copilot CLI support with monthly budget tracking +- [`3337893`] Fix copilot cli integration (PR #39) +- [`c703999`] fix: resolve lint warnings in copilot provider and helpers + +### 2.5 Budget (`internal/budget`) + +- [`5d6779e`] feat(budget): initial calculation algorithm +- [`cf8c770`] feat(budget): wire Codex daily token usage in +- [`b9f7cdf`] fix: budget codex daily mode uses token-based calculation over + 5h window +- [`e5c7c88`] fix(budget): improve display clarity +- [`24f54cd`] feat(budget): display reset times from scraped snapshots +- [`b9b07b4`] fix: show budget period label (today/this week) on usage bar +- [`d78cb36`] fix: use week-aware avg daily usage in budget projection + +### 2.6 Tasks (`internal/tasks`) + +- [`b486f26`] feat(tasks): task definitions and cost estimation +- [`a9ab729`] feat(tasks): task selection and priority scoring +- [`59ecb25`] feat: `RegisterCustom`, `IsCustom`, `ClearCustom` +- [`93a34c7`] feat: wire custom task registration into `run` and `preview` +- [`5baae99`] feat: add `SelectRandom` method to task selector +- [`073c05a`] feat(tasks): detailed agent instructions for PII Exposure Scanner + +### 2.7 Agents & Orchestrator (`internal/agents`, `internal/orchestrator`) + +- [`367ac50`] / [`b6e8084`] Claude Code CLI adapter +- [`2f57b0d`] Codex CLI adapter +- [`a492d86`] / [`f98c1ca`] plan-implement-review loop +- [`8c85a3a`] fix: switch codex non-interactive mode to `exec` (community PR + #11, brandon93s) +- [`b13164a`] fix: capture partial output on timeout and kill process groups + +### 2.8 Scheduler (`internal/scheduler`) + +- [`c4a831f`] feat(scheduler): cron and time-window scheduling +- [`82baf56`] Clarification of budgets for nighttime vs daytime + +### 2.9 Reporting & Stats (`internal/reporting`, `internal/stats`) + +- [`6162ec8`] feat(reporting): morning summary generator +- [`5226c88`] feat: run reports + provider preference +- [`cd5c7b7`] feat: task detail rows in report overview +- [`91c5f7a`] feat: show PR links prominently +- [`f36e30a`] feat: extract PR URLs from agent output +- [`93d616e`] feat: polish report summary card (duration, success rate) +- [`9c0b25b`] feat: stats computation +- [`512d11b`] fix: derive stats run count and dates from report files +- [`74875ec`] fix: show "exhausted" for past projected timestamps + +### 2.10 Setup & Calibration (`internal/setup`, `internal/calibrator`) + +- [`903dfd5`] feat: setup wizard +- [`a487ad1`] feat: improve setup UX +- [`e3170de`] Add provider calibration tooling and run_history provider + tracking +- [`3348c51`] fix: stats budget projection uses calibrator instead of + snapshot inference + +### 2.11 Security & Safety (`internal/security`) + +- [`af56947`] feat(security): security and safety features +- [`f070c93`] fix: guard sensitive paths (community PR #14, davemac) +- [`5e80e71`] fix: block task run in sensitive directories (home, root, tmp) +- [`f30eee7`] feat: enforce branch+PR prompts and dangerous-flags defaults +- [`8eb2cb5`] feat: inject nightshift run metadata into PRs and commits +- See also: `docs/permissions-auth-surface-map.md` for the current + permissions/auth landscape, derived from the code above. + +### 2.12 Projects (`internal/projects`) + +- [`d0f5c19`] feat(projects): multi-project discovery and budget allocation +- [`18815f1`] feat: branch selection ability (community PR #12) +- [`9237154`] fix: apply `--max-projects` limit **after** processed-today + filter (fixes #21) — the off-by-one of project filtering. + +### 2.13 Bus-Factor Analyzer (`internal/analysis`) + +- [`7f24d07`] feat: implement bus-factor analyzer for code ownership + concentration (merged via PR #4) +- [`1f7ecc8`] fix: correct markdown rendering of recommendations +- See also: `docs/bus-factor.md`. + +### 2.14 Website & Docs + +- [`3cb3bf2`] feat: add Docusaurus website with warm nighttime theme +- [`939c50f`] docs: comprehensive task reference page for all 59 built-in + tasks +- [`ebbee00`] docs: remove auto-generated implementation docs (cleanup) +- [`91c25ba`] docs: update guides, website docs, README + +### 2.15 TUI (removed) + +A Bubbletea-based TUI was implemented and later removed once the CLI matured +and live colored output covered most needs: + +- [`057d616`] feat(ui): implement bubbletea TUI for monitoring +- [`aba9798`] Remove TUI and related docs + +--- + +## 3. Release Milestones + +| Version | Commit | Notes | +|---------|--------------|------------------------------------------------------| +| v0.2.0 | [`d051ca8`] | Run UX, preflight, dry-run, confirmation, flags | +| v0.3.0 | [`c59dcb9`] | Bus-factor, security hardening, backward-compat tests| +| v0.3.1 | (migration: `docs/MIGRATION-v0.3.0-to-v0.3.1.md`) | | +| v0.3.2 | [`0b5e905`] | Codex exec fix, sensitive-path guards | +| v0.3.3 | [`1e89a6c`] | Branch selection, gofmt fixes | +| v0.3.4 | [`2ebd805`] | Provider config serialization, max-projects filter, | +| | | mobile navbar fix, pre-commit hook | + +The release process (goreleaser, version bumping, tag/push) is documented in +the `nightshift-release` skill — see CLAUDE-CODE skill metadata, not a file in +this repo. + +--- + +## 4. Recent Stabilization Pass (around v0.3.4) + +The most recent commits on `main` are a focused stabilization pass rather than +new features. They sit at the top of `git log --first-parent main`: + +- [`68bbf11`] **fix: mobile navbar sidebar hidden behind secondary panel** + (#46) — Docusaurus site z-index/layering on small viewports. +- [`9237154`] **fix: apply `--max-projects` limit after processed-today + filter** (#45, fixes #21) — limit was being applied before filtering, so + users could end up with zero projects to run. +- [`13720ae`] **Add pre-commit hook for gofmt/vet/build** (#44) — keep + lint-fix churn out of `main` by failing the commit locally. +- [`519821c`] **fix: serialize provider config with correct YAML key names** + (#43, fixes #20) — write path didn't match the read schema. +- [`615e2b9`] **fix(#19): config `max_projects`, budget day-boundary, codex + fallback flags** (#42) — bundled config/budget/provider polish. +- [`c703999`] **fix: resolve lint warnings in copilot provider and helpers** + (#38) — long tail of the Copilot integration landing. +- [`6d34c71`] **refactor: replace `WriteString(fmt.Sprintf(...))` with + `fmt.Fprintf`** (#41) — cosmetic but caught by linters. + +A series of `codex/lint-fix-*` and `chore/lint-fix-*` branches in +`git branch -a` are the staging areas for this work; the squashed result is +what shows up on `main`. + +--- + +## 5. Architectural Decisions Worth Knowing + +These aren't documented in any single ADR file but are visible in the commit +history above: + +1. **Provider abstraction with explicit billable-token semantics.** + `63a297d` cemented the rule that cached input tokens are excluded from + billing math; all later budget code assumes that contract. +2. **Stats-cache primary, JSONL fallback** for Claude token counting. + `0b2fec0 → d88b989 → 3430191` walked from "JSONL first" back to + "stats-cache first," because the JSONL path was slower and noisier than + the cache in practice. +3. **Branch + PR is mandatory.** `f30eee7` made branch/PR prompts and + dangerous-flag defaults part of orchestrator behavior, not an opt-in. + This is enforced at the agent layer so all providers inherit it. +4. **Sensitive-directory guard.** `5e80e71` + `f070c93` block runs in + `$HOME`, `/`, `/tmp` — accidental "run in the wrong directory" is the + single largest blast-radius failure mode for an unattended overnight + agent. +5. **Calibrator is the source of truth for projection.** `3348c51` moved + stats projection off scraped snapshots onto the calibrator, since + snapshots lag and the calibrator already integrates run history. + +--- + +## 6. Cross-References + +- `README.md` — user-facing project intro, install/quick-start. +- `docs/permissions-auth-surface-map.md` — present-day permissions/auth + surface, derived from `internal/security`, the orchestrator, and provider + adapters above. +- `docs/bus-factor.md` — bus-factor analyzer output format and methodology + (see §2.13). +- `docs/MIGRATION-v0.3.0-to-v0.3.1.md` — the only formal migration guide so + far. +- `docs/guides/` — task-author / user guides. +- `docs/implemented/` — historical implementation notes for landed features. +- `docs/deprecated/` — features removed but worth preserving context for. + +--- + +*This file is hand-curated and will drift over time. When in doubt, trust +`git log --first-parent main` over this document.*