From f11ae3637d7ec77a44fe6c9b217dbd308a19d4ba Mon Sep 17 00:00:00 2001 From: yimin12 Date: Wed, 13 May 2026 02:12:50 -0400 Subject: [PATCH] Backfill missing documentation Nightshift-Task: docs-backfill Nightshift-Ref: https://github.com/marcus/nightshift --- docs/COPILOT_INTEGRATION.md | 136 +++++++++++++++++++ docs/SPEC.md | 255 ++++++++++++++++++++++++++++++++++++ 2 files changed, 391 insertions(+) create mode 100644 docs/COPILOT_INTEGRATION.md create mode 100644 docs/SPEC.md diff --git a/docs/COPILOT_INTEGRATION.md b/docs/COPILOT_INTEGRATION.md new file mode 100644 index 0000000..e388e88 --- /dev/null +++ b/docs/COPILOT_INTEGRATION.md @@ -0,0 +1,136 @@ +# GitHub Copilot Integration + +Nightshift can run tasks with GitHub Copilot through either the GitHub CLI +Copilot extension or the standalone `copilot` binary. + +## Requirements + +- A GitHub account with a Copilot subscription. +- Either `gh` with the `github/gh-copilot` extension, or the standalone + `copilot` command. +- A working login for the CLI you choose. + +Install with the GitHub CLI: + +```bash +gh extension install github/gh-copilot +gh auth login +``` + +Or install the standalone CLI: + +```bash +npm install -g @github/copilot +# or +curl -fsSL https://gh.io/copilot-install | bash +``` + +Nightshift prefers the standalone `copilot` binary when it is available. If it +is not in `PATH`, it falls back to `gh copilot`. + +## Configuration + +Copilot uses the same provider configuration model as Claude Code and Codex: + +```yaml +providers: + preference: + - claude + - codex + - copilot + copilot: + enabled: true + data_path: "~/.copilot" + dangerously_skip_permissions: false +``` + +`providers.preference` controls provider selection order. Copilot is enabled by +default in config loading, but it is only selected when its CLI is available and +its budget allowance is positive. + +`providers.copilot.data_path` is where Nightshift reads and writes its local +Copilot budget tracking file. The default is `~/.copilot`. + +## Execution Model + +Nightshift invokes Copilot in non-interactive prompt mode: + +```bash +copilot -p "" --no-ask-user --silent +``` + +When using `gh`, the same flags are passed through to the Copilot extension: + +```bash +gh copilot -- -p "" --no-ask-user --silent +``` + +The `--no-ask-user` flag disables Copilot's ask-user tool so scheduled runs do +not block waiting for input. The `--silent` flag keeps output focused on the +agent response. + +If `providers.copilot.dangerously_skip_permissions: true`, Nightshift also adds: + +```bash +--allow-all-tools --allow-all-urls +``` + +Leave this disabled unless the target repository and execution environment are +trusted. It allows Copilot to use tools and URLs without interactive approval. + +## Budget Tracking + +GitHub Copilot does not expose an authoritative local usage file or API endpoint +for Nightshift to query. Nightshift therefore models Copilot as a monthly, +request-based provider: + +- each counted request is treated as one premium request; +- the local counter resets on the first day of each month at 00:00 UTC; +- the counter is stored at `~/.copilot/nightshift-usage.json`, or under the + configured `providers.copilot.data_path`; +- `budget.weekly_tokens` or `budget.per_provider.copilot` is converted to an + approximate monthly request limit by multiplying the configured value by four. + +Current limitation: the provider includes `IncrementRequestCount`, but the +`nightshift run` and `nightshift task run` execution paths do not call it after +successful Copilot tasks. Budget displays and provider selection can read an +existing local counter, but Nightshift does not yet automatically count Copilot +executions it launches. External Copilot usage is also not visible to +Nightshift. + +Because of those limits, Copilot budget enforcement is an estimate. Use a +conservative `budget.per_provider.copilot` value if you rely on Copilot monthly +request limits: + +```yaml +budget: + per_provider: + copilot: 100 +``` + +## Troubleshooting + +Check that Nightshift can see the CLI: + +```bash +nightshift doctor +``` + +If Copilot is skipped during provider selection: + +- confirm `providers.copilot.enabled: true`; +- confirm `providers.preference` includes `copilot` if you customized the list; +- run `which copilot` or `which gh`; +- for `gh`, run `gh extension list` and confirm `github/gh-copilot` is present; +- run the Copilot CLI directly to confirm authentication. + +If a scheduled run hangs or fails on permissions, either keep Copilot later in +`providers.preference` and use Claude/Codex for unattended runs, or explicitly +enable `providers.copilot.dangerously_skip_permissions` for trusted projects. + +## Related Docs + +- [Technical specification](SPEC.md) +- [Run lifecycle](guides/run-lifecycle.md) +- [Configuration docs](../website/docs/configuration.md) +- [Integrations docs](../website/docs/integrations.md) diff --git a/docs/SPEC.md b/docs/SPEC.md new file mode 100644 index 0000000..b4fc626 --- /dev/null +++ b/docs/SPEC.md @@ -0,0 +1,255 @@ +# Nightshift Technical Specification + +This document is a current implementation reference for Nightshift's runtime +configuration, provider model, budget model, task selection, and storage +locations. + +## Architecture + +Nightshift is a Go CLI that selects maintenance tasks, chooses an available AI +provider, and runs a plan, implement, review loop in a target repository. + +Main components: + +- `cmd/nightshift/commands`: Cobra commands such as `run`, `task`, `budget`, + `doctor`, `setup`, and `daemon`. +- `internal/config`: YAML loading, defaults, environment overrides, and + validation. +- `internal/providers`: provider-side usage and budget data for Claude, Codex, + and Copilot. +- `internal/agents`: command execution wrappers for Claude Code, Codex, and + GitHub Copilot. +- `internal/budget`: provider allowance calculation. +- `internal/tasks`: built-in and custom task definitions plus scoring. +- `internal/orchestrator`: plan, implement, review execution loop and PR + metadata handling. +- `internal/state` and `internal/db`: SQLite-backed run state. + +## Configuration Loading + +Nightshift loads configuration in this order: + +1. global config at `~/.config/nightshift/config.yaml`; +2. project config named `nightshift.yaml` in the current or selected project; +3. environment variables with the `NIGHTSHIFT_` prefix. + +Run `nightshift setup` for interactive configuration. Per-project config +overrides selected global fields during project resolution. + +## Configuration Schema + +Representative schema: + +```yaml +schedule: + cron: "0 2 * * *" + interval: "8h" + window: + start: "22:00" + end: "06:00" + timezone: "America/New_York" + max_projects: 1 + max_tasks: 1 + +budget: + mode: daily + max_percent: 75 + aggressive_end_of_week: false + reserve_percent: 5 + weekly_tokens: 700000 + per_provider: + claude: 700000 + codex: 700000 + copilot: 100 + billing_mode: subscription + calibrate_enabled: true + snapshot_interval: 30m + snapshot_retention_days: 90 + week_start_day: monday + db_path: "~/.local/share/nightshift/nightshift.db" + +providers: + preference: + - claude + - codex + - copilot + claude: + enabled: true + data_path: "~/.claude" + dangerously_skip_permissions: false + codex: + enabled: true + data_path: "~/.codex" + dangerously_bypass_approvals_and_sandbox: false + copilot: + enabled: true + data_path: "~/.copilot" + dangerously_skip_permissions: false + +projects: + - path: ~/code/project + priority: 1 + tasks: + - lint-fix + - pattern: ~/code/oss/* + exclude: + - ~/code/oss/archived + +tasks: + enabled: + - lint-fix + - docs-backfill + disabled: [] + priorities: + lint-fix: 1 + intervals: + lint-fix: "24h" + custom: + - type: dependency-review + name: "Dependency Review" + description: "Review dependency health and open a PR for safe updates." + category: safe + cost_tier: medium + risk_level: low + interval: "168h" + +integrations: + claude_md: true + agents_md: true + task_sources: + - td: + enabled: true + teach_agent: true + +logging: + level: info + path: "~/.local/share/nightshift/logs" + format: json + +reporting: + morning_summary: true + email: + slack_webhook: +``` + +`schedule.cron` and `schedule.interval` are mutually exclusive. `budget.mode` +must be `daily` or `weekly`. `budget.billing_mode` must be `subscription` or +`api`. `budget.week_start_day` must be `monday` or `sunday`. + +## Provider Model + +Nightshift separates providers from agents: + +- providers expose usage and budget data; +- agents execute prompts through local CLIs. + +Supported execution providers: + +- `claude`: Claude Code CLI, data path default `~/.claude`; +- `codex`: Codex CLI, data path default `~/.codex`; +- `copilot`: GitHub Copilot CLI, data path default `~/.copilot`. + +Provider selection follows `providers.preference`, skipping disabled providers, +providers whose CLI is unavailable, and providers with no budget allowance unless +`--ignore-budget` is set. + +## Budget Model + +Nightshift calculates an allowance before selecting tasks: + +- daily mode uses `weekly_budget / 7` as the daily budget; +- weekly mode uses the remaining weekly budget divided by days until reset; +- `max_percent` limits how much of the available budget a run may use; +- `reserve_percent` is subtracted from the calculated allowance; +- predicted daytime usage may be reserved when trend data is available. + +Claude and Codex are token-based providers. Codex can use rate-limit data from +session JSONL files, with token totals as a fallback. Subscription calibration +uses snapshots to infer a weekly token budget and falls back to +`budget.weekly_tokens` or `budget.per_provider`. + +Copilot is modeled as monthly request usage because GitHub does not expose +authoritative usage data to Nightshift. The local Copilot counter is stored in +`nightshift-usage.json` under the Copilot data path. The current task execution +paths do not automatically increment that counter after Copilot runs, so Copilot +budget enforcement is approximate. + +## Task Selection + +Tasks are selected per project after provider allowance is known. + +Selection inputs: + +- task definitions and estimated token ranges; +- `tasks.enabled` and `tasks.disabled`; +- per-task priorities from `tasks.priorities`; +- per-project task overrides; +- cooldowns from task defaults or `tasks.intervals`; +- project state recorded in SQLite; +- optional `--task`, `--random-task`, `--max-projects`, and `--max-tasks`. + +Custom tasks require `type`, `name`, and `description`. Optional metadata +includes `category`, `cost_tier`, `risk_level`, and `interval`. + +## Runtime Workflow + +`nightshift run` performs these steps: + +1. load config and initialize logging; +2. open the SQLite database; +3. clear stale task assignments; +4. initialize providers and budget manager; +5. resolve projects; +6. select a base branch; +7. register custom tasks; +8. build and display a preflight plan; +9. confirm execution unless skipped by `--yes` or a non-TTY environment; +10. run each selected task through the orchestrator; +11. record state and write reports. + +`nightshift task run --provider ` bypasses automatic provider +selection and runs a single task in the selected project. + +## Storage Locations + +Default locations: + +| Data | Location | +|------|----------| +| Global config | `~/.config/nightshift/config.yaml` | +| Project config | `nightshift.yaml` | +| SQLite database | `~/.local/share/nightshift/nightshift.db` | +| Logs | `~/.local/share/nightshift/logs` | +| Reports | `~/.local/share/nightshift/reports` | +| Summaries | `~/.local/share/nightshift/summaries` | +| Claude data | `~/.claude` | +| Codex data | `~/.codex` | +| Copilot data | `~/.copilot` | + +`budget.db_path`, `logging.path`, and provider `data_path` fields can override +the defaults. + +## Safety Model + +Nightshift is designed for unattended execution but keeps dangerous provider +flags explicit: + +- Claude only receives `--dangerously-skip-permissions` when + `providers.claude.dangerously_skip_permissions` is enabled. +- Codex agent construction defaults to headless execution for unattended use; + the config field is still written and displayed by setup. +- Copilot only receives `--allow-all-tools --allow-all-urls` when + `providers.copilot.dangerously_skip_permissions` is enabled. + +Interactive `nightshift run` displays a preflight summary and asks for +confirmation. Non-TTY runs auto-confirm. Use `--dry-run` to inspect planned work +without executing it. + +## Related Docs + +- [Copilot integration](COPILOT_INTEGRATION.md) +- [Run lifecycle](guides/run-lifecycle.md) +- [Codex budget tracking](guides/codex-budget-tracking.md) +- [Configuration docs](../website/docs/configuration.md) +- [Budget docs](../website/docs/budget.md) +- [Tasks docs](../website/docs/tasks.md)