Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .bully.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,25 @@ rules:
engine: semantic
scope: "pipeline/tests/test_*.py"
severity: warning

prefer-perf-counter:
description: >
Use `time.perf_counter()` for measuring elapsed time, not `time.time()`.
`time.time()` is a wall-clock that can jump backwards on NTP sync;
`perf_counter()` is monotonic and ideal for latency measurement.
engine: ast
scope: ["pipeline/pipeline.py", "pipeline/analyzer.py"]
severity: error
pattern: "time.time()"

no-shell-true-subprocess:
description: >
Do not pass `shell=True` to `subprocess.run`. Shell-mode invocation
mixes interpolation with shell parsing and is the most common path
to command injection. Pass a list of args instead, or if the call
is genuinely safe (input is `shlex.quote`'d, etc.), add a per-line
`bully-disable: no-shell-true-subprocess <reason>`.
engine: ast
scope: ["pipeline/pipeline.py", "pipeline/analyzer.py"]
severity: error
pattern: "subprocess.run($$$, shell=True, $$$)"
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ name: CI

on:
push:
branches: [main]
pull_request:

jobs:
Expand Down Expand Up @@ -31,5 +30,8 @@ jobs:
- name: pytest
run: pytest -q

- name: install ast-grep
run: pip install ast-grep-cli

- name: bully dogfood
run: bash scripts/dogfood.sh
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ __pycache__/
# Per-developer telemetry (see docs/telemetry.md)
.bully/

# Backlog.md local state
.backlog/

# Python build artifacts
build/
dist/
Expand Down
29 changes: 29 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

<!-- BACKLOG.MD MCP GUIDELINES START -->

<CRITICAL_INSTRUCTION>

## BACKLOG WORKFLOW INSTRUCTIONS

This project uses Backlog.md MCP for all task and project management activities.

**CRITICAL GUIDANCE**

- If your client supports MCP resources, read `backlog://workflow/overview` to understand when and how to use Backlog for this project.
- If your client only supports tools or the above request fails, call `backlog.get_backlog_instructions()` to load the tool-oriented overview. Use the `instruction` selector when you need `task-creation`, `task-execution`, or `task-finalization`.

- **First time working here?** Read the overview resource IMMEDIATELY to learn the workflow
- **Already familiar?** You should have the overview cached ("## Backlog.md Overview (MCP)")
- **When to read it**: BEFORE creating tasks, or when you're unsure whether to track work

These guides cover:
- Decision framework for when to create tasks
- Search-first workflow to avoid duplicates
- Links to detailed guides for task creation, execution, and finalization
- MCP tools reference

You MUST read the overview resource to understand the complete workflow. The information is NOT summarized here.

</CRITICAL_INSTRUCTION>

<!-- BACKLOG.MD MCP GUIDELINES END -->
45 changes: 30 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Bully is a lint pipeline for Claude Code. Every `Edit` / `Write` hits a `PostToo

## The config

A `.bully.yml` is a flat list of rules. Each rule says what to check, where it applies, how bad it is, and which engine runs it `script` (deterministic shell command) or `semantic` (natural-language rule the agent evaluates against the diff):
A `.bully.yml` is a flat list of rules. Each rule says what to check, where it applies, how bad it is, and which engine runs it -- `script` (deterministic shell command), `ast` (structural pattern via [ast-grep](https://ast-grep.github.io/)), or `semantic` (natural-language rule the agent evaluates against the diff):

```yaml
schema_version: 1
Expand All @@ -28,6 +28,13 @@ rules:
severity: error
script: "grep -nE 'console\\.log\\(' {file} && exit 1 || exit 0"

no-any-cast:
description: "No `as any` casts -- use a precise type or `unknown` plus narrowing."
engine: ast
scope: ["src/**/*.ts", "src/**/*.tsx"]
severity: error
pattern: "$EXPR as any"

prefer-derived-state:
description: >
React components should not use `useEffect` to derive state from
Expand All @@ -39,7 +46,9 @@ rules:
severity: warning
```

The first rule runs a grep on every edited `.ts`/`.tsx`; the second ships the diff to the agent with the description as the evaluation prompt. No plugins, no DSL -- just globs, shell, and prose.
The first rule runs a grep on every edited `.ts`/`.tsx`. The second matches the structural pattern with ast-grep -- ignores comments, strings, and formatting variants. The third ships the diff to the agent with the description as the evaluation prompt. No plugins, no DSL -- just globs, shell, ast patterns, and prose.

`engine: ast` requires `ast-grep` on `$PATH` (`brew install ast-grep`, `cargo install ast-grep`, or `pip install ast-grep-cli`). If missing, ast rules are skipped at runtime with a one-line stderr hint and `bully doctor` flags it.

Need to share rules across your own repos? `extends:` accepts any relative or absolute path to another `.bully.yml`:

Expand All @@ -59,10 +68,10 @@ Local rules override inherited rules of the same id.
<img src="bully-flow.png" alt="Bully flow: Edit/Write → PostToolUse hook → script phase → semantic phase → block or pass" width="500">
</p>

1. **Script phase** -- deterministic checks (grep, awk, shell-out to a linter). Fast. Fails the tool call on error-severity violations via exit code 2.
2. **Semantic phase** -- if the script phase passes, the pipeline hands a unified diff plus rule descriptions to the evaluator subagent. Structured verdicts come back; the parent session surfaces them.
1. **Script + AST phase** -- deterministic checks. `script` rules shell out (grep, awk, linters); `ast` rules run structural patterns through ast-grep. Both are fast and fail the tool call on error-severity violations via exit code 2.
2. **Semantic phase** -- if the deterministic phase passes, the pipeline hands a unified diff plus rule descriptions to the evaluator subagent. Structured verdicts come back; the parent session surfaces them.

Deterministic rules stay as shell. Judgment rules ("inline single-use variables", "don't derive state with `useEffect`") live as plain English the agent evaluates against the diff. Same trigger, same output format, same fix loop -- across every language in the repo.
Deterministic rules stay as shell or ast patterns. Judgment rules ("inline single-use variables", "don't derive state with `useEffect`") live as plain English the agent evaluates against the diff. Same trigger, same output format, same fix loop -- across every language in the repo.

## Prerequisites

Expand All @@ -87,10 +96,16 @@ To change the evaluator model, set the plugin's agent override or edit `model:`
### Verify the install

```bash
python3 "$(ls -d ~/.claude/plugins/cache/*/bully/*/ | tail -1)pipeline/pipeline.py" --doctor
bully doctor
```

`--doctor` checks Python version, config presence and parse-ability, hook wiring, evaluator-agent registration, and each skill. One line per check, `[OK]` or `[FAIL]`.
`doctor` checks Python version, config presence and parse-ability, hook wiring, evaluator-agent registration, and each skill. One line per check, `[OK]` or `[FAIL]`.

If `bully` isn't on `$PATH` (e.g., you skipped `pip install -e .`), call the pipeline directly as a fallback:

```bash
python3 "$(ls -d ~/.claude/plugins/cache/*/bully/*/ | tail -1)pipeline/pipeline.py" --doctor
```

### Manual install (fallback)

Expand Down Expand Up @@ -140,7 +155,7 @@ The init skill detects your stack, scans for existing linter configs, asks a cou
A fresh rule across an existing codebase lights up every pre-existing problem. Baseline the current state so only _new_ violations block edits:

```bash
python3 ~/.bully/pipeline/pipeline.py --baseline-init --glob "src/**/*.ts"
bully baseline-init --glob "src/**/*.ts"
```

That writes `.bully/baseline.json`. Future runs ignore anything recorded there. See [docs/design.md](docs/design.md) for the contract.
Expand Down Expand Up @@ -178,15 +193,15 @@ The `bully-author` skill walks through engine choice, drafts the rule, tests it
For authoring and debugging rules without triggering an Edit:

```bash
PIPE=~/.bully/pipeline/pipeline.py

python3 "$PIPE" --validate # parse + enum checks
python3 "$PIPE" --file src/foo.php # full pipeline on a file
python3 "$PIPE" --file src/foo.php --rule no-compact # isolate one rule
python3 "$PIPE" --file src/foo.php --print-prompt # see the semantic prompt
python3 "$PIPE" --show-resolved-config # rules after extends:
bully validate # parse + enum checks
bully lint src/foo.php # full pipeline on a file
bully lint src/foo.php --rule no-compact # isolate one rule
bully lint src/foo.php --print-prompt # see the semantic prompt
bully show-resolved-config # rules after extends:
```

`bully` is the console script installed by `pip install -e .`. If you can't install the package, call the pipeline directly: `python3 ~/.bully/pipeline/pipeline.py --validate` (or with `--file`, `--show-resolved-config`, etc.).

## Uninstall

Plugin install:
Expand Down
46 changes: 41 additions & 5 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,42 @@
# Security Policy

## Reporting a Vulnerability
## Trust boundary: `.bully.yml`

bully runs deterministic shell commands and dispatches a Claude subagent. If you find a way to escape the sandboxed scope or inject commands through a crafted `.bully.yml` or through file contents that bypass the script-rule scoping, please report it privately.
**A `.bully.yml` can execute arbitrary shell commands on your machine.** Every `engine: script` rule runs its `script:` field through the shell against files you edit. That's the whole point of the tool -- but it also means a malicious or careless `.bully.yml` in a repo you clone is equivalent to running a setup script you didn't review.

bully gates this with a per-machine trust allowlist:

1. The first time you edit a file in a repo, the hook sees an untrusted `.bully.yml` and **refuses to execute any rules**. It prints a one-line stderr hint and does **not** block the edit itself -- your tool call succeeds, nothing runs.
2. To enable the config on this machine, review it, then run:
```
bully trust # from the repo root, or: --config <path>
```
This records the SHA256 of the config (and every file it `extends:`) in `~/.bully-trust.json`.
3. Every subsequent run re-verifies the checksum. If the config changes, you see a "changed since last trust" message and rules stop running until you re-review:
```
bully trust --refresh
```

The trust state is **machine-local**. It is never committed to repos and never shared between machines. A teammate cloning a repo you trust must trust it themselves.

### Bypass for CI and automation

Set `BULLY_TRUST_ALL=1` to disable the gate unconditionally. Use this only in environments where the config is already trusted through other means:

- **CI pipelines** -- the config is reviewed as part of the repo's code review; the CI runner treats the repo as trusted.
- **Dogfood / test scripts** -- the `.bully.yml` is the repo's own, and changes arrive through normal PR review.

Do **not** set `BULLY_TRUST_ALL=1` in an interactive developer shell as a default. That defeats the gate.

### What trust does NOT protect

- If you trust a config that was already malicious, you authorized the scripts. Trust gates subsequent changes, not initial review.
- Running `script:` rules that shell out to third-party linters (eslint, phpstan, etc.) inherits whatever risks those tools carry. Trust the tools, not just bully.
- The semantic evaluator subagent sees diff content. If your diffs contain secrets, those secrets are sent to the subagent. Linting is not exfiltration-safe by design.

## Reporting a vulnerability

bully runs deterministic shell commands and dispatches a Claude subagent. If you find a way to escape the sandboxed scope or inject commands through a crafted `.bully.yml`, through file contents that bypass the script-rule scoping, or through a bug in the trust gate, please report it privately.

**Preferred:** [GitHub private vulnerability reporting](https://github.com/dynamik-dev/bully/security/advisories/new)

Expand All @@ -13,16 +47,18 @@ Expected response: acknowledgement within 72 hours.
## Scope

In scope:
- Command injection through config parsing or diff handling.
- Path traversal in rule `scope` globs.
- Command injection through config parsing, filename handling, or diff content.
- Path traversal in rule `scope` globs or `extends:` targets.
- Trust gate bypass: getting rules to run against an untrusted config without `BULLY_TRUST_ALL` and without an entry in `~/.bully-trust.json`.
- Telemetry file tampering causing the analyzer to crash or misclassify.
- Hook exit-code bypass.

Out of scope:
- Rules themselves being poorly written (that's a config bug, not a security issue).
- The `bully` skill making a judgment error on a semantic rule.
- Third-party linters invoked by a rule's `script:` field.
- A user trusting a malicious config after reviewing it. Trust is a gate, not a sandbox.

## Supported Versions
## Supported versions

Only `main` is supported. Tagged releases may receive security fixes at maintainer discretion.
39 changes: 28 additions & 11 deletions docs/design.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Design

`bully` is a Claude Code skill system that enforces project coding standards through a two-phase evaluation pipeline: deterministic script checks for pattern-matchable rules, followed by LLM semantic evaluation for judgment-based rules. It fires on every `Edit` and `Write` tool call via the `PostToolUse` hook.
`bully` is a Claude Code skill system that enforces project coding standards through a two-phase evaluation pipeline: deterministic checks (`script` rules shell out; `ast` rules run structural patterns through ast-grep) for pattern-matchable rules, followed by LLM semantic evaluation for judgment-based rules. It fires on every `Edit` and `Write` tool call via the `PostToolUse` hook.

## Core principles

Expand All @@ -21,10 +21,12 @@ Rules live in `.bully.yml` at the project root.
|-------|----------|-------------|
| `id` | yes | Unique identifier. Used in violation payloads and `passed_checks` context. |
| `description` | yes | What the rule enforces. Natural language. For semantic rules, this IS the evaluation prompt. |
| `engine` | yes | `script` or `semantic`. |
| `engine` | yes | `script`, `ast`, or `semantic`. |
| `scope` | yes | File glob or list of globs this rule applies to. See [Scope](#scope). |
| `severity` | yes | `error` (blocks) or `warning` (reported, non-blocking). |
| `script` | script engine only | Shell command to run. `{file}` is replaced with the file path. Diff provided on stdin. Exit non-zero on violation. |
| `pattern` | ast engine only | An [ast-grep pattern](https://ast-grep.github.io/guide/pattern-syntax.html). Literal code with `$NAME` for single-node captures, `$$$REST` for variadic. |
| `language` | ast engine, optional | Explicit ast-grep language (`ts`, `python`, `php`, ...). Inferred from the matched file extension when omitted. |

### Scope

Expand Down Expand Up @@ -75,7 +77,8 @@ rules:

- **No prescribed transformations.** `fix_hint` is a one-liner, not a codemod. The agent still performs the edit — prescribed transformations couple the rule to specific syntax and rot fast.
- **No rule dependencies or ordering.** Rules are independent. If two rules interact, the LLM sees both via the `passed_checks` context.
- **No `language` field.** Scope globs handle targeting.
- **`language` is optional even on ast rules.** Scope globs do most of the targeting work; `language` is escape-hatch for cases where the file extension can't disambiguate (e.g. `.h` could be C or C++).
- **ast-grep is an optional runtime dep.** Without it, `engine: ast` rules are skipped at runtime with a stderr hint and `bully doctor` reports `[FAIL]`. The pipeline itself stays stdlib-only.

## Validation

Expand All @@ -92,7 +95,7 @@ Checks performed on load:
Run the checks without firing the hook:

```bash
python3 pipeline/pipeline.py --validate
bully validate
```

Exits 0 on a clean config, 1 with a human-readable report otherwise. `hook.sh` runs `--validate` once per session (keyed off a tmpdir marker) so malformed configs surface on the first edit instead of silently dropping a rule forever.
Expand Down Expand Up @@ -159,15 +162,26 @@ Directives are scoped to the line they appear on. There is no file-level or bloc

## Short-circuit

Before any rule loads, the pipeline skips auto-generated files outright. The list lives in `pipeline/pipeline.py:AUTO_GENERATED_PATTERNS`:
Before any rule loads, the pipeline skips files matching a merged set of skip globs. The merge order is:

```
package-lock.json, yarn.lock, pnpm-lock.yaml, poetry.lock, Cargo.lock,
composer.lock, Gemfile.lock, go.sum, *.min.*, *.generated.*,
dist/**, build/**, __pycache__/**, node_modules/**, vendor/**
1. **Built-in defaults** (`pipeline/pipeline.py:SKIP_PATTERNS`):

```
package-lock.json, yarn.lock, pnpm-lock.yaml, poetry.lock, Cargo.lock,
*.min.js, *.min.css, *.min.*, dist/**, build/**, __pycache__/**,
*.generated.*, *.pb.go, *.g.dart, *.freezed.dart
```

2. **User-global**: `~/.bully-ignore` (one glob per line, `#` comments allowed). Per-machine, never committed.
3. **Project-local**: a top-level `skip:` key in `.bully.yml` -- inline (`skip: ["_build/**", "vendor/**"]`) or block-list form. Inherited from anything the config `extends:`.

```yaml
schema_version: 1
skip: ["_build/**", "vendor/**"]
rules: ...
```

Matches return `pass` before config parse, scope match, or script dispatch. No config flag, no override — lockfiles and build artifacts never benefit from lint and firing on them burns cycles on every edit.
Matches return `pass` before config parse, scope match, or script dispatch. The built-in list is always active; the user-global and project-local lists merge on top. Lockfiles and build artifacts never benefit from lint and firing on them burns cycles on every edit.

## Evaluation pipeline

Expand All @@ -192,7 +206,10 @@ PostToolUse fires (Edit or Write)
+-- No matching rules? --> exit 0 (pass)
|
v
Phase 1: run script rules (per-rule 30s timeout)
Phase 1: run script + ast rules (per-rule 30s timeout)
- script rules shell out (`{file}` substituted in)
- ast rules invoke ast-grep with rule.pattern
(skipped with stderr hint if ast-grep is missing)
|
+-- Any error-severity violations?
| --> print text to stderr
Expand Down
Loading
Loading