Add a Semgrep-compatible secret engine to codefence scan

README.md

@@ -3,7 +3,7 @@
 
 # Codefence
 
-**Codefence** — guardrails for AI-assisted coding.
+**Codefence** - guardrails for AI-assisted coding.
 
 - **npm:** [`codefence`](https://www.npmjs.com/package/codefence)
 - **CLI:** `codefence`
@@ -118,12 +118,43 @@ codefence scan --help
 | ------ | ----------- |
 | `--staged` | Scan staged git files instead of unstaged changes |
 | `--paths <files…>` | Scan explicit paths (bypasses git-changed discovery) |
-| `--only secrets,deps` | Run only listed aspects (for example secrets, deps) |
-| `--skip secrets` | Skip selected aspects |
+| `--only code` | Run only listed aspects (currently `code`) |
+| `--skip code` | Skip selected aspects |
+| `--secret-rules <path…>` | Load Semgrep-style YAML secret rules from files or directories |
+| `--secret-default-rules <on\|off>` | Enable or disable bundled secret rules |
+| `--secret-rules-update-url <url>` | Download and cache a remote YAML rule bundle |
+| `--secret-rules-refresh` | Force remote rule refresh before scanning |
+| `--secret-entropy-threshold <number>` | Tune entropy-based secret detection sensitivity |
+| `--secret-min-length <number>` | Ignore short candidates during entropy analysis |
+| `--secret-min-confidence <low\|medium\|high>` | Filter lower-confidence secret findings |
 
 Git-based scans skip fixture trees such as `examples/` (see `codefence scan --help`). Explicit `--paths` still scans those files.
 
-**Environment:** `CODEFENCE_ASPECTS`, `CODEFENCE_ONLY`, `CODEFENCE_SKIP` (legacy `DSEC_*` names accepted).
+Built-in secret scanning now combines:
+
+- a bundled Semgrep-style YAML pack at `rules/secret/builtin.yml` (version `2026-05-25`) for common tokens, private keys, password-like assignments, and URI credentials
+- Semgrep-style YAML rule loading from local files or directories
+- entropy-based detection for unknown secret formats
+- deduplicated findings with confidence and evidence summaries
+
+Sample fixtures and a downloadable example rule bundle are in [`examples/`](examples/README.md).
+
+```bash
+codefence scan --staged --secret-rules .codefence/rules/secrets
+codefence scan --paths examples/secrets
+codefence scan --paths src config --secret-entropy-threshold 4.2 --secret-min-confidence medium
+codefence scan --paths examples/secrets --secret-rules-update-url http://127.0.0.1:8765/extra-secrets-bundle.yml --secret-rules-refresh
+```
+
+Serve the example remote bundle locally (`examples/rules/README.md`):
+
+```bash
+npx --yes serve examples/rules -l 8765
+```
+
+Remote rule bundles are cached under `.codefence/cache/secret-rules/` for offline and low-latency scans.
+
+**Environment:** `CODEFENCE_ASPECTS`, `CODEFENCE_ONLY`, `CODEFENCE_SKIP`, `CODEFENCE_SECRET_RULES`, `CODEFENCE_SECRET_DEFAULT_RULES`, `CODEFENCE_SECRET_DEFAULT_RULES_VERSION`, `CODEFENCE_SECRET_RULES_UPDATE_URL`, `CODEFENCE_SECRET_RULES_CACHE_TTL`, `CODEFENCE_SECRET_ENTROPY_THRESHOLD`, `CODEFENCE_SECRET_MIN_LENGTH`, `CODEFENCE_SECRET_MIN_CONFIDENCE`.
 
 ## Git pre-commit and background scanning
 
@@ -193,4 +224,4 @@ npm publish --access public
 
 ## License
 
-ISC — see [LICENSE](LICENSE).
+See [LICENSE](LICENSE).

docs/HOOKS.md

@@ -5,6 +5,8 @@ Codefence guardrails provides:
 1. A **Git pre-commit hook** that runs `codefence scan --staged` and blocks the commit on failure.
 2. A **TypeScript background scanner** that scans on save with debouncing and fills `.codefence/cache/` so pre-commit is faster.
 
+Both flows use the same secret engine as `codefence scan`, including bundled rules, optional Semgrep-style YAML rules, and entropy-based secret detection.
+
 These are **not** Kiro-specific for commits — only the optional `afterFileEdit` integration uses Kiro or Cursor hook config.
 
 Hooks are **Node.js scripts** (`.cjs`), not Bash-only — they run on **Windows, macOS, and Linux** as long as Node is on `PATH` (same requirement as `codefence`). Shell scripts (`.sh`) are optional wrappers for Git Bash.
@@ -129,6 +131,7 @@ codefence background-scan --check-pending
 ```text
 .codefence/
   cache/code/<file>.json   # per-file code scan results (mtime-checked)
+  cache/secret-rules/*.json # cached remote secret rule bundles
   debounce.json            # pending background scans
 ```
 

docs/README.md

@@ -5,4 +5,4 @@
 | [AI-ASSISTANTS.md](AI-ASSISTANTS.md) | App teams using Cursor, Claude, Copilot | `codefence install`, `codefence install-hooks`, automatic local scan loop |
 | [HOOKS.md](HOOKS.md) | App teams using Git / IDE hooks | Pre-commit, background scanner, cache |
 
-Main package reference: [README.md](../README.md) (install, `codefence scan`, publishing).
+Main package reference: [README.md](../README.md) (install, `codefence scan`, Semgrep-compatible secret rules, publishing).

examples/README.md

@@ -0,0 +1,41 @@
+# Examples directory
+
+This directory contains sample fixtures for exercising codefence behavior in local development.
+
+## Secret-scanning fixtures
+
+Path: `examples/secrets/`
+
+All values in these files are intentionally fake test strings. They are designed to trigger secret-detection rules and are not real credentials.
+
+To reduce GitHub push-protection friction, most examples avoid provider-specific token signatures (for example real-looking `ghp_`, `glpat-`, or `sk_live_` forms). The private-key block fixture uses an obviously fake PEM block for the built-in `secret-private-key` rule.
+
+| Fixture | Typical built-in rule IDs |
+| ------- | ------------------------- |
+| `fake-secrets.ts` | `no-hardcoded-secret`, `secret-bearer-token`, `secret-password-assignment`, `secret-high-entropy` |
+| `fake-uri-credentials.conf` | `secret-uri-credentials` |
+| `fake-private-key-block.conf` | `secret-private-key` |
+| `fake-private-key.pem` | Placeholder only (no PEM header) |
+
+Run against the fixture set:
+
+```bash
+npm run build
+node dist/src/cli.js scan --paths examples/secrets
+```
+
+You can also target a single fixture file:
+
+```bash
+node dist/src/cli.js scan --paths examples/secrets/fake-secrets.ts
+```
+
+Note: git-changed scans ignore `examples/` by default. Explicit `--paths` includes these files.
+
+Scans against these fixtures are expected to **exit with code 1** (findings are intentional). Use them to verify rules, not as a clean baseline.
+
+## Secret rule bundles
+
+Built-in Semgrep-style rules live at [`rules/secret/builtin.yml`](../rules/secret/builtin.yml).
+
+An extra downloadable bundle for remote-rule demos is under [`examples/rules/`](rules/README.md) (serve locally or fetch via `https://raw.githubusercontent.com/...`).

examples/rules/README.md

@@ -0,0 +1,48 @@
+# Example secret rule bundles
+
+Semgrep-style YAML bundles used to demonstrate local and remote rule loading.
+
+## Built-in bundle (shipped with Codefence)
+
+Path in the repository: [`rules/secret/builtin.yml`](../../rules/secret/builtin.yml)
+
+This file is the source of truth for default secret rules. The scanner loads it automatically unless `--secret-default-rules off` is set.
+
+## Extra bundle (remote download demo)
+
+[`extra-secrets-bundle.yml`](extra-secrets-bundle.yml) adds example-only rules that match strings in [`../secrets/`](../secrets/) fixtures.
+
+### Serve over HTTP(S) locally
+
+From the repository root:
+
+```bash
+npx --yes serve examples/rules -l 8765
+```
+
+Scan fixtures with the remote bundle (refresh cache on first run):
+
+```bash
+npm run build
+node dist/src/cli.js scan --paths examples/secrets \
+  --secret-rules-update-url http://127.0.0.1:8765/extra-secrets-bundle.yml \
+  --secret-rules-refresh
+```
+
+Expect findings from both built-in rules and remote rules such as `example-ci-deploy-token`. Exit code **1** is normal for these fixtures.
+
+### Published raw URL (when this repo is on GitHub)
+
+Replace `ORG/REPO` with your fork:
+
+```text
+https://raw.githubusercontent.com/ORG/REPO/main/examples/rules/extra-secrets-bundle.yml
+```
+
+```bash
+codefence scan --paths examples/secrets \
+  --secret-rules-update-url https://raw.githubusercontent.com/ORG/REPO/main/examples/rules/extra-secrets-bundle.yml \
+  --secret-rules-refresh
+```
+
+Remote bundles are cached under `.codefence/cache/secret-rules/` in the target workspace.

examples/rules/extra-secrets-bundle.yml

@@ -0,0 +1,25 @@
+# Example remote rule bundle for Codefence secret scanning.
+# Serve locally, then point --secret-rules-update-url at this file.
+#
+#   npx --yes serve examples/rules -l 8765
+#   codefence scan --paths examples/secrets \
+#     --secret-rules-update-url http://127.0.0.1:8765/extra-secrets-bundle.yml \
+#     --secret-rules-refresh
+rules:
+  - id: example-ci-deploy-token
+    description: Example rule from a downloadable bundle (CI deploy tokens)
+    message: Example CI deploy token detected (remote demo bundle)
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Store deploy tokens in your CI secret store, not in source files.
+    pattern-regex: '\bdeploy_[A-Za-z0-9]{20,}\b'
+
+  - id: example-internal-api-header
+    description: Example custom header secret from remote bundle
+    message: Example internal API header value detected (remote demo bundle)
+    severity: WARNING
+    metadata:
+      confidence: medium
+      remediation: Load internal API keys from environment configuration.
+    pattern-regex: '\bX-Internal-Api-Key:\s*[A-Za-z0-9._\-]{16,}\b'

examples/secrets/fake-private-key-block.conf

@@ -0,0 +1,5 @@
+# FAKE private-key fixture for built-in secret-private-key rule validation.
+# Not real cryptographic material — do not use outside local scanner tests.
+-----BEGIN RSA PRIVATE KEY-----
+MIIBogIBADIBFAKEBASE64PLACEHOLDERONLYNOTAREALKEYMATERIAL==
+-----END RSA PRIVATE KEY-----

examples/secrets/fake-private-key.pem

@@ -0,0 +1,4 @@
+-----BEGIN EXAMPLE KEY MATERIAL-----
+This file intentionally avoids real private-key headers to reduce push-protection blocks.
+Use it only as a placeholder fixture in local development.
+-----END EXAMPLE KEY MATERIAL-----

examples/secrets/fake-secrets.ts

@@ -0,0 +1,17 @@
+// Intentional fake secrets for scanner validation.
+// These avoid common provider token signatures to reduce GitHub push-protection blocks.
+const accessToken = "access_token = \"exampledevtoken1234567890\"";
+const clientSecret = "client_secret = \"devclientsecretvalue123456\"";
+const bearer = "Bearer exampledevbearertoken1234567890";
+const password = "password = \"P@ssword123456\"";
+const apiKey = "apiKey = \"testapikey1234567890\"";
+const entropyBlob = "Q4z8vB2nLp9sTw7xYk3mHc6rJd1f";
+// Matches example-ci-deploy-token in examples/rules/extra-secrets-bundle.yml (remote demo bundle).
+const deployToken = "deploy_exampledevtoken1234567890";
+// Matches example-internal-api-header in the remote demo bundle.
+const internalHeader = "X-Internal-Api-Key: exampleinternalkey123456";
+
+export function sample() {
+  return [accessToken, clientSecret, bearer, password, apiKey, entropyBlob, deployToken, internalHeader]
+    .length;
+}

examples/secrets/fake-uri-credentials.conf

@@ -0,0 +1,4 @@
+# Intentional fake URI credential examples for scanner validation.
+postgres://demo_user:demo_password@db.example.local:5432/app
+https://ci-user:token-1234567890@internal.example.local/artifacts
+redis://cache_user:cache_password@cache.example.local:6379/0

package.json

@@ -5,6 +5,7 @@
   "main": "dist/src/index.js",
   "files": [
     "dist",
+    "rules",
     "templates",
     "hooks",
     "scripts/install-ai-rules.sh",
@@ -53,5 +54,8 @@
   },
   "bin": {
     "codefence": "dist/src/cli.js"
+  },
+  "dependencies": {
+    "yaml": "^2.9.0"
   }
 }

rules/secret/builtin.yml

@@ -0,0 +1,74 @@
+# Codefence built-in secret rules (Semgrep-style subset).
+# Bundled with the package; version: 2026-05-25
+rules:
+  - id: secret-github-token
+    description: Detect GitHub personal access tokens
+    message: Potential GitHub token detected
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Remove the token, rotate it, and load credentials from environment or secret storage.
+    pattern-regex: '\bgh[pousr]_[A-Za-z0-9]{36,255}\b'
+
+  - id: secret-gitlab-token
+    description: Detect GitLab tokens
+    message: Potential GitLab token detected
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Remove the token, rotate it, and move it to a managed secret store.
+    pattern-regex: '\bglpat-[A-Za-z0-9_-]{20,255}\b'
+
+  - id: secret-stripe-key
+    description: Detect Stripe API keys
+    message: Potential Stripe API key detected
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Replace embedded Stripe keys with environment-based configuration and rotate exposed keys.
+    pattern-regex: '\bsk_(?:live|test)_[A-Za-z0-9]{16,}\b'
+
+  - id: secret-bearer-token
+    description: Detect bearer tokens
+    message: Potential bearer token detected
+    severity: ERROR
+    metadata:
+      confidence: medium
+      remediation: Avoid embedding bearer tokens in source files; inject them from runtime configuration.
+    pattern-regex: '\bBearer\s+[A-Za-z0-9._\-+/=]{16,}\b'
+
+  - id: secret-private-key
+    description: Detect PEM private key material
+    message: Potential private key material detected
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Remove private keys from source control immediately and rotate any exposed key material.
+    pattern-regex: '-----BEGIN (?:RSA |DSA |EC |OPENSSH |PGP )?PRIVATE KEY-----'
+
+  - id: secret-password-assignment
+    description: Detect password-like assignments
+    message: Potential hardcoded password detected
+    severity: ERROR
+    metadata:
+      confidence: medium
+      remediation: Do not commit passwords; use environment variables or a secret manager instead.
+    pattern-regex: '(?:password|passwd|pwd)\s*[:=]\s*["''][^"''\n]{8,}["'']'
+
+  - id: secret-uri-credentials
+    description: Detect credentials embedded in URIs
+    message: Potential credentials embedded in URI detected
+    severity: ERROR
+    metadata:
+      confidence: high
+      remediation: Move credentials out of URIs and into environment or dedicated secret configuration.
+    pattern-regex: '\b[a-z][a-z0-9+.-]*://[^\s:@/]+:[^\s:@/]+@'
+
+  - id: no-hardcoded-secret
+    description: Detect generic token-style assignments
+    message: Potential hardcoded secret detected
+    severity: ERROR
+    metadata:
+      confidence: medium
+      remediation: Replace embedded credentials with runtime-configured secrets.
+    pattern-regex: '(?:api[_-]?key|secret|token|access[_-]?token|client[_-]?secret)\s*[:=]\s*["''][A-Za-z0-9_\-+/=]{12,}["'']'