Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
204a050
feat: add persistent style guidelines and output format validation
csoceanu Jun 22, 2026
49ff799
fix: update file selection prompts to also cover new functionality
csoceanu Jun 22, 2026
501006d
fix: load_all_indexes maps hyphenated folder names correctly
csoceanu Jun 22, 2026
fa3c215
feat: index sub-folders for more granular area selection
csoceanu Jun 22, 2026
7ff7f50
fix: use non-recursive glob in get_files_in_areas
csoceanu Jun 22, 2026
3920641
fix: revert prompts to original with minimal new-functionality addition
csoceanu Jun 22, 2026
73180f5
fix: use balanced file selection prompt for better recall
csoceanu Jun 22, 2026
17496cb
fix: index builder only covers direct files, not sub-folder files
csoceanu Jun 22, 2026
f9773f7
feat: index root-level docs as _root instead of including uncondition…
csoceanu Jun 22, 2026
28d54d8
feat: select files directly from index descriptions in one LLM step
csoceanu Jun 22, 2026
72b0015
fix: instruct LLM not to group files or use wildcards in indexes
csoceanu Jun 22, 2026
16b7060
fix: same-repo docs PR branches from base instead of PR HEAD
csoceanu Jun 22, 2026
5a57b07
fix: save doc content before branch switch in same-repo flow
csoceanu Jun 22, 2026
bcc38ec
feat: same-repo update-docs pushes to code PR branch directly
csoceanu Jun 22, 2026
112a86e
fix: confirmation message says 'committed to this PR' for same-repo
csoceanu Jun 22, 2026
af67af9
fix: ensure LLM output ends with a trailing newline
csoceanu Jun 22, 2026
4854248
fix: address review findings (high + medium priority)
csoceanu Jun 22, 2026
2d21d54
fix: cap validation error output to prevent prompt injection
csoceanu Jun 22, 2026
4fabdf1
fix: address remaining low-priority review findings
csoceanu Jun 22, 2026
96acc0d
fix: address second round of review findings
csoceanu Jun 22, 2026
32c6db8
fix: update stale docstring, clarify dead code path
csoceanu Jun 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,11 @@ RUN npm install -g @googleworkspace/cli
# Install uv (Python package runner, needed for mcp-atlassian)
RUN pip install --no-cache-dir -U uv

# Install AsciiDoc validator (asciidoctor.js)
RUN npm install -g @asciidoctor/core @asciidoctor/cli

# Install Python dependencies
RUN pip install --no-cache-dir -U openai mcp mcp-atlassian
RUN pip install --no-cache-dir -U openai mcp mcp-atlassian markdown docutils

# Set up working directory
WORKDIR /app
Expand Down
37 changes: 33 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,27 @@ You can guide how the AI generates doc updates by adding instructions in your `[
config-ref.rst: only update the CLI usage example
```

## Persistent Style Guidelines

You can define repository-level documentation style rules that are automatically applied to every AI-generated update. Create the following file in your repository root:

- **`.code-to-docs/style.md`** — Freeform Markdown/prose, passed to the LLM as-is

The action auto-detects this file. To use a custom path instead, set the `style-config-path` action input.

**Example `.code-to-docs/style.md`:**
```markdown
# Documentation Style

- Use **active voice** and a professional tone
- Use sentence case for headings
- Use dashes for bullet lists
- Keep paragraphs short
- Use present tense
```

Per-comment instructions (`[update-docs] keep changes minimal`) continue to work alongside the persistent config, appearing as additional guidance after the style guidelines. If per-comment or per-file instructions contradict the style guidelines, the reviewer's instructions take precedence.

## How It Works

1. **Triggered by PR Comments** - When someone comments `[review-docs]`, `[update-docs]`, or `[review-feature]` on a Pull Request
Expand Down Expand Up @@ -111,6 +132,7 @@ jobs:
jira-api-token: ${{ secrets.JIRA_API_TOKEN }}
google-sa-key: ${{ secrets.GOOGLE_SA_KEY }}
max-context-chars: ${{ secrets.MAX_CONTEXT_CHARS }}
style-config-path: '.code-to-docs/style.md'
```

### 2. Configure Secrets
Expand All @@ -132,6 +154,14 @@ Add these in **Settings → Secrets → Actions**:
| `GOOGLE_SA_KEY` | _(Optional, for `[review-feature]`)_ Google service account JSON key for fetching Google Docs. Docs must be shared with the service account email. |
| `MAX_CONTEXT_CHARS` | _(Optional)_ Maximum characters for LLM prompt content (default: `400000`, ~100K tokens). Decrease for models with smaller context windows (e.g., `32000` for an 8K-token model). |

### 3. Optional Action Inputs

These are set as `with:` parameters in the workflow step (not as secrets):

| Input | Description |
|-------|-------------|
| `style-config-path` | _(Optional)_ Path to a Markdown style configuration file (`.md`) containing documentation style guidelines. If not set, auto-detects `.code-to-docs/style.md`. |

### Supported Model Backends

Any OpenAI-compatible API works. Common examples:
Expand Down Expand Up @@ -174,9 +204,8 @@ The `[review-feature]` comment includes content from the Jira ticket and linked

## Performance Optimization

The action uses a two-stage caching system stored in `.doc-index/`:
The action builds semantic indexes stored in `.doc-index/`:

1. **Folder Indexes** - AI-generated semantic summaries of each documentation folder, used to quickly identify relevant areas without scanning all files
2. **File Summaries** - Cached summaries of long documentation files, reused across runs
- **Folder Indexes** — AI-generated summaries of each documentation folder, including per-file descriptions. The LLM selects relevant files directly from these descriptions without loading the actual file content, reducing API calls and runtime.

These are automatically committed to your main branch and shared across all PRs, reducing runtime from ~20 minutes to ~4 minutes on large projects.
Indexes are automatically committed to your main branch and shared across all PRs, reducing runtime from ~20 minutes to ~4 minutes on large projects.
5 changes: 5 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ inputs:
description: 'Maximum characters for LLM prompt content (default: 400000, ~100K tokens). Decrease for models with smaller context windows (e.g. 32000 for an 8K-token model).'
required: false
default: '400000'
style-config-path:
description: 'Path to a Markdown style configuration file (.md) containing documentation style guidelines. If not set, auto-detects .code-to-docs/style.md in the repository root.'
required: false
default: ''

outputs:
status:
Expand Down Expand Up @@ -99,3 +103,4 @@ runs:
JIRA_API_TOKEN: ${{ inputs.jira-api-token }}
GOOGLE_SA_KEY: ${{ inputs.google-sa-key }}
MAX_CONTEXT_CHARS: ${{ inputs.max-context-chars }}
STYLE_CONFIG_PATH: ${{ inputs.style-config-path }}
4 changes: 4 additions & 0 deletions entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ if [ -n "$JIRA_API_TOKEN" ]; then
export JIRA_API_TOKEN="$JIRA_API_TOKEN"
fi

if [ -n "$STYLE_CONFIG_PATH" ]; then
export STYLE_CONFIG_PATH="$STYLE_CONFIG_PATH"
fi

if [ -n "$GOOGLE_SA_KEY" ]; then
# Write service account JSON to temp file for gws CLI
GWS_CREDS_FILE=$(mktemp /tmp/gws-sa-XXXXXX.json)
Expand Down
59 changes: 59 additions & 0 deletions src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,19 @@

All environment variable access for configuration lives here.
Runtime env vars (GH_TOKEN, PR_NUMBER, etc.) are still read where needed.
Also handles loading persistent style guidelines from .code-to-docs/style.md
or a user-specified STYLE_CONFIG_PATH.
"""

import os
import re
from pathlib import Path

import openai
from openai import OpenAI

from security_utils import sanitize_output, validate_file_path


def get_client():
"""Get the shared OpenAI-compatible client."""
Expand Down Expand Up @@ -126,6 +131,60 @@ def truncate_diff(diff_text, max_chars, label="diff"):
return result + suffix


# =============================================================================
# STYLE CONFIGURATION
# =============================================================================

_AUTO_DETECT_PATHS = [
".code-to-docs/style.md",
]

_ALLOWED_STYLE_EXTENSIONS = (".md",)


def load_style_config(config_path=None):
"""Load documentation style guidelines from a config file."""
if not config_path:
config_path = os.environ.get("STYLE_CONFIG_PATH", "")

if config_path:
if not validate_file_path(config_path):
print(f"Warning: Style config path rejected by security check: '{config_path}', skipping")
return ""
if not config_path.endswith(_ALLOWED_STYLE_EXTENSIONS):
print(f"Warning: Style config must be a .md file, got '{config_path}', skipping")
return ""
config_file = Path(config_path)
if not config_file.is_file():
print(f"Warning: Style config not found at '{config_path}', skipping")
return ""
else:
config_file = None
for candidate in _AUTO_DETECT_PATHS:
p = Path(candidate)
if p.is_file() and validate_file_path(str(p)):
config_file = p
break
if config_file is None:
return ""

config_path_str = str(config_file)
print(f"Loading style config from: {config_path_str}")

try:
raw = config_file.read_text(encoding="utf-8").strip()
except Exception as e:
print(f"Warning: Could not read style config '{config_path_str}': {sanitize_output(str(e))}")
return ""

if not raw:
print(f"Warning: Style config '{config_path_str}' is empty, skipping")
return ""

print(f"Loaded style config ({len(raw):,} chars)")
return raw


def check_context_error(e):
"""
If e is a context-window error, print actionable guidance.
Expand Down
156 changes: 29 additions & 127 deletions src/discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
File discovery and selection for documentation updates.

This module handles finding which documentation files are relevant to a given
code change. It provides AI-powered file selection (via batched parallel calls),
long-file summarization, and an optimized two-stage discovery pipeline that uses
semantic indexes to narrow candidates before asking the AI to pick exact files.
code change. It provides AI-powered file selection via semantic indexes that
identify relevant files directly from per-folder index descriptions, and a
fallback full-scan path for repos without indexes.
"""

import os
Expand All @@ -28,10 +28,8 @@
fetch_indexes_from_main,
build_all_indexes,
update_indexes_if_needed,
find_relevant_areas_from_indexes,
get_files_in_areas,
find_relevant_files_from_indexes,
commit_indexes_to_repo,
get_or_generate_summary,
)


Expand Down Expand Up @@ -133,29 +131,27 @@ def get_file_content_or_summaries(line_threshold=300):
return file_data

_FILE_SELECTION_PROMPT_TEMPLATE = """
You are an ULTRA-CONSERVATIVE documentation assistant. Select ONLY files that DIRECTLY document the EXACT code being changed.
You are a precise documentation assistant. Select files that document the feature, component, or behavior being changed or extended in the diff.

Git diff from this PR:
{DIFF_PLACEHOLDER}

Documentation files to evaluate:
{CONTEXT_PLACEHOLDER}

STRICT SELECTION RULES:
1. ONLY select files that document the EXACT code, module, or component being modified in the diff
2. DO NOT select files just because they mention related concepts or technologies
3. DO NOT select overview or index files unless absolutely necessary
4. Select the MINIMUM number of files necessary
5. When in doubt, DO NOT select the file
6. Prefer returning NONE over selecting uncertain files

AVOID COMMON OVER-SELECTION MISTAKES:
7. If a doc file mentions the same technology (e.g., a library, tool, or protocol) but for a DIFFERENT component or purpose, DO NOT select it
8. If a doc file is about USER-CONFIGURED items (e.g., custom configs, user containers, plugins) but the code change is about INTERNAL/SYSTEM behavior, DO NOT select it
9. If a doc file is for a different subsystem that happens to share dependencies with the changed code, DO NOT select it
10. Release notes and changelogs should ONLY be selected if explicitly requested or if the change is a breaking change

Return ONLY file paths (one per line) that DIRECTLY match the code changes.
SELECT a file if ANY of these apply:
1. The diff MODIFIES existing behavior that the file documents (docs would become incorrect)
2. The diff ADDS new functionality that falls within the scope of what the file documents (docs would become incomplete)
3. The diff CHANGES defaults, error messages, or output that the file references

DO NOT select a file if:
4. The connection between the diff and the doc is only superficial (shared keywords but different context or component)
5. It is for a different subsystem that happens to share dependencies with the changed code
6. It is a release notes or changelog file (unless the change is breaking)

When genuinely uncertain, DO NOT select.

Return ONLY file paths (one per line) that match the criteria above.
If no files need updates, return "NONE".
"""

Expand Down Expand Up @@ -310,9 +306,9 @@ def find_relevant_files_optimized(diff):
"""
Optimized file discovery using semantic indexes.

This is a two-stage approach:
1. Use indexes to find relevant documentation AREAS (1 API call)
2. Load files from those areas and use AI to pick exact files (1 API call)
Uses per-folder indexes (which include per-file descriptions) to identify
the exact files that need updating in a single LLM step — without loading
the actual file content for filtering.

Falls back to full scan if indexes don't exist or AI requests it.

Expand All @@ -322,123 +318,29 @@ def find_relevant_files_optimized(diff):
Returns:
list: List of relevant file paths, or None to signal full scan needed
"""
# Try to fetch indexes from main branch if they don't exist locally
fetch_indexes_from_main()

# Check if indexes exist
indexes_changed = False
if not indexes_exist():
print("No indexes found. Building indexes first...")
build_all_indexes()
indexes_changed = True
else:
# Update indexes for any changed docs
updated = update_indexes_if_needed()
if updated:
print(f"Updated indexes for: {updated}")
indexes_changed = True

# Stage 1: Find relevant AREAS using indexes (1 API call)
print("Finding relevant documentation areas from indexes...")
relevant_areas = find_relevant_areas_from_indexes(diff, get_client())
# Commit new/updated indexes so they're cached for future runs
if indexes_changed:
print("Committing indexes to repository...")
commit_indexes_to_repo(content_type="indexes")

print("Finding relevant documentation files from indexes...")
relevant_files = find_relevant_files_from_indexes(diff, get_client())

if relevant_areas is None:
# AI requested full scan or error occurred
if relevant_files is None:
print("Falling back to full scan...")
return None

if not relevant_areas:
print("No relevant areas found")
return []

# Stage 2: Get files from relevant areas
candidate_files = get_files_in_areas(relevant_areas)
print(f"Found {len(candidate_files)} candidate files in areas: {relevant_areas}")

if not candidate_files:
return []

# Load content/summaries for candidate files only (parallel for speed)
file_previews = []
files_needing_summary = []
files_with_content = []

# First pass: identify which files need summaries
for file_path in candidate_files:
try:
content = Path(file_path).read_text(encoding='utf-8')
line_count = len(content.split('\n'))

if line_count > 300:
files_needing_summary.append((file_path, content))
else:
files_with_content.append((file_path, content))
except Exception as e:
print(f"Skipping {file_path}: {sanitize_output(str(e))}")

# Generate summaries in parallel for long files (with caching)
summaries_generated = False
if files_needing_summary:
# Check how many need actual generation vs cached
cached_count = 0
to_generate = []

for file_path, content in files_needing_summary:
from doc_index import load_cached_summary
cached = load_cached_summary(file_path)
if cached:
file_previews.append((file_path, cached))
cached_count += 1
else:
to_generate.append((file_path, content))

if cached_count > 0:
print(f"Using {cached_count} cached summaries")

if to_generate:
print(f"Generating {len(to_generate)} new summaries in parallel...")
summaries_generated = True

def generate_summary_task(args):
file_path, content = args
try:
# Generate and cache the summary
summary = get_or_generate_summary(file_path, content, summarize_long_file)
return (file_path, summary)
except Exception as e:
print(f"Error summarizing {file_path}: {sanitize_output(str(e))}")
# Fallback to full content — downstream prompt handles truncation
return (file_path, content)

with ThreadPoolExecutor(max_workers=5) as executor:
futures = {executor.submit(generate_summary_task, args): args[0]
for args in to_generate}
for future in as_completed(futures):
result = future.result()
if result:
file_previews.append(result)

# Add files that didn't need summaries
file_previews.extend(files_with_content)

# Commit indexes and summaries in a single push to avoid the branch
# going stale between two separate pushes
if indexes_changed or summaries_generated:
content_parts = []
if indexes_changed:
content_parts.append("indexes")
if summaries_generated:
content_parts.append("summaries")
content_label = " and ".join(content_parts)
print(f"Committing {content_label} to repository...")
commit_indexes_to_repo(content_type=content_label)

if not file_previews:
return []

# Stage 3: Use AI to pick exact files from candidates (1 API call typically)
# Since we have fewer files now, this is much faster
print(f"AI selecting exact files from {len(file_previews)} candidates...")
relevant_files = ask_ai_for_relevant_files(diff, file_previews)

return relevant_files
Loading
Loading