feat: add local daemon git layers#171
Open
rudironsoni wants to merge 13 commits into
Open
Conversation
rudironsoni
force-pushed
the
feature/local-daemon-git-layers
branch
2 times, most recently
from
29da63d to
51df1e1
Compare
There was a problem hiding this comment.
Pull request overview
Adds daemon-owned, Git-aware indexing layers so multiple worktrees can share a durable “base” index while indexing branch and dirty worktree changes as cheaper overlays, plus CLI/MCP support for --cwd, --base, and overlay status/prune operations.
Changes:
- Extend the daemon protocol/client/CLI/MCP surface to carry
cwdandbase_ref, and add overlay status/prune requests. - Introduce Git worktree context resolution, a SQLite layer metadata store, and a
LayeredProjectfacade that materializes and queries base/branch/dirty layers. - Refactor
Project/indexer/query paths to support indexing from a source tree distinct from the project root (needed for materialized layers).
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_protocol.py | Updates protocol encode/decode tests for new request/result fields. |
| tests/test_git_layers.py | Adds coverage for daemon state dir, remote URL normalization, repo/worktree identity, layer store persistence, and layered indexing. |
| src/cocoindex_code/shared.py | Adds PROJECT_ROOT context key to distinguish logical project root vs indexed source tree. |
| src/cocoindex_code/server.py | Threads cwd into MCP server calls so daemon can resolve worktree context correctly. |
| src/cocoindex_code/query.py | Factors out query_codebase_with_embedding to reuse a single query embedding across layered searches. |
| src/cocoindex_code/protocol.py | Adds cwd/base_ref to requests; adds overlay status/prune RPCs; adds layer lineage fields to SearchResult. |
| src/cocoindex_code/project.py | Allows indexing/searching against a separate source_root and db_dir; adds sync search with precomputed embeddings. |
| src/cocoindex_code/layered_project.py | Implements layer materialization, indexing orchestration, tombstone shadowing, and layered search/merge. |
| src/cocoindex_code/layer_store.py | Adds SQLite-backed metadata store for layers and overlay manifests, including TTL-based pruning. |
| src/cocoindex_code/indexer.py | Uses PROJECT_ROOT for settings/gitignore while indexing CODEBASE_DIR (materialized layer source). |
| src/cocoindex_code/git_context.py | Adds Git worktree context resolution, remote normalization, and dirty snapshot hashing. |
| src/cocoindex_code/daemon.py | Switches daemon project registry to choose between Project and LayeredProject; adds overlay status/prune dispatch. |
| src/cocoindex_code/client.py | Adds client plumbing for cwd/base_ref and overlay status/prune RPCs. |
| src/cocoindex_code/cli.py | Adds --cwd/--base options and ccc overlay status/prune commands. |
| src/cocoindex_code/_daemon_paths.py | Introduces durable daemon state dir resolution under XDG data home (with env override). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+428
to
+431
| def _materialize_commit(repo_root: Path, commit: str, source_dir: Path) -> None: | ||
| raw = _git_bytes(repo_root, "archive", "--format=tar", commit) | ||
| with tarfile.open(fileobj=io.BytesIO(raw), mode="r:") as archive: | ||
| archive.extractall(source_dir) |
Comment on lines
+449
to
+452
|
|
||
| def _materialize_paths_from_worktree(repo_root: Path, paths: list[str], source_dir: Path) -> None: | ||
| for path in paths: | ||
| source = repo_root / path |
Comment on lines
+89
to
+103
| status = item[:2] | ||
| path = item[3:] | ||
| original_path: str | None = None | ||
| if status[0] in {"R", "C"} or status[1] in {"R", "C"}: | ||
| if i < len(parts): | ||
| original_path = parts[i] or None | ||
| i += 1 | ||
| entries.append( | ||
| GitStatusEntry( | ||
| index_status=status[0], | ||
| worktree_status=status[1], | ||
| path=path, | ||
| original_path=original_path, | ||
| ) | ||
| ) |
Comment on lines
+107
to
+120
| def _dirty_snapshot_hash(repo_root: Path, entries: tuple[GitStatusEntry, ...]) -> str | None: | ||
| if not entries: | ||
| return None | ||
| digest = hashlib.sha256() | ||
| for entry in sorted(entries, key=lambda e: (e.path, e.original_path or "")): | ||
| digest.update(entry.index_status.encode()) | ||
| digest.update(entry.worktree_status.encode()) | ||
| digest.update(entry.path.encode()) | ||
| if entry.original_path is not None: | ||
| digest.update(entry.original_path.encode()) | ||
| path = repo_root / entry.path | ||
| if path.is_file(): | ||
| digest.update(hashlib.sha256(path.read_bytes()).digest()) | ||
| return digest.hexdigest()[:24] |
Comment on lines
+633
to
+636
| for layer in pruned: | ||
| import shutil | ||
|
|
||
| shutil.rmtree(layer.source_dir.parent, ignore_errors=True) |
rudironsoni
force-pushed
the
feature/local-daemon-git-layers
branch
from
6fd5eb4 to
2b75ca8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Git-aware local daemon layers so multiple worktrees can share a durable base index while branch and dirty worktree changes are indexed as cheaper overlays.
Refs #170.
This follows the design discussed in #170:
--cwd,--base,ccc overlay status, andccc overlay pruneValidation
uv run ruff check .uv run mypy .uv run pytest tests/(198 passed, 8 deselected)