Explore holotree packfiles to reduce filesystem churn (post-zstd Phase 1)

[joshyorko]

Context: Following PR #64 (holotree zstd compression, Phase 1), profiling may show that environment restore time is dominated not by decompression or hashing, but by filesystem churn (creating + chmod/chown + AV scanning of thousands of small files), especially on Windows, macOS, and networked filesystems.

This issue proposes exploring a holotree packfile / packed-object format, inspired by Git packfiles, as a future performance optimization that preserves RCC’s security and enterprise guarantees.

Motivation:

• Reduce number of filesystem operations during env restore
• Verify integrity once per pack instead of per-file
• Improve restore speed on AV-heavy and networked filesystems
• Keep current secure defaults (no hardlinks by default)

High-level idea:

• Store holotree content as chunked packfiles (large blobs)
• Single hash verification per pack
• Streamed decode (zstd-friendly)
• Materialize files only at the final destination
• Compatible with existing holotree semantics (immutability, relocations, copy-patterns)

Non-goals (v1):

• No default hardlinks or reflinks
• No skipping hash verification
• No change to existing hololib format without migration plan

Open questions:

• How to represent relocations inside a pack
• Optimal chunk sizing vs memory usage
• Interaction with existing catalog format
• Windows/macOS AV behavior with streamed writes

This is an exploration / design issue only, intended to complement (not replace) current optimizations like zstd compression.