Kyrex is a Python environment infrastructure layer for caching, portability, offline restoration, and reproducible environments.
Kyrex is a decentralized environment fabric designed to streamline dependency packaging and transfer. By caching wheels in a Content-Addressable Storage (CAS) database, sharing packages across local networks (LAN), and bundling complete environments, Kyrex enables developers to replicate configurations instantly and offline.
- Requirements
- Installation
- Quick Start
- Why Kyrex Exists
- Design Principles
- Who is Kyrex For?
- Architecture
- Core Features
- CLI Reference
- Comparison with Alternative Tools
- Security Model & Threat Model
- Testing
- Validation Status
- Compatibility Matrix
- Performance Philosophy
- FAQ
- Known Limitations
- Development Principles
- Release Roadmap
- Long-Term Vision
- Release Validation
- Contributing
- License
Kyrex v1.0 targets support for:
- Windows: Windows 10, Windows 11, or Windows Server (x86_64).
- Linux: Ubuntu, Debian, Red Hat, Fedora, Arch Linux, etc. (x86_64, aarch64).
- macOS: Apple Silicon (M1/M2/M3) and Intel-based architectures.
Note: Official support is granted only after completing automated CI validation and manual smoke testing for that platform.
Before installing Kyrex, ensure the following tools and system configurations are available:
- Python: Python 3.10–3.13 (3.14 will be supported once upstream dependencies officially support it.)
Check via:
python --version - Git: Installed and registered in your system PATH.
Check via:
git --version - Rust Toolchain: Stable compiler (
rustc) and package manager (cargo). Check via:rustc --versionandcargo --version
- Windows: Install Visual Studio Build Tools, MSVC C++ Compiler, and Windows SDK.
- Linux:
Install
build-essential,gcc,clang, andpkg-config. - macOS:
Install Xcode Command Line Tools via:
xcode-select --install
- uv: Check via
uv --version. - maturin: Check via
maturin --version(required for local extension compilation).
- TCP Port 8990: Required only for local LAN sharing server (
kyrex share). Must be allowed in firewall. - mDNS Multicast: Multicast traffic must be enabled on the local network for P2P peer discovery.
- Hardware: SSD (Solid State Drive) is highly recommended for optimal hard-link restoration speeds.
- Free Disk Space:
- Minimum: 2 GB
- Recommended: 10 GB+ (depending on the size and count of cached machine learning wheel binaries).
- Internet Connection: Required only for initial installation, dependency resolution, or downloading uncached packages. All subsequent environment builds can run completely offline.
To install the pre-built distribution of Kyrex directly:
pip install kyrexTo compile and install Kyrex locally:
git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
pip install maturin
maturin build --release
pip install target/wheels/kyrex-*.whlTo set up an editable local workspace for code changes:
git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\Activate.ps1
pip install maturin click rich packaging tomli pytest zeroconf uv
maturin develop# Resolve, download, and cache dependencies globally
kyrex install numpy pandas torch
# Share your local cache over the LAN
kyrex share --port 8990
# Compress the active virtual environment into a portable bundle
kyrex bundle --output dev_env.kyx
# Recreate the virtual environment on another machine from the bundle
kyrex restore dev_env.kyx --path .restored_venvModern Python development involves repeated downloads of identical package binaries across different local virtual environments, developer machines, and CI/CD pipelines. This redundancy wastes internet bandwidth, slows down team onboarding, and creates fragility when deploying to staging or air-gapped systems. Kyrex treats Python virtual environments not as disposable, arbitrary directory trees, but as verifiable, relocatable, and shareable software layers.
Kyrex follows five core principles:
- Reliability before features: Prioritize absolute stability, deterministic behaviors, and rigorous pre-release validation.
- Reproducible environments: Ensure virtual environments can be reconstituted offline identically across workstations.
- Offline-first workflows: Minimize external network dependency by caching wheels in a global content-addressed local store.
- Security by default: Cryptographic verification on CAS file writes and zero-trust sandbox execution limits.
- Performance through Rust: Focus core filesystem, database queries, and bundler compression loops inside performance-optimized Rust.
- Data Scientists & ML Engineers: Working with large binary dependencies (like PyTorch or CUDA wheels) who want to avoid reinstall delays and automatic hardware mapping.
- Research Labs & Universities: Setting up compute clusters where gigabytes of packages need to be distributed across many machines without saturated gateways.
- CI/CD Platforms: Replicating staging environments rapidly using local offline bundles rather than rebuilding environments from scratch.
- Offline & Air-gapped Developers: Working without internet connectivity (defense, banking, healthcare) who need a portable, self-contained way to package dependencies.
- Users who need a lightweight pip replacement for tiny projects with minimal, pure-Python dependencies (where standard
piporuvis already sufficient). - Single-use, disposable scripts that do not require environment persistence or portability.
graph TD
PyPI[PyPI Registry] -->|Download| Resolver[uv Resolver]
Resolver -->|Cache Write| CAS[Local CAS Cache]
Resolver -->|Query/Fetch| LAN[LAN Peers]
CAS -->|Hard-Link| Venv[Virtual Environment]
LAN -->|Download/Cache| CAS
Venv -->|Bundle| Kyx[Bundle .kyx]
Kyx -->|Restore| Destination[Restore Anywhere]
For a detailed analysis of components (Rust, SQLite, CAS, .kyx), execution flows, and design choices, see docs/architecture.md.
| Category | Capability | Description |
|---|---|---|
| Cache | Content-Addressable Storage | Caches wheel binaries globally using SHA-256 hashes to prevent duplicate storage. |
| Networking | mDNS P2P Sharing | Automatically advertises and discovers local peers on the network to pull cached wheels over the LAN. |
| Environment | Teleportation (.kyx) | Bundles environment manifests and wheels into a compressed archive for instant offline restoration. |
| Performance | Hard-Link Installation | Links wheels directly from the local CAS to virtual environments, avoiding duplicate copies and file writes. |
| Security | Subprocess Safety | Standardizes list-based subprocess calls and verifies hashes at system boundaries to prevent directory escapes. |
| Diagnostics | Environment Doctor | Audits cache databases, validates file integrity, and identifies virtual environment drift. |
| Machine Learning | CUDA Auto-Mapping | Detects GPU details on the host and automatically overrides installation indexes to fetch compatible CUDA wheels. |
Resolves and installs packages into a target virtual environment.
kyrex install [PACKAGES]... [OPTIONS]--offline: Disables PyPI lookups, installing only from local CAS wheels.--path, -p TEXT: Path to the virtual environment (default:.venv).
Archives the current virtual environment and all its cached wheels into a single .kyx bundle.
kyrex bundle --output TEXT [OPTIONS]--path, -p TEXT: Path to the virtual environment directory to bundle.
Restores a virtual environment from a .kyx bundle.
kyrex restore [BUNDLE_FILE] [OPTIONS]--path, -p TEXT: Path to recreate the virtual environment.
Starts the P2P wheel distribution server on the local network.
kyrex share [OPTIONS]--port, -p INTEGER: Port to bind the HTTP sharing server.
Clones a git repository and automatically bootstraps its dependencies.
kyrex clone [REPO_URL] [DEST_DIR]Initializes a virtual environment and installs dependencies detected in the current path (pyproject.toml or requirements.txt).
kyrex bootstrapDiagnoses and repairs local cache DBs, verifying file hashes and identifying environment drift.
kyrex doctorBelow are example outputs showing how Kyrex presents diagnostics and peer sharing status in the terminal:
$ kyrex doctor --fix
╔══════════════════════════════════════════════════════════════════════════╗
║ Kyrex Diagnostics ║
╚══════════════════════════════════════════════════════════════════════════╝
Running Kyrex Diagnostics...
-> Checking SQLite catalog path: C:\Users\Aaryan Rawat\.cache\kyrex\metadata.db
-> Checking Content-Addressable Storage (CAS): C:\Users\Aaryan Rawat\.cache\kyrex\cas
! Warning: Found 2 orphaned wheel files in CAS not registered in SQLite.
v Doctor repair option (--fix) active.
-> Re-indexing orphaned wheels into SQLite database catalog...
v Registered wheel: click-8.4.2-py3-none-any.whl (128 KB)
v Registered wheel: rich-15.0.0-py3-none-any.whl (320 KB)
v All integrity checks passed. Database and cache are healthy.
$ kyrex share --port 8990
╔══════════════════════════════════════════════════════════════════════════╗
║ Kyrex Peer-to-Peer Fabric ║
╚══════════════════════════════════════════════════════════════════════════╝
-> Advertising local package sharing service on port 8990...
-> mDNS Service Broadcast: _kyrex._tcp.local. (host: Aaryan-PC)
-> HTTP wheel server listening on http://127.0.0.1:8990
⚡ Peer fabric is active. Press Ctrl+C to stop sharing.
Kyrex does not replace dependency resolvers; it adds a portability and caching layer on top of them. Below is a factual comparison of features and capabilities:
| Feature | pip | uv | Poetry | PDM | Conda | Kyrex |
|---|---|---|---|---|---|---|
| Core Role | Package Installer | Package Resolver | Project Manager | Project Manager | System Environment | Environment Fabric |
| Deduplication Caching | ❌ (HTTP cache) | ✅ (Content-Address) | ❌ (HTTP cache) | ❌ (HTTP cache) | ✅ (Hard links) | ✅ (CAS Hard-link) |
| Offline Restore | ❌ (Manual flags) | ✅ (Local cache) | ❌ (Manual flags) | ❌ (Manual flags) | ✅ (Local channels) | ✅ (Zero-setup local) |
| LAN Peer Sharing | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ (mDNS Discovery) |
| Relocatable Bundles | ❌ | ❌ | ❌ | ❌ | ⚠ (Conda-pack) | ✅ (.kyx Archives) |
| GPU Auto-Mapping | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ (CUDA detection) |
| Reproducible lockfile | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ (.kyx.lock) |
Kyrex is engineered with a multi-layered security model to protect system hosts and developer environments.
- SHA-256 CAS Verification: All wheels are indexed and verified by their cryptographic SHA-256 hash to prevent package spoofing.
- Strict Input Sanitization: Python and Rust API boundaries reject any hash parameters that do not strictly match the 64-character hexadecimal format (
^[a-fA-F0-9]{64}$). - Path Traversal Prevention: Bundle extraction and file read/write routines validate destination paths to block directory escape and tar-slip vulnerability exploits.
- Subprocess Isolation: External helper programs (such as
gitoruv) are executed as structured lists withshell=False. This eliminates shell injection vectors. - Parameterized SQL Queries: All metadata transactions in the SQLite catalog use parameterized queries, neutralizing SQL injection attempts.
Kyrex protects against:
- Path traversal and directory escapes during bundle extraction or CAS file registration.
- Command injection through structured subprocess lists running with
shell=False. - Corrupted environment bundles or tampered files through cryptographic verification.
- Local cache database corruption or tampered package hashes.
Kyrex does not currently protect against:
- Malicious packages intentionally published to and downloaded from upstream PyPI registries.
- Supply-chain attacks originating from compromised source code repositories before they are built into wheels.
- Compromised host platforms where the root system has already been broken.
Kyrex contains an extensive test suite verifying the correctness of core operations across Rust and Python layers.
- CAS engine: Verifies file hashing and directory tree layout.
- SQLite metadata: Asserts table initialization and environment registrations.
- Bundle creation: Packs environment manifest and wheels into compressed archives.
- Bundle restoration: Recreates environments from archived
.kyxpackages. - Hash validation: Confirms boundary checks and format constraints are enforced.
- CLI installation: Tests click routing and subprocess environment linking.
- Bundle restore: Reconstitutes complete environments on clean simulated systems.
- Cache rebuild: Restores missing database paths from orphaned local wheels.
- Doctor repair: Runs automatic doctor fixes using
--fix.
- Path traversal: Asserts traversal attempts are detected and blocked.
- Invalid hashes: Validates that non-hexadecimal or incorrect length inputs raise value errors.
- Corrupted bundles: Verifies that tampered archives trigger verification failure.
- Injection attempts: Confirms that subprocess arguments ignore shell meta-character injections.
Execute the test suite locally:
python -m pytest -vLatest release v1.0.0 has been successfully verified under the following conditions:
- Fresh Windows installation validation.
- Python 3.13.7 virtual environment bootstrapping.
- Maturin compilation and PyPI-compatible wheel creation.
- CLI subcommands registration (
install,bundle,restore,share,doctor,clone,bootstrap). - Cryptographic verification on CAS writes and SQLite indexing.
- Portable
.kyxbundle creation and restoration. - Command-line version check and diagnostic checks.
Kyrex v1.0 categorizes platform and interpreter compatibility as follows:
Verified via direct local validation and testing on host hardware:
- Windows 11 (x86_64)
- Python 3.13
Built, compiled, and tested via the GitHub Actions CI matrix:
- Windows 10
- Ubuntu Linux
- Debian Linux
- Fedora Linux
- Arch Linux
- macOS Intel
- macOS Apple Silicon
- Python 3.10–3.12
- Additional Linux distributions validation.
- Extended Python interpreter compatibility (such as Python 3.14).
- Target platform ARM optimizations.
- Large-scale enterprise validation.
Kyrex is designed around three goals:
- Minimize repeated package downloads over the internet.
- Maximize local cache reuse across workspaces.
- Reduce setup time for Python virtual environments.
Performance benchmarks will be published after standardized testing on identical hardware and network configurations. We do not publish synthetic benchmarks.
Rust provides the memory safety, concurrent execution models, and compiler optimizations needed to compute cryptographic hashes and write Content-Addressable Storage (CAS) files concurrently, alongside high-performance tar/zstd archiving capabilities for portable .kyx bundles.
No. Kyrex leverages uv under the hood as its primary resolution engine. It sits on top of these tools to manage caching layers, P2P network sharing, bundle packaging, and hardware auto-mapping.
Yes. Once packages are cached in the global CAS, kyrex install --offline installs dependencies and recreates environments completely without internet connectivity.
Yes. Kyrex is fully compatible with Windows 10/11 and Windows Server platforms, utilizing standard registry and system calls to identify GPU configurations.
The .kyx bundle is portable across machines running the same host operating system and architecture. It bundles the package wheels and environment metadata. Bundles created on Linux should not be restored directly on Windows.
Yes. Teleporting environments via .kyx bundles allows pipelines to unpack and restore dependency environments in seconds, bypassing resolution and download phases.
Packages are stored in a central global cache directory at ~/.cache/kyrex/cas/ grouped by the first two characters of their SHA-256 hash.
Kyrex uses mDNS (via the zeroconf protocol) to broadcast a local service named _kyrex._tcp.local.. Discovered peers are queried over HTTP for their package manifests.
Yes. When Kyrex downloads a wheel from a LAN peer, it calculates its SHA-256 hash and compares it against the index-resolved hash to prevent tampering.
Yes. Kyrex's bootstrap command parses requirements.txt or pyproject.toml configurations to resolve and install dependencies.
Kyrex automatically falls back to fetching the wheel from PyPI if a peer download fails or times out.
No. Kyrex works with pre-built wheel distributions (.whl). If a package only has source distributions (.tar.gz), it is compiled and resolved into a wheel by the underlying uv engine before caching.
Kyrex checks system drivers, paths, and registry keys to identify the system's CUDA version. If installing torch, it overrides the target index url to the PyTorch wheels index matching the CUDA version.
Yes. You can use the kyrex doctor command to verify cache integrity and remove corrupted packages.
All global metadata, registered environments, and package-to-hash mappings are stored in a SQLite database file located at ~/.cache/kyrex/metadata.db.
- Windows wheels are currently published directly.
- Linux/macOS wheels are generated through CI.
- Bundle portability depends on platform-compatible binary wheels.
- Large environments may require significant disk space.
- Peer discovery currently operates within the same local network.
Every feature added to Kyrex must satisfy:
- Performance: High speed through Rust-compiled IO and SQLite indexing.
- Reliability: Deterministic environment recreation from portable archives.
- Reproducibility: Verifiable builds with lock-free file sharing.
- Security: Cryptographic verification and secure subprocess execution.
- Minimal Configuration: Out-of-the-box setups without manual configurations.
For a detailed technical roadmap and market gap analysis, see docs/ROADMAP.md.
Note: These are planned areas of exploration and may evolve based on community feedback.
- Content-Addressable Storage (CAS) cache engine.
- Portable
.kyxenvironment bundling. - Declarative
.kyx.locklockfile formats. - ThreadPool parallel downloads & local P2P LAN sharing.
- Diagnostics exporter (
doctor --report) and automatic recovery (doctor --fix).
- Full cross-platform validation (Windows 10, Ubuntu, Debian, Fedora, Arch, macOS Intel & Apple Silicon).
- Complete Python 3.10–3.12 validation.
- Multi-platform bundle validations.
- Custom plugin system templates.
- Shell autocompletion profiles.
- Distributed cache mirroring.
- Authenticated peer sharing.
Kyrex aims to become a foundational infrastructure layer for Python dependency management and environment portability. Rather than being another package installer, Kyrex provides a unified layer for caching, environment portability, offline development, LAN package distribution, reproducible builds, and hardware-aware dependency management. The goal is to make Python environments fast, reliable, and reproducible across laptops, workstations, CI pipelines, and enterprise infrastructure.
Kyrex undergoes validation before every public release.
See the RELEASE_VALIDATION_REPORT.md and FINAL_RELEASE_REPORT.md files for test logs, compatibility matrices, and validation summaries.
Contributions of all sizes are welcome.
Whether you're fixing bugs, improving documentation, proposing new features, or optimizing performance, we'd love your help.
See CONTRIBUTING.md for setup instructions, coding guidelines, testing requirements, and the pull request workflow.
Kyrex builds upon the excellent work of the Python and Rust ecosystems. Special thanks to the communities behind:
- Python
- Rust
- PyO3
- Maturin
- uv
- SQLite
- PyPI
Without these projects, Kyrex would not be possible.
Kyrex is licensed under the MIT License. See the LICENSE file for complete details.
Build once. Cache forever. Restore anywhere.