Skip to content

Sigilweaver/SigilYX

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

143 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SigilYX

Alteryx YXDB reader and writer for Rust, with Python bindings.

CI DOI crates.io PyPI docs.rs License: Apache-2.0 Rust MSRV

YXDB is the native binary format used by Alteryx Designer. SigilYX is a standalone, cross-platform library that reads and writes .yxdb files. The core is written in Rust; Python bindings are built on top with PyO3 and integrate with Polars, PyArrow, and Pandas.

Format scope: Full read/write support for the E1 (original engine) YXDB layout. Experimental read support for E2 (AMP engine) is included: 13 field types have been verified against real E2 files; 4 rarer types (Blob, SpatialObj, Time, WString) have speculative decoders behind an opt-in flag. E2 writing is not yet supported. See SPECIFICATION-E1.md and SPECIFICATION-E2.md.

Packages

Package Install Docs
Python (sigilyx) pip install sigilyx PyPI - API docs
Rust (sigilyx) sigilyx = "0.3" in Cargo.toml crates.io - docs.rs

What's in the box

  • E1 + E2 format support - read both original (E1) and AMP-engine (E2) layouts; full E1 write support.
  • All 17 E1 field types - including FixedDecimal, WString, Blob, and SpatialObj.
  • Multiple output formats - Polars, PyArrow, or Pandas from the same call.
  • Streaming - batched reads with constant memory, and Polars LazyFrame scans with projection and row-limit pushdown.
  • Spatial - SpatialObj columns decoded to ISO WKB (compatible with Shapely, PostGIS, GDAL).
  • Cross-platform - Linux, macOS, and Windows wheels for x64 and ARM; no native Alteryx install required.

Quick look

Python Rust
import polars as pl
import sigilyx  # registers pl.read_yxdb() etc.

df = pl.read_yxdb("data.yxdb")
df.yxdb.write("output.yxdb")
use sigilyx::{read_yxdb, write_yxdb, SpatialMode};

let df = read_yxdb("data.yxdb", SpatialMode::Wkb)?;
write_yxdb("output.yxdb", &df, &[])?;

Project layout

sigilyx/
  sigilyx/            # Rust core library (published to crates.io)
  sigilyx-python/     # PyO3 + pyo3-polars bindings
  python/sigilyx/     # Python wrapper (__init__.py - Polars/PyArrow/Pandas API)
  tests/              # Python test suite
  docs/               # Docusaurus site (sigilweaver.app/sigilyx/)
  SPECIFICATION-E1.md # YXDB E1 format spec
  SPECIFICATION-E2.md # YXDB E2 format spec (experimental)
Building from source

Requires Rust (>= 1.75) and Python 3.9+.

git clone https://github.com/Sigilweaver/SigilYX.git
cd SigilYX
python -m venv .venv
.venv\Scripts\activate         # Windows
# source .venv/bin/activate    # macOS / Linux
pip install maturin polars pyarrow pandas
maturin develop --release
Running tests
# Rust tests
cargo test --workspace

# Python tests
pytest tests/ -v

License

Apache License 2.0.

The format specification was derived from publicly available open-source YXDB implementations, including Alteryx's own OpenYXDB (MIT License); the implementation itself is original. See THIRD_PARTY_LICENSES.md for attribution of vendored components.

About

High-performance YXDB reader and writer

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages