Now available on PyPI! Install with:
pip install calm-data-generator
CALM-Data-Generator is a comprehensive Python library for synthetic data generation with advanced features for:
- Clinical/Medical Data - Generate realistic patient demographics, genes, proteins
- Tabular Data Synthesis - CTGAN, TVAE, CART, and more
- Time Series - TimeGAN, TimeVAE, FourierFlows
- Single-Cell - scVI, GEARS (Perturbation Prediction)
- Latent Differentiation - Force class separability in TVAE & scVI
- Flexible Clipping - Strict or permissive data range enforcement
- Drift Injection - Test ML model robustness with controlled drift
- Privacy Assessment - DCR metrics for re-identification risk
- Scenario Evolution - Feature evolution and target construction
Calm-Data-Generator is optimized for structured tabular data. It is designed to handle:
- Classification (Binary & Multi-class)
- Regression (Continuous variables)
- Multi-label problems
- Clustering (Preserving natural groupings)
- Time Series (Temporal correlations and patterns)
- Single-Cell / Genomics (scRNA-seq expression data)
Important
This library is NOT designed for unstructured data such as Images, Videos, or Audio. It does not include Computer Vision or Signal Processing models.
CALM-Data-Generator is not just another synthetic data tool—it's a unified ecosystem that brings together the best open-source libraries under a single, consistent API:
Instead of learning and managing multiple complex libraries separately, CALM-Data-Generator provides:
- One API for 15+ synthesis methods from different sources (Synthcity, scvi-tools, GEARS, imbalanced-learn, etc.)
- Seamless interoperability between tabular, time-series, streaming, and genomic data generators
- Consistent configuration across all methods with automatic parameter validation
- Integrated reporting with YData Profiling for all generation methods
- Extensible generator hierarchy:
BaseGenerator->ComplexGenerator-> domain generators. New domains (Finance, IoT, Insurance) inherit three reusable mathematical engines (Gaussian Copula unconditional, Gaussian Copula conditional, stochastic effects) without duplicating code.
The DriftInjector module is one of the most comprehensive drift simulation tools available:
- 14+ drift types: Feature drift (gradual, abrupt, incremental, recurrent), label drift, concept drift, correlation drift, outlier injection, and more
- Correlation-aware drift: Propagate realistic drift across correlated features (e.g., increase income → increase spending)
- Multi-modal drift profiles: Sigmoid, linear, cosine transitions for gradual drift
- Conditional drift: Apply drift only to specific data subsets based on business rules
- Functional drift (Pilar 5): Drift magnitude varies per row as a function of another column (e.g., sensor noise that scales exponentially with temperature)
- Causal cascades (Pilar 5): Define a causal DAG and propagate perturbations through non-linear transfer functions (
CausalEngine) - Integrated with generators: Inject drift directly during synthesis or post-hoc on existing data
- Perfect for MLOps: Test data drift monitoring, concept drift detection, and model robustness before production
The ScenarioInjector creates deterministic or stochastic temporal patterns:
- 7 evolution types:
linear,exponential_growth,decay,seasonal,step,noise,random_walk - driven_by (Pilar 5): Feature delta per row = f(current value of another column) — couples variables without a full DAG
- Target construction: Build synthetic ground-truth target variables from feature formulas or callables
- Future projection: Extend historical datasets into future time periods
In summary: While other tools focus on a single approach (e.g., just GANs, just statistical methods), CALM-Data-Generator unifies the ecosystem and adds production-grade drift simulation that most libraries don't offer.
This library leverages and unifies best-in-class open-source tools to provide a seamless data generation experience:
| Library | Methods / Features | Docs |
|---|---|---|
| Synthcity | CTGAN, TVAE, DDPM, TimeGAN, TimeVAE, FFlows, GREAT, RTVAE, BN, DPGAN, PATEGAN, Conditional Drift | synthcity docs |
| scikit-learn | CART, RF, KDE, GMM, metrics, preprocessing | sklearn docs |
| LightGBM | LGBM synthesis (FCS-style) | lgbm docs |
| XGBoost | XGBoost synthesis (FCS-style) | xgb docs |
| Copulae | Copula, Windowed Copula | copulae docs |
| imbalanced-learn | SMOTE, ADASYN | imblearn docs |
| hmmlearn | HMM synthesis | hmmlearn docs |
| PyTorch | Diffusion (custom), backend for all deep learning methods | torch docs |
| Library | Methods / Features | Docs |
|---|---|---|
| scvi-tools | scVI, scANVI | scvi docs |
| GEARS | Perturbation prediction (GEARS method) | gears docs |
| AnnData | Single-cell data structures for scVI/scANVI/GEARS | anndata docs |
| Library | Methods / Features | Docs |
|---|---|---|
| SDMetrics | QualityReporter — statistical quality scores | sdmetrics docs |
| YData Profiling | Automated data profiling reports | ydata docs |
| Plotly | Interactive visualizations and dashboards | plotly docs |
| Library | Methods / Features | Docs |
|---|---|---|
| SciPy | Gaussian Copula (ComplexGenerator), PSD matrix repair, statistics | scipy docs |
| NumPy | Numerical arrays, all generators | numpy docs |
| Pandas | DataFrames, all generators | pandas docs |
| Library | Methods / Features | Docs |
|---|---|---|
| River | StreamGenerator — Agrawal, SEA, Hyperplane, Sine, etc. ([stream] extra) |
river docs |
Calm-Data-Generator includes 19 ready-to-use Presets covering the most common synthetic data scenarios. Each preset encapsulates a generator configuration (method, hyperparameters, reporting) and exposes a single .generate() call.
Tip
Presets are starting points: instantiate a preset, call .generate(), and override any parameter you need via __init__ arguments (random_state, verbose, fast_dev_run).
Speed & Prototyping
| Preset | Method | Use Case |
|---|---|---|
FastPreset |
LightGBM | Fastest generation, configurable via kwargs |
FastPrototypePreset |
LightGBM | CI/CD pipelines, integration tests (fixed 10 iterations) |
Quality & Fidelity
| Preset | Method | Use Case |
|---|---|---|
HighFidelityPreset |
CTGAN (1000 epochs) | Production-quality tabular data |
DiffusionPreset |
TabDDPM (1000 steps) | Complex multi-modal distributions |
CopulaPreset |
Gaussian Copula | Fast statistical baseline, dependency modeling |
DataQualityAuditPreset |
TVAE + full report | Quality audit with comprehensive automated reporting |
Class Distribution
| Preset | Method | Use Case |
|---|---|---|
ImbalancedGeneratorPreset |
CTGAN | Force a specific minority/majority ratio (binary) |
BalancedDataGeneratorPreset |
SMOTE | Oversample minority classes to balance a skewed dataset |
Time Series
| Preset | Method | Use Case |
|---|---|---|
TimeSeriesPreset |
TimeGAN / TimeVAE / FourierFlows | Sequential data with temporal dynamics |
SeasonalTimeSeriesPreset |
TimeGAN + ScenarioInjector | Time series with injected sinusoidal seasonality |
Drift & Scenarios
| Preset | Method | Use Case |
|---|---|---|
DriftScenarioPreset |
CTGAN + DriftConfig | Stress-test drift detection systems |
GradualDriftPreset |
CTGAN + linear drift | Simulate slow linear feature drift |
ConceptDriftPreset |
CTGAN + concept drift | Alter P(y|x) relationships |
ScenarioInjectorPreset |
ScenarioInjector | Apply custom evolution scenarios to existing data |
Clinical & Omics
| Preset | Method | Use Case |
|---|---|---|
LongitudinalHealthPreset |
ClinicalDataGenerator | Multi-visit patient records |
RareDiseasePreset |
ClinicalDataGenerator | Cohorts with rare condition (default 1% prevalence) |
OmicsIntegrationPreset |
ClinicalDataGenerator | Multi-omics (clinical + gene expression + proteomics) |
SingleCellQualityPreset |
scVI (400 epochs) | High-quality single-cell RNA-seq data |
Full parameter reference:
calm_data_generator/docs/PRESETS_REFERENCE.md
from calm_data_generator.presets import FastPreset, HighFidelityPreset, ImbalancedGeneratorPreset
# --- Fast generation ---
preset = FastPreset(random_state=42)
synthetic_df = preset.generate(data=real_df, n_samples=1000)
# --- High-fidelity production data ---
preset = HighFidelityPreset(random_state=42)
synthetic_df = preset.generate(data=real_df, n_samples=5000)
# --- Imbalanced dataset (5% minority) ---
preset = ImbalancedGeneratorPreset(random_state=42)
synthetic_df = preset.generate(
data=real_df, n_samples=2000,
target_col="label", imbalance_ratio=0.05
)from calm_data_generator.presets import TimeSeriesPreset, SeasonalTimeSeriesPreset
# --- Time series with TimeGAN ---
preset = TimeSeriesPreset(random_state=42)
synthetic_df = preset.generate(
data=ts_df, n_samples=500, sequence_key="patient_id", time_key="visit_date"
)
# --- Seasonal time series (monthly pattern) ---
preset = SeasonalTimeSeriesPreset(random_state=42)
synthetic_df = preset.generate(
data=ts_df, n_samples=500,
time_col="date", seasonal_cols=["sales", "demand"],
period=12, amplitude=2.0
)from calm_data_generator.presets import LongitudinalHealthPreset, SingleCellQualityPreset
# --- Clinical longitudinal data (multi-visit) ---
preset = LongitudinalHealthPreset(random_state=42)
result = preset.generate(n_samples=200, n_visits=6)
# --- Single-cell RNA-seq ---
preset = SingleCellQualityPreset(random_state=42)
synthetic_df = preset.generate(data=adata_df, n_samples=500)| Library | Role | Usage in Calm-Data-Generator |
|---|---|---|
| Synthcity | Deep Learning Engine | Powers CTGAN, TVAE, DDPM, TimeGAN. Handling privacy & fidelity. |
| scvi-tools | Single-Cell Analysis | Powers scvi method for high-dimensional genomic/transcriptomic data. |
| GEARS | Graph Perturbation | Powers gears method for predicting single-cell perturbation effects. |
| River | Streaming ML | Powers StreamGenerator for concept drift simulation and real-time data flow. |
| YData Profiling | Reporting | Generates automated quality reports (QualityReporter). |
| Pydantic | Validation | Ensures strict type checking and configuration management. |
| PyTorch | Backend | Underlying tensor computation for all deep learning models. |
| Copulae | Statistical Modeling | Powers the copula and windowed_copula methods for multivariate dependence modeling and drift-aware generation. |
| hmmlearn | Statistical Modeling | Powers the hmm method for drift-aware generation via Hidden Markov Model regime transitions. |
| SciPy | Mathematical Core | Powers the Gaussian Copula engines inside ComplexGenerator (unconditional and conditional) for correlated multi-variate generation across domains. |
A key advantage of Calm-Data-Generator is enabling the use of private data in public or collaborative environments:
- Private Origin: You start with sensitive data (e.g., GDPR/HIPAA restricted) that cannot leave your secure environment.
- Synthetic Twin: The library generates a synthetic dataset that statistically mirrors the original but contains no real individuals.
- Safe Distribution: Once validated (using
QualityReporter's privacy checks), this synthetic dataset allows for risk-free sharing, model training, and testing without exposing confidential information.
- MLOps Monitoring Validation: Use StreamGenerator and DriftInjector to simulate data drift (gradual, abrupt) and verify if your monitoring alerts trigger correctly before deployment.
- Biomedical Research (HealthTech): Generate synthetic patient cohorts with ClinicalDataGenerator that preserve complex biological correlations (e.g., gene-age relationships) for collaborative studies without compromising patient privacy.
- Stress Testing ("What-If" Analysis): Use ScenarioInjector to simulate future scenarios (e.g., "What if the customer age base increases by 10 years?") and measure model performance degradation under stress.
- Correlation-Aware Drift: Inject drift that realistically propagates to correlated features (e.g., increasing income also proportionally increases spending) using the
correlations=Trueparameter. - Development Data: Provide developers with high-fidelity synthetic replicas of production databases, allowing them to build and test features safely without accessing sensitive real-world data.
Minimalist view of the system's core components and data flow.
Warning
The installation is heavy (~2-3 GB) and may take several minutes. Use a fresh virtual environment.
- GitHub (Recommended for latest features): The
mainbranch contains the most up-to-date version with the latest bug fixes and features. - PyPI (Stable): Releases on PyPI are stable versions updated less frequently for major changes.
PyTorch must be installed before the library so pip resolves the correct wheel (CPU, CUDA, or ROCm).
# CPU — Mac Intel, CI environments, no GPU
pip install "torch>=2.2.0" "torchvision>=0.17.0"
# CUDA 12.4 — Linux / Windows with NVIDIA GPU
pip install "torch>=2.2.0" "torchvision>=0.17.0" --index-url https://download.pytorch.org/whl/cu124
# ROCm 6.1 — Linux with AMD GPU
pip install "torch>=2.2.0" "torchvision>=0.17.0" --index-url https://download.pytorch.org/whl/rocm6.1
# Mac Apple Silicon (M1/M2/M3) — MPS is auto-detected, no special wheel needed
pip install "torch>=2.2.0" "torchvision>=0.17.0"Replace
cu124with your installed CUDA version (e.g.cu118,cu121). Find yours withnvidia-smi.
pip install --upgrade pip setuptools wheel
pip install calm-data-generator# River streaming support
pip install "calm-data-generator[stream]"
# GEARS perturbation prediction (requires PyG wheels — see below)
pip install "calm-data-generator[gears]"
# Singling-Out privacy risk metric (anonymeter)
pip install "calm-data-generator[privacy]"
# Full suite (stream + gears + privacy)
pip install "calm-data-generator[full]"GEARS / PyTorch Geometric requires platform-specific wheels from the PyG index. Install them before
calm-data-generator[gears]:# Get your torch version first python -c "import torch; print(torch.__version__)" # CPU example (replace torch-2.2.0 and +cpu with your version and platform) pip install torch-geometric --find-links https://data.pyg.org/whl/torch-2.2.0+cpu.html # CUDA 12.4 example pip install torch-geometric --find-links https://data.pyg.org/whl/torch-2.2.0+cu124.html pip install "calm-data-generator[gears]"
From source (GitHub - Latest Updates):
# Option A: Install directly from GitHub
pip install git+https://github.com/CALMDataGenerator/Calm-Data-Generator.git
# Option B: Clone and install (for development)
git clone https://github.com/CALMDataGenerator/Calm-Data-Generator.git
cd Calm-Data-Generator
pip install .Zsh shell (macOS/Linux): If brackets cause errors, use quotes:
pip install "calm-data-generator[stream]"River compilation errors (Linux/macOS):
# Ubuntu/Debian
sudo apt install build-essential python3-dev
# macOS
xcode-select --install
# Then retry
pip install calm-data-generatorWindows users: Install Visual Studio Build Tools first:
- Download Visual Studio Build Tools
- Install "Desktop development with C++"
- Then retry installation
Windows — Long Path error during installation:
Some packages (e.g. orbax-checkpoint) contain very deep directory structures that exceed the Windows default 260-character path limit. If you see an error like:
OSError: [Errno 2] No such file or directory: 'C:\...\very\long\path'
HINT: This error might have occurred since this system does not have Windows Long Path support enabled.
Enable long paths via PowerShell (run as Administrator):
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -ForceThen restart your machine and retry installation. Alternatively, install your virtual environment in a short root path (e.g. C:\venv\) to reduce total path length.
Dependency conflicts: Use a clean virtual environment:
python -m venv venv
source venv/bin/activate # Linux/macOS
# or: venv\Scripts\activate # Windows
pip install calm-data-generatorfrom calm_data_generator import RealGenerator
import pandas as pd
# Your real dataset (can be a DataFrame, path to .csv, .h5, or .h5ad)
data = pd.read_csv("your_data.csv") # or "your_data.h5ad"
# Initialize generator
gen = RealGenerator()
# Generate 1000 synthetic samples using CTGAN
synthetic = gen.generate(
data=data,
n_samples=1000,
method='ctgan',
target_col='label',
differentiation_factor=2.0, # NEW: Enhance class separability
clipping_mode='permissive', # NEW: Manage data ranges
epochs=300,
batch_size=500,
discriminator_steps=1,
)
print(f"Generated {len(synthetic)} samples")For the common case — train once, sample as many times as you want — use the sklearn-style wrapper:
gen = RealGenerator(auto_report=False).fit(data, method="cart", target_col="label")
small_batch = gen.sample(100)
large_batch = gen.sample(10_000) # reuses the fitted model, no retraining
# Chaining works too:
synthetic = RealGenerator().fit(data, method="ctgan").sample(1000)GPU is auto-detected at runtime in the order CUDA → MPS → CPU. No parameter is needed for most methods.
| Method | Device support | Notes |
|---|---|---|
scvi, scanvi |
CUDA · MPS · CPU | Auto-detected via Lightning accelerator |
gears |
CUDA · MPS · CPU | Auto-detected via PyTorch device |
ctgan, tvae, rtvae, ddpm, timegan, timevae, fflows, dpgan, pategan, great |
CUDA · CPU | Auto-detected internally by Synthcity |
cart, rf, lgbm, xgboost, gmm, copula, windowed_copula, kde, smote, adasyn, hmm, bn |
CPU only | Statistical / tree-based, no GPU path |
Mac Apple Silicon (M1/M2/M3): MPS is used automatically for scvi, scanvi, and gears when torch.backends.mps.is_available() returns True. Install the standard CPU wheel — no special MPS wheel is required.
To override the device explicitly:
# Force CPU for scVI/scANVI
gen.generate(data=data, n_samples=1000, method='scvi', accelerator='cpu')
# Force a specific device for GEARS
gen.generate(data=data, n_samples=1000, method='gears', device='cuda')
gen.generate(data=data, n_samples=1000, method='gears', device='mps')from calm_data_generator import ClinicalDataGenerator
from calm_data_generator.generators.configs import DateConfig
gen = ClinicalDataGenerator()
# Generate patient data with genes and proteins
result = gen.generate(
n_samples=100,
n_genes=500,
n_proteins=200,
date_config=DateConfig(start_date="2024-01-01")
)
demographics = result['demographics']
genes = result['genes']
proteins = result['proteins']Option 1: Directly from generate() (recommended)
from calm_data_generator import RealGenerator
gen = RealGenerator()
# Generate synthetic data WITH drift in one call
synthetic = gen.generate(
data=real_data,
n_samples=1000,
method='ctgan',
target_col='label',
drift_injection_config=[
{
"method": "inject_drift",
"params": {
"columns": ["age", "income", "label"],
"drift_mode": "gradual", # Auto-detects column types
"drift_magnitude": 0.3,
"center": 500,
"width": 200
}
}
]
)Option 2: Standalone DriftInjector
from calm_data_generator import DriftInjector
injector = DriftInjector()
# Unified drift injection (auto-detects types)
drifted_data = injector.inject_drift(
df=data,
columns=['feature1', 'feature2', 'status'],
drift_mode='gradual',
drift_magnitude=0.5,
# Optional specific configs
numeric_operation='shift',
categorical_operation='frequency',
boolean_operation='flip'
)
**Available drift methods:** `inject_feature_drift`, `inject_feature_drift_gradual`, `inject_feature_drift_incremental`, `inject_feature_drift_recurrent`, `inject_label_drift`, `inject_concept_drift`, `inject_categorical_frequency_drift`, and more. See [DRIFT_INJECTOR_REFERENCE.md](calm_data_generator/docs/DRIFT_INJECTOR_REFERENCE.md).
### Single-Cell / Gene Expression Data
Generate synthetic single-cell RNA-seq-like data using specialized VAE models:
```python
from calm_data_generator import RealGenerator
gen = RealGenerator()
# scVI: Generate new cells from scratch
synthetic = gen.generate(
data="expression_data.h5ad", # Paths to .h5 or .h5ad are supported directly
n_samples=1000,
method='scvi',
target_col='cell_type',
epochs=100,
n_latent=10,
)
| Method | Use Case |
|---|---|
scvi |
Generate new cells from learned distribution |
Once you have synthetic single-cell data, validate its quality using scGFT Evaluator via QualityReporter:
from calm_data_generator.reports.QualityReporter import QualityReporter
from calm_data_generator.generators.configs import ReportConfig
reporter = QualityReporter(verbose=True)
reporter.generate_comprehensive_report(
real_df=real_df,
synthetic_df=synthetic_df,
generator_name="SingleCell_Example",
report_config=ReportConfig(
output_dir="./sc_report",
use_scgft=True,
target_column="cell_type",
),
)See REPORTS_REFERENCE.md for full details.
from calm_data_generator import StreamGenerator
stream_gen = StreamGenerator()
# Generate a data stream with Concept Drift
stream_data = stream_gen.generate(
n_chunks=10,
chunk_size=1000,
concept_drift=True, # Simulate concept drift over time
n_features=10
)
print(f"Generated stream with {len(stream_data)} total samples")from calm_data_generator import QualityReporter
# Generate a quality report comparing real vs synthetic data
reporter = QualityReporter()
reporter.generate_report(
real_data=data,
synthetic_data=synthetic,
output_dir="./quality_report",
target_col="target"
)
# Report saved to ./quality_report/report.html
# Results JSON (including compared_data_files) saved to ./quality_report/report_results.jsonFor a fast, in-memory check without writing any file (quality scores, KS/MMD/Wasserstein
statistical tests, and TSTR if you pass a target_column), use evaluate() instead:
result = reporter.evaluate(real_data, synthetic, target_column="target")
print(result["quality_scores"], result["statistical_metrics"])For single-cell data, enable scGFT evaluation (Graph Fourier Transform-based manifold preservation metrics):
from calm_data_generator.generators.configs import ReportConfig
reporter.generate_comprehensive_report(
real_df=real_df,
synthetic_df=synthetic_df,
generator_name="MyGen",
report_config=ReportConfig(
output_dir="./sc_report",
use_scgft=True,
target_column="cell_type",
),
)
# Generates scgft_report.html with ARI, MMD, Jaccard, Kendall Tau metrics| Module | Import | Description |
|---|---|---|
| Tabular | generators.tabular |
RealGenerator, QualityReporter |
| Clinical | generators.clinical |
ClinicalDataGenerator, ClinicalDataGeneratorBlock |
| Stream | `generators.stream` | StreamGenerator, StreamBlockGenerator |
| Blocks | generators.tabular |
RealBlockGenerator |
| Drift | generators.drift |
DriftInjector |
| Dynamics | generators.dynamics |
ScenarioInjector |
| Reports | reports |
Visualizer |
| Method | Type | Description | Requirements |
|---|---|---|---|
cart |
ML | CART-based iterative synthesis (fast) | Base |
rf |
ML | Random Forest synthesis | Base |
lgbm |
ML | LightGBM-based synthesis | Base |
xgboost |
ML | XGBoost-based synthesis | Base |
gmm |
Statistical | Gaussian Mixture Models | Base |
kde |
Statistical | Kernel Density Estimation (numeric only) | Base |
copula |
Statistical | Gaussian Copula synthesis | Base |
windowed_copula |
Drift-Aware | Gaussian Copula interpolated across time windows | Base |
hmm |
Drift-Aware | Hidden Markov Model — drift via regime transitions | Base |
smote |
Augmentation | SMOTE oversampling | Base |
adasyn |
Augmentation | ADASYN adaptive sampling | Base |
resample |
Augmentation | Weighted resampling from original data | Base |
ctgan |
DL | Conditional GAN for tabular data | synthcity |
tvae |
DL | Variational Autoencoder | synthcity |
rtvae |
DL | Robust TVAE | synthcity |
great |
DL | GReaT — LLM-based tabular generation | synthcity |
bn |
Probabilistic | Bayesian Network — causal dependencies | synthcity |
diffusion |
DL | Tabular Diffusion (custom, lightweight) | Base (PyTorch) |
ddpm |
DL | TabDDPM — advanced diffusion | synthcity |
dpgan |
DL + Privacy | Differentially private GAN | synthcity |
pategan |
DL + Privacy | PATE-GAN — teacher-ensemble privacy | synthcity |
timegan |
Time Series | TimeGAN for sequential data | synthcity |
timevae |
Time Series | TimeVAE for sequential data | synthcity |
fflows |
Time Series | FourierFlows — normalizing flows, stable for periodic series | synthcity |
conditional_drift |
Drift-Aware | Temporal stage conditioning via TVAE/CTGAN | synthcity |
scvi |
Single-Cell | scVI (VAE) for RNA-seq expression data | scvi-tools |
scanvi |
Single-Cell | scANVI — semi-supervised, label-aware scVI | scvi-tools |
gears |
Single-Cell | GEARS — perturbation prediction via GNN | [gears] extra |
# List all tutorials
calm-data-generator tutorials
# Show a specific tutorial
calm-data-generator tutorials show 1
# Run a tutorial
calm-data-generator tutorials run 1
# Show version
calm-data-generator version| # | Tutorial | Description |
|---|---|---|
| 1 | Real Generator | Tabular data synthesis |
| 2 | Clinical Generator | Clinical/medical data |
| 3 | Drift Injector | Drift injection for ML |
| 4 | Stream Generator | Stream-based generation |
| 5 | Scenario Injector | Feature evolution |
Explore the full documentation in the calm_data_generator/docs/ directory:
| Document | Description |
|---|---|
| DOCUMENTATION.md | Main User Guide. Comprehensive manual covering all modules, concepts, and advanced usage. |
| REAL_GENERATOR_REFERENCE.md | API Reference for RealGenerator. Detailed parameters for all synthesis methods (ctgan, lgbm, scvi, etc.). |
| DRIFT_INJECTOR_REFERENCE.md | API Reference for DriftInjector. Guide to using inject_drift and specialized drift capabilities. |
| STREAM_GENERATOR_REFERENCE.md | API Reference for StreamGenerator. Details on stream simulation and drift integration. |
| CLINICAL_GENERATOR_REFERENCE.md | API Reference for ClinicalGenerator. Configuration for genes, proteins, and patient data. |
| API.md | Technical API Index. High-level index of classes and functions. |
Originally created by Alejandro Belda Fernandez (Universidad de Murcia), who led the development of the library through version 2.3.0.
Currently maintained by Antonio Guillen Teruel (Universidad de Murcia), from version 2.3.0 onwards.
The single-cell evaluation integrated via use_scgft is powered by scGFT Evaluator, authored by Nasim El Arifi Ahmed (Universidad de Murcia).
See AUTHORS for the full attribution and CITATION.cff if you need to cite this software in academic work.
Contact: calmdatagenerator@outlook.es — for questions about the library. For bugs and feature requests, please open an issue.
MIT License - see LICENSE file
See CHANGELOG.md for the full history. Summary of recent releases:
- ComplexGenerator: new abstract layer with 3 reusable mathematical engines (Gaussian Copula unconditional/conditional + stochastic effects).
ClinicalDataGeneratornow inherits from it. - CausalEngine: DAG-based causal cascade with topological sort (Kahn's algorithm) and non-linear transfer functions.
inject_functional_drift(): per-row drift magnitude = f(driver column value).inject_causal_cascade(): full causal propagation integrated intoDriftInjector.driven_byevolve type:ScenarioInjectorfeature delta driven by another column per row.- Bug fixes: CART datetime handling,
bnmethod dispatch,conditional_driftSynthcity API,windowed_copula1D array reshape. - Tests: 186 passed, 0 failed — all
unittest.TestCasefiles converted to pytest.
differentiation_factor,clipping_mode,use_latent_samplingparameters.- Windowed Copula, Conditional Drift, DPGAN, PATEGAN synthesis methods.
We stand on the shoulders of giants. This library is possible thanks to these amazing open-source projects:
- Synthcity (Apache 2.0) - The engine behind our deep learning models.
- River (BSD-3-Clause) - Powering our streaming capabilities.
- YData Profiling (MIT) - Providing comprehensive data reporting.
- scvi-tools (BSD-3-Clause) - Enabling single-cell analysis.
- GEARS (MIT) - Supporting graph-based perturbation prediction.
- Imbalanced-learn (MIT) - Providing SMOTE and ADASYN implementations.
- SDMetrics (MIT) - Powering the standardized metrics in our QualityReporter.
- Copulae (MIT) - Enabling multivariate dependence modeling via Gaussian Copulas.
- AnnData (BSD-3-Clause) - Providing the core data structure for single-cell and omics integration.
- LightGBM (MIT) - Powering our gradient boosting synthesis method.
- PyTorch (BSD-3-Clause) - The deep learning framework powering our generative models.
- PyTorch Geometric (MIT) - Enabling Graph Neural Network operations for relational data.
- XGBoost (Apache-2.0) - optimized distributed gradient boosting library.
- Hugging Face Hub (Apache-2.0) - Facilitating model sharing and versioning.
- Plotly (MIT) - Enabling interactive data visualizations.
- hmmlearn (BSD-3-Clause) - Powering the
hmmmethod for drift-aware generation via Hidden Markov Models. - scgft-evaluator - Providing Graph Fourier Transform-based evaluation for single-cell synthetic data quality assessment.