🏔️ aspect

aspect is a lightweight featurisation layer for tabular machine-learning data.

It lets you define independent feature pipelines for different input columns, then apply them to dictionaries, Pandas data frames, local files, Hugging Face datasets, or saved dataset checkpoints.

The core idea is:

input table
  ├── numeric column  → log / identity / custom transforms
  ├── categorical col → hash / one-hot / custom transforms
  ├── SMILES column   → chemistry features, if installed
  ├── species column  → taxonomic features, if installed
  └── text column     → deep embeddings, if installed

output dataset with named feature columns

Quick start

Python API

from aspect.data import DataPipeline

data = {
    "compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
    "assay": ["MIC", "MIC", "IC50"],
    "mwt": [46.07, 60.10, 78.11],
    "pIC50": [5.0, 6.2, 4.8],
}

pipe = DataPipeline(
    column_transforms={
        "log_mwt": ("mwt", "log"),
        "assay_hash": (
            "assay",
            {"name": "hash", "kwargs": {"ndim": 8}},
        ),
        "target": ("pIC50", "identity"),
    },
    columns_to_keep=["compound_id"],
)

features = pipe(data, drop_unused_columns=True)

print(features.column_names)
print(pipe.data_out_shape)
print(features[:2])

Expected column structure:

['compound_id', 'log_mwt', 'assay_hash', 'target']

log_mwt and target are (n, 1) arrays. assay_hash is an (n, 8) array. The output is a Hugging Face Dataset, so it can be saved, sliced, converted to Pandas, or passed into a downstream PyTorch/data-loader layer.

Command-line quick start

Create a small CSV:

cat > compounds.csv <<'CSV'
compound_id,assay,mwt,pIC50
cmpd-1,MIC,46.07,5.0
cmpd-2,MIC,60.10,6.2
cmpd-3,IC50,78.11,4.8
CSV

Featurise it:

aspect featurize compounds.csv \
    --features mwt:log@log_mwt assay:hash@assay_hash pIC50@target \
    --extras compound_id \
    --output features.parquet

The feature specification means:

mwt:log@log_mwt       # read mwt, apply log, write column name `log_mwt`
assay:hash@assay_hash # read assay, apply deterministic hash vector, write column name `assay_hash`
pIC50@target          # read pIC50, identity transform, write column name `target`
--extras compound_id  # retain compound_id without transforming it

Reuse a saved feature specification

aspect serialize \
    --features mwt:log@log_mwt assay:hash@assay_hash pIC50@target \
    --extras compound_id \
    --output feature-spec.as

Then apply the same specification later:

aspect featurize compounds.csv \
    --config feature-spec.as \
    --output features-from-config.parquet

Installation

Basic installation

pip install aspect-data

Chemistry features

Install chemistry support if you want SMILES featurisation via schemist:

pip install "aspect-data[chem]"

Chemprop support

Install Chemprop support if you want to prepare Chemprop-style molecular data:

pip install "aspect-data[chemprop]"

Taxonomic features

Install taxonomy support if you want species or taxon-ID features via vectome:

pip install "aspect-data[bio]"

Deep-learning features

Install transformer support if you want Hugging Face model embeddings:

pip install "aspect-data[deep]"

Development installation

git clone https://github.com/scbirlab/aspect.git
cd aspect
pip install -e ".[dev]"

Command-line interface

The CLI has two main commands:

aspect serialize
aspect featurize

Use help for the full option list:

aspect --help
aspect serialize --help
aspect featurize --help

`aspect serialize`

Create a reusable pipeline checkpoint from a feature specification.

aspect serialize \
    --features mwt:log@log_mwt assay:hash@assay_hash \
    --extras compound_id pIC50 \
    --output feature-spec.as

`aspect featurize`

Apply a feature specification or saved config to a dataset.

aspect featurize compounds.csv \
    --features mwt:log@log_mwt assay:hash@assay_hash \
    --extras compound_id pIC50 \
    --output features.parquet

or:

aspect featurize compounds.csv \
    --config feature-spec.as \
    --output features.parquet

Slice a dataset

This is useful for testing on large datasets.

aspect featurize large-table.csv \
    --start 1000 \
    --end 2000 \
    --features assay:hash@assay_hash mwt:log@log_mwt \
    --output slice.parquet

Cache location

Use --cache to control where intermediate Hugging Face dataset files are cached.

aspect featurize compounds.csv \
    --features assay:hash@assay_hash \
    --cache .aspect-cache \
    --output features.parquet

Python API

The main class is DataPipeline.

from aspect.data import DataPipeline

A DataPipeline maps one or more input columns to named output feature columns.

pipe = DataPipeline(
    column_transforms={
        "log_mwt": ("mwt", "log"),
        "assay_hash": ("assay", "hash"),
    },
)

Apply it to a dictionary:

out = pipe({
    "mwt": [46.07, 60.10, 78.11],
    "assay": ["MIC", "MIC", "IC50"],
})

Apply it to a Pandas data frame:

import pandas as pd
from aspect.data import DataPipeline

df = pd.DataFrame({
    "mwt": [46.07, 60.10, 78.11],
    "assay": ["MIC", "MIC", "IC50"],
})

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": ("assay", {"name": "hash", "kwargs": {"ndim": 16}}),
})

out = pipe(df)

Apply it to a local file:

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})

out = pipe("compounds.csv")

Apply it to a Hugging Face dataset reference:

pipe = DataPipeline({
    "assay_hash": ("assay", "hash"),
})

out = pipe("hf://datasets/scbirlab/fang-2023-biogen-adme~scaffold-split:train")

Feature specifications

A feature specification defines:

the input column;
the transform or chain of transforms;
the output column name.

Python feature specs

The recommended Python form is:

{
    "output_column": ("input_column", "transform_name")
}

Example:

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})

For transform arguments, use a dictionary with a nested kwargs field:

pipe = DataPipeline({
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 32, "seed": 123}},
    ),
})

For chained transforms, pass a list:

pipe = DataPipeline({
    "log_hashed_assay": (
        "assay",
        [
            {"name": "hash", "kwargs": {"ndim": 16}},
            "log",
        ],
    ),
})

Only use transform chains when the output of one transform is mathematically valid input for the next. For example, hash → log is usually not sensible because hash values may be negative.

CLI feature specs

CLI specs use a compact string form:

input_column:transform@output_column

Examples:

mwt:log@log_mwt
assay:hash@assay_hash
pIC50@target

Supported forms:

Spec	Meaning
`column`	keep `column` as an extra column
`column@new_name`	identity transform from `column` to `new_name`
`column:transform`	transform `column`; auto-name the output
`column:transform@new_name`	transform `column`; write `new_name`
`column:t1:t2@new_name`	apply chained transforms

For parameterised transforms, prefer the Python API. The Python API supports structured keyword arguments cleanly through {"name": ..., "kwargs": ...}.

Available transforms

Base transforms

These are available with the standard installation.

Transform	Input	Output	Notes
`identity`	any column	array	Pass-through transform
`log`	numeric	array	Natural logarithm, `np.log`
`hash`	string-like	fixed-length vector	Deterministic string hashing
`one-hot`	categorical	vector	Python API recommended because categories must be supplied

Example:

from aspect.data import DataPipeline

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 16}},
    ),
    "assay_onehot": (
        "assay",
        {"name": "one-hot", "kwargs": {"categories": ["MIC", "IC50"]}},
    ),
})

Chemistry transforms

Require:

pip install "aspect-data[chem]"

Transform	Input	Output
`morgan-fingerprint`	SMILES	molecular fingerprint
`descriptors-2d`	SMILES	2D molecular descriptors
`descriptors-3d`	SMILES	3D molecular descriptors

Example:

from aspect.data import DataPipeline

data = {
    "compound_id": ["ethanol", "ethylamine", "benzene"],
    "smiles": ["CCO", "CCN", "c1ccccc1"],
}

pipe = DataPipeline(
    {
        "morgan": ("smiles", "morgan-fingerprint"),
        "desc2d": ("smiles", "descriptors-2d"),
    },
    columns_to_keep=["compound_id"],
)

features = pipe(data, drop_unused_columns=True)

Taxonomic transforms

Require:

pip install "aspect-data[bio]"

Transform	Input	Output
`vectome-fingerprint`	species name or taxon ID	taxonomic fingerprint

Example:

from aspect.data import DataPipeline

data = {
    "species": [
        "Mycobacterium tuberculosis",
        "Escherichia coli",
        "Staphylococcus aureus",
    ],
}

pipe = DataPipeline({
    "species_fp": (
        "species",
        {"name": "vectome-fingerprint", "kwargs": {"ndim": 128}},
    ),
})

features = pipe(data)

Deep embedding transforms

Require:

pip install "aspect-data[deep]"

Transform	Input	Output
`hf-bart`	text	aggregated BART encoder/decoder embedding

Example:

from aspect.data import DataPipeline

data = {
    "description": [
        "cell wall inhibitor",
        "protein synthesis inhibitor",
        "DNA gyrase inhibitor",
    ],
}

pipe = DataPipeline({
    "text_embedding": (
        "description",
        {
            "name": "hf-bart",
            "kwargs": {
                "ref": "facebook/bart-base",
                "aggregator": ["mean", "max"],
            },
        },
    ),
})

features = pipe(data)

The model will be loaded through Hugging Face transformers, so the first run may download model weights.

Chemprop transform

Require:

pip install "aspect-data[chemprop]"

chemprop-mol is intended for converting SMILES into Chemprop-style molecular datapoints, with optional labels and extra dense features.

Because Chemprop models often need special collation and graph batching, use this transform together with a downstream model adapter or data-loader layer rather than assuming the output should be horizontally stacked with ordinary numeric arrays.

Saving and loading pipelines

DataPipeline objects can be checkpointed.

from aspect.data import DataPipeline

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 16}},
    ),
})

pipe.save_checkpoint("feature-spec.as")

Load the checkpoint later:

from aspect.data import DataPipeline

pipe = DataPipeline().load_checkpoint("feature-spec.as")

out = pipe({
    "mwt": [46.07, 60.10, 78.11],
    "assay": ["MIC", "MIC", "IC50"],
})

If the pipeline has already been applied to data, the checkpoint can also store input/output datasets unless skipped:

pipe.save_checkpoint(
    "feature-spec-and-data.as",
    skip_data_in=False,
    skip_data_out=False,
)

For reusable feature specifications, it is usually cleaner to save the pipeline before applying it to a dataset, or to skip data when checkpointing.

Supported input and output formats

Python inputs

DataPipeline accepts:

Input type	Example
dictionary / mapping	`{"smiles": ["CCO", "CCN"]}`
Pandas `DataFrame`	`pipe(df)`
Hugging Face `Dataset`	`pipe(dataset)`
Hugging Face `IterableDataset`	`pipe(iterable_dataset)`
local file path	`pipe("data.csv")`
Hugging Face dataset ref	`pipe("hf://datasets/org/name~config:split@revision")`

Local file inputs

Supported local input extensions include:

.csv, .csv.gz
.tsv, .tsv.gz
.txt, .txt.gz
.json
.parquet
.arrow
.xml
.hf saved Hugging Face datasets

CLI outputs

aspect featurize --output can write:

Extension	Output
`.csv`, `.csv.gz`	CSV
`.tsv`, `.txt`, optionally gzipped	delimited text
`.json`	JSON
`.parquet`	Parquet
`.sql`	SQL
`.hf`	Hugging Face dataset saved to disk
unknown extension	Hugging Face dataset, with `.hf` appended

Working with output columns

By default, DataPipeline keeps the original input columns and appends feature columns.

out = pipe(data)

To keep only feature outputs and selected metadata:

pipe = DataPipeline(
    {
        "log_mwt": ("mwt", "log"),
        "assay_hash": ("assay", "hash"),
    },
    columns_to_keep=["compound_id"],
)

out = pipe(data, drop_unused_columns=True)

You can also provide extra columns at call time:

out = pipe(
    data,
    drop_unused_columns=True,
    keep_extra_columns=["compound_id", "target"],
)

After running a pipeline, inspect inferred output shapes:

print(pipe.data_out_shape)

Example:

{
    "compound_id": (),
    "log_mwt": (1,),
    "assay_hash": (256,)
}

Custom transforms

Custom transforms can be registered using register_function.

A registered transform should be a factory: it receives configuration arguments and returns a callable with signature:

fn(data, input_column) -> numpy.ndarray

Example:

import numpy as np

from aspect.data import DataPipeline
from aspect.transform.registry import register_function


@register_function("square")
def Square():
    def _square(data, input_column):
        x = np.asarray(data[input_column], dtype=float)
        return x ** 2

    return _square


pipe = DataPipeline({
    "mwt_squared": ("mwt", "square"),
})

out = pipe({
    "mwt": [46.07, 60.10, 78.11],
})

With arguments:

@register_function("scale")
def Scale(factor=1.0):
    def _scale(data, input_column):
        x = np.asarray(data[input_column], dtype=float)
        return x * factor

    return _scale


pipe = DataPipeline({
    "scaled_mwt": (
        "mwt",
        {"name": "scale", "kwargs": {"factor": 0.001}},
    ),
})

If you save a pipeline using a custom transform, make sure the module that registers the transform is imported before loading the checkpoint.

Optional chemistry, taxonomy, and deep-learning examples

Mixed ADME-style table

from aspect.data import DataPipeline

data = {
    "compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
    "smiles": ["CCO", "CCN", "c1ccccc1"],
    "assay": ["solubility", "clearance", "microsome"],
    "species": ["M. tuberculosis", "E. coli", "S. aureus"],
    "mwt": [46.07, 45.08, 78.11],
    "label": [0.1, 0.4, 0.8],
}

pipe = DataPipeline(
    {
        "morgan": ("smiles", "morgan-fingerprint"),
        "desc2d": ("smiles", "descriptors-2d"),
        "assay_hash": (
            "assay",
            {"name": "hash", "kwargs": {"ndim": 32}},
        ),
        "species_fp": (
            "species",
            {"name": "vectome-fingerprint", "kwargs": {"ndim": 64}},
        ),
        "log_mwt": ("mwt", "log"),
        "y": ("label", "identity"),
    },
    columns_to_keep=["compound_id"],
)

features = pipe(data, drop_unused_columns=True)

This produces separate feature columns for chemical structure, descriptors, assay identity, species identity, molecular weight, and target label.

A downstream model adapter can decide whether to:

concatenate dense arrays;
route different columns into different neural-network towers;
collate graph-like objects separately;
keep labels and metadata separate from model inputs.

Development

Install development dependencies:

pip install -e ".[dev]"

Run tests:

pytest

Run the CLI smoke test script:

bash test/scripts/run-tests.sh

Run selected doctests:

python -m doctest aspect/data.py
python -m doctest aspect/transform/base.py
python -m doctest aspect/transform/functions.py

Issues, problems, suggestions

Please open an issue on GitHub:

https://github.com/scbirlab/aspect/issues

Documentation

(To come at ReadTheDocs.)

Name		Name	Last commit message	Last commit date
Latest commit History 242 Commits
.github/workflows		.github/workflows
aspect		aspect
docs		docs
test/scripts		test/scripts
.gitignore		.gitignore
.readthedocs.yml		.readthedocs.yml
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml

Folders and files

Latest commit

History

Repository files navigation

🏔️ aspect

Contents

Quick start

Python API

Command-line quick start

Reuse a saved feature specification

Installation

Basic installation

Chemistry features

Chemprop support

Taxonomic features

Deep-learning features

Development installation

Command-line interface

aspect serialize

aspect featurize

Slice a dataset

Cache location

Python API

Feature specifications

Python feature specs

CLI feature specs

Available transforms

Base transforms

Chemistry transforms

Taxonomic transforms

Deep embedding transforms

Chemprop transform

Saving and loading pipelines

Supported input and output formats

Python inputs

Local file inputs

CLI outputs

Working with output columns

Custom transforms

Optional chemistry, taxonomy, and deep-learning examples

Mixed ADME-style table

Development

Issues, problems, suggestions

Documentation

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 4

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`aspect serialize`

`aspect featurize`

Packages