Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.13"]
python-version: ["3.9", "3.14"]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -27,13 +27,14 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
- name: Run Cargo Tests
run: cargo test --features python -- --nocapture --test-threads=1
run: cargo test -- --nocapture --test-threads=1
- name: Run pytest
run: |
# just venv pytest
rm -rf .venv
python3 -m venv .venv
. .venv/bin/activate
.venv/bin/pip install --upgrade pip
.venv/bin/pip install wheel pytest maturin
maturin develop
.venv/bin/pytest python/tests
Expand All @@ -46,11 +47,11 @@ jobs:
matrix:
platform:
- target: x64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
- target: aarch64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
- target: armv7
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -77,13 +78,13 @@ jobs:
platform:
- target: x86_64-unknown-linux-musl
arch: x86_64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
- target: i686-unknown-linux-musl
arch: x86
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
- target: aarch64-unknown-linux-musl
arch: aarch64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
# all values: [x86_64, x86, aarch64, armhf, armv7, ppc64le, riscv64, s390x]
# { target: "armv7-unknown-linux-musleabihf", image_tag: "armv7" },
# { target: "powerpc64le-unknown-linux-musl", image_tag: "ppc64le" },
Expand Down Expand Up @@ -113,7 +114,7 @@ jobs:
strategy:
matrix:
target: [x64, x86]
interpreter: ["3.9", "3.10", "3.11", "3.12", "3.13"]
interpreter: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -151,9 +152,9 @@ jobs:
matrix:
platform:
- target: x64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
- target: aarch64
interpreter: 3.9 3.10 3.11 3.12 3.13
interpreter: 3.9 3.10 3.11 3.12 3.13 3.14
steps:
- uses: actions/checkout@v4
with:
Expand Down
16 changes: 14 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,23 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
## [1.1.0] - Unreleased

### Added

- Add opt-in transitive cost closure via `WeightedLevenshtein.transitive_closure()`.
- Support for Python 3.14.
- `__str__` and `__repr__` methods for `WeightedLevenshtein`.

### Changed

- Small runtime improvements in Rust backend.
- Cap `default_substitution_cost` at `default_insertion_cost + default_deletion_cost`, since a substitution can always be expressed as a deletion followed by an insertion.
- Treat empty-sided substitution costs as insertion/deletion aliases.
- Reject non-finite (NaN, infinite) costs during validation.

### Fixed

- Use the minimum cost for conflicting symmetric substitutions.

## [1.0.1] - 2025-09-21

Expand Down
66 changes: 14 additions & 52 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 2 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ocr_stringdist"
version = "1.0.1"
version = "1.1.0"
edition = "2021"
description = "String distances considering OCR errors."
authors = ["Niklas von Moers <niklasvmoers@protonmail.com>"]
Expand All @@ -14,8 +14,5 @@ name = "ocr_stringdist"
crate-type = ["cdylib"]

[dependencies]
pyo3 = { version = "0.24.0", features = ["auto-initialize"] }
pyo3 = { version = "0.28.3", features = ["auto-initialize"] }
rayon = "1.10.0"

[features]
python = []
9 changes: 7 additions & 2 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,20 @@ pytest:
uv run pytest --cov=python/ocr_stringdist python/tests

test:
cargo llvm-cov --features python
#cargo test --features python
cargo llvm-cov
#cargo test

mypy:
uv run mypy .

lint:
cargo clippy
uv run ruff check . --fix

format:
cargo fmt
uv run ruff format

doc:
uv run make -C docs html

Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Contents
getting-started
examples
end_to_end_example
transitive_closure
cost_learning_model
api/index
changelog
37 changes: 37 additions & 0 deletions docs/source/transitive_closure.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
===========================
Transitive Cost Closure
===========================

By default, :class:`~ocr_stringdist.WeightedLevenshtein` only considers the
costs you explicitly provide. If `("6", "G")` costs `0.5` and deleting `"G"`
costs `0.01`, the engine does **not** automatically know that deleting `"6"`
in context costs `0.51`.

:meth:`~ocr_stringdist.WeightedLevenshtein.transitive_closure` returns a new
instance whose cost dictionaries are filled with these effective (transitive)
costs. The closure also materializes mixed chains, e.g. a `del("y") + ins("x")`
sequence becoming an effective `("y", "x")` substitution.

Example
=======

.. code-block:: python

from ocr_stringdist import WeightedLevenshtein

wl = WeightedLevenshtein(
substitution_costs={("6", "G"): 0.5},
deletion_costs={"G": 0.01},
).transitive_closure()

# The chain "6" -> "G" -> ε is now a single effective deletion at 0.51.
print(wl.distance("06", "0")) # 0.51

After closure, :meth:`~ocr_stringdist.WeightedLevenshtein.explain` returns a
single flat operation at the effective cost; the underlying chain is not
preserved.

You may pass ``prune=True`` to the ``transitive_closure`` method to remove
generated substitutions whose costs are already represented by matches,
insertions, deletions, or shorter substitutions. This shrinks the resulting
cost map but is significantly more expensive to compute.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ documentation = "https://niklasvonm.github.io/ocr-stringdist/"


[tool.maturin]
features = ["pyo3/extension-module", "python"]
features = ["pyo3/extension-module"]
python-source = "python"
module-name = "ocr_stringdist._rust_stringdist"

Expand Down
Loading
Loading