[Feature Request] Ship precomputed distance map / sample training data

[flolep2607]

Great project.

The main friction for new users is the cold-start: learn_from() requires labelled (ocr_string, ground_truth) pairs that most people don't have.
Would it be possible to ship a precomputed substitution_costs JSON trained on a public OCR dataset (e.g. OCR-D, Tesseract UNLV)? Something like:

from ocr_stringdist import WeightedLevenshtein, PRETRAINED_COSTS

wl = WeightedLevenshtein(substitution_costs=PRETRAINED_COSTS)

OR

from ocr_stringdist import WeightedLevenshtein

wl = WeightedLevenshtein() # it used default pretrained costs

Since the model is just a dict[tuple[str, str], float], it'd be trivial to bundle as a JSON file or release asset.
Even a training script + dataset pointer would help.

[NiklasvonM]

Thanks a lot for the feedback!

There are already manually set (i.e., not learned and not fleshed out) default substitution costs:

from ocr_stringdist.default_ocr_distances import ocr_distance_map

These are used as the default costs:

from ocr_stringdist import WeightedLevenshtein

weighted_levenshtein = WeightedLevenshtein() # uses ocr_distance_map

levenshtein = WeightedLevenshtein.unweighted() # normal Levenshtein distance

I am open to changing these to more sensible defaults or adding different sets of learned costs, perhaps using an API such as

WeightedLevenshtein.pretrained(OCRModel.TESSERACT)

As for the training script: Is the end-to-end-example from the documentation what you're looking for? I could definitely add that example to the examples folder as well.

Also, I am open to adding scripts for generating training data (and saving the artefacts). Feel free to contribute!