Skip to content

[Feature] Bits per bytes metric #60

Description

@ilkerkesen

Implementation of the metric bits per bytes (BPB). BPB will allow us to compare both WELT and tokenizer-based methods in similar terms. This should be implement for both WELT's and tokenizer-based models' training scripts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions