Skip to content

Language detection #1

Description

@matichorvat

Hey,

I am not sure this is working as intended (https://github.com/aurelienwaite/wmt15/blob/master/preprocessing/src/main/scala/uk/ac/cam/eng/preprocessing/LangDetect.scala#L20)

What will happen is that if languages match, but the probability is not above the threshold, it will return false. Whereas the intended behavior was, I presume, to return false if languages do not match and the probability is above the threshold.

In Python:

if langs[0].lang != expected_lang and langs[0].prob > treshold:
    return False
else:
    return True

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions