Hey,
I am not sure this is working as intended (https://github.com/aurelienwaite/wmt15/blob/master/preprocessing/src/main/scala/uk/ac/cam/eng/preprocessing/LangDetect.scala#L20)
What will happen is that if languages match, but the probability is not above the threshold, it will return false. Whereas the intended behavior was, I presume, to return false if languages do not match and the probability is above the threshold.
In Python:
if langs[0].lang != expected_lang and langs[0].prob > treshold:
return False
else:
return True
Hey,
I am not sure this is working as intended (https://github.com/aurelienwaite/wmt15/blob/master/preprocessing/src/main/scala/uk/ac/cam/eng/preprocessing/LangDetect.scala#L20)
What will happen is that if languages match, but the probability is not above the threshold, it will return false. Whereas the intended behavior was, I presume, to return false if languages do not match and the probability is above the threshold.
In Python: