Skip to content

bug: TREC 2020 qrel_binary.json, score 1 should be treated as negative instead of positive #63

@zhiyuanpeng

Description

@zhiyuanpeng

Hi,

For TREC, 0,1 are negative, 2,3 are positive. However, in qrel_binary.json, 1 is mapped to positive (1). For instance, in qrel.json:

{"23849": {"1020327": 2, "1034183": 3, "1120730": 0, "1139571": 1

in qrel_binary.json:

{"23849": {"1020327": 1, "1034183": 1, "1120730": 0, "1139571": 1,

doc 1139571 should be mapped to 0 in qrel_binary.json.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions