Fails to learn?

[rowanhm]

Hi there

I've installed all the requirements, and am trying to train a model. I get am issue when it counts cooccurrences... essentially a worker gets chunks_ but it's an empty list (the data seems to be getting in fine though). Any idea what is going on here?

Any help would be much appreciated!

Full output when I run spvec.py learn --corpus-file bnc_mincount100.txt:

redacted/SPVec/lib/spvec.py:70: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
making vocab...
done (0:00:19)
counting co-occurrences...
redacted/SPVec/lib/spvec.py:70: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
redacted/SPVec/lib/spvec.py:70: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
redacted/SPVec/lib/spvec.py:70: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
redacted/SPVec/lib/spvec.py:70: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  import numpy as np
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "redacted/.pyenv/versions/3.8.16/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "redacted/SPVec/lib/spvec.py", line 619, in coocs_worker
    for line_no in chunks_[chunk_id]:
IndexError: list index out of range
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "redacted/SPVec/spvec.py", line 221, in <module>
    learn(sub_args)
  File "redacted/SPVec/spvec.py", line 89, in learn
    spvec = SPVec(corpus_filename=args.corpus_file,
  File "redacted/SPVec/lib/spvec.py", line 190, in __init__
    self.count_coocs()
  File "redacted/SPVec/lib/spvec.py", line 264, in count_coocs
    coocs_chunk = future.result()
  File "redacted/.pyenv/versions/3.8.16/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "redacted/.pyenv/versions/3.8.16/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
IndexError: list index out of range```