Skip to content

Endless warning message when use imap #269

@BeiYaaa

Description

@BeiYaaa

Hi there,

I'm running below script on Google Vertex AI Jupyter Notebook.

import pathos as pa
with PPool(pa.helpers.cpu_count()) as p:
                    # make sure the pool is started
                    try:
                        p.restart()
                    except:
                        pass
                    
                    record_df_ls = []
                    record_df_ls = p.imap(self._fit_n_evaluate, param_grid)
                    
                    # close the process pool
                    p.close()
                    # wait for all tasks to complete
                    p.join()

I was able to run it successfully before. However, today I notice that there are warning messages printed on the screen and I couldn't kill the kernel. Below is part of the recurring and endless output.
File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 286, in loads return load(file, ignore, **kwds) Process ForkPoolWorker-9: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 272, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 419, in load obj = StockUnpickler.load(self) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 942, in _create_namedtuple t = collections.namedtuple(name, fieldnames, defaults=defaults, module=modulename) Process ForkPoolWorker-10: File "/opt/conda/lib/python3.7/collections/__init__.py", line 370, in namedtuple raise ValueError('Field names cannot start with an underscore: ' Traceback (most recent call last): ValueError: Field names cannot start with an underscore: '_1' File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) Process ForkPoolWorker-11: File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res)

I noticed that it says ValueError can't start with an underscore. But I don't have file starting with an underscore in the fit function. This confuses me a lot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions