[Bug] Actor unavailable error

[zhusq20]

Bug Description

Traceback (most recent call last):
2026-03-18 05:25:02

2855e530…vdumckda
File "/root/slime_siqi/train.py", line 100, in
2026-03-18 05:25:02

2855e530…vdumckda
train(args)
2026-03-18 05:25:02

2855e530…vdumckda
File "/root/slime_siqi/train.py", line 69, in train
2026-03-18 05:25:02

2855e530…vdumckda
rollout_data_ref = ray.get(rollout_manager.generate.remote(rollout_id))
2026-03-18 05:25:02

2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:02

2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
2026-03-18 05:25:02

2855e530…vdumckda
return fn(*args, **kwargs)
2026-03-18 05:25:02

2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03

2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
2026-03-18 05:25:03

2855e530…vdumckda
return func(*args, **kwargs)
2026-03-18 05:25:03

2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03

2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 2981, in get
2026-03-18 05:25:03

2855e530…vdumckda
values, debugger_breakpoint = worker.get_objects(
2026-03-18 05:25:03

2855e530…vdumckda
^^^^^^^^^^^^^^^^^^^
2026-03-18 05:25:03

2855e530…vdumckda
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 1014, in get_objects
2026-03-18 05:25:03

2855e530…vdumckda
raise value
2026-03-18 05:25:03

2855e530…vdumckda
ray.exceptions.ActorUnavailableError: The actor ed2ff6eb7e211cc420c657b002000000 is unavailable: The actor is temporarily unavailable: RpcError: RPC error: Socket closed rpc_code: 14. The task may or may not have been executed on the actor.

Steps to Reproduce

I ran qwen3-1.7b training on math dataset, in step 9, this error occurrs

Expected Behavior

The program shouldn't collapse

Actual Behavior

the program exit

Environment

• slime version:
• Python version:
• PyTorch version:
• CUDA/ROCm version:
• GPU type and count:
• OS:
• SGLang version (if relevant):
• Megatron-LM version (if relevant):

Logs

Additional Context

No response

Pre-submission Checklist

• I have read the CONTRIBUTING.md and understand the collaboration scope.
• I have read the documentation and my issue is not addressed there.
• I have searched for existing issues and this is not a duplicate.
• I have provided a minimal, reproducible example.

[Ariya12138]

the same question

[zengxingchen]

Try increasing ulimit -n. Hope that helps.

[ZhexiLu]

the same question

[qiuuuuu622]

**I have the same question，Did you solve the issue? **
(MegatronTrainRayActor pid=73409) [2026-05-28 08:45:25] actor.py:173 - after destroy process groups
Traceback (most recent call last):
File "/root/slime/train.py", line 107, in
train(args)
File "/root/slime/train.py", line 24, in train
actor_model, critic_model = create_training_models(args, pgs, rollout_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/slime/slime/ray/placement_group.py", line 154, in create_training_models
actor_start_rollout_ids = ray.get(
^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/ray/_private/client_mode_hook.py", line 107, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 2980, in get
values, debugger_breakpoint = worker.get_objects(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/ray/_private/worker.py", line 1025, in get_objects
raise value
ray.exceptions.ActorUnavailableError: The actor 11fdd72ac83639b1f2bbcc2102000000 is unavailable: The actor is temporarily unavailable: RpcError: RPC error: Socket closed rpc_code: 14. The task may or may not have been executed on the actor.

[Qi2yU]

same issue，cao！

[Qi2yU]

你好，你有在报错前打印日志http_utils.py:185 - Error: ConnectTimeout: ConnectTimeout(''), retrying... (attempt 1/60, url=http://10.144.205.100:4403/generate, response=None)类似的吗