Skip to content

[BUG] UnboundLocalError in OllamaLLM.agenerate when Ollama call raises an exception #1175

@belgrano9

Description

@belgrano9

Describe the bug

When using OllamaLLM and the Ollama async call raises any exception (e.g. ReadTimeout with slow or large models), agenerate() crashes with an UnboundLocalError because completion is only assigned inside the try block but referenced unconditionally after it. The pipeline silently fills the batch with None generations instead of raising a clear error.

To reproduce

(code from docs)

from distilabel.models.llms import OllamaLLM
from distilabel.pipeline import Pipeline
from distilabel.steps import make_generator_step
from distilabel.steps.tasks import TextGeneration

dataset = [{"instruction": "What is the difference between supervised and unsupervised learning?"}]

with Pipeline("ollama-local") as pipeline:
    loader = make_generator_step(dataset)
    generate = TextGeneration(llm=OllamaLLM(model="gemma4:e2b"))
    loader >> generate

distiset = pipeline.run(use_cache=False)

Note: any instruction that causes the model to generate a long response and exceed the default 60s timeout will trigger this bug.

Expected behavior

When the Ollama call fails or times out, the step should log the warning and return None for that generation gracefully — not crash with UnboundLocalError.

Screenshots

No response

Environment

  • Distilabel Version: 1.5.3
  • Python Version: 3.12

Additional context

Root cause is in distilabel/models/llms/ollama.py in agenerate().
completion is only assigned inside the try block but used unconditionally after it:

return prepare_output([text], **self._get_llm_statistics(completion))

The fix is to initialize completion = None before the try block and guard the call:

    completion = None
    try:
        ...
    except Exception as e:
        ...

    stats = (
        self._get_llm_statistics(completion)
        if completion is not None
        else {"input_tokens": [None], "output_tokens": [None]}
    )
    return prepare_output([text], **stats)

I am happy to open a PR with this fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions