Skip to content

EdgeModel fails to import — LadeSpecDecLLM is undefined in cloud-edge-collaborative-inference-for-llm #568

Description

@Priyanshu6968

What happened:

The EdgeModel module in the cloud-edge-collaborative-inference-for-llm example fails to import. edge_model.py imports LadeSpecDecLLM from the models package at the top of the file:

# testalgorithms/query-routing/edge_model.py:21
from models import HuggingfaceLLM, APIBasedLLM, VllmLLM, EagleSpecDecModel, LadeSpecDecLLM
But LadeSpecDecLLM is not defined or exported anywhere in the repo.

What you expected to happen:

edge_model.py should import cleanly and EdgeModel should load for the supported backends. A missing/optional speculative-decoding backend should not break the whole example at import time.

How to reproduce it (as minimally and precisely as possible):

cd examples/cloud-edge-collaborative-inference-for-llm/testalgorithms/query-routing
python -c "import edge_model"

-> ImportError: cannot import name 'LadeSpecDecLLM' from 'models'

Metadata

Metadata

Assignees

Labels

kind/bugCategorizes issue or PR as related to a bug.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions