What happened:
The EdgeModel module in the cloud-edge-collaborative-inference-for-llm example fails to import. edge_model.py imports LadeSpecDecLLM from the models package at the top of the file:
# testalgorithms/query-routing/edge_model.py:21
from models import HuggingfaceLLM, APIBasedLLM, VllmLLM, EagleSpecDecModel, LadeSpecDecLLM
But LadeSpecDecLLM is not defined or exported anywhere in the repo.
What you expected to happen:
edge_model.py should import cleanly and EdgeModel should load for the supported backends. A missing/optional speculative-decoding backend should not break the whole example at import time.
How to reproduce it (as minimally and precisely as possible):
cd examples/cloud-edge-collaborative-inference-for-llm/testalgorithms/query-routing
python -c "import edge_model"
-> ImportError: cannot import name 'LadeSpecDecLLM' from 'models'
What happened:
The
EdgeModelmodule in thecloud-edge-collaborative-inference-for-llmexample fails to import.edge_model.pyimportsLadeSpecDecLLMfrom themodelspackage at the top of the file:What you expected to happen:
edge_model.py should import cleanly and EdgeModel should load for the supported backends. A missing/optional speculative-decoding backend should not break the whole example at import time.
How to reproduce it (as minimally and precisely as possible):
cd examples/cloud-edge-collaborative-inference-for-llm/testalgorithms/query-routing
python -c "import edge_model"
-> ImportError: cannot import name 'LadeSpecDecLLM' from 'models'