Skip to content
 
 

Repository files navigation

Changed spacy to support vllm api through openai

Endpoint & Behavior Changes

Aspect Change Modified File
Chat endpoint http://localhost:8001/v1/chat/completions model.py
Completions endpoint http://localhost:8001/v1/completions model.py
Models healthcheck URL http://localhost:8001/v1/models model.py
Additional OSS model Added registry gpt-oss-20b (openai/gpt-oss-20b) registry.py
Additional gemma3 model Added registry gemma3-27b (openai/gemma3-27b) registry.py

Added example for NER using VLLM and gpt-oss

Entity Recognition using VLLM and GPT-oss

Available Models:

- gpt-oss-20b
- gemma3-27b

To install with pip or as a dep through setup.py

setup(
    name="my-project",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "spacy-llm @ git+https://github.com/f-buerckel/spacy-llm.git@main",
    ],
)

Or with Pip:

pip install git+https://github.com/f-buerckel/spacy-llm.git@main

About

🦙 Integrating LLMs into structured NLP pipelines - modified to support vllm

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages