Skip to content

Add VS Code local MLX model example#1425

Open
ahnafyy wants to merge 1 commit into
ml-explore:mainfrom
ahnafyy:add-vscode-mlx-local-model-example
Open

Add VS Code local MLX model example#1425
ahnafyy wants to merge 1 commit into
ml-explore:mainfrom
ahnafyy:add-vscode-mlx-local-model-example

Conversation

@ahnafyy

@ahnafyy ahnafyy commented Jun 9, 2026

Copy link
Copy Markdown

Summary

Add a new vscode_lm example that serves a local MLX coding model through an OpenAI-compatible chat endpoint for use with VS Code language model integration.

Screenshot 2026-06-09 at 4 06 21 PM Screenshot 2026-06-09 at 4 06 35 PM

What changed

  • add a new vscode_lm/server.py example built on mlx-lm
  • expose GET /health, GET /v1/models, and POST /v1/chat/completions
  • add basic prompt-mediated tool-calling support for agent-style requests
  • document the VS Code setup flow in vscode_lm/README.md
  • add a sample chatLanguageModels.example.json config
  • link the new example from the top-level README.md
  • ignore the local virtualenv created for this example in .gitignore

Notes

  • the example defaults to no-auth local usage for simpler setup
  • the default model is mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit to fit local disk and memory constraints better than the 7B model
  • the example was validated end-to-end locally with /health, /v1/models, and a real /v1/chat/completions request

Validation

  • start server from the local Python 3.12 virtualenv
  • verify GET /health
  • verify GET /v1/models
  • verify POST /v1/chat/completions returns a completion without auth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant