[FEATURE] Add inline code suggestions with selectable AI model (Copilot-like experience)

[lorelore789]

✨ The feature

Add inline code suggestions (ghost text / Copilot-style) similar to GitHub Copilot and other extensions like Continue. The idea is to provide real-time completions while the user types, directly inside the editor.

Also allow selecting which AI model powers these suggestions.

🎯 The problem

Right now the extension is mostly chat-based, which breaks the coding flow and is slower compared to inline suggestions used in tools like Copilot or Continue.

💡 Proposed solution

Implement inline completions that appear as ghost text while typing, similar to Copilot/Continue behavior.

Add a setting to choose the model used for suggestions, for example:

{
  "inlineSuggestions": true,
  "inlineSuggestionsModel": "deepseek-v4-flash"
}

[Wallacy]

I explored this exact path on a separate branch: https://github.com/Wallacy/opencode-copilot-chat/tree/feat/persistent-autocomplete

What was built:

• src/autocomplete/ module with 6 files (types, session, streamer, throttle, provider, index)
• Implemented as a VS Code InlineCompletionItemProvider — the standard API for ghost text
• Uses the same /chat/completions endpoint with stream: true, but with a reduced context window (10 lines before + 300 chars after cursor)
• Minimal system prompt: "Return only the missing code at <CURSOR>."
• 500ms throttle, shared AbortController to prevent cascading cancellations

Why this hasn't been PR'd yet:

• Prohibitive latency (~10s): All OpenCode Go models are reasoning-first. Even with reasoning: false, MiMo V2.5 and DeepSeek V4 Flash still consume 400–700 hidden reasoning tokens before producing any text. This makes it unviable for real-time ghost text.
• Faster models (GLM, Kimi, Qwen) are too expensive to use as an autocomplete engine — the per-request cost would defeat the purpose of a Go subscription.
• No /completions endpoint: OpenCode only exposes /chat/completions. There's no support for echo, prefix, suffix, or logprobs — features that a proper FIM (Fill-in-the-Middle) implementation would need. This forces a "fake chat" prompting approach that's slower and less accurate than a dedicated endpoint.

TL;DR: The implementation works technically, but the intrinsic latency of the OpenCode Go ecosystem (reasoning models + no FIM endpoint) prevents an acceptable autocomplete experience in its current state. If OpenCode adds non-reasoning models or a /completions endpoint, this branch can be picked back up.

I plan to keep looking into this time to time, maybe someone can give some idea on how to speedup for a max of 2s per call... then will be usable.

[Wallacy]

Just to make sure if i was not clear….

The best that i got was 2s per completion for a not userfull , or 5s to a reasonable, and 10s to a good one.

Both models are close to 60tps; If a push a ver small input (less than 1k tokens), and ask for a very small output ( ~128k), i can make then to not reason/think about they will still need 2s to write the response. But to get a more reasonable completion i will need to push little bit more context, then even trying to disable the thinking, both models will reason for 300+ tokens… Then will push at least 5s just writing the thinking, then 2 or more seconds for the output. Most of the time is a 10+ seconds problem.

So we need a model like DeepSeek Coder or Qwen-Coder to push little bit more tps and not waste time on reasoning to be able to push a broader input context over the /completions endpoint

[ltmoerdani]

@Wallacy Your prototype analysis holds up. The OpenCode Go reasoning-first architecture, combined with the missing /completions endpoint, is a real blocker, not something to route around at the adapter level.

I looked at paths that skip waiting for OpenCode to ship FIM. Short version: there's a route open now, but autocomplete goes through a separate provider, not the Go subscription.

Best candidate: DeepSeek FIM Beta (/beta/completions)

• Native prompt + suffix endpoint, no fake-chat workaround needed
• Non-thinking mode only, which eliminates hidden reasoning tokens from the latency equation
• Disk-based context cache on by default: repeated prefixes (imports, signatures) hit cache, sub-2s in practice
• Pricing: $0.0028/1M input on cache hit, $0.28/1M output. Cheap enough for high-frequency autocomplete
• 2500 RPM on deepseek-v4-flash
• One naming trap worth flagging: use deepseek-v4-flash, not deepseek-chat (deprecated 2026-07-24)

Other working options:

• Qwen3-Coder via NIM/DashScope: FIM via <|fim_prefix|>...<|fim_suffix|>...<|fim_middle|> special tokens. Coder-Next is non-thinking by design.
• Ollama local (/v1/completions with suffix): free, offline, 100-300ms with qwen3-coder:1.7b or qwen2.5-coder:1.5b. Probably the right default for users without an API key.

Full writeup, pricing tables, decision matrix, settings schema: docs/issues/26-20260617-inline-completions-fim-research.md (commit c7a56c5).

My read: ship a P0 DeepSeek FIM adapter (BYOK) with Ollama as the zero-config fallback. The chat codepath stays untouched; InlineCompletionItemProvider from your branch carries over directly.

Two things to settle before we start:

1. Should we default to opt-in (autocomplete.enabled = false) to avoid surprise network calls mid-typing?
2. Cerebras adapter for Qwen3-Coder-480B at ~2000 TPS: worth it, or overkill for autocomplete?

[Wallacy]

DeepSeek FIM Beta will need another plan.... i will probably not go for this route.

Ollama is good, but for that the person can just use continue.dev or other extension. I dont know if the #10 is also asking that, but ollama is already directly supported by the VSCode, so i will prefer do nothing until we can do using OpenCode Go / Zen;

MiMo V2.5 and DeepSeek V4 Flash has no reasoning variants... we can already set "off" on the cat extension. The problem is most of the times the model just ignore and do a very low reasoning anyway... maybe is a problem of the upstream opencode endpoints. We already do several paths to handle the Go/Zen endpoints... For example DeepSeek V4 Flash dos support low/medium reasoning effort, but models.dev is not returning that... if we sent that will just work. Maybe there's another way to call both of those models without reasoning and a cheap system prompt that will emulate the FIM via <|fim_prefix|>...<|fim_suffix|>...<|fim_middle|>

[hu3bi]

PTAL: microsoft/vscode#318545

As mentioned here by me: #48 (reply in thread)
we could also join the discord and not just advertise https://github.com/ltmoerdani/opencode-copilot-chat in order to grow the community, but get back to https://opencode.ai/ and request a suitable inline completion model that satisfies the needs outlined by @Wallacy .

Most likely they already have that on their radar since it would be product offering from https://opencode.ai/ that attracts more customers and separate them from their competitors.

[ltmoerdani]

Circling back to this with an update.

I spent time this week reading through arbs-io/github-copilot-llm-gateway (v1.5.0, on the Marketplace). They've shipped inline completions via BYOK, and their architecture is worth studying. The setup: a standalone InlineCompletionItemProvider that calls the server's /v1/completions with prompt (text before cursor) and suffix (text after). It runs alongside Copilot, not through it. Opt-in, 300ms debounce, 3s timeout, 256 max tokens. One detail I liked: they auto-detect servers that reject the suffix parameter, vLLM does this, and silently fall back to prefix-only completions for the rest of the session instead of failing on every keystroke. 330 unit tests, clean separation between the FIM logic and the VS Code surface.

Here's what hasn't moved, though. Their solution works because they target self-hosted servers (vLLM, Ollama, llama.cpp) that natively expose /v1/completions. We don't have that endpoint. OpenCode only gives us /chat/completions, and every Go model is reasoning-first. @Wallacy, your latency numbers from the prototype branch still hold: 5-10s per completion even with thinking forced off, because the models burn 300+ hidden reasoning tokens before any visible text appears. That hasn't changed.

So the picture is lopsided. The piece we thought was the hard part, the VS Code API gap, turns out to be solvable at the extension level. arbs-io proved that. The piece we thought would eventually resolve itself, the OpenCode endpoint, hasn't moved at all.

@Wallacy, in your last comment you said you'd rather wait for OpenCode to ship a /completions endpoint or a non-reasoning model than go BYOK with DeepSeek FIM or Ollama. I've been going back and forth on that stance. I understand the reluctance. If someone wants Ollama autocomplete, continue.dev already does that well. DeepSeek FIM means asking users for a second API key separate from their Go subscription, which feels like it fragments the experience we've built. But arbs-io is getting real traction, 53 stars and active development, by shipping exactly the pattern we researched in doc #25. And the microsoft/vscode#318545 thread keeps growing. There's demand we're watching go to other extensions.

One middle path I keep coming back to: ship the InlineCompletionItemProvider as experimental and opt-in, default it to DeepSeek FIM BYOK (sub-2s with cache hits, $0.0028/1M input, cheap enough that high-frequency autocomplete won't scare anyone), but build the backend behind an interface so that when OpenCode eventually adds a /completions endpoint or ships a non-reasoning model, we swap the backend without touching the provider layer. Not blocked now. Not locked in permanently either.

Full research with the arbs-io architecture breakdown, DeepSeek FIM pricing tables, and the provider abstraction design is at 25-20260617-inline-completions-fim-research.md. I'm updating it with the arbs-io findings this week.

Genuinely want your read on this, @Wallacy. I don't want to ship something that fragments what we have. But I also don't want to sit on this for months waiting on an endpoint that may not come, while users install other extensions to fill the gap.

[hu3bi]

@ltmoerdani I would just wait till the external APIs/deps are ready.
This sounds like implementing a workaround because other stakeholders are not ready (yet).

If you got a bunch of free time and enjoy the experience go for an implementation.

I recommend to just wait till opencode ships deepseek FIM. And if we are lucky microsoft/vscode#318545 will also be ready.
In the meantime we can ask them via discord, ... when/how/if deepseek FIM or inline code completions model will be shipped.

That might save us building a functioning solution which would have to be updated the moment external dependencies are ready. Thus saving time and work.

[Wallacy]

Circling back to this with an update.

Genuinely want your read on this, @Wallacy. I don't want to ship something that fragments what we have. But I also don't want to sit on this for months waiting on an endpoint that may not come, while users install other extensions to fill the gap.

Yes, while my branch was very basic i was convinced that is possible to code the capability layer at the time. But for now, like on the arbs-io case, only using self hosted models or just subscribing directly to the DeepSeek FIM; I dont think that is a big problem to do at least one stop gap for this; OpenCode may never ship any model/endpoint with FIM; And if they ship will be easy to just adopt.