Context size problem

[feanor91]

Hello

I have strange problem, I asked for a long task, but even if I add size to the context, start with 16K no at 64K I have this message : HTTP 400: {"error":{"code":400,"message":"request (65561 tokens) exceeds the available context size (65536 tokens), try increasing it","type":"exceed_context_size_error","n_prompt_tokens":65561,"n_ctx":65536}}

I understand I ask a "big" update on the code, but the context sent seems not to be compacted. Perhaps it is I wanted, but, if yes, on a bunch of big code, it will not be usable.

I'm looking on the llam.cpp side if problem is here, but I don't know...Or something I did wrong in OpenFox?

Regards.

Bruno

[feanor91]

Had my answer...Dum me :(

[co-l]

Hello!

This shouldn't happen though, it looks like a bug in OpenFox or in your configuration.

Make sure the context window size is properly configured:

If this value is not 65536 right now for your config, that's the cause of the bug.

Otherwise I need to fix the issue.

[feanor91]

It was why I said I was a dumb guy, I put it to 65536, and it works....for a moment but later, the problem hapoend again. Contexte aboie was 0/65536. Compact buttons do nothing but chat said contexte full. Not sure but I think it occurs just after an auto compact

[co-l]

Yeah since the value was set higher before, the conversation reached a point of no-return, where it's higher than available context size, even for compaction.

To get it back working, you can either try to relaunch your LLM with a few thousand tokens more in the max context size ; or ask an agent to mess with your OpenFox database x) (not recommended), or - what I would do in your situation - put 5$ in DeepSeek API and configure it in OpenFox, use it to compact the session, and then switch back to the local model. That should cost your 0.01$ and then you'll have an easy "bigger brother" model to help you out when things hit limitations.

[feanor91]

Yes, it is an idea....Aniway, I strugle to find a local lmm for dev. Not sure it exist yet. And I have'nt 8000€ to create a sparks cluster 🤣