Skip to content

feat(sdk): add ignore_eos to force full-length decode in geniex-bench#1162

Draft
RemiliaForever (RemiliaForever) wants to merge 1 commit into
mainfrom
feat/bench-ignore-eos
Draft

feat(sdk): add ignore_eos to force full-length decode in geniex-bench#1162
RemiliaForever (RemiliaForever) wants to merge 1 commit into
mainfrom
feat/bench-ignore-eos

Conversation

@RemiliaForever

Copy link
Copy Markdown
Contributor

Summary

Adds an --ignore-eos option to geniex-bench that keeps generation running
past EOS/EOG so decode always produces the full --n-gen tokens. This makes
decode-speed numbers comparable across models that would otherwise stop early,
mirroring llama-bench --ignore-eos.

The whole behavioral change is one guard in the llama_cpp decode loop; the rest
is plumbing a single bool through the one public conduit that reaches it.

Changes

  • sdk/include/geniex.h — append bool ignore_eos to geniex_GenerationConfig (field appended at the end, so existing offsets are unchanged).
  • sdk/plugins/llama_cpp/src/{llm,vlm}.cpp — guard the llama_vocab_is_eog break with !cfg.ignore_eos. When set, the EOG token falls through, the loop runs to max_tokens, and stop_reason becomes "length" via the existing post-loop path. Both the text-LLM and VLM decode loops are covered.
  • sdk/benchmark/benchmark.c--ignore-eos flag → options_tfill_gen_config, plus help text.
  • FFI mirrors (required by CONTRIBUTING.md §4 — public header changed):
    • Python ctypes _types.py — append ('ignore_eos', c_bool) (correctness-critical: the SDK copies *config by value, so a shorter Structure would read past the allocation).
    • Go common.goIgnoreEos bool + wire into toCPtr().
    • Android JNI jniutils.cpp + Kotlin GenerationConfig.kt — read ignoreEos.

qairt

qairt drives EOS inside the third-party genie engine
(third-party/geniex-qairt/.../llm_model.cpp), which has no ignore-eos seam and
must not be modified. The flag is therefore a documented no-op on qairt — the
plugin simply ignores the new field, consistent with how it already ignores
--ngl / --n-ctx / --stop.

Test plan

  • geniex.h field compiles; a C consumer can set g.ignore_eos = true.
  • benchmark.c passes gcc -fsyntax-only.
  • Python _types.py loads; geniex_GenerationConfig._fields_ ends with ignore_eos and set/get round-trips.
  • bindings/go/common.go is gofmt-clean.
  • clang-format, ruff check, ruff format --check clean on touched files (pre-existing unrelated format drift in llm.cpp left untouched).
  • Full SDK bridge build + on-device bench run (--ignore-eos yields gen == n_gen and stop_reason == "length") — pending, needs the toolchain container / device.

geniex-bench --ignore-eos keeps generation running past EOS/EOG so decode
always produces --n-gen tokens, making decode-speed numbers comparable
across models. Adds a bool ignore_eos to geniex_GenerationConfig (guarded
in the llama_cpp llm/vlm decode loops) and updates the Python, Go, and
Android FFI surfaces. qairt drives EOS inside the third-party genie engine
and cannot honor it, so the flag is a documented no-op there.

Signed-off-by: RemiliaForever <remilia@koumakan.cc>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant