fix(context): align isItemTooBig budget with the message compiler

[MukundaKatta]

Description

@file rejects items that @CurrentFile accepts. Core.isItemTooBig reserves the full maxTokens (contextLength - maxTokens) when deciding whether a context item fits, but compileChatMessages only reserves min(MIN_RESPONSE_TOKENS, maxTokens) plus the counting safety buffer. On small context windows, or on models with no known completion limit where maxTokens defaults to 4096, a single file looks too big even though the compiler would include it. @CurrentFile skips isItemTooBig entirely, which is why it was unaffected.

This pulls the compiler's budget into getAvailableInputTokens and uses it in both places so the check matches what actually gets sent.

Fixes #12165

Checklist

• I've read the contributing guide
• The relevant docs, if any, have been updated or created
• The relevant tests, if any, have been updated or created

Tests

Added a getAvailableInputTokens unit test block in core/llm/countTokens.test.ts: one case for a small window with the default 4096 maxTokens, one for when maxTokens is below the minimum response allowance. The core/llm/countTokens Jest suite passes.

---

Summary by cubic

Aligns context-size checks with the message compiler so @file no longer gets rejected when the compiler would include it. Uses the compiler’s input budget (safety buffer + minimum response allowance) instead of reserving the full maxTokens, fixing false “too big” results on small windows and models with unknown completion limits.

• Bug Fixes
  • Added getAvailableInputTokens(contextLength, maxTokens) to mirror compiler budgeting (safety buffer + min(1000, maxTokens)).
  • Updated Core.isItemTooBig to use it; @file now matches @CurrentFile.
  • Added unit tests for small-window and low-maxTokens scenarios.

^{Written for commit 6e5827b. Summary will update on new commits.}

Review in cubic

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[cubic-dev-ai]

No issues found across 3 files

_{Re-trigger cubic}