Skip to content

perf(context): skip O(N) calculateTokenBreakdown when tracer is disabled#26489

Open
rushikeshsakharleofficial wants to merge 1 commit intogoogle-gemini:mainfrom
rushikeshsakharleofficial:perf/fix-tracer-eager-eval
Open

perf(context): skip O(N) calculateTokenBreakdown when tracer is disabled#26489
rushikeshsakharleofficial wants to merge 1 commit intogoogle-gemini:mainfrom
rushikeshsakharleofficial:perf/fix-tracer-eager-eval

Conversation

@rushikeshsakharleofficial
Copy link
Copy Markdown

Summary

  • Bug: render.ts eagerly evaluated env.tokenCalculator.calculateTokenBreakdown(nodes) as an argument to tracer.logEvent(...) on every API turn, even though logEvent immediately returns if (!this.enabled). JS evaluates arguments before the function call, so the O(N×parts) computation always ran.
  • Fix: Added get isEnabled(): boolean to ContextTracer and guarded the two expensive calls behind if (tracer.isEnabled):
    • calculateTokenBreakdown(nodes) — iterates all nodes, calls estimateTokenCountSync per part for categorization
    • Object.fromEntries(protectionReasons) — O(N) map copy
  • Both calls now execute only when GEMINI_CONTEXT_TRACE_DIR is set (tracing opt-in, off by default).

Impact: Applies to users with contextManagement.enabled: true. The generalistProfile budget is configured, so the budget branch of render() fires on every turn for those users.

Test plan

  • Existing context pipeline tests pass (no behavior change when tracing disabled)
  • When GEMINI_CONTEXT_TRACE_DIR is set, trace logs still contain Estimation Calibration and Protection Audit events
  • Nightly perf CI should show improvement for long-conversation scenarios with context management enabled

@rushikeshsakharleofficial rushikeshsakharleofficial requested a review from a team as a code owner May 5, 2026 06:13
@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 5, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request optimizes the context rendering pipeline by ensuring that resource-intensive diagnostic operations are only performed when tracing is explicitly enabled. By checking the tracer's status before evaluating arguments for log events, the application avoids unnecessary computations on every API turn, leading to improved performance for users with context management enabled.

Highlights

  • Performance Optimization: Added a guard condition to prevent expensive O(N) token breakdown calculations and map copies when the tracer is disabled.
  • API Update: Introduced an isEnabled getter to the ContextTracer class to allow efficient checking of the tracing state before executing logging logic.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces a check for the tracer's enabled state before logging events during the rendering process in packages/core/src/context/graph/render.ts. To support this, a new isEnabled getter was added to the ContextTracer class in packages/core/src/context/tracer.ts. I have no feedback to provide.

@gemini-cli gemini-cli Bot added the status/need-issue Pull requests that need to have an associated issue. label May 5, 2026
JS evaluates function arguments before calling the function, so
calculateTokenBreakdown(nodes) was running on every single API turn
even though tracer.logEvent immediately returned when tracing is
disabled (the common case).

Add get isEnabled() to ContextTracer and guard the two expensive
tracer.logEvent calls in render.ts:
- calculateTokenBreakdown(nodes) — O(N*parts) loop with redundant
  estimateTokenCountSync calls per part
- Object.fromEntries(protectionReasons) — O(N) map copy

Both calls now run only when GEMINI_CONTEXT_TRACE_DIR is set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status/need-issue Pull requests that need to have an associated issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant