Skip to content

Add the search metrics for the quick setup onboarding experiment according to the experiment design#8658

Open
catalinradoiu wants to merge 2 commits into
developfrom
feature/cradoiu/add-quick-setup-search-metrics
Open

Add the search metrics for the quick setup onboarding experiment according to the experiment design#8658
catalinradoiu wants to merge 2 commits into
developfrom
feature/cradoiu/add-quick-setup-search-metrics

Conversation

@catalinradoiu
Copy link
Copy Markdown
Contributor

@catalinradoiu catalinradoiu commented May 21, 2026

Task/Issue URL: https://app.asana.com/1/137249556945/project/1211724162604201/task/1214753739886022

Description

Adds the search-count metrics for the quick-setup onboarding experiment (onboardingQuickSetupExperimentMay26).

The plugin lives alongside the experiment's feature flag in com.duckduckgo.app.onboardingquicksetup and is wired as an AtbLifecyclePlugin that fires on onSearchRetentionAtbRefreshed. It injects OnboardingQuickSetupToggles directly (not via inventory-by-name) so removing the flag at experiment end will surface this code as a compile-time usage to clean up.

Metrics emitted per search-refresh ATB:

  • search value 1 on a single D1..D3 conversion window (mirrors the experiment's "first search" milestone).
  • search values 2..10 on daily D0..D7 conversion windows.

Both gates are required: the experiment toggle must be isEnabled() and isEnrolled() — otherwise no metrics are emitted.

Steps to test this PR

Quick-setup search metrics

  • On a build where the onboardingQuickSetupExperimentMay26 toggle is disabled or the user is not enrolled, refresh the search retention ATB and confirm no search_* experiment pixels are sent for this toggle.
  • Enroll a user into the experiment (either cohort) and enable the toggle remotely, then trigger search retention ATB refreshes and confirm search pixels are emitted with values 1..10 and the documented conversion windows.
  • Verify the unit tests in OnboardingQuickSetupSearchMetricsAtbLifecyclePluginTest pass (./gradlew :app:testPlayDebugUnitTest --tests "com.duckduckgo.app.onboardingquicksetup.OnboardingQuickSetupSearchMetricsAtbLifecyclePluginTest").

UI changes

Before After
No UI changes No UI changes

Note

Medium Risk
Adds a new AtbLifecyclePlugin that emits multiple experiment metrics on each search-retention ATB refresh; while gated by enrollment/enabled state, it changes analytics/pixel behavior and could increase event volume if misconfigured.

Overview
Adds a new OnboardingQuickSetupSearchMetricsAtbLifecyclePlugin wired via multibinding to send experiment search metrics when onSearchRetentionAtbRefreshed fires, but only when onboardingQuickSetupExperimentMay26 is both enabled and the user is enrolled.

When active, it emits COUNT_WHEN_IN_WINDOW pixels for search values 1..10, using a single D1..D3 window for value 1 and daily D0..D7 conversion windows for values 2..10. Includes unit tests validating gating, metric contents, and that no pixels are sent for other ATB refresh callbacks.

Reviewed by Cursor Bugbot for commit 3b44237. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Contributor Author

Move the search-count pixels for the quick-setup onboarding experiment
out of feature-toggles-impl into the app module, alongside the
OnboardingQuickSetupToggles feature flag. The plugin now injects the
toggle directly instead of looking it up by name in the inventory, so
removing the flag at experiment end will surface this code as a
compile-time usage to clean up.

The metrics are now only emitted when the experiment toggle is enabled
and the user is enrolled. Adds value=6 to the daily-windowed counts
(was missing from the 2..10 sequence).

Task/Issue URL: https://app.asana.com/1/137249556945/project/1211724162604201/task/1214753739886022

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@catalinradoiu catalinradoiu marked this pull request as ready for review May 22, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant