Skip to content

fix(source-google-analytics-data-api): scope stream naming mapping to top-level name - #83343

Open
devin-ai-integration[bot] wants to merge 3 commits into
masterfrom
devin/1785783022-ga4-name-field-path
Open

fix(source-google-analytics-data-api): scope stream naming mapping to top-level name#83343
devin-ai-integration[bot] wants to merge 3 commits into
masterfrom
devin/1785783022-ga4-name-field-path

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

What

Related to https://github.com/airbytehq/oncall/issues/13231:

The dynamic-stream naming ComponentMappingDefinition in manifest.yaml used field_path: ["**", "name"]. The CDK passes field_path segments straight to dpath.set (airbyte_cdk/sources/declarative/resolvers/config_components_resolver.py), and dpath treats ** as a recursive glob — so this mapping sets every name key at any depth in the resolved stream config, not just the stream name.

Today that happens to be harmless: at the point this mapping runs, the only name key in the template is the top-level stream name (request_body_json.metrics/dimensions are still empty lists and are populated by later mappings). So the current correctness depends on mapping order and on the template containing no other name key. Adding any nested name: to the stream template, or reordering components_mapping, would silently corrupt request bodies. Injecting a single nested name into the template reproduces the clobber: retriever.requester.request_parameters.name comes back as daily_active_usersProperty222 instead of the injected value.

This PR scopes the mapping to the top-level stream name. Naming semantics are unchanged — the positional Property<id> suffixing behavior is preserved exactly.

How

One-line manifest change:

 - type: ComponentMappingDefinition
-  field_path: ["**", "name"]
+  field_path: ["name"]
   value: "{{ components_values['name'] + 'Property' + ... }}"

The interpolated value is untouched.

Equivalence proof

The resolved dynamic_streams output was generated with both the old and new field_path and diffed as canonical JSON — full resolved stream configs, not just names (including retriever.requester.request_body_json metrics/dimensions, schema_loader paths, transformations, paginator, and incremental sync). Three configs were compared: property_ids: ["111"], property_ids: ["111","222","333"], and a config with a custom_reports_array (cohort report) entry.

=== CONFIG 1 ===  property_ids: ["111"]
OLD STREAM NAMES: ['daily_active_users', 'weekly_active_users', ..., 'tech_os_with_version_report']  (57 streams)
NEW STREAM NAMES: ['daily_active_users', 'weekly_active_users', ..., 'tech_os_with_version_report']  (57 streams)
FULL CONFIG DIFF (old vs new):
<empty>

=== CONFIG 2 ===  property_ids: ["111", "222", "333"]
OLD STREAM NAMES: ['daily_active_users', 'daily_active_usersProperty222', 'daily_active_usersProperty333', ...]  (171 streams)
NEW STREAM NAMES: ['daily_active_users', 'daily_active_usersProperty222', 'daily_active_usersProperty333', ...]  (171 streams)
FULL CONFIG DIFF (old vs new):
<empty>

=== CONFIG 3 ===  property_ids: ["111"] + custom_reports_array (cohort_report)
OLD STREAM NAMES: ['cohort_report', 'daily_active_users', ...]  (58 streams)
NEW STREAM NAMES: ['cohort_report', 'daily_active_users', ...]  (58 streams)
FULL CONFIG DIFF (old vs new):
<empty>

And the behavior that actually differs — with a nested name present in the template:

field_path=['**', 'name'] -> nested name = 'daily_active_users'   (clobbered)
field_path=['name']       -> nested name = 'keep-me'              (preserved)
stream names identical in both cases: ['daily_active_users', 'daily_active_usersProperty222']

Declarative-First Evaluation

This is a declarative-only fix: a single field_path correction on an existing ComponentMappingDefinition. No custom Python component was added or modified.

Review guide

  1. airbyte-integrations/connectors/source-google-analytics-data-api/manifest.yaml — the one-line field_path change.
  2. airbyte-integrations/connectors/source-google-analytics-data-api/unit_tests/test_components.pytest_dynamic_stream_naming_preserves_nested_name resolves the real manifest with an injected nested name and asserts it survives while stream names stay positional. Verified this test fails on the old ["**", "name"] path and passes with the fix. Full suite: 16 passed.
  3. metadata.yaml + docs/integrations/sources/google-analytics-data-api.md — patch bump to 2.9.46 and changelog row.

Serhii Lazebnyi (@lazebnyi) — you introduced this mapping in #61550; flagging you for the naming semantics, which this PR deliberately leaves unchanged.

User Impact

None. The resolved stream configs are byte-identical for all tested configurations, so no stream is renamed and no request body changes. This removes an order-dependent footgun so future edits to the stream template can safely introduce nested name keys.

Out of scope (tracked on the oncall issue)

  • Switching positional Property<id> suffixing to ID-stable suffixing — that renames existing streams and needs the breaking-change process plus a product decision.
  • Duplicate stream names when a custom report name collides with a built-in report name (e.g. property_ids: ["111"] plus a custom report named pages yields two streams named pages). Real defect, but the remedy is config validation, which is not cleanly expressible in the declarative resolver.
  • The stream fan-out redesign discussed in https://github.com/airbytehq/oncall/issues/13116.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Requested via the /ai-fix workflow on https://github.com/airbytehq/oncall/issues/13231.
Devin session

…pping to top-level name

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
      • Bump types: patch (default), minor, major, major_rc, rc, promote.
      • The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
      • The promote type strips the RC suffix to finalize a release.
      • Example: /bump-version type=rc or /bump-version type=minor
    • /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 📝 AI Documentation:
    • /ai-docs-review - AI-powered documentation review for PRs with connector changes.
    • /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
    • /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
      • Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
      • Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

Co-Authored-By: bot_apk <apk@cognition.ai>
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Note

Autopilot progressive rollouts are not enabled for the following modified connector(s):

  • source-google-analytics-data-api

This is a courtesy heads-up only — it does not block merge or fail any check.
To enable automatic progressive rollouts for the connector(s) above, comment
/enable-autopilot-rollouts on this PR. This sets defaultRolloutMode: autopilot
and enableProgressiveRollout: true in each connector's metadata.yaml,
preserving any existing autopilotConfig.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Deploy preview for airbyte-docs ready!

Project:airbyte-docs
Status: ✅  Deploy successful!
Preview URL:https://airbyte-docs-9xg1rw7nf-airbyte-growth.vercel.app
Latest Commit:c170a91

Deployed with vercel-action

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

source-google-analytics-data-api Connector Test Results

21 tests   18 ✅  15s ⏱️
 2 suites   3 💤
 2 files     0 ❌

Results for commit c170a91.

♻️ This comment has been updated with latest results.

Co-Authored-By: bot_apk <apk@cognition.ai>
@airbyte-support-bot

Copy link
Copy Markdown
Contributor

↪️ Triggering /ai-prove-fix per Hands-Free AI Triage Project triage next step.

Reason: Draft /ai-fix PR with green CI and no prove-fix run on record; validating the scoped stream-naming change against live data is the next pipeline gate.

Devin session

@octavia-bot

octavia-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

🔍 AI Prove Fix session starting... Running readiness checks and testing against customer connections. View playbook

Devin AI session created successfully!

@airbyte-support-bot

Airbyte Support Bot (airbyte-support-bot) commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

✅ Fix Proven — source-google-analytics-data-api 2.9.46

The change narrows the dynamic-stream naming mapping from field_path: ["**", "name"] to ["name"]. dpath.set treats ** as a recursive glob, so the old path wrote the computed stream name into every name key at any depth of the resolved stream config. Validated on two axes: the fix removes that over-write, and it changes nothing that is reachable today.

Check Result
Regression test (comparison mode, all streams) SPEC · ✅ CHECK · ✅ DISCOVER · ✅ READ
SPEC / DISCOVER output, target vs control byte-identical (59 streams, identical name set)
READ records, target vs control byte-identical — all 1799 records, identical per-stream counts
New unit test pins the fix ✅ fails on the old field_path, passes on the new one
Resolved-config equivalence, 3 configs ✅ identical, including the 171-stream / 114-suffixed-name multi-property case
🔬 Evidence

1. Live regression testworkflow run. Target = connector built from PR head c170a91 (2.9.46, SDM base 7.23.8); control = published 2.9.45. All four commands passed, no both_failed, no infrastructure annotation. Going beyond the summary table into the artifacts: spec.jsonl and catalog.jsonl are byte-identical after canonical sort, and every one of the 1799 READ records matches between versions, with identical RECORD/STATE/LOG/TRACE counts and identical per-stream record counts across all 53 streams that returned data.

2. The added unit test actually pins the fix (mutation test). Locally, the connector unit suite is 16 passed on the PR head. Reverting only the manifest line back to ["**", "name"] turns that into 1 failed, 15 passed, with test_dynamic_stream_naming_preserves_nested_name failing on its nested-name assertion. So the test fails on the old behavior and passes on the new — it is not a tautological test. It also asserts the first two positional names, so it covers the multi-property branch.

3. Independent equivalence check. Resolving the real manifest.yaml under both field paths and diffing the entire resolved dynamic-stream config list (not just names) reproduces the PR's claim:

Config Streams Suffixed names Resolved configs identical?
1 property 57 0
3 properties 171 114
1 property + custom report 58 0

Positional naming (daily_active_users, daily_active_usersProperty222, …) is preserved exactly.

4. Was the change exercised? Yes. Every stream's name is written through this mapping, so all 59 discovered streams in the regression run went through the corrected path — and the differing behavior itself is covered by (2). The differing behavior is not reachable from any customer config: it requires a second, nested name key in the stream template (or a reordered components_mapping), which is a property of the manifest, not of user config. That is why this is validated by regression equivalence plus the unit test rather than by pinning a live connection — a pinned sync would reproduce the byte-identical result already shown, with customer risk and no extra signal. Testing detail is recorded in the internal oncall issue.

✅ Pre-flight checks
  • Viability — one-line declarative correction plus a regression test, patch bump 2.9.45 → 2.9.46, changelog row. It addresses the reported defect directly.
  • Safety — manifest + unit test + metadata + docs only. No new network calls, no credential handling, no new dependencies, no obfuscation.
  • Breaking change: none. No spec, schema, primary-key, cursor, or state-format change; no stream added, removed, or renamed. The interpolated value producing the Property<id> suffix is untouched.
  • Reversibility — trivially revertible; downgrading to 2.9.45 resolves identical stream configs, with no config or state migration.

Notes

  • The positional Property<id> naming this mapping produces stays unchanged by design — this PR removes an order-dependent over-write, it does not change naming semantics. The ID-stable renaming discussed on the linked issue remains a separate, breaking change.
  • Pre-release 2.9.46-preview.c170a91 is published for downstream use. The first publish attempt failed on a transient GitHub 429 Too Many Requests and was re-triggered; no evidence above depends on that image, since the regression test builds the connector from source.
  • Recommended next step: /ai-canary-prerelease before promoting to a normal release.

Devin session

@airbyte-support-bot Airbyte Support Bot (airbyte-support-bot) added the hyd-prove Hydra: ai-prove-fix stage has run label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Pre-release Connector Publish Started

Publishing pre-release build for connector source-google-analytics-data-api.
PR: #83343

Pre-release versions will be tagged as {version}-preview.c170a91
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish FAILED for source-google-analytics-data-api.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Pre-release Connector Publish Started

Publishing pre-release build for connector source-google-analytics-data-api.
PR: #83343

Pre-release versions will be tagged as {version}-preview.c170a91
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-google-analytics-data-api:2.9.46-preview.c170a91

Docker Hub: https://hub.docker.com/layers/airbyte/source-google-analytics-data-api/2.9.46-preview.c170a91

Registry JSON:

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review August 4, 2026 13:10
@airbyte-support-bot

Copy link
Copy Markdown
Contributor

↪️ Triggering /ai-review per Hands-Free AI Triage Project triage next step.

Reason: /ai-prove-fix returned ✅ Fix Proven on the current HEAD, so I marked the PR ready for review; an AI review is the next pipeline gate.

Devin session

@octavia-bot

octavia-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

AI PR Review starting...

Reviewing PR for connector safety and quality.
View playbook

Devin AI session created successfully!

@airbyte-support-bot

Airbyte Support Bot (airbyte-support-bot) commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

🛡️ AI PR Review Report

🟢 Review Action: APPROVED

All 12 gates passed.

🔶 Risk Level: 2/5

Single-connector declarative correction with regression-proven identical resolved stream configs; no existing behavior changed.


📋 PR Details

Connector(s): source-google-analytics-data-api
PR: #83343
HEAD SHA: c170a911436d6e431b6489ab689154e08f33ecc9
Session: https://app.devin.ai/sessions/9f76a263fb424bc7b32e195d9a5c7f86

🔍 Gate Evaluation Details

Gate-by-Gate Analysis

Gate Status Enforced? Details
PR Hygiene PASS Yes PR body is ~5.4k chars of substantive content (What/How/Review guide/User Impact); changelog row 2.9.46 added to docs/integrations/sources/google-analytics-data-api.md; no unresolved human review comments.
Code Hygiene PASS WARNING Source change (manifest.yaml) is accompanied by a test change (unit_tests/test_components.py).
Test Coverage PASS Yes Behavioral-change indicator: title starts with fix(...). New test test_dynamic_stream_naming_preserves_nested_name with new assertions added.
Code Security PASS Yes No auth/credential/secret file paths touched. manifest.yaml diff hunk is a single field_path line; metadata.yaml diff hunk is only dockerImageTag — no allowedHosts/dockerRepository/authenticator keywords in the hunks.
Per-Record Performance PASS WARNING No changes to record-processing paths (record_selector, transformations, source/streams code); the change affects one-time dynamic-stream resolution.
Breaking Dependencies PASS WARNING No pyproject.toml, poetry.lock, or Gradle dependency changes.
Backwards Compatibility PASS Yes (enforced) No spec, schema, format, stream-name, primary-key, or cursor change. metadata.yaml change is a patch bump only. /ai-prove-fix reports byte-identical SPEC/DISCOVER output (59 streams, identical name set) target vs. control.
Forwards Compatibility PASS Warning (elevates Risk Level) Diff hunks contain no state/cursor/pagination/partition-router/transformation keywords; rollback to 2.9.45 resolves identical stream configs with no state migration.
Behavioral Changes PASS Warning (elevates Risk Level) No rate-limit, retry/backoff, timeout, error-handler, or page-size keywords in the diff hunks.
Out-of-Scope Changes PASS Skip All 4 changed files are under airbyte-integrations/connectors/source-google-analytics-data-api/ or docs/.
CI Checks PASS Yes Core checks green on c170a91: Test source-google-analytics-data-api Connector, Lint source-google-analytics-data-api Connector, Build and Verify Artifacts (source-google-analytics-data-api), Connector CI Checks Summary, Format Check, Check Changelog Updated.
Live / E2E Tests PASS Yes Validation required (bug fix + stream discovery/catalog change). /ai-prove-fix posted ✅ Fix Proven on this HEAD: regression test in comparison mode passed SPEC/CHECK/DISCOVER/READ, with byte-identical catalogs and all 1799 READ records matching control.

Change surface: manifest.yaml narrows one ComponentMappingDefinition from field_path: ["**", "name"] to ["name"] under dynamic_streams. Because dpath treats ** as a recursive glob, the old path wrote the computed stream name into every name key at any depth of the resolved stream config; the new path targets only the top-level stream name. The interpolated value (positional Property<id> suffixing) is untouched, so naming semantics are unchanged.

📚 Evidence Consulted

Evidence

  • Changed files: 4 files (+21 / -2) — manifest.yaml, metadata.yaml, unit_tests/test_components.py, docs/integrations/sources/google-analytics-data-api.md
  • CI checks: Test source-google-analytics-data-api Connector ✅, Lint source-google-analytics-data-api Connector ✅, Build and Verify Artifacts (source-google-analytics-data-api) ✅, Connector CI Checks Summary ✅, source-google-analytics-data-api Pre-Release Checks ✅, Progressive Rollout Gate ✅, Format Check ✅, Check Changelog Updated ✅, Docs / Vale ✅, Docs / MarkDownLint ✅, CodeQL ✅/neutral
  • PR labels: connectors/source/google-analytics-data-api, hyd-fix, hyd-prove, hyd-review
  • PR description: present (substantive, with equivalence proof)
  • /ai-prove-fix conclusion: ✅ Fix Proven (session c79cdc177d244567a56e190921ecb818), pre-release 2.9.46-preview.c170a91 published
  • Existing bot reviews: none

@airbyte-support-bot

Copy link
Copy Markdown
Contributor

↪️ Triggering /ai-ready per Hands-Free AI Triage Project triage next step.

Reason: /ai-review returned APPROVE for the exact current HEAD c170a911 on top of a ✅ Fix Proven, CI is green, and the branch is not conflicted.

Devin session

@octavia-bot octavia-bot Bot added the hyd-ready Hydra: ready for human review or merge label Aug 4, 2026
@octavia-bot-hoard

octavia-bot-hoard Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Auto-merge evaluation: FAIL

Preconditions (all must pass)

Status Check Reasoning
No Breaking Changes Patch version bump (2.9.45 → 2.9.46). One-line manifest change scoping field_path ["**","name"] → ["name"]; prove-fix and PR evidence show byte-identical SPEC/DISCOVER/READ and resolved stream configs vs control. No spec, schema, or stream-name changes; no breakingChanges entry needed.
Ai Review Passed Comment 13 by airbyte-support-bot contains '<!-- pr_ai_review_result: APPROVE; head_sha: c170a91...' with all 12 gates passed, matching the current HEAD.
Airbyte Support Bot Assigned Airbyte Support Bot assignment confirms automated merge eligibility.
No Destination Changes No destination-related changes detected.

Change scope (at least one must pass)

❌ No matching change scope detected.

Devin session | Workflow run

@airbyte-support-bot

Copy link
Copy Markdown
Contributor

🙋 Escalated to #human-in-the-loop per Hands-Free AI Triage Project triage next step.

Reason: /ai-ready returned FAIL with every precondition passing — no breaking changes, AI review APPROVE matching the exact current HEAD, Airbyte Support Bot assigned, no destination changes. The single failure is No matching change scope detected, so the auto-merge scope classifier does not recognise this change shape. This is the fourth PR today blocked on that same check, alongside #82773, so it reads as an auto-merge scope-rule gap rather than a problem with the PR. A human needs to merge directly, or the scope rules need widening.

Devin session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

connectors/source/google-analytics-data-api hyd-fix Hydra: ai-fix stage has run hyd-prove Hydra: ai-prove-fix stage has run hyd-ready Hydra: ready for human review or merge hyd-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants