ARO-26057: add ADX Grafana datasource provisioning by swiencki · Pull Request #4878 · Azure/ARO-HCP

swiencki · 2026-04-14T21:56:30Z

What

Adds ADX/Kusto Grafana datasource provisioning through the geography rollout using the typed GrafanaDatasources action. The step forwards ADX desired state to grafanactl modify datasource reconcile, grants the Grafana managed identity Kusto Viewer access to ServiceLogs, and removes the prior rollout-time go run shell path.

Why

This follows the existing typed/prebuilt-binary pattern for Grafana datasource rollout while enabling SRE dashboards to query Kusto data from Grafana.

Testing

GOWORK=/tmp/aro-hcp-adx-local.work go test ./pkg/pipeline from tooling/templatize
git diff --check
Local validation against the paired ARO-Tools change

Special notes for your reviewer

Depends on Azure/ARO-Tools#228. Downstream sdp-pipelines rollout packaging is https://dev.azure.com/msazure/AzureRedHatOpenShift/_git/sdp-pipelines/pullrequest/15393796.

Required merge order: ARO-Tools first, then this ARO-HCP PR with the ARO-Tools pin bump, then sdp-pipelines with the ARO-Tools/ARO-HCP pin bumps and make -C hcp/ update run twice.

The Kusto role-assignment rename fixes a name collision between ingest and viewer grants. After first deployment, old grant-${guid(...)} Kusto role assignments may remain as harmless duplicates; ops can do a one-time cleanup if desired.

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds ARO-HCP support to provision and clean up Azure Data Explorer (Kusto) data sources in the shared Managed Grafana workspace, integrated into the existing geography rollout flow.

Changes:

Extend Kusto IaC to optionally grant Grafana’s managed identity DB-level Viewer access and output the cluster URI.
Add geography pipeline steps to fetch global Grafana outputs and provision/update ADX Grafana datasources via Azure CLI.
Introduce configuration flags/schema + docs, and update the Kusto delete cleanup to remove the matching Grafana datasource.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
docs/monitoring.md	Documents ADX/Kusto datasource provisioning gates, naming, validation, and teardown behavior.
dev-infrastructure/templates/kusto.bicep	Threads optional Grafana resource ID into the Kusto module and exposes `kustoUri` output.
dev-infrastructure/modules/logs/kusto/main.bicep	Grants Grafana MI Viewer on `ServiceLogs` when provided and surfaces cluster URI output.
dev-infrastructure/modules/logs/kusto/cluster.bicep	Exposes Kusto cluster `uri` as an output.
dev-infrastructure/geography-pipeline.yaml	Adds global output step and a shell step to create/update ADX Grafana datasources.
dev-infrastructure/configurations/kusto.tmpl.bicepparam	Adds `grafanaResourceId` parameter placeholder for pipeline substitution.
dev-infrastructure/cleanup/delete.kusto.instance.sh	Extends cleanup to also delete the corresponding Grafana datasource.
dev-infrastructure/cleanup/delete.kusto.instance.pipeline.yaml	Wires Grafana resource ID from global outputs into the cleanup script run.
config/rendered/dev/swft/uksouth.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/rendered/dev/prow/westus3.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/rendered/dev/pers/westus3.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/rendered/dev/perf/westus3.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/rendered/dev/dev/westus3.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/rendered/dev/cspr/westus3.yaml	Adds default monitoring flags for ADX datasource provisioning (disabled).
config/config.yaml	Introduces new monitoring defaults for ADX provisioning flags.
config/config.schema.json	Defines schema/validation for the new monitoring configuration knobs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

janboll · 2026-04-16T07:26:50Z

/hold
please use https://github.com/Azure/ARO-Tools/blob/main/tools/grafanactl/cmd/modify/cmd.go#L32
for doing modifications to grafana. Ideally we should update grafana ONCE in the pipeline, cause running an update on the instance will block it for several minutes.

swiencki · 2026-04-16T16:06:29Z

Thanks for the feedback, I'll move this to grafanactl in Azure/ARO-Tools.

On the "update once" concern: the existing modify datasource reconcile runs globally because UpdateGrafanaIntegrations is an ARM-level mutation that blocks the Grafana instance for several minutes. ADX datasources go through the Grafana REST API directly (POST/PUT /api/datasources), which is a lightweight config write, no ARM-level instance lock. So per-geography invocation is safe here. Is this good with you?

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

swiencki · 2026-04-28T21:19:54Z

/unhold

Updated to use grafanactl.

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…peline Per PR #4878 review. Drops the new GrafanaDatasources call from geography-pipeline.yaml and extends the existing add-grafana-datasource in region-pipeline.yaml with the ADX block. clusterUrl sources from regional kusto-lookup output. AzureMonitor reconcile preserved via the ARO-Tools default-true. Triple-AND gate keeps the three feature flags agreeing by construction. No Go/Bicep/config/test changes.

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

Replace the temporary shell datasource script with a direct grafanactl invocation and keep cleanup datasource deletion out of this rollout scope. Depends on Azure/ARO-Tools#228.

…peline Per PR #4878 review. Drops the new GrafanaDatasources call from geography-pipeline.yaml and extends the existing add-grafana-datasource in region-pipeline.yaml with the ADX block. clusterUrl sources from regional kusto-lookup output. AzureMonitor reconcile preserved via the ARO-Tools default-true. Triple-AND gate keeps the three feature flags agreeing by construction. No Go/Bicep/config/test changes.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

openshift-ci · 2026-06-09T19:55:38Z

@swiencki: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/cspr	`8d14303`	link	true	`/test cspr`
ci/prow/mega-linter	`4d47166`	link	true	`/test mega-linter`
ci/prow/images	`4d47166`	link	true	`/test images`
ci/prow/e2e-parallel	`4d47166`	link	true	`/test e2e-parallel`
ci/prow/config-change-detection	`4d47166`	link	true	`/test config-change-detection`
ci/prow/bicep-lint	`4d47166`	link	true	`/test bicep-lint`
ci/prow/test-unit	`4d47166`	link	true	`/test test-unit`
ci/prow/lint	`4d47166`	link	true	`/test lint`
ci/prow/verify	`4d47166`	link	true	`/test verify`
ci/prow/e2e-images	`4d47166`	link	true	`/test e2e-images`
ci/prow/image-updater-images	`4d47166`	link	true	`/test image-updater-images`
ci/prow/periodic-healthcheck-images	`4d47166`	link	true	`/test periodic-healthcheck-images`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

mmazur · 2026-06-17T15:29:11Z

+      defaultDatabase:
+        configRef: kusto.serviceLogsDatabase
+      datasourceName:
+        value: "kusto-{{ .environmentName }}-{{ .azureGeoShortId }}"


This should reflect the actual source name.

openshift-ci · 2026-07-03T15:20:15Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copilot AI review requested due to automatic review settings April 14, 2026 21:56

openshift-ci Bot requested review from geoberle and janboll April 14, 2026 21:56

Copilot AI reviewed Apr 14, 2026

View reviewed changes

Comment thread dev-infrastructure/geography-pipeline.yaml Outdated

Comment thread dev-infrastructure/geography-pipeline.yaml Outdated

Comment thread dev-infrastructure/configurations/kusto.tmpl.bicepparam

Comment thread dev-infrastructure/cleanup/delete.kusto.instance.sh Outdated

swiencki changed the title ~~ARO-22279: add ADX Grafana datasource provisioning~~ ARO-26057: add ADX Grafana datasource provisioning Apr 15, 2026

openshift-ci Bot added the do-not-merge/hold label Apr 16, 2026

openshift-ci Bot added the needs-rebase label Apr 22, 2026

Copilot AI review requested due to automatic review settings April 28, 2026 18:45

swiencki force-pushed the adx-grafana-datasource-upstream branch from 8d14303 to a80a076 Compare April 28, 2026 18:45

openshift-ci Bot removed the needs-rebase label Apr 28, 2026

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Comment thread dev-infrastructure/geography-pipeline.yaml

Copilot AI review requested due to automatic review settings April 28, 2026 19:51

Copilot started reviewing on behalf of swiencki April 28, 2026 19:52 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

openshift-ci Bot added needs-rebase and removed do-not-merge/hold labels Apr 28, 2026

janboll requested changes Apr 30, 2026

View reviewed changes

Comment thread dev-infrastructure/geography-pipeline.yaml Outdated

Copilot AI review requested due to automatic review settings April 30, 2026 21:02

swiencki force-pushed the adx-grafana-datasource-upstream branch from 5d069ae to c091e9c Compare April 30, 2026 21:02

openshift-ci Bot removed the needs-rebase label Apr 30, 2026

Copilot started reviewing on behalf of swiencki April 30, 2026 21:02 View session

swiencki mentioned this pull request Apr 30, 2026

grafanactl: reconcile ADX datasources Azure/ARO-Tools#228

Open

Copilot AI reviewed Apr 30, 2026

View reviewed changes

Comment thread tooling/templatize/pkg/pipeline/grafana_datasources_test.go

Copilot AI review requested due to automatic review settings May 8, 2026 15:56

Copilot started reviewing on behalf of swiencki May 8, 2026 15:57 View session

Copilot AI reviewed May 8, 2026

View reviewed changes

Comment thread docs/monitoring.md

Comment thread docs/monitoring.md

Copilot AI review requested due to automatic review settings June 3, 2026 17:33

Copilot started reviewing on behalf of swiencki June 3, 2026 17:33 View session

Copilot AI reviewed Jun 3, 2026

View reviewed changes

Comment thread tooling/templatize/pkg/pipeline/grafana_datasources.go Outdated

Copilot AI review requested due to automatic review settings June 4, 2026 19:50

Copilot started reviewing on behalf of swiencki June 4, 2026 19:50 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

geoberle reviewed Jun 9, 2026

View reviewed changes

Comment thread dev-infrastructure/region-pipeline.yaml

Comment thread dev-infrastructure/region-pipeline.yaml

Comment thread dev-infrastructure/modules/logs/kusto/grant-access.bicep Outdated

Comment thread dev-infrastructure/templates/kusto-lookup.bicep Outdated

openshift-ci Bot added the needs-rebase label Jun 9, 2026

swiencki added 12 commits June 9, 2026 11:54

monitoring: add adx grafana datasource rollout

f942386

monitoring: clean up kusto grafana datasources

ede7c32

monitoring: harden kusto cleanup grafana wait

c137440

monitoring: fail adx datasource shell pipelines

4725fbb

monitoring: externalize adx datasource shell step

07579fa

monitoring: use grafanactl for ADX datasources

d8fe167

Replace the temporary shell datasource script with a direct grafanactl invocation and keep cleanup datasource deletion out of this rollout scope. Depends on Azure/ARO-Tools#228.

monitoring: constrain geography global output

b3d6d5f

monitoring: fix geography pipeline metadata

0b702b5

monitoring: restore shared global metadata

aa18f33

monitoring: use typed GrafanaDatasources for ADX

297e861

monitoring: use real geo short ID in ADX tests

1b32952

Copilot AI review requested due to automatic review settings June 9, 2026 18:54

swiencki force-pushed the adx-grafana-datasource-upstream branch from 6131235 to 4d47166 Compare June 9, 2026 18:54

openshift-ci Bot removed the needs-rebase label Jun 9, 2026

Copilot started reviewing on behalf of swiencki June 9, 2026 18:54 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread dev-infrastructure/region-pipeline.yaml

Comment thread dev-infrastructure/region-pipeline.yaml

Comment thread dev-infrastructure/region-pipeline.yaml

swiencki requested a review from geoberle June 9, 2026 19:56

mmazur reviewed Jun 17, 2026

View reviewed changes

openshift-ci Bot added the needs-rebase label Jul 3, 2026

Uh oh!

Conversation

swiencki commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Testing

Special notes for your reviewer

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janboll commented Apr 16, 2026

Uh oh!

swiencki commented Apr 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

swiencki commented Apr 28, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

openshift-ci Bot commented Jun 9, 2026

Uh oh!

mmazur Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

swiencki commented Apr 14, 2026 •

edited

Loading