From 27b5a5253b73ac84471b770f65884127198bb19c Mon Sep 17 00:00:00 2001 From: DENGXUELIN <37065511+DENGXUELIN@users.noreply.github.com> Date: Tue, 9 Jun 2026 05:42:08 +0800 Subject: [PATCH] Add Azure diagnostic pipeline fixtures --- skills/cloud/azure-review/SKILL.md | 36 +++++- .../cloud/azure-review/benchmark-checklist.md | 39 ++++++ ...e-review-complete-diagnostic-pipeline.yaml | 120 ++++++++++++++++++ ...re-review-partial-diagnostic-pipeline.yaml | 88 +++++++++++++ 4 files changed, 280 insertions(+), 3 deletions(-) create mode 100644 tests/benign/azure-review-complete-diagnostic-pipeline.yaml create mode 100644 tests/vulnerable/azure-review-partial-diagnostic-pipeline.yaml diff --git a/skills/cloud/azure-review/SKILL.md b/skills/cloud/azure-review/SKILL.md index ac6d6ac7..71d4265d 100644 --- a/skills/cloud/azure-review/SKILL.md +++ b/skills/cloud/azure-review/SKILL.md @@ -13,7 +13,7 @@ phase: [assess, operate] frameworks: [CIS-Azure-v2.1.0] difficulty: intermediate time_estimate: "60-90min" -version: "1.0.0" +version: "1.0.1" author: unitoneai license: MIT allowed-tools: Read, Grep, Glob @@ -39,6 +39,7 @@ If a target is provided via arguments, focus the review on: $ARGUMENTS - Assessing an existing Azure environment's security posture against CIS benchmarks - Preparing for a CIS benchmark audit or compliance assessment - Evaluating Entra ID configurations, NSG rules, Defender for Cloud, Storage account security, or Key Vault access policies +- Validating diagnostic settings coverage, category selection, destination hardening, retention, and sample delivery evidence - Onboarding a new Azure subscription into a security program --- @@ -88,6 +89,26 @@ For detailed CIS benchmark checklist items with specific Terraform patterns, Bic --- +### Step 10A: Diagnostic Pipeline Integrity Review + +For CIS Section 5 findings, do not pass logging controls solely because an `azurerm_monitor_diagnostic_setting` resource exists. Require evidence that the pipeline covers the right Azure scope, emits security-relevant categories, lands in a hardened destination, retains data long enough, and has delivered sample events. + +#### Diagnostic Evidence Gates + +| Gate | Requirement | Fail / Escalate When | +|------|-------------|----------------------| +| AZ-DIAG-01 | Subscription Activity Log and in-scope resource diagnostics are inventoried with coverage denominator. | Only subscription Activity Log exists, or critical resources such as Key Vault, Storage, SQL, NSG, or App Service lack diagnostics. | +| AZ-DIAG-02 | Required log categories or `category_group = "allLogs"` are enabled where supported. | Security, Administrative, Policy, ResourceHealth, Key Vault `AuditEvent`, or service-specific audit categories are missing. | +| AZ-DIAG-03 | Every diagnostic setting has a destination mapped to Log Analytics, Event Hub, or Storage with workspace/account/hub identity. | Destination is missing, points to a non-production sink, or cannot be tied to the reviewed subscription/resource. | +| AZ-DIAG-04 | Destination retention meets policy for both source settings and downstream storage/consumer retention. | Retention is disabled, below policy, or Event Hub consumers drop events before the required window. | +| AZ-DIAG-05 | Destination access is hardened: no public diagnostic storage, least-privilege readers, private access/CMK where policy requires it. | Diagnostic storage is public, broadly readable, lacks required CMK/private access, or Event Hub authorization is over-scoped. | +| AZ-DIAG-06 | Sample delivery evidence exists for high-value categories such as Activity Log `Administrative` and Key Vault `AuditEvent`. | No recent sample event is observed at the destination and the control is marked Pass instead of Not Evaluable. | +| AZ-DIAG-07 | Cross-region and multi-subscription resources route to approved regional or central destinations without unsupported gaps. | Diagnostics cover only a pilot region/subscription while production regions or subscriptions are omitted. | +| AZ-DIAG-08 | Exceptions include owner, expiry, remediation plan, monitoring, and retest trigger. | Missing categories, short retention, or absent sample delivery are accepted without governance evidence. | + +**Finding classification:** Missing diagnostics for critical production resources, omitted security-relevant categories, broadly accessible diagnostic destinations, or no destination for emitted logs are **High**. Missing sample delivery evidence, retention gaps, incomplete coverage denominator, or ungoverned exceptions are **Medium** unless they block incident response for regulated or critical workloads. + +--- --- @@ -102,8 +123,8 @@ Produce the final report using the structure defined in the Output Format sectio | Severity | Definition | Examples | |----------|-----------|----------| | **Critical** | Immediate risk of data breach or unauthorized access | NSGs open to 0.0.0.0/0 on RDP/SSH, SQL databases publicly accessible, Defender for Cloud disabled | -| **High** | Significant security gap that materially weakens posture | Missing MFA enforcement, storage accounts with public access, Key Vault without purge protection | -| **Medium** | Control gap that should be addressed in normal cycle | Missing activity log alerts, soft delete not enabled, TLS below 1.2 | +| **High** | Significant security gap that materially weakens posture | Missing MFA enforcement, storage accounts with public access, Key Vault without purge protection, critical resources without diagnostics, diagnostic destination broadly accessible | +| **Medium** | Control gap that should be addressed in normal cycle | Missing activity log alerts, soft delete not enabled, TLS below 1.2, diagnostic retention below policy, missing sample delivery evidence | | **Low** | Hardening recommendation or defense-in-depth measure | HTTP/2 not enabled, FTP not fully disabled, missing CMK on non-sensitive storage | | **Informational** | Best practice observation, no direct security impact | Naming conventions, tag policies, documentation gaps | @@ -142,6 +163,12 @@ Produce the final report using the structure defined in the Output Format sectio | 8 | Key Vault | X | Y | Z | nn% | | 9 | App Service | X | Y | Z | nn% | +### Diagnostic Pipeline Evidence + +| Scope | Resource Type | Categories / Groups | Destination | Retention | Destination Hardening | Sample Delivery | Status | +|-------|---------------|---------------------|-------------|-----------|-----------------------|-----------------|--------| +| Subscription / Resource ID | Activity Log / Key Vault / Storage / SQL / NSG / App Service | Administrative, Security, AuditEvent, allLogs | Log Analytics / Event Hub / Storage | N days | Private/CMK/RBAC status | Observed / Missing / Not Evaluable | Pass / Fail / Not Evaluable | + ### Detailed Findings #### [CIS X.Y.Z] @@ -200,6 +227,8 @@ Produce the final report using the structure defined in the Output Format sectio 4. **NSG rules using service tags.** A rule with `source_address_prefix = "Internet"` is equivalent to `0.0.0.0/0`. Both must be flagged for CIS 6.1 and 6.2. 5. **Key Vault purge protection is irreversible.** CIS 8.5 requires `purge_protection_enabled = true`. Note this cannot be disabled once enabled -- flag this for awareness during remediation. 6. **App Service TLS version on both Linux and Windows.** Check `azurerm_linux_web_app` and `azurerm_windows_web_app` resources separately. +7. **Treating one subscription Activity Log diagnostic setting as full logging coverage.** Resource-level diagnostics are still required for services such as Key Vault, Storage, SQL, NSG flow logs, and App Service. Record the coverage denominator before passing CIS Section 5 controls. +8. **Passing diagnostics without destination proof.** A diagnostic setting can exist while logs are routed to short-retention storage, a non-production workspace, an over-scoped Event Hub policy, or nowhere useful. Verify destination identity, retention, hardening, and sample delivery. --- @@ -231,4 +260,5 @@ Produce the final report using the structure defined in the Output Format sectio ## Changelog +- **1.0.1** -- Added diagnostic pipeline integrity gates for scope coverage, category coverage, destination mapping, retention, destination hardening, sample delivery, cross-region/subscription coverage, and exception governance. - **1.0.0** -- Initial release. Full coverage of CIS Microsoft Azure Foundations Benchmark v2.1.0 sections 1 through 9. diff --git a/skills/cloud/azure-review/benchmark-checklist.md b/skills/cloud/azure-review/benchmark-checklist.md index 41a67846..2b2445cd 100644 --- a/skills/cloud/azure-review/benchmark-checklist.md +++ b/skills/cloud/azure-review/benchmark-checklist.md @@ -409,10 +409,32 @@ resource "azurerm_monitor_diagnostic_setting" { Verify that Administrative, Security, ServiceHealth, Alert, Recommendation, Policy, Autoscale, and ResourceHealth categories are enabled. +For each diagnostic setting, record whether the setting uses explicit categories or `category_group = "allLogs"` where supported: + +```hcl +resource "azurerm_monitor_diagnostic_setting" "key_vault" { + target_resource_id = azurerm_key_vault.example.id + log_analytics_workspace_id = azurerm_log_analytics_workspace.security.id + + enabled_log { + category = "AuditEvent" + } +} +``` + +Treat partial category coverage as a failure when security-relevant categories are omitted. If a provider supports `category_group = "allLogs"`, verify whether policy expects the category group instead of a hand-picked subset. + #### CIS 5.1.3 -- Ensure the storage container storing the activity logs is not publicly accessible Check storage account access level for the diagnostic logs container. +Also verify the diagnostic destination itself: + +- Storage account public network access, blob public access, RBAC assignments, and container ACLs. +- Log Analytics workspace retention and table-level retention where configured. +- Event Hub authorization policy scope and downstream consumer retention. +- CMK/private endpoint evidence when policy or data classification requires it. + #### CIS 5.1.4 -- Ensure the storage account containing the container with activity logs is encrypted with a Customer Managed Key Cross-reference the diagnostics storage account with CMK encryption. @@ -428,6 +450,23 @@ resource "azurerm_monitor_diagnostic_setting" { } ``` +#### Diagnostic Pipeline Integrity Checklist + +Use this checklist before marking CIS Section 5 diagnostic coverage as Pass: + +| Gate | Check | +|------|-------| +| AZ-DIAG-01 | Build a coverage denominator for subscription Activity Log plus each in-scope Key Vault, Storage, SQL, NSG, App Service, and other monitored resource type. | +| AZ-DIAG-02 | Verify required categories/category groups, including Administrative, Security, Policy, ResourceHealth, and Key Vault AuditEvent. | +| AZ-DIAG-03 | Map each diagnostic setting to a production Log Analytics workspace, Event Hub, or Storage Account destination. | +| AZ-DIAG-04 | Verify source retention and downstream retention meet policy, including Event Hub consumer retention when logs are streamed. | +| AZ-DIAG-05 | Verify destination hardening: no public diagnostic storage, least-privilege readers, private access, and CMK where required. | +| AZ-DIAG-06 | Confirm sample delivery at the destination, or mark the control Not Evaluable when sample evidence is unavailable. | +| AZ-DIAG-07 | Confirm all production regions and subscriptions are covered, not only a pilot subscription or one region. | +| AZ-DIAG-08 | Document owner, expiry, remediation plan, monitoring, and retest trigger for any exception. | + +Do not downgrade missing sample delivery or retention evidence to Pass. Use Not Evaluable when configuration exists but delivery cannot be proven from the available evidence. + ### CIS 5.2 -- Activity Log Alerts #### CIS 5.2.1 -- Ensure that Activity Log Alert exists for Create Policy Assignment diff --git a/tests/benign/azure-review-complete-diagnostic-pipeline.yaml b/tests/benign/azure-review-complete-diagnostic-pipeline.yaml new file mode 100644 index 00000000..123a2dcc --- /dev/null +++ b/tests/benign/azure-review-complete-diagnostic-pipeline.yaml @@ -0,0 +1,120 @@ +case: azure-review-complete-diagnostic-pipeline +skill: azure-review +expected_risk: Low +description: > + Azure diagnostic settings include subscription and resource-level coverage, + required categories, hardened destinations, policy-compliant retention, and + recent sample delivery evidence. +environment: + subscription_id: sub-prod-001 + policy_retention_days: 365 + regulated: true +diagnostic_inventory: + expected_resources: + key_vaults: 4 + storage_accounts: 6 + sql_servers: 2 + nsgs: 18 + app_services: 9 + configured_settings: + - name: sub-activity-security + target_scope: subscription + target_resource_id: /subscriptions/sub-prod-001 + enabled_categories: + - Administrative + - Security + - Policy + - ResourceHealth + - ServiceHealth + - Alert + destination: + type: log_analytics + workspace: law-sec-prod + retention_days: 400 + private_link: true + table_retention_verified: true + sample_delivery: + Administrative: observed + Security: observed + Policy: observed + - name: kv-alllogs + target_scope: resource + resource_type: Microsoft.KeyVault/vaults + covered_resources: 4 + coverage_denominator: 4 + enabled_category_group: allLogs + required_category_sample: + AuditEvent: observed + destination: + type: event_hub + namespace: eh-sec-prod + hub: azure-diagnostics + retention_days: 7 + consumer_archive_days: 400 + authorization_scope: send_only + - name: storage-sql-nsg-appservice + target_scope: resource + resource_types: + - Microsoft.Storage/storageAccounts + - Microsoft.Sql/servers/databases + - Microsoft.Network/networkSecurityGroups + - Microsoft.Web/sites + covered_resources: 35 + coverage_denominator: 35 + enabled_category_group: allLogs + destination: + type: storage + account: diagsecure001 + retention_days: 400 + public_network_access: Disabled + blob_public_access: false + cmk_enabled: true + private_endpoint: true + sample_delivery: + StorageRead: observed + SQLSecurityAuditEvents: observed + NetworkSecurityGroupFlowEvent: observed + AppServiceHTTPLogs: observed +coverage_gaps: + key_vaults_without_audit_event: [] + sql_servers_without_diagnostics: [] + nsgs_without_flow_logs: 0 + app_services_without_http_logs: 0 + regions_omitted: [] +exception: + owner: cloud-security-monitoring + expiry: none + monitoring: diagnostic-drift-policy + retest_trigger: + - new-subscription + - new-critical-resource + - retention-policy-change +expected_findings: + - id: AZ-DIAG-01 + status: pass + reason: subscription and critical resource coverage denominators are complete + - id: AZ-DIAG-02 + status: pass + reason: required categories and allLogs groups are enabled + - id: AZ-DIAG-03 + status: pass + reason: each setting maps to an approved production destination + - id: AZ-DIAG-04 + status: pass + reason: workspace, storage, and consumer retention meet policy + - id: AZ-DIAG-05 + status: pass + reason: destinations use private access, least privilege, and CMK where required + - id: AZ-DIAG-06 + status: pass + reason: sample delivery is observed for security-relevant categories + - id: AZ-DIAG-07 + status: pass + reason: all production regions and resources are covered + - id: AZ-DIAG-08 + status: pass + reason: drift monitoring and retest triggers are documented +expected_output: + diagnostic_pipeline_evidence: Pass + section_5_status: Pass + severity: Low diff --git a/tests/vulnerable/azure-review-partial-diagnostic-pipeline.yaml b/tests/vulnerable/azure-review-partial-diagnostic-pipeline.yaml new file mode 100644 index 00000000..f41ee92e --- /dev/null +++ b/tests/vulnerable/azure-review-partial-diagnostic-pipeline.yaml @@ -0,0 +1,88 @@ +case: azure-review-partial-diagnostic-pipeline +skill: azure-review +expected_risk: High +description: > + A review passes Azure logging because one subscription diagnostic setting + exists, but critical resource diagnostics, categories, retention, destination + hardening, and sample delivery evidence are missing. +environment: + subscription_id: sub-prod-001 + policy_retention_days: 365 + regulated: true +diagnostic_inventory: + expected_resources: + key_vaults: 4 + storage_accounts: 6 + sql_servers: 2 + nsgs: 18 + app_services: 9 + configured_settings: + - name: sub-activity-basic + target_scope: subscription + target_resource_id: /subscriptions/sub-prod-001 + enabled_categories: + - Administrative + - Alert + missing_categories: + - Security + - Policy + - ResourceHealth + destination: + type: storage + account: diagpublic001 + retention_days: 30 + public_network_access: Enabled + blob_public_access: true + cmk_enabled: false + sample_delivery: + Administrative: observed + Security: missing + Policy: missing +coverage_gaps: + key_vaults_without_audit_event: + - kv-payments-prod + - kv-token-prod + - kv-support-prod + - kv-ops-prod + sql_servers_without_diagnostics: + - sql-customer-prod + - sql-ledger-prod + nsgs_without_flow_logs: 14 + app_services_without_http_logs: 7 + regions_omitted: + - eastus2 + - westeurope +exception: + owner: missing + expiry: missing + remediation_plan: missing + retest_trigger: missing +expected_findings: + - id: AZ-DIAG-01 + status: fail + reason: subscription Activity Log exists but critical resource diagnostics are missing + - id: AZ-DIAG-02 + status: fail + reason: Security, Policy, ResourceHealth, and Key Vault AuditEvent categories are missing + - id: AZ-DIAG-03 + status: pass + reason: the single subscription setting has a storage destination + - id: AZ-DIAG-04 + status: fail + reason: 30 day retention is below the 365 day policy + - id: AZ-DIAG-05 + status: fail + reason: diagnostic storage is public and lacks required CMK + - id: AZ-DIAG-06 + status: fail + reason: no sample delivery exists for security-relevant categories + - id: AZ-DIAG-07 + status: fail + reason: production regions are omitted from the diagnostic coverage + - id: AZ-DIAG-08 + status: fail + reason: exceptions have no owner, expiry, remediation plan, or retest trigger +expected_output: + diagnostic_pipeline_evidence: Failed + section_5_status: Fail + severity: High