Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 45 additions & 2 deletions skills/cloud/azure-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ phase: [assess, operate]
frameworks: [CIS-Azure-v2.1.0]
difficulty: intermediate
time_estimate: "60-90min"
version: "1.0.0"
version: "1.0.1"
author: unitoneai
license: MIT
allowed-tools: Read, Grep, Glob
Expand Down Expand Up @@ -91,7 +91,39 @@ For detailed CIS benchmark checklist items with specific Terraform patterns, Bic

---

### Step 11: Compile Assessment Report
### Step 11: Qualify Evidence Scope and Confidence

Before compiling findings, qualify the evidence source and Azure scope for each evaluated control. Azure CIS evidence can span tenant-level Entra ID settings, management-group policy assignments, subscription-scoped Defender plans, resource-scoped diagnostic/private-network settings, and sampled exports. Do not claim tenant-wide, management-group-wide, or subscription-wide compliance from a single Terraform module, report-only policy, or one resource export unless the denominator and effective scope are proven.

**Evidence confidence levels:**

| Level | Meaning | Example |
|-------|---------|---------|
| `iac-only` | Repository configuration shows intended state, but live effective state is not proven | Terraform defines one Conditional Access policy |
| `live-export` | Azure CLI, Microsoft Graph, Defender for Cloud, Policy, Resource Graph, or service export confirms deployed state | Graph export shows Conditional Access policy state and assignments |
| `policy-assignment` | Azure Policy assignment or initiative evidence proves enforcement at a management group, subscription, or resource scope | Management-group initiative with effect and exemption export |
| `defender-export` | Defender for Cloud export confirms plan and coverage state for the supplied subscriptions/resource types | Defender plans enabled for Servers, Storage, Key Vault, and SQL across listed subscriptions |
| `sampled` | Evidence covers selected tenants, subscriptions, resource groups, or resources only | Two subscriptions sampled from an enterprise landing zone |
| `unknown` | No supplied evidence proves the control | Break-glass exclusion details or tenant setting export is absent |

**Azure evidence-scope gates:**

| Gate | Requirement |
|------|-------------|
| `AZ-SCOPE-01` | Record evidence source, capture date, confidence level, and reviewed tenant/management-group/subscription/resource scope for every detailed finding. |
| `AZ-SCOPE-02` | Record the scope denominator: tenant ID, management groups, subscriptions, resource groups, resources, and exclusions when available. |
| `AZ-SCOPE-03` | For Conditional Access and Security Defaults, record policy state, report-only versus enabled mode, included/excluded users or roles, break-glass accounts, named locations, and equivalent-control rationale. |
| `AZ-SCOPE-04` | For Azure Policy evidence, record assignment scope, initiative/control mapping, effect, compliance state, exemption owner, expiry, and inherited management-group coverage. |
| `AZ-SCOPE-05` | For Defender for Cloud, record each subscription/resource-type plan, tier, auto-provisioning state, and whether evidence is IaC intent or live Defender export. |
| `AZ-SCOPE-06` | For diagnostic, private endpoint, and network controls, record target resource/subscription scope and whether evidence proves category/sink/private-link effectiveness or only resource presence. |
| `AZ-SCOPE-07` | Mark controls as Not Evaluable with a reason code when evidence is missing: `tenant-live-only`, `missing-management-group-export`, `missing-subscription-export`, `missing-resource-export`, `report-only-policy`, `sample-only`, `paid-plan-not-enabled`, or `not-in-scope`. |
| `AZ-SCOPE-08` | Surface evidence age, sampled coverage, policy exemptions, Defender paid-plan limits, owner, expiry, and retest trigger before assigning full Pass. |

**Classification guidance:** Claiming tenant-wide or enterprise subscription Pass from `iac-only`, `sampled`, report-only Conditional Access, or one subscription export is at least **Medium** for report integrity. For release-blocking identity, Defender, diagnostic, Key Vault, storage, or public-network controls, missing tenant/management-group/subscription denominator evidence can be **High**. Paid Defender plan observations should be reported from available evidence; do not enable paid plans or change subscription settings without explicit approval.

---

### Step 12: Compile Assessment Report

Produce the final report using the structure defined in the Output Format section.

Expand Down Expand Up @@ -148,6 +180,10 @@ Produce the final report using the structure defined in the Output Format sectio
- **Status:** Pass / Fail / Not Evaluable
- **Severity:** Critical / High / Medium / Low
- **CIS Profile:** Level 1 / Level 2
- **Evidence Source:** iac-only / live-export / policy-assignment / defender-export / sampled / unknown
- **Evidence Captured:** <date/time or export identifier>
- **Scope Coverage:** <tenant, management group, subscription, resource group, resource, or sample scope>
- **Not Evaluable Reason:** <reason code if applicable>
- **File:** <path to relevant config>
- **Line(s):** <line numbers if applicable>
- **Description:** <what was found>
Expand Down Expand Up @@ -200,6 +236,10 @@ Produce the final report using the structure defined in the Output Format sectio
4. **NSG rules using service tags.** A rule with `source_address_prefix = "Internet"` is equivalent to `0.0.0.0/0`. Both must be flagged for CIS 6.1 and 6.2.
5. **Key Vault purge protection is irreversible.** CIS 8.5 requires `purge_protection_enabled = true`. Note this cannot be disabled once enabled -- flag this for awareness during remediation.
6. **App Service TLS version on both Linux and Windows.** Check `azurerm_linux_web_app` and `azurerm_windows_web_app` resources separately.
7. **Treating report-only Conditional Access as enforcement.** A policy in report-only mode is evidence of intent, not an enforced tenant control. Record state, assignments, exclusions, and break-glass handling.
8. **Inferring enterprise coverage from one subscription.** Defender plans, diagnostics, Key Vault, storage, networking, and App Service controls need subscription/resource denominators or inherited policy evidence.
9. **Ignoring policy exemptions and inheritance.** Management-group policy assignments can enforce many subscriptions, but exemptions, disabled effects, and lower-scope overrides change the effective result.
10. **Treating Defender plan IaC as live subscription state.** `azurerm_security_center_subscription_pricing` shows intended configuration for one resource type; live Defender exports are needed before broad coverage claims.

---

Expand All @@ -222,6 +262,8 @@ Produce the final report using the structure defined in the Output Format sectio
- CIS Microsoft Azure Foundations Benchmark v2.1.0: https://www.cisecurity.org/benchmark/azure
- Microsoft Defender for Cloud Documentation: https://learn.microsoft.com/en-us/azure/defender-for-cloud/
- Microsoft Entra ID Security: https://learn.microsoft.com/en-us/entra/identity/
- Microsoft Entra Conditional Access: https://learn.microsoft.com/en-us/entra/identity/conditional-access/overview
- Azure Policy Assignment Structure: https://learn.microsoft.com/en-us/azure/governance/policy/concepts/assignment-structure
- Azure Storage Security: https://learn.microsoft.com/en-us/azure/storage/common/storage-security-guide
- Azure Key Vault Best Practices: https://learn.microsoft.com/en-us/azure/key-vault/general/best-practices
- Azure App Service Security: https://learn.microsoft.com/en-us/azure/app-service/overview-security
Expand All @@ -231,4 +273,5 @@ Produce the final report using the structure defined in the Output Format sectio

## Changelog

- **1.0.1** -- Added evidence confidence, tenant/management-group/subscription/resource scope gates, Conditional Access state handling, Azure Policy/Defender scope guidance, and Not Evaluable reason codes.
- **1.0.0** -- Initial release. Full coverage of CIS Microsoft Azure Foundations Benchmark v2.1.0 sections 1 through 9.
63 changes: 63 additions & 0 deletions tests/benign/azure-review-tenant-subscription-scope-verified.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
id: azure-review-tenant-subscription-scope-verified
skill: azure-review
expected: benign
description: >
Azure review preserves tenant, management-group, subscription, resource, and
Not Evaluable boundaries before assigning scoped Pass results.
scenario:
review_scope:
claimed_scope: "tenant contoso, management groups platform/prod, 14 active subscriptions"
evidence_inventory:
- name: tenant_conditional_access_export
source_type: live-export
captured: "2026-06-01T09:00:00Z"
coverage: "tenant-wide CA policies, enabled state, included roles, excluded break-glass accounts"
- name: management_group_subscription_denominator
source_type: live-export
captured: "2026-06-01T09:10:00Z"
coverage: "14 active subscriptions, 2 decommissioned subscriptions excluded"
- name: policy_assignment_compliance_export
source_type: policy-assignment
captured: "2026-06-01T09:20:00Z"
coverage: "platform/prod management groups, effects, exemptions, noncompliant resources"
- name: defender_plan_export
source_type: defender-export
captured: "2026-06-01T09:35:00Z"
coverage: "14 subscriptions and required resource-type plans"
- name: resource_graph_private_endpoint_export
source_type: live-export
captured: "2026-06-01T09:50:00Z"
coverage: "storage, Key Vault, SQL, and App Service resource denominators"
reported_findings:
- cis_id: "1.1.2"
title: "Ensure MFA is enabled for all privileged users"
status: Pass
evidence_source: live-export
evidence_captured: tenant_conditional_access_export
scope_coverage: "tenant-wide enabled CA policy; privileged roles included; break-glass exceptions documented"
notes:
- "Report-only policies are excluded from Pass evidence."
- "Break-glass accounts have owner, monitoring, and review date."
- cis_id: "2.1"
title: "Microsoft Defender for Cloud plans"
status: Pass
evidence_source: defender-export
evidence_captured: defender_plan_export
scope_coverage: "14 active subscriptions and required resource types"
notes:
- "Paid plan state is observed, not changed by the reviewer."
- cis_id: "1.2.5"
title: "Ensure SSPR methods required is set to two"
status: Not Evaluable
severity: Informational
evidence_source: unknown
not_evaluable_reason: tenant-live-only
scope_coverage: "SSPR settings export was not supplied"
should_not_trigger:
- "tenant-wide pass from iac-only Conditional Access"
- "all-subscription Defender pass from one pricing resource"
- "policy assignment treated as compliance without exemption export"
expected_result: >
The skill should accept scoped Pass findings where live tenant, management
group, subscription, policy, Defender, and resource evidence are supplied, and
preserve Not Evaluable reason codes for missing tenant-only evidence.
80 changes: 80 additions & 0 deletions tests/vulnerable/azure-review-iac-only-tenant-scope-overclaim.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
id: azure-review-iac-only-tenant-scope-overclaim
skill: azure-review
expected: vulnerable
description: >
IaC-only Conditional Access and one subscription Defender resource are
overclaimed as tenant-wide and enterprise subscription compliance.
scenario:
review_scope:
claimed_scope: "tenant-wide Entra ID and all landing-zone subscriptions"
supplied_evidence:
- path: terraform/identity/conditional-access.tf
source_type: iac-only
scope: "tenant module only"
captured: "2026-06-01"
- path: terraform/subscriptions/prod-a/defender.tf
source_type: iac-only
scope: "subscription prod-a only"
captured: "2026-06-01"
- path: terraform/policies/diagnostics.bicep
source_type: iac-only
scope: "one management group assignment"
captured: "2026-06-01"
missing_evidence:
- tenant_id_and_directory_settings_export
- conditional_access_state_export
- included_excluded_users_roles_and_groups
- break_glass_exception_evidence
- management_group_subscription_denominator
- azure_policy_compliance_and_exemption_export
- defender_plan_live_export_all_subscriptions
- resource_graph_target_resource_denominator
terraform:
conditional_access_policy:
resource_type: azuread_conditional_access_policy
display_name: Require MFA for administrators
state: reportOnly
conditions:
users:
included_roles:
- Global Administrator
excluded_groups:
- break-glass-accounts
grant_controls:
built_in_controls:
- mfa
defender_plan:
resource_type: azurerm_security_center_subscription_pricing
subscription: prod-a
resource_type_name: VirtualMachines
tier: Standard
reported_findings:
- cis_id: "1.1.2"
title: "Ensure MFA is enabled for all privileged users"
status: Pass
evidence_source: iac-only
scope_coverage: "tenant-wide"
problem: "Report-only CA policy and missing assignment/export evidence are treated as enforced tenant coverage."
- cis_id: "2.1"
title: "Microsoft Defender for Cloud plans"
status: Pass
evidence_source: iac-only
scope_coverage: "all subscriptions and resource types"
problem: "One subscription/resource-type pricing resource is treated as all Defender plan coverage."
- cis_id: "5.1.1"
title: "Ensure that a Diagnostic Setting exists"
status: Pass
evidence_source: policy-assignment
scope_coverage: "all subscriptions"
problem: "Policy assignment exists, but compliance state, exemptions, and subscription denominator are absent."
should_trigger:
- AZ-SCOPE-01
- AZ-SCOPE-02
- AZ-SCOPE-03
- AZ-SCOPE-04
- AZ-SCOPE-05
- AZ-SCOPE-07
expected_result: >
The skill should flag the report as overconfident and require live tenant,
management-group, subscription, policy-compliance, and Defender evidence before
tenant-wide or enterprise subscription Pass results are assigned.