Recorded Future Identity: Migrate from Data Collector API#14495
Open
aommm wants to merge 7 commits into
Open
Conversation
Renamed to azuredeploy-incident-creation-analytic-rule.json and moved to Data Connectors/ to pass DetectionTemplateSchemaValidation which requires all files in Analytic Rules/ to be .yaml. Updated deploy button URL in Playbooks/readme.md to match new path.
Adds the custom table definition so KqlValidations CI passes (KS204).
The ARM engine accepts string expressions (e.g. "[parameters('flag')]")
for the resource condition field — this is the standard pattern for
conditional resource deployment in ARM templates. The schema validator
was incorrectly limiting condition to literal boolean values only,
causing PlaybooksValidations CI to fail on any template that uses an
ARM expression for condition (e.g. role assignments gated on a
create_role_assignment parameter).
This change aligns the schema with actual ARM behavior by accepting
both boolean and string types for the resource-level condition field.
The output-level condition (a separate definition) is unchanged.
This was referenced Jun 16, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Migrates the Recorded Future Identity alert-ingestion path from the deprecated Log Analytics Data Collector connector to the Azure Monitor Logs Ingestion API (DCE/DCR), and updates deployment/docs to support the new ingestion + incident creation flow.
Changes:
- Added shared DCE/DCR + custom table + Sentinel connector tile ARM template for the alert importer flow.
- Updated
RFI-Playbook-Alert-Importer-LAWto ingest via MSI-authenticated HTTP call to the Logs Ingestion API and optionally assign required RBAC. - Updated the incident-creation analytic rule + playbooks documentation to use the new
_V2table/field names.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| Solutions/Recorded Future Identity/Playbooks/readme.md | Documents migration steps, new deployment option flow, and new table name. |
| Solutions/Recorded Future Identity/Playbooks/RFI-Playbook-Alert-Importer-LAW/azuredeploy.json | Switches playbook ingestion to Logs Ingestion API and adds optional RBAC assignment. |
| Solutions/Recorded Future Identity/Data Connectors/azuredeploy-incident-creation-analytic-rule.json | Adds an ARM-deployable NRT analytic rule targeting the new table. |
| Solutions/Recorded Future Identity/Data Connectors/azuredeploy-alert-importer.json | Adds shared DCE/DCR/table and a Sentinel “Data Connector” tile for the playbook ingestion. |
| Solutions/Recorded Future Identity/Analytic Rules/IncidentCreation/RecordedFutureIdentityExposure.yaml | Updates table/field references and bumps rule version. |
| .script/utils/schemas/ARM_DeploymentTemplateSchema.json | Changes the internal ARM template JSON schema for resource.condition. |
| .script/tests/KqlvalidationsTests/CustomTables/RFI_PlaybookAlertResults_V2_CL.json | Adds a custom table schema for KQL validation tests. |
Comment on lines
+279
to
+285
| "title": "Step 1 \u2014 Deploy Data Connectors infrastructure", | ||
| "description": "Deploys the shared Data Collection Endpoint (DCE), Data Collection Rule (DCR), Log Analytics table (`RFI_PlaybookAlertResults_V2_CL`), and this connector definition tile.\n\n[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Frecordedfuture%2FAzure-Sentinel%2FRFPD-77178-log-ingestion-api%2FSolutions%2FRecorded%20Future%20Identity%2FData%20Connectors%2Fazuredeploy-alert-importer.json)" | ||
| }, | ||
| { | ||
| "title": "Step 2 \u2014 Deploy the RFI-Playbook-Alert-Importer-LAW playbook", | ||
| "description": "Deploys the Logic App that imports Recorded Future Identity Playbook Alerts and writes them to the Log Analytics table via the Logs Ingestion API using Managed Identity.\n\n[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Frecordedfuture%2FAzure-Sentinel%2FRFPD-77178-log-ingestion-api%2FSolutions%2FRecorded%20Future%20Identity%2FPlaybooks%2FRFI-Playbook-Alert-Importer-LAW%2Fazuredeploy.json)\n\nAfter deployment, open the Logic App and authorize the different connectors, then enable the Logic App." | ||
| } |
Comment on lines
+40
to
+49
| "variables": { | ||
| "DceName": "recorded-future-identity-dce", | ||
| "DcrName": "recorded-future-identity-dcr-playbook-alerts", | ||
| "TableName": "RFI_PlaybookAlertResults_V2_CL", | ||
| "StreamName": "Custom-RFI_PlaybookAlertResults_V2_CL", | ||
| "workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces', parameters('log_analytics_workspace_name'))]", | ||
| "dceResourceId": "[resourceId('Microsoft.Insights/dataCollectionEndpoints', variables('DceName'))]", | ||
| "dcrResourceId": "[resourceId('Microsoft.Insights/dataCollectionRules', variables('DcrName'))]", | ||
| "tableResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces/tables', parameters('log_analytics_workspace_name'), variables('TableName'))]" | ||
| }, |
Comment on lines
21
to
23
| "support": { | ||
| "tier": "Partner" | ||
| }, |
Comment on lines
+8
to
10
| "Deploy the Data Connectors infrastructure (DCE, DCR, Log Analytics table, and connector definition) by deploying the azuredeploy-alert-importer.json template from the Data Connectors folder before deploying this playbook.", | ||
| "First install the RFI-CustomConnector-0-2-0 custom connector", | ||
| "To use the Recorded Future Identity connector, you will need a valid API token from Recorded Future as described in the [documentation](https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/Recorded%20Future%20Identity/Playbooks/readme.md#how-to-obtain-recorded-future-api-token)" |
Comment on lines
+294
to
306
| "type": "Http", | ||
| "inputs": { | ||
| "host": { | ||
| "connection": { | ||
| "name": "@parameters('$connections')['azureloganalyticsdatacollector']['connectionId']" | ||
| } | ||
| }, | ||
| "method": "post", | ||
| "body": "@string(body('Playbook_Alerts_-_Detailed_Identity_Novel_Exposures_alert_data')?['result'])", | ||
| "method": "POST", | ||
| "uri": "@{parameters('DceEndpoint')}/dataCollectionRules/@{parameters('DcrImmutableId')}/streams/@{parameters('StreamName')}?api-version=2023-01-01", | ||
| "headers": { | ||
| "Log-Type": "@parameters('playbook_alert_log_analytics_custom_log_name')" | ||
| "Content-Type": "application/json" | ||
| }, | ||
| "path": "/api/logs" | ||
| "body": "@createArray(body('Playbook_Alerts_-_Detailed_Identity_Novel_Exposures_alert_data')?['result'])", | ||
| "authentication": { | ||
| "type": "ManagedServiceIdentity", | ||
| "audience": "https://monitor.azure.com" | ||
| } | ||
| } |
Comment on lines
+44
to
+46
| "type": "Microsoft.OperationalInsights/workspaces/providers/alertRules", | ||
| "name": "[concat(parameters('workspace'), '/Microsoft.SecurityInsights/b1c2d3e4-5678-90ab-cdef-444444444444')]", | ||
| "apiVersion": "2023-12-01-preview", |
Comment on lines
71
to
76
| "description": "Resource type" | ||
| }, | ||
| "condition": { | ||
| "type": "boolean", | ||
| "type": ["boolean", "string"], | ||
| "description": "Condition of the resource" | ||
| }, |
Comment on lines
+1
to
+11
| { | ||
| "Name": "RFI_PlaybookAlertResults_V2_CL", | ||
| "Properties": [ | ||
| { | ||
| "Name": "TimeGenerated", | ||
| "Type": "DateTime" | ||
| }, | ||
| { | ||
| "Name": "playbook_alert_id", | ||
| "Type": "String" | ||
| }, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change(s):
RFI-Playbook-Alert-Importer-LAWlogic app to import data via DCE/DCRReason for Change(s):
Version Updated:
In this PR we've only migrated one of the logic apps of Recorded Future Identity.
We would be glad to hear your feedback on our approach.
We're planning to submit further PRs later for the rest of the logic apps, as well as for our main solution (which will be a lot bigger).