Recorded Future Identity: Migrate from Data Collector API#14479
Closed
aommm wants to merge 4 commits into
Closed
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR migrates the Recorded Future Identity “Alert Importer” ingestion path away from the deprecated Log Analytics Data Collector connector to the Azure Monitor Logs Ingestion API (DCE/DCR), and updates related deployment/docs assets.
Changes:
- Added a shared ARM template to deploy DCE/DCR + custom table + connector tile for the new ingestion pipeline.
- Updated
RFI-Playbook-Alert-Importer-LAWplaybook to write to Log Ingestion API using MSI and optional role assignment. - Updated incident-creation analytic rule artifacts to query the new
RFI_PlaybookAlertResults_V2_CLtable/schema.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| Solutions/Recorded Future Identity/Playbooks/readme.md | Documents the Log Ingestion API migration, new deployment flow (Option A/B), and updated table name. |
| Solutions/Recorded Future Identity/Playbooks/RFI-Playbook-Alert-Importer-LAW/azuredeploy.json | Switches playbook ingestion from Data Collector connector to Log Ingestion API + MSI, adds optional RBAC assignment and DCE/DCR references. |
| Solutions/Recorded Future Identity/Data Connectors/azuredeploy-alert-importer.json | Introduces shared DCE/DCR/table deployment and a Sentinel data connector tile with queries/instructions. |
| Solutions/Recorded Future Identity/Analytic Rules/IncidentCreation/azuredeploy.json | Adds an ARM deployable NRT analytic rule targeting the new table. |
| Solutions/Recorded Future Identity/Analytic Rules/IncidentCreation/RecordedFutureIdentityExposure.yaml | Updates the rule to use the new table and updated field names; bumps version. |
Comments suppressed due to low confidence (1)
Solutions/Recorded Future Identity/Data Connectors/azuredeploy-alert-importer.json:1
- The connector tile instructions link to a
recordedfuture/Azure-Sentinelfork + branch (RFPD-77178-log-ingestion-api) rather than the canonicalAzure/Azure-Sentinelrepo path used elsewhere in this PR. This will break deployments for users once the branch/fork is unavailable (and will also diverge from the marketplace/solution packaging expectation). Update these URLs to point to the merged location inhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/...(or the correct default branch) for both Step 1 and Step 2.
{
Comment on lines
+242
to
+249
| "connectivityCriteria": [ | ||
| { | ||
| "type": "IsConnectedQuery", | ||
| "value": [ | ||
| "RFI_PlaybookAlertResults_V2_CL\n| summarize LastLog = max(TimeGenerated)\n| where LastLog >= ago(24h)" | ||
| ] | ||
| } | ||
| ], |
Comment on lines
+45
to
+53
| "name": "[concat(parameters('workspace'), '/Microsoft.SecurityInsights/b1c2d3e4-5678-90ab-cdef-444444444444')]", | ||
| "apiVersion": "2023-12-01-preview", | ||
| "kind": "NRT", | ||
| "properties": { | ||
| "displayName": "Recorded Future Identity - Credential Exposure Detected", | ||
| "description": "Creates incidents when Recorded Future Identity detects compromised credentials for users in your organization.", | ||
| "severity": "High", | ||
| "enabled": true, | ||
| "query": "RFI_PlaybookAlertResults_V2_CL\n| where TimeGenerated >= now(-15m)", |
Comment on lines
+15
to
+34
| RFI_PlaybookAlertResults_V2_CL | ||
| | where TimeGenerated >= now(-15m) | ||
| eventGroupingSettings: | ||
| aggregationKind: AlertPerResult | ||
| alertDetailsOverride: | ||
| alertDisplayNameFormat: 'Identity Exposure: {{panel_status_entity_name_s}} with priority: {{panel_status_priority_s}}' | ||
| alertDisplayNameFormat: 'Identity Exposure: {{panel_status_entity_name}} with priority: {{panel_status_priority}}' | ||
| alertDescriptionFormat: | | ||
| _Recorded Future Identity Alert_ | ||
|
|
||
| **Rule Name:** {{panel_status_alert_rule_name_s}} | ||
| **Rule Name:** {{panel_status_alert_rule_name}} | ||
|
|
||
| **Alert ID:** {{playbook_alert_id_s}} | ||
| **Alert ID:** {{playbook_alert_id}} | ||
|
|
||
| **Evidence Summary:** {{alert_description_s}} | ||
| **Evidence Summary:** {{alert_description}} | ||
|
|
||
| Investigate this identity by searching in log analytics workspace for the Alert ID. | ||
| alertDynamicProperties: [] | ||
| customDetails: | ||
| RFLabel: Type | ||
| AlertId: playbook_alert_id_s | ||
| AlertId: playbook_alert_id |
Comment on lines
+8
to
10
| "Deploy the Data Connectors infrastructure (DCE, DCR, Log Analytics table, and connector definition) by deploying the azuredeploy-alert-importer.json template from the Data Connectors folder before deploying this playbook.", | ||
| "First install the RFI-CustomConnector-0-2-0 custom connector", | ||
| "To use the Recorded Future Identity connector, you will need a valid API token from Recorded Future as described in the [documentation](https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/Recorded%20Future%20Identity/Playbooks/readme.md#how-to-obtain-recorded-future-api-token)" |
Comment on lines
+296
to
+305
| "method": "POST", | ||
| "uri": "@{parameters('DceEndpoint')}/dataCollectionRules/@{parameters('DcrImmutableId')}/streams/@{parameters('StreamName')}?api-version=2023-01-01", | ||
| "headers": { | ||
| "Log-Type": "@parameters('playbook_alert_log_analytics_custom_log_name')" | ||
| "Content-Type": "application/json" | ||
| }, | ||
| "path": "/api/logs" | ||
| "body": "@createArray(body('Playbook_Alerts_-_Detailed_Identity_Novel_Exposures_alert_data')?['result'])", | ||
| "authentication": { | ||
| "type": "ManagedServiceIdentity", | ||
| "audience": "https://monitor.azure.com" | ||
| } |
Contributor
Author
|
Sorry, had to re-create this PR to get the changes to show properly: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change(s):
RFI-Playbook-Alert-Importer-LAWlogic app to import data via DCE/DCRReason for Change(s):
Version Updated:
In this PR we've only migrated one of the logic apps of Recorded Future Identity.
We would be glad to hear your feedback on our approach.
We're planning to submit further PRs later for the rest of the logic apps, as well as for our main solution (which will be a lot bigger).