crowdstrike: add support for NG-SIEM Correlation Detection events

[navnit-elastic]

Proposed commit message

crowdstrike: add support for NG-SIEM Correlation Detection events

- Parse NG-SIEM correlation-detection alerts in the alert data stream.
  Add ECS categorization, field definitions and pipeline tests.
- Populate event.duration ECS field from event.start and event.end
  fields in the Alert data streams.

Test samples are derived from live CrowdStrike instance.

Note

Acceptance criteria

• event.severity, event.start, event.end, and event.duration are populated - event.severity is derived from severity_name to align with other CrowdStrike alert types.
• host.name, user.name, and rule.name are populated for NG-SIEM events - host.name is derived from host_names[], user.name is derived from user_names[] (fall back to users[].user_name), and rule.name is derived from display_name.
• related.hosts, related.user, and related.ip are populated
• rule.id is populated from correlation_rule_id
• All correlation_rule_* fields land under crowdstrike.alert.*
• json.* is removed at the end of pipeline processing - this is not required as the pipeline renames the json.* object to crowdstrike.alert.*
• Pipeline test added with expected output validated against sample events
• Field definitions added to fields.yml for any new crowdstrike.alert.* fields introduced
• Existing EPP alert pipeline tests continue to pass

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

• Clone integrations repo.
• Install elastic package locally.
• Start elastic stack using elastic-package.
• Move to integrations/packages/crowdstrike directory.
• Run the following command to run tests.

elastic-package test pipeline -d alert

Related issues

• Closes [CrowdStrike] Add support for NG-SIEM in alert pipeline #19048

Screenshots

[github-actions]

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[jamiehynds]

@navnit-elastic this is the incorrect event.category can you adjust to event.category:alert

[navnit-elastic]

@jamiehynds, Would below event categorization meet expectation?

event.kind: alert
event.category: threat
event.type: indicator

[jamiehynds]

@navnit-elastic event.kind: alert is correct but the category and type fields are incorrect as they're more suited to IOC's from Threat Intel feeds.

Can we set to:
event.kind: alert
event.type: info

[navnit-elastic]

@jamiehynds, agreed. event.kind: alert set by default.yml, adjusted correlation_detection.yml to set only event.type: info.

[jamiehynds]

can drop this as we're adjusting the event.category to alert

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

Package crowdstrike 👍(7) 💚(5) 💔(2)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
falcon	6802.72	3133.81	-3668.91 (-53.93%)	💔
falcon	6802.72	5524.86	-1277.86 (-18.78%)	💔

To see the full report comment with /test benchmark fullreport

[jamiehynds]

Is the event.severity field being mapped with this pipeline? If not, can we ensure its mapped and aligned to the other CrowdStrike events? Related issue for context: https://github.com/elastic/dev/issues/3018

[navnit-elastic]

Is the event.severity field being mapped with this pipeline? If not, can we ensure its mapped and aligned to the other CrowdStrike events? Related issue for context: https://github.com/elastic/dev/issues/3018

@jamiehynds, The event.severity is set in default.yml from severity_name to align with other CrowdStrike alert types.

[kcreddy]

LGTM for my comments.

[elastic-vault-github-plugin-prod]

✅ All changelog entries have the correct PR link.

[github-actions]

TL;DR

Buildkite build 44534 failed before tests because the repository post-checkout hook could not merge commit f38f65a87b601ec9170319e60ceae48b8df96715 with main: packages/crowdstrike/changelog.yml had a content conflict. The current PR head 97d19ae4e36dd564ce3948703df553270ca49550 appears to have already addressed this by moving the PR changelog entries to a new 3.25.0 section, so the next action is to let the replacement Buildkite build 44535 complete.

Remediation

• Keep the PR changelog entries in their own top-level 3.25.0 section above the existing 3.24.0 entry from crowdstrike: make max_executions configurable for CEL data streams #19530.
• Validate with the new Buildkite run for 97d19ae4e36dd564ce3948703df553270ca49550; if it fails, investigate that newer build's logs rather than build 44534.

Investigation details

Root Cause

This is a merge-conflict/configuration failure during Buildkite checkout, not a package test failure. In build 44534, the post-checkout hook tried to merge the PR commit into main at a800785d9a31333b3c81c56e5c99848db7dfe2b5 and failed on packages/crowdstrike/changelog.yml.

At main commit a800785d9a31333b3c81c56e5c99848db7dfe2b5, packages/crowdstrike/changelog.yml starts with the existing 3.24.0 entry from #19530 at lines 2-6:

2 - version: "3.24.0"
3   changes:
4     - description: Make max_executions configurable for all CEL data streams to allow high-volume environments to complete pagination within a single interval.
5       type: enhancement
6       link: https://github.com/elastic/integrations/pull/19530

The failed PR commit f38f65a87b601ec9170319e60ceae48b8df96715 also edited the top 3.24.0 block at lines 2-9 for this PR's changes, so Git could not auto-merge the same changelog section.

The current PR head 97d19ae4e36dd564ce3948703df553270ca49550 now has the expected ordering: this PR's entries under 3.25.0 at lines 2-9, followed by the existing 3.24.0 entry at lines 10-14.

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/44534
• Job/step: :pipeline::arrow_up: Upload Pipeline: .buildkite/pipeline.yml
• Key log excerpt:

Auto-merging packages/crowdstrike/changelog.yml
CONFLICT (content): Merge conflict in packages/crowdstrike/changelog.yml
Automatic merge failed; fix conflicts and then commit the result.
Merge failed: 1
Error: running "repository post-checkout" shell hook: The repository post-checkout hook exited with status 1

Verification

• Checked the failed Buildkite log and compared packages/crowdstrike/changelog.yml at main (a800785d9a31333b3c81c56e5c99848db7dfe2b5), failed commit f38f65a87b601ec9170319e60ceae48b8df96715, and current PR head 97d19ae4e36dd564ce3948703df553270ca49550.
• Did not run package tests because the failed Buildkite job stopped before pipeline upload/test execution; the relevant validation is whether Buildkite build 44535 for the updated head passes checkout and pipeline upload.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 97d19ae

History

• 💔 Build #44534 failed f38f65a
• 💚 Build #44480 succeeded 2b0fc3d
• 💚 Build #44476 succeeded 35c30d5
• 💚 Build #44466 succeeded 06d00e9
• 💚 Build #44406 succeeded 0db1c8d

cc @navnit-elastic

[efd6]

LGTM, but I'd like to know the answer. Also, please wait for @jamiehynds.