[AWS CloudTrail] Remove host.id assignment from target entities — host.id represents the actor, not the target

[alexreal1314]

Motivation
Populating host.id from hostTargets.first() conflates the actor with the target and additionally picks an arbitrary, order-dependent value when multiple target instances are present. The assignment should be removed entirely.

Definition of done

• The field("host.id").set(hostTargets.first()); line is removed from the CloudTrail ingest pipeline.
• host.target.entity.id continues to be populated with the full list of host targets (unchanged).
• No host.id is derived from CloudTrail target entities anywhere in the pipeline.
• Changelog entry added for the aws package.

Team tag

@elastic/cloud-security-posture

[github-actions]

tl;dr: This is a valid, narrowly scoped bug; remove only the CloudTrail host.id assignment from target enrichment, keep host.target.entity.id unchanged, update the five affected expected pipeline fixtures, and add an aws bugfix changelog entry.

Recommendation

Implement this as a reversal of the host.id portion of PR #17827, not as a duplicate/noop. The current pipeline still correctly builds the host target set, but then copies the first sorted target into ECS host.id, which makes a target host look like the document host/actor and drops all but one target.

Findings

Code evidence

packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml:

• :848-851 initializes the type-specific target sets, including hostTargets.
• :884-885 adds ARN targets whose resource type is in hostResourceTypes.
• :905-907 adds simple IDs matching hostIdPrefixes.
• :936-938 currently does both target and host assignment:

if (!hostTargets.isEmpty()) {
  field("host.target.entity.id").set(hostTargets);
  field("host.id").set(hostTargets.first());
}

A local search under packages/aws/data_stream/cloudtrail found no other host.id assignment:

packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml:938:            field("host.id").set(hostTargets.first());

So deleting only field("host.id").set(hostTargets.first()); satisfies the core request while preserving host.target.entity.id at line 937.

Fixtures impacted

These expected pipeline outputs currently include top-level host.id from target classification and should drop only that field while retaining host.target.entity.id:

File	Line
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-get-password-data-json.log-expected.json	55
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-send-command-targets-json.log-expected.json	102
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-send-serial-console-ssh-public-key-json.log-expected.json	63
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-send-ssh-public-key-json.log-expected.json	63
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-start-session-json.log-expected.json	58

Docs/changelog notes

• packages/aws/changelog.yml:137-141 records PR [AWS-cloudtrail] set host.id field #17827: “Set host.id alongside host.target.entity.id in CloudTrail pipeline for entity store host entity resolution.” This issue should add a new bugfix entry at the top, with packages/aws/manifest.yml:4 currently at 6.20.0, so the next patch version is the likely target.
• packages/aws/data_stream/cloudtrail/fields/fields.yml:268 and packages/aws/docs/cloudtrail.md:188 still describe host.target.entity.id as including EC2 instance IDs, EBS volumes, snapshots, and AMI IDs. Current pipeline behavior only classifies EC2 instance targets as host targets; non-host resources fall through to entity.target.id. This is related docs drift from [docs-patrol] Update AWS CloudTrail docs for host/entity target classification change #17897, but [docs-patrol] Update AWS CloudTrail docs for host/entity target classification change #17897’s suggestion to document host.id should not be followed if this issue is implemented.

Verification

Commands run

Static search with the repository search tool confirmed the single CloudTrail host.id assignment above.

I also attempted the expected package pipeline command, but this runner does not have elastic-package installed:

$ cd packages/aws && elastic-package test pipeline -v --data-streams cloudtrail
bash: elastic-package: command not found

The implementation PR should run:

cd packages/aws && elastic-package test pipeline -v --data-streams cloudtrail

Detailed Action Plan

Implementation steps

1. In packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml:936-938, delete only:

field("host.id").set(hostTargets.first());

2. Regenerate or manually update the five expected JSON fixtures listed above so top-level host.id is removed and host.target.entity.id remains present.
3. Add a top-level bugfix entry to packages/aws/changelog.yml and bump packages/aws/manifest.yml from 6.20.0 to the next appropriate patch version.
4. Update the stale host.target.entity.id descriptions in packages/aws/data_stream/cloudtrail/fields/fields.yml and regenerated docs so they do not claim EBS volumes, snapshots, and AMIs are host targets.
5. Run cd packages/aws && elastic-package test pipeline -v --data-streams cloudtrail and confirm the CloudTrail pipeline tests pass.

Related Items

Type	Item	Relevance
PR	#17827	Introduced the current host.id = hostTargets.first() assignment and related fixture updates.
Issue	#17897	Related docs drift; partially superseded because host.id should be removed, not documented.
File	packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml:936-938	Exact pipeline block to change.
File	packages/aws/changelog.yml:137-141	Historical entry for the behavior being reversed.

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.