[Streams] Create Import from other Stream flow

[SoniaSanzV]

Closes #267913

Summary

• Adds an “Import from another stream” flow to reuse another stream’s lifecycle configuration from the Data lifecycle tab.
• Keeps import behavior privilege-aware: the import action is only available when the target stream can update both successful data lifecycle and failure store settings.
• Avoids partial imports from sources where failure store cannot be read: source streams without read_failure_store are excluded from import candidates.

Test plan

Run this setup in Dev Tools as an admin user:

PUT _ilm/policy/logs-import-ilm-downsample-policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": { "max_primary_shard_size": "50gb" }
        }
      },
      "warm": {
        "min_age": "1d",
        "actions": {
          "downsample": { "fixed_interval": "1h" }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": { "delete": {} }
      }
    }
  }
}

PUT _index_template/logs-import-tsdb-template
{
  "index_patterns": ["logs.import-tsdb-*"],
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series",
      "index.routing_path": ["service.name"],
      "index.lifecycle.prefer_ilm": false
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "service.name": { "type": "keyword", "time_series_dimension": true },
        "message": { "type": "match_only_text" }
      }
    }
  }
}

PUT _index_template/logs-import-standard-template
{
  "index_patterns": ["logs.import-standard-*"],
  "data_stream": {},
  "template": {
    "settings": {
      "index.lifecycle.prefer_ilm": false
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "message": { "type": "match_only_text" }
      }
    }
  }
}

POST kbn:/internal/streams/_create_classic
{
  "name": "logs.import-tsdb-source-downsample",
  "description": "DSL source with downsampling",
  "ingest": {
    "lifecycle": {
      "dsl": {
        "data_retention": "30d",
        "downsample": [{ "after": "1d", "fixed_interval": "1h" }]
      }
    },
    "processing": { "steps": [] },
    "settings": {},
    "failure_store": { "lifecycle": { "enabled": { "data_retention": "14d" } } },
    "classic": {}
  }
}

POST kbn:/internal/streams/_create_classic
{
  "name": "logs.import-tsdb-source-ilm-downsample",
  "description": "ILM source with downsampling",
  "ingest": {
    "lifecycle": { "ilm": { "policy": "logs-import-ilm-downsample-policy" } },
    "processing": { "steps": [] },
    "settings": {},
    "failure_store": { "lifecycle": { "enabled": { "data_retention": "21d" } } },
    "classic": {}
  }
}

POST kbn:/internal/streams/_create_classic
{
  "name": "logs.import-tsdb-target",
  "description": "Time series target",
  "ingest": {
    "lifecycle": { "dsl": { "data_retention": "7d" } },
    "processing": { "steps": [] },
    "settings": {},
    "failure_store": { "lifecycle": { "enabled": { "data_retention": "7d" } } },
    "classic": {}
  }
}

POST kbn:/internal/streams/_create_classic
{
  "name": "logs.import-standard-target",
  "description": "Standard target",
  "ingest": {
    "lifecycle": { "dsl": { "data_retention": "7d" } },
    "processing": { "steps": [] },
    "settings": {},
    "failure_store": { "lifecycle": { "enabled": { "data_retention": "7d" } } },
    "classic": {}
  }
}

Then:

• Open Streams and navigate to logs.import-tsdb-target.
• Go to the Data lifecycle tab.
• Open the tab actions menu and click Import from another stream.
• Select logs.import-tsdb-source-downsample and verify the import option shows the downsampling summary.
• Click Apply and verify the target stream imports retention, downsampling, and failure store settings.
• Reopen the import flyout, select logs.import-tsdb-source-ilm-downsample, inspect the ILM policy, and verify the policy includes a downsample action.
• Apply the ILM source and verify the target stream uses logs-import-ilm-downsample-policy.
• Navigate to logs.import-standard-target, import from logs.import-tsdb-source-downsample, and verify downsampling is omitted for the non-time-series target.
• Optional privilege check: test with a user that has lifecycle privileges but no manage_failure_store; verify Import from another stream is not shown.

Evidence

Screen.Recording.2026-07-10.at.12.50.01.mov

Release note

Adds the ability to import another stream's lifecycle configuration from the Streams data lifecycle tab.

[github-actions]

One reliability concern on the ILM policy fetch effect (potential infinite refetch loop when the cluster has no ILM policies or the request fails) — details inline. Otherwise the flow is well-structured and the pure helpers have good unit coverage.

Generated by Claude Reviewer for issue #275948 · 101.7 AIC · ⌖ 10 AIC · ⊞ 3.9K

[ElenaStoeva]

Great work overall @SoniaSanzV! I tested locally and the UI works well. I have a few concerns/questions:

1. On logs.import-standard-target, importing logs.import-tsdb-source-ilm-downsample shows the warning “downsampling steps from the selected ILM policy will be excluded,” but after applying, the target is assigned logs-import-ilm-downsample-policy unchanged. That policy still contains warm.actions.downsample, and ES reports it in use by the target stream's backing index.
   Code-wise, getImportedLifecycle() always returns { ilm: { policy } } for ILM sources regardless of targetIsTimeSeries, so there is no exclusion step for ILM policies. We could either block/down-rank ILM policies with downsampling for standard targets, or change the warning/copy to make clear the policy is applied as-is and may be invalid for non-TSDB streams.

Screen.Recording.2026-07-13.at.23.01.54.mov

2. After I apply an import, I see the value in the Lifecycle summary section flickering:

Screen.Recording.2026-07-13.at.18.35.15.mov

3. Can you clarify how you create the role for testing with a user that shouldn't see the import action? I created a user that can manage ilm but not failure store, but this doesn't display the stream details at all, so maybe I'm missing something.

4. I see there is a backport label for version 9.5.0 but the FF was last week so not sure if we can merge this for 9.5 - was this the plan?

[SoniaSanzV]

Thank you for the feedback @ElenaStoeva!!

1. On logs.import-standard-target, importing logs.import-tsdb-source-ilm-downsample shows the warning “downsampling steps from the selected ILM policy will be excluded,” but after applying, the target is assigned logs-import-ilm-downsample-policy unchanged. That policy still contains warm.actions.downsample, and ES reports it in use by the target stream's backing index.
   Code-wise, getImportedLifecycle() always returns { ilm: { policy } } for ILM sources regardless of targetIsTimeSeries, so there is no exclusion step for ILM policies. We could either block/down-rank ILM policies with downsampling for standard targets, or change the warning/copy to make clear the policy is applied as-is and may be invalid for non-TSDB streams.

this is a wording issue. The policy won't be modified when applying, but the downsampling won't have any effect on that Stream. I'll consult with docs a better copy. But this callout was introduced before with the component itself so i'll handle that outside this PR

2. After I apply an import, I see the value in the Lifecycle summary section flickering:

This is an issue that already existed and that we need to address, but it is not within the scope of this PR. I'm going to try to add this to this PR to improve transitions in general #277149

3. Can you clarify how you create the role for testing with a user that shouldn't see the import action? I created a user that can manage ilm but not failure store, but this doesn't display the stream details at all, so maybe I'm missing something.

I used:

PUT _security/role/streams_lifecycle_no_failure_store
{
  "cluster": [
    "monitor",
    "read_ilm",
    "manage_index_templates",
    "manage_ingest_pipelines",
    "manage_pipeline",
    "read_pipeline",
    "monitor_text_structure"
  ],
  "indices": [
    {
      "names": ["logs*"],
      "privileges": [
        "read",
        "monitor",
        "view_index_metadata",
        "manage_data_stream_lifecycle",
        "manage_ilm"
      ]
    }
  ]
}
POST _security/user/streams_test_user
{
  "password": "changeme",
  "roles": ["streams_lifecycle_no_failure_store", "kibana_admin"]"
}

I also made some privileged improvement here c546c95

4. I see there is a backport label for version 9.5.0 but the FF was last week so not sure if we can merge this for 9.5 - was this the plan?

Yes, this was meant to go into 9.5.0 and since we didn't finish this on time is supposed to go into next 9.5.x, but 9.5.0 is the only tag so far, I guess in case there is another BC?

[ElenaStoeva]

Thanks for addressing my feedback @SoniaSanzV! Updating the ILM downsampling warning copy sounds good to me - I'll leave this up to you to update in this PR or in a follow-up PR if you want to merge this first.

Regarding the backport label, let's first check if there are more BC's for 9.5.0 and if it's okay to merge this as is, given that this is a feature PR and usually we should only merge fixes after FF.

Code changes lgtm. Just left a nit about one of the tests but approving to unblock the PR.

[ElenaStoeva]

nit: Could we add a focused test that documents the intended ILM behavior for non-time-series targets? The DSL path has coverage for dropping downsample when targetIsTimeSeries is false, but the ILM path intentionally keeps only the policy reference unchanged. A test name like “keeps ILM policy references unchanged for non-time-series targets” would make that distinction explicit and protect the copy/behavior contract.

[SoniaSanzV]

Added in 335fa6f

[kibanamachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 8c74a6f
• Build duration: 47 mins

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
streamsApp	2656	2666	+10

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
streamsApp	2.2MB	2.2MB	+13.8KB

History

• 💔 Build #470080 failed 4d69fbd
• 💔 Build #469424 failed 3ebe126
• 💔 Build #469178 failed 634ad4e
• 💛 Build #468809 was flaky 335fa6f
• 💛 Build #468461 was flaky c546c95

cc @SoniaSanzV

[juliaElastic]

LGTM

[kibanamachine]

Starting backport for target branches: 9.5

https://github.com/elastic/kibana/actions/runs/29483068436

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	9.5	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 275948

Questions ?

Please refer to the Backport tool documentation

[SoniaSanzV]

💚 All backports created successfully

Status	Branch	Result
✅	9.5

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 275948 locally
cc: @SoniaSanzV

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 275948 locally
cc: @SoniaSanzV

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 275948 locally
cc: @SoniaSanzV

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 275948 locally
cc: @SoniaSanzV

[kibanamachine]

Starting backport for target branches: 9.5

https://github.com/elastic/kibana/actions/runs/30081164755

[kibanamachine]

Starting backport for target branches: 9.5

https://github.com/elastic/kibana/actions/runs/30081165262

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	9.5	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 275948

Questions ?

Please refer to the Backport tool documentation

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	9.5	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 275948

Questions ?

Please refer to the Backport tool documentation

[damian-polewski]

💚 All backports created successfully

Status	Branch	Result
✅	9.5

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation