[BUG] Monitor actions (indices:monitor/stats, cluster:monitor/health) denied under securitytenant scope when * resolves to tenant .kibana_* indices — regression from #6202

[SuZhou-Joe]

What is the bug?

When a request carries a securitytenant header (which OpenSearch Dashboards always sends when multitenancy is enabled), cluster/index monitor actions whose index pattern (*) resolves to a set that incidentally includes tenant .kibana_* indices are denied with a security_exception, even for the admin superuser.

Concretely, GET _list/indices (and the underlying IndicesStatsRequest / ClusterHealthRequest) returns 403:

no permissions for [indices:monitor/stats] and User [name=admin, backend_roles=[admin], requestedTenant=__user__]
  suppressed: no permissions for [cluster:monitor/health] and User [name=admin, backend_roles=[admin], requestedTenant=__user__]

This is a regression between 3.7.0 and 3.8.0. In 3.7.0 the same request succeeds (200); in 3.8.0 it returns 403.

Note _cat/indices was fixed by #6284, but _list/indices and the IndicesStatsRequest/ClusterHealthRequest monitor actions are still affected because the * is expanded to concrete indices before the interceptor sees it — so #6284's getOriginalRequested() change does not help those actions (its "original requested" already contains the resolved .kibana_* indices).

Real-world impact: this surfaced as a release integration-test failure for dashboards-assistant (OpenSearch Dashboards 3.8.0, with-security). The ml-commons ListIndexTool (used by the chatbot) mirrors _list/indices and issues IndicesStatsRequest + ClusterHealthRequest. Because Dashboards proxies the request with the user's securitytenant header, those monitor calls are denied, the tool returns an empty result, and the assistant answers with no index data. Any agent/tool that lists indices via the monitor APIs while a tenant header is present is broken.

How can one reproduce the bug?

Steps to reproduce the behavior:

1. Start OpenSearch 3.8.0 with the security demo configuration (multitenancy enabled by default).
2. Log in to Dashboards at least once so a .kibana_<tenant> index exists (e.g. .kibana_92668751_admin_1).
3. As admin, issue a _list/indices request with a tenant header:

   # 3.8.0 -> 403
   curl -sk -u admin:<password> -H "securitytenant: __user__" "https://localhost:9200/_list/indices"
   
   # same request WITHOUT the tenant header -> 200
   curl -sk -u admin:<password> "https://localhost:9200/_list/indices"

4. See the 403 security_exception for [indices:monitor/stats] / [cluster:monitor/health]. securitytenant: Global and securitytenant: Private reproduce it as well.

What is the expected behavior?

admin (and users with cluster_monitor / all-indices monitor permissions) should be able to run monitor actions such as indices:monitor/stats and cluster:monitor/health regardless of the securitytenant header. A tenant header should not cause a broad, read-only monitor request to be denied just because the resolved index set incidentally includes other tenants' .kibana_* indices — this matches the behavior in 3.7.0 and the intent of #6284 for _cat/indices.

What is your host/environment?

• OS: Linux x64
• Version: OpenSearch 3.8.0 (distribution build 12101), tar distribution. Regressed relative to OpenSearch 3.7.0.
• Plugins: security 3.8 (commit 30e36a1742ae1bf339d8c5b3681387e31b9dcfcb); compared against security 3.7.0.0 (tag fbe05437cd405392827d9bcbde8b0371e5885cef)

Do you have any screenshots?

Not applicable (API-level failure). DEBUG trace (logger.org.opensearch.security.privileges=DEBUG) for the denied indices:monitor/stats call on 3.8:

Action: indices:monitor/stats (IndicesStatsRequest)
IndexResolverReplacer: Resolved pattern [...] to indices: [..., .kibana_1, .kibana_92668751_admin_1, ...]
RequestedResolved: allIndices=[..., .kibana_1, .kibana_92668751_admin_1, ...],
                   originalRequested=[..., .kibana_1, .kibana_92668751_admin_1, ...]
Result from privileges interceptor: ReplaceResult{continueEvaluation=false, accessDenied=true, ...}

For comparison, indices:monitor/settings/get and cluster:monitor/state in the same request keep allIndices=[*] / originalRequested=[*] (not pre-expanded), so isLocalAll() is true, the interceptor returns continueEvaluation=true, and they are allowed. The problem is specific to the actions whose * is expanded to concrete indices before privilege evaluation.

Do you have any additional context?

Bisected to PR #6202 ("Handle Dashboards multitenancy for m* cluster actions in legacy evaluator", merge commit 0004de08beb382699e738f265462e55265ef5680, merged 2026-06-08). It is present in the 3.8 build and absent from the 3.7.0.0 tag. Two changes in that PR combine to produce the denial:

1. PrivilegesEvaluatorImpl (legacy evaluator) added a missing return PrivilegesEvaluatorResponse.insufficient(action0) in the cluster-permission path (around PrivilegesEvaluatorImpl.java:435). Previously this denied path fell through and the request was allowed.
2. PrivilegesInterceptor.replaceDashboardsIndex() added a check that returns ACCESS_DENIED_REPLACE_RESULT when the resolved indices include any .kibana_* (dashboards tenant) index.

Suggested direction: allow read-only monitor actions to be evaluated as "local all" (or otherwise skip the cross-tenant .kibana_* denial) when the request is a broad/wildcard monitor request rather than an explicit reference to another tenant's .kibana_* index — extending the #6284 fix to cover the pre-expanded monitor actions.

[Hailong-am]

@cwperks Got a new one from index refresh

{
  "ok": false,
  "error": "Authorization Exception",
  "body": {
    "_shards": {
      "total": 16,
      "successful": 14,
      "failed": 2,
      "failures": [
        {
          "shard": 0,
          "index": ".kibana_1",
          "status": "FORBIDDEN",
          "reason": {
            "type": "security_exception",
            "reason": "no permissions for [indices:admin/refresh[s]] and User [name=admin, backend_roles=[admin], requestedTenant=__user__]"
          }
        }
      ]
    }
  }
}

[cwperks]

Thank you @Hailong-am and @SuZhou-Joe for reporting these issues. I have created a tactical PR to unblock the release, but all of this is a result of OpenSearch Dashboards not truly treating the .kibana alias (and .kibana_* concrete indices as system indices)

All of these issues would not have occurred if .kibana_* indices were hidden indices as they would not be resolved by the request unless ?expand_wildcards=all was passed as a url param.

IMO over the longer term we need to figure out how dashboards can only send securitytenant header when accessing the .kibana alias as its not relevant to other requests. Semantically, the security plugin replaces the .kibana alias with the corresponding tenant index behind the scenes so unless the request is to .kibana alias the securitytenant header is not needed.