Add pagination support to Security configuration collection APIs

[cwperks]

Description

The Security configuration collection APIs currently return every caller-visible configuration entity in a single response. Large deployments may have thousands of internal users, roles, role mappings, action groups, tenants, or node-DN groups, which can produce large payloads and add memory, serialization, and client-side processing overhead.

OpenSearch introduced a common cursor-based pagination model for collection APIs in OpenSearch#14641 and OpenSearch#14718. The Security plugin should adopt a consistent version of that model for its configuration collection APIs.

Proposed scope

Add opt-in pagination to collection GETs for:

• /_plugins/_security/api/internalusers (and the legacy /user alias)
• /_plugins/_security/api/roles
• /_plugins/_security/api/rolesmapping
• /_plugins/_security/api/actiongroups (and the legacy /actiongroup alias)
• /_plugins/_security/api/tenants
• /_plugins/_security/api/nodesdn

Singleton configuration APIs such as securityconfig, audit, and allowlist do not need pagination. Other non-configuration collection APIs (for example API tokens, certificates, versions, and resource APIs) can adopt the contract separately if appropriate.

Proposed API contract

Use the same concepts as the OpenSearch _list APIs:

• size: positive page size, defaulting to 100 when pagination is requested
• next_token: opaque cursor returned by the preceding page
• sort: asc or desc, sorting by configuration entity name and defaulting to asc

For example:

GET /_plugins/_security/api/roles?size=2&sort=asc

{
  "next_token": "<opaque token>",
  "roles": {
    "role_a": { "...": "..." },
    "role_b": { "...": "..." }
  }
}

The final page should return "next_token": null.

The cursor should be bound to at least the configuration type, sort direction, and last returned entity name. Invalid or tampered cursors, and cursors used with a different endpoint or sort direction, should return 400 Bad Request. Name-based cursor continuation is preferable to offsets so additions or deletions before the cursor do not shift every later page.

Backward compatibility

Requests without pagination parameters must retain the exact existing response shape and continue returning the complete visible collection. Single-entity GET endpoints must retain their current behavior; pagination parameters on those endpoints should be rejected with 400 Bad Request.

Security and correctness requirements

Pagination must operate on the caller-visible result set:

1. authorize the request;
2. redact hashes and other sensitive fields;
3. remove hidden entities;
4. apply endpoint-specific filtering, such as filterBy for internal users;
5. apply endpoint-specific augmentation, such as show_all for dynamic node DNs;
6. sort and select the page.

This ordering is important so page contents, counts, and cursor values cannot leak hidden configuration entity names. Visible reserved and static entities should participate in deterministic ordering.

The behavior when the entity named by a cursor has since been deleted should be lexical continuation from that name, rather than requiring the entity to still exist.

Acceptance criteria

• All six collection APIs support the same pagination parameters and response contract.
• Requests without pagination parameters are fully backward compatible.
• Single-entity GET behavior is unchanged.
• Ordering is deterministic and based on entity name.
• Tests cover:
  • ascending and descending traversal across multiple pages;
  • a null token on the terminal page;
  • invalid size, sort, malformed tokens, and cross-endpoint tokens;
  • additions and deletions between page requests;
  • authorization and hidden-entity filtering without information leakage;
  • internal-user filterBy;
  • node-DN show_all.
• REST API documentation and specifications are updated.