Skip to content

Fix type and schema mismatches across the spec - #1110

Open
sean- wants to merge 9 commits into
opensearch-project:mainfrom
sean-:fix-search-size-int64-to-int32
Open

Fix type and schema mismatches across the spec#1110
sean- wants to merge 9 commits into
opensearch-project:mainfrom
sean-:fix-search-size-int64-to-int32

Conversation

@sean-

@sean- sean- commented May 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes type and schema mismatches across the spec, verified against server source code. Two classes of bug are addressed:

  1. Integer type mismatches. Code generators use the format specifier to choose native integer types — wrong formats produce incorrect generated code (e.g., int64 emits long in Java / int64 in Go where the server expects int).
  2. Path parameter cardinality. Several path parameters referenced the singular Name / Id schemas where the server splits the value on commas (or vice versa). The singular schemas accept a string; Names / Ids are oneOf single-or-array, matching the comma-separated syntax the server actually accepts.

All changes were verified by reading the Java/Kotlin server declarations for each field.

Bug fix: notifications HeaderParamsMap has wrong type (integer → string)

HeaderParamsMap.additionalProperties was defined as type: integer, format: int32, but the server declares this as Map<String, String> (CustomWebhookDestination.kt). This is not a width mismatch — the type is entirely wrong. Code generators emit integer-typed map values where the server sends and expects strings.

Bug fix: path parameter cardinality (Name/IdNames/Ids)

Three path parameters had a schema reference that did not match server behavior. The singular Name and Id schemas accept a single string; Names and Ids are oneOf single-or-array, matching the comma-separated path syntax accepted by Strings.splitStringByCommaToArray. Code generators emit the wrong type — a single string where the server accepts a list, or a list where the server accepts only a single name.

Spec parameter Before After Server behavior
snapshot.delete::path.snapshot Name Names Server splits on commas via Strings.splitStringByCommaToArray, stored as String[] snapshots.
ingest.get_pipeline::path.id Id Ids Server splits on commas via Strings.splitStringByCommaToArray.
snapshot.delete_repository::path.repository Names Name (reverse) Server reads a single repository string. Wildcard matching is performed on a single name pattern, not a comma-separated list.

Server references (OpenSearch 27333365):

Changes by area

ML plugin (ml.yaml, ml._common.yaml) — int64 → int32

  • size (×10): _search request bodies — SearchSourceBuilder.size is Java int
  • total_chunks (×2), chunk_number (×2): Java Integer
  • num_outputs: Java int

Security Analytics (security_analytics.yaml) — int64 → int32

  • size, startIndex (×2 each): request.paramAsInt() in alert and findings handlers
  • nearby_findings: request.paramAsInt() in correlation handler

Core OpenSearch schemas — mixed direction

int64 → int32 (server uses Java int):

  • indices.forcemerge max_num_segments
  • cluster.stats ClusterStatsIndices.count
  • cluster.remote_info num_nodes_connected
  • nodes.stats ThreadCount active/largest/queue/threads
  • snapshot file_count, SnapshotShardsStats 6 fields
  • indices.recovery RecoveryFiles recovered/reused/total
  • indices.stats ShardCommit.num_docs
  • _common RecoveryStats current_as_source/current_as_target, RefreshStats.listeners

int32 → int64 (server uses Java long):

  • indices.stats ShardCommit.generation
  • nodes.info NodeInfoJvmInfo.pid, NodeInfoNetwork.refresh_interval
  • _common SegmentsStats.count

integer/int32 → number/double:

  • cluster.stats ClusterIndicesShards.replication

Core search APIs — mixed direction

int64 → int32: TermSuggestion.Option.freq, MultiSearchItem.status, ReindexStatus.batches
int32 → int64: AggregationProfileDebug.total_buckets, FetchProfileBreakdown (8 fields), Term.ttf
number → integer/int64: MultiSearchResult.took
number → integer/int32: MultiSearchItem.status

Flow Framework (flow_framework.common.yaml) — add missing int64 format

11 type: integer fields lacked a format specifier. Without format: int64, code generators default to int32, but the server uses Java long:

  • created_time, last_updated_timeInstant.toEpochMilli()
  • total.valueTotalHits.value
  • _version, _seq_no, _primary_term (×6) — long in GetResult.java
  • took (×2) — long tookInMillis

Notifications (notifications._common.yaml) — integer → string (bug)

  • HeaderParamsMap additionalProperties: type: integer, format: int32type: string

Snapshot and ingest namespaces — path parameter cardinality

  • snapshot.delete::path.snapshot: NameNames
  • snapshot.delete_repository::path.repository: NamesName (reverse)
  • ingest.get_pipeline::path.id: IdIds

Server references

All changes verified against these commits:

Repo SHA
OpenSearch 27333365da0d55a16bea2ebaf34f6f2a691f6d50
ml-commons be86ad045dac49f35d811b74f2928a27536da2a4
security-analytics 4c3d533dd7d71d734f94deb9038fd9b46ae368f2
flow-framework 30ed527ae0b121ee7e47db8326d6672ca4d76aa8
notifications 690f3df226f2da10da23bb717ff151406be8c436

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@sean-
sean- force-pushed the fix-search-size-int64-to-int32 branch from 134e175 to 60b80f8 Compare May 19, 2026 06:28
@sean- sean- changed the title Fix type mismatches across the spec Fix type and schema mismatches across the spec May 22, 2026
sean- added 9 commits May 22, 2026 02:02
Several ML fields were typed as int64 but the server uses Java `int`
(32-bit). Code generators emit 64-bit integers for these fields, which
is wider than the server accepts.

Fields fixed (all int64 → int32):
- size (×10): _search request bodies and search input schema
  SearchSourceBuilder.size is Java int, parsed via parser.intValue()
- total_chunks (×2): MLRegisterModelMetaInput.totalChunks (Integer)
- chunk_number (×2): MLUploadModelChunkInput.chunkNumber (Integer)
- num_outputs: AnomalyLocalizationInput.numOutputs (int)

Server references (ml-commons be86ad04):
- https://github.com/opensearch-project/ml-commons/blob/be86ad045dac49f35d811b74f2928a27536da2a4/common/src/main/java/org/opensearch/ml/common/transport/register/MLRegisterModelMetaInput.java#L81
- https://github.com/opensearch-project/ml-commons/blob/be86ad045dac49f35d811b74f2928a27536da2a4/common/src/main/java/org/opensearch/ml/common/input/execute/anomalylocalization/AnomalyLocalizationInput.java#L148
- https://github.com/opensearch-project/ml-commons/blob/be86ad045dac49f35d811b74f2928a27536da2a4/common/src/main/java/org/opensearch/ml/common/transport/upload_chunk/MLUploadModelChunkInput.java#L32

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Query parameters `size`, `startIndex`, and `nearby_findings` were typed
as int64 but the server parses all of them with `request.paramAsInt()`,
which returns Java `int` (32-bit).

Fields fixed (all int64 → int32):
- get_alerts: size, startIndex
- get_findings: size, startIndex
- search_finding_correlations: nearby_findings

Server references (security-analytics 4c3d533d):
- RestGetAlertsAction.java:41-42 — request.paramAsInt()
  https://github.com/opensearch-project/security-analytics/blob/4c3d533dd7d71d734f94deb9038fd9b46ae368f2/src/main/java/org/opensearch/securityanalytics/resthandler/RestGetAlertsAction.java#L41-L42
- RestGetFindingsAction.java:41-42 — request.paramAsInt()
  https://github.com/opensearch-project/security-analytics/blob/4c3d533dd7d71d734f94deb9038fd9b46ae368f2/src/main/java/org/opensearch/securityanalytics/resthandler/RestGetFindingsAction.java#L41-L42
- RestSearchCorrelationAction.java:60 — request.paramAsInt()
  https://github.com/opensearch-project/security-analytics/blob/4c3d533dd7d71d734f94deb9038fd9b46ae368f2/src/main/java/org/opensearch/securityanalytics/resthandler/RestSearchCorrelationAction.java#L60

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Multiple schema files had integer widths or types that don't match the
server. Code generators emit incorrect types for these fields.

int64 → int32 (server uses Java int):
- indices.forcemerge max_num_segments — ForceMergeRequest.java:77
- cluster.stats ClusterStatsIndices.count — int indexCount
- cluster.remote_info num_nodes_connected — int numNodesConnected
- nodes.stats ThreadCount active/largest/queue/threads — private final int
- snapshot file_count — int incrementalFileCount
- snapshot SnapshotShardsStats done/failed/finalizing/initializing/started/total — private int
- indices.recovery RecoveryFiles recovered/reused/total — return int
- indices.stats ShardCommit.num_docs — int numDocs
- _common RecoveryStats current_as_source/current_as_target — AtomicInteger
- _common RefreshStats.listeners — int listeners

int32 → int64 (server uses Java long):
- indices.stats ShardCommit.generation — long generation
- nodes.info NodeInfoJvmInfo.pid — long pid
- nodes.info NodeInfoNetwork.refresh_interval — long refreshInterval
- _common SegmentsStats.count — long count

integer/int32 → number/double:
- cluster.stats ClusterIndicesShards.replication — getReplication() returns double

Server references (OpenSearch 27333365):
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/action/admin/indices/forcemerge/ForceMergeRequest.java#L77
-
https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/action/admin/cluster/stats/ClusterStatsIndices.java#L61
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/transport/SniffConnectionStrategy.java#L614
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/threadpool/ThreadPoolStats.java#L67-L71
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/snapshots/SnapshotStats.java#L61-L62
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/snapshots/SnapshotShardsStats.java#L56-L61
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/indices/recovery/ReplicationLuceneIndex.java
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/engine/CommitStats.java#L57-L59
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/monitor/jvm/JvmInfo.java#L222
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/monitor/os/OsInfo.java#L50
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/recovery/RecoveryStats.java#L55-L56
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/refresh/RefreshStats.java#L65
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/engine/SegmentsStats.java#L61

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
int64 → int32 (server uses Java int):
- TermSuggestion.Option.freq — int freq in TermSuggestion.java:229
- MultiSearchItem.status — int getStatus() in RestStatus.java
- ReindexStatus.batches — int batches in BulkByScrollTask.java:456

int32 → int64 (server uses Java long):
- AggregationProfileDebug.total_buckets — bucketOrds.size() returns long
- FetchProfileBreakdown (8 fields) — Map<String, Long> in ProfileResult.java:88
- Term.ttf — Lucene TermsEnum.totalTermFreq() returns long

number → integer/int64:
- MultiSearchResult.took — long tookInMillis in MultiSearchResponse.java:145

number → integer/int32:
- MultiSearchItem.status — int getStatus() in RestStatus.java

Server references (OpenSearch 27333365):
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/search/suggest/term/TermSuggestion.java#L229
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/search/profile/ProfileResult.java#L88
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/action/search/MultiSearchResponse.java#L145
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/rest/RestStatus.java#L526
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/reindex/BulkByScrollTask.java#L456

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Eleven integer fields were declared as `type: integer` without a format
specifier. The server uses Java `long` for all of them, so they need
`format: int64`. Without it, code generators default to int32.

Fields fixed (all add format: int64):
- created_time, last_updated_time — Instant.toEpochMilli() returns long
- total.value — TotalHits.value is long
- _version, _seq_no, _primary_term (×6) — long in GetResult.java:87-89
- took (×2) — long tookInMillis in SearchResponse

Server references:
- flow-framework 30ed527a: https://github.com/opensearch-project/flow-framework/blob/30ed527ae0b121ee7e47db8326d6672ca4d76aa8/src/main/java/org/opensearch/flowframework/model/Template.java
- OpenSearch 27333365: https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/index/get/GetResult.java#L87-L89

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
BUG: HeaderParamsMap.additionalProperties was defined as
type: integer / format: int32, but the server declares this field as
Map<String, String>. This is not an integer width mismatch — the type
is entirely wrong. Code generators emit integer-typed map values where
the server sends and expects strings.

Field fixed:
- HeaderParamsMap additionalProperties: integer/int32 → string

Server reference (notifications 690f3df2):
- CustomWebhookDestination.kt: Map<String, String>
  https://github.com/opensearch-project/notifications/blob/690f3df226f2da10da23bb717ff151406be8c436/notifications/core/src/main/kotlin/org/opensearch/notifications/core/setting/PluginSettings.kt

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Two snapshot path parameters had a schema reference that did not match
server behavior. The Name schema accepts a single string; Names is
oneOf single-or-array, matching the comma-separated path syntax accepted
by Strings.splitStringByCommaToArray.

Fields fixed:
- snapshot.delete::path.snapshot: Name -> Names
  Server splits on commas via Strings.splitStringByCommaToArray and
  stores the result as String[] snapshots.
- snapshot.delete_repository::path.repository: Names -> Name (reverse)
  Server reads a single repository string. Wildcard matching is
  performed on a single name pattern, not a comma-separated list.

Server references (OpenSearch 27333365):
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/rest/action/admin/cluster/RestDeleteSnapshotAction.java#L73
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/action/admin/cluster/snapshots/delete/DeleteSnapshotRequest.java#L58-L71
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/rest/action/admin/cluster/RestDeleteRepositoryAction.java#L70
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/action/admin/cluster/repositories/delete/DeleteRepositoryRequest.java#L55-L70

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
The ingest.get_pipeline {id} path parameter referenced the singular Id
schema, but the server splits the value on commas and treats it as a
list. The parameter description already documents this ("A
comma-separated list of pipeline IDs to retrieve. Wildcard expressions
are supported"), so the singular ref was a long-standing schema bug.

Field fixed:
- ingest.get_pipeline::path.id: Id -> Ids
  Server splits on commas via Strings.splitStringByCommaToArray and
  passes the resulting array to GetPipelineRequest.

Server reference (OpenSearch 27333365):
- https://github.com/opensearch-project/OpenSearch/blob/27333365da0d55a16bea2ebaf34f6f2a691f6d50/server/src/main/java/org/opensearch/rest/action/ingest/RestGetPipelineAction.java#L71

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant