Add x-error-responses extension for partial-failure response shapes - #1137
Open
sean- wants to merge 1 commit into
Open
Add x-error-responses extension for partial-failure response shapes#1137sean- wants to merge 1 commit into
sean- wants to merge 1 commit into
Conversation
sean-
requested review from
Tokesh,
VachaShah,
Xtansia,
aabeshov,
harshavamsi,
karenyrx,
lucy66hw and
sachetalva
as code owners
May 29, 2026 02:14
sean-
force-pushed
the
worktree-partial-failure-mode
branch
from
May 29, 2026 02:16
cd63338 to
38354e8
Compare
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 29, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
May 30, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 1, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 1, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 2, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 2, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 2, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 2, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 3, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 3, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
added a commit
to sean-/opensearch-go
that referenced
this pull request
Jun 3, 2026
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
force-pushed
the
worktree-partial-failure-mode
branch
from
June 4, 2026 00:43
38354e8 to
0c6dbf0
Compare
sean-
added a commit
to opensearch-project/opensearch-go
that referenced
this pull request
Jun 4, 2026
…n classifier, default router, and union decode overhaul Consolidates the gh-816 burn-down: a partial-failure error model with fine-grained masking, an HTTP-layer operation classifier, blocking node discovery with default-router injection, a generator overhaul (idiomatic naming, single-pass union decode, request-body union constructors), and the v5preview regeneration plus integration coverage and user docs that go with it. ## Partial-failure error model OpenSearch returns HTTP 200 for partial successes (bulk item failures, shard failures, single-doc replica failures), forcing callers to double-check responses after `err == nil`. New typed errors expose these through the standard Go `if err != nil` idiom; both `(resp, err)` are non-nil on partial failure and the response is fully populated. - Adds `PartialBulkError`, `PartialSearchError`, `ShardFailureError`, and helpers `IsPartialFailure`, `ToleratePartialFailures`, `RequireSuccessRate` for threshold-based tolerance. - Replaces the initial `Config.ReturnQueryErrors` boolean with `internal/errmask.ErrorMask`, a 15-bit field where each bit masks one wrapper schema in the proposed `x-error-responses` OpenAPI extension (`BulkItems`, `SearchShards`, `WriteShards`, `BroadcastShards`, `NodeFailures`, `BulkByScrollFailures`, `TaskFailures`, `MultiSearchItems`, `MultiDocItems`, `Snapshot{Create,Get}ShardFailures`, `SimulateDocFailures`, `RankEvalFailures`, `IngestionShardFailures`, `PitNodeFailures`). A set bit suppresses that category; the zero value reports every category. - Callers configure policy via `Config.Errors = errmask.BulkItems | errmask.SearchShards` or `OPENSEARCH_GO_ERROR_MASK` with `+/-` tokens (e.g. `+all,-bulk_items`). Lifecycle: - v4 default `errmask.All` preserves existing behavior. `Config.ReturnQueryErrors=true` is honored as a deprecated alias for `errmask.None`. - v5 default flips to `errmask.None` (safe by default). - v6 removes `Config.Errors` / `OPENSEARCH_GO_ERROR_MASK`; behavior is unconditionally `errmask.None`. Spec side: `opensearch-openapi.yaml` is patched with 15 `_common.errors___<Wrapper>` schemas and 115 operations gain an `x-error-responses` annotation, mirroring the upstream proposal in `opensearch-api-specification`. The local patch goes away cleanly once that PR lands and we re-bundle from source. Generator side: `cmd/osgen` reads `x-error-responses` into `ir.Operation.ErrorWrappers`; `cmd/osgen/errwrap` supplies a fallback for plugin operations the spec does not yet annotate. The dispatch fragment carries a data-driven `wrappers` map of `{Template, Applies}`, so generated code stays compilable when annotations land before the underlying response schema models the relevant field. ## OperationClassifier and bit-packed OperationID Adds a zero-allocation HTTP method+path classifier (reusing the existing `routeTrie`) that maps requests to structured `OperationID` values. Enables transparent metrics, tracing, and access-control middleware at the `http.RoundTripper` layer without per-operation wrappers. - `OperationID` is a bit-packed `int64` encoding R/W flag, category, and minor operation. `IsWrite`, `Category`, `Minor` support efficient bitwise filtering. `String()` returns Prometheus-friendly labels. - `OperationClassifier` is built from the canonical route table and is safe for concurrent use. Returns `OpOther` for unrecognized patterns. - `OperationID` field added to `trieLeaf`/`trieMatch`, `OpID()` to `Route`, `.Op()` to `RouteBuilder`. All 124 routes are tagged. ## Blocking DiscoverNodes and default Router injection - `DiscoverNodes` now waits for an in-flight discovery to complete (or for the context to be cancelled) instead of returning `nil` immediately, letting callers block until topology data is available after client construction. - `DiscoverNodesOnStart` becomes `*bool`. Auto-enabled when `OPENSEARCH_GO_ROUTER=true` and the caller did not set it. - `v5preview/opensearchapi.NewClient` and `NewDefaultClient` inject `opensearchtransport.NewDefaultRouter` when `config.Client.Router` is `nil`, opting v5preview clients into role-aware dispatch, RTT-based scoring, AIMD congestion-window, and shard-cost weighting by default. - `OPENSEARCH_GO_ROUTER` is a symmetric override: a falsy value (`false`/`0`) suppresses injection so `Router` stays `nil`, matching v4 behavior. Caller-provided Routers are preserved. - `internal/envvars.Falsy(name)` distinguishes "unset" from "explicitly opted out" so the v5preview rule reads as `!envvars.Falsy(envvars.Router)` without re-implementing parse logic. ## Generator: idiomatic naming, request-body unions, single-pass decode Naming rewrites at PascalCase boundaries: | Spec form | Idiomatic Go | |---------------|-----------------| | `Msearch` | `MSearch` | | `Mget` | `MGet` | | `Mtermvectors`| `MTermVectors` | | `Termvectors` | `TermVectors` | | `Forcemerge` | `ForceMerge` | | `Response` | `Resp` | The `Response -> Resp` rule additionally requires the trailing character to be uppercase, so standalone `SearchResponse` (a spec wrapper) does not collide with the operation-level `<Op>Resp` name. Hand-written types renamed for v4/v5 consistency: `BulkResponseItem -> BulkRespItem`, `ErrorResponseBase -> ErrorRespBase`, `MsearchErrors -> MSearchErrors`, `MsearchTemplateErrors -> MSearchTemplateErrors`. Request-body unions: - `splitUnionsFromSiblings` partitions request-body subtree unions (e.g. `ReindexSourceSort`) so they are routed to `UnionFragment` instead of being emitted as empty structs. - Plumbs `Op + Registry` into `UnionFragment` so plugin-package unions qualify cross-package branches as `opensearchapi.FieldSort`. - Each generated discriminated union gains `New<Union>From<Branch>(v <branch type>) <Union>` constructors per branch and a `SetRaw(json.RawMessage)` typed escape hatch. `SetRaw` clears the typed branch so `MarshalJSON` returns the raw bytes verbatim. Single-pass union decode: - Case A (merged): object unions with one permissive primary branch plus discriminated branch(es) (mget, msearch, indices-open). The primary is embedded and the common case decodes in a single `json.Unmarshal`; each discriminated branch is detected by the presence of its distinguishing key and decoded only when matched. Drops the `build.HasJSONKeys` map probe and per-item raw copy. - Case B (lazy `As<T>()`): aggregation/suggest result unions carry no wire discriminator, so they cannot be auto-selected. `UnmarshalJSON` only retains the raw bytes; generated `As<ConcreteType>()` accessors decode on demand. - Unions fitting neither (reindex bodies, plugin-defined task status) keep the existing try-each decoder; the classifier logs once per union name when it declines. - All union `UnmarshalJSON` aliases the owned response buffer (`u.raw = data`) rather than copying it; `RawJSON()` documents the borrowed-buffer contract. Measured impact on mget (1000 docs): decode allocations down ~2.7x (~43k -> ~16k allocs/op), time down ~1.7x (4.2ms -> 2.5ms). Bulk wrapper template now resolves its walked element type from the IR (via `bulkInnerItemType`) instead of hardcoding `BulkRespItem`, so future spec or naming changes propagate automatically. ## v5preview regeneration Mechanical output of `cmd/osgen` against the patched spec: - `clients_gen.go` declares `errors errmask.ErrorMask` on `Client` and `clientInit` takes the mask as a second arg. - 19 operation dispatch files emit per-wrapper post-`do()` blocks that return typed errors when the corresponding `errmask` bit is unset and the wire data carries a partial failure (`BulkItems`, `SearchShards`, `WriteShards`, `MultiSearchItems`). - Reserved-but-unemitted wrappers (`BroadcastShards`, `NodeFailures`, `BulkByScrollFailures`, `TaskFailures`, `MultiDocItems`, `Snapshot{Create,Get}ShardFailures`, `SimulateDocFailures`, `RankEvalFailures`, `IngestionShardFailures`, `PitNodeFailures`) carry annotations but no emission until detection logic lands. - Operations whose typed response shape lacks the field path a wrapper needs (`CreateResp` lacks `_shards`; msearch union items lack a top-level `Shards`) are silently skipped by the `Applies` guard; emission starts automatically once the response types catch up. - Idiomatic-naming pass touches every generated type containing `Msearch`, `Mget`, `Mtermvectors`, `Termvectors`, `Forcemerge`, or compound `*Response*` substrings. - Single-pass union decode applied to mget/msearch/indices-open success|error items and `As<T>()` accessors generated for aggregation and suggest result unions. ## Integration coverage and CI hardening v5preview integration tests drive real requests and assert decoded shape per server version: - aggregation: lazy `As<T>()` accessors (terms, date_histogram, stats, avg, sum, min, max, value_count, cardinality). - mget: merged success|error decode (`GetResult` found/not-found vs `MGetMultiGetError`). - msearch: merged success|error decode (`MSearchMultiSearchItem` vs `ErrorRespBase`), the first-byte-switch `SearchHitsMetadataTotal` union, and the `MultiSearchItemError` partial-failure surface. CI matrix changes: - Skip multi-node clusters on OpenSearch <=2.17.x to avoid the node-join/node-left coordinator race (`opensearch-project/OpenSearch#15521`, backported via #16118 on the 2.x line). Affected versions run with `OPENSEARCH_NODE_COUNT=1`. - `.github/workflows/test-compatibility.yml` adds a per-entry `node_count` matrix field: `1` for 1.3.20 through 2.17.1, `3` for 2.18.0 and later. Comment cites the upstream PRs so the cutoff is greppable. - `Makefile cluster.docker-up` respects `OPENSEARCH_NODE_COUNT` and passes `--scale opensearch-nodeN=0` to docker compose. Defaults to 3 for local development. - Widen the existing 2.1.0-only shard-routing test skip to all versions below 2.2.0 (security plugin `java.io.OptionalDataException` from non-thread-safe `User` serialization, fixed by `opensearch-project/security#1970`). - Generated-code emit/path test fixtures converted from raw `"GET"`/`"POST"` to `net/http http.Method*` constants for consistency with the rest of the suite. - `testutil.PollUpdate()` decouples jitter calculation from runtime backoff execution to fix a flake. ## Dependency bumps - `github.com/aws/aws-sdk-go-v2/config` 1.32.18 -> 1.32.20 - `github.com/aws/aws-sdk-go-v2/credentials` 1.19.17 -> 1.19.19 - `github.com/getkin/kin-openapi` v0.139.0 -> v0.140.0 Fixes: #852, #850 ## Documentation - `v5preview/opensearchapi/README.md`: Partial Failure Errors (Config.Errors, errmask, `OPENSEARCH_GO_ERROR_MASK`, typed errors, `opensearchapi.Errors` helper, per-Resp helpers, helper functions, operation constants) and Default Router Injection (truth table for `OPENSEARCH_GO_ROUTER`, opt-out semantics). - `v5preview/opensearchapi/MIGRATING.md`: v4 -> v5preview surface delta (import path, `Indices -> Index` on multi-index Req types, `DocumentID -> ID` on `IndexReq`, optional `Params` becomes `*Params`, optional `bool` query params become `*bool`, partial-failure type renames, errmask default flip, default Router injection). - `guides/error_handling.md`: Bulk/Search/Write partial-failure examples as paired v4/v5preview blocks; per-Resp helper subsection (`BulkItemFailures`, `SearchShardFailures`, `WriteShardFailures`, `MultiSearchItemFailures`, `PartialFailures(mask)`); error type reference covering v4 vs v5preview internal-field-type divergence. - `guides/bulk.md`: v4/v5preview field-name and `BulkResp.Items` shape divergences with paired error-iteration examples for v4's `[]map[string]BulkRespItem` and v5preview's `[]BulkItem`. - `UPGRADING.md`: keeps version-history essentials for the >=5.0 partial-failure model and v5preview Router injection; forward-links to the new package docs. Ref: #816 Ref: opensearch-project/opensearch-api-specification/pull/1137
sean-
force-pushed
the
worktree-partial-failure-mode
branch
from
June 4, 2026 21:52
0c6dbf0 to
ccf8f79
Compare
Introduce a per-operation `x-error-responses` vendor extension whose value is an array of `$ref` objects pointing at wrapper schemas in `spec/schemas/_common.errors.yaml`. Each wrapper describes one auxiliary failure shape an endpoint may carry alongside its primary 2xx response — including arity (array vs object vs map) and any parallel-array grouping (e.g. `task_failures[]` + `node_failures[]` together for the Tasks API) — using ordinary JSON Schema. Client generators walk an operation's `x-error-responses` array, look up each wrapper, and emit one typed handler per wrapper that detects and extracts the embedded failure content. The new `_common.errors.yaml` defines 15 wrappers covering the distinct partial-failure encodings present in OpenSearch today: BulkItems, SearchShards, WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures, TaskFailures, MultiSearchItems, MultiDocItems, SnapshotCreateShardFailures, SnapshotGetShardFailures, SimulateDocFailures, RankEvalFailures, IngestionShardFailures, and PitNodeFailures. Each wrapper composes existing element schemas (`ShardSearchFailure`, `ShardFailure`, `NodeStatistics`, `TaskFailure`, `SnapshotShardFailure`, `IngestionStateShardFailure`, `BulkByScrollFailure`, etc.) — no new element types are added. Annotate 115 operation variants across 10 namespace files (_core, asynchronous_search, cluster, dangling_indices, indices, ingest, ingestion, nodes, snapshot, tasks). `indices.recovery` and `indices.get_upgrade` are intentionally not annotated: their response classes extend `BroadcastResponse` but suppress the `_shards` block in their `toXContent` overrides. Document the extension in DEVELOPER_GUIDE.md alongside the existing vendor extensions list. Motivated by `opensearch-go`'s port of partial-failure error handling into its generated `v5preview/opensearchapi/` package, which needs a generator-driven mapping from operation to auxiliary failure types. Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
sean-
force-pushed
the
worktree-partial-failure-mode
branch
from
June 4, 2026 22:08
ccf8f79 to
465440c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a per-operation
x-error-responsesvendor extension whose value is an array of$refobjects pointing at wrapper schemas inspec/schemas/_common.errors.yaml. Each wrapper describes one auxiliary failure shape an endpoint may carry alongside its primary 2xx response — including arity (array vs object vs map) and any parallel-array grouping (e.g.task_failures[]+node_failures[]together for the Tasks API) — using ordinary JSON Schema. Client generators walk an operation'sx-error-responsesarray, look up each wrapper, and emit one typed handler per wrapper that detects and extracts the embedded failure content.The new
_common.errors.yamldefines 15 wrappers covering the distinct partial-failure encodings present in OpenSearch today: BulkItems, SearchShards, WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures, TaskFailures, MultiSearchItems, MultiDocItems, SnapshotCreateShardFailures, SnapshotGetShardFailures, SimulateDocFailures, RankEvalFailures, IngestionShardFailures, and PitNodeFailures. Each wrapper composes existing element schemas (ShardSearchFailure,ShardFailure,NodeStatistics,TaskFailure,SnapshotShardFailure,IngestionStateShardFailure,BulkByScrollFailure, etc.) — no new element types are added.Annotate 115 operation variants across 10 namespace files (_core, asynchronous_search, cluster, dangling_indices, indices, ingest, ingestion, nodes, snapshot, tasks).
indices.recoveryandindices.get_upgradeare intentionally not annotated: their response classes extendBroadcastResponsebut suppress the_shardsblock in theirtoXContentoverrides.Document the extension in DEVELOPER_GUIDE.md alongside the existing vendor extensions list.
Motivated by
opensearch-go's port of partial-failure error handling into its generatedv5preview/opensearchapi/package, which needs a generator-driven mapping from operation to auxiliary failure types.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.