[Bug] OpenSearchVectorStore fails against Amazon OpenSearch Serverless (AOSS)

[avichaym]

Search before asking

• I searched in the issues and found nothing similar.

Description

The OpenSearchVectorStore integration (introduced in #533) supports both Amazon OpenSearch Service (provisioned domains) and Amazon OpenSearch Serverless (AOSS) via a service_type parameter that defaults to "serverless". However, for serverless , some operations fail because AOSS has some differences specifically around SigV4 signing requirements, supported index APIs, bulk ingestion constraints, and KNN engine capabilities.

How to reproduce

1. Create an AOSS collection of type VECTORSEARCH in any region.
   Create a data-access policy granting aoss:* to your IAM principal for both collection/ and index//*.
   2)Wait for the collection to reach ACTIVE status.
   3)Run any pipeline that uses OpenSearchVectorStore with the default service_type=serverless (or explicitly set it), pointing at the AOSS endpoint.

Some operations will fail with issues -

POST //_refresh after add/delete - API surface issue

filtered KNN ( createKnnIndex omits method.engine, AOSS defaults to NMSLIB which cannot serve filtered KNN)

Version and environment

Flink Agents: 0.3-SNAPSHOT (current main)
Flink: 1.20.0
Java: JDK 17+
OS: macOS (arm64) / Linux

Are you willing to submit a PR?

• I'm willing to submit a PR!