Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/vale/styles/Vocab/OpenSearch/Products/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ Adoptium
AI21 Labs Jurassic
Amazon
Amazon OpenSearch Serverless
Amazon OpenSearch Serverless NextGen
Amazon OpenSearch Service
Amazon Bedrock
Amazon Kinesis
Expand Down
24 changes: 12 additions & 12 deletions _migration-assistant/amazon-opensearch-serverless.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
layout: default
title: Migrate to OpenSearch Serverless
title: Migrate to OpenSearch Serverless NextGen
nav_order: 55
permalink: /migration-assistant/amazon-opensearch-serverless/
---

# Migrate to OpenSearch Serverless
# Migrate to OpenSearch Serverless NextGen

If your target is an Amazon OpenSearch Serverless collection, Serverless works as a target for every source Migration Assistant supports: Elasticsearch 1.x--2.x, OpenSearch 1.x--2.x, Amazon OpenSearch Service, and Apache Solr 6.x--9.x (backfill only). The migration steps are the same as for any other OpenSearch target. Follow your source's playbook and use the Serverless-specific target configuration from this page.
If your target is an Amazon OpenSearch Serverless NextGen collection, Serverless NextGen works as a target for every source Migration Assistant supports: Elasticsearch 1.x--2.x, OpenSearch 1.x--2.x, Amazon OpenSearch Service, and Apache Solr 6.x--9.x (backfill only). The migration steps are the same as for any other OpenSearch target. Follow your source's playbook and use the Serverless NextGen target configuration from this page.

| Source | Backfill | Capture and Replay |
|:-------|:--------:|:------------------:|
Expand All @@ -19,21 +19,21 @@ If your target is an Amazon OpenSearch Serverless collection, Serverless works a

## Collection types

Amazon OpenSearch Serverless supports the following collection types. Migration Assistant auto-detects the collection type and adjusts behavior accordingly.
Amazon OpenSearch Serverless NextGen supports the following collection types. Migration Assistant auto-detects the collection type and adjusts behavior accordingly.

| Collection type | Document IDs |
|:---------------|:------------|
| `SEARCH` | Preserves source document IDs |
| `TIMESERIES` | Server-generated IDs (source IDs not preserved) |
| `TIMESERIES` | Server-generated IDs (source IDs not preserved). Not supported on NextGen at launch (available with Classic). |
| `VECTORSEARCH` | Server-generated IDs (source IDs not preserved) |

If your source data relies on specific document IDs (for example, for lookups or deduplication), use a `SEARCH` collection.

When migrating to a `VECTORSEARCH` collection, `knn_vector` field mappings are automatically converted to Faiss HNSW for Serverless compatibility, and `model_id` references are removed (Serverless does not support training APIs).
When migrating to a `VECTORSEARCH` collection, `knn_vector` field mappings are automatically converted to Faiss HNSW for Serverless NextGen compatibility, and `model_id` references are removed (Amazon OpenSearch Serverless NextGen does not support training APIs).

## Connecting your collection to Migration Assistant

Migration Assistant requires the following configuration to access your OpenSearch Serverless collection:
Migration Assistant requires the following configuration to access your Amazon OpenSearch Serverless NextGen collection:

1. The migration IAM role must have `aoss:APIAccessAll` in its IAM policy (the Amazon Elastic Kubernetes Service (EKS) deployment handles this automatically).
2. The migration IAM role must be listed as a `Principal` in your collection's data access policy (you must configure this).
Expand Down Expand Up @@ -94,7 +94,7 @@ If your collection already has a data access policy, use `update-access-policy`

### Step 3: Configure the workflow

Configure your OpenSearch Serverless collection as the target in your workflow configuration. The key difference from a managed OpenSearch Service target is `service: aoss` (instead of `service: es`):
Configure your Amazon OpenSearch Serverless NextGen collection as the target in your workflow configuration. The key difference from a managed OpenSearch Service target is `service: aoss` (instead of `service: es`):

```bash
workflow configure edit
Expand All @@ -107,7 +107,7 @@ Set the target cluster:
{
"targetClusters": {
"target": {
"endpoint": "https://<collection-id>.<region>.aoss.amazonaws.com",
"endpoint": "https://<collection-id>.aoss.<region>.on.aws",
"authConfig": {
"sigv4": {
"region": "<region>",
Expand All @@ -126,6 +126,6 @@ Then follow the steps in [Using the Workflow CLI]({{site.url}}{{site.baseurl}}/m

Choose the playbook for your source type:

- **Self-managed/third-party Elasticsearch or OpenSearch** -- Follow the [Elasticsearch 6.8 → OpenSearch 3.5 playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-elasticsearch-6-8-to-opensearch-3/), then replace the target cluster block with the preceding OpenSearch Serverless configuration.
- **Amazon OpenSearch Service/legacy Elasticsearch Service** -- Follow the [Amazon OpenSearch Service → Amazon OpenSearch Serverless playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-amazon-opensearch-service-to-serverless/).
- **Apache Solr** -- Follow the [Apache Solr 8.11 → OpenSearch 3.5 playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-solr-8.11-to-opensearch-3/), then replace the target cluster block with the preceding OpenSearch Serverless configuration.
- **Self-managed/third-party Elasticsearch or OpenSearch** -- Follow the [Elasticsearch 6.8 → OpenSearch 3.5 playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-elasticsearch-6-8-to-opensearch-3/), then replace the target cluster block with the preceding Serverless NextGen configuration.
- **Amazon OpenSearch Service/legacy Elasticsearch Service** -- Follow the [Amazon OpenSearch Service → Amazon OpenSearch Serverless NextGen playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-amazon-opensearch-service-to-serverless/).
- **Apache Solr** -- Follow the [Apache Solr 8.11 → OpenSearch 3.5 playbook]({{site.url}}{{site.baseurl}}/migration-assistant/playbook-solr-8.11-to-opensearch-3/), then replace the target cluster block with the preceding Serverless NextGen configuration.
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The following table lists supported source and target platforms.
|:---------|:-------|:-------|
| Self-managed (on-premises) | Yes | Yes |
| Amazon OpenSearch Service | Yes | Yes |
| [Amazon OpenSearch Serverless]({{site.url}}{{site.baseurl}}/migration-assistant/amazon-opensearch-serverless/) | No | Yes |
| [Amazon OpenSearch Serverless NextGen]({{site.url}}{{site.baseurl}}/migration-assistant/amazon-opensearch-serverless/) | No | Yes |
| Third-party cloud providers | Yes | Yes |
| AWS EC2 | Yes | Yes |
| Apache Solr (SolrCloud/Standalone) | Yes | No |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,11 @@ Migration Assistant supports the following authentication methods on generic Kub

Basic authentication works the same way as EKS. Put the credentials in a Kubernetes secret and reference the secret name in `authConfig.basic.secretName`.

### Use AWS Signature Version 4 for Amazon OpenSearch Service or Serverless
### Use AWS Signature Version 4 for Amazon OpenSearch Service or Serverless NextGen

AWS Signature Version 4 is supported by the workflow configuration, but **generic Kubernetes does not automatically create AWS pod identity for you**.

If your source or target uses Amazon OpenSearch Service or OpenSearch Serverless, you must make AWS credentials available to two sets of pods:
If your source or target uses Amazon OpenSearch Service or OpenSearch Serverless NextGen, you must make AWS credentials available to two sets of pods:

- The Migration Console pod (`migration-console-0`), running under the `migration-console-access-role` service account, which runs CLI commands such as `console clusters connection-check`
- The Argo workflow executor pods, running under the `argo-workflow-executor` service account, which perform the actual migration steps.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Before you build custom logic, verify whether the migration is already covered b
- `string` to `text` and `keyword`
- `flattened` to `flat_object`
- `dense_vector` to `knn_vector`
- Additional vector compatibility adjustments for newer OpenSearch and Serverless targets.
- Additional vector compatibility adjustments for newer OpenSearch and Serverless NextGen targets.

## Custom field type transformer

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The current metadata migration path already includes built-in transformations fo
- `string` to `text` and `keyword`
- `flattened` to `flat_object`
- `dense_vector` to `knn_vector`
- Additional vector compatibility transformations for newer OpenSearch and Serverless targets.
- Additional vector compatibility transformations for newer OpenSearch and Serverless NextGen targets.

If your migration needs more than the built-ins, you can also supply custom metadata transformers through `transformerConfig`, `transformerConfigBase64`, or `transformerConfigFile`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The metadata migration path can:
- Translate vector dimensions and related settings.
- Prepare the target mapping for OpenSearch vector search.

Depending on the target, additional vector compatibility transforms may also apply, including Serverless-specific adjustments.
Depending on the target, additional vector compatibility transforms may also apply, including Serverless NextGen-specific adjustments.

## Identifying dense_vector fields

Expand Down Expand Up @@ -52,6 +52,6 @@ Even when the mapping migration succeeds, query behavior may still need to chang

Run a pilot migration with representative queries before cutover in the following cases:

- The target is OpenSearch Serverless.
- The target is Amazon OpenSearch Serverless NextGen.
- The target version has vector-engine compatibility constraints.
- The application relies on specific vector query syntax or ranking behavior.
Loading
Loading