Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/vale/styles/Vocab/OpenSearch/Products/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ Microsoft Power BI
Migration Assistant
Migration Console
Minikube
Mustache
Nagle
Nagios
NVMe
Expand Down
18 changes: 16 additions & 2 deletions _search-plugins/search-relevance/query-sets.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ All query sets comprise one or more entries. Each entry is a JSON object contain
| :--- | :--- | :--- |
| `queryText` | String | The user query string. Required. |
| `referenceAnswer` | String | The expected or correct answer to the user query. This field is used for generating judgments, especially with large language models (LLMs). Optional. |
| Custom fields | String | Any field other than `queryText` and `referenceAnswer` is stored as a custom field in the entry. You can reference a custom field by its name as a [Mustache template variable]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/search-configurations/#using-mustache-templates) in a search configuration. Optional. |

### Basic query set example

Expand All @@ -102,7 +103,7 @@ A basic query set contains only the `queryText` field for each entry. It is suit

### Query set with reference answers example

This format includes the `referenceAnswer` field alongside the `queryText`. It is ideal for evaluating applications designed to provide specific answers, such as chatbots or question-answering systems.
In this format, each entry pairs a `queryText` with a `referenceAnswer`. Use it to evaluate applications that return a specific answer, such as chatbots or question-answering systems.

#### Example query set with reference answers

Expand All @@ -114,9 +115,22 @@ This format includes the `referenceAnswer` field alongside the `queryText`. It i
{"queryText": "When was the first iPhone released?", "referenceAnswer": "June 29, 2007"}
```


The `referenceAnswer` field is particularly useful when using [LLMs to generate judgments]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/judgments/). The LLM can use the reference answer as a ground truth to compare against the retrieved search results, allowing it to accurately score the relevance of the response.

### Query set with custom fields example

In addition to `queryText`, each entry can include custom fields. You can reference these fields as Mustache template variables in a [search configuration]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/search-configurations/#using-mustache-templates) so that one search configuration applies different filters or parameters to each query.

#### Example query set with custom fields

In the following example, each entry includes a `category` custom field. If a search configuration references `{{category}}` in its query, it filters the results for each query by the `category` value in the corresponding entry:

```json
{"queryText": "phone", "category": "electronics"}
{"queryText": "steel", "category": "materials"}
{"queryText": "keyboard", "category": "electronics"}
```

## Managing query sets

You can retrieve or delete query sets using the following APIs.
Expand Down
29 changes: 28 additions & 1 deletion _search-plugins/search-relevance/search-configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
:--- | :--- | :---
`name` | String | The name of the search configuration.
`description` | String | Description of the search configuration.
`query` | Object | Defines the query in OpenSearch query DSL. Use `%SearchText%` as a placeholder for the user query. Needs to be escaped.
`query` | Object | Defines the query in OpenSearch query DSL, provided as a JSON string with the inner quotation marks escaped (for example, `"query": "{\"query\":{\"multi_match\":{...}}}"`). Use the `%SearchText%` placeholder or [Mustache](https://mustache.github.io/) template variables (such as {% raw %}`{{queryText}}`{% endraw %}) to substitute query set values at runtime. For more information, see [Using Mustache templates](#using-mustache-templates).
`index` | String | The target index queried by this search configuration.
`searchPipeline` | String | Specifies an existing search pipeline. Optional.

Expand All @@ -45,6 +45,33 @@
}
```

### Using Mustache templates

Instead of the `%SearchText%` placeholder, you can use [Mustache](https://mustache.github.io/) template variables in a query. If the `query` parameter contains double curly braces ({% raw %}`{{}}`{% endraw %}), OpenSearch renders the `query` as a Mustache template; otherwise, OpenSearch substitutes the `%SearchText%` placeholder. Existing `%SearchText%` configurations continue to work unchanged.

When OpenSearch renders a template, the following variables are available:

- {% raw %}`{{queryText}}`{% endraw %}: The `queryText` value from the current query set entry.

Check failure on line 54 in _search-plugins/search-relevance/search-configurations.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.HeadingPunctuation] Don't use punctuation at the end of a heading. Raw Output: {"message": "[OpenSearch.HeadingPunctuation] Don't use punctuation at the end of a heading.", "location": {"path": "_search-plugins/search-relevance/search-configurations.md", "range": {"start": {"line": 54, "column": 95}}}, "severity": "ERROR"}
- {% raw %}`{{<field_name>}}`{% endraw %}: Any custom field from the query set entry, referenced by its field name (for example, {% raw %}`{{category}}`{% endraw %}). For more information, see [Query sets]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/query-sets/).

OpenSearch automatically escapes substituted values, so query text that contains quotation marks or other special characters remains valid in JSON.

Mustache partials ({% raw %}`{{>...}}`{% endraw %}) are not supported and cause the query to be rejected.
{: .note}

For example, the following request creates a search configuration that matches the user query against the `title` field and filters the results by the `category` custom field from each query set entry:

```json
PUT _plugins/_search_relevance/search_configurations
{
"name": "mustache_filter",
"query": "{\"query\":{\"bool\":{\"must\":[{\"match\":{\"title\":\"{% raw %}{{queryText}}{% endraw %}\"}}],\"filter\":[{\"term\":{\"category\":\"{% raw %}{{category}}{% endraw %}\"}}]}}}",
"description": "Title match filtered by a category custom field",
"index": "ecommerce"
}
```
{% include copy-curl.html %}

## Managing search configurations

You can retrieve or delete configurations using the following APIs.
Expand Down
Loading