diff --git a/.github/vale/styles/Vocab/OpenSearch/Products/accept.txt b/.github/vale/styles/Vocab/OpenSearch/Products/accept.txt index e4be8846456..0ebd8ee56ef 100644 --- a/.github/vale/styles/Vocab/OpenSearch/Products/accept.txt +++ b/.github/vale/styles/Vocab/OpenSearch/Products/accept.txt @@ -79,6 +79,7 @@ Google Normalized Distance Google Workspace Gradle Grafana +Grok Debugger Hanja Helm Hjson diff --git a/_dashboards/discover/run-queries.md b/_dashboards/discover/run-queries.md index d1beca18ffe..4dbf1ac3ca8 100644 --- a/_dashboards/discover/run-queries.md +++ b/_dashboards/discover/run-queries.md @@ -151,4 +151,20 @@ To update your preferences, select **Settings** from the top menu: ## Using keyboard shortcuts -To view all available keyboard shortcuts, select **Help** from the top menu. \ No newline at end of file +To view all available keyboard shortcuts, select **Help** from the top menu. + +## Using the Grok Debugger + +The **Grok Debugger** is a built-in tool in Dev Tools that helps you build and test [Grok patterns]({{site.url}}{{site.baseurl}}/ingest-pipelines/processors/grok/) before using them in ingest pipelines. Grok patterns parse unstructured log data into structured fields. + +To use the Grok Debugger, follow these steps: + +1. On the top menu in Dev Tools, select **Grok Debugger**. +1. Enter a sample log message in the **Sample Data** field. +1. Enter your Grok pattern in the **Grok Pattern** field. +1. Select **Simulate** to test the pattern. + +The debugger displays the parsed fields in the output pane, as shown in the following image. + +![Grok Debugger]({{site.url}}{{site.baseurl}}/images/dev-tools/grok-debugger.png) + diff --git a/_ingest-pipelines/processors/grok.md b/_ingest-pipelines/processors/grok.md index 76ad9906503..1ef0f038540 100644 --- a/_ingest-pipelines/processors/grok.md +++ b/_ingest-pipelines/processors/grok.md @@ -2,16 +2,16 @@ layout: default title: Grok parent: Ingest processors -nav_order: 140 +nav_order: 120 --- -This documentation describes using the `grok` processor in OpenSearch ingest pipelines. Consider using the [Data Prepper `grok` processor]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/grok/), which runs on the OpenSearch cluster, if your use case involves large or complex datasets. -{: .note} - # Grok processor The `grok` processor is used to parse and structure unstructured data using pattern matching. You can use the `grok` processor to extract fields from log messages, web server access logs, application logs, and other log data that follows a consistent format. +This documentation describes using the `grok` processor in OpenSearch ingest pipelines. Consider using the [Data Prepper `grok` processor]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/grok/), which runs on the OpenSearch cluster, if your use case involves large or complex datasets. +{: .note} + ## Grok basics The `grok` processor uses a set of predefined patterns to match parts of the input text. Each pattern consists of a name and a regular expression. For example, the pattern `%{IP:ip_address}` matches an IP address and assigns it to the field `ip_address`. You can combine multiple patterns to create more complex expressions. For example, the pattern `%{IP:client} %{WORD:method} %{URIPATHPARM:request} %{NUMBER:bytes %NUMBER:duration}` matches a line from a web server access log and extracts the client IP address, the HTTP method, the request URI, the number of bytes sent, and the duration of the request. @@ -19,7 +19,7 @@ The `grok` processor uses a set of predefined patterns to match parts of the inp For a list of available predefined patterns, see [Grok patterns](https://github.com/opensearch-project/OpenSearch/blob/main/libs/grok/src/main/resources/patterns/grok-patterns). {: .tip} -The `grok` processor is built on the [Oniguruma regular expression library](https://github.com/kkos/oniguruma/blob/master/doc/RE) and supports all the patterns from that library. You can use the [Grok Debugger](https://grokdebugger.com/) tool to test and debug your grok expressions. +The `grok` processor is built on the [Oniguruma regular expression library](https://github.com/kkos/oniguruma/blob/master/doc/RE) and supports all the patterns from that library. You can use the built-in [Grok Debugger]({{site.url}}{{site.baseurl}}/dashboards/discover/run-queries/#using-the-grok-debugger) in OpenSearch Dashboards Dev Tools to test and debug your grok expressions. Note that patterns are *not anchored*. For performance and reliability, include a start-of-line anchor (`^`) in your pattern. {: .note} diff --git a/images/dev-tools/grok-debugger.png b/images/dev-tools/grok-debugger.png new file mode 100644 index 00000000000..0e28e2ec858 Binary files /dev/null and b/images/dev-tools/grok-debugger.png differ