From ae060c6786ca50aa232b5334fcce6daac3eb405d Mon Sep 17 00:00:00 2001 From: Bonusree Date: Mon, 15 Jun 2026 16:19:06 +0600 Subject: [PATCH 1/6] combined cluster Signed-off-by: Bonusree --- .../elasticsearch/reconfigure/_index.md | 10 + .../reconfigure/elasticsearch-combined.md | 398 ++++++++++++++ .../reconfigure/elasticsearch-topology.md | 512 ++++++++++++++++++ .../elasticsearch/reconfigure/overview.md | 65 +++ 4 files changed, 985 insertions(+) create mode 100644 docs/guides/elasticsearch/reconfigure/_index.md create mode 100644 docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md create mode 100644 docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md create mode 100644 docs/guides/elasticsearch/reconfigure/overview.md diff --git a/docs/guides/elasticsearch/reconfigure/_index.md b/docs/guides/elasticsearch/reconfigure/_index.md new file mode 100644 index 000000000..cc0d80742 --- /dev/null +++ b/docs/guides/elasticsearch/reconfigure/_index.md @@ -0,0 +1,10 @@ +--- +title: Reconfigure +menu: + docs_{{ .version }}: + identifier: es-reconfigure + name: Reconfigure + parent: es-elasticsearch-guides + weight: 55 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md new file mode 100644 index 000000000..6bd9f3c1f --- /dev/null +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md @@ -0,0 +1,398 @@ +--- +title: Reconfigure Elasticsearch Combined Cluster +menu: + docs_{{ .version }}: + identifier: es-reconfigure-combined + name: Combined Cluster + parent: es-reconfigure + weight: 20 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Reconfigure Elasticsearch Combined Cluster + +This guide will show you how to use `KubeDB` Ops-manager operator to reconfigure an Elasticsearch Combined cluster. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. + +- Install `KubeDB` Provisioner and Ops-manager operator in your cluster following the steps [here](/docs/setup/README.md). + +- You should be familiar with the following `KubeDB` concepts: + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [Combined Cluster](/docs/guides/elasticsearch/clustering/combined-cluster/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + - [Reconfigure Overview](/docs/guides/elasticsearch/reconfigure/overview.md) + +To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> **Note:** YAML files used in this tutorial are stored in [docs/examples/elasticsearch](/docs/examples/elasticsearch) directory of [kubedb/docs](https://github.com/kubedb/docs) repository. + +Now, we are going to deploy an `Elasticsearch` Combined cluster using a supported version by `KubeDB` operator. Then we are going to apply `ElasticsearchOpsRequest` to reconfigure its configuration. + +### Prepare Elasticsearch Combined Cluster + +Now, we are going to deploy an `Elasticsearch` combined cluster with version `xpack-8.19.9`. + +### Deploy Elasticsearch + +At first, we will create a secret with the `elasticsearch.yml` file containing required configuration settings. + +**elasticsearch.yml:** + +```yaml +indices.query.bool.max_clause_count: 2048 +``` + +Let's create a k8s secret containing the above configuration where the file name will be the key and the file-content as the value: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: es-combined-custom-config + namespace: demo +stringData: + elasticsearch.yml: |- + indices.query.bool.max_clause_count: 2048 +``` + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-combined-custom-config.yaml +secret/es-combined-custom-config created +``` + +In this section, we are going to create an Elasticsearch object specifying `spec.configuration.secretName` field to apply this custom configuration. Below is the YAML of the `Elasticsearch` CR that we are going to create, + +```yaml +apiVersion: kubedb.com/v1 +kind: Elasticsearch +metadata: + name: es-combined + namespace: demo +spec: + version: xpack-8.19.9 + enableSSL: true + replicas: 2 + configuration: + secretName: es-combined-custom-config + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Let's create the `Elasticsearch` CR we have shown above, + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-combined.yaml +elasticsearch.kubedb.com/es-combined created +``` + +Now, wait until `es-combined` has status `Ready`. i.e, + +```bash +$ kubectl get es -n demo +NAME VERSION STATUS AGE +es-combined xpack-8.19.9 Ready 20m +``` + +Now, we will check if the Elasticsearch has started with the custom configuration we have provided. + +Exec into the Elasticsearch pod and query the cluster settings to see the configuration: + +```bash +$ kubectl exec -it -n demo es-combined-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:X4gzeLWqUHKMoQT7" | grep max_clause_count + "max_clause_count" : "2048" + "max_clause_count" : "2048" +``` + +Here, we can see that our given configuration is applied to the Elasticsearch cluster for all nodes. `indices.query.bool.max_clause_count` is set to `2048` from the default value `1024`. + +### Reconfigure using new config secret + +Now we will reconfigure this cluster to set `indices.query.bool.max_clause_count` to `4096`. + +Update our `elasticsearch.yml` file with the new configuration. + +**elasticsearch.yml:** + +```yaml +indices.query.bool.max_clause_count: 4096 +``` + +Then, we will create a new secret with this configuration file. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-es-combined-custom-config + namespace: demo +stringData: + elasticsearch.yml: |- + indices.query.bool.max_clause_count: 4096 +``` + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/new-es-combined-custom-config.yaml +secret/new-es-combined-custom-config created +``` + +#### Create ElasticsearchOpsRequest + +Now, we will use this secret to replace the previous secret using an `ElasticsearchOpsRequest` CR. The `ElasticsearchOpsRequest` yaml is given below, + +```yaml +apiVersion: ops.kubedb.com/v1alpha1 +kind: ElasticsearchOpsRequest +metadata: + name: esops-reconfigure-combined + namespace: demo +spec: + type: Reconfigure + databaseRef: + name: es-combined + configuration: + configSecret: + name: new-es-combined-custom-config + timeout: 5m + apply: IfReady +``` + +Here, + +- `spec.databaseRef.name` specifies that we are reconfiguring `es-combined` database. +- `spec.type` specifies that we are performing `Reconfigure` on our database. +- `spec.configuration.configSecret.name` specifies the name of the new secret. + +Let's create the `ElasticsearchOpsRequest` CR we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-reconfigure-update-combined.yaml +elasticsearchopsrequest.ops.kubedb.com/esops-reconfigure-combined created +``` + +#### Verify the new configuration is working + +If everything goes well, `KubeDB` Ops-manager operator will update the `configSecret` of `Elasticsearch` object. + +Let's wait for `ElasticsearchOpsRequest` to be `Successful`. Run the following command to watch `ElasticsearchOpsRequest` CR, + +```bash +$ kubectl get elasticsearchopsrequests -n demo +NAME TYPE STATUS AGE +esops-reconfigure-combined Reconfigure Successful 73s +``` + +We can see from the above output that the `ElasticsearchOpsRequest` has succeeded. If we describe the `ElasticsearchOpsRequest` we will get an overview of the steps that were followed to reconfigure the database. + +```bash +$ kubectl describe elasticsearchopsrequest -n demo esops-reconfigure-combined +Name: esops-reconfigure-combined +Namespace: demo +Labels: +Annotations: +API Version: ops.kubedb.com/v1alpha1 +Kind: ElasticsearchOpsRequest +Metadata: + Creation Timestamp: 2026-06-15T09:43:13Z + Generation: 1 + Resource Version: 431415 + UID: 25c9da3e-6e1f-4073-a98c-8df9061307e5 +Spec: + Apply: IfReady + Configuration: + Config Secret: + Name: new-es-combined-custom-config + Restart: auto + Database Ref: + Name: es-combined + Max Retries: 1 + Timeout: 5m + Type: Reconfigure +Status: + Conditions: + Last Transition Time: 2026-06-15T09:43:13Z + Message: Elasticsearch ops request is Reconfiguring + Observed Generation: 1 + Reason: Reconfigure + Status: True + Type: Reconfigure + Last Transition Time: 2026-06-15T09:43:18Z + Message: Successfully updated petSets + Observed Generation: 1 + Reason: UpdatePetSets + Status: True + Type: UpdatePetSets + Last Transition Time: 2026-06-15T09:43:28Z + Message: pod exists; ConditionStatus:True; PodName:es-combined-0 + Observed Generation: 1 + Status: True + Type: PodExists--es-combined-0 + Last Transition Time: 2026-06-15T09:43:28Z + Message: create es client; ConditionStatus:True; PodName:es-combined-0 + Observed Generation: 1 + Status: True + Type: CreateEsClient--es-combined-0 + Last Transition Time: 2026-06-15T09:43:29Z + Message: evict pod; ConditionStatus:True; PodName:es-combined-0 + Observed Generation: 1 + Status: True + Type: EvictPod--es-combined-0 + Last Transition Time: 2026-06-15T09:43:53Z + Message: create es client; ConditionStatus:True + Observed Generation: 1 + Status: True + Type: CreateEsClient + Last Transition Time: 2026-06-15T09:43:43Z + Message: pod exists; ConditionStatus:True; PodName:es-combined-1 + Observed Generation: 1 + Status: True + Type: PodExists--es-combined-1 + Last Transition Time: 2026-06-15T09:43:43Z + Message: create es client; ConditionStatus:True; PodName:es-combined-1 + Observed Generation: 1 + Status: True + Type: CreateEsClient--es-combined-1 + Last Transition Time: 2026-06-15T09:43:44Z + Message: evict pod; ConditionStatus:True; PodName:es-combined-1 + Observed Generation: 1 + Status: True + Type: EvictPod--es-combined-1 + Last Transition Time: 2026-06-15T09:43:58Z + Message: Successfully restarted all nodes + Observed Generation: 1 + Reason: RestartNodes + Status: True + Type: RestartNodes + Last Transition Time: 2026-06-15T09:43:59Z + Message: Successfully completed the modification process. + Observed Generation: 1 + Reason: Successful + Status: True + Type: Successful + Observed Generation: 1 + Phase: Successful +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal PauseDatabase 98s KubeDB Ops-manager Operator Pausing Elasticsearch demo/es-combined + Warning pod exists; ConditionStatus:True; PodName:es-combined-0 85s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-combined-0 + Warning create es client; ConditionStatus:True; PodName:es-combined-0 85s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-combined-0 + Warning evict pod; ConditionStatus:True; PodName:es-combined-0 84s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-combined-0 + Warning create es client; ConditionStatus:False 80s KubeDB Ops-manager Operator create es client; ConditionStatus:False + Warning create es client; ConditionStatus:True 75s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Warning pod exists; ConditionStatus:True; PodName:es-combined-1 70s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-combined-1 + Warning create es client; ConditionStatus:True; PodName:es-combined-1 70s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-combined-1 + Warning evict pod; ConditionStatus:True; PodName:es-combined-1 69s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-combined-1 + Warning create es client; ConditionStatus:False 65s KubeDB Ops-manager Operator create es client; ConditionStatus:False + Warning create es client; ConditionStatus:True 60s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Normal RestartNodes 55s KubeDB Ops-manager Operator Successfully restarted all nodes + Normal Successful 54s KubeDB Ops-manager Operator Successfully reconfigured all elasticsearch nodes. +``` + +Now let's exec into one of the instances and query the cluster settings to check the new configuration. + +```bash +$ kubectl exec -it -n demo es-combined-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:X4gzeLWqUHKMoQT7" | grep max_clause_count + "max_clause_count" : "4096" + "max_clause_count" : "4096" +``` + +As we can see from the configuration of the ready Elasticsearch, the value of `indices.query.bool.max_clause_count` has been changed from `2048` to `4096`. So the reconfiguration of the cluster is successful. + + +### Reconfigure using apply config + +Now we will reconfigure this cluster again to set `indices.query.bool.max_clause_count` to `8192`. This time we won't use a new secret. We will use the `applyConfig` field of the `ElasticsearchOpsRequest`. This will merge the new config into the existing secret. + +#### Create ElasticsearchOpsRequest + +Now, we will use the new configuration in the `applyConfig` field in the `ElasticsearchOpsRequest` CR. The `ElasticsearchOpsRequest` yaml is given below, + +```yaml +apiVersion: ops.kubedb.com/v1alpha1 +kind: ElasticsearchOpsRequest +metadata: + name: esops-reconfigure-apply-combined + namespace: demo +spec: + type: Reconfigure + databaseRef: + name: es-combined + configuration: + applyConfig: + elasticsearch.yml: | + indices.query.bool.max_clause_count: 8192 + timeout: 5m + apply: IfReady +``` + +Here, + +- `spec.databaseRef.name` specifies that we are reconfiguring `es-combined` cluster. +- `spec.type` specifies that we are performing `Reconfigure` on Elasticsearch. +- `spec.configuration.applyConfig` specifies the new configuration that will be merged into the existing secret. + +Let's create the `ElasticsearchOpsRequest` CR we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-reconfigure-apply-combined.yaml +elasticsearchopsrequest.ops.kubedb.com/esops-reconfigure-apply-combined created +``` + +#### Verify the new configuration is working + +If everything goes well, `KubeDB` Ops-manager operator will merge this new config with the existing configuration. + +Let's wait for `ElasticsearchOpsRequest` to be `Successful`. Run the following command to watch `ElasticsearchOpsRequest` CR, + +```bash +$ kubectl get elasticsearchopsrequests -n demo esops-reconfigure-apply-combined +NAME TYPE STATUS AGE +esops-reconfigure-apply-combined Reconfigure Successful 118s +``` + +We can see from the above output that the `ElasticsearchOpsRequest` has succeeded. Now let's exec into one of the instances and check the new configuration. + +```bash +$ kubectl exec -it -n demo es-combined-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:X4gzeLWqUHKMoQT7" | grep max_clause_count + "max_clause_count" : "8192" + "max_clause_count" : "8192" +``` + +As we can see from the configuration of the ready Elasticsearch, the value of `indices.query.bool.max_clause_count` has been changed from `4096` to `8192`. So the reconfiguration of the database using the `applyConfig` field is successful. + + +## Cleaning Up + +To clean up the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete es -n demo es-combined +kubectl delete elasticsearchopsrequest -n demo esops-reconfigure-apply-combined esops-reconfigure-combined +kubectl delete secret -n demo es-combined-custom-config new-es-combined-custom-config +kubectl delete namespace demo +``` + +## Next Steps + +- Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md new file mode 100644 index 000000000..b6e711d7a --- /dev/null +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md @@ -0,0 +1,512 @@ +--- +title: Reconfigure Elasticsearch Topology Cluster +menu: + docs_{{ .version }}: + identifier: es-reconfigure-topology + name: Topology Cluster + parent: es-reconfigure + weight: 30 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Reconfigure Elasticsearch Topology Cluster + +This guide will show you how to use `KubeDB` Ops-manager operator to reconfigure an Elasticsearch Topology cluster. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. + +- Install `KubeDB` Provisioner and Ops-manager operator in your cluster following the steps [here](/docs/setup/README.md). + +- You should be familiar with the following `KubeDB` concepts: + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [Topology Cluster](/docs/guides/elasticsearch/clustering/topology-cluster/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + - [Reconfigure Overview](/docs/guides/elasticsearch/reconfigure/overview.md) + +To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> **Note:** YAML files used in this tutorial are stored in [docs/examples/elasticsearch](/docs/examples/elasticsearch) directory of [kubedb/docs](https://github.com/kubedb/docs) repository. + +Now, we are going to deploy an `Elasticsearch` Topology cluster using a supported version by `KubeDB` operator. Then we are going to apply `ElasticsearchOpsRequest` to reconfigure its configuration. + +### Prepare Elasticsearch Topology Cluster + +Now, we are going to deploy an `Elasticsearch` topology cluster with version `xpack-8.19.9`. + +### Deploy Elasticsearch + +At first, we will create a secret with role-prefixed configuration files. In a topology cluster, you can target specific node roles by prefixing the filename with the role name (e.g., `master-elasticsearch.yml` applies only to master nodes, `data-elasticsearch.yml` applies only to data nodes). + +**master-elasticsearch.yml:** + +```yaml +cluster.max_shards_per_node: 2000 +``` + +**data-elasticsearch.yml:** + +```yaml +indices.query.bool.max_clause_count: 2048 +``` + +**ingest-elasticsearch.yml:** + +```yaml +http.max_content_length: 200mb +``` + +Here, `cluster.max_shards_per_node` is set to `2000` (default `1000`) for master nodes, `indices.query.bool.max_clause_count` is set to `2048` (default `1024`) for data nodes, and `http.max_content_length` is set to `200mb` (default `100mb`) for ingest nodes. + +Let's create a k8s secret containing the above configuration where the file name will be the key and the file-content as the value: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: es-topology-custom-config + namespace: demo +stringData: + master-elasticsearch.yml: |- + cluster.max_shards_per_node: 2000 + data-elasticsearch.yml: |- + indices.query.bool.max_clause_count: 2048 + ingest-elasticsearch.yml: |- + http.max_content_length: 200mb +``` + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-topology-custom-config.yaml +secret/es-topology-custom-config created +``` + +In this section, we are going to create an Elasticsearch object specifying `spec.configuration.secretName` field to apply this custom configuration. Below is the YAML of the `Elasticsearch` CR that we are going to create, + +```yaml +apiVersion: kubedb.com/v1 +kind: Elasticsearch +metadata: + name: es-topology + namespace: demo +spec: + version: xpack-8.19.9 + enableSSL: true + configuration: + secretName: es-topology-custom-config + topology: + master: + replicas: 1 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + data: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + ingest: + replicas: 1 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Let's create the `Elasticsearch` CR we have shown above, + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-topology.yaml +elasticsearch.kubedb.com/es-topology created +``` + +Now, wait until `es-topology` has status `Ready`. i.e, + +```bash +$ kubectl get es -n demo -w +NAME VERSION STATUS AGE +es-topology xpack-8.19.9 Provisioning 0s +es-topology xpack-8.19.9 Provisioning 24s +. +. +es-topology xpack-8.19.9 Ready 92s +``` + +Now, we will check if the Elasticsearch has started with the custom configuration we have provided. + +Exec into the master node and check the cluster setting: + +```bash +$ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node + "cluster.max_shards_per_node" : "2000" +``` + +Exec into a data node and check the index settings: + +```bash +$ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count + "indices.query.bool.max_clause_count" : "2048" +``` + +Exec into the ingest node and check the HTTP settings: + +```bash +$ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length + "max_content_length" : "200mb" +``` + +Here, we can see that our given configurations are applied to the respective node roles. + +### Reconfigure using new config secret + +Now we will reconfigure this cluster to update `cluster.max_shards_per_node` to `3000` for master nodes, `indices.query.bool.max_clause_count` to `4096` for data nodes, and `http.max_content_length` to `300mb` for ingest nodes. + +Update our configuration files with the new values. + +**master-elasticsearch.yml:** + +```yaml +cluster.max_shards_per_node: 3000 +``` + +**data-elasticsearch.yml:** + +```yaml +indices.query.bool.max_clause_count: 4096 +``` + +**ingest-elasticsearch.yml:** + +```yaml +http.max_content_length: 300mb +``` + +Then, we will create a new secret with these configuration files. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-es-topology-custom-config + namespace: demo +stringData: + master-elasticsearch.yml: |- + cluster.max_shards_per_node: 3000 + data-elasticsearch.yml: |- + indices.query.bool.max_clause_count: 4096 + ingest-elasticsearch.yml: |- + http.max_content_length: 300mb +``` + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/new-es-topology-custom-config.yaml +secret/new-es-topology-custom-config created +``` + +#### Create ElasticsearchOpsRequest + +Now, we will use this secret to replace the previous secret using an `ElasticsearchOpsRequest` CR. The `ElasticsearchOpsRequest` yaml is given below, + +```yaml +apiVersion: ops.kubedb.com/v1alpha1 +kind: ElasticsearchOpsRequest +metadata: + name: esops-reconfigure-topology + namespace: demo +spec: + type: Reconfigure + databaseRef: + name: es-topology + configuration: + configSecret: + name: new-es-topology-custom-config + timeout: 5m + apply: IfReady +``` + +Here, + +- `spec.databaseRef.name` specifies that we are reconfiguring `es-topology` database. +- `spec.type` specifies that we are performing `Reconfigure` on our database. +- `spec.configuration.configSecret.name` specifies the name of the new secret. + +Let's create the `ElasticsearchOpsRequest` CR we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-reconfigure-update-topology.yaml +elasticsearchopsrequest.ops.kubedb.com/esops-reconfigure-topology created +``` + +#### Verify the new configuration is working + +If everything goes well, `KubeDB` Ops-manager operator will update the `configSecret` of `Elasticsearch` object. + +Let's wait for `ElasticsearchOpsRequest` to be `Successful`. Run the following command to watch `ElasticsearchOpsRequest` CR, + +```bash +$ kubectl get elasticsearchopsrequests -n demo +NAME TYPE STATUS AGE +esops-reconfigure-topology Reconfigure Successful 4m55s +``` + +We can see from the above output that the `ElasticsearchOpsRequest` has succeeded. If we describe the `ElasticsearchOpsRequest` we will get an overview of the steps that were followed to reconfigure the database. + +```bash +$ kubectl describe elasticsearchopsrequest -n demo esops-reconfigure-topology +Name: esops-reconfigure-topology +Namespace: demo +Labels: +Annotations: +API Version: ops.kubedb.com/v1alpha1 +Kind: ElasticsearchOpsRequest +Metadata: + Creation Timestamp: 2024-08-02T05:08:37Z + Generation: 1 + Resource Version: 332491 + UID: b6e8cb1b-d29f-445e-bb01-60d29012c7eb +Spec: + Apply: IfReady + Configuration: + Config Secret: + Name: new-es-topology-custom-config + Database Ref: + Name: es-topology + Timeout: 5m + Type: Reconfigure +Status: + Conditions: + Last Transition Time: 2024-08-02T05:08:37Z + Message: Elasticsearch ops-request has started to reconfigure elasticsearch nodes + Observed Generation: 1 + Reason: Reconfigure + Status: True + Type: Reconfigure + Last Transition Time: 2024-08-02T05:09:42Z + Message: successfully reconciled the Elasticsearch with new configure + Observed Generation: 1 + Reason: UpdatePetSets + Status: True + Type: UpdatePetSets + Last Transition Time: 2024-08-02T05:09:47Z + Message: get pod; ConditionStatus:True; PodName:es-topology-master-0 + Observed Generation: 1 + Status: True + Type: GetPod--es-topology-master-0 + Last Transition Time: 2024-08-02T05:09:47Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-master-0 + Observed Generation: 1 + Status: True + Type: EvictPod--es-topology-master-0 + Last Transition Time: 2024-08-02T05:10:02Z + Message: check pod running; ConditionStatus:True; PodName:es-topology-master-0 + Observed Generation: 1 + Status: True + Type: CheckPodRunning--es-topology-master-0 + Last Transition Time: 2024-08-02T05:10:07Z + Message: get pod; ConditionStatus:True; PodName:es-topology-data-0 + Observed Generation: 1 + Status: True + Type: GetPod--es-topology-data-0 + Last Transition Time: 2024-08-02T05:10:07Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-data-0 + Observed Generation: 1 + Status: True + Type: EvictPod--es-topology-data-0 + Last Transition Time: 2024-08-02T05:10:22Z + Message: check pod running; ConditionStatus:True; PodName:es-topology-data-0 + Observed Generation: 1 + Status: True + Type: CheckPodRunning--es-topology-data-0 + Last Transition Time: 2024-08-02T05:10:27Z + Message: get pod; ConditionStatus:True; PodName:es-topology-data-1 + Observed Generation: 1 + Status: True + Type: GetPod--es-topology-data-1 + Last Transition Time: 2024-08-02T05:10:27Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-data-1 + Observed Generation: 1 + Status: True + Type: EvictPod--es-topology-data-1 + Last Transition Time: 2024-08-02T05:10:42Z + Message: check pod running; ConditionStatus:True; PodName:es-topology-data-1 + Observed Generation: 1 + Status: True + Type: CheckPodRunning--es-topology-data-1 + Last Transition Time: 2024-08-02T05:10:47Z + Message: get pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Observed Generation: 1 + Status: True + Type: GetPod--es-topology-ingest-0 + Last Transition Time: 2024-08-02T05:10:47Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Observed Generation: 1 + Status: True + Type: EvictPod--es-topology-ingest-0 + Last Transition Time: 2024-08-02T05:11:02Z + Message: check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 + Observed Generation: 1 + Status: True + Type: CheckPodRunning--es-topology-ingest-0 + Last Transition Time: 2024-08-02T05:11:07Z + Message: Successfully restarted all nodes + Observed Generation: 1 + Reason: RestartNodes + Status: True + Type: RestartNodes + Last Transition Time: 2024-08-02T05:11:09Z + Message: Successfully completed reconfigure elasticsearch + Observed Generation: 1 + Reason: Successful + Status: True + Type: Successful + Observed Generation: 1 + Phase: Successful +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Starting 3m7s KubeDB Ops-manager Operator Start processing for ElasticsearchOpsRequest: demo/esops-reconfigure-topology + Normal Starting 3m7s KubeDB Ops-manager Operator Pausing Elasticsearch database: demo/es-topology + Normal Successful 3m7s KubeDB Ops-manager Operator Successfully paused Elasticsearch database: demo/es-topology for ElasticsearchOpsRequest: esops-reconfigure-topology + Normal UpdatePetSets 2m2s KubeDB Ops-manager Operator successfully reconciled the Elasticsearch with new configure + Warning get pod; ConditionStatus:True; PodName:es-topology-master-0 117s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-master-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-master-0 117s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-master-0 + Warning check pod running; ConditionStatus:False; PodName:es-topology-master-0 112s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-master-0 + Warning check pod running; ConditionStatus:True; PodName:es-topology-master-0 102s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-master-0 + Warning get pod; ConditionStatus:True; PodName:es-topology-data-0 97s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-data-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-data-0 97s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-0 + Warning check pod running; ConditionStatus:False; PodName:es-topology-data-0 92s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-data-0 + Warning check pod running; ConditionStatus:True; PodName:es-topology-data-0 82s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-data-0 + Warning get pod; ConditionStatus:True; PodName:es-topology-data-1 77s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-data-1 + Warning evict pod; ConditionStatus:True; PodName:es-topology-data-1 77s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-1 + Warning check pod running; ConditionStatus:False; PodName:es-topology-data-1 72s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-data-1 + Warning check pod running; ConditionStatus:True; PodName:es-topology-data-1 62s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-data-1 + Warning get pod; ConditionStatus:True; PodName:es-topology-ingest-0 57s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 57s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Warning check pod running; ConditionStatus:False; PodName:es-topology-ingest-0 52s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-ingest-0 + Warning check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 42s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 + Normal RestartNodes 37s KubeDB Ops-manager Operator Successfully restarted all nodes + Normal Starting 35s KubeDB Ops-manager Operator Resuming Elasticsearch database: demo/es-topology + Normal Successful 35s KubeDB Ops-manager Operator Successfully resumed Elasticsearch database: demo/es-topology for ElasticsearchOpsRequest: esops-reconfigure-topology +``` + +Now let's exec into a master node, a data node, and the ingest node to verify the new configuration. + +```bash +$ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node + "cluster.max_shards_per_node" : "3000" + +$ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count + "indices.query.bool.max_clause_count" : "4096" + +$ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length + "max_content_length" : "300mb" +``` + +As we can see, the values have been updated on their respective node roles. So the reconfiguration of the cluster is successful. + + +### Reconfigure using apply config + +Now we will reconfigure this cluster again using the `applyConfig` field. This will merge the new config into the existing secret without requiring a new secret. + +#### Create ElasticsearchOpsRequest + +```yaml +apiVersion: ops.kubedb.com/v1alpha1 +kind: ElasticsearchOpsRequest +metadata: + name: esops-reconfigure-apply-topology + namespace: demo +spec: + type: Reconfigure + databaseRef: + name: es-topology + configuration: + applyConfig: + master-elasticsearch.yml: | + cluster.max_shards_per_node: 4000 + data-elasticsearch.yml: | + indices.query.bool.max_clause_count: 8192 + ingest-elasticsearch.yml: | + http.max_content_length: 400mb + timeout: 5m + apply: IfReady +``` + +Here, + +- `spec.databaseRef.name` specifies that we are reconfiguring `es-topology` cluster. +- `spec.type` specifies that we are performing `Reconfigure` on Elasticsearch. +- `spec.configuration.applyConfig` specifies the new configuration that will be merged into the existing secret. + +Let's create the `ElasticsearchOpsRequest` CR we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure/es-reconfigure-apply-topology.yaml +elasticsearchopsrequest.ops.kubedb.com/esops-reconfigure-apply-topology created +``` + +#### Verify the new configuration is working + +Let's wait for `ElasticsearchOpsRequest` to be `Successful`. + +```bash +$ kubectl get elasticsearchopsrequests -n demo esops-reconfigure-apply-topology +NAME TYPE STATUS AGE +esops-reconfigure-apply-topology Reconfigure Successful 55s +``` + +Now let's verify the updated values on each node role. + +```bash +$ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node + "cluster.max_shards_per_node" : "4000" + +$ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count + "indices.query.bool.max_clause_count" : "8192" + +$ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length + "max_content_length" : "400mb" +``` + +As we can see, `cluster.max_shards_per_node` has been changed from `3000` to `4000` on master nodes, `indices.query.bool.max_clause_count` has been changed from `4096` to `8192` on data nodes, and `http.max_content_length` has been changed from `300mb` to `400mb` on ingest nodes. So the reconfiguration using the `applyConfig` field is successful. + + +## Cleaning Up + +To clean up the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete es -n demo es-topology +kubectl delete elasticsearchopsrequest -n demo esops-reconfigure-apply-topology esops-reconfigure-topology +kubectl delete secret -n demo es-topology-custom-config new-es-topology-custom-config +kubectl delete ns demo +``` + +## Next Steps + +- Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/reconfigure/overview.md b/docs/guides/elasticsearch/reconfigure/overview.md new file mode 100644 index 000000000..0621bb5e9 --- /dev/null +++ b/docs/guides/elasticsearch/reconfigure/overview.md @@ -0,0 +1,65 @@ +--- +title: Reconfiguring Elasticsearch +menu: + docs_{{ .version }}: + identifier: es-reconfigure-overview + name: Overview + parent: es-reconfigure + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Reconfiguring Elasticsearch + +This guide will give an overview on how KubeDB Ops-manager operator reconfigures `Elasticsearch` components such as Combined, Topology (master, data, ingest), etc. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + +## How Reconfiguring Elasticsearch Process Works + +The following diagram shows how KubeDB Ops-manager operator reconfigures `Elasticsearch` components. Open the image in a new tab to see the enlarged version. + +
+ Reconfiguring process of Elasticsearch +
Fig: Reconfiguring process of Elasticsearch
+
+ +The Reconfiguring Elasticsearch process consists of the following steps: + +1. At first, a user creates an `Elasticsearch` Custom Resource (CR). + +2. `KubeDB` Provisioner operator watches the `Elasticsearch` CR. + +3. When the operator finds an `Elasticsearch` CR, it creates the required number of `PetSets` and related necessary stuff like secrets, services, etc. + +4. Then, in order to reconfigure the various components (ie. Combined, Topology) of the `Elasticsearch`, the user creates an `ElasticsearchOpsRequest` CR with desired information. + +5. `KubeDB` Ops-manager operator watches the `ElasticsearchOpsRequest` CR. + +6. When it finds an `ElasticsearchOpsRequest` CR, it halts the `Elasticsearch` object which is referred from the `ElasticsearchOpsRequest`. So, the `KubeDB` Provisioner operator doesn't perform any operations on the `Elasticsearch` object during the reconfiguring process. + +7. Then the `KubeDB` Ops-manager operator will replace the existing configuration with the new configuration provided or merge the new configuration with the existing configuration according to the `ElasticsearchOpsRequest` CR. + +8. Then the `KubeDB` Ops-manager operator will restart the related PetSet Pods so that they restart with the new configuration defined in the `ElasticsearchOpsRequest` CR. + +9. After the successful reconfiguring of the `Elasticsearch` components, the `KubeDB` Ops-manager operator resumes the `Elasticsearch` object so that the `KubeDB` Provisioner operator resumes its usual operations. +### spec.configuration Fields + +The `ElasticsearchOpsRequest` with `type: Reconfigure` uses the following sub-fields under `spec.configuration`: + +| Field | Description | +|---|---| +| `configSecret` | Reference to a `Secret` containing the full custom configuration file(s) for the database. | +| `secureConfigSecret` | Reference to a `Secret` containing secure settings (e.g., keystore passwords). | +| `applyConfig` | Inline map of `filename: \| content` entries. Merged into the existing `ConfigSecret`. If no `ConfigSecret` exists, a new secret named `{db-name}-user-config` is created. | +| `removeCustomConfig` | Set to `true` to remove all user-provided configuration and revert to the operator-generated defaults. | +| `removeSecureCustomConfig` | Set to `true` to remove user-provided secure settings and revert to the default empty keystore. | + +In the next docs, we are going to show a step by step guide on reconfiguring Elasticsearch components using `ElasticsearchOpsRequest` CRD. From 2d6ff739d6a96f0bc69427eff9d3cca7b2f926b3 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Mon, 15 Jun 2026 17:33:32 +0600 Subject: [PATCH 2/6] tls init, topo reconfig Signed-off-by: Bonusree --- .../reconfigure/elasticsearch-topology.md | 179 ++++++------ docs/guides/elasticsearch/tls/_index.md | 10 + .../tls/elasticsearch-combined.md | 235 ++++++++++++++++ .../tls/elasticsearch-topology.md | 255 ++++++++++++++++++ docs/guides/elasticsearch/tls/overview.md | 70 +++++ 5 files changed, 664 insertions(+), 85 deletions(-) create mode 100644 docs/guides/elasticsearch/tls/_index.md create mode 100644 docs/guides/elasticsearch/tls/elasticsearch-combined.md create mode 100644 docs/guides/elasticsearch/tls/elasticsearch-topology.md create mode 100644 docs/guides/elasticsearch/tls/overview.md diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md index b6e711d7a..92830b1bb 100644 --- a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md @@ -159,21 +159,22 @@ Exec into the master node and check the cluster setting: ```bash $ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node - "cluster.max_shards_per_node" : "2000" + "max_shards_per_node" : "2000", ``` Exec into a data node and check the index settings: ```bash $ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count - "indices.query.bool.max_clause_count" : "2048" + "max_clause_count" : "2048" + "max_clause_count" : "2048" ``` Exec into the ingest node and check the HTTP settings: ```bash $ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length - "max_content_length" : "200mb" + "max_content_length" : "200mb", ``` Here, we can see that our given configurations are applied to the respective node roles. @@ -281,101 +282,108 @@ Annotations: API Version: ops.kubedb.com/v1alpha1 Kind: ElasticsearchOpsRequest Metadata: - Creation Timestamp: 2024-08-02T05:08:37Z + Creation Timestamp: 2026-06-15T11:13:59Z Generation: 1 - Resource Version: 332491 - UID: b6e8cb1b-d29f-445e-bb01-60d29012c7eb + Resource Version: 434719 + UID: a050a578-420c-4bb8-a390-3b32b369ec65 Spec: Apply: IfReady Configuration: Config Secret: - Name: new-es-topology-custom-config + Name: new-es-topology-custom-config + Restart: auto Database Ref: - Name: es-topology - Timeout: 5m - Type: Reconfigure + Name: es-topology + Max Retries: 1 + Timeout: 5m + Type: Reconfigure Status: Conditions: - Last Transition Time: 2024-08-02T05:08:37Z - Message: Elasticsearch ops-request has started to reconfigure elasticsearch nodes + Last Transition Time: 2026-06-15T11:13:59Z + Message: Elasticsearch ops request is Reconfiguring Observed Generation: 1 Reason: Reconfigure Status: True Type: Reconfigure - Last Transition Time: 2024-08-02T05:09:42Z - Message: successfully reconciled the Elasticsearch with new configure + Last Transition Time: 2026-06-15T11:14:09Z + Message: Successfully updated petSets Observed Generation: 1 Reason: UpdatePetSets Status: True Type: UpdatePetSets - Last Transition Time: 2024-08-02T05:09:47Z - Message: get pod; ConditionStatus:True; PodName:es-topology-master-0 + Last Transition Time: 2026-06-15T11:14:19Z + Message: pod exists; ConditionStatus:True; PodName:es-topology-ingest-0 Observed Generation: 1 Status: True - Type: GetPod--es-topology-master-0 - Last Transition Time: 2024-08-02T05:09:47Z - Message: evict pod; ConditionStatus:True; PodName:es-topology-master-0 + Type: PodExists--es-topology-ingest-0 + Last Transition Time: 2026-06-15T11:14:19Z + Message: create es client; ConditionStatus:True; PodName:es-topology-ingest-0 Observed Generation: 1 Status: True - Type: EvictPod--es-topology-master-0 - Last Transition Time: 2024-08-02T05:10:02Z - Message: check pod running; ConditionStatus:True; PodName:es-topology-master-0 + Type: CreateEsClient--es-topology-ingest-0 + Last Transition Time: 2026-06-15T11:14:19Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Observed Generation: 1 + Status: True + Type: EvictPod--es-topology-ingest-0 + Last Transition Time: 2026-06-15T11:14:59Z + Message: create es client; ConditionStatus:True + Observed Generation: 1 + Status: True + Type: CreateEsClient + Last Transition Time: 2026-06-15T11:14:34Z + Message: pod exists; ConditionStatus:True; PodName:es-topology-data-0 Observed Generation: 1 Status: True - Type: CheckPodRunning--es-topology-master-0 - Last Transition Time: 2024-08-02T05:10:07Z - Message: get pod; ConditionStatus:True; PodName:es-topology-data-0 + Type: PodExists--es-topology-data-0 + Last Transition Time: 2026-06-15T11:14:34Z + Message: create es client; ConditionStatus:True; PodName:es-topology-data-0 Observed Generation: 1 Status: True - Type: GetPod--es-topology-data-0 - Last Transition Time: 2024-08-02T05:10:07Z + Type: CreateEsClient--es-topology-data-0 + Last Transition Time: 2026-06-15T11:14:34Z Message: evict pod; ConditionStatus:True; PodName:es-topology-data-0 Observed Generation: 1 Status: True Type: EvictPod--es-topology-data-0 - Last Transition Time: 2024-08-02T05:10:22Z - Message: check pod running; ConditionStatus:True; PodName:es-topology-data-0 + Last Transition Time: 2026-06-15T11:14:49Z + Message: pod exists; ConditionStatus:True; PodName:es-topology-data-1 Observed Generation: 1 Status: True - Type: CheckPodRunning--es-topology-data-0 - Last Transition Time: 2024-08-02T05:10:27Z - Message: get pod; ConditionStatus:True; PodName:es-topology-data-1 + Type: PodExists--es-topology-data-1 + Last Transition Time: 2026-06-15T11:14:49Z + Message: create es client; ConditionStatus:True; PodName:es-topology-data-1 Observed Generation: 1 Status: True - Type: GetPod--es-topology-data-1 - Last Transition Time: 2024-08-02T05:10:27Z + Type: CreateEsClient--es-topology-data-1 + Last Transition Time: 2026-06-15T11:14:49Z Message: evict pod; ConditionStatus:True; PodName:es-topology-data-1 Observed Generation: 1 Status: True Type: EvictPod--es-topology-data-1 - Last Transition Time: 2024-08-02T05:10:42Z - Message: check pod running; ConditionStatus:True; PodName:es-topology-data-1 + Last Transition Time: 2026-06-15T11:15:04Z + Message: pod exists; ConditionStatus:True; PodName:es-topology-master-0 Observed Generation: 1 Status: True - Type: CheckPodRunning--es-topology-data-1 - Last Transition Time: 2024-08-02T05:10:47Z - Message: get pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Type: PodExists--es-topology-master-0 + Last Transition Time: 2026-06-15T11:15:04Z + Message: create es client; ConditionStatus:True; PodName:es-topology-master-0 Observed Generation: 1 Status: True - Type: GetPod--es-topology-ingest-0 - Last Transition Time: 2024-08-02T05:10:47Z - Message: evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 - Observed Generation: 1 - Status: True - Type: EvictPod--es-topology-ingest-0 - Last Transition Time: 2024-08-02T05:11:02Z - Message: check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 + Type: CreateEsClient--es-topology-master-0 + Last Transition Time: 2026-06-15T11:15:04Z + Message: evict pod; ConditionStatus:True; PodName:es-topology-master-0 Observed Generation: 1 Status: True - Type: CheckPodRunning--es-topology-ingest-0 - Last Transition Time: 2024-08-02T05:11:07Z + Type: EvictPod--es-topology-master-0 + Last Transition Time: 2026-06-15T11:15:14Z Message: Successfully restarted all nodes Observed Generation: 1 Reason: RestartNodes Status: True Type: RestartNodes - Last Transition Time: 2024-08-02T05:11:09Z - Message: Successfully completed reconfigure elasticsearch + Last Transition Time: 2026-06-15T11:15:15Z + Message: Successfully completed the modification process. Observed Generation: 1 Reason: Successful Status: True @@ -383,44 +391,44 @@ Status: Observed Generation: 1 Phase: Successful Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal Starting 3m7s KubeDB Ops-manager Operator Start processing for ElasticsearchOpsRequest: demo/esops-reconfigure-topology - Normal Starting 3m7s KubeDB Ops-manager Operator Pausing Elasticsearch database: demo/es-topology - Normal Successful 3m7s KubeDB Ops-manager Operator Successfully paused Elasticsearch database: demo/es-topology for ElasticsearchOpsRequest: esops-reconfigure-topology - Normal UpdatePetSets 2m2s KubeDB Ops-manager Operator successfully reconciled the Elasticsearch with new configure - Warning get pod; ConditionStatus:True; PodName:es-topology-master-0 117s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-master-0 - Warning evict pod; ConditionStatus:True; PodName:es-topology-master-0 117s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-master-0 - Warning check pod running; ConditionStatus:False; PodName:es-topology-master-0 112s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-master-0 - Warning check pod running; ConditionStatus:True; PodName:es-topology-master-0 102s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-master-0 - Warning get pod; ConditionStatus:True; PodName:es-topology-data-0 97s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-data-0 - Warning evict pod; ConditionStatus:True; PodName:es-topology-data-0 97s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-0 - Warning check pod running; ConditionStatus:False; PodName:es-topology-data-0 92s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-data-0 - Warning check pod running; ConditionStatus:True; PodName:es-topology-data-0 82s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-data-0 - Warning get pod; ConditionStatus:True; PodName:es-topology-data-1 77s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-data-1 - Warning evict pod; ConditionStatus:True; PodName:es-topology-data-1 77s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-1 - Warning check pod running; ConditionStatus:False; PodName:es-topology-data-1 72s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-data-1 - Warning check pod running; ConditionStatus:True; PodName:es-topology-data-1 62s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-data-1 - Warning get pod; ConditionStatus:True; PodName:es-topology-ingest-0 57s KubeDB Ops-manager Operator get pod; ConditionStatus:True; PodName:es-topology-ingest-0 - Warning evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 57s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 - Warning check pod running; ConditionStatus:False; PodName:es-topology-ingest-0 52s KubeDB Ops-manager Operator check pod running; ConditionStatus:False; PodName:es-topology-ingest-0 - Warning check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 42s KubeDB Ops-manager Operator check pod running; ConditionStatus:True; PodName:es-topology-ingest-0 - Normal RestartNodes 37s KubeDB Ops-manager Operator Successfully restarted all nodes - Normal Starting 35s KubeDB Ops-manager Operator Resuming Elasticsearch database: demo/es-topology - Normal Successful 35s KubeDB Ops-manager Operator Successfully resumed Elasticsearch database: demo/es-topology for ElasticsearchOpsRequest: esops-reconfigure-topology + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal PauseDatabase 4m47s KubeDB Ops-manager Operator Pausing Elasticsearch demo/es-topology + Warning pod exists; ConditionStatus:True; PodName:es-topology-ingest-0 4m29s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-topology-ingest-0 + Warning create es client; ConditionStatus:True; PodName:es-topology-ingest-0 4m29s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-topology-ingest-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 4m29s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-ingest-0 + Warning create es client; ConditionStatus:False 4m24s KubeDB Ops-manager Operator create es client; ConditionStatus:False + Warning create es client; ConditionStatus:True 4m19s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Warning pod exists; ConditionStatus:True; PodName:es-topology-data-0 4m14s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-topology-data-0 + Warning create es client; ConditionStatus:True; PodName:es-topology-data-0 4m14s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-topology-data-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-data-0 4m14s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-0 + Warning create es client; ConditionStatus:False 4m9s KubeDB Ops-manager Operator create es client; ConditionStatus:False + Warning create es client; ConditionStatus:True 4m4s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Warning pod exists; ConditionStatus:True; PodName:es-topology-data-1 3m59s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-topology-data-1 + Warning create es client; ConditionStatus:True; PodName:es-topology-data-1 3m59s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-topology-data-1 + Warning evict pod; ConditionStatus:True; PodName:es-topology-data-1 3m59s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-data-1 + Warning create es client; ConditionStatus:False 3m54s KubeDB Ops-manager Operator create es client; ConditionStatus:False + Warning create es client; ConditionStatus:True 3m49s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Warning pod exists; ConditionStatus:True; PodName:es-topology-master-0 3m44s KubeDB Ops-manager Operator pod exists; ConditionStatus:True; PodName:es-topology-master-0 + Warning create es client; ConditionStatus:True; PodName:es-topology-master-0 3m44s KubeDB Ops-manager Operator create es client; ConditionStatus:True; PodName:es-topology-master-0 + Warning evict pod; ConditionStatus:True; PodName:es-topology-master-0 3m44s KubeDB Ops-manager Operator evict pod; ConditionStatus:True; PodName:es-topology-master-0 + Warning create es client; ConditionStatus:True 3m39s KubeDB Ops-manager Operator create es client; ConditionStatus:True + Normal RestartNodes 3m34s KubeDB Ops-manager Operator Successfully restarted all nodes + Normal Successful 3m33s KubeDB Ops-manager Operator Successfully reconfigured all elasticsearch nodes. ``` Now let's exec into a master node, a data node, and the ingest node to verify the new configuration. ```bash $ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node - "cluster.max_shards_per_node" : "3000" + "max_shards_per_node" : "3000", $ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count - "indices.query.bool.max_clause_count" : "4096" + "max_clause_count" : "4096" + "max_clause_count" : "4096" $ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length - "max_content_length" : "300mb" + "max_content_length" : "300mb", ``` As we can see, the values have been updated on their respective node roles. So the reconfiguration of the cluster is successful. @@ -472,22 +480,23 @@ elasticsearchopsrequest.ops.kubedb.com/esops-reconfigure-apply-topology created Let's wait for `ElasticsearchOpsRequest` to be `Successful`. ```bash -$ kubectl get elasticsearchopsrequests -n demo esops-reconfigure-apply-topology +$ kubectl get elasticsearchopsrequests -n demo esops-reconfigure-apply-topology NAME TYPE STATUS AGE -esops-reconfigure-apply-topology Reconfigure Successful 55s +esops-reconfigure-apply-topology Reconfigure Successful 6m52s ``` Now let's verify the updated values on each node role. ```bash $ kubectl exec -it -n demo es-topology-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.cluster&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_shards_per_node - "cluster.max_shards_per_node" : "4000" + "max_shards_per_node" : "4000", $ kubectl exec -it -n demo es-topology-data-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.indices&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_clause_count - "indices.query.bool.max_clause_count" : "8192" + "max_clause_count" : "8192" + "max_clause_count" : "8192" -$ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:$ELASTIC_USER_PASSWORD" | grep max_content_length - "max_content_length" : "400mb" +$ kubectl exec -it -n demo es-topology-ingest-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_nodes/settings?filter_path=nodes.*.settings.http&pretty" --user "elastic:X4am_*ihVy~M)m0j" | grep max_content_length + "max_content_length" : "400mb", ``` As we can see, `cluster.max_shards_per_node` has been changed from `3000` to `4000` on master nodes, `indices.query.bool.max_clause_count` has been changed from `4096` to `8192` on data nodes, and `http.max_content_length` has been changed from `300mb` to `400mb` on ingest nodes. So the reconfiguration using the `applyConfig` field is successful. diff --git a/docs/guides/elasticsearch/tls/_index.md b/docs/guides/elasticsearch/tls/_index.md new file mode 100644 index 000000000..d38b896e8 --- /dev/null +++ b/docs/guides/elasticsearch/tls/_index.md @@ -0,0 +1,10 @@ +--- +title: Run Elasticsearch with TLS +menu: + docs_{{ .version }}: + identifier: es-tls + name: TLS/SSL Encryption + parent: es-elasticsearch-guides + weight: 70 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/elasticsearch/tls/elasticsearch-combined.md b/docs/guides/elasticsearch/tls/elasticsearch-combined.md new file mode 100644 index 000000000..5cb11b6c1 --- /dev/null +++ b/docs/guides/elasticsearch/tls/elasticsearch-combined.md @@ -0,0 +1,235 @@ +--- +title: Elasticsearch Combined Cluster TLS/SSL Encryption +menu: + docs_{{ .version }}: + identifier: es-tls-combined + name: Combined Cluster + parent: es-tls + weight: 20 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Run Elasticsearch with TLS/SSL (Transport Encryption) + +KubeDB supports providing TLS/SSL encryption for Elasticsearch. This tutorial will show you how to use KubeDB to run an Elasticsearch combined cluster with TLS/SSL encryption. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install [`cert-manager`](https://cert-manager.io/docs/installation/) v1.0.0 or later to your cluster to manage your SSL/TLS certificates. + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +KubeDB uses the following crd fields to enable SSL/TLS encryption in Elasticsearch. + +- `spec:` + - `enableSSL` + - `tls:` + - `issuerRef` + - `certificates` + +Read about the fields in details in [elasticsearch concept](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). + +`tls` is applicable for all types of Elasticsearch (i.e., `combined` and `topology`). + +Users must specify the `tls.issuerRef` field. KubeDB uses the `issuer` or `clusterIssuer` referenced in the `tls.issuerRef` field, and the certificate specs provided in `tls.certificates` to generate certificate secrets. These certificate secrets are then used to configure TLS for both the transport layer (node-to-node communication) and the HTTP layer (client-to-node communication), containing `ca.crt`, `tls.crt` and `tls.key`. + +## Create Issuer/ ClusterIssuer + +We are going to create an example `Issuer` that will be used throughout the duration of this tutorial to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificate using openssl. + +```bash +openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=elasticsearch/O=kubedb" +``` + +- Now create a ca-secret using the certificate files you have just generated. + +```bash +kubectl create secret tls es-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/es-ca created +``` + +Now, create an `Issuer` using the `ca-secret` you have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: es-ca-issuer + namespace: demo +spec: + ca: + secretName: es-ca +``` + +Apply the `YAML` file: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/tls/es-issuer.yaml +issuer.cert-manager.io/es-ca-issuer created +``` + +## TLS/SSL encryption in Elasticsearch Combined Cluster + +```yaml +apiVersion: kubedb.com/v1 +kind: Elasticsearch +metadata: + name: es-combined-tls + namespace: demo +spec: + version: xpack-8.19.9 + enableSSL: true + tls: + issuerRef: + apiGroup: "cert-manager.io" + kind: Issuer + name: es-ca-issuer + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + storageType: Durable + deletionPolicy: WipeOut +``` + +### Deploy Elasticsearch Combined Cluster + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/tls/es-combined-tls.yaml +elasticsearch.kubedb.com/es-combined-tls created +``` + +Now, wait until `es-combined-tls` has status `Ready`. i.e, + +```bash +$ kubectl get es -n demo -w +NAME VERSION STATUS AGE +es-combined-tls xpack-8.19.9 Provisioning 0s +es-combined-tls xpack-8.19.9 Provisioning 15s +. +. +es-combined-tls xpack-8.19.9 Ready 82s +``` + +### Verify TLS/SSL in Elasticsearch Combined Cluster + +KubeDB creates a client certificate secret for Elasticsearch. Let's check it: + +```bash +$ kubectl describe secret -n demo es-combined-tls-client-cert + +Name: es-combined-tls-client-cert +Namespace: demo +Labels: app.kubernetes.io/component=database + app.kubernetes.io/instance=es-combined-tls + app.kubernetes.io/managed-by=kubedb.com + app.kubernetes.io/name=elasticsearches.kubedb.com + controller.cert-manager.io/fao=true +Annotations: cert-manager.io/alt-names: + *.es-combined-tls-pods.demo.svc.cluster.local,es-combined-tls,es-combined-tls.demo.svc,es-combined-tls.demo.svc.cluster.local,localhost + cert-manager.io/certificate-name: es-combined-tls-client-cert + cert-manager.io/common-name: es-combined-tls.demo.svc + cert-manager.io/ip-sans: 127.0.0.1 + cert-manager.io/issuer-group: cert-manager.io + cert-manager.io/issuer-kind: Issuer + cert-manager.io/issuer-name: es-ca-issuer + +Type: kubernetes.io/tls + +Data +==== +ca.crt: 1184 bytes +tls.crt: 1452 bytes +tls.key: 1704 bytes +``` + +Now, let's exec into an Elasticsearch pod and verify the configuration that TLS is enabled for both transport and HTTP layers. + +```bash +$ kubectl exec -n demo es-combined-tls-0 -c elasticsearch -- \ + cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.security + +Defaulted container "elasticsearch" out of: elasticsearch, init-sysctl (init), config-merger (init) +xpack.security.enabled: true + +xpack.security.transport.ssl.enabled: true +xpack.security.transport.ssl.verification_mode: certificate +xpack.security.transport.ssl.key: certs/transport/tls.key +xpack.security.transport.ssl.certificate: certs/transport/tls.crt +xpack.security.transport.ssl.certificate_authorities: [ "certs/transport/ca.crt" ] + +xpack.security.http.ssl.enabled: true +xpack.security.http.ssl.key: certs/http/tls.key +xpack.security.http.ssl.certificate: certs/http/tls.crt +xpack.security.http.ssl.certificate_authorities: [ "certs/http/ca.crt" ] +``` + +We can see from the above output that both `xpack.security.transport.ssl.enabled: true` and `xpack.security.http.ssl.enabled: true` are set, which means TLS is enabled for both node-to-node and client-to-node communication. + +Now, let's connect to the Elasticsearch cluster using HTTPS to confirm it is accessible with TLS. + +```bash +$ kubectl exec -it -n demo es-combined-tls-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:$ELASTIC_USER_PASSWORD" +{ + "cluster_name" : "es-combined-tls", + "status" : "green", + "timed_out" : false, + "number_of_nodes" : 3, + "number_of_data_nodes" : 3, + "active_primary_shards" : 1, + "active_shards" : 2, + "relocating_shards" : 0, + "initializing_shards" : 0, + "unassigned_shards" : 0, + "delayed_unassigned_shards" : 0, + "number_of_pending_tasks" : 0, + "number_of_in_flight_fetch" : 0, + "task_max_waiting_in_queue_millis" : 0, + "active_shards_percent_as_number" : 100.0 +} +``` + +From the above output, we can see that we are able to connect to the Elasticsearch cluster using the TLS configuration. + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete es -n demo es-combined-tls +kubectl delete issuer -n demo es-ca-issuer +kubectl delete secret -n demo es-ca +kubectl delete ns demo +``` + +## Next Steps + +- Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/tls/elasticsearch-topology.md b/docs/guides/elasticsearch/tls/elasticsearch-topology.md new file mode 100644 index 000000000..ab806d092 --- /dev/null +++ b/docs/guides/elasticsearch/tls/elasticsearch-topology.md @@ -0,0 +1,255 @@ +--- +title: Elasticsearch Topology Cluster TLS/SSL Encryption +menu: + docs_{{ .version }}: + identifier: es-tls-topology + name: Topology Cluster + parent: es-tls + weight: 30 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Run Elasticsearch with TLS/SSL (Transport Encryption) + +KubeDB supports providing TLS/SSL encryption for Elasticsearch. This tutorial will show you how to use KubeDB to run an Elasticsearch topology cluster with TLS/SSL encryption. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install [`cert-manager`](https://cert-manager.io/docs/installation/) v1.0.0 or later to your cluster to manage your SSL/TLS certificates. + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +KubeDB uses the following crd fields to enable SSL/TLS encryption in Elasticsearch. + +- `spec:` + - `enableSSL` + - `tls:` + - `issuerRef` + - `certificates` + +Read about the fields in details in [elasticsearch concept](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). + +`tls` is applicable for all types of Elasticsearch (i.e., `combined` and `topology`). + +Users must specify the `tls.issuerRef` field. KubeDB uses the `issuer` or `clusterIssuer` referenced in the `tls.issuerRef` field, and the certificate specs provided in `tls.certificates` to generate certificate secrets. These certificate secrets are then used to configure TLS for both the transport layer (node-to-node communication) and the HTTP layer (client-to-node communication), containing `ca.crt`, `tls.crt` and `tls.key`. + +## Create Issuer/ ClusterIssuer + +We are going to create an example `Issuer` that will be used throughout the duration of this tutorial to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificate using openssl. + +```bash +openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=elasticsearch/O=kubedb" +``` + +- Now create a ca-secret using the certificate files you have just generated. + +```bash +kubectl create secret tls es-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/es-ca created +``` + +Now, create an `Issuer` using the `ca-secret` you have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: es-ca-issuer + namespace: demo +spec: + ca: + secretName: es-ca +``` + +Apply the `YAML` file: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/tls/es-issuer.yaml +issuer.cert-manager.io/es-ca-issuer created +``` + +## TLS/SSL encryption in Elasticsearch Topology Cluster + +```yaml +apiVersion: kubedb.com/v1 +kind: Elasticsearch +metadata: + name: es-topology-tls + namespace: demo +spec: + version: xpack-8.19.9 + enableSSL: true + tls: + issuerRef: + apiGroup: "cert-manager.io" + kind: Issuer + name: es-ca-issuer + topology: + master: + replicas: 1 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + data: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + ingest: + replicas: 1 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: standard + storageType: Durable + deletionPolicy: WipeOut +``` + +### Deploy Elasticsearch Topology Cluster with TLS/SSL + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/tls/es-topology-tls.yaml +elasticsearch.kubedb.com/es-topology-tls created +``` + +Now, wait until `es-topology-tls` has status `Ready`. i.e, + +```bash +$ kubectl get es -n demo -w +NAME VERSION STATUS AGE +es-topology-tls xpack-8.19.9 Provisioning 0s +es-topology-tls xpack-8.19.9 Provisioning 18s +. +. +es-topology-tls xpack-8.19.9 Ready 2m5s +``` + +### Verify TLS/SSL in Elasticsearch Topology Cluster + +KubeDB creates a client certificate secret for Elasticsearch. Let's check it: + +```bash +$ kubectl describe secret -n demo es-topology-tls-client-cert + +Name: es-topology-tls-client-cert +Namespace: demo +Labels: app.kubernetes.io/component=database + app.kubernetes.io/instance=es-topology-tls + app.kubernetes.io/managed-by=kubedb.com + app.kubernetes.io/name=elasticsearches.kubedb.com + controller.cert-manager.io/fao=true +Annotations: cert-manager.io/alt-names: + *.es-topology-tls-pods.demo.svc.cluster.local,es-topology-tls,es-topology-tls.demo.svc,es-topology-tls.demo.svc.cluster.local,localhost + cert-manager.io/certificate-name: es-topology-tls-client-cert + cert-manager.io/common-name: es-topology-tls.demo.svc + cert-manager.io/ip-sans: 127.0.0.1 + cert-manager.io/issuer-group: cert-manager.io + cert-manager.io/issuer-kind: Issuer + cert-manager.io/issuer-name: es-ca-issuer + +Type: kubernetes.io/tls + +Data +==== +ca.crt: 1184 bytes +tls.crt: 1460 bytes +tls.key: 1708 bytes +``` + +Now, let's exec into the master node and verify the configuration that TLS is enabled for both transport and HTTP layers. + +```bash +$ kubectl exec -n demo es-topology-tls-master-0 -c elasticsearch -- \ + cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.security + +Defaulted container "elasticsearch" out of: elasticsearch, init-sysctl (init), config-merger (init) +xpack.security.enabled: true + +xpack.security.transport.ssl.enabled: true +xpack.security.transport.ssl.verification_mode: certificate +xpack.security.transport.ssl.key: certs/transport/tls.key +xpack.security.transport.ssl.certificate: certs/transport/tls.crt +xpack.security.transport.ssl.certificate_authorities: [ "certs/transport/ca.crt" ] + +xpack.security.http.ssl.enabled: true +xpack.security.http.ssl.key: certs/http/tls.key +xpack.security.http.ssl.certificate: certs/http/tls.crt +xpack.security.http.ssl.certificate_authorities: [ "certs/http/ca.crt" ] +``` + +We can see from the above output that both `xpack.security.transport.ssl.enabled: true` and `xpack.security.http.ssl.enabled: true` are set, which means TLS is enabled for both node-to-node and client-to-node communication across all topology node roles. + +Now, let's exec into the master node and connect using HTTPS to confirm the topology cluster is accessible with TLS. + +```bash +$ kubectl exec -it -n demo es-topology-tls-master-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:$ELASTIC_USER_PASSWORD" +{ + "cluster_name" : "es-topology-tls", + "status" : "green", + "timed_out" : false, + "number_of_nodes" : 4, + "number_of_data_nodes" : 2, + "active_primary_shards" : 1, + "active_shards" : 2, + "relocating_shards" : 0, + "initializing_shards" : 0, + "unassigned_shards" : 0, + "delayed_unassigned_shards" : 0, + "number_of_pending_tasks" : 0, + "number_of_in_flight_fetch" : 0, + "task_max_waiting_in_queue_millis" : 0, + "active_shards_percent_as_number" : 100.0 +} +``` + +From the above output, we can see that we are able to connect to the Elasticsearch topology cluster using the TLS configuration. The cluster has 4 nodes total (1 master + 2 data + 1 ingest) and is reporting `green` status. + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete es -n demo es-topology-tls +kubectl delete issuer -n demo es-ca-issuer +kubectl delete secret -n demo es-ca +kubectl delete ns demo +``` + +## Next Steps + +- Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/tls/overview.md b/docs/guides/elasticsearch/tls/overview.md new file mode 100644 index 000000000..cc7c12689 --- /dev/null +++ b/docs/guides/elasticsearch/tls/overview.md @@ -0,0 +1,70 @@ +--- +title: Elasticsearch TLS/SSL Encryption Overview +menu: + docs_{{ .version }}: + identifier: es-tls-overview + name: Overview + parent: es-tls + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Elasticsearch TLS/SSL Encryption + +**Prerequisite :** To configure TLS/SSL in `Elasticsearch`, `KubeDB` uses `cert-manager` to issue certificates. So first you have to make sure that the cluster has `cert-manager` installed. To install `cert-manager` in your cluster following steps [here](https://cert-manager.io/docs/installation/kubernetes/). + +To issue a certificate, the following crds of `cert-manager` are used: + +- `Issuer/ClusterIssuer`: Issuers, and ClusterIssuers represent certificate authorities (CAs) that are able to generate signed certificates by honoring certificate signing requests. All cert-manager certificates require a referenced issuer that is in a ready condition to attempt to honor the request. You can learn more details [here](https://cert-manager.io/docs/concepts/issuer/). + +- `Certificate`: `cert-manager` has the concept of Certificates that define a desired x509 certificate which will be renewed and kept up to date. You can learn more details [here](https://cert-manager.io/docs/concepts/certificate/). + +**Elasticsearch CRD Specification :** + +KubeDB uses the following crd fields to enable SSL/TLS encryption in `Elasticsearch`. + +- `spec:` + - `enableSSL` + - `tls:` + - `issuerRef` + - `certificates` + +Read about the fields in details from [elasticsearch concept](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). + +When `enableSSL` is set to `true`, the users must specify the `tls.issuerRef` field. `KubeDB` uses the `issuer` or `clusterIssuer` referenced in the `tls.issuerRef` field, and the certificate specs provided in `tls.certificates` to generate certificate secrets using `Issuer/ClusterIssuer` specification. These certificate secrets including `ca.crt`, `tls.crt` and `tls.key` are used to configure both transport-layer (node-to-node) and HTTP-layer (client-to-node) TLS in `Elasticsearch`. + +## How TLS/SSL configures in Elasticsearch + +The following figure shows how `KubeDB` enterprise used to configure TLS/SSL in Elasticsearch. Open the image in a new tab to see the enlarged version. + +
+Deploy Elasticsearch with TLS/SSL +
Fig: Deploy Elasticsearch with TLS/SSL
+
+ +Deploying Elasticsearch with TLS/SSL configuration process consists of the following steps: + +1. At first, a user creates an `Issuer/ClusterIssuer` cr. + +2. Then the user creates an `Elasticsearch` CR with `enableSSL: true` which refers to the `Issuer/ClusterIssuer` CR that the user created in the previous step. + +3. `KubeDB` Provisioner operator watches for the `Elasticsearch` cr. + +4. When it finds one, it creates `Secret`, `Service`, etc. for the `Elasticsearch` cluster. + +5. `KubeDB` Ops-manager operator watches for `Elasticsearch`(5c), `Issuer/ClusterIssuer`(5b), `Secret` and `Service`(5a). + +6. When it finds all the resources(`Elasticsearch`, `Issuer/ClusterIssuer`, `Secret`, `Service`), it creates `Certificates` by using `tls.issuerRef` and `tls.certificates` field specification from `Elasticsearch` cr. + +7. `cert-manager` watches for certificates. + +8. When it finds one, it creates certificate secrets `tls-secrets` (transport and HTTP secrets etc.) that hold the actual certificate signed by the CA, containing `ca.crt`, `tls.crt` and `tls.key`. + +9. `KubeDB` Provisioner operator watches for the Certificate secrets `tls-secrets`. + +10. When it finds all the tls-secrets, it creates the related `PetSets` so that Elasticsearch can be configured with TLS/SSL for both transport and HTTP layers. + +In the next docs, we are going to show a step-by-step guide on how to configure an `Elasticsearch` cluster with TLS/SSL. From 0d107bba8e6e7fe23c6fab6bdebf66e8e2cc5cb2 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Mon, 15 Jun 2026 18:23:06 +0600 Subject: [PATCH 3/6] combined tls Signed-off-by: Bonusree --- .../tls/elasticsearch-combined.md | 65 +++++++++++-------- 1 file changed, 37 insertions(+), 28 deletions(-) diff --git a/docs/guides/elasticsearch/tls/elasticsearch-combined.md b/docs/guides/elasticsearch/tls/elasticsearch-combined.md index 5cb11b6c1..0c3d4e534 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-combined.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-combined.md @@ -62,7 +62,7 @@ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.c - Now create a ca-secret using the certificate files you have just generated. ```bash -kubectl create secret tls es-ca \ +$ kubectl create secret tls es-ca \ --cert=ca.crt \ --key=ca.key \ --namespace=demo @@ -142,7 +142,6 @@ KubeDB creates a client certificate secret for Elasticsearch. Let's check it: ```bash $ kubectl describe secret -n demo es-combined-tls-client-cert - Name: es-combined-tls-client-cert Namespace: demo Labels: app.kubernetes.io/component=database @@ -151,40 +150,40 @@ Labels: app.kubernetes.io/component=database app.kubernetes.io/name=elasticsearches.kubedb.com controller.cert-manager.io/fao=true Annotations: cert-manager.io/alt-names: - *.es-combined-tls-pods.demo.svc.cluster.local,es-combined-tls,es-combined-tls.demo.svc,es-combined-tls.demo.svc.cluster.local,localhost + *.es-combined-tls-pods.demo.svc,*.es-combined-tls-pods.demo.svc.cluster.local,es-combined-tls,es-combined-tls.demo.svc,localhost cert-manager.io/certificate-name: es-combined-tls-client-cert - cert-manager.io/common-name: es-combined-tls.demo.svc + cert-manager.io/common-name: es-combined-tls cert-manager.io/ip-sans: 127.0.0.1 cert-manager.io/issuer-group: cert-manager.io cert-manager.io/issuer-kind: Issuer cert-manager.io/issuer-name: es-ca-issuer + cert-manager.io/subject-organizations: kubedb + cert-manager.io/uri-sans: Type: kubernetes.io/tls Data ==== -ca.crt: 1184 bytes -tls.crt: 1452 bytes -tls.key: 1704 bytes +tls.key: 1708 bytes +ca.crt: 1172 bytes +tls.crt: 1387 bytes ``` Now, let's exec into an Elasticsearch pod and verify the configuration that TLS is enabled for both transport and HTTP layers. ```bash $ kubectl exec -n demo es-combined-tls-0 -c elasticsearch -- \ - cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.security - -Defaulted container "elasticsearch" out of: elasticsearch, init-sysctl (init), config-merger (init) + cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.security xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.key: certs/transport/tls.key -xpack.security.transport.ssl.certificate: certs/transport/tls.crt +xpack.security.transport.ssl.certificate: certs/transport/tls.crt xpack.security.transport.ssl.certificate_authorities: [ "certs/transport/ca.crt" ] xpack.security.http.ssl.enabled: true -xpack.security.http.ssl.key: certs/http/tls.key +xpack.security.http.ssl.key: certs/http/tls.key xpack.security.http.ssl.certificate: certs/http/tls.crt xpack.security.http.ssl.certificate_authorities: [ "certs/http/ca.crt" ] ``` @@ -194,23 +193,33 @@ We can see from the above output that both `xpack.security.transport.ssl.enabled Now, let's connect to the Elasticsearch cluster using HTTPS to confirm it is accessible with TLS. ```bash -$ kubectl exec -it -n demo es-combined-tls-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:$ELASTIC_USER_PASSWORD" +$ kubectl exec -it -n demo es-combined-tls-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:es-combined-tls-auth" { - "cluster_name" : "es-combined-tls", - "status" : "green", - "timed_out" : false, - "number_of_nodes" : 3, - "number_of_data_nodes" : 3, - "active_primary_shards" : 1, - "active_shards" : 2, - "relocating_shards" : 0, - "initializing_shards" : 0, - "unassigned_shards" : 0, - "delayed_unassigned_shards" : 0, - "number_of_pending_tasks" : 0, - "number_of_in_flight_fetch" : 0, - "task_max_waiting_in_queue_millis" : 0, - "active_shards_percent_as_number" : 100.0 + "error" : { + "root_cause" : [ + { + "type" : "security_exception", + "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]", + "header" : { + "WWW-Authenticate" : [ + "Basic realm=\"security\", charset=\"UTF-8\"", + "Bearer realm=\"security\"", + "ApiKey" + ] + } + } + ], + "type" : "security_exception", + "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]", + "header" : { + "WWW-Authenticate" : [ + "Basic realm=\"security\", charset=\"UTF-8\"", + "Bearer realm=\"security\"", + "ApiKey" + ] + } + }, + "status" : 401 } ``` From 1b23397036ea0a8002fbc4fbe38e209dba30638f Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 16 Jun 2026 12:17:20 +0600 Subject: [PATCH 4/6] es tls Signed-off-by: Bonusree --- .../reconfigure/elasticsearch-topology.md | 4 +- .../elasticsearch/reconfigure/overview.md | 2 +- .../tls/elasticsearch-topology.md | 21 ++-- docs/guides/elasticsearch/tls/overview.md | 2 +- docs/images/elasticsearch/es-reconfigure.svg | 110 ++++++++++++++++++ docs/images/elasticsearch/es-tls.svg | 4 + 6 files changed, 128 insertions(+), 15 deletions(-) create mode 100644 docs/images/elasticsearch/es-reconfigure.svg create mode 100644 docs/images/elasticsearch/es-tls.svg diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md index 92830b1bb..20c84717c 100644 --- a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md @@ -24,7 +24,7 @@ This guide will show you how to use `KubeDB` Ops-manager operator to reconfigure - You should be familiar with the following `KubeDB` concepts: - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) - - [Topology Cluster](/docs/guides/elasticsearch/clustering/topology-cluster/index.md) + - [Topology Cluster](/docs/guides/elasticsearch/clustering/topology-cluster/simple-dedicated-cluster/index.md) - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) - [Reconfigure Overview](/docs/guides/elasticsearch/reconfigure/overview.md) @@ -516,6 +516,6 @@ kubectl delete ns demo ## Next Steps - Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). -- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering). - Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/reconfigure/overview.md b/docs/guides/elasticsearch/reconfigure/overview.md index 0621bb5e9..012e8f216 100644 --- a/docs/guides/elasticsearch/reconfigure/overview.md +++ b/docs/guides/elasticsearch/reconfigure/overview.md @@ -27,7 +27,7 @@ This guide will give an overview on how KubeDB Ops-manager operator reconfigures The following diagram shows how KubeDB Ops-manager operator reconfigures `Elasticsearch` components. Open the image in a new tab to see the enlarged version.
- Reconfiguring process of Elasticsearch + Reconfiguring process of Elasticsearch
Fig: Reconfiguring process of Elasticsearch
diff --git a/docs/guides/elasticsearch/tls/elasticsearch-topology.md b/docs/guides/elasticsearch/tls/elasticsearch-topology.md index ab806d092..fea6e0b4a 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-topology.md @@ -162,7 +162,6 @@ KubeDB creates a client certificate secret for Elasticsearch. Let's check it: ```bash $ kubectl describe secret -n demo es-topology-tls-client-cert - Name: es-topology-tls-client-cert Namespace: demo Labels: app.kubernetes.io/component=database @@ -171,40 +170,40 @@ Labels: app.kubernetes.io/component=database app.kubernetes.io/name=elasticsearches.kubedb.com controller.cert-manager.io/fao=true Annotations: cert-manager.io/alt-names: - *.es-topology-tls-pods.demo.svc.cluster.local,es-topology-tls,es-topology-tls.demo.svc,es-topology-tls.demo.svc.cluster.local,localhost + *.es-topology-tls-pods.demo.svc,*.es-topology-tls-pods.demo.svc.cluster.local,es-topology-tls,es-topology-tls.demo.svc,localhost cert-manager.io/certificate-name: es-topology-tls-client-cert - cert-manager.io/common-name: es-topology-tls.demo.svc + cert-manager.io/common-name: es-topology-tls cert-manager.io/ip-sans: 127.0.0.1 cert-manager.io/issuer-group: cert-manager.io cert-manager.io/issuer-kind: Issuer cert-manager.io/issuer-name: es-ca-issuer + cert-manager.io/subject-organizations: kubedb + cert-manager.io/uri-sans: Type: kubernetes.io/tls Data ==== -ca.crt: 1184 bytes -tls.crt: 1460 bytes -tls.key: 1708 bytes +ca.crt: 1172 bytes +tls.crt: 1387 bytes +tls.key: 1704 bytes ``` Now, let's exec into the master node and verify the configuration that TLS is enabled for both transport and HTTP layers. ```bash $ kubectl exec -n demo es-topology-tls-master-0 -c elasticsearch -- \ - cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.security - -Defaulted container "elasticsearch" out of: elasticsearch, init-sysctl (init), config-merger (init) + cat /usr/share/elasticsearch/config/elasticsearch.yml | grep -A 2 -i xpack.securit xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.key: certs/transport/tls.key -xpack.security.transport.ssl.certificate: certs/transport/tls.crt +xpack.security.transport.ssl.certificate: certs/transport/tls.crt xpack.security.transport.ssl.certificate_authorities: [ "certs/transport/ca.crt" ] xpack.security.http.ssl.enabled: true -xpack.security.http.ssl.key: certs/http/tls.key +xpack.security.http.ssl.key: certs/http/tls.key xpack.security.http.ssl.certificate: certs/http/tls.crt xpack.security.http.ssl.certificate_authorities: [ "certs/http/ca.crt" ] ``` diff --git a/docs/guides/elasticsearch/tls/overview.md b/docs/guides/elasticsearch/tls/overview.md index cc7c12689..6bf6fdc89 100644 --- a/docs/guides/elasticsearch/tls/overview.md +++ b/docs/guides/elasticsearch/tls/overview.md @@ -41,7 +41,7 @@ When `enableSSL` is set to `true`, the users must specify the `tls.issuerRef` fi The following figure shows how `KubeDB` enterprise used to configure TLS/SSL in Elasticsearch. Open the image in a new tab to see the enlarged version.
-Deploy Elasticsearch with TLS/SSL +Deploy Elasticsearch with TLS/SSL
Fig: Deploy Elasticsearch with TLS/SSL
diff --git a/docs/images/elasticsearch/es-reconfigure.svg b/docs/images/elasticsearch/es-reconfigure.svg new file mode 100644 index 000000000..2fcde210e --- /dev/null +++ b/docs/images/elasticsearch/es-reconfigure.svg @@ -0,0 +1,110 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1. Create Elasticsearch + + Elasticsearch + + Elasticsearch + + Elasticsearch + + diff --git a/docs/images/elasticsearch/es-tls.svg b/docs/images/elasticsearch/es-tls.svg new file mode 100644 index 000000000..074cb2c1a --- /dev/null +++ b/docs/images/elasticsearch/es-tls.svg @@ -0,0 +1,4 @@ + + + +            Enterprise            Operator              Community            Operator
service
se...
secret
se...
tls-secret
tls-secret
Cert- manager
Cert- ma...
StatefulSet
Statef...
Issuer/Cluster Issuer
Issuer...
Elasticsearch
Elasticsearch
Certificates
Certif...
User
User
2.Create
2.Create
1.Create
1.Create
5a.Watch
5a.Watch
3.Watch
3.Watch
4.Create
4.Create
5c.Watch
5c.Watch
6.Create
6.Create
7.Watch
7.Watch
uses
uses
8.Create
8.Create
9.Watch
9.Watch
10.Create
10.Create
5b.Watch
5b.Watch
refers to
refers toText is not SVG - cannot display \ No newline at end of file From 2477c1c733bcf1333fc0da93f0bc1475b06cb636 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 16 Jun 2026 15:06:25 +0600 Subject: [PATCH 5/6] url correction Signed-off-by: Bonusree --- docs/guides/druid/rotate-auth/guide.md | 4 +- .../reconfigure/elasticsearch-combined.md | 2 +- .../reconfigure/elasticsearch-topology.md | 2 +- .../tls/elasticsearch-combined.md | 45 ++++++++----------- .../tls/elasticsearch-topology.md | 2 +- 5 files changed, 23 insertions(+), 32 deletions(-) diff --git a/docs/guides/druid/rotate-auth/guide.md b/docs/guides/druid/rotate-auth/guide.md index 9955a5bbd..0bc5b7f06 100644 --- a/docs/guides/druid/rotate-auth/guide.md +++ b/docs/guides/druid/rotate-auth/guide.md @@ -374,7 +374,7 @@ spec: authentication: secretRef: kind: Secret - name: sample-druid-auth-user + name: druid-quickstart-auth-user timeout: 5m apply: IfReady ``` @@ -417,7 +417,7 @@ Spec: Apply: IfReady Authentication: Secret Ref: - Name: sample-druid-auth-user + Name: druid-quickstart-auth-user Database Ref: Name: druid-quickstart Timeout: 5m diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md index 6bd9f3c1f..9db90f883 100644 --- a/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-combined.md @@ -393,6 +393,6 @@ kubectl delete namespace demo ## Next Steps - Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). -- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/combined-cluster/index.md). - Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md index 20c84717c..8628e5951 100644 --- a/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/reconfigure/elasticsearch-topology.md @@ -516,6 +516,6 @@ kubectl delete ns demo ## Next Steps - Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). -- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/topology-cluster/hot-warm-cold-cluster/index.md). - Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/tls/elasticsearch-combined.md b/docs/guides/elasticsearch/tls/elasticsearch-combined.md index 0c3d4e534..f431359c9 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-combined.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-combined.md @@ -193,33 +193,24 @@ We can see from the above output that both `xpack.security.transport.ssl.enabled Now, let's connect to the Elasticsearch cluster using HTTPS to confirm it is accessible with TLS. ```bash -$ kubectl exec -it -n demo es-combined-tls-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:es-combined-tls-auth" +$ kubectl exec -it -n demo es-combined-tls-0 -c elasticsearch -- curl -k -XGET "https://localhost:9200/_cluster/health?pretty" --user "elastic:$ELASTIC_USER_PASSWORD" { - "error" : { - "root_cause" : [ - { - "type" : "security_exception", - "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]", - "header" : { - "WWW-Authenticate" : [ - "Basic realm=\"security\", charset=\"UTF-8\"", - "Bearer realm=\"security\"", - "ApiKey" - ] - } - } - ], - "type" : "security_exception", - "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]", - "header" : { - "WWW-Authenticate" : [ - "Basic realm=\"security\", charset=\"UTF-8\"", - "Bearer realm=\"security\"", - "ApiKey" - ] - } - }, - "status" : 401 + "cluster_name" : "es-combined-tls", + "status" : "green", + "timed_out" : false, + "number_of_nodes" : 3, + "number_of_data_nodes" : 3, + "active_primary_shards" : 4, + "active_shards" : 8, + "relocating_shards" : 0, + "initializing_shards" : 0, + "unassigned_shards" : 0, + "unassigned_primary_shards" : 0, + "delayed_unassigned_shards" : 0, + "number_of_pending_tasks" : 0, + "number_of_in_flight_fetch" : 0, + "task_max_waiting_in_queue_millis" : 0, + "active_shards_percent_as_number" : 100.0 } ``` @@ -239,6 +230,6 @@ kubectl delete ns demo ## Next Steps - Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). -- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/combined-cluster/index.md). - Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/tls/elasticsearch-topology.md b/docs/guides/elasticsearch/tls/elasticsearch-topology.md index fea6e0b4a..72566a2ce 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-topology.md @@ -249,6 +249,6 @@ kubectl delete ns demo ## Next Steps - Detail concepts of [Elasticsearch object](/docs/guides/elasticsearch/concepts/elasticsearch/index.md). -- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/_index.md). +- Different Elasticsearch topology clustering modes [here](/docs/guides/elasticsearch/clustering/topology-cluster/hot-warm-cold-cluster/index.md). - Monitor your Elasticsearch database with KubeDB using [out-of-the-box Prometheus operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). From b023fbe9d92f07d8e92130015a9631c5a6464c18 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 16 Jun 2026 16:01:22 +0600 Subject: [PATCH 6/6] gitops reconfig change, note add Signed-off-by: Bonusree --- docs/guides/elasticsearch/gitops/gitops.md | 8 +++----- docs/guides/elasticsearch/tls/elasticsearch-combined.md | 2 ++ docs/guides/elasticsearch/tls/elasticsearch-topology.md | 2 ++ 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/guides/elasticsearch/gitops/gitops.md b/docs/guides/elasticsearch/gitops/gitops.md index dd9ecc615..e29d930b1 100644 --- a/docs/guides/elasticsearch/gitops/gitops.md +++ b/docs/guides/elasticsearch/gitops/gitops.md @@ -345,15 +345,13 @@ At first, we will create a secret containing `user.conf` file with required conf To know more about this configuration file, check [here](/docs/guides/elasticsearch/configuration/combined-cluster/index.md) ```yaml apiVersion: v1 -stringData: - user.conf: | - max_connections=200 - shared_buffers=256MB kind: Secret metadata: name: es-configuration namespace: demo -type: Opaque +stringData: + elasticsearch.yml: |- + indices.query.bool.max_clause_count: 2048 ``` Now, we will add this file to `kubedb/es-configuration.yaml`. diff --git a/docs/guides/elasticsearch/tls/elasticsearch-combined.md b/docs/guides/elasticsearch/tls/elasticsearch-combined.md index f431359c9..8f15d958b 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-combined.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-combined.md @@ -49,6 +49,8 @@ Read about the fields in details in [elasticsearch concept](/docs/guides/elastic Users must specify the `tls.issuerRef` field. KubeDB uses the `issuer` or `clusterIssuer` referenced in the `tls.issuerRef` field, and the certificate specs provided in `tls.certificates` to generate certificate secrets. These certificate secrets are then used to configure TLS for both the transport layer (node-to-node communication) and the HTTP layer (client-to-node communication), containing `ca.crt`, `tls.crt` and `tls.key`. +> Note: `tls.issuerRef` is optional. A user can deploy Elasticsearch without creating an `Issuer`/`ClusterIssuer` by just setting `enableSSL: true`. In that case, the KubeDB Elasticsearch operator automatically creates a self-signed CA and the necessary certificate secrets. + ## Create Issuer/ ClusterIssuer We are going to create an example `Issuer` that will be used throughout the duration of this tutorial to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. diff --git a/docs/guides/elasticsearch/tls/elasticsearch-topology.md b/docs/guides/elasticsearch/tls/elasticsearch-topology.md index 72566a2ce..f943feed4 100644 --- a/docs/guides/elasticsearch/tls/elasticsearch-topology.md +++ b/docs/guides/elasticsearch/tls/elasticsearch-topology.md @@ -49,6 +49,8 @@ Read about the fields in details in [elasticsearch concept](/docs/guides/elastic Users must specify the `tls.issuerRef` field. KubeDB uses the `issuer` or `clusterIssuer` referenced in the `tls.issuerRef` field, and the certificate specs provided in `tls.certificates` to generate certificate secrets. These certificate secrets are then used to configure TLS for both the transport layer (node-to-node communication) and the HTTP layer (client-to-node communication), containing `ca.crt`, `tls.crt` and `tls.key`. +> Note: `tls.issuerRef` is optional. A user can deploy Elasticsearch without creating an `Issuer`/`ClusterIssuer` by just setting `enableSSL: true`. In that case, the KubeDB Elasticsearch operator automatically creates a self-signed CA and the necessary certificate secrets. + ## Create Issuer/ ClusterIssuer We are going to create an example `Issuer` that will be used throughout the duration of this tutorial to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`.