From dac180424d6673d3cfae7e3e5d5c2451f19c4fa6 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 4 Aug 2025 14:05:43 +0000 Subject: [PATCH 01/11] Add step to docs that explains how to change database host from localhost to 127.0.0.1 --- docs/src/user-guide/guides-using-presto.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 59d464bc04..96cb561025 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -36,14 +36,24 @@ sample dataset that works well with Presto is [postgresql]. ### Setting up Presto -1. Clone the CLP repository: +1. Navigate to `etc/clp-config.yml` and uncomment the `database` section (lines 16 to 20 inclusive). +Change the `database.host` parameter to `"127.0.0.1"` so that the `database` section looks like +this: + ```yaml + database: + type: "mariadb" # "mariadb" or "mysql" + host: "127.0.0.1" + port: 3306 + name: "clp-db" + ``` +2. Clone the CLP repository: ```bash git clone https://github.com/y-scope/clp.git ``` -2. Navigate to the `tools/deployment/presto-clp` directory in your terminal. -3. Generate the necessary config for Presto to work with CLP: +3. Navigate to the `tools/deployment/presto-clp` directory in your terminal. +4. Generate the necessary config for Presto to work with CLP: ```bash scripts/set-up-config.sh @@ -52,7 +62,7 @@ sample dataset that works well with Presto is [postgresql]. * Replace `` with the location of the clp-json package you set up in the previous section. -4. Configure Presto to use CLP's metadata database as follows: +5. Configure Presto to use CLP's metadata database as follows: * Open and edit `coordinator/config-template/metadata-filter.json`. * For each dataset you want to query, add a filter config of the form: @@ -78,7 +88,7 @@ sample dataset that works well with Presto is [postgresql]. this particular dataset. * The complete syntax for this file is [here][clp-connector-docs]. -5. Start a Presto cluster by running: +6. Start a Presto cluster by running: ```bash docker compose up From 29e4fc14f792a810847e0e6f77296412bbedb403 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 4 Aug 2025 14:14:07 +0000 Subject: [PATCH 02/11] Make IP address non-specific --- docs/src/user-guide/guides-using-presto.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 96cb561025..380340a551 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -37,17 +37,19 @@ sample dataset that works well with Presto is [postgresql]. ### Setting up Presto 1. Navigate to `etc/clp-config.yml` and uncomment the `database` section (lines 16 to 20 inclusive). -Change the `database.host` parameter to `"127.0.0.1"` so that the `database` section looks like -this: +Change the `database.host` parameter to a non-localhost hostname/IP so that the `database` section +looks like this: + ```yaml database: type: "mariadb" # "mariadb" or "mysql" - host: "127.0.0.1" + host: "" port: 3306 name: "clp-db" ``` 2. Clone the CLP repository: + ```bash git clone https://github.com/y-scope/clp.git ``` From 0f9b453cfcfe249659a6779e1000c7e924c4d825 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 4 Aug 2025 15:03:16 +0000 Subject: [PATCH 03/11] Change stale numbering --- docs/src/user-guide/guides-using-presto.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 380340a551..ff6836e1f8 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -36,9 +36,9 @@ sample dataset that works well with Presto is [postgresql]. ### Setting up Presto -1. Navigate to `etc/clp-config.yml` and uncomment the `database` section (lines 16 to 20 inclusive). -Change the `database.host` parameter to a non-localhost hostname/IP so that the `database` section -looks like this: +1. Navigate to `etc/clp-config.yml`. Locate and uncomment the `database:` section and change the +`database.host` parameter to a non-localhost hostname/IP. The `database:` section should look like +this: ```yaml database: From 6f6bbd3211e3e8a89ffa207415bbe2c65dca71d2 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Tue, 5 Aug 2025 14:51:37 +0000 Subject: [PATCH 04/11] Move instruction to 'setting up CLP'; add rationale --- docs/src/user-guide/guides-using-presto.md | 33 +++++++++++++--------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index ff6836e1f8..fbd7b9ce00 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -31,14 +31,10 @@ Using Presto with CLP requires: ### Setting up CLP -Follow the [quick-start](./quick-start/index.md) guide to set up CLP and compress your logs. A -sample dataset that works well with Presto is [postgresql]. - -### Setting up Presto - -1. Navigate to `etc/clp-config.yml`. Locate and uncomment the `database:` section and change the -`database.host` parameter to a non-localhost hostname/IP. The `database:` section should look like -this: +1. Follow the [quick-start](./quick-start/index.md) guide to download and extract the CLP package. +Before proceeding, navigate to `etc/clp-config.yml` in the package. Locate and uncomment the +`database:` section, and change the`database.host` parameter to a non-localhost hostname/IP. The +`database:` section should look like this: ```yaml database: @@ -48,14 +44,25 @@ this: name: "clp-db" ``` -2. Clone the CLP repository: + For CLP’s database, `localhost` indicates the host itself, but for the Presto coordinator and + worker(s), `localhost` points at the container Presto is running within, as the coordinator and + worker(s) run on a kind of virtual network. + Infrastructure that will support CLP and the Presto container running within the same network is + in development. + +2. Continue the [quick-start](./quick-start/index.md#using-clp) guide to start up CLP and compress +your logs. A sample dataset that works well with Presto is [postgresql]. + +### Setting up Presto + +1. Clone the CLP repository: ```bash git clone https://github.com/y-scope/clp.git ``` -3. Navigate to the `tools/deployment/presto-clp` directory in your terminal. -4. Generate the necessary config for Presto to work with CLP: +2. Navigate to the `tools/deployment/presto-clp` directory in your terminal. +3. Generate the necessary config for Presto to work with CLP: ```bash scripts/set-up-config.sh @@ -64,7 +71,7 @@ this: * Replace `` with the location of the clp-json package you set up in the previous section. -5. Configure Presto to use CLP's metadata database as follows: +4. Configure Presto to use CLP's metadata database as follows: * Open and edit `coordinator/config-template/metadata-filter.json`. * For each dataset you want to query, add a filter config of the form: @@ -90,7 +97,7 @@ this: this particular dataset. * The complete syntax for this file is [here][clp-connector-docs]. -6. Start a Presto cluster by running: +5. Start a Presto cluster by running: ```bash docker compose up From 4b21f3c5a14422bdd31e95f8333eb2904adfe69c Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Tue, 5 Aug 2025 16:04:27 +0000 Subject: [PATCH 05/11] Rabbit --- docs/src/user-guide/guides-using-presto.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index fbd7b9ce00..1b53d396d2 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -33,7 +33,7 @@ Using Presto with CLP requires: 1. Follow the [quick-start](./quick-start/index.md) guide to download and extract the CLP package. Before proceeding, navigate to `etc/clp-config.yml` in the package. Locate and uncomment the -`database:` section, and change the`database.host` parameter to a non-localhost hostname/IP. The +`database:` section, and change the `database.host` parameter to a non-localhost hostname/IP. The `database:` section should look like this: ```yaml From cf983c30a4b501c78b5063b9e2cc455673303cca Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Fri, 8 Aug 2025 20:34:39 +0000 Subject: [PATCH 06/11] Address Kirk's comments --- docs/src/user-guide/guides-using-presto.md | 45 ++++++++++++---------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 1b53d396d2..5a12eb87e0 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -31,27 +31,30 @@ Using Presto with CLP requires: ### Setting up CLP -1. Follow the [quick-start](./quick-start/index.md) guide to download and extract the CLP package. -Before proceeding, navigate to `etc/clp-config.yml` in the package. Locate and uncomment the -`database:` section, and change the `database.host` parameter to a non-localhost hostname/IP. The -`database:` section should look like this: - - ```yaml - database: - type: "mariadb" # "mariadb" or "mysql" - host: "" - port: 3306 - name: "clp-db" - ``` - - For CLP’s database, `localhost` indicates the host itself, but for the Presto coordinator and - worker(s), `localhost` points at the container Presto is running within, as the coordinator and - worker(s) run on a kind of virtual network. - Infrastructure that will support CLP and the Presto container running within the same network is - in development. - -2. Continue the [quick-start](./quick-start/index.md#using-clp) guide to start up CLP and compress -your logs. A sample dataset that works well with Presto is [postgresql]. +1. Follow the [quick-start](./quick-start/index.md) guide to download and extract the CLP package, + but don't start the package just yet. +2. Before starting the package, update the package's config as follows: + + * Open `etc/clp-config.yml` located within the package. + * Uncomment the `database` section. + * Change `database.host` value to a non-localhost hostname/IP. + * After the change, the `database` section should look something like this: + ```yaml + database: + type: "mariadb" # "mariadb" or "mysql" + host: "" + port: 3306 + name: "clp-db" + ``` + :::{note} + This change is necessary since the Presto containers run on a Docker network whereas CLP's + database runs on the host network. So `localhost` refers to two different entities in those + networks. This limitation will be addressed in the future when we unify Presto and CLP's + deployment infrastructure. + ::: + +3. Continue following the [quick-start](./quick-start/index.md#using-clp) guide to start CLP and + compress your logs. A sample dataset that works well with Presto is [postgresql]. ### Setting up Presto From 8dc755f88a4265e435742cfb061513c4b9e2a93c Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Fri, 8 Aug 2025 20:43:05 +0000 Subject: [PATCH 07/11] Rabbit --- docs/src/user-guide/guides-using-presto.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 5a12eb87e0..056abf96bc 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -46,11 +46,12 @@ Using Presto with CLP requires: port: 3306 name: "clp-db" ``` + :::{note} - This change is necessary since the Presto containers run on a Docker network whereas CLP's + This change is necessary since the Presto containers run on a Docker network, whereas CLP's database runs on the host network. So `localhost` refers to two different entities in those networks. This limitation will be addressed in the future when we unify Presto and CLP's - deployment infrastructure. + deployment infrastructure. ::: 3. Continue following the [quick-start](./quick-start/index.md#using-clp) guide to start CLP and From 6f1fb197fb0eeb37dd8766c3c90e082b64dbc780 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 11 Aug 2025 13:51:02 +0000 Subject: [PATCH 08/11] Address Kirk's comment --- docs/src/user-guide/guides-using-presto.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 056abf96bc..5100e3ff4f 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -39,13 +39,13 @@ Using Presto with CLP requires: * Uncomment the `database` section. * Change `database.host` value to a non-localhost hostname/IP. * After the change, the `database` section should look something like this: - ```yaml - database: - type: "mariadb" # "mariadb" or "mysql" - host: "" - port: 3306 - name: "clp-db" - ``` + ```yaml + database: + type: "mariadb" # "mariadb" or "mysql" + host: "" + port: 3306 + name: "clp-db" + ``` :::{note} This change is necessary since the Presto containers run on a Docker network, whereas CLP's From 623d6ace0f917de35d3245331dd15b73b47318be Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 11 Aug 2025 18:44:29 +0000 Subject: [PATCH 09/11] Add missing newline --- docs/src/user-guide/guides-using-presto.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 5100e3ff4f..4d1185e0fa 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -39,6 +39,7 @@ Using Presto with CLP requires: * Uncomment the `database` section. * Change `database.host` value to a non-localhost hostname/IP. * After the change, the `database` section should look something like this: + ```yaml database: type: "mariadb" # "mariadb" or "mysql" From 2155dc36b96cf26b0777dc80d6f13264a7f41165 Mon Sep 17 00:00:00 2001 From: Quinn Taylor Mitchell Date: Mon, 11 Aug 2025 14:53:45 -0400 Subject: [PATCH 10/11] Update docs/src/user-guide/guides-using-presto.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- docs/src/user-guide/guides-using-presto.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 4d1185e0fa..5100e3ff4f 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -39,7 +39,6 @@ Using Presto with CLP requires: * Uncomment the `database` section. * Change `database.host` value to a non-localhost hostname/IP. * After the change, the `database` section should look something like this: - ```yaml database: type: "mariadb" # "mariadb" or "mysql" From 43aad5eac8155eed25b7dc58ef91d5e035b39b80 Mon Sep 17 00:00:00 2001 From: quinntaylormitchell Date: Mon, 11 Aug 2025 19:11:16 +0000 Subject: [PATCH 11/11] Adding newline again --- docs/src/user-guide/guides-using-presto.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/src/user-guide/guides-using-presto.md b/docs/src/user-guide/guides-using-presto.md index 5100e3ff4f..c68434dcfb 100644 --- a/docs/src/user-guide/guides-using-presto.md +++ b/docs/src/user-guide/guides-using-presto.md @@ -39,6 +39,7 @@ Using Presto with CLP requires: * Uncomment the `database` section. * Change `database.host` value to a non-localhost hostname/IP. * After the change, the `database` section should look something like this: + ```yaml database: type: "mariadb" # "mariadb" or "mysql"