From 6030b57ca944939014ed5b93ab44e4ea88c26ed5 Mon Sep 17 00:00:00 2001 From: Josh Kauffman Date: Fri, 19 Jun 2026 11:52:40 -0400 Subject: [PATCH 1/6] docs(hosted-sinks): add provider walkthroughs for Supabase, Neon, and ClickHouse Cloud Co-Authored-By: Claude Sonnet 4.6 --- .../sinks/hosted-sinks/clickhouse-cloud.md | 78 +++++++++++++++++++ .../sinks/hosted-sinks/hosted-sinks.md | 11 +++ docs/how-to-guides/sinks/hosted-sinks/neon.md | 76 ++++++++++++++++++ .../sinks/hosted-sinks/supabase.md | 71 +++++++++++++++++ 4 files changed, 236 insertions(+) create mode 100644 docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md create mode 100644 docs/how-to-guides/sinks/hosted-sinks/neon.md create mode 100644 docs/how-to-guides/sinks/hosted-sinks/supabase.md diff --git a/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md b/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md new file mode 100644 index 000000000..381592c35 --- /dev/null +++ b/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md @@ -0,0 +1,78 @@ +# Hosted Sink: ClickHouse Cloud + +This walkthrough connects a Hosted Sink to a [ClickHouse Cloud](https://clickhouse.com/cloud) managed ClickHouse service. + +## 1. Create a ClickHouse Cloud Service + +1. Sign in to [clickhouse.cloud](https://clickhouse.cloud) and open your organization. +2. Click **New service**, choose a cloud provider and region close to StreamingFast's infrastructure, and select a service tier. +3. Set the **default user** password during creation — save it immediately, it is only shown once. +4. Wait for the service to reach **Running** status (typically 1–3 minutes). + +## 2. Get Connection Credentials + +In your service, go to **Connect → Native (TCP)**. + +| Field | Example | +|---|---| +| Host | `abc123def456.us-east-1.aws.clickhouse.cloud` | +| Port | `9440` (native TLS) | +| Database | `default` | +| User | `default` | +| Password | Set during service creation | + +{% hint style="info" %} +ClickHouse Cloud exposes the native protocol on port **9440** (TLS) rather than the standard `9000`. Enable **Secure (TLS)** in the sink configuration. +{% endhint %} + +## 3. Open the IP Access List + +By default, ClickHouse Cloud restricts inbound connections. You must allow StreamingFast's infrastructure to reach your service. Hosted Sinks runs in **us-central1 (Iowa)**. + +1. In your service, go to **Settings → Security → IP Access List**. +2. Click **Add entry** and add `0.0.0.0/0` to allow connections from any IP. +3. Save the access list. + +## 4. Create a Dedicated User and Database (Recommended) + +Connect to your service using the ClickHouse Cloud SQL console (**Connect → SQL console**) and run: + +```sql +CREATE DATABASE IF NOT EXISTS substreams; + +CREATE USER substreams_sink + IDENTIFIED BY 'a-strong-password'; + +GRANT SELECT, INSERT, CREATE TABLE, DROP TABLE, ALTER, TRUNCATE + ON substreams.* + TO substreams_sink; +``` + +Replace `substreams` with your intended database name. + +## 5. Configure the Hosted Sink + +In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: + +| Field | Value | +|---|---| +| Host | `..aws.clickhouse.cloud` | +| Port | `9440` | +| Database | `substreams` (or your database name) | +| User | `substreams_sink` | +| Password | Password for that user | +| Secure (TLS) | **Enabled** | + +## 6. Deploy + +Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your ClickHouse Cloud service and begin writing data. + +## Troubleshooting + +**Connection refused** — Confirm the IP Access List includes StreamingFast's egress IPs or is set to allow `0.0.0.0/0`. Also confirm port `9440` is selected, not `9000`. + +**Authentication failed** — Double-check the user and password. ClickHouse Cloud passwords are case-sensitive. If you reset the default user password in the console, the old credential is immediately invalidated. + +**TLS handshake error** — Ensure **Secure (TLS)** is enabled. ClickHouse Cloud's native port `9440` requires TLS; connections without it will fail. + +**Database or table not found** — Ensure the sink user has been granted `CREATE TABLE` on the target database so the sink can initialize the schema on first run. diff --git a/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md b/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md index bbe8e3f27..41383c5b1 100644 --- a/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md +++ b/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md @@ -86,6 +86,17 @@ Credentials are stored securely and used only to connect the sink process to you Click **Deploy sink**. The sink enters the `Deploying` state while the infrastructure provisions. Once pods are ready it transitions to `Deployed` and begins indexing. +## Provider Walkthroughs + +Step-by-step guides for connecting a Hosted Sink to common managed database providers: + +**Postgres** +- [Supabase](./supabase.md) — managed Postgres with a generous free tier. +- [Neon](./neon.md) — serverless Postgres that scales to zero. + +**ClickHouse** +- [ClickHouse Cloud](./clickhouse-cloud.md) — fully managed ClickHouse from the creators of ClickHouse. + ## Next Steps - [Monitor and manage your sink](./manage-your-sink.md) — check status, view logs, edit config, stop, or delete. diff --git a/docs/how-to-guides/sinks/hosted-sinks/neon.md b/docs/how-to-guides/sinks/hosted-sinks/neon.md new file mode 100644 index 000000000..d84497625 --- /dev/null +++ b/docs/how-to-guides/sinks/hosted-sinks/neon.md @@ -0,0 +1,76 @@ +# Hosted Sink: Neon + +This walkthrough connects a Hosted Sink to a [Neon](https://neon.tech) serverless Postgres database. + +## 1. Create a Neon Project + +1. Sign in to [console.neon.tech](https://console.neon.tech) and click **New project**. +2. Choose a name, Postgres version, and a region close to StreamingFast's infrastructure. +3. Neon creates a default database (`neondb`) and branch (`main`) automatically. + +## 2. Get Connection Credentials + +In your project, go to the **Dashboard** and open the **Connection Details** panel. Select: +- **Branch**: `main` +- **Database**: your target database (default: `neondb`) +- **Role**: `neondb_owner` or a role you create + +The connection string looks like: + +``` +postgres://:@.neon.tech/neondb?sslmode=require +``` + +Pull the individual fields from it: + +| Field | Example | +|---|---| +| Host | `ep-quiet-forest-a1b2c3d4.us-east-2.aws.neon.tech` | +| Port | `5432` | +| Database | `neondb` | +| User | `neondb_owner` | +| Password | shown once — copy it now | + +## 3. Create a Dedicated Role and Schema (Recommended) + +Open the Neon **SQL Editor** and run: + +```sql +CREATE ROLE substreams_sink WITH LOGIN PASSWORD 'a-strong-password'; +CREATE SCHEMA IF NOT EXISTS substreams; +GRANT ALL PRIVILEGES ON SCHEMA substreams TO substreams_sink; +GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA substreams TO substreams_sink; +ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink; +``` + +Replace `substreams` with your intended schema name. + +## 4. Configure the Hosted Sink + +In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: + +| Field | Value | +|---|---| +| Host | `ep-..aws.neon.tech` | +| Port | `5432` | +| Database | `neondb` (or your database name) | +| Schema | `substreams` (or the schema you created above) | +| User | `substreams_sink` | +| Password | Password for that role | +| SSL Mode | `require` | + +{% hint style="warning" %} +Neon endpoints **suspend after a period of inactivity** on the free tier. A Hosted Sink running continuously will keep the endpoint active, but if the sink is stopped for an extended period the first reconnect may be slow. Consider upgrading to a paid Neon plan for production workloads. +{% endhint %} + +## 5. Deploy + +Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your Neon database and begin writing data. + +## Troubleshooting + +**Endpoint suspended on first connect** — Neon free-tier endpoints auto-suspend. The sink will retry on reconnect but may log an initial connection error. This is transient; the endpoint wakes within a few seconds. + +**SSL required** — Neon requires SSL. Set SSL Mode to `require`; `disable` will be rejected. + +**Permission denied on new tables** — Run `ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink;` so that tables created by the sink are automatically accessible to the role. diff --git a/docs/how-to-guides/sinks/hosted-sinks/supabase.md b/docs/how-to-guides/sinks/hosted-sinks/supabase.md new file mode 100644 index 000000000..81be51917 --- /dev/null +++ b/docs/how-to-guides/sinks/hosted-sinks/supabase.md @@ -0,0 +1,71 @@ +# Hosted Sink: Supabase + +This walkthrough connects a Hosted Sink to a [Supabase](https://supabase.com) Postgres database. + +## 1. Create a Supabase Project + +1. Sign in to [supabase.com](https://supabase.com) and open your organization. +2. Click **New project**, choose a region close to StreamingFast's infrastructure, and set a strong database password. +3. Wait for the project to finish provisioning (roughly 1–2 minutes). + +## 2. Get Connection Credentials + +In your project, go to **Settings → Database → Connection parameters**. + +| Supabase field | Value to copy | +|---|---| +| Host | `db..supabase.co` | +| Port | `5432` | +| Database name | `postgres` (default) | +| Username | `postgres` (or a dedicated user you create) | +| Password | The password you set during project creation | + +{% hint style="info" %} +Use the **direct connection** host (`db..supabase.co`) rather than the Session or Transaction pooler hosts. The pooler ports (5432/6543 on `aws-0-.pooler.supabase.com`) use PgBouncer, which can cause issues with the sink's prepared statements. +{% endhint %} + +## 3. Create a Dedicated Database User (Recommended) + +Run the following in the Supabase **SQL Editor** to create a user scoped to your target schema: + +```sql +CREATE USER substreams_sink WITH PASSWORD 'a-strong-password'; +CREATE SCHEMA IF NOT EXISTS substreams; +GRANT ALL PRIVILEGES ON SCHEMA substreams TO substreams_sink; +GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA substreams TO substreams_sink; +ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink; +``` + +Replace `substreams` with your intended schema name. + +## 4. Allow External Connections + +Supabase Postgres accepts connections from the public internet by default. No additional firewall steps are needed for the Hosted Sink to reach your database. + +## 5. Configure the Hosted Sink + +In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: + +| Field | Value | +|---|---| +| Host | `db..supabase.co` | +| Port | `5432` | +| Database | `postgres` | +| Schema | `substreams` (or the schema you created above) | +| User | `substreams_sink` | +| Password | Password for that user | +| SSL Mode | `require` | + +Supabase requires SSL — use `require` at minimum. Use `verify-full` if you want certificate validation. + +## 6. Deploy + +Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your Supabase database and begin writing data. + +## Troubleshooting + +**Connection refused / timeout** — Confirm you are using the direct connection host, not the pooler host. Check that port `5432` is not blocked by any additional network policy in your Supabase project (**Settings → Database → Network restrictions**). + +**Permission denied** — Ensure the sink user has been granted privileges on the target schema and that `ALTER DEFAULT PRIVILEGES` was applied so future tables are also accessible. + +**SSL errors** — Supabase requires SSL. Set SSL Mode to `require` or higher; `disable` will be rejected. From a4bf765c75a944d07629fbad0f9be644afd80a7f Mon Sep 17 00:00:00 2001 From: Josh Kauffman Date: Fri, 19 Jun 2026 11:58:25 -0400 Subject: [PATCH 2/6] docs(hosted-sinks): add walkthrough pages to SUMMARY.md Co-Authored-By: Claude Sonnet 4.6 --- docs/SUMMARY.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 8287cfc82..d1bc80cdd 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -65,6 +65,9 @@ * [Consuming Substreams](how-to-guides/sinks/sinks.md) * [Hosted Sinks](how-to-guides/sinks/hosted-sinks/hosted-sinks.md) * [Managing Your Sink](how-to-guides/sinks/hosted-sinks/manage-your-sink.md) + * [Supabase](how-to-guides/sinks/hosted-sinks/supabase.md) + * [Neon](how-to-guides/sinks/hosted-sinks/neon.md) + * [ClickHouse Cloud](how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md) * [Substreams:SQL](how-to-guides/sinks/sql/sql.md) * [Using Relational Mappings](how-to-guides/sinks/sql/relational-mappings.md) * [Using Database Changes](how-to-guides/sinks/sql/db_out.md) From 2017c2bccff94e37efc4c5f9131e1b9965359e64 Mon Sep 17 00:00:00 2001 From: Josh Kauffman <40213896+joshkauffman@users.noreply.github.com> Date: Fri, 19 Jun 2026 15:53:09 -0400 Subject: [PATCH 3/6] Update ClickHouse Cloud setup instructions Clarified instructions for creating a ClickHouse Cloud service and added note on IP access restrictions. --- docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md b/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md index 381592c35..c793fe32d 100644 --- a/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md +++ b/docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md @@ -5,7 +5,7 @@ This walkthrough connects a Hosted Sink to a [ClickHouse Cloud](https://clickhou ## 1. Create a ClickHouse Cloud Service 1. Sign in to [clickhouse.cloud](https://clickhouse.cloud) and open your organization. -2. Click **New service**, choose a cloud provider and region close to StreamingFast's infrastructure, and select a service tier. +2. Click **New service**, choose a cloud provider and region close to StreamingFast's infrastructure, and select a service tier. Hosted Sinks are provided from us-central1 in Iowa. 3. Set the **default user** password during creation — save it immediately, it is only shown once. 4. Wait for the service to reach **Running** status (typically 1–3 minutes). @@ -30,7 +30,7 @@ ClickHouse Cloud exposes the native protocol on port **9440** (TLS) rather than By default, ClickHouse Cloud restricts inbound connections. You must allow StreamingFast's infrastructure to reach your service. Hosted Sinks runs in **us-central1 (Iowa)**. 1. In your service, go to **Settings → Security → IP Access List**. -2. Click **Add entry** and add `0.0.0.0/0` to allow connections from any IP. +2. Click **Add entry** and add `0.0.0.0/0` to allow connections from any IP. Note, if you require a restricted IP range, please contact us at support@streamingfast.io. 3. Save the access list. ## 4. Create a Dedicated User and Database (Recommended) @@ -52,7 +52,7 @@ Replace `substreams` with your intended database name. ## 5. Configure the Hosted Sink -In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: +In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section: | Field | Value | |---|---| From f54fc992f2caa062c0092f66bf292ac5c7aaf6da Mon Sep 17 00:00:00 2001 From: Josh Kauffman <40213896+joshkauffman@users.noreply.github.com> Date: Fri, 19 Jun 2026 15:55:44 -0400 Subject: [PATCH 4/6] Enhance Hosted Sink guide with SQL service walkthroughs Added walkthroughs for SQL services to the Hosted Sink guide. --- docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md b/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md index 41383c5b1..e0925bcd2 100644 --- a/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md +++ b/docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md @@ -38,7 +38,8 @@ Choose how you want to supply the `.spkg`: ### 3. Output -Choose **Postgres** or **ClickHouse** and fill in the connection details. +Choose **Postgres** or **ClickHouse** and fill in the connection details. +See walkthroughs provided below for different SQL services you can rely on, with guided setups to run your Hosted Sink. **Postgres fields:** From 5a12a9b1914a56f497cb0decd454d433e643dab1 Mon Sep 17 00:00:00 2001 From: Josh Kauffman <40213896+joshkauffman@users.noreply.github.com> Date: Fri, 19 Jun 2026 15:56:58 -0400 Subject: [PATCH 5/6] Fix link to create a new sink in The Graph Market Updated the link for creating a new sink in The Graph Market. --- docs/how-to-guides/sinks/hosted-sinks/neon.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to-guides/sinks/hosted-sinks/neon.md b/docs/how-to-guides/sinks/hosted-sinks/neon.md index d84497625..d421f14db 100644 --- a/docs/how-to-guides/sinks/hosted-sinks/neon.md +++ b/docs/how-to-guides/sinks/hosted-sinks/neon.md @@ -47,7 +47,7 @@ Replace `substreams` with your intended schema name. ## 4. Configure the Hosted Sink -In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: +In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section: | Field | Value | |---|---| From 0122bb3bb42676bd5a799ab7808e1e49a46433b7 Mon Sep 17 00:00:00 2001 From: Josh Kauffman <40213896+joshkauffman@users.noreply.github.com> Date: Fri, 19 Jun 2026 15:57:37 -0400 Subject: [PATCH 6/6] Update sink creation link in Supabase guide --- docs/how-to-guides/sinks/hosted-sinks/supabase.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to-guides/sinks/hosted-sinks/supabase.md b/docs/how-to-guides/sinks/hosted-sinks/supabase.md index 81be51917..ac27374c3 100644 --- a/docs/how-to-guides/sinks/hosted-sinks/supabase.md +++ b/docs/how-to-guides/sinks/hosted-sinks/supabase.md @@ -44,7 +44,7 @@ Supabase Postgres accepts connections from the public internet by default. No ad ## 5. Configure the Hosted Sink -In [The Graph Market](https://thegraph.market/sinks), create a new sink and fill in the **Output** section: +In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section: | Field | Value | |---|---|