Skip to content
Open
3 changes: 3 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@
* [Consuming Substreams](how-to-guides/sinks/sinks.md)
* [Hosted Sinks](how-to-guides/sinks/hosted-sinks/hosted-sinks.md)
* [Managing Your Sink](how-to-guides/sinks/hosted-sinks/manage-your-sink.md)
* [Supabase](how-to-guides/sinks/hosted-sinks/supabase.md)
* [Neon](how-to-guides/sinks/hosted-sinks/neon.md)
* [ClickHouse Cloud](how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md)
* [Substreams:SQL](how-to-guides/sinks/sql/sql.md)
* [Using Relational Mappings](how-to-guides/sinks/sql/relational-mappings.md)
* [Using Database Changes](how-to-guides/sinks/sql/db_out.md)
Expand Down
78 changes: 78 additions & 0 deletions docs/how-to-guides/sinks/hosted-sinks/clickhouse-cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Hosted Sink: ClickHouse Cloud

This walkthrough connects a Hosted Sink to a [ClickHouse Cloud](https://clickhouse.com/cloud) managed ClickHouse service.

## 1. Create a ClickHouse Cloud Service

1. Sign in to [clickhouse.cloud](https://clickhouse.cloud) and open your organization.
2. Click **New service**, choose a cloud provider and region close to StreamingFast's infrastructure, and select a service tier. Hosted Sinks are provided from us-central1 in Iowa.
3. Set the **default user** password during creation — save it immediately, it is only shown once.
4. Wait for the service to reach **Running** status (typically 1–3 minutes).

## 2. Get Connection Credentials

In your service, go to **Connect → Native (TCP)**.

| Field | Example |
|---|---|
| Host | `abc123def456.us-east-1.aws.clickhouse.cloud` |
| Port | `9440` (native TLS) |
| Database | `default` |
| User | `default` |
| Password | Set during service creation |

{% hint style="info" %}
ClickHouse Cloud exposes the native protocol on port **9440** (TLS) rather than the standard `9000`. Enable **Secure (TLS)** in the sink configuration.
{% endhint %}

## 3. Open the IP Access List

By default, ClickHouse Cloud restricts inbound connections. You must allow StreamingFast's infrastructure to reach your service. Hosted Sinks runs in **us-central1 (Iowa)**.

1. In your service, go to **Settings → Security → IP Access List**.
2. Click **Add entry** and add `0.0.0.0/0` to allow connections from any IP. Note, if you require a restricted IP range, please contact us at support@streamingfast.io.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could be turn into a mailto: link which usually in browser auto open email client, easy to check on GitBook preview

3. Save the access list.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everywhere we have this 0.0.0.0, add a hint to contact us if someone has increased enterprise needs where they need to restrict to a given IP range

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added here. Reviewing if found in other pages.

## 4. Create a Dedicated User and Database (Recommended)

Connect to your service using the ClickHouse Cloud SQL console (**Connect → SQL console**) and run:

```sql
CREATE DATABASE IF NOT EXISTS substreams;

CREATE USER substreams_sink
IDENTIFIED BY 'a-strong-password';

GRANT SELECT, INSERT, CREATE TABLE, DROP TABLE, ALTER, TRUNCATE
ON substreams.*
TO substreams_sink;
```

Replace `substreams` with your intended database name.

## 5. Configure the Hosted Sink

In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section:

| Field | Value |
|---|---|
| Host | `<hash>.<region>.aws.clickhouse.cloud` |
| Port | `9440` |
| Database | `substreams` (or your database name) |
| User | `substreams_sink` |
| Password | Password for that user |
| Secure (TLS) | **Enabled** |

## 6. Deploy

Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your ClickHouse Cloud service and begin writing data.

## Troubleshooting

**Connection refused** — Confirm the IP Access List includes StreamingFast's egress IPs or is set to allow `0.0.0.0/0`. Also confirm port `9440` is selected, not `9000`.

**Authentication failed** — Double-check the user and password. ClickHouse Cloud passwords are case-sensitive. If you reset the default user password in the console, the old credential is immediately invalidated.

**TLS handshake error** — Ensure **Secure (TLS)** is enabled. ClickHouse Cloud's native port `9440` requires TLS; connections without it will fail.

**Database or table not found** — Ensure the sink user has been granted `CREATE TABLE` on the target database so the sink can initialize the schema on first run.
14 changes: 13 additions & 1 deletion docs/how-to-guides/sinks/hosted-sinks/hosted-sinks.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ Choose how you want to supply the `.spkg`:

### 3. Output

Choose **Postgres** or **ClickHouse** and fill in the connection details.
Choose **Postgres** or **ClickHouse** and fill in the connection details.
See walkthroughs provided below for different SQL services you can rely on, with guided setups to run your Hosted Sink.

**Postgres fields:**

Expand Down Expand Up @@ -86,6 +87,17 @@ Credentials are stored securely and used only to connect the sink process to you

Click **Deploy sink**. The sink enters the `Deploying` state while the infrastructure provisions. Once pods are ready it transitions to `Deployed` and begins indexing.

## Provider Walkthroughs

Step-by-step guides for connecting a Hosted Sink to common managed database providers:

**Postgres**
- [Supabase](./supabase.md) — managed Postgres with a generous free tier.
- [Neon](./neon.md) — serverless Postgres that scales to zero.

**ClickHouse**
- [ClickHouse Cloud](./clickhouse-cloud.md) — fully managed ClickHouse from the creators of ClickHouse.

## Next Steps

- [Monitor and manage your sink](./manage-your-sink.md) — check status, view logs, edit config, stop, or delete.
76 changes: 76 additions & 0 deletions docs/how-to-guides/sinks/hosted-sinks/neon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Hosted Sink: Neon

This walkthrough connects a Hosted Sink to a [Neon](https://neon.tech) serverless Postgres database.

## 1. Create a Neon Project

1. Sign in to [console.neon.tech](https://console.neon.tech) and click **New project**.
2. Choose a name, Postgres version, and a region close to StreamingFast's infrastructure.
3. Neon creates a default database (`neondb`) and branch (`main`) automatically.

## 2. Get Connection Credentials

In your project, go to the **Dashboard** and open the **Connection Details** panel. Select:
- **Branch**: `main`
- **Database**: your target database (default: `neondb`)
- **Role**: `neondb_owner` or a role you create

The connection string looks like:

```
postgres://<user>:<password>@<endpoint-host>.neon.tech/neondb?sslmode=require
```

Pull the individual fields from it:

| Field | Example |
|---|---|
| Host | `ep-quiet-forest-a1b2c3d4.us-east-2.aws.neon.tech` |
| Port | `5432` |
| Database | `neondb` |
| User | `neondb_owner` |
| Password | shown once — copy it now |

## 3. Create a Dedicated Role and Schema (Recommended)

Open the Neon **SQL Editor** and run:

```sql
CREATE ROLE substreams_sink WITH LOGIN PASSWORD 'a-strong-password';
CREATE SCHEMA IF NOT EXISTS substreams;
GRANT ALL PRIVILEGES ON SCHEMA substreams TO substreams_sink;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA substreams TO substreams_sink;
ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink;
```

Replace `substreams` with your intended schema name.

## 4. Configure the Hosted Sink

In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section:

| Field | Value |
|---|---|
| Host | `ep-<name>.<region>.aws.neon.tech` |
| Port | `5432` |
| Database | `neondb` (or your database name) |
| Schema | `substreams` (or the schema you created above) |
| User | `substreams_sink` |
| Password | Password for that role |
| SSL Mode | `require` |

{% hint style="warning" %}
Neon endpoints **suspend after a period of inactivity** on the free tier. A Hosted Sink running continuously will keep the endpoint active, but if the sink is stopped for an extended period the first reconnect may be slow. Consider upgrading to a paid Neon plan for production workloads.
{% endhint %}

## 5. Deploy

Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your Neon database and begin writing data.

## Troubleshooting

**Endpoint suspended on first connect** — Neon free-tier endpoints auto-suspend. The sink will retry on reconnect but may log an initial connection error. This is transient; the endpoint wakes within a few seconds.

**SSL required** — Neon requires SSL. Set SSL Mode to `require`; `disable` will be rejected.

**Permission denied on new tables** — Run `ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink;` so that tables created by the sink are automatically accessible to the role.
71 changes: 71 additions & 0 deletions docs/how-to-guides/sinks/hosted-sinks/supabase.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Hosted Sink: Supabase

This walkthrough connects a Hosted Sink to a [Supabase](https://supabase.com) Postgres database.

## 1. Create a Supabase Project

1. Sign in to [supabase.com](https://supabase.com) and open your organization.
2. Click **New project**, choose a region close to StreamingFast's infrastructure, and set a strong database password.
3. Wait for the project to finish provisioning (roughly 1–2 minutes).

## 2. Get Connection Credentials

In your project, go to **Settings → Database → Connection parameters**.

| Supabase field | Value to copy |
|---|---|
| Host | `db.<project-ref>.supabase.co` |
| Port | `5432` |
| Database name | `postgres` (default) |
| Username | `postgres` (or a dedicated user you create) |
| Password | The password you set during project creation |

{% hint style="info" %}
Use the **direct connection** host (`db.<project-ref>.supabase.co`) rather than the Session or Transaction pooler hosts. The pooler ports (5432/6543 on `aws-0-<region>.pooler.supabase.com`) use PgBouncer, which can cause issues with the sink's prepared statements.
{% endhint %}

## 3. Create a Dedicated Database User (Recommended)

Run the following in the Supabase **SQL Editor** to create a user scoped to your target schema:

```sql
CREATE USER substreams_sink WITH PASSWORD 'a-strong-password';
CREATE SCHEMA IF NOT EXISTS substreams;
GRANT ALL PRIVILEGES ON SCHEMA substreams TO substreams_sink;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA substreams TO substreams_sink;
ALTER DEFAULT PRIVILEGES IN SCHEMA substreams GRANT ALL ON TABLES TO substreams_sink;
```

Replace `substreams` with your intended schema name.

## 4. Allow External Connections

Supabase Postgres accepts connections from the public internet by default. No additional firewall steps are needed for the Hosted Sink to reach your database.

## 5. Configure the Hosted Sink

In [The Graph Market](https://thegraph.market/sinks/new), create a new sink and fill in the **Output** section:

| Field | Value |
|---|---|
| Host | `db.<project-ref>.supabase.co` |
| Port | `5432` |
| Database | `postgres` |
| Schema | `substreams` (or the schema you created above) |
| User | `substreams_sink` |
| Password | Password for that user |
| SSL Mode | `require` |

Supabase requires SSL — use `require` at minimum. Use `verify-full` if you want certificate validation.

## 6. Deploy

Complete the remaining sink configuration (package, execution settings) and click **Deploy sink**. The sink will connect to your Supabase database and begin writing data.

## Troubleshooting

**Connection refused / timeout** — Confirm you are using the direct connection host, not the pooler host. Check that port `5432` is not blocked by any additional network policy in your Supabase project (**Settings → Database → Network restrictions**).

**Permission denied** — Ensure the sink user has been granted privileges on the target schema and that `ALTER DEFAULT PRIVILEGES` was applied so future tables are also accessible.

**SSL errors** — Supabase requires SSL. Set SSL Mode to `require` or higher; `disable` will be rejected.
Loading