diff --git a/docs/indexer.md b/docs/indexer.md index ccba7b90..78641afe 100644 --- a/docs/indexer.md +++ b/docs/indexer.md @@ -195,6 +195,7 @@ services: --bootnodes "/dns/bootnode2.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWSvSQNVHGmK965dKcCDGaxtyeY1XPCMwFUSLC8opguG1T" --bootnodes "/ip4/51.210.3.173/tcp/30333/p2p/12D3KooWRUd3BqRyiGfhxVb2BSyUDLK5nHHNXTddZpqzqvQ73C9u" --bootnodes "/ip4/141.95.65.179/tcp/30683/p2p/12D3KooWQ8xPXuBww4qSumnjycjjKDThFUj4nDgGS3UPoLyRBvqJ" + --database paritydb --validator --port 30333 --log info diff --git a/docs/mainnet.md b/docs/mainnet.md index 00086cab..03665147 100644 --- a/docs/mainnet.md +++ b/docs/mainnet.md @@ -92,7 +92,43 @@ The generated key should now be in a file called `subkey.key` in the sxt-node-ke Here we assume the setup uses the following volumes: `sxt-mainnet-data` is the block storage volume and the volume where the generated node key is stored is `sxt-node-key`. -### 2.1. Docker Run +### 2.1. Snapshot Download and Extraction + +When starting a new node, the initial syncing of blocks with the rest of the network can take multiple days. +To alleviate this, it is recommended to download and extract a snapshot of the `paritydb` database prior to booting your node. +Space and Time roughly publishes snapshots on a two-week cadence. + +The date of the latest snapshot (in `YYYY-MM-DD` format) can be found at https://snapshots.mainnet.sxt.network/latest.txt +The snapshot and checksum for the corresponding date can then be downloaded at the following URLs, respectively +(where `${DATE}` is once again in `YYYY-MM-DD` format). + +``` +https://snapshots.mainnet.sxt.network/${DATE}/sxt-mainnet.tar.gz +https://snapshots.mainnet.sxt.network/${DATE}/sxt-mainnet.sha1 +``` + +Once downloaded, extract the contents of the archive onto the `/chains` directory in the block storage volume. +The following snippet downloads and extracts the snapshot as describe above leveraging the [aria2](https://aria2.github.io/) +utility for parallelizing and retrying the download process. + +```bash +LATEST_DATE="$(curl -s https://snapshots.mainnet.sxt.network/latest.txt)" +pushd sxt-mainnet-data # Directory that will be mounted as the block storage volume in docker +aria2c -x 16 -s 16 "https://snapshots.mainnet.sxt.network/${LATEST_DATE}/sxt-mainnet.tar.gz" +curl -O "https://snapshots.mainnet.sxt.network/${LATEST_DATE}/sxt-mainnet.sha1" +sha1sum --check "sxt-mainnet.sha1" +mkdir -p chains +tar xf sxt-mainnet.tar.gz -C chains +rm sxt-mainnet.tar.gz +popd +``` + +Note that before the `sxt-mainnet.tar.gz` archive gets deleted in the snippet above, you'll be required to have enough disk space +to host both the archive and the exploded directory, which roughly equates to 2.5TB. To get around this requirement, a separate +volume that gets temporarily mounted to download the snapshot is recommended. A streaming solution would likely be the most space efficient, +but it has not yet been developed. + +### 2.2. Docker Run Make sure to set VALIDATOR_NAME to make it easier to identify your node in the telemetry dashboard. @@ -121,6 +157,7 @@ docker run -d --restart always \ --bootnodes "/dns/bootnode2.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWSvSQNVHGmK965dKcCDGaxtyeY1XPCMwFUSLC8opguG1T" \ --bootnodes "/ip4/51.210.3.173/tcp/30333/p2p/12D3KooWRUd3BqRyiGfhxVb2BSyUDLK5nHHNXTddZpqzqvQ73C9u" \ --bootnodes "/ip4/141.95.65.179/tcp/30683/p2p/12D3KooWQ8xPXuBww4qSumnjycjjKDThFUj4nDgGS3UPoLyRBvqJ" \ + --database "paritydb" \ --validator \ --port 30333 \ --log info \ @@ -130,7 +167,7 @@ docker run -d --restart always \ --name ${VALIDATOR_NAME?} ``` -### 2.2. Docker Compose +### 2.3. Docker Compose Prepare a `docker-compose.yaml` file as follows: @@ -164,11 +201,12 @@ services: --keystore-path /key --node-key-file /node-key/subkey.key --bootnodes "/dns/validator0.mainnet.sxt.network/tcp/30333/p2p/12D3KooWK4MUYTiz8H6gG98JwN3bT11keivvFLYjtwEv5sqhwkAt" - --bootnodes "/dns/validator.mainnet-sxt.ethernodes.io/tcp/30333/p2p/12D3KooWMEq9xSwmr8uTthCb6KM86av1T7AYe44kDPxWovEqzq1w" \ - --bootnodes "/dns/bootnode1.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWN8ZsZDNVr1ooMWJtsqMNHCVaQ4cvxvB8kELqrZeqct79" \ - --bootnodes "/dns/bootnode2.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWSvSQNVHGmK965dKcCDGaxtyeY1XPCMwFUSLC8opguG1T" \ - --bootnodes "/ip4/51.210.3.173/tcp/30333/p2p/12D3KooWRUd3BqRyiGfhxVb2BSyUDLK5nHHNXTddZpqzqvQ73C9u" \ - --bootnodes "/ip4/141.95.65.179/tcp/30683/p2p/12D3KooWQ8xPXuBww4qSumnjycjjKDThFUj4nDgGS3UPoLyRBvqJ" \ + --bootnodes "/dns/validator.mainnet-sxt.ethernodes.io/tcp/30333/p2p/12D3KooWMEq9xSwmr8uTthCb6KM86av1T7AYe44kDPxWovEqzq1w" + --bootnodes "/dns/bootnode1.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWN8ZsZDNVr1ooMWJtsqMNHCVaQ4cvxvB8kELqrZeqct79" + --bootnodes "/dns/bootnode2.sxt.blockhunters.services/tcp/30333/p2p/12D3KooWSvSQNVHGmK965dKcCDGaxtyeY1XPCMwFUSLC8opguG1T" + --bootnodes "/ip4/51.210.3.173/tcp/30333/p2p/12D3KooWRUd3BqRyiGfhxVb2BSyUDLK5nHHNXTddZpqzqvQ73C9u" + --bootnodes "/ip4/141.95.65.179/tcp/30683/p2p/12D3KooWQ8xPXuBww4qSumnjycjjKDThFUj4nDgGS3UPoLyRBvqJ" + --database "paritydb" --validator --port 30333 --log info diff --git a/docs/testnet.md b/docs/testnet.md index 9c5621a5..1eadd053 100644 --- a/docs/testnet.md +++ b/docs/testnet.md @@ -120,7 +120,43 @@ The generated key should now be in a file called `subkey.key` in the sxt-node-ke Here we assume the setup uses the following volumes: `sxt-testnet-data` is the block storage volume and the volume where the generated node key is stored is `sxt-node-key`. -### 2.1. Docker Run +### 2.1. Snapshot Download and Extraction + +When starting a new node, the initial syncing of blocks with the rest of the network can take multiple days. +To alleviate this, it is recommended to download and extract a snapshot of the `paritydb` database prior to booting your node. +Space and Time roughly publishes snapshots on a two-week cadence. + +The date of the latest snapshot (in `YYYY-MM-DD` format) can be found at https://snapshots.testnet.sxt.network/latest.txt +The snapshot and checksum for the corresponding date can then be downloaded at the following URLs, respectively +(where `${DATE}` is once again in `YYYY-MM-DD` format). + +``` +https://snapshots.testnet.sxt.network/${DATE}/sxt-mainnet.tar.gz +https://snapshots.testnet.sxt.network/${DATE}/sxt-mainnet.sha1 +``` + +Once downloaded, extract the contents of the archive onto the `/chains` directory in the block storage volume. +The following snippet downloads and extracts the snapshot as describe above leveraging the [aria2](https://aria2.github.io/) +utility for parallelizing and retrying the download process. + +```bash +LATEST_DATE="$(curl -s https://snapshots.testnet.sxt.network/latest.txt)" +pushd sxt-testnet-data # Directory that will be mounted as the block storage volume in docker +aria2c -x 16 -s 16 "https://snapshots.testnet.sxt.network/${LATEST_DATE}/sxt-testnet.tar.gz" +curl -O "https://snapshots.testnet.sxt.network/${LATEST_DATE}/sxt-testnet.sha1" +sha1sum --check "sxt-testnet.sha1" +mkdir -p chains +tar xf sxt-testnet.tar.gz -C chains +rm sxt-testnet.tar.gz +popd +``` + +Note that before the `sxt-testnet.tar.gz` archive gets deleted in the snippet above, you'll be required to have enough disk space +to host both the archive and the exploded directory, which roughly equates to 2.5TB. To get around this requirement, a separate +volume that gets temporarily mounted to download the snapshot is recommended. A streaming solution would likely be the most space efficient, +but it has not yet been developed. + +### 2.2. Docker Run Make sure to set VALIDATOR_NAME to make it easier to identify your node in the telemetry dashboard. @@ -146,6 +182,7 @@ docker run -d --restart always \ --bootnodes "/dns/new-validator0.testnet.sxt.network/tcp/30333/p2p/12D3KooWFSan1dfyXvyJkGoTf3Jyk7pLmWJpHEMriSYSt5JoqmDB" \ --bootnodes "/dns/new-validator1.testnet.sxt.network/tcp/30333/p2p/12D3KooWLcAKtSNC2fQM8SEPwsLSrNijfihQ7ENFdb3TpqS2WCas" \ --bootnodes "/dns/new-validator2.testnet.sxt.network/tcp/30333/p2p/12D3KooWHEJSqiT9KGVdU3HC7dvodY2DG8E8cHoVkSevA4ZVaM2d" \ + --database "paritydb" \ --validator \ --port 30333 \ --log info \ @@ -155,7 +192,7 @@ docker run -d --restart always \ --name ${VALIDATOR_NAME?} ``` -### 2.2. Docker Compose +### 2.3. Docker Compose Prepare a `docker-compose.yaml` file as follows: @@ -191,6 +228,7 @@ services: --bootnodes "/dns/new-validator0.testnet.sxt.network/tcp/30333/p2p/12D3KooWFSan1dfyXvyJkGoTf3Jyk7pLmWJpHEMriSYSt5JoqmDB" --bootnodes "/dns/new-validator1.testnet.sxt.network/tcp/30333/p2p/12D3KooWLcAKtSNC2fQM8SEPwsLSrNijfihQ7ENFdb3TpqS2WCas" --bootnodes "/dns/new-validator2.testnet.sxt.network/tcp/30333/p2p/12D3KooWHEJSqiT9KGVdU3HC7dvodY2DG8E8cHoVkSevA4ZVaM2d" + --database paritydb --validator --port 30333 --log info