Skip to content

Commit

Permalink
fix(devx) updated S3 URLs
Browse files Browse the repository at this point in the history
  • Loading branch information
lucas-tortora committed Nov 18, 2024
1 parent cc46ba5 commit c4fd016
Showing 1 changed file with 8 additions and 17 deletions.
25 changes: 8 additions & 17 deletions docs/content/operator/snapshots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,12 @@ To restore from a RocksDB snapshot, follow these steps:
- `--path`: Path to snapshot directory on local filesystem.
- `--no-sign-request`: If set, `--snapshot-bucket` and `--snapshot-bucket-type` are ignored, and Cloudflare R2 is used.
- `--snapshot-bucket`: Source snapshot bucket name, eg `iota-mainnet-snapshots`. This cannot be used with `--no-sign-request`.
- `--snapshot-bucket-type`: Snapshot bucket type. GCS and S3 currently supported. This cannot be used with `--no-sign-request`.
- `--snapshot-bucket-type`: Snapshot bucket type. S3 currently supported. This cannot be used with `--no-sign-request`.
- `--skip-indexes`: Skips downloading the very large `indexes/` dir, used by jsonrpc on the fullnode.

The following environment variables are used if `--no-sign-request` is not set:
* *AWS*: `AWS_SNAPSHOT_ACCESS_KEY_ID`, `AWS_SNAPSHOT_SECRET_ACCESS_KEY`, `AWS_SNAPSHOT_REGION`
* *GCS*: `GCS_SNAPSHOT_SERVICE_ACCOUNT_FILE_PATH`
* *AZURE*: `AZURE_SNAPSHOT_STORAGE_ACCOUNT`, `AZURE_SNAPSHOT_STORAGE_ACCESS_KEY`
*

1. When using `iota-tool download-db-snapshot` the database is copied to the location you pass to `--path`, in a directory named `epoch_[NUM]`. Move this directory to the `live/` Full node database directory, for example `/opt/iota/db/authorities_db/full_node_db/live`.
1. Make sure you update the ownership of the downloaded directory to the `iota` user (or whichever linux user you run `iota-node` as):

Expand Down Expand Up @@ -105,37 +103,30 @@ The following steps can be used to restore a node from a Formal snapshot:
- `--path`: Path to snapshot directory on local filesystem.
- `--no-sign-request`: If set, `--snapshot-bucket` and `--snapshot-bucket-type` are ignored, and Cloudflare R2 is used.
- `--snapshot-bucket`: Source snapshot bucket name, eg `iota-mainnet-snapshots`. This cannot be used with `--no-sign-request`.
- `--snapshot-bucket-type`: Snapshot bucket type. GCS and S3 currently supported. This cannot be used with `--no-sign-request`.
- `--snapshot-bucket-type`: Snapshot bucket type. S3 currently supported. This cannot be used with `--no-sign-request`.

The following environment variables are used if `--no-sign-request` is not set:
* *AWS*: `AWS_SNAPSHOT_ACCESS_KEY_ID`, `AWS_SNAPSHOT_SECRET_ACCESS_KEY`, `AWS_SNAPSHOT_REGION`
* *GCS*: `GCS_SNAPSHOT_SERVICE_ACCOUNT_FILE_PATH`
* *AZURE*: `AZURE_SNAPSHOT_STORAGE_ACCOUNT`, `AZURE_SNAPSHOT_STORAGE_ACCESS_KEY`


## IOTA Foundation managed snapshots

The IOTA Foundation hosts two tiers of snapshot storage access. **High throughput, Requester Pays enabled buckets**, and **free, permissionless buckets**.

**High throughput, Requester Pays enabled buckets:**
* GCS and S3 are both setup with requester pays. This means that you'll need to provide a set of valid AWS/GCP credentials when downloading from these buckets.
[Requester Pays](https://cloud.google.com/storage/docs/requester-pays) means you are charged for the egress costs of pulling the snapshot data.
* S3 are both setup with requester pays. This means that you'll need to provide a set of valid AWS credentials when downloading from these buckets.
* If you are looking for the best download speeds, we recommend using the S3 buckets with [transfer acceleration](https://aws.amazon.com/s3/transfer-acceleration/) enabled.

**Free, permissionless buckets:**
* These are currently hosted on Cloudflare R2, currently only in North America, but we plan on adding more regions soon.
* Since the bucket is open to the internet, there's no need to provide any cloud credentials.

### Bucket Names
**S3**
Testnet: `s3://iota-testnet-snapshots/`, `s3://iota-testnet-formal/`
Mainnet: `s3://iota-mainnet-snapshots/`, `s3://iota-mainnet-formal/`

**GCS**
Testnet: `gs://iota-testnet-snapshots/`, `gs://iota-testnet-formal/`
Mainnet: `gs://iota-mainnet-snapshots/`, `gs://iota-mainnet-formal/`
**S3**
Testnet: `s3://iota-rebased-testnet-snapshots/`, `s3://iota-rebased-testnet-formal/`
Devnet: `s3://iota-rebased-devnet-snapshots/`, `s3://iota-rebased-devnet-formal/`

![IOTA Managed Snapshots](/img/operator/snapshots/iota-cloud-snapshots.png "A diagram that shows the current architecture of iota snapshot availability")

## Enabling snapshots

Expand Down Expand Up @@ -181,6 +172,6 @@ Add an entry to the config file for `state-snapshot-write-config`. Using Amazon'
aws-region: "<BUCKET-REGION>"
object-store-connection-limit: 200
```
The configuration settings shown in the example are specific to AWS S3, but GCS, Azure Storage, and Cloudflare R2 are all supported.
The configuration settings shown in the example are specific to AWS S3.

<Quiz questions={questions} />

0 comments on commit c4fd016

Please sign in to comment.