Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(devx): Split and Edit full node guide #4182

Merged
merged 8 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 71 additions & 55 deletions docker/fullnode/README.md
Original file line number Diff line number Diff line change
@@ -1,98 +1,132 @@
# Use Docker to Run a Iota Full node Locally
# Use Docker to Run an IOTA Full Node Locally

Follow the steps in this Readme to install and configure a Iota Full node for testing locally using Docker. The instructions were validated on the following operating system/processor combinations:
Use this guide to install and configure an IOTA Full Node locally using Docker for testing. The instructions apply to the following operating system and processor combinations:

- Linux/AMD64
- Darwin/AMD64
- Darwin/ARM64

## Prerequisites

- [Install Docker](https://docs.docker.com/get-docker/)
- [Install Docker Compose](https://docs.docker.com/compose/install/)
- Download the Full node [docker-compose.yaml](https://github.com/iotaledger/iota/blob/develop/docker/fullnode/docker-compose.yaml) file.
Before you begin, ensure you have the following:

## Configure Iota Full node
- [Docker](https://docs.docker.com/get-docker/) installed.
- [Docker Compose](https://docs.docker.com/compose/install/) installed.
- Download the full node [docker-compose.yaml](https://github.com/iotaledger/iota/blob/develop/docker/fullnode/docker-compose.yaml) file.

Download the latest version of the Iota Full node configuration file [fullnode-template.yaml](https://github.com/iotaledger/iota/raw/develop/crates/iota-config/data/fullnode-template.yaml). Use the following command to download the file:
## Configure the IOTA Full Node

Download the latest version of the IOTA Full Node configuration file [fullnode-template.yaml](https://github.com/iotaledger/iota/raw/develop/crates/iota-config/data/fullnode-template.yaml). You can use the following command to download the file:

```shell
wget https://github.com/iotaledger/iota/raw/develop/crates/iota-config/data/fullnode-template.yaml
```

To ensure you add the appropriate peers for your full node, please refer to the instructions provided at the following link:
https://docs.iota.org/operator/iota-full-node#set-up-from-source and proceed with step 4.

### Download the Iota genesis blob
### Add Peers

The genesis blob contains the information that defined the Iota network configuration. Before you can start the Full node, you need to download the most recent file to ensure compatibility with the version of Iota you use.
For `Testnet` or `Devnet` nodes, edit the `fullnode.yaml` file to include peer nodes for state synchronization. Append
the following to the end of the current configuration:

To download the appropriate genesis.blob file, please refer to the instructions provided at the following link:
https://docs.iota.org/operator/iota-full-node#set-up-from-source. and proceed with step 3.
#### Devnet

## Start your Iota Full node
```yaml
p2p-config:
seed-peers:
- address: /dns/access-0.r.devnet.iota.cafe/udp/8084
peer-id: 01589ac910a5993f80fbc34a6e0c8b2041ddc5526a951c838df3037e11ab0188
- address: /dns/access-1.r.devnet.iota.cafe/udp/8084
peer-id: 32875c547ea3b44fa08a711646cedb70fa0c97959d236578131505da09723add
```

Run the following command to start the Iota fullnode in Docker:
#### Testnet

```shell
docker compose up
```yaml
p2p-config:
seed-peers:
- address: /dns/access-0.r.testnet.iota.cafe/udp/8084
peer-id: 46064108d0b689ed89d1f44153e532bb101ce8f8ca3a3d01ab991d4dea122cfc
- address: /dns/access-1.r.testnet.iota.cafe/udp/8084
peer-id: 8ffd25fa4e86c30c3f8da7092695e8a103462d7a213b815d77d6da7f0a2a52f5
```

**Important:** The commands in this document assume you use Docker Compose V2. The `docker compose` command uses a dash (`docker-compose`) in Docker Compose V1. If you use Docker Compose V1, replace the space in each `docker compose` command with a dash (`docker-compose`). For more information, see [Docker Compose V2](https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command).
### Download the IOTA Genesis Blob

The genesis blob defines the IOTA network configuration. Before starting the Full Node, download the latest
`genesis.blob` file to ensure compatibility with your version of IOTA. Keep in mind that the [IOTA Mainnet](https://wiki.iota.org/build/networks-endpoints/#iota) runs the
[Stardust Protocol](https://wiki.iota.org/learn/protocols/stardust/introduction/) at the moment, so only Testnet and Devnet blobs are available.

## Test the Iota Full node
- [Testnet genesis blob](https://dbfiles.testnet.iota.cafe/genesis.blob):
`curl -fLJO https://dbfiles.testnet.iota.cafe/genesis.blob`
- [Devnet genesis blob](https://dbfiles.devnet.iota.cafe/genesis.blob):
`curl -fLJO https://dbfiles.devnet.iota.cafe/genesis.blob`
- [Devnet migration blob](https://dbfiles.devnet.iota.cafe/migration.blob):
`curl -fLJO https://dbfiles.devnet.iota.cafe/migration.blob`

After the Full node starts you can test the JSON-RPC interfaces.
## Start Your IOTA Full Node

## View activity on your local Full node with Iota Explorer
Run the following command to start the IOTA Full Node in Docker:

Iota Explorer supports connecting to a local network. To view activity on your local Full node, open the URL: [https://explorer.iota.org/?network=local](https://explorer.iota.org/?network=local).
```shell
docker compose up
```

You can also change the network that Iota Explorer connects to by select it in the Iota Explorer interface.
**Important:** These commands assume you are using Docker Compose V2. In Docker Compose V1, the `docker compose` command uses a dash (`docker-compose`). If you use Docker Compose V1, replace the space in each `docker compose` command with a dash (`docker-compose`). For more information, see [Docker Compose V2](https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command).

### Stop the Full node
## Stop the Full Node

Run the following command to stop the Full node when you finish using it:
Run the following command to stop the Full Node when you finish using it:

```shell
docker compose stop
```

## Test the IOTA Full Node

After the Full Node starts, you can test the [JSON-RPC interfaces](https://docs.iota.org/iota-api-ref).

## Troubleshooting

If you encounter errors or your Full node stops working, run the commands in the following section to resolve the issue.
If you encounter errors or your Full Node stops working, run the commands in the following section to resolve the issue.

### Start the Full node in detached mode
### Start the Full Node in Detached Mode

First, try starting the Full node in detached mode:
First, try starting the Full Node in detached mode:

```shell
docker compose up -d
```

### Reset the environment
### Reset the Environment

If you continue to see issues, stop the Full node (`docker compose stop`) and delete the Docker container and volume. Then run the following command to start a new instance of the Full node using the same genesis blob.
If you continue to see issues, stop the Full Node (`docker compose stop`) and delete the Docker container and volume. Then run the following command to start a new instance of the Full Node using the same genesis blob:

```shell
docker compose down --volumes
```

### Stats (CPU/MEM USAGE %)
## Monitoring

### View Activity on Your Local Full Node with IOTA Explorer

To view usage details for the Full node running in Docker, run the following command:
The IOTA Explorer supports connecting to any network as long as it has `https` enabled. To view activity on your local
Full Node, open the URL: [https://explorer.rebased.iota.org/](https://explorer.rebased.iota.org/), and select Custom
RPC URL from the network dropdown in the top right.

### View Resource Usage (CPU and Memory)

To view resource usage details for the Full Node running in Docker, run the following command:

```shell
docker stats
```

This command shows a live data stream of the Docker container resource usage, such as CPU and memory. To view data for all containers, use the following command:
This command shows a live data stream of the Docker container's resource usage, such as CPU and memory. To view data for all containers, use the following command:

```shell
docker stats -a
```

### Inspect the state of a running Full node
### Inspect the State of a Running Full Node

Get the running container ID:

Expand All @@ -112,9 +146,9 @@ Inspect the database:
ls -la iotadb/
```

### Investigate local RPC connectivity issues
### Investigate Local RPC Connectivity Issues

Update the `json-rpc-address` in the Full node config to listen on all addresses:
Update the `json-rpc-address` in the Full Node config to listen on all addresses:

```shell
sed -i 's/127.0.0.1/0.0.0.0/' fullnode-template.yaml
Expand All @@ -124,21 +158,3 @@ sed -i 's/127.0.0.1/0.0.0.0/' fullnode-template.yaml
-json-rpc-address: "127.0.0.1:9000"
+json-rpc-address: "0.0.0.0:9000"
```

### Install wget and curl

Download each package. For example, on macOS use [homebrew](https://brew.sh/):

`brew install wget curl`

### Learn more about Iota

- https://docs.iota.io/learn

### Learn more about building and running a Full node from source code

- https://docs.iota.io/build/fullnode

### Learn more about Docker Compose

- https://docs.docker.com/compose/gettingstarted/
7 changes: 7 additions & 0 deletions docs/content/_snippets/node-hardware-requirements.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
:::info Minimum Hardware Requirements

- CPUs: 8 physical cores / 16 vCPUs
- RAM: 128 GB
- Storage (SSD): 4 TB NVMe drive

:::
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
:::warning Advanced

These instructions are for setting up a Full node for network participation. If you just need a local development environment,
you should instead follow the instructions in [Create a Local IOTA Network](/developer/getting-started/local-network) to create a local Full node, validators, and faucet.

:::
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The following diagram describes the architectural structure for IOTA's solution.
The core components are:

- [Execution Layer](../../references/execution-architecture/execution-layer.mdx)
- [IOTA Node](../../operator/iota-full-node.mdx)
- [IOTA Node](../../operator/iota-full-node/overview.mdx)
- [IOTA RPC](../../references/iota-api)
- [IOTA CLI](../../references/cli)

Expand Down
2 changes: 1 addition & 1 deletion docs/content/developer/advanced/custom-indexer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ This is suitable for setups with a single ingestion pipeline where progress trac

Colocate the data ingestion daemon with a Full node and enable checkpoint dumping on the latter to set up a local stream source. After enabling, the Full node starts dumping executed checkpoints as files to a local directory, and the data ingestion daemon subscribes to changes in the directory through an inotify-like mechanism. This approach allows minimizing ingestion latency (checkpoint are processed immediately after a checkpoint executor on a Full node) and getting rid of dependency on an externally managed bucket.

To enable, add the following to your [Full node configuration](../../operator/iota-full-node.mdx) file:
To enable, add the following to your [Full node configuration](../../operator/iota-full-node/overview.mdx) file:

```yaml
checkpoint-executor-config:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ You can develop and test your integration using a local development server. It w

## Configure a IOTA Full node

You can set up and configure a [IOTA Full node](../../operator/iota-full-node.mdx) using Docker or directly from source code in the IOTA GitHub repository.
You can set up and configure a [IOTA Full node](../../operator/iota-full-node/overview.mdx) using Docker or directly from source code in the IOTA GitHub repository.

## Set up IOTA addresses

Expand Down
6 changes: 3 additions & 3 deletions docs/content/operator/data-management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: A high-level description of data management on the IOTA network tha
import Quiz from '@site/src/components/Quiz';
import questions from '/json/node-operators/data-management.json';

Managing the data on your IOTA Full node is critical to ensuring a healthy IOTA network. This topic provides a high-level description of data management on IOTA Full nodes that you can use to optimize your Full node configuration. For more information about IOTA Full nodes, such as pruning policies and archival settings, see [Run an IOTA Full Node](iota-full-node.mdx).
Managing the data on your IOTA Full node is critical to ensuring a healthy IOTA network. This topic provides a high-level description of data management on IOTA Full nodes that you can use to optimize your Full node configuration. For more information about IOTA Full nodes, such as pruning policies and archival settings, see [Run an IOTA Full Node](iota-full-node/overview.mdx).

## Basic IOTA Full node functionality

Expand Down Expand Up @@ -75,7 +75,7 @@ As described previously, sustainable disk usage requires IOTA Full nodes to prun

Both transaction and object pruners run in the background. The logical deletion of entries from RocksDB ultimately triggers the physical compaction of data on disk, which is governed by RocksDB background jobs: the pruning effect on disk usage is not immediate and might take multiple days.

To learn more about object pruning policies, see [Object pruning](iota-full-node.mdx#object-pruning). You can configure the pruner in two modes:
To learn more about object pruning policies, see [Object pruning](iota-full-node/monitoring-and-pruning.mdx#object-pruning). You can configure the pruner in two modes:
* **aggressive pruning** (`num-epochs-to-retain: 0`): Preferred option. IOTA prunes old object versions as soon as possible.
* **epoch-based pruning** (`num-epochs-to-retain: X`): IOTA prunes old object versions after X epochs.

Expand All @@ -85,7 +85,7 @@ Testing indicates that aggressive pruning results in more efficient Full Node op

:::

To learn more about transaction pruning policies, see [Transaction pruning](iota-full-node.mdx#transaction-pruning). To configure transaction pruning, specify the `num-epochs-to-retain-for-checkpoints: X` config option. The checkpoints, including their transactions, effects and events are pruned up to X epochs ago. We suggest setting transaction pruning to 2 epochs.
To learn more about transaction pruning policies, see [Transaction pruning](iota-full-node/monitoring-and-pruning.mdx#transaction-pruning). To configure transaction pruning, specify the `num-epochs-to-retain-for-checkpoints: X` config option. The checkpoints, including their transactions, effects and events are pruned up to X epochs ago. We suggest setting transaction pruning to 2 epochs.

### Set an archiving watermark

Expand Down
2 changes: 1 addition & 1 deletion docs/content/operator/genesis.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ The genesis.blob files for each network are in:
- [Devnet genesis blob](https://dbfiles.devnet.iota.cafe/genesis.blob):
`curl -fLJO https://dbfiles.devnet.iota.cafe/genesis.blob`

See [IOTA Full Node](iota-full-node.mdx#setting-up-a-full-node-from-source) for how to get the genesis.blob file for each network.
See [IOTA Full Node](iota-full-node/overview.mdx) for how to get the genesis.blob file for each network.
Loading