-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker local-testnet scripts #5998
Changes from 15 commits
41d8908
29367f3
d13ca14
42d2b78
a77aef7
b8cf372
4f53303
c6caf6c
092ef55
01e02fd
24c1ac4
ecd2a34
9bf2d7c
96adb1f
e3cd364
d3cf4f8
71d48d4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# Setting up a local-testnet with Docker | ||
|
||
First and foremost, one needs to build the **seednode** & **node** images. Hence, the **_build.sh_** | ||
script is provided. This can be done, by invoking the script or building the images manually. | ||
|
||
``` | ||
./build.sh # (Optional) Can be ignored if you already have the images stored in the local registry. | ||
./setup.sh # Will setup the local-testnet. | ||
./clean.sh # Will stop and remove the containers related to the local-testnet. | ||
|
||
Optionally | ||
./stop.sh # Will stop all the containers in the local-testnet. | ||
./start.sh # Will start all stopped containers from the initial local-testnet. | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eux | ||
|
||
pushd ../.. | ||
|
||
docker build -f docker/seednode/Dockerfile . -t seednode:dev | ||
|
||
docker build -f docker/node/Dockerfile . -t node:dev | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eux | ||
|
||
# Delete the entire testnet folder, which includes configuration, executables and logs. | ||
|
||
export MULTIVERSXTESTNETSCRIPTSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
|
||
source "$MULTIVERSXTESTNETSCRIPTSDIR/variables.sh" | ||
|
||
# Get the IDs of containers attached to the network | ||
CONTAINER_IDS=$(docker network inspect -f '{{range .Containers}}{{.Name}} {{end}}' "$DOCKER_NETWORK_NAME") | ||
|
||
# Stop each container | ||
echo "Removing containers..." | ||
for CONTAINER_ID in $CONTAINER_IDS; do | ||
docker stop "$CONTAINER_ID" | ||
docker rm "$CONTAINER_ID" | ||
done | ||
|
||
echo "Removing network..." | ||
docker network rm ${DOCKER_NETWORK_NAME} | ||
|
||
echo "Removing $TESTNETDIR..." | ||
rm -rf $TESTNETDIR |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,231 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Starts from 3, if the DOCKER_NETWORK_SUBNET ends with a 0. The first IP address is reserved for the gateway and the | ||
# second one is allocated to the seednode. Therefore the counting starts from 3. If you modify the DOCKER_NETWORK_SUBNET | ||
# variable, make sure to change this one accordingly too. | ||
IP_HOST_BYTE=3 | ||
|
||
cloneRepositories() { | ||
cd $(dirname $MULTIVERSXDIR) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why this change? Does this function will always clone the proxy and deploy repos? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, it will. But if the repository already exists, it won't clone them again. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. confirmed that is working properly 👍
|
||
|
||
git clone [email protected]:multiversx/mx-chain-deploy-go.git || true | ||
git clone [email protected]:multiversx/mx-chain-proxy-go.git || true | ||
} | ||
|
||
buildNodeImages() { | ||
cd $MULTIVERSXDIR | ||
|
||
docker build -f docker/seednode/Dockerfile . -t seednode:dev | ||
|
||
docker build -f docker/node/Dockerfile . -t node:dev | ||
} | ||
|
||
createDockerNetwork() { | ||
docker network create -d bridge --subnet=${DOCKER_NETWORK_SUBNET} ${DOCKER_NETWORK_NAME} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe check if network There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. here I was referring to the fact that if the network is already created, it will fail because it is not able to create docker network again |
||
|
||
# this variable is used to keep track of the allocated IP addresses in the network, by removing the last byte | ||
# of the DOCKER_NETWORK_SUBNET. One can consider this the host network address without the last byte at the end. | ||
export NETWORK_ADDRESS=$(echo "$DOCKER_NETWORK_SUBNET" | rev | cut -d. -f2- | rev) | ||
} | ||
|
||
startSeedNode() { | ||
local publishPortArgs="" | ||
if [[ "$DOCKER_PUBLISH_PORTS" -gt 0 ]]; then | ||
publishPortArgs="-p $DOCKER_PUBLISH_PORT_RANGE:10000" | ||
(( DOCKER_PUBLISH_PORT_RANGE++ )) | ||
fi | ||
|
||
docker run -d --name seednode \ | ||
-v ${TESTNETDIR}/seednode/config:/go/mx-chain-go/cmd/seednode/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
$publishPortArgs \ | ||
seednode:dev \ | ||
--rest-api-interface=0.0.0.0:10000 | ||
} | ||
|
||
startObservers() { | ||
local observerIdx=0 | ||
local publishPortArgs="" | ||
|
||
# Example for loop with injected variables in Bash | ||
for ((i = 0; i < SHARDCOUNT; i++)); do | ||
for ((j = 0; j < SHARD_OBSERVERCOUNT; j++)); do | ||
# Your commands or code to be executed in each iteration | ||
KEY_INDEX=$((TOTAL_NODECOUNT - observerIdx - 1)) | ||
|
||
if [[ "$DOCKER_PUBLISH_PORTS" -gt 0 ]]; then | ||
publishPortArgs="-p $DOCKER_PUBLISH_PORT_RANGE:10000" | ||
fi | ||
|
||
docker run -d --name "observer${observerIdx}-${NETWORK_ADDRESS}.${IP_HOST_BYTE}-10200-shard${i}" \ | ||
-v $TESTNETDIR/node/config:/go/mx-chain-go/cmd/node/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
$publishPortArgs \ | ||
node:dev \ | ||
--destination-shard-as-observer $i \ | ||
--rest-api-interface=0.0.0.0:10200 \ | ||
--config ./config/config_observer.toml \ | ||
--sk-index=${KEY_INDEX} \ | ||
$EXTRA_OBSERVERS_FLAGS | ||
|
||
|
||
(( IP_HOST_BYTE++ )) | ||
(( DOCKER_PUBLISH_PORT_RANGE++ )) | ||
((observerIdx++)) || true | ||
done | ||
done | ||
|
||
for ((i = 0; i < META_OBSERVERCOUNT; i++)); do | ||
KEY_INDEX=$((TOTAL_NODECOUNT - observerIdx - 1)) | ||
|
||
if [[ "$DOCKER_PUBLISH_PORTS" -gt 0 ]]; then | ||
publishPortArgs="-p $DOCKER_PUBLISH_PORT_RANGE:10000" | ||
fi | ||
|
||
docker run -d --name "observer${observerIdx}-${NETWORK_ADDRESS}.${IP_HOST_BYTE}-10200-metachain" \ | ||
-v $TESTNETDIR/node/config:/go/mx-chain-go/cmd/node/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
$publishPortArgs \ | ||
node:dev \ | ||
--destination-shard-as-observer "metachain" \ | ||
--rest-api-interface=0.0.0.0:10200 \ | ||
--config ./config/config_observer.toml \ | ||
--sk-index=${KEY_INDEX} \ | ||
$EXTRA_OBSERVERS_FLAGS | ||
|
||
(( IP_HOST_BYTE++ )) | ||
(( DOCKER_PUBLISH_PORT_RANGE++ )) | ||
((observerIdx++)) || true | ||
done | ||
} | ||
|
||
startValidators() { | ||
local validatorIdx=0 | ||
local publishPortArgs="" | ||
# Example for loop with injected variables in Bash | ||
for ((i = 0; i < SHARDCOUNT; i++)); do | ||
for ((j = 0; j < SHARD_VALIDATORCOUNT; j++)); do | ||
|
||
if [[ "$DOCKER_PUBLISH_PORTS" -gt 0 ]]; then | ||
publishPortArgs="-p $DOCKER_PUBLISH_PORT_RANGE:10000" | ||
fi | ||
|
||
docker run -d --name "validator${validatorIdx}-${NETWORK_ADDRESS}.${IP_HOST_BYTE}-10200-shard${i}" \ | ||
-v $TESTNETDIR/node/config:/go/mx-chain-go/cmd/node/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
$publishPortArgs \ | ||
node:dev \ | ||
--rest-api-interface=0.0.0.0:10200 \ | ||
--config ./config/config_validator.toml \ | ||
--sk-index=${validatorIdx} \ | ||
|
||
(( IP_HOST_BYTE++ )) | ||
(( DOCKER_PUBLISH_PORT_RANGE++ )) | ||
((validatorIdx++)) || true | ||
done | ||
done | ||
|
||
for ((i = 0; i < META_VALIDATORCOUNT; i++)); do | ||
|
||
if [[ "$DOCKER_PUBLISH_PORTS" -gt 0 ]]; then | ||
publishPortArgs="-p $DOCKER_PUBLISH_PORT_RANGE:10000" | ||
fi | ||
|
||
docker run -d --name "validator${validatorIdx}-${NETWORK_ADDRESS}.${IP_HOST_BYTE}-10200-metachain" \ | ||
-v $TESTNETDIR/node/config:/go/mx-chain-go/cmd/node/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
$publishPortArgs \ | ||
node:dev \ | ||
--rest-api-interface=0.0.0.0:10200 \ | ||
--config ./config/config_observer.toml \ | ||
--sk-index=${validatorIdx} \ | ||
|
||
(( IP_HOST_BYTE++ )) | ||
(( DOCKER_PUBLISH_PORT_RANGE++ )) | ||
((validatorIdx++)) || true | ||
done | ||
} | ||
|
||
updateProxyConfigDocker() { | ||
pushd $TESTNETDIR/proxy/config | ||
cp config.toml config_edit.toml | ||
|
||
# Truncate config.toml before the [[Observers]] list | ||
sed -i -n '/\[\[Observers\]\]/q;p' config_edit.toml | ||
|
||
if [ "$SHARD_OBSERVERCOUNT" -le 0 ]; then | ||
generateProxyValidatorListDocker config_edit.toml | ||
else | ||
generateProxyObserverListDocker config_edit.toml | ||
fi | ||
|
||
mv config_edit.toml config.toml | ||
echo "Updated configuration for the Proxy." | ||
popd | ||
} | ||
|
||
generateProxyObserverListDocker() { | ||
local ipByte=3 | ||
|
||
for ((i = 0; i < SHARDCOUNT; i++)); do | ||
for ((j = 0; j < SHARD_OBSERVERCOUNT; j++)); do | ||
|
||
echo "[[Observers]]" >> config_edit.toml | ||
echo " ShardId = $i" >> config_edit.toml | ||
echo " Address = \"http://${NETWORK_ADDRESS}.${ipByte}:10200\"" >> config_edit.toml | ||
echo ""$'\n' >> config_edit.toml | ||
|
||
(( ipByte++ )) || true | ||
done | ||
done | ||
|
||
for META_OBSERVER in $(seq $META_OBSERVERCOUNT); do | ||
echo "[[Observers]]" >> config_edit.toml | ||
echo " ShardId = $METASHARD_ID" >> config_edit.toml | ||
echo " Address = \"http://${NETWORK_ADDRESS}.${ipByte}:10200\"" >> config_edit.toml | ||
echo ""$'\n' >> config_edit.toml | ||
|
||
(( ipByte++ )) || true | ||
done | ||
} | ||
|
||
generateProxyValidatorListDocker() { | ||
local ipByte=3 | ||
|
||
for ((i = 0; i < SHARDCOUNT; i++)); do | ||
for ((j = 0; j < SHARD_VALIDATORCOUNT; j++)); do | ||
|
||
echo "[[Observers]]" >> config_edit.toml | ||
echo " ShardId = $i" >> config_edit.toml | ||
echo " Address = \"http://${NETWORK_ADDRESS}.${ipByte}:10200\"" >> config_edit.toml | ||
echo " Type = \"Validator\"" >> config_edit.toml | ||
echo ""$'\n' >> config_edit.toml | ||
|
||
(( ipByte++ )) || true | ||
done | ||
done | ||
|
||
for META_OBSERVER in $(seq $META_VALIDATORCOUNT); do | ||
echo "[[Observers]]" >> config_edit.toml | ||
echo " ShardId = $METASHARD_ID" >> config_edit.toml | ||
echo " Address = \"http://${NETWORK_ADDRESS}.${ipByte}:10200\"" >> config_edit.toml | ||
echo " Type = \"Validator\"" >> config_edit.toml | ||
echo ""$'\n' >> config_edit.toml | ||
|
||
(( ipByte++ )) || true | ||
done | ||
} | ||
|
||
buildProxyImage() { | ||
pushd ${PROXYDIR} | ||
cd ../.. | ||
docker build -f docker/Dockerfile . -t proxy:dev | ||
} | ||
|
||
startProxyDocker() { | ||
docker run -d --name "proxy" \ | ||
-v $TESTNETDIR/proxy/config:/mx-chain-proxy-go/cmd/proxy/config \ | ||
--network ${DOCKER_NETWORK_NAME} \ | ||
-p $PORT_PROXY:8080 \ | ||
proxy:dev | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eux | ||
|
||
export DOCKERTESTNETDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
|
||
MULTIVERSXTESTNETSCRIPTSDIR="$(dirname "$DOCKERTESTNETDIR")/testnet" | ||
|
||
source "$DOCKERTESTNETDIR/variables.sh" | ||
source "$DOCKERTESTNETDIR/functions.sh" | ||
source "$MULTIVERSXTESTNETSCRIPTSDIR/include/config.sh" | ||
source "$MULTIVERSXTESTNETSCRIPTSDIR/include/build.sh" | ||
|
||
cloneRepositories | ||
|
||
prepareFolders | ||
|
||
buildConfigGenerator | ||
|
||
generateConfig | ||
|
||
copyConfig | ||
|
||
copySeednodeConfig | ||
updateSeednodeConfig | ||
|
||
copyNodeConfig | ||
updateNodeConfig | ||
|
||
createDockerNetwork | ||
|
||
startSeedNode | ||
startObservers | ||
startValidators | ||
|
||
if [ $USE_PROXY -eq 1 ]; then | ||
buildProxyImage | ||
prepareFolders_Proxy | ||
copyProxyConfig | ||
updateProxyConfigDocker | ||
startProxyDocker | ||
fi | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eux | ||
|
||
file_path="./tmp/stopped_containers" | ||
|
||
# Check if the file exists | ||
if [ ! -f "$file_path" ]; then | ||
echo "File $file_path not found." | ||
exit 1 | ||
fi | ||
|
||
# Read the file line by line | ||
while IFS= read -r line; do | ||
docker start $line | ||
done < "$file_path" | ||
|
||
rmdir ./tmp | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. add empty line? |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eux | ||
|
||
# Delete the entire testnet folder, which includes configuration, executables and logs. | ||
|
||
export MULTIVERSXTESTNETSCRIPTSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
|
||
source "$MULTIVERSXTESTNETSCRIPTSDIR/variables.sh" | ||
|
||
# Get the IDs of containers attached to the network | ||
export CONTAINER_IDS=$(docker network inspect -f '{{range $k, $v := .Containers}}{{printf "%s\n" $k}}{{end}}' "$DOCKER_NETWORK_NAME") | ||
|
||
mkdir -p ./tmp | ||
|
||
# Stop each container | ||
echo "Stopping containers..." | ||
for CONTAINER_ID in $CONTAINER_IDS; do | ||
docker stop "$CONTAINER_ID" | ||
echo "$CONTAINER_ID" >> ./tmp/stopped_containers | ||
done | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. add empty line? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add
#!/usr/bin/env bash
here also? i think pushd is bash specificThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this script intended to be run separately?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it is. I could have added the build step inside the
./start.sh
script, but it was easier for me to debug, since it didn't build the images at every run. Don't have a strong opinion on this, I can add it to the wrapper script instead. I was planning on adding aREADME.md
nevertheless.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is better that we keep the build separate from the run script. Not a strong opinion, thus.