This project is based on Bitnodes.
Ravennodes is currently being developed to estimate the size of the Ravencoin network by finding all the reachable nodes in the network. These are the nodes that accept incoming connections. Why you should run a full node is explained here on the Bitcoin wiki. The current methodology involves sending getaddr messages recursively to find all the reachable nodes in the network, starting from a set of seed nodes. It is worth mentioning that this method of estimating network size does not list all full nodes because not all nodes have an open port that can be probed using Ravennodes. These nodes are either behind firewalls or they are configured to not listen for connections.
The goal is to run the service as efficient as possible. The ravennodes crawler, web_updater, and website currently all run on an a1 medium AWS server (1 vCPu, 2GB RAM).
- Changed parameters to support Ravencoin
- Turned off Tor network support.
- Python 2.7 virtual environment
- Redis
| Package | Version |
|---|---|
| dpkt | 1.9.1 |
| flake8 | 3.5.0 |
| geoip2 | 2.9.0 |
| scipy | 0.13.0 |
| gevent | 1.3.4 |
| ipaddress | 1.0.22 |
| PySocks | 1.6.8 |
| pytest | 3.7.3 |
| redis | 2.10.6 |
| requests | 2.20.0 |
sudo apt update && sudo apt upgrade
cd ~/
sudo apt install redis
# I prefer running redis manually
# Optional, remove the service: sudo service redis-server disable
#Move redis conf file
sudo cp [LOCATION]/crawler/depends/redis/redis.conf /etc/redis/
#Install Python requirements
cd [LOCATION]/crawler
pip install -r requirements.txt
#Update GeoIP
bash geoip/update.sh
NOTE: For GeoIP, you now need an account at maxmind.com and acquire a license key. Add this key to ~/.bashrc:
export MAXMIND_LICENSE_KEY=YOUR-KEY-HERE
# Edit the following file:
sudo nano /etc/security/limits.conf
#Add the following:
* soft nofile 1000000
* hard nofile 1000000
root soft nofile 1000000
root hard nofile 1000000
# If you run an Ubuntu GUI, Edit the following files too:
sudo nano /etc/systemd/user.conf
#Change the following setting:
DefaultLimitNOFILE=1000000
sudo nano /etc/systemd/system.conf
#Change the following setting:
DefaultLimitNOFILE=1000000
#Login as admin
sudo -i
#Update /proc/sys/net/core/somaxconn
sysctl -w net.core.somaxconn=511
#Disable THP
echo never > /sys/kernel/mm/transparent_hugepage/enabled
# Set overcommit_memory to 1
sysctl vm.overcommit_memory=1
#Start Redis Server
redis-server /etc/redis/redis.conf
#Alternatively, run redis in the background with redis-server /etc/redis/redis.conf --daemonize yes
#Make sure the file redis.sock is made in /tmp/
Open a new console, activate the python 2.7 virtual environment and start the crawler:
cd [LOCATION]/Ravennodes
./start.sh
Data output will be available in ~/Ravennodes/data/export/5241564e
Process logs will be available in ~/Ravennodes/log
- Data change watcher adaptation from Michael Cho looking for crawler updates
- Data updater generating ravencoin nodes tables and world maps for the website
- Python 3.7 virtual environment
| Package | Version |
|---|---|
| pandas | 0.24.1 |
| watchdog | 0.9.0 |
| numpy | 1.16.1 |
| plotly | 3.6.1 |
Open a new console, activate the python 3.7 virtual environment and start the updater:
cd [LOCATION]/web_updater
./start.sh
- Website built using flask
- Python 3.7 virtual environment
See website/requirements.txt
Website is hosted according to this guide
Everyone is welcome to propose changes. Currently, the whole system is set up using python for self-educational purposes. Using an SQL type database, for example, will drastically improve performance.
-Jeroz
https://github.com/jeroz1/Ravennodes https://github.com/RavenCommunity/Ravennodes