First, install anaconda from here (make sure to pick the python 3 version): Website. If you would like a smaller download, check out miniconda: Miniconda.
Next, we're going to add some channels that we need for certain software:
conda config --add channels conda-forge # For ONNX/tensorboardX conda config --add channels pytorch # For PyTorch
If you get an error saying that the "conda" command could not be found, make sure that anaconda is installed and your path is set correctly.
Clone and enter ReAgent repo:
git clone --recurse-submodules https://github.com/facebookresearch/ReAgent.git cd ReAgent/
If you already cloned the repo without submodules, they can be added by running this command inside the repository"
git submodule update --init --recursive
Install dependencies:
conda install --file requirements.txt
Set JAVA_HOME to the location of your anaconda install
export JAVA_HOME="$(dirname $(dirname -- `which conda`))" echo $JAVA_HOME # Should see something like "/home/jjg/miniconda3"
Install Spark (the mv command may need to be done as root):
wget https://archive.apache.org/dist/spark/spark-2.3.3/spark-2.3.3-bin-hadoop2.7.tgz tar -xzf spark-2.3.3-bin-hadoop2.7.tgz sudo mv spark-2.3.3-bin-hadoop2.7 /usr/local/spark
Add the spark bin directory to your path so your terminal can find spark-submit:
export PATH=$PATH:/usr/local/spark/bin
Install OpenAI Gym if you plan on following our tutorial:
pip install "gym[classic_control,box2d,atari]"
Download libtorch from https://pytorch.org/get-started/locally/ and extract it to $HOME/libtorch
As of pytorch 1.3, libtorch is broken on OS/X. To fix (mac only):
cp ~/miniconda3/lib/libiomp5.dylib $HOME/libtorch/lib/
And now, you are ready to install ReAgent itself. To install the serving platform:
mkdir serving/build cd serving/build cmake -DCMAKE_PREFIX_PATH=$HOME/libtorch .. make
Next we must package the models. We use "pip install -e" on the root directory of the repository to create an ephemral package. This means that you can make changes to ReAgent and they will be reflected in the package immediately.
pip install -e .
At this point, you should be able to run all unit tests:
pytest
We have included a Dockerfile for the CPU-only build and CUDA build under the docker directory. The CUDA build will need nvidia-docker to run.
To build, clone repository and cd into the respective directory:
git clone https://github.com/facebookresearch/ReAgent.git cd ReAgent/
On macOS you will need to increase the default memory allocation as the default of 2G is not enough. You can do this by clicking the whale icon in the task bar. We recommend using at least 8G of memory.
On macOS, you can then build the image:
docker build -f docker/cpu.Dockerfile -t horizon:dev .
On Linux you can build the image with specific memory allocations from command line:
docker build -f docker/cpu.Dockerfile -t horizon:dev --memory=8g --memory-swap=8g .
To build with CUDA support, use the corresponding dockerfile:
docker build -f docker/cuda.Dockerfile -t horizon:dev .
Once the Docker image is built you can start an interactive shell in the container and run the unit tests. To have the ability to edit files locally and have changes be available in the Docker container, mount the local ReAgent repo as a volume using the -v flag. We also add -p for port mapping so we can view Tensorboard visualizations locally.
docker run -v $PWD:/home/ReAgent -p 0.0.0.0:6006:6006 -it horizon:dev
To run with GPU, include --runtime=nvidia after installing nvidia-docker.
docker run --runtime=nvidia -v $PWD:/home/ReAgent -p 0.0.0.0:6006:6006 -it horizon:dev
If you have SELinux (Fedora, Redhat, etc.) you will have to start docker with the following command (notice the :Z at the end of path):
docker run -v $PWD:/home/ReAgent:Z -p 0.0.0.0:6006:6006 -it horizon:dev
To run with GPU, include --runtime=nvidia after installing nvidia-docker.
docker run --runtime=nvidia -v $PWD:/home/ReAgent:Z -p 0.0.0.0:6006:6006 -it horizon:dev
Depending on where your local ReAgent copy is, you may need to white list your shared path via Docker -> Preferences... -> File Sharing.
Once inside the container, run the setup file:
cd ReAgent ./scripts/setup.sh
Now you can run all the tests:
python setup.py test
or try running one specific test:
python setup.py test -s ml.rl.test.constant_reward.test_constant_reward.TestConstantReward.test_trainer_maxq