This task aims to develop robust and effective trading agents for cryptocurrencies using ensemble methods. Participants are expected to explore innovative ensemble methods for single cryptocurrency trading. They are also encouraged to take advantage of the power of massively parallel simulations by utilizing the provided vectorized environments.
The code in this repo contains a comprehensive starter kit to get started with training single agents in massively parallel environments.
There is an example of ensemble method to use the majority voting approach in the tutorial, Task 1 Crypto Trading Ensemble. Later, we will provide submission format and example, which will use this tutorial.
A dataset containing second-level Limit Order Book (LOB) data for Bitcoin is provided. Please download here. All of the datasets required to train DRL agents are in the data directory, please download this into the Task_1 starter dir
The dataset contains the following files:
- BTC_1sec.csv : 1 second level LOB BTC data
- BTC_1sec_predict.npy : strong RNN factors generated on the csv dataset to train the DRL agent
We generate the strong RNN features by first training a RNN model on 60% of the dataset. The RNN is trained to predict future price movements using 101 formulaic alphas [1]. The strong factors generated by RNN are used as technical factors to train the FinRL agent. The FinRL agent will be trained on the remaining dataset, and then evaluated on a separate test set. The RNN we use combines LSTM and GRU networks for better data modeling. It uses a MLP input layer to process the alpha101 features followed by two RNN networks to model the data.
The BTC_1sec.csv contains all data used to train the RNN model and FinRL agent. Notice that the timestamps in this dataset has been processed and are not the true timestamps.
This starter kit demonstrates how to use the provided code. We provide you with RNN generated strong factors to use for the DRL agent. You are welcome to experiment with various ensemble configurations that yield optimal results.
The starter kit includes:
-
trade_simulator.py: Contains a market replay simulator for single commodity.- Class
TradeSimulator: A training-focused market replay simulator, complying with the older gym-style API requirements. - Class
EvalTradeSimulator: A market replay simulator for evaluation.
- Class
-
erl_config.py: Configuration file for reinforcement learning training. -
erl_replay_buffer.py: Serves as a training dataset for the reinforcement learning market replay simulator. -
erl_agent.py: Contains the DQN class algorithm for reinforcement learning. -
erl_net.py: Neural network structures used in the reinforcement learning algorithm. -
erl_run.py: Loads the simulator and trains the reinforcement learning agent. -
erl_evaluator.py: Evaluates the performance of the reinforcement learning agent. -
metrics.py: Contains some metrics for evaluation.
We will provide the evaluation code soon.
Please submit all your models and the scripts to load and test the models.
Please provide a readme that describes your submission and explains important things to note when running it so that we can ensure we run your submission as it was intended.
├── finrl-contest-task-2
│ ├── trained_models # Your trained component agent weights.
│ ├── task1_ensemble.py # File for implementing the ensemble method
│ ├── task1_test.py # File to load your model and test the ensemble method
│ ├── trade_simulator.py # File for the environment. Please submit it if you modified the provided env.
│ ├── READE.md # File to explain the your code
│ ├── requirements.txt # Have it if adding any new packages
│ ├── And any additional scripts you create
We will provide the template for task1_ensemble.py and task1_test.py based on the tutorial example solution soon.
For evaluatiuon, we will run your ensemble agents on a test set and compare the results using metrics like cumulative return, win loss rate and sharpe ratio (subject to change).
[1] Zura Kakushadze. 2016. 101 formulaic alphas. arXiv preprint arXiv:1601.00991