About me

I am a machine learning engineer working on the MAMEToolkit python library as a hobby. I am well versed in C/C++, Java, Python and Go. And I've developed deep learning agents with both TensorFlow and PyTorch.

MAMEToolkit

The MAMEToolkit is a Python 3 library used to train reinforcement learning algorithms against arcade games. It works as a wrapper around the MAME arcade game simulator, but allows your algorithms to interface with the game directly. I originally created the MAMEToolkit framework as part of my undergraduate thesis "Playing Street Fighter with Deep Reinforcement Learning". However, after I graduated I realised it could be a useful tool for many researchers, so I cleaned it up and released it on Github. Since then it has had a lot of attention, as it has helped many people learn and develop reinforcement learning algorithms. People love getting to go back and look at classic games in a whole new way.
I want to make this library more user friendly so more people can get started.

Future Plans

Although the library has been helpful for many, it still has room for improvement.
I have received lots of feedback from users about issues they've encountered, and have managed to resolve them as they arise. However, these improvements will not only reduce user problems, but will also improve the efficiency of the library. We will now go through all the objectives I hope to achieve through the encouragement of sponsors.

Additional Games

A lot of users have already developed there own game environments, and a lot of users have requested that additional games be added to the library. I hope to work with users to integrate their independent environments into the library, as well as take sponsors requests to develop new environments. As more games begin to be added, we should be able to simplify the implementation requirements for each new game.

Example Deep Reinforcement Learning Implementations/API

I get a lot of users asking how to develop their own deep reinforcement learning algorithms. However, the main purpose of this library is to help people learn how they can get started. In the future we may add a simple API for creating basic models using TensorFlow and PyTorch. I would like to create a set of example implementations with full documentation to explain to users how they can use TensorFlow or PyTorch to implement their own agents. I have already developed some example implementations, and I am learning and implementing new algorithms all the time for my PhD. I use MAMEToolkit as a test bed for all these implementations, so with a some extra work I could create an API and documentation to simplify the implementation for others.

Documentation Overhaul

At the moment the framework is fairly small so all documentation fits nicely on the README file. However, as more games and implementations get added to the toolkit, the library will become more complicated. We will need to have a proper documentation website outlining everything the library can do,
as well as documentation for the specific game and algorithm implementations. We may also get more developers working on the project, so we will need developer documentation.

C/C++ Implementation Optimization

The main reason the library is limited to Linux is because the library uses named pipes. The pipes transfer the data from mame to the toolkit. This system is somewhat inefficient and limits the compatibility of the framework. An alternative plan is to turn the python code into a C++ implementation that works more directly with mame to do the same thing. This new implementation will require a lot of work, but it won't require pipes, and it will be a lot faster. This will reduce the algorithm training times of all the users, and allow users to develop on more operating systems.

Compatibility Improvements

One of the main issues with this library is that is isn't very compatible. It only works on specific Linux machines with python 3. Eventually we would want this library to be available to every machine that can run MAME. This would mean the library could be used on Windows, Mac, and Linux machines, regardless of the libraries already installed. There are two ways in which we could achieve this. We could either setup a docker image with the minimum libraries required to make the library work. However, that would require the user also install docker on their computer. The other option would be to cross-compile many implementations of the bundled mame executable, and then also do the C/C++ implementation optimization