AlbumSpotter

These scripts can be used to train a machine learning model to identify music albums by cover art photos.

• collect information of saved albums from Spotify
• download cover art of the albums
• transforming and augmenting images for training the machine learning model
• inferencing with the trained model and provide the album info

Technical details

albums.py

This script uses the Spotipy library to collect information of your saved albums on Spotify. Follow the instructions and go to Spotify Dashboard to setup the credentials.

The script uses the authorization code flow so you will need to setup the SPOTIPY_REDIRECT_URI, SPOTIPY_CLIENT_ID, and SPOTIPY_CLIENT_SECRET environment variables when running the script.

By default album information will be cached in the file my_albums.json. The config.py has parameters to control the file name and which fields to keep.

images.py

This script selects and downloads cover art image of albums. Images are saved in the folder images/cover_arts/ by default and can be configured in config.py.

The script also saves a copy of each album art cover into the training folder in preparation of the transformation and augmentation.

To limit what albums to be processed (e.g. testing out the workflow with a smaller sample of albums), edit LIMIT_IMAGES_FILE in config.py and create a json file to list out those album IDs.

transform.py

This script creates additional images for each album art for training the model. Some transformations used are rotation, blurring, and color shifting etc. It is suggested to experiment and tweak both the transformation used and the parameters to better train the model.

training.py

This script uses the prepared images to train a model based on ResNet50. The number of epoch, batch size, and learning rate should be tuned based on the environment.

Trained models are saved with a timestamp in the file name. Together there is also a "mapping" file to lookup the album ID by the model output class index.

inference.py

This script captures an image from webcam and guesses what that album is. Modify the MODEL_TIMESTAMP constant to choose which trained model to use.

The mapping file and cached album data is also loaded so that the script can provide information (e.g. URL to play the album on Spotify) about the identified album.

testing.py

This script runs all models from the models directory against manually prepared testing data. By default testing data is in the images/testing folder and configurable in config.py. The sub-folder names are the album IDs and testing images can be prepared in those sub-folders.