Experiment with Spark locally using a Jupyter Notebook, with Docker. The project uses Almond, which is based on ammonite-spark and uses Ivy for managing packages.
Use docker-compose, i.e. docker-compose up
It's recommended to use an .env
file that Docker will automatically parse and substitute tags on runtime. See an example here.