Refactor load_mongodump.sh script (#150)

* Refactor load_mongodump.sh script to support better error handling, debugging information, and integration with Docker Compose configuration file for database host configuration.

* Update the docker-compose.dev.yml's environment DB_HOST to mongo://db/DB_NAME. Add docker-compose.dev.yml into gitignore. Update README.md file with information about not to make changes into docker-compose.dev.yml and update about forced push if changes are made to docker-compose.dev.yml.

* Update docker-compose-dev.yml to docker-compose.dev.yml.

* Remove stupid prefix on dashboard's title. Remove extra bullet point.

* Update DB_HOST to mongodb from mongo

* Include both docker-compose.dev.yml and docker-compose.yml in git ignore.

* Resolve merge conflict in .gitignore with addition of .DS_Store

* Remove comment for docker compose yml in gitignore

* Remove docker-compose.yml from .gitignore

* Revert to DB_HOST=db for docker-compose.dev.yml.

* Remove the check for correct number of args in load_mongodump.sh

* Update path to config file commentto two levels up instead of one level up in load_mongodump.sh

* Re-introduce check for correct number of args

* Update the sed command from DB_HOST:xx to DB_HOST=xx

.gitignore

@@ -133,4 +133,6 @@ dmypy.json
 # Pyre type checker
 .pyre/
 
-.DS_Store
+docker-compose.dev.yml
+
+.DS_Store

README.md

@@ -1,4 +1,4 @@
-# A simple and stupid dashboard for e-mission
+# A simple dashboard for e-mission
 
 Issues: Since this repository is part of a larger project, all issues are tracked in the central docs repository. If you have a question, as suggested by the open source guide, please file an issue instead of sending an email. Since issues are public, other contributors can try to answer the question and benefit from the answer.
 
@@ -70,6 +70,24 @@ Note that this expects a standard setup with:
 - this repository checked out under the `em-public-dashboard` directory, which makes the database name `em-public-dashboard_db_1`
 - the incoming mongodump is in tar gz format. This should be true of all canbikeco dumps, you may need to change the `tar xvf` to `unzip` otherwise.  The mongo container typically doesn't have zip installed, so using tar is more portable.
 
+## Working with `docker compose` and `.gitignore`
+
+### Using `docker compose`
+
+When working with `docker compose`, it's generally recommended to avoid committing changes to the `docker-compose.dev.yml` file, especially if you're running the `./load_mongodump <dump tar>` script. This file is typically configured to work in a specific way for your development environment, and changes might not be applicable or useful for others working on the same project.
+
+### `.gitignore` Configuration
+
+To streamline your workflow, we have added the `docker-compose.dev.yml` file to the `.gitignore` file. This means that by default, changes to `docker-compose.dev.yml` will not be tracked by Git. This setup helps to avoid unnecessary commits and ensures that your `docker-compose.dev.yml` remains consistent with the intended configuration for the project.
+
+### Committing Changes to `docker-compose.dev.yml`
+
+If you do need to make changes to `docker-compose.dev.yml` and want to commit those changes, you can override the ignore settings by using the following Git command:
+
+```bash
+git add -f docker-compose.dev.yml
+```
+
 **If you have a non-standard setup, please use your expertise to change the script appropriately.**
 
 #### Happy visualizations!
@@ -121,7 +139,6 @@ You may need to increase the resources avaliable to Docker if:
 - you believe you've loaded the data but there is none when running the notebooks
 - the notebook can't connect to the database
 - when you try and start the container for the database it exits with code 14
-- 
 
 ## Large Dataset Workaround
 

viz_scripts/docker/load_mongodump.sh

@@ -1,9 +1,70 @@
+#!/bin/bash
+
+# Directory of the script
+SCRIPT_DIR="$(dirname "$0")"
+
+# Path to the configuration file (two levels up)
+CONFIG_FILE="$SCRIPT_DIR/../../docker-compose.dev.yml"
+
+# Check if the correct number of arguments is provided
+if [ "$#" -ne 1 ]; then
+    echo "Usage: $0 <mongodump-file>"
+    echo "  <mongodump-file> : The path to the MongoDB dump file to be restored."
+    exit 1
+fi
+
 MONGODUMP_FILE=$1
 
-echo "Copying file to docker container"
-docker cp $MONGODUMP_FILE em-public-dashboard-db-1:/tmp
+# Print debug information
+echo "Script Directory: $SCRIPT_DIR"
+echo "Configuration File Path: $CONFIG_FILE"
+echo "MongoDump File Path: $MONGODUMP_FILE"
+
+# Check if the provided file exists
+if [ ! -f "$MONGODUMP_FILE" ]; then
+    echo "Error: File '$MONGODUMP_FILE' does not exist."
+    exit 1
+fi
+
+# Check if the configuration file exists
+if [ ! -f "$CONFIG_FILE" ]; then
+    echo "Error: Configuration file '$CONFIG_FILE' does not exist."
+    exit 1
+fi
+
+# Print details about the configuration file
+echo "Configuration file details:"
+ls -l "$CONFIG_FILE"
+
+# Extract the database name from the mongodump file
+DB_NAME=$(tar -tf "$MONGODUMP_FILE" | grep '^dump/' | sed 's|^dump/||' | awk -F'/' '{if (NF > 0) {print $1; exit}}')
+
+# Output the database name
+echo "$DB_NAME"
+
+if [ -z "$DB_NAME" ]; then
+    echo "Error: Failed to extract database name from mongodump."
+    exit 1
+fi
+
+echo "Database Name: $DB_NAME"
+
+# Update the docker-compose configuration file with the actual DB_HOST
+DB_HOST="mongodb://db/$DB_NAME"
+sed -i.bak "s|DB_HOST=.*|DB_HOST=$DB_HOST|" "$CONFIG_FILE"
+
+echo "Updated docker-compose file:"
+cat "$CONFIG_FILE"
+
+echo "Copying file to Docker container"
+docker cp "$MONGODUMP_FILE" em-public-dashboard-db-1:/tmp
+
+FILE_NAME=$(basename "$MONGODUMP_FILE")
+
+echo "Clearing existing database"
+docker exec em-public-dashboard-db-1 bash -c "mongo $DB_NAME --eval 'db.dropDatabase()'"
 
-FILE_NAME=`basename $MONGODUMP_FILE`
+echo "Restoring the dump from $FILE_NAME to database $DB_NAME"
+docker exec -e MONGODUMP_FILE=$FILE_NAME em-public-dashboard-db-1 bash -c "cd /tmp && tar xvf $FILE_NAME && mongorestore -d $DB_NAME dump/$DB_NAME"
 
-echo "Restoring the dump from $FILE_NAME"
-docker exec -e MONGODUMP_FILE=$FILE_NAME em-public-dashboard-db-1 bash -c 'cd /tmp && tar xvf $MONGODUMP_FILE && mongorestore'
+echo "Database restore complete."