Add an advanced search option to "display results in a map"

DEVELOPERS.md

@@ -144,7 +144,7 @@ If a new search engine backend class is to be implemented, it must closely follo
 utils.search.SearchEngineBase docstrings. There is a Django management command that can be used in order to test
 the implementation of a search backend. You can run it like:
 
-    docker-compose run --rm web python manage.py test_search_engine_backend -fsw --backend utils.search.backends.solr9pysolr.Solr9PySolrSearchEngine
+    docker compose run --rm web python manage.py test_search_engine_backend -fsw --backend utils.search.backends.solr9pysolr.Solr9PySolrSearchEngine
 
 Please read carefully the documentation of the management command to better understand how it works and how is it
 doing the testing.
@@ -217,7 +217,7 @@ https://github.com/mtg/freesound-audio-analyzers. The docker compose of the main
 services for the external analyzers which depend on docker images having been previously built from the 
 `freesound-audio-analyzers` repository. To build these images you simply need to checkout the code repository and run 
 `make`. Once the images are built, Freesound can be run including the external analyzer services by of the docker compose 
-file by running `docker-compose --profile analyzers up`
+file by running `docker compose --profile analyzers up`
 
 The new analysis pipeline uses a job queue based on Celery/RabbitMQ. RabbitMQ console can be accessed at port `5673`
 (e.g. `http://localhost:5673/rabbitmq-admin`) and using `guest` as both username and password. Also, accessing 
@@ -231,7 +231,7 @@ for Freesound async tasks other than analysis).
 
 - Make sure that there are no outstanding deprecation warnings for the version of django that we are upgrading to.
 
-      docker-compose run --rm web python -Wd manage.py test
+      docker compose run --rm web python -Wd manage.py test
 
 Check for warnings of the form `RemovedInDjango110Warning` (TODO: Make tests fail if a warning occurs)
 

README.md

@@ -65,35 +65,35 @@ Below are instructions for setting up a local Freesound installation for develop
 
 8. Build all Docker containers. The first time you run this command can take a while as a number of Docker images need to be downloaded and things need to be installed and compiled. 
 
-       docker-compose build
+       docker compose build
 
 9. Download the [Freesound development database dump](https://drive.google.com/file/d/11z9s8GyYkVlmWdEsLSwUuz0AjZ8cEvGy/view?usp=share_link) (~6MB), uncompress it and place the resulting `freesound-small-dev-dump-2023-09.sql` in the `freesound-data/db_dev_dump/` directory. Then run the database container and load the data into it using the commands below. You should get permission to download this file from Freesound admins.
 
-       docker-compose up -d db
-       docker-compose run --rm db psql -h db -U freesound  -d freesound -f freesound-data/db_dev_dump/freesound-small-dev-dump-2023-09.sql
+       docker compose up -d db
+       docker compose run --rm db psql -h db -U freesound  -d freesound -f freesound-data/db_dev_dump/freesound-small-dev-dump-2023-09.sql
        # or if the above command does not work, try this one 
-       docker-compose run --rm --no-TTY db psql -h db -U freesound -d freesound < freesound-data/db_dev_dump/freesound-small-dev-dump-2023-09.sql
+       docker compose run --rm --no-TTY db psql -h db -U freesound -d freesound < freesound-data/db_dev_dump/freesound-small-dev-dump-2023-09.sql
 
 10. Update database by running Django migrations
 
-        docker-compose run --rm web python manage.py migrate
+        docker compose run --rm web python manage.py migrate
 
 11. Create a superuser account to be able to log in to the local Freesound website and to the admin site
 
-        docker-compose run --rm web python manage.py createsuperuser
+        docker compose run --rm web python manage.py createsuperuser
 
 12. Install static build dependencies
 
-        docker-compose run --rm web npm install --force
+        docker compose run --rm web npm install --force
 
 13. Build static files. Note that this step will need to be re-run every time there are changes in Freesound's static code (JS, CSS and static media files).
 
-        docker-compose run --rm web npm run build
-        docker-compose run --rm web python manage.py collectstatic --noinput
+        docker compose run --rm web npm run build
+        docker compose run --rm web python manage.py collectstatic --noinput
 
 14. Run services 🎉
 
-        docker-compose up
+        docker compose up
 
     When running this command, the most important services that make Freesound work will be run locally.
     This includes the web application and database, but also the search engine, cache manager, queue manager and asynchronous workers, including audio processing. 
@@ -102,24 +102,24 @@ Below are instructions for setting up a local Freesound installation for develop
 15. Build the search index, so you can search for sounds and forum posts
 
         # Open a new terminal window so the services started in the previous step keep running
-        docker-compose run --rm web python manage.py reindex_search_engine_sounds
-        docker-compose run --rm web python manage.py reindex_search_engine_forum
+        docker compose run --rm web python manage.py reindex_search_engine_sounds
+        docker compose run --rm web python manage.py reindex_search_engine_forum
 
     After following the steps, you'll have a functional Freesound installation up and running, with the most relevant services properly configured. 
     You can run Django's shell plus command like this:
 
-        docker-compose run --rm web python manage.py shell_plus
+        docker compose run --rm web python manage.py shell_plus
 
     Because the `web` container mounts a named volume for the home folder of the user running the shell plus process, command history should be kept between container runs :)
 
-16. (extra step) The steps above will get Freesound running, but to save resources in your local machine some non-essential services will not be started by default. If you look at the `docker-compose.yml` file, you'll see that some services are marked with the profile `analyzers` or `all`. These services include sound similarity, search results clustering and the audio analyzers. To run these services you need to explicitly tell `docker-compose` using the `--profile` (note that some services need additional configuration steps (see *Freesound analysis pipeline* section in `DEVELOPERS.md`):
+16. (extra step) The steps above will get Freesound running, but to save resources in your local machine some non-essential services will not be started by default. If you look at the `docker compose.yml` file, you'll see that some services are marked with the profile `analyzers` or `all`. These services include sound similarity, search results clustering and the audio analyzers. To run these services you need to explicitly tell `docker compose` using the `--profile` (note that some services need additional configuration steps (see *Freesound analysis pipeline* section in `DEVELOPERS.md`):
 
-        docker-compose --profile analyzers up   # To run all basic services + sound analyzers
-        docker-compose --profile all up         # To run all services
+        docker compose --profile analyzers up   # To run all basic services + sound analyzers
+        docker compose --profile all up         # To run all services
 
 
 ### Running tests
 
 You can run tests using the Django test runner in the `web` container like that:
 
-    docker-compose run --rm web python manage.py test --settings=freesound.test_settings
+    docker compose run --rm web python manage.py test --settings=freesound.test_settings

_docs/api/source/resources.rst

@@ -80,7 +80,7 @@ Filter name             Type           Description
 ``avg_rating``          numerical      Average rating for the sound in the range [0, 5].
 ``num_ratings``         integer        Number of times the sound has been rated.
 ``comment``             string         Textual content of the comments of a sound  (tokenized). The filter is satisfied if sound contains the filter value in at least one of its comments.
-``comments``            integer        Number of times the sound has been commented.
+``num_comments``            integer        Number of times the sound has been commented.
 ======================  =============  ====================================================
 
 

accounts/tests/test_views.py

@@ -287,14 +287,9 @@ def test_sound_search_response(self):
         resp = self.client.get(reverse('sounds-search'))
         self.assertEqual(resp.status_code, 200)
 
-    def test_geotags_box_response(self):
-        # 200 response on geotag box page access
-        resp = self.client.get(reverse('geotags-box'))
-        self.assertEqual(resp.status_code, 200)
-
-    def test_geotags_box_iframe_response(self):
+    def test_geotags_embed_response(self):
         # 200 response on geotag box iframe
-        resp = self.client.get(reverse('embed-geotags-box-iframe'))
+        resp = self.client.get(reverse('embed-geotags'))
         self.assertEqual(resp.status_code, 200)
 
     def test_accounts_manage_pages(self):

clustering/interface.py

@@ -109,5 +109,32 @@ def cluster_sound_results(request, features=DEFAULT_FEATURES):
         return {'finished': False, 'error': False}
 
 
+def get_ids_in_cluster(request, requested_cluster_id):
+    """Get the sound ids in the requested cluster. Used for applying a filter by id when using a cluster facet.
+    """
+    try:
+        requested_cluster_id = int(requested_cluster_id) - 1
+
+        # results are cached in clustering_utilities, available features are defined in the clustering settings file.
+        result = cluster_sound_results(request, features=DEFAULT_FEATURES)
+        results = result['result']
+
+        sounds_from_requested_cluster = results[int(requested_cluster_id)]
+
+    except ValueError:
+        return []
+    except IndexError:
+        return []
+    except KeyError:
+        # If the clustering is not in cache the 'result' key won't exist
+        # This means that the clustering computation will be triggered asynchronously.
+        # Moreover, the applied clustering filter will have no effect.
+        # Somehow, we should inform the user that the clustering results were not available yet, and that
+        # he should try again later to use a clustering facet.
+        return []
+
+    return sounds_from_requested_cluster
+
+
 def hash_cache_key(key):
     return create_hash(key, limit=32)

freesound.code-workspace

@@ -34,28 +34,23 @@
 	"tasks": {
 		"version": "2.0.0",
 		"tasks": [
-			{
-				"label": "Run web and search",
-				"type": "shell",
-				"command": "docker-compose up web search",
-				"problemMatcher": []
-			},
+
 			{
 				"label": "Docker compose build",
 				"type": "shell",
-				"command": "docker-compose build",
+				"command": "docker compose build",
 				"problemMatcher": []
 			},
 			{
 				"label": "Build static",
 				"type": "shell",
-				"command": "docker-compose run --rm web npm run build && docker-compose run --rm web python manage.py collectstatic --clear --noinput",
+				"command": "docker compose run --rm web npm run build && docker compose run --rm web python manage.py collectstatic --clear --noinput",
 				"problemMatcher": []
 			},
 			{
 				"label": "Install static",
 				"type": "shell",
-				"command": "docker-compose run --rm web npm install --force",
+				"command": "docker compose run --rm web npm install --force",
 				"problemMatcher": []
 			},
 			{
@@ -67,37 +62,55 @@
 			{
 				"label": "Create caches",
 				"type": "shell",
-				"command": "docker-compose run --rm web python manage.py create_front_page_caches && docker-compose run --rm web python manage.py create_random_sounds && docker-compose run --rm web python manage.py generate_geotags_bytearray",
+				"command": "docker compose run --rm web python manage.py create_front_page_caches && docker compose run --rm web python manage.py create_random_sounds && docker compose run --rm web python manage.py generate_geotags_bytearray",
 				"problemMatcher": []
 			},
 			{
 				"label": "Run tests",
 				"type": "shell",
-				"command": "docker-compose run --rm web python manage.py test --settings=freesound.test_settings",
+				"command": "docker compose run --rm web python manage.py test --settings=freesound.test_settings",
 				"problemMatcher": []
 			},
 			{
 				"label": "Run tests verbose with warnings",
 				"type": "shell",
-				"command": "docker-compose run --rm web python -Wa manage.py test -v3 --settings=freesound.test_settings",
+				"command": "docker compose run --rm web python -Wa manage.py test -v3 --settings=freesound.test_settings",
 				"problemMatcher": []
 			},
 			{
 				"label": "Migrate",
 				"type": "shell",
-				"command": "docker-compose run --rm web python manage.py migrate",
+				"command": "docker compose run --rm web python manage.py migrate",
 				"problemMatcher": []
 			},
 			{
 				"label": "Make migrations",
 				"type": "shell",
-				"command": "docker-compose run --rm web python manage.py makemigrations",
+				"command": "docker compose run --rm web python manage.py makemigrations",
 				"problemMatcher": []
 			},
 			{
 				"label": "Shell plus",
 				"type": "shell",
-				"command": "docker-compose run --rm web python manage.py shell_plus",
+				"command": "docker compose run --rm web python manage.py shell_plus",
+				"problemMatcher": []
+			},
+			{
+				"label": "Reindex search engine",
+				"type": "shell",
+				"command": "docker compose run --rm web python manage.py reindex_search_engine_sounds && docker compose run --rm web python manage.py reindex_search_engine_forum",
+				"problemMatcher": []
+			},
+			{
+				"label": "Post dirty sounds to search engine",
+				"type": "shell",
+				"command": "docker compose run --rm web python manage.py post_dirty_sounds_to_search_engine",
+				"problemMatcher": []
+			},
+			{
+				"label": "Orchestrate analysis",
+				"type": "shell",
+				"command": "docker compose run --rm web python manage.py orchestrate_analysis",
 				"problemMatcher": []
 			}
 		]

freesound/settings.py

@@ -638,6 +638,23 @@
 SOLR5_BASE_URL = "http://search:8983/solr"
 SOLR9_BASE_URL = "http://search:8983/solr"
 
+SEARCH_ENGINE_SIMILARITY_ANALYZERS = {
+    FSDSINET_ANALYZER_NAME: {
+        'vector_property_name': 'embeddings', 
+        'vector_size': 100,
+    },
+    FREESOUND_ESSENTIA_EXTRACTOR_NAME: {
+        'vector_property_name': 'sim_vector', 
+        'vector_size': 100,
+    }
+}
+SEARCH_ENGINE_DEFAULT_SIMILARITY_ANALYZER = FREESOUND_ESSENTIA_EXTRACTOR_NAME
+SEARCH_ENGINE_NUM_SIMILAR_SOUNDS_PER_QUERY = 500
+USE_SEARCH_ENGINE_SIMILARITY = False  # Does not currently apply to API
+
+SEARCH_ALLOW_DISPLAY_RESULTS_IN_MAP = True
+MAX_SEARCH_RESULTS_IN_MAP_DISPLAY = 10000  # This is the maximum number of sounds that will be shown when using "display results in map" mode
+
 # -------------------------------------------------------------------------------
 # Similarity client settings
 SIMILARITY_ADDRESS = 'similarity'

freesound/static/bw-frontend/src/components/mapsMapbox.js

@@ -292,7 +292,7 @@ function makeSoundsMap(geotags_url, map_element_id, on_built_callback, on_bounds
                     if (nSounds > 1){
                         // The padding and offset "manual" adjustments of bounds below are to make the boudns more similar to
                         // those created in the mapbox static maps
-                        map.fitBounds(bounds, {duration:0, offset:[-10, 0],  padding: {top:60, right:60, left:0, bottom:50}});
+                        map.fitBounds(bounds, {duration:0, offset:[0, 0],  padding: {top:60, right:60, left:60, bottom:60}});
                     } else {
                         map.setZoom(3);
                         if (nSounds > 0){