From 099b778676a86cee90b1fd9b813f78760d463fa4 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 15:10:52 +0100 Subject: [PATCH 01/35] chore: upgrade tooling --- DOCKER_README.md | 46 +++------------------ Dockerfile | 4 +- MAINTAINERS.md | 2 +- README.md | 105 ++++++++++++++++++----------------------------- requirements.txt | 4 +- setup.py | 14 ++++--- tox.ini | 34 ++++----------- 7 files changed, 67 insertions(+), 142 deletions(-) diff --git a/DOCKER_README.md b/DOCKER_README.md index 27c94e5..8d1b0aa 100644 --- a/DOCKER_README.md +++ b/DOCKER_README.md @@ -1,41 +1,7 @@ -In this page you will find our recommended way of installing Docker on your machine. -This guide is made for OSX users. - -## Install Docker - -First install Docker using [Homebrew](https://brew.sh/) -``` -$ brew install docker -``` - -You can then install [Docker Desktop](https://docs.docker.com/get-docker/) if you wish, or use `docker-machine`. As we prefer the second option, we will only document this one. - -## Setup your Docker - -Install `docker-machine` -``` -$ brew install docker-machine -``` - -Then install [VirtualBox](https://www.virtualbox.org/) with [Homebrew Cask](https://github.com/Homebrew/homebrew-cask) to get a driver for your Docker machine -``` -$ brew cask install virtualbox -``` - -You may need to enter your password and authorize the application in your `System Settings` > `Security & Privacy`. - -Create now a new machine, set it up as default and connect your shell to it (here we use zsh. The commands should anyway be displayed in each steps' output) - -``` -$ docker-machine create --driver virtualbox default -$ docker-machine env default -$ eval "$(docker-machine env default)" -``` - -Now you're all setup to use our provided Docker image! - ## Build the image +> Make sure to have [Docker installed](https://docs.docker.com/engine/install/) + ```bash docker build -t algoliasearch-django . ``` @@ -53,8 +19,8 @@ docker run -it --rm --env ALGOLIA_APPLICATION_ID=XXXXXX \ However, we advise you to export them. That way, you can use [Docker's shorten syntax](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file) to set your variables. ```bash -export ALGOLIA_APPLICATION_ID=XXXXXX -export ALGOLIA_API_KEY=XXX +export ALGOLIA_APPLICATION_ID=XXXXXX +export ALGOLIA_API_KEY=XXX docker run -it --rm --env ALGOLIA_APPLICATION_ID --env ALGOLIA_API_KEY -v $PWD:/code -w /code algoliasearch-django bash ``` @@ -64,8 +30,8 @@ Once your container is running, any changes you make in your IDE are directly re To launch the tests, you can use this command ```bash -tox -e py36-django31 +tox -e py313-django51 ``` -If you'd like to sue an env other that `py36-django31`, run `tox --listenvs` to see the list of available envs. +If you'd like to sue an env other that `py313-django51`, run `tox --listenvs` to see the list of available envs. Feel free to contact us if you have any questions. diff --git a/Dockerfile b/Dockerfile index d9a39ee..97b1d66 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM python:3.8-slim +FROM python:3.13-slim # Force the stdout and stderr streams to be unbuffered. # Ensure python output goes to your terminal @@ -7,6 +7,6 @@ ENV PYTHONUNBUFFERED=1 WORKDIR /code COPY requirements.txt /code/ -RUN pip install -r requirements.txt +RUN pip3 install --upgrade pip && pip3 install -r requirements.txt COPY . /code/ diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 33c7143..0091440 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -2,4 +2,4 @@ | Name | Email | |-----------------|---------------------| -| Paul-Louis Nech | support@algolia.com | +| Algolia | support@algolia.com | diff --git a/README.md b/README.md index abcc4e6..94a3625 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,6 @@

-

Documentation • Community Forum • @@ -26,55 +25,56 @@ You can find the full reference on [Algolia's website](https://www.algolia.com/doc/framework-integration/django/). - - 1. **[Setup](#setup)** - * [Introduction](#introduction) - * [Install](#install) - * [Setup](#setup) - * [Quick Start](#quick-start) + + - [Introduction](#introduction) + - [Install](#install) + - [Setup](#setup) + - [Quick Start](#quick-start) 1. **[Commands](#commands)** - * [Commands](#commands) + + - [Commands](#commands) 1. **[Search](#search)** - * [Search](#search) + + - [Search](#search) 1. **[Geo-Search](#geo-search)** - * [Geo-Search](#geo-search) + + - [Geo-Search](#geo-search) 1. **[Tags](#tags)** - * [Tags](#tags) + + - [Tags](#tags) 1. **[Options](#options)** - * [Custom objectID](#custom-codeobjectidcode) - * [Custom index name](#custom-index-name) - * [Field Preprocessing and Related objects](#field-preprocessing-and-related-objects) - * [Index settings](#index-settings) - * [Restrict indexing to a subset of your data](#restrict-indexing-to-a-subset-of-your-data) - * [Multiple indices per model](#multiple-indices-per-model) - * [Temporarily disable the auto-indexing](#temporarily-disable-the-auto-indexing) -1. **[Tests](#tests)** - * [Run Tests](#run-tests) + - [Custom objectID](#custom-codeobjectidcode) + - [Custom index name](#custom-index-name) + - [Field Preprocessing and Related objects](#field-preprocessing-and-related-objects) + - [Index settings](#index-settings) + - [Restrict indexing to a subset of your data](#restrict-indexing-to-a-subset-of-your-data) + - [Multiple indices per model](#multiple-indices-per-model) + - [Temporarily disable the auto-indexing](#temporarily-disable-the-auto-indexing) -1. **[Troubleshooting](#troubleshooting)** - * [Frequently asked questions](#frequently-asked-questions) +1. **[Tests](#tests)** + - [Run Tests](#run-tests) +1. **[Troubleshooting](#troubleshooting)** + - [Frequently asked questions](#frequently-asked-questions) # Setup - - ## Introduction This package lets you easily integrate the Algolia Search API to your [Django](https://www.djangoproject.com/) project. It's based on the [algoliasearch-client-python](https://github.com/algolia/algoliasearch-client-python) package. You might be interested in this sample Django application providing a typeahead.js based auto-completion and Google-like instant search: [algoliasearch-django-example](https://github.com/algolia/algoliasearch-django-example). -- Compatible with **Python 2.7** and **Python 3.4+**. -- Supports **Django 1.7+**, **2.x** and **3.x**. +- Compatible with **Python 3.8+**. +- Supports **Django 4.x** and **5.x**. ## Install @@ -95,10 +95,10 @@ ALGOLIA = { There are several optional settings: -* `INDEX_PREFIX`: prefix all indices. Use it to separate different applications, like `site1_Products` and `site2_Products`. -* `INDEX_SUFFIX`: suffix all indices. Use it to differentiate development and production environments, like `Location_dev` and `Location_prod`. -* `AUTO_INDEXING`: automatically synchronize the models with Algolia (default to **True**). -* `RAISE_EXCEPTIONS`: raise exceptions on network errors instead of logging them (default to **settings.DEBUG**). +- `INDEX_PREFIX`: prefix all indices. Use it to separate different applications, like `site1_Products` and `site2_Products`. +- `INDEX_SUFFIX`: suffix all indices. Use it to differentiate development and production environments, like `Location_dev` and `Location_prod`. +- `AUTO_INDEXING`: automatically synchronize the models with Algolia (default to **True**). +- `RAISE_EXCEPTIONS`: raise exceptions on network errors instead of logging them (default to **settings.DEBUG**). ## Quick Start @@ -134,25 +134,17 @@ class YourModelIndex(AlgoliaIndex): ``` - - # Commands - - ## Commands -* `python manage.py algolia_reindex`: reindex all the registered models. This command will first send all the record to a temporary index and then moves it. - * you can pass ``--model`` parameter to reindex a given model -* `python manage.py algolia_applysettings`: (re)apply the index settings. -* `python manage.py algolia_clearindex`: clear the index - - +- `python manage.py algolia_reindex`: reindex all the registered models. This command will first send all the record to a temporary index and then moves it. + - you can pass `--model` parameter to reindex a given model +- `python manage.py algolia_applysettings`: (re)apply the index settings. +- `python manage.py algolia_clearindex`: clear the index # Search - - ## Search We recommend using our [InstantSearch.js library](https://www.algolia.com/doc/guides/building-search-ui/what-is-instantsearch/js/) to build your search @@ -169,12 +161,8 @@ params = { "hitsPerPage": 5 } response = raw_search(Contact, "jim", params) ``` - - # Geo-Search - - ## Geo-Search Use the `geo_field` attribute to localize your record. `geo_field` should be a callable that returns a tuple (latitude, longitude). @@ -195,12 +183,8 @@ class ContactIndex(AlgoliaIndex): algoliasearch.register(Contact, ContactIndex) ``` - - # Tags - - ## Tags Use the `tags` attributes to add tags to your record. It can be a field or a callable. @@ -212,16 +196,12 @@ class ArticleIndex(AlgoliaIndex): At query time, specify `{ tagFilters: 'tagvalue' }` or `{ tagFilters: ['tagvalue1', 'tagvalue2'] }` as search parameters to restrict the result set to specific tags. - - # Options - - ## Custom `objectID` You can choose which field will be used as the `objectID `. The field should be unique and can - be a string or integer. By default, we use the `pk` field of the model. +be a string or integer. By default, we use the `pk` field of the model. ```python class ArticleIndex(AlgoliaIndex): @@ -279,8 +259,8 @@ class ContactIndex(AlgoliaIndex): - With this configuration, you can search for a `Contact` using its `Account` names - You can use the associated `account_ids` at search-time to fetch more data from your -model (you should **only proxy the fields relevant for search** to keep your records' size -as small as possible) + model (you should **only proxy the fields relevant for search** to keep your records' size + as small as possible) ## Index settings @@ -400,12 +380,8 @@ with disable_auto_indexing(MyModel): ``` - - # Tests - - ## Run Tests To run the tests, first find your Algolia application id and Admin API key (found on the Credentials page). @@ -414,8 +390,8 @@ To run the tests, first find your Algolia application id and Admin API key (foun ALGOLIA_APPLICATION_ID={APPLICATION_ID} ALGOLIA_API_KEY={ADMIN_API_KEY} tox ``` - To override settings for some tests, use the [settings method](https://docs.djangoproject.com/en/1.11/topics/testing/tools/#django.test.SimpleTestCase.settings): + ```python class OverrideSettingsTestCase(TestCase): def setUp(self): @@ -433,15 +409,12 @@ class OverrideSettingsTestCase(TestCase): # ... ``` - - # Troubleshooting # Use the Dockerfile + If you want to contribute to this project without installing all its dependencies, you can use our Docker image. Please check our [dedicated guide](DOCKER_README.md) to learn more. ## Frequently asked questions Encountering an issue? Before reaching out to support, we recommend heading to our [FAQ](https://www.algolia.com/doc/framework-integration/django/faq/) where you will find answers for the most common issues and gotchas with the package. - - diff --git a/requirements.txt b/requirements.txt index c8cb9b4..cf996c8 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ -django>=1.7 -algoliasearch>=3.0,<4.0 +django>=4.0 +algoliasearch>=4.0,<5.0 # dev dependencies pypandoc wheel diff --git a/setup.py b/setup.py index 55a7bda..051d80b 100644 --- a/setup.py +++ b/setup.py @@ -20,10 +20,10 @@ path_version = os.path.join(os.path.dirname(__file__), 'algoliasearch_django/version.py') -if sys.version_info[0] == 3: - exec(open(path_version).read()) +if sys.version_info < (3, 8): + raise RuntimeError("algoliasearch_django 4.x requires Python 3.8+") else: - execfile(path_version) + exec(open(path_version).read()) setup( @@ -31,7 +31,7 @@ version=VERSION, license='MIT License', packages=find_packages(exclude=['tests']), - install_requires=['django>=1.7', 'algoliasearch>=3.0,<4.0'], + install_requires=['django>=4.0', 'algoliasearch>=4.0,<5.0'], description='Algolia Search integration for Django', long_description=README, long_description_content_type='text/markdown', @@ -47,8 +47,12 @@ 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python', - 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.8', + 'Programming Language :: Python :: 3.9', + 'Programming Language :: Python :: 3.10', + 'Programming Language :: Python :: 3.11', + 'Programming Language :: Python :: 3.12', 'Topic :: Internet :: WWW/HTTP', ] ) diff --git a/tox.ini b/tox.ini index 939d0c3..cc5123d 100644 --- a/tox.ini +++ b/tox.ini @@ -1,19 +1,10 @@ [tox] envlist = - {py34}-django17 - {py34,py35,py36}-django18 - {py34,py35,py36}-django19 - {py34,py35,py36}-django110 - {py34,py35,py36}-django111 - {py34,py35,py36}-django20 - {py34,py35,py36}-django21 - {py34,py35,py36}-django22LTS - {py36,py37,py38,py39}-django30 - {py36,py37,py38,py39}-django31 - {py36,py37,py38,py39,py310}-django32 {py38,py39,py310}-django40 {py38,py39,py310,py311}-django41 - {py38,py39,py310,py311}-django42 + {py38,py39,py310,py311,py312}-django42 + {py310,py311,py312}-django50 + {py310,py311,py312,py313}-django51 coverage skip_missing_interpreters = True @@ -22,28 +13,19 @@ deps = six mock factory_boy - py{34,311}: Faker>=1.0,<2.0 - django17: Django>=1.7,<1.8 - django18: Django>=1.8,<1.9 - django19: Django>=1.9,<1.10 - django110: Django>=1.10,<1.11 - django111: Django>=1.11,<2.0 - django20: Django>=2.0,<2.1 - django21: Django>=2.1,<2.2 - django22LTS: Django>=2.2,<3.0 - django30: Django>=3.0,<3.1 - django31: Django>=3.1,<3.2 - django32: Django>=3.2,<3.3 + py{38,313}: Faker>=5.0,<6.0 django40: Django>=4.0,<4.1 django41: Django>=4.1,<4.2 django42: Django>=4.2,<4.3 + django50: Django>=5.0,<5.1 + django51: Django>=5.1,<5.2 passenv = ALGOLIA* commands = python runtests.py [versions] -twine = >=1.13,<2.0 -wheel = >=0.34,<1.0 +twine = >=5.1,<6.0 +wheel = >=0.45,<1.0 [testenv:coverage] basepython = python3.8 From 6b398ad705fcef3f858e41d968ad95655b18c2f0 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 15:15:43 +0100 Subject: [PATCH 02/35] chore: update ci --- .github/workflows/main.yml | 97 ++++++++++++++++++++++---------------- 1 file changed, 57 insertions(+), 40 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 70632b3..32462aa 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -17,51 +17,68 @@ jobs: strategy: matrix: include: - - version: "3.5.4" - toxenv: py35-django20 - os: ubuntu-20.04 - - version: "3.6.7" - toxenv: py36-django32 - os: ubuntu-20.04 - - version: "3.7.5" - toxenv: py37-django32 - os: ubuntu-20.04 - - version: "3.8.15" - toxenv: py38-django32 - os: ubuntu-20.04 - - version: "3.9" - toxenv: py39-django30 - os: ubuntu-latest - - version: "3.9" - toxenv: py39-django31 - os: ubuntu-latest - - version: "3.9" - toxenv: py39-django32 - os: ubuntu-latest - - version: "3.9" + # django 4.0 + - version: "3.8.20" + toxenv: py38-django40 + os: ubuntu-20.04 + - version: "3.9.20" toxenv: py39-django40 - os: ubuntu-latest - - version: "3.9" - toxenv: py39-django41 - os: ubuntu-latest - - version: "3.9" - toxenv: py39-django42 - os: ubuntu-latest - - version: "3.10" + os: ubuntu-20.04 + - version: "3.10.15" toxenv: py310-django40 - os: ubuntu-latest - - version: "3.10" + os: ubuntu-20.04 + # django 4.1 + - version: "3.8.20" + toxenv: py38-django41 + os: ubuntu-20.04 + - version: "3.9.20" + toxenv: py39-django41 + os: ubuntu-20.04 + - version: "3.10.15" toxenv: py310-django41 - os: ubuntu-latest - - version: "3.10" - toxenv: py310-django42 - os: ubuntu-latest - - version: "3.11" + os: ubuntu-20.04 + - version: "3.11.10" toxenv: py311-django41 - os: ubuntu-latest - - version: "3.11" + os: ubuntu-20.04 + # django 4.2 + - version: "3.8.20" + toxenv: py38-django42 + os: ubuntu-20.04 + - version: "3.9.20" + toxenv: py39-django42 + os: ubuntu-20.04 + - version: "3.10.15" + toxenv: py310-django42 + os: ubuntu-20.04 + - version: "3.11.10" toxenv: py311-django42 - os: ubuntu-latest + os: ubuntu-20.04 + - version: "3.12.7" + toxenv: py312-django42 + os: ubuntu-20.04 + # django 5.0 + - version: "3.10.15" + toxenv: py310-django50 + os: ubuntu-20.04 + - version: "3.11.10" + toxenv: py311-django50 + os: ubuntu-20.04 + - version: "3.12.7" + toxenv: py312-django50 + os: ubuntu-20.04 + # django 5.1 + - version: "3.10.15" + toxenv: py310-django51 + os: ubuntu-20.04 + - version: "3.11.10" + toxenv: py311-django51 + os: ubuntu-20.04 + - version: "3.12.7" + toxenv: py312-django51 + os: ubuntu-20.04 + - version: "3.13.0" + toxenv: py313-django51 + os: ubuntu-20.04 steps: - uses: actions/checkout@v3 From 2587b37302f950f52b61b63d9b6da972fdd4c997 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 15:18:59 +0100 Subject: [PATCH 03/35] chore: update pr and issue template --- .github/ISSUE_TEMPLATE.md | 89 +++++++++++++++++++++++++------- .github/PULL_REQUEST_TEMPLATE.md | 17 +++--- 2 files changed, 76 insertions(+), 30 deletions(-) diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 8869d73..da2ba62 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,22 +1,73 @@ - + If you are not sure about the origin of the issue, or if it impacts your customer experience, please contact [our support team](https://alg.li/support). + - type: textarea + attributes: + label: Description + description: A clear and concise description of what the bug is. + validations: + required: true + - type: dropdown + id: python version + attributes: + label: python version + description: What is the Python version you've reproduced the error with + options: + - 3.8 + - 3.9 + - 3.10 + - 3.11 + - 3.12 + - 3.13 + validations: + required: true + - type: dropdown + id: django version + attributes: + label: Django version + description: What is the Django version you've reproduced the error with + options: + - 4.0 + - 4.1 + - 4.2 + - 5.0 + - 5.1 + validations: + required: true + - type: textarea + attributes: + label: Steps to reproduce + description: Write down the steps to reproduce the bug, please include any information that seems relevant for us to reproduce it properly + placeholder: | + 1. Use method `...` + 2. With parameters `...` + 3. See error + validations: + required: true + - type: textarea + id: logs + attributes: + label: Relevant log output + description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. + render: shell + - type: checkboxes + attributes: + label: Self-service + description: | + If you feel like you could contribute to this issue, please check the box below. This would tell us and other people looking for contributions that someone's working on it. + If you do check this box, please send a pull request within 7 days so we can still delegate this to someone else. + options: + - label: I'd be willing to fix this bug myself. -- Django version: -- Algolia Django integration version: -- Algolia Client Version: #.#.# -- Language Version: - -### Description - - -### Steps To Reproduce diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 24da431..3e0f401 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,22 +1,17 @@ -| Q | A -| ----------------- | ---------- -| Bug fix? | yes/no -| New feature? | yes/no -| BC breaks? | no -| Related Issue | Fix #... -| Need Doc update | yes/no +## 🧭 What and Why +🎟 JIRA Ticket: -## Describe your change +### Changes included: -## What problem is this fixing? +## 🧪 Test From 9d9c8761141013a6138a03019c8b6cebb589f9b8 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 15:24:02 +0100 Subject: [PATCH 04/35] chore: minor python version --- .github/workflows/main.yml | 76 +++++++++++++++++++------------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 32462aa..0ecf25b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -18,67 +18,67 @@ jobs: matrix: include: # django 4.0 - - version: "3.8.20" + - version: "3.8" toxenv: py38-django40 - os: ubuntu-20.04 - - version: "3.9.20" + os: ubuntu-22.04 + - version: "3.9" toxenv: py39-django40 - os: ubuntu-20.04 - - version: "3.10.15" + os: ubuntu-22.04 + - version: "3.10" toxenv: py310-django40 - os: ubuntu-20.04 + os: ubuntu-22.04 # django 4.1 - - version: "3.8.20" + - version: "3.8" toxenv: py38-django41 - os: ubuntu-20.04 - - version: "3.9.20" + os: ubuntu-22.04 + - version: "3.9" toxenv: py39-django41 - os: ubuntu-20.04 - - version: "3.10.15" + os: ubuntu-22.04 + - version: "3.10" toxenv: py310-django41 - os: ubuntu-20.04 - - version: "3.11.10" + os: ubuntu-22.04 + - version: "3.11" toxenv: py311-django41 - os: ubuntu-20.04 + os: ubuntu-22.04 # django 4.2 - - version: "3.8.20" + - version: "3.8" toxenv: py38-django42 - os: ubuntu-20.04 - - version: "3.9.20" + os: ubuntu-22.04 + - version: "3.9" toxenv: py39-django42 - os: ubuntu-20.04 - - version: "3.10.15" + os: ubuntu-22.04 + - version: "3.10" toxenv: py310-django42 - os: ubuntu-20.04 - - version: "3.11.10" + os: ubuntu-22.04 + - version: "3.11" toxenv: py311-django42 - os: ubuntu-20.04 - - version: "3.12.7" + os: ubuntu-22.04 + - version: "3.12" toxenv: py312-django42 - os: ubuntu-20.04 + os: ubuntu-22.04 # django 5.0 - - version: "3.10.15" + - version: "3.10" toxenv: py310-django50 - os: ubuntu-20.04 - - version: "3.11.10" + os: ubuntu-22.04 + - version: "3.11" toxenv: py311-django50 - os: ubuntu-20.04 - - version: "3.12.7" + os: ubuntu-22.04 + - version: "3.12" toxenv: py312-django50 - os: ubuntu-20.04 + os: ubuntu-22.04 # django 5.1 - - version: "3.10.15" + - version: "3.10" toxenv: py310-django51 - os: ubuntu-20.04 - - version: "3.11.10" + os: ubuntu-22.04 + - version: "3.11" toxenv: py311-django51 - os: ubuntu-20.04 - - version: "3.12.7" + os: ubuntu-22.04 + - version: "3.12" toxenv: py312-django51 - os: ubuntu-20.04 - - version: "3.13.0" + os: ubuntu-22.04 + - version: "3.13" toxenv: py313-django51 - os: ubuntu-20.04 + os: ubuntu-22.04 steps: - uses: actions/checkout@v3 From 2f9aa46b5a474caa07a6090f4b50bc0ff9410390 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 15:46:45 +0100 Subject: [PATCH 05/35] chore: use pip3 --- .github/workflows/main.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 0ecf25b..9d44ae2 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -92,7 +92,7 @@ jobs: run: | python -m venv python-ci-run source python-ci-run/bin/activate - python -m pip install --upgrade pip - python -m pip install tox - python -m pip install -r requirements.txt + pip3 install --upgrade pip + pip3 install tox + pip3 install -r requirements.txt TOXENV=${{ matrix.toxenv }} ALGOLIA_APPLICATION_ID=${{ secrets.ALGOLIA_APPLICATION_ID }} ALGOLIA_API_KEY=${{ secrets.ALGOLIA_API_KEY }} tox From ca426a7f16c848c37013aa9925872a87a8319afc Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 16:47:11 +0100 Subject: [PATCH 06/35] chore: format and setup linter --- algoliasearch_django/__init__.py | 12 +- algoliasearch_django/apps.py | 2 +- algoliasearch_django/decorators.py | 17 +- .../commands/algolia_applysettings.py | 11 +- .../management/commands/algolia_clearindex.py | 11 +- .../management/commands/algolia_reindex.py | 15 +- algoliasearch_django/models.py | 212 ++++--- algoliasearch_django/registration.py | 34 +- algoliasearch_django/settings.py | 2 +- algoliasearch_django/version.py | 2 +- requirements.txt | 2 + runtests.py | 7 +- setup.py | 71 ++- tests/factories.py | 26 +- tests/models.py | 7 +- tests/settings.py | 59 +- tests/test_commands.py | 55 +- tests/test_decorators.py | 70 +-- tests/test_engine.py | 24 +- tests/test_index.py | 560 ++++++++++-------- tests/test_signal.py | 40 +- tox.ini | 18 +- 22 files changed, 672 insertions(+), 585 deletions(-) diff --git a/algoliasearch_django/__init__.py b/algoliasearch_django/__init__.py index 95c7c7d..08751ea 100644 --- a/algoliasearch_django/__init__.py +++ b/algoliasearch_django/__init__.py @@ -5,6 +5,7 @@ from django.utils.module_loading import autodiscover_modules +import logging from . import models from . import registration from . import settings @@ -30,13 +31,10 @@ delete_record = algolia_engine.delete_record update_records = algolia_engine.update_records raw_search = algolia_engine.raw_search -clear_index = algolia_engine.clear_index # TODO: deprecate +clear_index = algolia_engine.clear_index # TODO: deprecate clear_objects = algolia_engine.clear_objects reindex_all = algolia_engine.reindex_all -# Default log handler -import logging - class NullHandler(logging.Handler): def emit(self, record): @@ -44,9 +42,9 @@ def emit(self, record): def autodiscover(): - autodiscover_modules('index') + autodiscover_modules("index") -logging.getLogger(__name__.split('.')[0]).addHandler(NullHandler()) +logging.getLogger(__name__.split(".")[0]).addHandler(NullHandler()) -default_app_config = 'algoliasearch_django.apps.AlgoliaConfig' +default_app_config = "algoliasearch_django.apps.AlgoliaConfig" diff --git a/algoliasearch_django/apps.py b/algoliasearch_django/apps.py index a614999..26ae6c3 100644 --- a/algoliasearch_django/apps.py +++ b/algoliasearch_django/apps.py @@ -4,7 +4,7 @@ class AlgoliaConfig(AppConfig): """Simple AppConfig which does not do automatic discovery.""" - name = 'algoliasearch_django' + name = "algoliasearch_django" def ready(self): super(AlgoliaConfig, self).ready() diff --git a/algoliasearch_django/decorators.py b/algoliasearch_django/decorators.py index 23c4eae..a85e268 100644 --- a/algoliasearch_django/decorators.py +++ b/algoliasearch_django/decorators.py @@ -26,11 +26,13 @@ class ContextDecorator: """ A base class that enables a context manager to also be used as a decorator. """ + def __call__(self, func): @wraps(func, assigned=available_attrs(func)) def inner(*args, **kwargs): with self: return func(*args, **kwargs) + return inner @@ -47,11 +49,12 @@ class AuthorIndex(AlgoliaIndex): def _algolia_engine_wrapper(index_class): if not issubclass(index_class, AlgoliaIndex): - raise ValueError('Wrapped class must subclass AlgoliaIndex.') + raise ValueError("Wrapped class must subclass AlgoliaIndex.") register(model, index_class) return index_class + return _algolia_engine_wrapper @@ -76,21 +79,17 @@ def __init__(self, model=None): def __enter__(self): for model in self.models: post_save.disconnect( - algolia_engine._AlgoliaEngine__post_save_receiver, - sender=model + algolia_engine._AlgoliaEngine__post_save_receiver, sender=model ) pre_delete.disconnect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, - sender=model + algolia_engine._AlgoliaEngine__pre_delete_receiver, sender=model ) def __exit__(self, exc_type, exc_value, traceback): for model in self.models: post_save.connect( - algolia_engine._AlgoliaEngine__post_save_receiver, - sender=model + algolia_engine._AlgoliaEngine__post_save_receiver, sender=model ) pre_delete.connect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, - sender=model + algolia_engine._AlgoliaEngine__pre_delete_receiver, sender=model ) diff --git a/algoliasearch_django/management/commands/algolia_applysettings.py b/algoliasearch_django/management/commands/algolia_applysettings.py index cb034e4..69df31b 100644 --- a/algoliasearch_django/management/commands/algolia_applysettings.py +++ b/algoliasearch_django/management/commands/algolia_applysettings.py @@ -5,18 +5,17 @@ class Command(BaseCommand): - help = 'Apply index settings.' + help = "Apply index settings." def add_arguments(self, parser): - parser.add_argument('--model', nargs='+', type=str) + parser.add_argument("--model", nargs="+", type=str) def handle(self, *args, **options): """Run the management command.""" - self.stdout.write('Apply settings to index:') + self.stdout.write("Apply settings to index:") for model in get_registered_model(): - if options.get('model', None) and not (model.__name__ in - options['model']): + if options.get("model", None) and model.__name__ not in options["model"]: continue get_adapter(model).set_settings() - self.stdout.write('\t* {}'.format(model.__name__)) + self.stdout.write("\t* {}".format(model.__name__)) diff --git a/algoliasearch_django/management/commands/algolia_clearindex.py b/algoliasearch_django/management/commands/algolia_clearindex.py index a2c4ade..3e5dff0 100644 --- a/algoliasearch_django/management/commands/algolia_clearindex.py +++ b/algoliasearch_django/management/commands/algolia_clearindex.py @@ -5,18 +5,17 @@ class Command(BaseCommand): - help = 'Clear index.' + help = "Clear index." def add_arguments(self, parser): - parser.add_argument('--model', nargs='+', type=str) + parser.add_argument("--model", nargs="+", type=str) def handle(self, *args, **options): """Run the management command.""" - self.stdout.write('Clear index:') + self.stdout.write("Clear index:") for model in get_registered_model(): - if options.get('model', None) and not (model.__name__ in - options['model']): + if options.get("model", None) and model.__name__ not in options["model"]: continue clear_objects(model) - self.stdout.write('\t* {}'.format(model.__name__)) + self.stdout.write("\t* {}".format(model.__name__)) diff --git a/algoliasearch_django/management/commands/algolia_reindex.py b/algoliasearch_django/management/commands/algolia_reindex.py index ce9b9da..bd1ad50 100644 --- a/algoliasearch_django/management/commands/algolia_reindex.py +++ b/algoliasearch_django/management/commands/algolia_reindex.py @@ -5,25 +5,24 @@ class Command(BaseCommand): - help = 'Reindex all models to Algolia' + help = "Reindex all models to Algolia" def add_arguments(self, parser): - parser.add_argument('--batchsize', nargs='?', default=1000, type=int) - parser.add_argument('--model', nargs='+', type=str) + parser.add_argument("--batchsize", nargs="?", default=1000, type=int) + parser.add_argument("--model", nargs="+", type=str) def handle(self, *args, **options): """Run the management command.""" - batch_size = options.get('batchsize', None) + batch_size = options.get("batchsize", None) if not batch_size: # py34-django18: batchsize is set to None if the user don't set # the value, instead of not be present in the dict batch_size = 1000 - self.stdout.write('The following models were reindexed:') + self.stdout.write("The following models were reindexed:") for model in get_registered_model(): - if options.get('model', None) and not (model.__name__ in - options['model']): + if options.get("model", None) and model.__name__ not in options["model"]: continue counts = reindex_all(model, batch_size=batch_size) - self.stdout.write('\t* {} --> {}'.format(model.__name__, counts)) + self.stdout.write("\t* {} --> {}".format(model.__name__, counts)) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index bb3730e..63e7cb3 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -5,7 +5,6 @@ from itertools import chain import logging -import sys from algoliasearch.exceptions import AlgoliaException from django.db.models.query_utils import DeferredAttribute @@ -26,8 +25,7 @@ def check_and_get_attr(model, name): else: return get_model_attr(name) except AttributeError: - raise AlgoliaIndexError( - '{} is not an attribute of {}'.format(name, model)) + raise AlgoliaIndexError("{} is not an attribute of {}".format(name, model)) def get_model_attr(name): @@ -43,7 +41,7 @@ class AlgoliaIndex(object): # Use to specify a custom field that will be used for the objectID. # This field should be unique. - custom_objectID = 'pk' + custom_objectID = "pk" # Use to specify the fields that should be included in the index. fields = () @@ -80,11 +78,15 @@ def __init__(self, model, client, settings): self.__named_fields = {} self.__translate_fields = {} - if self.settings is None: # Only set settings if the actual index class does not define some + if ( + self.settings is None + ): # Only set settings if the actual index class does not define some self.settings = {} try: - all_model_fields = [f.name for f in model._meta.get_fields() if not f.is_relation] + all_model_fields = [ + f.name for f in model._meta.get_fields() if not f.is_relation + ] except AttributeError: # get_fields requires Django >= 1.8 all_model_fields = [f.name for f in model._meta.local_fields] @@ -93,13 +95,11 @@ def __init__(self, model, client, settings): elif isinstance(self.fields, (list, tuple, set)): self.fields = tuple(self.fields) else: - raise AlgoliaIndexError('Fields must be a str, list, tuple or set') + raise AlgoliaIndexError("Fields must be a str, list, tuple or set") # Check fields for field in self.fields: - # unicode is a type in python < 3.0, which we need to support (e.g. dev uses unicode_literals) - # noinspection PyUnresolvedReferences - if sys.version_info < (3, 0) and isinstance(field, unicode) or isinstance(field, str): + if isinstance(field, str): attr = field name = field elif isinstance(field, (list, tuple)) and len(field) == 2: @@ -107,7 +107,8 @@ def __init__(self, model, client, settings): name = field[1] else: raise AlgoliaIndexError( - 'Invalid fields syntax: {} (type: {})'.format(field, type(field))) + "Invalid fields syntax: {} (type: {})".format(field, type(field)) + ) self.__translate_fields[attr] = name if attr in all_model_fields: @@ -118,21 +119,25 @@ def __init__(self, model, client, settings): # If no fields are specified, index all the fields of the model if not self.fields: self.fields = set(all_model_fields) - for elt in ('pk', 'id', 'objectID'): + for elt in ("pk", "id", "objectID"): try: self.fields.remove(elt) except KeyError: continue self.__translate_fields = dict(zip(self.fields, self.fields)) - self.__named_fields = dict(zip(self.fields, map(get_model_attr, - self.fields))) + self.__named_fields = dict( + zip(self.fields, map(get_model_attr, self.fields)) + ) # Check custom_objectID - if self.custom_objectID in chain(['pk'], all_model_fields) or hasattr(model, self.custom_objectID): + if self.custom_objectID in chain(["pk"], all_model_fields) or hasattr( + model, self.custom_objectID + ): self.objectID = get_model_attr(self.custom_objectID) else: - raise AlgoliaIndexError('{} is not a model field of {}'.format( - self.custom_objectID, model)) + raise AlgoliaIndexError( + "{} is not a model field of {}".format(self.custom_objectID, model) + ) # Check tags if self.tags: @@ -149,34 +154,37 @@ def __init__(self, model, client, settings): if self.should_index: if hasattr(model, self.should_index): attr = getattr(model, self.should_index) - if type(attr) is not bool: # if attr is a bool, we keep attr=name to getattr on instance + if ( + type(attr) is not bool + ): # if attr is a bool, we keep attr=name to getattr on instance self.should_index = attr if callable(self.should_index): self._should_index_is_method = True else: try: model._meta.get_field_by_name(self.should_index) - except: - raise AlgoliaIndexError('{} is not an attribute nor a field of {}.'.format( - self.should_index, model)) + except Exception: + raise AlgoliaIndexError( + "{} is not an attribute nor a field of {}.".format( + self.should_index, model + ) + ) def __init_index(self, client, model, settings): if not self.index_name: self.index_name = model.__name__ - tmp_index_name = '{index_name}_tmp'.format(index_name=self.index_name) + tmp_index_name = "{index_name}_tmp".format(index_name=self.index_name) - if 'INDEX_PREFIX' in settings: - self.index_name = settings['INDEX_PREFIX'] + '_' + self.index_name - tmp_index_name = '{index_prefix}_{tmp_index_name}'.format( - tmp_index_name=tmp_index_name, - index_prefix=settings['INDEX_PREFIX'] + if "INDEX_PREFIX" in settings: + self.index_name = settings["INDEX_PREFIX"] + "_" + self.index_name + tmp_index_name = "{index_prefix}_{tmp_index_name}".format( + tmp_index_name=tmp_index_name, index_prefix=settings["INDEX_PREFIX"] ) - if 'INDEX_SUFFIX' in settings: - self.index_name += '_' + settings['INDEX_SUFFIX'] - tmp_index_name = '{tmp_index_name}_{index_suffix}'.format( - tmp_index_name=tmp_index_name, - index_suffix=settings['INDEX_SUFFIX'] + if "INDEX_SUFFIX" in settings: + self.index_name += "_" + settings["INDEX_SUFFIX"] + tmp_index_name = "{tmp_index_name}_{index_suffix}".format( + tmp_index_name=tmp_index_name, index_suffix=settings["INDEX_SUFFIX"] ) self.tmp_index_name = tmp_index_name @@ -189,7 +197,7 @@ def _validate_geolocation(geolocation): """ Make sure we have the proper geolocation format. """ - if set(geolocation) != {'lat', 'lng'}: + if set(geolocation) != {"lat", "lng"}: raise AlgoliaIndexError( 'Invalid geolocation format, requires "lat" and "lng" keys only got {}'.format( geolocation @@ -204,7 +212,7 @@ def get_raw_record(self, instance, update_fields=None): the objectID and the given fields. Also, `_geoloc` and `_tags` will not be included. """ - tmp = {'objectID': self.objectID(instance)} + tmp = {"objectID": self.objectID(instance)} if update_fields: if isinstance(update_fields, str): @@ -222,21 +230,21 @@ def get_raw_record(self, instance, update_fields=None): loc = self.geo_field(instance) if isinstance(loc, tuple): - tmp['_geoloc'] = {'lat': loc[0], 'lng': loc[1]} + tmp["_geoloc"] = {"lat": loc[0], "lng": loc[1]} elif isinstance(loc, dict): self._validate_geolocation(loc) - tmp['_geoloc'] = loc + tmp["_geoloc"] = loc elif isinstance(loc, list): [self._validate_geolocation(geo) for geo in loc] - tmp['_geoloc'] = loc + tmp["_geoloc"] = loc if self.tags: if callable(self.tags): - tmp['_tags'] = self.tags(instance) - if not isinstance(tmp['_tags'], list): - tmp['_tags'] = list(tmp['_tags']) + tmp["_tags"] = self.tags(instance) + if not isinstance(tmp["_tags"], list): + tmp["_tags"] = list(tmp["_tags"]) - logger.debug('BUILD %s FROM %s', tmp['objectID'], self.model) + logger.debug("BUILD %s FROM %s", tmp["objectID"], self.model) return tmp def _has_should_index(self): @@ -276,11 +284,16 @@ def _should_really_index(self, instance): elif attr_type is property: attr_value = self.should_index.__get__(instance) else: - raise AlgoliaIndexError('{} should be a boolean attribute or a method that returns a boolean.'.format( - self.should_index)) + raise AlgoliaIndexError( + "{} should be a boolean attribute or a method that returns a boolean.".format( + self.should_index + ) + ) if type(attr_value) is not bool: - raise AlgoliaIndexError("%s's should_index (%s) should be a boolean" % ( - instance.__class__.__name__, self.should_index)) + raise AlgoliaIndexError( + "%s's should_index (%s) should be a boolean" + % (instance.__class__.__name__, self.should_index) + ) return attr_value def save_record(self, instance, update_fields=None, **kwargs): @@ -301,33 +314,32 @@ def save_record(self, instance, update_fields=None, **kwargs): try: if update_fields: - obj = self.get_raw_record(instance, - update_fields=update_fields) + obj = self.get_raw_record(instance, update_fields=update_fields) result = self.__index.partial_update_object(obj) else: obj = self.get_raw_record(instance) result = self.__index.save_object(obj) - logger.info('SAVE %s FROM %s', obj['objectID'], self.model) + logger.info("SAVE %s FROM %s", obj["objectID"], self.model) return result except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('%s FROM %s NOT SAVED: %s', obj['objectID'], - self.model, e) + logger.warning( + "%s FROM %s NOT SAVED: %s", obj["objectID"], self.model, e + ) def delete_record(self, instance): """Deletes the record.""" objectID = self.objectID(instance) try: self.__index.delete_object(objectID) - logger.info('DELETE %s FROM %s', objectID, self.model) + logger.info("DELETE %s FROM %s", objectID, self.model) except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('%s FROM %s NOT DELETED: %s', objectID, - self.model, e) + logger.warning("%s FROM %s NOT DELETED: %s", objectID, self.model, e) def update_records(self, qs, batch_size=1000, **kwargs): """ @@ -350,9 +362,10 @@ def update_records(self, qs, batch_size=1000, **kwargs): batch = [] objectsIDs = qs.only(self.custom_objectID).values_list( - self.custom_objectID, flat=True) + self.custom_objectID, flat=True + ) for elt in objectsIDs: - tmp['objectID'] = elt + tmp["objectID"] = elt batch.append(dict(tmp)) if len(batch) >= batch_size: @@ -362,7 +375,7 @@ def update_records(self, qs, batch_size=1000, **kwargs): if len(batch) > 0: self.__index.partial_update_objects(batch) - def raw_search(self, query='', params=None): + def raw_search(self, query="", params=None): """Performs a search query and returns the parsed JSON.""" if params is None: params = {} @@ -373,19 +386,18 @@ def raw_search(self, query='', params=None): if DEBUG: raise e else: - logger.warning('ERROR DURING SEARCH ON %s: %s', self.index_name, e) + logger.warning("ERROR DURING SEARCH ON %s: %s", self.index_name, e) def get_settings(self): """Returns the settings of the index.""" try: - logger.info('GET SETTINGS ON %s', self.index_name) + logger.info("GET SETTINGS ON %s", self.index_name) return self.__index.get_settings() except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('ERROR DURING GET_SETTINGS ON %s: %s', - self.model, e) + logger.warning("ERROR DURING GET_SETTINGS ON %s: %s", self.model, e) def set_settings(self): """Applies the settings to the index.""" @@ -394,24 +406,23 @@ def set_settings(self): try: self.__index.set_settings(self.settings) - logger.info('APPLY SETTINGS ON %s', self.index_name) + logger.info("APPLY SETTINGS ON %s", self.index_name) except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('SETTINGS NOT APPLIED ON %s: %s', - self.model, e) + logger.warning("SETTINGS NOT APPLIED ON %s: %s", self.model, e) def clear_objects(self): """Clears all objects of an index.""" try: self.__index.clear_objects() - logger.info('CLEAR INDEX %s', self.index_name) + logger.info("CLEAR INDEX %s", self.index_name) except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('%s NOT CLEARED: %s', self.model, e) + logger.warning("%s NOT CLEARED: %s", self.model, e) def clear_index(self): # TODO: add deprecated warning @@ -420,12 +431,12 @@ def clear_index(self): def wait_task(self, task_id): try: self.__index.wait_task(task_id) - logger.info('WAIT TASK %s', self.index_name) + logger.info("WAIT TASK %s", self.index_name) except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('%s NOT WAIT: %s', self.model, e) + logger.warning("%s NOT WAIT: %s", self.model, e) def delete(self): self.__index.delete() @@ -445,9 +456,13 @@ def reindex_all(self, batch_size=1000): try: if not self.settings: self.settings = self.get_settings() - logger.debug('Got settings for index %s: %s', self.index_name, self.settings) + logger.debug( + "Got settings for index %s: %s", self.index_name, self.settings + ) else: - logger.debug("index %s already has settings: %s", self.index_name, self.settings) + logger.debug( + "index %s already has settings: %s", self.index_name, self.settings + ) except AlgoliaException as e: if any("Index does not exist" in arg for arg in e.args): pass # Expected, let's clear and recreate from scratch @@ -455,21 +470,21 @@ def reindex_all(self, batch_size=1000): raise e # Unexpected error while getting settings try: if self.settings: - replicas = self.settings.get('replicas', None) - slaves = self.settings.get('slaves', None) + replicas = self.settings.get("replicas", None) + slaves = self.settings.get("slaves", None) should_keep_replicas = replicas is not None should_keep_slaves = slaves is not None if should_keep_replicas: - self.settings['replicas'] = [] + self.settings["replicas"] = [] logger.debug("REMOVE REPLICAS FROM SETTINGS") if should_keep_slaves: - self.settings['slaves'] = [] + self.settings["slaves"] = [] logger.debug("REMOVE SLAVES FROM SETTINGS") self.__tmp_index.set_settings(self.settings).wait() - logger.debug('APPLY SETTINGS ON %s_tmp', self.index_name) + logger.debug("APPLY SETTINGS ON %s_tmp", self.index_name) rules = [] synonyms = [] for r in self.__index.browse_rules(): @@ -477,19 +492,19 @@ def reindex_all(self, batch_size=1000): for s in self.__index.browse_synonyms(): synonyms.append(s) if len(rules): - logger.debug('Got rules for index %s: %s', self.index_name, rules) + logger.debug("Got rules for index %s: %s", self.index_name, rules) should_keep_rules = True if len(synonyms): - logger.debug('Got synonyms for index %s: %s', self.index_name, rules) + logger.debug("Got synonyms for index %s: %s", self.index_name, rules) should_keep_synonyms = True self.__tmp_index.clear_objects() - logger.debug('CLEAR INDEX %s_tmp', self.index_name) + logger.debug("CLEAR INDEX %s_tmp", self.index_name) counts = 0 batch = [] - if hasattr(self, 'get_queryset'): + if hasattr(self, "get_queryset"): qs = self.get_queryset() else: qs = self.model.objects.all() @@ -501,41 +516,50 @@ def reindex_all(self, batch_size=1000): batch.append(self.get_raw_record(instance)) if len(batch) >= batch_size: self.__tmp_index.save_objects(batch) - logger.info('SAVE %d OBJECTS TO %s_tmp', len(batch), - self.index_name) + logger.info( + "SAVE %d OBJECTS TO %s_tmp", len(batch), self.index_name + ) batch = [] counts += 1 if len(batch) > 0: self.__tmp_index.save_objects(batch) - logger.info('SAVE %d OBJECTS TO %s_tmp', len(batch), - self.index_name) + logger.info("SAVE %d OBJECTS TO %s_tmp", len(batch), self.index_name) - self.__client.move_index(self.tmp_index_name, - self.index_name) - logger.info('MOVE INDEX %s_tmp TO %s', self.index_name, - self.index_name) + self.__client.move_index(self.tmp_index_name, self.index_name) + logger.info("MOVE INDEX %s_tmp TO %s", self.index_name, self.index_name) if self.settings: if should_keep_replicas: - self.settings['replicas'] = replicas + self.settings["replicas"] = replicas logger.debug("RESTORE REPLICAS") if should_keep_slaves: - self.settings['slaves'] = slaves + self.settings["slaves"] = slaves logger.debug("RESTORE SLAVES") if should_keep_replicas or should_keep_slaves: self.__index.set_settings(self.settings) if should_keep_rules: - response = self.__index.save_rules(rules, {'forwardToReplicas': True}) + response = self.__index.save_rules( + rules, {"forwardToReplicas": True} + ) response.wait() - logger.info("Saved rules for index %s with response: {}".format(response), self.index_name) + logger.info( + "Saved rules for index %s with response: {}".format(response), + self.index_name, + ) if should_keep_synonyms: - response = self.__index.save_synonyms(synonyms, {'forwardToReplicas': True}) + response = self.__index.save_synonyms( + synonyms, {"forwardToReplicas": True} + ) response.wait() - logger.info("Saved synonyms for index %s with response: {}".format(response), self.index_name) + logger.info( + "Saved synonyms for index %s with response: {}".format( + response + ), + self.index_name, + ) return counts except AlgoliaException as e: if DEBUG: raise e else: - logger.warning('ERROR DURING REINDEXING %s: %s', self.model, - e) + logger.warning("ERROR DURING REINDEXING %s: %s", self.model, e) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index 2616511..bd93ac9 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -30,13 +30,12 @@ def __init__(self, settings=SETTINGS): """Initializes the Algolia engine.""" try: - app_id = settings['APPLICATION_ID'] - api_key = settings['API_KEY'] + app_id = settings["APPLICATION_ID"] + api_key = settings["API_KEY"] except KeyError: - raise AlgoliaEngineError( - 'APPLICATION_ID and API_KEY must be defined.') + raise AlgoliaEngineError("APPLICATION_ID and API_KEY must be defined.") - self.__auto_indexing = settings.get('AUTO_INDEXING', True) + self.__auto_indexing = settings.get("AUTO_INDEXING", True) self.__settings = settings self.__registered_models = {} @@ -56,21 +55,22 @@ def register(self, model, index_cls=AlgoliaIndex, auto_indexing=None): # Check for existing registration. if self.is_registered(model): raise RegistrationError( - '{} is already registered with Algolia engine'.format(model)) + "{} is already registered with Algolia engine".format(model) + ) # Perform the registration. if not issubclass(index_cls, AlgoliaIndex): raise RegistrationError( - '{} should be a subclass of AlgoliaIndex'.format(index_cls)) + "{} should be a subclass of AlgoliaIndex".format(index_cls) + ) index_obj = index_cls(model, self.client, self.__settings) self.__registered_models[model] = index_obj - if (isinstance(auto_indexing, bool) and - auto_indexing) or self.__auto_indexing: + if (isinstance(auto_indexing, bool) and auto_indexing) or self.__auto_indexing: # Connect to the signalling framework. post_save.connect(self.__post_save_receiver, model) pre_delete.connect(self.__pre_delete_receiver, model) - logger.info('REGISTER %s', model) + logger.info("REGISTER %s", model) def unregister(self, model): """ @@ -81,14 +81,15 @@ def unregister(self, model): """ if not self.is_registered(model): raise RegistrationError( - '{} is not registered with Algolia engine'.format(model)) + "{} is not registered with Algolia engine".format(model) + ) # Perform the unregistration. del self.__registered_models[model] # Disconnect from the signalling framework. post_save.disconnect(self.__post_save_receiver, model) pre_delete.disconnect(self.__pre_delete_receiver, model) - logger.info('UNREGISTER %s', model) + logger.info("UNREGISTER %s", model) def get_registered_models(self): """ @@ -101,7 +102,8 @@ def get_adapter(self, model): """Returns the adapter associated with the given model.""" if not self.is_registered(model): raise RegistrationError( - '{} is not registered with Algolia engine'.format(model)) + "{} is not registered with Algolia engine".format(model) + ) return self.__registered_models[model] @@ -145,7 +147,7 @@ def update_records(self, model, qs, batch_size=1000, **kwargs): adapter = self.get_adapter(model) adapter.update_records(qs, batch_size=batch_size, **kwargs) - def raw_search(self, model, query='', params=None): + def raw_search(self, model, query="", params=None): """Performs a search query and returns the parsed JSON.""" if params is None: params = {} @@ -183,12 +185,12 @@ def reset(self, settings=None): def __post_save_receiver(self, instance, **kwargs): """Signal handler for when a registered model has been saved.""" - logger.debug('RECEIVE post_save FOR %s', instance.__class__) + logger.debug("RECEIVE post_save FOR %s", instance.__class__) self.save_record(instance, **kwargs) def __pre_delete_receiver(self, instance, **kwargs): """Signal handler for when a registered model has been deleted.""" - logger.debug('RECEIVE pre_delete FOR %s', instance.__class__) + logger.debug("RECEIVE pre_delete FOR %s", instance.__class__) self.delete_record(instance) diff --git a/algoliasearch_django/settings.py b/algoliasearch_django/settings.py index abbaa61..354f292 100644 --- a/algoliasearch_django/settings.py +++ b/algoliasearch_django/settings.py @@ -1,4 +1,4 @@ from django.conf import settings SETTINGS = settings.ALGOLIA -DEBUG = SETTINGS.get('RAISE_EXCEPTIONS', settings.DEBUG) +DEBUG = SETTINGS.get("RAISE_EXCEPTIONS", settings.DEBUG) diff --git a/algoliasearch_django/version.py b/algoliasearch_django/version.py index aaa4264..189c03b 100644 --- a/algoliasearch_django/version.py +++ b/algoliasearch_django/version.py @@ -1 +1 @@ -VERSION = '3.0.0' +VERSION = "4.0.0" diff --git a/requirements.txt b/requirements.txt index cf996c8..c98c7fa 100644 --- a/requirements.txt +++ b/requirements.txt @@ -7,3 +7,5 @@ tox twine factory_boy mock +ruff>=0.7.4,<1.0 +pyright>=1.1.389,<2.0 diff --git a/runtests.py b/runtests.py index 4318d94..3e3da0c 100755 --- a/runtests.py +++ b/runtests.py @@ -9,12 +9,13 @@ def main(): - os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.settings' + os.environ["DJANGO_SETTINGS_MODULE"] = "tests.settings" django.setup() TestRunner = get_runner(settings) test_runner = TestRunner() - failures = test_runner.run_tests(['tests']) + failures = test_runner.run_tests(["tests"]) sys.exit(bool(failures)) -if __name__ == '__main__': + +if __name__ == "__main__": main() diff --git a/setup.py b/setup.py index 051d80b..208ae50 100644 --- a/setup.py +++ b/setup.py @@ -10,16 +10,18 @@ # Allow setup.py to be run from any path os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) -path_readme = os.path.join(os.path.dirname(__file__), 'README.md') +path_readme = os.path.join(os.path.dirname(__file__), "README.md") try: import pypandoc - README = pypandoc.convert_file(path_readme, 'rst') + + README = pypandoc.convert_file(path_readme, "rst") except (IOError, ImportError): with open(path_readme) as readme: README = readme.read() -path_version = os.path.join(os.path.dirname(__file__), - 'algoliasearch_django/version.py') +path_version = os.path.join( + os.path.dirname(__file__), "algoliasearch_django/version.py" +) if sys.version_info < (3, 8): raise RuntimeError("algoliasearch_django 4.x requires Python 3.8+") else: @@ -27,32 +29,41 @@ setup( - name='algoliasearch-django', - version=VERSION, - license='MIT License', - packages=find_packages(exclude=['tests']), - install_requires=['django>=4.0', 'algoliasearch>=4.0,<5.0'], - description='Algolia Search integration for Django', + name="algoliasearch-django", + version="4.0.0", + license="MIT License", + packages=find_packages(exclude=["tests"]), + install_requires=["django>=4.0", "algoliasearch>=4.0,<5.0"], + description="Algolia Search integration for Django", long_description=README, - long_description_content_type='text/markdown', - author='Algolia Team', - author_email='support@algolia.com', - url='https://github.com/algolia/algoliasearch-django', - keywords=['algolia', 'pyalgolia', 'search', 'backend', 'hosted', 'cloud', - 'full-text search', 'faceted search', 'django'], + long_description_content_type="text/markdown", + author="Algolia Team", + author_email="support@algolia.com", + url="https://github.com/algolia/algoliasearch-django", + keywords=[ + "algolia", + "pyalgolia", + "search", + "backend", + "hosted", + "cloud", + "full-text search", + "faceted search", + "django", + ], classifiers=[ - 'Environment :: Web Environment', - 'Framework :: Django', - 'Intended Audience :: Developers', - 'License :: OSI Approved :: MIT License', - 'Operating System :: OS Independent', - 'Programming Language :: Python', - 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.8', - 'Programming Language :: Python :: 3.9', - 'Programming Language :: Python :: 3.10', - 'Programming Language :: Python :: 3.11', - 'Programming Language :: Python :: 3.12', - 'Topic :: Internet :: WWW/HTTP', - ] + "Environment :: Web Environment", + "Framework :: Django", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Topic :: Internet :: WWW/HTTP", + ], ) diff --git a/tests/factories.py b/tests/factories.py index ad237d7..395e833 100644 --- a/tests/factories.py +++ b/tests/factories.py @@ -1,37 +1,33 @@ import factory -from .models import ( - Example, - User, - Website -) +from .models import Example, User, Website class ExampleFactory(factory.django.DjangoModelFactory): uid = factory.Sequence(lambda n: n) - name = factory.Sequence(lambda n: 'Example name-{}'.format(n)) - address = factory.Sequence(lambda n: 'Example address-{}'.format(n)) - lat = factory.Faker('latitude') - lng = factory.Faker('longitude') + name = factory.Sequence(lambda n: "Example name-{}".format(n)) + address = factory.Sequence(lambda n: "Example address-{}".format(n)) + lat = factory.Faker("latitude") + lng = factory.Faker("longitude") class Meta: model = Example class UserFactory(factory.django.DjangoModelFactory): - name = factory.Sequence(lambda n: 'User name-{}'.format(n)) - username = factory.Sequence(lambda n: 'User username-{}'.format(n)) + name = factory.Sequence(lambda n: "User name-{}".format(n)) + username = factory.Sequence(lambda n: "User username-{}".format(n)) - _lat = factory.Faker('latitude') - _lng = factory.Faker('longitude') + _lat = factory.Faker("latitude") + _lng = factory.Faker("longitude") class Meta: model = User class WebsiteFactory(factory.django.DjangoModelFactory): - name = factory.Sequence(lambda n: 'Website name-{}'.format(n)) - url = factory.Faker('url') + name = factory.Sequence(lambda n: "Website name-{}".format(n)) + url = factory.Faker("url") class Meta: model = Website diff --git a/tests/models.py b/tests/models.py index 58e5723..f216d49 100644 --- a/tests/models.py +++ b/tests/models.py @@ -19,7 +19,7 @@ def location(self): return self._lat, self._lng def permissions(self): - return self._permissions.split(',') + return self._permissions.split(",") class Website(models.Model): @@ -70,8 +70,5 @@ def property_string(self): class BlogPost(models.Model): - author = models.ForeignKey( - User, - on_delete=models.CASCADE - ) + author = models.ForeignKey(User, on_delete=models.CASCADE) text = models.TextField(default="") diff --git a/tests/settings.py b/tests/settings.py index e1066c2..746243d 100644 --- a/tests/settings.py +++ b/tests/settings.py @@ -15,30 +15,30 @@ BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) -SECRET_KEY = 'MillisecondsMatter' +SECRET_KEY = "MillisecondsMatter" DEBUG = False # Application definition INSTALLED_APPS = ( - 'django.contrib.admin', - 'django.contrib.auth', - 'django.contrib.contenttypes', - 'django.contrib.sessions', - 'django.contrib.messages', - 'django.contrib.staticfiles', - 'algoliasearch_django', - 'tests' + "django.contrib.admin", + "django.contrib.auth", + "django.contrib.contenttypes", + "django.contrib.sessions", + "django.contrib.messages", + "django.contrib.staticfiles", + "algoliasearch_django", + "tests", ) MIDDLEWARE = [ - 'django.contrib.sessions.middleware.SessionMiddleware', - 'django.middleware.common.CommonMiddleware', - 'django.middleware.csrf.CsrfViewMiddleware', - 'django.contrib.auth.middleware.AuthenticationMiddleware', - 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', - 'django.contrib.messages.middleware.MessageMiddleware', - 'django.middleware.clickjacking.XFrameOptionsMiddleware', - 'django.middleware.security.SecurityMiddleware', + "django.contrib.sessions.middleware.SessionMiddleware", + "django.middleware.common.CommonMiddleware", + "django.middleware.csrf.CsrfViewMiddleware", + "django.contrib.auth.middleware.AuthenticationMiddleware", + "django.contrib.auth.middleware.SessionAuthenticationMiddleware", + "django.contrib.messages.middleware.MessageMiddleware", + "django.middleware.clickjacking.XFrameOptionsMiddleware", + "django.middleware.security.SecurityMiddleware", ] TEMPLATES = [ @@ -57,32 +57,33 @@ }, ] -ROOT_URLCONF = 'tests.urls' +ROOT_URLCONF = "tests.urls" # Database DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), + "default": { + "ENGINE": "django.db.backends.sqlite3", + "NAME": os.path.join(BASE_DIR, "db.sqlite3"), } } # Internationalization -LANGUAGE_CODE = 'en-us' -TIME_ZONE = 'UTC' +LANGUAGE_CODE = "en-us" +TIME_ZONE = "UTC" USE_I18N = True USE_L10N = True USE_TZ = True def safe_index_name(name): - return '{}_ci-{}'.format(name, time.time()) + return "{}_ci-{}".format(name, time.time()) + # AlgoliaSearch settings ALGOLIA = { - 'APPLICATION_ID': os.getenv('ALGOLIA_APPLICATION_ID'), - 'API_KEY': os.getenv('ALGOLIA_API_KEY'), - 'INDEX_PREFIX': 'test', - 'INDEX_SUFFIX': safe_index_name('django'), - 'RAISE_EXCEPTIONS': True + "APPLICATION_ID": os.getenv("ALGOLIA_APPLICATION_ID"), + "API_KEY": os.getenv("ALGOLIA_API_KEY"), + "INDEX_PREFIX": "test", + "INDEX_SUFFIX": safe_index_name("django"), + "RAISE_EXCEPTIONS": True, } diff --git a/tests/test_commands.py b/tests/test_commands.py index 5b7e300..6f124a3 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -21,12 +21,14 @@ def tearDownClass(cls): def setUp(self): # Create some records - User.objects.create(name='James Bond', username="jb") - User.objects.create(name='Captain America', username="captain") - User.objects.create(name='John Snow', username="john_snow", - _lat=120.2, _lng=42.1) - User.objects.create(name='Steve Jobs', username="genius", - followers_count=331213) + User.objects.create(name="James Bond", username="jb") + User.objects.create(name="Captain America", username="captain") + User.objects.create( + name="John Snow", username="john_snow", _lat=120.2, _lng=42.1 + ) + User.objects.create( + name="Steve Jobs", username="genius", followers_count=331213 + ) self.out = StringIO() @@ -35,101 +37,96 @@ def tearDown(self): clear_index(User) def test_reindex(self): - call_command('algolia_reindex', stdout=self.out) + call_command("algolia_reindex", stdout=self.out) result = self.out.getvalue() - regex = r'Website --> 0' + regex = r"Website --> 0" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User --> 4' + regex = r"User --> 4" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) def test_reindex_with_args(self): - call_command('algolia_reindex', stdout=self.out, model=['Website']) + call_command("algolia_reindex", stdout=self.out, model=["Website"]) result = self.out.getvalue() - regex = r'Website --> \d+' + regex = r"Website --> \d+" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User --> \d+' + regex = r"User --> \d+" try: self.assertNotRegex(result, regex) except AttributeError: self.assertNotRegexpMatches(result, regex) def test_clearindex(self): - call_command('algolia_clearindex', stdout=self.out) + call_command("algolia_clearindex", stdout=self.out) result = self.out.getvalue() - regex = r'Website' + regex = r"Website" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User' + regex = r"User" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) def test_clearindex_with_args(self): - call_command( - 'algolia_clearindex', - stdout=self.out, - model=['Website'] - ) + call_command("algolia_clearindex", stdout=self.out, model=["Website"]) result = self.out.getvalue() - regex = r'Website' + regex = r"Website" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User' + regex = r"User" try: self.assertNotRegex(result, regex) except AttributeError: self.assertNotRegexpMatches(result, regex) def test_applysettings(self): - call_command('algolia_applysettings', stdout=self.out) + call_command("algolia_applysettings", stdout=self.out) result = self.out.getvalue() - regex = r'Website' + regex = r"Website" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User' + regex = r"User" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) def test_applysettings_with_args(self): - call_command('algolia_applysettings', stdout=self.out, - model=['Website']) + call_command("algolia_applysettings", stdout=self.out, model=["Website"]) result = self.out.getvalue() - regex = r'Website' + regex = r"Website" try: self.assertRegex(result, regex) except AttributeError: self.assertRegexpMatches(result, regex) - regex = r'User' + regex = r"User" try: self.assertNotRegex(result, regex) except AttributeError: diff --git a/tests/test_decorators.py b/tests/test_decorators.py index 53a9e16..46de051 100644 --- a/tests/test_decorators.py +++ b/tests/test_decorators.py @@ -1,8 +1,4 @@ -from mock import ( - ANY, - call, - patch -) +from mock import ANY, call, patch from django.test import TestCase @@ -26,28 +22,31 @@ def non_decorated_operation(): WebsiteFactory() UserFactory() - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: decorated_operation() # The decorated method should have prevented the indexing operations mocked_save_record.assert_not_called() - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: non_decorated_operation() # The non-decorated method is not preventing the indexing operations # (the signal was correctly re-connected for both of the models) - mocked_save_record.assert_has_calls([ - call( - ANY, - created=True, - raw=False, - sender=ANY, - signal=ANY, - update_fields=None, - using=ANY - ) - ] * 2) + mocked_save_record.assert_has_calls( + [ + call( + ANY, + created=True, + raw=False, + sender=ANY, + signal=ANY, + update_fields=None, + using=ANY, + ) + ] + * 2 + ) def test_disable_auto_indexing_as_decorator_for_model(self): """Test that the `disable_auto_indexing` should work as a decorator for a specific model""" @@ -61,7 +60,7 @@ def non_decorated_operation(): WebsiteFactory() UserFactory() - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: decorated_operation() # The decorated method should have prevented the indexing operation for the `User` model @@ -73,36 +72,39 @@ def non_decorated_operation(): sender=ANY, signal=ANY, update_fields=None, - using=ANY + using=ANY, ) - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: non_decorated_operation() # The non-decorated method is not preventing the indexing operations # (the signal was correctly re-connected for both of the models) - mocked_save_record.assert_has_calls([ - call( - ANY, - created=True, - raw=False, - sender=ANY, - signal=ANY, - update_fields=None, - using=ANY - ) - ] * 2) + mocked_save_record.assert_has_calls( + [ + call( + ANY, + created=True, + raw=False, + sender=ANY, + signal=ANY, + update_fields=None, + using=ANY, + ) + ] + * 2 + ) def test_disable_auto_indexing_as_context_manager(self): """Test that the `disable_auto_indexing` should work as a context manager""" - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: with disable_auto_indexing(): WebsiteFactory() mocked_save_record.assert_not_called() - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: WebsiteFactory() mocked_save_record.assert_called_once() diff --git a/tests/test_engine.py b/tests/test_engine.py index 1d97a79..c2d6d15 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -26,8 +26,8 @@ def tearDown(self): def test_init_exception(self): algolia_settings = dict(settings.ALGOLIA) - del algolia_settings['APPLICATION_ID'] - del algolia_settings['API_KEY'] + del algolia_settings["APPLICATION_ID"] + del algolia_settings["API_KEY"] with self.settings(ALGOLIA=algolia_settings): with self.assertRaises(AlgoliaEngineError): @@ -36,10 +36,10 @@ def test_init_exception(self): def test_user_agent(self): user_agent = UserAgent.get() - parts = re.split('\s*;\s*', user_agent) + parts = re.split("\s*;\s*", user_agent) - self.assertIn('Django (%s)' % django_version(), parts) - self.assertIn('Algolia for Django (%s)' % VERSION, parts) + self.assertIn("Django (%s)" % django_version(), parts) + self.assertIn("Algolia for Django (%s)" % VERSION, parts) def test_auto_discover_indexes(self): """Test that the `index` module was auto-discovered and the models registered""" @@ -50,7 +50,7 @@ def test_auto_discover_indexes(self): User, # Registered using the `register` decorator Website, # Registered using the `register` method ], - algolia_engine.get_registered_models() + algolia_engine.get_registered_models(), ) def test_is_register(self): @@ -60,8 +60,7 @@ def test_is_register(self): def test_get_adapter(self): self.engine.register(Website) - self.assertEquals(AlgoliaIndex, - self.engine.get_adapter(Website).__class__) + self.assertEquals(AlgoliaIndex, self.engine.get_adapter(Website).__class__) def test_get_adapter_exception(self): with self.assertRaises(RegistrationError): @@ -71,8 +70,8 @@ def test_get_adapter_from_instance(self): self.engine.register(Website) instance = Website() self.assertEquals( - AlgoliaIndex, - self.engine.get_adapter_from_instance(instance).__class__) + AlgoliaIndex, self.engine.get_adapter_from_instance(instance).__class__ + ) def test_register(self): self.engine.register(Website) @@ -92,8 +91,9 @@ class WebsiteIndex(AlgoliaIndex): pass self.engine.register(Website, WebsiteIndex) - self.assertEqual(WebsiteIndex.__name__, - self.engine.get_adapter(Website).__class__.__name__) + self.assertEqual( + WebsiteIndex.__name__, self.engine.get_adapter(Website).__class__.__name__ + ) def test_register_with_custom_index_exception(self): class WebsiteIndex(object): diff --git a/tests/test_index.py b/tests/test_index.py index 07b5622..c5136a3 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -15,41 +15,44 @@ class IndexTestCase(TestCase): def setUp(self): self.client = algolia_engine.client - self.user = User(name='Algolia', username="algolia", - bio='Milliseconds matter', followers_count=42001, - following_count=42, _lat=123, _lng=-42.24, - _permissions='read,write,admin') + self.user = User( + name="Algolia", + username="algolia", + bio="Milliseconds matter", + followers_count=42001, + following_count=42, + _lat=123, + _lng=-42.24, + _permissions="read,write,admin", + ) self.contributor = User( - name='Contributor', + name="Contributor", username="contributor", - bio='Contributions matter', + bio="Contributions matter", followers_count=7, following_count=5, _lat=52.0705, _lng=-4.3007, - _permissions='contribute,follow' + _permissions="contribute,follow", ) - self.example = Example(uid=4, - name='SuperK', - address='Finland', - lat=63.3, - lng=-32.0, - is_admin=True) - self.example.category = ['Shop', 'Grocery'] + self.example = Example( + uid=4, name="SuperK", address="Finland", lat=63.3, lng=-32.0, is_admin=True + ) + self.example.category = ["Shop", "Grocery"] self.example.locations = [ - {'lat': 10.3, 'lng': -20.0}, - {'lat': 22.3, 'lng': 10.0}, + {"lat": 10.3, "lng": -20.0}, + {"lat": 22.3, "lng": 10.0}, ] def tearDown(self): - if hasattr(self, 'index'): + if hasattr(self, "index"): self.index.delete() def test_default_index_name(self): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - regex = r'^test_Website_django(_ci-\d+.\d+)?$' + regex = r"^test_Website_django(_ci-\d+.\d+)?$" try: self.assertRegex(self.index.index_name, regex) except AttributeError: @@ -57,10 +60,10 @@ def test_default_index_name(self): def test_custom_index_name(self): class WebsiteIndex(AlgoliaIndex): - index_name = 'customName' + index_name = "customName" self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) - regex = r'^test_customName_django(_ci-\d+.\d+)?$' + regex = r"^test_customName_django(_ci-\d+.\d+)?$" try: self.assertRegex(self.index.index_name, regex) except AttributeError: @@ -73,12 +76,12 @@ def test_index_model_with_foreign_key_reference(self): def test_index_name_settings(self): algolia_settings = dict(settings.ALGOLIA) - del algolia_settings['INDEX_PREFIX'] - del algolia_settings['INDEX_SUFFIX'] + del algolia_settings["INDEX_PREFIX"] + del algolia_settings["INDEX_SUFFIX"] with self.settings(ALGOLIA=algolia_settings): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - regex = r'^Website$' + regex = r"^Website$" try: self.assertRegex(self.index.index_name, regex) except AttributeError: @@ -90,56 +93,44 @@ def test_tmp_index_name(self): algolia_settings = dict(settings.ALGOLIA) # With no suffix nor prefix - del algolia_settings['INDEX_PREFIX'] - del algolia_settings['INDEX_SUFFIX'] + del algolia_settings["INDEX_PREFIX"] + del algolia_settings["INDEX_SUFFIX"] with self.settings(ALGOLIA=algolia_settings): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - self.assertEqual( - self.index.tmp_index_name, - 'Website_tmp' - ) + self.assertEqual(self.index.tmp_index_name, "Website_tmp") # With only a prefix - algolia_settings['INDEX_PREFIX'] = 'prefix' + algolia_settings["INDEX_PREFIX"] = "prefix" with self.settings(ALGOLIA=algolia_settings): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - self.assertEqual( - self.index.tmp_index_name, - 'prefix_Website_tmp' - ) + self.assertEqual(self.index.tmp_index_name, "prefix_Website_tmp") # With only a suffix - del algolia_settings['INDEX_PREFIX'] - algolia_settings['INDEX_SUFFIX'] = 'suffix' + del algolia_settings["INDEX_PREFIX"] + algolia_settings["INDEX_SUFFIX"] = "suffix" with self.settings(ALGOLIA=algolia_settings): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - self.assertEqual( - self.index.tmp_index_name, - 'Website_tmp_suffix' - ) + self.assertEqual(self.index.tmp_index_name, "Website_tmp_suffix") # With a prefix and a suffix - algolia_settings['INDEX_PREFIX'] = 'prefix' - algolia_settings['INDEX_SUFFIX'] = 'suffix' + algolia_settings["INDEX_PREFIX"] = "prefix" + algolia_settings["INDEX_SUFFIX"] = "suffix" with self.settings(ALGOLIA=algolia_settings): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) - self.assertEqual( - self.index.tmp_index_name, - 'prefix_Website_tmp_suffix' - ) + self.assertEqual(self.index.tmp_index_name, "prefix_Website_tmp_suffix") def test_reindex_with_replicas(self): self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): settings = { - 'replicas': [ - self.index.index_name + '_name_asc', - self.index.index_name + '_name_desc' + "replicas": [ + self.index.index_name + "_name_asc", + self.index.index_name + "_name_desc", ] } @@ -148,25 +139,25 @@ class WebsiteIndex(AlgoliaIndex): def test_reindex_with_should_index_boolean(self): Website.objects.create( - name='Algolia', - url='https://algolia.com', - is_online=True + name="Algolia", url="https://algolia.com", is_online=True ) self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): settings = { - 'replicas': [ - self.index.index_name + '_name_asc', - self.index.index_name + '_name_desc' + "replicas": [ + self.index.index_name + "_name_asc", + self.index.index_name + "_name_desc", ] } - should_index = 'is_online' + should_index = "is_online" self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - @unittest.skip(reason="FIXME: it's a known issue that reindex all might not work properly") + @unittest.skip( + reason="FIXME: it's a known issue that reindex all might not work properly" + ) def test_reindex_no_settings(self): self.maxDiff = None @@ -185,34 +176,57 @@ class WebsiteIndex(AlgoliaIndex): time.sleep(10) # FIXME: Refactor reindex_all to return taskID # Expect the former settings to be kept across reindex - self.assertEqual(self.index.get_settings(), existing_settings, - "An index whose model has no settings should keep its settings after reindex") + self.assertEqual( + self.index.get_settings(), + existing_settings, + "An index whose model has no settings should keep its settings after reindex", + ) - @unittest.skip(reason="FIXME: it's a known issue that reindex all might not work properly") + @unittest.skip( + reason="FIXME: it's a known issue that reindex all might not work properly" + ) def test_reindex_with_settings(self): import uuid + id = str(uuid.uuid4()) self.maxDiff = None - index_settings = {'searchableAttributes': ['name', 'email', 'company', 'city', 'county', 'account_names', - 'unordered(address)', 'state', 'zip_code', 'phone', 'fax', - 'unordered(web)'], 'attributesForFaceting': ['city', 'company'], - 'customRanking': ['desc(followers)'], - 'queryType': 'prefixAll', - 'highlightPreTag': '', - 'ranking': [ - 'asc(name)', - 'typo', - 'geo', - 'words', - 'filters', - 'proximity', - 'attribute', - 'exact', - 'custom' - ], - 'replicas': ['WebsiteIndexReplica_' + id + '_name_asc', - 'WebsiteIndexReplica_' + id + '_name_desc'], - 'highlightPostTag': '', 'hitsPerPage': 15} + index_settings = { + "searchableAttributes": [ + "name", + "email", + "company", + "city", + "county", + "account_names", + "unordered(address)", + "state", + "zip_code", + "phone", + "fax", + "unordered(web)", + ], + "attributesForFaceting": ["city", "company"], + "customRanking": ["desc(followers)"], + "queryType": "prefixAll", + "highlightPreTag": "", + "ranking": [ + "asc(name)", + "typo", + "geo", + "words", + "filters", + "proximity", + "attribute", + "exact", + "custom", + ], + "replicas": [ + "WebsiteIndexReplica_" + id + "_name_asc", + "WebsiteIndexReplica_" + id + "_name_desc", + ], + "highlightPostTag": "", + "hitsPerPage": 15, + } # Given an existing index defined with settings class WebsiteIndex(AlgoliaIndex): @@ -236,27 +250,22 @@ class WebsiteIndex(AlgoliaIndex): former_settings["hitsPerPage"] = 15 self.assertDictEqual(self.index.get_settings(), former_settings) - @unittest.skip(reason="FIXME: it's a known issue that reindex all might not work properly") + @unittest.skip( + reason="FIXME: it's a known issue that reindex all might not work properly" + ) def test_reindex_with_rules(self): # Given an existing index defined with settings class WebsiteIndex(AlgoliaIndex): - settings = {'hitsPerPage': 42} + settings = {"hitsPerPage": 42} self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) underlying_index = self.index._AlgoliaIndex__index # Given some existing query rules on the index rule = { - 'objectID': 'my-rule', - 'condition': { - 'pattern': 'some text', - 'anchoring': 'is' - }, - 'consequence': { - 'params': { - 'query': 'other text' - } - } + "objectID": "my-rule", + "condition": {"pattern": "some text", "anchoring": "is"}, + "consequence": {"params": {"query": "other text"}}, } underlying_index.save_rule(rule).wait() @@ -277,17 +286,24 @@ def remove_metadata(rule): self.assertEqual(len(rules), 1, "There should only be one rule") self.assertIn(rule, rules, "The existing rule should be kept over reindex") - @unittest.skip(reason="FIXME: it's a known issue that reindex all might not work properly") + @unittest.skip( + reason="FIXME: it's a known issue that reindex all might not work properly" + ) def test_reindex_with_synonyms(self): # Given an existing index defined with settings class WebsiteIndex(AlgoliaIndex): - settings = {'hitsPerPage': 42} + settings = {"hitsPerPage": 42} self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) underlying_index = self.index._AlgoliaIndex__index # Given some existing synonyms on the index - synonym = {'objectID': 'street', 'type': 'altCorrection1', 'word': 'Street', 'corrections': ['St']} + synonym = { + "objectID": "street", + "type": "altCorrection1", + "word": "Street", + "corrections": ["St"], + } underlying_index.save_synonyms([synonym]).wait() # When reindexing with no settings on the instance @@ -298,7 +314,9 @@ class WebsiteIndex(AlgoliaIndex): # Expect the synonyms to be kept across reindex synonyms = [s for s in underlying_index.browse_synonyms()] self.assertEqual(len(synonyms), 1, "There should only be one synonym") - self.assertIn(synonym, synonyms, "The existing synonym should be kept over reindex") + self.assertIn( + synonym, synonyms, "The existing synonym should be kept over reindex" + ) def apply_some_settings(self, index): """ @@ -308,358 +326,392 @@ def apply_some_settings(self, index): :return: the new settings """ # When reindexing with settings on the instance - old_hpp = index.settings['hitsPerPage'] if 'hitsPerPage' in index.settings else None - index.settings['hitsPerPage'] = 42 + old_hpp = ( + index.settings["hitsPerPage"] if "hitsPerPage" in index.settings else None + ) + index.settings["hitsPerPage"] = 42 index.reindex_all() - index.settings['hitsPerPage'] = old_hpp + index.settings["hitsPerPage"] = old_hpp time.sleep(10) # FIXME: Refactor reindex_all to return taskID index_settings = index.get_settings() # Expect the instance's settings to be applied at reindex - self.assertEqual(index_settings['hitsPerPage'], 42, - "An index whose model has settings should apply those at reindex") + self.assertEqual( + index_settings["hitsPerPage"], + 42, + "An index whose model has settings should apply those at reindex", + ) return index_settings def test_custom_objectID(self): class UserIndex(AlgoliaIndex): - custom_objectID = 'username' + custom_objectID = "username" self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertEqual(obj['objectID'], 'algolia') + self.assertEqual(obj["objectID"], "algolia") def test_custom_objectID_property(self): class UserIndex(AlgoliaIndex): - custom_objectID = 'reverse_username' + custom_objectID = "reverse_username" self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertEqual(obj['objectID'], 'ailogla') + self.assertEqual(obj["objectID"], "ailogla") def test_invalid_custom_objectID(self): class UserIndex(AlgoliaIndex): - custom_objectID = 'uid' + custom_objectID = "uid" with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_geo_fields(self): class UserIndex(AlgoliaIndex): - geo_field = 'location' + geo_field = "location" self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertEqual(obj['_geoloc'], {'lat': 123, 'lng': -42.24}) + self.assertEqual(obj["_geoloc"], {"lat": 123, "lng": -42.24}) def test_several_geo_fields(self): class ExampleIndex(AlgoliaIndex): - geo_field = 'geolocations' + geo_field = "geolocations" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.example) - self.assertEqual(obj['_geoloc'], [ - {'lat': 10.3, 'lng': -20.0}, - {'lat': 22.3, 'lng': 10.0}, - ]) + self.assertEqual( + obj["_geoloc"], + [ + {"lat": 10.3, "lng": -20.0}, + {"lat": 22.3, "lng": 10.0}, + ], + ) def test_geo_fields_already_formatted(self): class ExampleIndex(AlgoliaIndex): - geo_field = 'geolocations' + geo_field = "geolocations" - self.example.locations = {'lat': 10.3, 'lng': -20.0} + self.example.locations = {"lat": 10.3, "lng": -20.0} self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.example) - self.assertEqual(obj['_geoloc'], {'lat': 10.3, 'lng': -20.0}) + self.assertEqual(obj["_geoloc"], {"lat": 10.3, "lng": -20.0}) def test_none_geo_fields(self): class ExampleIndex(AlgoliaIndex): - geo_field = 'location' + geo_field = "location" Example.location = lambda x: None self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.example) - self.assertIsNone(obj.get('_geoloc')) + self.assertIsNone(obj.get("_geoloc")) def test_invalid_geo_fields(self): class UserIndex(AlgoliaIndex): - geo_field = 'position' + geo_field = "position" with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_tags(self): class UserIndex(AlgoliaIndex): - tags = 'permissions' + tags = "permissions" self.index = UserIndex(User, self.client, settings.ALGOLIA) # Test the users' tag individually obj = self.index.get_raw_record(self.user) - self.assertListEqual(obj['_tags'], ['read', 'write', 'admin']) + self.assertListEqual(obj["_tags"], ["read", "write", "admin"]) obj = self.index.get_raw_record(self.contributor) - self.assertListEqual(obj['_tags'], ['contribute', 'follow']) + self.assertListEqual(obj["_tags"], ["contribute", "follow"]) def test_invalid_tags(self): class UserIndex(AlgoliaIndex): - tags = 'tags' + tags = "tags" with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_one_field(self): class UserIndex(AlgoliaIndex): - fields = 'name' + fields = "name" self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertIn('name', obj) - self.assertNotIn('username', obj) - self.assertNotIn('bio', obj) - self.assertNotIn('followers_count', obj) - self.assertNotIn('following_count', obj) - self.assertNotIn('_lat', obj) - self.assertNotIn('_lng', obj) - self.assertNotIn('_permissions', obj) - self.assertNotIn('location', obj) - self.assertNotIn('_geoloc', obj) - self.assertNotIn('permissions', obj) - self.assertNotIn('_tags', obj) + self.assertIn("name", obj) + self.assertNotIn("username", obj) + self.assertNotIn("bio", obj) + self.assertNotIn("followers_count", obj) + self.assertNotIn("following_count", obj) + self.assertNotIn("_lat", obj) + self.assertNotIn("_lng", obj) + self.assertNotIn("_permissions", obj) + self.assertNotIn("location", obj) + self.assertNotIn("_geoloc", obj) + self.assertNotIn("permissions", obj) + self.assertNotIn("_tags", obj) self.assertEqual(len(obj), 2) def test_multiple_fields(self): class UserIndex(AlgoliaIndex): - fields = ('name', 'username', 'bio') + fields = ("name", "username", "bio") self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertIn('name', obj) - self.assertIn('username', obj) - self.assertIn('bio', obj) - self.assertNotIn('followers_count', obj) - self.assertNotIn('following_count', obj) - self.assertNotIn('_lat', obj) - self.assertNotIn('_lng', obj) - self.assertNotIn('_permissions', obj) - self.assertNotIn('location', obj) - self.assertNotIn('_geoloc', obj) - self.assertNotIn('permissions', obj) - self.assertNotIn('_tags', obj) + self.assertIn("name", obj) + self.assertIn("username", obj) + self.assertIn("bio", obj) + self.assertNotIn("followers_count", obj) + self.assertNotIn("following_count", obj) + self.assertNotIn("_lat", obj) + self.assertNotIn("_lng", obj) + self.assertNotIn("_permissions", obj) + self.assertNotIn("location", obj) + self.assertNotIn("_geoloc", obj) + self.assertNotIn("permissions", obj) + self.assertNotIn("_tags", obj) self.assertEqual(len(obj), 4) def test_fields_with_custom_name(self): # tuple syntax class UserIndex(AlgoliaIndex): - fields = ('name', ('username', 'login'), 'bio') + fields = ("name", ("username", "login"), "bio") self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertIn('name', obj) - self.assertNotIn('username', obj) - self.assertIn('login', obj) - self.assertEqual(obj['login'], 'algolia') - self.assertIn('bio', obj) - self.assertNotIn('followers_count', obj) - self.assertNotIn('following_count', obj) - self.assertNotIn('_lat', obj) - self.assertNotIn('_lng', obj) - self.assertNotIn('_permissions', obj) - self.assertNotIn('location', obj) - self.assertNotIn('_geoloc', obj) - self.assertNotIn('permissions', obj) - self.assertNotIn('_tags', obj) + self.assertIn("name", obj) + self.assertNotIn("username", obj) + self.assertIn("login", obj) + self.assertEqual(obj["login"], "algolia") + self.assertIn("bio", obj) + self.assertNotIn("followers_count", obj) + self.assertNotIn("following_count", obj) + self.assertNotIn("_lat", obj) + self.assertNotIn("_lng", obj) + self.assertNotIn("_permissions", obj) + self.assertNotIn("location", obj) + self.assertNotIn("_geoloc", obj) + self.assertNotIn("permissions", obj) + self.assertNotIn("_tags", obj) self.assertEqual(len(obj), 4) # list syntax class UserIndex(AlgoliaIndex): - fields = ('name', ['username', 'login'], 'bio') + fields = ("name", ["username", "login"], "bio") self.index = UserIndex(User, self.client, settings.ALGOLIA) obj = self.index.get_raw_record(self.user) - self.assertIn('name', obj) - self.assertNotIn('username', obj) - self.assertIn('login', obj) - self.assertEqual(obj['login'], 'algolia') - self.assertIn('bio', obj) - self.assertNotIn('followers_count', obj) - self.assertNotIn('following_count', obj) - self.assertNotIn('_lat', obj) - self.assertNotIn('_lng', obj) - self.assertNotIn('_permissions', obj) - self.assertNotIn('location', obj) - self.assertNotIn('_geoloc', obj) - self.assertNotIn('permissions', obj) - self.assertNotIn('_tags', obj) + self.assertIn("name", obj) + self.assertNotIn("username", obj) + self.assertIn("login", obj) + self.assertEqual(obj["login"], "algolia") + self.assertIn("bio", obj) + self.assertNotIn("followers_count", obj) + self.assertNotIn("following_count", obj) + self.assertNotIn("_lat", obj) + self.assertNotIn("_lng", obj) + self.assertNotIn("_permissions", obj) + self.assertNotIn("location", obj) + self.assertNotIn("_geoloc", obj) + self.assertNotIn("permissions", obj) + self.assertNotIn("_tags", obj) self.assertEqual(len(obj), 4) def test_invalid_fields(self): class UserIndex(AlgoliaIndex): - fields = ('name', 'color') + fields = ("name", "color") with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_invalid_fields_syntax(self): class UserIndex(AlgoliaIndex): - fields = {'name': 'user_name'} + fields = {"name": "user_name"} with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_invalid_named_fields_syntax(self): class UserIndex(AlgoliaIndex): - fields = ('name', {'username': 'login'}) + fields = ("name", {"username": "login"}) with self.assertRaises(AlgoliaIndexError): UserIndex(User, self.client, settings.ALGOLIA) def test_get_raw_record_with_update_fields(self): class UserIndex(AlgoliaIndex): - fields = ('name', 'username', ['bio', 'description']) + fields = ("name", "username", ["bio", "description"]) self.index = UserIndex(User, self.client, settings.ALGOLIA) - obj = self.index.get_raw_record(self.user, - update_fields=('name', 'bio')) - self.assertIn('name', obj) - self.assertNotIn('username', obj) - self.assertNotIn('bio', obj) - self.assertIn('description', obj) - self.assertEqual(obj['description'], 'Milliseconds matter') - self.assertNotIn('followers_count', obj) - self.assertNotIn('following_count', obj) - self.assertNotIn('_lat', obj) - self.assertNotIn('_lng', obj) - self.assertNotIn('_permissions', obj) - self.assertNotIn('location', obj) - self.assertNotIn('_geoloc', obj) - self.assertNotIn('permissions', obj) - self.assertNotIn('_tags', obj) + obj = self.index.get_raw_record(self.user, update_fields=("name", "bio")) + self.assertIn("name", obj) + self.assertNotIn("username", obj) + self.assertNotIn("bio", obj) + self.assertIn("description", obj) + self.assertEqual(obj["description"], "Milliseconds matter") + self.assertNotIn("followers_count", obj) + self.assertNotIn("following_count", obj) + self.assertNotIn("_lat", obj) + self.assertNotIn("_lng", obj) + self.assertNotIn("_permissions", obj) + self.assertNotIn("location", obj) + self.assertNotIn("_geoloc", obj) + self.assertNotIn("permissions", obj) + self.assertNotIn("_tags", obj) self.assertEqual(len(obj), 3) def test_should_index_method(self): class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'has_name' + fields = "name" + should_index = "has_name" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertTrue(self.index._should_index(self.example), - "We should index an instance when should_index(instance) returns True") + self.assertTrue( + self.index._should_index(self.example), + "We should index an instance when should_index(instance) returns True", + ) instance_should_not = Example(name=None) - self.assertFalse(self.index._should_index(instance_should_not), - "We should not index an instance when should_index(instance) returns False") + self.assertFalse( + self.index._should_index(instance_should_not), + "We should not index an instance when should_index(instance) returns False", + ) def test_should_index_unbound(self): class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'static_should_index' + fields = "name" + should_index = "static_should_index" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertTrue(self.index._should_index(self.example), - "We should index an instance when should_index() returns True") + self.assertTrue( + self.index._should_index(self.example), + "We should index an instance when should_index() returns True", + ) class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'static_should_not_index' + fields = "name" + should_index = "static_should_not_index" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) instance_should_not = Example() - self.assertFalse(self.index._should_index(instance_should_not), - "We should not index an instance when should_index() returns False") + self.assertFalse( + self.index._should_index(instance_should_not), + "We should not index an instance when should_index() returns False", + ) def test_should_index_attr(self): class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'index_me' + fields = "name" + should_index = "index_me" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertTrue(self.index._should_index(self.example), - "We should index an instance when its should_index attr is True") + self.assertTrue( + self.index._should_index(self.example), + "We should index an instance when its should_index attr is True", + ) instance_should_not = Example() instance_should_not.index_me = False - self.assertFalse(self.index._should_index(instance_should_not), - "We should not index an instance when its should_index attr is False") + self.assertFalse( + self.index._should_index(instance_should_not), + "We should not index an instance when its should_index attr is False", + ) class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'category' + fields = "name" + should_index = "category" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - with self.assertRaises(AlgoliaIndexError, msg="We should raise when the should_index attr is not boolean"): + with self.assertRaises( + AlgoliaIndexError, + msg="We should raise when the should_index attr is not boolean", + ): self.index._should_index(self.example) def test_should_index_field(self): class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'is_admin' + fields = "name" + should_index = "is_admin" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertTrue(self.index._should_index(self.example), - "We should index an instance when its should_index field is True") + self.assertTrue( + self.index._should_index(self.example), + "We should index an instance when its should_index field is True", + ) instance_should_not = Example() instance_should_not.is_admin = False - self.assertFalse(self.index._should_index(instance_should_not), - "We should not index an instance when its should_index field is False") + self.assertFalse( + self.index._should_index(instance_should_not), + "We should not index an instance when its should_index field is False", + ) class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'name' + fields = "name" + should_index = "name" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - with self.assertRaises(AlgoliaIndexError, msg="We should raise when the should_index field is not boolean"): + with self.assertRaises( + AlgoliaIndexError, + msg="We should raise when the should_index field is not boolean", + ): self.index._should_index(self.example) def test_should_index_property(self): class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'property_should_index' + fields = "name" + should_index = "property_should_index" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertTrue(self.index._should_index(self.example), - "We should index an instance when its should_index property is True") + self.assertTrue( + self.index._should_index(self.example), + "We should index an instance when its should_index property is True", + ) class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'property_should_not_index' + fields = "name" + should_index = "property_should_not_index" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - self.assertFalse(self.index._should_index(self.example), - "We should not index an instance when its should_index property is False") + self.assertFalse( + self.index._should_index(self.example), + "We should not index an instance when its should_index property is False", + ) class ExampleIndex(AlgoliaIndex): - fields = 'name' - should_index = 'property_string' + fields = "name" + should_index = "property_string" self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) - with self.assertRaises(AlgoliaIndexError, msg="We should raise when the should_index property is not boolean"): + with self.assertRaises( + AlgoliaIndexError, + msg="We should raise when the should_index property is not boolean", + ): self.index._should_index(self.example) def test_save_record_should_index_boolean(self): website = Website.objects.create( - name='Algolia', - url='https://algolia.com', - is_online=True + name="Algolia", url="https://algolia.com", is_online=True ) self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): settings = { - 'replicas': [ - self.index.index_name + '_name_asc', - self.index.index_name + '_name_desc' + "replicas": [ + self.index.index_name + "_name_asc", + self.index.index_name + "_name_desc", ] } - should_index = 'is_online' + should_index = "is_online" self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.save_record(website) def test_cyrillic(self): class CyrillicIndex(AlgoliaIndex): - fields = ['bio', 'name'] + fields = ["bio", "name"] settings = { - 'searchableAttributes': ['name', 'bio'], + "searchableAttributes": ["name", "bio"], } index_name = "test_cyrillic" @@ -668,5 +720,7 @@ class CyrillicIndex(AlgoliaIndex): self.index = CyrillicIndex(User, self.client, settings.ALGOLIA) self.index.save_record(self.user).wait() result = self.index.raw_search("крупнейших") - self.assertEqual(result['nbHits'], 1, "Search should return one result") - self.assertEqual(result['hits'][0]['name'], 'Algolia', "The result should be self.user") + self.assertEqual(result["nbHits"], 1, "Search should return one result") + self.assertEqual( + result["hits"][0]["name"], "Algolia", "The result should be self.user" + ) diff --git a/tests/test_signal.py b/tests/test_signal.py index e576bd3..f09b659 100644 --- a/tests/test_signal.py +++ b/tests/test_signal.py @@ -1,7 +1,7 @@ import time from mock import patch, call, ANY -from django.test import TestCase, override_settings +from django.test import TestCase from algoliasearch_django import algolia_engine from algoliasearch_django import get_adapter @@ -14,7 +14,6 @@ class SignalTestCase(TestCase): - @classmethod def tearDownClass(cls): get_adapter(Website).delete() @@ -23,7 +22,7 @@ def tearDown(self): clear_index(Website) def test_save_signal(self): - with patch.object(algolia_engine, 'save_record') as mocked_save_record: + with patch.object(algolia_engine, "save_record") as mocked_save_record: websites = WebsiteFactory.create_batch(3) mocked_save_record.assert_has_calls( @@ -35,7 +34,7 @@ def test_save_signal(self): sender=ANY, signal=ANY, update_fields=None, - using=ANY + using=ANY, ) for website in websites ] @@ -44,31 +43,26 @@ def test_save_signal(self): def test_delete_signal(self): websites = WebsiteFactory.create_batch(3) - with patch.object(algolia_engine, 'delete_record') as mocked_delete_record: + with patch.object(algolia_engine, "delete_record") as mocked_delete_record: websites[0].delete() websites[1].delete() - mocked_delete_record.assert_has_calls( - [ - call(websites[0]), - call(websites[1]) - ] - ) + mocked_delete_record.assert_has_calls([call(websites[0]), call(websites[1])]) def test_update_records(self): - Website.objects.create(name='Algolia', url='https://www.algolia.com') - Website.objects.create(name='Google', url='https://www.google.com') - Website.objects.create(name='Facebook', url='https://www.facebook.com') - Website.objects.create(name='Facebook', url='https://www.facebook.fr') - Website.objects.create(name='Facebook', url='https://fb.com') + Website.objects.create(name="Algolia", url="https://www.algolia.com") + Website.objects.create(name="Google", url="https://www.google.com") + Website.objects.create(name="Facebook", url="https://www.facebook.com") + Website.objects.create(name="Facebook", url="https://www.facebook.fr") + Website.objects.create(name="Facebook", url="https://fb.com") - qs = Website.objects.filter(name='Facebook') - update_records(Website, qs, url='https://facebook.com') + qs = Website.objects.filter(name="Facebook") + update_records(Website, qs, url="https://facebook.com") time.sleep(10) - qs.update(url='https://facebook.com') + qs.update(url="https://facebook.com") time.sleep(10) - result = raw_search(Website, 'Facebook') - self.assertEqual(result['nbHits'], qs.count()) - for res, url in zip(result['hits'], qs.values_list('url', flat=True)): - self.assertEqual(res['url'], url) + result = raw_search(Website, "Facebook") + self.assertEqual(result["nbHits"], qs.count()) + for res, url in zip(result["hits"], qs.values_list("url", flat=True)): + self.assertEqual(res["url"], url) diff --git a/tox.ini b/tox.ini index cc5123d..ddc98c9 100644 --- a/tox.ini +++ b/tox.ini @@ -26,9 +26,11 @@ commands = python runtests.py [versions] twine = >=5.1,<6.0 wheel = >=0.45,<1.0 +ruff = >=0.7.4,<1.0 +pyright = >=1.1.389,<2.0 [testenv:coverage] -basepython = python3.8 +basepython = python3.11 deps = coverage passenv = ALGOLIA* @@ -37,7 +39,7 @@ commands = coverage report [testenv:coveralls] -basepython = python3.8 +basepython = python3.11 deps = coverage coveralls @@ -49,7 +51,7 @@ commands = coveralls [testenv:release] -basepython = python3.8 +basepython = python3.11 deps = twine {[versions]twine} wheel {[versions]wheel} @@ -60,3 +62,13 @@ commands = python setup.py sdist bdist_wheel twine check dist/* twine upload -u {env:PYPI_USER} -p {env:PYPI_PASSWORD} --repository-url https://upload.pypi.org/legacy/ dist/* + +[testenv:lint] +basepython = python3.11 +deps = + ruff {[versions]ruff} + pyright {[versions]pyright} +commands = + pip3 freeze > requirements.txt + ruff check --fix --unsafe-fixes + ruff format . From 5907b3efdca6ff63aa6df9c6e0dbb2f281fa14cc Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 20 Nov 2024 16:57:04 +0100 Subject: [PATCH 07/35] chore: setup pyright --- algoliasearch_django/models.py | 2 +- tox.ini | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 63e7cb3..e48da08 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -5,7 +5,7 @@ from itertools import chain import logging -from algoliasearch.exceptions import AlgoliaException +from algoliasearch.http.exceptions import AlgoliaException from django.db.models.query_utils import DeferredAttribute from .settings import DEBUG diff --git a/tox.ini b/tox.ini index ddc98c9..d835757 100644 --- a/tox.ini +++ b/tox.ini @@ -72,3 +72,4 @@ commands = pip3 freeze > requirements.txt ruff check --fix --unsafe-fixes ruff format . + pyright From 523958e563d4774b2313cebfe19ff6d5f9ee917b Mon Sep 17 00:00:00 2001 From: shortcuts Date: Thu, 21 Nov 2024 15:58:01 +0100 Subject: [PATCH 08/35] fix: use algoliasearch v4 syntax --- algoliasearch_django/decorators.py | 25 +----- algoliasearch_django/models.py | 115 +++++++++++++++++---------- algoliasearch_django/registration.py | 11 ++- tests/test_commands.py | 4 +- tests/test_engine.py | 12 +-- 5 files changed, 86 insertions(+), 81 deletions(-) diff --git a/algoliasearch_django/decorators.py b/algoliasearch_django/decorators.py index a85e268..a8056fb 100644 --- a/algoliasearch_django/decorators.py +++ b/algoliasearch_django/decorators.py @@ -1,9 +1,5 @@ -try: - # ContextDecorator was introduced in Python 3.2 - from contextlib import ContextDecorator -except ImportError: - ContextDecorator = None -from functools import WRAPPER_ASSIGNMENTS, wraps +from contextlib import ContextDecorator +from functools import WRAPPER_ASSIGNMENTS from django.db.models.signals import post_save, pre_delete @@ -19,23 +15,6 @@ def available_attrs(fn): return WRAPPER_ASSIGNMENTS -if ContextDecorator is None: - # ContextDecorator was introduced in Python 3.2 - # See https://docs.python.org/3/library/contextlib.html#contextlib.ContextDecorator - class ContextDecorator: - """ - A base class that enables a context manager to also be used as a decorator. - """ - - def __call__(self, func): - @wraps(func, assigned=available_attrs(func)) - def inner(*args, **kwargs): - with self: - return func(*args, **kwargs) - - return inner - - def register(model): """ Register the given model class and wrapped AlgoliaIndex class with the Algolia engine: diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index e48da08..38b62a0 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -6,6 +6,9 @@ import logging from algoliasearch.http.exceptions import AlgoliaException +from algoliasearch.search.models.operation_index_params import OperationIndexParams +from algoliasearch.search.models.operation_type import OperationType +from algoliasearch.search.models.search_params_object import SearchParamsObject from django.db.models.query_utils import DeferredAttribute from .settings import DEBUG @@ -71,7 +74,7 @@ class AlgoliaIndex(object): def __init__(self, model, client, settings): """Initializes the index.""" - self.__init_index(client, model, settings) + self.__init_index(model, settings) self.model = model self.__client = client @@ -170,7 +173,7 @@ def __init__(self, model, client, settings): ) ) - def __init_index(self, client, model, settings): + def __init_index(self, model, settings): if not self.index_name: self.index_name = model.__name__ @@ -189,9 +192,6 @@ def __init_index(self, client, model, settings): self.tmp_index_name = tmp_index_name - self.__index = client.init_index(self.index_name) - self.__tmp_index = client.init_index(self.tmp_index_name) - @staticmethod def _validate_geolocation(geolocation): """ @@ -315,10 +315,14 @@ def save_record(self, instance, update_fields=None, **kwargs): try: if update_fields: obj = self.get_raw_record(instance, update_fields=update_fields) - result = self.__index.partial_update_object(obj) + result = self.__client.partial_update_object( + self.index_name, obj.get("objectID"), obj + ) else: obj = self.get_raw_record(instance) - result = self.__index.save_object(obj) + result = self.__client.save_object( + self.index_name, obj.get("objectID"), obj + ) logger.info("SAVE %s FROM %s", obj["objectID"], self.model) return result except AlgoliaException as e: @@ -333,7 +337,7 @@ def delete_record(self, instance): """Deletes the record.""" objectID = self.objectID(instance) try: - self.__index.delete_object(objectID) + self.__client.delete_object(self.index_name, objectID) logger.info("DELETE %s FROM %s", objectID, self.model) except AlgoliaException as e: if DEBUG: @@ -369,19 +373,21 @@ def update_records(self, qs, batch_size=1000, **kwargs): batch.append(dict(tmp)) if len(batch) >= batch_size: - self.__index.partial_update_objects(batch) + self.__client.partial_update_objects(self.index_name, batch) batch = [] if len(batch) > 0: - self.__index.partial_update_objects(batch) + self.__client.partial_update_objects(self.index_name, batch) def raw_search(self, query="", params=None): """Performs a search query and returns the parsed JSON.""" if params is None: - params = {} + params = SearchParamsObject() + + params.query = query try: - return self.__index.search(query, params) + return self.__client.search_single_index(self.index_name, params) except AlgoliaException as e: if DEBUG: raise e @@ -392,7 +398,7 @@ def get_settings(self): """Returns the settings of the index.""" try: logger.info("GET SETTINGS ON %s", self.index_name) - return self.__index.get_settings() + return self.__client.get_settings(self.index_name) except AlgoliaException as e: if DEBUG: raise e @@ -405,7 +411,7 @@ def set_settings(self): return try: - self.__index.set_settings(self.settings) + self.__client.set_settings(self.index_name, self.settings) logger.info("APPLY SETTINGS ON %s", self.index_name) except AlgoliaException as e: if DEBUG: @@ -416,7 +422,7 @@ def set_settings(self): def clear_objects(self): """Clears all objects of an index.""" try: - self.__index.clear_objects() + self.__client.clear_objects(self.index_name) logger.info("CLEAR INDEX %s", self.index_name) except AlgoliaException as e: if DEBUG: @@ -430,7 +436,7 @@ def clear_index(self): def wait_task(self, task_id): try: - self.__index.wait_task(task_id) + self.__client.wait_for_task(self.index_name, task_id) logger.info("WAIT TASK %s", self.index_name) except AlgoliaException as e: if DEBUG: @@ -439,9 +445,9 @@ def wait_task(self, task_id): logger.warning("%s NOT WAIT: %s", self.model, e) def delete(self): - self.__index.delete() - if self.__tmp_index: - self.__tmp_index.delete() + self.__client.delete_index(self.index_name) + if self.tmp_index_name: + self.__client.delete_index(self.tmp_index_name) def reindex_all(self, batch_size=1000): """ @@ -469,6 +475,11 @@ def reindex_all(self, batch_size=1000): else: raise e # Unexpected error while getting settings try: + should_keep_replicas = False + should_keep_slaves = False + replicas = None + slaves = None + if self.settings: replicas = self.settings.get("replicas", None) slaves = self.settings.get("slaves", None) @@ -483,22 +494,31 @@ def reindex_all(self, batch_size=1000): self.settings["slaves"] = [] logger.debug("REMOVE SLAVES FROM SETTINGS") - self.__tmp_index.set_settings(self.settings).wait() + set_settings_response = self.__client.set_settings( + self.tmp_index_name, self.settings + ) + self.__client.wait_for_task( + self.tmp_index_name, set_settings_response.task_id + ) logger.debug("APPLY SETTINGS ON %s_tmp", self.index_name) + rules = [] - synonyms = [] - for r in self.__index.browse_rules(): - rules.append(r) - for s in self.__index.browse_synonyms(): - synonyms.append(s) + self.__client.browse_rules( + self.index_name, lambda _resp: rules.append(_resp) + ) if len(rules): logger.debug("Got rules for index %s: %s", self.index_name, rules) should_keep_rules = True + + synonyms = [] + self.__client.browse_synonyms( + self.index_name, lambda _resp: synonyms.append(_resp) + ) if len(synonyms): logger.debug("Got synonyms for index %s: %s", self.index_name, rules) should_keep_synonyms = True - self.__tmp_index.clear_objects() + self.__client.clear_objects(self.tmp_index_name) logger.debug("CLEAR INDEX %s_tmp", self.index_name) counts = 0 @@ -515,18 +535,23 @@ def reindex_all(self, batch_size=1000): batch.append(self.get_raw_record(instance)) if len(batch) >= batch_size: - self.__tmp_index.save_objects(batch) + self.__client.save_objects(self.tmp_index_name, batch, True) logger.info( - "SAVE %d OBJECTS TO %s_tmp", len(batch), self.index_name + "SAVE %d OBJECTS TO %s", len(batch), self.tmp_index_name ) batch = [] counts += 1 if len(batch) > 0: - self.__tmp_index.save_objects(batch) - logger.info("SAVE %d OBJECTS TO %s_tmp", len(batch), self.index_name) - - self.__client.move_index(self.tmp_index_name, self.index_name) - logger.info("MOVE INDEX %s_tmp TO %s", self.index_name, self.index_name) + self.__client.save_objects(self.tmp_index_name, batch, True) + logger.info("SAVE %d OBJECTS TO %s", len(batch), self.tmp_index_name) + + self.__client.operation_index( + self.tmp_index_name, + OperationIndexParams( + operation=OperationType.MOVE, destination=self.index_name + ), + ) + logger.info("MOVE INDEX %s TO %s", self.tmp_index_name, self.index_name) if self.settings: if should_keep_replicas: @@ -536,24 +561,30 @@ def reindex_all(self, batch_size=1000): self.settings["slaves"] = slaves logger.debug("RESTORE SLAVES") if should_keep_replicas or should_keep_slaves: - self.__index.set_settings(self.settings) + self.__client.set_settings(self.index_name, self.settings) if should_keep_rules: - response = self.__index.save_rules( - rules, {"forwardToReplicas": True} + save_rules_response = self.__client.save_rules( + self.index_name, rules, True + ) + self.__client.wait_for_task( + self.index_name, save_rules_response.task_id ) - response.wait() logger.info( - "Saved rules for index %s with response: {}".format(response), + "Saved rules for index %s with response: {}".format( + save_rules_response + ), self.index_name, ) if should_keep_synonyms: - response = self.__index.save_synonyms( - synonyms, {"forwardToReplicas": True} + save_synonyms_response = self.__client.save_synonyms( + self.index_name, synonyms, True + ) + self.__client.wait_for_task( + self.index_name, save_synonyms_response.task_id ) - response.wait() logger.info( "Saved synonyms for index %s with response: {}".format( - response + save_synonyms_response ), self.index_name, ) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index bd93ac9..9b1a79c 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -3,8 +3,7 @@ from django.db.models.signals import post_save from django.db.models.signals import pre_delete -from algoliasearch.search_client import SearchClient -from algoliasearch.user_agent import UserAgent +from algoliasearch.search.client import SearchClientSync from .models import AlgoliaIndex from .settings import SETTINGS @@ -13,9 +12,6 @@ logger = logging.getLogger(__name__) -UserAgent.add("Algolia for Django", VERSION) -UserAgent.add("Django", django_version()) - class AlgoliaEngineError(Exception): """Something went wrong with Algolia Engine.""" @@ -39,7 +35,10 @@ def __init__(self, settings=SETTINGS): self.__settings = settings self.__registered_models = {} - self.client = SearchClient.create(app_id, api_key) + self.client = SearchClientSync(app_id, api_key) + self.client._config.user_agent.add("Algolia for Django", VERSION).add( + "Django", django_version() + ) def is_registered(self, model): """Checks whether the given models is registered with Algolia engine""" diff --git a/tests/test_commands.py b/tests/test_commands.py index 6f124a3..964d642 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -16,8 +16,8 @@ def tearDownClass(cls): user_index_name = get_adapter(User).index_name website_index_name = get_adapter(Website).index_name - algolia_engine.client.init_index(user_index_name).delete() - algolia_engine.client.init_index(website_index_name).delete() + algolia_engine.client.delete_index(user_index_name) + algolia_engine.client.delete_index(website_index_name) def setUp(self): # Create some records diff --git a/tests/test_engine.py b/tests/test_engine.py index c2d6d15..0e417a5 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -1,10 +1,8 @@ import six -import re from django.conf import settings from django.test import TestCase -from algoliasearch.user_agent import UserAgent from django import get_version as django_version from algoliasearch_django.version import VERSION from algoliasearch_django import algolia_engine @@ -34,12 +32,10 @@ def test_init_exception(self): AlgoliaEngine(settings=settings.ALGOLIA) def test_user_agent(self): - user_agent = UserAgent.get() - - parts = re.split("\s*;\s*", user_agent) - - self.assertIn("Django (%s)" % django_version(), parts) - self.assertIn("Algolia for Django (%s)" % VERSION, parts) + self.assertIn( + "Algolia for Django (%s); Django (%s)" % VERSION % django_version(), + self.engine.client._config.user_agent.get(), + ) def test_auto_discover_indexes(self): """Test that the `index` module was auto-discovered and the models registered""" From f6dd4e7087f0775f858e3993b98df5de297d0b9c Mon Sep 17 00:00:00 2001 From: shortcuts Date: Thu, 21 Nov 2024 17:21:05 +0100 Subject: [PATCH 09/35] fix: some types --- algoliasearch_django/registration.py | 5 ++--- tests/models.py | 12 ++++++------ 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index 9b1a79c..026d331 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -36,9 +36,8 @@ def __init__(self, settings=SETTINGS): self.__registered_models = {} self.client = SearchClientSync(app_id, api_key) - self.client._config.user_agent.add("Algolia for Django", VERSION).add( - "Django", django_version() - ) + self.client.add_user_agent("Algolia for Django", VERSION) + self.client.add_user_agent("Django", django_version()) def is_registered(self, model): """Checks whether the given models is registered with Algolia engine""" diff --git a/tests/models.py b/tests/models.py index f216d49..679d4d7 100644 --- a/tests/models.py +++ b/tests/models.py @@ -5,10 +5,10 @@ class User(models.Model): name = models.CharField(max_length=30) username = models.CharField(max_length=30, unique=True) bio = models.CharField(max_length=140, blank=True) - followers_count = models.BigIntegerField(default=0) - following_count = models.BigIntegerField(default=0) - _lat = models.FloatField(default=0) - _lng = models.FloatField(default=0) + followers_count = models.BigIntegerField(0) + following_count = models.BigIntegerField(0) + _lat = models.FloatField(0) + _lng = models.FloatField(0) _permissions = models.CharField(max_length=30, blank=True) @property @@ -25,7 +25,7 @@ def permissions(self): class Website(models.Model): name = models.CharField(max_length=100) url = models.URLField() - is_online = models.BooleanField(default=False) + is_online = models.BooleanField(False) class Example(models.Model): @@ -34,7 +34,7 @@ class Example(models.Model): address = models.CharField(max_length=200) lat = models.FloatField() lng = models.FloatField() - is_admin = models.BooleanField(default=False) + is_admin = models.BooleanField(False) category = [] locations = [] index_me = True From 1e0054b4b4fa82537f1718463c90a72c615d2afc Mon Sep 17 00:00:00 2001 From: shortcuts Date: Fri, 22 Nov 2024 11:47:22 +0100 Subject: [PATCH 10/35] fix: remove user agent for now --- algoliasearch_django/registration.py | 6 ++---- tests/test_engine.py | 12 +++++------- 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index 026d331..5ac0ec0 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -7,8 +7,6 @@ from .models import AlgoliaIndex from .settings import SETTINGS -from .version import VERSION -from django import get_version as django_version logger = logging.getLogger(__name__) @@ -36,8 +34,8 @@ def __init__(self, settings=SETTINGS): self.__registered_models = {} self.client = SearchClientSync(app_id, api_key) - self.client.add_user_agent("Algolia for Django", VERSION) - self.client.add_user_agent("Django", django_version()) + # self.client.add_user_agent("Algolia for Django", VERSION) + # self.client.add_user_agent("Django", django_version()) def is_registered(self, model): """Checks whether the given models is registered with Algolia engine""" diff --git a/tests/test_engine.py b/tests/test_engine.py index 0e417a5..9a41b43 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -3,8 +3,6 @@ from django.conf import settings from django.test import TestCase -from django import get_version as django_version -from algoliasearch_django.version import VERSION from algoliasearch_django import algolia_engine from algoliasearch_django import AlgoliaIndex from algoliasearch_django import AlgoliaEngine @@ -31,11 +29,11 @@ def test_init_exception(self): with self.assertRaises(AlgoliaEngineError): AlgoliaEngine(settings=settings.ALGOLIA) - def test_user_agent(self): - self.assertIn( - "Algolia for Django (%s); Django (%s)" % VERSION % django_version(), - self.engine.client._config.user_agent.get(), - ) + # def test_user_agent(self): + # self.assertIn( + # "Algolia for Django (%s); Django (%s)" % VERSION % django_version(), + # self.engine.client._config.user_agent.get(), + # ) def test_auto_discover_indexes(self): """Test that the `index` module was auto-discovered and the models registered""" From d232e1a2141ba58cc276783b9e75a0ee973c0f81 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Fri, 22 Nov 2024 16:28:08 +0100 Subject: [PATCH 11/35] fix: some tests and types --- algoliasearch_django/models.py | 114 +++++++++++++++++---------------- tests/factories.py | 3 + tests/test_commands.py | 16 +++-- tests/test_index.py | 69 ++++++++++---------- tests/test_signal.py | 10 +-- 5 files changed, 111 insertions(+), 101 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 38b62a0..9e306bc 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -260,20 +260,23 @@ def _should_index(self, instance): def _should_really_index(self, instance): """Return True if according to should_index the object should be indexed.""" + if self.should_index is None: + raise AlgoliaIndexError("{} should be defined.".format(self.should_index)) + if self._should_index_is_method: is_method = inspect.ismethod(self.should_index) try: - count_args = len(inspect.signature(self.should_index).parameters) + count_args = len(inspect.signature(self.should_index).parameters) # pyright: ignore -- should_index_is_method except AttributeError: # noinspection PyDeprecation - count_args = len(inspect.getargspec(self.should_index).args) + count_args = len(inspect.getfullargspec(self.should_index).args) if is_method or count_args == 1: # bound method, call with instance - return self.should_index(instance) + return self.should_index(instance) # pyright: ignore -- should_index_is_method else: # unbound method, simply call without arguments - return self.should_index() + return self.should_index() # pyright: ignore -- should_index_is_method else: # property/attribute/Field, evaluate as bool attr_type = type(self.should_index) @@ -312,19 +315,19 @@ def save_record(self, instance, update_fields=None, **kwargs): self.delete_record(instance) return + obj = {} try: if update_fields: obj = self.get_raw_record(instance, update_fields=update_fields) - result = self.__client.partial_update_object( - self.index_name, obj.get("objectID"), obj + self.__client.partial_update_objects( + index_name=self.index_name, objects=[obj], wait_for_tasks=True ) else: obj = self.get_raw_record(instance) - result = self.__client.save_object( - self.index_name, obj.get("objectID"), obj + self.__client.save_objects( + index_name=self.index_name, objects=[obj], wait_for_tasks=True ) logger.info("SAVE %s FROM %s", obj["objectID"], self.model) - return result except AlgoliaException as e: if DEBUG: raise e @@ -337,7 +340,9 @@ def delete_record(self, instance): """Deletes the record.""" objectID = self.objectID(instance) try: - self.__client.delete_object(self.index_name, objectID) + self.__client.delete_objects( + index_name=self.index_name, object_ids=[objectID], wait_for_tasks=True + ) logger.info("DELETE %s FROM %s", objectID, self.model) except AlgoliaException as e: if DEBUG: @@ -373,11 +378,15 @@ def update_records(self, qs, batch_size=1000, **kwargs): batch.append(dict(tmp)) if len(batch) >= batch_size: - self.__client.partial_update_objects(self.index_name, batch) + self.__client.partial_update_objects( + index_name=self.index_name, objects=batch, wait_for_tasks=True + ) batch = [] if len(batch) > 0: - self.__client.partial_update_objects(self.index_name, batch) + self.__client.partial_update_objects( + index_name=self.index_name, objects=batch, wait_for_tasks=True + ) def raw_search(self, query="", params=None): """Performs a search query and returns the parsed JSON.""" @@ -387,7 +396,7 @@ def raw_search(self, query="", params=None): params.query = query try: - return self.__client.search_single_index(self.index_name, params) + return self.__client.search_single_index(self.index_name, params).to_dict() except AlgoliaException as e: if DEBUG: raise e @@ -398,7 +407,7 @@ def get_settings(self): """Returns the settings of the index.""" try: logger.info("GET SETTINGS ON %s", self.index_name) - return self.__client.get_settings(self.index_name) + return self.__client.get_settings(self.index_name).to_dict() except AlgoliaException as e: if DEBUG: raise e @@ -411,7 +420,8 @@ def set_settings(self): return try: - self.__client.set_settings(self.index_name, self.settings) + _resp = self.__client.set_settings(self.index_name, self.settings) + self.__client.wait_for_task(self.index_name, _resp.task_id) logger.info("APPLY SETTINGS ON %s", self.index_name) except AlgoliaException as e: if DEBUG: @@ -422,7 +432,8 @@ def set_settings(self): def clear_objects(self): """Clears all objects of an index.""" try: - self.__client.clear_objects(self.index_name) + _resp = self.__client.clear_objects(self.index_name) + self.__client.wait_for_task(self.index_name, _resp.task_id) logger.info("CLEAR INDEX %s", self.index_name) except AlgoliaException as e: if DEBUG: @@ -430,10 +441,6 @@ def clear_objects(self): else: logger.warning("%s NOT CLEARED: %s", self.model, e) - def clear_index(self): - # TODO: add deprecated warning - self.clear_objects() - def wait_task(self, task_id): try: self.__client.wait_for_task(self.index_name, task_id) @@ -445,9 +452,11 @@ def wait_task(self, task_id): logger.warning("%s NOT WAIT: %s", self.model, e) def delete(self): - self.__client.delete_index(self.index_name) + _resp = self.__client.delete_index(self.index_name) + self.__client.wait_for_task(self.index_name, _resp.task_id) if self.tmp_index_name: - self.__client.delete_index(self.tmp_index_name) + _resp = self.__client.delete_index(self.tmp_index_name) + self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) def reindex_all(self, batch_size=1000): """ @@ -494,17 +503,13 @@ def reindex_all(self, batch_size=1000): self.settings["slaves"] = [] logger.debug("REMOVE SLAVES FROM SETTINGS") - set_settings_response = self.__client.set_settings( - self.tmp_index_name, self.settings - ) - self.__client.wait_for_task( - self.tmp_index_name, set_settings_response.task_id - ) + _resp = self.__client.set_settings(self.tmp_index_name, self.settings) + self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) logger.debug("APPLY SETTINGS ON %s_tmp", self.index_name) rules = [] self.__client.browse_rules( - self.index_name, lambda _resp: rules.append(_resp) + self.index_name, lambda _resp: rules.extend(_resp.hits) ) if len(rules): logger.debug("Got rules for index %s: %s", self.index_name, rules) @@ -512,20 +517,21 @@ def reindex_all(self, batch_size=1000): synonyms = [] self.__client.browse_synonyms( - self.index_name, lambda _resp: synonyms.append(_resp) + self.index_name, lambda _resp: synonyms.extend(_resp.hits) ) if len(synonyms): logger.debug("Got synonyms for index %s: %s", self.index_name, rules) should_keep_synonyms = True - self.__client.clear_objects(self.tmp_index_name) - logger.debug("CLEAR INDEX %s_tmp", self.index_name) + _resp = self.__client.clear_objects(self.tmp_index_name) + self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) + logger.debug("CLEAR INDEX %s", self.tmp_index_name) counts = 0 batch = [] - if hasattr(self, "get_queryset"): - qs = self.get_queryset() + if hasattr(self, "get_queryset") and callable(self.get_queryset): # pyright: ignore + qs = self.get_queryset() # pyright: ignore else: qs = self.model.objects.all() @@ -535,22 +541,29 @@ def reindex_all(self, batch_size=1000): batch.append(self.get_raw_record(instance)) if len(batch) >= batch_size: - self.__client.save_objects(self.tmp_index_name, batch, True) + self.__client.save_objects( + index_name=self.tmp_index_name, + objects=batch, + wait_for_tasks=True, + ) logger.info( "SAVE %d OBJECTS TO %s", len(batch), self.tmp_index_name ) batch = [] counts += 1 if len(batch) > 0: - self.__client.save_objects(self.tmp_index_name, batch, True) + self.__client.save_objects( + index_name=self.tmp_index_name, objects=batch, wait_for_tasks=True + ) logger.info("SAVE %d OBJECTS TO %s", len(batch), self.tmp_index_name) - self.__client.operation_index( + _resp = self.__client.operation_index( self.tmp_index_name, OperationIndexParams( operation=OperationType.MOVE, destination=self.index_name ), ) + self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) logger.info("MOVE INDEX %s TO %s", self.tmp_index_name, self.index_name) if self.settings: @@ -561,31 +574,20 @@ def reindex_all(self, batch_size=1000): self.settings["slaves"] = slaves logger.debug("RESTORE SLAVES") if should_keep_replicas or should_keep_slaves: - self.__client.set_settings(self.index_name, self.settings) + _resp = self.__client.set_settings(self.index_name, self.settings) + self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) if should_keep_rules: - save_rules_response = self.__client.save_rules( - self.index_name, rules, True - ) - self.__client.wait_for_task( - self.index_name, save_rules_response.task_id - ) + _resp = self.__client.save_rules(self.index_name, rules, True) + self.__client.wait_for_task(self.index_name, _resp.task_id) logger.info( - "Saved rules for index %s with response: {}".format( - save_rules_response - ), + "Saved rules for index %s with response: {}".format(_resp), self.index_name, ) if should_keep_synonyms: - save_synonyms_response = self.__client.save_synonyms( - self.index_name, synonyms, True - ) - self.__client.wait_for_task( - self.index_name, save_synonyms_response.task_id - ) + _resp = self.__client.save_synonyms(self.index_name, synonyms, True) + self.__client.wait_for_task(self.index_name, _resp.task_id) logger.info( - "Saved synonyms for index %s with response: {}".format( - save_synonyms_response - ), + "Saved synonyms for index %s with response: {}".format(_resp), self.index_name, ) return counts diff --git a/tests/factories.py b/tests/factories.py index 395e833..3f79076 100644 --- a/tests/factories.py +++ b/tests/factories.py @@ -17,6 +17,8 @@ class Meta: class UserFactory(factory.django.DjangoModelFactory): name = factory.Sequence(lambda n: "User name-{}".format(n)) username = factory.Sequence(lambda n: "User username-{}".format(n)) + following_count = 0 + followers_count = 0 _lat = factory.Faker("latitude") _lng = factory.Faker("longitude") @@ -28,6 +30,7 @@ class Meta: class WebsiteFactory(factory.django.DjangoModelFactory): name = factory.Sequence(lambda n: "Website name-{}".format(n)) url = factory.Faker("url") + is_online = False class Meta: model = Website diff --git a/tests/test_commands.py b/tests/test_commands.py index 964d642..18da393 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -21,14 +21,16 @@ def tearDownClass(cls): def setUp(self): # Create some records - User.objects.create(name="James Bond", username="jb") - User.objects.create(name="Captain America", username="captain") - User.objects.create( - name="John Snow", username="john_snow", _lat=120.2, _lng=42.1 - ) - User.objects.create( - name="Steve Jobs", username="genius", followers_count=331213 + User(name="James Bond", username="jb", followers_count=0) + User(name="Captain America", username="captain", followers_count=0) + User( + name="John Snow", + username="john_snow", + _lat=120.2, + _lng=42.1, + followers_count=0, ) + User(name="Steve Jobs", username="genius", followers_count=331213) self.out = StringIO() diff --git a/tests/test_index.py b/tests/test_index.py index c5136a3..4f491ae 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -129,8 +129,8 @@ def test_reindex_with_replicas(self): class WebsiteIndex(AlgoliaIndex): settings = { "replicas": [ - self.index.index_name + "_name_asc", - self.index.index_name + "_name_desc", + self.index.index_name + "_name_asc", # pyright: ignore + self.index.index_name + "_name_desc", # pyright: ignore ] } @@ -138,16 +138,14 @@ class WebsiteIndex(AlgoliaIndex): self.index.reindex_all() def test_reindex_with_should_index_boolean(self): - Website.objects.create( - name="Algolia", url="https://algolia.com", is_online=True - ) + Website(name="Algolia", url="https://algolia.com", is_online=True) self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): settings = { "replicas": [ - self.index.index_name + "_name_asc", - self.index.index_name + "_name_desc", + self.index.index_name + "_name_asc", # pyright: ignore + self.index.index_name + "_name_desc", # pyright: ignore ] } should_index = "is_online" @@ -173,7 +171,6 @@ class WebsiteIndex(AlgoliaIndex): # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - time.sleep(10) # FIXME: Refactor reindex_all to return taskID # Expect the former settings to be kept across reindex self.assertEqual( @@ -243,7 +240,6 @@ class WebsiteIndex(AlgoliaIndex): # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - time.sleep(10) # FIXME: Refactor reindex_all to return taskID # Expect the settings to be reset to model definition over reindex former_settings = existing_settings @@ -259,7 +255,6 @@ class WebsiteIndex(AlgoliaIndex): settings = {"hitsPerPage": 42} self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) - underlying_index = self.index._AlgoliaIndex__index # Given some existing query rules on the index rule = { @@ -268,12 +263,11 @@ class WebsiteIndex(AlgoliaIndex): "consequence": {"params": {"query": "other text"}}, } - underlying_index.save_rule(rule).wait() + self.index.__client.save_rule(self.index.index_name, rule["objectID"], rule) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - time.sleep(10) # FIXME: Refactor reindex_all to return taskID # Expect the rules to be kept across reindex def remove_metadata(rule): @@ -281,7 +275,10 @@ def remove_metadata(rule): del copy["_metadata"] return copy - rules = [r for r in underlying_index.browse_rules()] + rules = [] + self.index.__client.browse_rules( + self.index.index_name, lambda _resp: rules.extend(_resp.hits) + ) rules = list(map(remove_metadata, rules)) self.assertEqual(len(rules), 1, "There should only be one rule") self.assertIn(rule, rules, "The existing rule should be kept over reindex") @@ -295,7 +292,6 @@ class WebsiteIndex(AlgoliaIndex): settings = {"hitsPerPage": 42} self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) - underlying_index = self.index._AlgoliaIndex__index # Given some existing synonyms on the index synonym = { @@ -304,15 +300,19 @@ class WebsiteIndex(AlgoliaIndex): "word": "Street", "corrections": ["St"], } - underlying_index.save_synonyms([synonym]).wait() + self.index.__client.save_synonyms( + self.index.index_name, synonym["objectID"], synonym + ) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - time.sleep(10) # FIXME: Refactor reindex_all to return taskID # Expect the synonyms to be kept across reindex - synonyms = [s for s in underlying_index.browse_synonyms()] + synonyms = [] + self.index.__client.browse_synonyms( + self.index.index_name, lambda _resp: synonyms.extend(_resp.hits) + ) self.assertEqual(len(synonyms), 1, "There should only be one synonym") self.assertIn( synonym, synonyms, "The existing synonym should be kept over reindex" @@ -658,31 +658,31 @@ class ExampleIndex(AlgoliaIndex): self.index._should_index(self.example) def test_should_index_property(self): - class ExampleIndex(AlgoliaIndex): + class ExampleIndex1(AlgoliaIndex): fields = "name" should_index = "property_should_index" - self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) + self.index = ExampleIndex1(Example, self.client, settings.ALGOLIA) self.assertTrue( self.index._should_index(self.example), "We should index an instance when its should_index property is True", ) - class ExampleIndex(AlgoliaIndex): + class ExampleIndex2(AlgoliaIndex): fields = "name" should_index = "property_should_not_index" - self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) + self.index = ExampleIndex2(Example, self.client, settings.ALGOLIA) self.assertFalse( self.index._should_index(self.example), "We should not index an instance when its should_index property is False", ) - class ExampleIndex(AlgoliaIndex): + class ExampleIndex3(AlgoliaIndex): fields = "name" should_index = "property_string" - self.index = ExampleIndex(Example, self.client, settings.ALGOLIA) + self.index = ExampleIndex3(Example, self.client, settings.ALGOLIA) with self.assertRaises( AlgoliaIndexError, msg="We should raise when the should_index property is not boolean", @@ -690,16 +690,16 @@ class ExampleIndex(AlgoliaIndex): self.index._should_index(self.example) def test_save_record_should_index_boolean(self): - website = Website.objects.create( - name="Algolia", url="https://algolia.com", is_online=True - ) + website = Website(name="Algolia", url="https://algolia.com", is_online=True) self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): + self.assertIsNotNone(self.index.index_name) + settings = { "replicas": [ - self.index.index_name + "_name_asc", - self.index.index_name + "_name_desc", + self.index.index_name + "_name_asc", # pyright: ignore + self.index.index_name + "_name_desc", # pyright: ignore ] } should_index = "is_online" @@ -718,9 +718,12 @@ class CyrillicIndex(AlgoliaIndex): self.user.bio = "крупнейших" self.user.save() self.index = CyrillicIndex(User, self.client, settings.ALGOLIA) - self.index.save_record(self.user).wait() + self.index.save_record(self.user) result = self.index.raw_search("крупнейших") - self.assertEqual(result["nbHits"], 1, "Search should return one result") - self.assertEqual( - result["hits"][0]["name"], "Algolia", "The result should be self.user" - ) + self.assertIsNotNone(result) + + if result is not None: + self.assertEqual(result["nbHits"], 1, "Search should return one result") + self.assertEqual( + result["hits"][0]["name"], "Algolia", "The result should be self.user" + ) diff --git a/tests/test_signal.py b/tests/test_signal.py index f09b659..ccd3364 100644 --- a/tests/test_signal.py +++ b/tests/test_signal.py @@ -50,11 +50,11 @@ def test_delete_signal(self): mocked_delete_record.assert_has_calls([call(websites[0]), call(websites[1])]) def test_update_records(self): - Website.objects.create(name="Algolia", url="https://www.algolia.com") - Website.objects.create(name="Google", url="https://www.google.com") - Website.objects.create(name="Facebook", url="https://www.facebook.com") - Website.objects.create(name="Facebook", url="https://www.facebook.fr") - Website.objects.create(name="Facebook", url="https://fb.com") + Website(name="Algolia", url="https://www.algolia.com", is_online=False) + Website(name="Google", url="https://www.google.com", is_online=False) + Website(name="Facebook", url="https://www.facebook.com", is_online=False) + Website(name="Facebook", url="https://www.facebook.fr", is_online=False) + Website(name="Facebook", url="https://fb.com", is_online=False) qs = Website.objects.filter(name="Facebook") update_records(Website, qs, url="https://facebook.com") From 2efcc03d9f978263e40b34d7dc31bb2b714ca9b2 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Mon, 25 Nov 2024 14:34:52 +0100 Subject: [PATCH 12/35] chore: remove deprecation notices --- README.md | 4 ++-- algoliasearch_django/__init__.py | 1 - algoliasearch_django/registration.py | 4 ---- tests/test_commands.py | 6 +++--- tests/test_index.py | 17 +---------------- tests/test_signal.py | 4 ++-- 6 files changed, 8 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 94a3625..e430ff3 100644 --- a/README.md +++ b/README.md @@ -338,9 +338,9 @@ class MyModelMetaIndex(AlgoliaIndex): for index in self.indices: index.set_settings() - def clear_index(self): + def clear_objects(self): for index in self.indices: - index.clear_index() + index.clear_objects() def save_record(self, instance, update_fields=None, **kwargs): for index in self.indices: diff --git a/algoliasearch_django/__init__.py b/algoliasearch_django/__init__.py index 08751ea..c4f1ef0 100644 --- a/algoliasearch_django/__init__.py +++ b/algoliasearch_django/__init__.py @@ -31,7 +31,6 @@ delete_record = algolia_engine.delete_record update_records = algolia_engine.update_records raw_search = algolia_engine.raw_search -clear_index = algolia_engine.clear_index # TODO: deprecate clear_objects = algolia_engine.clear_objects reindex_all = algolia_engine.reindex_all diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index 5ac0ec0..99e7d5b 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -156,10 +156,6 @@ def clear_objects(self, model): adapter = self.get_adapter(model) adapter.clear_objects() - def clear_index(self, model): - # TODO: add deprecatd warning - self.clear_objects(model) - def reindex_all(self, model, batch_size=1000): """ Reindex all the records. diff --git a/tests/test_commands.py b/tests/test_commands.py index 18da393..f975f5d 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -4,7 +4,7 @@ from algoliasearch_django import algolia_engine from algoliasearch_django import get_adapter -from algoliasearch_django import clear_index +from algoliasearch_django import clear_objects from .models import Website from .models import User @@ -35,8 +35,8 @@ def setUp(self): self.out = StringIO() def tearDown(self): - clear_index(Website) - clear_index(User) + clear_objects(Website) + clear_objects(User) def test_reindex(self): call_command("algolia_reindex", stdout=self.out) diff --git a/tests/test_index.py b/tests/test_index.py index 4f491ae..fa5be02 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -1,9 +1,7 @@ # coding=utf-8 -import time from django.conf import settings from django.test import TestCase -import unittest from algoliasearch_django import AlgoliaIndex from algoliasearch_django import algolia_engine @@ -153,9 +151,6 @@ class WebsiteIndex(AlgoliaIndex): self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - @unittest.skip( - reason="FIXME: it's a known issue that reindex all might not work properly" - ) def test_reindex_no_settings(self): self.maxDiff = None @@ -179,9 +174,6 @@ class WebsiteIndex(AlgoliaIndex): "An index whose model has no settings should keep its settings after reindex", ) - @unittest.skip( - reason="FIXME: it's a known issue that reindex all might not work properly" - ) def test_reindex_with_settings(self): import uuid @@ -232,7 +224,7 @@ class WebsiteIndex(AlgoliaIndex): self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) # Given some existing query rules on the index - # index.__index.save_rule() # TODO: Check query rules are kept + self.index.__client.save_rule() # Given some existing settings on the index existing_settings = self.apply_some_settings(self.index) @@ -246,9 +238,6 @@ class WebsiteIndex(AlgoliaIndex): former_settings["hitsPerPage"] = 15 self.assertDictEqual(self.index.get_settings(), former_settings) - @unittest.skip( - reason="FIXME: it's a known issue that reindex all might not work properly" - ) def test_reindex_with_rules(self): # Given an existing index defined with settings class WebsiteIndex(AlgoliaIndex): @@ -283,9 +272,6 @@ def remove_metadata(rule): self.assertEqual(len(rules), 1, "There should only be one rule") self.assertIn(rule, rules, "The existing rule should be kept over reindex") - @unittest.skip( - reason="FIXME: it's a known issue that reindex all might not work properly" - ) def test_reindex_with_synonyms(self): # Given an existing index defined with settings class WebsiteIndex(AlgoliaIndex): @@ -332,7 +318,6 @@ def apply_some_settings(self, index): index.settings["hitsPerPage"] = 42 index.reindex_all() index.settings["hitsPerPage"] = old_hpp - time.sleep(10) # FIXME: Refactor reindex_all to return taskID index_settings = index.get_settings() # Expect the instance's settings to be applied at reindex self.assertEqual( diff --git a/tests/test_signal.py b/tests/test_signal.py index ccd3364..4ad9d02 100644 --- a/tests/test_signal.py +++ b/tests/test_signal.py @@ -6,7 +6,7 @@ from algoliasearch_django import algolia_engine from algoliasearch_django import get_adapter from algoliasearch_django import raw_search -from algoliasearch_django import clear_index +from algoliasearch_django import clear_objects from algoliasearch_django import update_records from .factories import WebsiteFactory @@ -19,7 +19,7 @@ def tearDownClass(cls): get_adapter(Website).delete() def tearDown(self): - clear_index(Website) + clear_objects(Website) def test_save_signal(self): with patch.object(algolia_engine, "save_record") as mocked_save_record: From f0db28dd25a0a6a6aa410ce74de83ac7a1760fdb Mon Sep 17 00:00:00 2001 From: shortcuts Date: Mon, 25 Nov 2024 15:36:03 +0100 Subject: [PATCH 13/35] fix: assertions --- algoliasearch_django/models.py | 5 +++-- tests/test_engine.py | 4 ++-- tests/test_index.py | 2 +- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 9e306bc..4c9fc5b 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -324,6 +324,7 @@ def save_record(self, instance, update_fields=None, **kwargs): ) else: obj = self.get_raw_record(instance) + print(obj) self.__client.save_objects( index_name=self.index_name, objects=[obj], wait_for_tasks=True ) @@ -391,9 +392,9 @@ def update_records(self, qs, batch_size=1000, **kwargs): def raw_search(self, query="", params=None): """Performs a search query and returns the parsed JSON.""" if params is None: - params = SearchParamsObject() + params = SearchParamsObject().to_dict() - params.query = query + params["query"] = query try: return self.__client.search_single_index(self.index_name, params).to_dict() diff --git a/tests/test_engine.py b/tests/test_engine.py index 9a41b43..3963841 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -54,7 +54,7 @@ def test_is_register(self): def test_get_adapter(self): self.engine.register(Website) - self.assertEquals(AlgoliaIndex, self.engine.get_adapter(Website).__class__) + self.assertEqual(AlgoliaIndex, self.engine.get_adapter(Website).__class__) def test_get_adapter_exception(self): with self.assertRaises(RegistrationError): @@ -63,7 +63,7 @@ def test_get_adapter_exception(self): def test_get_adapter_from_instance(self): self.engine.register(Website) instance = Website() - self.assertEquals( + self.assertEqual( AlgoliaIndex, self.engine.get_adapter_from_instance(instance).__class__ ) diff --git a/tests/test_index.py b/tests/test_index.py index fa5be02..d7b24a9 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -252,7 +252,7 @@ class WebsiteIndex(AlgoliaIndex): "consequence": {"params": {"query": "other text"}}, } - self.index.__client.save_rule(self.index.index_name, rule["objectID"], rule) + self.client.save_rule(self.index.index_name, rule["objectID"], rule) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) From ace44b124e623107668222a2c3076742d3b953c4 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Mon, 25 Nov 2024 22:16:33 +0100 Subject: [PATCH 14/35] fix: django settings upgrade and some cleanup --- algoliasearch_django/models.py | 10 ++----- runtests.py | 6 ++-- tests/settings.py | 50 +++++++++++++++++++++++++--------- tests/test_index.py | 8 +++--- 4 files changed, 48 insertions(+), 26 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 4c9fc5b..19f31cc 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -86,12 +86,9 @@ def __init__(self, model, client, settings): ): # Only set settings if the actual index class does not define some self.settings = {} - try: - all_model_fields = [ - f.name for f in model._meta.get_fields() if not f.is_relation - ] - except AttributeError: # get_fields requires Django >= 1.8 - all_model_fields = [f.name for f in model._meta.local_fields] + all_model_fields = [ + f.name for f in model._meta.get_fields() if not f.is_relation + ] if isinstance(self.fields, str): self.fields = (self.fields,) @@ -324,7 +321,6 @@ def save_record(self, instance, update_fields=None, **kwargs): ) else: obj = self.get_raw_record(instance) - print(obj) self.__client.save_objects( index_name=self.index_name, objects=[obj], wait_for_tasks=True ) diff --git a/runtests.py b/runtests.py index 3e3da0c..90dc364 100755 --- a/runtests.py +++ b/runtests.py @@ -12,8 +12,10 @@ def main(): os.environ["DJANGO_SETTINGS_MODULE"] = "tests.settings" django.setup() TestRunner = get_runner(settings) - test_runner = TestRunner() - failures = test_runner.run_tests(["tests"]) + test_runner = TestRunner(failfase=True) + # kept here to run a single test + failures = test_runner.run_tests(["tests.test_index.IndexTestCase"]) + # failures = test_runner.run_tests(["tests"]) sys.exit(bool(failures)) diff --git a/tests/settings.py b/tests/settings.py index 746243d..c967350 100644 --- a/tests/settings.py +++ b/tests/settings.py @@ -1,25 +1,36 @@ """ Django settings for core project. -Generated by 'django-admin startproject' using Django 1.8.2. +Generated by 'django-admin startproject' using Django 5.1.3. For more information on this file, see -https://docs.djangoproject.com/en/1.8/topics/settings/ +https://docs.djangoproject.com/en/5.1/topics/settings/ For the full list of settings and their values, see -https://docs.djangoproject.com/en/1.8/ref/settings/ +https://docs.djangoproject.com/en/5.1/ref/settings/ """ import os import time +from pathlib import Path +# Build paths inside the project like this: BASE_DIR / 'subdir'. +BASE_DIR = Path(__file__).resolve().parent.parent -BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +# Quick-start development settings - unsuitable for production +# See https://docs.djangoproject.com/en/5.1/howto/deployment/checklist/ + +# SECURITY WARNING: keep the secret key used in production secret! SECRET_KEY = "MillisecondsMatter" -DEBUG = False + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +ALLOWED_HOSTS = [] # Application definition -INSTALLED_APPS = ( + +INSTALLED_APPS = [ "django.contrib.admin", "django.contrib.auth", "django.contrib.contenttypes", @@ -28,19 +39,21 @@ "django.contrib.staticfiles", "algoliasearch_django", "tests", -) +] MIDDLEWARE = [ - "django.contrib.sessions.middleware.SessionMiddleware", - "django.middleware.common.CommonMiddleware", - "django.middleware.csrf.CsrfViewMiddleware", "django.contrib.auth.middleware.AuthenticationMiddleware", "django.contrib.auth.middleware.SessionAuthenticationMiddleware", "django.contrib.messages.middleware.MessageMiddleware", + "django.contrib.sessions.middleware.SessionMiddleware", "django.middleware.clickjacking.XFrameOptionsMiddleware", + "django.middleware.common.CommonMiddleware", + "django.middleware.csrf.CsrfViewMiddleware", "django.middleware.security.SecurityMiddleware", ] +ROOT_URLCONF = "tests.urls" + TEMPLATES = [ { "BACKEND": "django.template.backends.django.DjangoTemplates", @@ -57,23 +70,34 @@ }, ] -ROOT_URLCONF = "tests.urls" - # Database +# https://docs.djangoproject.com/en/5.1/ref/settings/#databases + DATABASES = { "default": { "ENGINE": "django.db.backends.sqlite3", - "NAME": os.path.join(BASE_DIR, "db.sqlite3"), + "NAME": BASE_DIR / "db.sqlite3", } } # Internationalization +# https://docs.djangoproject.com/en/5.1/topics/i18n/ + LANGUAGE_CODE = "en-us" + TIME_ZONE = "UTC" + USE_I18N = True + USE_L10N = True + USE_TZ = True +# Default primary key field type +# https://docs.djangoproject.com/en/5.1/ref/settings/#default-auto-field + +DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField" + def safe_index_name(name): return "{}_ci-{}".format(name, time.time()) diff --git a/tests/test_index.py b/tests/test_index.py index d7b24a9..e682e24 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -23,6 +23,7 @@ def setUp(self): _lng=-42.24, _permissions="read,write,admin", ) + self.website = Website(name="Algolia", url="https://algolia.com") self.contributor = User( name="Contributor", @@ -675,12 +676,10 @@ class ExampleIndex3(AlgoliaIndex): self.index._should_index(self.example) def test_save_record_should_index_boolean(self): - website = Website(name="Algolia", url="https://algolia.com", is_online=True) self.index = AlgoliaIndex(Website, self.client, settings.ALGOLIA) class WebsiteIndex(AlgoliaIndex): - self.assertIsNotNone(self.index.index_name) - + custom_objectID = "name" settings = { "replicas": [ self.index.index_name + "_name_asc", # pyright: ignore @@ -689,8 +688,9 @@ class WebsiteIndex(AlgoliaIndex): } should_index = "is_online" + self.website.is_online = True self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) - self.index.save_record(website) + self.index.save_record(self.website) def test_cyrillic(self): class CyrillicIndex(AlgoliaIndex): From 466d528a810c9bc442e1069085f832d1e16cfba1 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 11:32:08 +0100 Subject: [PATCH 15/35] fix: prepare for next python release --- algoliasearch_django/models.py | 14 ++---------- algoliasearch_django/registration.py | 4 ++-- runtests.py | 4 ++-- tests/test_engine.py | 13 ++++++----- tests/test_index.py | 34 ++++++++++++++++++++-------- 5 files changed, 38 insertions(+), 31 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 19f31cc..381f5fb 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -400,7 +400,7 @@ def raw_search(self, query="", params=None): else: logger.warning("ERROR DURING SEARCH ON %s: %s", self.index_name, e) - def get_settings(self): + def get_settings(self) -> dict | None: """Returns the settings of the index.""" try: logger.info("GET SETTINGS ON %s", self.index_name) @@ -482,23 +482,16 @@ def reindex_all(self, batch_size=1000): raise e # Unexpected error while getting settings try: should_keep_replicas = False - should_keep_slaves = False replicas = None - slaves = None if self.settings: replicas = self.settings.get("replicas", None) - slaves = self.settings.get("slaves", None) should_keep_replicas = replicas is not None - should_keep_slaves = slaves is not None if should_keep_replicas: self.settings["replicas"] = [] logger.debug("REMOVE REPLICAS FROM SETTINGS") - if should_keep_slaves: - self.settings["slaves"] = [] - logger.debug("REMOVE SLAVES FROM SETTINGS") _resp = self.__client.set_settings(self.tmp_index_name, self.settings) self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) @@ -567,10 +560,7 @@ def reindex_all(self, batch_size=1000): if should_keep_replicas: self.settings["replicas"] = replicas logger.debug("RESTORE REPLICAS") - if should_keep_slaves: - self.settings["slaves"] = slaves - logger.debug("RESTORE SLAVES") - if should_keep_replicas or should_keep_slaves: + if should_keep_replicas: _resp = self.__client.set_settings(self.index_name, self.settings) self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) if should_keep_rules: diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index 99e7d5b..c217565 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -34,8 +34,8 @@ def __init__(self, settings=SETTINGS): self.__registered_models = {} self.client = SearchClientSync(app_id, api_key) - # self.client.add_user_agent("Algolia for Django", VERSION) - # self.client.add_user_agent("Django", django_version()) + self.client.add_user_agent("Algolia for Django", VERSION) + self.client.add_user_agent("Django", django_version()) def is_registered(self, model): """Checks whether the given models is registered with Algolia engine""" diff --git a/runtests.py b/runtests.py index 90dc364..f05758f 100755 --- a/runtests.py +++ b/runtests.py @@ -14,8 +14,8 @@ def main(): TestRunner = get_runner(settings) test_runner = TestRunner(failfase=True) # kept here to run a single test - failures = test_runner.run_tests(["tests.test_index.IndexTestCase"]) - # failures = test_runner.run_tests(["tests"]) + # failures = test_runner.run_tests(["tests.test_index.IndexTestCase.test_reindex_with_synonyms"]) + failures = test_runner.run_tests(["tests"]) sys.exit(bool(failures)) diff --git a/tests/test_engine.py b/tests/test_engine.py index 3963841..c67672e 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -1,9 +1,10 @@ import six +from django import __version__ as __django__version__ from django.conf import settings from django.test import TestCase -from algoliasearch_django import algolia_engine +from algoliasearch_django import algolia_engine, __version__ from algoliasearch_django import AlgoliaIndex from algoliasearch_django import AlgoliaEngine from algoliasearch_django.registration import AlgoliaEngineError @@ -29,11 +30,11 @@ def test_init_exception(self): with self.assertRaises(AlgoliaEngineError): AlgoliaEngine(settings=settings.ALGOLIA) - # def test_user_agent(self): - # self.assertIn( - # "Algolia for Django (%s); Django (%s)" % VERSION % django_version(), - # self.engine.client._config.user_agent.get(), - # ) + def test_user_agent(self): + self.assertIn( + "Algolia for Django (%s); Django (%s)" % __version__ % __django__version__, + self.engine.client._config.user_agent.get(), + ) def test_auto_discover_indexes(self): """Test that the `index` module was auto-discovered and the models registered""" diff --git a/tests/test_index.py b/tests/test_index.py index e682e24..91f5dd1 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -225,7 +225,7 @@ class WebsiteIndex(AlgoliaIndex): self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) # Given some existing query rules on the index - self.index.__client.save_rule() + # index.__index.save_rule() # TODO: Check query rules are kept # Given some existing settings on the index existing_settings = self.apply_some_settings(self.index) @@ -237,7 +237,13 @@ class WebsiteIndex(AlgoliaIndex): # Expect the settings to be reset to model definition over reindex former_settings = existing_settings former_settings["hitsPerPage"] = 15 - self.assertDictEqual(self.index.get_settings(), former_settings) + + new_settings = self.index.get_settings() + + self.assertIsNotNone(new_settings) + + if new_settings is not None: + self.assertDictEqual(new_settings, former_settings) def test_reindex_with_rules(self): # Given an existing index defined with settings @@ -253,6 +259,11 @@ class WebsiteIndex(AlgoliaIndex): "consequence": {"params": {"query": "other text"}}, } + self.assertIsNotNone(self.index.index_name) + + if self.index.index_name is None: + return + self.client.save_rule(self.index.index_name, rule["objectID"], rule) # When reindexing with no settings on the instance @@ -280,6 +291,11 @@ class WebsiteIndex(AlgoliaIndex): self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) + self.assertIsNotNone(self.index.index_name) + + if self.index.index_name is None: + return + # Given some existing synonyms on the index synonym = { "objectID": "street", @@ -287,25 +303,25 @@ class WebsiteIndex(AlgoliaIndex): "word": "Street", "corrections": ["St"], } - self.index.__client.save_synonyms( - self.index.index_name, synonym["objectID"], synonym - ) + save_synonyms_response = self.client.save_synonyms(self.index.index_name, synonym_hit=[synonym]) + self.client.wait_for_task(self.index.index_name, save_synonyms_response.task_id) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() # Expect the synonyms to be kept across reindex + resp = self.client.search_synonyms(index_name=self.index.index_name) + print(resp, self.index.index_name) + synonyms = [] - self.index.__client.browse_synonyms( - self.index.index_name, lambda _resp: synonyms.extend(_resp.hits) - ) + self.client.browse_synonyms(self.index.index_name, lambda _resp: print(_resp)) self.assertEqual(len(synonyms), 1, "There should only be one synonym") self.assertIn( synonym, synonyms, "The existing synonym should be kept over reindex" ) - def apply_some_settings(self, index): + def apply_some_settings(self, index) -> dict: """ Applies a sample setting to the index. From bf70da45ee972f16e1cbbf718cf37e293ab67156 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 11:33:42 +0100 Subject: [PATCH 16/35] chore: format --- algoliasearch_django/registration.py | 6 ++++-- tests/test_index.py | 4 +++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index c217565..c0e8176 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -1,8 +1,10 @@ from __future__ import unicode_literals import logging +from django import __version__ as __django__version__ from django.db.models.signals import post_save from django.db.models.signals import pre_delete +from algoliasearch_django import __version__ from algoliasearch.search.client import SearchClientSync from .models import AlgoliaIndex @@ -34,8 +36,8 @@ def __init__(self, settings=SETTINGS): self.__registered_models = {} self.client = SearchClientSync(app_id, api_key) - self.client.add_user_agent("Algolia for Django", VERSION) - self.client.add_user_agent("Django", django_version()) + self.client.add_user_agent("Algolia for Django", __version__) + self.client.add_user_agent("Django", __django__version__) def is_registered(self, model): """Checks whether the given models is registered with Algolia engine""" diff --git a/tests/test_index.py b/tests/test_index.py index 91f5dd1..6dc5e80 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -303,7 +303,9 @@ class WebsiteIndex(AlgoliaIndex): "word": "Street", "corrections": ["St"], } - save_synonyms_response = self.client.save_synonyms(self.index.index_name, synonym_hit=[synonym]) + save_synonyms_response = self.client.save_synonyms( + self.index.index_name, synonym_hit=[synonym] + ) self.client.wait_for_task(self.index.index_name, save_synonyms_response.task_id) # When reindexing with no settings on the instance From 2cf07af2b5c5b4c9162fe0618515ec4f7780c972 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 12:03:58 +0100 Subject: [PATCH 17/35] chore: some types improvements --- algoliasearch_django/models.py | 59 ++++++++++++++++------------------ tox.ini | 2 +- 2 files changed, 29 insertions(+), 32 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 381f5fb..0363c82 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -4,6 +4,7 @@ from functools import partial from itertools import chain import logging +from typing import Callable, Iterable, Optional from algoliasearch.http.exceptions import AlgoliaException from algoliasearch.search.models.operation_index_params import OperationIndexParams @@ -57,7 +58,7 @@ class AlgoliaIndex(object): tags = None # Use to specify the index to target on Algolia. - index_name = None + index_name: str | None = None # Use to specify the settings of the index. settings = None @@ -72,9 +73,27 @@ class AlgoliaIndex(object): # Name of the attribute to check on instances if should_index is not a callable _should_index_is_method = False + get_queryset: Optional[Callable[[], Iterable]] = None + def __init__(self, model, client, settings): """Initializes the index.""" - self.__init_index(model, settings) + if not self.index_name: + self.index_name = model.__name__ + + tmp_index_name = "{index_name}_tmp".format(index_name=self.index_name) + + if "INDEX_PREFIX" in settings: + self.index_name = settings["INDEX_PREFIX"] + "_" + self.index_name + tmp_index_name = "{index_prefix}_{tmp_index_name}".format( + tmp_index_name=tmp_index_name, index_prefix=settings["INDEX_PREFIX"] + ) + if "INDEX_SUFFIX" in settings: + self.index_name += "_" + settings["INDEX_SUFFIX"] + tmp_index_name = "{tmp_index_name}_{index_suffix}".format( + tmp_index_name=tmp_index_name, index_suffix=settings["INDEX_SUFFIX"] + ) + + self.tmp_index_name = tmp_index_name self.model = model self.__client = client @@ -170,25 +189,6 @@ def __init__(self, model, client, settings): ) ) - def __init_index(self, model, settings): - if not self.index_name: - self.index_name = model.__name__ - - tmp_index_name = "{index_name}_tmp".format(index_name=self.index_name) - - if "INDEX_PREFIX" in settings: - self.index_name = settings["INDEX_PREFIX"] + "_" + self.index_name - tmp_index_name = "{index_prefix}_{tmp_index_name}".format( - tmp_index_name=tmp_index_name, index_prefix=settings["INDEX_PREFIX"] - ) - if "INDEX_SUFFIX" in settings: - self.index_name += "_" + settings["INDEX_SUFFIX"] - tmp_index_name = "{tmp_index_name}_{index_suffix}".format( - tmp_index_name=tmp_index_name, index_suffix=settings["INDEX_SUFFIX"] - ) - - self.tmp_index_name = tmp_index_name - @staticmethod def _validate_geolocation(geolocation): """ @@ -239,7 +239,7 @@ def get_raw_record(self, instance, update_fields=None): if callable(self.tags): tmp["_tags"] = self.tags(instance) if not isinstance(tmp["_tags"], list): - tmp["_tags"] = list(tmp["_tags"]) + tmp["_tags"] = list(tmp["_tags"]) # pyright: ignore logger.debug("BUILD %s FROM %s", tmp["objectID"], self.model) return tmp @@ -374,12 +374,7 @@ def update_records(self, qs, batch_size=1000, **kwargs): tmp["objectID"] = elt batch.append(dict(tmp)) - if len(batch) >= batch_size: - self.__client.partial_update_objects( - index_name=self.index_name, objects=batch, wait_for_tasks=True - ) - batch = [] - + # TODO: pass batch_size to partial_update_objects if len(batch) > 0: self.__client.partial_update_objects( index_name=self.index_name, objects=batch, wait_for_tasks=True @@ -519,9 +514,10 @@ def reindex_all(self, batch_size=1000): counts = 0 batch = [] + qs = [] - if hasattr(self, "get_queryset") and callable(self.get_queryset): # pyright: ignore - qs = self.get_queryset() # pyright: ignore + if hasattr(self, "get_queryset") and callable(self.get_queryset): + qs = self.get_queryset() else: qs = self.model.objects.all() @@ -550,7 +546,8 @@ def reindex_all(self, batch_size=1000): _resp = self.__client.operation_index( self.tmp_index_name, OperationIndexParams( - operation=OperationType.MOVE, destination=self.index_name + operation=OperationType.MOVE, + destination=self.index_name, # pyright: ignore ), ) self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) diff --git a/tox.ini b/tox.ini index d835757..33fc455 100644 --- a/tox.ini +++ b/tox.ini @@ -72,4 +72,4 @@ commands = pip3 freeze > requirements.txt ruff check --fix --unsafe-fixes ruff format . - pyright + pyright algoliasearch_django From bc74b9d020747010a065257f1fdbbb1803545f76 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 13:15:25 +0100 Subject: [PATCH 18/35] fix: decorators --- algoliasearch_django/decorators.py | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/algoliasearch_django/decorators.py b/algoliasearch_django/decorators.py index a8056fb..d1a075c 100644 --- a/algoliasearch_django/decorators.py +++ b/algoliasearch_django/decorators.py @@ -53,22 +53,14 @@ def __init__(self, model=None): if model is not None: self.models = [model] else: - self.models = algolia_engine._AlgoliaEngine__registered_models + self.models = algolia_engine.get_registered_models() def __enter__(self): for model in self.models: - post_save.disconnect( - algolia_engine._AlgoliaEngine__post_save_receiver, sender=model - ) - pre_delete.disconnect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, sender=model - ) + post_save.disconnect(algolia_engine.__post_save_receiver, sender=model) + pre_delete.disconnect(algolia_engine.__pre_delete_receiver, sender=model) def __exit__(self, exc_type, exc_value, traceback): for model in self.models: - post_save.connect( - algolia_engine._AlgoliaEngine__post_save_receiver, sender=model - ) - pre_delete.connect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, sender=model - ) + post_save.connect(algolia_engine.__post_save_receiver, sender=model) + pre_delete.connect(algolia_engine.__pre_delete_receiver, sender=model) From 19f93b7a09cbfc50694367f067f91a0c1a70ae42 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 13:52:11 +0100 Subject: [PATCH 19/35] fix: requirements and install from source --- requirements.txt | 19 +++++++++++-------- tox.ini | 2 +- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/requirements.txt b/requirements.txt index c98c7fa..0f30fda 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,14 @@ -django>=4.0 -algoliasearch>=4.0,<5.0 +Django>=4.0 +# algoliasearch>=4.0,<5.0 +algoliasearch @ git+https://github.com/algolia/algoliasearch-client-python@main # dev dependencies -pypandoc -wheel +factory_boy>=3.0,<4.0 +mock>=5.0,<6.0 +pypandoc>=1.0,<2.0 +pyright>=1.1.389,<2.0 +ruff>=0.7.4,<1.0 +setuptools>=75.0,<76.0 +six>=1.16,<2.0 tox twine -factory_boy -mock -ruff>=0.7.4,<1.0 -pyright>=1.1.389,<2.0 +wheel diff --git a/tox.ini b/tox.ini index 33fc455..9e7ddc5 100644 --- a/tox.ini +++ b/tox.ini @@ -69,7 +69,7 @@ deps = ruff {[versions]ruff} pyright {[versions]pyright} commands = - pip3 freeze > requirements.txt + pip3 install -r requirements.txt ruff check --fix --unsafe-fixes ruff format . pyright algoliasearch_django From 3b5b25b3c84ff99cd84ca6eabf87607d38cf9941 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 13:58:02 +0100 Subject: [PATCH 20/35] fix: circular import --- algoliasearch_django/registration.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/algoliasearch_django/registration.py b/algoliasearch_django/registration.py index c0e8176..166a0bc 100644 --- a/algoliasearch_django/registration.py +++ b/algoliasearch_django/registration.py @@ -4,7 +4,7 @@ from django import __version__ as __django__version__ from django.db.models.signals import post_save from django.db.models.signals import pre_delete -from algoliasearch_django import __version__ +from algoliasearch_django.version import VERSION as __version__ from algoliasearch.search.client import SearchClientSync from .models import AlgoliaIndex From c85fce28d11818a259b815a42a5ae6d0557fe107 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 14:09:40 +0100 Subject: [PATCH 21/35] fix: pull from source until end of bfcm --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index 208ae50..ff7cd0f 100644 --- a/setup.py +++ b/setup.py @@ -33,7 +33,7 @@ version="4.0.0", license="MIT License", packages=find_packages(exclude=["tests"]), - install_requires=["django>=4.0", "algoliasearch>=4.0,<5.0"], + install_requires=["django>=4.0"], description="Algolia Search integration for Django", long_description=README, long_description_content_type="text/markdown", From 8f1b5617d51e4ac3ebb185014c9fd33664da3c1c Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 14:22:27 +0100 Subject: [PATCH 22/35] fix: lint before tests --- tox.ini | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/tox.ini b/tox.ini index 9e7ddc5..d57176e 100644 --- a/tox.ini +++ b/tox.ini @@ -7,12 +7,15 @@ envlist = {py310,py311,py312,py313}-django51 coverage skip_missing_interpreters = True +allowlist_externals = True [testenv] deps = six mock factory_boy + ruff {[versions]ruff} + pyright {[versions]pyright} py{38,313}: Faker>=5.0,<6.0 django40: Django>=4.0,<4.1 django41: Django>=4.1,<4.2 @@ -21,7 +24,12 @@ deps = django51: Django>=5.1,<5.2 passenv = ALGOLIA* -commands = python runtests.py +commands = + pip3 install -r requirements.txt + ruff check --fix --unsafe-fixes + ruff format . + pyright algoliasearch_django + python runtests.py [versions] twine = >=5.1,<6.0 @@ -62,14 +70,3 @@ commands = python setup.py sdist bdist_wheel twine check dist/* twine upload -u {env:PYPI_USER} -p {env:PYPI_PASSWORD} --repository-url https://upload.pypi.org/legacy/ dist/* - -[testenv:lint] -basepython = python3.11 -deps = - ruff {[versions]ruff} - pyright {[versions]pyright} -commands = - pip3 install -r requirements.txt - ruff check --fix --unsafe-fixes - ruff format . - pyright algoliasearch_django From 790dce6f4a2f3631ac4f555268f3b99f782f5a6e Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 14:33:48 +0100 Subject: [PATCH 23/35] chore: add timeout --- .github/workflows/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 9d44ae2..766d58a 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -89,6 +89,7 @@ jobs: python-version: ${{ matrix.version }} - name: Install dependencies and run tests + timeout-minutes: 5 run: | python -m venv python-ci-run source python-ci-run/bin/activate From 893f3f4d928da5dc98c239496e7a926b8e87b17a Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 15:23:39 +0100 Subject: [PATCH 24/35] fix: types and --- .github/workflows/main.yml | 2 +- algoliasearch_django/models.py | 4 ++-- tox.ini | 15 ++++++++++----- 3 files changed, 13 insertions(+), 8 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 766d58a..98d2624 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -89,7 +89,7 @@ jobs: python-version: ${{ matrix.version }} - name: Install dependencies and run tests - timeout-minutes: 5 + timeout-minutes: 3 run: | python -m venv python-ci-run source python-ci-run/bin/activate diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 0363c82..af4050c 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -58,7 +58,7 @@ class AlgoliaIndex(object): tags = None # Use to specify the index to target on Algolia. - index_name: str | None = None + index_name: Optional[str] = None # Use to specify the settings of the index. settings = None @@ -395,7 +395,7 @@ def raw_search(self, query="", params=None): else: logger.warning("ERROR DURING SEARCH ON %s: %s", self.index_name, e) - def get_settings(self) -> dict | None: + def get_settings(self) -> Optional[dict]: """Returns the settings of the index.""" try: logger.info("GET SETTINGS ON %s", self.index_name) diff --git a/tox.ini b/tox.ini index d57176e..83a3d63 100644 --- a/tox.ini +++ b/tox.ini @@ -14,8 +14,6 @@ deps = six mock factory_boy - ruff {[versions]ruff} - pyright {[versions]pyright} py{38,313}: Faker>=5.0,<6.0 django40: Django>=4.0,<4.1 django41: Django>=4.1,<4.2 @@ -26,9 +24,6 @@ passenv = ALGOLIA* commands = pip3 install -r requirements.txt - ruff check --fix --unsafe-fixes - ruff format . - pyright algoliasearch_django python runtests.py [versions] @@ -70,3 +65,13 @@ commands = python setup.py sdist bdist_wheel twine check dist/* twine upload -u {env:PYPI_USER} -p {env:PYPI_PASSWORD} --repository-url https://upload.pypi.org/legacy/ dist/* + +[testenv:lint] +deps = + ruff {[versions]ruff} + pyright {[versions]pyright} +commands = + pip3 install -r requirements.txt + ruff check --fix --unsafe-fixes + ruff format . + pyright algoliasearch_django From 999f1158aa19951011b98a45808e7d631aef6d1a Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 17:27:45 +0100 Subject: [PATCH 25/35] fix: more tests --- algoliasearch_django/decorators.py | 22 ++++++++++++++---- algoliasearch_django/models.py | 2 +- runtests.py | 6 ++++- tests/test_engine.py | 6 +++-- tests/test_index.py | 37 +++++++++++++++++------------- 5 files changed, 48 insertions(+), 25 deletions(-) diff --git a/algoliasearch_django/decorators.py b/algoliasearch_django/decorators.py index d1a075c..b5e422e 100644 --- a/algoliasearch_django/decorators.py +++ b/algoliasearch_django/decorators.py @@ -53,14 +53,26 @@ def __init__(self, model=None): if model is not None: self.models = [model] else: - self.models = algolia_engine.get_registered_models() + self.models = algolia_engine._AlgoliaEngine__registered_models # pyright: ignore def __enter__(self): for model in self.models: - post_save.disconnect(algolia_engine.__post_save_receiver, sender=model) - pre_delete.disconnect(algolia_engine.__pre_delete_receiver, sender=model) + post_save.disconnect( + algolia_engine._AlgoliaEngine__post_save_receiver, + sender=model, # pyright: ignore + ) + pre_delete.disconnect( + algolia_engine._AlgoliaEngine__pre_delete_receiver, + sender=model, # pyright: ignore + ) def __exit__(self, exc_type, exc_value, traceback): for model in self.models: - post_save.connect(algolia_engine.__post_save_receiver, sender=model) - pre_delete.connect(algolia_engine.__pre_delete_receiver, sender=model) + post_save.connect( + algolia_engine._AlgoliaEngine__post_save_receiver, + sender=model, # pyright: ignore + ) + pre_delete.connect( + algolia_engine._AlgoliaEngine__pre_delete_receiver, + sender=model, # pyright: ignore + ) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index af4050c..2d12355 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -559,7 +559,7 @@ def reindex_all(self, batch_size=1000): logger.debug("RESTORE REPLICAS") if should_keep_replicas: _resp = self.__client.set_settings(self.index_name, self.settings) - self.__client.wait_for_task(self.tmp_index_name, _resp.task_id) + self.__client.wait_for_task(self.index_name, _resp.task_id) if should_keep_rules: _resp = self.__client.save_rules(self.index_name, rules, True) self.__client.wait_for_task(self.index_name, _resp.task_id) diff --git a/runtests.py b/runtests.py index f05758f..a2de077 100755 --- a/runtests.py +++ b/runtests.py @@ -14,7 +14,11 @@ def main(): TestRunner = get_runner(settings) test_runner = TestRunner(failfase=True) # kept here to run a single test - # failures = test_runner.run_tests(["tests.test_index.IndexTestCase.test_reindex_with_synonyms"]) + # failures = test_runner.run_tests( + # [ + # "tests.test_index.IndexTestCase.test_reindex_with_rules" + # ] + # ) failures = test_runner.run_tests(["tests"]) sys.exit(bool(failures)) diff --git a/tests/test_engine.py b/tests/test_engine.py index c67672e..97be25c 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -32,8 +32,10 @@ def test_init_exception(self): def test_user_agent(self): self.assertIn( - "Algolia for Django (%s); Django (%s)" % __version__ % __django__version__, - self.engine.client._config.user_agent.get(), + "Algolia for Django ({}); Django ({})".format( + __version__, __django__version__ + ), + self.engine.client._config._user_agent.get(), ) def test_auto_discover_indexes(self): diff --git a/tests/test_index.py b/tests/test_index.py index 6dc5e80..684d0af 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -256,7 +256,7 @@ class WebsiteIndex(AlgoliaIndex): rule = { "objectID": "my-rule", "condition": {"pattern": "some text", "anchoring": "is"}, - "consequence": {"params": {"query": "other text"}}, + "consequence": {"params": {"hitsPerPage": 42}}, } self.assertIsNotNone(self.index.index_name) @@ -264,25 +264,30 @@ class WebsiteIndex(AlgoliaIndex): if self.index.index_name is None: return - self.client.save_rule(self.index.index_name, rule["objectID"], rule) + self.client.save_rule_with_http_info( + self.index.index_name, rule["objectID"], rule + ) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) self.index.reindex_all() - # Expect the rules to be kept across reindex - def remove_metadata(rule): - copy = dict(rule) - del copy["_metadata"] - return copy - rules = [] - self.index.__client.browse_rules( - self.index.index_name, lambda _resp: rules.extend(_resp.hits) + self.client.browse_rules( + self.index.index_name, + lambda _resp: rules.extend([_hit.to_dict() for _hit in _resp.hits]), ) - rules = list(map(remove_metadata, rules)) self.assertEqual(len(rules), 1, "There should only be one rule") - self.assertIn(rule, rules, "The existing rule should be kept over reindex") + self.assertEqual( + rules[0]["consequence"], + rule["consequence"], + "The existing rule should be kept over reindex", + ) + self.assertEqual( + rules[0]["objectID"], + rule["objectID"], + "The existing rule should be kept over reindex", + ) def test_reindex_with_synonyms(self): # Given an existing index defined with settings @@ -313,11 +318,11 @@ class WebsiteIndex(AlgoliaIndex): self.index.reindex_all() # Expect the synonyms to be kept across reindex - resp = self.client.search_synonyms(index_name=self.index.index_name) - print(resp, self.index.index_name) - synonyms = [] - self.client.browse_synonyms(self.index.index_name, lambda _resp: print(_resp)) + self.client.browse_synonyms( + self.index.index_name, + lambda _resp: synonyms.extend([_hit.to_dict() for _hit in _resp.hits]), + ) self.assertEqual(len(synonyms), 1, "There should only be one synonym") self.assertIn( synonym, synonyms, "The existing synonym should be kept over reindex" From f7e12822f72492eb5330c6c0bb1090ef408a83d1 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 18:01:46 +0100 Subject: [PATCH 26/35] fix: commands and index tests --- runtests.py | 2 +- tests/test_commands.py | 18 ++++++++---------- tests/test_index.py | 5 ++--- 3 files changed, 11 insertions(+), 14 deletions(-) diff --git a/runtests.py b/runtests.py index a2de077..8d97daa 100755 --- a/runtests.py +++ b/runtests.py @@ -12,7 +12,7 @@ def main(): os.environ["DJANGO_SETTINGS_MODULE"] = "tests.settings" django.setup() TestRunner = get_runner(settings) - test_runner = TestRunner(failfase=True) + test_runner = TestRunner(failfast=True) # kept here to run a single test # failures = test_runner.run_tests( # [ diff --git a/tests/test_commands.py b/tests/test_commands.py index f975f5d..373c605 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -21,16 +21,14 @@ def tearDownClass(cls): def setUp(self): # Create some records - User(name="James Bond", username="jb", followers_count=0) - User(name="Captain America", username="captain", followers_count=0) - User( - name="John Snow", - username="john_snow", - _lat=120.2, - _lng=42.1, - followers_count=0, - ) - User(name="Steve Jobs", username="genius", followers_count=331213) + u = User(name="James Bond", username="jb", followers_count=0, following_count=0, _lat=0, _lng=0) + u.save() + u = User(name="Captain America", username="captain", followers_count=0, following_count=0, _lat=0, _lng=0) + u.save() + u = User(name="John Snow", username="john_snow", _lat=120.2, _lng=42.1, followers_count=0, following_count=0) + u.save() + u = User(name="Steve Jobs", username="genius", followers_count=331213, following_count=0, _lat=0, _lng=0) + u.save() self.out = StringIO() diff --git a/tests/test_index.py b/tests/test_index.py index 684d0af..33fbad3 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -264,9 +264,8 @@ class WebsiteIndex(AlgoliaIndex): if self.index.index_name is None: return - self.client.save_rule_with_http_info( - self.index.index_name, rule["objectID"], rule - ) + _resp = self.client.save_rule(self.index.index_name, rule["objectID"], rule) + self.client.wait_for_task(self.index.index_name, _resp.task_id) # When reindexing with no settings on the instance self.index = WebsiteIndex(Website, self.client, settings.ALGOLIA) From be333a169bbbbc0992e6b702b90e7ef417e84770 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 22:21:40 +0100 Subject: [PATCH 27/35] trigger From 651654e1cd307338af8117ac8ac22623d3956bb1 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Tue, 26 Nov 2024 22:54:47 +0100 Subject: [PATCH 28/35] fix: lint and ci --- .github/workflows/main.yml | 3 +-- tests/test_commands.py | 36 ++++++++++++++++++++++++++++++++---- 2 files changed, 33 insertions(+), 6 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 98d2624..1af0c10 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -89,11 +89,10 @@ jobs: python-version: ${{ matrix.version }} - name: Install dependencies and run tests - timeout-minutes: 3 + timeout-minutes: 10 run: | python -m venv python-ci-run source python-ci-run/bin/activate pip3 install --upgrade pip pip3 install tox - pip3 install -r requirements.txt TOXENV=${{ matrix.toxenv }} ALGOLIA_APPLICATION_ID=${{ secrets.ALGOLIA_APPLICATION_ID }} ALGOLIA_API_KEY=${{ secrets.ALGOLIA_API_KEY }} tox diff --git a/tests/test_commands.py b/tests/test_commands.py index 373c605..7dd8097 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -21,13 +21,41 @@ def tearDownClass(cls): def setUp(self): # Create some records - u = User(name="James Bond", username="jb", followers_count=0, following_count=0, _lat=0, _lng=0) + u = User( + name="James Bond", + username="jb", + followers_count=0, + following_count=0, + _lat=0, + _lng=0, + ) u.save() - u = User(name="Captain America", username="captain", followers_count=0, following_count=0, _lat=0, _lng=0) + u = User( + name="Captain America", + username="captain", + followers_count=0, + following_count=0, + _lat=0, + _lng=0, + ) u.save() - u = User(name="John Snow", username="john_snow", _lat=120.2, _lng=42.1, followers_count=0, following_count=0) + u = User( + name="John Snow", + username="john_snow", + _lat=120.2, + _lng=42.1, + followers_count=0, + following_count=0, + ) u.save() - u = User(name="Steve Jobs", username="genius", followers_count=331213, following_count=0, _lat=0, _lng=0) + u = User( + name="Steve Jobs", + username="genius", + followers_count=331213, + following_count=0, + _lat=0, + _lng=0, + ) u.save() self.out = StringIO() From f4f601257301b5591d08c95220e29669ccacde9c Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 27 Nov 2024 10:36:08 +0100 Subject: [PATCH 29/35] trigger From 4a5d0542633c757d77ed51878c03ed21eef44c18 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 27 Nov 2024 11:21:23 +0100 Subject: [PATCH 30/35] chore: longer timeout? --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 1af0c10..cb72981 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -89,7 +89,7 @@ jobs: python-version: ${{ matrix.version }} - name: Install dependencies and run tests - timeout-minutes: 10 + timeout-minutes: 20 run: | python -m venv python-ci-run source python-ci-run/bin/activate From d4b2085777f24910de716ec9361352885745e479 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 27 Nov 2024 11:40:54 +0100 Subject: [PATCH 31/35] chore: ignore at the correct spot --- algoliasearch_django/decorators.py | 16 ++++++++-------- setup.py | 1 + tox.ini | 7 +++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/algoliasearch_django/decorators.py b/algoliasearch_django/decorators.py index b5e422e..9b9adea 100644 --- a/algoliasearch_django/decorators.py +++ b/algoliasearch_django/decorators.py @@ -58,21 +58,21 @@ def __init__(self, model=None): def __enter__(self): for model in self.models: post_save.disconnect( - algolia_engine._AlgoliaEngine__post_save_receiver, - sender=model, # pyright: ignore + algolia_engine._AlgoliaEngine__post_save_receiver, # pyright: ignore + sender=model, ) pre_delete.disconnect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, - sender=model, # pyright: ignore + algolia_engine._AlgoliaEngine__pre_delete_receiver, # pyright: ignore + sender=model, ) def __exit__(self, exc_type, exc_value, traceback): for model in self.models: post_save.connect( - algolia_engine._AlgoliaEngine__post_save_receiver, - sender=model, # pyright: ignore + algolia_engine._AlgoliaEngine__post_save_receiver, # pyright: ignore + sender=model, ) pre_delete.connect( - algolia_engine._AlgoliaEngine__pre_delete_receiver, - sender=model, # pyright: ignore + algolia_engine._AlgoliaEngine__pre_delete_receiver, # pyright: ignore + sender=model, ) diff --git a/setup.py b/setup.py index ff7cd0f..69affeb 100644 --- a/setup.py +++ b/setup.py @@ -64,6 +64,7 @@ "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", "Topic :: Internet :: WWW/HTTP", ], ) diff --git a/tox.ini b/tox.ini index 83a3d63..592e645 100644 --- a/tox.ini +++ b/tox.ini @@ -7,7 +7,6 @@ envlist = {py310,py311,py312,py313}-django51 coverage skip_missing_interpreters = True -allowlist_externals = True [testenv] deps = @@ -33,7 +32,7 @@ ruff = >=0.7.4,<1.0 pyright = >=1.1.389,<2.0 [testenv:coverage] -basepython = python3.11 +basepython = python3.13 deps = coverage passenv = ALGOLIA* @@ -42,7 +41,7 @@ commands = coverage report [testenv:coveralls] -basepython = python3.11 +basepython = python3.13 deps = coverage coveralls @@ -54,7 +53,7 @@ commands = coveralls [testenv:release] -basepython = python3.11 +basepython = python3.13 deps = twine {[versions]twine} wheel {[versions]wheel} From 2fb46fff2828c79989314ea682c50e089e9fcc81 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Wed, 27 Nov 2024 12:10:56 +0100 Subject: [PATCH 32/35] chore: issue template dir --- .github/{ISSUE_TEMPLATE.md => ISSUE_TEMPLATE/Bug_report.yml} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/{ISSUE_TEMPLATE.md => ISSUE_TEMPLATE/Bug_report.yml} (100%) diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE/Bug_report.yml similarity index 100% rename from .github/ISSUE_TEMPLATE.md rename to .github/ISSUE_TEMPLATE/Bug_report.yml From 5df5c84b7e82538941a503af93ff1fe5b569b4b7 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Thu, 28 Nov 2024 10:05:09 +0100 Subject: [PATCH 33/35] chore: review --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- MAINTAINERS.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 3e0f401..6894e86 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,6 +1,6 @@ ## 🧭 What and Why -🎟 JIRA Ticket: +🎟 Related Issue: ### Changes included: diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 0091440..ab0e2c7 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,5 +1,5 @@ ## `algolia/algoliasearch-django` maintainers -| Name | Email | -|-----------------|---------------------| -| Algolia | support@algolia.com | +| Name | Email | +|-----------------|------------------------| +| Algolia | https://alg.li/support | From 4a4031f57cda545faf603e9cf3c19c42498214fe Mon Sep 17 00:00:00 2001 From: shortcuts Date: Mon, 16 Dec 2024 14:26:43 +0100 Subject: [PATCH 34/35] chore: pull latest --- requirements.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index 0f30fda..8aa678e 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,5 @@ Django>=4.0 -# algoliasearch>=4.0,<5.0 -algoliasearch @ git+https://github.com/algolia/algoliasearch-client-python@main +algoliasearch>=4.0,<5.0 # dev dependencies factory_boy>=3.0,<4.0 mock>=5.0,<6.0 From 5a07fe6d9e3e8d066f0985c728f6bbddf0d2cb44 Mon Sep 17 00:00:00 2001 From: shortcuts Date: Mon, 16 Dec 2024 14:27:45 +0100 Subject: [PATCH 35/35] chore: remove TODO --- algoliasearch_django/models.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/algoliasearch_django/models.py b/algoliasearch_django/models.py index 2d12355..5e55822 100644 --- a/algoliasearch_django/models.py +++ b/algoliasearch_django/models.py @@ -374,10 +374,9 @@ def update_records(self, qs, batch_size=1000, **kwargs): tmp["objectID"] = elt batch.append(dict(tmp)) - # TODO: pass batch_size to partial_update_objects if len(batch) > 0: self.__client.partial_update_objects( - index_name=self.index_name, objects=batch, wait_for_tasks=True + index_name=self.index_name, objects=batch, wait_for_tasks=True, batch_size=batch_size, ) def raw_search(self, query="", params=None):