Skip to content
This repository has been archived by the owner on May 10, 2023. It is now read-only.

Commit

Permalink
indexing records: review and update for Invenio v3.2 and ES 7
Browse files Browse the repository at this point in the history
  • Loading branch information
topless authored and Pablo Panero committed May 14, 2020
1 parent d33e068 commit a0c31f2
Show file tree
Hide file tree
Showing 7 changed files with 99 additions and 107 deletions.
10 changes: 5 additions & 5 deletions 10-indexing-records/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The goal of this tutorial is to learn how to take advantage of Elasticsearch by manipulating records fields when indexing.

### Table of Contents
## Table of Contents

- [Step 1: Bootstrap exercise](#step-1-bootstrap-exercise)
- [Step 2: Modify the record before indexing](#step-2-modify-the-record-before-indexing)
Expand Down Expand Up @@ -62,7 +62,7 @@ $ ./start-from.sh 09-deposit-form
We are going to take advantage of the `invenio-indexer` signal [before_record_index](https://github.com/inveniosoftware/invenio-indexer/blob/master/invenio_indexer/signals.py) to modify the record fields before indexing.
This signal [is called](https://github.com/inveniosoftware/invenio-indexer/blob/master/invenio_indexer/api.py#L305) every time and just before indexing a record.

Create a new file `indexer.py` and copy the following code:
If it doesn't exist, create a new file `indexer.py` and copy the following code:

`my-site/my_site/records/indexer.py`

Expand Down Expand Up @@ -100,7 +100,7 @@ def indexer_receiver(

```

Now we need to register the signal in our Invenio instance. Change the `ext.py` to connect the signal.
Now we need to register the signal in our Invenio instance. We have to connect the signal with our indexer at `ext.py` in the `init_app` of our extension.

`my-site/my_site/records/ext.py`

Expand All @@ -122,7 +122,7 @@ from . import config

Finally, let's change the Elasticsearch mappings to update the fields that we have changed.

`my-site/my_site/records/mappings/v6/records/record-v1.0.0.json`
`my-site/my_site/records/mappings/v7/records/record-v1.0.0.json`

```diff
"id": {
Expand Down Expand Up @@ -173,7 +173,7 @@ $ pipenv run invenio index reindex --pid-type recid --yes-i-know
$ pipenv run invenio index run
```

We can now create a new record, using the deposit of the previous exercise, and verify that in Elasticsearch we will can see the modified fields.
We can now create a new record, using the deposit of the previous exercise, and verify that in Elasticsearch we can see the modified fields.

```bash
$ ./scripts/server
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,3 @@
in Elasticsearch. You need to provide one mapping per major version of
Elasticsearch you want to support.
"""

from __future__ import absolute_import, print_function

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,4 @@
# My site is free software; you can redistribute it and/or modify it under
# the terms of the MIT License; see LICENSE file for more details.

"""Mappings for Elasticsearch 5.x."""

from __future__ import absolute_import, print_function
"""Mappings for Elasticsearch 7.x."""
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{
"mappings": {
"date_detection": false,
"numeric_detection": false,
"properties": {
"$schema": {
"type": "text",
"index": false
},
"id": {
"type": "keyword"
},
"name": {
"type": "text"
},
"organization": {
"type": "text"
},
"_created": {
"type": "date"
},
"_updated": {
"type": "date"
}
}
}
}

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
{
"mappings": {
"date_detection": false,
"numeric_detection": false,
"properties": {
"$schema": {
"type": "text",
"index": false
},
"title": {
"type": "text",
"copy_to": "suggest_title"
},
"suggest_title": {
"type": "completion"
},
"id": {
"type": "keyword"
},
"owner": {
"type": "integer"
},
"publication_date": {
"type": "date",
"format": "date"
},
"contributors_count": {
"type": "short"
},
"contributors": {
"type": "object",
"properties": {
"ids": {
"type": "object",
"properties": {
"source": {
"type": "text"
},
"value": {
"type": "keyword"
}
}
},
"affiliations": {
"type": "text"
},
"role": {
"type": "keyword"
},
"email": {
"type": "text"
},
"name": {
"type": "text"
}
}
},
"_created": {
"type": "date"
},
"_updated": {
"type": "date"
}
}
}
}

0 comments on commit a0c31f2

Please sign in to comment.