Annif: Explore for subject field metadata (epic)

[hortongn]

• Demo at https://annif.org
• Code at https://github.com/NatLibFi/Annif (Docker container available)
• Subject Vocabularies and Training Data : https://github.com/NatLibFi/Annif-corpora
• Tutorials : https://github.com/NatLibFi/Annif-tutorial

• Get Annif running locally
  https://github.com/NatLibFi/Annif/wiki
  Annif: Explore for subject field metadata (epic) #14 (comment)

• Annif: identify a subject vocabulary #15

• Annif: Find existing content in Scholar to be used as a training set #3

• Annif: train the model #6

• Annif: Test the model and improve it #7

• Explore API calls / Document API calls
  The Swagger UI documentation for the REST API is at http://localhost:5000/v1/ui

• Explore Docker Option

• Annif: get running on a test server #16

• Explore how this service could be integrated into Scholar@UC #8

[scherztc]

https://github.com/NatLibFi/Annif-tutorial/blob/master/exercises/README.md

[scherztc]

How to get ANNIF running locally (Development Install): https://github.com/NatLibFi/Annif/wiki

Note: Must be using python version between 3.8 to 3.10

1. git clone [email protected]:NatLibFi/Annif.git
2. Install pipx and Poetry if you don't have them. First pipx:

python3 -m pip install --user pipx
python3 -m pipx ensurepath

3. Open a new shell, and then install Poetry:

pipx install poetry

4. Create a virtual environment and install dependencies:

poetry install --all-extras

5. Enter the virtual environment:

poetry shell

If you are restarting

6. You will also need NLTK data files:

python -m nltk.downloader punkt

7. Copy projects.cfg.dist to projects.cfg

8. Load Subject Vocabulary

CD to directory above ANNIF
git clone : [email protected]:NatLibFi/Annif-corpora.git

CD into ANNIF directory

annif load-vocab yso ../Annif-corpora/vocab/yso-skos.ttl

9. We will train the model using the the English language training data generated from Finna.fi metadata:

annif train tfidf-en ../Annif-corpora/training/yso-finna-en.tsv.gz

10. Start up the application:

annif run

10b. Restarting the application.

You will need to run poetry shell and annif run each time if you exit the poetry shell

11. Test Document (GUI)

Visit localhost:5000
Copy/Paste Text
Select TF-IDF English
Run

or CLI

cat document.txt | annif suggest tfidf-en

or CLI with folder of documents

annif eval tfidf-en /path/to/documents/

12. Explore API
    The Swagger UI documentation for the REST API is at http://localhost:5000/v1/ui/