Machine Learning pipeline for the United Nations Digital Library 🇺🇳

• Machine Learning pipeline for the United Nations Digital Library 🇺🇳
  • Pipeline description
  • Summarization models
  • NER models
  • Usage

Pipeline description

Summarization models

Model	File	Paper	Authors	Year	HuggingFace 🤗 model
DistillBART-CNN	DistillBARTCNN.py	arXiv link	Shleifer et al.	2020	Link
DistillBART-XSUM	DistillBARTXSUM.py	arXiv link	Shleifer et al.	2020	Link
DistilPegasusCNN	DistilPegasusCNN.py	arXiv link	Shleifer et al.	2020	Link
Longformer	LED.py	arXiv link	Beltagy et al.	2020	Link
LongT5	LongT5.py	arXiv link	Guo et al.	2022	Link

NER models

Model	File	Paper	Authors	Year	HuggingFace 🤗 model
RoBERTa	RoBERTa.py	arXiv link	Liu et al.	2019	Link
FLERT	FLERT.py	arXiv link	Akbik et al.	2020	Link
spaCyNER	spaCy.py	-	spaCy	2023	Link (on spaCy website)

Usage

TODO