Hugging Face-based NLP Sandbox PHI Annotators

Introduction

NLPSandbox.io is an open platform for benchmarking modular natural language processing (NLP) tools on both public and private datasets. Academics, students, and industry professionals are invited to browse the available tasks and participate by developing and submitting an NLP Sandbox tool.

This repository packages dslim-bert as an NLP Sandbox PHI annotator. The performance of this tool can be viewed and compared to the performance of other PHI annotators on NLPSandbox.io.

Annotations supported by dslim-bert:

Annotation	Schema	Supported
Contact	TextContactAnnotation	No
Date	TextDateAnnotation	No
ID	TextIdAnnotation	No
Location	TextLocationAnnotation	Yes
Person Name	TextPersonNameAnnotation	Yes

Contents

• Specification
• Requirements
• Usage
  • Running with Docker
  • Running with Python
  • Accessing the user interface
• Development
• Versioning
  • GitHub release tags
  • Docker image tags
• Benchmarking on NLPSandbox.io
• Citation
• Contributing
• License

Specification

• NLP Sandbox schemas version: 1.2.0
• Tool version: 1.2.0
• Tool images:
  • docker.synapse.org/syn22277123/phi-annotator-huggingface-bert-base-ner
  • docker.synapse.org/syn22277123/phi-annotator-huggingface-bert-base-ner-uncased
  • docker.synapse.org/syn22277123/phi-annotator-huggingface-bert-large-ner

Requirements

• Docker Engine >=19.03.0

Usage

Running with Docker

The command below starts this NLP Sandbox PHI annotator locally.

docker compose up --build

You can stop the container run with Ctrl+C, followed by docker compose down.

Running with Python

Create a Conda environment:

conda create --name phi-annotator python=3.9 -y
conda activate phi-annotator

Install dependencies and download an NER model from HuggingFace (default: David S. Lim's fine-tuned BERT model):

Install and start this NLP Sandbox tool.

cd server && pip install -r requirements.txt
python save_bert.py
python -m openapi_server

Accessing this NLP Sandbox tool User Interface

This NLP Sandbox tool provides a web interface that you can use to annotate clinical notes. This web client has been automatically generated by openapi-generator. To access the UI, open a new tab in your browser and navigate to one of the following address depending on whether you are running the tool using Docker (production) or Python (development).

• Using Docker: http://localhost/ui
• Using Python: http://localhost:8080/ui

Development

Please refer to the section Development of the NLP Sandbox PHI Annotator example for information on how to develop an NLP Sandbox PHI annotator in Python-Flask and other programming languages-frameworks.

Versioning

GitHub release tags

This repository uses semantic versioning to track the releases of this tool. This repository uses "non-moving" GitHub tags, that is, a tag will always point to the same git commit once it has been created.

Docker image tags

The artifact published by the CI/CD workflow of this GitHub repository is a Docker image pushed to the Synapse Docker Registry. This table lists the image tags pushed to the registry.

Tag name	Moving	Description
latest	Yes	Latest stable release.
edge	Yes	Latest commit made to the default branch.
edge-<sha>	No	Same as above with the reference to the git commit.
<major>.<minor>.<patch>	No	Stable release.

You should avoid using a moving tag like latest when deploying containers in production, because this makes it hard to track which version of the image is running and hard to roll back.

Benchmarking on NLPSandbox.io

Visit nlpsandbox.io for instructions on how to submit your NLP Sandbox tool and evaluate its performance.

Citation

• If you use dslim-bert in your publications, please follow the citation guidelines given by the authors of dslim-bert.
• If you use this NLP Sandbox tool or resources from NLPSandbox.io, please follow these [citation guidelines].

Contributing

Thinking about contributing to this project? Get started by reading our contribution guide.

License

Apache License 2.0