This is a Master's thesis for modelling patient trajectories using Transformers at Halmstad University togehter with my collegue and friend Linus Lerjebo, and our supervisiors.
There exists some novel methods which have shown on new state-of-art results for modelling patient trajectories, some of them are highlighted in the table below:
| Method | Description | Paper |
|---|---|---|
| BEHRT | Training the BERT model on EHR data. Given a patient's medical history, predict future diseases. As input, BEHRT add an additional feature apart from orignial BERT, age embedding. | Link |
| ClinicalBERT | ClinicalBERT trains BERT on clinical notes in order to predict if a patient should be readmitted or not. | Link |
| G-BERT | Combines graph neural networks with BERT for medication recommendation. The medical onotology between medical codes is extracted from their GNN, and then fed into the BERT model. | Link |
| Hi-BEHRT | Use BEHRT model to implement an hierarchy using a sliding window, this for capturing long-term dependency. The original BEHRT can only capture sentences of token length 512, while Hi-BEHRT can train on token length 1220. | Link |
In this project, we are combining the Hi-BEHRT and G-BERT, for both modelling for multimodal transformers (text and numerical inputs), capture long-term dependencies, and learn the hierarchical medical ontology of medical codes. As ouput of our model, sequence classification. Given a patient's medical history, can we predict future heart failure patients, and different patient attributes (medication recommendation, readmission, etc...).
For this project we use MIMIC-IV dataset, which contains data for over 60,000 patients.
The dataset is accessible by taking a course at CITI, and here is a more in-depth guide for accessing MIMIC dataset, though this guide covers MIMIC-III, but the steps for accessing MIMIC-IV should be the same as accessing MIMIC-III.
- Most of the requirements are listed in the environment.yml file.
- Spark version = 2.4.7
- Hadoop version = 2.7
- Java jdk = 1.8.0_281
- Python = 3.7.11