mimic_medication_analysis/03_integrate_med_flag.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Integration of Medication Flag\n",
    "\n",
    "Goal: Integrate medication flag based on `inputevents_based_medications.parquet` (produced by `inputevents_mv_analysis.ipynb`) indicating whether alarm is more likely to be relevant. An alarm is more likely to be relevant if a medication was given within a certain period of time around the alarm (1h before to 1h after the alarm, aka a time window of 120min around the alarm). This time window can be adjusted with the variable `minutes_around_alarm`.\n",
    "\n",
    "Note: The merging was necessary because otherwise the two data frames would not be comparable due to their different sizes."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Reading of Alarm and Medication Data"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "alarm_data = pd.read_csv('../../data/alarm_data.csv')\n",
    "alarm_data.CHARTTIME = pd.to_datetime(alarm_data.CHARTTIME)\n",
    "\n",
    "medication = pd.read_parquet('../../data/inputevents_based_medications.parquet', engine='pyarrow')\n",
    "medication.STARTTIME = pd.to_datetime(medication.STARTTIME)\n",
    "medication.ENDTIME = pd.to_datetime(medication.ENDTIME)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Addition of Column for Medical Flag"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Add MEDICATION column for flag and set default to 'False' for all alarms\n",
    "alarm_data['MEDICATION'] = False"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Merging of Alarm and Medication Data"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Filter for ICU stays that are in medication data\n",
    "unique_med_stays = medication['ICUSTAY_ID'].unique().tolist()\n",
    "alarm_data_subset = alarm_data[alarm_data['ICUSTAY_ID'].isin(unique_med_stays)]\n",
    "\n",
    "# Remove columns that are not needed for comparison after merge\n",
    "alarm_data_subset = alarm_data_subset[['ROW_ID', 'ICUSTAY_ID', 'CHARTTIME', 'MEDICATION']]\n",
    "medication = medication[['ICUSTAY_ID', 'STARTTIME', 'ENDTIME']]\n",
    "\n",
    "# Merge data frames based on ICUSTAY_ID\n",
    "alarms_with_med = alarm_data_subset.merge(medication, left_on='ICUSTAY_ID', right_on='ICUSTAY_ID')\n",
    "alarms_with_med.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Setting of Medication Flag"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Set time window around alarm (in minutes)\n",
    "# Half of this duration is applied to time before and after an alarm respectively\n",
    "minutes_around_alarm = 120\n",
    "\n",
    "# Set medication flag if medication was administered in right time period\n",
    "alarms_with_med.loc[\n",
    "    (alarms_with_med['CHARTTIME'] >= alarms_with_med['STARTTIME'] - pd.Timedelta(minutes=minutes_around_alarm/2)) &\n",
    "    (alarms_with_med['CHARTTIME'] <= alarms_with_med['ENDTIME'] + pd.Timedelta(minutes=minutes_around_alarm/2)),\n",
    "    'MEDICATION'] = True"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Extraction of Alarm Data with Medication Flag"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Extract relevant row IDs from merged data frame\n",
    "relevant_row_ids = alarms_with_med[alarms_with_med['MEDICATION'] == True].ROW_ID.unique().tolist()\n",
    "\n",
    "# Set medication flag in original alarm data\n",
    "alarm_data.loc[alarm_data['ROW_ID'].isin(relevant_row_ids), 'MEDICATION'] = True\n",
    "\n",
    "# Save as parquet file\n",
    "pd.DataFrame(alarm_data).to_parquet(f'../../data/alarm_data_with_med_{str(minutes_around_alarm)}min.parquet', engine='pyarrow')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Checking of Overall Amount of Likely Relevant Alarms"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "alarm_data_with_med_120min = pd.read_parquet('../../data/alarm_data_with_med_120min.parquet', engine='pyarrow')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Extract how many alarms were actually relevant\n",
    "relevant_alarms_120min = alarm_data_with_med_120min[alarm_data_with_med_120min['MEDICATION'] == True]\n",
    "\n",
    "print(f'# alarms: {len(alarm_data_with_med_120min)}')\n",
    "print(f'# relevant alarms: {len(relevant_alarms_120min)}')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "For 189,415 of the 285,350 alarms, a medication is administered between one hour before and one hour after the alarm. Maybe the choosen time window of 2 hours around an alarm should be reduced."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "alarm_data_with_med_60min = pd.read_parquet('../../data/alarm_data_with_med_60min.parquet', engine='pyarrow')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "relevant_alarms_60min = alarm_data_with_med_60min[alarm_data_with_med_60min['MEDICATION'] == True]\n",
    "\n",
    "print(f'# relevant alarms: {len(relevant_alarms_60min)}')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "relevant_alarms_60min['ICUSTAY_ID'].value_counts().describe()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "For 173,151 of the 285,350 alarms, a medication is administered between 30min before and 30min after the alarm which is still a lot. With this time window, there are 1 to 12 relevant alarms in 75% of the ICU stays."
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}