-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mappings for patient, family history, social history and location #4
Open
janhaviel
wants to merge
3
commits into
main
Choose a base branch
from
ACEP_mappings
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
## Table name: care_site | ||
|
||
### Reading from location.csv | ||
|
||
![](md_files/image4.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| care_site_id | service_location_id | | service_location_id is is the unique identifier for the encounter place of service. If it is an integer it can be mapped to the care_site_id directly. If a string it has to be stored in care_site_source_value and then generated.<br> | | ||
| care_site_name | service_location_name | | service_location_name is the text description of the encounter place of service. This will directly provide the care_site_name.<br> | | ||
| place_of_service_concept_id | | | | | ||
| location_id | | | This has to be auto generated for each of the care sites so that it can be linked to the location table and the location information can be mapped there. | | ||
| care_site_source_value | service_location_id | | The original service_location_id has to be stored in the care_site_source_value.<br> | | ||
| place_of_service_source_value | | | | | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
## Table name: death | ||
|
||
### Reading from patient.csv | ||
|
||
![](md_files/image2.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| person_id | patient_id | | The patient_id is the primary key linking all tables. This can be mapped back to the the person_source_value/person_id of the person table depending on the logic used.<br> | | ||
| death_date | patient_date_of_death | | The format of patient_date_of_death is not provided in the dictionary and has to be checked.<br> | | ||
| death_datetime | | | | | ||
| death_type_concept_id | | | | | ||
| cause_concept_id | | | | | ||
| cause_source_value | | | | | ||
| cause_source_concept_id | | | | | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Source Data Mapping Approach to CDMV5.4 | ||
|
||
![](md_files/image7.png) | ||
|
||
## Contents | ||
|
||
[person](person.md) | ||
|
||
[death](death.md) | ||
|
||
[location](location.md) | ||
|
||
[care_site](care_site.md) | ||
|
||
[observation](observation.md) | ||
|
||
[source_appendix](source_appendix.md) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
## Table name: location | ||
|
||
### Reading from location.csv | ||
|
||
![](md_files/image3.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| location_id | | | This information has to be mapped from the care_site table. The service_location_id will act as care_site_source_value/care_site_id. If not provided this has to be autogenerated. | | ||
| address_1 | service_location_street_address | | service_location_street_address contains the street address of the service location where services were provided. The first line of address will be stored in address_1<br> | | ||
| address_2 | service_location_street_address | | service_location_street_address contains the street address of the service location where services were provided. The second line of address will be stored in address_2<br> | | ||
| city | service_location_city | | service_location_city is the city of the service location where services were provided. This can be directly stored in city.<br> | | ||
| state | service_location_state | | service_location_state is the state of the service location where services were provided. This can be directly stored in state.<br> | | ||
| zip | service_location_zip_code | | service_location_zip_code is the zip code of the service location where services were provided. This can be directly stored in zip.<br> | | ||
| county | | | | | ||
| location_source_value | | | | | ||
| country_concept_id | | | | | ||
| country_source_value | | | | | ||
| latitude | | | | | ||
| longitude | | | | | ||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
## Table name: observation | ||
|
||
### Reading from family_history.csv | ||
|
||
![](md_files/image5.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| observation_id | | | This will be generated for each unique observation if not given in the source table. | | ||
| person_id | patient_id | | The patient_id is the primary key linking all tables. This can be mapped back to the the person_source_value/person_id of the person table depending on the logic used<br> | | ||
| visit_occurrence_id | encounter_id | | encounter_id is an identifier unique to the encounter. It is the primary key for the visit occurence table. This can be mapped back to the the visit_source_value/visit_occurrence_id of the person table depending on the logic used.<br> | | ||
| observation_type_concept_id | | | This has to be mapped to the respective Type Concept. Needs more investigation. | | ||
| observation_concept_id | family_history_type_code | | family_history_type_code has SNOMED CT code representing family medical history hierarchy. This can be directly mapped to observation_source_concept_id. The observation_source_concept_id is used to map to the standard observation_concept_id<br> | | ||
| observation_source_concept_id | family_history_type_code | | family_history_type_code has SNOMED CT code representing family medical history hierarchy. This can be directly mapped to observation_source_concept_id.<br> | | ||
| observation_source_value | family_history_observed_value | | family_history_observed_value is the unstructured/non-standard family history documentation. This can be stored directly as observation_source_value.<br> | | ||
| observation_date | family_history_documentation_date_time | | family_history_documentation_date_time is the date and time when the family history is recorded. The date element can be extracted and stored in observation_date. The standard format is not specified in the dictionary this has to be checked when we get the data.<br> | | ||
| observation_datetime | family_history_documentation_date_time | | family_history_documentation_date_time is the date and time when the family history is recorded. This can be stored in observation_datetime. The standard format is not specified in the dictionary this has to be checked when we get the data. | | ||
| provider_id | service_provider_npi | | The service_provider_npi is the unique NPI for the clinician providing the encounter services. This will map to npi in provider table, the provider_id is the primary key for it which can be added to the observation table.<br> | | ||
| value_as_string | | | | | ||
| value_as_number | | | | | ||
| value_as_concept_id | | | | | ||
| qualifier_concept_id | | | | | ||
| unit_concept_id | | | | | ||
| visit_detail_id | | | | | ||
| unit_source_value | | | | | ||
| qualifier_source_value | | | | | ||
| value_source_value | | | | | ||
| observation_event_id | | | | | ||
| obs_event_field_concept_id | | | | | ||
|
||
### Reading from social_history.csv | ||
|
||
![](md_files/image6.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| observation_id | | | This will be generated for each unique observation if not given in the source table. | | ||
| person_id | patient_id | | The patient_id is the primary key linking all tables. This can be mapped back to the the person_source_value/person_id of the person table depending on the logic used<br> | | ||
| visit_occurrence_id | encounter_id | | encounter_id is an identifier unique to the encounter. It is the primary key for the visit occurence table. This can be mapped back to the the visit_source_value/visit_occurrence_id of the person table depending on the logic used.<br> | | ||
| observation_type_concept_id | | | This has to be mapped to the respective Type Concept. Needs more investigation. | | ||
| observation_concept_id | social_history_type_code | | social_history_type_code has standard code representing social history. This can be directly mapped to observation_source_concept_id. The observation_source_concept_id is used to map to the standard observation_concept_id.<br> | | ||
| observation_source_concept_id | social_history_type_code | | social_history_type_code has standard code representing social history. This can be directly mapped to observation_source_concept_id.<br> | | ||
| observation_source_value | social_history_observed_value | | social_history_observed_value is the unstructured/non-standard social history documentation. This can be stored directly as observation_source_value.<br> | | ||
| observation_date | social_history_documentation_date_time | | social_history_documentation_date_time is the date and time when the social history is recorded. The date element can be extracted and stored in observation_date. The standard format is not specified in the dictionary this has to be checked when we get the data.<br> | | ||
| observation_datetime | social_history_documentation_date_time | | social_history_documentation_date_time is the date and time when the social history is recorded. This can be stored in observation_datetime. The standard format is not specified in the dictionary this has to be checked when we get the data.<br>family_history_documentation_date_time is the date and time when the family history is recorded. This can be stored in observation_datetime. The standard format is not specified in the dictionary this has to be checked when we get the data. | | ||
| provider_id | service_provider_npi | | The service_provider_npi is the unique NPI for the clinician providing the encounter services. This will map to npi in provider table, the provider_id is the primary key for it which can be added to the observation table.<br> | | ||
| value_as_string | | | | | ||
| value_as_number | | | | | ||
| value_as_concept_id | | | | | ||
| qualifier_concept_id | | | | | ||
| unit_concept_id | | | | | ||
| visit_detail_id | | | | | ||
| unit_source_value | | | | | ||
| qualifier_source_value | | | | | ||
| value_source_value | | | | | ||
| observation_event_id | | | | | ||
| obs_event_field_concept_id | | | | | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
## Table name: person | ||
|
||
### Reading from patient.csv | ||
|
||
![](md_files/image1.png) | ||
|
||
| Destination Field | Source field | Logic | Comment field | | ||
| --- | --- | --- | --- | | ||
| person_id | patient_id | | If all the patient_id are integers then it could directly be used as a person_id, but if they are strings, the patient_id would go to person_source_value and the person_id will have to be generated.<br> | | ||
| gender_concept_id | patient_gender | | The gender_concept_id would have to be mapped from the concept table. The description to be mapped would be coming from patient_gender/gender_source_value. This logic can be derived once we have the data.<br> | | ||
| year_of_birth | patient_date_of_birth | | year_of_birth will be extracted from patient_date_of_birth which is represented as yyyy-mm-dd<br> | | ||
| month_of_birth | patient_date_of_birth | | month_of_birth will be extracted from patient_date_of_birth which is represented as yyyy-mm-dd<br> | | ||
| day_of_birth | patient_date_of_birth | | day_of_birth will be extracted from patient_date_of_birth which is represented as yyyy-mm-dd<br> | | ||
| birth_datetime | | | | | ||
| race_concept_id | patient_race | | The race_concept_id would have to be mapped from the concept table. The description to be mapped would be coming from patient_race/race_source_value. This logic can be derived once we have the data.<br> | | ||
| ethnicity_concept_id | patient_ethnicity | | The ethnicity_concept_id would have to be mapped from the concept table. The description to be mapped would be coming from patient_ethnicity/ethnicity_source_value. This logic can be derived once we have the data.<br> | | ||
| location_id | | | | | ||
| provider_id | | | | | ||
| care_site_id | service_location_id | | service_location_id is is the unique identifier for the encounter place of service.It has to be mapped to the care_site_source_value/care_site_id in care_site_table.<br> | | ||
| person_source_value | patient_id | | The original patient_id from the source would go to person_source_value.<br> | | ||
| gender_source_value | patient_gender | | The patient_gender can be directly stored in gender_source_value<br> | | ||
| gender_source_concept_id | | | | | ||
| race_source_value | patient_race | | The patient_race can be directly stored in race_source_value<br> | | ||
| race_source_concept_id | | | | | ||
| ethnicity_source_value | patient_ethnicity | | The patient_ethnicity can be directly stored in ethnicity_source_value<br> | | ||
| ethnicity_source_concept_id | | | | | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Appendix: source tables | ||
|
||
### Table: patient.csv | ||
|
||
| Field | Type | Most freq. value | Comment | | ||
| --- | --- | --- | --- | | ||
| patient_id | VARCHAR | SMH3467 | | | ||
| patient_mrn | VARCHAR | SMH3467 | MRN refers to a medical record number, it can be used as the person_id if in the data the MRN is unique for all the individuals. | | ||
| patient_ssn | VARCHAR | 8989-3456-787 | Should be NULL | | ||
| patient_first_name | VARCHAR | Thomus | Should be NULL | | ||
| patient_last_name | VARCHAR | Bogle | Should be NULL | | ||
| patient_gender | VARCHAR | Male | | | ||
| patient_date_of_birth | DATE | 1999-12-30 | | | ||
| patient_date_of_death | VARCHAR | NULL | Mapped to death table | | ||
| patient_race | VARCHAR | white | | | ||
| patient_ethnicity | VARCHAR | Non-Hispanic | | | ||
| patient_language | VARCHAR | English | There are no specific fields present in the CDM. This would have to be passed as NULL. | | ||
| service_location_id | INT | 786353893 | | | ||
| service_location_name | VARCHAR | City Hospital | The service_location_id corresponding to the care_site_id will map it to the care_site_table | | ||
| payer_id | INT | 1112 | payer_id is the unique identifier number assigned to the insured person’s insurance company. Ideally it should have a corresponding field to map it to the billing data table, but this cannot be derivied from the information provided in the data dictionary. | | ||
|
||
### Table: location.csv | ||
|
||
| Field | Type | Most freq. value | Comment | | ||
| --- | --- | --- | --- | | ||
| service_location_id | INT | 848383005 | The location id has to be generated | | ||
| service_location_name | VARCHAR | Smile Hospital | | | ||
| service_location_street_address | VARCHAR | 100 Country Road | | | ||
| service_location_city | VARCHAR | London | | | ||
| service_location_state | VARCHAR | Texas | | | ||
| service_location_zip_code | VARCHAR | 75261 | | | ||
|
||
### Table: family_history.csv | ||
|
||
| Field | Type | Most freq. value | Comment | | ||
| --- | --- | --- | --- | | ||
| patient_id | VARCHAR | 121212 | | | ||
| encounter_id | INT | 101 | | | ||
| family_history_type_code | INT | 266919005 | | | ||
| family_history_type_description | VARCHAR | no one in family ever had AIDs | family_history_type_description contains the text description for Family History Type Code. As this is directly mapping to the concept table through the concept_source_value_id, this description is not required additionally. However, the data needs to be inspected again to derive to this conclusion. | | ||
| family_history_observed_value | VARCHAR | no_use_of_tobacco | | | ||
| family_history_documentation_date_time | VARCHAR | 2014-11-17 9:20:00 | | | ||
| service_provider_npi | INT | | | | ||
| service_provider_first_name | VARCHAR | Ahmed | This has to be deidentified. Pass as NULL. | | ||
| service_provider_last_name | VARCHAR | White | This has to be deidentified. Pass as NULL. | | ||
| onset_date | DATE | 2006-09-14 | There is not specific mapping present for the field. Observation period does not accept values from family or clinical history. | | ||
| service_location_id | INT | 848383005 | This will be the care_site_source_value/care_site_id (based on logic) linked to the visit occurence table or the person table through the primary keys of visit_occurence_id and person_id respectively. | | ||
| service_location_name | VARCHAR | Smile Hospital | This will be care_site_name mapping to the care_site_id linked to the visit occurence table or the person table through the primary keys of visit_occurence_id and person_id respectively. | | ||
|
||
### Table: social_history.csv | ||
|
||
| Field | Type | Most freq. value | Comment | | ||
| --- | --- | --- | --- | | ||
| patient_id | VARCHAR | 121212 | | | ||
| encounter_id | INT | 101 | | | ||
| social_history_type_code | INT | 266919005 | | | ||
| social_history_type_description | VARCHAR | neved smoked | social_history_type_description contains the text description for Social History Type Code. As this is directly mapping to the concept table through the concept_source_value_id, this description is not required additionally. However, the data needs to be inspected again to derive to this conclusion. | | ||
| social_history_observed_value | VARCHAR | no_use_of_tobacco | | | ||
| social_history_documentation_date_time | VARCHAR | 2019-12-19 11:00:00 | | | ||
| effective_start_date | DATE | 2019-12-17 | There is not specific mapping present for the field. Observation period does not accept values from family, social or clinical history. | | ||
| effective_stop_date | DATE | 2019-12-18 | There is not specific mapping present for the field. Observation period does not accept values from family, social or clinical history. | | ||
| service_provider_npi | INT | | | | ||
| service_provider_first_name | VARCHAR | Ahmed | This has to be deidentified. Pass as NULL. | | ||
| service_provider_last_name | VARCHAR | Mahmood | This has to be deidentified. Pass as NULL. | | ||
| service_location_id | INT | 848383005 | This will be the care_site_source_value/care_site_id (based on logic) linked to the visit occurence table or the person table through the primary keys of visit_occurence_id and person_id respectively. | | ||
| service_location_name | VARCHAR | Smile Hospital | This will be care_site_name mapping to the care_site_id linked to the visit occurence table or the person table through the primary keys of visit_occurence_id and person_id respectively. | | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
social_history_observed_value
andfamily_history_observed_value
will map toobersvation_concept_id
alongwithobservation_source_value
. Reason: a source value is mapped to a concept ID, whereas source codes are mapped tosource_concept_ids
. "A source code may not always be a concept ID" , therefore all source codes/concept IDs are mapped to source_concept_id attributesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
obersvation_concept_id
for the same reason as above. They are mapped toobservation_source_concept_id
onlyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense thank you! We got the rationale behind using the observed_value to derive concept_id but we were wondering if we can implement a logic here that if the social_history_type_code or family_history_type_code does not have a corresponding concept_code in the concept table then we should derive the observation_concept_id from the social_history_observed_value and family_history_observed_value? Or would that cause any issues?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the code is available and the choice of standard vocabulary is same as source, then we use the code to fetch the concept ID, else if code is unavailable or missing then we use the value to map to the corresponding concept ID. That logic is part of the vocabulary mapping process which will be discussed later.
@ishitasati11 FYI