This package contain programs for cleaning NIPT phenotype record data.
Install
$ git clone [email protected]:ShujiaHuang/nipt_cleaner.git
Some usages:
- Clean the NIFTY record data
$ time python clean_nipt_data.py tsv2json -c "ID,SAMPLE_NUM,SAMPLE_LONG_ID,SAMPLE_AGE,WEIGHT,HEIGHT,CUSTOMER_NATION,CUSTOMER_COUNTRY,BLOOD_TIME,CUSTOMER_ADDRESS,PAST_MEDIC_HISTORY,PRESENT_MEDIC_HISTORY,FOETUS_SEX,FOETUS_TYPE,GEST_WEEKS,NIFTY13,NIFTY18,NIFTY21,NIFTY_SEX,IVF_ET,VISIT_GEST_WEEKS,VISIT_DELIVERY_HOSPITAL,VISIT_DELIVERY_WAY,VISIT_DELIVERY_RESULT,VISIT_CESAREAN_CAUSE,BABY_CHECK_HOSPITAL,ILL_IN_GEST,ILL_IN_GEST_DESC,CHECK_CAUSE_DESC,LEARN_CHANNEL_DESC,NIFTY_RESULT_CAUSE,DELIVERY_B_EXCEPTION,DELIVERY_CHR_EXCEPTION,DELIVERY_ILL_END,DELIVERY_BABY_DIE,DELIVERY_SPON_ABOR,DELIVERY_REFUSED,DELIVERY_OTHER,DELIVERY_OTHER_CAUSE,HYPERTENSION,ECLAMPSIA,GLYCURESIS,OTHER_ILL,CHECK_TANG_HIGH,CHECK_AGE35,CHECK_YANG_RISK,CHECK_YANG_TIMEOUT,CHECK_YANG_DANGER,CHECK_SAFETY,CHECK_OTHER,LEARN_CHANNEL_DOCTOR,LEARN_CHANNEL_FRIEND,LEARN_CHANNEL_MEDIA,LEARN_CHANNEL_OTHER,NIFTY_RESULT_T21,NIFTY_RESULT_T18,NIFTY_RESULT_T13,NIFTY_RESULT_OTHER,YANG_PUNCTURE,YANG_PUNCTURE_RESULT,CARYO_RESULT,CARYO_RESULT_EXCEPTION,NIFTY_RESULT_EQUAL,VISIT_DATE,VISITOR_NAME,VISITOR_ID,VISIT_STATUE,NO_CALL_REASON,VISIT_REMARK,SATISFACTION,SATISFACTION_HOSPITAL_BLOOD,SATISFACTION_ATTITUDE,SATISFACTION_CYCLE_LONG,SATISFACTION_EXPENSIVE,SATISFACTION_REPORT_CONTENT,SATISFACTION_REPORT_SEND_TYPE,SATISFACTION_REPORT_SEND_TIME,SATISFACTION_INVOICE_SEND_TIME,SATISFACTION_REPORT_REMIND,SATISFACTION_INFORMATION_INPUT,SATISFACTION_INFORMATION_WRONG,SATISFACTION_OTHER,EXPECTED_VALUE,SATISFACTION_OTHER_DESC,id_card_number" -i million_nipt_phenotype.filter.tsv > million_nipt_phenotype.json
$ python clean_nipt_data.py extraction_million -i million_nipt_phenotype.json -f "sample_num,id_card_number,customer_nation,sample_age,height,weight,blood_time,gest_weeks,visit_gest_weeks,visit_date,foetus_sex,foetus_type,ivf_et,nifty13,nifty18,nifty21,nifty_result_t13,nifty_result_t18,nifty_result_t21" --json > million_nipt_info.json 2> un.log
- Clean the HPV record data
$ time python clean_nipt_data.py csv2json -i HPV_2016.6.28.csv -c sample_num,person_age,check_result,id_card > HPV_2016.6.28.part.json
$ python clean_nipt_data.py extraction -i HPV_2016.6.28.part.json -f 'sample_num,person_age,id_card,check_result' --json > HPV_2016.6.28.info.json