Determine the difference between charges with duplicate descriptions but different expunge ability

[jeremylang]

Notes

• no marker when joined to the large data set
• some sort of indicator for charges that have two rows
• originally thought it was just misdemeanor version vs felony version, but we don't know that for a fact and need to do more research

List of contradictory offenses: https://docs.google.com/spreadsheets/d/1FPj1SIdiif8W1qVwghBSTya79ZTr7WVPwO2Qtg1p3eA/edit#gid=649639468

Steps

1. Figure out what the criteria is

[bffcorianne]

I’ve been doing a lot of research as how we’d determine expungibility for some of the charges isn’t really cut and dry. It looks like important info we need are the staircasing, offense level, and probably grid which is found in the sentencing guidelines document.

• Felony and Misdemeanor Master Crime List by M.G.L. Reference as PDF
  I think it would be a good idea to contact the court to get that as a csv or pdf which would have benefits

• Traceable, official source of truth

• Reduce manual manipulation. We might be reinventing the wheel.

IANAL; do we plan to involve an SME (court person/attorney) at some point to ensure our interpretation and choices are correct?

[jeremylang]

We've been working with Pauline at GBLS as our expungement SME.

Next step is reaching out to acquire the documentation mentioned above.

[emilymo008]

Just added a column for extra criteria needed to determine expungeability in data/processed/prosecution_charges_detailed.csv, and fixed charges that were erroneously duplicated because we previously ignored such details. Next, we can either manually input the expungeability for these charges, or we can modify our master crime list and then join by chapter, section, and extra criteria.

[laurafeeney]

this is effectively closed by the same updates we made in issue #155
The challenges with building an effective and comprehensive crosswalk of offenses and expungability are:

• Offenses have a text based description, and the chapter and section where the offense is defined in massachusetts general law.
• within a chapter and section, there can be multiple versions of an offense. For example, robbery with or without a dangerous weapon. Assault or other crimes on someone under or over age 65. And lots of weird stuff in ch272 s53 (prostitution vs disorderly conduct, etc).
  --> so, matching just by chapter and section will lead to some errors, since within a chapter and section some offenses are expungable and others aren't. you have to rely on the text.
• the text based description seems "mostly" standard within a district, but with some variations across districts. it does not line up perfectly with the master crime list in Drive, and likely not the one linked to above. Even when we reconciled everything found in NW + Suffolk, Middlesex had new variations.

But, all of the records that were unmatched were relatively straightforward (with or without a firearm, involving a person >65 years or not, etc). Others we made an educated guess and checked with Sana at CfJJ. There are very very few left that we're unsure about, likely even fewer once we cut out the >21 individuals.