PNW Census: Pipeline development

[MichaelHanksSF]

This is the issue to create the PNW Census pipeline.
The pipeline will need to:

• validate a single csv file
• incorporate the new "year_month" element from the filename in exactly the same way "year" is currently handled; for the purpose of differentiating between different returns e.g. PNW Census will allow 2024_Jan, 2024_Feb.... where 903 only has 2024; adding the column "year_month" instead of "year" to all files
• produce a cleanfile per input following a pipeline.json to be defined with client
• hash the values for unique identifiers in the census; and apply the same hash to 903 identifiers such that a single data model can be created at the end
• concatenate cleanfiles at the la level
• create a reports output for each list for the region
• make the usual logs and outputs available in the standard places in the infrastructure

[MichaelHanksSF]

3 days