Implement Model Specific Data Loader for each of the CM Models

[Polichinel]

Objective

Implement a model-specific data loader for each country-month (cm) model, building on the base data loader from common_utils.

NOTE: this cannot be done before these issue have been resolved!!!!

• Fill Out Configuration Files for CM Models #63
• Create Subdirectories for CM Models #62
• Create Catalog of Country-Month Level Models #61
• ... auto main generation
• ... common_dataloader
• ...

Requirements:

• common_data_loader: The model-specific data loader will extend the functionality of the base loader in common_utils.

• CM Model Configurations: Each model will have a specific configuration dictating which data should be fetched. The data loader script for each model should be minimal, with any additional processing limited to sanity checks or minor adjustments.

• File Structure:

  • Each model-specific data loader should be placed in the respective model directory:

    model/src/dataloader/
    

• Integration:

  • The data loader must interact with the model-specific main file, which will be automatically generated.

Additional Information:

• Naming Convention for raw data: Follow the existing naming conventions for raw data as outlined in the ADR.
• Processing: Any required data processing is done server-side or in the base data loader. Minimal additional processing may be required at the model-specific level.
• Testing: No unit or integration tests are required at this stage unless the data goes through model-specific processing
• File Formats: Data formats are standardized through the base loader, so there is no need to handle multiple formats.
• Storage: the data should be stored in model/data/raw but that should happen automatically at this point.

---

Actionable Steps:

• Create model-specific data loaders for each cm model based on the configurations.
• Place the data loader scripts in the appropriate directory: model/src/dataloader/.
• Ensure the data loaders interact with the model-specific main files.
• Follow the naming conventions outlined in the ADR for any raw data.
• Implement minimal processing steps only if required, following the model configurations.