Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Catalog of Country-Month Level Models #61

Closed
5 of 7 tasks
Polichinel opened this issue Sep 12, 2024 · 0 comments · Fixed by #71 or #114 · May be fixed by prio-data/viewsforecasting#55
Closed
5 of 7 tasks

Create Catalog of Country-Month Level Models #61

Polichinel opened this issue Sep 12, 2024 · 0 comments · Fixed by #71 or #114 · May be fixed by prio-data/viewsforecasting#55
Assignees

Comments

@Polichinel
Copy link
Collaborator

Polichinel commented Sep 12, 2024

Objective

Compile a catalog of all country-month (cm) level of analysis (loa) models currently in the old pipeline that are being migrated to the new pipeline.

Requirements:

  • Catalog Content: Each model entry in the catalog must include the following details:

    • Algorithm: Specify the model's algorithm.
    • Target: Define the model's forecasting target.
    • Input Features: List the input features (query set) used by the model.
    • Non-default Hyperparameters: Include all non-default hyperparameters.
    • Forecasting Type: Specify the type of forecasting (e.g., direct multi-step forecasting for step-shifting models).
    • Implementation Status: Indicate whether the model has been implemented in the new pipeline (note: this does not refer to whether it is activated).
    • Implementation Date & Author: These should be listed as "NA" for models not yet implemented in the new pipeline.
  • File Format:

    • Create the catalog in a markdown file named cm_model_catalog.md.
    • The catalog should be structured as a table.
    • Store the file in the following directory:
      views_pipeline/documentation/catalogs/cm_model_catalog.md
      
  • Branch:

    • Create a new branch named create_cm_catalog_01 to work on this task.
  • Information Sources:

    • Use the current pipeline notebook to gather information about the country-month level models.
    • Consult Dr. Jim Dale if additional clarification is needed.
  • Review Process:

    • Upon completion, the catalog must be reviewed by HH, JD, and MC before final approval.

Additional Actions:

  • Glossary & ADRs:

    • Follow the existing templates for ADRs and glossary entries.
    • Document and create any necessary ADRs or glossary entries that emerge during the process.
  • Issues:

    • If any issues arise or further definitions are needed, document them as new GitHub issues.

Exemple of table (draft)

Model Name Algorithm Target Input Features Non-default Hyperparameters Forecasting Type Implementation Status Implementation Date Author
NA Random Forest ln_sb_best ... n_estimators=500, max_depth=10 Direct multi-step No NA NA
magical_mystery XGBoost ln_sb_best ... learning_rate=0.1, n_estimators=300 Direct multi-step Yes 15-09-2023 Jim Dale

With model name, implementation date, and author will be NA as long as the implementation is No


Summary of Actionable Steps:

  • Create a new branch named create_cm_catalog_01.
  • Compile the model catalog with all required details: algorithm, target, input features, non-default hyperparameters, forecasting type, implementation status, and implementation date/author (where applicable).
  • Format the catalog as a markdown table in cm_model_catalog.md, and store it in views_pipeline/documentation/catalogs/.
  • Consult the current pipeline notebook and reach out to Dr. Jim Dale as needed.
  • Submit the catalog for review by HH, JD, and MC.
  • Note and create ADRs or glossary entries following existing templates.
  • Log any issues encountered as GitHub issues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment