Merge pull request #159 from microsoft/mitokic/04272024/multistep_bug

fix feature engineering lags with multitep horizon

R/prep_data.R

@@ -419,7 +419,7 @@ prep_data <- function(run_info,
             multivariate_prep_recipe_1(external_regressors,
               xregs_future_values_list = xregs_future_list,
               get_fourier_periods(fourier_periods, date_type),
-              get_lag_periods(lag_periods, date_type, forecast_horizon, multistep_horizon),
+              get_lag_periods(lag_periods, date_type, forecast_horizon, multistep_horizon, TRUE),
               get_rolling_window_periods(rolling_window_periods, date_type),
               hist_end_date,
               date_type
@@ -591,7 +591,7 @@ prep_data <- function(run_info,
             multivariate_prep_recipe_1(external_regressors,
               xregs_future_values_list = xregs_future_list,
               get_fourier_periods(fourier_periods, date_type),
-              get_lag_periods(lag_periods, date_type, forecast_horizon, multistep_horizon),
+              get_lag_periods(lag_periods, date_type, forecast_horizon, multistep_horizon, TRUE),
               get_rolling_window_periods(rolling_window_periods, date_type)
             ) %>%
             dplyr::mutate(Target = base::ifelse(Date > hist_end_date, NA, Target))
@@ -929,13 +929,15 @@ get_fourier_periods <- function(fourier_periods,
 #' @param date_type date type
 #' @param forecast_horizon forecast horizon
 #' @param multistep_horizon multistep horizon
+#' @param feature_engineering adjust lags used for feature engineering
 #'
 #' @return list of lag periods
 #' @noRd
 get_lag_periods <- function(lag_periods,
                             date_type,
                             forecast_horizon,
-                            multistep_horizon = FALSE) {
+                            multistep_horizon = FALSE,
+                            feature_engineering = FALSE) {
   if (!is.null(lag_periods)) {
     return(lag_periods)
   }
@@ -950,11 +952,23 @@ get_lag_periods <- function(lag_periods,
   # change multistep horizons to run based on date type
   if (multistep_horizon) {
     if (date_type == "day") {
-      oplist <- c(28, 90, 180)
+      if (feature_engineering) {
+        oplist <- c(28, 60, 90, 180, 365)
+      } else {
+        oplist <- c(28, 90, 180)
+      }
     } else if (date_type == "week") {
-      oplist <- c(4, 12, 24)
+      if (feature_engineering) {
+        oplist <- c(4, 8, 12, 24, 48, 52)
+      } else {
+        oplist <- c(4, 12, 24)
+      }
     } else if (date_type == "month") {
-      c(1, 2, 3, 6, 12)
+      if (feature_engineering) {
+        oplist <- c(1, 2, 3, 6, 9, 12)
+      } else {
+        oplist <- c(1, 2, 3, 6, 12)
+      }
     }
 
     if (max(oplist) < forecast_horizon) {

tests/testthat/test-multistep_horizon.R

@@ -28,7 +28,7 @@ test_that("multistep_horizon monthly data", {
     back_test_scenarios = 2,
     models_to_run = "cubist",
     run_ensemble_models = FALSE,
-    num_hyperparameters = 1, 
+    num_hyperparameters = 1,
     pca = TRUE
   )
 

vignettes/models-used-in-finnts.Rmd

@@ -77,7 +77,7 @@ Ensemble models are trained on predictions made by individual models. For exampl
 
 ### Multistep Horizon Models
 
-By default within `prep_models()`, the `multistep_horizon` argument is set to TRUE. Meaning a multistep horizon approach is taken for specific multivariate models trained on the R1 feature engineering recipe. Below are the models that can run as multistep. 
+By default within `prep_models()`, the `multistep_horizon` argument is set to FALSE. If set to TRUE, a multistep horizon approach is taken for specific multivariate models trained on the R1 feature engineering recipe. Below are the models that can run as multistep. 
 
 - cubist
 - glmnet
@@ -86,7 +86,7 @@ By default within `prep_models()`, the `multistep_horizon` argument is set to TR
 - svm-rbf
 - xgboost
 
-A multistep model optimizes for each period in a forecast horizon. Let's take an example of a monthly data set with a forecast horizon of 3. When creating the features for the R1 recipe, finnts will create lags of 1, 2, 3, 6, 9, 12 months. Then when training a mulitstep model it will iteratively use specific features to train the model. First it will train a model on the first forecast horizon (H1), where it will use all available feature lags. Then for H2 it will use lags of 2 or more. Finally for H3 it will use lags of 3 or more. So the final model is actually a collection of multiple models that each trained on a specific horizon. This let's the model optimize for using all available data when creating the forecast. so in our example one glmnet model actually has three separate horizon specific models under the hood. 
+A multistep model optimizes for each period in a forecast horizon. Let's take an example of a monthly data set with a forecast horizon of 3. When creating the features for the R1 recipe, finnts will create lags of 1, 2, 3, 6, 9, 12 months. Then when training a mulitstep model it will iteratively use specific features to train the model. First it will train a model on the first forecast horizon (H1), where it will use all available feature lags. Then for H2 it will use lags of 2 or more. Finally for H3 it will use lags of 3 or more. So the final model is actually a collection of multiple models that each trained on a specific horizon. This lets the model optimize for using all available data when creating the forecast. So in our example, one glmnet model actually has three separate horizon specific models under the hood. 
 
 A few more things to mention. If `multistep_horizon` is TRUE then other multivariate models like arima-boost or prophet-xregs will not run a multistep horizon approach. Instead they will use lags that are equal to or greater than the forecast horizon. One set of hyperparameters will be chosen for each multistep model, meaning glmnet will only use one combination of final hyperparameters and apply it to each horizon model. Multistep models are not ran for the R2 recipe, since it has it's own way of dealing with multiple horizons. Finally if `feature_selection` is turned on, it will be ran for each horizon specific model, meaning for a 3 month forecast horizon the feature selection process will be ran 3 times. One for each combination of features tied to a specific horizon.