hierarchical iterations #52
Conversation
cyclic_boosting/base.py
Outdated
| idea of such hierarchical iterations is to support the modeling of | ||
| hierarchical or causal effects (e.g., mitigate confounding). | ||
|
|
||
| If this argument is omitted, such no hierarchical iterations are run. |
There was a problem hiding this comment.
A rather pedantic comment. This argument cannot be omitted (in the strict sense that it is not defined) because in the __init__ of CyclicBoostingBase, hierarchical_feature_groups is set to None and not =Optional[Union[str, int, Tuple, FeatureID]]. This means that hierarchical_feature_groups = Nonehas to be passed to the child classes ofCyclicBoostingBase`.
TL:DR, omitted makes it sound like this argument is Optional but is not. It is mandatory but has a default None. Perhaps "If this argument is not explicitly set, such no hierarchical iterations are run." is better?
There was a problem hiding this comment.
Will change it, thanks.
cyclic_boosting/base.py
Outdated
| @@ -887,8 +920,11 @@ def _check_stop_criteria(self, iterations: int, convergence_parameters: Converge | |||
| "analysis plots." | |||
| ) | |||
|
|
|||
| if iterations <= 3 and self.hierarchical_feature_groups is not None: | |||
There was a problem hiding this comment.
Rather ignorant question, why is 3 the magical number of iterations here? Is it a good balance between two criteria?
If this is in fact a parameter one can tune, than 3 should be replaced by a constant, say "TRAINING_ITERATIONS_HIER_FEATURES" and the docstring above has to be adjusted.
There was a problem hiding this comment.
I think 3 is a good value, but a parameter with default 3 might be better, yes. I will change it.
| yhat = CB_est.predict(X.copy()) | ||
|
|
||
| mad = np.nanmean(np.abs(y - yhat)) | ||
| np.testing.assert_almost_equal(mad, 1.699, 3) |
There was a problem hiding this comment.
Can I read this slight improvement in MAD (compared to the previous test) as a result of feature hierarchization, or is the difference too small to attribute it to that?
There was a problem hiding this comment.
The improvement is small, yes, but I think it is thanks to the hierarchical training. Here it helps to describe the strong confounding of the price due to different products. I couldn't find a better example in our integration test.
In the first three iterations of the training, only selected feature groups are used, i.e., all other feature groups are excluded. From the fourth iteration onwards, all feature groups are used. The idea of such hierarchical iterations is to support the modeling of hierarchical or causal effects (e.g., mitigate confounding).