Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix as_result_df for nested rbind #818

Merged
merged 9 commits into from
Feb 21, 2024

Conversation

Melkiades
Copy link
Contributor

@Melkiades Melkiades commented Jan 23, 2024

Fixes #815 and #809

Having multiple nested rbind is usually a problem as it breaks the structure of the table completely. In other words, we should find a way to flatten rbind_roots nodes, or remove them in {tern} (tabulate_survival_subgroups uses them), or establish a way to represent these in the table tree.

I would suggest going for the following:

  • Harmonize flattening of rbinds with proper representation of the "split" with at least its visible label rows as split values (different representation)
  • Remove rbinds from {tern} when not from different data (I think the example case I got could be done with a split as the filtering variable can be the split) Remove rbind nesting from {tern} - e.g. tabulate_survival_subgroups tern#1181
  • Add LabelRows to the result df (probably with another result_df_* function)

Copy link
Contributor

github-actions bot commented Jan 23, 2024

badge

Code Coverage Summary

Filename                     Stmts    Miss  Cover    Missing
-------------------------  -------  ------  -------  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
R/00tabletrees.R               745      62  91.68%   21, 102, 105, 412, 496-497, 500, 656, 757, 849-850, 951, 953-954, 977-980, 1002, 1114-1117, 1212-1217, 1365, 1462-1465, 1529-1532, 1568-1571, 1577-1582, 1632, 1639, 1739, 1855, 1869, 1872-1875, 1878-1881, 1909, 1940-1941
R/as_html.R                    161      25  84.47%   5-10, 74, 131-136, 141-146, 161-165, 253
R/colby_constructors.R         560      20  96.43%   71, 123, 181-184, 244-247, 387, 404, 1213, 1306, 1468, 1506, 1528, 1552, 1573, 1729
R/compare_rtables.R             83      17  79.52%   93-96, 99-102, 115-118, 137, 157-158, 189, 194
R/format_rcell.R                12       0  100.00%
R/indent.R                      13       2  84.62%   39-40
R/index_footnotes.R             66       0  100.00%
R/make_split_fun.R             119      23  80.67%   22-25, 51-54, 57-60, 115, 119, 280, 283-286, 291-294, 313, 412
R/make_subset_expr.R           136      14  89.71%   34-48, 126-133, 168, 250, 266, 274
R/simple_analysis.R              5       1  80.00%   55
R/split_funs.R                 505      66  86.93%   143, 148, 154-159, 164, 181-185, 366-371, 388-393, 474, 526, 544-547, 564, 631, 641-642, 644, 658, 702, 727, 903, 910, 936-939, 950-951, 953, 955, 1126-1128, 1142-1146, 1210-1213, 1276-1279
R/summary.R                    215      24  88.84%   38, 85, 192, 200, 271-276, 287-288, 307-308, 418, 465-481, 516, 549
R/tree_accessors.R             952     102  89.29%   109, 251, 269, 292, 330, 344, 360, 465, 492-493, 774-779, 907, 925, 949, 999, 1054-1055, 1094, 1127, 1163-1167, 1223, 1298-1302, 1320-1330, 1399, 1504-1507, 1532, 1552-1553, 1562, 1603, 1621-1625, 1646-1650, 1729, 1771, 1875, 1979, 1992, 2005, 2019, 2027, 2036-2040, 2382, 2740, 2853, 2886-2908, 2997-3004, 3159, 3232-3237, 3447-3448, 3455, 3458-3461, 3465, 3516, 3576, 3601-3625
R/tt_afun_utils.R              411      32  92.21%   50, 164, 171, 181-194, 260, 271-272, 504, 512-515, 597-601, 622, 637-639
R/tt_compare_tables.R           70       4  94.29%   56, 178, 257, 261
R/tt_compatibility.R           510      56  89.02%   19, 142-143, 192, 197, 332-333, 337-340, 346, 350, 398, 520, 568, 601, 621, 654-657, 701, 718-722, 809, 837-840, 849, 912, 920, 931-934, 1044, 1051, 1080-1094, 1125-1126
R/tt_dotabulation.R           1122      96  91.44%   54, 252, 257, 259, 310, 334, 338-341, 373-376, 399, 434-437, 465-468, 565, 702-706, 756, 760, 789-792, 802, 822-826, 833-836, 1095, 1099, 1130, 1241-1244, 1449-1457, 1598, 1682-1691, 1771-1774, 1785, 1790, 1795-1796, 1798, 1809, 1814, 1837, 1932-1951
R/tt_export.R                  512      31  93.95%   44, 181-185, 234-237, 289-292, 437, 443, 469, 523, 816, 825, 850-854, 1021-1024, 1027, 1058, 1064
R/tt_from_df.R                  16       0  100.00%
R/tt_paginate.R                440      37  91.59%   45, 70, 107-115, 396, 518-521, 541-545, 695-698, 748-755, 824, 827, 837, 844, 847
R/tt_pos_and_access.R          571      43  92.47%   77, 79-81, 106, 166, 212-216, 262, 515, 517, 525, 531, 545, 555-558, 740, 751-755, 760-763, 790, 843-844, 856, 1022-1023, 1081-1109, 1389, 1466
R/tt_showmethods.R             144      21  85.42%   60, 97-120, 183, 209, 218, 226, 229-233, 326-327
R/tt_sort.R                     88       5  94.32%   223-226, 234
R/tt_toString.R                387      27  93.02%   119, 323-326, 332, 347, 357, 364, 367, 373-383, 471, 536, 542, 777-803
R/utils.R                       29       0  100.00%
R/validate_table_struct.R       84      10  88.10%   79-83, 92-93, 140, 150-151
R/Viewer.R                      61       9  85.25%   47, 51, 61-65, 85, 119
TOTAL                         8017     727  90.93%

Diff against main

Filename                     Stmts    Miss  Cover
-------------------------  -------  ------  --------
R/00tabletrees.R              +121     +21  -1.75%
R/as_html.R                    +73     +18  -7.57%
R/colby_constructors.R         +78      +3  -0.04%
R/compare_rtables.R             +5      +6  -6.38%
R/index_footnotes.R            +16       0  +100.00%
R/make_split_fun.R            +119     +23  +80.67%
R/make_subset_expr.R           +30      +2  +1.03%
R/split_funs.R                 +84     +11  -0.01%
R/summary.R                    +32      +8  -2.42%
R/tree_accessors.R            +159     +34  -2.14%
R/tt_afun_utils.R              +66      +7  -0.54%
R/tt_compare_tables.R           +5       0  +0.44%
R/tt_compatibility.R           +97      +6  +1.13%
R/tt_dotabulation.R           +385     +52  -2.59%
R/tt_export.R                 +283     -42  +25.82%
R/tt_from_df.R                  +7       0  +100.00%
R/tt_paginate.R                +57     +22  -4.49%
R/tt_pos_and_access.R          +51      +6  -0.42%
R/tt_showmethods.R             +23       0  +2.77%
R/tt_sort.R                     +7      -1  +1.73%
R/tt_toString.R                +82      +5  +0.24%
R/utils.R                      +18      -1  +9.09%
R/validate_table_struct.R      +84     +10  +88.10%
R/Viewer.R                      +5       0  +1.32%
TOTAL                        +1887    +190  -0.31%

Results for commit: bd4b284

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Jan 23, 2024

Unit Tests Summary

    1 files     24 suites   1m 30s ⏱️
  201 tests   201 ✅ 0 💤 0 ❌
1 524 runs  1 524 ✅ 0 💤 0 ❌

Results for commit bd4b284.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jan 23, 2024

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
Result Data Frames 💔 $2.03$ $+2.66$ $+15$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
Result Data Frames 👶 $+1.19$ as_result_df_as_is_is_producing_a_data.frame_that_is_compatible_with_df_to_tt
Result Data Frames 👶 $+1.15$ as_result_df_keeps_label_rows
Result Data Frames 👶 $+0.44$ as_result_df_works_fine_also_with_multiple_rbind_root

Results for commit c1714cb

♻️ This comment has been updated with latest results.

Comment on lines +215 to +221
init_tbl <- df_to_tt(mtcars)
end_tbl <- init_tbl %>% as_result_df(as_is = TRUE) %>% df_to_tt()

expect_equal(
matrix_form(init_tbl)$strings,
matrix_form(end_tbl)$strings
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @shajoezhu could you check this? Tell me if it is as you want it. The above example works when there is a complex table and non-unique rownames

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brilliant! i was wondering why it has non-unique rownames (what about appending the multi-level rownames, and make them unique?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rownames need to be unique but if you create a table using {rtables} it happens that they are not unique. If that is the case as_result_df makes a label_name column with the final row names and if that column is present df_to_tt uses it for the table

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would suggest we take look into the tree structure, i feel that we need to go deeper, and adding more elements at the node level

if I do the following, i see it starts to appending new columsn

> df_to_tt(mtcars) %>% as_result_df()
             avar_name            row_name          label_name row_num is_group_summary node_class  mpg cyl  disp  hp drat    wt  qsec vs am gear carb
1            Mazda RX4           Mazda RX4           Mazda RX4       1            FALSE    DataRow   21   6   160 110  3.9  2.62 16.46  0  1    4    4
2        Mazda RX4 Wag       Mazda RX4 Wag       Mazda RX4 Wag       2            FALSE    DataRow   21   6   160 110  3.9 2.875 17.02  0  1    4    4
3           Datsun 710          Datsun 710          Datsun 710       3            FALSE    DataRow 22.8   4   108  93 3.85  2.32 18.61  1  1    4    1
4       Hornet 4 Drive      Hornet 4 Drive      Hornet 4 Drive       4            FALSE    DataRow 21.4   6   258 110 3.08 3.215 19.44  1  0    3    1
5    Hornet Sportabout   Hornet Sportabout   Hornet Sportabout       5            FALSE    DataRow 18.7   8   360 175 3.15  3.44 17.02  0  0    3    2
6              Valiant             Valiant             Valiant       6            FALSE    DataRow 18.1   6   225 105 2.76  3.46 20.22  1  0    3    1
7           Duster 360          Duster 360          Duster 360       7            FALSE    DataRow 14.3   8   360 245 3.21  3.57 15.84  0  0    3    4
8            Merc 240D           Merc 240D           Merc 240D       8            FALSE    DataRow 24.4   4 146.7  62 3.69  3.19    20  1  0    4    2
9             Merc 230            Merc 230            Merc 230       9            FALSE    DataRow 22.8   4 140.8  95 3.92  3.15  22.9  1  0    4    2
10            Merc 280            Merc 280            Merc 280      10            FALSE    DataRow 19.2   6 167.6 123 3.92  3.44  18.3  1  0    4    4
11           Merc 280C           Merc 280C           Merc 280C      11            FALSE    DataRow 17.8   6 167.6 123 3.92  3.44  18.9  1  0    4    4
12          Merc 450SE          Merc 450SE          Merc 450SE      12            FALSE    DataRow 16.4   8 275.8 180 3.07  4.07  17.4  0  0    3    3
13          Merc 450SL          Merc 450SL          Merc 450SL      13            FALSE    DataRow 17.3   8 275.8 180 3.07  3.73  17.6  0  0    3    3
14         Merc 450SLC         Merc 450SLC         Merc 450SLC      14            FALSE    DataRow 15.2   8 275.8 180 3.07  3.78    18  0  0    3    3
15  Cadillac Fleetwood  Cadillac Fleetwood  Cadillac Fleetwood      15            FALSE    DataRow 10.4   8   472 205 2.93  5.25 17.98  0  0    3    4
16 Lincoln Continental Lincoln Continental Lincoln Continental      16            FALSE    DataRow 10.4   8   460 215    3 5.424 17.82  0  0    3    4
17   Chrysler Imperial   Chrysler Imperial   Chrysler Imperial      17            FALSE    DataRow 14.7   8   440 230 3.23 5.345 17.42  0  0    3    4
18            Fiat 128            Fiat 128            Fiat 128      18            FALSE    DataRow 32.4   4  78.7  66 4.08   2.2 19.47  1  1    4    1
19         Honda Civic         Honda Civic         Honda Civic      19            FALSE    DataRow 30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
20      Toyota Corolla      Toyota Corolla      Toyota Corolla      20            FALSE    DataRow 33.9   4  71.1  65 4.22 1.835  19.9  1  1    4    1
21       Toyota Corona       Toyota Corona       Toyota Corona      21            FALSE    DataRow 21.5   4 120.1  97  3.7 2.465 20.01  1  0    3    1
22    Dodge Challenger    Dodge Challenger    Dodge Challenger      22            FALSE    DataRow 15.5   8   318 150 2.76  3.52 16.87  0  0    3    2
23         AMC Javelin         AMC Javelin         AMC Javelin      23            FALSE    DataRow 15.2   8   304 150 3.15 3.435  17.3  0  0    3    2
24          Camaro Z28          Camaro Z28          Camaro Z28      24            FALSE    DataRow 13.3   8   350 245 3.73  3.84 15.41  0  0    3    4
25    Pontiac Firebird    Pontiac Firebird    Pontiac Firebird      25            FALSE    DataRow 19.2   8   400 175 3.08 3.845 17.05  0  0    3    2
26           Fiat X1-9           Fiat X1-9           Fiat X1-9      26            FALSE    DataRow 27.3   4    79  66 4.08 1.935  18.9  1  1    4    1
27       Porsche 914-2       Porsche 914-2       Porsche 914-2      27            FALSE    DataRow   26   4 120.3  91 4.43  2.14  16.7  0  1    5    2
28        Lotus Europa        Lotus Europa        Lotus Europa      28            FALSE    DataRow 30.4   4  95.1 113 3.77 1.513  16.9  1  1    5    2
29      Ford Pantera L      Ford Pantera L      Ford Pantera L      29            FALSE    DataRow 15.8   8   351 264 4.22  3.17  14.5  0  1    5    4
30        Ferrari Dino        Ferrari Dino        Ferrari Dino      30            FALSE    DataRow 19.7   6   145 175 3.62  2.77  15.5  0  1    5    6
31       Maserati Bora       Maserati Bora       Maserati Bora      31            FALSE    DataRow   15   8   301 335 3.54  3.57  14.6  0  1    5    8
32          Volvo 142E          Volvo 142E          Volvo 142E      32            FALSE    DataRow 21.4   4   121 109 4.11  2.78  18.6  1  1    4    2

however, if i start appending to this operation, it does not go back

 df_to_tt(mtcars) %>% as_result_df() %>% df_to_tt()  %>% as_result_df()

Copy link
Collaborator

@shajoezhu shajoezhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thanks @Melkiades

@Melkiades Melkiades merged commit 4908f3a into main Feb 21, 2024
17 checks passed
@Melkiades Melkiades deleted the 815_fix_as_result_df_for_rbind@main branch February 21, 2024 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants