Fix table grouping

[ebardelli]

Description

This PR removes the groupDataPopulated flag from _DataTable. This flag wasn't updated when the data object was modified, leading to the DataTable being frozen after the first execution, making grouped DataTables non-interactive.

Closes #2850

Checklist

• For UI or styling changes, I have added a screenshot or gif showing before & after
• I have added a changeset
• I have added to the docs where applicable
• I have added to the VS Code extension where applicable

[archiewood]

Thank you for your contribution! No problems in CI here.

Are you able to create a simple story to test this functionality?

[ebardelli]

I'm not sure about a story (I assume it's an automated test for the component, right?). Is there a way to change an input through a story?

[archiewood]

It's more a playground so you can set up the component with context and then check it delivers the expected behaviour.

I assume to confirm this works, you need a grouped table wired up to an input that filters it?

[archiewood]

Here's a simple example:

https://github.com/evidence-dev/evidence/blob/bf7b469f166c5ae9c99b3a21d030a749c1b45adf/packages/ui/core-components/src/lib/unsorted/viz/table/DataTable.stories.svelte#L42C1-L45C9

[ebardelli]

Ok, let me see what I can do. Is there a way to set the expected output?

[zachstence]

Ok, let me see what I can do. Is there a way to set the expected output?

We use Chromatic which takes screenshots of our components and then compares them to ensure nothing changes unless its meant to. So the "expected output" will be whatever the result of your story is, and will be validated on every PR going forward.

[ebardelli]

Sorry for the slow response.

Here's the thing. I don't think I can write a story for this because the bug appears only after the table is rendered and an input is changed.

See this for a minimal reproduction page using the template database:

    ```sql orders
    select * from needful_things.orders
    ```
    
    <Dropdown
        data={orders} 
        name=filter_group
        value=category
    />
    
    ```sql orders_filtered
    select category, channel, sum(sales) as Sales
    from needful_things.orders
    where category like '${inputs.filter_group.value}'
    group by all
    ```
    
    Filtered Row Count: {orders_filtered.length}
    
    <DataTable data={orders_filtered} totalRow=true/>
    
    <DataTable data={orders_filtered} groupBy=category totalRow=true/>

Here is a visual:

The groupDataPopulated variable pins the values in the tables and prevents the table from refreshing when the dropdown changes the data. The ungrouped table and the total row in the grouped data correctly display the results because they aren't pinned.

Is there a way to code a change in a dropdown in a story?

[archiewood]

It’s enough to just set it up i think. In this case you (and we) can verify the fix by manipulating the story in the browser

[kwongz]

Added Chromatic Group by FilterData Story

Screen.Recording.2024-12-20.161719.mp4

[kwongz]

Looking at the UI test, one of the groupBy Sort test failed, did not sort the group by column properly. I'll look into this, but the removal of the prop seems to have fixed the dataTable groupBy with filtered Data

[ebardelli]

Thank you for the storybook update. I didn't account for the new group sorting behavior. I'll also take a look after Christmas if it's still open.

[ebardelli]

I pushed an update. It looks like moving the grouping logic before the sorting block fixes the grouped sorting. Let's see if all the tests pass now.

[kwongz]

I was still experiencing the issue with group sort issue with the moved grouping logic.

Russia doesn't sort into ascending or descending in Europe Group

Screen.Recording.2024-12-30.160554.mp4

Looking into the code, the groupDataPopulated prop was added to help properly handle the group sort in a previous issue. But it did introduce this new problem @ebardelli has been experiencing.

A change I found that helped is adding a check for when data changes to update the groupDataPopulated prop

Filtered Data from Input

Screen.Recording.2024-12-30.155101.mp4

Group Sorting Still Works

Screen.Recording.2024-12-30.155751.mp4

[kwongz]

The chromatic test Looks good, no new changes to sorting, and filtering data behavior looks solid

Merging

@ebardelli Thanks for your contribution!