You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There's an issue with the parquet files generated by to_parquet when by_layer=False
Copied from code comment:
Divisions cannot be generated from a parquet file that stores nested information without a reset_index().set_index() loop. It seems like this happens at the to_parquet level rather than in read_parquet as dropping the nested columns from the dataframe to save does enable divisions to be found, but removing the nested columns from the set of columns to load does not. Divisions are going to be crucial, and so I think it's best to not support this until this is resolved.
The text was updated successfully, but these errors were encountered:
There's an issue with the parquet files generated by to_parquet when
by_layer=False
Copied from code comment:
Divisions cannot be generated from a parquet file that stores nested information without a reset_index().set_index() loop. It seems like this happens at the to_parquet level rather than in read_parquet as dropping the nested columns from the dataframe to save does enable divisions to be found, but removing the nested columns from the set of columns to load does not. Divisions are going to be crucial, and so I think it's best to not support this until this is resolved.
The text was updated successfully, but these errors were encountered: