Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten tool parameters need updating for BODS 0.4 #107

Open
kd-ods opened this issue Nov 26, 2024 · 4 comments
Open

Flatten tool parameters need updating for BODS 0.4 #107

kd-ods opened this issue Nov 26, 2024 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@kd-ods
Copy link
Collaborator

kd-ods commented Nov 26, 2024

Bug description

At the moment, the flattened version of submitted BODS 0.4 data isn't quite right. If you run this file through the tool, you'll see that there is no 'statementId' column on sheets 2 - 5 which would be needed to join the rows with the relevant rows on sheet 1.

Suggested fix

The problem is that the parameters passed to flatten-tool here are those for 0.3 and earlier. For 0.4, I think that id_name and root_id need to be 'statementId' and not 'statementID'.

@radix0000
Copy link
Collaborator

This specific problem should now be fixed in: http://bods-cove-addbods04.dokkubodsdev1.ods.mobi

Will close this issue once the inverse problem is also fixed.

@kd-ods
Copy link
Collaborator Author

kd-ods commented Nov 28, 2024

Yes - let's ultimately test this @radix0000 by round-tripping some data from JSON -> xlsx -> JSON and checking that the checks give the same output along the way.

@radix0000
Copy link
Collaborator

Inverse has been added. Interestingly JSON -> xlsx -> JSON creates a load of extra errors even for BODS 0.3 data with the old code. Investigating.

@radix0000
Copy link
Collaborator

The obvious issues with this round trip in BODS 0.3 are the booleans (true/false in JSON) end up being strings of the Python boolean names ("True"/"False"), and also keys with a value of an empty list disappear along the way.

@kd-ods kd-ods moved this from Ready to In Progress in Data Review Tool update (for BODS 0.4) Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

2 participants