Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilities for reading exported spreadsheets #102

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

clbarnes
Copy link

I had some code I found useful for parsing the human-readable (blegh) spreadsheet exports, and thought they might be a useful addition here.

@elliot-100
Copy link
Collaborator

Hi, thanks for this.

I'm not sure this fits in this package as it doesn't seem to directly add to or use any of its code, and is outside the core purpose which is to extract data via the API; in fact it relies on having previously extracted data by other means (spreadsheet download).

Could you advise if there info that can be retrieved this way that isn't available from the API?

@Olen what do you think?

@Olen
Copy link
Owner

Olen commented Apr 20, 2024

I was about to ask the same. I have not had time to look at the code or docs yet, so I am not sure what this does.

@elliot-100 elliot-100 marked this pull request as draft April 20, 2024 12:54
@clbarnes
Copy link
Author

clbarnes commented Apr 21, 2024

I am not sure what this does

It provides 2 functions for parsing the excel sheets which spond can generate; one representing poll results, one for group members. It produces an iterator of well-defined objects in each case.

it doesn't seem to directly add to or use any of its code, and is outside the core purpose which is to extract data via the API; in fact it relies on having previously extracted data by other means (spreadsheet download).

You're right that it doesn't intersect with any current functionality. However, it seemed a good idea to offer the code here as a one-stop shop for people interacting with spond data, rather than having it languish in my own minuscule repo.

Could you advise if there info that can be retrieved this way that isn't available from the API?

There isn't currently a method for getting poll results. Getting human-readable group memberships and user info fields is also a fair bit of effort because you have to manually match up IDs. I also saw a use case for doing analysis of static snapshots of data rather than fetching the data with every run.

I would understand if you prefer not to accept this contribution, but I would encourage the expansion of the scope of this repo to allow the very small intersection of spond admins / python users to have just the one package they need :)

@elliot-100
Copy link
Collaborator

elliot-100 commented Apr 22, 2024

There isn't currently a method for getting poll results.

True - I haven't looked into it and I don't think anyone else has; not sure how easy it would be.

Getting human-readable group memberships and user info fields is also a fair bit of effort because you have to manually match up IDs.

This was one reason I started on Spond-classes - it derives members, subgroups and roles (and their relationships) from group data.

Member.profile_uid can be used to definitively identify a Spond user across Group memberships. See where it comes from in https://github.com/elliot-100/Spond-classes/blob/main/spond_classes/member.py

I also saw a use case for doing analysis of static snapshots of data rather than fetching the data with every run.

It's not built in (yet) but I use something like https://github.com/Olen/Spond/blob/main/groups.py to dump JSON periodically, then Spond-classes to analyse and reconcile against spreadsheet data from another system without API.

@Olen
Copy link
Owner

Olen commented Apr 22, 2024

I would very much like to include spond-classes into this library, but I don't know how mature it is yet.
I think it is the right direction.

It will be breaking change, but that will always happen from time to time.

Regarding the parsing of polls etc, I think it's fine to include a script to do so. Just give it a descriptive name and please add some documentation.

@clbarnes
Copy link
Author

I think it's more useful as an importable function than as a standalone script, where users of this library can incorporate into their own scripts. My use case was cross-referencing two different polls with subgroup memberships - it wouldn't have been very useful to have needed one script to dump the data to JSON and then another to read that JSON into the necessary data structures.

What do you think about using something like pdoc3 to generate documentation automatically based on type annotations and docstrings? The exports module I added is typed and documented in the code. For my CLIs, rather than restate their usage in the README, I use https://pypi.org/project/pipe2codeblock/ to pipe the tool's own help text into a literal block.

If spond-classes or something equivalent does make it in here (e.g. using pydantic, as I suggested on that repo), then the exports module could re-use the same data types.

@Olen
Copy link
Owner

Olen commented Apr 22, 2024

Self-documenting is fine, but I would also like to have at least a summary in the README to allow for a quick understanding of the possibilities without installing and running any code.

@elliot-100
Copy link
Collaborator

elliot-100 commented Apr 22, 2024

On reflection my comments above were fairly tangential/irrelevant. I didn't mean to suggest including spond-classes in the library at this point (or necessarily any point). I think for some reason I misread your User class as being general purpose, rather than specific to this spreadsheet parsing case. That's why I deviated into using profile id to match users....

I shall try it out.

Only thought for now is that perhaps a more specific package name like xls_export_processing or read_xls_export would be useful.

@elliot-100 elliot-100 added the enhancement New feature or request label Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants