Plaintext Export

[LiamMorrow]

Would be good if users could export their data in a plaintext format like json or CSV.

JSON would be easiest, let's tackle that first

Lots of discussion here. It seems like CSV is the general consensus we won't entirely dismiss json, but the protobuf format is easily convertible to JSON so the first pass will be CSV. That being said, a JSON format would be EASILY added once the groundwork for exporting has been laid. The initial format will try to export as described by @bschwehn here. :

SessionId | TimeStamp           | Exercise Name      | Weight | Reps | Target Reps | Notes | Weight Units
|---------+---------------------+--------------------+--------+------+-------------+-------|
| 12344   | 2024-08-11T12:12:12 | Deadlift, Romanian | 50   |   12 |          12 |
| 12344   | 2024-08-11T12:14:12 | Deadlift, Romanian | 50   |   10 |          12 |
| 12344   | 2024-08-11T12:16:12 | Deadlift, Romanian | 50   |    8 |          12 |

What this IS

A feature which allows a user to EXPORT their data into a plaintext data format (CSV), uncompressed and human readable. There may be a couple trigger mechanisms for this, the MVP is a manual trigger which creates a file which the user can share. A potential trigger down the line could be to hook into the automated remote backups feature.

What this IS NOT

A mechanism of creating a back up of data. LiftLog will likely NEVER be able to read these files and import them. That is the sole responsibility of the BACKUP feature.
It is also NOT a stable format.

What it COULD be

These exports could be used for data analysis in various tools. However while the format will likely be fairly stable, there is always the chance that a field is renamed or the format changes in a slightly incompatible way. Such is the curse of CSV. An example of this is if Time based exercises are ever added

[LiamMorrow]

Note - I don't want this to serve as a backup option. Supporting backwards compatibility on plaintext is a pain (big reason I went with protobuf for the storage format).

This is a pure export

[LiamMorrow]

Might be good to investigate if there are any standard specifications for exporting gym tracking data.

[bschwehn]

I hope it is not too impertinent to describe what I, personally, would hope to achieve with such an export.

You are absolutely right that the purpose is not for this to function as a round-trippable backup. Instead it is to enable users to maintain a long-lived historic record of their exercise routine and progress, without locking them into using any particular single application that might no longer be available years down the line.

So I think the most important feature of the export is that it should make it easy to extract just the exercise data into a flat list like:

| TimeStamp           | Exercise Name      | Weight | Reps | Target Reps |
|---------------------+--------------------+--------+------+-------------|
| 2024-08-11T12:12:12 | Deadlift, Romanian | 50kg   |   12 |          12 |
| 2024-08-11T12:14:12 | Deadlift, Romanian | 50kg   |   10 |          12 |
| 2024-08-11T12:16:12 | Deadlift, Romanian | 50kg   |    8 |          12 |

I don't know if you record the data in such a way that the target reps can be derived easily for every exercise. If not, just don't include it.

Then maybe if LiftLog adds supports for taking notes for each exercise or to track isometric exercises:

| TimeStamp           | Exercise Name      | Weight | Reps/Time | Target Reps | Notes                 |
|---------------------+--------------------+--------+-----------+-------------+-----------------------|
| 2024-08-11T12:12:12 | Deadlift, Romanian | 50kg   |        10 |          12 | Backpain, aborted set |
| 2024-08-11T12:14:12 | Deadlift, Romanian | 50kg   |         0 |          12 |                       |
| 2024-08-11T12:14:12 | Deadlift, Romanian | 50kg   |         0 |          12 |                       |
| 2024-08-11T12:14:15 | Plank              | 0kg    |       60s |             |                       |

(not sure how skipped exercises could be logged)

or maybe:

| TimeStamp           | Exercise Name      | WeightInKg | Reps | TimeInSec | Target Reps | Target Time | Notes    |
|---------------------+--------------------+------------+------+-----------+-------------+-------------+----------|
| 2024-08-11T12:12:12 | Deadlift, Romanian |         50 |   10 |           |          12 |             | Backpain |
| 2024-08-11T12:14:15 | Plank              |          0 |      |        60 |             |          80 |          |

Exporting the sessions plans and exercise configurations might be nice to have, but is of secondary importance. Maybe they can be in separate files with the goal being that the schema of the primary exercise list does not change too often and things don't get too complicated.
For example, if the sessions are part of the export, they would really need to have the full history of the session data to be useful.

Exercises:

| TimeStamp           | Exercise Name      | Reps | Session      |
|---------------------+--------------------+------+--------------|
| 2024-08-11T12:12:12 | Deadlift, Romanian |   12 | A@2024-08-11 |
| 2024-08-11T12:14:12 | Deadlift, Romanian |   10 | A@2024-08-11 |
| 2024-08-11T12:16:12 | Deadlift, Romanian |    8 | A@2024-08-11 |
| 2024-08-13T15:12:12 | Deadlift, Romanian |   12 | A@2024-08-13 |
| 2024-08-13T15:15:12 | Deadlift, Romanian |   10 | A@2024-08-13 |
| 2024-08-13T15:18:12 | Deadlift, Romanian |    8 | A@2024-08-13 |

Sessions:

| TimeStamp           | Session Name | Session ID   | Weight | Target Reps | Break Time |
|---------------------+--------------+--------------+--------+-------------+------------|
| 2024-08-11T12:12:12 | Session  A   | A@2024-08-11 | 50kg   |          12 | 120s       |
| 2024-08-13T15:16:12 | Session  A   | A@2024-08-13 | 55kg   |          12 | 180s       |

And while this retains more information, it would also be more work to extract the data for the users. And for every feature you add (drop sets, other pyramid configurations) the schema would have to change. Therefore also here I think including as much data as is easily available (like the target reps / weight) in the primary list of exercises would be more convenient -- users that don't care about the details would be unaffected by any changes to the sessions list / format.

Now, if it is easiest to implement, I think it is totally fine to just dump everything into a JSON file of whatever format is most convenient. But I think for users that are less technically inclined a CSV export that they can simply copy in some Spreadsheet would be the easiest.

Exporting into a standardized format that allows import into other programs would be great of course, but I don't think it is realistic unfortunately. At least I am also not aware of any widely used standard format.

[LiamMorrow]

Important

Do note that this is me just being a stream of consciousness in the early planning stage of this feature. I'm not trying to convince you of "the best way", just saying ideas as they come into my head so that maybe we can use the best ideas from a bunch of ideas. I'll clean up what I aim to do once we get close to the ideal feature.

I hope it is not too impertinent to describe what I, personally, would hope to achieve with such an export.

Impertinent is not how I'd describe this at all. I appreciate the thorough requirements, it means I don't have to do double work trying to figure out what might be good for people, especially as this is a feature that I likely wouldn't use.

Then maybe if LiftLog adds supports for taking notes for each exercise or to track isometric exercises:

Just a side "note", this is already actually supported

Screen.Recording.2024-08-13.at.08.06.56.mov

(not sure how skipped exercises could be logged)

Yeah interesting. There IS a difference between doing 0 reps and not attempting a set in LiftLog (shows a dash when you have put nothing, compared to explicitly putting 0), so it would be possible to represent this.

Exporting the sessions plans and exercise configurations might be nice to have, but is of secondary importance. Maybe they can be in separate files with the goal being that the schema of the primary exercise list does not change too often and things don't get too complicated.

Yeah I'd probably suggest the same in that they go in separate files. Unfortunately their structure is something I do change often enough, and would like to keep doing. Maybe they can just have the stripped back form. As you say - this is for archival, not for restoration.

CLI Tangent

I'll have to really chew on this one for a bit I think. My thoughts are that I don't want to introduce a CSV format, then have it break over time just cause the app updated, as there's no easy way to downgrade apps in this world. I am toying with the idea of maybe having a CLI tool which can read .liftlogbackup files (which are just a gzipped protobuf file described by this). This CLI could have options to output to different formats, and different parts of a backup.

The pros being:

• I can put CLI versions as things change, but older versions are always downloadable. I'm also fairly careful not to break this format - because yeah backups can be years before they are used.
• No need to bundle a CSV/JSON library into the app - minor pro, but app size does matter somewhat - even if this is already a fairly large app
• Less complexity in the app - my fear is someone uses plaintext export to and assumes it acts as a roundtrip backup (mitigatable with warnings, but people ALWAYS ignore warnings)
• Whatever system you are using for backups has the actual backup file, so it could go back into liftlog with no issues

cons:

• Complexity for YOUR use case - you now need to be managing the conversion process. As you said you'd like it to be stored as plaintext
• Another tool to maintain and add features to

XML

I don't think I want to do XML, it's just a pain to read and write. However it does have merits for this structure of data. That's all I'll say on it.

[bschwehn]

Splitting the extraction of the data to various formats into a separate tool is an interesting idea. It would allow playing around with this idea without having to add too much to the main application. And should there prove to be demand, it could always be added to the main application later -- but for my use-case a separate tool would be a perfectly good solution long-term also.

It is something I would be much more comfortable to work on also, as I already know dotnet but not Maui / Blazor. I should have some time to POC something this weekend.

[bschwehn]

So I wrote a PoC for converting the export file to CSV, and with a little more polish this would work fine for me. https://github.com/bschwehn/LiftLogExport

[LiamMorrow]

That's awesome, if you like I can package up the types into a nuget package so that you don't need to copy them over into the repo

[bschwehn]

if you like I can package up the types into a nuget package

That would be useful actually if it is easy for you to do. Not a high priority as long as this is just something useful for only me...
I want to slowly add some improvements -- maybe at some point it will become more generally useful, but no promises :)

[flako-dd]

Hi!
Just my 2 cents from another advanced user perspective:
I think changing the export format to json makes most sense. I would love to be able to post-process the data with for example python. a csv table would be less universal from my perspective and more of a end-format. Would also be easier with feature like the http auto backup.

My flow would be: Auto backup to my server, on update extract and convert json to csv with python, correlate data with other apps (weight tracking e.g.)

[ddibiasi]

I'm also quite interested in this feature, to use the workout data in Jupyter notebooks.

In my professional opinion as data scientist and software engineer, a simple CSV or even JSON export would suffice. You must not necessarily conform to a standard, if there even is one, to make exported data useful. The usefulness of exported data is determined by the data / model description. Thinking about how your data can best be stored in DB tables is often the best approach.

I think exporting the data to plaintext should be done by the app itself and should not require a separate script. That said, unfortunately I can't support the development of this feature, as MAUI Blazor is not available for Linux.

[LiamMorrow]

That said, unfortunately I can't support the development of this feature, as MAUI Blazor is not available for Linux.

Oh, this app is also built (primarily) as a standalone blazor app. It is very possible to run on linux and much of the development was done on a linux box actually (until the speed of the m2 chips got the better of me and I crossed over to the dark side). If you're interested in having a go just pull it down and run the LiftLog.Web project (worth following the directions in the readme first).

However this does seem to be the most requested feature right now, so I may be able to find some time to work something out, likely based on @bschwehn 's suggested format above.
#244 (comment)

My MAIN hesitation for doing this feature is that I don't want people to build work off of a certain format, then I break that format as features are added. Not a huge issue with a single storage format that I have for the backups, since I try to NEVER break compatibility there (yknow, backups should be pretty resilient) , but protobuf lends itself well to refactors. But really the fact is it probably won't break or have to change for a long time.

Consider this prioritised.

[LiamMorrow]

I have merged the first pass of this, which enables CSV exporting in #316 🎉 Thanks for everyone's feedback! Please have a play with it in the browser at https://app.liftlog.online and see if it meets your needs. Feel free to suggest additional fields or a different format, as this is a feature which I personally won't be using it was a bit of a stab in the dark.

If there are no glaring issues I will try and get a release out with it early next week