Historic version of category data in database dumps

[aaron-lohmann]

I am using in a research project the database dump crates. Though, the category data is not time-stamped and only refers to the most recent point in time. Is it possible to either make available older data dumps or add a time stamp like 'created_at' which shows when a certain category was created for a crate?

Thanks for the great work!

[Turbo87]

Hi @aaron-lohmann, if you are interested in the history of the categories the easiest way to retrieve that would be to look at the git history of the corresponding definition file in the repository: https://github.com/rust-lang/crates.io/commits/main/src/boot/categories.toml

[aaron-lohmann]

@Turbo87 Thanks for reaching out and your answer. This is the option I am considering at the moment and might end up going for regardless, the problem is only the scale as I will need to do that for the entire ecosystem of packages hosted on crates. Thus, I was checking whether there might an easier possibility.

[Turbo87]

make available older data dumps

we don't archive the old data dumps, so that is unfortunately not an option

or add a time stamp like 'created_at' which shows when a certain category was created for a crate?

we don't have that data either and would have to infer it from the git history too

[aaron-lohmann]

Thanks for clarifying! Will try my best to retrieve the information from the .toml files. For anyone how might be interested, I follow a rather wacky solution of downloading old files like so

https://raw.githubusercontent.com/dtolnay/syn/HEAD@%7B2019-04-29%7D/Cargo.toml

If anyone has a better idea, happy for suggestions.