Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Awesome work! Here's 57 million run history records. #1

Open
JakeRabinowitz opened this issue Jul 8, 2020 · 2 comments
Open

Awesome work! Here's 57 million run history records. #1

JakeRabinowitz opened this issue Jul 8, 2020 · 2 comments

Comments

@JakeRabinowitz
Copy link

JakeRabinowitz commented Jul 8, 2020

Hi there, this is super impressive work! I run the Slay the Spire internal metrics dashboard, and was running a run history export script from October 2018 -> July 2019. Over that time I accrued about 57 million runs worth of metrics, with the idea that they could be useful for projects like this one. I turned off the data export a year ago, but I'll turn it on again for a while to get some more up to date data for you.

Here's a link to the run files I have now: https://drive.google.com/drive/folders/1c7MwTdLxnPgvmPbBEfNWa45YAUU53H0l?usp=sharing

There are over 35,000 files, each contains ~1,600 runs in a json list. The file names are [timestamp]#[runcount].json.gz. I recommend you use a faster json library than the default 😉 I find ujson to be pretty effective.

@alexdriedger
Copy link
Owner

Thank you so much! I'll try training the model with the new data

@JakeRabinowitz
Copy link
Author

Oh just fyi: If you're trying to split runs within a file into training/test sets, you need to select runs randomly instead of picking a range of runs for the test set, since the runs files are organized/grouped by the DB. If you just designate entire run files as test data, that should be fine since they're just exports of the past 10 minutes worth of runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants