Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data set wont load on Windows #3

Open
wellscrosby opened this issue Feb 21, 2024 · 2 comments
Open

Data set wont load on Windows #3

wellscrosby opened this issue Feb 21, 2024 · 2 comments

Comments

@wellscrosby
Copy link

wellscrosby commented Feb 21, 2024

On Windows (didn't test Linux) I was getting the error:
Training data file "datasets\mnist\train-images-idx3-ubyte" not found; did you remember to download and extract it?: Os { code: 5, kind: permissiondenied, message: "access is denied." }
from running Mnist::new(r"datasets\mnist");

To get Mnist::new(r"datasets\mnist"); to work I had to move each of the idx3-ubyte files (from their folders with only the idx3-ubyte file inside after being extracted) into datasets\mnist\ and change the filenames const values in lib.rs to the following:

// Filenames
const TRAIN_DATA_FILENAME: &str = "train-images.idx3-ubyte";
const TEST_DATA_FILENAME: &str = "t10k-images.idx3-ubyte";
const TRAIN_LABEL_FILENAME: &str = "train-labels.idx1-ubyte";
const TEST_LABEL_FILENAME: &str = "t10k-labels.idx1-ubyte";

the only difference being a '.' instead of a '-' before 'idx3-ubyte' in each.

I'm not sure if I was doing anything wrong or if there is a better way for solving this. I used the built in file extraction tool in Windows 11 to decompress the .gz file, which might not be behaving how you expect (it created a folder with the name "train-images-idx3-ubyte" that has a file named "train-images.idx3-ubyte" in it from decompressing "train-images-idx3-ubyte.gz").

@busyboredom
Copy link
Owner

Hmm thanks for reporting this, I'll check it out. I never tested this on windows 11, so could definitely be a quirk with some change in default behavior for windows 11 vs 10. Or it could be a change in the mnist archive upstream.

@busyboredom
Copy link
Owner

Just tested on NixOS/KDE Plasma, and the files from http://yann.lecun.org/exdb/mnist/index.html are extracted as expected. So this does seem to be a Windows quirk. I'll make a fix this weekend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants