extensions cannot be used when scanning

[alexcanana]

Hello,

I tried using your tool but I get an "extensions cannot be used when scanning" error.
Here's the command I used:

python generate.py -r USA,EUR,JPN -l en -e zip --exclude "Virtual Console,GameCube" --no-all --all-regions- -d "../Nintendo - Nintendo Entertainment System (Parent-Clone) (20200405-041607).dat" -i "../Nintendo - Nintendo Entertainment System" -o "../NES"

Of course, I tried removing the extension, but it turns into a invalid DAT file error.

Any idea on why this happens?

Thank you!

[alexcanana]

Hi again,

Managed to make it work by removing the -e option.

In the wiki, there's a mention of a --use-hashes option, but adding it to my command will trigger a option --use-hashes not recognized error.

Thank you.

[andrebrait]

Hey there,

Yes, you have to remove the whole option (which means both the key -e and its value).

Regarding --use-hashes, the wiki is outdated. It is now the default option. I have to update a lot of things here :-(

Now that it is the default, you can disable it by using --no-scan, but I wouldn't recommend it, unless you know your file names are 100% up-to-date. Matching by hash (i.e. scanning files) is a superior method.

[alexcanana]

Thanks for your reply.

No worries, it's working now, no problem.
Thank you for a useful tool!

[HVR88]

Is this an error/bug in 1.9.1? Or was there a bug in earlier versions? Or that extensions are no longer needed because of the default hash checking? The behavior is now different of course.

The script fails when specifying a file extension. I'm using the same command line I used on all previous versions and now receiving this error only after updating to 1.9.1

As I understood it, the extension is specified to match the files being scanned, as no such archive extension may exist in the dat.

If this is because of hash checking not needing the file extension, what is the remaining purpose of the -e option?

[andrebrait]

Yes, the behavior changed with 1.9.0, I think (1.9.1 contains no breaking changes relative to 1.9.0)

The extension is now only used for pure filename matching (i.e. when scanning is disabled).

Otherwise, it tries to find matches by file hashes. Since this is a superior method (and it doesn't matter if your .sfc roms are .smc in your set, for example) that's guaranteed to get the right ROMs every time, it's now the default.

So, the good news is that the extension parameter is totally unnecessary unless you don't want the tool to calculate hashes at all.

[HVR88]

OK, sounds good. I'd recommend ignoring it if present in this situation then, maybe showing a soft error, but not stopping.

[HVR88]

Also, are you doing some renaming when copying?

[andrebrait]

OK, sounds good. I'd recommend ignoring it if present in this situation then, maybe showing a soft error, but not stopping.

I usually tend to write software in a way that prevents the user from getting an unexpected result. If you add a flag, it should do something, unless explicitly ignored in the docs, for example.

Normally, I would add a note in the documentation and be done with it, but since the extension option was there from the start, way before scanning was ever even an idea for me, I decided to make its usage while scanning be prohibited.

[andrebrait]

Also, are you doing some renaming when copying?

Yes.

With scanning, all files are renamed to "Game Name" + proper extension if uncompressed, or .zip if compressed (because it currently only support zips)

[andrebrait]

So, if you have file

SupErMarioWorldUsa.bla

and its hash matches that of the real ROM, it becomes copied as

Super Mario World (USA) (Rev 1).sfc

(If that is the name of the ROM in the DAT)

[HVR88]

So if the archive format isn't zip then your tool won't be able to open it to check the hashes of the files I suppose...

What's the correct procedure then? Go back to the older non-scan behavior by just checking filenames against p/clone data?

[mspykerez]

OH thats why my .7z files didn't work.

[andrebrait]

Yep ;-)

Support for other archive formats is planned but I don't know when I'm gonna implement those

[andrebrait]

So if the archive format isn't zip then your tool won't be able to open it to check the hashes of the files I suppose...

What's the correct procedure then? Go back to the older non-scan behavior by just checking filenames against p/clone data?

Well, either that or convert them to zip :-(

Opening other formats with Python is not as trivial as opening a zip file, and they might depend on external modules, which is why I haven't implemented them yet