[Bug] FDedup failing with latest release mmh3==5.1.0

[touma-I]

Search before asking

• I searched the issues and found no similar issues.

Component

Transforms/universal/fdedup

What happened + What you expected to happen

When building the transform with current requirements.txt, we are pulling in the latest release of mmh3 that was published to pypi on Jan 25 and causing the UT of fdedup to fail.

Reproduction script

To produce the error, run test-src with mmh3>=4.1.0

To prevent the error, modify transforms/universal/fdedup/requirements.txt to add:
mmh3>=4.1.0, <=5.0.1

Anything else

Also investigate if relaxing polars can be achieved. It is currently set to polars==1.9.0 and the preferred approach would be to use polars>=1.9.0

OS

Ubuntu

Python

3.11.x

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[touma-I]

Stop gap in current dev branch: mmh3>=4.1.0, <=5.0.1

Long term, investigate how we can make it work for latest release of mmh3

[cmadam]

I have fixed the code so that it works with any version of mmh3

[cmadam]

The code worked with polars releases up to, and including, 1.9.0. Then, it stopped working with releases 1.10.0, 1.11.0, and 1.12.0. The problem went away and the tests passed successfully in all the subsequent releases of polars (1.13.0, 1.13.1, 1.14.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, and the latest, 1.21.0). So I changed the requirements.txt file with:

polars>=1.13.0

to eliminate all the problematic versions (1.10.0 - 1.12.0)