Fix portal & server generate password test sometimes fail

[pkong-ds]

Failing examples

• https://github.com/authgear/authgear-server/actions/runs/10572893051/job/29291367108
• https://github.com/authgear/authgear-server/actions/runs/10465349585/job/28980452492?pr=4619

The math

Why it happened

Think it happened at least 3 times since it got merged. Was curious why so I did some math.

all combinations of `[1-9]{8}` = 9^8
all combinations of `[0-9]{8}` = 10^8


P(1st trial fail) = 1
    because `if (!ranOnce) return 0`
P(2nd trial fail) = [1 - (9^8 / 10^8)] = 0.5695
P(3nd trial fail) = [1 - (9^8 / 10^8)]^2 = 0.5695^2 = 0.3243
...
P(10th trial fail) = [1 - (9^8 / 10^8)]^9 = 0.5695^9 = 0.00630

0.00630 = test will fail 6 times out of 1000 CI runs, which is not that unlikely, so it failing 3 times is kind of expected.

How to prevent

Say we aim for only fail once in 100,000 CI runs

[1 - (9^8 / 10^8)]^n <= 0.000001
n >= 24.53

Which gives the solution n = 25. So I increased to 25 here

How do I know it works?

Ran below locally, let's see if it fails 🕵️

for i in {1..100000}; do npx jest passwordGenerator.test.ts --silent || (echo "Failed after $i attempts" && break); echo $i; done

EDIT2: ran to 816710774 w/o failing, will stop for now

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.534 s, estimated 2 s
10774
 PASS  src/util/passwordGenerator.test.ts

[louischan-oursky]

Thanks for the calculation. But we do not want to modify the original behavior of 10 times. Instead we should make it able to use N instead of 10 in the test only. Can you also apply the same logic to the server generator? They are twin.

[pkong-ds]

okok, lemme update after linter is ready 🙏

[louischan-oursky]

Do not make it optional. Make it required. But the public API must remain unchanged.

[pkong-ds]

Updated, thanks! CI pass locally