Make ISEMAIL and ISURL more flexible for longer TLD

[vviers]

Context

More and more local governments around the world use custom TLD (.amsterdam, .melbourne, .corsica, .nyc just to name a few, full list here) and the ISEMAIL function doesn't consider email@cityof.{TLD} as valid if it is of length > 6.

Proposal

I know validating emails is very tricky and that there is no "right way" to do this, but I think allowing longer TLDs makes sense, especially since application for gTLD was made easier back in 2011 and there are now more than 1400 TLDs (out of which 466 are longer than 6 characters).

I used 24 as the upper limit as that's the current max, though in theory TLDs could be 63 bytes as per RFC 1034.

Check my numbers

Here's a quick python script that backs up my numbers

import requests
r = requests.get("http://data.iana.org/TLD/tlds-alpha-by-domain.txt")
tlds = r.text.split("\n")[1:]

# Longest TLD
max(tlds, key=len)
# > 'XN--VERMGENSBERATUNG-PWB'
len(max(tlds, key=len))
# > 24

# TLDs longer than 6
sum([len(tld) > 6 for tld in tlds])
# > 466

# Total number of TLDs
len(tlds)
# > 1453

[paulfitz]

Thanks a lot @vviers ! Makes sense.

There seems to be still a genuine test failure

[vviers]

I think I found the issue, I didn't realize docstrings were actual tests

[vviers]

Weird... the "should distinguish valid and invalid emails" tests pass on my machine but not here

[vviers]

I found what was wrong, I needed to update the JS isEmail function in app/common/gutil.ts as well.

@paulfitz I can't help but notice that this function is never called in the code, do you know if it is still needed ? Is this used in your enterprise codebase perhaps ?

[paulfitz]

@paulfitz I can't help but notice that this function is never called in the code, do you know if it is still needed ? Is this used in your enterprise codebase perhaps ?

Just checked, we don't use it. It was introduced in 2018, and used in an invite mechanism that has since been ripped out. So it could just be removed. Thanks for spotting this.

[paulfitz]

Looks good, thanks @vviers !