Deduplicate emoji

[treyhunner]

Some emoji have an old-style plain variant and a newer fancy variant. Both are shown in the terminal and (on my machine at least) look the same.

Example:

Those two will copy different character sequences though!

🗃️

versus

🗃

We just want the fancy-looking one.

Looking at the two in the REPL, it looks like a variant character is sent after one to make it fancy:

>>> import unicodedata
>>> len(x)
2
>>> len(y)
1
>>> x[-1]
'️'
>>> unicodedata.name(x[-1])
'VARIATION SELECTOR-16'

Unicode.party shows only the fancy one: https://unicode.party/?query=file

utf8.xyz shows only the unfancy one (which makes sense because it only searches characters, not multi-character glyphs: https://utf8.xyz/file

I'm not sure if Unicode Party removes the unfancy ones or if we're getting both because we currently merge the emojilib data (which powers Unicode Party) with the official Unicode Consortium's character data (which powers utf8.xyz).