Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLDR-16251 Add constraint on duplicate variant tags #3323

Merged
merged 2 commits into from
Oct 5, 2023
Merged

CLDR-16251 Add constraint on duplicate variant tags #3323

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6b9b154
CLDR-16251 Add constraint on duplicate variant tags
macchiati Oct 5, 2023
af5bbb2
CLDR-16251 Add note about tlang.
macchiati Oct 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/ldml/tr35.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,9 @@ A _Unicode language identifier_ has the following structure (provided in EBNF (P
<tr><td><code>alphanum</code></td><td><pre>= [0-9 A-Z a-z] ;</pre></td></tr>
</tbody></table>

> As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there is a further condition:
> The sequence of variant subtags must not have any duplicates (eg, de-1996-fonipa-1996 is not syntactically well-formed).
Comment on lines +320 to +321
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
> As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there is a further condition:
> The sequence of variant subtags must not have any duplicates (eg, de-1996-fonipa-1996 is not syntactically well-formed).
As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there is a further condition:
The sequence of variant subtags must not have any duplicates (eg, de-1996-fonipa-1996 is not syntactically well-formed).

(and for consistency with Unicode Locale Identifier, this should probably precede the grammar table)


The semantics of the various subtags is explained in _[Language Identifier Field Definitions](#Field_Definitions)_ ; there are also direct links from [`unicode_language_subtag`](#unicode_language_subtag) , etc. While theoretically the [`unicode_language_subtag`](#unicode_language_subtag) may have more than 3 letters through the IANA registration process, in practice that has not occurred. The [`unicode_language_subtag`](#unicode_language_subtag) "und" may be omitted when there is a [`unicode_script_subtag`](#unicode_script_subtag) ; for that reason [`unicode_language_subtag`](#unicode_language_subtag) values with 4 letters are not permitted. However, such [`unicode_language_id`](#unicode_language_id) values are not intended for general interchange, because they are not valid BCP 47 tags. Instead, they are intended for certain protocols such as the identification of transliterators or font ScriptLangTag values. For more information on language subtags with 4 letters, see [BCP 47 Language Tag to Unicode BCP 47 Locale Identifier](#Language_Tag_to_Locale_Identifier).

For example, "en-US" (American English), "en_GB" (British English), "es-419" (Latin American Spanish), and "uz-Cyrl" (Uzbek in Cyrillic) are all valid Unicode language identifiers.
Expand Down Expand Up @@ -347,6 +350,9 @@ As is often the case, the complete syntactic constraints are not easily captured
| `tkey` | `= alpha digit ;` |
| `tvalue` | `= (sep alphanum{3,8})+ ;` |

> As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there is a further condition:
> The sequence of variant subtags in a tlang must not have any duplicates.

Comment on lines +353 to +355
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be merged with the comment above the table, e.g.

-As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there is a further condition: There cannot be more than one extension with the same singleton (-a-, …, -t-, -u-, …). Note that the private use extension (-x-) must come after all other extensions.
+As is often the case, the complete syntactic constraints are not easily captured by ABNF, so there are further conditions:
+There cannot be more than one extension with the same singleton (-a-, …, -t-, -u-, …), and
+the sequence of variant subtags in a tlang must not have any duplicates.
+Note that the private use extension (-x-) must come after all other extensions, and can be followed by any subtags.

For historical reasons, this is called a Unicode locale identifier. However, it really functions (with few exceptions) as a language identifier, and accesses language-based data. Except where it would be unclear, this document uses the term "locale" data loosely to encompass both types of data: for more information, see _[Language and Locale IDs](#Language_and_Locale_IDs)_.

As of the release of this specification, there were no other_extensions defined. The other_extensions are present in the syntax to allow implementations to preserve that information.
Expand Down