Fix the IndexUnicodeProperties Joining_Type

[eggrobin]

Treat column 2 of ArabicShaping.txt as an unofficial contributory property and Joining_Type as a derived property in IndexUnicodePropertySource. When retiring ToolUnicodePropertySource, we should switch to using that derivation.

See the test failure prior to the fix.

[macchiati]

clever approach. A few items:
The UcdProperty and UcdProperty values are generated by GenerateEnums, so that needs to be modified to produce the right values. Otherwise the next time it is run, the changes will be overridden.

Short term, I think making the derived property "occupy" field 1 is a clever hack. Longer term I think we need to add a better mechanism for derived IndexUnicodeProperties.

[eggrobin]

The UcdProperty and UcdProperty values are generated by GenerateEnums, so that needs to be modified to produce the right values. Otherwise the next time it is run, the changes will be overridden.

I ran GenerateEnums already (a CI check tests that it does not change anything).

I think we need to add a better mechanism for derived IndexUnicodeProperties.

We have plenty of those in IndexUnicodeProperties.txt already, see e.g.

unicodetools/unicodetools/src/main/resources/org/unicode/props/IndexUnicodeProperties.txt

Lines 124 to 143 in 77e3ce6

	DerivedCoreProperties ; Lowercase
	DerivedCoreProperties ; Uppercase
	DerivedCoreProperties ; Cased
	DerivedCoreProperties ; Case_Ignorable
	DerivedCoreProperties ; Changes_When_Lowercased
	DerivedCoreProperties ; Changes_When_Uppercased
	DerivedCoreProperties ; Changes_When_Titlecased
	DerivedCoreProperties ; Changes_When_Casefolded
	DerivedCoreProperties ; Changes_When_Casemapped
	DerivedCoreProperties ; Alphabetic
	DerivedCoreProperties ; Default_Ignorable_Code_Point
	DerivedCoreProperties ; Grapheme_Base
	DerivedCoreProperties ; Grapheme_Extend
	DerivedCoreProperties ; Grapheme_Link
	DerivedCoreProperties ; Math
	DerivedCoreProperties ; ID_Start
	DerivedCoreProperties ; ID_Continue
	DerivedCoreProperties ; XID_Start
	DerivedCoreProperties ; XID_Continue
	DerivedCoreProperties ; Indic_Conjunct_Break

[eggrobin]

making the derived property "occupy" field 1 is a clever hack

You may have misread that line of IndexUnicodeProperties.txt, it is

DerivedJoiningType; Joining_Type; 1

that is, we are getting it from the derived file (like the other derived properties), where it actually does reside in field 1.

[macchiati]

Ah, got it. Ok, will approve

…

On Thu, Jan 18, 2024 at 8:41 AM Robin Leroy ***@***.***> wrote: making the derived property "occupy" field 1 is a clever hack You may have misread that line of IndexUnicodeProperties.txt, it is *DerivedJoiningType*; Joining_Type; 1 that is, we are getting it from the derived file (like the other derived properties), where it actually does reside in field 1. — Reply to this email directly, view it on GitHub <#657 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMEQPIW2DAJH4O7OPPDYPFGB5AVCNFSM6AAAAABB77BMWWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJYHAZTMMBVG4> . You are receiving this because your review was requested.Message ID: ***@***.***>