CLDR-17371 stream filter to replace xmlns with DOCTYPE

[srl295]

• optimized
• assumes current version DTD
• supports spec change in CLDR-17177 kbd: update per feedback #3495

CLDR-17371

• This PR completes the ticket.

ALLOW_MANY_COMMITS=true

[srl295]

kbd-check fails because of the xmlns, to be addressed on the keyman side in keymanapp/keyman#10803

[srl295]

@macchiati I don't think this PR needs to block the spec, this PR is to show that the spec is supportable in Java. (once tests pass!)

So, I'm OK to leave this PR open and make improvements if needed. API changes, optimization etc

[macchiati]

@macchiati I don't think this PR needs to block the spec, this PR is to show that the spec is supportable in Java. (once tests pass!)

So, I'm OK to leave this PR open and make improvements if needed. API changes, optimization etc

Makes sense

[srl295]

I measured 11s±1s for 1,000 iterations. - about 11ms per iteration to load and validate common/main/mt.xml. No significant difference between using and not using the wrap() function.

There is perhaps even a slight improvement (due to read-ahead or inefficiencies in the XML parser?) when using the wrap. The quick-check for <!DOCTYPE is now at the byte or char array level and runs pretty quickly.

[macchiati]

I'm a bit confused. Are you saying that it takes 11s / 10000 iterations with the wrap and approximately the same without?

…

On Thu, Feb 22, 2024 at 9:50 AM Steven R. Loomis ***@***.***> wrote: Measured 11s±1s for 10,000 iterations. Perhaps slight improvement (due to read-ahead or inefficiencies in the XML parser?) when using the wrap. The quick-check is now at the byte or char array level. — Reply to this email directly, view it on GitHub <#3511 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMDPHMLXDJLG7374ZFDYU6AOJAVCNFSM6AAAAABDTXCLSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZHE3DCMRSGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[srl295]

I'm a bit confused. Are you saying that it takes 11s / 1000 iterations with the wrap and approximately the same without?

11s/1000, sorry. I'm re-running the tests.

[jira-pull-request-webhook]

Notice: the branch changed across the force-push!

• tools/cldr-code/src/test/java/org/unicode/cldr/util/TestDoctypeXmlStreamWrapper.java is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

[srl295]

I'm getting setup to run on a less busy system

[srl295]

Raw results.
10,000 iterations (faster, different, nearly idle system)

  <testcase name="TestBytePerf(boolean)[1] wrapped=false" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="74.146"/>
  <testcase name="TestBytePerf(boolean)[2] wrapped=true" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="74.809"/>
  <testcase name="TestBytePerf(boolean)[3] wrapped=false" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="76.95"/>
  <testcase name="TestBytePerf(boolean)[4] wrapped=true" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="75.434"/>
  <testcase name="TestBytePerf(boolean)[5] wrapped=false" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="77.519"/>
  <testcase name="TestBytePerf(boolean)[6] wrapped=true" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="74.999"/>
  <testcase name="TestBytePerf(boolean)[7] wrapped=false" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="77.698"/>
  <testcase name="TestBytePerf(boolean)[8] wrapped=true" classname="org.unicode.cldr.util.TestDoctypeXmlStreamWrapper" time="77.906"/>

[srl295]

Thanks. @macchiati ok to move ticket to accepted ? (Had sent mail to TC)

[macchiati]

Ok, you can. Assuming pk =ok

…

On Fri, Feb 23, 2024, 18:02 Steven R. Loomis ***@***.***> wrote: Thanks. @macchiati <https://github.com/macchiati> pk to move ticket to accepted ? (Had sent mail to TC) — Reply to this email directly, view it on GitHub <#3511 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMCM3CLDE46WKP3BU7LYVFC47AVCNFSM6AAAAABDTXCLSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSGIYTIMZVGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[srl295]

Ok, you can. Assuming pk =ok

Failing build step is not CLDR tests, it's the Keyman compiler now out of date with these PRs. Once they merge I can fix it.

[jira-pull-request-webhook]

Hooray! The files in the branch are the same across the force-push. 😃

~ Your Friendly Jira-GitHub PR Checker Bot

[macchiati]

Looks like the tests fail

[mcdurdin]

Looks like the tests fail

as @srl295 wrote previously:

Failing build step is not CLDR tests, it's the Keyman compiler now out of date with these PRs. Once they merge I can fix it.

He is working on a PR to the Keyman compiler to allow the <keyboard3 xmlns attribute in the validation phase in the compiler, once that merges then that will resolve the test here.

[srl295]

Looks like the tests fail

CLDR's unit tests pass.

[srl295]

Looks like the tests fail

CLDR's unit tests pass.

I've disabled the keyboard check workflow for now. I'll re enable it once these merge and I can update the compiler to the changed spec.

[srl295]

ping, can i get a review on this? All unit tests pass.