Changes to support VoiceOver

[dpvc]

Sorry for the long delay in getting this to you. I continued to work on it after our call and our the weekend. I have eliminated the worst of the hacks that I had added, and am much happier with this version than what I had on Thursday. There is no longer a need for the the speech string as text content now that we aren't going to support paragraph reading in VoiceOver (which ignored aria-label among other things). I've also worked around the need I had to delay during an up-arrow event. So that is all much cleaner.

There is still the need for a separate child node for speech, however, in order to handle the stepped reading modes. If the top-level container has a role and aria-label, the container gets the focus and then the explorer steps into the expression and the automatic stepping is terminated. So I've put the role and label on a child to avoid that. But at least it uses aria-label and aria-braillelabel instead of text content.

The other important change is that only the active node in the expression tree is given a role and aria-label. That is what is avoiding the up-arrow difficulties, and makes the results work more reliably across browsers. So the role and labels are added and removed as the walking takes place.

This allows the use of roles other than treeitem, which was invalid the way we were using it. Also, the presentation role that I was using technically was not correct, either, as aria-label isn't allowed on it. Instead, I'm now using role math, which seems to do the trick (though more on that in a bit).

Because the roles and labels are added and removed on the fly, we don't get the problem I had before about the parent node's label being spoken along with the child node when you move down a level. And I already mentioned that moving up a level now works without the extra re-focusing hack.

Along with the roles and labels being added dynamically, I also set aria-hidden on the children of the active node. This isn't really necessary in CHTML exploration, but for SVG when in-line breaking is allowed, it is needed, and I didn't want to special-case the SVG output. Note, however, that walking an in-line expression with line breaking in SVG is not working very well, as that involves multiple SVG child elements of the mjx-container. I haven't tried to fix any of that yet.

There are some differences between what is spoken among the different browsers. In particular, Safari will add the role at the end of the speech string (like 'x, math'). Worse, when walking through a sentence containing math, one will get x = a plus b, empty math because the extra node that has the labels doesn't have any content. I would like to get rid of the empty, but haven't spent time trying to figure out what might be needed for that. When walking into the expression, you don't get empty, even though everything inside it is hidden, so it can probably be done.

Alternatively, one could use role button or img, which don't get the role appended. This makes for the most consistent experience across browsers, but is a bit of a misuse of those roles (of course, so was treeitem). This means that x, math is just x in all browsers (it was x in Firefox, x, math or x, empty math in Safari, and x, math or x, group in Chrome). I personally like this experience better, but the drawback is that the added element with the labels will also be a button, and if you want to navigate the page by buttons, that will include all the math. Image might be the better choices s I don't think you navigate by images, but I might be wrong. The math role is safest, I would guess, but has the extra , math in some cases.

I also added aria-roledescription as MathJax expression rather than math, so you get x, MathJax expression. That would allow the reader to know that MathJax navigation keys are available (there should be some extra explanation of this that we can talk about). That may be too verbose, however.

In VoiceOver, if you pause on an expression, you will get a message about "You are on a MathJax expression in web content. To exit this web area, press ..." and so on. With the button role, you get "To click this button, press..." before the "To exit" message. Similarly, for the img role, you get "To interact with this image, press...", so those are a bit unpleasant. But I suspect that you are right that most serious users will have turned those messages off (I haven't look into how hard that is).

So the choice of role is not ideal not matter what. You either get the extra , MathJax expression or get extra messages that indicate the wrong type, and get things listed in the button or image lists.

The text used for the role and role description are obtained from a function so that they can be localized (or changed easily for checking different settings).

Because the role is no longer present on most nodes, I had to change the navigation query. I added data-speech-node attributes to mark what used to have roles, and use that for navigation instead.

There is a section in semantic-enrich.ts that alters the results from SRE to remove the roles and add the needed data-speech-node attributes. It also removes aria-level, aria-posinset, etc., which don't seem to be needed. If they really are needed, you can comment out those lines, but note that the aria-owns values are incorrect, as they are supposed to be IDs if the elements owned, not indices, and we don't have IDs on any of the child nodes, so these are pointing to non-existent elements.

The actual text in the mjx-c nodes are marked as aria-hidden, so they are never read directly, and the data-keep-hidden attribute tells the code that adds and removes aria-hidden not to change that.

I hope the rest of the code is clear otherwise.