Thinking About Access

[gbishop]

Raw signals available

The signals available from the browser include:

• keyboard
  • keydown
  • keyup (we'll need to disable autorepeat)
• mouse and touch
  • click
  • dblclick
  • pointerdown
  • pointermove
  • pointerout
  • pointerover
  • pointerup
• timers
• acceleration and orientation
• location
• sound
• video

Conditioning

We need to condition the raw inputs to prevent accidental triggers.

Conditioners could include:

• Minimum on time: time the signal must be on before it is considered active.
• Minimum off time: time the signal must be off before it is considered
  inactive.
• Minimum idle time: time between successive activations.
• Hover padding: area outside of a button that activates hover state. Something
  like hysteresis or stickiness for eye-gaze users.

Scanned access

The most interesting part of scanned access is the access pattern.

I'd like for researchers to be able to define their own new and general patterns
but I don't want those who only want something standard to be confronted with
too much complexity.

We could use scan groups that define the order of visitation. A scan group
might be defined as a nested list of buttons or other scan groups. Each group
has a single parent and one or more children. You start at the top of the
hierarchy choosing among elements at the top level. If you select a nested
group, you then scan it.

A designer should be free to explicitly list every item if they wish but
shouldn't be required to. Perhaps we could have common patterns represented as
functions with a few parameters.

On complicated user interfaces with tab controls, radio buttons and such, I
think we should adapt the strategies recommended by the Web Accessibility Initiative. They assume multiple user
inputs (tab, enter, arrow keys) but I think we can adapt their guidance. They
suggest scaning components first in an order specified by the designer, then
elements in the component. In the Contact simulation you might visit the tab
control as a group, followed by the Quick grid tone buttons as a group, then
the Quick grid.

I'm thinking when a group includes only one component, that component is
immediately selected. Another example would be a grid that contains only 1 row
or a row that contains only 1 active column. Maybe you start with the individual
items instead of selecting the single row first. If a group has only one member,
an optimization would be to go directly to that member.

No:
  Cycle through current scan group
    up to (number) times or
  then
    [offer to] go back to parent

Yes on a scan group:
  make it current

Yes on a button:
  activate it
  then

You might specify the scanning of a grid like this with increasing indentation
indicating the user choosing YES.

grid
  scan through rows 1 time then ask if they want to continue
    scan columns up to 3 times then ask if they want to continue
      issue "press" event and start over with rows

You change the order of components by naming them.

Quick
  Quick grid
  Quick tone
Chat Menu
Chat
Task Menu
Task

If you don't specify the children, you get the default order so you only have to
name the parts you want to change.

Perhaps you can specify a default for all components of a given type. Or maybe
you can name patterns and then apply them to components.

1-switch parameters

• input: which conditioned input is used to control
• step interval: time between automatic steps
• restart scan: time interval or restart on button hit

2-switch parameters

• inputs: which conditioned inputs are used to control

more switches

You should be able to bind events to conditioned inputs to do things like

• move up in the hierarchy
• go backward
• jump to somewhere else

Highlighting groups

Designers should be able to specify how groups are to be highlighted to make it
clear what is currently being chosen. This could be done with explicit CSS with
a few examples already defined. Highlights might include:

• border color
• background color
• drop shadow
• size

Eye tracker access

I'm assuming that the eye tracker looks to us like a mouse with no buttons. That
is, you can move the pointer but you can't click. If they are using an Eye-Gaze
device they likely should use its ability to click on dwell.

I think we can easily support adjustable gaps between buttons.

Buttons might grow to fill the gap when the pointer is hovered over them. This
could provide some hysteresis for selections making it easy to avoid rapidly
cycling between choices.

It should be possible to support effects like filling the button from the
outside-in to draw the eye to the middle during the hover interval.

Eye tracker parameters

• dwell interval for click
• dwell effects

Touch access

We should be able to highlight (or announce?) choices when the user touches a
button but only activate it when they release without touching another.

We should also be able to emulate iOS VoiceOver; you find the choice you want,
release, and then tap again to activate.

[cdsjeff]

Very nicely laid out! One suggestion is to consider alternative click methods like a switch click for head tracking. Find this contracting experience can we give some able to use a switch independently head tracker

[gbishop]

Clicking (or hitting a key) after pointing with the mouse should be easy to implement.

[cdsjeff]

Also is this the right What is the talk about two-step access in which the user clicks an area of the keyboard which the magnifies around user to select From a key in the magnified area area?

[gbishop]

Zooming is pretty easy as is scrolling. It will be interesting to think about how to enable designers to program such interactions in a declarative way.

[gbishop]

I cleaned it up some. It appeared to be two copies of my original smashed together. And their markdown is strange. Those paragraphs should be filled but they are simply breaking at every linebreak in the input.

It seems like every github tool uses a different version of markdown.

[gtrogdon]

One thing that jumps out to me is how our use case differs from the use case of the WAI and its conventions. Particularly, in the latter case, discoverability is everything, and you rightly point out that the default behavior should correspond with those conventions. With that in mind, once a user interface is designed, it becomes a much more predictable piece of content than websites generally are, and, if possible, should be afforded some means by which one can create shortcuts to navigate around faster. It looks like you mention this, but it is a good representation of a larger pattern of thinking about the design of these interfaces that not only conforms to the standards of the environment it inhabits (the browser), but it also extends the logic given that there is less variance in our designs compared to the entire gamut of websites that can exist.

Will write more soon.

[gbishop]

Yes! Well put. I think we should, at least, support actions that jump around in the scan groups. I think that would allow designers to implement shortcuts (good term) for their users. The contrast with the full web is apt also. Good thinking.

…

On Tue, Feb 22, 2022 at 8:32 AM gtrogdon ***@***.***> wrote: One thing that jumps out to me is how our use case differs from the use case of the WAI and its conventions. Particularly, in the latter case, discoverability is everything, and you rightly point out that the default behavior should correspond with those conventions. With that in mind, once a user interface is designed, it becomes a much more predictable piece of content than websites generally are, and, if possible, should be afforded some means by which one can create shortcuts to navigate around faster. It looks like you mention this, but it is a good representation of a larger pattern of thinking about the design of these interfaces that not only conforms to the standards of the environment it inhabits (the browser), but it also extends the logic given that there is less variance in our designs compared to the entire gamut of websites that can exist. Will write more soon. — Reply to this email directly, view it on GitHub <#74 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7E63DAWPQYZ4H3LZ7TU4OF6DANCNFSM5O7AFFWQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[gbishop]

Access scenario: Touch with announcements

• The user touches the screen.
• When they enter a button and remain there for at least T_highlight seconds we announce the button name and/or begin the visual highlight sequence.
• We ignore all additional touches on multi-touch devices until this touch ends.
• If they leave the button (lift or move out) for less than T_off seconds we ignore it (like it didn't happen).
• If they leave for longer we cancel the action.
• If they remain on the button for T_activate seconds we issue a press event.
• If they remain after the press event for T_repeat seconds we issue another press event.

[gbishop]

Access scenario: Eye tracker controlling the pointer location

This should be like VoiceOver on iOS.

• When the pointer enters a button and remains there for at least T_highlight seconds we announce the button name and/or begin the visual highlight sequence.
• If the pointer leaves the button for less than T_off seconds we ignore it (that is we treat it like the leave did not happen).
• If the pointer leaves for longer we cancel the action.
• If the pointer remains on the button for T_activate seconds we issue a press event.
• If they remain after the press event for T_repeat seconds we issue another press event.

[lageist]

Additional eye tracking parameters/selection options:

1. Detecting a user's blink to indicate selection.
2. Gaze plus a switch. Less common, but highly effective for some.
3. Zoom options.

[gbishop]

1. The eye tracker itself would have to tell us about blink. I'm not proposing that we do the eye tracking. But if the device gives us an event it'll be easy to use.

[lageist]

Scanned Access Follow-up

Additional parameters to consider:

1. Select on switch release. Pressing switch initiates the scan, release selects.
2. Screen as switch:

• Automatic scan. User presses/releases anywhere on screen to start, presses/releases anywhere on screen to select when desired item is presented.
• Inverse scan. Press/hold touch on screen until desired item is presented, release to select.

Notes about scan groups and patterns:

• Ability to combine patterns. Initial scan of groups, once a group is selected, scan in linear or row/column pattern.
• Determine when various components are included in the scan/group (e.g., ability to group the message window and speak buttons based on function versus location on layout).
• Set where a scan pattern restarts (e.g., from the last selection or restarts at beginning).
• Option to set a delay after selection to allow a user time to prepare for next selection.
• Option to set/enable auditory cues that may differ from the button label (e.g., abbreviation, initial sounds).

[gbishop]

Thanks. I'll include both of those in my thinking.

I'm thinking rows and columns are groups. So if you're doing row/column scanning you've got a group listing the rows and each row is a group listing the buttons in the row. It is always 1D, building up to higher dimensions by nesting.

Yes, we need some rules for what is included. Rows and Columns are easy (#row = 1 say). But could build groups based on arbitrary fields in the content (#function = 'verb').

Yes, we need a way to say where we restart in a group. Where we left off, at the beginning of this group, in the parent of this group, etc.

Yes, we need adjustable delays.

I'm thinking the "highlight" handled various cues. You can have it play a sound, change some aspect of the display, etc.

[cdsjeff]

Since the eye tracker is a separate piece of hardware and software, it’s going to have to send a separate message for blink recognition

…

On Wed, Mar 2, 2022 at 9:26 AM Gary Bishop ***@***.***> wrote: 1. The eye tracker itself would have to tell us about blink. I'm not proposing that we do the eye tracking. But if the device gives us an event it'll be easy to use. — Reply to this email directly, view it on GitHub <#74 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHYBQ5OVLWGXXRVB64SKNTDU553AVANCNFSM5O7AFFWQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.*** com>