Eyegaze Access Solution: Block Scanning to Direct Access

[sofia-bg]

Design aim: A solution to improve access for an eye-gaze user where navigation on home screen would be similar to block scanning. Once user dwells within the block for any period of time, then the screen fills with only the cells from within that block. Then the cells would all be accessed via direct selection (for the most part, perhaps depending on size of initial block another round of block would be appropriate).

I created a new folder in design solutions called IDD01 Access Solutions

Three drawings of prototype ideas are here:
Example 1_ Quadrants to Direct Select
Example2- Halves to Direct
Example 3 - Thirds to Halves to Direct

Initial excel file of data from home page, groups, and school page is here.

Have not yet transitioned to building in OS-DPI. Still thinking through how that will work. Have programmed in excel which block each cell is in (for quadrants, halves and thirds) but want to avoid having to sustain dwell on a single cell for block to trigger, but rather entrance into block acting as trigger for dwell and dwell sustaining even if move across cells within the block.

[gbishop]

I have some ideas for this but I'm too tired now. Will respond later.

[gbishop]

I can think of several approaches here. I'll describe them below.

Fake it

You could set up the full grid then take a screenshot. With that image you could create a VSD that looks just like the grid but has huge blocks. The big buttons would trigger actions that would switch to tabs with the smaller sub grids.

Pros: No special skills required.
Cons: It would be pain to change the buttons around and would be boring work.

Patterns

The right way would be for me to figure out how to get Patterns to work with pointer events.

As an aside. I'm thinking Patterns is the wrong name for what I have implemented. Perhaps that panel should be named Scanning? That is all it really works for currently. I'd love to figure out how to generalize to pointer events.

I'd love to do some brainstorming with some of you. That might break my mental logjam.

Pros: It is a much needed feature.
Cons: I don't yet know how to do it.

CSS Tricks

It might be possible to fake this with some clever CSS in the Customize component. We could select groups using the
button[#Group=1] rules and an animation to implement the dwell.

Pros: It might work with the existing system.
Cons: It would be a hack.

[sofia-bg]

Ok I also had the same idea re: faking it with VSD but wondered if that was the "easy way out." Was wondering too about top level being VSD and second level the grid... but yes would love to figure out how to use patterns to get it going at some point.

[gbishop]

I think I might see how to implement a link between Patterns and Methods that could accomplish this.

I don't yet see how to generalize it to a hierarchical pattern. When you are scanning with switches you might first select a group, then a row in the group, and finally a button. How does that work with a pointer? Your proposal would not need this but we computer guys are always thinking about edge cases; that's what we do.

Consider the layout below. It has 4 top-level groups and the top 2 each contain 3 rows. Imagine we hovered over the top left quadrant to select it and it became yellow. Now we could hover over the middle row and turn it orange.

What happens if we now move out of this top left quadrant into the top right one? We are two levels down in the hierarchy of groups.

If we were scanning with switches the only things we could activate would be the buttons inside that orange rectangle.

Should we limit the user to only selecting buttons in the orange rectangle? That would, I think, be really annoying but might be a goal... Or should we back up in the hierarchy until we find a group that includes the top right box and start over?

Or maybe we should make it more like scanning and move up one level if you remain outside the current level. It'd be a little like having to scan N times through a row before you could go back to row selection.

Your input please?

[gbishop]

Of course, Sofia's idea wouldn't work this way. She'd only need 1 level. You remain within a group long enough and you jump to a new panel with larger buttons just for that set of choices.

[gbishop]

Another diagram. This time yellow indicates activation.

I start out looking at A (or anywhere inside the group). If I look long enough the group becomes activated. Now I look at B for long enough and it gets activated. Now I look at C. What happens?

One scenario is for the selection to first move back to the outer left hand group, then to the outer right hand group, then to the group containing C.

If you didn't want these multiple levels you could simply define the rows as the top level group. But if you define the hierarchy it seems like it should mean something.

[gbishop]

I now see how to integrate patterns with pointer access. I think it will be pretty neat.

I do need your input on how a hierarchical pattern should behave with pointers. I'm currently thinking in the diagram above in the 4th frame, after you looked anywhere in the C group it would become selected with no backing out of the hierarchy.

That is, I think the hierarchy applies going in but not out.

But, I'm just making this stuff up.

[sofia-bg]

I am not quite following, I have a hard time with static images when we are discussing something dynamic. In the images you made it seems to me that you are scanning blocks but that once a block is "selected" you still need to be able to dwell on a single icon because everything stays on the page. This doesn't really solve the problem of not being able to sustain the dwell on the icon when it is it's original size. In my mind when you dwell on a "block" then what happens is everything else from that page disappears and everything in that block fills the page. Now you can look at a cell to select with (i.e., point and dwell) and it is easier because the cell is bigger.

As far as hierarchy, it would depend on the number of cells in the original block and user preference... Perhaps if breaking up into chunks of 9, then you select blocks on first level but not on second level. However if doing half the screen (so two blocks each 6x5) then perhaps subsequently need to scan two blocks again (3x5 or some uneven combo) to get to a cell size that is easier to navigate?

Not sure if that makes sense or is what you meant by hierarchy. Might be an easier zoom conversation than text.

[gbishop]

Yes, let's meet on Zoom. I was not solving your problem, I think that one is easy. You'll have 4 groups (1 for each quadrant) and they will produce an event when one is selected. Then you'll switch tabs or whatever to show bigger buttons. I'm talking about the general case of hierarchy without any Zoom. I don't know why you would want it but somebody might and I like to implement the general thing. So suppose nothing disappears, only the group refines. Just like in switch scanning. At first you had only 4 choices. After dwelling on one, you now have the choices inside it and so on down to the deepest level where you are choosing individual buttons. My question is, if you're several levels deep in the hierarchy how do you get out. In switch scanning you cycle enough times and pop up level by level. With pointers we don't have cycles. So what is the equivalent? I think I'll have your quadrant/zoom example working soon.

…

On Mon, Jan 9, 2023 at 4:05 PM Sofia B-G ***@***.***> wrote: I am not quite following, I have a hard time with static images when we are discussing something dynamic. In the images you made it seems to me that you are scanning blocks but that once a block is "selected" you still need to be able to dwell on a single icon because everything stays on the page. This doesn't really solve the problem of not being able to sustain the dwell on the icon when it is it's original size. In my mind when you dwell on a "block" then what happens is everything else from that page disappears and everything in that block fills the page. Now you can look at a cell to select with (i.e., point and dwell) and it is easier because the cell is bigger. As far as hierarchy, it would depend on the number of cells in the original block and user preference... Perhaps if breaking up into chunks of 9, then you select blocks on first level but not on second level. However if doing half the screen (so two blocks each 6x5) then perhaps subsequently need to scan two blocks again (3x5 or some uneven combo) to get to a cell size that is easier to navigate? Not sure if that makes sense or is what you meant by hierarchy. Might be an easier zoom conversation than text. — Reply to this email directly, view it on GitHub <#142 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7FG7YBOJYHNNEZIZ6LWRR4QTANCNFSM6AAAAAATSJUF2Y> . You are receiving this because you commented.Message ID: ***@***.*** com>

[sofia-bg]

@gbishop I remember you got a better version of this working than my fake it idea, but I cannot seem to find it. Do you think you know where it might be?

[gbishop]

Maybe in that discussion? I can't look now

…

On Wed, Feb 28, 2024, 15:02 Sofia B-G ***@***.***> wrote: @gbishop <https://github.com/gbishop> I remember you got a better version of this working than my fake it idea, but I cannot seem to find it. Do you think you know where it might be? — Reply to this email directly, view it on GitHub <#142 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7CZLPKIDFLYZ22NQOLYV6ELRAVCNFSM6AAAAAATSJUF22VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DMMRSGY3TE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[gbishop]

#234

…

On Wed, Feb 28, 2024 at 3:07 PM Gary Bishop ***@***.***> wrote: Maybe in that discussion? I can't look now On Wed, Feb 28, 2024, 15:02 Sofia B-G ***@***.***> wrote: > @gbishop <https://github.com/gbishop> I remember you got a better > version of this working than my fake it idea, but I cannot seem to find it. > Do you think you know where it might be? > > — > Reply to this email directly, view it on GitHub > <#142 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABQK7CZLPKIDFLYZ22NQOLYV6ELRAVCNFSM6AAAAAATSJUF22VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DMMRSGY3TE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.*** > .com> >