Scanning on VSD/Grid Combo

[sagostine]

Hi Y'all,

As we talked about during our last meeting here is my idea about scanning in my Interoception VSD/Grid Combo that I would love steps on how to make this a reality!

Let me know if anything doesn't make sense and absolutely open to ideas on how to make it even better than what I am picturing!

Here is the OSDPI file in case you want to mess with it :)

FINAL Introception_Grid_7_28_23 - Copy.zip

@lageist @sofia-bg @aetkie @hsiaotingsu

Thank you!!!
Susie

[gbishop]

Things to note about Patterns.

1. Empty Groups are ignored
2. You can define as many Groups and you need
3. Filter takes an expression which can include a State variable
4. You can Order By a Field to visit buttons in specified order
5. The same buttons can be in multiple groups

Suggestions.

To order the buttons in the VSD, add a column to your spreadsheet so you can use Order By on it. Call it #order.

To jump from one area to another, I suggest using a State variable along with a Filter. Call the State variable $S (you should choose a better name). When scanning the VSD $S could be 'v'. The VSD group might start with a Filter $S = 'v', then another Filter #ComponentName = 'vsd' and finally an Order By #order. This group will only visit buttons in the VSD if the $S variable is 'v' otherwise it will be empty.

The grid might be represented by multiple Groups that are mutually exclusive. The Group for 1-by-1 scan might start with Filter $S = '1', Filter #ComponentName = 'hp'. The Group for row access might start with Filter $S = '2', Filter #ComponentName = 'hp', Group By #row

You could put that last row of buttons in a separate group or not.

You'd manipulate the $S variable in your Actions ideally by including the new value for $S in your spreadsheet.

This is not a complete solution but I hope it gives you the hints you need.

[gbishop]

@sagostine did you get it working the way you wanted? I'll be pleased to help if you need it.

[sagostine]

@gbishop I am definitely closer and have gotten it to scan but not consistently. I am not sure what is wrong. It will sometimes let me scan the VSD but I have to go through the whole VSD before it jumps over to the grid. I also haven't ever gotten it to row by row scan, just 1 by 1 (which is what I want for the VSD and first group of the grid but then I want row by row...). Maybe something isn't matching up with my spreadsheet?? Interested to hear your thoughts/suggestions!

FINAL_Introception_Grid_9.1.23.zip

[sofia-bg]

I think this is working pretty close to what you want. Did quite a bit but tried to keep your spreadsheet the same. Added a second tab with the grid and a second picture of the VSD (see I added a second sheet) but with NO coordinates. This way it scans the VSD first and then switches tab and scans only in the grid...

I kind of liked just doing it straight by components. To start in your original I just added a column to the VSD to group by left, middle, right (1, 2, 3) and with scanning by components it was kind of nice to have the option to go through the body in case you made a mistake, instead of having to scan all the way back home.

OH -- I got rid of speak and made the display speak (just my preference) but realizing I did not add it to the scanning pattern -- maybe you can try to add it to patterns in the order you want it to be

FINAL_Introception_Grid_9_1_23_SBG.zip

[gbishop]

Super clever solution! I had it working with patterns but lost it chasing the export bug. I will recreate when I recover though I like your solution better.

…

On Fri, Sep 1, 2023, 18:14 Sofia B-G ***@***.***> wrote: I think this is working pretty close to what you want. Did quite a bit but tried to keep your spreadsheet the same. Added a second tab with the grid and a second picture of the VSD (see I added a second sheet) but with NO coordinates. This way it scans the VSD first and then switches tab and scans only in the grid... I kind of liked just doing it straight by components. To start in your original I just added a column to the VSD to group by left, middle, right (1, 2, 3) and with scanning by components it was kind of nice to have the option to go through the body in case you made a mistake, instead of having to scan all the way back home. FINAL_Introception_Grid_9_1_23_SBG.zip <https://github.com/UNC-Project-Open-AAC/OS-DPI/files/12501749/FINAL_Introception_Grid_9_1_23_SBG.zip> — Reply to this email directly, view it on GitHub <#215 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7FX5PDMHUG3WGM6NCLXYJM4JANCNFSM6AAAAAA3VV356M> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[gbishop]

It occurs to me you should be able to do something similar with a filter on the vsd, simply remove all the vsd rows when you want to focus on the grid

…

On Fri, Sep 1, 2023, 18:25 Gary Bishop ***@***.***> wrote: Super clever solution! I had it working with patterns but lost it chasing the export bug. I will recreate when I recover though I like your solution better. On Fri, Sep 1, 2023, 18:14 Sofia B-G ***@***.***> wrote: > I think this is working pretty close to what you want. Did quite a bit > but tried to keep your spreadsheet the same. Added a second tab with the > grid and a second picture of the VSD (see I added a second sheet) but with > NO coordinates. This way it scans the VSD first and then switches tab and > scans only in the grid... > > I kind of liked just doing it straight by components. To start in your > original I just added a column to the VSD to group by left, middle, right > (1, 2, 3) and with scanning by components it was kind of nice to have the > option to go through the body in case you made a mistake, instead of having > to scan all the way back home. > > FINAL_Introception_Grid_9_1_23_SBG.zip > <https://github.com/UNC-Project-Open-AAC/OS-DPI/files/12501749/FINAL_Introception_Grid_9_1_23_SBG.zip> > > — > Reply to this email directly, view it on GitHub > <#215 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABQK7FX5PDMHUG3WGM6NCLXYJM4JANCNFSM6AAAAAA3VV356M> > . > You are receiving this because you were mentioned.Message ID: > ***@***.*** > .com> >

[sofia-bg]

Do you mean in the second selector tell it to also ignore something? Will need to see to understand I think. My solution took too long to make - I knew there had to be an easier way in patterns but it just wasn't doing it for me. But like I said, I thought using components was a nice solution so starting back at beginning wasn't a hassle AND if you made mistake in the first pass of the body you could try again... curious what @sagostine will decide she wants to do

[gbishop]

In the vsd there is a filter like #sheetname = vsd or something similar. You could add another filter something like $mode = vsd. When that condition is true the vsd will have hotspots, when it is false there won't be any.

…

On Fri, Sep 1, 2023, 18:31 Sofia B-G ***@***.***> wrote: Do you mean in the second selector tell it to also ignore something? Will need to see to understand I think. My solution took too long to make - I knew there had to be an easier way in patterns but it just wasn't doing it for me. But like I said, I thought using components was a nice solution so starting back at beginning wasn't a hassle AND if you made mistake in the first pass of the body you could try again... curious what @sagostine <https://github.com/sagostine> will decide she wants to do — Reply to this email directly, view it on GitHub <#215 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7FIRFUZ4TZ2BA6R4XLXYJO2PANCNFSM6AAAAAA3VV356M> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[gbishop]

These various approaches could make a good example

…

On Fri, Sep 1, 2023, 18:34 Gary Bishop ***@***.***> wrote: In the vsd there is a filter like #sheetname = vsd or something similar. You could add another filter something like $mode = vsd. When that condition is true the vsd will have hotspots, when it is false there won't be any. On Fri, Sep 1, 2023, 18:31 Sofia B-G ***@***.***> wrote: > Do you mean in the second selector tell it to also ignore something? Will > need to see to understand I think. My solution took too long to make - I > knew there had to be an easier way in patterns but it just wasn't doing it > for me. But like I said, I thought using components was a nice solution so > starting back at beginning wasn't a hassle AND if you made mistake in the > first pass of the body you could try again... curious what @sagostine > <https://github.com/sagostine> will decide she wants to do > > — > Reply to this email directly, view it on GitHub > <#215 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABQK7FIRFUZ4TZ2BA6R4XLXYJO2PANCNFSM6AAAAAA3VV356M> > . > You are receiving this because you were mentioned.Message ID: > ***@***.*** > .com> >

[sofia-bg]

I agree. We have been talking about the best way to share exemplars so that people can clearly see the approach, one was screen recordings if I remember correctly. I thought about it, but I knew it would take a awhile to get it working that it could start to be confusing... screenshots work but I want to make sure people know how the spreadsheet gets augmented to make actions etc. work... for now starting by adding more tags so at very least people can search for the types of things they are working on

[gbishop]

Here is a version using patterns. I modified your spreadsheet a little to adjust values in the scan column and I modified the pattern. I did not fix them all but I hope you'll see how to correct the others.

FINAL_Introception_Grid_9_1_23_patterns.zip

[sofia-bg]

@sagostine

[gbishop]

I hacked the aesthetics of it a bit to make the highlights on the VSD circular. The magic happens in the Customize block.

FINAL_Introception_Grid_9_1_23_patterns_aesthetics.zip

[sagostine]

Thanks y'all! This was so helpful. I do have some questions about why you did what you did.

1. In patterns, you both put three different selectors in the same pattern instead of adding a new pattern for the different scanning groups. Why?
2. You changed the scan to 1 instead of v, but the first selector is still set as v. Yet, it scans in the correct order. Why?
3. I changed my patterns to match yours and now it has stopped scanning all together. No idea why and I cannot for the life of me get it to work again...no error message is coming up so I have no idea why it isn't working. Something in my spreadsheet maybe??
4. I love the customize block, the circle's look so much better than the red squares. Sofia recommended googling the CSS language to get it to do what I want (font bigger, change font, change shape, etc.). Gary do you have any recommendations for best places to look for this or is it just something you've learned over time?
5. Andrea and I were also discussing the differences between group by and order by. From what we gathered, group by is for bigger groups such as row by row scanning whereas order by is for smaller groups such as individual scanning. Is that correct?
6. And is there a way to set it up so you don't have to direct select the box in order for it to start scanning?

Here is my most updated version (where scanning doesn't work for me...)
FINAL_Introception_Grid_9_6_23.zip

Thank you!! Looking forward to your thoughts!
Susie

[sofia-bg]

Just responding to CSS now, will read other questions later.

Check this out for ways that I changed color per button and font size/bold... I added a column to the spreadsheet and went through and tagged each word with the color it was supposed to be (imitating a commercially available layout).

halves_v3_colors_7_3_23_23-2.zip

And in this one I changed button colors, font sizes and boldness and used the code Gary gave me about hiding the arrow buttons for prediction
.page-control { display: none; height: 10px;}

kb_thirds_v1.zip

[sofia-bg]

re: scanning not working, export the spreadsheets and compare to yours. I know I made changes to make different things work and if you are starting with your previous version that we hacked from it is likely that our scanning patterns rely on the changes made to the spreadsheet.

[gbishop]

1. In patterns, you both put three different selectors in the same pattern instead of adding a new pattern for the different scanning groups. Why?
   • Only 1 Pattern is active at a time thus you have to combine the various scans into one Pattern.
2. You changed the scan to 1 instead of v, but the first selector is still set as v. Yet, it scans in the correct order. Why?
   • When $scan is 1 the first selector is empty because the Filter is not satisfied instead the second selector becomes active.
3. I changed my patterns to match yours and now it has stopped scanning all together. No idea why and I cannot for the life of me get it to work again...no error message is coming up so I have no idea why it isn't working. Something in my spreadsheet maybe??
   • You have misspelled #ComponentName in your first selector is your first problem. Second, you need to initialize the value of $scan in your init action. You also need to change your other actions to update the value of $scan; look at the Actions in the example I sent. I added columns to the content spreadsheet as well.
4. I love the customize block, the circle's look so much better than the red squares. Sofia recommended googling the CSS language to get it to do what I want (font bigger, change font, change shape, etc.). Gary do you have any recommendations for best places to look for this or is it just something you've learned over time?
   • CSS tutorials abound. W3Schools has one. I usually refer to MDN. There are tons of videos on YouTube. You also need to adjust the positions of the VSD hotspots to make the lineup. A better choice would be to remove the circles from your image, then alignment wouldn't be such an issue or you could hide the outlines of the VSD buttons.
5. Andrea and I were also discussing the differences between group by and order by. From what we gathered, group by is for bigger groups such as row by row scanning whereas order by is for smaller groups such as individual scanning. Is that correct?
   • No. Order by is about sorting. You are sorting the hot spots in your VSD with the Order By #order clause in the first selector. Group by is for Grouping some buttons into a Group that is highlighted together.
6. And is there a way to set it up so you don't have to direct select the box in order for it to start scanning?
   • I don't know what you are asking. You may be asking about focus. When you click with the mouse it is obvious that the event should go to the button you clicked. When you type is isn't so obvious where to send the event. Are you typing into one of the fields in the editor or are you sending a key event your user interface? The easiest and surest way of making sure keyboard events go to your user interface is to type Alt followed by u. When you type the Alt you'll see letters popup over different areas of the interface. You'll see a U over the user interface you are designing; that's the hint that typing U will move the focus there.

[aetkie]

Trying to understand group by vs order by more clearly. This is how I understand it:

Order By determines the order in which elements will be scanned/highlighted. In Susie’s example, she ordered the body parts on the VSD in numerical order (first the eyes, then nose, etc.). Group By is for grouping elements to be scanned/highlighted together. If Susie wants the row "I feel... interested, confident, and annoyed" to be scanned together, then she would group by #row. If you want all the control buttons to be scanned, group by #controls.

Am I understanding correctly @gbishop ?

[aetkie]

@sofia-bg incorporated the resources that @gbishop noted and your examples into the Customize page in the wiki. It's a work in progress but the info is there.

[gbishop]

Yes, that is correct.

…

On Thu, Sep 7, 2023 at 1:09 PM Andrea Etkie ***@***.***> wrote: Trying to understand group by vs order by more clearly. This is how I understand it: Order By determines the order in which elements will be scanned/highlighted. In Susie’s example, she ordered the body parts on the VSD in numerical order (first the eyes, then nose, etc.). Group By is for grouping elements to be scanned/highlighted together. If Susie wants the row "I feel... interested, confident, and annoyed" to be scanned together, then she would group by #row. If you want all the control buttons to be scanned, group by #controls. Am I understanding correctly @gbishop <https://github.com/gbishop> ? — Reply to this email directly, view it on GitHub <#215 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7ETOVUA2POR7WFM4GDXZH5VPANCNFSM6AAAAAA3VV356M> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[aetkie]

Thanks. Next question - where does the filter #ComponentName come from since it is not a field in the sheet?

[gbishop]

Excellent question! I added a few fields to make it easier to select components. They include #ComponentName, #sheetName and #ComponentType. #ComponentName is the value of the Name property on the component. #ComponentType is Grid, Button, etc.

…

On Thu, Sep 7, 2023 at 1:39 PM Andrea Etkie ***@***.***> wrote: Thanks. Next question - where does the filter #ComponentName come from since it is not a field in the sheet? — Reply to this email directly, view it on GitHub <#215 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7HY4SJJLZWJMKQGSCLXZIBGTANCNFSM6AAAAAA3VV356M> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[sofia-bg]

there are a few of these sprinkled through OS-DPI that it might be worth it to figure out where to collate in one spot in documentation (as well as link to where they are needed). So maybe a document of pre-establishd fields but then also refer to those when documenting how to do something?

for example reference #sheetName would be defined as representing the sheet you want to pull data from in the file you uploaded; then reference #sheetName in example of using it as a filter or condition...

[sofia-bg]

@gbishop off top of my head there is also add_word( ) and add_letter( ) that my assumption is are things you "built in" for us to use in updates are there others?

[aetkie]

Was just thinking the same thing. I am working on a word doc that I will share with the group later. Documenting + explanation would be helpful. For instance, I learned that #ComponentName is linked to whatever the designer named the component in the Layout tab.

[gbishop]

Good idea. Probably should normalize the names also. Why #sheetName rather than #SheetName?

The functions like add_word, etc. are documented in Filter (or somewhere).

[sofia-bg]

That would be helpful @aetkie for you to think about once you compile them, if there could be a pattern that makes sense. I know with my own variables/naming of components I can get sloppy and then forget across designs what my naming conventions were. It is easier for me if I try to stick to a pattern (all lower, initial upper, etc.) so seems that would help for things we make.

Also strikes me that then we could have some "suggestions" re: naming conventions and that might help clarify that sometimes things are "broken" because you typed in the name of the component, state variable, etc. slightly wrong from how you named it? I think that has been hard to communicate and so thinking some support for considering naming conventions might help solve/clarify that for new users?

[gbishop]

Correct. They are added internally.

…

On Thu, Sep 7, 2023 at 1:45 PM Andrea Etkie ***@***.***> wrote: So those 3 fields are built into the OS-DPI and we do not have to specify them in our spreadsheets? — Reply to this email directly, view it on GitHub <#215 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABQK7G7WLFCV3XWNGZBEEDXZIB2PANCNFSM6AAAAAA3VV356M> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>