[CAIA A11y Research]: Public Websites - New Federal Standardized Header and Footer for Va.gov - Phase 2 (Feb. 2024) #73242
Comments
sara-amanda
assigned coforma-terry, sara-amanda, cindymerrill, SarahKay8, laflannery and coforma-jamie
|
@cindymerrill Here is your research ticket. I will plan to coordinate with our CAIA a11ys to see who can first join your two pilot sessions. If you would like to review this ticket, and let us know if you need any adjustments, that would be great! We look forward to working with you, @laflannery and your team! |
|
Thank you @sara-amanda ! I would like to also have an accessibility specialist review my conversation guide (and maybe research plan if you do that, too), so I added that into the description above along with links. It would be great if that could happen this week. The actual dates for the pilot test(s) and research sessions may change but are targeted for the 4th and 5th week of January. |
|
@sara-amanda I also updated other information in the Description, including adding names of people, fixing dates (goes through Feb 7, not 6), and links to the research plan, convo guide, and epic. |
|
Hi @cindymerrill, I took a look at the conversation guide! There are a few additions you'll need to make for screen reader users. Let me know if you have any questions about what I've written here. :) cc @sara-amanda FIRST: Technology - SR, OS, BrowserIt's really important for the a11ys to know which screen reader is being used, what operating system, and what browser. This will let us know what support we need to offer. Please ask this up front, even if Perigean has provided this information to you already - we've found in previous studies that participants sometimes use different technology than they list in the screener (i.e., using an iPad instead of their desktop). SECOND: Level of comfort with Zoom + SRYou'll want to get a sense early of how comfortable they are using Zoom with their screen reader. Have they ever shared their screen before while using a SR? That sort of thing. Folks who are advanced SR users still may not have experience with Zoom specifically. When you ask, note that there are folks on the call that are here to help them through the process. :) For reference, there's a real mix of experience levels - folks could be great on computers but new to SRs, or have many years of experience with SRs, or not be good on computers at all AND have to use an SR. THIRD: If the participant needs support, we'll help them screen shareSome SR users are sighted, but those who aren't won't have any visual cues to direct them to the share screen button. The a11ys on the call will be able to help! In short, the participant needs to do the following:
We'll be able to guide them through this process, if they need the help, via keyboard shortcuts. For the conversation guide, I think the key is just to let the participant know that we're here to help, and to say what they're doing out loud so that if there are any issues, we know what they are. FOURTH: Sharing the link to the prototypeYou'll want to ask them how they are most comfortable receiving the link. Some SR users are fine with Zoom chat, but others will prefer getting it in their email, and some may find it easier to have it dictated to them.
|
|
@cindymerrill Just looked at the research plan, looks great. :) Nothing to change, but a couple of things to keep in mind as you move forward:
|
|
@coforma-jamie Thank you for your feedback! I have a few responses/questions...
I already asked what assistive tech they're using in the Warm-up Questions, but I can add questions about their browser and operating system.
I already have this question in the Warm-up Questions, but I can add "email" as another option that Perigean can do for me--just as they can text the participant.
I assume you mean to say screen magnifier users?
Yes, and also the placement of logo and whether it contributes to their understanding that this is an official government website. |
Perfect!
Great!
Yes, forgive the mistype. :)
That's an excellent point. |
|
@sara-amanda I'd like to confirm that there can be an accessibility specialist to attend each of my research session times, including the ones late in the day Pacific time (6-7:30 pm ET). I assume that's what this means, but just wanted to check: I'll need to know this before I submit my research plan to Research Ops for recruiting. Thanks! |
|
@coforma-jamie I've revised my conversation guide to incorporate your suggestions :). |
|
@coforma-jamie @sara-amanda I was reading through more of the ticket, and I saw the following:
Ahead of time I will know what screen reader or screen magnifier technology they plan to use, but not what operating system and browser. I plan to ask all of these at the beginning of each session, but the only information I'll know beforehand is what technology they'll be using. I also plan to ask at the beginning of each session about their experience using assistive tech. The only thing I'll know about their experience beforehand is that they have at least 2 years of experience (per screening question). Is this all OK for the accessibility specialist(s) who will be attending my research sessions? If not, I could ask more in the screening questions. |
Additional Feedback@coforma-jamie thank you for reviewing this for @cindymerrill ! Evan Burnett (our newest CAIA a11y) and I also reviewed your research plan and your conversation guide, and it appears you have taken into consideration and gone to lengths to insure that this study incorporates and supports users of assistive technology, while also planning to be accommodating to their unique needs in each session. Thank you so very much! We have just had a few remaining recommendations and questions, listed below. Note: Cindy, Evan is in DSVA Slack, but not in GitHub just yet, but will be soon!) cc: @coforma-terry MockupsWhen it becomes available, it may be helpful to have an a11y specialist test the mock up/prototype, prior to the pilot and research sessions, to make sure that basic functionality is present. Comfort Levels
Session Time
AccommodationsPrior to starting, be sure to DisqualificationsIt mentions that AT users with less than
Screen SharingIn @coforma-jamie 's feedback she mentions screen sharing and our ability to assist in Zoom.
Participant Information for a11ysHere is a handy dandy table of information we would love to collect about each user. @cindymerrill - in response to your comment yes, you can absolutely collect the more detailed answers at the start of the session. We would just need the higher-level information you mentioned prior to the sessions to prepare. Device & AT Combinations
Next StepsSession InvitesPlease invite CAIA’s a11ys to your research sessions, and we will work to support as many sessions as possible. @cindymerrill, regarding your question about the late sessions 6-7:30 p.m. ET, we will do our very best to accommodate those sessions.
Late Sessions for ET
@coforma-jamie can you also add your availability her for @cindymerrill ? What to ExpectIn these sessions you can expect Zoom support for users of Assistive Technology, in addition to observations of the session with an a11y lens, which will be synthesized and provided in a deliverable post-research. Research Support Disclaimer
|
|
@sara-amanda Thank you for this additional feedback on accessibility! I have questions and comments about it, which I've listed below...
Accessibility Specialist @laflannery is building the prototype, so I think this is already covered.
What do you recommend? If they don't want a transcript saved, then I assume that we simply proceed without it or a video recording. The notetaker, observers, and I will capture whatever we can in our notes.
I don't think we need to do this, given that the non-AT sessions often lasted only 30-45 minutes, in which case 90 minutes is 2x that. I'd like to have Sitewide UX Lead @aklausmeier weigh in on this. If we do extend the sessions to 120 minutes, then the session times will be even later (e.g., 12-2 pm, 3:30-5:30 pm, and 6-8 pm ET).
What are you envisioning here? I can ask the question, but I don't have a good idea of what might be desired that we could accommodate. Here's what I just added into the conversation guide: Is there anything else that we can do to support you during this session?
@aklausmeier suggested testing with only experienced AT users so that we don't have participants who don't know how to use a screen reader (which I've had get in the way of what we were testing in a past research study).
Not sure. I've never done this in my 3 years of research studies with Veterans, both with and without AT. I don't feel like we should bother, especially because participants often join the session on a different device than they signed up with. But if anyone feels strongly about this, I could ask Perigean if they'd be willing to send instructions to each participant. Would you provide instructions that are customized to the screen reader/magnifier that they tell us that they use? I'm concerned that we won't have enough details to be able to write concise enough instructions, and then the long "if you have this, do this. else if you have this, do this" instructions may be overwhelming. I'd prefer that the accessibility specialist in the research session advise the participant about what to do based on the technology that they bring into the session with them.
I don't think I want to run a screen reader session without an accessibility specialist present, so perhaps @laflannery might be available during the times that CAIA folks are not? This is currently Jan 29 and Feb 1 from 6-7:30 pm ET. Maybe @laflannery could add her availability into the above chart of late sessions? Are there any CAIA A11y folks who are in time zones farther west than ET? If no one is available, then I would need to remove some of the sessions and extend the study for an additional day (Feb 8). I'd like to see which sessions could be accommodated so I'll specify those in my research plan for recruiting. Once Perigean has recruited the participants, I will create an online signup sheet where A11y folks and other observers can sign up for the sessions that are convenient for them, and then I'll send out zoom invites for each session.
Does this mean that you're unable or unwilling to support users with screen magnifiers instead of screen readers? I expect to have about half of each. I believe I could handle screen magnifiers on my own, and then specify that time slots without A11y support are ONLY for screen magnifier users, not screen reader users. |
Feedback on FeedbackHi, @cindymerrill, thank you for your great questions! See our replies below: Recordings
Yes, that is what we were thinking as well. Session Duration
That works, Cindy. Thanks so much! We were just going off of the 60 minutes in the plan, but if they only last 30-45, we are just looking to make sure there is enough extra time set aside, which usually equates to double. Additional Accomodations
It is hard to plan for this because every disability is unique, has different needs and an individual can have one or multiple disabilities. I think what you have added is perfect. If you ask that question, we can pivot and likely accommodate them. For example, they might ask you to read more slowly, or repeat the questions more than once. Some may need live captions turned on. Some may ask for you to avoid cross-talk, and make it difficult if more than one person is talking. They may request extra time to verbally respond to the questions. AT Experience Levels
We would still recommend, for future studies to incorporate varying skill levels with AT, when possible, so that it can be tested for users, who span the wide spectrum of AT skills. This is because you will likely have a real user who may not have much experience with screen readers, yet. Screen Sharing Instructions in Advance
Those are all valid points. We recommended this because we have seen in the sessions we have been sitting in on, where that has been a problem area. If you have not, perhaps this is just something to keep on your radar for future studies, in the event this becomes a stronger need? CAIA SupportSession Times PST
Yes, let's see if @laflannery might have availability, for Jan 29 and Feb 1 from 6-7:30 pm ET. We are also seeing if @SarahKay8 is available during those time slots as well. Unfortunately, we no longer have a CAIA a11y on the West Coast. They moved on to a new project. AT Technical Support
Not at all. We can sit in on any and all AT sessions, and can (in most instances) provide a level of support. Our primary support is with screen readers, since those are pretty standardly used. Additional AT software can vary, and we just don't have the access or ability to be specialized in them all; however, we can do our best to support those users as well. Additional QuestionsPlease let us know if you have any other questions. |
|
Thank you for your responses, @sara-amanda ! I'll wait to hear whether @laflannery or @SarahKay8 might have availability for Jan 29 and/or Feb 1 from 6-7:30 pm ET. If neither does, then I'll restrict those two sessions to screen magnifier users only. Please let me know ASAP so I can launch recruiting. |
|
I can be available |
|
I'll be out the week of the 29th |
|
Yay, thanks, @laflannery ! I'll add you to the schedule above to cover Jan 29 and Feb 1 from 6-7:30 pm ET, if we have sessions scheduled then. |
|
@sara-amanda Our prototype is all ready for pilot testing next week. Could we schedule a pilot participant or two with someone using a screen reader? Not sure if it should be one pilot test or two. Ideally, I'd like to get the prototype tested with a screen reader and a screen magnifier, and with a smartphone and with a computer. Timing should be on Jan 23, 24, and/or 25 during a Pacific-friendly time; I'll get scheduling constraints from my team and then post them here. UPDATE: Here are 2 potential times:
Please let me know if/when we could have a pilot with a CAIA accessibility specialist. Thanks! |
Pilot Sessions@cindymerrill that is great news! We are looking forward to assisting you with your research sessions and pilots. CAIA a11ys and AT Available
Next Steps
|
If you can cross-post that link here too @cindymerrill that would be great! Thanks so much! |
|
@sara-amanda Here's the link to sign up, but it currently has ALL possible time slots. I'm planning to update it when I know which 11 slots have been scheduled by Perigean. You need to be logged into the VA computer system to view and edit it because it's on SharePoint. You can sign up by entering your email address in place of "Need A11y", and then I'll add whoever signs up to the calendar invite and an optional debrief afterwards. |
Calendar Invites@cindymerrill Right now, we have holds on our calendar for all the slots provided above.
CAIA will then be connected to the official invite (should it change at all) with the assignees, and have visibility on the calendar, that shows our time is held with your team on our 😄 Thanks so much! |
|
Hi @sara-amanda! I can add you to all of the AT research sessions that have been already scheduled (9 of 11)--DONE. I'll let you know when the last 2 have been scheduled and then the schedule will be set on my end. My schedule on SharePoint now has the screen reader/magnifier software and device for each participant that has been scheduled so far. |
|
@sara-amanda I noticed that in your table of time slots above, some have more than one accessibility specialist listed. I assume that there will be only one of them assigned to each slot because I am limited in the number of observers that I can allow. Since 1 of the 3 available observer slots will be for an accessibility specialist, that means that there can be only 2 other observers. I would like my team and stakeholders to be able to fill both of these remaining observer slots, but if there are open slots after that, then it's fine if a second accessibility specialist is interested in observing (@laflannery told me that Evan might want to observe some sessions but isn't ready to be the accessibility specialist providing tech support). Hope that works for you. |
|
@sara-amanda Accessibility specialists could start signing up now to attend research sessions in this spreadsheet on SharePoint. We have 9 of 11 participants confirmed, and we’ll update the schedule as soon as the last two participants are confirmed. |
Thank you! @cindymerrill |
No worries! Thanks for the information, @cindymerrill we typically put all of our availability in the table, then once we have the invites, we assign them out. @EvanAtCoforma would like to observe, but if there are no open slots, we understand. He can jump on another research study that has the open slots. We have a few running concurrently at the moment. 😄 |
|
@sara-amanda Since the research sessions start on Monday and the remaining 2 sessions still haven't been scheduled and are unlikely to today, I'd like to start getting accessibility specialists assigned to the sessions. Then I can add them to the calendar invites. |
Updated Session TableThe CAIA availability table has been updated as of today, Jan. 26, 2024. Invites Received
Slots Available
No Official Invites
In this Ticket*Edited to remove pilot invites from the total. |
|
@sara-amanda A 10th session was just scheduled for Mon Feb 5 from 3:30-5 pm. We're still waiting for an 11th session to be scheduled. How did you count that 11 sessions had already been scheduled (because I think there were only 9 and now 10)? Please don't delete the other holds because we don't know when the 11th participant will be scheduled. |
Session Total
cc: @coforma-terry |
|
@sara-amanda Can you confirm that I'm covering the following, sorry I'm just very confused and I don't think I have all the invites that the table indicates:
|
|
Quick Question: @cindymerrill - I updated the links to the conversation guide and the research plan in the original ticket, under supporting artifacts, because they were giving me a 404 for each. Should I update the product outline link with this link? I wanted to confirm this one. Thanks so much! |
|
Yes, the links changed yesterday (unfortunate timing). Here's the convo guide and the research plan. The product outline link you have seems OK. @sara-amanda |
Invite Forward Confirmations@laflannery I went ahead and cross-referenced the ones from your list below, and included the list of holds you are on, where we are waiting for invites, if this helps.
Outstanding Holds, Pending Invites to ForwardThere are two more out there you are on holds for, that I don't have invites on just yet:
|
|
I'm all under control now, thank you so much @sara-amanda! |
|
@laflannery 2/5 3:30-5 has a confirmed participant in it now, so you're on for that one! FYI @sara-amanda UPDATE: Just added you both to the new session invite. Now we're just waiting for Perigean to schedule one more session, and then all the holds can go away. |
|
@sara-amanda You can release the holds on 1/31 because Perigean is NOT going to schedule any sessions on that day. |
|
@sara-amanda I have two questions about some of the updates you made to the session invites:
|
|
@cindymerrill The 2/2 6-7:30 was my mistake. It's confusing with the signup sheet and this table because it's 2 sources of truth and I am trying to make sure I am where I need to be and you have someone at each session. Jamie was on the 5th and I accidently mistook that when I was looking at the table |
|
Thank you for clarifying, @laflannery ! |
|
Here is a quick update @cindymerrill: Holds RemovedJan. 31, 2024
✅ Thank you! Jan. 31, 2024, no longer has holds on the CAIA A11y Capacity Calendar Table UpdatedI also broke out our CAIA A11y Availability Table in this ticket by:
The CAIA A11Y Capacity Calendar will reflect this, now. Availability of A11ysIf you see more than one name on the holds, that is just the number of a11ys who are available to cover that session. We will only assign one, based on your limit of observers. If our availability changes, we have a note that there is another a11y available to cover, so you will see an "or" in the table, to reflect that. Daily DebriefInvites for the optional daily debrief sessions received. This has been added to the CAIA A11y Capacity Calendar. |
|
@sara-amanda My last header/footer research session has finally been scheduled--for Tuesday 2/6 from 3:30-5 pm ET. So I'd like to have an accessibility specialist assigned to that session if possible, and all the other time slot holds can be released. |
|
Thank you @cindymerrill ! Ticket Update
|
|
@coforma-jamie @EvanAtCoforma @sara-amanda Could you please read these observer instructions before you provide accessibility support in my upcoming header/footer research sessions? Thanks! The purpose of this research is to get user feedback on the design prototype so we can see whether/how the new design affects wayfinding and task success. Here are some tips on being an observer:
|
|
THANK YOU so much for your excellent accessibility tech support, @laflannery @sara-amanda ! Laura covered 8 of the 9 sessions, which was amazing! Sorry that I didn't get to work with @EvanAtCoforma and @coforma-jamie this time because their sessions had no-show participants :(. |
Project Overview
Note
High-Level Overview: Public Websites
11Assistive Tech Usersscreen magnificationandscreen readersscreen magnificationandscreen readeron 1/23 and 1/25/2024 detailed in ticket: 74018Team Info.
Toggle to view Public Websites Team information
Shane Strassberg (VA Research Ops)Initial Slack Request
Toggle to view original intake information
Do you have an accessibility champion?
Yes, we have a champion
If you have an accessibility champion, what's their name?
Laura Flannery
Is anyone on your team interested in becoming an accessibility champion?
No one else at this time
Calendar checking
Have you reviewed the CAIA capacity calendar?
When is the timeframe for your testing?
Research Sessions
Tip
Additional scheduling information can be found in the comments of this ticket.
Toggle to view initial research session information
Scheduling 1-2 pilot tests:
See task list for link to pilot session ticket.
Scheduling Research Sessions
Planned dates of research: Jan 29 - Feb 7, 2024
We would like to request that Perigean calls each participant to remind them about the session, in addition to emailing them. And please include the session time in each participant's own time zone (from their address).
Length of sessions
Team Availability: Public Websites
Final Invites Received
Toggle to view the final schedule
SCREEN READER-PILOT@SarahKay8SCREEN MAG-PILOT@eli-oatopen slot@EvanAtCoforma@EvanAtCoforma@EvanAtCoformaOR@sara-amandaFeb 6, 20243:30-5 pmuser didn't showLaura's Assigned Sessions
Thank you for helping out with these sessions, Laura, since we had 5 studies that week!
open slot@EvanAtCoforma@EvanAtCoforma@EvanAtCoformaCAIA's Assigned Sessions
SCREEN READER-PILOT@SarahKay8SCREEN MAG-PILOT@eli-oatFeb 6, 20243:30-5 pmuser didn't showImportant
Table copied over as of 1/30/2024, following the last session being added. The working comment is now closed, and this is the table to reference.
Supporting artifacts
Toggle to view supporting artifacts
Broken LinkNONETBDRelinked 1/26/2024Relinked 1/26/2024TBDNext steps
Sitewide Public Websites - Next Steps
CAIA - Next Steps
The text was updated successfully, but these errors were encountered: