[PM] Determine who will conduct QA and accessibility reviews about dhs-oneapp HOT 3 OPEN

Neil can help with accessibility, still figuring out QA.

QA, as I understand it, involves...

• design review
• usability testing
• NJ testing
• UAT, which involves testing every interaction and if the data is getting from user input all the way through the various systems (also work on #134 will help us figure out the plan for this)

from dhs-oneapp.

This is how I define things, which may or may not be how everyone defines these things.

Accessibility

• Accessibility makes sure that our build meets legal guidelines so that everyone can use the app regardless of physical or cognitive abilities.
• Accessibility reviews can start pre-build in the design phase by reviewing mockups to catch any potential patterns that might be problematic (which hopefully shouldn't be too much of an issue because of USWDS). Depending on availability, I would say this is optional. It can be good to catch stuff early on, but that doesn't mean there won't be issues later.
• Accessibility can also work with frontend pre-build to discuss strategy, but I would say this is optional, too. This might be good, though, so Mike and Neil can get on the same page about requirements.
• Mostly, I think of accessibility as a review of the build that happens when it is code-stable. Someone, whether that is Neil, Mike, or both, should understand the legal accessibility requirements and what is considered a launch blocker or not.

QA

• In my experience, QA is when someone formally reviews a stable build pre-launch in a staging/pre-prod environment by making sure the build looks right/isn't broken; checking to make sure all links, fields, and functionality works as expected; and making sure data is pulled/saved as expected. This usually happens in multiple browsers and on multiple devices to make sure the build works across different circumstances.
• To me, QA is about making sure the build works, so I don't qualify usability testing and design reviews as part of QA. Those things are obviously important and should happen, and they ensure that users can navigate the build once it's live, but they don't have anything to do with making sure that clicking link A takes you to B, or submitting form C actually saves on the backend.

UAT

• In my experience, UAT is when real users test that a build actually works. So, this is different than usability testing in that usability testing assesses whether people understand the design, while UAT only assesses whether the build works regardless of whether users find it intuitive or not. To be clear, usability testing should always be done, and UAT is not a replacement for it. They're two very different things, and while you might fix a usability issue if it surfaces in UAT, UAT is so far along in the process that it's not the time to be deeply evaluating usability. It's essentially the last effort to find bugs or problems in the system before launch.

At the VA, we do UAT behind a feature flag in production. This is because all the backend systems at the VA have totally separate staging and production environments. For NJ, I don't know what the backend set up is like, and I think that will determine whether we want to consider UAT for this project or not. If our staging site either connects to the real database or something that is pretty much a copy of it, we probably don't need to run UAT because it's not going to be that much different than QA. If we are connected to a totally different staging environment, then we could consider running UAT since it would happen in a different environment (production) than QA (staging).

Let me know if you have any questions about this!

from dhs-oneapp.

We basically need new USDSers to start, or the team is on the hook to do this work.

from dhs-oneapp.