Orange digital accessibility guidelines

Accessibility user testing #

In order to evaluate the accessibility of a site, application, or product, accessibility experts conduct audits with the objective to check their compliance against standard or best practices rules (WCAG 2.1 Level AA for web).

Once the application is compliant or mostly compliant with guidelines, experts can complete their audit with tests performed by users with disabilities.

In ergonomics: «User testing is the main method for evaluating the user experience in an iterative process. The evaluation relies on the observation of users performing a set of tasks while interacting with a system. Data collected regarding their behavior, reactions and performances provide information about strengths and weaknesses of the evaluated system, as well as about the experience of users.» (translated from Méthodes de design UX. Carine Lallemand & Guillaume Gronier. Eyrolles, 2016).
The goal of the accessibility expert is not so sharp. User tests will mainly allow to:

• Ensure people with disabilities can use the application (absence of blocking points)
• (If difficulties) List new anomalies identified thanks to the user test in progress
• Confirm and illustrate the remaining anomalies detected during previous audit
• Have concrete elements to prioritize corrections

To enable non-ergonomic accessibility experts to realise relevant user tests, we have defined a detailed protocol that is reproducible whatever the application. The protocol consists of 5 parts; only the third stage (application interaction) depends on the application being tested.

1. Reception of the participant (context, objective, instructions)
2. Information regarding the participant profile
   • His/her characteristics (age, digital experience…), equipment of his computer station and job
   • His/her experience with using the application
3. Testing: users interacts with the application following scenarios of use
   • 2 to 5 scenarios are defined based on the main tasks of the application
4. User feedback
   • The feeling after interaction (appreciations, difficulties, suggestions)
   • The ergonomic qualities of the application
5. Conclusion
   • Opinion on the test and information on the continuation of the study

Users are volunteers, ideally 4 or 5 users with different impairments using different assistive technologies (or not using assistive technologies). Tests are performed with the user's own equipment, face-to-face (in person or remotely) or over the phone. Each test lasts no more than 1:30 hours. Sessions are not recorded and the anonymity of the participant is preserved.