This is a transcript of one of a series of discussions with W4A researchers that took place during W4A 2015. It features Giorgio Brajnik, University of Udine, Italy.
Giorgio discusses his work with evaluation and QA testing in industry, and meeting the challenge of ensuring accessibility is integrated effectively into design, development and testing processes.
David Sloan: Giorgio, you’ve done a lot of research in the area of accessibility evaluation and quality assurance testing. Can you talk a bit more about your work?
Giorgio: Sure. I’ve been working on methodological aspects, dealing with what kind of method you can use to find different kinds of problems. And the fact is that different methods have different power, so to speak, and different weak points.
And so it’s important to understand how you choose a method depending on the kind of situations and the kind of goals that you are pursuing, because different methods and different guidelines and different strategies reflect on different, say, kind of validity. Are you really spotting accessibility problems? Are you spotting problems that, say, deal with other kinds of issues, like usability for example.
And, different methods also reflect different kinds of reliability…how repeatable are the results? If you give the same guidelines to different groups of evaluators, with the same web site, on different days, probably you get different results. Most of the experts usually don’t agree 100% on the findings that they find on web pages, on web sites.
And so this is probably also in part due to the fact that accessibility’s very subjective, it depends heavily on how people interpret information, how people do problem-solving while using a web application, a web site. And that’s very difficult to characterize in a very objective way.
DS: So when it comes to applying the knowledge that you’re gathering in an industrial setting, let’s say in a software development practice, how can your work help to optimize development practices, help someone to save money, or help somebody meet their legal obligations?
Giorgio: Well, I would say that the first question somebody should ask is: why you are doing something? What is the purpose of an accessibility auditing or investigation—what do you want to achieve? Is it for legal purposes, and what’s the standard behind it? Or are you doing it to improve the quality of the web site for some reason? Are you doing it for marketing purposes? Are you doing it for improving the quality of the image of the brand that is associated to the web site?
So depending on the goal, of course you have to choose different strategies. And, as we were talking this morning, the strategy for web accessibility, could be closely tied with the strategy other teams within the company pursue for testing with, say, functionality or performance of the web site.
So I’m saying accessibility testing should be part of quality assurance in general. And that’s a very complex question, a complex matter, because quality means different things to different stakeholders.
DS: That’s a very good question—so do you have any examples of where you’ve worked with people in industry to either understand existing practices or help them with tools or methodologies?
Giorgio: Yes, I’ve been working recently with a company that does…they develop mobile applications. And they started asking me, while I was doing some consultancy work in helping them set up a testing strategy for functional purposes, for functional testing. Then, someday, the discussion drifted onto accessibility and usability.
And, so the problem of what is quality and what is quality assurance strategy and what kind of goals would you like to pursue with that facet of problem…these kinds of questions popped out. And we had some solutions…we suggested some solutions to them to how to do usability/accessibility investigations while doing design of the web pages, or screens of the application.
DS: So, integrating testing at the right stages of design…?
Giorgio: Yes, so some of them could be done using early prototypes, like wireframes, and some of them could be done only when using a highly refined prototype, so perhaps almost the final product, before releasing it.
DS: So how’s that advice been received? Or are they still working on it?
Giorgio. They’re still working on it. It’s been well-received, and I must say that in some cases I got developers and designers telling me: “we tried to follow the good practices in doing early prototypes and so on, and then we did that…sketches and so on, we did informal user testing…and then when we developed the final product we discovered that there are people who are color-blind, and because we did this prototype using handmade prototypes, we didn’t pay attention to the color, of course.”
And so, they discovered that these strange issues with accessibility that pop up now and then, and which would be better if somebody would use better principles at the beginning.
DS: Yes, so in that case, they’re making a mistake, and discovering the cost of fixing the mistake, and also “ah we could have sorted this much earlier”
Giorgio. Yes, but at least they are discovering that they put together a mechanism that through which they collected feedback and were able to fix the problem before releasing the application, which was good.
DS: And do you see a company like that as being way ahead of the curve, or is there a greater demand for the sort of support that you’re providing?
Giorgio: I wouldn’t say it’s a great demand, but I would say that the money is slightly increasing in the part of the world in which I work, which is northeastern Italy. There are more and more companies that are aware of accessibility and they are aware of the potential of accessibility in terms of enlarging the market and improving the image, the brand image.
