Simplifying Accessibility Evaluation

This is a transcript of one of a series of discussions with W4A researchers that took place during W4A 2015. It features Chris Bailey (Vodafone, UK; formerly AbilityNet and Teesside University).

Chris talks about his PhD research on helping teach students about accessibility using an evaluation tool, and how that research influences his work as an accessibility consultant

David Sloan: So, Chris, tell us a little bit about the web accessibility research you did as a student.

Chris: I’d be happy to. Fundamentally, the current approach to ensuring accessibility is mainly about evaluation. So you take a resource, and you test it, and you say “is it accessible or not?” Now, accessibility is really about making something available to everyone, and for me one of the biggest ironies, or paradoxes in this world is that the amount of knowledge and expertise required to actually use current resources for accessibility evaluation is huge.

The tools are complex, and…take a look at WCAG [Web Content Accessibility Guidelines], read them, and honestly ask yourself “Can someone take these guidelines and easily use them to develop an accessible web site?” Research shows they can’t.

So here’s what happened when I did my research. I was a novice. I didn’t know anything about accessibility. So I read…I read your work, David, I read other people’s work, I read the guidelines. A lot. And then, part of my job was actually testing web sites.

And sometimes, I would take the guidelines, I would apply them to the web site, and I’d get them wrong. But that’s okay, that’s how you learn. So, for me, evaluation was a learning experience.

And that’s really when I had a “light-bulb” moment. And I thought: hang on, I’m experiencing a learning experience…I didn’t know anything about accessibility, and now I think I know quite a lot.

So, hang on, let’s look at what I do, let’s look at the processes that I perform when I’m doing an evaluation, and let’s actually break that down, and let’s break it down into steps, and let’s break it down into so many steps that it’s easy to use. And that if you actually break it down, define it…

DS: Then, you’re deconstructing it into something that more people can use…you’re simplifying the process, right?

Chris: Thankyou, yeah, I think you got me back on track there! It needs to be a simple process…accessibility evaluation needs to be simple. So what I did, I developed a method. When I was performing evaluations…you’re looking for barriers, okay? You’re looking for problems that would present issues to real users.

So what I did, I thought: right, let’s break down a process to check for each individual barrier. Let’s define that process, and let’s make it something that a novice can use. And that’s what I did.

I developed a tool, and embedded in that tool was a method, the Structured Walkthrough Method. It was based on Barrier Walkthrough, which was developed by Giorgio Brajnik, but I developed this method to be used by novices.

And because I was working in a university, these novices were students of computing, so they were like me, they dodn’t know anything about accessibility when they started, but they wanted to…

DS: And they needed to?

Chris: And they needed to, yes, because they’re going to be working in industry, they’re going to move into industry, they’re going to be making real sites. And we’ll never solve the problem of accessibility if people don’t do it from the very start, and learn about it from the start of their career.

So, here’s what I did. I did a lot of experiments, a lot of testing, I worked with students, and basically we embedded accessibility evaluation into their assignments.

DS: Wow.

Chris: So, all of a sudden, they were motivated to perform an evaluation, and they were motivated to use my tool and my method.

DS: And that’s because you’d lowered the barrier…you’d reduced the mystique or the sort of specialism around accessibility that they maybe felt before?

Chris: Of course, because during the lectures, we’d show them the guidelines, we’d show them the tools, and they’d think: well, I don’t understand them…I don’t know what that means…how am I supposed to use that in the real world?

So I said okay, well, there’s the WCAG approach, how about you use my tool? And they did. So, during lessons, they would use the tool to perform evaluations. And, okay, they didn’t get it all right, but they got a lot of it right.

I’ll tell you a story. So, part of the assignment was for students to pick a web site and evaluate it. And, the assignment was pretty tough, I tell you, because I was using it for my PhD research, so the results had to be quite detailed, I needed lots of information. So these poor students had to perform this huge evaluation, it was amazing.

DS: We’ve all been there, we work in accessibility…! (laughs)

Chris: And there was a great story, one girl came up to me after we’d had a lecture, after the assignment, so the assignment had been done. She’d used the tool, she’d used my method, and obviously she’d been present in my lectures.

And she said to me “Chris, I’m going to be honest. I found that assignment really, really hard. But I feel like I learned something.” And for me, that’s the most valuable part of my research. It made people learn about it, and it made people understand accessibility.

DS: Right. And that’s all you can ask for, as a researcher, that you’re making a difference?

Chris: So that for me was the impact, yes. I was making a difference and people were learning. It still happens now, so I work now in industry. I used to work at an agency, AbilityNet, actually a charity, and I would show my colleagues…these were colleagues who wouldn’t be consultants, but maybe admin or project managers, and I would show them this tool and say: this is what I used to show my students, this is how they learned about accessibility. And we had an intern once, and I showed her it, and she said “yes, I’m learning about accessibility now, I understand it.”

And for me again, that was brilliant.

DS: So you’ve effectively taken knowledge and made it accessible, made it possible for people to learn and apply accessibility out there in the real world.

Chris: Yes, I believe I have, and you know, I really should…the tool has been sat there for a few years, not doing anything and I think now’s the time to do something with it, because I think it will help solve the problem of accessibility.

The 13th Web for All Conference

11–13 April 2016. Co-located with WWW2016. Montreal, Canada.

Important dates

» Camera Ready papers: 10 March

Register for W4A

» Register usingWWW2016 Conference Registration

WWW/W4A Hackathon

» Hackathon details
» Hackathon Signup Form


Intuit will award $2000 and $1000 to the best technical and communication papers

Google will sponsor 6 PhD students to participate in the W4A Doctoral Consortium

IBM will provide travel grants to the winners of the People with Disabilities Student Award

The Paciello Group will give the Judges' and Delegates' awards to the winners of the Accessibility Challenge

Canvas and Intuit will award student grants to attend W4A

The ABILITY Magazine will feature the W4A 2016 award winners in their printed and online publications. See the 2015 winners report