Back to Writing

Writeup

An ADA compliance agent that won't touch code without asking

I'd already started using AI to help implement ADA compliance on a task I'd been assigned — mostly because it was doing a decent job and saving me time. What turned it from a personal shortcut into something worth building properly happened in a meeting, when a developer mentioned the standard estimate for this kind of work: about 50 hours per system. Resourcing for it was tight too — not every developer was cleared or available to take it on, so the hours didn't just cost time, they cost a scarce kind of time. That's when it stopped being a productivity hack and became something I wanted to build for real.

The estimate

Fifty hours doesn't go where you'd expect. A meaningful chunk is research — cross-referencing WCAG criteria to confirm what actually needs to change and why. Another chunk is the low-hanging fruit: hunting down every div or input that's quietly acting like a button, missing an ARIA label, or relying on visual cues a screen reader can't see. The rest is the harder stuff — JavaScript that needs to behave differently for assistive tech, not just markup that needs a tag swapped. None of it is hard to understand in isolation. It's just a lot of files, and not every developer was available to do it — which is its own kind of expensive.

How it actually works

It's built as an AGENT.md file — GitHub Copilot's way of letting you define a persistent agent with its own instructions, run on Codex. I tested it at a couple of effort settings and landed on medium; it consistently gave better code than low, without the cost of going higher than I needed.

The first version reviewed entire folders at once, and it didn't work well — the agent lost track of finer details across that much code, and the proposals that came back missed context that mattered. Scoping it down to one file at a time fixed that. It loses the broader picture of the codebase, but it gets the close reading right, which matters more for this kind of work.

Within a file, it's looking for two main things: HTML that isn't semantic — divs and inputs standing in for buttons, missing ARIA labels, elements with no accessible name — and JavaScript that touches the DOM in ways that affect screen readers, since plenty of markup gets generated dynamically rather than written by hand.

Why it proposes instead of applies

Every run produces two tables instead of a diff. One lists the WCAG rules it thinks apply. The other lists the proposed changes, each scored for impact and feasibility — both judged by the agent itself. The semantic HTML fixes — converting a fake button, adding a missing label — usually score high on both: high impact because they're foundational to screen-reader navigation, high feasibility because they're low-risk to implement.

The clearest example of something genuinely harder to spot manually was a focus-handling bug. When a client submitted a form, the resulting message — success or error — wouldn't get read by the screen reader. Sometimes it would restart from the top of the page; sometimes it wouldn't announce anything at all. The fix was small — a JavaScript update that moves focus to the message after submission completes — but the difference for someone actually relying on a screen reader is significant. That's the kind of issue that's easy to miss if you're not testing with one, and easy for an agent to catch consistently if you ask it to look.

The WCAG table is the part I'm most deliberate about. The agent generates those citations without any actual grounded access to the WCAG spec — it's working from what it knows, not querying the real rule text. So I built it to be a reference for cross-checking, not a source of truth. The output doesn't ask you to believe the agent got the citation right. It hands you exactly what to go verify.

From there, the developer picks which proposals to actually implement — not all-or-nothing, just the ones worth doing. It also reports a current ADA/WCAG compliance estimate and a projected one if the selected changes go in, so there's a number attached to "is this worth doing" before any code changes.

The test run

The test was 10 files, 50+ individual changes applied. By hand, with the WCAG research and the implementation both factored in, that was roughly a 3-hour job. The agent got through it in about 30 minutes.

It wasn't flawless. A couple of the proposed updates conflicted with existing code in a way that meant the screen reader update didn't actually take effect as written. Not a big deal — once flagged, the agent revised the fix without much back-and-forth. It's the kind of thing I'd expect to catch in review regardless, and catching it is the point of reviewing.

I hadn't run this past anyone before that point. I built the first working version on my own, and only reached out to one developer once I had something I was confident enough to put my name on — more a test of whether it was worth pursuing further than a formal proposal. Before he'd had the chance to actually use it, we ran out of GitHub Copilot tokens. It wasn't the outcome I was hoping for. But it taught me something I needed — that token efficiency in how the agent's output is structured isn't optional, it's part of the design itself.

What's next

The output format is close to where I want it — efficient enough that running out of tokens mid-test shouldn't happen again. The actual validation loop with another developer is still unfinished; that's the next thing to pick back up, now that the problem that interrupted it the first time is mostly solved. Beyond that, the open question is the same one underneath every agent I build: how much of this should stay something only I run, and how much is worth turning into something the rest of the team can pick up themselves.

Update: that validation loop happened — see the follow-up on teaching the agent to generate SVN patch files.

Back to Writing Questions? Let's talk →