Rule checkers ask is there a label? Signpost asks does the label mean anything?
A share button named Save passes every rule in axe-core and every check in Xcode's
Accessibility Inspector — it has a name, and the name is specific, unique and correctly
punctuated. It is also wrong, and nothing that reads only the string can tell.
Code & full evaluation · Model · Dataset
Starting Python in your browser — nothing is uploaded anywhere.
missing class in the evaluation shows it. It does
not replace testing with assistive technology, or review by people who use it. It narrows what a
person has to look at. Every finding cites a documented guideline, marked
normative (a WCAG success criterion) or
guidance (a best-practice guide), rather than asserting what someone
will experience.
On 860 held-out labels from sites the model never saw, with 309 planted defects: rule checkers find 23%, Signpost adds 42%, and 35% are found by neither. Of the defects the rules missed, Signpost recovers 54.6% [48.3, 60.8] while reporting 0.59% of known-good labels (1.27% on the widest denominator — both are published). The threshold slider above is that trade-off; move it and watch the count change.
The measurements come from a synthetic corpus composed for this project, not from crawled pages. What that does and does not license you to conclude is set out in the dataset card.