Accessibility standards & contribution guidelines for Django

[Thibaud Colas]

Hi django-developers,

This is a long-overdue follow up to DEP 11 and the creation of Django’s accessibility team. As part of implementing what this DEP outlines, we need to discuss what accessibility standards should apply to Django, among other things. In my opinion this is both a matter of deciding what pre-existing standards / guidelines would be relevant in theory, but also how to make sure whichever ones we choose are achievable to follow in practice. We need to find some consensus on what Django as a project should aim for, so our team and all contributors can take it into consideration, and in particular so we can document what we want to commit to as a reference for accessibility work.

Here are accessibility team responsibilities I would like to bring up for discussion in particular:

• Deciding on any relevant accessibility guidelines to follow, such as WCAG, and at which conformance level.
• Coordinating regular manual accessibility audits on all relevant projects.
• Writing and maintaining documentation relating to accessibility, such as a statement of commitment to accessibility issues, and contribution guidelines.

Now I’ll try to outline what I’d suggest for each area, which people here are very welcome to disagree with as relevant. 

Accessibility guidelines

In my opinion, the most important standard for _sites built with Django_ is WCAG 2.1, AA level. Anything resulting in AA level issues should be considered as a bug. Wherever possible, we should also strive to not get in the way of sites built with Django aiming for AAA level compliance.

For the Django admin, I would recommend the same target of WCAG 2.1 AA level as a baseline, aiming for AAA wherever possible. For example, the link text issue discussed in Default change password UX: area for improvements is a case where I’d expect us to want to follow WCAG SC 2.4.9: Link Purpose (Link Only), even though it’s level AAA.

This distinction between "sites built with Django" and the "Django admin" highlights an important point, which is that for lots of sites the Django admin would be what accessibility standards call an authoring tool. There is a separate standard for authoring tools that builds upon WCAG 2.1: ATAG 2.0. I think we should also explicitly aim for compliance with ATAG 2.0 wherever possible, as it has a lot of useful guidelines we could follow for the Django admin. For Django itself, there might be very little to do to implement those guidelines, but for the wider ecosystem of CMS-like features in Django it would make a clear difference if Django was clearly defined as an authoring tool. WCAG itself is a very thin baseline as standards go, so anything that goes beyond it is well worth considering in my opinion.

For everything else within the Django ecosystem – the documentation, the djangoproject.com website, the bug tracker, IRC, plain-text documentation, etc. – we should follow the exact same standard. I would also really like to see the same standard being explicitly recommended for consideration for third-party Django projects (similarly to Django projects wanting to provide the same level of browser and language support as Django itself, I imagine). This probably warrants a separate conversation but I would also recommend the Django Software Foundation explicitly mandating the same standard for any project or event it endorses.

It’s worth mentioning as well that the next version of WCAG, WCAG 2.2, is due to be released by the end of 2021. I’d recommend we don’t aim for support with WCAG 2.2 right now, but do so as soon as it’s released. In the meantime, we should also consider any WCAG 2.2-only guideline wherever reasonable (for example in audits, design of new features, etc.).

Contribution guidelines

Aiming for compliance with specific standards is good, but we shouldn’t expect Django contributors to be familiar with WCAG, or any other standard. I think it’s important we update the contribution guidelines to propose practical support targets:

• Which automated or semi-automated testing tools to use when testing changes to Django. My go-to is anything Axe-based, as it’s ubiquitous, and has very few false positives. In particular with the Accessibility Insights browser extension.
• Which assistive technologies should we explicitly strive to test Django with, so contributors have clear guidance on how to test their changes. For example here are the testing guidelines for public sector sites in the UK.
• Whether / how best for the accessibility team to be involved with UX or design changes in Django & other projects, or other changes that would require accessibility considerations.

Manual audits

This would be very dependent on the above two points. Beyond this, it would be good to discuss what audit methodologies would be satisfactory (whole of the admin, sampling, what demo site to test, etc.) – and what would be the most useful on the reporting side.

Short-term actions

There are a couple more things I wanted to raise for others to feed back on, relevant to all of the above but that I think would make a bigger difference in the short term:

• Currently I’m only able to test Django with VoiceOver on macOS, and as such can’t test with a lot of other popular assistive technologies. There is a SaaS called Assistiv Labs that provides VMs with pre-installed assistive technologies, with a free plan for open-source projects. Could we ask them to sponsor Django? I imagine this might be more of a question for the relevant DSF team(s) but wanted to see what people thought here first. Note this isn’t specific to my situation – assistive technologies are very segmented by operating system, so we would need to find strategies to test across different ones one way or another.
• There is an EU-funded research project on authoring tools’ accessibility currently under way, We4Authors Cluster. I’m interested in filling out a survey from this project as a way to assess Django’s position with their early findings. I don’t think this engages anyone in any way but thought it would be good to bring it up nonetheless in case it raises any red flags.

---

How does all of this sound? Anything I missed that should be brought up at the same time? I appreciate this might be a lot of things to comment on over email – assuming there is some high-level agreement, it might be easier for people to review as PR(s) to the Django contributing docs, but thought I’d try here first.

Best regards,

Thibaud

[Carlton Gibson]

Hi Thibaud. 

Thanks for this, and for your efforts here generally! 🥇

> ...a long-overdue follow up...

This is the only bit I can't agree with 😀 — ticking along, slow-and-steady, is realistic/sustainable/desirable/... 🙂

You mention a lot of things, so that'll take a little while to absorb — links to click, things to read. (If you have stronger opinions on "We should do X, using tool Y" then that's worth putting forward.)

The thought that comes up is, what can we do on the documentation front? We have guides/how-tos on security, testing, and so on. Could we add similar for accessibility? 

* These are the standards to follow

* Here's some of the basics of making sure your Django site is accessible. 

* Here are some tools you can use to test/ensure that. 

(Lots of this is in your email here.) 

I've read various articles on this over the years, but don't feel confident on it at all. I think a trusted guide in the Django docs would be an awesome resource. (As a Django user I want a guide in the docs so that I can make my site accessible from the start.) 

That doesn't directly address your specific questions. 

Kind Regards,

Carlton

[Sarah A]

Hi folks,

I'm interested in accessibility lately, I discovered a tool that can probably be added to the list: pa11y. It was used for automated tests.

I don't have enough experience to give my opinion but @Thibaud you might have an opinion on it.

I wanted to check the issue on password UX but maybe I can help on others things related to this before. Tell me what do you think.

Best regards,

Sarah

[Tobias Bengfort]

Hi Thibaud,

thanks for the follow up. +1 on basically everything you wrote!

On 15/06/2021 02.59, Thibaud Colas wrote:
> This distinction between "sites built with Django" and the "Django
> admin" highlights an important point, which is that for lots of sites
> the Django admin would be what accessibility standards call an authoring
> tool. There is a separate standard for authoring tools that builds upon
> WCAG 2.1: ATAG 2.0. I think we should also explicitly aim for compliance
> with ATAG 2.0 wherever possible, as it has a lot of useful guidelines we
> could follow for the Django admin. For Django itself, there might be
> very little to do to implement those guidelines, but for the wider
> ecosystem of CMS-like features in Django it would make a clear
> difference if Django was clearly defined as an authoring tool. WCAG
> itself is a very thin baseline as standards go, so anything that goes
> beyond it is well worth considering in my opinion.

I am not completely sure about the definition of an "authoring tool",
but it might make sense to consider django as a whole as such an
authoring tool, not just the Django admin. ATAG A. (Make the authoring
tool user interface accessible) would then only apply to Django admin,
but ATAG B. (Support the production of accessible content) would apply
to everything.

> * *Which automated or semi-automated testing tools to use when testing
> changes to Django*. My go-to is anything Axe
> <https://www.deque.com/axe/>-based, as it’s ubiquitous, and has very

> few false positives. In particular with the Accessibility Insights

> <https://accessibilityinsights.io/> browser extension.

AFAIK the accessibility checker in chrome's lighthouse is also based on
Axe. As it is already built into a very popular browser this might lower
the threshold for testing even further.

tobias

[Thibaud Colas]

Thank you all for the feedback :) I have a feeling it might indeed be easier for me to propose this for review in the form of a pull request to the Django docs, as I have quite a clear idea of what I’d propose at a high level.

On testing tools, yes, both Pa11y and Lighthouse would be worth considering. They both use Axe to do the actual checks, which means results will be largely comparable between Lighthouse, Pa11y, and Accessibility Insights. I’ve set up automated tests for the Django admin based on Pa11y + Lighthouse, which you can see here: django_admin_tests. My preference as "the one tool" to recommend still is Accessibility Insights, as it contains a good mixture of automated and semi-automated checks, as well as a very thorough wizard to guide people through manual checks.

> I wanted to check the issue on password UX but maybe I can help on others things related to this before. Tell me what do you think.

There’s quite a lot of ground to cover, so any and all help is welcome :) If you’re interested in UX work generally – once we start properly auditing the Django admin, reporting issues will be straightforward enough, but proposing solutions will be much harder. Anyone with a good mix of Django and UX expertise will be in a great position to contribute. For example, last week I audited django-postgres-metrics and found a few issues relating to table sorting. Fixing those issues could be an opportunity to generally improve the table sorting UX for all.

> I am not completely sure about the definition of an "authoring tool",

but it might make sense to consider django as a whole as such an
authoring tool, not just the Django admin. ATAG A. (Make the authoring
tool user interface accessible) would then only apply to Django admin,
but ATAG B. (Support the production of accessible content) would apply
to everything.

Indeed! There are a few things that likely couldn’t apply (Guideline B.3.2: Assist authors in repairing accessibility problems) but they feel like they might be the exception rather than the rule. It’s certainly a helpful lens to consider accessibility improvements through.

Best regards,

Thibaud

[sapt...@gmail.com]

On Thursday, 17 June 2021 at 14:01:15 UTC+5:30 Thibaud Colas wrote:

Thank you all for the feedback :) I have a feeling it might indeed be easier for me to propose this for review in the form of a pull request to the Django docs, as I have quite a clear idea of what I’d propose at a high level.

On testing tools, yes, both Pa11y and Lighthouse would be worth considering. They both use Axe to do the actual checks, which means results will be largely comparable between Lighthouse, Pa11y, and Accessibility Insights. I’ve set up automated tests for the Django admin based on Pa11y + Lighthouse, which you can see here: django_admin_tests. My preference as "the one tool" to recommend still is Accessibility Insights, as it contains a good mixture of automated and semi-automated checks, as well as a very thorough wizard to guide people through manual checks.

I love the Django admin accessibility tests and repo that you made. The demo report looks pretty decent to me! I think that would be super useful when issues are created and for fixing the issues. There's also this web app https://beinclusive.app/ which allows you to manage accessibility audits better which might be helpful in some form when we have a concrete accessibility audit.

Might I also add that once we have a decent accessibility improvement, we add something like accessLint in the CI to assure that future PRs don't have any glaring accessibility issues?