Change:

Each dimension will contain stages” based on the proof points that are demonstrated.

To:

Each dimension will contain stages based on the proof points that are demonstrated.

The "proof points" and "ratings for evaluation" are currently in different subsections of each section. it isn't clear, for instance, which proof points relate to which level of maturity. The reader also has to move back and forth between the subsections to understand what evidence establishes which outcomes.

Can this material be reorganized and, perhaps, combined to make it clearer?

Although section 3.4 includes proof points and ratings related to design, implementation, and evaluation, it doesn't relate these to actual outcomes with respect to the accessibility achieved for people with disabilities. For example, it doesn't specify that the organization has identified the standards to which it is going to conform, or the criteria of accessibility (e.g., in terms of effectiveness/efficiency/satisfaction) to be achieved.

Nowhere does it state that the design, implementation and testing procedures are sufficient to achieve the desired standards and outcomes, if they are implemented correctly.

Describing requirements for ICT design, development, testing and maintenance is one of the areas in which the ISO 30071-1:2019 standard is particularly strong - but it isn't the only focus of that standard.

I suggest reworking this section to relate it more clearly to the suitability of the processes and procedures for actually achieving defined outcomes (e.g., meeting technical standards or satisfying criteria of usability/accessibility for users who have disabilities). For example, the design and testing procedures need to enable achievement of the organization's accessibility objectives (defined in terms of standards or other criteria).

As currently written, it seems to me that one could do everything described in this section and not achieve good outcomes or standards conformance.

Nothing in the proposed maturity model is specific to "Web Content Accessibility Guidelines". However, section 2 can be read as limiting its scope:

"This Accessibility Maturity Model is organized around seven “Dimensions.” Each dimension represents an aspect of an organization where maturity in accessibility is needed to improve compliance with Web Content Accessibility Guidelines."

In practice, there are various other technical standards relevant to an organization's approach to ICT accessibility, as well as technologies or applications that are not well addressed by current standards. The standards include ATAG, UAAG and WCAG (from the W3C), PDF-UA from the ISO, EN 301 549 from the European Union, 36 CFR Part 1194 in the United states, and so on.

I think the maturity model should be adjusted to clarify the range of applicable sources of technical requirements, some of which may be established by law - depending on the regulatory context in which the organization operates.

Also, one of the advantages of ISO/IEC 30071-1:2019 is that it goes beyond issues of conformance with technical standards to address how well users with disabilities can achieve their goals in using a technology. Conformance with technical standards is acknowledged as among the means of achieving accessibility, but it isn't the end sought, and organizations are encouraged to advance beyond thinking and acting in terms of compliance with technical requirements.

I think this maturity model should acknowledge that technical standards don't solve everything, and that WCAG certainly doesn't (which is why we have ATAG, UAAG, and others). In short, this should be considered an independent project from and not tied to WCAG, or any version thereof.

The plural "points" should be "point" (single) in:
https://w3c.github.io/wcag/understanding/dragging-movements.html

Change:

For example, two separate single tap or click actions may define the rectangle coordinates or the start and end points of a connecting line.

To:

For example, two separate single tap or click actions may define the rectangle coordinates or the start and end point of a connecting line.

The proof points should include measures of the extent to which the processes pursued by the organization lead to successful outcomes over time with respect to accessibility.

In the ICT development life cycle section, for instance, measures should be taken of the extent to which what is actually produced satisfies the technical accessibility standards that have been adopted by the organization. An additional, relevant measure is concerned with the timeliness and correctness of changes made to resolve accessibility-related defects found during testing or after deployment.

In the procurement section, there should be a proof point that measures whether the technological products actually acquired consistently satisfy the accessibility standards adopted by the organization. Improvement in this metric over time would furnish evidence that the procurement procedures and requirements are effective.

User research is discussed (section 3.4.1.3), but the outcomes of the user research aren't. Does the research indicate high levels of accessibility? Are accessibility-related issues identified via user research actually addressed in a timely and effective manner?

The only reference to the degree to which technical standards are met that I could find is in the "Accessible direct Communication" section (i.e., section 3.1.1.2 of the current draft). There does not appear to be any proof point related to the organization's handling of defects; yet, the degree to which accessibility-related defects are resolved in a timely and effective manner is a central distinction between organizations at different levels of maturity in this domain.

I am concerned that an organization could satisfy many of the proof points without consistently succeeding in its accessibility efforts - and without consistently improving its performance over time. None of the other indicators matters if the organization's success in acquiring and delivering accessible technology-based products and services remains consistently low, yet this is not measured in most of the contexts to which it is relevant. Much of the "evidence" of maturity described in the document is circumstantial, even in those cases in which more direct evidence exists (e.g., improvements in technical standards conformance over time, or the effective and resolution of accessibility-related defects).

The ratings in section 3.7.2 Ratings for Evaluation include things that I would consider proof points, but these proof points are not included in the list.

e.g. cultural programs, communities of practice, governance.

It isn't entirely clear how the maturity model would apply to a decentralized organization in which different components are at different maturity stages along each of the dimensions. For example, a university may have faculties and departments which are relatively autonomous in their technology acquisition and Web content development, as well as a department responsible for the institution's main Web site. The group responsible for the main Web site may, for instance, be at a relatively advanced stage in all dimensions, whereas the faculties and departments could vary considerably along the dimensions.

Until consistent practices are developed and implemented across the entire institution, the results of evaluating the dimensions with respect to each faculty/department are going to yield varying outcomes. It isn't clear from the document how the model should be applied in such cases.

For instance, is the stage achieved by the entire organization along a given dimension determined by the stage reached by the worst-performing organizational component? For the purpose of improving and implementing a strategy, having a much more detailed record of maturity for each of the components of the organization could actually be more useful than an aggregate for the entire organization.

Is one also supposed to aggregate the results across all of the dimensions to assess the maturity of the entire organization? If so, is the maturity stage achieved by the organization the lowest stage reached in any of the dimensions?

As these questions suggest, I think the document could better define what the outputs of the maturity model are (e.g., a list of evaluations - one for each dimension, for the entire organization, as well as an aggregate maturity level - a single maturity "stage" reached). There are cases in which having evaluations for components of the organization would be useful, as suggested above, so if all of these different evaluations - and hence different outputs of the model are valid for different purposes, some discussion of this in the document would help to clarify the role of the model considerably.

Defining what the valid outputs of evaluation are and why each of them would be useful would contribute significantly to clarifying the use of the maturity model.

Change...

A Maturity Model is:

A tool;

That helps people assess the current effectiveness of a person, group, or an organization;
And supports figuring out what capabilities they have and need to acquire next;
In order to improve their performance.

To something like...

[... Next sentence... ]

A Maturity Model is a tool that helps:

Assess the current effectiveness of a person, group, or an organization;
Supports figuring out what capabilities they have and need to acquire next;
Improve performance.

In the Open Questions (1.4) section, there is mention of a third spreadsheet format for Policy, but there is no spreadsheet that exists for Policy since that is not one of the dimensions. Maybe I'm missing something, though.

I'm using the spreadsheet found at https://docs.google.com/spreadsheets/d/1fzPlHjHIaSQn9GvpJd86SDpV49U8Beg_iXj6K5aeyGg/edit#gid=1195503796 .

Flow change needed for:

We have proposed three variations on the spreadsheet formats.
Procurement is uses alternative 1
Knowledge & Skills is alternative 2
Policy is alternative 3

Change to:

We have proposed three variations on the spreadsheet format, the following dimensions all have their own format to be looked at:

1. Procurement
2. Knowledge & Skills
3. Policy

I find the proof points in procurement a bit confusing and it doesn't really say what is really needed and who does it. These bullets are too brief to explain it.

In procurement, it's the vendor that should provide proof of due diligence in meeting accessibility requirements. If a procuring organization wants accessible products, they may be specific about what standard(s) they expect the ICT to meet and what proof they wants to see from the vendor. That should be built into the procurement and/or solicitation language.

For the program management part, this is about establishing and managing the process, making sure the procurement steps and language the organization requires are being done and collecting and monitoring metrics around that. Whether or not they actually have a "dashboard" isn't the important part.

What is the "Accessibility audit" part about? Need to be more specific about what you are auditing for. Are you testing the procured ICT, auditing that procurement personnel are following the process, and/or measuring the number of employee complaints? Not sure how "Issue management" comes into play here. Issues on what and reported by whom?

​Change:

Most maturity models contain some number of levels with increasing levels of maturity.

To:

Most maturity models contain some number of levels with increasing maturity.

Please add a link to https://w3c.github.io/wcag-maturity-model/ in the About section of github

See screenshot below for where to set this (showing ARIA as an example)

According to literature that I have read, some countries have laws which establish quotas of employees with disabilities that employers are expected to fill. In certain other countries, there are no quotas, but there are anti-discrimination laws in place which apply to all stages of the employment process.

Are the activities described in section 3.5 actually allowed under anti-discrimination laws related to employment practices? Are they consistent with different regimes of employment law around the world?

Especially in section 3, there are numerous terms and occasional abbreviations introduced without definition or explanation.

Adding a glossary, or including brief explanations of terms that may not be familiar to members of the intended audience would contribute greatly to the readability of this document.

There are statements in section 3 that read like notes for internal use by its authors rather than as clear communications suitable for a wide audience. I can decode much of the language and abbreviations, but anyone not deeply embedded in accessibility work would probably find this much more difficult.

" Dimensions include:" change to " Dimensions included:"

Also...

1. Delete space before "Dimensions include:"
2. "enter" before "Dimensions" so it goed to next sentence

Change:

Dimensions include:

Communications
Knowledge and Skills
Support
ICT Development Lifecycle
Personnel
Procurement
Culture

To:

Dimensions included:

Communications
Knowledge and Skills
Support
ICT Development Lifecycle
Personnel
Procurement
Culture

It seems to me that the proof points and ratings in section 3.7 are all indicative of the quality of an organization's culture with respect to accessibility, but that they don't address core issues one would reasonably consider important, such as:

To what extent are issues and concerns raised by employees, clients etc., with disabilities actually paid attention to and addressed through concrete actions?

To what degree is the expertise of accessibility specialists or others who have accessibility-related knowledge within the organization not only respected by colleagues, but acted upon in the decisions which are made about the design, development and procurement of ICT systems?

How consistently is a collaborative approach taken in which all parties seek to ensure accessibility-related needs are met?

On the negative side, if there is a culture of ignoring, disrespecting, or not acting on the advice of people with accessibility expertise, then the organization doesn't have an inclusive culture - even if other indicators are present. Likewise if the needs of people with disabilities are frequently sidelined, ignored, or not treated as priorities (whether those people are internal or external to the organization).

In future drafts of this section, I think it would be helpful to give closer attention to the ways in which culture is manifest in concrete behaviours (i.e., reflected in how people make decisions and treat each other in practice) - especially in contexts that affect accessibility-related outcomes.

In the Understanding of 2.5.1: Pointer Gestures there's a link to Dragging SC, the name changed and so should the link in this Understanding doc...

https://w3c.github.io/wcag/understanding/pointer-gestures

From:
Dragging motions are covered in Success Criterion 2.5.7: Dragging.

To:
Dragging motions are covered in Success Criterion 2.5.7: Dragging Movements.

(also change 'motions' to 'movements'... ?)

Change:

Culture: The attitudes, sensitivity, and behaviors around accessibility, including internal interaction, perception, and decision-making.

To:

Culture: Attitudes, sensitivity, and behaviors around accessibility. Including internal interaction, perception, and decision-making.

Read as three separate parts, while everything after "disability-related employee resource groups" applies to all

Change:

Personnel: Job descriptions, recruiting, disability-related employee resource groups necessary to provide lived-experience to accessibility efforts.

To:

Personnel: Lived-experience to accessibility efforts for job descriptions, recruiting, and disability-related employee resource groups.

Change:

The results of a maturity modeling assessment provide a holistic picture of an organization’s accessibility initiatives...where it is doing accessibility well, and where improvements can be made that remove barriers.

To:

The results of a maturity modeling assessment provide a holistic picture of an organization’s accessibility initiatives...where it is doing accessibility well, and where improvements can be made removing and, even better, preventing barriers.

Stage 1 "Inactive" should be changed as there might be awareness and (some) recognition, but still the decisions can be to deliberately be "Inactive" (for all kind of different reasons)

comma used in this way is not correct...

Change:

This is a first draft, and a number of issues are still to be determined that we would like comment from the public on:

To:

In this draft there are a number of issues still to be determined that we would like comment from the public on:

The last bullet of the outcomes for the Launch maturity stage of the Support dimension says "No external support for K&S". Does that mean Knowledge and Skills? Why is that being talked about in the ratings for the Support Dimension?

Change

level(s)of

To

level(s) of

There's an editors' note regarding the need to add in research references. Sheri Byrne-Haber said she'd add this, so opening this issue to track that so she can be assigned an issue.

It is designed to work for many different size organizations from s to large corporations or government agencies.

letter "s" must become "small"

It is designed to work for many different size organizations from small to large corporations or government agencies.

The outcomes for the support dimension optimize stage says:

Employees/Talent Acquisition: Candidates are offered accommodations for their interviews. Disability Employee Resource Group(s) provide social and professional support to employees with disabilities.

This is confusing since all of the proof points in the support dimension are about supporting employees with disabilities and supporting customers with disabilities. and are not related to talent acquisition at all, which are covered instead in the Personnel dimension.

Many of the proof points and outcomes in section 3.3 appear to be covered under Communications and under Knowledge and Skill (sections 3.1 and 3.2).

Is this overlap desirable? If so, perhaps it should be noted somewhere in the document. I suppose one could argue that it is desirable to enable 3.3 to be evaluated completely independenty of having evaluated 3.1 or 3.2.

https://w3c.github.io/wcag/understanding/pointer-gestures

As we have a definition now (which explains dragging movements a bit better than in this Understanding doc) we must 'link' the first instance to the definition.

Sentence starts with:

Dragging is a movement where the user picks up

Links "Dragging is a movement" to the definition? Or re-write the sentence a bit?

Between sentences is a double space, I see a weird thingy in Github code (a red dot)... to be deleted

Maturity Modeling is a big part of a “shift-left” methodology of preventing problems from recurring, not fixing them after they have happened.

​
Most maturity models contain some number of levels with increasing levels of maturity.

Change:

Policy and business process subject matter experts responsible for putting plans, actions and metrics in place.

To:

Policy and business process subject matter experts responsible for putting plans, actions, and metrics in place.

Section 3.6 describes practices which can support the procurement of ICT systems that are broadly accessible to users with disabilities. However, the success of these practices depends on having standards (including, but not limited to, international technical standards) and criteria of adequacy related to accessibility which the acquired systems are expected to meet.

That is, the organization should specify appropriate standards and criteria (including international technical standards from the W3C, ISO, or elsewhere, as applicable to the project) as the basis of its procurement contracts and evaluation of prospective suppliers. Without a well defined list of concrete requirements, it is difficult to judge success or to hold vendors accountable.

The proof points and ratings in this section should more clearly connect procurement-related activities with the organization's chosen accessibility standards and criteria of adequacy. For example, the first point in section 3.6.1.1 states that there should be an accessibility policy, but not what such a policy should specify. Later points should refer back to the standards and criteria specified in the policy.

Compare the approach taken in ISO/IEC 30071-1:2019, which is much clearer about the role of an accessibility policy and in relating the organization's development and procurement activities to the prescriptions of the policy.

Change:

Stages loosely correspond to the following criteria. Each dimension will contain defined criteria that pertain specifically to the dimension being defined. Stages are cumulative, so stage advancement cannot be achieved without first meeting the specific criteria of a lower level. The terms for the stages were adopted for consistency with the Policy Driven Adoption Maturity Model, currently being used by multiple government agencies.

To:

Stages loosely correspond to the following criteria:

• Each dimension will contain defined criteria that pertain specifically to the dimension being defined.
• Stages are cumulative, so stage advancement cannot be achieved without first meeting the specific criteria of a lower level.

The terms for the stages were adopted for consistency with the Policy Driven Adoption Maturity Model, currently being used by multiple government agencies.

The paragraph beginning "Measures:" isn't clear, and I don't understand it.

Section 3.2.1.1: the heading is about employing people with desired knowledge and skills, but the text appears to be about identifying what knowledge and skills are necessary, and the items in the list don't make sense.

Later subsections under "proof points" also have a lack of clarity - especially in the unordered lists, which are just brief phrases that often don't state clearly what is needed. I suggest rewriting them as phrases or, if necessary, full sentences that describe what each point means (appropriately for a reader who doesn't already know). Essentially, review for clarity and revise accordingly.

There are two topics in this section: (1) communication about accessibility, and (2) the accessibility of communications themselves - whether or not they are about accessibility.

I think the two issues should be clarified and better distinguished in a revision of the text.

The headings of subsections under 3.1.1 don't clearly describe the content of the subsections. The proof points are only briefly stated, and it isn't clear what is required to satisfy them (e.g., what a "statement of commitment" or an "accessibility statement" is, and what either of them should contain).

"Voluntary Product Accessibility Template" should include a bibliography reference that cites the official documentation. The same holds for "model accessibility statement".

In section 3.1.2, it isn't clear that the items listed under the "Optimize" level relate to communications - they appear to be more about organizational governance.

The relationship between "proof points" and "ratings for evaluation" could be clarified in general (e.g., the "proof points" describe what evidence to acquire, and the ratings describe what the evidence needs to show to achieve different levels of maturity along the dimension).

There are aspects of the maturity model which assume that the organization has a hierarchical structure of authority along conventional corporate lines. However, some organizations don't fit this paradigm.

For example, an open-source software project may have leaders who are elected at regular intervals by the members, and who have limited decision-making powers. Decision-making authority may reside in elected committees or ultimately in votes taken by the project's members as a whole. The Debian Project serves as one example that comes to mind. Open-source projects exist to create ICT products, so it would be valuable for a maturity model to be applicable to them. However, open-source project governance need not involve a corporate entity or a hierarchically organized social structure.

Can the maturity model be written in such a way that it accounts for, but doesn't presuppose, a hierarchical organizational structure? It seems to me this wouldn't require large changes, but that the aspects of the document which deal with the authority structure (internal power relations and management) of the organization should be reviewed and revised to ensure they can be applied to democratically governed or other non-hierarchical structures.

Depending on the method chosen (spreadsheet), the stages in NOT by definition based on the proof points...

Change:

Each dimension will contain stages” based on the proof points that are demonstrated.

To:

Each dimension will contain stages” based on the proof points that are demonstrated.

(btw, we've chosen the stages 'based' on other factors than proof points at all)

We have n o figure 1

Figure 1 shows the core structure of the Maturity Model.

K&S definition is missing 'insight of knowledge'

Change:

Knowledge and Skills: Education and outsourcing practices to fill gaps for accessibility operations.

To:

Knowledge and Skills: Insight of working experience, education, and outsourcing practices to fill gaps for accessibility operations.

Maybe I missed something or it is not updated yet but 2.4.13 Focus Appearance (Enhanced) still got "Not obscured" as last bullet

https://w3c.github.io/wcag/guidelines/22/#focus-appearance-enhanced

There is considerable overlap between this document and the practices recommended in ISO/IEC 30071-1:2019, in that both are concerned with an organization's policies and processes. However, the ISO standard is more detailed and specific in important respects. For example, it specifies what should be done at each stage of the ICT system development and maintenance process, and what matters should be addressed in an organization's accessibility policy (then reflected in its policies and procedures with respect to ICT).

The ISO standard also extends beyond conformance with technical accessibility guidelines such as WCAG to address issues of "efficiency", "effectiveness" and "satisfaction" of users with disabilities in interacting with ICT-based systems.

It isn't clear from the maturity model document how it is meant to relate to the ISO work. While it doesn't treat all of the same issues as ISO/IEC 30071-1:2019, the overlap of concerns is sufficient to raise questions about consistency between the two, whether organizations should use both, and how maturity model analysis relates to an organization's decisions about conforming to the ISO standard.

For consistency, all maturity stages should have a definition and outcomes for every dimension. This is true except for two cases:

• 3.3.2 Ratings for Evaluation - Support: Inactive stage has no outcomes listed.
• 3.6.2 Ratings for Evaluation - Procurement: Inactive state text that is there should be identified as a definition. It is also missing outcomes.

This was a suggestion that Jeff Kline proposed in the Google doc that has not yet been carried out in the maturity model document. I don't know if the group had reached consensus on adding this or where to add it, as the discussion in Google had different ideas on placement of the link: at the end of each ratings for evaluation section or at the beginning of that section were both mentioned. There's also the possibility of adding it at the beginning of the section that breaks down the dimensions.

Change:

The Maturity Model has three levels of content with associated documentation.

to:

The Maturity Model has four levels of content with associated documentation.

Inactive
Launch
Integrate
Optimize

= 4

See 1.3:

1.3 Existing Research
Reference appendix with summary of the existing six maturity models outside of WCAG.

Now...

1. There is no appendix in the document
2. There are more than 6 'in the wild', it reads like "THE 6 Models", which should not be mentioned that way.

Delete 1.3 OR mention "Some models researched"

There's some aspect of this in all of the ratings sections, but it became super-obvious to me in 3.4.2 Ratings for Evaluation for the ICT development lifecycle that the outcomes:

• The language does't seem to align in the type of language used in Silver for outcomes. (incompletely stated as well) For example:

  Launch: "May be limited in scope to new products, apps, websites."

  • This might be better stated as, "Accessibility efforts in the development process limited to new products, apps, and websites, or starting to assess/address a small number of existing high priority ICT.

• The outcomes don't fully cover the situations that would be occur at that level of maturity. For example:

  Inactive: "If ACRs/VPATs are required, they are not being produced."

  • IMO, ACRs aren't the only poor outcome if inactive. I would think that none of the organization-developed ICT would be accessible would be an outcome at this stage. They may also be receiving customer complaints or requests for accessible products that they are unable to address.

Level "Launch" mentions:

Launch | Recognized need organization-wide. Planning initiated, but activities not well organized.

Often ideas get launched while there is not a 'recognized need :"organization-wide"', and still counts as a good starting point to get accessibility on the agenda putting you at level 1.

Also the AMM can be used for parts of the organization, making this even more important NOT to demand organization-wide

Should be changed to:

Launch | Recognized need in the organization. Planning initiated, but activities not well organized.

The proof points for ICT development lifecycle section needs a sub-section on planning. Some things that come to mind:

• Planning process with templates, checklists, and output that ensures accessibility is incorporated
• Requirements and goals for accessibility of work products are documented and communicated to the team.