thoughtbot / capybara_accessibility_audit Goto Github PK

Accessibility tooling for Capybara

License: MIT License

Ruby 84.72% CSS 0.06% HTML 15.22%

capybara_accessibility_audit's Issues

Confirmation of alert box is not working

Problem:
When using the accept_confirm or accept_alert methods along with the capybara_accessibility_audit gem, a Selenium::WebDriver::Error::UnexpectedAlertOpenError error is raised during the test execution.

Expected Behavior:
The test should run without any errors when using the accept_confirm or accept_alert methods, even with the capybara_accessibility_audit gem enabled.

Actual Behavior:
The test is raising a Selenium::WebDriver::Error::UnexpectedAlertOpenError when using the accept_confirm or accept_alert methods with the capybara_accessibility_audit gem.

Steps to Reproduce:
I've created a GitHub repository that demonstrates the issue. The repository contains a simple system spec (system/root_spec.rb) utilising the accept_confirm method:

require "rails_helper"

describe "Root" do
  around do |example|
    with_accessibility_audit_options skipping: %w[
    html-has-lang
    landmark-one-main
    page-has-heading-one
    region
    ] do
      example.run
    end
  end

  it "raises an error", js: true do
    visit root_path

    accept_confirm do
      click_on "Alert"
    end

    expect(page).to have_content("Root Show")
  end
end

Run in reporting mode vs failure mode

Thanks for writing this - we look forward to using it widely to help us with the new Title II regulations.

One thing that would help us move forward is to have the ability to add the gem to all of our projects, but instead of having every system test fail when it encountered an accessibility issue to instead save the failure and instead give us a report of all the accessibility issues. We could then use that for reporting to our management and tracking our progress towards closing accessibility gaps.

Ideally we'd love to have something where we set a configuration setting, like config.capybara_accessibility_audit.reporting_only that, when run would give us output something like


Finished in 5 minutes 5 seconds (files took 2.45 seconds to load)
238 examples, 0 failures, 24 failed accessibility tests

Failed accessibility tests examples:
... detailed breakdown of the accessibility tests that failed...

Add min dependency of rails 7 to gemspec

Gemfile specifies this is rails 7, and rails 6.1+ nicities are used.

     NoMethodError:
       undefined method `compact_blank!' for {}:Hash
       Did you mean?  compact!

Worth adding this to the .gemspec

wrong number of arguments (given 2, expected 1) (v0.1.1)

Backtrace:

  1) Admin - Users #show with default calendar view shows user details
     Failure/Error: visit(path)
     
     ArgumentError:
       wrong number of arguments (given 2, expected 1)
     
     
     [Screenshot]: /home/clintel/Clintel/careright/tmp/screenshots/failures_r_spec_example_groups_admin_users_show_with_default_calendar_view_shows_user_details_12.png

     
     
     # /home/clintel/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/capybara-3.36.0/lib/capybara/session.rb:258:in `visit'
     # /home/clintel/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/capybara-3.36.0/lib/capybara/dsl.rb:58:in `block (2 levels) in <module:DSL>'
     # /home/clintel/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/capybara_accessibility_audit-0.1.1/lib/capybara_accessibility_audit/audit_system_test_extensions.rb:39:in `block (2 levels) in accessibility_audit_after'

For interest

def login_as(user, path = "/")
    reset_session!
    visit(path)
    expect(page).to have_current_path("/users/sign_in")
end

is what is invoking it.

Not really clear why two arguments are passed to visit, it's only got 1x param (https://github.com/teamcapybara/capybara/blob/master/lib/capybara/session.rb#L260 + our older version)

https://github.com/thoughtbot/capybara_accessibility_audit/blob/main/lib/capybara_accessibility_audit/audit_system_test_extensions.rb#L39

From: /home/clintel/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/capybara_accessibility_audit-0.1.1/lib/capybara_accessibility_audit/audit_system_test_extensions.rb:40 RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView#visit:

    38: define_method method do |*arguments, **options, &block|
    39:   require "pry"
 => 40:   binding.pry
    41:   super(*arguments, **options, &block).tap { Auditor.new(self).audit!(method) }
    42: end

[1] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> arguments
=> ["/admin/users"]
[2] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> method
=> :visit
[3] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> options
=> {}
[4] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> block
=> nil
[5] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> *arguments
SyntaxError: unexpected '\n', expecting '='
[5] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> puts *arguments.inspect
["/admin/users"]
=> nil

[2] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> method(:visit)
=> #<Method: #<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView "shows user details" (./spec/system/admin/users_spec.rb:12)>.visit>
[3] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> method(:visit).super_method
=> #<Method: Capybara::DSL#visit>
[4] pry(#<RSpec::ExampleGroups::AdminUsers::Show::WithDefaultCalendarView>)> puts method(:visit).super_method.source
        define_method method do |*args, &block|
          page.send method, *args, &block
        end
=> nil

If its just something funky with older ruby, maybe adding in some min versions might be worht doing.

Audit failures should not stop the test from running

I've observed that if a test fails because a page is not accessible, the remainder of the test block does not run. This results in issues being reported for a single page, but not for any other pages loaded in a test block.

Expected behavior:
The test should run to the end (barring other non-accessibility failures), even if a few of the pages visited fail the accessibility audit. Audit errors should be reported across all pages visited.

Actual behavior:
The test halts on the first accessibility error that it finds

Example:

RSpec.describe "Playbook", type: :system do
  around do |example|
    with_accessibility_audit_options do
      example.run
    end
  end

  it "renders all pages without error" do
    visit "/playbook"

    playbook_links = all(".application-main-content a[href^='/playbook']").map do |link|
      link[:href]
    end
    playbook_links.each do |url|
     # the first page visited that has an accessibility error will end the test
      visit url
    end
  end
end

I do realize that this issue/suggestion goes against the default behavior of Capybara - Capybara normally does not run the entire test if an assertion fails, or if it fails to visit a page. But it would be a great enhancement.

thoughtbot / capybara_accessibility_audit Goto Github PK

capybara_accessibility_audit's Issues

Confirmation of alert box is not working

Run in reporting mode vs failure mode

Add min dependency of rails 7 to gemspec

wrong number of arguments (given 2, expected 1) (v0.1.1)

Audit failures should not stop the test from running

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent