acook / uspec Goto Github PK

Currently if you do a puts or similar inside a spec block it will dump the output wherever in the middle of the test output. Instead, capture the output and display it indented after the test results.

• Indented string display (completed in a6d72c0)
• Hijack $stdout similar to https://github.com/acook/uspec/blob/main/uspec/uspec_helper.rb#L7

Currently if you put break or return in the body of a spec block then it will immediately end the test, creating an unpleasant visual artifact in the test output, and without showing that the test was skipped or counting it in the summary.

While it is easy to avoid this, mistakes could be hidden in larger test suites. Additionally being able to do an intentional early return would be useful, if it were possible to capture the early return value.

I've been using uspec for years and it never occurred to me to try until today when I was doing a step-by-step integration test that seemed to call for multiple assertions. By design, uspec is philosophically opposed to multiple assertions for unit tests. However, for integration tests, it makes more sense. Either way, it should still show the proper test count and at the very least show a failed or pending test.

Example code:

spec "does things" do
  break false
  true
end

Output:

integration_spec:
 -- does thingstest summary: 0 successful, 0 failed, 0 pending 

Note that the return values (true and false in this example) are immaterial and have no behavioral effect on this issue.

Once the formatters are decoupled from the core, add a TAP formatter.

https://testanything.org

Similar to my blacklight test runner scripts.

Uspec catches all Exceptions at certain points in order to provide useful error messages, however this may also prevent responding to SIGINT or other SIGNALs during the execution of those blocks which could be annoying.

Tests won't run because semantics of require_relative changed in Ruby 2.x.
https://travis-ci.org/acook/uspec/jobs/545994949

There are certainly other issues as well. I am unlikely to fix them but I'm open to someone else taking a stab at it.

There are currently a few work-arounds I've had to implement to make at_exit behave right:

• f024c36
• c1e215d#diff-63eceac4e3e881697cb521eb9cf57b99L17

Having it in the library is also a problem since we can't easily test without overriding the exit code of another test framework or easily switch between testing and development modes.

A better solution would be, if we really needed at_exit that it would be in the runner only. But most likely it is not necessary at all.

I came across this article talking about it, though it provides examples of use cases, it doesn't explain alternatives.

Depending on the patch level, the tests fail in different ways, so the failures on CI are different than on my development VM.

https://travis-ci.org/acook/uspec/jobs/546236040#L554

It has something to do with BasicObject, ancestors, or binding methods.

This same issue probably breaks 1.9.x as well.

I don't consider fixing pre Ruby 2.1.x issues high priority but I'm open to PRs that don't slow down newer Rubies or add a lot of code complexity.

The proxy object would catch certain methods passed to it, like a to or should method as a basis of comparison, but could also pass normal methods through to the object inside, mutate it and store the result, returning self - the proxy.

Example:

spec Cheese do
  subject :swiss do
    spec_object.new type: 'swiss'
  end

  spec 'type is swiss' do
    swiss.type.should == 'swiss'
  end
end

It is supposed to be completely transparent. The swiss variable if you accessed it with pry, would reveal itself to be a Proxy object.

Alternatively, we avoid the full proxy and just wrap it, allowing only explicit method passing through a gateway for better encapsulation.

Example:

spec Cheese do
  subject :swiss do
    spec_object.new type: 'swiss'
  end

  spec 'type is swiss' do
    swiss.has :type, eq('swiss')
    # or as a method(!?)
    swiss :type, eq('swiss')
    # or returning another proxy and abusing ruby syntax even further for more DSLing
    swiss :type eq('swiss')
  end
end

Testing Impasta I noticed that CI is passing but uspec is showing errors:
https://circleci.com/gh/acook/impasta/8

It looks like since there are no error results uspec is exiting with a 0 status. We need to catch all errors, or at least not exit with a 0 when Ruby falls over.

Related to 6c61652