perform_under is run several times about rspec-benchmark HOT 1 CLOSED

Hi 👋

Thanks for using rspec-benchmark and submitting this issue report.

This is expected behaviour and by design. Any performance comparison triggers a warmup run of a given expectation to get Ruby going. This helps establish a baseline before the actual performance test runs. In the case of perform_under a single iteration is run. Then the actual test measurements are taken. You can influence this with the warmup matcher as per the documentation. So in your case, you could do:

expect { ... }.to perform_under(5).ms.warmup(0)

You could wrap the above with a RSpec matcher like perform_under_without_warmup.

As a side note, performance tests are geared towards checking computation that can be run many times without having any side effects. This way you're testing an algorithm and its characteristics. Any setup to make sure that the test has expected data needs to happen before the assertion. Creating a performance test for saving a record to a database has many issues. One of them is what is actually being tested. The web framework performance of saving a record? Database responsiveness? My suggestion would be to focus on testing an algorithm that represents a business logic rather than external services whose performance is not so much under your control.

I hope this helps. I'm going to close this issue as this is not a bug. Please submit a PR if you think the documentation should provide a better explanation of this.

from rspec-benchmark.