some examples in Readme are failing about rspec-benchmark HOT 1 CLOSED

Hi Jan,

Thanks for trying out rspec-benchmark. 👍

Given the options, I believe this is

a configuration issue on my side

I can see that you're testing inside the Rails app from the error messages. Please note that all the examples in the readme are tested using Ruby without any frameworks. So I'd encourage you to set up the following file outside of the Rails context for your future explorations:

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem "rspec-benchmark"
end

RSpec.describe do
  include RSpec::Benchmark::Matchers
  ...
end

The first test works as expected.

it "how many objects are left" do
  expect { ["foo", "bar", "baz"].sort[1] }.to perform_allocation(3).and_retain(3)
end

So why is your expectation failing? My guess is that Rails autoloads and potentially monkey patches some behaviour resulting in more objects being created. For example, running Object.new.methods.size in Ruby 2.7 irb session returns 58. The same code run in Rails 7 bare-bones app console returns 87.

The second test fails.

it "perform_constant" do
  sizes = bench_range(8, 100_000) # => [8, 64, 512, 4096, 32768, 100000]
  number_arrays = sizes.map { |n| Array.new(n) { rand(n) } }
  expect { |n, i| number_arrays[i].max }.to perform_linear.in_range(8, 100_000).ratio(2)
end

However, this is a configuration issue. The expectation range generates more steps than the number_arrays has elements, hence the error on nil. When you generate number_arrays you need to ensure that it matches the range you run the performance expectation with. Specifically, the bench_range(8, 100_000) generates [8, 64, 512, 4096, 32768, 100000] which uses ratio of 8 by default. However, the in_range(8, 100_00).ratio(2) generates [8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 100000] which uses ratio of 2. So to fix the test, one way is to update bench_range to use ratio: 2 as well and use generated sizes to specify the expectation range.

The following test passes:

it "perform_constant" do
  sizes = bench_range(8, 100_000, ratio: 2)
  number_arrays = sizes.map { |n| Array.new(n) { rand(n) } }
  expect { |n, i| number_arrays[i].max }.to perform_linear.in_range(sizes.first, sizes.last).ratio(2)
end

The benchark-trend has more explanation as to how the API works.

from rspec-benchmark.