realgeeks / lab_tech Goto Github PK

Current gemspec doesn't allow Rails 7.0 https://github.com/RealGeeks/lab_tech/blob/main/lab_tech.gemspec#L20

It blocks our 6.1 app from upgrading to Rails 7.0.

• move the "installation" section to the top LIKE A NORMAL GEM ;P
• generate a TOC as with check_please

Prior to using LabTech, our Dyno Load on Heroku was a gentle 0.8~1.5 for our 1m load average.

Once installing LabTech and running an experiment which accumulated 2.5M observations over the course of a week, our 1m avg dyno load increased to 6~8!

On the one hand, this may mean that we have reached ROFLSCALE and would be better equipped by the scientist package directly.

On the other hand, it seems possible that pulling the complex computations onto the application's Background Job queue could be sufficient to raise the ceiling at which folks need to expand their infrastructure footprint.

It's unlikely I'll take the time to identify and make the adjustment, but maybe @bhaibel will take a stab if we decide to stay on LabTech.

I just added LabTech to my Rails project and getting this error on migration.

Caused by this line:

Changing it to t.text "value", limit: 1.gigabyte - 1.byte will help in my case.

I am working on macOS with M1 chip and Docker:
ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]
Rails 6.1.6
lab_tech 0.1.9
scientist 1.6.3

Maybe related to my postgis 14.3.2 on Alpine?

I have written an experiment to compare the output of 2 rails serializers (lightly edited from the production code, but the logic is identical):

def index
    LabTech.science 'PagesController#index' do |experiment|
      experiment.use do
        @pages = Page.all
        @json = ActiveModelSerializers::SerializableResource.new(@pages, each_serializer: PageSerializer, root: 'pages', include: '**').as_json
      end
      experiment.try do
        @json = { pages: QueryService.all_pages }.as_json
      end

      experiment.context params: page_params.to_h
      experiment.compare do |control, candidate|
        control_8601 = control[:pages].map { |hash| hash.deep_transform_values! { |value| value.is_a?(ActiveSupport::TimeWithZone) ? value.iso8601(3).to_s : value } }
        # binding.pry
        control_8601.to_json == candidate['pages'].to_json
      end
      experiment.clean { |value| value.with_indifferent_access[:pages] }
    end
    render json: @json
  end

When I uncomment the binding.pry and check control_8601.to_json == candidate['pages'].to_json, I get true. But when I get the summarized_results, they are always marked as mismatched (this is in my dev environment):

> LabTech.summarize_results 'PagesController#index'

--------------------------------------------------------------------------------
Experiment: PagesController#index
--------------------------------------------------------------------------------
Earliest results: 2023-07-21T22:03:56Z
Latest result:    2023-08-09T05:19:14Z (18 days)

24 of 24 (100.00%) mismatched
--------------------------------------------------------------------------------

When I check LabTech.compare_mismatches 'PagesController#index', there is no difference between the control and candidate.

I'm running the comparison with the deep_transform_values because in my control, the created_at value is coming back as an ActiveSupport::TimeWithZone and in the candidate it is returned as a string - that's the only difference between the two that I can find, however they seem to be identical after the transform. I has noticed this before doing the transform, compare_mismatches would still come back without any difference, so my theory is that once serialized to Postgres and deserialized later, the dates would be normalized.

My Gemfile.lock has the latest versions of the gems:

lab_tech (0.1.9)
scientist (1.6.4)

What am I doing wrong here? Or is there a bug? Could this be related to #41?