joinhandshake / knockoff Goto Github PK

Add instrumentation

sidekiq, example

For easier debugging and informational purposes.

Hi everyone,

We recently started using Knockoff to offload specific read-only workloads to a read replica. We're running into occasional exceptions when using Knockoff to connect to a read replica within a Sidekiq job:

ActiveRecord::ConnectionNotEstablished: No connection pool with 'Knockoff::KnockoffReplica0' found.
  from active_record/connection_adapters/abstract/connection_pool.rb:930:in `retrieve_connection'
  from active_record/connection_handling.rb:116:in `retrieve_connection'
  from active_record/connection_handling.rb:88:in `connection'
  from knockoff/active_record/base.rb:11:in `connection'
  from active_record/associations/association.rb:97:in `association_scope'
  from active_record/associations/association.rb:86:in `scope'
  from active_record/associations/collection_association.rb:285:in `scope'
  from active_record/associations/collection_proxy.rb:961:in `scope'
  from active_record/associations/collection_proxy.rb:1133:in `scoping'
  from active_record/relation/delegation.rb:90:in `method_missing'

For the job that recently raised this error, we're at a Sidekiq concurrency of 3 and have specified a pool of 30 for the read replica's database url.

Reading similar reports of this error, the reports seem to point to thread safety issues in the database connection adapter.

Is there anything that we need to do before using the Knockoff.on_replica { ... } method in Sidekiq? I saw the forking note in the README - do we need to call Knockoff.disconnect_all! somewhere in the Sidekiq boot process?

Any advice would be appreciated. I can submit a PR to update relevant docs if we get to the bottom of this

Thanks!

Is it possible to disable a model from never using replicas without going through Knockoff.on_primary on each query, for example User.find()?

Right now I'm wrapping an action with around_action to use replication only on GET-requests:

  def force_master_database_on_post(&block)
    if request.method_symbol != :get
      DatabaseReplica.with_primary_replication(&block)
    else
      yield
    end
  end

But now looking into that I don't want replication on key models like the User model, or anything related to payments as examples.

Right now it's purely URI based. There are a few issues with that 1) very likely there is an interest to inherit from the primary database.yml config 2) url params is not a very clean way for editing params 3) version controlling some configuration is valuable.

• config means prepared_statements, db_pool, reaping_frequency, etc.

Some assumptions:

• each replica will have same config (aside from the db url itself)

Maybe an assumption?

• replicas have same config as primary

• basics
• pgbouncer
• heroku
• implementing "sticky" to master functionality
• connect/disconnect before and after forking
• changing the config real-time

After getting frustrated with octopus, octoshark, db-charmer, makara finally I found you gem. But this gem too have the same issue. I configured everything as per your suggestion. I have 2 slaves

For example the below query, selects slave properly,

Knockoff.on_replica { User.find(1) }

But the below query is always hitting to the master,

Knockoff.on_replica { User.where(id: 1) }

I just need few search queries to hit to our 2 slave database instead of master. We do not need any other features or customisation. So I though your gem would be perfect for that. But still struggling..

I am on Rails 5.