elmassimo / mongoid_includes Goto Github PK

View Code? Open in Web Editor NEW

46.0 6.0 17.0 145 KB

🌿 Improves eager loading support for Mongoid

Home Page: https://maximomussini.com/posts/mongoid-n+1/

License: MIT License

Ruby 100.00%

eager-loading mongoid performance queries ruby rails

mongoid_includes's People

Stargazers

Watchers

Forkers

feedbackfruits stefangomez anilmaurya bhernez dileeppdkr madale xenodata-lab sagarkt unused asgerb cotocisternas tedma4 playlist neongrau vitmf rightspend danarnold

mongoid_includes's Issues

Bullet does not detect N+1 queries when using mongoid_includes

Issue 🚨

Hello @ElMassimo

I was trying to test your Gem, but looks like the preload is not working. ⚠️

The issue is that: The Bullet gem is still accusing N+1 Query Problem even after applying the .includes inside the query.
(So looks like the N+1 Query Problem is still happening after using .includes.)

The example is pretty straightforward:

records →(has_many) attachments →(belongs_to) project

Scenario 🌍

class Record < ApplicationRecord
  include Mongoid::Document
  include Mongoid::Timestamps

  has_many :attachments
end

class Attachment < ApplicationRecord
  include Mongoid::Document
  include Mongoid::Timestamps

  belongs_to :project
  belongs_to :record, optional: true
end

class Project < ApplicationRecord
  include Mongoid::Document
  include Mongoid::Timestamps

  has_many :records
  has_many :attachments

  field :id, type: String
end

Steps to Reproduce 🚶 ...

To this topic, I'll present several queries and their output on the bullet.log, comparing the output before mongoid_includes and after mongoid_includes.

Just enter on rails console and execute the following commands.

1. With N+1 Query Problem

Bullet.profile do
  query = Record.where(project_id: '621ea9b069ca9c07aec7e36d')
  query.map do |r| 
    r.attachments.map do |a|
      a.project.id
    end
  end
end

Before mongoid_includes: USE eager loading detected → Record => [:attachments], Attachment => [:project]
After mongoid_includes: USE eager loading detected → Record => [:attachments], Attachment => [:project]

2. Attempt to fix N+1 for Attachments

Bullet.profile do
  query = Record.where(project_id: '621ea9b069ca9c07aec7e36d')
  query.includes(:attachments).map do |r| 
    r.attachments.map do |a|
      a.project.id
    end
  end
end

Before mongoid_includes: USE eager loading detected → Attachment => [:project]
After mongoid_includes: USE eager loading detected → Record => [:attachments], Attachment => [:project] 😱

3. Attempt to fix N+1 for Attachments and Projects (nested includes)

Bullet.profile do
  query = Record.where(project_id: '621ea9b069ca9c07aec7e36d')
  query.includes(:attachments).includes(:project, from: :attachments).map do |r| 
    r.attachments.includes(:project).map do |a|
      a.project.id
    end
  end
end

Before mongoid_includes: _Error: NameError: undefined local variable or method `klass' for #Mongoid::Criteria:0x00007fe2f6d787b0
After mongoid_includes: USE eager loading detected → Record => [:attachments], Attachment => [:project] 😱😱

Conclusion 💬

The mongoid_includes gem is not working even for the most basic .includes, as showing in 1. and 2. 🚨
It is also not working for nested includes, as shown in 3. 🚨
The builtin .includes from mongoid is working for 1. and 2., but after we add the mongoid_inclues gem to our Gemfile and install it, the builtin .includes method stops to work 😱 🚨

Versions 📍

Ruby: ruby 2.6.8p205
Rails: Rails 6.0.5.1
Mongoid: 7.0.13
Mongoid Includes: 3.0.0

Mongoid version restriction too high

mongoid_includes v2.0.0 fixed eager loading issues that arose from changes in mongoid 5 and 6. However, mongoid_includes now requires mongoid 6, which stops users on mongoid 5 (and hence rails 4) from using this gem, even though it should still work with mongoid 5, given the main difference between 5 and 6 is their rails support.

mongoid_includes should still be able to support mongoid >= 5.2.0.

Support for mongoid 9.0

Hi, are there any plans to support mongoid 9.0? Looking at the changelog, there don't appear to be any breaking changes, so hopefully it's just a matter of updating the specifications and tests to support 9.0 like the update for 8.x.

Mongoid 9.0 release notes on MongoDB's site
Mongoid 9.0 release notes on GitHub

Error with mongoid 6.0.3

Hi,
Mongoid 6.0.3 raise an error with mongoid_includes:

There was an exception - NoMethodError(undefined method `first' for false:FalseClass)
/opt/rvm/gems/ruby-2.3.3/gems/mongoid-6.0.3/lib/mongoid/contextual/mongo.rb:271:in `find_first'
/opt/rvm/gems/ruby-2.3.3/gems/mongoid-6.0.3/lib/mongoid/contextual.rb:20:in `find_first'
/opt/rvm/gems/ruby-2.3.3/gems/mongoid-6.0.3/lib/mongoid/findable.rb:114:in `find_by'
/Projects/sofred2016/app/channels/application_cable/connection.rb:12:in `find_verified_user'
/Projects/sofred2016/app/channels/application_cable/connection.rb:6:in `connect'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/connection/base.rb:167:in `handle_open'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/server/worker.rb:58:in `block in invoke'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/server/worker.rb:39:in `block in work'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:126:in `call'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:506:in `block (2 levels) in compile'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:455:in `call'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:448:in `block (2 levels) in around'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:286:in `block (2 levels) in halting'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/engine.rb:60:in `block (4 levels) in <class:Engine>'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/execution_wrapper.rb:85:in `wrap'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/engine.rb:55:in `block (3 levels) in <class:Engine>'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:391:in `instance_exec'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:391:in `block in make_lambda'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:285:in `block in halting'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:447:in `block in around'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:455:in `call'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:101:in `__run_callbacks__'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:750:in `_run_work_callbacks'
/opt/rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:90:in `run_callbacks'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/server/worker.rb:38:in `work'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/server/worker.rb:56:in `invoke'
/opt/rvm/gems/ruby-2.3.3/gems/actioncable-5.0.1/lib/action_cable/server/worker.rb:51:in `block in async_invoke'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/rvm/gems/ruby-2.3.3/gems/concurrent-ruby-1.0.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'

Some compatibilities issues with bullet

The gem doesn't seem to work well with bullet, after declaring:

Trend.[...].includes(:page, :product_collection).includes(:products, from: :product_collection)

I get this from bullet:

N+1 Query detected
  Trend => [:page]
  Add to your finder: :includes => [:page]

N+1 Query detected
  Trend => [:product_collection]
  Add to your finder: :includes => [:product_collection]

N+1 Query detected
  ProductCollection => [:products]
  Add to your finder: :includes => [:products]

Mongoid 7 support

Hey @ElMassimo, thanks for this nice gem, works better than native includes for me :)
But since Mongoid 7 is out, it would be great to be able to use this gem there as well.

Support for mongoid 8.0

Are there any plans to support mongoid 8.0?

Scope with limit not working

Hello there!
I tried using the gem to eager load some users with their posts using a limit scope but it didn't work
Something like this :

User.includes(:posts, with: ->(posts) { posts.where(approved: true).limit(10) })

The snippet above returns only the 10 posts for the first user and empty results for the rest of the users

Is there a fix for this or I'm doing something wrong ?

Transfer mongoid_includes to github/mongoid? #49

Hi, related simi/mongoid_paranoia#49
@ElMassimo what do you thing?

Undefined method 'polymorphic_belongs_to?'

Thanks for the gem! It's a big help. I've been running into an issue while trying to use it:

After using the gem in a manner exactly like your Albums example in the readme (i.e. same relations), I receive this error: undefined method 'polymorphic_belongs_to?' for #<Array:0x00000005ce5798>

This is the only relevant part of the backtrace which pertains to either the gem or mongoid:

mongoid_includes (1.0.1) lib/mongoid/includes/criteria.rb:41:in `includes'

This is being used within an engine. I'm using Mongoid version 4.0.2.

Getting NoMethodError after adding 'mongoid_includes' to Gemfile

Hello,

I'm getting NoMethodError: undefined method 'first' for false:FalseClass every time I query for any document. Once I remove this gem from the Gemfile everything works.

The whole stack:

NoMethodError: undefined method 'first' for false:FalseClass
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/mongoid-6.0.2/lib/mongoid/contextual/mongo.rb:271:in 'find_first'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/mongoid-6.0.2/lib/mongoid/contextual.rb:20:in 'find_first'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/mongoid-6.0.2/lib/mongoid/findable.rb:114:in 'find_by'
  from (irb):3
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/railties-5.0.0.1/lib/rails/commands/console.rb:65:in 'start'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/railties-5.0.0.1/lib/rails/commands/console_helper.rb:9:in 'start'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/railties-5.0.0.1/lib/rails/commands/commands_tasks.rb:78:in 'console'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/railties-5.0.0.1/lib/rails/commands/commands_tasks.rb:49:in 'run_command!'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/railties-5.0.0.1/lib/rails/commands.rb:18:in '<top (required)>'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:293:in 'require'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:293:in 'block in require'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:259:in 'load_dependency'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:293:in 'require'
  from /Users/mityakoval/RubymineProjects/vaktdagboka/bin/rails:9:in '<top (required)>'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:287:in 'load'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:287:in 'block in load'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:259:in 'load_dependency'
  from /Users/mityakoval/.rvm/gems/ruby-2.3.0@vaktboka/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:287:in 'load'
  from /Users/mityakoval/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in 'require'
  from /Users/mityakoval/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in 'require'

Can't make it work

Hey man, your gem looks amazing but for some reasons (missing coffee maybe?) I can't make it work with the following code:

Projections::Tasks::SkillModeration.limit(1).includes(:user, from: :skill_application).entries
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | STARTED | {"find"=>"projections_tasks_bases", "filter"=>{"_type"=>"Projections::Tasks::SkillModeration"}, "limit"=>1}
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | SUCCEEDED | 0.001094886s
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | STARTED | {"find"=>"skill_applications", "filter"=>{"_id"=>{"$in"=>[BSON::ObjectId('542ef41ca3653cd949a52e09')]}}}
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | SUCCEEDED | 0.000665095s
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | STARTED | {"find"=>"users", "filter"=>{"_id"=>{"$in"=>[BSON::ObjectId('54278072ab40e20002002d31')]}}}
[2016-06-30T16:54:39] DEBUG Rails : MONGODB | mongo:27017 | tm_develop.find | SUCCEEDED | 0.0015331420000000001s

where models are something like this:

class User
end

class SkillApplication
  belongs_to :user
end

class Projections::Tasks::SkillModeration
  belongs_to :skill_application
end

I only have added the gem in the gemfile and ran the bundle command but nothing, I am missing something obvious?