Comments (13)
Weird, that's pretty close to what I'm running on, so it must be some other environmental factor.
If you're using the require "bootsnap/setup"
form, could you temporarily replace it with:
require 'bootsnap'
Bootsnap.setup(
cache_dir: 'tmp/cache',
development_mode: true,
disable_trace: false,
load_path_cache: false,
autoload_paths_cache: false,
compile_cache_iseq: false,
compile_cache_yaml: false
)
This should disable all features, and should make your boot time consistently similar to the time without bootsnap.
Then try toggling on the load_path_cache
and autoload_paths_cache
features to see if they make boot faster or slower.
Then do the same for the compile_cache_iseq
and compile_cache_yaml
features.
Once you've identified which pair of features is making it slower, see if you can isolate it to an individual one of them, or if it's both of them.
from bootsnap.
The output from ruby -v
and uname
:
~/src/cookpad[master] % ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
~/src/cookpad[master] % uname -msr
Darwin 16.5.0 x86_64
from bootsnap.
ok, reporting some results:
with all features disabled:
bin/rails runner "puts Rails.env" 10.17s user 3.92s system 51% cpu 27.452 total
load_path_cache: true, autoload_paths_cache: false
bin/rails runner "puts Rails.env" 7.78s user 0.98s system 51% cpu 17.012 total
load_path_cache: true, autoload_paths_cache: true
bin/rails runner "puts Rails.env" 8.09s user 0.96s system 51% cpu 17.679 total
compile_cache_iseq: true, compile_cache_yaml: false
bin/rails runner "puts Rails.env" 9.56s user 5.13s system 23% cpu 1:01.33 total
compile_cache_iseq: false, compile_cache_yaml: true
bin/rails runner "puts Rails.env" 9.66s user 3.97s system 49% cpu 27.367 total
Now, the best combination that I've found is enabling everything except compile_cache_iseq
bin/rails runner "puts Rails.env" 7.52s user 1.04s system 48% cpu 17.789 total
from bootsnap.
This confirm that RubyVM::InstructionSequence.load_from_binary
is running slower in my case than loading directly the source, right?
from bootsnap.
Can you replace lib/bootsnap/compile_cache/iseq.rb
with the following, run it again with compile_cache_iseq
enabled, and paste the last line it prints?:
require 'bootsnap/bootsnap'
require 'zlib'
module Bootsnap
module CompileCache
module ISeq
class << self
attr_accessor :cache_dir
end
$counts = { i2s: 0, s2o: 0, i2o: 0, uncompilable: 0 }
at_exit { puts $counts.inspect }
def self.input_to_storage(_, path)
$counts[:i2s] += 1
RubyVM::InstructionSequence.compile_file(path).to_binary
rescue SyntaxError
$counts[:uncompilable] += 1
raise Uncompilable, 'syntax error'
end
def self.storage_to_output(binary)
$counts[:s2o] += 1
RubyVM::InstructionSequence.load_from_binary(binary)
rescue RuntimeError => e
if e.message == 'broken binary format'
STDERR.puts "[Bootsnap::CompileCache] warning: rejecting broken binary"
return nil
else
raise
end
end
def self.input_to_output(_)
$counts[:i2o] += 1
nil # ruby handles this
end
module InstructionSequenceMixin
def load_iseq(path)
# Having coverage enabled prevents iseq dumping/loading.
return nil if defined?(Coverage) && Bootsnap::CompileCache::Native.coverage_running?
Bootsnap::CompileCache::Native.fetch(
Bootsnap::CompileCache::ISeq.cache_dir,
path.to_s,
Bootsnap::CompileCache::ISeq
)
rescue RuntimeError => e
if e.message =~ /unmatched platform/
puts "unmatched platform for file #{path}"
end
raise
end
def compile_option=(hash)
super(hash)
Bootsnap::CompileCache::ISeq.compile_option_updated
end
end
def self.compile_option_updated
option = RubyVM::InstructionSequence.compile_option
crc = Zlib.crc32(option.inspect)
Bootsnap::CompileCache::Native.compile_option_crc32 = crc
end
def self.install!(cache_dir)
Bootsnap::CompileCache::ISeq.cache_dir = cache_dir
Bootsnap::CompileCache::ISeq.compile_option_updated
class << RubyVM::InstructionSequence
prepend InstructionSequenceMixin
end
end
end
end
end
from bootsnap.
This is the output replacing the code:
{:i2s=>0, :s2o=>3300, :i2o=>0, :uncompilable=>0}
from bootsnap.
Huh, ok, can you change the definition of storage_to_output
to:
$t = 0
at_exit { puts $t }
def self.storage_to_output(binary)
$counts[:s2o] += 1
t=Time.now
b = RubyVM::InstructionSequence.load_from_binary(binary)
$t += (Time.now - t)
b
rescue RuntimeError => e
if e.message == 'broken binary format'
STDERR.puts "[Bootsnap::CompileCache] warning: rejecting broken binary"
return nil
else
raise
end
end
from bootsnap.
ok, running the app a couple of times I got this:
during first run after of deleting cache:
~/src/cookpad[master] % time bin/rails runner "puts Rails.env"
development
0.47127000000000047
{:i2s=>3301, :s2o=>3301, :i2o=>0, :uncompilable=>0}
bin/rails runner "puts Rails.env" 9.46s user 2.72s system 43% cpu 27.728 total
during subsequent runs:
~/src/cookpad[master] % time bin/rails runner "puts Rails.env"
development
0.6067550000000022
{:i2s=>0, :s2o=>3301, :i2o=>0, :uncompilable=>0}
bin/rails runner "puts Rails.env" 6.49s user 1.60s system 16% cpu 48.926 total
from bootsnap.
Oh wow, I wonder where that time is going then. Bizarre.
from bootsnap.
Hey, thanks for Bootsnap! I've also just tried it out today and experienced a similar issue - compile_cache_iseq
makes things slower. This is with Ruby 2.4.4 in an Alpine Linux Docker container.
from bootsnap.
Yeah. I just experienced this at Shopify too. I'll try to isolate the cause but I believe RubyVM::InstructionSequence.load_from_binary
is the cause.
from bootsnap.
Something I was pondering the other day is that maybe it is related to sharing the host's filesystem with the Docker container. I was trying this on Mac, and I know there is extra overhead on that platform. Might try to run some experiments when I get a chance.
from bootsnap.
Yeah, we could probably document this better, but I would expect performance weirdness when mounting the cache FS over the network, since it would just double the number of network roundtrips. If time-to-access latency is the dominating concern, that would slow it down for sure.
from bootsnap.
Related Issues (20)
- /usr/share/ruby/time.rb is not `.stable?` - path_test.rb failing HOT 9
- YAML cache "bypasses" permitted_classes in some cases HOT 4
- ibf_dump_object_unsupported […] T_NONE (NotImplementedError) HOT 26
- bootsnap compile-cache-yaml folder contents file permissions issue HOT 4
- windows rails demo project error HOT 2
- sharing remote caching bootsnap cache dir across machines HOT 2
- Facing `compile_file': "\\xE2" on US-ASCII (Encoding::InvalidByteSequenceError) error when trying to run any rails command HOT 14
- Bootsnap causes server to hang in Ruby 3.2.1 HOT 6
- undefined method `driver_path=' for Selenium::WebDriver::Chrome:Module HOT 1
- Testsuite fails with ruby3.3.0dev HOT 6
- warning: method redefined; discarding old require HOT 2
- Bootsnap LoadError for Prism HOT 8
- `bin/bootsnap precompile` not compiling everything HOT 7
- 1.18.x extension build failed HOT 11
- 1.18.x: `compile_cache/iseq.rb:64:in 'fetch': Invalid argument - (Errno::EINVAL)` HOT 1
- EOFError: end of buffer reached HOT 9
- Shopify IP allowlist restricting access to non-sensitive resource? HOT 3
- I feel like everybody is doing `gem "bootsnap", require: false` so why not make bootsnap.rb do nothing? HOT 2
- `bootsnap -v` or `bootsnap --version` could return the version of bootsnap? HOT 1
- Exception raised when upgraded to ruby 3.3.1 and deployed to Heroku HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bootsnap.