Git Product home page Git Product logo

gcbenchmarks's People

Contributors

chriselrod avatar d-netto avatar gbaraldi avatar kpamnany avatar nhdaly avatar oscardssmith avatar seelengrab avatar vchuravy avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

oscardssmith chflood seelengrab kpamnany chriselrod nhdaly udesou qinsoon tjljw2000

gcbenchmarks's Issues

Having trouble running ensemblesolve.jl

Tried running it but I'm getting a:

ERROR: LoadError: ArgumentError: Package DiffEqNoiseProcess does not have Markdown in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have DiffEqNoiseProcess checked out for development and have
  added Markdown as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with DiffEqNoiseProcess
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:1137 [inlined]
 [2] macro expansion
   @ ./lock.jl:267 [inlined]
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1114
 [4] include
   @ ./Base.jl:418 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1501
 [6] top-level scope
   @ stdin:1
in expression starting at /home/netto/.julia/packages/DiffEqNoiseProcess/iWugD/src/DiffEqNoiseProcess.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile DiffEqNoiseProcess [77a26b50-5914-5dd7-bc55-306e6241c503] to /home/netto/.julia/compiled/v1.9/DiffEqNoiseProcess/jl_Bd0eF3.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1652
  [3] compilecache
    @ ./loading.jl:1596 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1297
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1170
  [6] macro expansion
    @ ./loading.jl:1150 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1114
  [9] include
    @ ./Base.jl:418 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1501
 [11] top-level scope
    @ stdin:1
in expression starting at /home/netto/.julia/packages/DifferentialEquations/4jfQK/src/DifferentialEquations.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile DifferentialEquations [0c46a032-eb83-5123-abaf-570d42b7fbaa] to /home/netto/.julia/compiled/v1.9/DifferentialEquations/jl_62jKDA.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1652
 [3] compilecache
   @ ./loading.jl:1596 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1297
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1170
 [6] macro expansion
   @ ./loading.jl:1150 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1114
in expression starting at /home/netto/GCBenchmarks/benches/diffeq/ensemblesolve.jl:1
ERROR: LoadError: EOFError: read end of file
Stacktrace:
 [1] read(this::Base.PipeEndpoint, #unused#::Type{UInt8})
   @ Base ./stream.jl:977
 [2] deserialize
   @ ~/julia_pr/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
 [3] deserialize(s::Base.PipeEndpoint)
   @ Serialization ~/julia_pr/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:801
 [4] run_one_bench(runs::Int64, threads::Int64, file::String)
   @ Main ~/GCBenchmarks/run_benchmarks.jl:58
 [5] main()
   @ Main ~/GCBenchmarks/run_benchmarks.jl:113
 [6] top-level scope
   @ ~/GCBenchmarks/run_benchmarks.jl:117
in expression starting at /home/netto/GCBenchmarks/run_benchmarks.jl:117

Pkg.instantiate() and Pkg.resolve() don't seem to solve it.

Error when running compiler_stresstest benchmark

I've updated to the latest version of the GCBenchmarks repo and I'm getting the following error when running the compiler_stresstest.jl benchmark.

category = "compiler"
file = "compiler_stresstest.jl"
     Cloning git-repo `https://github.com/JuliaDebug/Cthulhu.jl.git`

signal (15): Terminated
in expression starting at none:0
calc_gnu_debuglink_crc32 at /home/eduardo/julia/src/debuginfo.cpp:678 [inlined]
openDebugInfo at /home/eduardo/julia/src/debuginfo.cpp:693
find_object_file at /home/eduardo/julia/src/debuginfo.cpp:982 [inlined]
jl_dylib_DI_for_fptr at /home/eduardo/julia/src/debuginfo.cpp:1132
jl_getDylibFunctionInfo at /home/eduardo/julia/src/debuginfo.cpp:1168 [inlined]
jl_getFunctionInfo_impl at /home/eduardo/julia/src/debuginfo.cpp:1240
ijl_lookup_code_address at /home/eduardo/julia/src/stackwalk.c:587
lookup at ./stacktraces.jl:107
stacktrace at ./stacktraces.jl:163
stacktrace at ./stacktraces.jl:161 [inlined]
scrub_repl_backtrace at ./client.jl:87
jfptr_scrub_repl_backtrace_34656 at /home/eduardo/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/eduardo/julia/src/gf.c:2392 [inlined]
ijl_apply_generic at /home/eduardo/julia/src/gf.c:2574
scrub_repl_backtrace at ./client.jl:94
exec_options at ./client.jl:306
_start at ./client.jl:518
jfptr__start_47387 at /home/eduardo/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/eduardo/julia/src/gf.c:2392 [inlined]
ijl_apply_generic at /home/eduardo/julia/src/gf.c:2574
jl_apply at /home/eduardo/julia/src/julia.h:1840 [inlined]
true_main at /home/eduardo/julia/src/jlapi.c:566
jl_repl_entrypoint at /home/eduardo/julia/src/jlapi.c:710
main at /home/eduardo/julia/cli/loader_exe.c:59
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /home/eduardo/julia/usr/bin/julia (unknown line)
unknown function (ip: (nil))
Allocations: 1293958 (Pool: 1293247; Big: 711); GC: 2
ERROR: LoadError: AssertionError: false
Stacktrace:
  [1] deserialize(s::Serializer{Base.Process}, #unused#::Type{Task})
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:1437
  [2] handle_deserialize(s::Serializer{Base.Process}, b::Int32)
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:960
  [3] deserialize
    @ ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
  [4] deserialize(s::Serializer{Base.Process}, #unused#::Type{Task})
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:1427
  [5] handle_deserialize(s::Serializer{Base.Process}, b::Int32)
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:960
  [6] deserialize
    @ ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
  [7] deserialize(s::Serializer{Base.Process}, #unused#::Type{Task})
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:1427
  [8] handle_deserialize(s::Serializer{Base.Process}, b::Int32)
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:960
  [9] deserialize
    @ ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
 [10] deserialize(s::Serializer{Base.Process}, #unused#::Type{Task})
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:1427
 [11] handle_deserialize(s::Serializer{Base.Process}, b::Int32)
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:960
 [12] deserialize
    @ ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
 [13] deserialize(s::Base.Process)
    @ Serialization ~/julia/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:801
 [14] open(::typeof(deserialize), ::Cmd; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./process.jl:427
 [15] open
    @ ./process.jl:415 [inlined]
 [16] run_one_bench(runs::Int64, threads::Int64, file::String)
    @ Main ~/GCBenchmarks/run_benchmarks.jl:36
 [17] run_all_benches(runs::Int64, threads::Int64)
    @ Main ~/GCBenchmarks/run_benchmarks.jl:59
 [18] main()
    @ Main ~/GCBenchmarks/run_benchmarks.jl:73
 [19] top-level scope
    @ ~/GCBenchmarks/run_benchmarks.jl:81
in expression starting at /home/eduardo/GCBenchmarks/run_benchmarks.jl:81

I'm not sure if this is the right place to report the error, but perhaps I'm missing some configuration or something. I'm using the latest master build for Julia updated up until this commit.

Reducing scale of rb_tree

The benchmark seems to be taking a few hours to complete on the machine I'm testing on. Wondering whether we can reduce the size/time scales without affect its quality as a benchmark.

Dependency free benchmarks

Right now the diffeq and flux benchmark are stale.

I think we should remove them and future benchmarks need to version their dependencies.
These benchmarks need to be comparable over time across multiple Julia versions.

MethodError in objarrays.jl 

category = "bigarrays"
bench = "objarray.jl"
(gcthreads, threads) = (1, 8)
ERROR: LoadError: MethodError: no method matching (::Colon)(::Int64, ::Cmd)

Closest candidates are:
  (::Colon)(::T, ::Any, ::T) where T<:Real
   @ Base range.jl:50
  (::Colon)(::A, ::Any, ::C) where {A<:Real, C<:Real}
   @ Base range.jl:10
  (::Colon)(::T, ::Any, ::T) where T
   @ Base range.jl:49
  ...

Stacktrace:
 [1] macro expansion
   @ Main ./threadingconstructs.jl:136 [inlined]
 [2] run(maxsize::Cmd)
   @ Main ~/RAI/GCBenchmarks/benches/multithreaded/bigarrays/objarray.jl:30
 [3] top-level scope
   @ ~/RAI/GCBenchmarks/utils.jl:38
in expression starting at /Users/dnetto/RAI/GCBenchmarks/benches/multithreaded/bigarrays/objarray.jl:35
ERROR: LoadError: EOFError: read end of file
Stacktrace:
 [1] read(this::Base.PipeEndpoint, ::Type{UInt8})
   @ Base ./stream.jl:980
 [2] deserialize(s::AbstractSerializer)
   @ Serialization ~/RAI/builds/julia-master/usr/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:813 [inlined]
 [3] deserialize(s::Base.PipeEndpoint)
   @ Serialization ~/RAI/builds/julia-master/usr/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:800
 [4] run_bench(runs::Int64, threads::Int64, gcthreads::Int64, file::String, show_json::Bool)
   @ Main ~/RAI/GCBenchmarks/run_benchmarks.jl:68
 [5] run_category_files(benches::Vector{String}, args::Dict{String, Any}, show_json::Bool)
   @ Main ~/RAI/GCBenchmarks/run_benchmarks.jl:145
 [6] run_all_categories(args::Dict{String, Any}, show_json::Bool)
   @ Main ~/RAI/GCBenchmarks/run_benchmarks.jl:157
 [7] main(args::Dict{String, Any})
   @ Main ~/RAI/GCBenchmarks/run_benchmarks.jl:183
 [8] top-level scope
   @ ~/RAI/GCBenchmarks/run_benchmarks.jl:195
in expression starting at /Users/dnetto/RAI/GCBenchmarks/run_benchmarks.jl:195

mark time = 0

Happened when running tree_mutable.jl on JuliaLang/julia#45639.

threads = 2
┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐
│         │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │
│         │         ms │      ms │        ms │         ms │           ms │                us │       MB │          % │
├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤
│ minimum │      17269 │    7583 │         0 │         31 │          227 │                55 │      612 │         44 │
│  median │      17269 │    7583 │         0 │         31 │          227 │                55 │      612 │         44 │
│ maximum │      17269 │    7583 │         0 │         31 │          227 │                55 │      612 │         44 │
└─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘

Discrepancies in reported GCBenchmarks times

GCBenchmarks seems to be reporting 2x time differences between two commits, but /usr/bin/time seems to be indicating similar results (arguably it has a low resolution, but 2x differences in a ~10s benchmark should be detectable). Wondering if there could be an accounting error somewhere.

  • commit A:
/usr/bin/time -v ../julia_pr/julia -t32 benches/multithreaded/binary_tree/tree_immutable.jl
(value = nothing, times = 0x000000023aaea7e6, gc_diff = Base.GC_Diff(9788654305, 0, 0, 305679705, 370, 20, 7128353568, 19, 0), gc_end = Base.GC_Num(2370784, 0, 0, 173, 0, 311177345, 43076, 180, 7291395542, 10421004013, 0, 0x000000000490286b, 32, 0, 981405336, 959996647, 57500, 12817147, 95871236, 1658846, 6839526771, 451688062))
	Command being timed: "../julia_pr/julia -t32 benches/multithreaded/binary_tree/tree_immutable.jl"
	User time (seconds): 126.50
	System time (seconds): 36.48
	Percent of CPU this job got: 1313%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:12.40
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 1211584
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 2431137
	Voluntary context switches: 1806860
	Involuntary context switches: 11766307
	Swaps: 0
	File system inputs: 0
	File system outputs: 32
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐
│         │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │
│         │         ms │      ms │        ms │         ms │           ms │                us │       MB │          % │
├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤
│ minimum │       7316 │    3504 │       849 │       2636 │          276 │                86 │     1700 │         47 │
│  median │       8147 │    4052 │      1051 │       2969 │          306 │              3124 │     1736 │         49 │
│ maximum │       8891 │    4376 │      1285 │       3181 │          354 │             19691 │     1749 │         50 │
└─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘
  • commit B:
/usr/bin/time -v ../julia_pr/julia -t32 benches/multithreaded/binary_tree/tree_immutable.jl
(value = nothing, times = 0x0000000200cbfdfd, gc_diff = Base.GC_Diff(9788654241, 0, 0, 305679705, 370, 20, 6344223929, 19, 0), gc_end = Base.GC_Num(61102312, 0, 0, 173, 0, 311177628, 43074, 180, 6445774786, 10362260197, 0, 0x00000000056dd36c, 32, 0, 744576242, 946989655, 8693209, 15107517, 93963581, 22333670, 5854424797, 591178541))
	Command being timed: "../julia_pr/julia -t32 benches/multithreaded/binary_tree/tree_immutable.jl"
	User time (seconds): 106.15
	System time (seconds): 32.55
	Percent of CPU this job got: 1222%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:11.34
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 1205228
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 2401199
	Voluntary context switches: 1859271
	Involuntary context switches: 12255069
	Swaps: 0
	File system inputs: 0
	File system outputs: 32
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐
│         │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │
│         │         ms │      ms │        ms │         ms │           ms │                us │       MB │          % │
├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤
│ minimum │      18189 │   14457 │      1049 │      13408 │         1511 │                65 │     1713 │         79 │
│  median │      18189 │   14457 │      1049 │      13408 │         1511 │                65 │     1713 │         79 │
│ maximum │      18189 │   14457 │      1049 │      13408 │         1511 │                65 │     1713 │         79 │
└─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘

Method mismatch in ensemblesolve.jl

Basically, just did a julia run_benchmarks.jl slow diffeq and got the following error:

ERROR: LoadError: MethodError: no method matching SparseArrays.UMFPACK.UmfpackLU(::Ptr{Nothing}, ::Ptr{Nothing}, ::Int64, ::Int64, ::Vector{Int64}, ::Vector{Int64}, ::Vector{Float64}, ::Int64)

Closest candidates are:
  SparseArrays.UMFPACK.UmfpackLU(::Ptr{Nothing}, ::Ptr{Nothing}, ::Int64, ::Int64, ::Vector{Ti}, ::Vector{Ti}, ::Vector{Tv}, ::Int64, ::SparseArrays.UMFPACK.UmfpackWS{Ti}, ::Vector{Float64}, ::Vector{Float64}, ::ReentrantLock) where {Tv<:Union{Float64, ComplexF64}, Ti<:Union{Int32, Int64}}
   @ SparseArrays ~/julia-45608/usr/share/julia/stdlib/v1.9/SparseArrays/src/solvers/umfpack.jl:163

Stacktrace:
  [1] init_cacheval(alg::LinearSolve.UMFPACKFactorization, A::SparseArrays.SparseMatrixCSC{Float64, Int64}, b::Vector{Float64}, u::Vector{Float64}, Pl::IterativeSolvers.Identity, Pr::IterativeSolvers.Identity, maxiters::Int64, abstol::Float64, reltol::Float64, verbose::Bool, assumptions::LinearSolve.OperatorAssumptions{Nothing})
    @ LinearSolve ~/.julia/packages/LinearSolve/dY1ir/src/factorization.jl:271
  [2] init(::SciMLBase.LinearProblem{Nothing, true, SparseArrays.SparseMatrixCSC{Float64, Int64}, Vector{Float64}, SciMLBase.NullParameters, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::LinearSolve.UMFPACKFactorization; alias_A::Bool, alias_b::Bool, abstol::Float64, reltol::Float64, maxiters::Int64, verbose::Bool, Pl::IterativeSolvers.Identity, Pr::IterativeSolvers.Identity, assumptions::LinearSolve.OperatorAssumptions{Nothing}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ LinearSolve ~/.julia/packages/LinearSolve/dY1ir/src/common.jl:108
  [3] init(::SciMLBase.LinearProblem{Nothing, true, SparseArrays.SparseMatrixCSC{Float64, Int64}, Vector{Float64}, SciMLBase.NullParameters, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::LinearSolve.UMFPACKFactorization)
    @ LinearSolve ~/.julia/packages/LinearSolve/dY1ir/src/common.jl:88
  [4] solve(::SciMLBase.LinearProblem{Nothing, true, SparseArrays.SparseMatrixCSC{Float64, Int64}, Vector{Float64}, SciMLBase.NullParameters, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::LinearSolve.UMFPACKFactorization; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ LinearSolve ~/.julia/packages/LinearSolve/dY1ir/src/common.jl:151
  [5] solve(::SciMLBase.LinearProblem{Nothing, true, SparseArrays.SparseMatrixCSC{Float64, Int64}, Vector{Float64}, SciMLBase.NullParameters, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::LinearSolve.UMFPACKFactorization)
    @ LinearSolve ~/.julia/packages/LinearSolve/dY1ir/src/common.jl:148
  [6] macro expansion
    @ ~/.julia/packages/LinearSolve/dY1ir/src/LinearSolve.jl:79 [inlined]
  [7] macro expansion
    @ ~/.julia/packages/SnoopPrecompile/UWvXF/src/SnoopPrecompile.jl:51 [inlined]
  [8] top-level scope
    @ ~/.julia/packages/LinearSolve/dY1ir/src/LinearSolve.jl:73
  [9] include
    @ ./Base.jl:424 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1614
 [11] top-level scope
    @ stdin:1
in expression starting at /home/netto/.julia/packages/LinearSolve/dY1ir/src/LinearSolve.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile LinearSolve [7ed4a6bd-45f5-4d41-b270-4a48e9bafcae] to /home/netto/.julia/compiled/v1.9/LinearSolve/jl_xhePzz.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1765
  [3] compilecache
    @ ./loading.jl:1709 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1397
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1260
  [6] macro expansion
    @ ./loading.jl:1240 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1204
  [9] include
    @ ./Base.jl:424 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1614
 [11] top-level scope
    @ stdin:1
in expression starting at /home/netto/.julia/packages/OrdinaryDiffEq/SRiFK/src/OrdinaryDiffEq.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile OrdinaryDiffEq [1dea7af3-3e70-54e6-95c3-0bf5283fa5ed] to /home/netto/.julia/compiled/v1.9/OrdinaryDiffEq/jl_gos6hk.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1765
  [3] compilecache
    @ ./loading.jl:1709 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1397
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1260
  [6] macro expansion
    @ ./loading.jl:1240 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1204
  [9] include
    @ ./Base.jl:424 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1614
 [11] top-level scope
    @ stdin:1
in expression starting at /home/netto/.julia/packages/StochasticDiffEq/MT7GU/src/StochasticDiffEq.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile StochasticDiffEq [789caeaf-c7a9-5a7d-9973-96adeb23e2a0] to /home/netto/.julia/compiled/v1.9/StochasticDiffEq/jl_0DXbMb.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1765
  [3] compilecache
    @ ./loading.jl:1709 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1397
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1260
  [6] macro expansion
    @ ./loading.jl:1240 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1204
  [9] include
    @ ./Base.jl:424 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1614
 [11] top-level scope
    @ stdin:1
in expression starting at /home/netto/.julia/packages/DifferentialEquations/yS3VA/src/DifferentialEquations.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile DifferentialEquations [0c46a032-eb83-5123-abaf-570d42b7fbaa] to /home/netto/.julia/compiled/v1.9/DifferentialEquations/jl_xebfK2.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1765
 [3] compilecache
   @ ./loading.jl:1709 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1397
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1260
 [6] macro expansion
   @ ./loading.jl:1240 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1204
in expression starting at /home/netto/GCBenchmarks/benches/slow/diffeq/ensemblesolve.jl:3
ERROR: LoadError: EOFError: read end of file
Stacktrace:
 [1] read(this::Base.PipeEndpoint, #unused#::Type{UInt8})
   @ Base ./stream.jl:977
 [2] deserialize
   @ ~/julia-45608/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:814 [inlined]
 [3] deserialize(s::Base.PipeEndpoint)
   @ Serialization ~/julia-45608/usr/share/julia/stdlib/v1.9/Serialization/src/Serialization.jl:801
 [4] run_bench(runs::Int64, threads::Int64, file::String, show_json::Bool)
   @ Main ~/GCBenchmarks/run_benchmarks.jl:53
 [5] run_category_files(benches::Vector{String}, args::Dict{String, Any}, show_json::Bool)
   @ Main ~/GCBenchmarks/run_benchmarks.jl:102
 [6] main(args::Dict{String, Any})
   @ Main ~/GCBenchmarks/run_benchmarks.jl:154
 [7] top-level scope
   @ ~/GCBenchmarks/run_benchmarks.jl:158
in expression starting at /home/netto/GCBenchmarks/run_benchmarks.jl:158

Significantly different output for GC times depending on whether benchmark is run directly or through the CLI

Example:

  • directly:
../julia-stock/julia --gcthreads=1 --project=. benches/serial/linked/tree.jl 

GC: pause 24.28ms. collected 63.577698MB. incr 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 128.03 MB,
heap_size 83.68 MB, heap_target 130.18 MB, Fragmentation 0.495

GC: pause 33.71ms. collected 64.622284MB. full 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 85.91 MB,
heap_size 83.96 MB, heap_target 120.18 MB, Fragmentation 0.554

GC: pause 129.62ms. collected 24.792351MB. incr 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 85.91 MB,
heap_size 105.57 MB, heap_target 152.59 MB, Fragmentation 0.446

GC: pause 257.01ms. collected 0.133347MB. incr 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 111.33 MB,
heap_size 152.43 MB, heap_target 235.51 MB, Fragmentation 0.493

GC: pause 760.00ms. collected 17.473351MB. full 
Heap stats: bytes_mapped 256.06 MB, bytes_resident 250.31 MB,
heap_size 228.08 MB, heap_target 321.53 MB, Fragmentation 0.548
(value = -4192633690078234733, times = 0x0000000100b63e3d, gc_diff = Base.GC_Diff(167772640, 1, 0, 4194314, 0, 101, 386626515, 2, 0), gc_start = Base.GC_Num(14647352, 0, 0, 2986, 0, 5276942, 1254, 2592, 325682843, 314985560, 0x0000000002bc0000, 6, 2, 112956088, 111855641, 10224, 15240, 62803, 20980681, 12730348, 50553001, 275064223, 9078620960059293, 9078620844326467), gc_end = Base.GC_Num(74668288, 0, 0, 2987, 0, 9471256, 1254, 2693, 712309358, 422737264, 0x0000000002bc0000, 8, 2, 257005408, 111855641, 15507, 15507, 91649, 6989427, 250015623, 61368525, 650854878, 9078620960059293, 9078623367010371))
  • CLI:
../julia-stock/julia --project=. run_benchmarks.jl serial linked tree -n1
bench = "tree.jl"

GC: pause 25.16ms. collected 62.320648MB. incr 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 128.03 MB,
heap_size 88.44 MB, heap_target 135.03 MB, Fragmentation 0.469

GC: pause 33.09ms. collected 67.997505MB. full 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 84.97 MB,
heap_size 114.39 MB, heap_target 156.54 MB, Fragmentation 0.408

GC: pause 359.39ms. collected 8.745331MB. incr 
Heap stats: bytes_mapped 128.03 MB, bytes_resident 109.05 MB,
heap_size 148.87 MB, heap_target 232.44 MB, Fragmentation 0.516

GC: pause 18.04ms. collected 171.679581MB. incr 
Heap stats: bytes_mapped 192.05 MB, bytes_resident 113.00 MB,
heap_size 76.92 MB, heap_target 115.11 MB, Fragmentation 1.565
┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐
│         │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │
│         │         ms │      ms │        ms │         ms │           ms │                us │       MB │          % │
├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤
│ minimum │       4376 │     392 │       363 │         30 │          359 │                22 │      140 │          9 │
│  median │       4376 │     392 │       363 │         30 │          359 │                22 │      140 │          9 │
│ maximum │       4376 │     392 │       363 │         30 │          359 │                22 │      140 │          9 │
│   stdev │        NaN │     NaN │       NaN │        NaN │          NaN │               NaN │      NaN │        NaN │
└─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘

Setup bpftrace variant of the benchmark script

Diogo and I were looking at the scaling of the mark code and measuring the pure GC time was not giving us enough insight.

Using:

#!/usr/bin/env bpftrace

BEGIN
{
    printf("Tracing Julia GC Times... Hit Ctrl-C to end.\n");
}

usdt:usr/lib/libjulia-internal.so:julia:gc__mark__begin
{
    @start[pid] = nsecs;
}

usdt:usr/lib/libjulia-internal.so:julia:gc__mark__end
/@start[pid]/
{
    @msecs[pid] = lhist((nsecs - @start[pid]) / 1000000, 0, 128, 8);
    delete(@start[pid]);
}

END
{
    clear(@start);
}

We got for one thread:

➜  julia sudo ~/src/julia/contrib/bpftrace/gc_simple.bt -p (pidof julia)
Attaching 4 probes...
Tracing Julia GC Times... Hit Ctrl-C to end.

@msecs[55902]: 
[0, 8)                68 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[8, 16)                1 |                                                    |
[16, 24)               3 |@@                                                  |
[24, 32)               0 |                                                    |
[32, 40)               0 |                                                    |
[40, 48)               0 |                                                    |
[48, 56)               0 |                                                    |
[56, 64)               0 |                                                    |
[64, 72)               0 |                                                    |
[72, 80)               0 |                                                    |
[80, 88)               0 |                                                    |
[88, 96)               0 |                                                    |
[96, 104)              0 |                                                    |
[104, 112)             1 |                                                    |
[112, 120)            44 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                   |
[120, 128)            13 |@@@@@@@@@                                           |
[128, ...)             9 |@@@@@@                                              |

gc_scrub benchmark 1 thread

For two threads:

➜  julia sudo ~/src/julia/contrib/bpftrace/gc_simple.bt -p (pidof julia)
Attaching 4 probes...
Tracing Julia GC Times... Hit Ctrl-C to end.

@msecs[56302]: 
[0, 8)                74 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[8, 16)                1 |                                                    |
[16, 24)               3 |@@                                                  |
[24, 32)               0 |                                                    |
[32, 40)               0 |                                                    |
[40, 48)               0 |                                                    |
[48, 56)               0 |                                                    |
[56, 64)               0 |                                                    |
[64, 72)              57 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@            |
[72, 80)              11 |@@@@@@@                                             |
[80, 88)               2 |@                                                   |
[88, 96)               2 |@                                                   |
[96, 104)              1 |                                                    |

But we need to find a better way then running tho processes next to each other an then using pidof to attach the traces.

n collections = -1?

happened when running red-black tree benchmark. @kpamnany

  2.115060 seconds (100.00% gc time)
elapsed time (ns):  2115059916
gc time (ns):       2115058458
bytes allocated:    0
pool allocs:        0
non-pool GC allocs: 0
minor collections:  -1
full collections:   2

Benchmark reports max_pause > 0 despite gc_time = 0

Example:

category = "TimeZones"
bench = "TimeZones.jl"
(gcthreads, threads) = (1, 1)
┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐
│         │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │
│         │         ms │      ms │        ms │         ms │           ms │                us │       MB │          % │
├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤
│ minimum │        927 │       0 │         0 │          0 │          120 │                 0 │     4638 │          0 │
│  median │        927 │       0 │         0 │          0 │          120 │                 0 │     4638 │          0 │
│ maximum │        927 │       0 │         0 │          0 │          120 │                 0 │     4638 │          0 │
│   stdev │        NaN │     NaN │       NaN │        NaN │          NaN │               NaN │      NaN │        NaN │
└─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘

We might be incorrectly caching some maximum pause value from before the benchmark runs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.