joachimbrand / rimu.jl Goto Github PK

View Code? Open in Web Editor NEW

20.0 3.0 1.0 93.29 MB

Random Integrators for many-body quantum systems

License: MIT License

Julia 100.00%

fciqmc projector-quantum-monte-carlo exact-diagonalization

rimu.jl's People

Contributors

Stargazers

Watchers

Forkers

nasir26

rimu.jl's Issues

`save_dvec()` and `load_dvec()` are not exported

Rimu.jl/src/RimuIO.jl

Line 16 in cf90d2c

export save_df, load_df

I guess this is not intentional, I see no reason to not export these two functions.

ERROR: UndefVarError: ProjectorMonteCarloProblem not defined

I want to use Rimu package but it give me error

julia> using Rimu
julia> ProjectorMonteCarloProblem

output
ERROR: UndefVarError: ProjectorMonteCarloProblem not defined
Could you help me?
Thank you!

Tests failing on Apple Silicon

Rimu tests fail on Apple MacBook Pro with M2 Pro processor.

The error seems to be triggered by MPI reductions:

AllOverlaps: Error During Test at /Users/brand/git/code/Rimu/test/lomc.jl:90
  Got exception outside of a @test
  User-defined reduction operators are currently not supported on non-Intel architectures.
  See https://github.com/JuliaParallel/MPI.jl/issues/404 for more details.
  Stacktrace:
    [1] error(s::String)
      @ Base ./error.jl:35
    [2] MPI.Op(f::Function, T::Type; iscommutative::Bool)
      @ MPI ~/.julia/packages/MPI/APiiL/src/operators.jl:95
    [3] MPI.Op(f::Function, T::Type)
      @ MPI ~/.julia/packages/MPI/APiiL/src/operators.jl:91
    [4] Allreduce!(rbuf::MPI.RBuffer{Base.RefValue{Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}}}, Base.RefValue{Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}}}}, op::Function, comm::MPI.Comm)
      @ MPI ~/.julia/packages/MPI/APiiL/src/collective.jl:668
    [5] Allreduce!(sendbuf::Base.RefValue{Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}}}, recvbuf::Base.RefValue{Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}}}, op::Function, comm::MPI.Comm)
      @ MPI ~/.julia/packages/MPI/APiiL/src/collective.jl:670
    [6] Allreduce(obj::Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}}, op::Function, comm::MPI.Comm)
      @ MPI ~/.julia/packages/MPI/APiiL/src/collective.jl:695
    [7] sort_into_targets!(dtarget::Rimu.RMPI.MPIData{DVec{BoseFS{5, 15, BitString{19, 1, UInt32}}, Float64, IsDynamicSemistochastic{Float64, Rimu.StochasticStyles.ThresholdCompression{Float64}, DynamicSemistochastic{Float64, WithReplacement{Float64}}}, Dict{BoseFS{5, 15, BitString{19, 1, UInt32}}, Float64}}, Rimu.RMPI.MPIPointToPoint{Pair{BoseFS{5, 15, BitString{19, 1, UInt32}}, Float64}, 1}}, w::DVec{BoseFS{5, 15, BitString{19, 1, UInt32}}, Float64, IsDynamicSemistochastic{Float64, Rimu.StochasticStyles.ThresholdCompression{Float64}, DynamicSemistochastic{Float64, WithReplacement{Float64}}}, Dict{BoseFS{5, 15, BitString{19, 1, UInt32}}, Float64}}, stats::Rimu.MultiScalar{Tuple{Int64, Int64, Int64, Float64}})
      @ Rimu.RMPI ~/git/code/Rimu/src/RMPI/helpers.jl:91
...

Related MPI issue: JuliaParallel/MPI.jl#404.

In short we do many reduction operations with MPI.Allreduce. The issue arises when the reduction operation does something else/more than just one of the few built-in reductions for scalars. Passing a general Julia function as reduction operator to MPI.Allreduce apparently works only on Intel processors at the moment.

E.g. MPI-enabled sum fails because sum uses a non-generic reduction operator that includes some type conversion

"""
    Base.add_sum(x, y)

The reduction operator used in `sum`. The main difference from [`+`](@ref) is that small
integers are promoted to `Int`/`UInt`.
"""
add_sum(x, y) = x + y
add_sum(x::SmallSigned, y::SmallSigned) = Int(x) + Int(y)
add_sum(x::SmallUnsigned, y::SmallUnsigned) = UInt(x) + UInt(y)
add_sum(x::Real, y::Real)::Real = x + y

Attempt to resolve (or work around) the issue: #196

MPI memory leakage

It has been observed that memory usage may increase linearly in time when running multiple MPI ranks on the same node leading to memory overflows. This was not an issue in the past and downgrading to MPI.jl v0.10 seems to help as a workaround.

Tests fail on Julia v1.8rc4

Pkg.test() fails on the latest release candidate for the new Julia version.

The error is thrown when attempting to compare columns from DataFrames with isapprox, where one of the DataFrames was loaded from an .arrow file. It looks like the culprit might be in the SentinalArrays package.

Here is the error message:

ReportToFile: Error During Test at /Users/brand/git/Rimu/test/lomc.jl:330
  Test threw exception
  Expression: df1.shift ≈ df2.shift
  MethodError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer
  Stacktrace:
    [1] reduce_empty(op::Base.MappingRF{typeof(norm), typeof(max)}, #unused#::Type{Float64})
      @ Base ./reduce.jl:348
    [2] reduce_empty_iter
      @ ./reduce.jl:371 [inlined]
    [3] mapreduce_empty_iter(f::Function, op::Function, itr::Arrow.Primitive{Float64, Vector{Float64}}, ItrEltype::Base.HasEltype)
      @ Base ./reduce.jl:367
    [4] _mapreduce(f::typeof(norm), op::typeof(max), #unused#::IndexLinear, A::Arrow.Primitive{Float64, Vector{Float64}})
      @ Base ./reduce.jl:419
    [5] _mapreduce_dim
      @ ./reducedim.jl:365 [inlined]
    [6] #mapreduce#764
      @ ./reducedim.jl:357 [inlined]
    [7] mapreduce
      @ ./reducedim.jl:357 [inlined]
    [8] #36
      @ ./none:0 [inlined]
    [9] MappingRF
      @ ./reduce.jl:95 [inlined]
   [10] _foldl_impl(op::Base.MappingRF{SentinelArrays.var"#36#37"{typeof(norm), typeof(max)}, Base.BottomRF{typeof(max)}}, init::Base._InitialValue, itr::Vector{Arrow.Primitive{Float64, Vector{Float64}}})
      @ Base ./reduce.jl:62
   [11] foldl_impl
      @ ./reduce.jl:48 [inlined]
   [12] mapfoldl_impl
      @ ./reduce.jl:44 [inlined]
   [13] #mapfoldl#258
      @ ./reduce.jl:162 [inlined]
   [14] mapfoldl
      @ ./reduce.jl:162 [inlined]
   [15] #mapreduce#262
      @ ./reduce.jl:294 [inlined]
   [16] mapreduce
      @ ./reduce.jl:294 [inlined]
   [17] #reduce#264
      @ ./reduce.jl:475 [inlined]
   [18] reduce
      @ ./reduce.jl:475 [inlined]
   [19] mapreduce
      @ ~/.julia/packages/SentinelArrays/EQtMp/src/chainedvector.jl:736 [inlined]
   [20] generic_normInf
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:453 [inlined]
   [21] normInf
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:522 [inlined]
   [22] generic_norm2(x::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}})
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:463
   [23] norm2
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:524 [inlined]
   [24] norm(itr::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}}, p::Int64)
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:593
   [25] norm
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:591 [inlined]
   [26] isapprox(x::Arrow.Primitive{Float64, Vector{Float64}}, y::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}}; atol::Int64, rtol::Float64, nans::Bool, norm::typeof(norm))
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:1702
   [27] isapprox(x::Arrow.Primitive{Float64, Vector{Float64}}, y::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}})
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:1696
   [28] eval_test(evaluated::Expr, quoted::Expr, source::LineNumberNode, negate::Bool)
      @ Test /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:307
   [29] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
   [30] macro expansion
      @ ~/git/Rimu/test/lomc.jl:330 [inlined]
   [31] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [32] macro expansion
      @ ~/git/Rimu/test/lomc.jl:306 [inlined]
   [33] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [34] macro expansion
      @ ~/git/Rimu/test/lomc.jl:289 [inlined]
   [35] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [36] top-level scope
      @ ~/git/Rimu/test/lomc.jl:15
ReportToFile: Error During Test at /Users/brand/git/Rimu/test/lomc.jl:331
  Test threw exception
  Expression: df2.norm ≈ df3.norm
  MethodError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer
  Stacktrace:
    [1] reduce_empty(op::Base.MappingRF{typeof(norm), typeof(max)}, #unused#::Type{Float64})
      @ Base ./reduce.jl:348
    [2] reduce_empty_iter
      @ ./reduce.jl:371 [inlined]
    [3] mapreduce_empty_iter(f::Function, op::Function, itr::Arrow.Primitive{Float64, Vector{Float64}}, ItrEltype::Base.HasEltype)
      @ Base ./reduce.jl:367
    [4] _mapreduce(f::typeof(norm), op::typeof(max), #unused#::IndexLinear, A::Arrow.Primitive{Float64, Vector{Float64}})
      @ Base ./reduce.jl:419
    [5] _mapreduce_dim
      @ ./reducedim.jl:365 [inlined]
    [6] #mapreduce#764
      @ ./reducedim.jl:357 [inlined]
    [7] mapreduce
      @ ./reducedim.jl:357 [inlined]
    [8] #36
      @ ./none:0 [inlined]
    [9] MappingRF
      @ ./reduce.jl:95 [inlined]
   [10] _foldl_impl(op::Base.MappingRF{SentinelArrays.var"#36#37"{typeof(norm), typeof(max)}, Base.BottomRF{typeof(max)}}, init::Base._InitialValue, itr::Vector{Arrow.Primitive{Float64, Vector{Float64}}})
      @ Base ./reduce.jl:62
   [11] foldl_impl
      @ ./reduce.jl:48 [inlined]
   [12] mapfoldl_impl
      @ ./reduce.jl:44 [inlined]
   [13] #mapfoldl#258
      @ ./reduce.jl:162 [inlined]
   [14] mapfoldl
      @ ./reduce.jl:162 [inlined]
   [15] #mapreduce#262
      @ ./reduce.jl:294 [inlined]
   [16] mapreduce
      @ ./reduce.jl:294 [inlined]
   [17] #reduce#264
      @ ./reduce.jl:475 [inlined]
   [18] reduce
      @ ./reduce.jl:475 [inlined]
   [19] mapreduce
      @ ~/.julia/packages/SentinelArrays/EQtMp/src/chainedvector.jl:736 [inlined]
   [20] generic_normInf
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:453 [inlined]
   [21] normInf
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:522 [inlined]
   [22] generic_norm2(x::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}})
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:463
   [23] norm2
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:524 [inlined]
   [24] norm(itr::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}}, p::Int64)
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:593
   [25] norm
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:591 [inlined]
   [26] isapprox(x::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}}, y::Arrow.Primitive{Float64, Vector{Float64}}; atol::Int64, rtol::Float64, nans::Bool, norm::typeof(norm))
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:1702
   [27] isapprox(x::SentinelArrays.ChainedVector{Float64, Arrow.Primitive{Float64, Vector{Float64}}}, y::Arrow.Primitive{Float64, Vector{Float64}})
      @ LinearAlgebra /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/LinearAlgebra/src/generic.jl:1696
   [28] eval_test(evaluated::Expr, quoted::Expr, source::LineNumberNode, negate::Bool)
      @ Test /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:307
   [29] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
   [30] macro expansion
      @ ~/git/Rimu/test/lomc.jl:331 [inlined]
   [31] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [32] macro expansion
      @ ~/git/Rimu/test/lomc.jl:306 [inlined]
   [33] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [34] macro expansion
      @ ~/git/Rimu/test/lomc.jl:289 [inlined]
   [35] macro expansion
      @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [36] top-level scope
      @ ~/git/Rimu/test/lomc.jl:15

(@v1.8) pkg> st --manifest
Status `~/.julia/environments/v1.8/Manifest.toml`
  [621f4979] AbstractFFTs v1.2.1
⌅ [1520ce14] AbstractTrees v0.3.4
  [7d9f7c33] Accessors v0.1.18
  [79e6a3ab] Adapt v3.4.0
  [dce04be8] ArgCheck v2.3.0
  [4fba245c] ArrayInterface v6.0.22
  [30b0a656] ArrayInterfaceCore v0.1.17
  [015c0d05] ArrayInterfaceOffsetArrays v0.1.6
  [b0d46f97] ArrayInterfaceStaticArrays v0.1.4
  [dd5226c6] ArrayInterfaceStaticArraysCore v0.1.0
  [69666777] Arrow v2.3.0
⌅ [31f734f8] ArrowTypes v1.2.1
  [bf4720bc] AssetRegistry v0.1.0
  [c52e3926] Atom v0.12.38
  [fbb218c0] BSON v0.3.5
  [198e06fe] BangBang v0.3.36
  [9718e550] Baselet v0.1.1
  [6e4b80f9] BenchmarkTools v1.3.1
  [c3b6d118] BitIntegers v0.2.6
  [62783981] BitTwiddlingConvenienceFunctions v0.1.4
  [fa961155] CEnum v0.4.2
  [2a0fbf3d] CPUSummary v0.1.25
⌅ [00ebfdb7] CSTParser v2.5.0
  [49dc2e85] Calculus v0.5.1
  [d360d2e6] ChainRulesCore v1.15.3
  [9e997f8a] ChangesOfVariables v0.1.4
  [53a63b46] CodeTools v0.7.1
  [da1fd8a2] CodeTracking v1.1.0
  [5ba52731] CodecLz4 v0.4.0
  [6b39b394] CodecZstd v0.7.2
  [3da002f7] ColorTypes v0.11.4
  [5ae59095] Colors v0.12.8
  [a80b9123] CommonMark v0.8.6
  [bbf7d656] CommonSubexpressions v0.3.0
⌅ [34da2185] Compat v3.45.0
  [a33af91c] CompositionsBase v0.1.1
  [88cd18e8] ConsoleProgressMonitor v0.1.2
  [187b0558] ConstructionBase v1.4.0
  [adafc99b] CpuId v0.3.1
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.10.0
  [a93c6f00] DataFrames v1.3.4
  [864edb3b] DataStructures v0.18.13
  [e2d170a0] DataValueInterfaces v1.0.0
  [244e2a9f] DefineSingletons v0.1.2
  [b429d917] DensityInterface v0.4.0
  [163ba53b] DiffResults v1.0.3
  [b552c78f] DiffRules v1.11.0
  [b4f34e82] Distances v0.10.7
  [31c24e10] Distributions v0.25.66
  [33d173f1] DocSeeker v0.4.3
⌅ [ffbed154] DocStringExtensions v0.8.6
⌅ [e30172f5] Documenter v0.26.3
  [fa6b7ba4] DualNumbers v0.6.8
  [e2ba6199] ExprTools v0.1.8
  [e189563c] ExternalDocstrings v0.1.1
  [7a1cc6ca] FFTW v1.5.0
  [5789e2e9] FileIO v1.15.0
  [1a297f60] FillArrays v0.13.2
  [53c48c17] FixedPointNumbers v0.8.4
  [08572546] FlameGraphs v0.2.10
  [41a02a25] Folds v0.2.8
  [1fa38f19] Format v1.3.2
  [59287772] Formatting v0.4.2
  [f6369f11] ForwardDiff v0.10.32
  [de31a74c] FunctionalCollections v0.5.0
  [fb4132e2] FuzzyCompletions v0.5.1
⌅ [cd3eb016] HTTP v0.9.17
  [9fb69e20] Hiccup v0.2.2
  [3e5b6fbb] HostCPUFeatures v0.1.8
  [34004b35] HypergeometricFunctions v0.3.11
⌅ [b5f81e59] IOCapture v0.1.1
  [615f187c] IfElse v0.1.1
  [9b13fd28] IndirectArrays v1.0.0
  [83e8ac13] IniFile v0.5.1
  [22cec73e] InitialValues v0.3.1
  [842dd82b] InlineStrings v1.1.4
  [3587e190] InverseFunctions v0.1.7
  [41ab1584] InvertedIndices v1.1.0
  [92d709cd] IrrationalConstants v0.1.1
  [82899510] IteratorInterfaceExtensions v1.0.0
  [1019f520] JLFzf v0.1.5
  [692b3bcd] JLLWrappers v1.4.1
  [682c06a0] JSON v0.21.3
⌅ [98e50ef6] JuliaFormatter v0.13.7
  [aa1ae85d] JuliaInterpreter v0.9.15
  [e5e0dc1b] Juno v0.8.4
  [7c4cb9fa] LNR v0.2.1
  [10f19ff3] LayoutPointers v0.1.10
  [50d2b5c4] Lazy v0.15.1
⌅ [1d6d02ad] LeftChildRightSiblingTrees v0.1.3
  [2ab3a3ac] LogExpFunctions v0.3.17
  [6f1432cf] LoweredCodeUtils v2.2.2
  [da04e1cc] MPI v0.19.2
  [3da0fdf6] MPIPreferences v0.1.3
  [1914dd2f] MacroTools v0.5.9
  [d125e4d3] ManualMemory v0.1.8
  [739be429] MbedTLS v1.1.3
  [eff96d63] Measurements v2.7.2
  [e89f7d12] Media v0.5.0
  [128add7d] MicroCollections v0.1.2
  [e1d29d7a] Missings v1.0.2
  [78c3b35d] Mocking v0.7.3
  [5cb8414e] ModuleInterfaceTools v1.0.1
  [0987c9cc] MonteCarloMeasurements v1.0.10
  [77ba4419] NaNMath v1.0.1
  [510215fc] Observables v0.5.1
  [6fe1bfb0] OffsetArrays v1.12.7
  [5fb14364] OhMyREPL v0.5.12
  [bac558e1] OrderedCollections v1.4.1
  [90014a1f] PDMats v0.11.16
  [d96e819e] Parameters v0.12.3
  [69de0a69] Parsers v2.3.2
  [fa939f87] Pidfile v1.3.0
  [b98c9c47] Pipe v1.3.0
  [2dfb63ee] PooledArrays v1.4.2
  [21216c6a] Preferences v1.3.0
  [08abe8d2] PrettyTables v1.3.1
  [33c8b6b6] ProgressLogging v0.1.4
  [92933f4c] ProgressMeter v1.7.2
  [1fd47b50] QuadGK v2.4.2
  [e6cf234a] RandomNumbers v1.5.3
  [3cdcf5f2] RecipesBase v1.2.1
  [189a3867] Reexport v1.2.2
  [42d2dcc6] Referenceables v0.1.2
  [ae029012] Requires v1.3.0
  [295af30f] Revise v3.4.0
  [c53c40cc] Rimu v0.8.0 `~/git/Rimu`
  [79098fc4] Rmath v0.7.0
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.33
  [6c6a2e73] Scratch v1.1.1
  [91c51154] SentinelArrays v1.3.13
⌅ [efcf1570] Setfield v0.8.2
  [a2af1166] SortingAlgorithms v1.0.1
  [276daf66] SpecialFunctions v2.1.7
  [171d559e] SplittablesBase v0.1.14
  [aedffcd0] Static v0.7.6
  [90137ffa] StaticArrays v1.5.2
  [1e83bf80] StaticArraysCore v1.0.1
  [82ae8749] StatsAPI v1.5.0
  [2913bbd2] StatsBase v0.33.21
  [4c63d2b9] StatsFuns v1.0.1
  [b5087856] StrFormat v1.0.1
  [68059f60] StrLiterals v1.1.0
⌅ [88034a9c] StringDistances v0.10.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.7.0
  [5d786b92] TerminalLoggers v0.1.5
  [24d252fe] ThreadedScans v0.1.0
  [ac1d9e8a] ThreadsX v0.1.10
  [f269a46b] TimeZones v1.9.0
⌅ [0796e94c] Tokenize v0.5.21
  [3bb67fe8] TranscodingStreams v0.9.6
  [28d57a85] Transducers v0.4.73
  [a2a6695c] TreeViews v0.3.0
  [30578b45] URIParser v0.4.1
  [5c2747f8] URIs v1.4.0
  [3a884ed6] UnPack v1.0.2
  [3d5dd08c] VectorizationBase v0.21.46
  [0f1e0344] WebIO v0.8.18
  [104b5d7c] WebSockets v1.5.9
  [cc8bc4a8] Widgets v0.6.6
  [700de1a5] ZygoteRules v0.2.2
  [f5851436] FFTW_jll v3.3.10+0
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+2
  [5ced341a] Lz4_jll v1.9.3+0
  [856f044c] MKL_jll v2022.0.0+0
  [7cb0a576] MPICH_jll v4.0.2+4
  [9237b28f] MicrosoftMPI_jll v10.1.3+2
  [fe0851c0] OpenMPI_jll v4.1.3+2
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.3.0+0
  [3161d3a3] Zstd_jll v1.5.2+0
⌅ [214eeab7] fzf_jll v0.29.0+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.3
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.8.0
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.0
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] LibCURL_jll v7.84.0+0
  [29816b5a] LibSSH2_jll v1.10.2+0
  [c8ffd9c3] MbedTLS_jll v2.28.0+0
  [14a3606d] MozillaCACerts_jll v2022.2.1
  [4536629a] OpenBLAS_jll v0.3.20+0
  [05823500] OpenLibm_jll v0.8.1+0
  [83775a58] Zlib_jll v1.2.12+3
  [8e850b90] libblastrampoline_jll v5.1.1+0
  [8e850ede] nghttp2_jll v1.48.0+0
  [3f19e933] p7zip_jll v17.4.0+0
Info Packages marked with ⌅ have new versions available but cannot be upgraded. To see why use `status --outdated -m`

julia> versioninfo()
Julia Version 1.8.0-rc4
Commit 7853436ccd7 (2022-08-08 07:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin21.5.0)
  CPU: 8 × Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores
Environment:
  JULIA_MPI_PATH = /usr/local/Cellar/mpich/3.3.1/

Random numbers and threading

Is the way we use thread-local still safe and consistent with changes in Julia, in particular with the task-local rngs in Julia v1.7?

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Compatibility with RandomNumbers v1.5.0

Seeding the RNG fails with the latest version (v1.5.0) of RandomNumbers.jl.

Documentation: consistently denote Fock-space operators with hats

All Fock-space operators should have hats.

So far docstrings have not used hats on canonical creation and destruction operators and used hats inconsistently for number operators. All should have hats, and unicode hats (access with "a\hattab" are preferred. Also, use unicode daggers if possible in order to improve legibility for terminal-based output where full LaTeX rendering is unavailable.

E.g.

a_i^\\dagger -> â_i^†
n_i(n_i-1) -> n̂_i(n̂_i-1)

`lomc!` test randomly failing

Sometimes, this happens:

┌ Error: population is dead. Aborting.
└ @ Rimu ~/work/Rimu.jl/Rimu.jl/src/lomc.jl:385
┌ Error: Attempted blocking on an empty vector.
└ @ Rimu.StatsTools ~/work/Rimu.jl/Rimu.jl/src/StatsTools/blocking.jl:146
Threading: Test Failed at /home/runner/work/Rimu.jl/Rimu.jl/test/lomc.jl:500
  Expression: ≈(E5, E6, rtol = 0.1)
   Evaluated: -16.25265259644408 ≈ 0.0 (rtol=0.1)

Rerunning the test fixes the issue.

Segfault on Maui

I am running into the following error on the NeSI Maui cluster when I use the G2Correlator with replica. It is reproducible and happens only on Maui, job works fine on both NeSI Mahuika and the new CTCP cluster.

signal (11): Segmentation fault
in expression starting at /scale_wlg_nobackup/filesets/nobackup/massey02373/Ray/maui_jobs/polaron/M50N20/single-impurity/V0.5/G2/P0/ros-new.jl:59
jl_gc_pool_alloc at /buildworker/worker/package_linux64/build/src/gc.c:1217
jl_gc_alloc_ at /buildworker/worker/package_linux64/build/src/julia_internal.h:285 [inlined]
jl_gc_alloc at /buildworker/worker/package_linux64/build/src/gc.c:3283
jl_gc_alloc_buf at /buildworker/worker/package_linux64/build/src/julia_internal.h:319 [inlined]
array_resize_buffer at /buildworker/worker/package_linux64/build/src/array.c:722
jl_array_grow_at_end at /buildworker/worker/package_linux64/build/src/array.c:911 [inlined]
jl_array_grow_end at /buildworker/worker/package_linux64/build/src/array.c:975
_growend! at ./array.jl:884 [inlined]
resize! at ./array.jl:1104 [inlined]
resize! at ./compiler/ssair/ir.jl:190
resize! at ./compiler/ssair/ir.jl:1081
batch_inline! at ./compiler/ssair/inlining.jl:531
ssa_inlining_pass! at ./compiler/ssair/inlining.jl:71
jfptr_ssa_inlining_passNOT._8455.clone_1 at /scale_wlg_persistent/filesets/opt_nesi/XC50_sles12_skl/Julia/1.6.0-CrayGNU-19.04/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
run_passes at ./compiler/ssair/driver.jl:129
optimize at ./compiler/optimize.jl:272 [inlined]
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:209
typeinf_ext at ./compiler/typeinfer.jl:892
typeinf_ext_toplevel at ./compiler/typeinfer.jl:925
typeinf_ext_toplevel at ./compiler/typeinfer.jl:921
jfptr_typeinf_ext_toplevel_11290.clone_1 at /scale_wlg_persistent/filesets/opt_nesi/XC50_sles12_skl/Julia/1.6.0-CrayGNU-19.04/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
jl_type_infer at /buildworker/worker/package_linux64/build/src/gf.c:298
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:340
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1970
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:2236 [inlined]
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2229 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
dot at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/RMPI/mpidata.jl:142
replica_stats at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/strategies_and_params/replicastrategy.jl:76
#lomc!#91 at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/lomc.jl:243
lomc! at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/lomc.jl:220
unknown function (ip: 0x2aaad175d3f6)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
#lomc!#90 at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/lomc.jl:216
lomc!##kw at /home/mingrui.yang/FCIQMC/Ray/git/Rimu.jl/src/lomc.jl:215
unknown function (ip: 0x2aaad1735651)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
top-level scope at ./timing.jl:287
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:871
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:825
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:929
eval at ./boot.jl:360 [inlined]
include_string at ./loading.jl:1094
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
_include at ./loading.jl:1148
include at ./Base.jl:386
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
exec_options at ./client.jl:285
_start at ./client.jl:485
jfptr__start_41020.clone_1 at /scale_wlg_persistent/filesets/opt_nesi/XC50_sles12_skl/Julia/1.6.0-CrayGNU-19.04/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
true_main at /buildworker/worker/package_linux64/build/src/jlapi.c:560
repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at /buildworker/worker/package_linux64/build/cli/loader_exe.c:51
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at /opt/nesi/XC50_sles12_skl/Julia/1.6.0-CrayGNU-19.04/bin/julia (unknown line)
Allocations: 40167670 (Pool: 40158954; Big: 8716); GC: 43
srun: error: nid00443: task 17: Segmentation fault
srun: Terminating job step 2008063.0
slurmstepd: error: *** STEP 2008063.0 ON nid00443 CANCELLED AT 2021-06-07T23:17:49 ***

`lomc!` fails with `LogUpdate`

s_strat_2 = LogUpdate(ζ)
df2, st2 = lomc!(ham, copy(svec); s_strat = s_strat_2, params = p2, post_step, maxlength=200)

LoadError: type LogUpdate has no field targetwalkers
type LogUpdate has no field targetwalkers

Stacktrace:
 [1] getproperty(x::LogUpdate, f::Symbol)
   @ Base ./Base.jl:33
 [2] default_working_memory
   @ ~/git/Rimu/src/lomc.jl:149 [inlined]
 [3] Rimu.QMCState(hamiltonian::HubbardReal1D{Float64, BoseFS{10, 10, BitString{19, 1}}, 6.0, 1.0}, v::DVec{BoseFS{10, 10, BitString{19, 1}}, Int64, IsStochasticInteger{Int64}, Dict{BoseFS{10, 10, BitString{19, 1}}, Int64}}; laststep::Nothing, dτ::Nothing, threading::Symbol, wm::Nothing, params::RunTillLastStep{Float64}, s_strat::LogUpdate, r_strat::ReportDFAndInfo, τ_strat::ConstantTimeStep, m_strat::NoMemory, replica::NoStats{1}, post_step::ProjectedEnergy{HubbardReal1D{Float64, BoseFS{10, 10, BitString{19, 1}}, 6.0, 1.0}, Rimu.DictVectors.FrozenDVec{BoseFS{10, 10, BitString{19, 1}}, Int64}, Rimu.DictVectors.FrozenDVec{BoseFS{10, 10, BitString{19, 1}}, Float64}}, maxlength::Int64)
   @ Rimu ~/git/Rimu/src/lomc.jl:93
 [4] lomc!(ham::HubbardReal1D{Float64, BoseFS{10, 10, BitString{19, 1}}, 6.0, 1.0}, v::DVec{BoseFS{10, 10, BitString{19, 1}}, Int64, IsStochasticInteger{Int64}, Dict{BoseFS{10, 10, BitString{19, 1}}, Int64}}; df::DataFrames.DataFrame, kwargs::Base.Iterators.Pairs{Symbol, Any, NTuple{4, Symbol}, NamedTuple{(:s_strat, :params, :post_step, :maxlength), Tuple{LogUpdate, RunTillLastStep{Float64}, ProjectedEnergy{HubbardReal1D{Float64, BoseFS{10, 10, BitString{19, 1}}, 6.0, 1.0}, Rimu.DictVectors.FrozenDVec{BoseFS{10, 10, BitString{19, 1}}, Int64}, Rimu.DictVectors.FrozenDVec{BoseFS{10, 10, BitString{19, 1}}, Float64}}, Int64}}})
   @ Rimu ~/git/Rimu/src/lomc.jl:225
 [5] top-level scope
   @ In[27]:2
 [6] eval
   @ ./boot.jl:360 [inlined]
 [7] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
   @ Base ./loading.jl:1116

Incorrect info in `lomc!()` docstring

Rimu.jl/src/lomc.jl

Line 210 in 2c8bca2

 * `s_strat::ShiftStrategy = DoubleLogUpdate(targetwalkers = 100, ζ = 0.08, ξ = ζ^2/4)` - 

Here targetwalkers = 100 should be targetwalkers = 1000 to be consistent with what is defned in shiftstrategy.jl:

Rimu.jl/src/strategies_and_params/shiftstrategy.jl

Line 94 in 2c8bca2

function DoubleLogUpdate(;targetwalkers = 1000, ζ = 0.08, ξ = ζ^2/4)

MPI one sided communication fails sometimes

MPI operations with mpi_one_sided() fails under certain conditions. While
mpirun -np 4 julia script_mpi_minimum.jl
runs fine with the test script in Rimu/test/, errors seem to crop up consistently when walker number and number of time steps are simultaneously ramped up.

Two patterns show up:

┌ Info: running on 4 ranks with  4 * 2  initial walkers
└   wn = 8.0

signal (11): Segmentation fault
in expression starting at /home/brand/tmp/script_mpi_minimum.jl:34
PMPI_Put at /usr/lib/libmpi.so (unknown line)
Put at /home/brand/.julia/packages/MPI/26D0s/src/onesided.jl:170
put at /home/brand/git/Rimu/src/RMPI.jl:317 [inlined]
sort_into_targets! at /home/brand/git/Rimu/src/RMPI.jl:473

and another version without segmentation fault:

┌ Info: running on 4 ranks with  4 * 2  initial walkers
└   wn = 8.0
[it075741:27266] *** An error occurred in MPI_Put
[it075741:27266] *** reported by process [4124508161,2]
[it075741:27266] *** on win pt2pt window 3
[it075741:27266] *** MPI_ERR_TYPE: invalid datatype
[it075741:27266] *** MPI_ERRORS_ARE_FATAL (processes in this win will now abort,
[it075741:27266] ***    and potentially your MPI job)
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------

signal (15): Terminated

Last, this happened when increasing

N = 10_000
laststep = 10_000

This error was observed after #9 was fixed by upgrading to MPI.jl v0.17.2.

MPI fails across multiple HPC nodes

Currently the code fails to run on more than one nodes on the NeSI Mahuika cluster. The error code looks like this:

mlx5: wbn235: got completion with error:
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00008813 08031011 0008acd2
mlx5: wbn011: got completion with error:
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00008813 080287c8 0008e1d2

Most likely this is not a Rimu problem. Need to look into it more.

The mpi implementation used is on NeSI is: impi/2019.6.166-GCC-9.2.0

Bug in `single_particle_density` with `CompositeFS`

julia> PDVec(CompositeFS(BoseFS((1,2,3)), BoseFS((0,1,0)))=>1)|>single_particle_density
ERROR: MethodError: no method matching +(::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, ::Rimu.MultiScalar{Tuple{Float64, Float64, Float64}})

Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...)
   @ Base operators.jl:578
  +(::Union{InitialValues.NonspecificInitialValue, InitialValues.SpecificInitialValue{typeof(+)}}, ::Any)
   @ InitialValues ~/.julia/packages/InitialValues/OWP8V/src/InitialValues.jl:154
  +(::Union{Tuple, StaticArraysCore.SVector}, ::Rimu.MultiScalar)
   @ Rimu ~/.julia/packages/Rimu/7bF5b/src/helpers.jl:51
  ...

Stacktrace:
  [1] next
    @ ~/.julia/packages/Transducers/KcCBR/src/core.jl:487 [inlined]
  [2] next
    @ ~/.julia/packages/Transducers/KcCBR/src/core.jl:794 [inlined]
  [3] next
    @ ~/.julia/packages/Transducers/KcCBR/src/core.jl:287 [inlined]
  [4] next
    @ ~/.julia/packages/Transducers/KcCBR/src/library.jl:54 [inlined]
  [5] next
    @ ~/.julia/packages/Transducers/KcCBR/src/library.jl:290 [inlined]
  [6] next(rf::Transducers.Reduction{Transducers.NoComplete, Transducers.Reduction{Transducers.Filter{ComposedFunction{typeof(!), typeof(isempty)}}, Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(+)}}}}}, result::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, input::Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/core.jl:777
  [7] foldlargs
    @ ~/.julia/packages/Transducers/KcCBR/src/core.jl:181 [inlined]
  [8] __foldl__
    @ ~/.julia/packages/Transducers/KcCBR/src/processes.jl:176 [inlined]
  [9] foldl_basecase
    @ ~/.julia/packages/Transducers/KcCBR/src/processes.jl:361 [inlined]
 [10] _reduce_basecase(rf::Transducers.Reduction{Transducers.Filter{ComposedFunction{typeof(!), typeof(isempty)}}, Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(+)}}}}, init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, reducible::Transducers.SizedReducible{Tuple{Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}, Int64})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/threading_utils.jl:58
 [11] _reduce(ctx::Transducers.CancellableDACContext, rf::Transducers.Reduction{Transducers.Filter{ComposedFunction{typeof(!), typeof(isempty)}}, Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(+)}}}}, init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, reducible::Transducers.SizedReducible{Tuple{Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}, Int64})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:139
 [12] _reduce(ctx::Transducers.CancellableDACContext, rf::Transducers.Reduction{Transducers.Filter{ComposedFunction{typeof(!), typeof(isempty)}}, Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(+)}}}}, init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, reducible::Transducers.SizedReducible{Tuple{Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}, Int64})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:148
 [13] _reduce(ctx::Transducers.CancellableDACContext, rf::Transducers.Reduction{Transducers.Filter{ComposedFunction{typeof(!), typeof(isempty)}}, Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(+)}}}}, init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, reducible::Transducers.SizedReducible{NTuple{5, Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}, Int64})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:148
 [14] _transduce_assoc_nocomplete
    @ ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:131 [inlined]
 [15] transduce_assoc(xform::Transducers.IdentityTransducer, step::Transducers.Completing{typeof(+)}, init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}}, coll0::Transducers.Eduction{Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(BangBang.push!!)}}}, Base.Iterators.Filter{ComposedFunction{typeof(!), typeof(isempty)}, NTuple{5, Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}}}; simd::Val{false}, basesize::Nothing, stoppable::Nothing, nestlevel::Nothing)
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:108
 [16] transduce_assoc
    @ ~/.julia/packages/Transducers/KcCBR/src/reduce.jl:84 [inlined]
 [17] transduce
    @ ~/.julia/packages/Transducers/KcCBR/src/executors.jl:152 [inlined]
 [18] fold(rf::typeof(+), coll::Transducers.Eduction{Transducers.Reduction{Transducers.Map{Rimu.DictVectors.var"#61#62"{Rimu.var"#23#25"{Int64}, typeof(+), Rimu.DictVectors.PDVecIterator{typeof(pairs), Pair{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}, PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}}}}, Transducers.BottomRF{Transducers.Completing{typeof(BangBang.push!!)}}}, Base.Iterators.Filter{ComposedFunction{typeof(!), typeof(isempty)}, NTuple{5, Dict{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64}}}}, exec::Transducers.ThreadedEx{NamedTuple{(), Tuple{}}}; init::Rimu.MultiScalar{Tuple{Int64, Int64, Int64}})
    @ Transducers ~/.julia/packages/Transducers/KcCBR/src/executors.jl:157
 [19] fold
    @ ~/.julia/packages/Transducers/KcCBR/src/executors.jl:157 [inlined]
 [20] #mapreduce#6
    @ ~/.julia/packages/Folds/ZayPF/src/reduce.jl:26 [inlined]
 [21] mapreduce
    @ ~/.julia/packages/Folds/ZayPF/src/reduce.jl:19 [inlined]
 [22] #mapreduce#5
    @ ~/.julia/packages/Folds/ZayPF/src/reduce.jl:10 [inlined]
 [23] mapreduce
    @ ~/.julia/packages/Folds/ZayPF/src/reduce.jl:10 [inlined]
 [24] #mapreduce#60
    @ ~/.julia/packages/Rimu/7bF5b/src/DictVectors/pdvec.jl:511 [inlined]
 [25] mapreduce
    @ ~/.julia/packages/Rimu/7bF5b/src/DictVectors/pdvec.jl:510 [inlined]
 [26] single_particle_density(dvec::PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}; component::Int64)
    @ Rimu ~/.julia/packages/Rimu/7bF5b/src/strategies_and_params/poststepstrategy.jl:275
 [27] single_particle_density
    @ ~/.julia/packages/Rimu/7bF5b/src/strategies_and_params/poststepstrategy.jl:269 [inlined]
 [28] |>(x::PDVec{CompositeFS{2, 7, 3, Tuple{BoseFS{6, 3, BitString{8, 1, UInt8}}, BoseFS{1, 3, BitString{3, 1, UInt8}}}}, Int64, 5, IsStochasticInteger{Int64}, Rimu.DictVectors.NonInitiator, Rimu.DictVectors.NotDistributed}, f::typeof(single_particle_density))
    @ Base ./operators.jl:907
 [29] top-level scope
    @ REPL[12]:1

julia> Rimu.PACKAGE_VERSION
v"0.10.2"

Saving `DVec`s and restarting long calculations from file

Rimu is providing the functions save_dvec and load_dvec for saving DVecs to disk. Apparently the package BSON.jl on newer Julia versions ≥ 1.8. Consequently these function should no longer be used. See

Rimu.jl/test/runtests.jl

Line 93 in 8f91d7d

# BSON is currently broken on 1.8

An alternative would be to use Serialization.serialize or JLD2.jl.

However, we should also think of a mechanism that would allow us to restart longer calculations that ran out of time, crashed due to power cuts or other emergency shutdowns, etc. For this we would need a robust way to save the current state of a long calculation on regular time intervals, and restarting from such data.

A related issue is the saving of metadata for a MC calculation to file. This would be useful for restarting calculations, but could also be useful for automated data analysis and plotting scripts, as the dataframes produced by lomc! currently lack information about most of the calculation parameters.

Potential bug in Bernoulli spawning

In a private conversation, Prof. Jiang-Min Zhang (@wdlang06) pointed out that the Bernoulli spawning implementation does not seem right.

Rimu.jl/src/StochasticStyles/spawning.jl

Line 322 in 12b61c2

if rand() > prob

where it should be rand() < prob.

I do know that the Bernoulli strategy is somewhat experimental, and WithReplacement() is the default spawning strategy (correct me if I am wrong). It is still worth checking though.