Comments (7)
MilesCranmer
commented on May 18, 2024
Awesome to hear you are liking it! There is the recorder::Bool option in Options, which you can also turn on by setting the environment variable PYSR_RECORDER=1. Then, recorder_file::String is the filename for this (a JSON file; default is pysr_recorder.json). This JSON file records everything, even each individual mutation, which is maybe too much info, but it's helpful for debugging the performance. Does that work?
Cheers,
Miles
from symbolicregression.jl.
cobac
commented on May 18, 2024
Thanks! (btw those options are undocumented I don't know if you were aware)
Hmm... I assume that'd be quite a bit of overhead (?). But I could benchmark in different runs how the loss function evolves over time vs the actual time. Great.
However, I've done a few test runs with recorder = true (with and without recorder_file = ...) but I can't find the .json file in my filesystem. Using [8254be44] SymbolicRegression v0.6.9. I can reproduce this in a clean environment.
Thanks again.
from symbolicregression.jl.
MilesCranmer
commented on May 18, 2024
No problem. That functionality is still experimental since I wrote it for myself to debug specific use-cases of the software in a project I'm using it on, but I can see now that others would find this useful too!
However, I've done a few test runs with recorder = true (with and without recorder_file = ...) but I can't find the .json file in my filesystem. Using [8254be44] SymbolicRegression v0.6.9. I can reproduce this in a clean environment.
Sorry, I see the bug. Just pushed a fix to master. It should work now. (previously it was only working for the environment variable method)
from symbolicregression.jl.
cobac
commented on May 18, 2024
Thanks!
I see that the program is trying to save the record now. However, all runs are failing at some point with ERROR: Inf not allowed to be written in JSON spec. Si I assume the program is trying to save an equation before checking if it produces proper results. I am not familiar enough with the code base to know what's the best workaround, but I assume you must have encountered the same issue in your own testing (?)
I've tested with current master and bc7b81f
from symbolicregression.jl.
MilesCranmer
commented on May 18, 2024
I’ve never seen that before. How are you running it? Do you have a MWE?
from symbolicregression.jl.
cobac
commented on May 18, 2024
I've created this repository that reproduces the error with current master, all packages updated and Julia 1.6.1. I can reproduce the error instantiating the Manifest.toml and including the .jl file. I'm runing Linux and I've tested with 1 thread and with multiple threads, but I don't think that should be relevant in this case.
I've noticed that it always fails when the progress meter is at 60%.
This is the stacktrace I get:
ERROR: Inf not allowed to be written in JSON spec)
Stacktrace:
[1] #write#78
@ ~/.julia/packages/JSON3/PgKj8/src/write.jl:225 [inlined]
[2] write(::StructTypes.NumberType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Float64)
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:225
[3] write(::StructTypes.DictType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Dict{String, Any}; kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:155
[4] write(::StructTypes.DictType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Dict{String, Any})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:146
[5] write(::StructTypes.DictType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Dict{String, Any}; kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:155
[6] write(::StructTypes.DictType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Dict{String, Any})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:146
[7] write(::StructTypes.DictType, buf::Vector{UInt8}, pos::Int64, len::Int64, x::Dict{String, Any}; kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:155
[8] write
@ ~/.julia/packages/JSON3/PgKj8/src/write.jl:146 [inlined]
[9] write(io::IOStream, obj::Dict{String, Any}; kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:32
[10] write(io::IOStream, obj::Dict{String, Any})
@ JSON3 ~/.julia/packages/JSON3/PgKj8/src/write.jl:30
[11] (::SymbolicRegression.var"#58#90")(io::IOStream)
@ SymbolicRegression ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegression.jl:572
[12] open(::SymbolicRegression.var"#58#90", ::String, ::Vararg{String, N} where N; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base ./io.jl:330
[13] open
@ ./io.jl:328 [inlined]
[14] macro expansion
@ ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegression.jl:571 [inlined]
[15] macro expansion
@ ~/.julia/packages/SymbolicRegression/vSShw/src/Recorder.jl:8 [inlined]
[16] _EquationSearch(::SymbolicRegression.../ProgramConstants.jl.SRDistributed, datasets::Vector{SymbolicRegression.../Dataset.jl.Dataset{Float64}}; niterations::Int64, options::Options{Tuple{typeof(+), typeof(-), typeof(*), typeof(/)}, Tuple{typeof(sin), typeof(cos), typeof(exp), typeof(sq), typeof(cb)}, L2DistLoss}, numprocs::Nothing, procs::Nothing, runtests::Bool)
@ SymbolicRegression ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegression.jl:570
[17] EquationSearch(datasets::Vector{SymbolicRegression.../Dataset.jl.Dataset{Float64}}; niterations::Int64, options::Options{Tuple{typeof(+), typeof(-), typeof(*), typeof(/)}, Tuple{typeof(sin), typeof(cos), typeof(exp), typeof(sq), typeof(cb)}, L2DistLoss}, numprocs::Nothing, procs::Nothing, multithreading::Bool, runtests::Bool)
@ SymbolicRegression ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegression.jl:181
[18] EquationSearch(X::Matrix{Float64}, y::Matrix{Float64}; niterations::Int64, weig
hts::Nothing, varMap::Nothing, options::Options{Tuple{typeof(+), typeof(-), typeof(*)
, typeof(/)}, Tuple{typeof(sin), typeof(cos), typeof(exp), typeof(sq), typeof(cb)}, L
2DistLoss}, numprocs::Nothing, procs::Nothing, multithreading::Bool, runtests::Bool)
@ SymbolicRegression ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegre
ssion.jl:145
[19] #EquationSearch#24
@ ~/.julia/packages/SymbolicRegression/vSShw/src/SymbolicRegression.jl:157 [inlin
ed]
Thanks again and sorry for the bother.
from symbolicregression.jl.
MilesCranmer
commented on May 18, 2024
Thanks! Just pushed a fix in 124020d; want to try it? (set --project=[path_to_repo] to use the non-versioned one)
from symbolicregression.jl.
Related Issues (20)
- SymbolicRegression.jl paper is out!
- [Feature]: can I design a custom operator for sequence data? HOT 1
- [Bug]: Threading always seems slower than multiprocessing
- [BUG]: method error when used in the debugger HOT 20
- Should use `errormonitor` on tasks for distributed mode
- [Feature]: MLJ integration HOT 3
- [Feature]: ternary operators HOT 3
- Should use scale of data when mutating or creating constants
- [BUG]: `Options` not updated in `MLJ.fit`
- Garbage collection too passive on worker processes HOT 3
- [Feature]: Including Units Evaluating Model Performance and Controlling Iterative Models Example :MLJ HOT 10
- Whether SR.jl can Implement this ARZ algorithm ? HOT 1
- Silence warnings for Optim.jl HOT 1
- Multidimensional equations HOT 4
- How do I set up a basis function consisting of three different inputs x, y, z? HOT 2
- Unique solution of Regression HOT 1
- How can I set the maximum number of nests? HOT 4
- [Feature]: Proper caching
- [Feature]: Training on high dimensions X HOT 1
- String representation of an expression changes after running a second regression HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from symbolicregression.jl.