jutho / optimkit.jl Goto Github PK
View Code? Open in Web Editor NEWOptimKit: A blissfully ignorant Julia package for gradient optimization
OptimKit: A blissfully ignorant Julia package for gradient optimization
One other minor thing, with a sufficiently poorly conditioned problem, it seems its possible to get stuck in an infinite loop in bisect
here (at least it happened to me).
I can try and get a MWE if needed, let me know, but perhaps a simple fix is to add a maxiter
type argument there as well, or simply use the same one from the HagerZhangLineSearch ?
Hello,
Can I use ConjugateGradient with Float32 function value and gradient? I have been running calculations with Float64 and everything has been fine, but with Float32 I get the following error:
MethodError: no method matching OptimKit.LineSearchPoint(::Float32, ::Float32, ::Float64, ::ITensor, ::Float32, ::ITensor, ::ITensor)
Closest candidates are:
OptimKit.LineSearchPoint(::T, ::T, ::T, ::X, ::T, ::G, ::G) where {T<:Real, X, G} at /mnt/home/astrashko/.julia/packages/OptimKit/xpmbV/src/linesearches.jl:6
For complex optimization, there is an error for missing method isless(::ComplexF64, ::ComplexF64). Is the complex number supported in OptimKit?
Sorry to sound like a broken record and I know we discussed in #10 but there's still an infinite loop path in bisect. I confirm because I run into it sporadically, and editing this line to read if c.dฯ >= 0 || numfg>10
to break out if it tries more than 10 bisect steps fixes it for me.
No doubt I have some numerically noisy gradients causing this, and I'm happy if the package sends me a warning or even errors when bisect fails due to that, but just infinite looping really kills my analysis (eg I run 100 sims and step away from my computer for a while, only to come back and find sim 3 got stuck in an infinite loop and it never went further).
I want to cite this package in my article.
As well as TensorOperations.jl
, DOI with ZENODO is helpful.
Line 7 in 6f08a89
This is really just for my curiosity. If appropriate, can you include a comment about why not just _inner(x, v1, v2) = LinearAlgebra.dot(v1, v2)
? I can't tell if it's about accuracy, or performance, or ....
I know that LinearAlgebra.generic_norm2
does some rescaling for better numerical accuracy, and maybe dot
does not.
First, thanks a ton for this package, it really fills in exactly the missing piece for me of other Julia optimization packages that only really work with Array
. However, at the moment OptimKit seems to only work with Float64
, e.g.
julia> using OptimKit
julia> optimize(x->(x'x,2x), ones(Float32,5), LBFGS())
ERROR: MethodError: no method matching OptimKit.HagerZhangLineSearchIterator(::var"#1#2", ::typeof(OptimKit._retract), ::typeof(OptimKit._inner), ::OptimKit.LineSearchPoint{Float32,Array{Float32,1},Array{Float32,1}}, ::Array{Float32,1}, ::Float64, ::Bool, ::HagerZhangLineSearch{Float64})
Would be great if OptimKit could work with any eltype <:Real
(not just Float32
, but BigFloat
, etc...). I unfortunately don't have the time to make a proper PR, but if its helpful, I have it working with Float32 here: marius311@ce2136a where with this you can do
optimize(x->(x'x,2x), ones(Float32,5), LBFGS(gradtol=1f-8))
and it works.
It would be helpful if one could add the option of terminating the optimization procedure in case a linesearch is not able to find a new set of variables that satisfy the Wolfe conditions.
Do you think that this could be included?
This could be done by giving the information about the success of the linesearch into the finalize! function such that the user can use it to terminate the optimization in that case.
Best wishes,
Erik
I am not sure if this is intentional or not, but all the optimisation algorithms exit with an error if the initial guess was exactly at the optimum. The simplest example:
using OptimKit; alg=GradientDescent(;); fg(x)=sum(x.^2), 2.0 .*x; optimize(fg, [0.],alg);
This code terminates with an error "linesearch was not given a descent direction!". The same for the conjugate gradient algorithm. The LBFGS algorithm tries to start the linear search with d
This package's test suite generates a lot of output, often triggering the PkgEval log limit. For example: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2022-11/19/OptimKit.primary.log
It would be nice if by default the logs weren't as verbose ๐
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.