Run solve on vectors in NNODE_tests.jl about neuralpde.jl HOT 16 CLOSED

So yeah actually, looking again... it's user error. Two errors to be exact. And the fact that the fix "works" is why it needs to be reverted. Let me explain.

The issue is chain = Flux.Chain(Dense(1, 5, σ), Dense(5, 1)) the chain only has one output, so it's not correctly a neural network representing u(t) since u(t) = [x(t),y(t)]. So first of all, that needs to change to Flux.Chain(Dense(1, 5, σ), Dense(5, 2)). Secondly, you'll get an error because the user made the initial condition defined on a row vector, not a regular vector, because of u0 = [0.0f0 0.0f0]. So they either need to do Flux.Chain(Dense(1, 5, σ), Dense(5, 2),x->reshape(x,1,2)) or, more sensibly, use a vector. So the working code is:

using NeuralNetDiffEq
linear = (u, p, t) -> @. t - u*t
tspan = (0.0f0, 1.0f0)
u0 = [0.0f0,0.0f0]
prob = ODEProblem(linear, u0 ,tspan)
chain = Flux.Chain(Dense(1, 5, σ), Dense(5, 2))

opt = Flux.ADAM(0.1, (0.9, 0.95))
sol = solve(prob, NeuralNetDiffEq.NNODE(chain, opt), dt=1/50f0, verbose=true, abstol=1e-10, maxiters=10)

from neuralpde.jl.

@vavrines I fixed that. Now your code should work in master.

from neuralpde.jl.

@KirillZubov why broadcast there? Why are the sizes not lining up?

from neuralpde.jl.

@ChrisRackauckas we used the phi function for scalars and not for vectors. where u0 is not Number 6183315#diff-01a78f91139818e96d7b261cd47ca62fL63

from neuralpde.jl.

Yes, but why is that an issue? rand(2) * 0.1rand(2) is a perfectly fine piece of code that doesn't error and matches the calculation that phi should be doing.

from neuralpde.jl.

Why do all of these tests on Lotka-Volterra work (#46) but for some reason this fails? I think this just needs more investigating before concluding that broadcast is the right answer.

from neuralpde.jl.

@ChrisRackauckas ok, I see. I more investigating it.

from neuralpde.jl.

phi = (t,θ) -> u0 + (t-tspan[1])*chain(adapt(typeof(θ),[t]),θ) @KirillZubov Here u0 is 1 X 2 array and the other part is a vector. Which in this case is 1 element.( the chain's output should be 2 in the first place).

from neuralpde.jl.

The difference arrises due to addition of different dimensions.

from neuralpde.jl.

Let's make u0 = [0.0f0 0.0f0] into u0 = [0.0f0,0.0f0]. Still errors I think.

To be clear why I think it's important to look into this, the fact that rand(2) * 0.1rand(2) works but this line of code in the package doesn't is quite odd: it means a shape is changing somewhere. If you accidentally broadcast a vector to a row vector, rand(2) .+ rand(2)', you get a matrix. If we then are summing after that, we can still get a scalar out, but just not the scalar we were hoping to calculate. So this could be pointing to something that is a deeper bug, and .+ could "fix it" but calculate the wrong thing and give wrong gradients, so we should be careful to know this is actually the fix we need.

from neuralpde.jl.

@vavrines @ChrisRackauckas oh, I understood it just needs chain = Flux.Chain(Dense(1, 5, σ), Dense(5, 2)) output neural network should be of the same dimension as u0.

from neuralpde.jl.

I think there's no need to put an update with .+ or reshape then.

from neuralpde.jl.

yep, I will push back the changes

from neuralpde.jl.

Thx for pointing out that and all the discussions : )

from neuralpde.jl.

I wonder if we can throw a better error here, since that error is quite obscure...

from neuralpde.jl.

This was fixed and we forgot to close the issue.

from neuralpde.jl.