Comments (6)
The function fwd_gradients_0
performs the automatic differentiation in forward mode where the Jacobain vector products (jvps) are calculated by composing two reverse mode vector Jacobian products (vjps), hence the double usage of tf.gradients
. Refer following links for more details:
renmengye/tensorflow-forward-ad#2
https://j-towns.github.io/2017/06/12/A-new-trick.html
https://en.wikipedia.org/wiki/Automatic_differentiation
from pinns.
The function fwd_gradients_0
is used to calculate the gradients involved in the nonlinear function N in Eq.(7), refer the literature. This is used in the function net_U0
which outputs the first term involved in SSE_n (Eq.(12)),
The dummy variables are optional and just used to hold the computed gradients, refer the documentation.
Additionally, note that the function fwd_gradients_1
is unused as there is no gradient term involved in the function net_U1
, which outputs the terms involved in in SSE_b (Eq.(12)),
Is it not sufficient to directly take gradients of U0 w.r.t x?
I didn't get what you mean by taking the gradient of U0
as it is used in SSE_n without any further manipulation required, see code.
from pinns.
Thank you for the reply,
What I meant was instead of using a function and dummy variable to calculate the gradients, is it not sufficient to use "tf.gradients(U0,x)"
from pinns.
Okay , Thank you , I get the approach.
But I still don't understand why we have to perform forward gradients instead of standard reverse gradients.
My problem is that I am solving the same problem with a custom Autodiff framework that only supports reverse gradients and when I call it twice using a dummy variable I am getting zero.
from pinns.
We use forward gradient because (from Wikipedia):
Forward accumulation is more efficient than reverse accumulation for functions f : ℝn → ℝm with m ≫ n as only n sweeps are necessary, compared to m sweeps for reverse accumulation.
In principle, the reverse gradient should be able to produce the desired derivatives. Take care of the arguments that are passed to your custom autodiff function which is different for the forward and reverse modes. The reverse gradient only requires one sweep of the computational graph but might be less efficient in our case.
from pinns.
Yes,
In principle, there shouldn't be any difference between forward and reverse gradients
But,
In the case of reverse gradients, i.e tf.grad(U0,X) , the output will be of shape X i.e N*1
In the case of forward gradients, as mentioned in the code, the output will be of the shape of the dummy Variable (i.e, shape of U0) i.e N*q
Hence, further construction of SSE will yield different results for fwd and reverse gradients.
Edit:
Can you please elaborate on what you mean by
Take care of the arguments that are passed to your custom autodiff function which is different for the forward and reverse modes. The reverse gradient only requires one sweep of the computational graph but might be less efficient in our case.
from pinns.
Related Issues (20)
- Don't quite understand the mechanism of encoding of PDE part
- A problem about experiment settings
- can pytorch replace tensorflow HOT 1
- Try in tensorflow2.6 by myself. But had some problems. Hope get some help! HOT 6
- New reposit: a starting step for CFD specialists
- Physics Informed Neural Networks: a starting step for CFD specialists
- burgers equation PINN idea HOT 2
- Faster response HOT 2
- error on tf.contrib.opt.ScipyOptimizerInterface HOT 4
- Darcy's flow in multiscale porous media
- Failed to process string with tex because latex could not be found HOT 1
- some improvement need to run on tensorflow v2 HOT 5
- How to add the training data? HOT 1
- Data in the vorticity file HOT 1
- Is there any code about PINN using pytorch? Can you give me a link HOT 3
- Shrodinger's equation
- encountered 'ValueError: Mat 4 mopt wrong format, byteswapping problem?' in the 'cylinder_nektar_t0_vorticity.mat' file
- AttributeError: module 'tensorflow' has no attribute 'set_random_seed' HOT 8
- How do I represent random data for each value of x?
- solving pinn
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pinns.