jamesyang007 / fastad Goto Github PK
View Code? Open in Web Editor NEWFastAD is a C++ implementation of automatic differentiation both forward and reverse mode.
License: MIT License
FastAD is a C++ implementation of automatic differentiation both forward and reverse mode.
License: MIT License
It was pretty straightforward to extend the example to also return a derivative with respect to the volatitliy parameter ("vega") (see here for the Rcpp-wrapped example).
But when I (somewhat mechanically) try to roll it to the time (and rate) derivatives (after taking care of the actual expression, i.e. using ad::sqrt()
and ad::exp()
) I am hitting a wall in the brace initialiser. Even in the simpler case of just adding "tau":
black_scholes.cpp: In function ‘Rcpp::NumericMatrix black_scholes(double, double, double, double, double)’:
black_scholes.cpp:66:25: error: cannot convert ‘<brace-enclosed initializer list>’ to ‘ad::core::ValueAdjView<double, ad::scl>::ptr_pack_t’ {aka ‘ad::util::PtrPack<double>’}
66 | call_expr.bind_cache({del_buf_c.data(), veg_buf_c.data(), tau_buf_c.data()});
| ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
biting with
94 | ptr_pack_t bind_cache(ptr_pack_t begin)
| ~~~~~~~~~~~^~~~~
I must be doing something obvious wrong. I glanced at the documentation and saw the richer vector and matrix valued expression but here I think I just want a larger list of scalars. Is that possible?
Change Eigen::sqrt to x.sqrt()
Currently, when LeafNodes get copied, their internal value and adjoint pointers get copied to point to the original value and adjoint. This is usually the desired behavior, but sometimes they may be pointing to garbage memory. We should be able to reset these pointers to point to its own value and adjoint.
This change should make the binding logic simpler by getting rid of the compiler check if an expression is var_view before binding.
Hey,
I am trying out your lib and it seems nice. I think it would be nice to have some convenience operators like += etc. Also I was wondering why it is not possible to use both normal floating point values in arithmetic expressions involving ad::Var? I think it would be nice to be able to do something like
Var<double> a, b{2}; a = 5. * b * b;
mat
transpose is very common in scientific computing. If we want to support that, can we write a function in reverse/core/unary.hpp
to directly transpose underneath values and adjoints?
I would like to make a PR but I'm not sure if the above idea is correct.
Vec<double> a, b {1,2,3};
auto expr = (a = b*b*b+b);
How do I get 3rd derivative of a wrt b? (d3a/db3)
I am getting the following messages when I try to compile one of the FastAD examples:
temp.cpp: In function ‘int main()’:
temp.cpp:12:22: error: no matching function for call to ‘autodiff(ad::core::ExprBind<ad::core::BinaryNode<ad::core::Add, ad::core::UnaryNode<ad::core::Sin, ad::VarView<double, ad::scl> >, ad::core::UnaryNode<ad::core::Cos, ad::VarView<double, ad::vec> > > >&)’
12 | auto f = autodiff(expr_bound);
| ~~~~~~~~^~~~~~~~~~~~
In file included from include/fastad_bits/reverse/core.hpp:7,
from include/fastad_bits/reverse.hpp:3,
from include/fastad:4,
from temp.cpp:1:
Below are the steps to reproduce this:
Step 1:
git clone https://github.com/JamesYang007/FastAD.git fastad.git
Step 2:
cd fastad.git ; ./setup.sh
Step 3:
Create the file temp.cpp with the following contents (from one of the FastAD readme examples):
#include <fastad>
#include <iostream>
int main()
{
using namespace ad;
Var<double, scl> x(2);
Var<double, vec> v(5);
auto expr_bound = bind(sin(x) + cos(v));
auto f = autodiff(expr_bound);
std::cout << x.get_adj() << std::endl;
std::cout << v.get_adj(2,0) << std::endl;
return 0;
}
Step 4:
g++ temp.cpp -o temp -I include -I libs/eigen-3.3.7/build/include/eigen3
MSVC is failing test
The + - * / operator overload of conventional data types such as double is not implemented, so it is not very convenient to use
Hi @JamesYang007!
This repo seems pretty nice, especially your philosophy as laid out in your paper of arguing the benefit of having a pair of pointers of matrices, rather than a matrix of dual numbers.
I had been using https://github.com/autodiff/autodiff for a while which overloads Eigen's scalar type (i.e. the latter approach) to use a matrix of dual numbers, and I think there are quite a bit of overhead (and cache misses) when compared to the reference function (matrix of doubles). I wanted to test out your repo, but realised that this repo had been mainly focusing on reversed mode rather than forward mode (which is the focus of https://github.com/autodiff/autodiff). Do you have any plans to make some of the implementations in https://github.com/JamesYang007/FastAD/tree/master/include/fastad_bits/reverse/core applicable to both modes? More than that, it seems like forward mode right now only works with scalar types (float/double) rather than matrix/vector type?
Finally, one huge benefit of https://github.com/autodiff/autodiff is that it is immediately applicable to a lot of existing function/solvers (since it is using custom scalar type to do autodiff), while your approach requires manual work to implements different operators (e.g. all the ad::XXX
custom operator rather than the Eigen::XXX
operator) and https://github.com/autodiff/autodiff immediately works with any function that takes a templated Eigen argument. Do you have any thoughts on that? (I had thought of a possible approach could be that we can extend MatrixBase
rather than using custom scalar type to keep track of the adjoints of the input variables.)
Can we get any example when we have an array of functions, rather than a single one. This is quite a common case, when we need to compute Jacobian. I think I have some idea how to do that, but I am not sure this would be efficient.
Currently benchmarks are using googletest framework. They should use googlebenchmark
Some functions may not be continuous or differentiable at a point, but can be made so by analytic continuation. For example, sin(x)/x is undefined at x = 0, but can be made continuous by defining the function at x = 0 to be 1.
Implement conditional-statements such that an expression may be evaluated differently depending on input x.
Remove armadillo dependency with self-implementation of matrix.
CMake, include/fastad_bits/ files, and test/ files, should not reference armadillo at all or USE_ARMA preprocessor flag.
There are many uses of CRTP such as ADExpression that can be simplified and made more robust using self-implemented Concepts.
Types to consider:
There are meta-programming tools such as is_glue_eq
, glue_size
that should be changed/moved.
Hello,
Is there any support for sparse matrices?
Thanks.
Create a new ad expression responsible for representing already-computed ad expression.
Currently, Travis CI only tests on linux (trusty distribution) with gcc-7, g++-7. We would like to test for other compilers and MacOS as well.
Is there a standard way to store a FastAD expression in a "generic" expression object?
The BaseExpr class is templated on the derived type, which means it's not suitable for use in a class that can store any FastAD expression.
Maybe a variant of the std::any concept?
To forward evaluate, feval is called, and then backward evaluation requires another feval before a beval.
Can we manage Vec such that we can separate these calls?
#18 this issue is related to current issue.
Should Var move its contents by copying base DualNum and setting the pointers current base members? Otherwise, default move ctor will make them point to de-allocated memory.
The current library only supports from C++17. It would be nice to have it support from C++14.
Is it possible to use FastAD reverse mode to compute the derivative of an algorithm
template <class Scalar> Scalar f( const std::vector<Scalar>& x)
where f(x) only uses the operators + , -, *, /, and = operations with Scalar objects.
If so, is there an example that demonstrates this ?
In practice, it may be useful to have delta-functions.
Implement delta-function.
This issue arrises in hessian
I sent these questions in an email to Dr Yang but I thought I would post here too
I have read the paper on FastAD and I am very interested in using fastAD for my algorithm
I was wondering does FastAD work with Rcpp? If so how can I install it? I think it should be possible but I just wanted to check (i'm new to C++)
I have used "autodiff" library (https://autodiff.github.io/) however I have found it to not be much faster than numerical differentiation for my application - have you used this before? I noticed in the paper you didn't benchmark against it
Also I was wondering if it possible to compute a gradient w.r.t a std::vector filled with eigen matrices? (or any other 3D or higher dimensional structure)? or will all the parameters need to be input into a vector or matrix and then reformatted back into the container needed for the rest of the model afterwards?
Is it possible to do use fastAD just within a standard function (rather than using "int main()" etc)? Im new to C++ and have just been using functions for everything (also using it through R via Rcpp)
It would be very convenient to enable CMake integration with FetchContent like so
include(FetchContent)
FetchContent_Declare(
FastAD
GIT_REPOSITORY https://github.com/JamesYang007/FastAD
GIT_TAG v3.2.1
GIT_SHALLOW TRUE
GIT_PROGRESS TRUE)
FetchContent_MakeAvailable(FastAD)
However, it would require to put FASTAD_ENABLE_TEST OFF by default in the CMakeLists.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.