Comments (3)
It would be really awesome to have automatic differentiation support for exactly the reasons you mentioned! But, unfortunately, it would be a huge amount of effort and scope creep for this project to implement that. If someone ever came along and wanted to implement an AD framework that would be great and I think we should include it (or perhaps package it as its own project), but as far as I know nobody is working on such a thing.
However, there might be an existing C++ autodiff package that can work with Armadillo matrices (and thus with ensmallen objective functions). Going off of this site, it seems like FunG and FunCy might work with Armadillo matrices? I don't see any exact examples, but maybe you could play with it and make it work, or open a Github issue there and see if you can the maintainer can provide an Armadillo example?
There may also be other AD libraries on autodiff.org that fit the bill---I am not sure. Personally I haven't [yet] needed AD for anything I've done with ensmallen.
Theoretically, if you have an objective function o(const arma::mat&)
, and an AD toolkit that can produce the Jacobian of o()
with respect to the input matrix, then you should be able to package that into a class like ensmallen requires, and use the AD-generated Jacobian as the Gradient()
implementation.
I hope this helps! Sorry we don't have a plug-and-play solution here.
from ensmallen.
Great initiative! Perhaps we could take inspiration from the new Flashlight library from FB-AI? Since they're the progenitor of this concept, I'd expect them to nail this concept in C++ perfectly. Further, I recall there were some attempts at this in the shogun library as well. In that blog, Gil said:
Nowadays, functionalities such as auto differentiation are frequently a requirement when working with kernels, see for example GPFlow. However, in Shogun we still rely on manual implementations of gradient calculations, which is both error prone and time consuming.
I think the same is true for mlpack as well. Although this means we will have to build an entire backend using graphs, treat everything as Tensor and work our way up from there. Let me know what you guys think?
from ensmallen.
This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!
from ensmallen.
Related Issues (20)
- disable building tests by default HOT 3
- Convert ensmallen from vcpkg to nuget and submit the package to nuget repository HOT 4
- [regression in 2.17.0] Tests aren't built HOT 2
- armadillo / bandicoot conv_to ambiguity HOT 6
- tests/catch.hpp update to anticipate failure to build with glibc 2.34 HOT 2
- C++20 redudant template argument disallowed in ctor HOT 4
- Debian build (test) failure on armel HOT 24
- Adam with Decay HOT 2
- Compiling errors ensmallen 02.18.02 using VS2019 armadillo.lib 10.08.02 and mlpack.lib 03.04.02 HOT 6
- The example to use the constrained function HOT 4
- AugLagrangian with constraints validation HOT 8
- Multi-objective functions both MOEA/D-DE and NSGA2 - Examples using SchafferFunctionN1 HOT 1
- Implementing Function Type for Partial Derivatives HOT 3
- Does ensmallen work with armadillo >=10.0? HOT 5
- Enable HTTPS on ensmallen.org HOT 1
- Debian build (test) failure on i386 HOT 7
- ProgressBar() does not work with L_BFGS HOT 3
- Bounds for search space HOT 5
- During AugLagrangian's optimization, L_BFGS sets coordinates to nan HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ensmallen.