Comments (6)
https://docs.ray.io/en/latest/tune/index.html
from dmol-book.
a bit more far-fetched would be experiment tracking.
I'm thinking of stuff like weights and biases which 1) has tools for hyperparameter sweeps 2) tools to visualize some chemistry
from dmol-book.
as framework, I really enjoy using Optuna
from dmol-book.
@whitead Do you want to use this package (https://docs.ray.io/en/latest/tune/index.html) in this chapter?
from dmol-book.
@mehradans92 I think that comment was me sharing some existing methods used. It would be better to be as package agnostic as possible though.
from dmol-book.
@mehradans92 Read through it briefly. Looks great, a lot of work went into it! Also I can tell it will be very helpful. A few proposed changes:
- Try to look at the layers chapter once more, there is some overlapping material (e.g., dropout, regularization, hyperparameters).
- Cite some papers on learning rate schedulers and maybe add some information on momentum, since it's related. Also some have mentioned warm-start, which I'm not familiar with. Maybe mention it..
- Fig 8.2 - does it need to be a movie? Can be distracting while reading. I can see the benefit for 8.1 certainly
- Batch size - would love to get 1-2 citations here on batch size and its connection to randomness in estimating gradient
- Dropout - can you cite the paper and maybe add a bit more on where it should be added (all layers?), if it should be combined with other regularization, etc.
- It is really critical to use validation data for a hyperparameter search - otherwise you're implicitly fitting to testing data. See here. You need to strongly emphasize this point early and make sure code/examples uses the word validation, instead of test, for the search.
- On Keras, can you reduce the output level of the logging (verbose=0) so the text isn't rendered in the chapter.
- You've split up the code nicely, but it'd be great to have some discussion, maybe showing how snippets of how the methods work too, before going right into training.
from dmol-book.
Related Issues (20)
- Alt Text HOT 1
- Issue on page /dl/gnn.html HOT 1
- Diffusion Models
- VAE Chapter
- Syntax Highlighting is broken
- Issue on page /dl/Equivariant.html HOT 1
- Intro chapter
- sklearn is a dependency somewhere HOT 1
- GraphSAGE
- GIF Controls
- Error when importing dmol-book in collab HOT 2
- Issue on page /dl/xai.html HOT 1
- Cite this in XAI
- Dependency Problems from Email
- Issue on page /dl/attention.html
- V2 Translations with GPT-4
- Confusing sentence in Equivariant.ipynb.
- Bijector with relu activation?
- Question about Chapter 19: e3nn on trajectory prediction HOT 1
- Equivariant Neural Network Chapter
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dmol-book.