maxim5 / hyper-engine Goto Github PK
View Code? Open in Web Editor NEWPython library for Bayesian hyper-parameters optimization
Home Page: https://pypi.python.org/pypi/hyperengine
License: Apache License 2.0
Python library for Bayesian hyper-parameters optimization
Home Page: https://pypi.python.org/pypi/hyperengine
License: Apache License 2.0
Hello, thanks for your great work, I found all examples implemented in TF codes, Now I want to optimize hyper-parameters on Pytorch. How do I use it?
Hello, thanks for your great work, I found all examples implemented in TF codes, Now I want to optimize hyper-parameters on Pytorch. How do I use it?
tensorflow (optional)
__init__ -> impl.tensorflow.__init__ -> impl.tensorflow.tensorflow_model_io -> tensorflow
Can you please add one example for DNN hyper-parameter tuning?
I am just a little confused.
When i changed the neural network setting with concrete parameters, because i just only want to optimize learning rate, the second training will use a new learning rate to train the neural network.
For example, after the first training, i got a final accuracy=0.8172.
And then, it will go for a second training with a new learning rate. But the final accuracy maybe be very bad.
I thought, after one training, the second training will use the old results to train this model, and then get a better result. But actually, it may get a worse result.
So i want know, how do this system choose the next learning rate?
Thank you so much.
Hello, can you please let me know the advantages and disadvantages of this library over Hyperopt? Also, it would be great if you can add support for PyTorch. Thank you very much.
hello,
i am trying to implement your example with 2_1_cnn_mnist.py.
and i want to add some layers.
like this:
layer = conv_layer(x, params.conv[0])
layer = conv_layer(layer, params.conv[1])
layer = conv_layer(layer, params.conv[2])
layer = dense_layer(layer, params.dense)
logits = tf.layers.dense(inputs=layer, units=10)
conv = [
# Layer 1
hype.spec.new(
filter_num = hype.spec.choice([32]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
# Layer 2
hype.spec.new(
filter_num = hype.spec.choice([32]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
# Layer 3
hype.spec.new(
filter_num = hype.spec.choice([64]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
]
i just add one layer and one params.con[]
so i got this issue:
Traceback (most recent call last):
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/examples/2_1_cnn_mnist_try.py", line 118, in
tuner.tune()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/model/hyper_tuner.py", line 45, in tune
point = self._strategy.next_proposal()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/strategy.py", line 150, in next_proposal
return self._maximizer.compute_max_point()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/maximizer.py", line 41, in compute_max_point
values = self._utility.compute_values(batch)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/utility.py", line 132, in compute_values
mu, sigma = self.mean_and_std(batch)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/utility.py", line 64, in mean_and_std
k_star = np.swapaxes(self.kernel.compute(self.points, batch), 0, 1)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/kernel.py", line 57, in compute
dist = cdist(batch_x, batch_y, **self._params)
File "/Users/dongyijie/Downloads/hyper-engine/venv/lib/python3.7/site-packages/scipy/spatial/distance.py", line 2721, in cdist
raise ValueError('XA and XB must have the same number of columns '
ValueError: XA and XB must have the same number of columns (i.e. feature dimension.)
so how can i fix it?
thank you so much.
Hi , i am from Stackoverflow. I am trying to understand your implementation from the paper " Extrapolating of Learning Curve .. ". As far as i understand , they use 11 different mathematic model to fit the learning curve and then predict with monte carlo estimator. But i can't find in your code where you built these model and where the monte carlo calculation are. Can you please clarify it ? Thanks
Hello! First of all, thank you for your work, it is really helpful! I am experimenting with code from 1_3_saving_best_models_mnist.py . Tell me, please, how can I use fine-tuned models directly from Tensorflow? I am asking, because I don't see any Tensorflow variable declaration in this code. How to do that easily?
Hi! Thank you for making a great library.
I was wondering how can I specify different methods (utilities) when optimizing?
It seems that "UpperConfidenceBound" is the default option and I tried to specify something like strategy_params = {
'utility': 'ExpectedImprovement'
}
and some other attempts but without much success. And in the source code, I have not found how I can do this. Could you kindly provide some explanation on this?
Thank you very much in advance!
I found this project from your post on stackoverflow. I'd like to experiment with it, but I'm unclear on how to use it - there isn't any obvious documentation describing the use of hyper-engine.
The README says hyper-engine provides a tensorflow binding. Depending on what that binding entails, it may or may not accommodate my use-case.
system:
platform: win10 1709
anaconda: 5.1.0
python:3.6.4
tensorflow:1.8
The following error occurred while installing the hyper-engine:
(base) C:\Users\Samantha>pip install hyperengine
Collecting hyperengine
Using cached https://files.pythonhosted.org/packages/d7/de/cc05d99e18ddb74012bf5d5ec8f7932fd5d667a5373c576d26dfad6f598a/hyperengine-0.1.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\setup.py", line 21, in <module>
long_description = read('README.rst'),
File "C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\setup.py", line 9, in read
return open(os.path.join(os.path.dirname(__file__), file_)).read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x93 in position 8018: illegal multibyte sequence
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\
Any idea about this? Thanks for your time!
How can I feed my own data instead of using mnist?
Like the example in this post
https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/5_DataManagement/build_an_image_dataset.ipynb
Thanks for any help!
import hyperengine
import numpy as np
def rosenbrock(hyperparams):
return (hyperparams["x"]-1)**2 + 10*(hyperparams["x"]**2-hyperparams["y"])**2
class BlackBoxSolver:
def __init__(self, func):
self.func=func
self._val_loss_curve = []
def train(self):
loss=self.func()
self._val_loss_curve.append(loss)
return self._reducer(self._val_loss_curve)
def _reducer(self, *args, **kwargs):
return np.min(*args, **kwargs)
def terminate(self):
pass
def solver_generator(hyperparams):
return BlackBoxSolver(partial(rosenbrock, hyperparams))
class IterLimitedHyperTuner(hyperengine.HyperTuner):
def __init__(self, hyper_params_spec, solver_generator, iterLimit, *args, **kwargs):
j=0
def solver_generator_limited(hyperparams):
nonlocal j
if j<iterLimit:
j+=1
return solver_generator(hyperparams)
else:
raise StopIteration()
super().__init__(hyper_params_spec, solver_generator_limited, *args, **kwargs)
def tune(self):
try:
super().tune()
except StopIteration as ex:
minLossPointNum=np.argmin(self.strategy.values)
return dict(zip(self.parsed._spec.keys(), self.strategy.points[minLossPointNum]))
spec=hp.new({
"x": hp.uniform(-10, 10),
"y": hp.uniform(-10, 10),
})
tuner=IterLimitedHyperTuner(spec, solver_generator, iterLimit=10, strategy='bayesian') #'portfolio'
tuner.tune()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.