maxim5 / hyper-engine Goto Github PK

Hello， thanks for your great work, I found all examples implemented in TF codes, Now I want to optimize hyper-parameters on Pytorch. How do I use it?

Hello， thanks for your great work, I found all examples implemented in TF codes, Now I want to optimize hyper-parameters on Pytorch. How do I use it?

tensorflow (optional)

__init__ -> impl.tensorflow.__init__ -> impl.tensorflow.tensorflow_model_io -> tensorflow

Can you please add one example for DNN hyper-parameter tuning?

I am just a little confused.

When i changed the neural network setting with concrete parameters, because i just only want to optimize learning rate, the second training will use a new learning rate to train the neural network.

For example, after the first training, i got a final accuracy=0.8172.
And then, it will go for a second training with a new learning rate. But the final accuracy maybe be very bad.
I thought, after one training, the second training will use the old results to train this model, and then get a better result. But actually, it may get a worse result.

So i want know, how do this system choose the next learning rate?
Thank you so much.

Hello, can you please let me know the advantages and disadvantages of this library over Hyperopt? Also, it would be great if you can add support for PyTorch. Thank you very much.

hello,

i am trying to implement your example with 2_1_cnn_mnist.py.
and i want to add some layers.

like this:
layer = conv_layer(x, params.conv[0])
layer = conv_layer(layer, params.conv[1])
layer = conv_layer(layer, params.conv[2])
layer = dense_layer(layer, params.dense)
logits = tf.layers.dense(inputs=layer, units=10)

conv = [
# Layer 1
hype.spec.new(
filter_num = hype.spec.choice([32]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
# Layer 2
hype.spec.new(
filter_num = hype.spec.choice([32]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
# Layer 3
hype.spec.new(
filter_num = hype.spec.choice([64]),
filter_size = [hype.spec.choice([5])] * 2,
activation = hype.spec.choice(ACTIVATIONS.keys()),
batch_norm = hype.spec.random_bool(),
dropout = hype.spec.uniform(0.0, 0.0),
),
]

i just add one layer and one params.con[]

so i got this issue:
Traceback (most recent call last):
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/examples/2_1_cnn_mnist_try.py", line 118, in
tuner.tune()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/model/hyper_tuner.py", line 45, in tune
point = self._strategy.next_proposal()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/strategy.py", line 150, in next_proposal
return self._maximizer.compute_max_point()
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/maximizer.py", line 41, in compute_max_point
values = self._utility.compute_values(batch)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/utility.py", line 132, in compute_values
mu, sigma = self.mean_and_std(batch)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/utility.py", line 64, in mean_and_std
k_star = np.swapaxes(self.kernel.compute(self.points, batch), 0, 1)
File "/Users/dongyijie/Downloads/hyper-engine/hyperengine/bayesian/kernel.py", line 57, in compute
dist = cdist(batch_x, batch_y, **self._params)
File "/Users/dongyijie/Downloads/hyper-engine/venv/lib/python3.7/site-packages/scipy/spatial/distance.py", line 2721, in cdist
raise ValueError('XA and XB must have the same number of columns '
ValueError: XA and XB must have the same number of columns (i.e. feature dimension.)

so how can i fix it?
thank you so much.

Hi , i am from Stackoverflow. I am trying to understand your implementation from the paper " Extrapolating of Learning Curve .. ". As far as i understand , they use 11 different mathematic model to fit the learning curve and then predict with monte carlo estimator. But i can't find in your code where you built these model and where the monte carlo calculation are. Can you please clarify it ? Thanks

Hello! First of all, thank you for your work, it is really helpful! I am experimenting with code from 1_3_saving_best_models_mnist.py . Tell me, please, how can I use fine-tuned models directly from Tensorflow? I am asking, because I don't see any Tensorflow variable declaration in this code. How to do that easily?

Hi! Thank you for making a great library.

I was wondering how can I specify different methods (utilities) when optimizing?
It seems that "UpperConfidenceBound" is the default option and I tried to specify something like strategy_params = {
'utility': 'ExpectedImprovement'
}

and some other attempts but without much success. And in the source code, I have not found how I can do this. Could you kindly provide some explanation on this?

Thank you very much in advance!

I found this project from your post on stackoverflow. I'd like to experiment with it, but I'm unclear on how to use it - there isn't any obvious documentation describing the use of hyper-engine.

The README says hyper-engine provides a tensorflow binding. Depending on what that binding entails, it may or may not accommodate my use-case.

system:
platform: win10 1709
anaconda: 5.1.0
python:3.6.4
tensorflow:1.8
The following error occurred while installing the hyper-engine:

(base) C:\Users\Samantha>pip install hyperengine
Collecting hyperengine
  Using cached https://files.pythonhosted.org/packages/d7/de/cc05d99e18ddb74012bf5d5ec8f7932fd5d667a5373c576d26dfad6f598a/hyperengine-0.1.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\setup.py", line 21, in <module>
        long_description = read('README.rst'),
      File "C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\setup.py", line 9, in read
        return open(os.path.join(os.path.dirname(__file__), file_)).read()
    UnicodeDecodeError: 'gbk' codec can't decode byte 0x93 in position 8018: illegal multibyte sequence

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Samantha\AppData\Local\Temp\pip-install-7qdfd7at\hyperengine\

Any idea about this? Thanks for your time!

How can I feed my own data instead of using mnist?
Like the example in this post
https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/5_DataManagement/build_an_image_dataset.ipynb
Thanks for any help!