keras-team / autokeras Goto Github PK
View Code? Open in Web Editor NEWAutoML library for deep learning
Home Page: http://autokeras.com/
License: Apache License 2.0
AutoML library for deep learning
Home Page: http://autokeras.com/
License: Apache License 2.0
Cool project!
But while I tried to accesso https://autokeras.com/ I'll receive an SSL error from browser, the certification is issued for github only.
You may change the configuration from website's repo at:
and fill autokeras.com into that field.
Traceback (most recent call last):
File "example.py", line 50, in
test_my_layer()
File "example.py", line 39, in test_my_layer
b = WeightedAdd()(a)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 590, in call
self.build(input_shapes[0])
File "/usr/local/lib/python2.7/dist-packages/keras/layers/merge.py", line 68, in build
raise ValueError('A merge layer should be called '
ValueError: A merge layer should be called on a list of inputs.
Keras on Tensorflow 1.5
Your /examples/mnist.py does not load GPUtil like your older experiments files do (like
https://github.com/jhfjhfj1/autokeras/blob/master/experiments/mnist.py ). Does it no longer need GPUtil?
It was not on your requirements.txt list.
import os
import GPUtil
def select_gpu():
try:
# Get the first available GPU
DEVICE_ID_LIST = GPUtil.getFirstAvailable()
DEVICE_ID = DEVICE_ID_LIST[0] # grab first element from list
# Set CUDA_VISIBLE_DEVICES to mask out all other GPUs than the first available device id
os.environ["CUDA_VISIBLE_DEVICES"] = str(DEVICE_ID)
except FileNotFoundError:
print("GPU not found")
I have a dataset of houses and sale prices that I'm playing around with, is autokeras capable of using for regression problems, or will it be in the future?
autokeras.classifier.load_image_dataset failed silently
x_train, y_train = load_image_dataset(csv_file_path="***_train.csv", images_path=".")
print(x_train.shape)
print(y_train.shape)
***_train.csv contains 100k lines of image path and labels
I try to do load_image_dataset, and wish to get x_train.shape. However, it doesn't print any value.
so is the load_image_dataset not able to handle 100k data size?
How do I use the parameters? If I use
clf = ImageClassifier(verbose=True, searcher_args={'trainer_args': {'max_iter_num': 10, 'augment': False}, 'default_model_len': 3})
I get this error: TypeError: init() got an unexpected keyword argument 'searcher_args'
flake8 testing of https://github.com/jhfjhfj1/autokeras on Python 3.7.0
$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./experiments/pytorch_cifar10.py:51:7: F821 undefined name 'ResNet18'
net = ResNet18()
^
./experiments/pytorch_cifar10.py:98:9: F821 undefined name 'progress_bar'
progress_bar(batch_idx, len(trainloader), 'Loss: %.3f | Acc: %.3f%% (%d/%d)'
^
./experiments/pytorch_cifar10.py:118:13: F821 undefined name 'progress_bar'
progress_bar(batch_idx, len(testloader), 'Loss: %.3f | Acc: %.3f%% (%d/%d)'
^
3 F821 undefined name 'ResNet18'
3
Trying to create an image classifier with ~1000 training samples and 7 classes but it throws a runtime error. Is there a way of reducing batch size or something else that can be done to circumvent this?
Following is the error.
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58/usr/lib/python3.5/multiprocessing/semaphore_tracker.py:129: UserWarning: semaphore_tracker: There appear to be 2 leaked semaphores to clean up at shutdown len(cache))
The example code on MNIST showed an issue on multiprocessing with CUDA.
I guess CUDA doesn't work well with the multiprocessing https://pytorch.org/docs/stable/notes/windows.html
I use Windows10, Python 3.6.5, CUDA 9.0, and tensorflow-gpu==1.8.0. I followed requirements.txt to install others.
Using commands "torch.version.cuda" and "torch.cuda.is_available()", I also checked CUDA 9.0 works well on my environment.
...........................................
Epoch 1: loss 1.8169708251953125, metric_value 0.987
...........................................
Epoch 2: loss 1.694014310836792, metric_value 0.9866
...........................................
Epoch 3: loss 1.5495505332946777, metric_value 0.9884
...........................................
Epoch 4: loss 1.7369670867919922, metric_value 0.9882
...........................................
Epoch 5: loss 2.2431726455688477, metric_value 0.9842
...........................................
Epoch 6: loss 1.821488618850708, metric_value 0.9864
...........................................
Epoch 7: loss 1.3172129392623901, metric_value 0.9908
...........................................
Epoch 8: loss 1.3580961227416992, metric_value 0.9906
...........................................
Epoch 9: loss 1.2147481441497803, metric_value 0.992
...........................................
Epoch 10: loss 1.3565152883529663, metric_value 0.9916
...........................................
Epoch 11: loss 1.201681137084961, metric_value 0.9916
...........................................
Epoch 12: loss 1.9307321310043335, metric_value 0.987
...........................................
Epoch 13: loss 1.2660953998565674, metric_value 0.993
...........................................
Epoch 14: loss 1.2389497756958008, metric_value 0.9924
...........................................
Epoch 15: loss 1.3451954126358032, metric_value 0.992
...........................................
Epoch 16: loss 2.506535291671753, metric_value 0.9844
...........................................
Epoch 17: loss 2.018087387084961, metric_value 0.989
...........................................
Epoch 18: loss 1.1533660888671875, metric_value 0.9934
...........................................
Epoch 19: loss 1.493323802947998, metric_value 0.9908
No loss decrease after 5 epochs
THCudaCheck FAIL file=c:\programdata\miniconda3\conda-bld\pytorch_1533090623466\work\torch\csrc\generic\StorageSharing.cpp line=231 error=63 : OS call failed or operation not supported on this OS
Traceback (most recent call last):
File "ak_mnist.py", line 10, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
File "C:\Users\kmbmjn\Anaconda3\lib\site-packages\autokeras\image_supervised.py", line 238, in fit
run_searcher_once(train_data, test_data, self.path, int(time_remain))
File "C:\Users\kmbmjn\Anaconda3\lib\site-packages\autokeras\image_supervised.py", line 41, in run_searcher_once
searcher.search(train_data, test_data, timeout)
File "C:\Users\kmbmjn\Anaconda3\lib\site-packages\autokeras\search.py", line 189, in search
metric_value, loss, graph = train_results.get(timeout=remaining_time)[0]
File "C:\Users\kmbmjn\Anaconda3\lib\multiprocessing\pool.py", line 644, in get
raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '[(0.9899199999999999, tensor(1.7033, device='cuda:0'), <autokeras.graph.Graph object at 0x00000174DB1BC7B8>)]'. Reason: 'RuntimeError('cuda runtime error (63) : OS call failed or operation not supported on this OS at c:\programdata\miniconda3\conda-bld\pytorch_1533090623466\work\torch\csrc\generic\StorageSharing.cpp:231',)'
Please add a link to the docs at http://autokeras.com/ in the readme of this repo. Thanks.
Describe which feature do you want to add to this project
Visual Regression would allow this project to search for an architecture adept at predicting values for images, rather than classifying them into categories.
Explain why this feature is needed
This would allow use cases like the following to work:
Describe the solution you'd like
Add an ImageRegressor
in a similar style as ImageClassifier
.
Describe alternatives you've considered
You could merge the classifier and regressor into one 'Visual' category, allowing other categories (e.g. 'Text', 'Audio') to come later.
Additional context
N/A
I see ImageClassifier
in the doc. Is there something like TextClassifier
to process text?
pip install autokeras Collecting autokeras Using cached https://files.pythonhosted.org/packages/cf/3f/ef52a7654445180ac174b6c159e44261e4fa4161bb28973229845deeedee/autokeras-0.2.0.tar.gz Collecting torch==0.4.0 (from autokeras) Could not find a version that satisfies the requirement torch==0.4.0 (from autokeras) (from versions: 0.1.2, 0.1.2.post1) No matching distribution found for torch==0.4.0 (from autokeras)
Windows 10
Python 3.6.2
Object detection is one of the most important research areas and applications in Computer Vision, such as face detection, vehicle detection, people counting. This module will provide easy access to people or group who needs to perform object detection in their work with limited Computer Vision or Machine Learning background.
Some add-ons on current AutoKeras to be considered:
Traceback (most recent call last):
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/search.py", line 327, in train
verbose).train_model(**trainer_args)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/utils.py", line 122, in train_model
self._train(train_loader, epoch)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/utils.py", line 143, in _train
outputs = self.model(inputs)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/graph.py", line 603, in forward
temp_tensor = torch_layer(edge_input_tensor)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight[64, 4, 3, 3], so expected input[128, 1, 20, 20] to have 4 channels, but got 1 channels instead
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "autokeras_alter.py", line 35, in
clf.fit(X_train, y_train, time_limit=246060)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/classifier.py", line 225, in fit
run_searcher_once(train_data, test_data, self.path)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/classifier.py", line 40, in run_searcher_once
searcher.search(train_data, test_data)
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/site-packages/autokeras/search.py", line 190, in search
accuracy, loss, graph = train_results.get()[0]
File "/public/users/liangqiaoxing/.conda/envs/tensorflow_py3.6/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
RuntimeError: Given groups=1, weight[64, 4, 3, 3], so expected input[128, 1, 20, 20] to have 4 channels, but got 1 channels instead
The error message seemed to come from Torch. I tried running the same ImageClassifier code on mnist data with shape [batch, 28, 28, 1] and the code worked fine.
Here is my training set. Does anyone know why or meet the same error? Thanks!
demo.zip
C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from
floatto
np.floatingis deprecated. In future, it will be treated as
np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Initializing search.
Initialization finished.
Training model 0
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 44, in mapstar
return list(map(*args))
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\search.py", line 323, in train
verbose).train_model(**trainer_args)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\utils.py", line 121, in train_model
self._train(train_loader, epoch)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\utils.py", line 140, in _train
outputs = self.model(inputs)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\graph.py", line 577, in forward
temp_tensor = torch_layer(edge_input_tensor)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
TypeError: 'float' object cannot be interpreted as an integer
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "mnist.py", line 10, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\classifier.py", line 224, in fit
run_searcher_once(train_data, test_data, self.path)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\classifier.py", line 40, in run_searcher_once
searcher.search(train_data, test_data)
File "C:\Users\Adam\Desktop\autokeras\autokeras\examples\autokeras\search.py", line 187, in search
accuracy, loss, graph = train_results.get()[0]
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
raise self._value
TypeError: 'float' object cannot be interpreted as an integer
`
mnist.py execution fails.
Windows 10
Python 3.6.2
it would be great to share a code for
Mining Mobile Internet Packets for Malware Detection
Multi-layer Anomaly Detection for Internet Traffic Based on Data Mining
thanks
C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from
floatto
np.floatingis deprecated. In future, it will be treated as
np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Initializing search.
Initialization finished.
Training model 0
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 44, in mapstar
return list(map(*args))
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\search.py", line 327, in train
verbose).train_model(**trainer_args)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\utils.py", line 122, in train_model
self._train(train_loader, epoch)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\utils.py", line 143, in _train
outputs = self.model(inputs)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\graph.py", line 603, in forward
temp_tensor = torch_layer(edge_input_tensor)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
TypeError: 'float' object cannot be interpreted as an integer
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "cifar10.py", line 8, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\classifier.py", line 225, in fit
run_searcher_once(train_data, test_data, self.path)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\classifier.py", line 40, in run_searcher_once
searcher.search(train_data, test_data)
File "C:\Users\Adam\Desktop\autokeras\examples\autokeras\search.py", line 190, in search
accuracy, loss, graph = train_results.get()[0]
File "C:\Users\Adam\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
raise self._value
TypeError: 'float' object cannot be interpreted as an integer
`
Windows 10
Python 3.6.2
cifar10.py example fails.
Hi --
Do you have any information about the expected performance of this project on standard benchmarks, eg MNIST or CIFAR10?
Thanks
Semantic Segmentation is one of the most important research areas and applications in Computer Vision, such as autonomous driving, indoor navigation. This module will provide easy access to people or group who needs to perform semantic segmentation in their work with limited Computer Vision or Machine Learning background.
Some add-ons on current AutoKeras to be considered:
Hi! HaiFeng, I tried to run the demo, but it failed, the code as shown below:
from keras.datasets import mnist
from autokeras.classifier import ImageClassifier
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))
clf = ImageClassifier(verbose=True)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y)
The error message as shown below:
Initializing search.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-23-077a1b1d90e1> in <module>()
1 clf = ImageClassifier(verbose=True)
----> 2 clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
3 clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
4 y = clf.evaluate(x_test, y_test)
5 print(y)
/Users/victor/virtualenvlist/mydlp2/lib/python2.7/site-packages/autokeras/classifier.pyc in fit(self, x_train, y_train, time_limit)
210 start_time = time.time()
211 while time.time() - start_time <= time_limit:
--> 212 run_searcher_once(x_train, y_train, x_test, y_test, self.path)
213 if len(self.load_searcher().history) >= constant.MAX_MODEL_NUM:
214 break
/Users/victor/virtualenvlist/mydlp2/lib/python2.7/site-packages/autokeras/classifier.pyc in run_searcher_once(x_train, y_train, x_test, y_test, path)
41 backend.set_session(sess)
42 searcher = pickle_from_file(os.path.join(path, 'searcher'))
---> 43 searcher.search(x_train, y_train, x_test, y_test)
44
45
/Users/victor/virtualenvlist/mydlp2/lib/python2.7/site-packages/autokeras/search.pyc in search(self, x_train, y_train, x_test, y_test)
156 def search(self, x_train, y_train, x_test, y_test):
157 if not self.history:
--> 158 self.init_search()
159
160 # Start the new process for training.
/Users/victor/virtualenvlist/mydlp2/lib/python2.7/site-packages/autokeras/search.pyc in init_search(self)
142 print('Initializing search.')
143 graph = DefaultClassifierGenerator(self.n_classes,
--> 144 self.input_shape).generate(self.default_model_len,
145 self.default_model_width)
146 model_id = self.model_count
/Users/victor/virtualenvlist/mydlp2/lib/python2.7/site-packages/autokeras/generator.pyc in __init__(self, n_classes, input_shape)
34 class DefaultClassifierGenerator(ClassifierGenerator):
35 def __init__(self, n_classes, input_shape):
---> 36 super().__init__(n_classes, input_shape)
37
38 def generate(self, model_len=constant.MODEL_LEN, model_width=constant.MODEL_WIDTH):
TypeError: super() takes at least 1 argument (0 given)
From the log, I think the fit()
function in class ImageClassifier
got something wrong.
Thanks in Advances.
Even after installing python3.6, tensorflow-gpu, keras, and autokeras, when I do
"import autokeras" or "from autokeras.classifier import ImageClassifier", it gives me
ModuleNotFoundError: No module named 'autokeras.classifier'; 'autokeras' is not a package
How can I fix this?
I have your /examples/mnist.py file working on an AWS GPU server. I needed to delete the "time_limit=12 * 60 * 60" part though because that was giving an error. My question is, does it use multiple cores by default? I read the Parallelism section of your paper, but am unclear if I have to do anything to make it parallel. What if for example I want to run it on a CPU-only server with 128 cores, would it automatically use all of them? Also, when you say it runs multi-GPU, do you mean within the same machine, or can it also run over a cluster of multiple GPU servers?
Also, in Table 1 of your paper, where you compare Auto-Keras with similar programs, are those accuracy results on the test set or validation set? In your github repo it seems to be only split into training and test sets. Should that also include a validation set?
from autokeras.image_classifier import ImageClassifier
This gives the following error:
ImportError Traceback (most recent call last)
in ()
1 from keras.datasets import mnist
----> 2 from autokeras.image_classifier import ImageClassifier
/usr/local/lib/python3.6/dist-packages/autokeras/image_classifier.py in ()
18 from autokeras.search import BayesianSearcher, train
19 from autokeras.utils import ensure_dir, has_file, pickle_from_file, pickle_to_file
---> 20 from autokeras.classifier import Classifier
21
22
ImportError: cannot import name 'Classifier'
Steps to reproduce the behavior:
Use this https://colab.research.google.com/drive/1it4U3H8t6fMhLBt_NFmB6vRRN2bVpxFv
Separating classifier.py and image_classifier.py has created this problem.
Include the details about the versions of:
Not sure if the argument time_limit is used here. This function is running for a very long time.
clf.fit(x_train, y_train, time_limit=1)
Any possible reason?
Mac:
Python 3ish
I am trying the example mnist.py but repeatedly getting this error. I have uninstalled tensorflow and reinstalled it's latest version but the problem still persists. How can I rectify this? P.S. all solutions I found online were for Windows, I am using Ubuntu 18.04
As stated here, Autokeras kills the Google Colaboratory kernel, in CPU or GPU mode.
Run the following notebook, which is a simple copy-paste of the AutoKeras tutorial, with a slight modification: in the AutoKeras API, the package autokeras.image_classifier seems to have been renamed to autokeras.image_supervised, and the documentation is not yet up to date.
It should run, since Colaboratory runs Python 3.6
Colaboratory seems incompatible with the Python multiprocessing API.
I modified the code of search.py/Searcher in order to remove it, and it worked.
More clearly, I replaced this:
multiprocessing.set_start_method('spawn', force=True)
pool = multiprocessing.Pool(1)
train_results = pool.map_async(train, [(graph, train_data, test_data, self.trainer_args, os.path.join(self.path, str(model_id) + '.png'), self.metric, self.loss, self.verbose)])
By this:
train_results = train((graph, train_data, test_data, self.trainer_args, os.path.join(self.path, str(model_id) + '.png'), self.metric, self.loss, self.verbose))
Maybe AutoKeras could have an option in order to desactivate the use of that API ?
Running examples/mnist.py
and got the following error:
...
Father ID: 2
[('to_wider_model', 1, 64), ('to_wider_model', 1, 128), ('to_wider_model', 1, 256), ('to_wider_model', 1, 512), ('to_conv_deeper_model', 6, 3), ('to_add_skip_model', 6, 19), ('to_conv_deeper_model', 1, 5), ('to_conv_deeper_model', 11, 5), ('to_wider_model', 19, 64), ('to_wider_model', 11, 64), ('to_conv_deeper_model', 25, 3), ('to_add_skip_model', 19, 11), ('to_wider_model', 1, 1024), ('to_conv_deeper_model', 33, 3)]
Traceback (most recent call last):
File "mnist.py", line 12, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/classifier.py", line 219, in fit
run_searcher_once(train_data, test_data, self.path)
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/classifier.py", line 41, in run_searcher_once
searcher.search(train_data, test_data)
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/search.py", line 168, in search
new_graph, new_father_id = self.maximize_acq()
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/search.py", line 229, in maximize_acq
getattr(nm_graph, args[0])(*list(args[1:]))
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/graph.py", line 311, in to_conv_deeper_model
new_layers = deeper_conv_block(target, kernel_size, self.weighted)
File "/home/fayw/anaconda3/lib/python3.6/site-packages/autokeras/layer_transformer.py", line 11, in deeper_conv_block
n_filters = conv_layer.filters
AttributeError: 'StubDropout' object has no attribute 'filters'
Please help!
Thank you for sharing your project sir.
I am about to use this source for my journal paper.
Original papers said that the authers used several hundreds of graphic card to run AutoML.
How about this source?
Ex: in my case, i am using only one Titan X graphic card with 5000 images (224x224x3).
Actually i already got memory error. What is minimum requirement of graphic card to run this source sir?
Your library can do 3D convolutional auto keras.
Input images are 3D images, such as 3D CT scan images.
If you add this feature, it will benefit many people.
@tl-yang Please add 2 items to the pull request guide in the file.
from queue import PriorityQueue
ImportError: No module named queue
CNN structure is becoming more and more widely used in text classification. Unlike image in shape [height, weight, channel], in this case the input will be in shape [1, n, embedding size], where n is the length of the text. Currently such input will cause error in autokeras, since "1 - filter_size" will be less than zero. So I think autokeras should automatically detect such input shape and then apply 1D architecture search like 1D convolution, 1D pooling, etc.
Autokeras now is mainly for 2D input like image. It could be more flexible.
Add some codes to detect the shape of the input, and to search specific network structures optimized for 1D, 2D or 3D tensor depending on the shape of the input.
Hey mr
i got following error (the code is from classifier.py in this source):
if x_train.shape[0] != y_train.shape[0]:
raise ValueError('x_train and y_train should have the same number of instances.')
where x_train.shape is 972 but y_train.shape is 48600 which is 50 times than x_train.
because, i have 50 classes, so the oneHot gives me array of 50 numbers ([0, 0, 1, 0 .....0]) per image.
When running image_classifier. If we use a big MAX_MODEL_NUM (1000). https://github.com/jhfjhfj1/autokeras/blob/master/autokeras/bayesian.py#L89 would throw an overflow warning. Which causes a FileNotFound Error to be raised.
Here is the stack trace:
File "/home/tsunglin/autokeras/autokeras/image_classifier.py", line 229, in fit
run_searcher_once(train_data, test_data, self.path, int(time_remain))
File "/home/tsunglin/autokeras/autokeras/image_classifier.py", line 42, in run_searcher_once
searcher.search(train_data, test_data, timeout)
File "/home/tsunglin/autokeras/autokeras/search.py", line 189, in search
new_graph, new_father_id = self.maximize_acq(timeout)
File "/home/tsunglin/autokeras/autokeras/search.py", line 268, in maximize_acq
nm_graph = self.load_model_by_id(father_id)
File "/home/tsunglin/autokeras/autokeras/search.py", line 109, in load_model_by_id
return pickle_from_file(os.path.join(self.path, str(model_id) + '.h5'))
File "/home/tsunglin/autokeras/autokeras/utils.py", line 190, in pickle_from_file
return pickle.load(open(path, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: 'tests/resources/temp/None.h5'
Steps to reproduce the behavior:
Without error
Include the details about the versions of:
Traceback (most recent call last):
File "C:/Users/lichunhong/Desktop/python_code/autokeras_try/1.py", line 10, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
File "C:\ProgramData\Anaconda3\lib\site-packages\autokeras\classifier.py", line 225, in fit
run_searcher_once(train_data, test_data, self.path)
File "C:\ProgramData\Anaconda3\lib\site-packages\autokeras\classifier.py", line 40, in run_searcher_once
searcher.search(train_data, test_data)
File "C:\ProgramData\Anaconda3\lib\site-packages\autokeras\search.py", line 190, in search
accuracy, loss, graph = train_results.get()[0]
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 644, in get
raise self._value
TypeError: 'float' object cannot be interpreted as an integer
Describe the bug:
Running an simple MNIST data classification.
I was able to run for the first time. Next day, I tried to run the same program, I got this error message
(asr) maybe@maybe1:/data/jugs/asr/languageclf$ ./trainalutokeras.py
from: can't read /var/mail/keras.datasets
from: can't read /var/mail/autokeras.classifier
./trainalutokeras.py: line 5: syntax error near unexpected token `,'
./trainalutokeras.py: line 5: ` (x_train, y_train),(x_test, y_test) = mnist.load_data()'
Setup Details:
Any details about your local setup that might be helpful in troubleshooting. Include the details about the versions of:
I am trying to run the code on windows, I have torch version 0.4.1, it's showing that requirements.txt requires torch version 0.4.0 which is unavailable. How do I go about this problem?
This is my error:
Using TensorFlow backend.
Traceback (most recent call last):
File "autok1.py", line 3, in
from autokeras.constant import Constant
ImportError: cannot import name 'Constant'
This looks like an internal error?
When running mnist.py after some time, suddenly an error occurred :
Traceback (most recent call last):
File "/home/jdiez/project/AutoKeras/Test_01.py", line 11, in
clf.fit(x_train, y_train, time_limit=20 * 60)
File "/home/jdiez/lib/anaconda3/envs/autokeras/lib/python3.6/site-packages/autokeras/image_supervised.py", line 238, in fit
run_searcher_once(train_data, test_data, self.path, int(time_remain))
File "/home/jdiez/lib/anaconda3/envs/autokeras/lib/python3.6/site-packages/autokeras/image_supervised.py", line 41, in run_searcher_once
searcher.search(train_data, test_data, timeout)
File "/home/jdiez/lib/anaconda3/envs/autokeras/lib/python3.6/site-packages/autokeras/search.py", line 175, in search
timeout)
File "/home/jdiez/lib/anaconda3/envs/autokeras/lib/python3.6/site-packages/autokeras/bayesian.py", line 272, in optimize_acq
raise TimeoutError
TimeoutError
Process finished with exit code 1
Steps to reproduce the behavior:
Run without error
Include the details about the versions of:
Torch, Tensorflow and Keras are all working fine insisde this conda env.
The same error occurs when adapting the code for CIFAR10, Fashion-MNIST
I've encountered with this error
File "/home/me/venv/lib64/python3.6/site-packages/autokeras/classifier.py", line 258, in save_searcher
pickle.dump(searcher, open(os.path.join(self.path, 'searcher'), 'wb'))
PermissionError: [Errno 13] Permission denied: '/tmp/autokeras/searcher'
I'm running this on linux server on virtual environment. Could you please give me the suggestion?
I want to save the trained model and make it usable for serving. Is it possible? How?
Thanks for Auto Keras.
I have one question.
Since searching model takes long time, I believe it would be awesome if the Autokeras supports trainining multiple models on distributed machines (parallel training). In this sense, searching time can be saved a lot.
I wonder if Auto Keras is already ready for it or have plan to support this?
Thanks!
Export the Autokeras model of its own type instead of exporting the Keras model.
More flexible than Keras model.
Create a base class which can only be loaded and do the "predict" and "evaluation" function.
A class named "PortableImageClassifer" extends the base class.
It has a member of an instance of "Graph".
Refer to other parts of the project to see how to do "predict" and "evaluation" with an instance of Graph.
NA
NA
from keras.datasets import mnist
from autokeras.image_supervised import ImageClassifier
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))
clf = ImageClassifier(verbose=True)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y)
Steps to reproduce the behavior:
Include the details about the versions of:
I tried fitting the data on amazon p2.8xlarge instance(which has 8 GPUs), but it just fills the VRAM of one GPU and crashes with the cuda out of memory error. Is there any way of using all 8 GPUs?
Following the instructions of https://autokeras.com/start/
I've found this error
Loading load_image_dataset with
from autokeras.image_classifier import load_image_dataset
rise an error " No module named 'autokeras.image_classifier'"
checking the script in the
rise the error from different module "from autokeras.classifier import load_image_dataset"
I've found the function in the module image_supervised.py
You only support the image classification problems. Am I right? If my problem is similar to the image classification, but my input images are 3D instead of 2D. Can I still use your AutoKeras to figure out the better architectures and hyperparameters for 3D convolutional neural networks? Thanks for sharing your library.
I am working on rnn predicting by keras. And I want to know if this autokeras can support it?
f[len_a][len_b] is always equal to f[-1][-1]=0
len_a = len(list_a)
len_b = len(list_b)
f = np.zeros((len_a + 1, len_b + 1))
f[-1][-1] = 0
for i in range(-1, len_a):
f[i][-1] = i + 1
for j in range(-1, len_b):
f[-1][j] = j + 1
for i in range(len_a):
for j in range(len_b):
f[i][j] = min(f[i][j - 1] + 1, f[i - 1][j] + 1, f[i - 1][j - 1] + layer_distance(list_a[i], list_b[j]))
print(f)
return f[len_a][len_b]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.