Comments (17)
This would be really neat to have working. As Google Colab gives free GPUs, this would let a lot more people try out Autokeras...
from autokeras.
I encounter the same issue.
from autokeras.
@PositronBeam Thank you for your report.
We haven't tested with Google Collaboratory yet.
Will come back to this soon.
Thanks.
from autokeras.
Hi @jhfjhfj1 , would you mind telling us whether you tested with Google Collaboratory?
Thank a lot
from autokeras.
@vincent-hui , Thank you for your follow up.
We have changed the multiprocessing into torch.multiprocessing.
But we haven't tested with the Google Colaboratory yet.
from autokeras.
@jhfjhfj1 I just tried autokeras-0.2.7 with the Google Colaboratory
Google Colaboratory runtime dies when I run
from keras.datasets import mnist
from autokeras import ImageClassifier
if __name__ == '__main__':
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))
clf = ImageClassifier(verbose=True, augment=False)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y * 100)
Here is the output message
Using TensorFlow backend.
Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 2s 0us/step
Initializing search.
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43]
Initialization finished.
Training model 0
from autokeras.
@jhfjhfj1 I just tried autokeras-0.2.11 with the Google Colaboratory
Google Colaboratory runtime dies when I use autokeras-0.2.11
Here is the output message
Using TensorFlow backend.
/usr/local/lib/python3.6/dist-packages/tqdm/autonotebook/__init__.py:14: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
" (e.g. in jupyter console)", TqdmExperimentalWarning)
Initializing search.
Initialization finished.
╒==============================================╕
| Training model 0 |
╘==============================================╛
from autokeras.
I tried autokeras-0.2.13 with the Google Colaboratory. Google Colaboratory runtime dies when running autokeras.
from autokeras.
Help needed.
Does anyone know why it cannot work with Google Collaboratory and how to fix it?
Thanks
from autokeras.
@jhfjhfj1
Sorry for my stupid question. In Additional context of the first post of @@PositronBeam , is it a workaround for running autokeras on Google Collaboratory?
from autokeras.
Add google colab check to @PositronBeam
search.py line175 - 180
BEFORE==============================================================
mp.set_start_method('spawn', force=True)
pool = mp.Pool(1)
try:
train_results = pool.map_async(train, [(graph, train_data, test_data, self.trainer_args,
os.path.join(self.path, str(model_id) + '.png'),
self.metric, self.loss, self.verbose)])
AFTER===============================================================
import sys
if "google.colab" in sys.modules:
gcolab = True
else:
gcolab = False
mp.set_start_method('spawn', force=True)
pool = mp.Pool(1)
try:
if gcolab:
train_results = train((graph, train_data, test_data, self.trainer_args,
os.path.join(self.path, str(model_id) + '.png'),
self.metric, self.loss, self.verbose))
else:
train_results = pool.map_async(train, [(graph, train_data, test_data, self.trainer_args,
os.path.join(self.path, str(model_id) + '.png'),
self.metric, self.loss, self.verbose)])
from autokeras.
Hi @compassNS ,
Would you mind telling me how to run autokeras with Google Colaboratory ?
Thank a lot.
from autokeras.
I tried making the change @compassNS suggested but I got this error:
local variable 'pool' referenced before assignment
from autokeras.
@chencc6566193 Since this issue is kind of urgent, @jhfjhfj1 reassigned it to me as well. Please feel free to let me know if you had any progress, then we two can co-work on this one. Thanks!
from autokeras.
Help needed.
Does anyone know why it cannot work with Google Collaboratory and how to fix it?
Thanks
@jhfjhfj1 In my case, it seems that lack of memory was the problem when I use google Colab.
So I think that, some kind of batch generator such as fit_generator something could be useful to handle with the problem.
Do you have any plan to implement this idea?
By the way, autokeras is really easy, and powerful for me to do NAS for image classification!
Thank you so much for your kindness and effort for this :)
Now I'm gonna also try to use autokeras for other data types, such as time-series neuronal / behavior data - convert them into kind of 2-d array or text data.
I saw that module for audio data is also being considered, and it will be really nice for them too!
from autokeras.
Support GPU on Google Colab
###Solution
- Disable multiprocessing
Reason
- Google Colab doesn't support multiprocessing "spwan" start method. However, cuda only support "spawn" start method. Thus, to enable GPU accelaration on Google Colab, we must run autokeras in sigle process.
from autokeras.
Sorry, I am confused about the final resolution. Is it changing the Google Colab Notes to single process or was there a change in the source code. Also, can you please post steps to disable multi-processing?
from autokeras.
Related Issues (20)
- Feature: Model Agnostic Explanability - Permutation Importance and Counterfactual
- Bug: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 205. in Chinese path, not in English path
- Bug: TypeError: can only concatenate str (not "NoneType") to st HOT 1
- Bug: Unable to predict data using the model trained (Node: 'model/multi_category_encoding/string_lookup_22/None_Lookup/LookupTableFindV2' Table not initialized. [[{{node model/multi_category_encoding/string_lookup_22/None_Lookup/LookupTableFindV2}}]] [Op:__inference_predict_function_10129638] )
- Bug: HOT 4
- AttributeError: module 'numpy' has no attribute 'object' when using autokeras with structured data HOT 5
- Autokers textinput text classifier does not work.
- ModuleNotFoundError: No module named 'tensorflow.keras.layers.experimental'Bug: HOT 5
- Bug: import autokeras as ak --> ModuleNotFoundError: No module named 'tensorflow.keras.layers.experimental' HOT 9
- Bug: FatalTypeError: Expected the model-building function, or HyperModel.build() to return a valid Keras Model instance
- Bug: StructuredDataClassifier and StructuredDataRegressor gone? HOT 9
- Bug: Autokeras/TF fails with CUDA_ERROR_ILLEGAL_ADDRESS/CUDA_ERROR_INVALID_HANDLE when max_trials is not 1
- Bug: "Best val_accuracy So Far" is mistaken for "Best val_loss So Far"
- Feature: 希望AutoKeras可以基于目前最新的Keras3,支持Keras3的新特性!
- Why was the TimeSeries parts of the package removed in the latest version?
- Feature:
- Bug: HOT 1
- Bug: ModuleNotFoundError: No module named 'tensorflow.keras.layers.experimental' HOT 2
- Bug: trouble loading and using tensorflow savedmodel with custom input layer in keras, has been working fine for 2+ years and now currently looking for the solutions with my team HOT 1
- module 'autokeras' has no attribute 'StructuredDataClassifier'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autokeras.