udacity / aind2-cnn Goto Github PK

View Code? Open in Web Editor NEW

252.0 25.0 557.0 39.3 MB

AIND Term 2 -- Lesson on Convolutional Neural Networks

License: MIT License

Jupyter Notebook 100.00%

aind2-cnn's Introduction

aind2-cnn

Instructions

Clone the repository and navigate to the downloaded folder.

git clone https://github.com/udacity/aind2-cnn.git
cd aind2-cnn

(Optional) If you plan to install TensorFlow with GPU support on your local machine, follow the guide to install the necessary NVIDIA software on your system. If you are using an EC2 GPU instance, you can skip this step.
(Optional) If you are running the project on your local machine (and not using AWS), create (and activate) a new environment.
- Linux (to install with GPU support, change requirements/dog-linux.yml to requirements/dog-linux-gpu.yml):
```
conda env create -f requirements/dog-linux.yml
source activate dog-project
```
- Mac (to install with GPU support, change requirements/dog-mac.yml to requirements/dog-mac-gpu.yml):
```
conda env create -f requirements/dog-mac.yml
source activate dog-project
```
- Windows (to install with GPU support, change requirements/dog-windows.yml to requirements/dog-windows-gpu.yml):
```
conda env create -f requirements/dog-windows.yml
activate dog-project
```
(Optional) If you are running the project on your local machine (and not using AWS) and Step 6 throws errors, try this alternative step to create your environment.
- Linux or Mac (to install with GPU support, change requirements/requirements.txt to requirements/requirements-gpu.txt):
```
conda create --name dog-project python=3.5
source activate dog-project
pip install -r requirements/requirements.txt
```
- Windows (to install with GPU support, change requirements/requirements.txt to requirements/requirements-gpu.txt):
```
conda create --name dog-project python=3.5
activate dog-project
pip install -r requirements/requirements.txt
```
(Optional) If you are using AWS, install Tensorflow.

sudo python3 -m pip install -r requirements/requirements-gpu.txt

Switch Keras backend to TensorFlow.

Linux or Mac:

 KERAS_BACKEND=tensorflow python -c "from keras import backend"

Windows:

 set KERAS_BACKEND=tensorflow
 python -c "from keras import backend"

(Optional) If you are running the project on your local machine (and not using AWS), create an IPython kernel for the dog-project environment.

python -m ipykernel install --user --name dog-project --display-name "dog-project"

Launch Jupyter notebook.

jupyter notebook

(Optional) If you are running the project on your local machine (and not using AWS), before running code, change the kernel to match the dog-project environment by using the drop-down menu (Kernel > Change kernel > dog-project).

aind2-cnn's People

Contributors

Stargazers

Watchers

Forkers

pradau ryosukehonda chaimpollak td2014 gongyanchao spil-peter-forgacs cmiller112000 swaption2009 angelmtenor behzadip parsonswlu rballet zzmjohn shupingr blcooley luzi82 lxwvictor thirupathipattipaka kvr777 bryantravissmith pbackx robsonmatos samindaa m7mdbadawy jquickgh arunkv chriscurrin danielgf3 bn2302 cfcdavidchan zf109 brendanvadacorabell vainarouski adrianyi pwangonline jrohnke nerush jcriscione msait oforero ebhtra isallam roman-v-l tsuresh83 xingdu1991 wailo ocefra lansator mrestuccia quantcompany khrm suribes knnaraghi ajdhole lxyea minkvsky personableduck dmitrychudo ppeiris romanroibu snua marianokamp zaher88abd warezt fses91 kwangshin joshnewnham dominicjsykes axsch chiajasper voyageth oserr mprilucik twopatient jamesandersen zmy adripurkayastha keithblaha leaprovenzano azureq enggen jeffadaniels omnigeeker recto jdoepfert herruzojm jaredbbowden fnakashima kscharpf heidekrueger gfrmin geofflangenderfer lukamu andres-root sladomic encomp anirudh838 vikhhr legohyl schrezraoeder

aind2-cnn's Issues

Kernel died on conv-visualization

Although I am using the dog-project kernel, it still died.
It appears that
ax.imshow(np.squeeze(model.predict(np.reshape(img, (1, img.shape[0], img.shape[1], 1)))), cmap='gray')
is the main culprit.

bottleneck_features.ipynb: OOM when allocating tensor

I am getting the following error when trying to run 'bottleneck_features.ipynb' on ml.p2.xlarge notebook instance:

ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[8,64,224,224] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node block1_conv1/convolution}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[block5_pool/MaxPool/_159]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[8,64,224,224] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node block1_conv1/convolution}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

tensorflow 2.0

I'm currently taking the machine learning nanodegree. I have a GPU geforce rtx 2070 runing on ubuntu. I had some errors when setting up the dog-project environment specifically when installing tensorflow. I removed the tensorflow-gpu line from dog-linux-gpu.yml, after that I was able to create the virtual environment as per step 3 in the instructions. After the environment was created, I executed "pip install tf-nightly-gpu-2.0-preview", and this time tensorflow was properly installed.

I guess my question is if using a different version of tensorflow will introduce me to some more issues in the future when using the notebooks provided in this repository.

I appreciate your feedback.
Thanks,
Oscar

It would be useful to show how the bottleneck npz files are saved

I think it would be useful to see the actual saving of the npz file to create the bottleneck features, as it completes the transfer learning picture. Otherwise the bottleneck .npz files are a bit magic ✨.

The transfer-learning/bottleneck_features.ipynb gets really close but does not include the steps to save the files to disk. Please can we have that code too?

Encountered some problem with step 5 ~ 6

Dear all

According to the guidance here:
https://github.com/udacity/aind2-cnn

Should I run in on the AWS or my macbook(no GPU)??

I tried to execute the step on the AWS according to the guidance, but failed at step 5

5.(Optional) If you are using AWS, install Tensorflow. sudo python3 -m pip install -r requirements/requirements-gpu.txt

But I got error message like this on the AWS:

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.5/dist-packages/Keras-2.0.8-py3.5.egg'

What should I do??

Another question is

What does the 'input_shape=(7, 7, 512)' means?

What does the 'input_shape=(7, 7, 512)' means?
What are the meanings of 7,7,512 respectively?
Looking forward to you reply. Thanks.

bug: requirements.txt install issues

I tried installing the requirements.txt file manually via pip but wasn't able to install opencv-python.

No module named 'keras' on AWS

I was trying to use AWS Notebook instance for this project as suggested in the lesson. I installed TensorFlow according to the guidelines in readme (step 5):

sudo python3 -m pip install -r requirements/requirements-gpu.txt

That seems to have worked fine. However, switching Keras backend to TensorFlow (step 6) fails.

KERAS_BACKEND=tensorflow python -c "from keras import backend"

It says "ModuleNotFoundError: No module named 'keras'".

It needs to notice that pip list does not show keras or tensorflow as installed packages. When I run sudo pip list I can see tensorflow-gpu and Keras 2.0.2 (starting with a capital K), but not "keras" (with a small letter). Could that be a cause of the issue and how to fix it?

Array count on file names incorrect

in first input cell :

load ordered list of dog names

dog_names = [item[25:-1] for item in glob('dogImages/train/*/')]

The item[25:-1] should be item[20:-1] it is truncating the first 5 chars of the file name when parsing the dog_names.