Light

ahmedshingaly / ione Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 0.0 0 B

AI for Architecture Design

ione's People

Contributors

Stargazers

Watchers

ione's Issues

Relu block

Open in MURAL
iFacade iOne - ahmedshingaly

If the derivative is 0, it means that we are in a stable minimum. Thus, no update on the weights is needed -> we reached a stable state.

Open in MURAL
iFacade iOne - ahmedshingaly

The forward and back propagation is performed until the loss becomes zero( Predicted result == True value).

Open in MURAL
iFacade iOne - ahmedshingaly

What is a dataset?Learn by doing: Create a small dataset in 5 minutesHow to analyze a dataset?

Open in MURAL
iFacade iOne - ahmedshingaly

How can we reduce the loss and increase the model accuracy?The answer is optimizers. The optimizer reduces the loss by finding the rate of change of loss with respect to learned parameters in each layer. The learned parameters are updated simultaneously for each layer according to the calculated rate. These learned parameters are also called as weights.The function of all the optimizers is to minimize the loss by updating the weight in such a way that the loss becomes zero.

Open in MURAL
iFacade iOne - ahmedshingaly

Learning rate

Open in MURAL
iFacade iOne - ahmedshingaly

What is a machine learning model?Forward propagationImage featuresFeature detection and Feature DescriptionLearn by doing: Understand convolution, filters, feature detector, and descriptor in 5 minutes visually.What does a Relu block do?What is max-pooling?What is FC block or fully connected block?What is a dropout layer?

Open in MURAL
iFacade iOne - ahmedshingaly

Dropout layer

Open in MURAL
iFacade iOne - ahmedshingaly

https://keras.io/losses/

Open in MURAL
iFacade iOne - ahmedshingaly

You need a compiled model to train (because training uses the loss function and the optimizer). But it’s not necessary to compile the model when testing/predicting the model on a new data

Open in MURAL
iFacade iOne - ahmedshingaly

https://docs.google.com/document/d/1oZ7LLRZAVA6Ufz4eCASUZNPGYuQs_-SdOf3JstTbgrQ/edit#

Open in MURAL
iFacade iOne - ahmedshingaly

After the 1000 nodes, we have a layer called a softmax layer. It is a function that provides probabilities for each possible class in a multi-class classification model. The probabilities add up to exactly 1.0. For example, softmax might determine that the probability of a particular image is a dog at 0.9, a cat at 0.08, and a horse at 0.02. For binary class classification, we use a sigmoid function, which gives the probability of one class.

Open in MURAL
iFacade iOne - ahmedshingaly

List of the optimizers and how fast they reduce the loss.As you can see that Adadelta is the fastest. But it is not always important to make the loss zero, it should also be reliable

Open in MURAL
iFacade iOne - ahmedshingaly

The above line of code is the way to represent fully connected layer which is called as dense layer in keras.Here 1000 is the number of nodes in the fully connected layer.

Open in MURAL
iFacade iOne - ahmedshingaly

A supervised neural network, at the highest and simplest representation, can be presented as a black box with 2 methods learn and predict as following:

Open in MURAL
iFacade iOne - ahmedshingaly

The above line of code is the way to represent dropout layer in keras. Here 0.5 stands for dropping 50 percent of neurons.

Open in MURAL
iFacade iOne - ahmedshingaly

We could have guessed this rate by calculating directly the derivative of the loss function. The advantage of using the mathematical derivative is that it is much faster and more precise to calculate (less floating point precision problems).Here is what our loss function looks like:If w=2, we have a loss of 0, since the neural network actual output will fit perfectly the training set.If w<2, we have a positive loss function, but the derivative is negative, meaning that an increase of weight will decrease the loss function.At w=2, the loss is 0 and the derivative is 0, we reached a perfect model, nothing is needed.If w>2, the loss becomes positive again, but the derivative is as well positive, meaning that any more increase in the weight, will increase the losses even more!

Open in MURAL
iFacade iOne - ahmedshingaly

The above line of code is the way to represent a the maxpooling layer in keras. In this case the size of maxpooling is 2*2.

Open in MURAL
iFacade iOne - ahmedshingaly

Step-2: Prepare the dataset

Open in MURAL
iFacade iOne - ahmedshingaly

activation Functions

Open in MURAL
iFacade iOne - ahmedshingaly

Where p(x) is the wanted probability, and q(x) the actual probability. The sum is over the three classes A, B, and C. In this case, the loss is 0.479 :H = - (0.0ln(0.228) + 1.0ln(0.619) + 0.0*ln(0.153)) = 0.479

Open in MURAL
iFacade iOne - ahmedshingaly

Max pooling

Open in MURAL
iFacade iOne - ahmedshingaly

If the derivative rate is negative, it means that an increase in weight will decrease the error, thus we need to increase the weights.

Open in MURAL
iFacade iOne - ahmedshingaly

delta ruleNew weight = old weight — Derivative * learning rate

Open in MURAL
iFacade iOne - ahmedshingaly

#The above line of code is the way to represent a convolution layer in Keras. It has 32 filters of each size 3*3. The shape of input image is height = 150, width = 150 and number of channels = 3

Open in MURAL
iFacade iOne - ahmedshingaly

loss = Absolute value of (desired — actual ).

Open in MURAL
iFacade iOne - ahmedshingaly

Step — 4 Compile the model

Open in MURAL
iFacade iOne - ahmedshingaly

Learn by doing: Perform data augmentation in 5 minute

Open in MURAL
iFacade iOne - ahmedshingaly

In order to validate this equation:

Open in MURAL
iFacade iOne - ahmedshingaly

from keras.layers import Conv2D(Conv2D(32, (3, 3), input_shape=(150, 150,3)))

Open in MURAL
iFacade iOne - ahmedshingaly

Step -3 Create the model

Open in MURAL
iFacade iOne - ahmedshingaly

Dropout is a technique for addressing the problem of overfitting. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting and overfitting too much. Generally, use a small dropout value of 20%-50% of neurons with 20% providing a good starting point

Open in MURAL
iFacade iOne - ahmedshingaly

Fully Connected block

Open in MURAL
iFacade iOne - ahmedshingaly

from keras.layers DropoutDropout(0.5)

Open in MURAL
iFacade iOne - ahmedshingaly

Step-1: Analyze the dataset

Open in MURAL
iFacade iOne - ahmedshingaly

The most intuitive loss function is simply loss = (Desired output — actual output). However this loss function returns positive values when the network undershoot (prediction < desired output), and negative values when the network overshoot (prediction > desired output). If we want the loss function to reflect an absolute error on the performance regardless if it’s overshooting or undershooting we can define it as:

Open in MURAL
iFacade iOne - ahmedshingaly

next weight model (2)

Open in MURAL
iFacade iOne - ahmedshingaly

The easiest way to do a full working example, is to take only one operator (matrix multiplication), one loss function (RMSE), one optimizer(gradient descent) and execute a full running example.

Open in MURAL
iFacade iOne - ahmedshingaly

optimizer

Open in MURAL
iFacade iOne - ahmedshingaly

Step — 4 Compile the model

Open in MURAL
iFacade iOne - ahmedshingaly

Machine learning

Open in MURAL
iFacade iOne - ahmedshingaly

If two fully connected layers (Dense)are stacked one after the another, it means every single node of the first FC layer is connected to every single node in the following FC layer.

Open in MURAL
iFacade iOne - ahmedshingaly

Input, hidden, output layers are fully connected layers where every node in the current layer is connected to every node in the next layer.

Open in MURAL
iFacade iOne - ahmedshingaly

What is a machine learning model?

Open in MURAL
iFacade iOne - ahmedshingaly

https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6

Open in MURAL
iFacade iOne - ahmedshingaly

Dropout Neural Net Model. Left: A standard neural net with 2 hidden layers. Right: An example of a thinned net produced by applying dropout to the network on the left. Crossed units have been dropped.

Open in MURAL
iFacade iOne - ahmedshingaly

convolutional neural network

Open in MURAL
iFacade iOne - ahmedshingaly

Learning rate is the rate with which the model should learn. The learning rate value is a small real value such as 0.1, 0.001 or 0.0001.The decision of how much our learning rate should depend on experimentation. Naive method for choosing learning rate is trying out a bunch of numbers and using the one that looks to work best, manually decreasing it over time when training doesn’t seem to improve the loss anymore. It tells how fast the weights must be learned.

Open in MURAL
iFacade iOne - ahmedshingaly

If the derivative rate is positive, it means that an increase in weight will increase the error, thus the new weight should be smaller.

Open in MURAL
iFacade iOne - ahmedshingaly

from keras.layers import MaxPooling2D(MaxPooling2D(pool_size=(2, 2)))

Open in MURAL
iFacade iOne - ahmedshingaly

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.