Git Product home page Git Product logo

echo's Introduction

Donate

Echo-AI

Python package containing all mathematical backend algorithms used in Machine Learning. The full documentation for Echo is provided here.

Table of Contents

About

Echo-AI Package is created to provide an implementation of the most promising mathematical algorithms, which are missing in the most popular deep learning libraries, such as PyTorch, Keras and TensorFlow.

Activation Functions

The package contains implementation for following activation functions (โœ… - implemented functions, ๐Ÿ•‘ - functions to be implemented soon, โฌœ - function is implemented in the original deep learning package):

# Function Equation Keras PyTorch TensorFlow-Keras TensorFlow - Core
1 Weighted Tanh equation โœ… โœ… โœ… ๐Ÿ•‘
2 Swish equation โœ… โœ… โœ… ๐Ÿ•‘
3 ESwish equation โœ… โœ… โœ… ๐Ÿ•‘
4 Aria2 equation โœ… โœ… โœ… ๐Ÿ•‘
5 ELiSH equation โœ… โœ… โœ… ๐Ÿ•‘
6 HardELiSH equation โœ… โœ… โœ… ๐Ÿ•‘
7 Mila equation โœ… โœ… โœ… ๐Ÿ•‘
8 SineReLU equation โœ… โœ… โœ… ๐Ÿ•‘
9 Flatten T-Swish equation โœ… โœ… โœ… ๐Ÿ•‘
10 SQNL equation โœ… โœ… โœ… ๐Ÿ•‘
11 ISRU equation โœ… โœ… โœ… ๐Ÿ•‘
12 ISRLU equation โœ… โœ… โœ… ๐Ÿ•‘
13 Bent's identity equation โœ… โœ… โœ… ๐Ÿ•‘
14 Soft Clipping equation โœ… โœ… โœ… ๐Ÿ•‘
15 SReLU equation โœ… โœ… โœ… ๐Ÿ•‘
15 BReLU equation ๐Ÿ•‘ โœ… โœ… ๐Ÿ•‘
16 APL equation ๐Ÿ•‘ โœ… โœ… ๐Ÿ•‘
17 Soft Exponential equation โœ… โœ… โœ… ๐Ÿ•‘
18 Maxout equation ๐Ÿ•‘ โœ… โœ… ๐Ÿ•‘
19 Mish equation โœ… โœ… โœ… ๐Ÿ•‘
20 Beta Mish equation โœ… โœ… โœ… ๐Ÿ•‘
21 RReLU equation ๐Ÿ•‘ โฌœ ๐Ÿ•‘ ๐Ÿ•‘
22 CELU equation โœ… โฌœ โœ… ๐Ÿ•‘
23 ReLU6 equation โœ… โฌœ ๐Ÿ•‘ ๐Ÿ•‘
24 HardTanh equation โœ… โฌœ โœ… ๐Ÿ•‘
25 GLU equation ๐Ÿ•‘ โฌœ ๐Ÿ•‘ ๐Ÿ•‘
26 LogSigmoid equation โœ… โฌœ โœ… ๐Ÿ•‘
27 TanhShrink equation โœ… โฌœ โœ… ๐Ÿ•‘
28 HardShrink equation โœ… โฌœ โœ… ๐Ÿ•‘
29 SoftShrink equation โœ… โฌœ โœ… ๐Ÿ•‘
30 SoftMin equation โœ… โฌœ โœ… ๐Ÿ•‘
31 LogSoftmax equation โœ… โฌœ โœ… ๐Ÿ•‘
32 Gumbel-Softmax ๐Ÿ•‘ โฌœ ๐Ÿ•‘ ๐Ÿ•‘
33 LeCun's Tanh โœ… โœ… โœ… ๐Ÿ•‘
34 TaLU โœ… ๐Ÿ•‘ โœ… ๐Ÿ•‘

Repository Structure

The repository has the following structure:

- echoAI # main package directory
| - Activation # sub-package containing activation functions implementation
| |- Torch  # sub-package containing implementation for PyTorch
| | | - functional.py # script which contains implementation of activation functions
| | | - weightedTanh.py # activation functions wrapper class for PyTorch
| | | - ... # PyTorch activation functions wrappers
| |- Keras  # sub-package containing implementation for Keras
| | | - custom_activations.py # script which contains implementation of activation functions
| |- TF_Keras  # sub-package containing implementation for Tensorflow-Keras
| | | - custom_activation.py # script which contains implementation of activation functions
| - __init__.py

- Observations # Folder containing other assets

- docs # Sphinx documentation folder

- LICENSE # license file
- README.md
- setup.py # package setup file
- Smoke_tests # folder, which contains scripts with demonstration of activation functions usage
- Unit_tests # folder, which contains unit test scripts

Setup Instructions

To install echoAI package from PyPI run the following command:

$ pip install echoAI

Code Examples:

Sample scripts are provided in Smoke_tests folder.

PyTorch:

# import PyTorch
import torch

# import activation function from echoAI
from echoAI.Activation.Torch.mish import Mish

# apply activation function
mish = Mish()
t = torch.tensor(0.1)
t_mish = mish(t)

Keras:

# import Keras
from keras import layers
from keras.models import Model
from keras.layers import Input, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D

# import activation function from echoAI
from echoAI.Activation.Keras.custom_activations import Mish

def CNNModel(input_shape):
    X_input = Input(input_shape)
    X = ZeroPadding2D((3, 3))(X_input)X
    X = Conv2D(32, (3, 3), strides = (1, 1), name = 'conv0')(X)
    X = BatchNormalization(axis = 3, name = 'bn0')(X)

    # apply the activation function
    X = Mish()(X)

    # MAXPOOL
    X = MaxPooling2D((2, 2), name='max_pool')(X)

    # FLATTEN X (means convert it to a vector) + FULLYCONNECTED
    X = Flatten()(X)
    X = Dense(10, activation='softmax', name='fc')(X)

    # Create model
    model = Model(inputs = X_input, outputs = X, name='CNNModel')

    return model

TensorFlow Keras:

#import tensorflow
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.layers import Dense, Flatten

# import activation function from echoAI
from echoAI.Activation.TF_Keras.custom_activation import Mish

model = tf.keras.Sequential([
    layers.Flatten(),
    layers.Dense(128, input_shape=(784,)),
    Mish(), # use the activation function
    layers.Dense(64, activation='relu'),
    layers.Dense(10, activation='softmax')])

# Compile the model
model.compile(optimizer = "adam", loss = "mean_squared_error", metrics = ["accuracy"])

# Fit the model
model.fit(x = X_train, y = y_train, epochs = 3, batch_size = 128)

echo's People

Contributors

digantamisra98 avatar lexie88rus avatar soumik12345 avatar

Forkers

thegodone

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.