This repo was moved from @motlabs group. Thanks for @jwkanggist who is a leader of motlabs community.

Awesome Machine Learning DEMOs with iOS

We tackle the challenge of using machine learning models on iOS via Core ML and ML Kit (TensorFlow Lite).

Machine Learning Framework for iOS
- Flow of Model When Using Core ML
- Flow of Model When Using Create ML
Baseline Projects
Application Projects
- Annotation Tool
Create ML Projects
Performance
- 📏Measure module
- Implements
See also

Machine Learning Framework for iOS

Flow of Model When Using Core ML

The overall flow is very similar for most ML frameworks. Each framework has its own compatible model format. We need to take the model created in TensorFlow and convert it into the appropriate format, for each mobile ML framework.

Once the compatible model is prepared, you can run the inference using the ML framework. Note that you must perform pre/postprocessing manually.

If you want more explanation, check this slide(Korean).

Flow of Model When Using Create ML

Baseline Projects

DONE

Using built-in model with Core ML
Using built-in on-device model with ML Kit
Using custom model for Vision with Core ML and ML Kit
Object Detection with Core ML

TODO

Object Detection with ML Kit
Using built-in cloud model on ML Kit
- Landmark recognition
Using custom model for NLP with Core ML and ML Kit
Using custom model for Audio with Core ML and ML Kit
- Audio recognition
- Speech recognition
- TTS

Image Classification

Name	DEMO	Note
ImageClassification-CoreML		-
MobileNet-MLKit		-

Object Detection & Recognition

Name	DEMO	Note
ObjectDetection-CoreML		-
TextDetection-CoreML		-
TextRecognition-MLKit		-
FaceDetection-MLKit		-

Pose Estimation

Name	DEMO	Note
PoseEstimation-CoreML		-
PoseEstimation-TFLiteSwift		-
PoseEstimation-MLKit		-
FingertipEstimation-CoreML		-

Depth Prediction


DepthPrediction-CoreML		-

Semantic Segmentation

Name	DEMO	Note
SemanticSegmentation-CoreML		-

Application Projects

Name	DEMO	Note
dont-be-turtle-ios		-
WordRecognition-CoreML-MLKit(preparing...)		Detect character, find a word what I point and then recognize the word using Core ML and ML Kit.

Annotation Tool

Name	DEMO	Note
KeypointAnnotation		Annotation tool for own custom estimation dataset

Create ML Projects

Name	Create ML DEMO	Core ML DEMO	Note
SimpleClassification-CreateML-CoreML			A Simple Classification Using Create ML and Core ML

Performance

Execution Time: Inference Time + Postprocessing Time

(with iPhone X)	Inference Time(ms)	Execution Time(ms)	FPS
ImageClassification-CoreML	40	40	23
MobileNet-MLKit	120	130	6
ObjectDetection-CoreML	100 ~ 120	110 ~ 130	5
TextDetection-CoreML	12	13	30(max)
TextRecognition-MLKit	35~200	40~200	5~20
PoseEstimation-CoreML	51	65	14
PoseEstimation-MLKit	200	217	3
DepthPrediction-CoreML	624	640	1
SemanticSegmentation-CoreML	178	509	1
WordRecognition-CoreML-MLKit	23	30	14
FaceDetection-MLKit	-	-	-

📏Measure module

You can see the measured latency time for inference or execution and FPS on the top of the screen.

If you have more elegant method for measuring the performance, suggest on issue!

Implements

	Measure📏	Unit Test	Bunch Test
ImageClassification-CoreML	O	X	X
MobileNet-MLKit	O	X	X
ObjectDetection-CoreML	O	O	X
TextDetection-CoreML	O	X	X
TextRecognition-MLKit	O	X	X
PoseEstimation-CoreML	O	O	X
PoseEstimation-MLKit	O	X	X
DepthPrediction-CoreML	O	X	X
SemanticSegmentation-CoreML	O	X	X

Repositories	iPhone 11 series	iPhone 12 series	iPhone SE (2nd)
PoseEstimation-CoreML	❌	❌	❌
PoseEstimation-TFLite	❌	❌	❌
ObjectDetection-CoreML	❌	❌	❌
DepthPrediction-CoreML	❌	❌	❌
SemanticSegmentation-CoreML	❌	❌	❌
ImageClassification-CoreML	❌	❌	❌
ImageClassification-TFLite	❌	❌	❌
TextDetection-CoreML	❌	❌	❌
TextRecognition-MLKit	❌	❌	❌

Repo	Model	XS, XS Max	XR	X, 8, 8+	7, 7+	6S, 6S+	6, 6+
PoseEstimation-CoreML	cpm	27	27	32	39	44	115
PoseEstimation-CoreML	hourhglass	6	7	29	42	48	94

Models

Vision

Model Name	Problem	Demo Project	Performance Test	Demo Gif	Added in here
MobileNetV2	Image Classification	Done	Done	Done	`TODO`
Resnet50	Image Classification	Done	Done	`TODO`	`TODO`
SqueezeNet	Image Classification	Done	Done	`TODO`	`TODO`
MNIST	Drawing Classification	`TODO`	`TODO`	`TODO`	`TODO`
YOLOv3-Tiny	Object Detection	Done	Done	`TODO`	`TODO`
YOLOv3	Object Detection)	Done	Done	`TODO`	`TODO`
DeeplabV3	Image Segmentation	Done	Done	Done	`TODO`
FCRN-DepthPrediction	Depth Estimation	Done	Done	Done	`TODO`

Text

Model Name	Problem	Demo Project	Performance Test	Demo Gif	Added in here
BERT-SQuAD	Question Answering	`TODO`	`TODO`	`TODO`	`TODO`

tucan9389 / awesome-ml-demos-with-ios Goto Github PK

awesome-ml-demos-with-ios's Introduction

Awesome Machine Learning DEMOs with iOS

Contents

Machine Learning Framework for iOS

Flow of Model When Using Core ML

Flow of Model When Using Create ML

Baseline Projects

DONE

TODO

Image Classification

Object Detection & Recognition

Pose Estimation

Depth Prediction

Semantic Segmentation

Application Projects

Annotation Tool

Create ML Projects

Performance

📏Measure module

Implements

See also

WWDC

Core ML

Create ML and Turi Create

Common ML

Metal

AR

Examples

awesome-ml-demos-with-ios's People

Contributors

Stargazers

Watchers

Forkers

awesome-ml-demos-with-ios's Issues

Repository Name

Category

Link

DEMO

Model Source

Mobile Machine Learning Framework Used

Inference Time(ms)

Link

Models

Vision

Text

Related Links

Core ML

Create ML and Turi Create

Common ML

Used script for picking the title and url

Recommend Projects

Recommend Topics

Recommend Org