Git Product home page Git Product logo

flutter_opencv's Introduction

flutter_opencv

A Flutter™ plug-in providing a binding to OpenCV-4.x.

Pub Read the Docs Build Status Coverage Status Discord Chat

Flutter OpenCV

Table of Contents

Usage

Installation

In the pubspec.yaml of your flutter project, add the following dependency:

dependencies:
  ...
  opencv: "^1.0.4"

In your library add the following import:

import 'package:opencv/opencv.dart';

Examples

Core concepts

All functions are currently contained in the ImgProc class. e.g. ImgProc.blur(...)

Variables for integer choices (read pre-defined values/enums for border types, threshold types, etc.) are stored in ImgProc & Core classes, exactly similar to how OpenCV itself does. However, these variables are renamed to suit the lowerCamelCase Dart fashion (just to objectively not lose "health suggestion points" on pub.dev.

i.e. One of the border types is Core.BORDER_REFLECT, can be used in ImgProc.blur(someBytes, someIntArray, someIntArray, Core.borderReflect);

So, if you refer to the OpenCV docs or a tutorial, the function you're implementing in OpenCV is very much identical to that in flutter_opencv.

The only difference is how the data is managed - OpenCV uses the Mat class to store every image as a matrix.

flutter_opencv instead only works on data as an array of bytes.

Basic usage

The only difference in OpenCV's methods and ours - OpenCV functions take values of the source & destination matrices by reference. In Flutter, we use byte arrays instead of Mat() & we do not need to provide a destination byte array.

So, basically, someFunction(sourceMatrix, destinationMatrix, otherValue1, otherValue2, ...) in OpenCV is now instead dynamic destinationMatrix = someFunction(sourceMatrix, otherValue1, otherValue2, ...) in flutter_opencv.

e.g. For simple thresholding, in OpenCV, you'd do Imgproc.threshold(sourceMatrix, destinationMatrix, 80, 255, Imgproc.THRESH_BINARY); Whereas in flutter_opencv you'd do res = await ImgProc.threshold(await file.readAsBytes(), 80, 255, ImgProc.CV_THRESH_BINARY);

Input images & formats

Since we're not representing images as Mat(), we're going to need the byte[] data. I'll write down methods for each (Flutter) Image class here. For now,

  1. Image from File: File file = await DefaultCacheManager().getSingleFile(_URL); will fetch the file from the internet. await file.readAsBytes() can be directly passed to any of the functions, since it returns a byte array.

Storing result, temporarily

dynamic outputMatrix = someFunction(...) will store a byte array in outputMatrix.

Output images & formats

For now,

  1. Image from memory: Image.memory(outputMatrix) will show the image obtained (as a byte array) from the above function.

Cascading functions/effects

Since you're never going to just implement one function, there's a need to be able to cascade. Inputs & outputs to all functions are common - byte arrays. Hence output from one function can directly be fed to the next.

FAQs related to the idea/concept behind this plug-in

Why not do a full binding?

I am actually planning to write a full binding and publish it separately later on. I understand that this implementation is lackluster & a lot of functions still need to be added. If you really need something, make a feature request.

Why not use dart:ffi?

I've read up a few bad things about performance benchmarking in ffi, [refer] (https://news.ycombinator.com/item?id=17219291). Due to lack of a better option & the simplicity of method channels, I chose not to go with dart:ffi. Maybe, in the future, someone would build the project on dart:ffi instead of Java/Swift bindings & make it work exactly like OpenCV does. If I ever get the time or resources, I'll start another project in the name of "opencv_advanced".

Why not use the core OpenCV classes like Mat?

Because I wanted to provide a simple interface - one that does not require the user to learn OpenCV or it's API. Instead, much of the way the library works is designed around Flutter/Dart, making it easier for Flutter users to pick it up.

Does lesser functions (in comparison to a full binding) mean lower build size?

Unfortunately, no. The build size would be the same for this, or a fully bound version (which I'll work on after this project reaches certain level of maturity)

Benchmarks?

None yet. I'm still profiling the plug-in for most use cases, and haven't faced any issues with rendering so far. Will add benchmarks soon. PR/PM me if interested.

What OpenCV version is flutter_opencv built on top of?

As of now, the plug-in is built on top of the builds provided for OpenCV v4.1. I know that OpenCV v4.3 is out already, but an issue with the android exportable AAR file prevents me from hosting multiple versions right now. In case you need to change your OpenCV version (read downgrade) while working with this library, you can change it the android/build.gradle file. Just change the dependency version to some other supported one for implementation 'com.quickbirdstudios:opencv:4.1.0'.

Please keep in mind that the above build files (AARs) are provided by someone else - https://github.com/quickbirdstudios/opencv-android. I'm grateful to these guys, but unsure of how often (and far) they would maintain the build repositories, so I'm also working on putting multiple different versions of OpenCV builds on bintray - but it's low on the priority list for now.

Notes

  • iOS not supported currently - I do not own a Mac, so deploying for iOS will be a little slow.

Todos

  • Setup CI for docs
  • Document using Sphinx, store in /docs
  • List future features in the doc
  • Create an issue template
  • Document basic usage
  • Setup CI for build quality/code cov
  • Document/blog about advanced usage
  • Create a feature request template
  • Write tests
  • Integrate iOS, setup projects/kanban board for iOS development
  • Build multiple OpenCV versions, host on bintray

Changelog

Please see the Changelog page to know what's recently changed.

Contributions

Feel free to contribute to this project.

If you find a bug or want a feature, but don't know how to fix/implement it, please fill an issue.
If you fixed a bug or implemented a feature, please send a pull request.

Sponsors

  • BrowserStack for providing their App Live & Automate services for free

As featured in:

Built with flutter_opencv

  • ScanIT Some undergraduate students built this & sent the link across to me. Considering the current flaky build of this plug-in, A+ for the effort.
  • Make a PR if you want to list something you made with flutter_opencv.

Trademarks

“Flutter and the related logo are trademarks of Google LLC. We are not endorsed by or affiliated with Google LLC.”

Copyright(s)

License Agreement For Open Source Computer Vision Library (3-clause BSD License)

Copyright (C) 2020, Aditya Mulgundkar, all rights reserved.
Copyright (C) 2000-2019, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2016, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015-2016, OpenCV Foundation, all rights reserved.
Copyright (C) 2015-2016, Itseez Inc., all rights reserved.
Third party copyrights are property of their respective owners.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the names of the copyright holders nor the names of the contributors may be used to endorse or promote products derived from this software without specific prior written permission.
This software is provided by the copyright holders and contributors “as is” and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall copyright holders or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.

flutter_opencv's People

Contributors

adityamulgundkar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

flutter_opencv's Issues

Consider using null safety

Describe the solution you'd like
It would be useful, if this library would use sound null safety, like basically every new library does nowadays.
This is annoying since flutter forces you to explicitly disable null safety, if ANY dependency does not use it, which means that all the work that I already put in, to make my code null safe is completely invalidated by this library.

Describe alternatives you've considered
I have considered using other dependencies such as opencv_4, however none of the dependencies support the feature that I need (warpPerspectiveTransform).

OpenCV's HoughCircles API.

I have a requirement to use hough transform to detect circles. How can I do that with opencv? OpenCV's HoughCircles API.

Image Stitching

Does this plugin support image stitching (horizontal and vertical)?

Windows Desktop Support

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
Yes, OpenCV Support of Windows, and even backed by Microsoft.

Describe the solution you'd like
As Flutter expands its support for more platforms, camera access on the Windows desktop platform is becoming a challenge, and to write a plugin we need OpenCV integration, which is a much more complex process, thus camera operations are the main concern for the request.

Describe alternatives you've considered
So far there are no packages in Flutter supporting camera support for Windows Desktop, and I have tried to create a plugin with OpenCV integration, which is very time-consuming as I am not from a C++ background.

Edge detection and crop

Is there a plan to add edge detection and crop images?
I am working on an application to take a photo of an ID or document, automatically detect the edges and crop the image.

Regards

[BUG] I/System.out(11892): OpenCV Error: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.Integer

Describe the bug
the following error occurs when using the warpPerspectiveTransform function
I/System.out(11892): OpenCV Error: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.Integer

To Reproduce

Future<dynamic> WarpPerspective() async {
    Uint8List image = widget.file.readAsBytesSync();
    List<dynamic> sourcepoints = [
      tl_x,
      tl_y,
      tr_x,
      tr_y,
      bl_x,
      bl_y,
      br_x,
      br_y
    ];
    double tld_x = widget.tl.dx;
    double tld_y = widget.tl.dy;
    double trd_x = widget.tr.dx;
    double trd_y = widget.tr.dy;
    double bld_x = widget.bl.dx;
    double bld_y = widget.bl.dy;
    double brd_x = widget.br.dx;
    double brd_y = widget.br.dy;
    List<dynamic> destinationpoints = [
      tld_x,
      tld_y,
      trd_x,
      trd_y,
      bld_x,
      bld_y,
      brd_x,
      brd_y
    ];
    List<double> size = [100.0, 200.0];
    print(sourcepoints);
    print(destinationpoints);
    var bytesArray = await ImgProc.warpPerspectiveTransform(image,
        sourcePoints: sourcepoints,
        destinationPoints: destinationpoints,
        outputSize: size);
    setState(() {
      bytes = bytesArray;
    });
    return await bytesArray;
  }

crop,rotate,normalize functionality

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
Yes, It is available in Opencv Python direct function available cv2.rotate or cv2.crop

Describe the solution you'd like
Here crop,rotate shall have direct function like others we have.

Can you guide me, how can I implement it in my own.

Add support of Hough Lines

In my project, I would need line detection. Therefore it would be awesome if you could extend the library with the houghLines-function as you already did with the houghCirlces-function.

The feature is implemented in the Andoird/IOS/Python libraries of open OpenCV (here) and allows detecting lines.
A solution should look like the pictures described in their documentation.

Add basic examples for function cascading/successive functions

Verbatim from the project README:
Since you're never going to just implement one function, there's a need to be able to cascade. Inputs & outputs to all functions are common - byte arrays. Hence output from one function can directly be fed to the next.

This means that all functions take the same byte arrays (exceptions include functions that need a grayscale image instead of a binary one, but the grayscale image too is in a byte array form!), e.g. houghCircles. In general, in OpenCV, a lot can be achieved by using 2-3 different functions in succession. For example, something as simple as Canny edge detection (https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_canny/py_canny.html) would require one to convert the source image to grayscale, then apply some blur to it, then run the Canny function.

Find any good use case for the same and add your function to the examples in example/lib/main.dart which showcases the demo app.

To add an example to the app:

  • Simply add a new switch case in runAFunction
  • Add the string calling this function in the Dropdown Widget
    After you do this, every time someone selects your new function name from the dropdown and clicks on run, your own cascaded/successive function(s) run in the runAFunction method and throw an output image.

Extract frames from a video

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
Yes.

# Python
cap = cv2.VideoCapture(Video_path)
while True:
    ok, frame_image = cap.read()

Describe the solution you'd like
Some way to extract the frames of the video in a loop similar to above implementation.

Describe alternatives you've considered
No alternatives found till now.

Questions about HoughCircle API

Hi, thank you for your great works. I'm a student currently doing a university project to perform Iris and pupil detection for covid patients. For HoughCircle API, could I ask whether there's any way to get the detected circles' radius only instead of the whole processed output picture?

Document usage of functions in the official documentation

Currently, we have no documentation of the existing functions, but they are demonstrated in the examples at example/lib/main.dart under the runAFunction method.

Write base level documentation for the same in Sphinx, in the doc/source/index.rst file. Making changes to this file (or the files in this directory) will directly affect the documentation generated at https://flutter-opencv.readthedocs.io/en/latest/

Just fork the code; write the documentation; send across a pull request to this repository. Once the request is accepted, the documentation pages will rebuild. (i.e. you might want to run Sphinx locally in order to ensure everything works fine, but this is an extra step).

Linux support

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
Yes, OpenCV supports Linux.

Describe the solution you'd like
I would like to use OpenCV in Flutter apps for Linux. Flutter supports Linux as a platform, would be great to be able use OpenCV in Linux Flutter apps.

Describe alternatives you've considered
An alternative might be to implement the web version instead (using WebAssembly or OpenCV.js. It would be just as accessible to Linux users; however, it would seem to be more complex to implement.

Hi. Could you add cv2.inRange func please? Thanks a lot!! By the way flutter_opencv package is excellent!!!

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
If yes, please describe the function/method & class that contains the feature. If not, please explain why Flutter/Dart requires said feature nonetheless.

Describe the solution you'd like
A clear and concise description of what you want to happen. Provide links to existing documentation, if possible.

Describe alternatives you've considered
Have you tried any alternative solutions? If yes, please provide links to the same. If not, can the same feature be achieved by cascading the existing set of effects/methods available in the latest build?

Additional context
Add any other context or screenshots about the feature request here.

Why not use dart:ffi?

The android and ios versions are just wrappers over opencv c++ library. Won't it be beneficial if you can use dart:ffi to create bindings directly to c++. That way you won't have to deal with writing platform specific codes, and at the same time make your library work with not just android and ios, but also on desktop platforms.

[BUG] Function calls block main UI thread

The function calls are futures, but they still block the main UI thread causing jank, frozen app.

Steps to reproduce the behavior:

  1. Load a bigger image as Uint8List. The bigger the image, the longer the freezing time.
  2. Use on this any of the ImgProc methods, ex ImgProc.resize, ImgProc.blur
  3. See freeze

Expected behavior
The method call shouldn't block the main UI thread.

Additional context
If the methods would be sync calls we could use compute, isolates avoiding this issue.

Need some guidance in adding bindings for `grabCut`

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
It is available in OpenCV and the bindings for Python and Java, example: https://www.docs.opencv.org/trunk/d7/d1b/group__imgproc__misc.html#ga909c1dda50efcbeaa3ce126be862b37f

I need it for a pet project I'm working on, and overall I think it would be beneficial to have it in flutter_opencv because it will make the bindings more complete.

I've followed the guidelines in the readme and here is my current draft implementation: #7

Note that I've hardcoded some of the things (number of iterations, the rectangle, etc.) because I am just exploring the feasibility of this approach. Although the code runs without an error, the result is a black image.

grabCut is a bit tricky, because it uses an InputOutputArray variable, which is not featured in any of the existing wrapper-functions and is not mentioned in the readme either. Something might be odd with dst, perhaps it has to be initialized somehow.

@AdityaMulgundkar, can you provide some guidance? Your assistance would be greatly appreciated, thank you forward.

Please add the Findcontour feature

Is the requested feature available in the C++/Java/Python interfaces for OpenCV?
If yes, please describe the function/method & class that contains the feature. If not, please explain why Flutter/Dart requires said feature nonetheless.

Describe the solution you'd like
A clear and concise description of what you want to happen. Provide links to existing documentation, if possible.

Describe alternatives you've considered
Have you tried any alternative solutions? If yes, please provide links to the same. If not, can the same feature be achieved by cascading the existing set of effects/methods available in the latest build?

Additional context
Add any other context or screenshots about the feature request here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.