Git Product home page Git Product logo

ftc-object-detection's Introduction

TensorFlowObjectDetector (TFOD) for FTC

Seek Example Thumbnail

What is this?

This repository contains an Android library which enables FTC teams to use machine learning in their OpModes. Specifically, this library makes it possible to use neural networks to do object detection on camera frames. This library requires very little setup, and once running will update recognitions in the background without user interaction, enabling the user to focus on other tasks. Furthermore, this repository contains scripts to enable teams to collect and annotate their own datasets, should they wish to learn more or experiment with different models.

Why did you do this?

Perhaps a better question is what we're hoping to accomplish by releasing this library. In no particular order, here are some of our goals.

  1. Democratize machine learning. We believe machine learning is becoming a fundamental technology that everyone (e.g. all FTC teams) should be able to benefit from and have access to, and this is a step in that direction.
  2. Enable higher levels of autonomy. It's no secret that the reason for machine learning's success has been its unparalleled performance and accuracy across a wide range of real world tasks. By bringing some of this technology to FTC, we hope to push the limits of what autonomous robots can do.
  3. Educate people about ML. We want students to learn about the whole process of using and developing machine learning models, to help them better understand the true extents of what is possible.

How do I get started?

Depending on when you are reading this, this library may or may not already be prepackaged with the robot controller app. If that is the case, then you can jump directly to the usage guidelines. If not, or if you're trying to use this library outisde of the FTC codebase (which is supported, acceptable, and encouraged!), you'll first want to follow the steps in the Installation section. You can then continue to the usage section, as above.

How does it work?

Definitions:

  • frame: A single image, from e.g. a camera or a video
  • model: A pretrained machine learned model which operates on frames
  • recognition: Output from the model, annotating all objects in a frame
  • tracker: An object tracker to smoothly interpolate recognitions

The library's operation can be thought of as being broken into a few distinct steps:

  1. Initialize a source for frames
  2. Load the labels and models into memmory
  3. Start background processing and other objects

Then, the following runs indefinitely:

  1. Get a new frame
  2. If possible, pass the frame to the model
  3. Feed the frame through the tracker
  4. If the model is done processing, feed its results into the tracker
  5. Make the most recent model results (recognitions) available to the client

A more detailed understanding of each of the steps involved can be found in the documentation and comments for each of the parts of the system.

Acknowledgements

We would like to thank Aakanksha Chowdhery, Vivek Rathod, and Ronny Votel for their help and support with TensorFlow Lite and the TensorFlow Object Detection API. We would also like to thank Vasu Agrawal for his tireless work in developing this library, David Konerding and Liz Looney for their mentorship throughout the development cycle, as well as the folks on the FTC SDK team (especially Tom Eng, Bob Atkinson, and Craig MacFarlane) for their help with FTC-speciic integration. Finally, we would like to thank everyone involved in coordinating and participating in our data collection event, including Patricia Cruz, Aaron Cunningham, Calvin Johnson, Nathan Mulcahey, and FTC teams 8381, 11039, 12454, 12635, 12869, and 13799.

ftc-object-detection's People

Contributors

lizlooney avatar vasuagrawal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ftc-object-detection's Issues

Incorrect specification of WINDOW_SCALE in labeler.py for vcxsrv users?

I am using a Windows 10 machine with Ubuntu 18.04 LTS installed as a Linux subsystem on my windows machine.

I am using VcXsrv as the X Windows server on my laptop so I can view XWindows applications that are run on my Ubuntu subsystem.

This works well, but I had a problem with the window being too small to use when I launched labeler.py. I noticed that in the labeler.py script, the window scale is specified as ".75". I compared this to the scale specified in find_bb.py (which runs properly on my VcXsrv server). In the script find_bb.py it specifies the scale as "0.5".

I modified the labeler.py script (see below) and it works well now. The window is large and visible. It appears that for scale values < 1, the value must include a leading "0" in the script:

WINDOW = "Tracking"

# On VcXsrv servers, it seems the scale has to be specified with a leading zero for a value less than 1.
# Otherwise, the displayed window is very small.
#WINDOW_SCALE = .75
WINDOW_SCALE = 0.75

CACHE_SIZE = 150 # 5 seconds worth of frames

I am not sure if this is a problem unique to VcXsrv, but when I ran labeler.py on a Windows machine (natively, using Python3 for windows) I did not encounter this problem.

Security Policy violation Binary Artifacts

This issue was automatically created by Allstar.

Security Policy Violation
Project is out of compliance with Binary Artifacts policy: binaries present in source code

Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.

Remediation Steps
To remediate, remove the generated executable artifacts from the repository.

Artifacts Found

  • TFObjectDetector/tfod/libs/common-1.1.0.jar
  • TFObjectDetector/tfod/libs/support-annotations-27.1.1.jar

Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.


Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

aaptOptions

Directive

aaptOptions {
noCompress "tflite"
}

added to top of TeamCode's build.gradle provokes error message on project sync:

Could not find method aaptOptions() for arguments [build_e5d9rdb1nwf42enje2of40kq2$_run_closure1@3e6d5bf3] on project ':TeamCode' of type org.gradle.api.Project.

Typos in google / ftc-object-detection / training / README.md

Full url to that README.md: https://github.com/google/ftc-object-detection/tree/master/training.

bazel run -c opt tensorflow/contrib/lite/toco:toco --
--input_file=[PATH TO THIS REPO]/training/models/sample_mobilenet_v1_0.5_ssd_quantized/tflite/tflite_graph.pb
--output_file=[PATH TO THIS REPO]/training/models/sample_mobilenet_v1_0.5_ssd_quantized/tflite/detect.tflite
--input_shapes=1,300,300,3
--input_arrays='normalized_input_image_tenor'
--
ouptut_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'
--inference_type=QUANTIZED_UINT8
--mean_values=128
--std_values=128
--change_concat_input_ranges=false
--allow_custom_ops

Look in the middle.

tenor should be tensor.
ouptut should be output.

Conversion of records to checkpoints

The training tutorial https://github.com/google/ftc-object-detection/tree/master/training says. "You can now take the .record files you generated and use them in the same training pipeline you were using earlier in the tutorials. As before, you'll almost certainly want to fine tune an existing model..." I'm not quite sure what are those earlier tutorials. The only tutorial mentioned on the training tutorial is a Medium one for training on the cloud.

I have a good video of poker chips and thumb drives, good records, and a pre-trained model -- the one you supply for Gold and Silver Minerals. My aim is to convert those poker chip and thumb drive records into checkpoints using your Gold and Silver Mineral model, so my phone can recognize poker chips and thumb drives. How do I do this?

I can make a good model using Tensorflow for Poets, but that model does not work with ftc_app version 4.3 ConceptTensorFlowObjectDetection.

Change Orientation of TFObjectDetector

I am relatively new to using TFObjectDetector and my team found it better for the phone to be laid on its side rather than upright. I have looked through the source code and can't seem to find anyway to change the orientation from portrait to landscape. Any help would be appreciated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.