Line Filter should take a minimum and a maximum.
It should indicate what the length's scale is, or alternatively it should be relative to the size of the mat so it doesn't matter if the resize operation changes.

Problem

Registering an object to the event bus inside of a constructor call can result in a subscriber being called prior to all of the fields in the class being fully constructed. This can lead to null pointer exceptions for fields that are final and should have been initialized in the classes constructor.

Relavent Gitter Discussion Discovery of problem and discussion of how to fix it.

Guava Google Groups Discussion with Dev

Don't publish the "this" reference during construction

One of the mistakes that can introduce a data race into your class is to expose the this reference to another thread before the constructor has completed. Sometimes the reference is explicit, such as directly storing this in a static field or collection, but other times it can be implicit, such as when you publish a reference to an instance of a non-static inner class in a constructor. Constructors are not ordinary methods -- they have special semantics for initialization safety. An object is assumed to be in a predictable, consistent state after the constructor has completed, and publishing a reference to an incompletely constructed object is dangerous. Listing 2 shows an example of introducing this sort of race condition into a constructor. It may look harmless, but it contains the seeds of serious concurrency problems.
-Brian Goetz

Resolution

This can be resolved by ensuring that constructors never call a register method

When you click preview the preview shows up on the right of the preview pane instead in the same order of the pipeline.

I can full screen and minimize but not resize the window manually. The split screen is working however.

It is impossible to remove a webcam source.

What were the actions performed prior to this error appearing?

System Info:

Stack Trace:

java.lang.IllegalStateException: Failed to grab image
    at edu.wpi.grip.core.sources.CameraSource.lambda$startVideo$23(CameraSource.java:110)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.bytedeco.javacv.FrameGrabber$Exception: cvGrabFrame() Error: Could not grab frame. (Has start() been called?)
    at org.bytedeco.javacv.OpenCVFrameGrabber.grab(OpenCVFrameGrabber.java:244)
    at edu.wpi.grip.core.sources.CameraSource.lambda$startVideo$23(CameraSource.java:108)
    ... 1 more

I created a FindLines and FilterLines operation and turned on the image previews for them. Nothing showed up in the preview area, but I did notice that the scrollbar changed, indicating that it was allocating an empty window about the right size. It did work the first time I tried it. I'll continue to play with it to see if I can find a pattern.

When using "mask" type mats, it would be super helpful to have some sort of simple editing functions in GRIP for creating simple mats. Single-channel rectangle drawing would be fine for this.

Also opening a camera that does not exist causes an exception.

edit: Opening cameras on windows does not work, if the camera exists on that slot or not. The problem seems to be in the way you handle the camera class with opencv. I have used a program written with java to process an image.

System Info:

Stack Trace:

Exception: cvCreateCameraCapture() Error: Could not create camera capture.

java.io.IOException: A problem occurred trying to start the frame grabber for Webcam 1
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:97)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:68)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:43)
    at edu.wpi.grip.ui.pipeline.AddSourceView.lambda$null$39(AddSourceView.java:94)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.bytedeco.javacv.FrameGrabber$Exception: cvCreateCameraCapture() Error: Could not create camera capture.
    at org.bytedeco.javacv.OpenCVFrameGrabber.start(OpenCVFrameGrabber.java:179)
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:95)
    ... 6 more

• minMaxLoc
• anything to find a line
• anything to find circles

Using this save file:
Move your camera very fast. The image will show through even though the "show original image" is unchecked.

<grip:Pipeline>
  <sources>
    <grip:Camera>
      <property name="deviceNumber" value="0"/>
    </grip:Camera>
  </sources>
  <steps>
    <grip:Step name="resize">
      <grip:Input step="0" socket="0"/>
      <grip:Input step="0" socket="1"/>
      <grip:Input step="0" socket="2">
        <value>0.25</value>
      </grip:Input>
      <grip:Input step="0" socket="3">
        <value>0.25</value>
      </grip:Input>
      <grip:Input step="0" socket="4">
        <value>INTER_LINEAR</value>
      </grip:Input>
      <grip:Output step="0" socket="0" published="false" previewed="true"/>
    </grip:Step>
    <grip:Step name="Find Blobs">
      <grip:Input step="1" socket="0"/>
      <grip:Input step="1" socket="1">
        <value>1</value>
      </grip:Input>
      <grip:Input step="1" socket="2">
        <value>
          <double>0.0</double>
          <double>1.0</double>
        </value>
      </grip:Input>
      <grip:Input step="1" socket="3">
        <value>false</value>
      </grip:Input>
      <grip:Output step="1" socket="0" published="false" previewed="true"/>
    </grip:Step>
  </steps>
  <connections>
    <grip:Connection>
      <grip:Output step="0" socket="0" published="false" previewed="true"/>
      <grip:Input step="1" socket="0"/>
    </grip:Connection>
    <grip:Connection>
      <grip:Output source="0" socket="0" published="false" previewed="false"/>
      <grip:Input step="0" socket="0"/>
    </grip:Connection>
  </connections>
  <sink>
    <value class="edu.wpi.grip.core.sinks.DummySink"/>
  </sink>
</grip:Pipeline>

Do you have any plans to package Kinect support with GRIP? I'm not sure how it would work, since every platform implements drivers differently, but it could potentially be very useful.

Each source will be associated with a specific source view.

After shutting down my IP webcam, GRIP threw a null pointer error. Ideally, GRIP would notice that it lost connection, and display the last frame it received as well as a "lost connection" notice somewhere in the UI. Ideal behavior would also include a UI control or intermittent pinging to restore connectivity when it becomes available.

Stack Trace:

java.lang.NullPointerException
    at org.bytedeco.javacv.IPCameraFrameGrabber.grab(IPCameraFrameGrabber.java:121)
    at edu.wpi.grip.core.sources.CameraSource.lambda$startVideo$23(CameraSource.java:108)
    at java.lang.Thread.run(Thread.java:745)

Codacy detected an issue:

Message: Switch statements should have a default label

Occurred on:

• Commit: ed3e9ee
• File: src/main/java/edu/wpi/grip/ui/util/ImageConverter.java
• LineNum: 60
• Code: switch (channels) {

Currently on:

• Commit: c0df380
• File: src/main/java/edu/wpi/grip/ui/util/ImageConverter.java
• LineNum: 60

It will only work on two images. However, OpenCV allows for a single image and a scalar. Is this something we should fix?

Don't know how to fix this. We should probably check the log file and try to figure out what is going wrong.

Chocolatey installed 0/1 package(s). 1 package(s) failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log

This is more intuitive than having to press the enter key in a field to commit the value to the step.

In aerial assist, you could get more points in autonomous if your robot scored in the "hot" goal. One of the approaches to this was using computer vision to identify which goal was lit up There's a page on screensteps explaining one approach (there's also c code using the NI libraries)

If add the following features, this program would theoretically be useful in 2014:

• HSV/HSL threshold operation
  • Can be implemented similarly to RGBThresholdOperation
• Particle analysis
  • Find a list of clusters
    • k-means might work for this
  • Filter particles
    • Minimum size
    • Aspect ratio range

The remaining math in the screensteps page would probably happen in the control system code, not in GRIP.

The step window for adaptiveThreshold has labels that are too long, causing them to be hard to read.

Sources should not be started until either the user chooses to add it in the UI, the core triggers a start event (headless. This will ensure that sources that are loaded from a

Starting a source:

• UI
  • New sources (added by the UI) should be verified that they can start without a problem
    • These sources will start running after the creation UI closes
  • Sources loaded from a save file will be created stopped
    • These sources will start running after the user presses the "play" button
    • There will also be a "start all sources" button that will post a StartAllSourcesEvent
• Headless mode
  • Sources loaded from a save file will be created stopped
    • These sources will be initialized with a post StartAllSourcesEvent

Closing a camera crashes Grip, the camera wont show up even if i put in the right values.

Pipeline should be run in a separate thread

The pipeline should have a timer and will run at a specific rate.
Setting the source value will no longer runs the pipeline; instead it flags a dirty bit.

for(;;) {
    if(sem.wait(time) || sources.dirty) {
        run();
    }
}

This will make everything sequential and makes mutable data safe.

Please create a list of operations and inputs that should be removed and leave them here.

• batch distance
• ddepth
• houghlines
• mask <== Better to keep and extend the functionality.
• cvtColor -> dstCn

While running Find Lines and Filter lines for an extended period of time I encountered a blue screen. I was running a TV show on a second monitor which might been related.

Currently it is possible to connect multiple things to an input of a step. This shouldn't be allowed - it makes the value of the socket ambiguous and dependent on the timing of the events in the application. Instead, when the user tries to make a connection to an input socket that is already connected to something, we could either not allow it, or just remove the old connection.

Dependency Injection

When implementing dependency injection there are two common patterns that are used. Constructor Injection and Field Injection.

Please use this post to collect evidence supporting/against one pattern or the other.

Field Injection

• Harder to test

  Field Injection
  Guice injects fields with the @Inject annotation. This is the most concise injection, but the least testable.
  -Guice wiki

• Slower

  Second issue is performance. A constructor call (direct or by reflection) is always faster than a bunch of reflection field assignments. Dependency injection frameworks must use reflection analysis to build dependency tree and create reflection constructors. This process causes additional performance hit.
  -Maciej Chałapuk

• Breaks the encapsulation paradigm

  From the philosophical point of view, field injection breaks encapsulation, which is one of four fundamentals of object-oriented programming, which is the main paradigm of Java.
  -Maciej Chałapuk

• Encourages Dependency hiding

  If there was one thing I could take away from #Spring it was being able to use @Autowired on fields… sigh #BedCon
  It encourages hiding dependencies, which is the opposite of what DI actually wants to achieve…

  • Oliver Gierke

• Considered 'Auto-Magic' by manny

  So i have always stayed away from field injection... There is a lot of what I consider automagic behind the scenes, and from my opinion I always prefer explicit over implicit.

  • Scott Hepburn (Kronos Inc. Software Engineer, uses Spring)

• Impossible to create instances of a class without using reflection

• Reduces the amount of "boilerplate" code you have to write.

• JavaFX uses field injection by default

Constructor Injection

• Promotes smaller classes

  • If you start adding too many constructor fields it becomes a sign to refactor your code.

• Promotes being able to define concrete state rules about a class.

  The core of the problem here is that code you’ve written allows clients to create instances of the class in an invalid state. The whole purpose of a type is clients being able to rely on the invariants it enforces. It’s one of the reasons you use an EmailAddress type over a plain String to represent email adresses in your code: clients can be sure the instance they get is a valid email address as the value object enforces this constraint during construction. A String can potentially be anything, validated or not - how do you know?
  -Oliver Gierke

• Provides a way to write tests without an injection framework

  At least with constructor and setter injection, you have the option of not using the frameworks and/or reflection in your tests.

  • Marjan Venema

Project Lombok will solve the code bloat problem.

Event: ErrorEvent

• Throwable
• Optional Message
• Origin

Event: ErrorClearedEvent

• Origin

ErrorWitness (final class)

• Constructor
  • EventBus
  • Origin
• Methods
  • error
    • throwable
    • Optional message
  • allClear
    • only posts the status when the witness's state has changed.
  • question
    • returns true if there is no error

Needs a few additional parameters for the operation that is there to be really useful. Then you can think about how to do a more general version of blob finding operations.

...At least in ubuntu. Is this something we want? Opencv allows for differently sized images, but the usage is complicated.

See:
http://docs.opencv.org/3.0.0/d7/d1b/group__imgproc__misc.html#ga72b913f352e4a1b1b397736707afcde3

Should only take THRESH_BINARY and THRESH_BINARY_INV, but has dropdown items for all the threshold types.

What were the actions performed prior to this error appearing?

yeah uh uh type - Jonathan Leitschuh

tried to use a webcam on windows

System Info:

Stack Trace:

java.io.IOException: A problem occurred trying to start the frame grabber for Webcam 0
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:97)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:68)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:43)
    at edu.wpi.grip.ui.pipeline.AddSourceView.lambda$null$39(AddSourceView.java:94)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.bytedeco.javacv.FrameGrabber$Exception: cvCreateCameraCapture() Error: Could not create camera capture.
    at org.bytedeco.javacv.OpenCVFrameGrabber.start(OpenCVFrameGrabber.java:179)
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:95)
    ... 6 more

Trying to use any webcam other then 0 when your system only has 1 webcam throws an exception. Ideally GRIP would enumerate the webcams on your system and only present ones that won't make it crash.

Stack Trace:

java.io.IOException: A problem occurred trying to start the frame grabber for Webcam 1
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:97)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:68)
    at edu.wpi.grip.core.sources.CameraSource.<init>(CameraSource.java:43)
    at edu.wpi.grip.ui.pipeline.AddSourceView.lambda$null$39(AddSourceView.java:94)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.bytedeco.javacv.FrameGrabber$Exception: cvCreateCameraCapture() Error: Could not create camera capture.
    at org.bytedeco.javacv.OpenCVFrameGrabber.start(OpenCVFrameGrabber.java:179)
    at edu.wpi.grip.core.sources.CameraSource.startVideo(CameraSource.java:95)
    ... 6 more

What were the actions performed prior to this error appearing?

I added a webcam. I'm using a USB webcam (Logitech 9000 Pro) on Windows 10 64-bit

Message: Camera Frame Grabber Thread crashed with uncaught exception

System Info:

Stack Trace:

java.lang.ArithmeticException: / by zero
    at edu.wpi.grip.core.sources.CameraSource.lambda$start$18(CameraSource.java:158)
    at java.lang.Thread.run(Thread.java:745)

Sources

1. Having multiple clients try to access the frames of a webcam at the same time can create problems (right??) so we have to ensure that when GRIP deploys we have to stop the local instances sources.
2. Currently it is impossible to create a source that is not valid on your machine. This source may, however, due to networking or webcams being plugged into different ports, be valid on the roboRIO, how do we handle this?

Sinks

If we have a sink that is publishing after we deploy GRIP over to the robot then there will be two network tables nodes publishing to the same values. As such, we should make it so that when when we push the jar over to the robot it should should suspend camera access on the local machine.

This instead should be handled with a more subtle notification.

Is this OK?

:ui:run
ID Type Value
Dec 10, 2015 9:50:18 AM com.sun.javafx.css.StyleConverterImpl getInstance
SEVERE: StyleConverterImpl : converter Class is null for : com.sun.javafx.scene.layout.region.CornerRadiiConverter
Dec 10, 2015 9:50:18 AM com.sun.javafx.css.StyleConverterImpl readBinary
SEVERE: could not deserialize com.sun.javafx.scene.layout.region.CornerRadiiConverter
could not deserialize com.sun.javafx.scene.layout.region.CornerRadiiConverter

Building 90% > :ui:run

If you supply a mask param you start getting a ghosting effect on the image.

This was broken by the refactor

When a connection to an input socket is removed, the socket still has the last value obtained from that connection. One of the reasons this is bad is because some types (like Mats) are basically pointers, so the value of the input socket keeps changing as long as the Mat keeps getting modified by the step it used to be connected to.

Since OpenCV's inRange method is prone to segfaulting with the wrong input it should be avoided. Instead, inRange should be wrapped in a helper function that provides certain safety guarantees to ensure that a segfault is not possible.

Let's collect all of the issues related to the generated OpenCV functions in one place so we can keep up with the progress on them easily. Here are the ones I've found by playing around with it a little.

• We probably shouldn't map an input socket to the dtype or ddepth parameter. We want to provide as low-level access as necessary, but these don't really seem to map to useful ideas in the application, even though they happen to parameters to many opencv functions.
• accumulateSquare (and other accumulate functions that we don't wrap) aren't really operations, they're more like implementations of the += operator. It doesn't make sense to make operations for them.
• batchDistance crashes. The domain for K should be 1 to MAX_VALUE, not MIN_VALUE to MAX_VALUE.
• canny crashes. We need to implement checkbox socket controls.
• compare, flipcode, etc... Many enums are shown as integers. We should detect which parameters seem to be enums.
• cornerHarris, cornerMinEigenVal, possible more output Mats of floats. We should allow float Mats to be previewed.
• dilate probably remove the border stuff, or at least make it actually work.
• normalize keeps printing errors about "Unknown/unsupported norm type". I guess the enums aren't being generated with the right values.
• pow, sqrt only work on floating point Mats. I'm also not sure if we need to support these operations right now, but if we discover that taking the square root of an image is useful, we should add a way to convert between pixel data types.

Codacy detected an issue:

Message: Use explicit scoping instead of the default package private level

Occurred on:

• Commit: b7a0568
• File: src/main/java/edu/wpi/grip/ui/pipeline/input/RangeInputSocketView.java
• LineNum: 18
• Code: final RangeSlider slider;

Currently on:

• Commit: c0df380
• File: src/main/java/edu/wpi/grip/ui/pipeline/input/RangeInputSocketView.java
• LineNum: 18

When GRIP is resized larger than its default size operations that are hovered will disappear. This happens on windows when full screened and also on OSX sometimes.

The function to search the palette list of steps does not search the step descriptions.