debrief / limpet_ops Goto Github PK

We wish to develop a self-documenting set of operations. This documentation will be presented to the user in the Operations Browser, and in auto-generated documentation.

Could you add a compulsory schema element that can contain HTML text? Then insert a bit of placeholder text for each one.

As the library of operations grows, we wish to store operations in subject-specific libraries, and let users switch on/off the libraries in use.

Hmm, this may be overkill, but I think we should support libraries in a similar way to Eclipse Views/Categories.

So, a library gets defined once. Then in assorted plugins, we we define an operation we can specify which library it fits into. This will allow libraries to be defined across multiple plugins, and for plugins to contribute operations to libraries defined in other plugins.

The library should have a documentation element, that describes the purpose/content of that library.

We will provide a Preferences page that allows a user to activate/de-activate libraries.

Our Operations Browser ( #2 ) will show a tree of operations, grouped according to the library. Selecting a library in a the list of operations will show the documentation for that library in the detail view.

Would you mind implementing the rest of the FailMessage attributes - I'm keen to check that we can provide applicability feedback for the range of types of operation that we support.

It will also improve the effectiveness of the technical demonstrator if/when I demo it to candidate users.

Dinko - it would make sense to put this fix in the existing Ops Browser PR ( #15 )

Dinko said:

   The sole responsibility of the operation is checking applicability and 
   providing the commands for certain context. My feeling is that Limpet 
   operations should be dropped, applicability will be provided in a declarative 
   way and the Commands should be renamed to Operations. That will 
   simplify the design.

This is good logic, good suggestion. There is an exception to the processing, where the outer class (the operation) does some more complex processing, including deciding the order in which to pass the items to the command, or offering the user multiple options for running the command.

For example, the subtraction isn't commutative. So, if two applicable datasets are selected we need to offer the user the choice of which one is subtracted from the other:
https://github.com/debrief/limpet/blob/master/info.limpet/src/info/limpet/data/operations/arithmetic/SubtractQuantityOperation.java#L85

Similarly, for the "Doppler" calculation, we can create an operation that could take any of the selection as the "Primary" object, and the other selected items as the "secondary track".

If we can handle instances like this declaratively, then we can certainly drop Operations.

Or, I guess we can remove the Operation wrapper for most commands, just keeping a programmatic solution when necessary.

In the current demo (11th April), the Subtract item is only visible in the Operations Browser when an item is selected. See video:
http://screencast.com/t/fjECtYRTGnxx

Dinko - it would make sense to put this fix in the existing Ops Browser PR ( #15 )

We've got an NPE in the unit tests:

In Limpet, most of the data model objects we'll be working with will be collections/series of data items.

So, in this tech demonstrator, could you introduce a couple of arrays as our model objects. We can then develop tests that are only applicable if the selections are both arrays/collections, and if they are of equal length.

I suggest these sample entities:

• [1, 2, 3, 4, 5]
• [2, 3, 4, 5, 6]
• [1, 2, 3]
• ["a", "b", "c", "d", "e"]

With these present, then "add" and "subtract" could apply when the first two are selected (since they contain the same number of elements). "Mean" could apply if one of the first three are selected. "Concatenate" if the list of strings is selected.

To add to the complexity, many of our data structures will be time series, so we could do with some kind of "couplet" entites that allow a value and a timestamp to be stored, such as:

• [ [0, 1.1], [1000, 2.2], [2000, 4.5], [3000, 3.0], [4000, 6.0] ]
• [ [0, 2.1], [1000, 3.2], [2000, 1.5], [3000, 1.0], [4000, 2.0] ]
• [ [0, 3.1], [3000, 2.0], [4000, 1.0] ]
• [ [0, 0.1], [1000, 4.2], [2000, 2.5], [3000, 2.0], [5000, 1.0] ]
• [ [0, 4.1], [1000, 1.2], [2000, 3.5], [3000, 4.0], [4000, 3.0], [5000, 6.0 ] ]

With these present, we could allow "add" for the first two - since they are at the same time stamps.

We could alternatively allow "Add (indexed from 1)", "Add (indexed from 2)", "Add (indexed from 3)" and "Add (indexed from 4) if 1&2&3&4 are selected. When the first one is selected, we will produce values to sum in the other series by interpolating the value at each timestamp in series 1.

This is currently theoretical...

But, as our library of operators grows, I think we should acknowledge that some operators may only relate to specific niches (testing for acoustic data).

It does look like the namespace attribute may be a placeholder for this.

We wish to let users learn more about the applicable elements for operations.

This will show them why the current selection isn't applicable to a particular operation. It will also teach them what kind of elements that an operation will apply to.

In my original spec, I did hope to include the text in the applicable tests:
debrief/limpet#65 (comment)

But, I appreciate that if they aren't available in the parent schema, then we can't use them.

It would be great, however, if we could add these string attributes to the applicable tests - then our logic code can display text to a user regarding the specific test that failed.

So, these appear to be the options (in descending preference order, IMHO):

1. Is it worth considering extending the Core Expressions Framework to allow these optional parameters?
2. Should we come up with a library of error messages, so that when an expression fails, we can lookup the human-readable explanation of why that test failed. But, the library text may not always be correct for the specific context of the test.
3. Alternatively, we can provide an additional "Failed" documentation element that explains why selection matching may have failed. But, it is a maintenance burden to keep this aligned with the actual tests.

Would you mind considering/investigating these?

I'm playing with your branch, Mac OSX.

I select two items. such as the first two numbers. Then I right-click on the second item, to see the list of operations. After about a second, the first item becomes unselected, so only the second one is selected.

I can't imagine that you would implement this, but would you mind checking to see if it happens under MS Windows?

I've had a look at our CollectionComplianceTests

We've got lots of tests that are right down in the weeds, it would be wrong to represent them in our PoC.

But, I do think there could be merit in some slightly more challenging tests, ones that still work for our existing data:

• Exact combination of data this would test that the selection includes the specified number of number items and the specified number of string items (such as two numbers and one string).
• No number data we will allow any selection of types (string & date) that don't contain number data

For both of these, the output will be a concatenation of the toString() operators.

We may choose to build these from the existing operators.

We have a requirement to display a browsable view of the operations.

This RCP view will contain a "master-detail view". The top part will be a list of registered operations. A toggle button will control whether all operations are shown, or just those that are applicable to the current selection.

Selecting an operation will show the documentation element for that operation in the "detail" view. The documentation element is being produced in Issue #1.

The view will also show why the selected operation isn't applicable to the current selection, using the documentation produced in Issue #3.

As you know, our current applicability is defined in code. We can create our operation classes in JUnit tests, and verify that the operations work as expected.

These tests verify:

• that our compliance tests work deliver the expected results
• that our operations are only offered for suitable selections (i.e. that the compliance tests work)

I can't immediately think how we would perform this testing in a declarative context.

Would we run the JUnit as an Eclipse Plugin Test? I guess this could retrieve the operations. We could then retrieve a specific one, and run it?

Or, do we break down our tests into separate blocks of functionality:

• test our high level logic works (currently in SampleView)
• test our applicability tests (properties, support classes)
• test our operations deliver the expected output.

For non associative operations, we need to provide UI options to specify the order of the arguments.

We also need to provide multiple options for some other operations. One such operation is to resample the data. For a resample operation, we offer a choice of frequencies (something like:: 1 sec, 5 secs, 10sec, 30 secs, 1 minute, 2 mins, 5 mins, 10 mins, 15 mins, 30 mins, 1 hour, 2 hours, 5 hrs, 12 hours, 1 day).

Hmm, do we need to represent this declarative? We "could" just handle it at the programmatic stage. But, if we handle it declaratively we can generate the menu options without loading the actual .jar file.

Other examples of operations that require multiple options are:

• moving average (with choice of window size)
• apply smooothing (choice of smoothing algorithm)
• apply filter (again, with choice of algorithm)
• [come on Ian, think of some more....]

IMHO we have an issue with how we display the subtract operation.

Here we have it with two numbers selected:

And here with three numbers selected:

In my opinion it's right to show "Subtract" as a tree/folder - whether there are applicable child operations, or not. It should be consistent across the two.

Hmm, maybe not the same folder as for "Sublibrary sample". That folder is clearly for organising child operations. I can work with my designer to come up with a new tree icon for an operation that contains children. Temporarily, could you use another standard Eclipse icon for these "Operations with permutations" objects?