testgroup-bnl / modacs Goto Github PK

Something was clearly lost in translation between machines/environments... Some section headers and some code/quotes need to be correctly formatted. Information on the "Events" system needs to be added.

Description

Currently, after resetting an instrument, the original output files are overwritten, resulting in the loss of any data from before the reset. This is especially undesirable for instruments with initialization procedures (for example, white references with the specs), since that data lost if they are reset mid-flight, potentially making the post-reset data unusable.

Possible Implementation

Instead of overwriting, the output filenames should be appended with a incremental number (or timestamp) after a reset, so that all original data is preserved.

@amcmahon01 When you push the next set of changes could you then also create a new develop branch that we then use for updates and when tested we can merge these into master to help with testing/v control? The idea would be to then have develop the default and master as a stable version plus we will then produce releases based on the master branch release candidates.

Sound good? Initially after you merge you updates master and develop will be identical until we do any bug fixes or additional development. Once we are then satisfied with any changes we can then merge to master.

Thermal camera calibration info should be copied to the output folder so that data source is completely traceable.

Possible Implementation

Thermal camera instrument module should be updated. Cal files are stored in the instrument's directory and already accessed directly, so paths/permissions should not be an issue.

Either 1) utilize on-board spectrometer hardware or 2) saved dark current spectra (or both) to be used as a correction (by subtraction) to the spectral measurements. This should remove instrument noise from the measurements

Utilize USB API for Canon camera to directly record file names with triggers. Ideally could also be used for previews, settings, etc.

Description

Currently, instrument modules on the Rasp Pi have been modified to not display data locally and to only send data to a remote dashboard if using a manual trigger. This should be made optional and assignable to different triggers per instrument.

Possible Implementation

Additional options should be added to instrument config files to determine this behavior. An instrument variable could also be added, allowing the option to be changed on the fly or to be assigned/overridden globally. There could a global flag "LowDataMode" which would result in sending only telemetry to a dashboard, rather than raw data, or "UseLocalDisplay" to enable/disable any display processing locally.

For example
http://www.sciencedirect.com/science/article/pii/S0034425715001248

Provide support for average multiple measurements together into a single saved spectra file.

Description

Currently, if a timed trigger is running and an instrument is reset, the timer continues running and as a result the old instrument object continues to exist since there is still an active reference to it. This results in both instrument objects attempting to run simultaneously and the original timed trigger cannot be stopped.

Possible Implementation

Timed trigger objects should be stopped and destroyed during instrument reset so that the instrument object can be properly destroyed without lingering references. Optionally, the status of the timer (running or not) could be transferred to the new object, but given that some instruments need manual initialization, this is probably best done manually.

Description

Allow a server instance to serve a dynamic web page with all of the dashboard features, eliminating the need for a client instance.

Possible Implementation

There are several packages aimed at Qt web interfaces which could be leveraged, however, the most challenging part will be translating UIs. Ideally, this "conversion" could done programmatically (maybe even at run-time) so that instruments require no new development for support.

Question: Is http://www.paramiko.org/installing.html required for MoDaCS?

If so, we should update the requirements list

Description

If so, we should update the requirements list

Context

Keep requirements up to date

@amcmahon01

Description

Shutdown should be separated into two stages: instrument shutdown, then framework shutdown. This would allow logs and remote dashboards to more accurately monitor instrument shutdowns and any post-processing procedures.

Possible Implementation

Create a separate shutdown signal for core component threads/modules, rather than sharing one between all threads/modules.

Description

A max buffer size should be imposed on the TCP transfer mechanism so that if a connection is dropped real-time display data does not accumulate to unreasonable sizes.

Context

This will reduce memory and swap space usage, improve performance and provide more up to date data when reconnected.

Possible Implementation

Implement a queuing system which either ignores calls after a certain point or a ring buffer that holds only to most recent values.

Instruments, such as the ICI thermal cam, that index their output files in YYYY-MM-DD_hhmmss format will overwrite samples within the same second, keeping only the last one acquired. This has only occurred during sampling "slow downs", usually caused by RAM limits and swap file usage.

Solution:
Add global record number to all filenames to ensure they are unique.

@amcmahon01 If you are comfortable with it, please add Bruce Cook from NASA Goddard (thebruce555) to this repo. He is the PI of NASA G-LiHT which uses Ocean OPtics spectrometers and he is interested in possibly adapting this to run their specs, etc. It may require that we 1) tell him to schedule a time to discuss the package and 2) remind him not to share it widely until we give th go ahead. Thoughts?