leppott / contdataqc Goto Github PK

Occurs during the "QC" step.

In the QC or Stats portion show the number of times a parameter exceeds a threshold value.

Include a table and a plot. The plot would have the threshold identified with a line on the plot.

Only the file version. "Bad file" error message.

Create data output options that will format the data for other data analysis packages.

Section title in Word QC document should be Sensor Depth not Water Level.

The plot is ok for Sensor Depth and Water Temp but the section title is old / inaccurate.

Get with Sys.getenv("USERNAME") and add to all report RMD files.

The QC tests are described but use default settings. Could you actual values from ContData.env.

Introduced with v2.0.1.9005 with fixes for offset time data. Only affects these types of files.

Add the ability to calculate RBI as a new function.

Jen. 2017-03-08. Confirmed on 23rd.

Want ability to create new thresholds and save them.

v2.0.1.9014
Report_QC.rmd (check Report_Aggregate.rmd).
Extra comma at end of FAIL flag assignment.

Example code assumes other steps have been done. Need a stand alone example.

In the example using non-ContDataQC data the summary stats processing text says item 4 of 1. Should be 1 of 1.

For some plots the legend covers up data. Move outside of the plot area. Below?

Aggregate report using seconds instead of minutes for sampling interval.

Add ability for the user to view/modify the thresholds and user defined values.

First step may be just to export to file so user can create a different config file.

Plot data with flags.

Red dots = "F"ail

Orange dots = "S"uspect

May need to be a separate Report, e.g., Report_QC_plotflags.rmd.

Fails on vignette creation. Needs temp data.

Moving from R 3.4.2 to R 3.4.3 the vignette does not install on a clean install.

In the non-File version of Aggregate the QC report is run inside the loop so it runs on each individual file. It should be run on the final file that was put together.

Fixed when creating fun.Aggregate.File(). Need to make the change in fun.Aggregate().

Not formatted for markdown and is old.

From Example:

Use "toupper" for error checking.

In the QC report output the legend is wrong (colors flipped) for Sensor Depth and Water Temp.

Season not being assigned for Dec 31.

Comment from user.

Files are using user input. May want to use content of files. For example, if request 2016-01-01 to 2016-12-31 but only have 3 months of data the file name could reflect that; 2016-06-01 to 2016-08-31.

v2.0.1.0002.

In Config.R the names were copied and not modified. Says pH and DO.

After working with more data decided upon some adjustments to the thresholds.

Jen Stamp mods added. 20170728.

With disparate air and water temperature data the Y-axis isn't large enough.

Test4 in the QC Report.

Add more to the summary stats. Right now just daily means.

Last argument (fun.myFile.Prefix) in ?ContDataQC is unused.

It should be removed.

7-day running mean, min, max.

Input and Output folders are restricted to a base/root directory.

Should modify so that users can have input and output in different locations.

Add the ability to work on a single file. Current work flow uses the user supplied variables (file name, file data, start date, and end date) to search a directory for all matching files.

Users aren't seeing and running the code for the "parameters" and "directories" when running examples. So things aren't working. Make the examples self contained within each block and don't need to run any previous blocks. Makes for longer examples but should be less confusing.

Flag count for each parameter.

Should probably include Fail and Suspect.

This would allow the user to review the file and sort on those with issues.

OR could have an overall flag count of Fail and Suspect for each row. Not sure if need by parameter.

Suggested by Jimmy while working on the TN files.

Make use of futile.logger library for a more developed logging of errors and printing to the screen for users.

Site - Parameter - Julian - MonthDay

other columns are Years (down and across).

Fill in dates at the intersections.

see box plotting as an option

If have dissimilar files will get errors when using the "file" versions of the functions.

For example, if use Aggregate and use files from one or more SiteID or one or more data type (Water or AW) then get a failure.

Need an explicit check and detailed error to the user.

When using the package with Shiny the reports are not working.

Jen. 2017.03.08.

Gaps not always visible in plots in reports. Ensure not interpolating and/or filling gaps.

Summary Stats example in ContDataQC() produces an error.

Need to change order of seasons on plot output from SummaryStats. Currently in alphabetical order.

Data from different sensors in the same file can have different start times (e.g., x:12 and x:17). When the files are combined this causes every other row for each parameter to be NA.

Test user David (email 20170607).

QC of gage data file not picking up all parameters. Only a select few.

Using "Dunfield" dataset the QC step fails to process (NA for time interval) and Reports show time interval as zero.

Using v2.0.1.9013.

Jen, 2017-03-23.