If many plots are to be displayed in a page (for comparisons) then it makes sense to have some way of fixing the x axis. A startDate and endDate with endDate also having options of 'last' and 'now'.

Can we please add the following typeIDs,
Thanks SS

typeID": "hum",
"name": "humidity",
"unit": "%",
"description": "amount of water vapour in atmosphere"

typeID": "mwindir",
"name": "mean wind direction",
"unit": "degrees, clockwise relative to north",
"description": "mean wind direction over the sampling period"

typeID": "mwindsp",
"name": "mean wind speed",
"unit": "m/s",
"description": "mean wind speed over the sampling period"

typeID": "rain",
"name": "rainfall",
"unit": "mm",
"description": "rainfall over the sampling period"

typeID": "solrad",
"name": "solar radiation",
"unit": "W/m2",
"description": "mean solar radiation over the sampling period"

And this methodID.
"methodID": "weathersta",
"name": "weather station",
"description": "An instrument recording different types of atmospheric conditions"
"reference": "https://en.wikipedia.org.nz/wiki/Weather_station"

Please add the following typeIDs
They all need a methodID of "lab", which is an existing methodID

typeID": "HCO3",
"name": "bicarbonate",
"unit": "mg/L",
"description": "HCO3 concentration in a sample"

typeID": "Li",
"name": "lithium",
"unit": "mg/L",
"description": "Li concentration in a sample"

typeID": "Na",
"name": "sodium",
"unit": "mg/L",
"description": "Na concentration in a sample"

typeID": "K",
"name": "potassium",
"unit": "mg/L",
"description": "K concentration in a sample"

typeID": "Ca",
"name": "calcium",
"unit": "mg/L",
"description": "Ca concentration in a sample"

typeID": "SiO2",
"name": "silicon dioxide",
"unit": "mg/L",
"description": "SiO2 concentration in a sample"

typeID": "H2S",
"name": "hydrogen sulphide",
"unit": "mg/L",
"description": "H2S concentration in a sample"

typeID": "Fe",
"name": "iron",
"unit": "mg/L",
"description": "Fe concentration in a sample"

typeID": "Al",
"name": "aluminium",
"unit": "mg/L",
"description": "Al concentration in a sample"

typeID": "As",
"name": "arsenic",
"unit": "mg/L",
"description": "As concentration in a sample"

typeID": "Rb",
"name": "rubidium",
"unit": "mg/L",
"description": "Rb concentration in a sample"

typeID": "Cs",
"name": "caesium",
"unit": "mg/L",
"description": "Cs concentration in a sample"

typeID": "NH3",
"name": "ammonia",
"unit": "mg/L",
"description": "NH3 concentration in a sample"

typeID": "B",
"name": "boron",
"unit": "mg/L",
"description": "B concentration in a sample"

typeID": "F",
"name": "flourine",
"unit": "mg/L",
"description": "F concentration in a sample"

typeID": "Br",
"name": "bromide",
"unit": "mg/L",
"description": "Br concentration in a sample"

typeID": "NO3-N",
"name": "nitrate as nitrogen",
"unit": "mg/L",
"description": "NO3-N concentration in a sample"

typeID": "PO4-P",
"name": "phosphate as as phosphorus",
"unit": "mg/L",
"description": "PO4-P concentration in a sample"

typeID": "SO4",
"name": "sulphate",
"unit": "mg/L",
"description": "SO4 concentration in a sample"

typeID": "d18O",
"name": "ratio 18O:16O",
"unit": "per mil",
"description": "d18O concentration relative to standard"

typeID": "d2H",
"name": "ratio 2H:1H",
"unit": "per mil",
"description": "d2H concentration relative to standard"

Add the ability to overlay two or more stations on the same plot/axis so they can be compared directly.

for typeID=z
methodID": "bubble",
"name": "bubbler system",
"description": "gas purge bubbler system, water level is proportional to gas pressure",
"reference": "http://automationwiki.com/index.php/Bubbler_Level_Measurement"

for typeID=fr
methodID": "encode",
"name": "stage encoder system",
"description": "a potentiometer with a wheel and pulley system connected to a float in a stilling well to provide an electronic reading of the water level",
"reference": "http://www.rickly.com/sm/Float-Type/ShaftEncoders/ShaftEncoders.htm"

Thanks

For the typeID=rs, can you please add the following methods

"methodID": "nofilt",
"name": "unfiltered",
"description": "data not filtered before RSAM calculated",
"reference": ""

"methodID": "nofilt-d",
"name": "unfiltered daily average",
"description": "data not filtered before RSAM calculated, daily average value",
"reference": ""

"methodID": "2-5hz",
"name": "2-5 Hz filter",
"description": "data band-pass filtered 4th order 2-5 Hz before RSAM calculated",
"reference": ""

"methodID": "2-5hz-d",
"name": "2-5 Hz filter daily average",
"description": "data band-pass filtered 4th order 2-5 Hz before RSAM calculated, daily average value",
"reference": ""

"methodID": "1-4hz",
"name": "1-4 Hz filter",
"description": "data band-pass filtered 4th order 1-4 Hz before RSAM calculated",
"reference": ""

"methodID": "1-4hz-d",
"name": "1-4 Hz filter daily average",
"description": "data band-pass filtered 4th order 1-4 Hz before RSAM calculated, daily average value",
"reference": ""

"methodID": "1.5-6hz",
"name": "1.5-6 Hz filter",
"description": "data band-pass filtered 4th order 1.5-6 Hz before RSAM calculated",
"reference": ""

"methodID": "1.5-6hz-d",
"name": "1.5-6 Hz filter daily average",
"description": "data band-pass filtered 4th order 1.5-6 Hz before RSAM calculated, daily average value",
"reference": ""

Thanks

Allow for showing more than one observation on a plot. By listing multiple sites and types in the query.

Please add a query endpoint that allows the user to retrieve a summary of statistics for the query. The query parameters should be the same as observation and the returned format should be a JSON summary of the stats parameters for the data in the query range. This is instead of returning the CSV data.

The url query should be like

http://fits.geonet.org.nz/observation/stats?typeID=e&siteID=HOLD&networkID=CG also allow for &days=100 etc

The following values should be returned for the data in the query

• min - the date time, value, and error for the minimum observation.
• max - the date time, value, and error for the maximum value observation.
• first - the date time, value, and error for the first observation.
• last - the date time, value, and error for the last observation.
• the mean - the statistical average from the postgres function avg() http://www.postgresql.org/docs/9.4/static/functions-aggregate.html
• the population standard deviation from the postgres function stddev_pop(expression)

I don't have the JSON format yet - we will have to ask some users and see what they think. We also want to allow for adding a number of other stats about the data from postgres. I initially suggest something like:

{
"Maximum": {
                    "DateTime": "2012-07-31T12:01:04.000000Z",
            "Value": 12.5,
            "Error": 0.0
                    },
"Minimum": {
                    "DateTime": "2012-07-31T12:01:04.000000Z",
            "Value": 12.5,
            "Error": 0.0
                    },
...
"Mean": float64,
"StddevPopulation": float64
}

Comment from Steve:

One thing we noticed was that the siites on the map are labelled by siteID, 
but in the site selection box it is by site name. The site name isn't necessarily 
obvious knowing the siteID. Can the selection box include siteID somehow? 

In the Select Sites dialogue could you please add the SiteID as a prefix before the name? Say like:

GISB - Gisbourne

And sort the list alphabetically (including the SiteID).

Add option to specify height and/or width of plot axes.
Although the output is a scalable vector graphic (SVG) all the features (text, line widths, etc) are scaled and distorted proportionally so that enlarging the output from the default size does not clarify any detail it simply plots the same image larger on the screen.
As an example, the image below shows the same plot for MAVL displayed at the default size (800x270) and a specified size of (800x800) in an attempt to exaggerate the y-scale. This only distorts the image without improving the visible detail because all line widths and font sizes are scaled proportionately.

The readme for Docker deployment is out of date. Correct it.

from Kevin:

Example Query and Response - replace empty grey panel with an image of the example query.

It's XML and I'm not totally sure what to display.

Would you be able to implement median, lower 20% percentile and upper 80% percentile to plots? This would then allow us to say something like 'the temperature of Ruapehu Crater Lake is 42 degC; it is above 37degC only 20% of the time, so we think it is really high' , or something like that. Reading the documentation I think that percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression) does that?

The reason for this is that no assumptions on the distribution or independence of data points are made.

The general idea of these kind of basic statistics on plots is really great and allows us to make some very quick assessments of the data very easily.

Hi Geoff, can you add me a couple of methodIDs when you get a chance. Sorry, every dataset I look at needs something new. It doesn't seem to matter that I thought about this quite a bit before starting to look at datasets.

for typeID=fr

methodID": "vela",
"name": "velocity - cross-sectional area",
"description": "flow rate is determined by the product of velocity and channel cross-sectional area, with a roughness factor (of 0.7)",
"reference": "http://en.wikipedia.org/wiki/Discharge_(hydrology)"

for typeID=mg and typeID=cl (and most other chemical analyses)

methodID": "lab",
"name": "geothermal water chemistry laboratory",
"description": "unspecified analysis in a geothermal water chemistry laboratory",
"reference": "http://www.gns.cri.nz/Home/Services/Laboratories-Facilities/New-Zealand-Geothermal-Analytical-Laboratory"

As there are so many parameters it might be more user friendly if the parameters were grouped. Currently to look at GPS/GNSS plots you select 'up' or 'east'

Also more than half of the first 10 parameters don't have any data ATM?

Can we query and calculate an average (mean) over a fixed time interval as part of the query? For example, if we have data sampled at an interval of one hour or at irregular intervals smaller than one day, can we query those data a get daily averages returned (and plotted), instead of the original data? I'm thinking of something like avg=d as part of the query url.

For volcanic flux data,we will implement the fluxes from individual scans, so that we have the potential to look at how flux changes over short time intervals. But for longer time periods a daily average value will still be required. If that can be done, it would also be nice to be able to query avg=m or avg=y (for month or year). Is &avg=10d (10 day average) possible? For these I am referring to a fixed window length with no overlap. So if we queried with avg=m, the dates would be the first of each month, and the averaged values would be all those in that month.

Can these queries return the standard deviation or the standard error of the values in the 'error' column?

Are some other query modifiers of a similar type possible? For example, count in the window, minimum value, maximum value

From Kevin:

Can the returned file 'observation' have an extension CSV ? Windows treated the file as an extensionless file.

There are situations where the same kind of data are collected by more than one method, and it is useful to distinguish the data by the method used, maybe because the errors are different or some other factor influencing the data values.

I'd like to be able to specify the method in the query, maybe not always, but optionally.

An example of the kind of data I am talking about can be found at http://volcano.gns.cri.nz/volcanoes/whiteis/lake_height.png

Thanks,

SS

Can the unit for typeID=SO2-flux-a be changed from t/d to kg/s (tonnes/day to kilograms/second).

Volcanic flux data already in FITS are daily averages with units of tonnes/day. Daily averages and tonnes/day as units are common international usage in volcanology.

But, we want FITS to contain the flux values calculated from individual instrument scans, as these are the basic data we collect. Those scans are currently every minute or every 4 minutes (depending on the instrument type). Not all scans produce data, so uploading individual scans does not imply a massive volume of data. For a scan that takes such as a short period of time, a unit of tonnes/day is not appropriate, and we want to use kg/s. This is better as it is a SI unit, and Agnes says it is becoming more commonly used in the community.

Can we please query for the data range for a particular SiteID-TypeID combination. It would return two date/times, being the first and last dates for data.

something like siteID, location, observation in a time window / spatial area.

Sites with a type of observation inside a polygon.

Add a service to get information for a single site.

Latest val label is getting trimmed for leveling data plots.

While trying dygraph I founded that the observation endpoint's output csv cannot be referred from other website's JavaScript.

If we only want users to download our csv then it's OK now. But if we want to provide our api (data) to any website on the fly maybe we should add "*" to Access-Control-Allow-Origin.

For typeID=SO2-flux-a we have an existing methodID=flsp. This is for a flyspec instrument either flown in a plane or driven in a car. We need to account for a "scanning flyspec" which is the same instrument operating in a different mode. This is what is used on Tongariro. Please add the following:

methodID: "sflsp",
name: "scanning-flyspec",
description: "miniaturized UV spectrometer system measures absorption of ultraviolet light by SO2. Stationary ground-based system scanning from horizon to horizon ".
reference: ""

Can you please add some new typeIDs and methodIDs.
New typeIDs. These are for airborne volcano gas measurements (at several volcanoes).

typeID": "CO2-flux-a",
"name": "Carbon Dioxide flux in air",
"unit": "t/d",
"description": "flux of carbon dioxide measured in the air"

typeID": "H2S-flux-a",
"name": "Hydrogen Sulphide flux in air",
"unit": "t/d",
"description": "flux of hydrogen sulphide measured in the air"

Can we replace the typeID for SO2-flux-a to be compatible with these as they are better described
typeID": "SO2-flux-a",
"name": "Sulphur Dioxide flux in air",
"unit": "t/d",
"description": "flux of sulphur dioxide measured in the air"

New methodIDs for SO2-flux-a and H2S-flux-a and CO2-flux-a
methodID": "cont",
"name": "contouring",
"description": "aircraft flies through a plume at different altitudes, a pump inside the aircraft sucks in gases and instruments measure the concentration",
"reference": "http://info.geonet.org.nz/display/volc/Gas"

New methodIDs for SO2-flux-a
methodID": "cosp",
"name": "cospec",
"description": "correlation spectrometer measures absorption of ultraviolet light by SO2, flying beneath a plume",
"reference": "http://info.geonet.org.nz/display/volc/Gas"

New methodIDs for SO2-flux-a
methodID": "flsp",
"name": "flyspec",
"description": "miniaturized UV spectrometer system measures absorption of ultraviolet light by SO2, flying beneath a plume",
"reference": "http://info.geonet.org.nz/display/volc/Gas, http://www.flyspec-inc.com/"

For a plot query I can add "days=100" to get the last 100 days in the plot. Can the same functionality be added for the observation query? Or possibly even someway of defining a date range, so the returned csv file contains only data between date1 and date2

Add visual observations.

• These might not be associated with an observation.
• They have free text comments.
• They have a URL link (for an image).

Are there multiple images?

as on this plot

http://fits.geonet.org.nz/plot?networkID=NZ&siteID=OTVZ&typeID=nve&type=scatter&showMethod=true

The max x axis and grid interval at the moment is 1 year. This needs increasing for some plots e.g.,

http://fits.geonet.org.nz/plot?networkID=VO&siteID=NA001&typeID=t&yrange=350&type=scatter

This should be more like every 5 or 10 years for the above plot.

This will serve as useful documentation for users and people loading data.

e.g., on this plot

http://fits.geonet.org.nz/plot?networkID=NZ&siteID=OTVZ&typeID=nve&type=scatter

The ability to specify the time period for the x axis (either start and end or start and duration).
The "days" parameter partially achieves this but does not allow for viewing a specific time period in the past. Possibly an "enddate" parameter would work in conjunction with the "days" parameter.

Add a ground relationship for sites.

standardize the request routing.
add api docs.

A program for users to load typeIDs and methodIDs from text file of a standard format would be a very useful addition. These would allow users who cannot write SQL the capability to load these parameters. To the user the program would operate in a similar manner to fits-loader.

A new GPS method needs to be added for the Gamit/GlobK time series.

"methodID": "gamit",
"name": "Gamit/GlobK",
"description": "Gamit/GlobK GNSS processing software",
"reference": "http://www-gpsg.mit.edu/~simon/gtgk/"

Not sure if we need to add a new type for Gamit time series, or can we use the same type e, n, u and e_rf, n_rf, u_rf but related to this new method?

From Kevin:

Geoff,

I've worked through these pages and I certainly got the hang of things fairly quickly. Only minor tweaks, really:

Response Properties: "This query returns an SVG image." (could link 'SVG image' to http://en.wikipedia.org/wiki/Scalable_Vector_Graphics)

Page http://fits.geonet.org.nz/api-docs/endpoint/method
Typo: "a valid type indentifier" -> "a valid type identifier"

Page http://fits.geonet.org.nz/api-docs/endpoint/observation

Could usefully insert hyperlink for 'ISO8601 format' -> http://en.wikipedia.org/wiki/ISO_8601

Page http://fits.geonet.org.nz/api-docs/endpoint/plot
Typo: "with the days query paramter." -> "with the days query parameter."

Page http://fits.geonet.org.nz/api-docs/endpoint/site
Typo: "the obseravation type." -> "the observation type."
Typo: "Site above ground level" -> "Sites above ground level".

Page http://fits.geonet.org.nz/api-docs/endpoint/type
Add full-stop to "a short name for the type"

Thanks very much for this, it really covers the ground well for end-users.
Kevin

Make Optional consistent and visually distinct.

This needs some more work for very small ranges. From Steve:

"I also noticed that the Taupo lake levelling plots have a very small yrange so that no labels appear at all as the range is less than the minimum label interval. See here http://fits.geonet.org.nz/plot?typeID=z&siteID=TP003&networkID=VO&yrange=.05
In this case we might argue that the problem is that the data are stored as heights in metres. If there were in mm we wouldn't have a problem. I can talk to Nico/Brad about that if you'd prefer me to deal with it that way. If not, can you account for small y-ranges too."

Can you please add TypeIDs for the following:

typeID": "SO2-flux-a",
"name": "Sulphur Dioxide",
"unit": "t/d",
"description": "Sulphur Dioxide"

method.name = multiple mini-doas stations
method.methodID = mdoas-m
method.description = miniature differential optical absorption spectrometer, multiple stations
method.reference = http://www.geo.mtu.edu/volcanoes/vc_web/tools/doas_1.html

method.name = single mini-doas station
method.methodID = mdoas-s
method.description = miniature differential optical absorption spectrometer, a single station
method.reference = http://www.geo.mtu.edu/volcanoes/vc_web/tools/doas_1.html

Add a detrend option. ie calculate a simple linear regression (trend line) and subtract this from the data. This was an option with the Java applet and is useful to see any deviations from the general trend (eg slow slip events) when the data is otherwise trending steeply. An example of where this would be useful is station GISB where the average velocity east is almost zero so the variations in the east component are obvious (http://fits.geonet.org.nz/plot?networkID=LI&siteID=GISB&typeID=e). Whereas, there is a significant velocity north which somewhat masks the variations in this component (http://fits.geonet.org.nz/plot?networkID=LI&siteID=GISB&typeID=n)). Detrending would highlight the variations (slow slips) more clearly.