Axis75 high-frequency noise about internal_waves_barkley_canyon HOT 8 CLOSED

• Hump gets worse above 900 m; appendix showing every 40m?
• Based on continuum amplitude and slope, the hump only affects continuum range above -680 m.
• Hump present through depth at frequencies above 1.2e-4 Hz.

Annual PSD for appendix - may affect continuum range above -680 m:

from internal_waves_barkley_canyon.

Raw vs 15-min data, for Oct 10, 2013

u:

Mean Backscatter:

Beam intensity:

Beam correlations:

from internal_waves_barkley_canyon.

• Smooth one day (Oct 10) of raw data to 15-minutes using both np.convolve (450 pt box car filter / 450 so sums to 1) and ds.coarsen and see how they compare with each other and ONC 15-minute data.
• Using xarray coarsen reproduces the ONC 15-minute data exactly (they probably used this function)
• Using numpy convolution reproduces the ONC 15-minute data with gaps where NaNs were present in the raw data.

Raw (2-second, ONC):

Averaged (15-minute, ONC):

Coarsened (15-minute, xarray from raw):

Convolved (15-minute, numpy from raw):

• PSD of raw data to see if hump present.
• PSD of two months of raw data show that the hump and spikes are present. They are not a result of averaging.
• As with the 15-minute data, there is little effect on the specific continuum range below the threshold depths (see end of this comment).

Raw PSD for appendix:

• Hump only 'goes away' at lower depths because the continuum amplitude rises to conceal it, seen in the 'zoomed out' PSD.
• There is also a very notable spike at 3e-3 Hz, that diminishes with depth.

'Zoomed out' raw PSD:

• Evaluate 15-min data for corr. and intensity thresholds using ds.where, see if significant portion of deep data is affected.
• Echo intensity is best for evaluating noise, as ds.where function begins eliminating data with just a slight threshold increase. Raw data colour bars were set to 55 - 120 counts, and significant data is eliminated at a threshold of just 65.
• Correlation is more difficult to assess, as significant data is not eliminated until a very high threshold of 120 counts, where raw data colour bars were set to 75-130.
• From both variables it is obvious that most of the 'bad' data is above -700 m, and rarely dips below this level.

from internal_waves_barkley_canyon.

@jklymak

See comment directly above this one for updates on data quality. If there are too many plots or the comments are tough to follow, we can discuss on Wednesday.

from internal_waves_barkley_canyon.

@kurtisanstey That looks fine. What happens when you combine these two into a flag on the velocity? It looks like not a lot is gained from screening on full res versus 15 mintue averages, which is good.

from internal_waves_barkley_canyon.

• Check long-term raw data for threshold consistency.

2 months screened intensity (raw):

• Combine corr. and intens. threshold data to make a 'flag' for the velocity data, see how it looks.
• Need to check long-term threshold values for better screening.
• Preliminary test of one day of 15-minute data results in a velocity screen (values of 1 or NaN, only) that could be applied as such:

Screen:

Velocity:

Velocity after applying screen:

from internal_waves_barkley_canyon.

• Check ds.coarsen for number of good data points per window.
• Coarsen function does not return # of good data points per window (checked xarray info and source code). It uses the 'mean()' function with skipna=True to determine window value. If set to skipna=False convolves data exactly as np.convolve(), with NaN for windows containing NaN. skipna() and np.convolve() both do not have a counter for # of good data points per window.
• Check long-term 15-minute data for threshold consistency, decide how to cut data (mask or cut depth). Use percent good vs depth to determine good cut-off depth.
• Percent good vs depth for two years of 15-min data suggests that an echo intensity threshold of 65 and/or correlation threshold of 115 are ideal for screening data, and that threshold depth is fairly consistent through time.
• For these threshold values, there is steep step-like jump from 20% to 90% between -600 and -700 m.
• Propose cutting plots at -600 m, with analysis only below -700 m.
• Some seasonal variation of good data in fall (more bio scatterers higher in water column, so better reflection).
• Cut plots at -600 m, and analysis line at -700 m? Why not just cut at -700 m?
• Update writing with details.

from internal_waves_barkley_canyon.

Updated in writing. Archiving for reference.

from internal_waves_barkley_canyon.