Data Analysis Methods in Physical Oceanography

1 Data Acquisition and Recording

1Introduction xBasic sampling requirements xSampling interval xSampling duration xSampling accuracy xBurst sampling versus continuous sampling xRegularly versus irregularly sampled dataRole of autonomous platforms in creating irregularly sampled data. X

1.2.7 Independent realizations x

1.3 Temperature xMercury thermometers xThe mechanical bathythermograph (MBT) xResistance thermometers (expendable bathythermograph: XBT)Salinity/conductivity-temperature-depth profilersThermodynamic Equation of Sea Water (TEOS-10)Dynamic response of temperature sensorsResponse times of CTD systems xxTemperature calibration of STD/CTD profilers xSea surface temperatureInfluence of sea ice on SSTUAV infrared and microwave SST measurements xx

1.3.12 The modern digital thermometer xxPotential temperature and density

1.3.14 Calibration of thermometers at NIST

1.3.15 Calculating the buoyancy frequency from CTD data

1.4 Salinity xxSalinity and electrical conductivity xxThe practical salinity scale xxNonconductive methods xxRemote sensing of salinity, accuracy and space/time coverage xx

1.5 Depth or pressure xxHydrostatic pressure xxFree-fall velocity..problem of XBT depth accuracy xxEcho soundingOptical satellite imagery in shallow areas, accuracy 481.5.5 Other depth sounding methods　(expand on LiDAR) 54

1.6 Sea-level measurement 55

1.6.0 Add discussion of Chart Datum, Geoid, Digital Elevation Models,Tide and pressure gauges　 58Satellite altimetry, accuracies and impacts, long term intercalibration 62verted echo sounder (IES) 63

1.7 Eulerian currents 68Early current meter technology, possibly remove or shorten 70Rotor-type current meters, intercomparison, VACM, VMCM,　update 70Nonmechanical current meters 78Acoustic Doppler current meters (ADCMs) 83Comparisons of current meters xxElectromagnetic methods 95Other methods of current measurement 96Mooring logistics xxAcoustic releasesData telemetry 99

1.8 Lagrangian current measurements 102Drift cards and bottlesEvolution of drifters 103

1.8.3. Modern drifters 104Processing satellite-tracked drifter data xxDrifter response

1.8.5 The Global Drifter Program 109

1.8.6 Other types of surface drifters

1.8.7. Computing diffusivity from drifters 115

1.8.8 Subsurface floatsEvoluton of sofar to rafos 116

1.8.9 Surface displacements in satellite imagery 119

1.8.10 Autonomous Underwater Vehicles (AUVs)

1.8.11 ARGO Floats updateWind in situ measurements, satellite measurements 119Precipitation. Satellite measurements 125Chemical tracers 127Conventional tracers 128Light attenuation and scattering 138Oxygen isotope:　d18O 142Helium-3; helium/heat ratioMapping pCO2 and heat content 143

1.12 Transient chemical tracers 145Tritium 146Radiocarbon 149Chlorofluorocarbons 153Radon-222 155Sulfur hexafluoride 157Strontium-90

1.13 Autonomous Platforms

1.13.1 Wave gliders

1.13.2 Saildrones

1.13.3 Unmanned Aerial Vehicles

1.13.4 Marine Mammals as Instrument Platforms

1.14 Observing Ocean Turbulence

1.14.1 Early direct measurements (thermistors, thermocouples)

1.14.2 Hot film (velocity) and cold film (temperature) measurements

1.14.3 Free falling airfoil sensors

1.14.4 Multi probes (airfoil plus thermistors plus　onductivity) from APL-UW

1.14.5 OSU Chameleon (shear probes, thermistors, conductivity)

1.14.6 Dye Injections

1.14.7 High-freequency Acoustics

1.14.8 Autonomous Platforms

1.14.9 Moored Instruments

1.14.10 SurfaceWave Instrument Float with Tracking

1.15 Data Formats

1.15.1 NetCDF

1.15.2 HDF

1.15.3 Comma separated values

1.15.4 PDF

1.15.5 XLS, CSV

1.15.6 ZIP

1.15.7 CDF


2 Data Processing and Presentation xxxIntroduction xxxCalibration. xxxInterpolation. Add bicubic, nearest neighbor, other methodsThe 2010 international thermodynamic equation of seawater (TEOS-10) 161Data presentation 162Introduction 162Vertical profiles 167Vertical sections 170Horizontal maps 172Map projections 177Characteristic or property versus property diagrams 181Time-series presentation 185Histograms 187New directions in graphical presentation 187


3 Statistical Methods and Error Handling xxxIntroduction xxxSample distributions 194Probability 197　

3.3.1 Cumulative probability functions 200Moments and expected values 201Unbiased estimators and moments xxCommon probability density functions 207Central limit theorem 211Estimation 214Confidence intervals 216Confidence interval for m (s known) 217Confidence interval for m (s unknown) 218Confidence interval for s2 219Goodness-of-fit test 220Selecting the sample size 224Confidence intervals for altimeter bias estimates 225Estimation methods 227Minimum variance unbiased estimation 228Maximum likelihood 230

3.12 Linear estimation (regression) 233Method of least squares 234Standard error of the estimate 238Multivariate regression 239A computational example of matrix regression 240Polynomial curve fitting with least squares 242Relationship between least-squares and maximum likelihood 242

3.13 Relationship between regression and correlation 243The effects of random errors on correlation 244The maximum likelihood correlation estimator 245Correlation and regression: cause and effect 246

3.14 Hypothesis testing. expand 249Significance levels and confidence intervals for correlation 253Analysis of variance and the F-distribution 254Effective degrees of freedom 257　

3.15.1 Simple effective degrees of freedom

3.15.2 Trend estimates and the integral time scale 261Editing and despiking techniques: the nature of errors 266Identifying and removing errors 266Propagation of error 273Dealing with numbers: the statistics of roundoff 274Gauss-Markov theorem 2773.17 Interpolation: filling the data gaps 277Equally and unequally spaced data 277Interpolation methods 279Interpolating gappy records: practical examples 286

3.18 Covariance and the covariance matrix 290Covariance and structure functions 291A computational example 291Multivariate distributions 293　

3.19 Bootstrap and jackknife methods 294Bootstrap method 295Jackknife method 301

3.20 Extreme Value Statistics

3.20.1 Basic theory

3.20.2 Statistical models and applicationsThe generalized extreme value (GEV) approach and the generalized Paretodistribution (GPD) approach


4 The Spatial Analyses of Data Fields xxTraditional block and bulk averaging xxObjective analysis xx　

4.2.1 Objective mapping: examples. Correct equations xxKriging compare two methods

4.3.1 Mathematical formulationEmpirical orthogonal functions 319Principal axes of a single vector time series (scatter plot) 325EOF computation using the scatter matrix method 328EOF computation using singular value decomposition 332An example: deep currents near a mid-ocean ridge 334Interpretation of EOFs 336Variations on conventional EOF analysis 340

4.5 Extended empirical orthogonal functions (EEOFs)

4.5.1 Applications of EEOFs

4.6 Cyclostationary Empirical Orthogonal Functions (CSEOFs)

4.7 Factor Analysis

4.8 Normal mode analysis 344

4.8.1 Vertical normal modes 344

4.8.2 An example: normal modes of semidiurnal frequency 347

4.8.3 Coastal-trapped waves (CTWs) 350

4.9 Self organizing maps

4.9.1. Basic formulation

4.9.2 SOM versus Principal Component Analysis

4.9.3 The self-organizing map (SOM)

4.9.4 Application to estuarine circulation in Juan de Fuca Strait

4.9.5. Growing hierarchical self-organizing maps

4.10 Kalman filters4.11 Mixed layer depth estimation

4.11.1 Threshold methods

4.11.2 Step-function least squares regression method

4.11.3 Integral depth-scale method

4.11.4. The Split-and-Merge Algorithm

4.11.5 Recent methods

4.11.6 Comparison of Methods

4.12 Inverse methods 356

4.12.1 General inverse theory 356

4.12.2 Inverse theory and absolute currents 361

4.12.3 The IWEX internal wave problem 366

4.12.4 Summary of inverse methodsNew. Review spatial mapping software tools 3705 Time-series Analysis Methods xBasic concepts 371Stochastic processes and stationarity 373Correlation functions 374


5.4 Spectral analysis 404　

5.4.1 Spectra of deterministic and stochastic processes

5.4.2 Spectra of discrete series 413

5.4.3 Conventional spectral methods 417

5.4.4 Spectra of vector series 425

5.4.5 Effect of sampling on spectral estimates 432

5.4.6 Smoothing spectral estimates (windowing) 441

5.4.7 Smoothing spectra in the frequency domain 450

5.4.8 Confidence intervals on spectra 454

5.4.9 Zero-padding and prewhitening 455

5.4.10 Spectral analysis of unevenly spaced time series 460

5.4.11 General spectral bandwidth and　Q　of the system 461

5.4.13 Summary of the standard spectral analysis approach 461

5.5 Spectral analysis (parametric methods) 464

5.5.1 Some basic concepts 467

5.5.2 Autoregressive power spectral estimation 468

5.5.3 Maximum likelihood spectral estimation 478

5.6 Cross-spectral analysis 480

5.6.1 Cross-correlation functions 480

5.6.2 Cross-covariance method 482

5.6.3 Fourier transform method 482

5.6.4 Phase and cross-amplitude functions 484

5.6.5 Coincident and quadrature spectra 485

5.6.6 Coherence spectrum (coherency) 486

5.6.7 Frequency response of a linear system 490

5.6.8 Rotary cross-spectral analysis 495

5.6.9 Admittance (transfer) functions (and use in extraction of masked data)

5.7 Wavelet analysis. expand 501

5.7.1 The wavelet transform 502

5.7.2 Wavelet algorithms 504

5.7.3 Oceanographic examples 505

5.7.4 The S-transformation 508

5.7.5 The multiple filter technique 511

5.7.6 Wavelet coherence (coherence amplitude and phase)

5.8 Fourier analysis xx

5.8.1 Mathematical formulation 381

5.8.2 Discrete time series

5.8.3 A computational example 387

5.8.4 Fourier analysis for specified frequencies 388

5.8.5 The fast Fourier transform 3905.9 Harmonic analysis 392

5.9.1 A least-squares method 392

5.9.2 A computational example 395

5.9.3 Harmonic analysis of tides 397

5.9.4 Choice of constituents 398

5.9.5 A computational example for tides 399

5.9.6 Complex demodulation

5.10 Regime shift detection

5.10.1 STARS

5.10.2 KZA

5.10.3 CUI

5.11 Vector regression

5.11.1 The 2-parameter complex functional approach

5.11.2 The vector regressional model

5.11.3 Wind versus surface drift: The six characteristic cases

5.12 Fractals expand 557

5.12.1 The scaling exponent method 561

5.12.2 The yardstick method 562

5.12.3 Box counting method 563

5.12.4 Correlation dimension 564

5.12.5 Dimensions of multifractal functions 564

5.12.6 Predictability

5.13 Analyzing Turbulence Measurements

5.13.1 Spectral analysis of temperature measurements

5.13.2. High resolution vertical profiles

5.13.3. Turbulent Kinetic energy spectra

5.13.4 Acoustical Flow Imaging

5.13.5 Time Series from Moored Arrays


6 Digital Filters

6.1 Introduction

6.2 Basic concepts

6.3 Ideal filters

6.3.1 Bandwidth

6.3.2 Gibb’s phenomenon

6.3.3 Recoloring

6.4 Design of oceanographic filters

6.4.1 Frequency versus time domain filtering

6.4.2 Filter cascades

6.5 Running mean filters

6.6 Godin-type filters

6.7 Lanczos-window cosine filters

6.7.1 Cosine filters

6.7.2 The Lanczos window

6.7.3 Practical filter design

6.7.4 The Hanning (von Hann) window

6.8 Butterworth filters

6.8.1 High-pass and band-pass filters

6.8.2 Digital formulation

6.8.3 Tangent versus sine filters

6.8.4 Filter design

6.8.5 Filter coefficients

6.9 Kaiser-Bessel filters

6.9.1 A low-pass Kaiser-Bessel filter

6.10 Frequency-domain (transform) filtering


7.0 Machine Learning Methods

7.1 Maximum likelihood spatial analysis

7.2 Decision Tree

7.3 Naïve Bayes

7.4 k-means, kNN

7.5 Principal Component Analysis

7.6 Random Forest

7.7 Support Vector Machines


8.0 Neural networks, convolutional neural networks and deep learning Definition, Overview History Artificial neural networks Deep neural nets

Appendices xx
Appendix A Units in physical oceanography (expand) xx
Appendix B Glossary of statistical terminology 572
Appendix C Means, variances and moment-generating functions for some common continuous variables 576
Appendix D Statistical tables 577
Appendix E Correlation coefficients at the 5% and 1% levels of significance for various degrees of freedom v 585
Appendix F Approximations and nondimensional numbers in physical oceanography (Expand) 586
Appendix G Convolution 593
Appendix H Computer software for data analyses