The Surface Water and Ocean Topography (SWOT) satellite is expected to observe the sea surface height (SSH) down to scales of ∼10-15 kilometers. While SWOT will reveal submesoscale SSH patterns that have never before been observed on global scales, how to extract the corresponding velocity fields and underlying dynamics from this data presents a new challenge. At these soon-to-be-observed scales, geostrophic balance is not sufficiently accurate, and the SSH will contain strong signals from inertial gravity waves — two problems that make estimating surface velocities non-trivial. Here we show that a data-driven approach can be used to estimate the surface flow, particularly the kinematic signatures of smaller scales flows, from SSH observations, and that it performs significantly better than directly using the geostrophic relationship. We use a Convolution Neural Network (CNN) trained on submesoscale-permitting high-resolution simulations to test the possibility of reconstructing surface vorticity, strain, and divergence from snapshots of SSH. By evaluating success using pointwise accuracy and vorticity-strain joint distributions, we show that the CNN works well when inertial gravity wave amplitudes are weak. When the wave amplitudes are strong, the model may produce distorted results; however, an appropriate choice of loss function can help filter waves from the divergence field, making divergence a surprisingly reliable field to reconstruct in this case. We also show that when applying the CNN model to realistic simulations, pretraining a CNN model with simpler simulation data improves the performance and convergence, indicating a possible path forward for estimating real flow statistics with limited observations.

The spatiotemporal evolution of marine heatwaves (MHWs) is explored using a tracking algorithm termed Ocetrac that provides objective characterization of MHW spatiotemporal evolution. Candidate MHW grid points are defined in detrended gridded sea temperature data using a seasonally varying temperature threshold. Identified MHW points are collected into spatially distinct objects using edge detection with weak sensitivity to edge detection and size threshold criteria. These MHW objects are followed in space and time while allowing objects to split and merge. Ocetrac is applied to monthly satellite sea surface temperature data from September 1981 through January 2021. The resulting MHWs are characterized by their intensity, duration, and total area covered. The global analysis shows that MHWs in the Gulf of Maine and Mediterranean Sea evolve within a relatively small region, while major MHWs in the Pacific and Indian Oceans are linked in space and time. The largest and most long lasting MHW using this method lasts for 60 months from November 2013 to October 2018, encompassing previously identified MHW events including those in the Northeast Pacific (2014-2015), the Tasman Sea (2015-2016, 2017-2018), and the Great Barrier Reef (2016).

A document by C Spencer Jones. Click on the document to view its contents.

We describe a new way to apply a spatial filter to gridded data from models or observations, focusing on low-pass filters. The new method is analogous to smoothing via diffusion, and its implementation requires only a discrete Laplacian operator appropriate to the data. The new method can approximate arbitrary filter shapes, including Gaussian filters, and can be extended to spatially-varying and anisotropic filters. The new diffusion-based smoother's properties are illustrated with examples from ocean model data and ocean observational products. An open-source python package implementing this algorithm, called gcm-filters, is currently under development.

The core tools of science (data, software, and computers) are undergoing a rapid and historic evolution, changing what questions scientists ask and how they find answers. Earth science data are being transformed into new formats optimized for cloud storage that enable rapid analysis of multi-petabyte datasets. Datasets are moving from archive centers to vast cloud data storage, adjacent to massive server farms. Open source cloud-based data science platforms, accessed through a web-browser window, are enabling advanced, collaborative, interdisciplinary science to be performed wherever scientists can connect to the internet. Specialized software and hardware for machine learning and artificial intelligence (AI/ML) are being integrated into data science platforms, making them more accessible to average scientists. Increasing amounts of data and computational power in the cloud are unlocking new approaches for data-driven discovery. For the first time, it is truly feasible for scientists to bring their analysis to data in the cloud without specialized cloud computing knowledge. This shift in paradigm has the potential to lower the threshold for entry, expand the science community, and increase opportunities for collaboration while promoting scientific innovation, transparency, and reproducibility. Yet, we have all witnessed promising new tools which seem harmless and beneficial at the outset become damaging or limiting. What do we need to consider as this new way of doing science is evolving?

Computational Oceanography is the study of ocean phenomena by numerical simulation, especially dynamical and physical phenomena. Progress in information technology has driven exponential growth in the number of global ocean observations and the fidelity of numerical simulations of the ocean in the past few decades. The growth has been exponentially faster for ocean simulations, however. We argue that this faster growth is shifting the importance of field measurements and numerical simulations for oceanographic research. It is leading to the maturation of Computational Oceanography as a branch of marine science on par with observational oceanography. One implication is that ultra-resolved ocean simulations are only loosely constrained by observations. Another implication is that barriers to analyzing the output of such simulations should be removed. Although some specific limits and challenges exist, many opportunities are identified for the future of Computational Oceanography. Most important is the prospect of hybrid computational and observational approaches to advance understanding of the ocean.

In this presentation, we will describe the [Pangeo Project](http://pangeo.io), a coordinated community effort with support from NASA, NSF, AWS, Microsoft Azure and Google Cloud, to develop interactive and reproducible open source workflows for discovery, visualization, and quantitative analysis of large datasets used for research in the Earth Sciences. The Pangeo computational platform is based on JupyterHub and deployed wherever the data is stored. Python libraries such as Xarray, Rasterio, and Dask enable distributed parallel computations on HPC and Kubernetes clusters. We will discuss the design concepts central to the Pangeo platform and highlight specific applications using NASA satellite data archives on AWS. We will discuss recent progress in the integration of data discovery tools (e.g. STAC, CMR, Intake) with cloud-native storage formats for multidimensional data types (Cloud-Optimized Geotiff, Zarr, etc.) and highlight how they can be used to construct elegant, robust and reproducible scientific workflows. Finally, we will discuss performance, security, transferability across public cloud platforms, cost to operate, and approaches to encourage a cultural shift in scientific computation through educational events.

Oceanic mesoscale motions including eddies, meanders, fronts, and filaments comprise a dominant fraction of oceanic kinetic energy and contribute to the redistribution of tracers in the ocean such as heat, salt, and nutrients. This reservoir of mesoscale energy is regulated by the conversion of potential energy and transfers of kinetic energy across spatial scales. Whether and under what circumstances mesoscale turbulence precipitates forward or inverse cascades, and the rates of these cascades, remain difficult to directly observe and quantify despite their impacts on physical and biological processes. Here we use global observations to investigate the seasonality of surface kinetic energy and upper ocean potential energy. We apply spatial filters to along-track satellite measurements of sea surface height to diagnose surface eddy kinetic energy across 60-300 km scales. A geographic and scale dependent seasonal cycle appears throughout much of the mid-latitudes, with eddy kinetic energy at scales less than 60 km peaking 1-4 months before that at 60-300 km scales. Spatial patterns in this lag align with geographic regions where the conversion of potential to kinetic energy are seasonally varying. In mid-latitudes, the conversion rate peaks 0-2 months prior to kinetic energy at scales less than 60 km. The consistent geographic patterns between the seasonality of potential energy conversion and kinetic energy across spatial scale provide observational evidence for the inverse cascade, and demonstrate that some component of it is seasonally modulated. Implications for mesoscale parameterizations and numerical modeling are discussed.

As the size of geoscience datasets grows, scientists are eager to move away from a download-based workflow, where data files are downloaded a local computer for analysis, towards a more cloud-native workflow, where data is loaded on demand over the network. On-demand data loading offers several advantages, including increased reproducibility, provenance tracking, and, potentially, scalability using distributed cloud computing. In this notebook, we demonstrate how to load data on-demand using three different remote data access protocols: - OPeNDAP, the most common, well-established protocol - NetCDF over HTTP, enabled by the h5py library - Zarr over HTTP, a new format optimized for cloud object storage (e.g. Amazon S3) We then conduct a simple benchmarking exercise to explore the throughput and scalability of each service. We use Dask to parallelize reads from each access protocol and calculate the throughput as a function of number of parallel reads. One conclusion is that Zarr over HTTP, coupled with cloud object storage, shows favorable scaling up to hundreds of parallel processes. Finally, we compare the throughput of Zarr over HTTP on a few different clouds, including Google Cloud Storage, Jetstream Cloud, Wasabi Cloud, and Open Storage Network.

The Pangeo project recently introduced large parts of the CMIP6 data archive into the cloud. This enables, for the first time, centralized, reproducible science of state-of-the-art climate simulations without the need to own large storage or a supercomputer as a user. The data itself however, still presents significant challenges for analysis, one of which is applying operations across many models. Two of the major hurdles are different naming/metadata between modeling centers and complex grid layouts, particularly for the ocean components of climate models. Common workflows in the past often included interpolating/remapping desired variables and creating new files, creating organizational burden, and increasing storage requirements. We will demonstrate two Pangeo tools which enable seamless calculation of common operations like vector calculus operators (grad, curl, div) and weighted averages/integrals across a wide range of CMIP6 models directly on the data stored in the cloud. cmip6_preprocessing provides numerous tools to unify naming conventions and parse grid information and metrics (like cell area). This information is used by xgcm to enable finite volume analysis on the native model grids. The combination of both tools facilitates fast analysis while ensuring a reproducible and accurate workflow.

As more analysis-ready datasets are provided on the cloud, we need to consider how researchers access data. To maximize performance and minimize costs, we move the analysis to the data. This notebook demonstrates a Pangeo deployment connected to multiple Dask Gateways to enable analysis, regardless of where the data is stored. Public clouds are partitioned into regions, a geographic location with a cluster of data centers. A dataset like the National Water Model Short-Range Forecast is provided in a single region of some cloud provider (e.g. AWS’s us-east-1). To analyze that dataset efficiently, we do the analysis in the same region as the dataset. That’s especially true for very large datasets. Making local “dark replicas” of the datasets is slow and expensive. In this notebook we demonstrate a few open source tools to compute “close” to cloud data. We use Intake as a data catalog, to discover the datasets we have available and load them as an xarray Dataset. With xarray, we’re able to write the necessary transformations, filtering, and reductions that compose our analysis. To process the large amounts of data in parallel, we use Dask. Behind the scenes, we’ve configured this Pangeo deployment with multiple Dask Gateways, which provide a secure, multi-tenant server for managing Dask clusters. Each Gateway is provisioned with the necessary permissions to access the data. By placing compute (the Dask workers) in the same region as the dataset, we achieve the highest performance: these worker machines are physically close to the machines storing the data and have the highest bandwidth. We minimize cost by avoiding egress costs: fees charged to the data provider when data leaves a cloud region.

Computer simulations of the Earth’s climate and weather generate huge amounts of data. These data are often persisted on HPC systems or in the cloud across multiple data assets of a variety of formats (netCDF, zarr, etc…). Finding, investigating, loading these data assets into compute-ready data containers costs time and effort. The data user needs to know what data sets are available, the attributes describing each data set, before loading a specific data set and analyzing it. In this notebook, we demonstrate the integration of data discovery tools such as intake and intake-esm (an intake plugin) with data stored in cloud optimized formats (zarr). We highlight (1) how these tools provide transparent access to local and remote catalogs and data, (2) the API for exploring arbitrary metadata associated with data, loading data sets into data array containers. We also showcase the Pangeo catalog, an open source project to enumerate and organize cloud optimized climate data stored across a variety of providers, and a place where several intake-esm collections are now publicly available. We use one of these public collections as an example to show how an end user would explore and interact with the data, and conclude with a short overview of the catalog’s online presence.