Posters

Abstract

An unprecedented expansion of offshore wind lease areas across the globe has led to a rise in offshore wind development, which requires properly assessing atmospheric and metocean conditions in each development area. Given the complex and chaotic nature of the atmosphere, large quantities of data are necessary to properly evaluate and quantify wind resources. Here we describe our use of an open-sourced energy-specific data format called MDIO, along with other open-sourced tools, to process terabyte-scale atmosphere and ocean datasets on the cloud for wind resource assessment applications. At TGS, we host 50TB of atmospheric data on our Wind AXIOM application including Weather Research & Forecasting (WRF) model simulations, the ERA5 reanalysis dataset and other numerical weather prediction (NWP) simulations. To make this possible, we developed MDIO, which is a cloud-native format with built-in compression that allows for fast access of spatiotemporal datasets while significantly reducing cloud storage and processing costs. MDIO is built on Zarr and optimized for energy applications to improve performance and scalability. Our processing workflow is built entirely on open-source tools such as Zarr, Dask, Xarray, Pangeo, FSSPEC, and Apache Airflow, with Google Cloud. All these tools have allowed us to calculate and host long-term statistics and other meaningful wind resource analytics at each grid cell of ERA5 on a global scale. Performant access to these long term timeseries is made possible with MDIO. The MDIO format provides many benefits for users of NWP model and reanalysis data. By making MDIO open-source, we hope to improve the quality of data processing and analysis across the wind energy industry and promote the growth and sustainability of wind energy markets. MDIO format is optimized for distributed computing, which enables deep learning and parallel processing that could accelerate our understanding of atmospheric processes relevant for the wind energy sector. We look forward to engaging with the wind resource assessment community to further refine MDIO and better serve the wind energy community.