The Geospatial Data Abstraction Library (GDAL; gdal.org) is a library and set of command line tools. GDAL supports reading the GOES-16 NetCDF files after a series of patches were recently upstreamed by the OCC team. The patches should be incorporated into GDAL version 2.2, and so for now GDAL must be built from source to work fully with the GOES-16 dataset.
Using GDAL with Docker
The easiest way to get started with GDAL is to use the docker image built by OCC which includes the HDF5 & NetCDF libraries. This image builds GDAL from source so it is always the latest version. The dockerfile can be found on the OCC GDAL container repo.
In the directory with the GOES-16 data you want to work with:
docker run -d --rm -ti --name gdal -v $(pwd):/data quay.io/occ_data/gdal:latest
This command will pull the docker container from Quay, run the container naming it “gdal” for future commands, and mount the current working directory as /data inside the container.
To test that it is working we can run the
gdalinfo command over a file. This command will tell us information about the file metadata.
docker exec -ti gdal gdalinfo OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc
Driver: netCDF/Network Common Data Format Files: OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc Size is 512, 512 Coordinate System is `' Metadata: NC_GLOBAL#cdm_data_type=Image NC_GLOBAL#Conventions=CF-1.7 NC_GLOBAL#dataset_name=OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc NC_GLOBAL#date_created=2017-07-11T15:56:18.3Z NC_GLOBAL#id=97e2d087-457b-4601-8a03-1bf0b0afa589 NC_GLOBAL#institution=DOC/NOAA/NESDIS > U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services NC_GLOBAL#instrument_ID=FM1 NC_GLOBAL#instrument_type=GOES R Series Advanced Baseline Imager NC_GLOBAL#iso_series_metadata_id=a70be540-c38b-11e0-962b-0800200c9a66 NC_GLOBAL#keywords=SPECTRAL/ENGINEERING > VISIBLE WAVELENGTHS > VISIBLE RADIANCE NC_GLOBAL#keywords_vocabulary=NASA Global Change Master Directory (GCMD) Earth Science Keywords, Version 22.214.171.124.0 NC_GLOBAL#license=Unclassified data. Access is restricted to approved users only. NC_GLOBAL#Metadata_Conventions=Unidata Dataset Discovery v1.0 NC_GLOBAL#naming_authority=gov.nesdis.noaa NC_GLOBAL#orbital_slot=GOES-Test NC_GLOBAL#platform_ID=G16 NC_GLOBAL#processing_level=National Aeronautics and Space Administration (NASA) L1b NC_GLOBAL#production_data_source=Realtime NC_GLOBAL#production_environment=OE NC_GLOBAL#production_site=WCDAS NC_GLOBAL#project=GOES NC_GLOBAL#scene_id=Full Disk NC_GLOBAL#spatial_resolution=0.5km at nadir NC_GLOBAL#standard_name_vocabulary=CF Standard Name Table (v25, 05 July 2013) NC_GLOBAL#summary=Single reflective band ABI L1b Radiance Products are digital maps of outgoing radiance values at the top of the atmosphere for visible and near-IR bands. NC_GLOBAL#timeline_id=ABI Mode 3 NC_GLOBAL#time_coverage_end=2017-07-11T15:56:14.9Z NC_GLOBAL#time_coverage_start=2017-07-11T15:45:38.2Z NC_GLOBAL#title=ABI L1b Radiances Subdatasets: SUBDATASET_1_NAME=NETCDF:"OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc":Rad SUBDATASET_1_DESC=[21696x21696] toa_outgoing_radiance_per_unit_wavelength (16-bit integer) SUBDATASET_2_NAME=NETCDF:"OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc":DQF SUBDATASET_2_DESC=[21696x21696] status_flag (8-bit integer) Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 512.0) Upper Right ( 512.0, 0.0) Lower Right ( 512.0, 512.0) Center ( 256.0, 256.0)
The NetCDF files contain subdatasets, in this example
NETCDF:"OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc":DQF. These strings can be used as input into GDAL to refer to the specific subsets.
Unscaling and Converting to PNG
To create a png image which we can view we should first extract the subdataset we are interested in and unscale the data values. The
gdal_translate program is what we will use for this step. It can extract the dataset and do the unscaling in a single step. The NetCDF files for GOES-16 store most data values as 16-bit integers, however working with 32-bit floating point numbers is significantly easier so we will do that conversion at this step as well.
docker exec -ti gdal gdal_translate -ot float32 -unscale -CO COMPRESS=deflate NETCDF:"OR_ABI-L1b-RadF-M3C02_G16_s20171921545382_e20171921556149_c20171921556183.nc":Rad fulldisk.tif
If this command runs successfully it should have created a
fulldisk.tif in your directory.
After you have this fulldisk file you can convert it to a png image again using
docker exec -ti gdal gdal_translate -ot Byte -of png -scale 0 650 0 255 fulldisk.tif fulldisk.png
This command will convert the tiff file to a png while scaling the data values from 0-650 in the tiff to 0-255 so that they fit in a byte for the png file.
Reprojecting GOES-16 Data
Now that you have a basic idea of how to work with the data you may be interested in using it in GIS applications such as ArcGIS or QGIS. To do this we need to reproject the data from the geostationary viewpoint into a rectilinear projection such as geographic (just lat-lon values). We will use the
gdalwarp utility to do this reprojection process while using the tiff file from the previous step.
docker exec -ti gdal gdalwarp -t_srs EPSG:4326 -dstnodata -999.0 fulldisk.tif fulldisk_geo.tif
This step will create a fulldisk_geo.tif file which uses the projection defined by
EPSG:4326 (geographic). This file can be readily loaded into GIS software and reprojected further or displayed with other datasets.
To compare the difference after reprojection we will also create a png file here using a step similar to that in the previous section.
docker exec -ti gdal gdal_translate -ot Byte -of png -scale 0 650 0 255 fulldisk_geo.tif fulldisk_geo.png
which will create a fulldisk_geo.png file in your current directory. Opening this file you can see the difference where the data has been turned from looking spherical into being elongated and stretched.