[ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels

De Vries Andries andries.devries at env.ethz.ch
Thu Mar 14 05:26:02 MDT 2019


Hi Dennis,

Thanks a lot, I think I found the right solution following your suggestion; to place the hybrid coefficients in a constant file using the nco command. In combination with using ncl version 6.5.0, there is no need anymore to convert the ERA20C data first to netcdf format. 

Briefly, my responses to your points:

1) Yes. I checked; the hybrid coefficients are identical for both the original N80 and regular 1x1 grid of the ERA-20C data. Also your description of the conversion hyam=hyam/p0 is clear; I had that implemented in my script already.

2) what I meant is that with NCL I can read in the hybrid coefficients from grib files of ERA-Interim data, I suppose cause this is the grib1 format. This works also with NCL version 6.3.0, for example. However, NCL does not find the hybrid coefficients in the ERA20C data, perhaps because (1) these are grib2 files and NCL (version 6.5.0.) misses those hybrid coefficients (a bug?), or (2) the hybrid coefficients are simply not present in ERA20C grib 2 files, and cdo adds them when these files are converted to netcdf format.

3) thank you. I understand now why the variable names in ERA20C files are messed up when I convert the grib2 format to grib1 format.

Thanks again for your help!! I really appreciate!

Cheers,
Andries
________________________________________
From: Dennis Shea [shea at ucar.edu]
Sent: 13 March 2019 21:04
To: De Vries  Andries
Cc: ncl-talk at ucar.edu
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels

[1] I speculate that the hybrid coefficients are identical for the original model grid and 1x1 interpolated grid. Is this not true?

[2]  You state that the CDO correctly extracts hybrid coefficients on the 1x1 interpolated grids.
The netCDF Operator (NCO) 'ncks' can be used to extract the CDO created hybrid coefficients

%> ncks -v hyai,hybi,hyam,hybm  e20c_temp_199410.nc<http://e20c_temp_199410.nc>   e20c_HybridCoef.nc

This file is attached. You can use this in companion with the grib2 file.

   fgrb = addfile("...grib", "r")
   fhy  = addfile("e20c_HybridCoef.nc", "r")
   hyam = fhy->hyam
   hybm = fhy->hybm

---
Note that hybrid {a,b} coefficients take two different forms

NCAR's CESM atmospheric model calculates pressure at the hybrid levels use the formula p = a(k)*p0 + b(k)*psfc

ECWMF uses p = ap(k)+ b(k)*psfc :  where ap(k) =a(k)*p0

All, NCL's built-in functions that use hybrid coefficients [<http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml> eg:  <http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml> pres_hybrid_ccm<http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml>] expect the CESM formulation.

So,

p0 = 100000.0

hyam = hyam/p0      ; convert ap(k) =a(k)*p0 to NCL expected values

===

[3] "moving target":

The WMO prescribes parameter IDs 0-127 to be fixed.

Parameter IDs 128 onward can be defined by each operational center.

NCL uses internal built-in tables to look-up the correct long_name and units associated with each parameter ID.

These tables must be updated with each NCL release.

.Good Luck

D


On Wed, Mar 13, 2019 at 9:48 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>> wrote:
Hi Dennis,

Thanks again for your comments and explanation. Do I understand correctly that NCL doesnt recognize the hybrid coefficients in the ERA-20C data?

I'm little surprise since NCL (also version 6.3.0) seems to recognize hybrid coefficients in the ERA-Interim data grib files of ECMWF, see again the print statements below. Any idea what is different between the ERA-20C and ERA-Interim data that leads to this problem with reading in hybrid coefficients? Could it be a difference of grib2 (is ERA-20C wgrib2?) and grib1 (ERA-Interim) format?

I hope to find a way to read in grib files of ERA-20C data with NCL, including the hybrid level coefficients, without needing additional tricks or conversion of data.

What do you mean by "The issue with GRIB is that it is a moving target."?

Kind regards,
Andries


-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
print statement of NCL 6.3.0 of ERA-Interim data, grib file, regular grid 1x1, temperature
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

a Copyright (C) 1995-2015 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.3.0
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
ncl 0> a=addfile("ei_an_temp_r1x1_ml_199410.grib","r")
print(a)
ncl 1> print(a)

Variable: a
Type: file
filename:       (null)
path:   ei_an_temp_r1x1_ml_199410.grib
   file global attributes:
   dimensions:
      initial_time0_hours = 124
      lv_HYBL1 = 60
      lv_HYBL_i2 = 61
      ncl_scalar = 1
      g0_lat_4 = 181
      g0_lon_5 = 360
   variables:
      float T_GDS0_HYBL ( initial_time0_hours, lv_HYBL1, g0_lat_4, g0_lon_5 )
         center :       European Center for Medium-Range Weather Forecasts (RSMC)
         long_name :    Temperature
         units :        K
         _FillValue :   1e+20
         level_indicator :      109
         gds_grid_type :        0
         parameter_table_version :      128
         parameter_number :     130
         forecast_time :        0
         forecast_time_units :  hours

      double initial_time0_hours ( initial_time0_hours )
         long_name :    initial time
         units :        hours since 1800-01-01 00:00

      double initial_time0_encoded ( initial_time0_hours )
         long_name :    initial time encoded as double
         units :        yyyymmddhh.hh_frac

      float g0_lat_4 ( g0_lat_4 )
         long_name :    latitude
         GridType :     Cylindrical Equidistant Projection Grid
         units :        degrees_north
         Dj :    1
         Di :    1
         Lo2 :  359
         La2 :  -90
         Lo1 :   0
         La1 :  90

      float g0_lon_5 ( g0_lon_5 )
         long_name :    longitude
         GridType :     Cylindrical Equidistant Projection Grid
         units :        degrees_east
         Dj :    1
         Di :    1
         Lo2 :  359
         La2 :  -90
         Lo1 :   0
         La1 :  90

      float P0 ( ncl_scalar )
         units :        Pa
         long_name :    reference pressure

      float lv_HYBL_i2_b ( lv_HYBL_i2 )
         note : layer interfaces associated with hybrid levels lv_HYBL1
         long_name :    hybrid B coefficient at layer interfaces

      float lv_HYBL_i2_a ( lv_HYBL_i2 )
         note : layer interfaces associated with hybrid levels lv_HYBL1
         long_name :    hybrid A coefficient at layer interfaces

      float lv_HYBL1_b ( lv_HYBL1 )
         note : derived from lv_HYBL_i2_b as average of layer interfaces above and below midpoints
         long_name :    hybrid B coefficient at layer midpoints

      float lv_HYBL1_a ( lv_HYBL1 )
         note : derived from lv_HYBL_i2_a as average of layer interfaces above and below midpoints
         long_name :    hybrid A coefficient at layer midpoints

      integer lv_HYBL1 ( lv_HYBL1 )
         formula_terms :        a: lv_HYBL1_a b: lv_HYBL1_b ps: unknown p0: P0
         standard_name :        atmosphere hybrid sigma pressure coordinate
         long_name :    hybrid level
         units :        number

      string initial_time0 ( initial_time0_hours )
         long_name :    Initial time of first record
         units :        mm/dd/yyyy (hh:mm)

________________________________________
From: Dennis Shea [shea at ucar.edu<mailto:shea at ucar.edu>]
Sent: 12 March 2019 20:53
To: De Vries  Andries
Cc: ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels

Yes. NCL does not recognize the hybrid coefficients. IMHO: It is a bug.

wgrib2 does 'see' the 92 interface coefficients.

%> wgrib2 -hybrid e20c_an_temp_r360x180_ml_199410.grib | less

The following are the (hyai, hybi) values:

1:0:Hybrid levels=92 1=( 0.000000, 2.000040) 2=( 3.980832, 7.387186) 3=(12.908319,21.413612) 4=(33.952858,51.746601) 5=(76.167656,108.715561) 6=(150.986023,204.637451) 7=(271.356506,352.824493) 8=(450.685791,566.519226) 9=(701.813354,857.945801) 10=(1036.166504,1237.585449) 11=(1463.163940,1713.709595) 12=(1989.874390,2292.155518) 13=(2620.898438,2976.302246) 14=(3358.425781,3767.196045) 15=(4202.416504,4663.776367) 16=(5150.859863,5663.156250) 17=(6199.839355,6759.727051) 18=(7341.469727,7942.926270) 19=(8564.624023,9208.305664) 20=(9873.560547,10558.881836) 21=(11262.484375,11982.662109) 22=(12713.897461,13453.225586) 23=(14192.009766,14922.685547) 24=(15638.053711,16329.560547) 25=(16990.623047,17613.281250) 26=(18191.029297,18716.968750) 27=(19184.544922,19587.513672) 28=(19919.796875,20175.394531) 29=(20348.916016,20434.158203) 30=(20426.218750,20319.011719) 31=(20107.031250,19785.357422) 32=(19348.775391,18798.822266) 33=(18141.296875,17385.595703) 34=(16544.585938,15633.566406) 35=(14665.645508,13653.219727) 36=(12608.383789,11543.166992) 37=(10471.310547,9405.222656) 38=(8356.252930,7335.164551) 39=(6353.920898,5422.802734) 40=(4550.215820,3743.464355) 41=(3010.146973,2356.202637) 42=(1784.854614,1297.656128) 43=(895.193542,576.314148) 44=(336.772369,162.043427) 45=(54.208336, 6.575628) 46=( 0.003160, 0.000000) 47=( 0.000000, 0.000000) 48=( 0.000000, 0.000000) 49=( 0.000000, 0.000000) 50=( 0.000000, 0.000000) 51=( 0.000000, 0.000000) 52=( 0.000000, 0.000000) 53=( 0.000000, 0.000000) 54=( 0.000000, 0.000000) 55=( 0.000000, 0.000000) 56=( 0.000000, 0.000000) 57=( 0.000000, 0.000000) 58=( 0.000000, 0.000000) 59=( 0.000000, 0.000000) 60=( 0.000000, 0.000000) 61=( 0.000000, 0.000000) 62=( 0.000000, 0.000000) 63=( 0.000000, 0.000000) 64=( 0.000000, 0.000014) 65=( 0.000055, 0.000131) 66=( 0.000279, 0.000548) 67=( 0.001000, 0.001701) 68=( 0.002765, 0.004267) 69=( 0.006322, 0.009035) 70=( 0.012508, 0.016860) 71=( 0.022189, 0.028610) 72=( 0.036227, 0.045146) 73=( 0.055474, 0.067316) 74=( 0.080777, 0.095964) 75=( 0.112979, 0.131935) 76=( 0.152934, 0.176091) 77=( 0.201520, 0.229315) 78=( 0.259554, 0.291993) 79=( 0.326329, 0.362203) 80=( 0.399205, 0.436906) 81=( 0.475016, 0.513280) 82=( 0.551458, 0.589317) 83=( 0.626559, 0.662934) 84=( 0.698224, 0.732224) 85=( 0.764679, 0.795385) 86=( 0.824185, 0.850950) 87=( 0.875518, 0.897767) 88=( 0.917651, 0.935157) 89=( 0.950274, 0.963007) 90=( 0.973466, 0.982238) 91=( 0.989153, 0.994204) 92=( 0.997630, 1.000000)

The mid-layer coefficients (91) are calculated as the mid-points and are calculated.
====
Still, something seems odd. My local CDO version

%> cdo -V
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)

%>cdo -f nc copy e20c_an_temp_r360x180_ml_199410.grib   e20c_temp_199410.nc<http://e20c_temp_199410.nc><http://e20c_temp_199410.nc>

cdo copy: Open failed on >e20c_an_temp_r360x180_ml_199410.grib<
Unsupported file type
====
CDO Versions 1.9.3 and 1.9.4 do recognize the hybrid coefficients.

====
The issue with GRIB is that it is a moving target.

D


On Tue, Mar 12, 2019 at 7:29 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>>> wrote:
Hi Dennis,

Thank you very much. The newer version of NCL helps indeed, hadn't thought of that simple solution. However, still I miss a few coordinate variables that I can read in the netcdf file, but not in the original grib file; the hyam and hybm variabels. Please, see the print statement of the netcdf file below. Any idea why and what can help?

Cheers,
Andries

Variable: b2
Type: file
filename:       e20c_an_temp_r360x180_ml_199410
path:   ../ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
   file global attributes:
      CDI : Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/cdi)
      Conventions : CF-1.6
      history : Thu Feb 14 11:02:46 2019: cdo -f nc copy /atmosdyn/devriesa/DATA/ERA20C/e20c_an_temp_r360x180_ml_199410.grib /atmosdyn/devriesa/DATA/ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
      institution : European Centre for Medium-Range Weather Forecasts
      CDO : Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)
   dimensions:
      time = 124  // unlimited
      lon = 360
      lat = 181
      lev = 91
      nhyi = 92
      nhym = 91
   variables:
      double time ( time )
         standard_name :        time
         units :        hours since 1994-10-1 00:00:00
         calendar :     proleptic_gregorian
         axis : T

      double lon ( lon )
         standard_name :        longitude
         long_name :    longitude
         units :        degrees_east
         axis : X

      double lat ( lat )
         standard_name :        latitude
         long_name :    latitude
         units :        degrees_north
         axis : Y

      double lev ( lev )
         standard_name :        hybrid_sigma_pressure
         long_name :    hybrid level at layer midpoints
         formula :      hyam hybm (mlev=hyam+hybm*aps)
         formula_terms :        ap: hyam b: hybm ps: aps
         units :        level
         positive :     down

      double hyai ( nhyi )
         long_name :    hybrid A coefficient at layer interfaces
         units :        Pa

      double hybi ( nhyi )
         long_name :    hybrid B coefficient at layer interfaces
         units :        1

      double hyam ( nhym )
         long_name :    hybrid A coefficient at layer midpoints
         units :        Pa

      double hybm ( nhym )
         long_name :    hybrid B coefficient at layer midpoints
         units :        1

      float t ( time, lev, lat, lon )
         standard_name :        air_temperature
         long_name :    Temperature
         units :        K
         param :        0.0.0

________________________________________
From: Dennis Shea [shea at ucar.edu<mailto:shea at ucar.edu><mailto:shea at ucar.edu<mailto:shea at ucar.edu>>]
Sent: 12 March 2019 13:09
To: De Vries  Andries
Cc: ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels

I am using NCL 6.5.0. I have no problem reading the grib file.

%> ncl

 NCAR Command Language Version 6.5.0

ncl 0> f = addfile("e20c_an_temp_r360x180_ml_199410.grib","r")
ncl 1> print(f)
ncl 2> t = f->TMP_P0_L105_GLL0
printVarSummary(t)
ncl 3> printVarSummary(t)

Variable: t
Type: float
Total Size: 2941061760 bytes
            735265440 values
Number of Dimensions: 4
Dimensions and sizes:    [initial_time0_hours | 124] x [lv_HYBL0 | 91] x [lat_0 | 181] x [lon_0 | 360]
Coordinates:
            initial_time0_hours: [1707120..1707858]
            lv_HYBL0: [ 1..91]
            lat_0: [90..-90]
            lon_0: [ 0..359]
Number Of Attributes: 11
  center :    European Center for Medium-Range Weather Forecasts
  production_status :    Operational products
  long_name :    Temperature
  units :    K
  _FillValue :    1e+20
  grid_type :    Latitude/longitude
  parameter_discipline_and_category :    Meteorological products, Temperature
  parameter_template_discipline_category_number :    ( 0, 0, 0, 0 )
  level_type :    Hybrid level
  forecast_time :    0
  forecast_time_units :    hours

ncl 4> printMinMax(t,0)
(0)     Temperature (K) : min=170.211   max=313.993

On Tue, Mar 12, 2019 at 5:30 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>>>> wrote:
Dear NCL Team & Users,

I have an issue with reading in ERA-20C grib files from ECMWF that contain data on model levels. No matter if it is data on a regular (1x1 degree grid) or a gaussian grid (N80), I can not read the variables within the file, see for example the first print statements below.

When I convert the grib file to netcdf format, using the tool cdo {cdo -f nc copy {infile} {outfile}}, there is no problem; I can read in the netcdf file and variables without any problem, see the second print statement below. Ideally, however, I would like to use NCL to read in the grib files without converting them first to netcdf format. Any idea how to achieve this?

Two side comments (1) I can read in ERA-20C grib files with data on pressure levels, surface, or isentropic levels without any problem problem; there is no need to convert them to netcdf format, and (2) Also, I can read in ERA-Interim grib files with data on model levels without any problems that are downloaded in the same manner & format as the ERA-20C data.

@NCL TEAM; I copied an example ERA-20C file in grib and netcdf format with temperature on model levels for the year 199410 to ftp.cgd.ucar.edu<http://ftp.cgd.ucar.edu><http://ftp.cgd.ucar.edu><http://ftp.cgd.ucar.edu>

Kind regards,
Andries

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
READING IN ERA-20C GRIB FILE WITH DATA ON MODEL LEVELS; CANNOT READ VARIABLES
-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Copyright (C) 1995-2015 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.3.0
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
ncl 0> a=addfile("e20c_an_temp_r360x180_ml_199410.grib","r")

ncl 1> print(a)

print(a)

Variable: a
Type: file
filename:       (null)
path:   e20c_an_temp_r360x180_ml_199410.grib
   file global attributes:
   dimensions:
   variables:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
READING IN ERA-20C NETCDF FILE WITH DATA ON MODEL LEVELS; READS VARIABLES
-----------------------------------------------------------------------------------------------------------------------------------------------------------------

 Copyright (C) 1995-2015 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.3.0
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.

ncl 0> a=addfile("e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>","r")

ncl 1> print(a)

Variable: a
Type: file
filename:       e20c_an_temp_r360x180_ml_199410
path:   e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
   file global attributes:
      CDI : Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/cdi)
      Conventions : CF-1.6
      history : Thu Feb 14 11:02:46 2019: cdo -f nc copy /atmosdyn/devriesa/DATA/ERA20C/e20c_an_temp_r360x180_ml_199410.grib /atmosdyn/devriesa/DATA/ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
      institution : European Centre for Medium-Range Weather Forecasts
      CDO : Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)
   dimensions:
      time = 124  // unlimited
      lon = 360
      lat = 181
      lev = 91
      nhyi = 92
      nhym = 91
   variables:
      double time ( time )
         standard_name :        time
         units :        hours since 1994-10-1 00:00:00
         calendar :     proleptic_gregorian
         axis : T

      double lon ( lon )
         standard_name :        longitude
         long_name :    longitude
         units :        degrees_east
         axis : X

      double lat ( lat )
         standard_name :        latitude
         long_name :    latitude
         units :        degrees_north
         axis : Y

      double lev ( lev )
         standard_name :        hybrid_sigma_pressure
         long_name :    hybrid level at layer midpoints
         formula :      hyam hybm (mlev=hyam+hybm*aps)
         formula_terms :        ap: hyam b: hybm ps: aps
         units :        level
         positive :     down

      double hyai ( nhyi )
         long_name :    hybrid A coefficient at layer interfaces
         units :        Pa

      double hybi ( nhyi )
         long_name :    hybrid B coefficient at layer interfaces
         units :        1

      double hyam ( nhym )
         long_name :    hybrid A coefficient at layer midpoints
         units :        Pa

      double hybm ( nhym )
         long_name :    hybrid B coefficient at layer midpoints
         units :        1

      float t ( time, lev, lat, lon )
         standard_name :        air_temperature
         long_name :    Temperature
         units :        K
         param :        0.0.0
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk


More information about the ncl-talk mailing list