[ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels
De Vries Andries
andries.devries at env.ethz.ch
Thu Mar 14 05:26:02 MDT 2019
Hi Dennis,
Thanks a lot, I think I found the right solution following your suggestion; to place the hybrid coefficients in a constant file using the nco command. In combination with using ncl version 6.5.0, there is no need anymore to convert the ERA20C data first to netcdf format.
Briefly, my responses to your points:
1) Yes. I checked; the hybrid coefficients are identical for both the original N80 and regular 1x1 grid of the ERA-20C data. Also your description of the conversion hyam=hyam/p0 is clear; I had that implemented in my script already.
2) what I meant is that with NCL I can read in the hybrid coefficients from grib files of ERA-Interim data, I suppose cause this is the grib1 format. This works also with NCL version 6.3.0, for example. However, NCL does not find the hybrid coefficients in the ERA20C data, perhaps because (1) these are grib2 files and NCL (version 6.5.0.) misses those hybrid coefficients (a bug?), or (2) the hybrid coefficients are simply not present in ERA20C grib 2 files, and cdo adds them when these files are converted to netcdf format.
3) thank you. I understand now why the variable names in ERA20C files are messed up when I convert the grib2 format to grib1 format.
Thanks again for your help!! I really appreciate!
Cheers,
Andries
________________________________________
From: Dennis Shea [shea at ucar.edu]
Sent: 13 March 2019 21:04
To: De Vries Andries
Cc: ncl-talk at ucar.edu
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels
[1] I speculate that the hybrid coefficients are identical for the original model grid and 1x1 interpolated grid. Is this not true?
[2] You state that the CDO correctly extracts hybrid coefficients on the 1x1 interpolated grids.
The netCDF Operator (NCO) 'ncks' can be used to extract the CDO created hybrid coefficients
%> ncks -v hyai,hybi,hyam,hybm e20c_temp_199410.nc<http://e20c_temp_199410.nc> e20c_HybridCoef.nc
This file is attached. You can use this in companion with the grib2 file.
fgrb = addfile("...grib", "r")
fhy = addfile("e20c_HybridCoef.nc", "r")
hyam = fhy->hyam
hybm = fhy->hybm
---
Note that hybrid {a,b} coefficients take two different forms
NCAR's CESM atmospheric model calculates pressure at the hybrid levels use the formula p = a(k)*p0 + b(k)*psfc
ECWMF uses p = ap(k)+ b(k)*psfc : where ap(k) =a(k)*p0
All, NCL's built-in functions that use hybrid coefficients [<http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml> eg: <http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml> pres_hybrid_ccm<http://www.ncl.ucar.edu/Document/Functions/Built-in/pres_hybrid_ccm.shtml>] expect the CESM formulation.
So,
p0 = 100000.0
hyam = hyam/p0 ; convert ap(k) =a(k)*p0 to NCL expected values
===
[3] "moving target":
The WMO prescribes parameter IDs 0-127 to be fixed.
Parameter IDs 128 onward can be defined by each operational center.
NCL uses internal built-in tables to look-up the correct long_name and units associated with each parameter ID.
These tables must be updated with each NCL release.
.Good Luck
D
On Wed, Mar 13, 2019 at 9:48 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>> wrote:
Hi Dennis,
Thanks again for your comments and explanation. Do I understand correctly that NCL doesnt recognize the hybrid coefficients in the ERA-20C data?
I'm little surprise since NCL (also version 6.3.0) seems to recognize hybrid coefficients in the ERA-Interim data grib files of ECMWF, see again the print statements below. Any idea what is different between the ERA-20C and ERA-Interim data that leads to this problem with reading in hybrid coefficients? Could it be a difference of grib2 (is ERA-20C wgrib2?) and grib1 (ERA-Interim) format?
I hope to find a way to read in grib files of ERA-20C data with NCL, including the hybrid level coefficients, without needing additional tricks or conversion of data.
What do you mean by "The issue with GRIB is that it is a moving target."?
Kind regards,
Andries
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
print statement of NCL 6.3.0 of ERA-Interim data, grib file, regular grid 1x1, temperature
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
a Copyright (C) 1995-2015 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.3.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.
ncl 0> a=addfile("ei_an_temp_r1x1_ml_199410.grib","r")
print(a)
ncl 1> print(a)
Variable: a
Type: file
filename: (null)
path: ei_an_temp_r1x1_ml_199410.grib
file global attributes:
dimensions:
initial_time0_hours = 124
lv_HYBL1 = 60
lv_HYBL_i2 = 61
ncl_scalar = 1
g0_lat_4 = 181
g0_lon_5 = 360
variables:
float T_GDS0_HYBL ( initial_time0_hours, lv_HYBL1, g0_lat_4, g0_lon_5 )
center : European Center for Medium-Range Weather Forecasts (RSMC)
long_name : Temperature
units : K
_FillValue : 1e+20
level_indicator : 109
gds_grid_type : 0
parameter_table_version : 128
parameter_number : 130
forecast_time : 0
forecast_time_units : hours
double initial_time0_hours ( initial_time0_hours )
long_name : initial time
units : hours since 1800-01-01 00:00
double initial_time0_encoded ( initial_time0_hours )
long_name : initial time encoded as double
units : yyyymmddhh.hh_frac
float g0_lat_4 ( g0_lat_4 )
long_name : latitude
GridType : Cylindrical Equidistant Projection Grid
units : degrees_north
Dj : 1
Di : 1
Lo2 : 359
La2 : -90
Lo1 : 0
La1 : 90
float g0_lon_5 ( g0_lon_5 )
long_name : longitude
GridType : Cylindrical Equidistant Projection Grid
units : degrees_east
Dj : 1
Di : 1
Lo2 : 359
La2 : -90
Lo1 : 0
La1 : 90
float P0 ( ncl_scalar )
units : Pa
long_name : reference pressure
float lv_HYBL_i2_b ( lv_HYBL_i2 )
note : layer interfaces associated with hybrid levels lv_HYBL1
long_name : hybrid B coefficient at layer interfaces
float lv_HYBL_i2_a ( lv_HYBL_i2 )
note : layer interfaces associated with hybrid levels lv_HYBL1
long_name : hybrid A coefficient at layer interfaces
float lv_HYBL1_b ( lv_HYBL1 )
note : derived from lv_HYBL_i2_b as average of layer interfaces above and below midpoints
long_name : hybrid B coefficient at layer midpoints
float lv_HYBL1_a ( lv_HYBL1 )
note : derived from lv_HYBL_i2_a as average of layer interfaces above and below midpoints
long_name : hybrid A coefficient at layer midpoints
integer lv_HYBL1 ( lv_HYBL1 )
formula_terms : a: lv_HYBL1_a b: lv_HYBL1_b ps: unknown p0: P0
standard_name : atmosphere hybrid sigma pressure coordinate
long_name : hybrid level
units : number
string initial_time0 ( initial_time0_hours )
long_name : Initial time of first record
units : mm/dd/yyyy (hh:mm)
________________________________________
From: Dennis Shea [shea at ucar.edu<mailto:shea at ucar.edu>]
Sent: 12 March 2019 20:53
To: De Vries Andries
Cc: ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels
Yes. NCL does not recognize the hybrid coefficients. IMHO: It is a bug.
wgrib2 does 'see' the 92 interface coefficients.
%> wgrib2 -hybrid e20c_an_temp_r360x180_ml_199410.grib | less
The following are the (hyai, hybi) values:
1:0:Hybrid levels=92 1=( 0.000000, 2.000040) 2=( 3.980832, 7.387186) 3=(12.908319,21.413612) 4=(33.952858,51.746601) 5=(76.167656,108.715561) 6=(150.986023,204.637451) 7=(271.356506,352.824493) 8=(450.685791,566.519226) 9=(701.813354,857.945801) 10=(1036.166504,1237.585449) 11=(1463.163940,1713.709595) 12=(1989.874390,2292.155518) 13=(2620.898438,2976.302246) 14=(3358.425781,3767.196045) 15=(4202.416504,4663.776367) 16=(5150.859863,5663.156250) 17=(6199.839355,6759.727051) 18=(7341.469727,7942.926270) 19=(8564.624023,9208.305664) 20=(9873.560547,10558.881836) 21=(11262.484375,11982.662109) 22=(12713.897461,13453.225586) 23=(14192.009766,14922.685547) 24=(15638.053711,16329.560547) 25=(16990.623047,17613.281250) 26=(18191.029297,18716.968750) 27=(19184.544922,19587.513672) 28=(19919.796875,20175.394531) 29=(20348.916016,20434.158203) 30=(20426.218750,20319.011719) 31=(20107.031250,19785.357422) 32=(19348.775391,18798.822266) 33=(18141.296875,17385.595703) 34=(16544.585938,15633.566406) 35=(14665.645508,13653.219727) 36=(12608.383789,11543.166992) 37=(10471.310547,9405.222656) 38=(8356.252930,7335.164551) 39=(6353.920898,5422.802734) 40=(4550.215820,3743.464355) 41=(3010.146973,2356.202637) 42=(1784.854614,1297.656128) 43=(895.193542,576.314148) 44=(336.772369,162.043427) 45=(54.208336, 6.575628) 46=( 0.003160, 0.000000) 47=( 0.000000, 0.000000) 48=( 0.000000, 0.000000) 49=( 0.000000, 0.000000) 50=( 0.000000, 0.000000) 51=( 0.000000, 0.000000) 52=( 0.000000, 0.000000) 53=( 0.000000, 0.000000) 54=( 0.000000, 0.000000) 55=( 0.000000, 0.000000) 56=( 0.000000, 0.000000) 57=( 0.000000, 0.000000) 58=( 0.000000, 0.000000) 59=( 0.000000, 0.000000) 60=( 0.000000, 0.000000) 61=( 0.000000, 0.000000) 62=( 0.000000, 0.000000) 63=( 0.000000, 0.000000) 64=( 0.000000, 0.000014) 65=( 0.000055, 0.000131) 66=( 0.000279, 0.000548) 67=( 0.001000, 0.001701) 68=( 0.002765, 0.004267) 69=( 0.006322, 0.009035) 70=( 0.012508, 0.016860) 71=( 0.022189, 0.028610) 72=( 0.036227, 0.045146) 73=( 0.055474, 0.067316) 74=( 0.080777, 0.095964) 75=( 0.112979, 0.131935) 76=( 0.152934, 0.176091) 77=( 0.201520, 0.229315) 78=( 0.259554, 0.291993) 79=( 0.326329, 0.362203) 80=( 0.399205, 0.436906) 81=( 0.475016, 0.513280) 82=( 0.551458, 0.589317) 83=( 0.626559, 0.662934) 84=( 0.698224, 0.732224) 85=( 0.764679, 0.795385) 86=( 0.824185, 0.850950) 87=( 0.875518, 0.897767) 88=( 0.917651, 0.935157) 89=( 0.950274, 0.963007) 90=( 0.973466, 0.982238) 91=( 0.989153, 0.994204) 92=( 0.997630, 1.000000)
The mid-layer coefficients (91) are calculated as the mid-points and are calculated.
====
Still, something seems odd. My local CDO version
%> cdo -V
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)
%>cdo -f nc copy e20c_an_temp_r360x180_ml_199410.grib e20c_temp_199410.nc<http://e20c_temp_199410.nc><http://e20c_temp_199410.nc>
cdo copy: Open failed on >e20c_an_temp_r360x180_ml_199410.grib<
Unsupported file type
====
CDO Versions 1.9.3 and 1.9.4 do recognize the hybrid coefficients.
====
The issue with GRIB is that it is a moving target.
D
On Tue, Mar 12, 2019 at 7:29 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>>> wrote:
Hi Dennis,
Thank you very much. The newer version of NCL helps indeed, hadn't thought of that simple solution. However, still I miss a few coordinate variables that I can read in the netcdf file, but not in the original grib file; the hyam and hybm variabels. Please, see the print statement of the netcdf file below. Any idea why and what can help?
Cheers,
Andries
Variable: b2
Type: file
filename: e20c_an_temp_r360x180_ml_199410
path: ../ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
file global attributes:
CDI : Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/cdi)
Conventions : CF-1.6
history : Thu Feb 14 11:02:46 2019: cdo -f nc copy /atmosdyn/devriesa/DATA/ERA20C/e20c_an_temp_r360x180_ml_199410.grib /atmosdyn/devriesa/DATA/ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
institution : European Centre for Medium-Range Weather Forecasts
CDO : Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)
dimensions:
time = 124 // unlimited
lon = 360
lat = 181
lev = 91
nhyi = 92
nhym = 91
variables:
double time ( time )
standard_name : time
units : hours since 1994-10-1 00:00:00
calendar : proleptic_gregorian
axis : T
double lon ( lon )
standard_name : longitude
long_name : longitude
units : degrees_east
axis : X
double lat ( lat )
standard_name : latitude
long_name : latitude
units : degrees_north
axis : Y
double lev ( lev )
standard_name : hybrid_sigma_pressure
long_name : hybrid level at layer midpoints
formula : hyam hybm (mlev=hyam+hybm*aps)
formula_terms : ap: hyam b: hybm ps: aps
units : level
positive : down
double hyai ( nhyi )
long_name : hybrid A coefficient at layer interfaces
units : Pa
double hybi ( nhyi )
long_name : hybrid B coefficient at layer interfaces
units : 1
double hyam ( nhym )
long_name : hybrid A coefficient at layer midpoints
units : Pa
double hybm ( nhym )
long_name : hybrid B coefficient at layer midpoints
units : 1
float t ( time, lev, lat, lon )
standard_name : air_temperature
long_name : Temperature
units : K
param : 0.0.0
________________________________________
From: Dennis Shea [shea at ucar.edu<mailto:shea at ucar.edu><mailto:shea at ucar.edu<mailto:shea at ucar.edu>>]
Sent: 12 March 2019 13:09
To: De Vries Andries
Cc: ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>
Subject: Re: [ncl-talk] reading in ERA-20C grib file with data on model/hybrid levels
I am using NCL 6.5.0. I have no problem reading the grib file.
%> ncl
NCAR Command Language Version 6.5.0
ncl 0> f = addfile("e20c_an_temp_r360x180_ml_199410.grib","r")
ncl 1> print(f)
ncl 2> t = f->TMP_P0_L105_GLL0
printVarSummary(t)
ncl 3> printVarSummary(t)
Variable: t
Type: float
Total Size: 2941061760 bytes
735265440 values
Number of Dimensions: 4
Dimensions and sizes: [initial_time0_hours | 124] x [lv_HYBL0 | 91] x [lat_0 | 181] x [lon_0 | 360]
Coordinates:
initial_time0_hours: [1707120..1707858]
lv_HYBL0: [ 1..91]
lat_0: [90..-90]
lon_0: [ 0..359]
Number Of Attributes: 11
center : European Center for Medium-Range Weather Forecasts
production_status : Operational products
long_name : Temperature
units : K
_FillValue : 1e+20
grid_type : Latitude/longitude
parameter_discipline_and_category : Meteorological products, Temperature
parameter_template_discipline_category_number : ( 0, 0, 0, 0 )
level_type : Hybrid level
forecast_time : 0
forecast_time_units : hours
ncl 4> printMinMax(t,0)
(0) Temperature (K) : min=170.211 max=313.993
On Tue, Mar 12, 2019 at 5:30 AM De Vries Andries <andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch><mailto:andries.devries at env.ethz.ch<mailto:andries.devries at env.ethz.ch>>>> wrote:
Dear NCL Team & Users,
I have an issue with reading in ERA-20C grib files from ECMWF that contain data on model levels. No matter if it is data on a regular (1x1 degree grid) or a gaussian grid (N80), I can not read the variables within the file, see for example the first print statements below.
When I convert the grib file to netcdf format, using the tool cdo {cdo -f nc copy {infile} {outfile}}, there is no problem; I can read in the netcdf file and variables without any problem, see the second print statement below. Ideally, however, I would like to use NCL to read in the grib files without converting them first to netcdf format. Any idea how to achieve this?
Two side comments (1) I can read in ERA-20C grib files with data on pressure levels, surface, or isentropic levels without any problem problem; there is no need to convert them to netcdf format, and (2) Also, I can read in ERA-Interim grib files with data on model levels without any problems that are downloaded in the same manner & format as the ERA-20C data.
@NCL TEAM; I copied an example ERA-20C file in grib and netcdf format with temperature on model levels for the year 199410 to ftp.cgd.ucar.edu<http://ftp.cgd.ucar.edu><http://ftp.cgd.ucar.edu><http://ftp.cgd.ucar.edu>
Kind regards,
Andries
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
READING IN ERA-20C GRIB FILE WITH DATA ON MODEL LEVELS; CANNOT READ VARIABLES
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Copyright (C) 1995-2015 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.3.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.
ncl 0> a=addfile("e20c_an_temp_r360x180_ml_199410.grib","r")
ncl 1> print(a)
print(a)
Variable: a
Type: file
filename: (null)
path: e20c_an_temp_r360x180_ml_199410.grib
file global attributes:
dimensions:
variables:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
READING IN ERA-20C NETCDF FILE WITH DATA ON MODEL LEVELS; READS VARIABLES
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Copyright (C) 1995-2015 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.3.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.
ncl 0> a=addfile("e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>","r")
ncl 1> print(a)
Variable: a
Type: file
filename: e20c_an_temp_r360x180_ml_199410
path: e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
file global attributes:
CDI : Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/cdi)
Conventions : CF-1.6
history : Thu Feb 14 11:02:46 2019: cdo -f nc copy /atmosdyn/devriesa/DATA/ERA20C/e20c_an_temp_r360x180_ml_199410.grib /atmosdyn/devriesa/DATA/ERA20C-MOD/NC/e20c_an_temp_r360x180_ml_199410.nc<http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc><http://e20c_an_temp_r360x180_ml_199410.nc>
institution : European Centre for Medium-Range Weather Forecasts
CDO : Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)
dimensions:
time = 124 // unlimited
lon = 360
lat = 181
lev = 91
nhyi = 92
nhym = 91
variables:
double time ( time )
standard_name : time
units : hours since 1994-10-1 00:00:00
calendar : proleptic_gregorian
axis : T
double lon ( lon )
standard_name : longitude
long_name : longitude
units : degrees_east
axis : X
double lat ( lat )
standard_name : latitude
long_name : latitude
units : degrees_north
axis : Y
double lev ( lev )
standard_name : hybrid_sigma_pressure
long_name : hybrid level at layer midpoints
formula : hyam hybm (mlev=hyam+hybm*aps)
formula_terms : ap: hyam b: hybm ps: aps
units : level
positive : down
double hyai ( nhyi )
long_name : hybrid A coefficient at layer interfaces
units : Pa
double hybi ( nhyi )
long_name : hybrid B coefficient at layer interfaces
units : 1
double hyam ( nhym )
long_name : hybrid A coefficient at layer midpoints
units : Pa
double hybm ( nhym )
long_name : hybrid B coefficient at layer midpoints
units : 1
float t ( time, lev, lat, lon )
standard_name : air_temperature
long_name : Temperature
units : K
param : 0.0.0
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu><mailto:ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
More information about the ncl-talk
mailing list