[Met_help] [rt.rap.ucar.edu #96494] History for How can I easily use gridded netcdf data

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi MET help,

We are looking into using MET for some statistical analysis of GMAO forecasts. However, we only use CF-compliant netcdf files, not grib1/2. The current FAQ page says there is currently no way to explicitly define a time slice other than using an index. However, time and level information in CF-compliant netcdf files are very clearly defined with attributes. My first question is about if there any effort underway to have met lookup indices automatically/internally by specifying time strings (e.g 2020-08-28T18:00:00) or levels information similar to grib (e.g P500)? It would be extremely helpful to have the netcdf interface work as easily as the grib interface. My second question is, is there any way for grid_stat to run statistics on all levels or a range of levels in the file without specifying each 2D plane? For example, if I have 137 levels in my forecast and observation/analysis files, there are occasions when I would like to compare all fields/levels with something like !
 “(0,*,*,*)” or maybe just the lower atmosphere “(0,120:137,*,*)”. Is there any way to do these things easily with netcdf files without making very long lists of indices?

Thanks,
Scott



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: How can I easily use gridded netcdf data
From: John Halley Gotway
Time: Fri Aug 28 13:52:28 2020

Scott,

I see you have a question about requesting data from CF-compliant
NetCDF
files using a time stamp instead of an index into the time dimension.

This functionality was actually added in MET version 8.1. At least,
that's
where I found it in the release notes:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
version-8-1#notes
*Enhanced the NetCDF library to request data using a timestring
instead of
requiring an explicit index for the time dimension.*

And scanning the release notes for met-9.0 and met-9.1 there were some
other enhancements/bugfixes somewhat related to this:
met-9.0:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
version-9-0#notes
met-9.1: https://dtcenter.github.io/MET/Users_Guide/release-notes.html

If you remember where you saw that outdated information, please point
me in
the right direction so I can update it.

By way of example, I used a sample CF-compliant NetCDF file containing
a
variable for:

float edr(time, z, y, x) ;


And I ran it through MET's plot_data_plane tool like this:


plot_data_plane gtg_obs_forecast.20130404.i12.f06.nc edr.ps
'name="edr";
level="(0,0,*,*)";'

plot_data_plane gtg_obs_forecast.20130404.i12.f06.nc edr.ps
'name="edr";
level="(20130404_18,0,*,*)";'


So replacing the time index of "0" with the timestring of
"20130404_18"
works fine. Note that the timestring should be in
"YYYYMMDD[_HH[MMSS]]"
format, where those brackets indicate optional elements. So
"20200807",
"20200807_00", "20200807_000000" are all equivalent.


So that should help with selecting time values. However, this logic
currently only applies to the time dimension, not others, such as
height.


As long as you are comparing CF forecasts to CF observations, I can
certainly see how support for ranges would be useful, such as in Grid-
Stat:


fcst = {

   field = [

      { name = "edr"; level = [ "(20130404_18,0-10,*,*)" ]; }

   ];

}

obs = fcst;


But this doesn't currently work and results in a config file parsing
error.
Instead, you'd need to list out each of these 11 levels separately.
This is
the closest issue I found in GitHub which relates this this idea:

https://github.com/dtcenter/MET/issues/654


I certainly can see how the NetCDF library logic could be extended to
accommodate this logic. Makes sense for Grid-Stat, Series-Analysis,
and
MODE Time Domain. It could also work for Point-Stat, but the
complication
is that it expects you to specify point observations following GRIB
conventions. So even when you use short-hand to request forecast
fields,
you'd likely need to explicitly specify the 11 matching observation
levels
to be used.


Hope that helps clarify.


Thanks,
John Halley Gotway

On Fri, Aug 28, 2020 at 10:12 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:

>
> Fri Aug 28 10:12:14 2020: Request 96494 was acted upon.
> Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
>        Queue: met_help
>      Subject: How can I easily use gridded netcdf data
>        Owner: Nobody
>   Requestors: scott.d.rabenhorst at nasa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96494 >
>
>
> Hi MET help,
>
> We are looking into using MET for some statistical analysis of GMAO
> forecasts. However, we only use CF-compliant netcdf files, not
grib1/2. The
> current FAQ page says there is currently no way to explicitly define
a time
> slice other than using an index. However, time and level information
in
> CF-compliant netcdf files are very clearly defined with attributes.
My
> first question is about if there any effort underway to have met
lookup
> indices automatically/internally by specifying time strings (e.g
> 2020-08-28T18:00:00) or levels information similar to grib (e.g
P500)? It
> would be extremely helpful to have the netcdf interface work as
easily as
> the grib interface. My second question is, is there any way for
grid_stat
> to run statistics on all levels or a range of levels in the file
without
> specifying each 2D plane? For example, if I have 137 levels in my
forecast
> and observation/analysis files, there are occasions when I would
like to
> compare all fields/levels with something like !
>  “(0,*,*,*)” or maybe just the lower atmosphere “(0,120:137,*,*)”.
Is
> there any way to do these things easily with netcdf files without
making
> very long lists of indices?
>
> Thanks,
> Scott
>
>
>

------------------------------------------------