[Met_help] [rt.rap.ucar.edu #75687] History for NetCDF Model Grids
John Halley Gotway via RT
met_help at ucar.edu
Wed Mar 30 13:57:58 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John or Julie, I know I am using up my question quota but I have a more general question. I have to take a binary model (flat ) file and convert it to NetCDF so MET can us it. The model file only has data values and not latitudes or longitudes. Will I have to include latitudes and longitudes for each grid point in the NetCDF file?
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Thu Mar 24 15:20:11 2016
Bob,
One option would be to make it look like the NetCDF output file from
the
pcp_combine tool. While that does include lat and lon variables, MET
does
not actually read them. So no, you could skip those. But the global
attributes are used to define the grid information.
Alternatively, you could write it as a CF-compliant NetCDF file. I
have
very few examples of those, but for the next release we do plan on
enabling
MET to read/write NetCDF4 and write CF-compliant files ourselves.
What's the source of this binary file?
Thanks,
John
On Thu, Mar 24, 2016 at 3:09 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Mar 24 15:09:27 2016: Request 75653 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: NetCDF Question
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75653 >
>
>
> John or Julie, I know I am using up my question quota but I have a
more
> general question. I have to take a binary model (flat ) file and
convert
> it to NetCDF so MET can us it. The model file only has data values
and
> not latitudes or longitudes. Will I have to include latitudes and
> longitudes for each grid point in the NetCDF file?
>
> Thanks
> Bob
>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75653] NetCDF Question
From: robert.craig.2 at us.af.mil
Time: Thu Mar 24 15:26:32 2016
It is a model file generated from ensemble members. Instead of the
ensemble mean they wanted the median, so I calculate that outside of
MET. Then I was planning to read it back into MET as a NetCDF file
and run point stat in it. It is part of a dust verification project
we are doing.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, March 24, 2016 4:20 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75653] NetCDF Question
Bob,
One option would be to make it look like the NetCDF output file from
the pcp_combine tool. While that does include lat and lon variables,
MET does not actually read them. So no, you could skip those. But
the global attributes are used to define the grid information.
Alternatively, you could write it as a CF-compliant NetCDF file. I
have very few examples of those, but for the next release we do plan
on enabling MET to read/write NetCDF4 and write CF-compliant files
ourselves.
What's the source of this binary file?
Thanks,
John
On Thu, Mar 24, 2016 at 3:09 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Mar 24 15:09:27 2016: Request 75653 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: NetCDF Question
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75653
> >
>
>
> John or Julie, I know I am using up my question quota but I have a
> more general question. I have to take a binary model (flat ) file
and convert
> it to NetCDF so MET can us it. The model file only has data values
and
> not latitudes or longitudes. Will I have to include latitudes and
> longitudes for each grid point in the NetCDF file?
>
> Thanks
> Bob
>
>
>
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Fri Mar 25 14:19:53 2016
John, is there any documentation for this? I not aware of what NetCDF
fields
I have to set for MET to be happy. Attached is list of fields a
colleague
used to convert hail to NetDCF. Also, I will be converting a model
forecast
file to NetCDF, not an ob file, so should I still be setting it up for
PCP
combine? Below is the header for the model file before I degribbed it:
RDREF= Radio Refractivity [no dim]
timerange 10 P1 12 P2 168 TimeU 0 nx 620 ny 210 GDS grid 0
num_in_ave 0
missing 0
center 57 subcenter 0 process 220 Table 5 scan: WE:SN winds(N/S)
latlon: lat -0.857000 to 34.843000 by 0.170000 nxny 130200
long -31.064000 to 80.480000 by 0.180000, (620 x 210) scan
64 mode
128 bdsgrid 1
min/max data 2765.8 999988 num bits 16 BDS_Ref 27658 DecScale 1
BinScale
8
By the way, this field is not Radio Refractivity (a huge pet peeve of
my that
I can't get my colleagues here to properly fill out the grib header)
but
visibility median from 12 members.
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, March 24, 2016 4:20 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75653] NetCDF Question
Bob,
One option would be to make it look like the NetCDF output file from
the
pcp_combine tool. While that does include lat and lon variables, MET
does not
actually read them. So no, you could skip those. But the global
attributes
are used to define the grid information.
Alternatively, you could write it as a CF-compliant NetCDF file. I
have very
few examples of those, but for the next release we do plan on enabling
MET to
read/write NetCDF4 and write CF-compliant files ourselves.
What's the source of this binary file?
Thanks,
John
On Thu, Mar 24, 2016 at 3:09 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Mar 24 15:09:27 2016: Request 75653 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: NetCDF Question
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75653
> >
>
>
> John or Julie, I know I am using up my question quota but I have a
> more general question. I have to take a binary model (flat ) file
and
> convert
> it to NetCDF so MET can us it. The model file only has data values
and
> not latitudes or longitudes. Will I have to include latitudes and
> longitudes for each grid point in the NetCDF file?
>
> Thanks
> Bob
>
>
>
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Mon Mar 28 15:36:02 2016
Bob,
Sorry for the delay in getting back to you on this. Thanks for
sending the
data. I'll take a look and let you know what I find.
John
On Mon, Mar 28, 2016 at 3:24 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 28 15:24:06 2016: Request 75687 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: NetCDF Model Grids
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
>
> I wrote a script to take a model file and convert to NETCDF
(NETCDF_MET)
> type and use in point stat. The model file was a lat lon grid so
that is
> the projection I used the NETCDF conversion. So after doing that
and
> then
> running MET, I am getting the following error:
>
> ['/h/WXQC/met-5.1/bin/point_stat',
>
'/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH36.nc',
> '/h/data/global/WXQC/data/met/nc_obs/nc_2016030300',
> '/h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km', '-
outdir',
> '/h/data/global/WXQC/data/met/data/', '-v', '5']
> DEBUG 1: Default Config File:
> /h/WXQC/met-5.1/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> ERROR :
> ERROR : read_netcdf_grid_v3() -> Projection type latlon not
currently
> supported.
> ERROR :
>
> My config file is attached. So, is the error correct and MET won't
read a
> NETCDF_MET file with a latlon projection? I placed the NETCDF
file on
> your ftp server.
>
> Thanks
> Bob
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Mon Mar 28 16:22:14 2016
Bob,
Here's an example of the NetCDF variable attributes that MET uses to
define
a LatLon grid...
:Projection = "LatLon" ;
:lat_ll = "25.063000 degrees_north" ;
:lon_ll = "-124.938000 degrees_east" ;
:delta_lat = "0.125000 degrees" ;
:delta_lon = "0.125000 degrees" ;
:Nlat = "224 grid_points" ;
:Nlon = "464 grid_points" ;
FYI, I got this by running the "regrid_data_plane" tool to regrid some
GFS
data to a LatLon grid:
met-5.1/bin/regrid_data_plane \
gfs_2012040900_F012.grib G110 \
gfs_g110.nc -field 'name="TMP"; level="Z2";'
And then I used ncdump to look at the attributes. Please try defining
these global attributes (and removing the other projection-related
ones)
and then try again.
If you get stuck, please let me know.
Thanks,
John
On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by RT_System
> Queue: met_help
> Subject: NetCDF Model Grids
> Owner: johnhg
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 08:24:15 2016
John, I made your changes and got a little further. The latest error
is
below:
DEBUG 1: User Config File:
/h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_NcMet".
DEBUG 4: Latitude/Longitude Grid Data:
DEBUG 4: lat_ll = -0.857
DEBUG 4: lon_ll = 31.064
DEBUG 4: delta_lat = 0.17
DEBUG 4: delta_lon = 0.18
DEBUG 4: Nlat = 620
DEBUG 4: Nlon = 210
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type
"FileType_NcMet".
DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains 0
entries.
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=1217264214
DEBUG 1: Forecast File:
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.nc
DEBUG 1: Observation File:
/h/data/global/WXQC/data/met/nc_obs/nc_2016030200
DEBUG 2:
DEBUG
2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for USWRFZ2.
ERROR :
ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
set
corner for variable "USWRF"
Note: the lon_ll is positive though it was negative in the NetCDF file
( I
sent the latest to the ftp site). I tried reversing the sign and
playing with
the longitude description, but still get the error above
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 28, 2016 5:22 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
Here's an example of the NetCDF variable attributes that MET uses to
define a
LatLon grid...
:Projection = "LatLon" ;
:lat_ll = "25.063000 degrees_north" ;
:lon_ll = "-124.938000 degrees_east" ;
:delta_lat = "0.125000 degrees" ;
:delta_lon = "0.125000 degrees" ;
:Nlat = "224 grid_points" ;
:Nlon = "464 grid_points" ;
FYI, I got this by running the "regrid_data_plane" tool to regrid some
GFS
data to a LatLon grid:
met-5.1/bin/regrid_data_plane \
gfs_2012040900_F012.grib G110 \
gfs_g110.nc -field 'name="TMP"; level="Z2";'
And then I used ncdump to look at the attributes. Please try defining
these
global attributes (and removing the other projection-related ones) and
then
try again.
If you get stuck, please let me know.
Thanks,
John
On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by RT_System
> Queue: met_help
> Subject: NetCDF Model Grids
> Owner: johnhg
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Tue Mar 29 09:43:03 2016
Bob,
The issue is that MET expects the dimensions of USWRF to be lat
followed by
lon, as opposed to lon, lat:
double USWRF(lat, lon) ;
It should work after switching those.
I realize that this format is overly restrictive. Part of our plans
for
the next release are moving to support the enhanced data model in
NetCDF4.
In that process, we also plan to change the NetCDF file format that
MET
writes to follow the CF-convention. We do still plan on being
backward
compatible, being able to read these types of NetCDF files. But we'd
eventually like to move away from them since they're overly
restrictive.
Thanks,
John
On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, I made your changes and got a little further. The latest
error is
> below:
>
> DEBUG 1: User Config File:
> /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> DEBUG 4: Latitude/Longitude Grid Data:
> DEBUG 4: lat_ll = -0.857
> DEBUG 4: lon_ll = 31.064
> DEBUG 4: delta_lat = 0.17
> DEBUG 4: delta_lon = 0.18
> DEBUG 4: Nlat = 620
> DEBUG 4: Nlon = 210
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type
> "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1217264214
> DEBUG 1: Forecast File:
>
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.nc
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> DEBUG 2:
> DEBUG
> 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for USWRFZ2.
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
set
> corner for variable "USWRF"
>
> Note: the lon_ll is positive though it was negative in the NetCDF
file ( I
> sent the latest to the ftp site). I tried reversing the sign and
playing
> with
> the longitude description, but still get the error above
>
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 28, 2016 5:22 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> Here's an example of the NetCDF variable attributes that MET uses to
> define a
> LatLon grid...
>
> :Projection = "LatLon" ;
>
> :lat_ll = "25.063000 degrees_north" ;
>
> :lon_ll = "-124.938000 degrees_east" ;
>
> :delta_lat = "0.125000 degrees" ;
>
> :delta_lon = "0.125000 degrees" ;
>
> :Nlat = "224 grid_points" ;
>
> :Nlon = "464 grid_points" ;
>
>
> FYI, I got this by running the "regrid_data_plane" tool to regrid
some GFS
> data to a LatLon grid:
>
> met-5.1/bin/regrid_data_plane \
>
> gfs_2012040900_F012.grib G110 \
>
> gfs_g110.nc -field 'name="TMP"; level="Z2";'
>
> And then I used ncdump to look at the attributes. Please try
defining
> these
> global attributes (and removing the other projection-related ones)
and then
> try again.
>
> If you get stuck, please let me know.
>
> Thanks,
> John
>
>
>
> On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > Queue: met_help
> > Subject: NetCDF Model Grids
> > Owner: johnhg
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 12:30:43 2016
John, I made the change but get the same results. Below is the dump
of the header for the model file:
[11351336 qcteam hpc9-lsn-02d
/h/data/global/WXQC/data/met/nc_mdl/0301] ncdump
MEPS20KM_20160301_CY12_FH24.nc | more
netcdf MEPS20KM_20160301_CY12_FH24 {
dimensions:
lat = 210 ;
lon = 620 ;
variables:
double USWRF(lat, lon) ;
USWRF:name = "USWRF" ;
USWRF:long_name = "USWRF" ;
USWRF:level = "Z0" ;
USWRF:units = "m" ;
USWRF:_FillValue = -9999. ;
USWRF:init_time = "20160301_120000" ;
USWRF:init_time_ut = 120000 ;
USWRF:valid_time = "20160302_120000" ;
USWRF:valid_time_ut = 120000 ;
USWRF:accum_time = "240000" ;
USWRF:accum_time_sec = 86400 ;
// global attributes:
:FileOrigins = "Model visibility" ;
:MET_version = "V5.1" ;
:MET_tool = "pcp_combine" ;
:RunCommand = "Median: 12 files" ;
:Projection = "LatLon" ;
:lat_ll = "-0.857 degrees_north" ;
:lon_ll = "-31.064 degrees_east" ;
:delta_lat = "0.17 degrees" ;
:delta_lon = "0.18 degrees" ;
:Nlat = "620 grid_points" ;
:Nlon = "210 grid_points" ;
data:
When I run this I get
DEBUG 1: User Config File: /h/WXQC/met-
5.1/data/config/PointStatConfig_dust_meps20km
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_NcMet".
DEBUG 4: Latitude/Longitude Grid Data:
DEBUG 4: lat_ll = -0.857
DEBUG 4: lon_ll = 31.064
DEBUG 4: delta_lat = 0.17
DEBUG 4: delta_lon = 0.18
DEBUG 4: Nlat = 620
DEBUG 4: Nlon = 210
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_NcMet".
DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains 0
entries.
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744073085188694
DEBUG 1: Forecast File:
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH48.nc
DEBUG 1: Observation File:
/h/data/global/WXQC/data/met/nc_obs/nc_2016030312
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for USWRFZ2.
ERROR :
ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
set corner for variable "USWRF"
ERROR :
Note the long_ll sign – positive. In the NC file it is negative.
Could this be the problem?
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 10:43 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
The issue is that MET expects the dimensions of USWRF to be lat
followed by lon, as opposed to lon, lat:
double USWRF(lat, lon) ;
It should work after switching those.
I realize that this format is overly restrictive. Part of our plans
for the next release are moving to support the enhanced data model in
NetCDF4.
In that process, we also plan to change the NetCDF file format that
MET writes to follow the CF-convention. We do still plan on being
backward compatible, being able to read these types of NetCDF files.
But we'd eventually like to move away from them since they're overly
restrictive.
Thanks,
John
On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, I made your changes and got a little further. The latest
error
> is
> below:
>
> DEBUG 1: User Config File:
> /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> DEBUG 4: Latitude/Longitude Grid Data:
> DEBUG 4: lat_ll = -0.857
> DEBUG 4: lon_ll = 31.064
> DEBUG 4: delta_lat = 0.17
> DEBUG 4: delta_lon = 0.18
> DEBUG 4: Nlat = 620
> DEBUG 4: Nlon = 210
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1217264214
> DEBUG 1: Forecast File:
>
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.n
> c
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> DEBUG 2:
> DEBUG
> 2:
>
----------------------------------------------------------------------
> ----------
> DEBUG 2:
> DEBUG 2: Reading data for USWRFZ2.
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
> set corner for variable "USWRF"
>
> Note: the lon_ll is positive though it was negative in the NetCDF
file
> ( I sent the latest to the ftp site). I tried reversing the sign
and
> playing with the longitude description, but still get the error
above
>
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 28, 2016 5:22 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> Here's an example of the NetCDF variable attributes that MET uses to
> define a LatLon grid...
>
> :Projection = "LatLon" ;
>
> :lat_ll = "25.063000 degrees_north" ;
>
> :lon_ll = "-124.938000 degrees_east" ;
>
> :delta_lat = "0.125000 degrees" ;
>
> :delta_lon = "0.125000 degrees" ;
>
> :Nlat = "224 grid_points" ;
>
> :Nlon = "464 grid_points" ;
>
>
> FYI, I got this by running the "regrid_data_plane" tool to regrid
some
> GFS data to a LatLon grid:
>
> met-5.1/bin/regrid_data_plane \
>
> gfs_2012040900_F012.grib G110 \
>
> gfs_g110.nc -field 'name="TMP"; level="Z2";'
>
> And then I used ncdump to look at the attributes. Please try
defining
> these global attributes (and removing the other projection-related
> ones) and then try again.
>
> If you get stuck, please let me know.
>
> Thanks,
> John
>
>
>
> On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > Queue: met_help
> > Subject: NetCDF Model Grids
> > Owner: johnhg
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75687] NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 13:32:36 2016
John, after further review, I found I had my grid dimensions reversed
and that got me passed that error. I wish I would have noticed that
earlier. I have a new error but I will review things for a while
before calling the cavalry.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 10:43 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
The issue is that MET expects the dimensions of USWRF to be lat
followed by lon, as opposed to lon, lat:
double USWRF(lat, lon) ;
It should work after switching those.
I realize that this format is overly restrictive. Part of our plans
for the next release are moving to support the enhanced data model in
NetCDF4.
In that process, we also plan to change the NetCDF file format that
MET writes to follow the CF-convention. We do still plan on being
backward compatible, being able to read these types of NetCDF files.
But we'd eventually like to move away from them since they're overly
restrictive.
Thanks,
John
On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, I made your changes and got a little further. The latest
error
> is
> below:
>
> DEBUG 1: User Config File:
> /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> DEBUG 4: Latitude/Longitude Grid Data:
> DEBUG 4: lat_ll = -0.857
> DEBUG 4: lon_ll = 31.064
> DEBUG 4: delta_lat = 0.17
> DEBUG 4: delta_lon = 0.18
> DEBUG 4: Nlat = 620
> DEBUG 4: Nlon = 210
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1217264214
> DEBUG 1: Forecast File:
>
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.n
> c
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> DEBUG 2:
> DEBUG
> 2:
>
----------------------------------------------------------------------
> ----------
> DEBUG 2:
> DEBUG 2: Reading data for USWRFZ2.
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
> set corner for variable "USWRF"
>
> Note: the lon_ll is positive though it was negative in the NetCDF
file
> ( I sent the latest to the ftp site). I tried reversing the sign
and
> playing with the longitude description, but still get the error
above
>
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 28, 2016 5:22 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> Here's an example of the NetCDF variable attributes that MET uses to
> define a LatLon grid...
>
> :Projection = "LatLon" ;
>
> :lat_ll = "25.063000 degrees_north" ;
>
> :lon_ll = "-124.938000 degrees_east" ;
>
> :delta_lat = "0.125000 degrees" ;
>
> :delta_lon = "0.125000 degrees" ;
>
> :Nlat = "224 grid_points" ;
>
> :Nlon = "464 grid_points" ;
>
>
> FYI, I got this by running the "regrid_data_plane" tool to regrid
some
> GFS data to a LatLon grid:
>
> met-5.1/bin/regrid_data_plane \
>
> gfs_2012040900_F012.grib G110 \
>
> gfs_g110.nc -field 'name="TMP"; level="Z2";'
>
> And then I used ncdump to look at the attributes. Please try
defining
> these global attributes (and removing the other projection-related
> ones) and then try again.
>
> If you get stuck, please let me know.
>
> Thanks,
> John
>
>
>
> On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > Queue: met_help
> > Subject: NetCDF Model Grids
> > Owner: johnhg
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75687] NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 14:50:39 2016
John, the next issue. When I run verify I notice it is rejecting all
my obs due to valid time. If I kick the verbosity up to 5, I see that
MET thinks the valid time of the model is 19700101. See below
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_NcMet".
DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains 0
entries.
DEBUG 5: Parsed forecast field number 1:
VarInfo::dump():
MagicStr = USWRFZ2
ReqName = USWRF
Name = USWRFZ2
LongName = USWRFZ2(*,*)
Units = NA
PFlag = 0
PName = (nul)
PUnits = (nul)
VFlag = 0
Init = 19700101_000000 (0)
Valid = 19700101_000000 (0)
Lead = NA (-9999)
LevelInfo::dump():
Type = LevelType_None
TypeNum = -9999
ReqName = *,*
Name = *,*
Units = (nul)
Upper = 0
Lower = 0
VarInfoNcMet::dump():
Dimension:
Nelements = 2
Nalloc = 10
AllocInc = 10
Element # 0 = -12345
Element # 1 = -12345
The header of the NETCDF model file is:
netcdf MEPS20KM_20160301_CY00_FH12 {
dimensions:
lat = 210 ;
lon = 620 ;
variables:
double USWRF(lat, lon) ;
USWRF:name = "USWRF" ;
USWRF:long_name = "USWRF" ;
USWRF:level = "Z0" ;
USWRF:units = "m" ;
USWRF:_FillValue = -9999. ;
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = 0 ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = 12 ;
USWRF:accum_time = "000000" ;
USWRF:accum_time_sec = 0 ;
// global attributes:
:FileOrigins = "Model visibility" ;
:MET_version = "V5.1" ;
:MET_tool = "pcp_combine" ;
:RunCommand = "Median: 12 files" ;
:Projection = "LatLon" ;
:lat_ll = "-0.857 degrees_north" ;
:lon_ll = "-31.064 degrees_east" ;
:delta_lat = "0.17 degrees" ;
:delta_lon = "0.18 degrees" ;
:Nlat = "210 grid_points" ;
:Nlon = "620 grid_points" ;
So why the mismatch in dates?
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 10:43 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
The issue is that MET expects the dimensions of USWRF to be lat
followed by lon, as opposed to lon, lat:
double USWRF(lat, lon) ;
It should work after switching those.
I realize that this format is overly restrictive. Part of our plans
for the next release are moving to support the enhanced data model in
NetCDF4.
In that process, we also plan to change the NetCDF file format that
MET writes to follow the CF-convention. We do still plan on being
backward compatible, being able to read these types of NetCDF files.
But we'd eventually like to move away from them since they're overly
restrictive.
Thanks,
John
On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, I made your changes and got a little further. The latest
error
> is
> below:
>
> DEBUG 1: User Config File:
> /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> DEBUG 4: Latitude/Longitude Grid Data:
> DEBUG 4: lat_ll = -0.857
> DEBUG 4: lon_ll = 31.064
> DEBUG 4: delta_lat = 0.17
> DEBUG 4: delta_lon = 0.18
> DEBUG 4: Nlat = 620
> DEBUG 4: Nlon = 210
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1217264214
> DEBUG 1: Forecast File:
>
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.n
> c
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> DEBUG 2:
> DEBUG
> 2:
>
----------------------------------------------------------------------
> ----------
> DEBUG 2:
> DEBUG 2: Reading data for USWRFZ2.
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &) const -> can't
> set corner for variable "USWRF"
>
> Note: the lon_ll is positive though it was negative in the NetCDF
file
> ( I sent the latest to the ftp site). I tried reversing the sign
and
> playing with the longitude description, but still get the error
above
>
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 28, 2016 5:22 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> Here's an example of the NetCDF variable attributes that MET uses to
> define a LatLon grid...
>
> :Projection = "LatLon" ;
>
> :lat_ll = "25.063000 degrees_north" ;
>
> :lon_ll = "-124.938000 degrees_east" ;
>
> :delta_lat = "0.125000 degrees" ;
>
> :delta_lon = "0.125000 degrees" ;
>
> :Nlat = "224 grid_points" ;
>
> :Nlon = "464 grid_points" ;
>
>
> FYI, I got this by running the "regrid_data_plane" tool to regrid
some
> GFS data to a LatLon grid:
>
> met-5.1/bin/regrid_data_plane \
>
> gfs_2012040900_F012.grib G110 \
>
> gfs_g110.nc -field 'name="TMP"; level="Z2";'
>
> And then I used ncdump to look at the attributes. Please try
defining
> these global attributes (and removing the other projection-related
> ones) and then try again.
>
> If you get stuck, please let me know.
>
> Thanks,
> John
>
>
>
> On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > Queue: met_help
> > Subject: NetCDF Model Grids
> > Owner: johnhg
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Tue Mar 29 14:56:51 2016
Bob,
MET is reading the init and valid times from the init_time_ut and
valid_time_ut attributes:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = 0 ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = 12 ;
"UT" means unixtime and is the number of seconds since 1970-01-01.
Please
set their real value and, actually, define them as a string instead of
an
integer. We switched to using strings after realizing that NetCDF
can't
store very large values as integers, but can handle them as strings.
So
please use:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = "1456790400" ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = "1456833600" ;
FYI, I used the following 2 date commands to convert from a time
strings to
a unixtime values:
[johnhg at number5]% date -ud ''2016-03-01' UTC '00:00:00'' +%s
1456790400
[johnhg at number5]% date -ud ''2016-03-01' UTC '12:00:00'' +%s
1456833600
Thanks,
John
On Tue, Mar 29, 2016 at 2:50 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, the next issue. When I run verify I notice it is rejecting
all my
> obs due to valid time. If I kick the verbosity up to 5, I see that
MET
> thinks the valid time of the model is 19700101. See below
>
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> DEBUG 5: Parsed forecast field number 1:
> VarInfo::dump():
> MagicStr = USWRFZ2
> ReqName = USWRF
> Name = USWRFZ2
> LongName = USWRFZ2(*,*)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> VFlag = 0
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Lead = NA (-9999)
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = *,*
> Name = *,*
> Units = (nul)
> Upper = 0
> Lower = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 2
> Nalloc = 10
> AllocInc = 10
> Element # 0 = -12345
> Element # 1 = -12345
>
> The header of the NETCDF model file is:
>
> netcdf MEPS20KM_20160301_CY00_FH12 {
> dimensions:
> lat = 210 ;
> lon = 620 ;
> variables:
> double USWRF(lat, lon) ;
> USWRF:name = "USWRF" ;
> USWRF:long_name = "USWRF" ;
> USWRF:level = "Z0" ;
> USWRF:units = "m" ;
> USWRF:_FillValue = -9999. ;
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = 0 ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = 12 ;
> USWRF:accum_time = "000000" ;
> USWRF:accum_time_sec = 0 ;
>
> // global attributes:
> :FileOrigins = "Model visibility" ;
> :MET_version = "V5.1" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Median: 12 files" ;
> :Projection = "LatLon" ;
> :lat_ll = "-0.857 degrees_north" ;
> :lon_ll = "-31.064 degrees_east" ;
> :delta_lat = "0.17 degrees" ;
> :delta_lon = "0.18 degrees" ;
> :Nlat = "210 grid_points" ;
> :Nlon = "620 grid_points" ;
>
> So why the mismatch in dates?
>
> Thanks
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 29, 2016 10:43 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> The issue is that MET expects the dimensions of USWRF to be lat
followed
> by lon, as opposed to lon, lat:
> double USWRF(lat, lon) ;
>
> It should work after switching those.
>
> I realize that this format is overly restrictive. Part of our plans
for
> the next release are moving to support the enhanced data model in
NetCDF4.
> In that process, we also plan to change the NetCDF file format that
MET
> writes to follow the CF-convention. We do still plan on being
backward
> compatible, being able to read these types of NetCDF files. But
we'd
> eventually like to move away from them since they're overly
restrictive.
>
> Thanks,
> John
>
> On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> >
> > John, I made your changes and got a little further. The latest
error
> > is
> > below:
> >
> > DEBUG 1: User Config File:
> > /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > DEBUG 4: Latitude/Longitude Grid Data:
> > DEBUG 4: lat_ll = -0.857
> > DEBUG 4: lon_ll = 31.064
> > DEBUG 4: delta_lat = 0.17
> > DEBUG 4: delta_lon = 0.18
> > DEBUG 4: Nlat = 620
> > DEBUG 4: Nlon = 210
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> > of type "FileType_NcMet".
> > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains 0
> > entries.
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=1217264214
> > DEBUG 1: Forecast File:
> >
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12.n
> > c
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> > DEBUG 2:
> > DEBUG
> > 2:
> >
----------------------------------------------------------------------
> > ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for USWRFZ2.
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &) const ->
can't
> > set corner for variable "USWRF"
> >
> > Note: the lon_ll is positive though it was negative in the NetCDF
file
> > ( I sent the latest to the ftp site). I tried reversing the sign
and
> > playing with the longitude description, but still get the error
above
> >
> >
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 28, 2016 5:22 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> >
> > Bob,
> >
> > Here's an example of the NetCDF variable attributes that MET uses
to
> > define a LatLon grid...
> >
> > :Projection = "LatLon" ;
> >
> > :lat_ll = "25.063000 degrees_north" ;
> >
> > :lon_ll = "-124.938000 degrees_east" ;
> >
> > :delta_lat = "0.125000 degrees" ;
> >
> > :delta_lon = "0.125000 degrees" ;
> >
> > :Nlat = "224 grid_points" ;
> >
> > :Nlon = "464 grid_points" ;
> >
> >
> > FYI, I got this by running the "regrid_data_plane" tool to regrid
some
> > GFS data to a LatLon grid:
> >
> > met-5.1/bin/regrid_data_plane \
> >
> > gfs_2012040900_F012.grib G110 \
> >
> > gfs_g110.nc -field 'name="TMP"; level="Z2";'
> >
> > And then I used ncdump to look at the attributes. Please try
defining
> > these global attributes (and removing the other projection-related
> > ones) and then try again.
> >
> > If you get stuck, please let me know.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > > Queue: met_help
> > > Subject: NetCDF Model Grids
> > > Owner: johnhg
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > > >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75687] NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 15:03:49 2016
Who would have thought :) - some great information for the users
manual.
Thanks
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 3:57 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
MET is reading the init and valid times from the init_time_ut and
valid_time_ut attributes:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = 0 ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = 12 ;
"UT" means unixtime and is the number of seconds since 1970-01-01.
Please set their real value and, actually, define them as a string
instead of an integer. We switched to using strings after realizing
that NetCDF can't store very large values as integers, but can handle
them as strings. So please use:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = "1456790400" ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = "1456833600" ;
FYI, I used the following 2 date commands to convert from a time
strings to a unixtime values:
[johnhg at number5]% date -ud ''2016-03-01' UTC '00:00:00'' +%s
1456790400
[johnhg at number5]% date -ud ''2016-03-01' UTC '12:00:00'' +%s
1456833600
Thanks,
John
On Tue, Mar 29, 2016 at 2:50 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, the next issue. When I run verify I notice it is rejecting
all
> my obs due to valid time. If I kick the verbosity up to 5, I see
that
> MET thinks the valid time of the model is 19700101. See below
>
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> DEBUG 5: Parsed forecast field number 1:
> VarInfo::dump():
> MagicStr = USWRFZ2
> ReqName = USWRF
> Name = USWRFZ2
> LongName = USWRFZ2(*,*)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> VFlag = 0
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Lead = NA (-9999)
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = *,*
> Name = *,*
> Units = (nul)
> Upper = 0
> Lower = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 2
> Nalloc = 10
> AllocInc = 10
> Element # 0 = -12345
> Element # 1 = -12345
>
> The header of the NETCDF model file is:
>
> netcdf MEPS20KM_20160301_CY00_FH12 {
> dimensions:
> lat = 210 ;
> lon = 620 ;
> variables:
> double USWRF(lat, lon) ;
> USWRF:name = "USWRF" ;
> USWRF:long_name = "USWRF" ;
> USWRF:level = "Z0" ;
> USWRF:units = "m" ;
> USWRF:_FillValue = -9999. ;
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = 0 ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = 12 ;
> USWRF:accum_time = "000000" ;
> USWRF:accum_time_sec = 0 ;
>
> // global attributes:
> :FileOrigins = "Model visibility" ;
> :MET_version = "V5.1" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Median: 12 files" ;
> :Projection = "LatLon" ;
> :lat_ll = "-0.857 degrees_north" ;
> :lon_ll = "-31.064 degrees_east" ;
> :delta_lat = "0.17 degrees" ;
> :delta_lon = "0.18 degrees" ;
> :Nlat = "210 grid_points" ;
> :Nlon = "620 grid_points" ;
>
> So why the mismatch in dates?
>
> Thanks
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 29, 2016 10:43 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> The issue is that MET expects the dimensions of USWRF to be lat
> followed by lon, as opposed to lon, lat:
> double USWRF(lat, lon) ;
>
> It should work after switching those.
>
> I realize that this format is overly restrictive. Part of our plans
> for the next release are moving to support the enhanced data model
in NetCDF4.
> In that process, we also plan to change the NetCDF file format that
> MET writes to follow the CF-convention. We do still plan on being
> backward compatible, being able to read these types of NetCDF files.
> But we'd eventually like to move away from them since they're overly
restrictive.
>
> Thanks,
> John
>
> On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> >
> > John, I made your changes and got a little further. The latest
> > error is
> > below:
> >
> > DEBUG 1: User Config File:
> > /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > DEBUG 4: Latitude/Longitude Grid Data:
> > DEBUG 4: lat_ll = -0.857
> > DEBUG 4: lon_ll = 31.064
> > DEBUG 4: delta_lat = 0.17
> > DEBUG 4: delta_lon = 0.18
> > DEBUG 4: Nlat = 620
> > DEBUG 4: Nlon = 210
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > object of type "FileType_NcMet".
> > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains
> > 0 entries.
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=1217264214
> > DEBUG 1: Forecast File:
> >
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12
> > .n
> > c
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> > DEBUG 2:
> > DEBUG
> > 2:
> >
--------------------------------------------------------------------
> > --
> > ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for USWRFZ2.
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &) const ->
can't
> > set corner for variable "USWRF"
> >
> > Note: the lon_ll is positive though it was negative in the NetCDF
> > file ( I sent the latest to the ftp site). I tried reversing the
> > sign and playing with the longitude description, but still get the
> > error above
> >
> >
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 28, 2016 5:22 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> >
> > Bob,
> >
> > Here's an example of the NetCDF variable attributes that MET uses
to
> > define a LatLon grid...
> >
> > :Projection = "LatLon" ;
> >
> > :lat_ll = "25.063000 degrees_north" ;
> >
> > :lon_ll = "-124.938000 degrees_east" ;
> >
> > :delta_lat = "0.125000 degrees" ;
> >
> > :delta_lon = "0.125000 degrees" ;
> >
> > :Nlat = "224 grid_points" ;
> >
> > :Nlon = "464 grid_points" ;
> >
> >
> > FYI, I got this by running the "regrid_data_plane" tool to regrid
> > some GFS data to a LatLon grid:
> >
> > met-5.1/bin/regrid_data_plane \
> >
> > gfs_2012040900_F012.grib G110 \
> >
> > gfs_g110.nc -field 'name="TMP"; level="Z2";'
> >
> > And then I used ncdump to look at the attributes. Please try
> > defining these global attributes (and removing the other
> > projection-related
> > ones) and then try again.
> >
> > If you get stuck, please let me know.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > > Queue: met_help
> > > Subject: NetCDF Model Grids
> > > Owner: johnhg
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > > >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75687] NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Tue Mar 29 16:11:05 2016
Okay, that command helps. That gets me to the next issue - it thinks
all my obs points are off grid. A plot of the obs shows the north
Africa area is covered and a plot of the grid shows north Africa. So
I will be looking into that next.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 3:57 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
MET is reading the init and valid times from the init_time_ut and
valid_time_ut attributes:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = 0 ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = 12 ;
"UT" means unixtime and is the number of seconds since 1970-01-01.
Please set their real value and, actually, define them as a string
instead of an integer. We switched to using strings after realizing
that NetCDF can't store very large values as integers, but can handle
them as strings. So please use:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = "1456790400" ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = "1456833600" ;
FYI, I used the following 2 date commands to convert from a time
strings to a unixtime values:
[johnhg at number5]% date -ud ''2016-03-01' UTC '00:00:00'' +%s
1456790400
[johnhg at number5]% date -ud ''2016-03-01' UTC '12:00:00'' +%s
1456833600
Thanks,
John
On Tue, Mar 29, 2016 at 2:50 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, the next issue. When I run verify I notice it is rejecting
all
> my obs due to valid time. If I kick the verbosity up to 5, I see
that
> MET thinks the valid time of the model is 19700101. See below
>
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> DEBUG 5: Parsed forecast field number 1:
> VarInfo::dump():
> MagicStr = USWRFZ2
> ReqName = USWRF
> Name = USWRFZ2
> LongName = USWRFZ2(*,*)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> VFlag = 0
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Lead = NA (-9999)
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = *,*
> Name = *,*
> Units = (nul)
> Upper = 0
> Lower = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 2
> Nalloc = 10
> AllocInc = 10
> Element # 0 = -12345
> Element # 1 = -12345
>
> The header of the NETCDF model file is:
>
> netcdf MEPS20KM_20160301_CY00_FH12 {
> dimensions:
> lat = 210 ;
> lon = 620 ;
> variables:
> double USWRF(lat, lon) ;
> USWRF:name = "USWRF" ;
> USWRF:long_name = "USWRF" ;
> USWRF:level = "Z0" ;
> USWRF:units = "m" ;
> USWRF:_FillValue = -9999. ;
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = 0 ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = 12 ;
> USWRF:accum_time = "000000" ;
> USWRF:accum_time_sec = 0 ;
>
> // global attributes:
> :FileOrigins = "Model visibility" ;
> :MET_version = "V5.1" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Median: 12 files" ;
> :Projection = "LatLon" ;
> :lat_ll = "-0.857 degrees_north" ;
> :lon_ll = "-31.064 degrees_east" ;
> :delta_lat = "0.17 degrees" ;
> :delta_lon = "0.18 degrees" ;
> :Nlat = "210 grid_points" ;
> :Nlon = "620 grid_points" ;
>
> So why the mismatch in dates?
>
> Thanks
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 29, 2016 10:43 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> The issue is that MET expects the dimensions of USWRF to be lat
> followed by lon, as opposed to lon, lat:
> double USWRF(lat, lon) ;
>
> It should work after switching those.
>
> I realize that this format is overly restrictive. Part of our plans
> for the next release are moving to support the enhanced data model
in NetCDF4.
> In that process, we also plan to change the NetCDF file format that
> MET writes to follow the CF-convention. We do still plan on being
> backward compatible, being able to read these types of NetCDF files.
> But we'd eventually like to move away from them since they're overly
restrictive.
>
> Thanks,
> John
>
> On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> >
> > John, I made your changes and got a little further. The latest
> > error is
> > below:
> >
> > DEBUG 1: User Config File:
> > /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > DEBUG 4: Latitude/Longitude Grid Data:
> > DEBUG 4: lat_ll = -0.857
> > DEBUG 4: lon_ll = 31.064
> > DEBUG 4: delta_lat = 0.17
> > DEBUG 4: delta_lon = 0.18
> > DEBUG 4: Nlat = 620
> > DEBUG 4: Nlon = 210
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > object of type "FileType_NcMet".
> > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains
> > 0 entries.
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=1217264214
> > DEBUG 1: Forecast File:
> >
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12
> > .n
> > c
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> > DEBUG 2:
> > DEBUG
> > 2:
> >
--------------------------------------------------------------------
> > --
> > ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for USWRFZ2.
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &) const ->
can't
> > set corner for variable "USWRF"
> >
> > Note: the lon_ll is positive though it was negative in the NetCDF
> > file ( I sent the latest to the ftp site). I tried reversing the
> > sign and playing with the longitude description, but still get the
> > error above
> >
> >
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 28, 2016 5:22 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> >
> > Bob,
> >
> > Here's an example of the NetCDF variable attributes that MET uses
to
> > define a LatLon grid...
> >
> > :Projection = "LatLon" ;
> >
> > :lat_ll = "25.063000 degrees_north" ;
> >
> > :lon_ll = "-124.938000 degrees_east" ;
> >
> > :delta_lat = "0.125000 degrees" ;
> >
> > :delta_lon = "0.125000 degrees" ;
> >
> > :Nlat = "224 grid_points" ;
> >
> > :Nlon = "464 grid_points" ;
> >
> >
> > FYI, I got this by running the "regrid_data_plane" tool to regrid
> > some GFS data to a LatLon grid:
> >
> > met-5.1/bin/regrid_data_plane \
> >
> > gfs_2012040900_F012.grib G110 \
> >
> > gfs_g110.nc -field 'name="TMP"; level="Z2";'
> >
> > And then I used ncdump to look at the attributes. Please try
> > defining these global attributes (and removing the other
> > projection-related
> > ones) and then try again.
> >
> > If you get stuck, please let me know.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > > Queue: met_help
> > > Subject: NetCDF Model Grids
> > > Owner: johnhg
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > > >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75687] NetCDF Model Grids
From: robert.craig.2 at us.af.mil
Time: Wed Mar 30 07:27:45 2016
John, I think it is all working now. When I passed the init_time_ut
as a string, it didn't work. But when I passed it as an integer, it
worked. Now the off grid points are reasonable for the window I
selected and MET date on files are now correct. Thanks for the help
getting this working.
Just to let you know, there is interest with management here that we
start using the HiRA framework proposed by Marion Mittermaier for
comparing models at different resolutions. This is the single ob -
neighborhood forecast where each grid point in the neighborhood
becomes a "ensemble member" for comparison to the ob point. I don't
think MET has this but there might be interest and money to add it.
More to come on this issue.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 29, 2016 3:57 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
Bob,
MET is reading the init and valid times from the init_time_ut and
valid_time_ut attributes:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = 0 ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = 12 ;
"UT" means unixtime and is the number of seconds since 1970-01-01.
Please set their real value and, actually, define them as a string
instead of an integer. We switched to using strings after realizing
that NetCDF can't store very large values as integers, but can handle
them as strings. So please use:
USWRF:init_time = "20160301_000000" ;
USWRF:init_time_ut = "1456790400" ;
USWRF:valid_time = "20160301_120000" ;
USWRF:valid_time_ut = "1456833600" ;
FYI, I used the following 2 date commands to convert from a time
strings to a unixtime values:
[johnhg at number5]% date -ud ''2016-03-01' UTC '00:00:00'' +%s
1456790400
[johnhg at number5]% date -ud ''2016-03-01' UTC '12:00:00'' +%s
1456833600
Thanks,
John
On Tue, Mar 29, 2016 at 2:50 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, the next issue. When I run verify I notice it is rejecting
all
> my obs due to valid time. If I kick the verbosity up to 5, I see
that
> MET thinks the valid time of the model is 19700101. See below
>
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_NcMet".
> DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list contains
0
> entries.
> DEBUG 5: Parsed forecast field number 1:
> VarInfo::dump():
> MagicStr = USWRFZ2
> ReqName = USWRF
> Name = USWRFZ2
> LongName = USWRFZ2(*,*)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> VFlag = 0
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Lead = NA (-9999)
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = *,*
> Name = *,*
> Units = (nul)
> Upper = 0
> Lower = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 2
> Nalloc = 10
> AllocInc = 10
> Element # 0 = -12345
> Element # 1 = -12345
>
> The header of the NETCDF model file is:
>
> netcdf MEPS20KM_20160301_CY00_FH12 {
> dimensions:
> lat = 210 ;
> lon = 620 ;
> variables:
> double USWRF(lat, lon) ;
> USWRF:name = "USWRF" ;
> USWRF:long_name = "USWRF" ;
> USWRF:level = "Z0" ;
> USWRF:units = "m" ;
> USWRF:_FillValue = -9999. ;
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = 0 ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = 12 ;
> USWRF:accum_time = "000000" ;
> USWRF:accum_time_sec = 0 ;
>
> // global attributes:
> :FileOrigins = "Model visibility" ;
> :MET_version = "V5.1" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Median: 12 files" ;
> :Projection = "LatLon" ;
> :lat_ll = "-0.857 degrees_north" ;
> :lon_ll = "-31.064 degrees_east" ;
> :delta_lat = "0.17 degrees" ;
> :delta_lon = "0.18 degrees" ;
> :Nlat = "210 grid_points" ;
> :Nlon = "620 grid_points" ;
>
> So why the mismatch in dates?
>
> Thanks
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 29, 2016 10:43 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> The issue is that MET expects the dimensions of USWRF to be lat
> followed by lon, as opposed to lon, lat:
> double USWRF(lat, lon) ;
>
> It should work after switching those.
>
> I realize that this format is overly restrictive. Part of our plans
> for the next release are moving to support the enhanced data model
in NetCDF4.
> In that process, we also plan to change the NetCDF file format that
> MET writes to follow the CF-convention. We do still plan on being
> backward compatible, being able to read these types of NetCDF files.
> But we'd eventually like to move away from them since they're overly
restrictive.
>
> Thanks,
> John
>
> On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> >
> > John, I made your changes and got a little further. The latest
> > error is
> > below:
> >
> > DEBUG 1: User Config File:
> > /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > DEBUG 4: Latitude/Longitude Grid Data:
> > DEBUG 4: lat_ll = -0.857
> > DEBUG 4: lon_ll = 31.064
> > DEBUG 4: delta_lat = 0.17
> > DEBUG 4: delta_lon = 0.18
> > DEBUG 4: Nlat = 620
> > DEBUG 4: Nlon = 210
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > object of type "FileType_NcMet".
> > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains
> > 0 entries.
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=1217264214
> > DEBUG 1: Forecast File:
> >
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12
> > .n
> > c
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> > DEBUG 2:
> > DEBUG
> > 2:
> >
--------------------------------------------------------------------
> > --
> > ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for USWRFZ2.
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &) const ->
can't
> > set corner for variable "USWRF"
> >
> > Note: the lon_ll is positive though it was negative in the NetCDF
> > file ( I sent the latest to the ftp site). I tried reversing the
> > sign and playing with the longitude description, but still get the
> > error above
> >
> >
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 28, 2016 5:22 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> >
> > Bob,
> >
> > Here's an example of the NetCDF variable attributes that MET uses
to
> > define a LatLon grid...
> >
> > :Projection = "LatLon" ;
> >
> > :lat_ll = "25.063000 degrees_north" ;
> >
> > :lon_ll = "-124.938000 degrees_east" ;
> >
> > :delta_lat = "0.125000 degrees" ;
> >
> > :delta_lon = "0.125000 degrees" ;
> >
> > :Nlat = "224 grid_points" ;
> >
> > :Nlon = "464 grid_points" ;
> >
> >
> > FYI, I got this by running the "regrid_data_plane" tool to regrid
> > some GFS data to a LatLon grid:
> >
> > met-5.1/bin/regrid_data_plane \
> >
> > gfs_2012040900_F012.grib G110 \
> >
> > gfs_g110.nc -field 'name="TMP"; level="Z2";'
> >
> > And then I used ncdump to look at the attributes. Please try
> > defining these global attributes (and removing the other
> > projection-related
> > ones) and then try again.
> >
> > If you get stuck, please let me know.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > > Queue: met_help
> > > Subject: NetCDF Model Grids
> > > Owner: johnhg
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > > >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: NetCDF Model Grids
From: John Halley Gotway
Time: Wed Mar 30 13:57:49 2016
Bob,
I apologize for misleading you on storing unix times as integers
versus
strings. I remembered that being an issue for reading climate data.
I
checked the versions and see...
- met-5.0 reads unixtime attributes as integers.
- met-5.1 reads unixtime attributes as integers *OR* character
strings.
Thanks for the information about HiRA. I'm guessing that this is what
I've
always thought of as "neighborhood point" verification. But we'll
definitely need to dig in deeper into the details. And I'll talk to
Tara
about this as well.
For now, I'll resolve this ticket.
Thanks,
John
On Wed, Mar 30, 2016 at 7:27 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
>
> John, I think it is all working now. When I passed the
init_time_ut as a
> string, it didn't work. But when I passed it as an integer, it
worked.
> Now the off grid points are reasonable for the window I selected and
MET
> date on files are now correct. Thanks for the help getting this
working.
>
> Just to let you know, there is interest with management here that we
start
> using the HiRA framework proposed by Marion Mittermaier for
comparing
> models at different resolutions. This is the single ob -
neighborhood
> forecast where each grid point in the neighborhood becomes a
"ensemble
> member" for comparison to the ob point. I don't think MET has
this but
> there might be interest and money to add it. More to come on this
issue.
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, March 29, 2016 3:57 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
>
> Bob,
>
> MET is reading the init and valid times from the init_time_ut and
> valid_time_ut attributes:
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = 0 ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = 12 ;
>
> "UT" means unixtime and is the number of seconds since 1970-01-01.
Please
> set their real value and, actually, define them as a string instead
of an
> integer. We switched to using strings after realizing that NetCDF
can't
> store very large values as integers, but can handle them as strings.
So
> please use:
>
> USWRF:init_time = "20160301_000000" ;
> USWRF:init_time_ut = "1456790400" ;
> USWRF:valid_time = "20160301_120000" ;
> USWRF:valid_time_ut = "1456833600" ;
>
> FYI, I used the following 2 date commands to convert from a time
strings
> to a unixtime values:
>
> [johnhg at number5]% date -ud ''2016-03-01' UTC '00:00:00'' +%s
> 1456790400
>
> [johnhg at number5]% date -ud ''2016-03-01' UTC '12:00:00'' +%s
> 1456833600
>
> Thanks,
> John
>
>
> On Tue, Mar 29, 2016 at 2:50 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> >
> > John, the next issue. When I run verify I notice it is rejecting
all
> > my obs due to valid time. If I kick the verbosity up to 5, I see
that
> > MET thinks the valid time of the model is 19700101. See below
> >
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> > of type "FileType_NcMet".
> > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains 0
> > entries.
> > DEBUG 5: Parsed forecast field number 1:
> > VarInfo::dump():
> > MagicStr = USWRFZ2
> > ReqName = USWRF
> > Name = USWRFZ2
> > LongName = USWRFZ2(*,*)
> > Units = NA
> > PFlag = 0
> > PName = (nul)
> > PUnits = (nul)
> > VFlag = 0
> > Init = 19700101_000000 (0)
> > Valid = 19700101_000000 (0)
> > Lead = NA (-9999)
> > LevelInfo::dump():
> > Type = LevelType_None
> > TypeNum = -9999
> > ReqName = *,*
> > Name = *,*
> > Units = (nul)
> > Upper = 0
> > Lower = 0
> > VarInfoNcMet::dump():
> > Dimension:
> > Nelements = 2
> > Nalloc = 10
> > AllocInc = 10
> > Element # 0 = -12345
> > Element # 1 = -12345
> >
> > The header of the NETCDF model file is:
> >
> > netcdf MEPS20KM_20160301_CY00_FH12 {
> > dimensions:
> > lat = 210 ;
> > lon = 620 ;
> > variables:
> > double USWRF(lat, lon) ;
> > USWRF:name = "USWRF" ;
> > USWRF:long_name = "USWRF" ;
> > USWRF:level = "Z0" ;
> > USWRF:units = "m" ;
> > USWRF:_FillValue = -9999. ;
> > USWRF:init_time = "20160301_000000" ;
> > USWRF:init_time_ut = 0 ;
> > USWRF:valid_time = "20160301_120000" ;
> > USWRF:valid_time_ut = 12 ;
> > USWRF:accum_time = "000000" ;
> > USWRF:accum_time_sec = 0 ;
> >
> > // global attributes:
> > :FileOrigins = "Model visibility" ;
> > :MET_version = "V5.1" ;
> > :MET_tool = "pcp_combine" ;
> > :RunCommand = "Median: 12 files" ;
> > :Projection = "LatLon" ;
> > :lat_ll = "-0.857 degrees_north" ;
> > :lon_ll = "-31.064 degrees_east" ;
> > :delta_lat = "0.17 degrees" ;
> > :delta_lon = "0.18 degrees" ;
> > :Nlat = "210 grid_points" ;
> > :Nlon = "620 grid_points" ;
> >
> > So why the mismatch in dates?
> >
> > Thanks
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, March 29, 2016 10:43 AM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> >
> > Bob,
> >
> > The issue is that MET expects the dimensions of USWRF to be lat
> > followed by lon, as opposed to lon, lat:
> > double USWRF(lat, lon) ;
> >
> > It should work after switching those.
> >
> > I realize that this format is overly restrictive. Part of our
plans
> > for the next release are moving to support the enhanced data model
in
> NetCDF4.
> > In that process, we also plan to change the NetCDF file format
that
> > MET writes to follow the CF-convention. We do still plan on being
> > backward compatible, being able to read these types of NetCDF
files.
> > But we'd eventually like to move away from them since they're
overly
> restrictive.
> >
> > Thanks,
> > John
> >
> > On Tue, Mar 29, 2016 at 8:24 AM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687 >
> > >
> > > John, I made your changes and got a little further. The latest
> > > error is
> > > below:
> > >
> > > DEBUG 1: User Config File:
> > > /h/WXQC/met-5.1/data/config/PointStatConfig_dust_meps20km
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > > Met2dDataFile object of type "FileType_NcMet".
> > > DEBUG 4: Latitude/Longitude Grid Data:
> > > DEBUG 4: lat_ll = -0.857
> > > DEBUG 4: lon_ll = 31.064
> > > DEBUG 4: delta_lat = 0.17
> > > DEBUG 4: delta_lon = 0.18
> > > DEBUG 4: Nlat = 620
> > > DEBUG 4: Nlon = 210
> > > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > > object of type "FileType_NcMet".
> > > DEBUG 4: parse_conf_sid_exc() -> Station ID exclusion list
contains
> > > 0 entries.
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=1217264214
> > > DEBUG 1: Forecast File:
> > >
/h/data/global/WXQC/data/met/nc_mdl/0301/MEPS20KM_20160301_CY12_FH12
> > > .n
> > > c
> > > DEBUG 1: Observation File:
> > > /h/data/global/WXQC/data/met/nc_obs/nc_2016030200
> > > DEBUG 2:
> > > DEBUG
> > > 2:
> > >
--------------------------------------------------------------------
> > > --
> > > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for USWRFZ2.
> > > ERROR :
> > > ERROR : MetNcFile::data(NcVar *, const LongArray &) const ->
can't
> > > set corner for variable "USWRF"
> > >
> > > Note: the lon_ll is positive though it was negative in the
NetCDF
> > > file ( I sent the latest to the ftp site). I tried reversing
the
> > > sign and playing with the longitude description, but still get
the
> > > error above
> > >
> > >
> > > Bob
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, March 28, 2016 5:22 PM
> > > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #75687] NetCDF Model Grids
> > >
> > > Bob,
> > >
> > > Here's an example of the NetCDF variable attributes that MET
uses to
> > > define a LatLon grid...
> > >
> > > :Projection = "LatLon" ;
> > >
> > > :lat_ll = "25.063000 degrees_north" ;
> > >
> > > :lon_ll = "-124.938000 degrees_east" ;
> > >
> > > :delta_lat = "0.125000 degrees" ;
> > >
> > > :delta_lon = "0.125000 degrees" ;
> > >
> > > :Nlat = "224 grid_points" ;
> > >
> > > :Nlon = "464 grid_points" ;
> > >
> > >
> > > FYI, I got this by running the "regrid_data_plane" tool to
regrid
> > > some GFS data to a LatLon grid:
> > >
> > > met-5.1/bin/regrid_data_plane \
> > >
> > > gfs_2012040900_F012.grib G110 \
> > >
> > > gfs_g110.nc -field 'name="TMP"; level="Z2";'
> > >
> > > And then I used ncdump to look at the attributes. Please try
> > > defining these global attributes (and removing the other
> > > projection-related
> > > ones) and then try again.
> > >
> > > If you get stuck, please let me know.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > > On Mon, Mar 28, 2016 at 3:36 PM, The RT System itself via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Mar 28 15:36:02 2016: Request 75687 was acted upon.
> > > > Transaction: Given to johnhg (John Halley Gotway) by RT_System
> > > > Queue: met_help
> > > > Subject: NetCDF Model Grids
> > > > Owner: johnhg
> > > > Requestors: robert.craig.2 at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75687
> > > > >
> > > >
> > > >
> > > > This transaction appears to have no content
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list