[Met_help] [rt.rap.ucar.edu #78783] History for pcp_combine data format
John Halley Gotway via RT
met_help at ucar.edu
Fri Dec 2 09:17:33 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi all,
I try to start MET with pcp_combine, I found wrfoutput can't be directly used. Form the former questions and answers , it is said :
Here's what's required:
- The dimensions should be named lat and lon.
- The lat and lon variables are actually not used by MET, but it's
fine if they're present.
- The variable attributes should contain the "name" and "level"
attributes. In this example "A1" means that this is precip
accumulated over 1 hour.
- The variable attributes should contain timing information:
init_time_ut, valid_time_ut, and accum_time_sec for the initialization
and valid times in unixtime (date +%s) and the accumulation interval
as a number of seconds.
- The global attributes should contain "MET_version" so that MET knows
how to parse the projection information.
- The global attributes for a lat/lon projection are: Projection,
lat_ll, lon_ll, delat_lat, delta_lon, Nlat, and Nlon (pretty self-
explanatory).
Give that a shot and just let me know if you get stuck.
Thanks,
John Halley Gotway
// global attributes:
:FileOrigins = "File ex_latlon.nc generated
20120726_134701 UTC on host rambler by the MET pcp_combine tool" ;
:MET_version = "V4.0" ;
:MET_tool = "pcp_combine" ;
:RunCommand = "Addition: 1 files." ;
:Projection = "LatLon" ;
:lat_ll = "25.000000 degrees_north" ;
:lon_ll = "-130.000000 degrees_east" ;
:delta_lat = "0.500000 degrees" ;
:delta_lon = "0.500000 degrees" ;
:Nlat = "60 grid_points" ;
:Nlon = "120 grid_points" ; }
************************************
So when I use UPP convert wrfoutput to grib2, whether the projection has to be "lat-lon" ?
I found lots of input data in MET is grib format, does that mean my input data has to be grib format ?
If I use the observations which is netcdf and generated by ourselfs , how can I convert it to the grib format?
Thank you very much!
Mandy
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: pcp_combine data format
From: John Halley Gotway
Time: Tue Nov 29 12:40:49 2016
Mandy,
MET can read gridded data in GRIB1, GRIB2, or 3 different flavors of
NetCDF:
- The "internal" NetCDF format that looks like the output of the
pcp_combine tool. That's what you've referenced above.
- CF-compliant NetCDF3 files.
- The output of the wrf_interp utility.
If you have the NetCDF output from WRF, yes, I would suggest using UPP
to
post-process it. It does not need to be "lat-lon"... it can be
post-processed to whatever projection you'd like.
There is in general no easy way to convert NetCDF to GRIB. If you
have
NetCDF data you've generated yourself, I'd suggest either making it
look
like the NetCDF output from pcp_combine, or preferably, make it
CF-compliant.
Hope that helps.
Thanks,
John Halley Gotway
On Tue, Nov 29, 2016 at 10:35 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Tue Nov 29 10:35:54 2016: Request 78783 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: pcp_combine data format
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78783 >
>
>
> Hi all,
>
> I try to start MET with pcp_combine, I found wrfoutput can't be
> directly used. Form the former questions and answers , it is said :
>
> Here's what's required:
>
> - The dimensions should be named lat and lon.
> - The lat and lon variables are actually not used by MET, but it's
> fine if they're present.
> - The variable attributes should contain the "name" and "level"
> attributes. In this example "A1" means that this is precip
> accumulated over 1 hour.
> - The variable attributes should contain timing information:
> init_time_ut, valid_time_ut, and accum_time_sec for the
initialization
> and valid times in unixtime (date +%s) and the accumulation interval
> as a number of seconds.
> - The global attributes should contain "MET_version" so that MET
knows
> how to parse the projection information.
> - The global attributes for a lat/lon projection are: Projection,
> lat_ll, lon_ll, delat_lat, delta_lon, Nlat, and Nlon (pretty self-
> explanatory).
>
> Give that a shot and just let me know if you get stuck.
>
> Thanks,
> John Halley Gotway
>
> // global attributes:
> :FileOrigins = "File ex_latlon.nc generated
> 20120726_134701 UTC on host rambler by the MET pcp_combine tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Addition: 1 files." ;
> :Projection = "LatLon" ;
> :lat_ll = "25.000000 degrees_north" ;
> :lon_ll = "-130.000000 degrees_east" ;
> :delta_lat = "0.500000 degrees" ;
> :delta_lon = "0.500000 degrees" ;
> :Nlat = "60 grid_points" ;
> :Nlon = "120 grid_points" ; }
>
>
>
> ************************************
>
> So when I use UPP convert wrfoutput to grib2, whether the projection
has
> to be "lat-lon" ?
>
> I found lots of input data in MET is grib format, does that mean my
input
> data has to be grib format ?
>
> If I use the observations which is netcdf and generated by ourselfs
, how
> can I convert it to the grib format?
>
> Thank you very much!
>
> Mandy
>
>
------------------------------------------------
Subject: pcp_combine data format
From: Zhang, Mandy
Time: Wed Nov 30 23:25:35 2016
Hi John,
I'm still working with pcp_combine test..
I have WRF output and postprocess with UPP. the data is grib format,
wrfprs_d01.00, wrfprs_d01.01,wrfprs_d01.02...then I do pcp_combine
like :
#~bin/bash
echo
/home/mzhang/tool/met/bin/pcp_combine \
00000000_000000 1 20160427_010000 1 \
pcp_20160427_24hr.nc \
-pcpdir /home/mzhang/output/model/wrfdata/grib/postprd/pcp
******************
it works well!
but if I do
#~bin/bash
echo
/home/mzhang/tool/met/bin/pcp_combine \
00000000_000000 1 20160427_020000 2 \
pcp_20160427_24hr.nc \
-pcpdir /home/mzhang/output/model/wrfdata/grib/postprd/pcp
******
the error follows blow:
ERROR : sum_data_files() -> Cannot find a file with a valid time of
20160427_020000 and accumulation time of 010000 matching the regular
expression ".*"
ERROR :
test_pcp.sh: line 7: /home/mzhang: Is a directory
test_pcp.sh: line 8: /home/mzhang: Is a directory
test_pcp.sh: line 9: /home/mzhang: Is a directory
test_pcp.sh: line 10: /home/mzhang: Is a directory
test_pcp.sh: line 11: /home/mzhang: Is a directory
*****************
the grib data wrfprs_d01.02 header can be show by Cdo -sinfo:
....
31 : generic (ltype=108) : : levels=1
lev : 12750
bounds : 25500-0
32 : top_of_atmosphere : levels=1
toa : 0
Time coordinate : 1 step
RefTime = 2016-04-27 00:00:00 Units = hours Calendar =
proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-
MM-DD hh:mm:ss
2016-04-27 02:00:00
cdo sinfo: Processed 85 variables over 1 timestep ( 1.71s )
**********
And I repeat it many times, the same case occurs each time.
Do you have clues what's wrong with it?
Thank you very much!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, November 29, 2016 1:40:49 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #78783] pcp_combine data format
Mandy,
MET can read gridded data in GRIB1, GRIB2, or 3 different flavors of
NetCDF:
- The "internal" NetCDF format that looks like the output of the
pcp_combine tool. That's what you've referenced above.
- CF-compliant NetCDF3 files.
- The output of the wrf_interp utility.
If you have the NetCDF output from WRF, yes, I would suggest using UPP
to
post-process it. It does not need to be "lat-lon"... it can be
post-processed to whatever projection you'd like.
There is in general no easy way to convert NetCDF to GRIB. If you
have
NetCDF data you've generated yourself, I'd suggest either making it
look
like the NetCDF output from pcp_combine, or preferably, make it
CF-compliant.
Hope that helps.
Thanks,
John Halley Gotway
On Tue, Nov 29, 2016 at 10:35 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Tue Nov 29 10:35:54 2016: Request 78783 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: pcp_combine data format
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78783 >
>
>
> Hi all,
>
> I try to start MET with pcp_combine, I found wrfoutput can't be
> directly used. Form the former questions and answers , it is said :
>
> Here's what's required:
>
> - The dimensions should be named lat and lon.
> - The lat and lon variables are actually not used by MET, but it's
> fine if they're present.
> - The variable attributes should contain the "name" and "level"
> attributes. In this example "A1" means that this is precip
> accumulated over 1 hour.
> - The variable attributes should contain timing information:
> init_time_ut, valid_time_ut, and accum_time_sec for the
initialization
> and valid times in unixtime (date +%s) and the accumulation interval
> as a number of seconds.
> - The global attributes should contain "MET_version" so that MET
knows
> how to parse the projection information.
> - The global attributes for a lat/lon projection are: Projection,
> lat_ll, lon_ll, delat_lat, delta_lon, Nlat, and Nlon (pretty self-
> explanatory).
>
> Give that a shot and just let me know if you get stuck.
>
> Thanks,
> John Halley Gotway
>
> // global attributes:
> :FileOrigins = "File ex_latlon.nc generated
> 20120726_134701 UTC on host rambler by the MET pcp_combine tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "pcp_combine" ;
> :RunCommand = "Addition: 1 files." ;
> :Projection = "LatLon" ;
> :lat_ll = "25.000000 degrees_north" ;
> :lon_ll = "-130.000000 degrees_east" ;
> :delta_lat = "0.500000 degrees" ;
> :delta_lon = "0.500000 degrees" ;
> :Nlat = "60 grid_points" ;
> :Nlon = "120 grid_points" ; }
>
>
>
> ************************************
>
> So when I use UPP convert wrfoutput to grib2, whether the projection
has
> to be "lat-lon" ?
>
> I found lots of input data in MET is grib format, does that mean my
input
> data has to be grib format ?
>
> If I use the observations which is netcdf and generated by ourselfs
, how
> can I convert it to the grib format?
>
> Thank you very much!
>
> Mandy
>
>
------------------------------------------------
Subject: pcp_combine data format
From: John Halley Gotway
Time: Thu Dec 01 15:35:15 2016
Mandy,
I'd suggest that you start by looking at the accumulation interval of
the
precipitation data in your WRF output files from UPP. I always use
the
"wgrib" utility to dump out that sort of information:
wgrib wrfprs_d01.02
wgrib wrfprs_d01.03
... and so on
The question is whether your output actually contains 1-hourly
accumulated
precip, or does it contain runtime accumulation. Runtime accumulation
means that the 6-hour wrf file contains 0 to 6 hours of precip; the 7-
hour
file contains 0 to 7 hours of precip; and so on. The precip values
just
accumulate over the course of the entire model integration.
The default for WRF-ARW is runtime accumulation. So if you're running
WRF-ARW and you didn't specifically change the output bucket interval,
then
that's very likely what you have.
If that's the case, it's totally fine. You should just use the
pcp_combine
"-subtract" option instead of "-sum". Suppose you want the 6 hours of
precip between the 6hr and 12hr forecasts, you'd run the following:
pcp_combine -subtract wrfprs_d01.12 12 wrfprs_d01.06 06
apcp_06_to_12.nc
That says... get 12 hours of precip from the first file and 6 hours
from
the second file and subtract them.
Hope that helps.
Thanks,
John
On Wed, Nov 30, 2016 at 11:25 PM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78783 >
>
> Hi John,
>
> I'm still working with pcp_combine test..
>
> I have WRF output and postprocess with UPP. the data is grib format,
> wrfprs_d01.00, wrfprs_d01.01,wrfprs_d01.02...then I do pcp_combine
like :
>
> #~bin/bash
> echo
> /home/mzhang/tool/met/bin/pcp_combine \
> 00000000_000000 1 20160427_010000 1 \
> pcp_20160427_24hr.nc \
> -pcpdir /home/mzhang/output/model/wrfdata/grib/postprd/pcp
> ******************
>
> it works well!
>
> but if I do
>
> #~bin/bash
> echo
> /home/mzhang/tool/met/bin/pcp_combine \
> 00000000_000000 1 20160427_020000 2 \
> pcp_20160427_24hr.nc \
> -pcpdir /home/mzhang/output/model/wrfdata/grib/postprd/pcp
> ******
>
> the error follows blow:
>
> ERROR : sum_data_files() -> Cannot find a file with a valid time of
> 20160427_020000 and accumulation time of 010000 matching the regular
> expression ".*"
> ERROR :
> test_pcp.sh: line 7: /home/mzhang: Is a directory
> test_pcp.sh: line 8: /home/mzhang: Is a directory
> test_pcp.sh: line 9: /home/mzhang: Is a directory
> test_pcp.sh: line 10: /home/mzhang: Is a directory
> test_pcp.sh: line 11: /home/mzhang: Is a directory
> *****************
>
> the grib data wrfprs_d01.02 header can be show by Cdo -sinfo:
>
> ....
>
> 31 : generic (ltype=108) : : levels=1
> lev : 12750
> bounds : 25500-0
> 32 : top_of_atmosphere : levels=1
> toa : 0
> Time coordinate : 1 step
> RefTime = 2016-04-27 00:00:00 Units = hours Calendar =
> proleptic_gregorian
> YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
> YYYY-MM-DD hh:mm:ss
> 2016-04-27 02:00:00
> cdo sinfo: Processed 85 variables over 1 timestep ( 1.71s )
> **********
>
> And I repeat it many times, the same case occurs each time.
>
> Do you have clues what's wrong with it?
>
> Thank you very much!
>
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, November 29, 2016 1:40:49 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #78783] pcp_combine data format
>
> Mandy,
>
> MET can read gridded data in GRIB1, GRIB2, or 3 different flavors of
> NetCDF:
> - The "internal" NetCDF format that looks like the output of the
> pcp_combine tool. That's what you've referenced above.
> - CF-compliant NetCDF3 files.
> - The output of the wrf_interp utility.
>
> If you have the NetCDF output from WRF, yes, I would suggest using
UPP to
> post-process it. It does not need to be "lat-lon"... it can be
> post-processed to whatever projection you'd like.
>
> There is in general no easy way to convert NetCDF to GRIB. If you
have
> NetCDF data you've generated yourself, I'd suggest either making it
look
> like the NetCDF output from pcp_combine, or preferably, make it
> CF-compliant.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
>
>
>
> On Tue, Nov 29, 2016 at 10:35 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue Nov 29 10:35:54 2016: Request 78783 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > Queue: met_help
> > Subject: pcp_combine data format
> > Owner: Nobody
> > Requestors: Man.Zhang-1 at ou.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78783 >
> >
> >
> > Hi all,
> >
> > I try to start MET with pcp_combine, I found wrfoutput can't
be
> > directly used. Form the former questions and answers , it is said
:
> >
> > Here's what's required:
> >
> > - The dimensions should be named lat and lon.
> > - The lat and lon variables are actually not used by MET, but it's
> > fine if they're present.
> > - The variable attributes should contain the "name" and "level"
> > attributes. In this example "A1" means that this is precip
> > accumulated over 1 hour.
> > - The variable attributes should contain timing information:
> > init_time_ut, valid_time_ut, and accum_time_sec for the
initialization
> > and valid times in unixtime (date +%s) and the accumulation
interval
> > as a number of seconds.
> > - The global attributes should contain "MET_version" so that MET
knows
> > how to parse the projection information.
> > - The global attributes for a lat/lon projection are: Projection,
> > lat_ll, lon_ll, delat_lat, delta_lon, Nlat, and Nlon (pretty self-
> > explanatory).
> >
> > Give that a shot and just let me know if you get stuck.
> >
> > Thanks,
> > John Halley Gotway
> >
> > // global attributes:
> > :FileOrigins = "File ex_latlon.nc generated
> > 20120726_134701 UTC on host rambler by the MET pcp_combine tool" ;
> > :MET_version = "V4.0" ;
> > :MET_tool = "pcp_combine" ;
> > :RunCommand = "Addition: 1 files." ;
> > :Projection = "LatLon" ;
> > :lat_ll = "25.000000 degrees_north" ;
> > :lon_ll = "-130.000000 degrees_east" ;
> > :delta_lat = "0.500000 degrees" ;
> > :delta_lon = "0.500000 degrees" ;
> > :Nlat = "60 grid_points" ;
> > :Nlon = "120 grid_points" ; }
> >
> >
> >
> > ************************************
> >
> > So when I use UPP convert wrfoutput to grib2, whether the
projection has
> > to be "lat-lon" ?
> >
> > I found lots of input data in MET is grib format, does that mean
my input
> > data has to be grib format ?
> >
> > If I use the observations which is netcdf and generated by
ourselfs ,
> how
> > can I convert it to the grib format?
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list