[Met_help] [rt.rap.ucar.edu #86195] History for how to automate Pcp_combine

John Halley Gotway via RT met_help at ucar.edu
Tue Jul 9 12:07:34 MDT 2019


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I'm working with MODE and to prove some cases I used the PCP_COMBINE to
generated the forecast and observation files that I need to run MODE

if I want to 24 hs accumulated:

bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24 pcp_combine/
wrf_20170707_24hs.nc -pcpdir /data/naranda/wrf

and the output is a netcdf file that has the first 24 hs precipitation
accumulate. If I want the next 24 hs precipitation accumulate, i'll put
this

bin/pcp_combine -sum 20170708_000000 1 20170709_000000 24 pcp_combine_wrf/
wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf

or

 bin/pcp_combine -sum 20170707_250000 1 20170707_480000 24 pcp_combine_wrf/
wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf

but pcp_combine doesn't work and I don't understand why. I would like to
automate MODE but for that I need to automate PCP_COMBINE and here is my
problem because I don't know how MET could sum files (the files are in the
same folder) every 24 hs, for example.

I sent you some wrf files that I use to run pcp_combine as an example,

Thanks,

Natalí


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: how to automate Pcp_combine
From: John Halley Gotway
Time: Mon Jul 16 14:58:40 2018

Natali,

I see you're look for some advice for how to set up calls to the
pcp_combine tool.

Thanks for sending your sample data, I grabbed these files from the
ftp
site:
   wrfprs02_20170707_00.001
   wrfprs02_20170707_00.002
   wrfprs02_20170707_00.003
   wrfprs02_20170707_00.004
   wrfprs02_20170707_00.005
   wrfprs02_20170707_00.006

I'm actually surprised to see that you're precip is hourly.  The
default
for WRF-ARW is to dump runtime accumulations.  But hourly is fine.
That
should work just as well in pcp_combine.  Since I only have hours 1
through
6, I obviously can't generate a 24-hour accumulation.

I'm guessing the misunderstanding is coming from the command line
arguments
for the SUM option:
        SUM_ARGS:
                init_time *** THIS IS THE MODEL INITIALIZATION TIME***
                in_accum # The input accumulation is 01 for your data
                valid_time # This is endpoint of the desired
accumulation
interval
                out_accum # This is the desired output accumulation
interval... 24 hours in your case
                out_file
                [-pcpdir path]
                [-pcprx reg_exp]

So here's what your 2 commands should look like:

# 0-24 hours accumulation
bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24 pcp_combine/
wrf_20170707_0_24hs.nc -pcpdir /data/naranda/wrf

# 24-48 hours accumulation
bin/pcp_combine -sum 20170707_000000 1 20170709_000000 24
pcp_combine_wrf/
wrf_20170707_24_48hs.nc -pcpdir /data/naranda/wrf

Notice that the model initialization time stays the same.  But we are
requesting a different valid time... 20170708 for the first day and 09
for
the second day.  Does that make sense?

Thanks,
John


On Fri, Jul 13, 2018 at 12:48 PM natali aranda via RT
<met_help at ucar.edu>
wrote:

>
> Fri Jul 13 12:47:37 2018: Request 86195 was acted upon.
> Transaction: Ticket created by natali.g.aranda at gmail.com
>        Queue: met_help
>      Subject: how to automate Pcp_combine
>        Owner: Nobody
>   Requestors: natali.g.aranda at gmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86195 >
>
>
> Hi,
>
> I'm working with MODE and to prove some cases I used the PCP_COMBINE
to
> generated the forecast and observation files that I need to run MODE
>
> if I want to 24 hs accumulated:
>
> bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24
pcp_combine/
> wrf_20170707_24hs.nc -pcpdir /data/naranda/wrf
>
> and the output is a netcdf file that has the first 24 hs
precipitation
> accumulate. If I want the next 24 hs precipitation accumulate, i'll
put
> this
>
> bin/pcp_combine -sum 20170708_000000 1 20170709_000000 24
pcp_combine_wrf/
> wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
>
> or
>
>  bin/pcp_combine -sum 20170707_250000 1 20170707_480000 24
pcp_combine_wrf/
> wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
>
> but pcp_combine doesn't work and I don't understand why. I would
like to
> automate MODE but for that I need to automate PCP_COMBINE and here
is my
> problem because I don't know how MET could sum files (the files are
in the
> same folder) every 24 hs, for example.
>
> I sent you some wrf files that I use to run pcp_combine as an
example,
>
> Thanks,
>
> Natalí
>
>

------------------------------------------------
Subject: how to automate Pcp_combine
From: natali aranda
Time: Wed Aug 01 15:07:21 2018

Hi John,

thanks for the advice and the observation about the init_time was the
key.

but now, I would like use pcp combine with imerg files.

In aranda_data I put 4 files as examples,

for example: if I would like to generate 2-hour accumulation,

bin/pcp_combine -sum 20180201_000000 0030 20180201_015959 2
pcp_combine/
imerg_acu.nc -pcpdir /data/naranda/imerg_201802

but it doesn't work. For the init_time info, I use the
StartGranuleDateTime
from the first file and for the valid_time, the StopGranuleDateTime
from
the last file.

I don't know where is my mistake.


I can use the function add, and it works, but I would like to use sum
because I will automate this process


Thanks for your help!

2018-07-16 17:58 GMT-03:00 John Halley Gotway via RT
<met_help at ucar.edu>:

> Natali,
>
> I see you're look for some advice for how to set up calls to the
> pcp_combine tool.
>
> Thanks for sending your sample data, I grabbed these files from the
ftp
> site:
>    wrfprs02_20170707_00.001
>    wrfprs02_20170707_00.002
>    wrfprs02_20170707_00.003
>    wrfprs02_20170707_00.004
>    wrfprs02_20170707_00.005
>    wrfprs02_20170707_00.006
>
> I'm actually surprised to see that you're precip is hourly.  The
default
> for WRF-ARW is to dump runtime accumulations.  But hourly is fine.
That
> should work just as well in pcp_combine.  Since I only have hours 1
through
> 6, I obviously can't generate a 24-hour accumulation.
>
> I'm guessing the misunderstanding is coming from the command line
arguments
> for the SUM option:
>         SUM_ARGS:
>                 init_time *** THIS IS THE MODEL INITIALIZATION
TIME***
>                 in_accum # The input accumulation is 01 for your
data
>                 valid_time # This is endpoint of the desired
accumulation
> interval
>                 out_accum # This is the desired output accumulation
> interval... 24 hours in your case
>                 out_file
>                 [-pcpdir path]
>                 [-pcprx reg_exp]
>
> So here's what your 2 commands should look like:
>
> # 0-24 hours accumulation
> bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24
pcp_combine/
> wrf_20170707_0_24hs.nc -pcpdir /data/naranda/wrf
>
> # 24-48 hours accumulation
> bin/pcp_combine -sum 20170707_000000 1 20170709_000000 24
pcp_combine_wrf/
> wrf_20170707_24_48hs.nc -pcpdir /data/naranda/wrf
>
> Notice that the model initialization time stays the same.  But we
are
> requesting a different valid time... 20170708 for the first day and
09 for
> the second day.  Does that make sense?
>
> Thanks,
> John
>
>
> On Fri, Jul 13, 2018 at 12:48 PM natali aranda via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Jul 13 12:47:37 2018: Request 86195 was acted upon.
> > Transaction: Ticket created by natali.g.aranda at gmail.com
> >        Queue: met_help
> >      Subject: how to automate Pcp_combine
> >        Owner: Nobody
> >   Requestors: natali.g.aranda at gmail.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86195 >
> >
> >
> > Hi,
> >
> > I'm working with MODE and to prove some cases I used the
PCP_COMBINE to
> > generated the forecast and observation files that I need to run
MODE
> >
> > if I want to 24 hs accumulated:
> >
> > bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24
pcp_combine/
> > wrf_20170707_24hs.nc -pcpdir /data/naranda/wrf
> >
> > and the output is a netcdf file that has the first 24 hs
precipitation
> > accumulate. If I want the next 24 hs precipitation accumulate,
i'll put
> > this
> >
> > bin/pcp_combine -sum 20170708_000000 1 20170709_000000 24
> pcp_combine_wrf/
> > wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
> >
> > or
> >
> >  bin/pcp_combine -sum 20170707_250000 1 20170707_480000 24
> pcp_combine_wrf/
> > wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
> >
> > but pcp_combine doesn't work and I don't understand why. I would
like to
> > automate MODE but for that I need to automate PCP_COMBINE and here
is my
> > problem because I don't know how MET could sum files (the files
are in
> the
> > same folder) every 24 hs, for example.
> >
> > I sent you some wrf files that I use to run pcp_combine as an
example,
> >
> > Thanks,
> >
> > Natalí
> >
> >
>
>

------------------------------------------------
Subject: how to automate Pcp_combine
From: John Halley Gotway
Time: Tue Aug 07 12:51:14 2018

Natali,

The pcp_combine "-sum" command only works for GRIB1 and GRIB2 input
data
files.  Since you're using NetCDF, the -sum option won't work.
Instead,
you can use the -add option:

pcp_combine -add \
  3B-HHR.MS.MRG.3IMERG.20180201-S000000-E002959.0000.V05B.nc-edit
'name="precipitationCal"; level="(*,*)"; file_type=NETCDF_NCCF;'  \
  3B-HHR.MS.MRG.3IMERG.20180201-S010000-E012959.0060.V05B.nc-edit
'name="precipitationCal"; level="(*,*)"; file_type=NETCDF_NCCF;'  \
  3B-HHR.MS.MRG.3IMERG.20180201-S003000-E005959.0030.V05B.nc-edit
'name="precipitationCal"; level="(*,*)"; file_type=NETCDF_NCCF;'  \
  3B-HHR.MS.MRG.3IMERG.20180201-S013000-E015959.0090.V05B.nc-edit
'name="precipitationCal"; level="(*,*)"; file_type=NETCDF_NCCF;' \
  3B-HHR.MS.MRG.3IMERG.20180201-S013000-E015959.00_to_02.V05B.nc

That will produce output, but I think you should be careful in it's
interpretation for two reasons:

(1) MET isn't parsing the valid time for these files.  So it's setting
it
to a Unix time value of 0 (Jan 1, 1970):
  WARNING: NcCfFile::open() -> could not determine the valid time,
using 0.

(2) Each 30 minute input file contains a precipitation *RATE* (not an
accumulation):
        float precipitationCal(lon, lat) ;
                precipitationCal:Units = "mm/hr" ;

   So you're adding together 4 precipitation rates.  Up to you to
interpret
the output values.

Thanks,
John

On Wed, Aug 1, 2018 at 3:07 PM natali aranda via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86195 >
>
> Hi John,
>
> thanks for the advice and the observation about the init_time was
the key.
>
> but now, I would like use pcp combine with imerg files.
>
> In aranda_data I put 4 files as examples,
>
> for example: if I would like to generate 2-hour accumulation,
>
> bin/pcp_combine -sum 20180201_000000 0030 20180201_015959 2
pcp_combine/
> imerg_acu.nc -pcpdir /data/naranda/imerg_201802
>
> but it doesn't work. For the init_time info, I use the
StartGranuleDateTime
> from the first file and for the valid_time, the StopGranuleDateTime
from
> the last file.
>
> I don't know where is my mistake.
>
>
> I can use the function add, and it works, but I would like to use
sum
> because I will automate this process
>
>
> Thanks for your help!
>
> 2018-07-16 17:58 GMT-03:00 John Halley Gotway via RT
<met_help at ucar.edu>:
>
> > Natali,
> >
> > I see you're look for some advice for how to set up calls to the
> > pcp_combine tool.
> >
> > Thanks for sending your sample data, I grabbed these files from
the ftp
> > site:
> >    wrfprs02_20170707_00.001
> >    wrfprs02_20170707_00.002
> >    wrfprs02_20170707_00.003
> >    wrfprs02_20170707_00.004
> >    wrfprs02_20170707_00.005
> >    wrfprs02_20170707_00.006
> >
> > I'm actually surprised to see that you're precip is hourly.  The
default
> > for WRF-ARW is to dump runtime accumulations.  But hourly is fine.
That
> > should work just as well in pcp_combine.  Since I only have hours
1
> through
> > 6, I obviously can't generate a 24-hour accumulation.
> >
> > I'm guessing the misunderstanding is coming from the command line
> arguments
> > for the SUM option:
> >         SUM_ARGS:
> >                 init_time *** THIS IS THE MODEL INITIALIZATION
TIME***
> >                 in_accum # The input accumulation is 01 for your
data
> >                 valid_time # This is endpoint of the desired
accumulation
> > interval
> >                 out_accum # This is the desired output
accumulation
> > interval... 24 hours in your case
> >                 out_file
> >                 [-pcpdir path]
> >                 [-pcprx reg_exp]
> >
> > So here's what your 2 commands should look like:
> >
> > # 0-24 hours accumulation
> > bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24
pcp_combine/
> > wrf_20170707_0_24hs.nc -pcpdir /data/naranda/wrf
> >
> > # 24-48 hours accumulation
> > bin/pcp_combine -sum 20170707_000000 1 20170709_000000 24
> pcp_combine_wrf/
> > wrf_20170707_24_48hs.nc -pcpdir /data/naranda/wrf
> >
> > Notice that the model initialization time stays the same.  But we
are
> > requesting a different valid time... 20170708 for the first day
and 09
> for
> > the second day.  Does that make sense?
> >
> > Thanks,
> > John
> >
> >
> > On Fri, Jul 13, 2018 at 12:48 PM natali aranda via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > Fri Jul 13 12:47:37 2018: Request 86195 was acted upon.
> > > Transaction: Ticket created by natali.g.aranda at gmail.com
> > >        Queue: met_help
> > >      Subject: how to automate Pcp_combine
> > >        Owner: Nobody
> > >   Requestors: natali.g.aranda at gmail.com
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86195
> >
> > >
> > >
> > > Hi,
> > >
> > > I'm working with MODE and to prove some cases I used the
PCP_COMBINE to
> > > generated the forecast and observation files that I need to run
MODE
> > >
> > > if I want to 24 hs accumulated:
> > >
> > > bin/pcp_combine -sum 20170707_000000 1 20170708_000000 24
pcp_combine/
> > > wrf_20170707_24hs.nc -pcpdir /data/naranda/wrf
> > >
> > > and the output is a netcdf file that has the first 24 hs
precipitation
> > > accumulate. If I want the next 24 hs precipitation accumulate,
i'll put
> > > this
> > >
> > > bin/pcp_combine -sum 20170708_000000 1 20170709_000000 24
> > pcp_combine_wrf/
> > > wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
> > >
> > > or
> > >
> > >  bin/pcp_combine -sum 20170707_250000 1 20170707_480000 24
> > pcp_combine_wrf/
> > > wrf_20170708_24hs.nc -pcpdir /data/naranda/wrf
> > >
> > > but pcp_combine doesn't work and I don't understand why. I would
like
> to
> > > automate MODE but for that I need to automate PCP_COMBINE and
here is
> my
> > > problem because I don't know how MET could sum files (the files
are in
> > the
> > > same folder) every 24 hs, for example.
> > >
> > > I sent you some wrf files that I use to run pcp_combine as an
example,
> > >
> > > Thanks,
> > >
> > > Natalí
> > >
> > >
> >
> >
>
>

------------------------------------------------


More information about the Met_help mailing list