[Met_help] [rt.rap.ucar.edu #77614] History for Question on supported data format
John Halley Gotway via RT
met_help at ucar.edu
Wed Oct 12 13:28:43 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello
I would like to run grid-stat tool using WRF model forecasts and TRMM
observations.
I converted original HDF-format TRMM observation to NetCDF-format, and I
also interpolated original lat/lon grid to WRF Lambert conformal grid using
NCL.
And I tried to run pcp_combine before running grid-stat tool to make
accumulated precipitation.
However, I got errors like
"get_field() -> can't open data file ***.nc"
Could you give me some advices on this problem?
And... Actually, I selected TRMM observations to verify WRF forecasts over
South Korea, where stage-4 analysis is not available.
Could you recommend other gridded precipitation observations over Southeast
Asia other than TRMM?
Thank you for your kindness.
Best regards
Yonghan Choi
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Fri Aug 19 16:40:20 2016
Yongchan,
I see that you're having trouble using gridded TRMM data in MET. I
read
that you've done some pre-processing of the TRMM data using NCL prior
to
passing it to the MET tools.
Can you please post a sample NetCDF file that you're passing to
pcp_combine
to our anonymous FTP site, following the instructions listed here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Please write once you've posted a sample file. I'll grab it and run
it
through the debugger to identify what's happening.
Thanks,
John Halley Gotway
On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT
<met_help at ucar.edu>
wrote:
>
> Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> Transaction: Ticket created by cyh082 at gmail.com
> Queue: met_help
> Subject: Question on supported data format
> Owner: Nobody
> Requestors: cyh082 at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>
>
> Hello
>
> I would like to run grid-stat tool using WRF model forecasts and
TRMM
> observations.
>
> I converted original HDF-format TRMM observation to NetCDF-format,
and I
> also interpolated original lat/lon grid to WRF Lambert conformal
grid using
> NCL.
>
> And I tried to run pcp_combine before running grid-stat tool to make
> accumulated precipitation.
>
> However, I got errors like
> "get_field() -> can't open data file ***.nc"
>
> Could you give me some advices on this problem?
>
> And... Actually, I selected TRMM observations to verify WRF
forecasts over
> South Korea, where stage-4 analysis is not available.
> Could you recommend other gridded precipitation observations over
Southeast
> Asia other than TRMM?
>
> Thank you for your kindness.
>
> Best regards
> Yonghan Choi
>
>
------------------------------------------------
Subject: Question on supported data format
From: Yonghan Choi
Time: Sun Aug 21 00:08:58 2016
Dear John Halley Gotway:
I uploaded my NetCDF-formatted TRMM data to your ftp server.
Actually, I interpolated (to WRF grids) and made accumulation using
original TRMM data, and I converted HDF-formatted original TRMM data
to
NetCDF format.
Thank you for your kindness.
Best regards
Yonghan Choi
On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Yongchan,
>
> I see that you're having trouble using gridded TRMM data in MET. I
read
> that you've done some pre-processing of the TRMM data using NCL
prior to
> passing it to the MET tools.
>
> Can you please post a sample NetCDF file that you're passing to
pcp_combine
> to our anonymous FTP site, following the instructions listed here:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Please write once you've posted a sample file. I'll grab it and run
it
> through the debugger to identify what's happening.
>
> Thanks,
> John Halley Gotway
>
> On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > Transaction: Ticket created by cyh082 at gmail.com
> > Queue: met_help
> > Subject: Question on supported data format
> > Owner: Nobody
> > Requestors: cyh082 at gmail.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> >
> >
> > Hello
> >
> > I would like to run grid-stat tool using WRF model forecasts and
TRMM
> > observations.
> >
> > I converted original HDF-format TRMM observation to NetCDF-format,
and I
> > also interpolated original lat/lon grid to WRF Lambert conformal
grid
> using
> > NCL.
> >
> > And I tried to run pcp_combine before running grid-stat tool to
make
> > accumulated precipitation.
> >
> > However, I got errors like
> > "get_field() -> can't open data file ***.nc"
> >
> > Could you give me some advices on this problem?
> >
> > And... Actually, I selected TRMM observations to verify WRF
forecasts
> over
> > South Korea, where stage-4 analysis is not available.
> > Could you recommend other gridded precipitation observations over
> Southeast
> > Asia other than TRMM?
> >
> > Thank you for your kindness.
> >
> > Best regards
> > Yonghan Choi
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Tue Aug 23 16:38:36 2016
Yonghan,
Thanks for sending that sample data file. Yes, it is definitely a
flavor
of NetCDF that the MET software is not expecting. Here's an option
for
accessing that same TRMM data, following links from the MET website:
http://www.dtcenter.org/met/users/downloads/observation_data.php
# Pull binary 3-hourly TRMM data file
wget
ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/201009/3B42.100921.00z.7.precipitation.bin
# Pull Rscript from MET website
wget http://www.dtcenter.org/met/users/downloads/Rscripts/trmmbin2nc.R
# Edit that Rscript by setting
out_lat_ll = -50
out_lon_ll = 0
out_lat_ur = 50
out_lon_ur = 359.75
# Run the Rscript
Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
3B42.100921.00z.7.precipitation.nc
# Plot the result
met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
3B42.100921.00z.7.precipitation.ps 'name="APCP_03"; level="(*,*)";'
I've attached a PNG version of resulting image.
I see that the domain of your data is much smaller. You have two
choices
here...
(1) In that Rscript, you can choose different boundaries (i.e.
out_lat/lon_ll/ur) to specify the tile of data to be selected.
(2) As of version 5.1, MET includes support for regridding the data it
reads. You could keep TRMM on it's native domain and use the MET
tools to
do the regridding. For example, the "regrid_data_plane" tool reads a
NetCDF file, regrids the data, and writes a NetCDF file.
Alternatively,
the "regrid" section of the configuration files for the MET tools may
be
used to do the regridding on the fly. For example, you could run
Grid-Stat
to compare your model output to TRMM and say "regrid = { field = FCST;
...}". That tells Grid-Stat to automatically regrid the TRMM
observations
to your model domain.
Hope that helps. Do this sounds like a workable path for you?
Thanks,
John
On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>
> Dear John Halley Gotway:
>
> I uploaded my NetCDF-formatted TRMM data to your ftp server.
>
> Actually, I interpolated (to WRF grids) and made accumulation using
> original TRMM data, and I converted HDF-formatted original TRMM data
to
> NetCDF format.
>
> Thank you for your kindness.
>
> Best regards
> Yonghan Choi
>
> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Yongchan,
> >
> > I see that you're having trouble using gridded TRMM data in MET.
I read
> > that you've done some pre-processing of the TRMM data using NCL
prior to
> > passing it to the MET tools.
> >
> > Can you please post a sample NetCDF file that you're passing to
> pcp_combine
> > to our anonymous FTP site, following the instructions listed here:
> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > Please write once you've posted a sample file. I'll grab it and
run it
> > through the debugger to identify what's happening.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > > Transaction: Ticket created by cyh082 at gmail.com
> > > Queue: met_help
> > > Subject: Question on supported data format
> > > Owner: Nobody
> > > Requestors: cyh082 at gmail.com
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614
> >
> > >
> > >
> > > Hello
> > >
> > > I would like to run grid-stat tool using WRF model forecasts and
TRMM
> > > observations.
> > >
> > > I converted original HDF-format TRMM observation to NetCDF-
format, and
> I
> > > also interpolated original lat/lon grid to WRF Lambert conformal
grid
> > using
> > > NCL.
> > >
> > > And I tried to run pcp_combine before running grid-stat tool to
make
> > > accumulated precipitation.
> > >
> > > However, I got errors like
> > > "get_field() -> can't open data file ***.nc"
> > >
> > > Could you give me some advices on this problem?
> > >
> > > And... Actually, I selected TRMM observations to verify WRF
forecasts
> > over
> > > South Korea, where stage-4 analysis is not available.
> > > Could you recommend other gridded precipitation observations
over
> > Southeast
> > > Asia other than TRMM?
> > >
> > > Thank you for your kindness.
> > >
> > > Best regards
> > > Yonghan Choi
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Tue Aug 23 16:39:12 2016
Oops, looks like I sent you two versions of that PNG file. I intended
it
to be only that global version.
John
On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Yonghan,
>
> Thanks for sending that sample data file. Yes, it is definitely a
flavor
> of NetCDF that the MET software is not expecting. Here's an option
for
> accessing that same TRMM data, following links from the MET website:
> http://www.dtcenter.org/met/users/downloads/observation_data.php
>
> # Pull binary 3-hourly TRMM data file
> wget ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> 201009/3B42.100921.00z.7.precipitation.bin
>
> # Pull Rscript from MET website
> wget
http://www.dtcenter.org/met/users/downloads/Rscripts/trmmbin2nc.R
>
> # Edit that Rscript by setting
> out_lat_ll = -50
> out_lon_ll = 0
> out_lat_ur = 50
> out_lon_ur = 359.75
>
> # Run the Rscript
> Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> 3B42.100921.00z.7.precipitation.nc
>
> # Plot the result
> met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
> 3B42.100921.00z.7.precipitation.ps 'name="APCP_03"; level="(*,*)";'
>
> I've attached a PNG version of resulting image.
>
> I see that the domain of your data is much smaller. You have two
choices
> here...
> (1) In that Rscript, you can choose different boundaries (i.e.
> out_lat/lon_ll/ur) to specify the tile of data to be selected.
> (2) As of version 5.1, MET includes support for regridding the data
it
> reads. You could keep TRMM on it's native domain and use the MET
tools to
> do the regridding. For example, the "regrid_data_plane" tool reads
a
> NetCDF file, regrids the data, and writes a NetCDF file.
Alternatively,
> the "regrid" section of the configuration files for the MET tools
may be
> used to do the regridding on the fly. For example, you could run
Grid-Stat
> to compare your model output to TRMM and say "regrid = { field =
FCST;
> ...}". That tells Grid-Stat to automatically regrid the TRMM
observations
> to your model domain.
>
> Hope that helps. Do this sounds like a workable path for you?
>
> Thanks,
> John
>
>
> On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>>
>> Dear John Halley Gotway:
>>
>> I uploaded my NetCDF-formatted TRMM data to your ftp server.
>>
>> Actually, I interpolated (to WRF grids) and made accumulation using
>> original TRMM data, and I converted HDF-formatted original TRMM
data to
>> NetCDF format.
>>
>> Thank you for your kindness.
>>
>> Best regards
>> Yonghan Choi
>>
>> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Yongchan,
>> >
>> > I see that you're having trouble using gridded TRMM data in MET.
I read
>> > that you've done some pre-processing of the TRMM data using NCL
prior to
>> > passing it to the MET tools.
>> >
>> > Can you please post a sample NetCDF file that you're passing to
>> pcp_combine
>> > to our anonymous FTP site, following the instructions listed
here:
>> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
>> >
>> > Please write once you've posted a sample file. I'll grab it and
run it
>> > through the debugger to identify what's happening.
>> >
>> > Thanks,
>> > John Halley Gotway
>> >
>> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT
<met_help at ucar.edu
>> >
>> > wrote:
>> >
>> > >
>> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
>> > > Transaction: Ticket created by cyh082 at gmail.com
>> > > Queue: met_help
>> > > Subject: Question on supported data format
>> > > Owner: Nobody
>> > > Requestors: cyh082 at gmail.com
>> > > Status: new
>> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614
>> >
>> > >
>> > >
>> > > Hello
>> > >
>> > > I would like to run grid-stat tool using WRF model forecasts
and TRMM
>> > > observations.
>> > >
>> > > I converted original HDF-format TRMM observation to NetCDF-
format,
>> and I
>> > > also interpolated original lat/lon grid to WRF Lambert
conformal grid
>> > using
>> > > NCL.
>> > >
>> > > And I tried to run pcp_combine before running grid-stat tool to
make
>> > > accumulated precipitation.
>> > >
>> > > However, I got errors like
>> > > "get_field() -> can't open data file ***.nc"
>> > >
>> > > Could you give me some advices on this problem?
>> > >
>> > > And... Actually, I selected TRMM observations to verify WRF
forecasts
>> > over
>> > > South Korea, where stage-4 analysis is not available.
>> > > Could you recommend other gridded precipitation observations
over
>> > Southeast
>> > > Asia other than TRMM?
>> > >
>> > > Thank you for your kindness.
>> > >
>> > > Best regards
>> > > Yonghan Choi
>> > >
>> > >
>> >
>> >
>>
>>
>
------------------------------------------------
Subject: Question on supported data format
From: Yonghan Choi
Time: Tue Aug 23 20:19:20 2016
Dear John Halley Gotway:
Yes, I can try the method you suggested after Yellowstone becomes
available.
I think your advices will be helpful to me.
Thank you very much.
Best regards
Yonghan Choi
On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Yonghan,
>
> Thanks for sending that sample data file. Yes, it is definitely a
flavor
> of NetCDF that the MET software is not expecting. Here's an option
for
> accessing that same TRMM data, following links from the MET website:
> http://www.dtcenter.org/met/users/downloads/observation_data.php
>
> # Pull binary 3-hourly TRMM data file
> wget
> ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> 201009/3B42.100921.00z.7.precipitation.bin
>
> # Pull Rscript from MET website
> wget
http://www.dtcenter.org/met/users/downloads/Rscripts/trmmbin2nc.R
>
> # Edit that Rscript by setting
> out_lat_ll = -50
> out_lon_ll = 0
> out_lat_ur = 50
> out_lon_ur = 359.75
>
> # Run the Rscript
> Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> 3B42.100921.00z.7.precipitation.nc
>
> # Plot the result
> met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
> 3B42.100921.00z.7.precipitation.ps 'name="APCP_03"; level="(*,*)";'
>
> I've attached a PNG version of resulting image.
>
> I see that the domain of your data is much smaller. You have two
choices
> here...
> (1) In that Rscript, you can choose different boundaries (i.e.
> out_lat/lon_ll/ur) to specify the tile of data to be selected.
> (2) As of version 5.1, MET includes support for regridding the data
it
> reads. You could keep TRMM on it's native domain and use the MET
tools to
> do the regridding. For example, the "regrid_data_plane" tool reads
a
> NetCDF file, regrids the data, and writes a NetCDF file.
Alternatively,
> the "regrid" section of the configuration files for the MET tools
may be
> used to do the regridding on the fly. For example, you could run
Grid-Stat
> to compare your model output to TRMM and say "regrid = { field =
FCST;
> ...}". That tells Grid-Stat to automatically regrid the TRMM
observations
> to your model domain.
>
> Hope that helps. Do this sounds like a workable path for you?
>
> Thanks,
> John
>
>
> On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> >
> > Dear John Halley Gotway:
> >
> > I uploaded my NetCDF-formatted TRMM data to your ftp server.
> >
> > Actually, I interpolated (to WRF grids) and made accumulation
using
> > original TRMM data, and I converted HDF-formatted original TRMM
data to
> > NetCDF format.
> >
> > Thank you for your kindness.
> >
> > Best regards
> > Yonghan Choi
> >
> > On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Yongchan,
> > >
> > > I see that you're having trouble using gridded TRMM data in MET.
I
> read
> > > that you've done some pre-processing of the TRMM data using NCL
prior
> to
> > > passing it to the MET tools.
> > >
> > > Can you please post a sample NetCDF file that you're passing to
> > pcp_combine
> > > to our anonymous FTP site, following the instructions listed
here:
> > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > Please write once you've posted a sample file. I'll grab it and
run it
> > > through the debugger to identify what's happening.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > > > Transaction: Ticket created by cyh082 at gmail.com
> > > > Queue: met_help
> > > > Subject: Question on supported data format
> > > > Owner: Nobody
> > > > Requestors: cyh082 at gmail.com
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=77614
> > >
> > > >
> > > >
> > > > Hello
> > > >
> > > > I would like to run grid-stat tool using WRF model forecasts
and TRMM
> > > > observations.
> > > >
> > > > I converted original HDF-format TRMM observation to NetCDF-
format,
> and
> > I
> > > > also interpolated original lat/lon grid to WRF Lambert
conformal grid
> > > using
> > > > NCL.
> > > >
> > > > And I tried to run pcp_combine before running grid-stat tool
to make
> > > > accumulated precipitation.
> > > >
> > > > However, I got errors like
> > > > "get_field() -> can't open data file ***.nc"
> > > >
> > > > Could you give me some advices on this problem?
> > > >
> > > > And... Actually, I selected TRMM observations to verify WRF
forecasts
> > > over
> > > > South Korea, where stage-4 analysis is not available.
> > > > Could you recommend other gridded precipitation observations
over
> > > Southeast
> > > > Asia other than TRMM?
> > > >
> > > > Thank you for your kindness.
> > > >
> > > > Best regards
> > > > Yonghan Choi
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: Yonghan Choi
Time: Fri Oct 07 01:31:03 2016
Dear John Halley Gotway:
Although you sent me the above email about 40 days ago, I hope you can
remember that email.
(I am sorry for the late reply.)
Actually, I successfully converted original binary TRMM file into
NetCDF
file using R script.
Let's assume that I obtained the below "4" 3-h accumulated
precipitation
files.
3B42.100921.00z.7.precipitation.nc
3B42.100921.03z.7.precipitation.nc
3B42.100921.06z.7.precipitation.nc
3B42.100921.09z.7.precipitation.nc
I would like to make 12-h accumulated (from 00 UTC to 12 UTC 21
September) precipitation file using "pcp_combine" tool.
I put all NetCDF files to "input" directory.
I executed "pcp_combine" by typing as follows.
pcp_combine -sum 00000000_000000 3 20100921_120000 12 sample_obs.nc
-pcpdir
./input
However, I got the error message as below.
sum_grib_files() -> Cannot find a file with a valid time of
20100921_120000
and accumulation time of 030000 matching the regular expression ".*"
Could you give me some advices?
Thank you for your kindness.
Best regards
Yonghan Choi
On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Oops, looks like I sent you two versions of that PNG file. I
intended it
> to be only that global version.
>
> John
>
> On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Yonghan,
> >
> > Thanks for sending that sample data file. Yes, it is definitely a
flavor
> > of NetCDF that the MET software is not expecting. Here's an
option for
> > accessing that same TRMM data, following links from the MET
website:
> >
http://www.dtcenter.org/met/users/downloads/observation_data.php
> >
> > # Pull binary 3-hourly TRMM data file
> > wget ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > 201009/3B42.100921.00z.7.precipitation.bin
> >
> > # Pull Rscript from MET website
> > wget
http://www.dtcenter.org/met/users/downloads/Rscripts/trmmbin2nc.R
> >
> > # Edit that Rscript by setting
> > out_lat_ll = -50
> > out_lon_ll = 0
> > out_lat_ur = 50
> > out_lon_ur = 359.75
> >
> > # Run the Rscript
> > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > 3B42.100921.00z.7.precipitation.nc
> >
> > # Plot the result
> > met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
> > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
level="(*,*)";'
> >
> > I've attached a PNG version of resulting image.
> >
> > I see that the domain of your data is much smaller. You have two
choices
> > here...
> > (1) In that Rscript, you can choose different boundaries (i.e.
> > out_lat/lon_ll/ur) to specify the tile of data to be selected.
> > (2) As of version 5.1, MET includes support for regridding the
data it
> > reads. You could keep TRMM on it's native domain and use the MET
tools
> to
> > do the regridding. For example, the "regrid_data_plane" tool
reads a
> > NetCDF file, regrids the data, and writes a NetCDF file.
Alternatively,
> > the "regrid" section of the configuration files for the MET tools
may be
> > used to do the regridding on the fly. For example, you could run
> Grid-Stat
> > to compare your model output to TRMM and say "regrid = { field =
FCST;
> > ...}". That tells Grid-Stat to automatically regrid the TRMM
> observations
> > to your model domain.
> >
> > Hope that helps. Do this sounds like a workable path for you?
> >
> > Thanks,
> > John
> >
> >
> > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> >>
> >> Dear John Halley Gotway:
> >>
> >> I uploaded my NetCDF-formatted TRMM data to your ftp server.
> >>
> >> Actually, I interpolated (to WRF grids) and made accumulation
using
> >> original TRMM data, and I converted HDF-formatted original TRMM
data to
> >> NetCDF format.
> >>
> >> Thank you for your kindness.
> >>
> >> Best regards
> >> Yonghan Choi
> >>
> >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Yongchan,
> >> >
> >> > I see that you're having trouble using gridded TRMM data in
MET. I
> read
> >> > that you've done some pre-processing of the TRMM data using NCL
prior
> to
> >> > passing it to the MET tools.
> >> >
> >> > Can you please post a sample NetCDF file that you're passing to
> >> pcp_combine
> >> > to our anonymous FTP site, following the instructions listed
here:
> >> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >> >
> >> > Please write once you've posted a sample file. I'll grab it
and run
> it
> >> > through the debugger to identify what's happening.
> >> >
> >> > Thanks,
> >> > John Halley Gotway
> >> >
> >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> met_help at ucar.edu
> >> >
> >> > wrote:
> >> >
> >> > >
> >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> >> > > Transaction: Ticket created by cyh082 at gmail.com
> >> > > Queue: met_help
> >> > > Subject: Question on supported data format
> >> > > Owner: Nobody
> >> > > Requestors: cyh082 at gmail.com
> >> > > Status: new
> >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=77614
> >> >
> >> > >
> >> > >
> >> > > Hello
> >> > >
> >> > > I would like to run grid-stat tool using WRF model forecasts
and
> TRMM
> >> > > observations.
> >> > >
> >> > > I converted original HDF-format TRMM observation to NetCDF-
format,
> >> and I
> >> > > also interpolated original lat/lon grid to WRF Lambert
conformal
> grid
> >> > using
> >> > > NCL.
> >> > >
> >> > > And I tried to run pcp_combine before running grid-stat tool
to make
> >> > > accumulated precipitation.
> >> > >
> >> > > However, I got errors like
> >> > > "get_field() -> can't open data file ***.nc"
> >> > >
> >> > > Could you give me some advices on this problem?
> >> > >
> >> > > And... Actually, I selected TRMM observations to verify WRF
> forecasts
> >> > over
> >> > > South Korea, where stage-4 analysis is not available.
> >> > > Could you recommend other gridded precipitation observations
over
> >> > Southeast
> >> > > Asia other than TRMM?
> >> > >
> >> > > Thank you for your kindness.
> >> > >
> >> > > Best regards
> >> > > Yonghan Choi
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Fri Oct 07 13:34:51 2016
Hello,
I see you're having trouble running pcp_combine on TRMM NetCDF files.
Rather than using the "sum" command, I'd suggest trying the "add"
option
for pcp_combine:
pcp_combine -add \
3B42.100921.00z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";' \
3B42.100921.03z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";' \
3B42.100921.06z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";' \
3B42.100921.09z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";' \
precip24.nc
Does that option work for you?
Thanks,
John
On Fri, Oct 7, 2016 at 1:31 AM, Yonghan Choi via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>
> Dear John Halley Gotway:
>
> Although you sent me the above email about 40 days ago, I hope you
can
> remember that email.
> (I am sorry for the late reply.)
>
> Actually, I successfully converted original binary TRMM file into
NetCDF
> file using R script.
>
> Let's assume that I obtained the below "4" 3-h accumulated
precipitation
> files.
> 3B42.100921.00z.7.precipitation.nc
> 3B42.100921.03z.7.precipitation.nc
> 3B42.100921.06z.7.precipitation.nc
> 3B42.100921.09z.7.precipitation.nc
>
> I would like to make 12-h accumulated (from 00 UTC to 12 UTC 21
> September) precipitation file using "pcp_combine" tool.
>
> I put all NetCDF files to "input" directory.
> I executed "pcp_combine" by typing as follows.
> pcp_combine -sum 00000000_000000 3 20100921_120000 12 sample_obs.nc
> -pcpdir
> ./input
>
> However, I got the error message as below.
> sum_grib_files() -> Cannot find a file with a valid time of
20100921_120000
> and accumulation time of 030000 matching the regular expression ".*"
>
> Could you give me some advices?
>
> Thank you for your kindness.
>
> Best regards
> Yonghan Choi
>
> On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Oops, looks like I sent you two versions of that PNG file. I
intended it
> > to be only that global version.
> >
> > John
> >
> > On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > Yonghan,
> > >
> > > Thanks for sending that sample data file. Yes, it is definitely
a
> flavor
> > > of NetCDF that the MET software is not expecting. Here's an
option for
> > > accessing that same TRMM data, following links from the MET
website:
> > >
http://www.dtcenter.org/met/users/downloads/observation_data.php
> > >
> > > # Pull binary 3-hourly TRMM data file
> > > wget ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > > 201009/3B42.100921.00z.7.precipitation.bin
> > >
> > > # Pull Rscript from MET website
> > > wget
http://www.dtcenter.org/met/users/downloads/Rscripts/trmmbin2nc.R
> > >
> > > # Edit that Rscript by setting
> > > out_lat_ll = -50
> > > out_lon_ll = 0
> > > out_lat_ur = 50
> > > out_lon_ur = 359.75
> > >
> > > # Run the Rscript
> > > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > > 3B42.100921.00z.7.precipitation.nc
> > >
> > > # Plot the result
> > > met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
> > > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
level="(*,*)";'
> > >
> > > I've attached a PNG version of resulting image.
> > >
> > > I see that the domain of your data is much smaller. You have
two
> choices
> > > here...
> > > (1) In that Rscript, you can choose different boundaries (i.e.
> > > out_lat/lon_ll/ur) to specify the tile of data to be selected.
> > > (2) As of version 5.1, MET includes support for regridding the
data it
> > > reads. You could keep TRMM on it's native domain and use the
MET tools
> > to
> > > do the regridding. For example, the "regrid_data_plane" tool
reads a
> > > NetCDF file, regrids the data, and writes a NetCDF file.
> Alternatively,
> > > the "regrid" section of the configuration files for the MET
tools may
> be
> > > used to do the regridding on the fly. For example, you could
run
> > Grid-Stat
> > > to compare your model output to TRMM and say "regrid = { field =
FCST;
> > > ...}". That tells Grid-Stat to automatically regrid the TRMM
> > observations
> > > to your model domain.
> > >
> > > Hope that helps. Do this sounds like a workable path for you?
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > >>
> > >> Dear John Halley Gotway:
> > >>
> > >> I uploaded my NetCDF-formatted TRMM data to your ftp server.
> > >>
> > >> Actually, I interpolated (to WRF grids) and made accumulation
using
> > >> original TRMM data, and I converted HDF-formatted original TRMM
data
> to
> > >> NetCDF format.
> > >>
> > >> Thank you for your kindness.
> > >>
> > >> Best regards
> > >> Yonghan Choi
> > >>
> > >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> > Yongchan,
> > >> >
> > >> > I see that you're having trouble using gridded TRMM data in
MET. I
> > read
> > >> > that you've done some pre-processing of the TRMM data using
NCL
> prior
> > to
> > >> > passing it to the MET tools.
> > >> >
> > >> > Can you please post a sample NetCDF file that you're passing
to
> > >> pcp_combine
> > >> > to our anonymous FTP site, following the instructions listed
here:
> > >> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >> >
> > >> > Please write once you've posted a sample file. I'll grab it
and run
> > it
> > >> > through the debugger to identify what's happening.
> > >> >
> > >> > Thanks,
> > >> > John Halley Gotway
> > >> >
> > >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> > met_help at ucar.edu
> > >> >
> > >> > wrote:
> > >> >
> > >> > >
> > >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > >> > > Transaction: Ticket created by cyh082 at gmail.com
> > >> > > Queue: met_help
> > >> > > Subject: Question on supported data format
> > >> > > Owner: Nobody
> > >> > > Requestors: cyh082 at gmail.com
> > >> > > Status: new
> > >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > Ticket/Display.html?id=77614
> > >> >
> > >> > >
> > >> > >
> > >> > > Hello
> > >> > >
> > >> > > I would like to run grid-stat tool using WRF model
forecasts and
> > TRMM
> > >> > > observations.
> > >> > >
> > >> > > I converted original HDF-format TRMM observation to NetCDF-
format,
> > >> and I
> > >> > > also interpolated original lat/lon grid to WRF Lambert
conformal
> > grid
> > >> > using
> > >> > > NCL.
> > >> > >
> > >> > > And I tried to run pcp_combine before running grid-stat
tool to
> make
> > >> > > accumulated precipitation.
> > >> > >
> > >> > > However, I got errors like
> > >> > > "get_field() -> can't open data file ***.nc"
> > >> > >
> > >> > > Could you give me some advices on this problem?
> > >> > >
> > >> > > And... Actually, I selected TRMM observations to verify WRF
> > forecasts
> > >> > over
> > >> > > South Korea, where stage-4 analysis is not available.
> > >> > > Could you recommend other gridded precipitation
observations over
> > >> > Southeast
> > >> > > Asia other than TRMM?
> > >> > >
> > >> > > Thank you for your kindness.
> > >> > >
> > >> > > Best regards
> > >> > > Yonghan Choi
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: Yonghan Choi
Time: Sun Oct 09 03:43:21 2016
Dear John Halley Gotway:
Yes, the option you suggested worked!
I have some questions on the R script.
1. One of attributes for variable, APCP_03 is "init_time_ut".
What is the difference between "init_time" and "init_time_ut"?
2. "FillValue" attribute for variable APCP_03 is set to "-9999".
In MET program, is fill value or missing value assigned to "-9999"?
I found that fill value for TRMM observations is -9999.9.
And I tried to run UPP and pcp_combine using WRF forecasts.
First. I ran UPP using 13 WRF outputs (wrfout_d01_2010-09-21_00:00:00
~
wrfout_d01_2010-09-21_12:00:00).
I included "ACM TOTAL PRECIP" variable in wrf_cntrl.parm file.
After running UPP, I obtained 13 outputs (WRFPRS_d01.00 ~
WRFPRS_d01.12).
Then, I tried to run pcp_combine by typing as belows.
pcp_combine -sum 20100921_000000 1 20100921_120000 12 sample_fcst.nc
-pcpdir ./sample
I got error messages.
DEBUG 1: [1] Reading input file: ./sample/WRFPRS_d01.12
WARNING:
WARNING: MetGrib1DataFile::data_plane() -> No exact match found for
VarInfo
"APCP/A010000" in GRIB file "./sample/WRFPRS_d01.12".
WARNING:
ERROR :
ERROR : get_field() -> can't get data plane from file
"./sample/WRFPRS_d01.12"
ERROR :
Could you give me some advices on the above problem?
Thank you for your kindness.
Best regards
Yonghan Choi
On Fri, Oct 7, 2016 at 1:34 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Hello,
>
> I see you're having trouble running pcp_combine on TRMM NetCDF
files.
> Rather than using the "sum" command, I'd suggest trying the "add"
option
> for pcp_combine:
>
> pcp_combine -add \
> 3B42.100921.00z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";'
\
> 3B42.100921.03z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";'
\
> 3B42.100921.06z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";'
\
> 3B42.100921.09z.7.precipitation.nc 'name="APCP_03"; level="(*,*)";'
\
> precip24.nc
>
> Does that option work for you?
>
> Thanks,
> John
>
> On Fri, Oct 7, 2016 at 1:31 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> >
> > Dear John Halley Gotway:
> >
> > Although you sent me the above email about 40 days ago, I hope you
can
> > remember that email.
> > (I am sorry for the late reply.)
> >
> > Actually, I successfully converted original binary TRMM file into
NetCDF
> > file using R script.
> >
> > Let's assume that I obtained the below "4" 3-h accumulated
precipitation
> > files.
> > 3B42.100921.00z.7.precipitation.nc
> > 3B42.100921.03z.7.precipitation.nc
> > 3B42.100921.06z.7.precipitation.nc
> > 3B42.100921.09z.7.precipitation.nc
> >
> > I would like to make 12-h accumulated (from 00 UTC to 12 UTC 21
> > September) precipitation file using "pcp_combine" tool.
> >
> > I put all NetCDF files to "input" directory.
> > I executed "pcp_combine" by typing as follows.
> > pcp_combine -sum 00000000_000000 3 20100921_120000 12
sample_obs.nc
> > -pcpdir
> > ./input
> >
> > However, I got the error message as below.
> > sum_grib_files() -> Cannot find a file with a valid time of
> 20100921_120000
> > and accumulation time of 030000 matching the regular expression
".*"
> >
> > Could you give me some advices?
> >
> > Thank you for your kindness.
> >
> > Best regards
> > Yonghan Choi
> >
> > On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Oops, looks like I sent you two versions of that PNG file. I
intended
> it
> > > to be only that global version.
> > >
> > > John
> > >
> > > On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > >
> > > > Yonghan,
> > > >
> > > > Thanks for sending that sample data file. Yes, it is
definitely a
> > flavor
> > > > of NetCDF that the MET software is not expecting. Here's an
option
> for
> > > > accessing that same TRMM data, following links from the MET
website:
> > > >
http://www.dtcenter.org/met/users/downloads/observation_data.php
> > > >
> > > > # Pull binary 3-hourly TRMM data file
> > > > wget ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > > > 201009/3B42.100921.00z.7.precipitation.bin
> > > >
> > > > # Pull Rscript from MET website
> > > > wget http://www.dtcenter.org/met/users/downloads/Rscripts/
> trmmbin2nc.R
> > > >
> > > > # Edit that Rscript by setting
> > > > out_lat_ll = -50
> > > > out_lon_ll = 0
> > > > out_lat_ur = 50
> > > > out_lon_ur = 359.75
> > > >
> > > > # Run the Rscript
> > > > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > > > 3B42.100921.00z.7.precipitation.nc
> > > >
> > > > # Plot the result
> > > > met-5.2/bin/plot_data_plane 3B42.100921.00z.7.precipitation.nc
> > > > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
level="(*,*)";'
> > > >
> > > > I've attached a PNG version of resulting image.
> > > >
> > > > I see that the domain of your data is much smaller. You have
two
> > choices
> > > > here...
> > > > (1) In that Rscript, you can choose different boundaries (i.e.
> > > > out_lat/lon_ll/ur) to specify the tile of data to be selected.
> > > > (2) As of version 5.1, MET includes support for regridding the
data
> it
> > > > reads. You could keep TRMM on it's native domain and use the
MET
> tools
> > > to
> > > > do the regridding. For example, the "regrid_data_plane" tool
reads a
> > > > NetCDF file, regrids the data, and writes a NetCDF file.
> > Alternatively,
> > > > the "regrid" section of the configuration files for the MET
tools may
> > be
> > > > used to do the regridding on the fly. For example, you could
run
> > > Grid-Stat
> > > > to compare your model output to TRMM and say "regrid = { field
=
> FCST;
> > > > ...}". That tells Grid-Stat to automatically regrid the TRMM
> > > observations
> > > > to your model domain.
> > > >
> > > > Hope that helps. Do this sounds like a workable path for you?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614
>
> > > >>
> > > >> Dear John Halley Gotway:
> > > >>
> > > >> I uploaded my NetCDF-formatted TRMM data to your ftp server.
> > > >>
> > > >> Actually, I interpolated (to WRF grids) and made accumulation
using
> > > >> original TRMM data, and I converted HDF-formatted original
TRMM data
> > to
> > > >> NetCDF format.
> > > >>
> > > >> Thank you for your kindness.
> > > >>
> > > >> Best regards
> > > >> Yonghan Choi
> > > >>
> > > >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> > Yongchan,
> > > >> >
> > > >> > I see that you're having trouble using gridded TRMM data in
MET.
> I
> > > read
> > > >> > that you've done some pre-processing of the TRMM data using
NCL
> > prior
> > > to
> > > >> > passing it to the MET tools.
> > > >> >
> > > >> > Can you please post a sample NetCDF file that you're
passing to
> > > >> pcp_combine
> > > >> > to our anonymous FTP site, following the instructions
listed here:
> > > >> >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >> >
> > > >> > Please write once you've posted a sample file. I'll grab
it and
> run
> > > it
> > > >> > through the debugger to identify what's happening.
> > > >> >
> > > >> > Thanks,
> > > >> > John Halley Gotway
> > > >> >
> > > >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> > > met_help at ucar.edu
> > > >> >
> > > >> > wrote:
> > > >> >
> > > >> > >
> > > >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > > >> > > Transaction: Ticket created by cyh082 at gmail.com
> > > >> > > Queue: met_help
> > > >> > > Subject: Question on supported data format
> > > >> > > Owner: Nobody
> > > >> > > Requestors: cyh082 at gmail.com
> > > >> > > Status: new
> > > >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > Ticket/Display.html?id=77614
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > > Hello
> > > >> > >
> > > >> > > I would like to run grid-stat tool using WRF model
forecasts and
> > > TRMM
> > > >> > > observations.
> > > >> > >
> > > >> > > I converted original HDF-format TRMM observation to
> NetCDF-format,
> > > >> and I
> > > >> > > also interpolated original lat/lon grid to WRF Lambert
conformal
> > > grid
> > > >> > using
> > > >> > > NCL.
> > > >> > >
> > > >> > > And I tried to run pcp_combine before running grid-stat
tool to
> > make
> > > >> > > accumulated precipitation.
> > > >> > >
> > > >> > > However, I got errors like
> > > >> > > "get_field() -> can't open data file ***.nc"
> > > >> > >
> > > >> > > Could you give me some advices on this problem?
> > > >> > >
> > > >> > > And... Actually, I selected TRMM observations to verify
WRF
> > > forecasts
> > > >> > over
> > > >> > > South Korea, where stage-4 analysis is not available.
> > > >> > > Could you recommend other gridded precipitation
observations
> over
> > > >> > Southeast
> > > >> > > Asia other than TRMM?
> > > >> > >
> > > >> > > Thank you for your kindness.
> > > >> > >
> > > >> > > Best regards
> > > >> > > Yonghan Choi
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Mon Oct 10 09:30:40 2016
Yongchan,
(1) What's the difference between init_time and init_time_ut?
The init_time and init_time_ut attributes indicate the exact same
timing
information, just formatted differently. Without the "ut", it's just
a
string in YYYYMMDD_HHMMSS format. The "ut" means unixtime which is a
count
of the number of seconds since Jan 1, 1970. It's a very convenient
way of
storing/processing timing information. In fact, the MET code only
reads
the "init_time_ut" attribute, not the "init_time" one. We include
"init_time" in the pcp_combine output for ease of the user. The same
is
true of "valid_time" and "valid_time_ut".
If we had it to do over again, we should probably have skipped the
unixtime
attributes because of the confusion they cause. Instead, we could
have
just parsed the YYYYMMDD_HHMMSS strings in the code. However in MET
version 6.0, we'll be switching to writing NetCDF version 4.0 output
files
and at the same time switching to using the NetCDF climate forecast
(CF)
convention.
(2) MET uses the value of "-9999" to indicate bad data. We don't
really
distinguish between fill value and missing value... it's all stored
internally as -9999. While MET uses -9999 internally, the input
NetCDF
files can use any value they please. When reading data from those
files,
any occurrences of the FillValue attribute will be stored as -9999.
(3) Lastly, I see you're having trouble accessing precip from the GRIB
output files of UPP. I would suggest using the "wgrib" utility to
list the
contents of those GRIB files. Try:
wgrib sample/WRFPRS_d01.12 | grep APCP
Accumulated precipitation is somewhat of a special case. State
variables,
such as temperature, are written out as an instantaneous value.
Precipitation, on the other hand, must be defined over some
accumulation
time interval. Here's my guess as to what might be happening...
If you're running WRF-ARW, the default accumulation scheme is to
accumulate
precipitation over the entire model integration. You can configure it
to
do otherwise, but runtime accumulation is the default. So the 6-hour
forecast file contains 6 hours of precip. The 12-hour file has 12
hours of
precip. And so on. But you can easily check this using wgrib command
listed above.
If that's the case, instead of the "sum" or "add" commands, you'll use
the
pcp_combine "subtract" option. For example, if you want the 6 hour
accumulation between the 6 and 12 hour forecasts, just subtract
them...
like this:
pcp_combine -subtract sample/WRFPRS_d01.12 12 sample/WRFPRS_d01.06 6
apcp_06_f06_to_f12.nc
Hope that helps.
Thanks,
John
On Sun, Oct 9, 2016 at 3:43 AM, Yonghan Choi via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>
> Dear John Halley Gotway:
>
> Yes, the option you suggested worked!
>
> I have some questions on the R script.
> 1. One of attributes for variable, APCP_03 is "init_time_ut".
> What is the difference between "init_time" and "init_time_ut"?
>
> 2. "FillValue" attribute for variable APCP_03 is set to "-9999".
> In MET program, is fill value or missing value assigned to "-9999"?
> I found that fill value for TRMM observations is -9999.9.
>
> And I tried to run UPP and pcp_combine using WRF forecasts.
>
> First. I ran UPP using 13 WRF outputs (wrfout_d01_2010-09-
21_00:00:00 ~
> wrfout_d01_2010-09-21_12:00:00).
> I included "ACM TOTAL PRECIP" variable in wrf_cntrl.parm file.
> After running UPP, I obtained 13 outputs (WRFPRS_d01.00 ~
WRFPRS_d01.12).
>
> Then, I tried to run pcp_combine by typing as belows.
> pcp_combine -sum 20100921_000000 1 20100921_120000 12 sample_fcst.nc
> -pcpdir ./sample
>
> I got error messages.
> DEBUG 1: [1] Reading input file: ./sample/WRFPRS_d01.12
> WARNING:
> WARNING: MetGrib1DataFile::data_plane() -> No exact match found for
VarInfo
> "APCP/A010000" in GRIB file "./sample/WRFPRS_d01.12".
> WARNING:
> ERROR :
> ERROR : get_field() -> can't get data plane from file
> "./sample/WRFPRS_d01.12"
> ERROR :
>
> Could you give me some advices on the above problem?
>
> Thank you for your kindness.
>
> Best regards
> Yonghan Choi
>
> On Fri, Oct 7, 2016 at 1:34 PM, John Halley Gotway via RT <
> met_help at ucar.edu
> > wrote:
>
> > Hello,
> >
> > I see you're having trouble running pcp_combine on TRMM NetCDF
files.
> > Rather than using the "sum" command, I'd suggest trying the "add"
option
> > for pcp_combine:
> >
> > pcp_combine -add \
> > 3B42.100921.00z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > 3B42.100921.03z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > 3B42.100921.06z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > 3B42.100921.09z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > precip24.nc
> >
> > Does that option work for you?
> >
> > Thanks,
> > John
> >
> > On Fri, Oct 7, 2016 at 1:31 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > >
> > > Dear John Halley Gotway:
> > >
> > > Although you sent me the above email about 40 days ago, I hope
you can
> > > remember that email.
> > > (I am sorry for the late reply.)
> > >
> > > Actually, I successfully converted original binary TRMM file
into
> NetCDF
> > > file using R script.
> > >
> > > Let's assume that I obtained the below "4" 3-h accumulated
> precipitation
> > > files.
> > > 3B42.100921.00z.7.precipitation.nc
> > > 3B42.100921.03z.7.precipitation.nc
> > > 3B42.100921.06z.7.precipitation.nc
> > > 3B42.100921.09z.7.precipitation.nc
> > >
> > > I would like to make 12-h accumulated (from 00 UTC to 12 UTC 21
> > > September) precipitation file using "pcp_combine" tool.
> > >
> > > I put all NetCDF files to "input" directory.
> > > I executed "pcp_combine" by typing as follows.
> > > pcp_combine -sum 00000000_000000 3 20100921_120000 12
sample_obs.nc
> > > -pcpdir
> > > ./input
> > >
> > > However, I got the error message as below.
> > > sum_grib_files() -> Cannot find a file with a valid time of
> > 20100921_120000
> > > and accumulation time of 030000 matching the regular expression
".*"
> > >
> > > Could you give me some advices?
> > >
> > > Thank you for your kindness.
> > >
> > > Best regards
> > > Yonghan Choi
> > >
> > > On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Oops, looks like I sent you two versions of that PNG file. I
> intended
> > it
> > > > to be only that global version.
> > > >
> > > > John
> > > >
> > > > On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway
<johnhg at ucar.edu
> >
> > > > wrote:
> > > >
> > > > > Yonghan,
> > > > >
> > > > > Thanks for sending that sample data file. Yes, it is
definitely a
> > > flavor
> > > > > of NetCDF that the MET software is not expecting. Here's an
option
> > for
> > > > > accessing that same TRMM data, following links from the MET
> website:
> > > > > http://www.dtcenter.org/met/users/downloads/observation_
> data.php
> > > > >
> > > > > # Pull binary 3-hourly TRMM data file
> > > > > wget ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > > > > 201009/3B42.100921.00z.7.precipitation.bin
> > > > >
> > > > > # Pull Rscript from MET website
> > > > > wget http://www.dtcenter.org/met/users/downloads/Rscripts/
> > trmmbin2nc.R
> > > > >
> > > > > # Edit that Rscript by setting
> > > > > out_lat_ll = -50
> > > > > out_lon_ll = 0
> > > > > out_lat_ur = 50
> > > > > out_lon_ur = 359.75
> > > > >
> > > > > # Run the Rscript
> > > > > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > > > > 3B42.100921.00z.7.precipitation.nc
> > > > >
> > > > > # Plot the result
> > > > > met-5.2/bin/plot_data_plane
3B42.100921.00z.7.precipitation.nc
> > > > > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
> level="(*,*)";'
> > > > >
> > > > > I've attached a PNG version of resulting image.
> > > > >
> > > > > I see that the domain of your data is much smaller. You
have two
> > > choices
> > > > > here...
> > > > > (1) In that Rscript, you can choose different boundaries
(i.e.
> > > > > out_lat/lon_ll/ur) to specify the tile of data to be
selected.
> > > > > (2) As of version 5.1, MET includes support for regridding
the data
> > it
> > > > > reads. You could keep TRMM on it's native domain and use
the MET
> > tools
> > > > to
> > > > > do the regridding. For example, the "regrid_data_plane"
tool
> reads a
> > > > > NetCDF file, regrids the data, and writes a NetCDF file.
> > > Alternatively,
> > > > > the "regrid" section of the configuration files for the MET
tools
> may
> > > be
> > > > > used to do the regridding on the fly. For example, you
could run
> > > > Grid-Stat
> > > > > to compare your model output to TRMM and say "regrid = {
field =
> > FCST;
> > > > > ...}". That tells Grid-Stat to automatically regrid the
TRMM
> > > > observations
> > > > > to your model domain.
> > > > >
> > > > > Hope that helps. Do this sounds like a workable path for
you?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > > > >>
> > > > >> Dear John Halley Gotway:
> > > > >>
> > > > >> I uploaded my NetCDF-formatted TRMM data to your ftp
server.
> > > > >>
> > > > >> Actually, I interpolated (to WRF grids) and made
accumulation
> using
> > > > >> original TRMM data, and I converted HDF-formatted original
TRMM
> data
> > > to
> > > > >> NetCDF format.
> > > > >>
> > > > >> Thank you for your kindness.
> > > > >>
> > > > >> Best regards
> > > > >> Yonghan Choi
> > > > >>
> > > > >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> > Yongchan,
> > > > >> >
> > > > >> > I see that you're having trouble using gridded TRMM data
in MET.
> > I
> > > > read
> > > > >> > that you've done some pre-processing of the TRMM data
using NCL
> > > prior
> > > > to
> > > > >> > passing it to the MET tools.
> > > > >> >
> > > > >> > Can you please post a sample NetCDF file that you're
passing to
> > > > >> pcp_combine
> > > > >> > to our anonymous FTP site, following the instructions
listed
> here:
> > > > >> >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > >> >
> > > > >> > Please write once you've posted a sample file. I'll grab
it and
> > run
> > > > it
> > > > >> > through the debugger to identify what's happening.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > John Halley Gotway
> > > > >> >
> > > > >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> > > > met_help at ucar.edu
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted upon.
> > > > >> > > Transaction: Ticket created by cyh082 at gmail.com
> > > > >> > > Queue: met_help
> > > > >> > > Subject: Question on supported data format
> > > > >> > > Owner: Nobody
> > > > >> > > Requestors: cyh082 at gmail.com
> > > > >> > > Status: new
> > > > >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > Ticket/Display.html?id=77614
> > > > >> >
> > > > >> > >
> > > > >> > >
> > > > >> > > Hello
> > > > >> > >
> > > > >> > > I would like to run grid-stat tool using WRF model
forecasts
> and
> > > > TRMM
> > > > >> > > observations.
> > > > >> > >
> > > > >> > > I converted original HDF-format TRMM observation to
> > NetCDF-format,
> > > > >> and I
> > > > >> > > also interpolated original lat/lon grid to WRF Lambert
> conformal
> > > > grid
> > > > >> > using
> > > > >> > > NCL.
> > > > >> > >
> > > > >> > > And I tried to run pcp_combine before running grid-stat
tool
> to
> > > make
> > > > >> > > accumulated precipitation.
> > > > >> > >
> > > > >> > > However, I got errors like
> > > > >> > > "get_field() -> can't open data file ***.nc"
> > > > >> > >
> > > > >> > > Could you give me some advices on this problem?
> > > > >> > >
> > > > >> > > And... Actually, I selected TRMM observations to verify
WRF
> > > > forecasts
> > > > >> > over
> > > > >> > > South Korea, where stage-4 analysis is not available.
> > > > >> > > Could you recommend other gridded precipitation
observations
> > over
> > > > >> > Southeast
> > > > >> > > Asia other than TRMM?
> > > > >> > >
> > > > >> > > Thank you for your kindness.
> > > > >> > >
> > > > >> > > Best regards
> > > > >> > > Yonghan Choi
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: Yonghan Choi
Time: Wed Oct 12 01:52:19 2016
Dear John Halley Gotway:
I really appreciate your detailed explanations on "init_time_ut" and
"fill/missing value".
I successfully ran "pcp_combine" for WRF outputs using the method you
suggested.
I think that I can run "grid_stat" now.
Thank you for your kindness.
Best regards
Yonghan Choi
On Mon, Oct 10, 2016 at 9:30 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Yongchan,
>
> (1) What's the difference between init_time and init_time_ut?
>
> The init_time and init_time_ut attributes indicate the exact same
timing
> information, just formatted differently. Without the "ut", it's
just a
> string in YYYYMMDD_HHMMSS format. The "ut" means unixtime which is
a count
> of the number of seconds since Jan 1, 1970. It's a very convenient
way of
> storing/processing timing information. In fact, the MET code only
reads
> the "init_time_ut" attribute, not the "init_time" one. We include
> "init_time" in the pcp_combine output for ease of the user. The
same is
> true of "valid_time" and "valid_time_ut".
>
> If we had it to do over again, we should probably have skipped the
unixtime
> attributes because of the confusion they cause. Instead, we could
have
> just parsed the YYYYMMDD_HHMMSS strings in the code. However in MET
> version 6.0, we'll be switching to writing NetCDF version 4.0 output
files
> and at the same time switching to using the NetCDF climate forecast
(CF)
> convention.
>
> (2) MET uses the value of "-9999" to indicate bad data. We don't
really
> distinguish between fill value and missing value... it's all stored
> internally as -9999. While MET uses -9999 internally, the input
NetCDF
> files can use any value they please. When reading data from those
files,
> any occurrences of the FillValue attribute will be stored as -9999.
>
> (3) Lastly, I see you're having trouble accessing precip from the
GRIB
> output files of UPP. I would suggest using the "wgrib" utility to
list the
> contents of those GRIB files. Try:
>
> wgrib sample/WRFPRS_d01.12 | grep APCP
>
> Accumulated precipitation is somewhat of a special case. State
variables,
> such as temperature, are written out as an instantaneous value.
> Precipitation, on the other hand, must be defined over some
accumulation
> time interval. Here's my guess as to what might be happening...
>
> If you're running WRF-ARW, the default accumulation scheme is to
accumulate
> precipitation over the entire model integration. You can configure
it to
> do otherwise, but runtime accumulation is the default. So the 6-
hour
> forecast file contains 6 hours of precip. The 12-hour file has 12
hours of
> precip. And so on. But you can easily check this using wgrib
command
> listed above.
>
> If that's the case, instead of the "sum" or "add" commands, you'll
use the
> pcp_combine "subtract" option. For example, if you want the 6 hour
> accumulation between the 6 and 12 hour forecasts, just subtract
them...
> like this:
>
> pcp_combine -subtract sample/WRFPRS_d01.12 12 sample/WRFPRS_d01.06 6
> apcp_06_f06_to_f12.nc
>
> Hope that helps.
>
> Thanks,
> John
>
> On Sun, Oct 9, 2016 at 3:43 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> >
> > Dear John Halley Gotway:
> >
> > Yes, the option you suggested worked!
> >
> > I have some questions on the R script.
> > 1. One of attributes for variable, APCP_03 is "init_time_ut".
> > What is the difference between "init_time" and "init_time_ut"?
> >
> > 2. "FillValue" attribute for variable APCP_03 is set to "-9999".
> > In MET program, is fill value or missing value assigned to "-
9999"?
> > I found that fill value for TRMM observations is -9999.9.
> >
> > And I tried to run UPP and pcp_combine using WRF forecasts.
> >
> > First. I ran UPP using 13 WRF outputs (wrfout_d01_2010-09-
21_00:00:00 ~
> > wrfout_d01_2010-09-21_12:00:00).
> > I included "ACM TOTAL PRECIP" variable in wrf_cntrl.parm file.
> > After running UPP, I obtained 13 outputs (WRFPRS_d01.00 ~
WRFPRS_d01.12).
> >
> > Then, I tried to run pcp_combine by typing as belows.
> > pcp_combine -sum 20100921_000000 1 20100921_120000 12
sample_fcst.nc
> > -pcpdir ./sample
> >
> > I got error messages.
> > DEBUG 1: [1] Reading input file: ./sample/WRFPRS_d01.12
> > WARNING:
> > WARNING: MetGrib1DataFile::data_plane() -> No exact match found
for
> VarInfo
> > "APCP/A010000" in GRIB file "./sample/WRFPRS_d01.12".
> > WARNING:
> > ERROR :
> > ERROR : get_field() -> can't get data plane from file
> > "./sample/WRFPRS_d01.12"
> > ERROR :
> >
> > Could you give me some advices on the above problem?
> >
> > Thank you for your kindness.
> >
> > Best regards
> > Yonghan Choi
> >
> > On Fri, Oct 7, 2016 at 1:34 PM, John Halley Gotway via RT <
> > met_help at ucar.edu
> > > wrote:
> >
> > > Hello,
> > >
> > > I see you're having trouble running pcp_combine on TRMM NetCDF
files.
> > > Rather than using the "sum" command, I'd suggest trying the
"add"
> option
> > > for pcp_combine:
> > >
> > > pcp_combine -add \
> > > 3B42.100921.00z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > > 3B42.100921.03z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > > 3B42.100921.06z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > > 3B42.100921.09z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";' \
> > > precip24.nc
> > >
> > > Does that option work for you?
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Oct 7, 2016 at 1:31 AM, Yonghan Choi via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614
>
> > > >
> > > > Dear John Halley Gotway:
> > > >
> > > > Although you sent me the above email about 40 days ago, I hope
you
> can
> > > > remember that email.
> > > > (I am sorry for the late reply.)
> > > >
> > > > Actually, I successfully converted original binary TRMM file
into
> > NetCDF
> > > > file using R script.
> > > >
> > > > Let's assume that I obtained the below "4" 3-h accumulated
> > precipitation
> > > > files.
> > > > 3B42.100921.00z.7.precipitation.nc
> > > > 3B42.100921.03z.7.precipitation.nc
> > > > 3B42.100921.06z.7.precipitation.nc
> > > > 3B42.100921.09z.7.precipitation.nc
> > > >
> > > > I would like to make 12-h accumulated (from 00 UTC to 12 UTC
21
> > > > September) precipitation file using "pcp_combine" tool.
> > > >
> > > > I put all NetCDF files to "input" directory.
> > > > I executed "pcp_combine" by typing as follows.
> > > > pcp_combine -sum 00000000_000000 3 20100921_120000 12
sample_obs.nc
> > > > -pcpdir
> > > > ./input
> > > >
> > > > However, I got the error message as below.
> > > > sum_grib_files() -> Cannot find a file with a valid time of
> > > 20100921_120000
> > > > and accumulation time of 030000 matching the regular
expression ".*"
> > > >
> > > > Could you give me some advices?
> > > >
> > > > Thank you for your kindness.
> > > >
> > > > Best regards
> > > > Yonghan Choi
> > > >
> > > > On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Oops, looks like I sent you two versions of that PNG file.
I
> > intended
> > > it
> > > > > to be only that global version.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway <
> johnhg at ucar.edu
> > >
> > > > > wrote:
> > > > >
> > > > > > Yonghan,
> > > > > >
> > > > > > Thanks for sending that sample data file. Yes, it is
definitely
> a
> > > > flavor
> > > > > > of NetCDF that the MET software is not expecting. Here's
an
> option
> > > for
> > > > > > accessing that same TRMM data, following links from the
MET
> > website:
> > > > > >
http://www.dtcenter.org/met/users/downloads/observation_
> > data.php
> > > > > >
> > > > > > # Pull binary 3-hourly TRMM data file
> > > > > > wget
ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > > > > > 201009/3B42.100921.00z.7.precipitation.bin
> > > > > >
> > > > > > # Pull Rscript from MET website
> > > > > > wget http://www.dtcenter.org/met/users/downloads/Rscripts/
> > > trmmbin2nc.R
> > > > > >
> > > > > > # Edit that Rscript by setting
> > > > > > out_lat_ll = -50
> > > > > > out_lon_ll = 0
> > > > > > out_lat_ur = 50
> > > > > > out_lon_ur = 359.75
> > > > > >
> > > > > > # Run the Rscript
> > > > > > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > > > > > 3B42.100921.00z.7.precipitation.nc
> > > > > >
> > > > > > # Plot the result
> > > > > > met-5.2/bin/plot_data_plane
3B42.100921.00z.7.precipitation.nc
> > > > > > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
> > level="(*,*)";'
> > > > > >
> > > > > > I've attached a PNG version of resulting image.
> > > > > >
> > > > > > I see that the domain of your data is much smaller. You
have two
> > > > choices
> > > > > > here...
> > > > > > (1) In that Rscript, you can choose different boundaries
(i.e.
> > > > > > out_lat/lon_ll/ur) to specify the tile of data to be
selected.
> > > > > > (2) As of version 5.1, MET includes support for regridding
the
> data
> > > it
> > > > > > reads. You could keep TRMM on it's native domain and use
the MET
> > > tools
> > > > > to
> > > > > > do the regridding. For example, the "regrid_data_plane"
tool
> > reads a
> > > > > > NetCDF file, regrids the data, and writes a NetCDF file.
> > > > Alternatively,
> > > > > > the "regrid" section of the configuration files for the
MET tools
> > may
> > > > be
> > > > > > used to do the regridding on the fly. For example, you
could run
> > > > > Grid-Stat
> > > > > > to compare your model output to TRMM and say "regrid = {
field =
> > > FCST;
> > > > > > ...}". That tells Grid-Stat to automatically regrid the
TRMM
> > > > > observations
> > > > > > to your model domain.
> > > > > >
> > > > > > Hope that helps. Do this sounds like a workable path for
you?
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >>
> > > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > > > > >>
> > > > > >> Dear John Halley Gotway:
> > > > > >>
> > > > > >> I uploaded my NetCDF-formatted TRMM data to your ftp
server.
> > > > > >>
> > > > > >> Actually, I interpolated (to WRF grids) and made
accumulation
> > using
> > > > > >> original TRMM data, and I converted HDF-formatted
original TRMM
> > data
> > > > to
> > > > > >> NetCDF format.
> > > > > >>
> > > > > >> Thank you for your kindness.
> > > > > >>
> > > > > >> Best regards
> > > > > >> Yonghan Choi
> > > > > >>
> > > > > >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via
RT <
> > > > > >> met_help at ucar.edu> wrote:
> > > > > >>
> > > > > >> > Yongchan,
> > > > > >> >
> > > > > >> > I see that you're having trouble using gridded TRMM
data in
> MET.
> > > I
> > > > > read
> > > > > >> > that you've done some pre-processing of the TRMM data
using
> NCL
> > > > prior
> > > > > to
> > > > > >> > passing it to the MET tools.
> > > > > >> >
> > > > > >> > Can you please post a sample NetCDF file that you're
passing
> to
> > > > > >> pcp_combine
> > > > > >> > to our anonymous FTP site, following the instructions
listed
> > here:
> > > > > >> >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > >> >
> > > > > >> > Please write once you've posted a sample file. I'll
grab it
> and
> > > run
> > > > > it
> > > > > >> > through the debugger to identify what's happening.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > John Halley Gotway
> > > > > >> >
> > > > > >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT <
> > > > > met_help at ucar.edu
> > > > > >> >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > >
> > > > > >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted
upon.
> > > > > >> > > Transaction: Ticket created by cyh082 at gmail.com
> > > > > >> > > Queue: met_help
> > > > > >> > > Subject: Question on supported data format
> > > > > >> > > Owner: Nobody
> > > > > >> > > Requestors: cyh082 at gmail.com
> > > > > >> > > Status: new
> > > > > >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > Ticket/Display.html?id=77614
> > > > > >> >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > Hello
> > > > > >> > >
> > > > > >> > > I would like to run grid-stat tool using WRF model
forecasts
> > and
> > > > > TRMM
> > > > > >> > > observations.
> > > > > >> > >
> > > > > >> > > I converted original HDF-format TRMM observation to
> > > NetCDF-format,
> > > > > >> and I
> > > > > >> > > also interpolated original lat/lon grid to WRF
Lambert
> > conformal
> > > > > grid
> > > > > >> > using
> > > > > >> > > NCL.
> > > > > >> > >
> > > > > >> > > And I tried to run pcp_combine before running grid-
stat tool
> > to
> > > > make
> > > > > >> > > accumulated precipitation.
> > > > > >> > >
> > > > > >> > > However, I got errors like
> > > > > >> > > "get_field() -> can't open data file ***.nc"
> > > > > >> > >
> > > > > >> > > Could you give me some advices on this problem?
> > > > > >> > >
> > > > > >> > > And... Actually, I selected TRMM observations to
verify WRF
> > > > > forecasts
> > > > > >> > over
> > > > > >> > > South Korea, where stage-4 analysis is not available.
> > > > > >> > > Could you recommend other gridded precipitation
observations
> > > over
> > > > > >> > Southeast
> > > > > >> > > Asia other than TRMM?
> > > > > >> > >
> > > > > >> > > Thank you for your kindness.
> > > > > >> > >
> > > > > >> > > Best regards
> > > > > >> > > Yonghan Choi
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Question on supported data format
From: John Halley Gotway
Time: Wed Oct 12 13:28:15 2016
Yongchan,
Great, thanks for following up and letting me know.
I'll go ahead and resolve this ticket.
John
On Wed, Oct 12, 2016 at 1:52 AM, Yonghan Choi via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
>
> Dear John Halley Gotway:
>
> I really appreciate your detailed explanations on "init_time_ut" and
> "fill/missing value".
>
> I successfully ran "pcp_combine" for WRF outputs using the method
you
> suggested.
>
> I think that I can run "grid_stat" now.
>
> Thank you for your kindness.
>
> Best regards
> Yonghan Choi
>
> On Mon, Oct 10, 2016 at 9:30 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Yongchan,
> >
> > (1) What's the difference between init_time and init_time_ut?
> >
> > The init_time and init_time_ut attributes indicate the exact same
timing
> > information, just formatted differently. Without the "ut", it's
just a
> > string in YYYYMMDD_HHMMSS format. The "ut" means unixtime which
is a
> count
> > of the number of seconds since Jan 1, 1970. It's a very
convenient way
> of
> > storing/processing timing information. In fact, the MET code only
reads
> > the "init_time_ut" attribute, not the "init_time" one. We include
> > "init_time" in the pcp_combine output for ease of the user. The
same is
> > true of "valid_time" and "valid_time_ut".
> >
> > If we had it to do over again, we should probably have skipped the
> unixtime
> > attributes because of the confusion they cause. Instead, we could
have
> > just parsed the YYYYMMDD_HHMMSS strings in the code. However in
MET
> > version 6.0, we'll be switching to writing NetCDF version 4.0
output
> files
> > and at the same time switching to using the NetCDF climate
forecast (CF)
> > convention.
> >
> > (2) MET uses the value of "-9999" to indicate bad data. We don't
really
> > distinguish between fill value and missing value... it's all
stored
> > internally as -9999. While MET uses -9999 internally, the input
NetCDF
> > files can use any value they please. When reading data from those
files,
> > any occurrences of the FillValue attribute will be stored as
-9999.
> >
> > (3) Lastly, I see you're having trouble accessing precip from the
GRIB
> > output files of UPP. I would suggest using the "wgrib" utility to
list
> the
> > contents of those GRIB files. Try:
> >
> > wgrib sample/WRFPRS_d01.12 | grep APCP
> >
> > Accumulated precipitation is somewhat of a special case. State
> variables,
> > such as temperature, are written out as an instantaneous value.
> > Precipitation, on the other hand, must be defined over some
accumulation
> > time interval. Here's my guess as to what might be happening...
> >
> > If you're running WRF-ARW, the default accumulation scheme is to
> accumulate
> > precipitation over the entire model integration. You can
configure it to
> > do otherwise, but runtime accumulation is the default. So the 6-
hour
> > forecast file contains 6 hours of precip. The 12-hour file has 12
hours
> of
> > precip. And so on. But you can easily check this using wgrib
command
> > listed above.
> >
> > If that's the case, instead of the "sum" or "add" commands, you'll
use
> the
> > pcp_combine "subtract" option. For example, if you want the 6
hour
> > accumulation between the 6 and 12 hour forecasts, just subtract
them...
> > like this:
> >
> > pcp_combine -subtract sample/WRFPRS_d01.12 12 sample/WRFPRS_d01.06
6
> > apcp_06_f06_to_f12.nc
> >
> > Hope that helps.
> >
> > Thanks,
> > John
> >
> > On Sun, Oct 9, 2016 at 3:43 AM, Yonghan Choi via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > >
> > > Dear John Halley Gotway:
> > >
> > > Yes, the option you suggested worked!
> > >
> > > I have some questions on the R script.
> > > 1. One of attributes for variable, APCP_03 is "init_time_ut".
> > > What is the difference between "init_time" and "init_time_ut"?
> > >
> > > 2. "FillValue" attribute for variable APCP_03 is set to "-9999".
> > > In MET program, is fill value or missing value assigned to "-
9999"?
> > > I found that fill value for TRMM observations is -9999.9.
> > >
> > > And I tried to run UPP and pcp_combine using WRF forecasts.
> > >
> > > First. I ran UPP using 13 WRF outputs (wrfout_d01_2010-09-
21_00:00:00
> ~
> > > wrfout_d01_2010-09-21_12:00:00).
> > > I included "ACM TOTAL PRECIP" variable in wrf_cntrl.parm file.
> > > After running UPP, I obtained 13 outputs (WRFPRS_d01.00 ~
> WRFPRS_d01.12).
> > >
> > > Then, I tried to run pcp_combine by typing as belows.
> > > pcp_combine -sum 20100921_000000 1 20100921_120000 12
sample_fcst.nc
> > > -pcpdir ./sample
> > >
> > > I got error messages.
> > > DEBUG 1: [1] Reading input file: ./sample/WRFPRS_d01.12
> > > WARNING:
> > > WARNING: MetGrib1DataFile::data_plane() -> No exact match found
for
> > VarInfo
> > > "APCP/A010000" in GRIB file "./sample/WRFPRS_d01.12".
> > > WARNING:
> > > ERROR :
> > > ERROR : get_field() -> can't get data plane from file
> > > "./sample/WRFPRS_d01.12"
> > > ERROR :
> > >
> > > Could you give me some advices on the above problem?
> > >
> > > Thank you for your kindness.
> > >
> > > Best regards
> > > Yonghan Choi
> > >
> > > On Fri, Oct 7, 2016 at 1:34 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu
> > > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I see you're having trouble running pcp_combine on TRMM NetCDF
files.
> > > > Rather than using the "sum" command, I'd suggest trying the
"add"
> > option
> > > > for pcp_combine:
> > > >
> > > > pcp_combine -add \
> > > > 3B42.100921.00z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";'
> \
> > > > 3B42.100921.03z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";'
> \
> > > > 3B42.100921.06z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";'
> \
> > > > 3B42.100921.09z.7.precipitation.nc 'name="APCP_03";
level="(*,*)";'
> \
> > > > precip24.nc
> > > >
> > > > Does that option work for you?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Fri, Oct 7, 2016 at 1:31 AM, Yonghan Choi via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614 >
> > > > >
> > > > > Dear John Halley Gotway:
> > > > >
> > > > > Although you sent me the above email about 40 days ago, I
hope you
> > can
> > > > > remember that email.
> > > > > (I am sorry for the late reply.)
> > > > >
> > > > > Actually, I successfully converted original binary TRMM file
into
> > > NetCDF
> > > > > file using R script.
> > > > >
> > > > > Let's assume that I obtained the below "4" 3-h accumulated
> > > precipitation
> > > > > files.
> > > > > 3B42.100921.00z.7.precipitation.nc
> > > > > 3B42.100921.03z.7.precipitation.nc
> > > > > 3B42.100921.06z.7.precipitation.nc
> > > > > 3B42.100921.09z.7.precipitation.nc
> > > > >
> > > > > I would like to make 12-h accumulated (from 00 UTC to 12 UTC
21
> > > > > September) precipitation file using "pcp_combine" tool.
> > > > >
> > > > > I put all NetCDF files to "input" directory.
> > > > > I executed "pcp_combine" by typing as follows.
> > > > > pcp_combine -sum 00000000_000000 3 20100921_120000 12
> sample_obs.nc
> > > > > -pcpdir
> > > > > ./input
> > > > >
> > > > > However, I got the error message as below.
> > > > > sum_grib_files() -> Cannot find a file with a valid time of
> > > > 20100921_120000
> > > > > and accumulation time of 030000 matching the regular
expression
> ".*"
> > > > >
> > > > > Could you give me some advices?
> > > > >
> > > > > Thank you for your kindness.
> > > > >
> > > > > Best regards
> > > > > Yonghan Choi
> > > > >
> > > > > On Tue, Aug 23, 2016 at 4:39 PM, John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Oops, looks like I sent you two versions of that PNG file.
I
> > > intended
> > > > it
> > > > > > to be only that global version.
> > > > > >
> > > > > > John
> > > > > >
> > > > > > On Tue, Aug 23, 2016 at 4:38 PM, John Halley Gotway <
> > johnhg at ucar.edu
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Yonghan,
> > > > > > >
> > > > > > > Thanks for sending that sample data file. Yes, it is
> definitely
> > a
> > > > > flavor
> > > > > > > of NetCDF that the MET software is not expecting.
Here's an
> > option
> > > > for
> > > > > > > accessing that same TRMM data, following links from the
MET
> > > website:
> > > > > > >
http://www.dtcenter.org/met/users/downloads/observation_
> > > data.php
> > > > > > >
> > > > > > > # Pull binary 3-hourly TRMM data file
> > > > > > > wget
ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/
> > > > > > > 201009/3B42.100921.00z.7.precipitation.bin
> > > > > > >
> > > > > > > # Pull Rscript from MET website
> > > > > > > wget
http://www.dtcenter.org/met/users/downloads/Rscripts/
> > > > trmmbin2nc.R
> > > > > > >
> > > > > > > # Edit that Rscript by setting
> > > > > > > out_lat_ll = -50
> > > > > > > out_lon_ll = 0
> > > > > > > out_lat_ur = 50
> > > > > > > out_lon_ur = 359.75
> > > > > > >
> > > > > > > # Run the Rscript
> > > > > > > Rscript trmmbin2nc.R 3B42.100921.00z.7.precipitation.bin
> > > > > > > 3B42.100921.00z.7.precipitation.nc
> > > > > > >
> > > > > > > # Plot the result
> > > > > > > met-5.2/bin/plot_data_plane
3B42.100921.00z.7.precipitation.nc
> > > > > > > 3B42.100921.00z.7.precipitation.ps 'name="APCP_03";
> > > level="(*,*)";'
> > > > > > >
> > > > > > > I've attached a PNG version of resulting image.
> > > > > > >
> > > > > > > I see that the domain of your data is much smaller. You
have
> two
> > > > > choices
> > > > > > > here...
> > > > > > > (1) In that Rscript, you can choose different boundaries
(i.e.
> > > > > > > out_lat/lon_ll/ur) to specify the tile of data to be
selected.
> > > > > > > (2) As of version 5.1, MET includes support for
regridding the
> > data
> > > > it
> > > > > > > reads. You could keep TRMM on it's native domain and
use the
> MET
> > > > tools
> > > > > > to
> > > > > > > do the regridding. For example, the "regrid_data_plane"
tool
> > > reads a
> > > > > > > NetCDF file, regrids the data, and writes a NetCDF file.
> > > > > Alternatively,
> > > > > > > the "regrid" section of the configuration files for the
MET
> tools
> > > may
> > > > > be
> > > > > > > used to do the regridding on the fly. For example, you
could
> run
> > > > > > Grid-Stat
> > > > > > > to compare your model output to TRMM and say "regrid = {
field
> =
> > > > FCST;
> > > > > > > ...}". That tells Grid-Stat to automatically regrid the
TRMM
> > > > > > observations
> > > > > > > to your model domain.
> > > > > > >
> > > > > > > Hope that helps. Do this sounds like a workable path
for you?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Aug 21, 2016 at 12:08 AM, Yonghan Choi via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >>
> > > > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77614
> >
> > > > > > >>
> > > > > > >> Dear John Halley Gotway:
> > > > > > >>
> > > > > > >> I uploaded my NetCDF-formatted TRMM data to your ftp
server.
> > > > > > >>
> > > > > > >> Actually, I interpolated (to WRF grids) and made
accumulation
> > > using
> > > > > > >> original TRMM data, and I converted HDF-formatted
original
> TRMM
> > > data
> > > > > to
> > > > > > >> NetCDF format.
> > > > > > >>
> > > > > > >> Thank you for your kindness.
> > > > > > >>
> > > > > > >> Best regards
> > > > > > >> Yonghan Choi
> > > > > > >>
> > > > > > >> On Fri, Aug 19, 2016 at 4:40 PM, John Halley Gotway via
RT <
> > > > > > >> met_help at ucar.edu> wrote:
> > > > > > >>
> > > > > > >> > Yongchan,
> > > > > > >> >
> > > > > > >> > I see that you're having trouble using gridded TRMM
data in
> > MET.
> > > > I
> > > > > > read
> > > > > > >> > that you've done some pre-processing of the TRMM data
using
> > NCL
> > > > > prior
> > > > > > to
> > > > > > >> > passing it to the MET tools.
> > > > > > >> >
> > > > > > >> > Can you please post a sample NetCDF file that you're
passing
> > to
> > > > > > >> pcp_combine
> > > > > > >> > to our anonymous FTP site, following the instructions
listed
> > > here:
> > > > > > >> > http://www.dtcenter.org/met/
> users/support/met_help.php#ftp
> > > > > > >> >
> > > > > > >> > Please write once you've posted a sample file. I'll
grab it
> > and
> > > > run
> > > > > > it
> > > > > > >> > through the debugger to identify what's happening.
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > John Halley Gotway
> > > > > > >> >
> > > > > > >> > On Thu, Aug 18, 2016 at 4:11 PM, Yonghan Choi via RT
<
> > > > > > met_help at ucar.edu
> > > > > > >> >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > > Thu Aug 18 16:11:55 2016: Request 77614 was acted
upon.
> > > > > > >> > > Transaction: Ticket created by cyh082 at gmail.com
> > > > > > >> > > Queue: met_help
> > > > > > >> > > Subject: Question on supported data format
> > > > > > >> > > Owner: Nobody
> > > > > > >> > > Requestors: cyh082 at gmail.com
> > > > > > >> > > Status: new
> > > > > > >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > > Ticket/Display.html?id=77614
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > Hello
> > > > > > >> > >
> > > > > > >> > > I would like to run grid-stat tool using WRF model
> forecasts
> > > and
> > > > > > TRMM
> > > > > > >> > > observations.
> > > > > > >> > >
> > > > > > >> > > I converted original HDF-format TRMM observation to
> > > > NetCDF-format,
> > > > > > >> and I
> > > > > > >> > > also interpolated original lat/lon grid to WRF
Lambert
> > > conformal
> > > > > > grid
> > > > > > >> > using
> > > > > > >> > > NCL.
> > > > > > >> > >
> > > > > > >> > > And I tried to run pcp_combine before running grid-
stat
> tool
> > > to
> > > > > make
> > > > > > >> > > accumulated precipitation.
> > > > > > >> > >
> > > > > > >> > > However, I got errors like
> > > > > > >> > > "get_field() -> can't open data file ***.nc"
> > > > > > >> > >
> > > > > > >> > > Could you give me some advices on this problem?
> > > > > > >> > >
> > > > > > >> > > And... Actually, I selected TRMM observations to
verify
> WRF
> > > > > > forecasts
> > > > > > >> > over
> > > > > > >> > > South Korea, where stage-4 analysis is not
available.
> > > > > > >> > > Could you recommend other gridded precipitation
> observations
> > > > over
> > > > > > >> > Southeast
> > > > > > >> > > Asia other than TRMM?
> > > > > > >> > >
> > > > > > >> > > Thank you for your kindness.
> > > > > > >> > >
> > > > > > >> > > Best regards
> > > > > > >> > > Yonghan Choi
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list