[Met_help] [rt.rap.ucar.edu #55783] History for Fwd: problems running MODE v3.1
Paul Oldenburg via RT
met_help at ucar.edu
Tue Apr 24 12:25:01 MDT 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I'm trying to run mode from the version 3.1 MET package. I have to convert
some HDF files to netCDF first. I do that, then attempt to run MODE using
a command like
bin/mode pcp_6hr_ARW_cn.netacc06_108000.01 pcp_6hr_OBS.netacc06_108000.01
MODE_config_6hr -outdir MODE_output
However, I get this output/error:
DEBUG 1: Default Config File:
/home/jdduda/METv3.1/data/config/MODEConfig_default
DEBUG 1: Match Config File: MODE_config_6hr
DEBUG 1: Merge Config File: MODE_config_6hr
DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
NetCDF: Attribute not found
The config file is attached.
Here is the output from ncdump -h on the forecast and observed data sets:
-*FCST*
[jdduda at sooner2 METv3.1]$ ncdump -h pcp_6hr_ARW_cn.netacc06_108000.01
netcdf pcp_6hr_ARW_cn.netacc06_108000 {
dimensions:
x = 1163 ;
y = 723 ;
z = 1 ;
variables:
float acc06_(z, y, x) ;
acc06_:standard_name = "acc06_" ;
acc06_:long_name = "acc06_" ;
acc06_:units = "mm" ;
}
-*OBS*
[jdduda at sooner2 METv3.1]$ ncdump -h pcp_6hr_OBS.netacc06_108000.01
netcdf pcp_6hr_OBS.netacc06_108000 {
dimensions:
x = 1163 ;
y = 723 ;
z = 1 ;
variables:
float acc06_(z, y, x) ;
acc06_:standard_name = "acc06_" ;
acc06_:long_name = "acc06_" ;
acc06_:units = "mm" ;
}
Jeff Duda
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Tue Apr 10 16:57:52 2012
Jeff,
MET is capable of reading gridded NetCDF data, but only for two
specific formats. NetCDF files can have arbitrary
structure including the number of dimensions of the variables
contained therein. The two formats read by MET are
2-dimensional grids that have certain attributes and dimensions. One
is the output of the WRF post-processing tool
p_interp and the other is a format used by MET internally.
You can see the structure of the MET internal NetCDF file format by
using ncdump on the output of the pcp_combine tool
that appears after you run the test scripts. For example, look at
$MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc where
$MET_BASE is the installation folder of MET on your
system.
In general, it is not an easy task to convert data into this file
format, but it has been done. In the DTC, we use the
R statistical computing language with the ncdf library to perform this
type of conversion. It is hard to give you more
specific advice without knowing anything about your model and obs
data. If your data consists of one or more
geographical planes with a known map projection, then it should be
possible to perform this conversion, but it may take
a lot of work to do so.
Please let me know if you have any questions.
Paul
On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>
> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> Transaction: Ticket created by jeffduda319 at gmail.com
> Queue: met_help
> Subject: Fwd: problems running MODE v3.1
> Owner: Nobody
> Requestors: jeffduda319 at gmail.com
> Status: new
> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>
>
> Hello,
> I'm trying to run mode from the version 3.1 MET package. I have to
convert
> some HDF files to netCDF first. I do that, then attempt to run MODE
using
> a command like
>
> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
pcp_6hr_OBS.netacc06_108000.01
> MODE_config_6hr -outdir MODE_output
>
> However, I get this output/error:
> DEBUG 1: Default Config File:
> /home/jdduda/METv3.1/data/config/MODEConfig_default
> DEBUG 1: Match Config File: MODE_config_6hr
> DEBUG 1: Merge Config File: MODE_config_6hr
> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> NetCDF: Attribute not found
>
> The config file is attached.
> Here is the output from ncdump -h on the forecast and observed data
sets:
>
> -*FCST*
> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> dimensions:
> x = 1163 ;
> y = 723 ;
> z = 1 ;
> variables:
> float acc06_(z, y, x) ;
> acc06_:standard_name = "acc06_" ;
> acc06_:long_name = "acc06_" ;
> acc06_:units = "mm" ;
> }
>
> -*OBS*
> [jdduda at sooner2 METv3.1]$ ncdump -h pcp_6hr_OBS.netacc06_108000.01
> netcdf pcp_6hr_OBS.netacc06_108000 {
> dimensions:
> x = 1163 ;
> y = 723 ;
> z = 1 ;
> variables:
> float acc06_(z, y, x) ;
> acc06_:standard_name = "acc06_" ;
> acc06_:long_name = "acc06_" ;
> acc06_:units = "mm" ;
> }
>
> Jeff Duda
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Wed Apr 11 10:44:11 2012
Paul,
I don't need to run pcp_combine since my files already contain the
precipitation on the interval I want. Also, I don't see this p_interp
tool
anywhere in the package or the documentation. What is it and how do I
use
it?
Jeff
On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> MET is capable of reading gridded NetCDF data, but only for two
specific
> formats. NetCDF files can have arbitrary
> structure including the number of dimensions of the variables
contained
> therein. The two formats read by MET are
> 2-dimensional grids that have certain attributes and dimensions.
One is
> the output of the WRF post-processing tool
> p_interp and the other is a format used by MET internally.
>
> You can see the structure of the MET internal NetCDF file format by
using
> ncdump on the output of the pcp_combine tool
> that appears after you run the test scripts. For example, look at
> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc where
> $MET_BASE is the installation folder of MET on your
> system.
>
> In general, it is not an easy task to convert data into this file
format,
> but it has been done. In the DTC, we use the
> R statistical computing language with the ncdf library to perform
this
> type of conversion. It is hard to give you more
> specific advice without knowing anything about your model and obs
data.
> If your data consists of one or more
> geographical planes with a known map projection, then it should be
> possible to perform this conversion, but it may take
> a lot of work to do so.
>
> Please let me know if you have any questions.
>
> Paul
>
>
> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> >
> > Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> > Transaction: Ticket created by jeffduda319 at gmail.com
> > Queue: met_help
> > Subject: Fwd: problems running MODE v3.1
> > Owner: Nobody
> > Requestors: jeffduda319 at gmail.com
> > Status: new
> > Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >
> >
> > Hello,
> > I'm trying to run mode from the version 3.1 MET package. I have
to
> convert
> > some HDF files to netCDF first. I do that, then attempt to run
MODE
> using
> > a command like
> >
> > bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
pcp_6hr_OBS.netacc06_108000.01
> > MODE_config_6hr -outdir MODE_output
> >
> > However, I get this output/error:
> > DEBUG 1: Default Config File:
> > /home/jdduda/METv3.1/data/config/MODEConfig_default
> > DEBUG 1: Match Config File: MODE_config_6hr
> > DEBUG 1: Merge Config File: MODE_config_6hr
> > DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> > DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> > NetCDF: Attribute not found
> >
> > The config file is attached.
> > Here is the output from ncdump -h on the forecast and observed
data sets:
> >
> > -*FCST*
> > [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
> > netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> > dimensions:
> > x = 1163 ;
> > y = 723 ;
> > z = 1 ;
> > variables:
> > float acc06_(z, y, x) ;
> > acc06_:standard_name = "acc06_" ;
> > acc06_:long_name = "acc06_" ;
> > acc06_:units = "mm" ;
> > }
> >
> > -*OBS*
> > [jdduda at sooner2 METv3.1]$ ncdump -h pcp_6hr_OBS.netacc06_108000.01
> > netcdf pcp_6hr_OBS.netacc06_108000 {
> > dimensions:
> > x = 1163 ;
> > y = 723 ;
> > z = 1 ;
> > variables:
> > float acc06_(z, y, x) ;
> > acc06_:standard_name = "acc06_" ;
> > acc06_:long_name = "acc06_" ;
> > acc06_:units = "mm" ;
> > }
> >
> > Jeff Duda
> >
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Wed Apr 11 10:57:37 2012
Jeff,
I was not suggesting that you process your data using pcp_combine,
only that you look at the format of the NetCDF output
from that tool. That format is one of the NetCDF gridded data formats
understood by mode, and so you must coerce your
data into that same format. Does that make sense?
I only mentioned p_interp because it is another NetCDF format of
gridded data understood by MET tools, but I think it
would be easier for you to use the MET internal format, unless your
model data is WRF output. In that case, I would
recommend the WRF Unified Post-Processor (UPP), anyway. The bottom
line is don't worry about p_interp.
Paul
On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>
> Paul,
> I don't need to run pcp_combine since my files already contain the
> precipitation on the interval I want. Also, I don't see this
p_interp tool
> anywhere in the package or the documentation. What is it and how do
I use
> it?
>
> Jeff
>
> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via
RT<met_help at ucar.edu>wrote:
>
>> Jeff,
>>
>> MET is capable of reading gridded NetCDF data, but only for two
specific
>> formats. NetCDF files can have arbitrary
>> structure including the number of dimensions of the variables
contained
>> therein. The two formats read by MET are
>> 2-dimensional grids that have certain attributes and dimensions.
One is
>> the output of the WRF post-processing tool
>> p_interp and the other is a format used by MET internally.
>>
>> You can see the structure of the MET internal NetCDF file format by
using
>> ncdump on the output of the pcp_combine tool
>> that appears after you run the test scripts. For example, look at
>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc where
>> $MET_BASE is the installation folder of MET on your
>> system.
>>
>> In general, it is not an easy task to convert data into this file
format,
>> but it has been done. In the DTC, we use the
>> R statistical computing language with the ncdf library to perform
this
>> type of conversion. It is hard to give you more
>> specific advice without knowing anything about your model and obs
data.
>> If your data consists of one or more
>> geographical planes with a known map projection, then it should be
>> possible to perform this conversion, but it may take
>> a lot of work to do so.
>>
>> Please let me know if you have any questions.
>>
>> Paul
>>
>>
>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>>>
>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
>>> Transaction: Ticket created by jeffduda319 at gmail.com
>>> Queue: met_help
>>> Subject: Fwd: problems running MODE v3.1
>>> Owner: Nobody
>>> Requestors: jeffduda319 at gmail.com
>>> Status: new
>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>
>>>
>>> Hello,
>>> I'm trying to run mode from the version 3.1 MET package. I have
to
>> convert
>>> some HDF files to netCDF first. I do that, then attempt to run
MODE
>> using
>>> a command like
>>>
>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
pcp_6hr_OBS.netacc06_108000.01
>>> MODE_config_6hr -outdir MODE_output
>>>
>>> However, I get this output/error:
>>> DEBUG 1: Default Config File:
>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
>>> DEBUG 1: Match Config File: MODE_config_6hr
>>> DEBUG 1: Merge Config File: MODE_config_6hr
>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
>>> NetCDF: Attribute not found
>>>
>>> The config file is attached.
>>> Here is the output from ncdump -h on the forecast and observed
data sets:
>>>
>>> -*FCST*
>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
>>> dimensions:
>>> x = 1163 ;
>>> y = 723 ;
>>> z = 1 ;
>>> variables:
>>> float acc06_(z, y, x) ;
>>> acc06_:standard_name = "acc06_" ;
>>> acc06_:long_name = "acc06_" ;
>>> acc06_:units = "mm" ;
>>> }
>>>
>>> -*OBS*
>>> [jdduda at sooner2 METv3.1]$ ncdump -h pcp_6hr_OBS.netacc06_108000.01
>>> netcdf pcp_6hr_OBS.netacc06_108000 {
>>> dimensions:
>>> x = 1163 ;
>>> y = 723 ;
>>> z = 1 ;
>>> variables:
>>> float acc06_(z, y, x) ;
>>> acc06_:standard_name = "acc06_" ;
>>> acc06_:long_name = "acc06_" ;
>>> acc06_:units = "mm" ;
>>> }
>>>
>>> Jeff Duda
>>>
>>
>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Wed Apr 11 13:19:36 2012
Jeff,
There is a sample R script that may be of interest to you if you
decide to format your HDF5 data into the MET internal
NetCDF format. The script is used to convert gridded text TRMM data
with a known map projection (lat/long). You can
download it from the MET website:
http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
Also, we are just curious, what is the source of your HDF5 data? If
it is an emerging and/or common data format, we may
consider adding support for it into MET.
Thanks,
Paul
On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> Jeff,
>
> I was not suggesting that you process your data using pcp_combine,
only that you look at the format of the NetCDF output
> from that tool. That format is one of the NetCDF gridded data
formats understood by mode, and so you must coerce your
> data into that same format. Does that make sense?
>
> I only mentioned p_interp because it is another NetCDF format of
gridded data understood by MET tools, but I think it
> would be easier for you to use the MET internal format, unless your
model data is WRF output. In that case, I would
> recommend the WRF Unified Post-Processor (UPP), anyway. The bottom
line is don't worry about p_interp.
>
> Paul
>
>
> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>
>> Paul,
>> I don't need to run pcp_combine since my files already contain the
>> precipitation on the interval I want. Also, I don't see this
p_interp tool
>> anywhere in the package or the documentation. What is it and how do
I use
>> it?
>>
>> Jeff
>>
>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via
RT<met_help at ucar.edu>wrote:
>>
>>> Jeff,
>>>
>>> MET is capable of reading gridded NetCDF data, but only for two
specific
>>> formats. NetCDF files can have arbitrary
>>> structure including the number of dimensions of the variables
contained
>>> therein. The two formats read by MET are
>>> 2-dimensional grids that have certain attributes and dimensions.
One is
>>> the output of the WRF post-processing tool
>>> p_interp and the other is a format used by MET internally.
>>>
>>> You can see the structure of the MET internal NetCDF file format
by using
>>> ncdump on the output of the pcp_combine tool
>>> that appears after you run the test scripts. For example, look at
>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc where
>>> $MET_BASE is the installation folder of MET on your
>>> system.
>>>
>>> In general, it is not an easy task to convert data into this file
format,
>>> but it has been done. In the DTC, we use the
>>> R statistical computing language with the ncdf library to perform
this
>>> type of conversion. It is hard to give you more
>>> specific advice without knowing anything about your model and obs
data.
>>> If your data consists of one or more
>>> geographical planes with a known map projection, then it should be
>>> possible to perform this conversion, but it may take
>>> a lot of work to do so.
>>>
>>> Please let me know if you have any questions.
>>>
>>> Paul
>>>
>>>
>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>>>>
>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
>>>> Transaction: Ticket created by jeffduda319 at gmail.com
>>>> Queue: met_help
>>>> Subject: Fwd: problems running MODE v3.1
>>>> Owner: Nobody
>>>> Requestors: jeffduda319 at gmail.com
>>>> Status: new
>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>
>>>>
>>>> Hello,
>>>> I'm trying to run mode from the version 3.1 MET package. I have
to
>>> convert
>>>> some HDF files to netCDF first. I do that, then attempt to run
MODE
>>> using
>>>> a command like
>>>>
>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
pcp_6hr_OBS.netacc06_108000.01
>>>> MODE_config_6hr -outdir MODE_output
>>>>
>>>> However, I get this output/error:
>>>> DEBUG 1: Default Config File:
>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
>>>> DEBUG 1: Match Config File: MODE_config_6hr
>>>> DEBUG 1: Merge Config File: MODE_config_6hr
>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
>>>> NetCDF: Attribute not found
>>>>
>>>> The config file is attached.
>>>> Here is the output from ncdump -h on the forecast and observed
data sets:
>>>>
>>>> -*FCST*
>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
>>>> dimensions:
>>>> x = 1163 ;
>>>> y = 723 ;
>>>> z = 1 ;
>>>> variables:
>>>> float acc06_(z, y, x) ;
>>>> acc06_:standard_name = "acc06_" ;
>>>> acc06_:long_name = "acc06_" ;
>>>> acc06_:units = "mm" ;
>>>> }
>>>>
>>>> -*OBS*
>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_OBS.netacc06_108000.01
>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
>>>> dimensions:
>>>> x = 1163 ;
>>>> y = 723 ;
>>>> z = 1 ;
>>>> variables:
>>>> float acc06_(z, y, x) ;
>>>> acc06_:standard_name = "acc06_" ;
>>>> acc06_:long_name = "acc06_" ;
>>>> acc06_:units = "mm" ;
>>>> }
>>>>
>>>> Jeff Duda
>>>>
>>>
>>>
>>>
>>
>>
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Wed Apr 11 13:35:02 2012
Paul,
My data is coming from the WRF, but on a very large domain and at 4 km
resolution so as to require extreme compression for long-term storage.
Specifically I'm verifying precip forecasts from the Center for
Analysis
and Prediction of Storms' (CAPS at the University of Oklahoma) storm
scale
ensemble forecast system run as a part of the NOAA Hazardous Weather
Testbed spring experiment conducted annually. Other people at CAPS
are
using other tools to convert the WRF history dumps to other formats
for
compression and storage, and those compressed files are what I have
available to me. I have conversion utilities to use, but apparently
the
structure of the resulting GRIB and netCDF files doesn't "jive" with
the
structure required by MET.
I've only heard of and briefly used R and am not at all proficient
with it,
nor would I say I have adequate knowledge of it to use it for the
purpose
you have mentioned. However, I think I can get help from other CAPS
scientists who have verified the same precip forecasts using MODE
previously for my purposes. So I will probably revert to working with
them
for now.
I'll make sure to ask if I have any other questions, but you can
probably
close this ticket. Thanks.
Jeff
On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> There is a sample R script that may be of interest to you if you
decide to
> format your HDF5 data into the MET internal
> NetCDF format. The script is used to convert gridded text TRMM data
with
> a known map projection (lat/long). You can
> download it from the MET website:
> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
>
> Also, we are just curious, what is the source of your HDF5 data? If
it is
> an emerging and/or common data format, we may
> consider adding support for it into MET.
>
> Thanks,
>
> Paul
>
>
> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> > Jeff,
> >
> > I was not suggesting that you process your data using pcp_combine,
only
> that you look at the format of the NetCDF output
> > from that tool. That format is one of the NetCDF gridded data
formats
> understood by mode, and so you must coerce your
> > data into that same format. Does that make sense?
> >
> > I only mentioned p_interp because it is another NetCDF format of
gridded
> data understood by MET tools, but I think it
> > would be easier for you to use the MET internal format, unless
your
> model data is WRF output. In that case, I would
> > recommend the WRF Unified Post-Processor (UPP), anyway. The bottom
line
> is don't worry about p_interp.
> >
> > Paul
> >
> >
> > On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>
> >> Paul,
> >> I don't need to run pcp_combine since my files already contain
the
> >> precipitation on the interval I want. Also, I don't see this
p_interp
> tool
> >> anywhere in the package or the documentation. What is it and how
do I
> use
> >> it?
> >>
> >> Jeff
> >>
> >> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> met_help at ucar.edu>wrote:
> >>
> >>> Jeff,
> >>>
> >>> MET is capable of reading gridded NetCDF data, but only for two
> specific
> >>> formats. NetCDF files can have arbitrary
> >>> structure including the number of dimensions of the variables
contained
> >>> therein. The two formats read by MET are
> >>> 2-dimensional grids that have certain attributes and dimensions.
One is
> >>> the output of the WRF post-processing tool
> >>> p_interp and the other is a format used by MET internally.
> >>>
> >>> You can see the structure of the MET internal NetCDF file format
by
> using
> >>> ncdump on the output of the pcp_combine tool
> >>> that appears after you run the test scripts. For example, look
at
> >>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
> >>> $MET_BASE is the installation folder of MET on your
> >>> system.
> >>>
> >>> In general, it is not an easy task to convert data into this
file
> format,
> >>> but it has been done. In the DTC, we use the
> >>> R statistical computing language with the ncdf library to
perform this
> >>> type of conversion. It is hard to give you more
> >>> specific advice without knowing anything about your model and
obs data.
> >>> If your data consists of one or more
> >>> geographical planes with a known map projection, then it should
be
> >>> possible to perform this conversion, but it may take
> >>> a lot of work to do so.
> >>>
> >>> Please let me know if you have any questions.
> >>>
> >>> Paul
> >>>
> >>>
> >>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> >>>>
> >>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> >>>> Transaction: Ticket created by jeffduda319 at gmail.com
> >>>> Queue: met_help
> >>>> Subject: Fwd: problems running MODE v3.1
> >>>> Owner: Nobody
> >>>> Requestors: jeffduda319 at gmail.com
> >>>> Status: new
> >>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>>
> >>>>
> >>>> Hello,
> >>>> I'm trying to run mode from the version 3.1 MET package. I have
to
> >>> convert
> >>>> some HDF files to netCDF first. I do that, then attempt to run
MODE
> >>> using
> >>>> a command like
> >>>>
> >>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> pcp_6hr_OBS.netacc06_108000.01
> >>>> MODE_config_6hr -outdir MODE_output
> >>>>
> >>>> However, I get this output/error:
> >>>> DEBUG 1: Default Config File:
> >>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> >>>> DEBUG 1: Match Config File: MODE_config_6hr
> >>>> DEBUG 1: Merge Config File: MODE_config_6hr
> >>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> >>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> >>>> NetCDF: Attribute not found
> >>>>
> >>>> The config file is attached.
> >>>> Here is the output from ncdump -h on the forecast and observed
data
> sets:
> >>>>
> >>>> -*FCST*
> >>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
> >>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> >>>> dimensions:
> >>>> x = 1163 ;
> >>>> y = 723 ;
> >>>> z = 1 ;
> >>>> variables:
> >>>> float acc06_(z, y, x) ;
> >>>> acc06_:standard_name = "acc06_" ;
> >>>> acc06_:long_name = "acc06_" ;
> >>>> acc06_:units = "mm" ;
> >>>> }
> >>>>
> >>>> -*OBS*
> >>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_OBS.netacc06_108000.01
> >>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> >>>> dimensions:
> >>>> x = 1163 ;
> >>>> y = 723 ;
> >>>> z = 1 ;
> >>>> variables:
> >>>> float acc06_(z, y, x) ;
> >>>> acc06_:standard_name = "acc06_" ;
> >>>> acc06_:long_name = "acc06_" ;
> >>>> acc06_:units = "mm" ;
> >>>> }
> >>>>
> >>>> Jeff Duda
> >>>>
> >>>
> >>>
> >>>
> >>
> >>
> >
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Wed Apr 11 13:40:43 2012
Jeff,
Have you tried using the GRIB version of your data with MET? That
should be supported by mode. If you still have
trouble with GRIB, can you please upload your data file(s) and config
file to our FTP site using these instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Thanks for the detailed information about your data. I have actually
verified the CAPS ensemble as part of the HWT
project myself using MET, although I have not yet tried to do so with
this year's data.
Thanks,
Paul
On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>
> Paul,
> My data is coming from the WRF, but on a very large domain and at 4
km
> resolution so as to require extreme compression for long-term
storage.
> Specifically I'm verifying precip forecasts from the Center for
Analysis
> and Prediction of Storms' (CAPS at the University of Oklahoma) storm
scale
> ensemble forecast system run as a part of the NOAA Hazardous Weather
> Testbed spring experiment conducted annually. Other people at CAPS
are
> using other tools to convert the WRF history dumps to other formats
for
> compression and storage, and those compressed files are what I have
> available to me. I have conversion utilities to use, but apparently
the
> structure of the resulting GRIB and netCDF files doesn't "jive" with
the
> structure required by MET.
>
> I've only heard of and briefly used R and am not at all proficient
with it,
> nor would I say I have adequate knowledge of it to use it for the
purpose
> you have mentioned. However, I think I can get help from other CAPS
> scientists who have verified the same precip forecasts using MODE
> previously for my purposes. So I will probably revert to working
with them
> for now.
>
> I'll make sure to ask if I have any other questions, but you can
probably
> close this ticket. Thanks.
>
> Jeff
>
> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via
RT<met_help at ucar.edu>wrote:
>
>> Jeff,
>>
>> There is a sample R script that may be of interest to you if you
decide to
>> format your HDF5 data into the MET internal
>> NetCDF format. The script is used to convert gridded text TRMM
data with
>> a known map projection (lat/long). You can
>> download it from the MET website:
>> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
>>
>> Also, we are just curious, what is the source of your HDF5 data?
If it is
>> an emerging and/or common data format, we may
>> consider adding support for it into MET.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
>>> Jeff,
>>>
>>> I was not suggesting that you process your data using pcp_combine,
only
>> that you look at the format of the NetCDF output
>>> from that tool. That format is one of the NetCDF gridded data
formats
>> understood by mode, and so you must coerce your
>>> data into that same format. Does that make sense?
>>>
>>> I only mentioned p_interp because it is another NetCDF format of
gridded
>> data understood by MET tools, but I think it
>>> would be easier for you to use the MET internal format, unless
your
>> model data is WRF output. In that case, I would
>>> recommend the WRF Unified Post-Processor (UPP), anyway. The bottom
line
>> is don't worry about p_interp.
>>>
>>> Paul
>>>
>>>
>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>
>>>> Paul,
>>>> I don't need to run pcp_combine since my files already contain
the
>>>> precipitation on the interval I want. Also, I don't see this
p_interp
>> tool
>>>> anywhere in the package or the documentation. What is it and how
do I
>> use
>>>> it?
>>>>
>>>> Jeff
>>>>
>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
>> met_help at ucar.edu>wrote:
>>>>
>>>>> Jeff,
>>>>>
>>>>> MET is capable of reading gridded NetCDF data, but only for two
>> specific
>>>>> formats. NetCDF files can have arbitrary
>>>>> structure including the number of dimensions of the variables
contained
>>>>> therein. The two formats read by MET are
>>>>> 2-dimensional grids that have certain attributes and dimensions.
One is
>>>>> the output of the WRF post-processing tool
>>>>> p_interp and the other is a format used by MET internally.
>>>>>
>>>>> You can see the structure of the MET internal NetCDF file format
by
>> using
>>>>> ncdump on the output of the pcp_combine tool
>>>>> that appears after you run the test scripts. For example, look
at
>>>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
>>>>> $MET_BASE is the installation folder of MET on your
>>>>> system.
>>>>>
>>>>> In general, it is not an easy task to convert data into this
file
>> format,
>>>>> but it has been done. In the DTC, we use the
>>>>> R statistical computing language with the ncdf library to
perform this
>>>>> type of conversion. It is hard to give you more
>>>>> specific advice without knowing anything about your model and
obs data.
>>>>> If your data consists of one or more
>>>>> geographical planes with a known map projection, then it should
be
>>>>> possible to perform this conversion, but it may take
>>>>> a lot of work to do so.
>>>>>
>>>>> Please let me know if you have any questions.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>>>>>>
>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
>>>>>> Queue: met_help
>>>>>> Subject: Fwd: problems running MODE v3.1
>>>>>> Owner: Nobody
>>>>>> Requestors: jeffduda319 at gmail.com
>>>>>> Status: new
>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>>>
>>>>>>
>>>>>> Hello,
>>>>>> I'm trying to run mode from the version 3.1 MET package. I have
to
>>>>> convert
>>>>>> some HDF files to netCDF first. I do that, then attempt to run
MODE
>>>>> using
>>>>>> a command like
>>>>>>
>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
>> pcp_6hr_OBS.netacc06_108000.01
>>>>>> MODE_config_6hr -outdir MODE_output
>>>>>>
>>>>>> However, I get this output/error:
>>>>>> DEBUG 1: Default Config File:
>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
>>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
>>>>>> NetCDF: Attribute not found
>>>>>>
>>>>>> The config file is attached.
>>>>>> Here is the output from ncdump -h on the forecast and observed
data
>> sets:
>>>>>>
>>>>>> -*FCST*
>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
>>>>>> dimensions:
>>>>>> x = 1163 ;
>>>>>> y = 723 ;
>>>>>> z = 1 ;
>>>>>> variables:
>>>>>> float acc06_(z, y, x) ;
>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>> acc06_:long_name = "acc06_" ;
>>>>>> acc06_:units = "mm" ;
>>>>>> }
>>>>>>
>>>>>> -*OBS*
>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_OBS.netacc06_108000.01
>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
>>>>>> dimensions:
>>>>>> x = 1163 ;
>>>>>> y = 723 ;
>>>>>> z = 1 ;
>>>>>> variables:
>>>>>> float acc06_(z, y, x) ;
>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>> acc06_:long_name = "acc06_" ;
>>>>>> acc06_:units = "mm" ;
>>>>>> }
>>>>>>
>>>>>> Jeff Duda
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Wed Apr 11 13:57:42 2012
Paul,
When trying to convert instead to GRIB format, I get this error when
running MODE:
[jdduda at sooner2 METv3.1]$ bin/mode pcp_6hr_ARW_cn.grbacc06_021600
pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
ERROR :
ERROR : VarInfoFactory::new_var_info() -> unsupported gridded data
file
type "0"
ERROR :
Jeff
On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> Have you tried using the GRIB version of your data with MET? That
should
> be supported by mode. If you still have
> trouble with GRIB, can you please upload your data file(s) and
config file
> to our FTP site using these instructions:
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Thanks for the detailed information about your data. I have
actually
> verified the CAPS ensemble as part of the HWT
> project myself using MET, although I have not yet tried to do so
with this
> year's data.
>
> Thanks,
>
> Paul
>
>
> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >
> > Paul,
> > My data is coming from the WRF, but on a very large domain and at
4 km
> > resolution so as to require extreme compression for long-term
storage.
> > Specifically I'm verifying precip forecasts from the Center for
Analysis
> > and Prediction of Storms' (CAPS at the University of Oklahoma)
storm
> scale
> > ensemble forecast system run as a part of the NOAA Hazardous
Weather
> > Testbed spring experiment conducted annually. Other people at
CAPS are
> > using other tools to convert the WRF history dumps to other
formats for
> > compression and storage, and those compressed files are what I
have
> > available to me. I have conversion utilities to use, but
apparently the
> > structure of the resulting GRIB and netCDF files doesn't "jive"
with the
> > structure required by MET.
> >
> > I've only heard of and briefly used R and am not at all proficient
with
> it,
> > nor would I say I have adequate knowledge of it to use it for the
purpose
> > you have mentioned. However, I think I can get help from other
CAPS
> > scientists who have verified the same precip forecasts using MODE
> > previously for my purposes. So I will probably revert to working
with
> them
> > for now.
> >
> > I'll make sure to ask if I have any other questions, but you can
probably
> > close this ticket. Thanks.
> >
> > Jeff
> >
> > On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via
RT<met_help at ucar.edu
> >wrote:
> >
> >> Jeff,
> >>
> >> There is a sample R script that may be of interest to you if you
decide
> to
> >> format your HDF5 data into the MET internal
> >> NetCDF format. The script is used to convert gridded text TRMM
data
> with
> >> a known map projection (lat/long). You can
> >> download it from the MET website:
> >> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
> >>
> >> Also, we are just curious, what is the source of your HDF5 data?
If it
> is
> >> an emerging and/or common data format, we may
> >> consider adding support for it into MET.
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >>
> >> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> >>> Jeff,
> >>>
> >>> I was not suggesting that you process your data using
pcp_combine, only
> >> that you look at the format of the NetCDF output
> >>> from that tool. That format is one of the NetCDF gridded data
formats
> >> understood by mode, and so you must coerce your
> >>> data into that same format. Does that make sense?
> >>>
> >>> I only mentioned p_interp because it is another NetCDF format of
> gridded
> >> data understood by MET tools, but I think it
> >>> would be easier for you to use the MET internal format, unless
your
> >> model data is WRF output. In that case, I would
> >>> recommend the WRF Unified Post-Processor (UPP), anyway. The
bottom line
> >> is don't worry about p_interp.
> >>>
> >>> Paul
> >>>
> >>>
> >>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> >>>>
> >>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>>
> >>>> Paul,
> >>>> I don't need to run pcp_combine since my files already contain
the
> >>>> precipitation on the interval I want. Also, I don't see this
p_interp
> >> tool
> >>>> anywhere in the package or the documentation. What is it and
how do I
> >> use
> >>>> it?
> >>>>
> >>>> Jeff
> >>>>
> >>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> >> met_help at ucar.edu>wrote:
> >>>>
> >>>>> Jeff,
> >>>>>
> >>>>> MET is capable of reading gridded NetCDF data, but only for
two
> >> specific
> >>>>> formats. NetCDF files can have arbitrary
> >>>>> structure including the number of dimensions of the variables
> contained
> >>>>> therein. The two formats read by MET are
> >>>>> 2-dimensional grids that have certain attributes and
dimensions. One
> is
> >>>>> the output of the WRF post-processing tool
> >>>>> p_interp and the other is a format used by MET internally.
> >>>>>
> >>>>> You can see the structure of the MET internal NetCDF file
format by
> >> using
> >>>>> ncdump on the output of the pcp_combine tool
> >>>>> that appears after you run the test scripts. For example, look
at
> >>>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
> >>>>> $MET_BASE is the installation folder of MET on your
> >>>>> system.
> >>>>>
> >>>>> In general, it is not an easy task to convert data into this
file
> >> format,
> >>>>> but it has been done. In the DTC, we use the
> >>>>> R statistical computing language with the ncdf library to
perform
> this
> >>>>> type of conversion. It is hard to give you more
> >>>>> specific advice without knowing anything about your model and
obs
> data.
> >>>>> If your data consists of one or more
> >>>>> geographical planes with a known map projection, then it
should be
> >>>>> possible to perform this conversion, but it may take
> >>>>> a lot of work to do so.
> >>>>>
> >>>>> Please let me know if you have any questions.
> >>>>>
> >>>>> Paul
> >>>>>
> >>>>>
> >>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> >>>>>>
> >>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> >>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
> >>>>>> Queue: met_help
> >>>>>> Subject: Fwd: problems running MODE v3.1
> >>>>>> Owner: Nobody
> >>>>>> Requestors: jeffduda319 at gmail.com
> >>>>>> Status: new
> >>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
> >
> >>>>>>
> >>>>>>
> >>>>>> Hello,
> >>>>>> I'm trying to run mode from the version 3.1 MET package. I
have to
> >>>>> convert
> >>>>>> some HDF files to netCDF first. I do that, then attempt to
run MODE
> >>>>> using
> >>>>>> a command like
> >>>>>>
> >>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> >> pcp_6hr_OBS.netacc06_108000.01
> >>>>>> MODE_config_6hr -outdir MODE_output
> >>>>>>
> >>>>>> However, I get this output/error:
> >>>>>> DEBUG 1: Default Config File:
> >>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> >>>>>> DEBUG 1: Match Config File: MODE_config_6hr
> >>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
> >>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> >>>>>> NetCDF: Attribute not found
> >>>>>>
> >>>>>> The config file is attached.
> >>>>>> Here is the output from ncdump -h on the forecast and
observed data
> >> sets:
> >>>>>>
> >>>>>> -*FCST*
> >>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> >>>>>> dimensions:
> >>>>>> x = 1163 ;
> >>>>>> y = 723 ;
> >>>>>> z = 1 ;
> >>>>>> variables:
> >>>>>> float acc06_(z, y, x) ;
> >>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>> acc06_:long_name = "acc06_" ;
> >>>>>> acc06_:units = "mm" ;
> >>>>>> }
> >>>>>>
> >>>>>> -*OBS*
> >>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_OBS.netacc06_108000.01
> >>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> >>>>>> dimensions:
> >>>>>> x = 1163 ;
> >>>>>> y = 723 ;
> >>>>>> z = 1 ;
> >>>>>> variables:
> >>>>>> float acc06_(z, y, x) ;
> >>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>> acc06_:long_name = "acc06_" ;
> >>>>>> acc06_:units = "mm" ;
> >>>>>> }
> >>>>>>
> >>>>>> Jeff Duda
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >
> >
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Wed Apr 11 14:24:31 2012
Jeff,
Can you please upload the model data files that you are passing to
mode to the RAL FTP site using the instructions in
the URL below? I'll take a look.
Thanks,
Paul
On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>
> Paul,
> When trying to convert instead to GRIB format, I get this error when
> running MODE:
>
> [jdduda at sooner2 METv3.1]$ bin/mode pcp_6hr_ARW_cn.grbacc06_021600
> pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
> ERROR :
> ERROR : VarInfoFactory::new_var_info() -> unsupported gridded data
file
> type "0"
> ERROR :
>
> Jeff
>
> On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via
RT<met_help at ucar.edu>wrote:
>
>> Jeff,
>>
>> Have you tried using the GRIB version of your data with MET? That
should
>> be supported by mode. If you still have
>> trouble with GRIB, can you please upload your data file(s) and
config file
>> to our FTP site using these instructions:
>>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> Thanks for the detailed information about your data. I have
actually
>> verified the CAPS ensemble as part of the HWT
>> project myself using MET, although I have not yet tried to do so
with this
>> year's data.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>
>>> Paul,
>>> My data is coming from the WRF, but on a very large domain and at
4 km
>>> resolution so as to require extreme compression for long-term
storage.
>>> Specifically I'm verifying precip forecasts from the Center for
Analysis
>>> and Prediction of Storms' (CAPS at the University of Oklahoma)
storm
>> scale
>>> ensemble forecast system run as a part of the NOAA Hazardous
Weather
>>> Testbed spring experiment conducted annually. Other people at
CAPS are
>>> using other tools to convert the WRF history dumps to other
formats for
>>> compression and storage, and those compressed files are what I
have
>>> available to me. I have conversion utilities to use, but
apparently the
>>> structure of the resulting GRIB and netCDF files doesn't "jive"
with the
>>> structure required by MET.
>>>
>>> I've only heard of and briefly used R and am not at all proficient
with
>> it,
>>> nor would I say I have adequate knowledge of it to use it for the
purpose
>>> you have mentioned. However, I think I can get help from other
CAPS
>>> scientists who have verified the same precip forecasts using MODE
>>> previously for my purposes. So I will probably revert to working
with
>> them
>>> for now.
>>>
>>> I'll make sure to ask if I have any other questions, but you can
probably
>>> close this ticket. Thanks.
>>>
>>> Jeff
>>>
>>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via
RT<met_help at ucar.edu
>>> wrote:
>>>
>>>> Jeff,
>>>>
>>>> There is a sample R script that may be of interest to you if you
decide
>> to
>>>> format your HDF5 data into the MET internal
>>>> NetCDF format. The script is used to convert gridded text TRMM
data
>> with
>>>> a known map projection (lat/long). You can
>>>> download it from the MET website:
>>>> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
>>>>
>>>> Also, we are just curious, what is the source of your HDF5 data?
If it
>> is
>>>> an emerging and/or common data format, we may
>>>> consider adding support for it into MET.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
>>>>> Jeff,
>>>>>
>>>>> I was not suggesting that you process your data using
pcp_combine, only
>>>> that you look at the format of the NetCDF output
>>>>> from that tool. That format is one of the NetCDF gridded data
formats
>>>> understood by mode, and so you must coerce your
>>>>> data into that same format. Does that make sense?
>>>>>
>>>>> I only mentioned p_interp because it is another NetCDF format of
>> gridded
>>>> data understood by MET tools, but I think it
>>>>> would be easier for you to use the MET internal format, unless
your
>>>> model data is WRF output. In that case, I would
>>>>> recommend the WRF Unified Post-Processor (UPP), anyway. The
bottom line
>>>> is don't worry about p_interp.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>>>
>>>>>> Paul,
>>>>>> I don't need to run pcp_combine since my files already contain
the
>>>>>> precipitation on the interval I want. Also, I don't see this
p_interp
>>>> tool
>>>>>> anywhere in the package or the documentation. What is it and
how do I
>>>> use
>>>>>> it?
>>>>>>
>>>>>> Jeff
>>>>>>
>>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
>>>> met_help at ucar.edu>wrote:
>>>>>>
>>>>>>> Jeff,
>>>>>>>
>>>>>>> MET is capable of reading gridded NetCDF data, but only for
two
>>>> specific
>>>>>>> formats. NetCDF files can have arbitrary
>>>>>>> structure including the number of dimensions of the variables
>> contained
>>>>>>> therein. The two formats read by MET are
>>>>>>> 2-dimensional grids that have certain attributes and
dimensions. One
>> is
>>>>>>> the output of the WRF post-processing tool
>>>>>>> p_interp and the other is a format used by MET internally.
>>>>>>>
>>>>>>> You can see the structure of the MET internal NetCDF file
format by
>>>> using
>>>>>>> ncdump on the output of the pcp_combine tool
>>>>>>> that appears after you run the test scripts. For example, look
at
>>>>>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
>>>>>>> $MET_BASE is the installation folder of MET on your
>>>>>>> system.
>>>>>>>
>>>>>>> In general, it is not an easy task to convert data into this
file
>>>> format,
>>>>>>> but it has been done. In the DTC, we use the
>>>>>>> R statistical computing language with the ncdf library to
perform
>> this
>>>>>>> type of conversion. It is hard to give you more
>>>>>>> specific advice without knowing anything about your model and
obs
>> data.
>>>>>>> If your data consists of one or more
>>>>>>> geographical planes with a known map projection, then it
should be
>>>>>>> possible to perform this conversion, but it may take
>>>>>>> a lot of work to do so.
>>>>>>>
>>>>>>> Please let me know if you have any questions.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>>>>>>>>
>>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
>>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
>>>>>>>> Queue: met_help
>>>>>>>> Subject: Fwd: problems running MODE v3.1
>>>>>>>> Owner: Nobody
>>>>>>>> Requestors: jeffduda319 at gmail.com
>>>>>>>> Status: new
>>>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>> I'm trying to run mode from the version 3.1 MET package. I
have to
>>>>>>> convert
>>>>>>>> some HDF files to netCDF first. I do that, then attempt to
run MODE
>>>>>>> using
>>>>>>>> a command like
>>>>>>>>
>>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
>>>> pcp_6hr_OBS.netacc06_108000.01
>>>>>>>> MODE_config_6hr -outdir MODE_output
>>>>>>>>
>>>>>>>> However, I get this output/error:
>>>>>>>> DEBUG 1: Default Config File:
>>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
>>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
>>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
>>>>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
>>>>>>>> NetCDF: Attribute not found
>>>>>>>>
>>>>>>>> The config file is attached.
>>>>>>>> Here is the output from ncdump -h on the forecast and
observed data
>>>> sets:
>>>>>>>>
>>>>>>>> -*FCST*
>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
>> pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
>>>>>>>> dimensions:
>>>>>>>> x = 1163 ;
>>>>>>>> y = 723 ;
>>>>>>>> z = 1 ;
>>>>>>>> variables:
>>>>>>>> float acc06_(z, y, x) ;
>>>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>>>> acc06_:long_name = "acc06_" ;
>>>>>>>> acc06_:units = "mm" ;
>>>>>>>> }
>>>>>>>>
>>>>>>>> -*OBS*
>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
pcp_6hr_OBS.netacc06_108000.01
>>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
>>>>>>>> dimensions:
>>>>>>>> x = 1163 ;
>>>>>>>> y = 723 ;
>>>>>>>> z = 1 ;
>>>>>>>> variables:
>>>>>>>> float acc06_(z, y, x) ;
>>>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>>>> acc06_:long_name = "acc06_" ;
>>>>>>>> acc06_:units = "mm" ;
>>>>>>>> }
>>>>>>>>
>>>>>>>> Jeff Duda
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Wed Apr 11 15:00:48 2012
Paul,
I've uploaded both GRIB and netCDF versions of the fcst and observed
precip
files, as well as a config file (use MODE_config_6hr). Note that the
config file is set so that the variable to verify is from the GRIB
file.
If you want to run it for the netCDF files, you'll have to change the
name
of the fcst and obs fields in the config file accordingly.
Jeff
On Wed, Apr 11, 2012 at 3:24 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> Can you please upload the model data files that you are passing to
mode to
> the RAL FTP site using the instructions in
> the URL below? I'll take a look.
>
> Thanks,
>
> Paul
>
>
> On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >
> > Paul,
> > When trying to convert instead to GRIB format, I get this error
when
> > running MODE:
> >
> > [jdduda at sooner2 METv3.1]$ bin/mode pcp_6hr_ARW_cn.grbacc06_021600
> > pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
> > ERROR :
> > ERROR : VarInfoFactory::new_var_info() -> unsupported gridded
data file
> > type "0"
> > ERROR :
> >
> > Jeff
> >
> > On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via
RT<met_help at ucar.edu
> >wrote:
> >
> >> Jeff,
> >>
> >> Have you tried using the GRIB version of your data with MET?
That
> should
> >> be supported by mode. If you still have
> >> trouble with GRIB, can you please upload your data file(s) and
config
> file
> >> to our FTP site using these instructions:
> >>
> >> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>
> >> Thanks for the detailed information about your data. I have
actually
> >> verified the CAPS ensemble as part of the HWT
> >> project myself using MET, although I have not yet tried to do so
with
> this
> >> year's data.
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >>
> >> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
> >>>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>
> >>> Paul,
> >>> My data is coming from the WRF, but on a very large domain and
at 4 km
> >>> resolution so as to require extreme compression for long-term
storage.
> >>> Specifically I'm verifying precip forecasts from the Center for
> Analysis
> >>> and Prediction of Storms' (CAPS at the University of Oklahoma)
storm
> >> scale
> >>> ensemble forecast system run as a part of the NOAA Hazardous
Weather
> >>> Testbed spring experiment conducted annually. Other people at
CAPS are
> >>> using other tools to convert the WRF history dumps to other
formats for
> >>> compression and storage, and those compressed files are what I
have
> >>> available to me. I have conversion utilities to use, but
apparently
> the
> >>> structure of the resulting GRIB and netCDF files doesn't "jive"
with
> the
> >>> structure required by MET.
> >>>
> >>> I've only heard of and briefly used R and am not at all
proficient with
> >> it,
> >>> nor would I say I have adequate knowledge of it to use it for
the
> purpose
> >>> you have mentioned. However, I think I can get help from other
CAPS
> >>> scientists who have verified the same precip forecasts using
MODE
> >>> previously for my purposes. So I will probably revert to
working with
> >> them
> >>> for now.
> >>>
> >>> I'll make sure to ask if I have any other questions, but you can
> probably
> >>> close this ticket. Thanks.
> >>>
> >>> Jeff
> >>>
> >>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT<
> met_help at ucar.edu
> >>> wrote:
> >>>
> >>>> Jeff,
> >>>>
> >>>> There is a sample R script that may be of interest to you if
you
> decide
> >> to
> >>>> format your HDF5 data into the MET internal
> >>>> NetCDF format. The script is used to convert gridded text TRMM
data
> >> with
> >>>> a known map projection (lat/long). You can
> >>>> download it from the MET website:
> >>>> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
> >>>>
> >>>> Also, we are just curious, what is the source of your HDF5
data? If
> it
> >> is
> >>>> an emerging and/or common data format, we may
> >>>> consider adding support for it into MET.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Paul
> >>>>
> >>>>
> >>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> >>>>> Jeff,
> >>>>>
> >>>>> I was not suggesting that you process your data using
pcp_combine,
> only
> >>>> that you look at the format of the NetCDF output
> >>>>> from that tool. That format is one of the NetCDF gridded data
formats
> >>>> understood by mode, and so you must coerce your
> >>>>> data into that same format. Does that make sense?
> >>>>>
> >>>>> I only mentioned p_interp because it is another NetCDF format
of
> >> gridded
> >>>> data understood by MET tools, but I think it
> >>>>> would be easier for you to use the MET internal format, unless
your
> >>>> model data is WRF output. In that case, I would
> >>>>> recommend the WRF Unified Post-Processor (UPP), anyway. The
bottom
> line
> >>>> is don't worry about p_interp.
> >>>>>
> >>>>> Paul
> >>>>>
> >>>>>
> >>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> >>>>>>
> >>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>>>>
> >>>>>> Paul,
> >>>>>> I don't need to run pcp_combine since my files already
contain the
> >>>>>> precipitation on the interval I want. Also, I don't see this
> p_interp
> >>>> tool
> >>>>>> anywhere in the package or the documentation. What is it and
how do
> I
> >>>> use
> >>>>>> it?
> >>>>>>
> >>>>>> Jeff
> >>>>>>
> >>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> >>>> met_help at ucar.edu>wrote:
> >>>>>>
> >>>>>>> Jeff,
> >>>>>>>
> >>>>>>> MET is capable of reading gridded NetCDF data, but only for
two
> >>>> specific
> >>>>>>> formats. NetCDF files can have arbitrary
> >>>>>>> structure including the number of dimensions of the
variables
> >> contained
> >>>>>>> therein. The two formats read by MET are
> >>>>>>> 2-dimensional grids that have certain attributes and
dimensions.
> One
> >> is
> >>>>>>> the output of the WRF post-processing tool
> >>>>>>> p_interp and the other is a format used by MET internally.
> >>>>>>>
> >>>>>>> You can see the structure of the MET internal NetCDF file
format by
> >>>> using
> >>>>>>> ncdump on the output of the pcp_combine tool
> >>>>>>> that appears after you run the test scripts. For example,
look at
> >>>>>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
> >>>>>>> $MET_BASE is the installation folder of MET on your
> >>>>>>> system.
> >>>>>>>
> >>>>>>> In general, it is not an easy task to convert data into this
file
> >>>> format,
> >>>>>>> but it has been done. In the DTC, we use the
> >>>>>>> R statistical computing language with the ncdf library to
perform
> >> this
> >>>>>>> type of conversion. It is hard to give you more
> >>>>>>> specific advice without knowing anything about your model
and obs
> >> data.
> >>>>>>> If your data consists of one or more
> >>>>>>> geographical planes with a known map projection, then it
should be
> >>>>>>> possible to perform this conversion, but it may take
> >>>>>>> a lot of work to do so.
> >>>>>>>
> >>>>>>> Please let me know if you have any questions.
> >>>>>>>
> >>>>>>> Paul
> >>>>>>>
> >>>>>>>
> >>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> >>>>>>>>
> >>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> >>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
> >>>>>>>> Queue: met_help
> >>>>>>>> Subject: Fwd: problems running MODE v3.1
> >>>>>>>> Owner: Nobody
> >>>>>>>> Requestors: jeffduda319 at gmail.com
> >>>>>>>> Status: new
> >>>>>>>> Ticket<URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
> >>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Hello,
> >>>>>>>> I'm trying to run mode from the version 3.1 MET package. I
have to
> >>>>>>> convert
> >>>>>>>> some HDF files to netCDF first. I do that, then attempt to
run
> MODE
> >>>>>>> using
> >>>>>>>> a command like
> >>>>>>>>
> >>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> >>>> pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>> MODE_config_6hr -outdir MODE_output
> >>>>>>>>
> >>>>>>>> However, I get this output/error:
> >>>>>>>> DEBUG 1: Default Config File:
> >>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> >>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
> >>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
> >>>>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>> NetCDF: Attribute not found
> >>>>>>>>
> >>>>>>>> The config file is attached.
> >>>>>>>> Here is the output from ncdump -h on the forecast and
observed
> data
> >>>> sets:
> >>>>>>>>
> >>>>>>>> -*FCST*
> >>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> >> pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> >>>>>>>> dimensions:
> >>>>>>>> x = 1163 ;
> >>>>>>>> y = 723 ;
> >>>>>>>> z = 1 ;
> >>>>>>>> variables:
> >>>>>>>> float acc06_(z, y, x) ;
> >>>>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>>>> acc06_:long_name = "acc06_" ;
> >>>>>>>> acc06_:units = "mm" ;
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> -*OBS*
> >>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> >>>>>>>> dimensions:
> >>>>>>>> x = 1163 ;
> >>>>>>>> y = 723 ;
> >>>>>>>> z = 1 ;
> >>>>>>>> variables:
> >>>>>>>> float acc06_(z, y, x) ;
> >>>>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>>>> acc06_:long_name = "acc06_" ;
> >>>>>>>> acc06_:units = "mm" ;
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> Jeff Duda
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >>
> >
> >
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55783] Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Wed Apr 11 15:19:15 2012
Jeff,
I think part of the problem with the GRIB data you sent me might be an
encoding issue. I tried to use the standard NCEP
wgrib tool to list out the records present in the GRIB file, but it
gives the following error:
[pgoldenb at orval 20120411.duda]$ wgrib pcp_6hr_ARW_cn.grbacc06_021600
Len of grib message is inconsistent.
missing end section
0 0 0 0
Are you able to list the records in that GRIB file using wgrib? If
so, perhaps the file is being corrupted by the FTP
transfer. Would you mind putting all the files you uploaded into a
tarball and then uploading that? That should
preserve the binary files and help avoid corruption. Sorry for the
trouble.
Paul
On 04/11/2012 03:00 PM, Jeff Duda via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>
> Paul,
> I've uploaded both GRIB and netCDF versions of the fcst and observed
precip
> files, as well as a config file (use MODE_config_6hr). Note that
the
> config file is set so that the variable to verify is from the GRIB
file.
> If you want to run it for the netCDF files, you'll have to change
the name
> of the fcst and obs fields in the config file accordingly.
>
> Jeff
>
> On Wed, Apr 11, 2012 at 3:24 PM, Paul Oldenburg via
RT<met_help at ucar.edu>wrote:
>
>> Jeff,
>>
>> Can you please upload the model data files that you are passing to
mode to
>> the RAL FTP site using the instructions in
>> the URL below? I'll take a look.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>
>>> Paul,
>>> When trying to convert instead to GRIB format, I get this error
when
>>> running MODE:
>>>
>>> [jdduda at sooner2 METv3.1]$ bin/mode pcp_6hr_ARW_cn.grbacc06_021600
>>> pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
>>> ERROR :
>>> ERROR : VarInfoFactory::new_var_info() -> unsupported gridded
data file
>>> type "0"
>>> ERROR :
>>>
>>> Jeff
>>>
>>> On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via
RT<met_help at ucar.edu
>>> wrote:
>>>
>>>> Jeff,
>>>>
>>>> Have you tried using the GRIB version of your data with MET?
That
>> should
>>>> be supported by mode. If you still have
>>>> trouble with GRIB, can you please upload your data file(s) and
config
>> file
>>>> to our FTP site using these instructions:
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> Thanks for the detailed information about your data. I have
actually
>>>> verified the CAPS ensemble as part of the HWT
>>>> project myself using MET, although I have not yet tried to do so
with
>> this
>>>> year's data.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>>
>>>>> Paul,
>>>>> My data is coming from the WRF, but on a very large domain and
at 4 km
>>>>> resolution so as to require extreme compression for long-term
storage.
>>>>> Specifically I'm verifying precip forecasts from the Center for
>> Analysis
>>>>> and Prediction of Storms' (CAPS at the University of Oklahoma)
storm
>>>> scale
>>>>> ensemble forecast system run as a part of the NOAA Hazardous
Weather
>>>>> Testbed spring experiment conducted annually. Other people at
CAPS are
>>>>> using other tools to convert the WRF history dumps to other
formats for
>>>>> compression and storage, and those compressed files are what I
have
>>>>> available to me. I have conversion utilities to use, but
apparently
>> the
>>>>> structure of the resulting GRIB and netCDF files doesn't "jive"
with
>> the
>>>>> structure required by MET.
>>>>>
>>>>> I've only heard of and briefly used R and am not at all
proficient with
>>>> it,
>>>>> nor would I say I have adequate knowledge of it to use it for
the
>> purpose
>>>>> you have mentioned. However, I think I can get help from other
CAPS
>>>>> scientists who have verified the same precip forecasts using
MODE
>>>>> previously for my purposes. So I will probably revert to
working with
>>>> them
>>>>> for now.
>>>>>
>>>>> I'll make sure to ask if I have any other questions, but you can
>> probably
>>>>> close this ticket. Thanks.
>>>>>
>>>>> Jeff
>>>>>
>>>>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT<
>> met_help at ucar.edu
>>>>> wrote:
>>>>>
>>>>>> Jeff,
>>>>>>
>>>>>> There is a sample R script that may be of interest to you if
you
>> decide
>>>> to
>>>>>> format your HDF5 data into the MET internal
>>>>>> NetCDF format. The script is used to convert gridded text TRMM
data
>>>> with
>>>>>> a known map projection (lat/long). You can
>>>>>> download it from the MET website:
>>>>>> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
>>>>>>
>>>>>> Also, we are just curious, what is the source of your HDF5
data? If
>> it
>>>> is
>>>>>> an emerging and/or common data format, we may
>>>>>> consider adding support for it into MET.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
>>>>>>> Jeff,
>>>>>>>
>>>>>>> I was not suggesting that you process your data using
pcp_combine,
>> only
>>>>>> that you look at the format of the NetCDF output
>>>>>>> from that tool. That format is one of the NetCDF gridded data
formats
>>>>>> understood by mode, and so you must coerce your
>>>>>>> data into that same format. Does that make sense?
>>>>>>>
>>>>>>> I only mentioned p_interp because it is another NetCDF format
of
>>>> gridded
>>>>>> data understood by MET tools, but I think it
>>>>>>> would be easier for you to use the MET internal format, unless
your
>>>>>> model data is WRF output. In that case, I would
>>>>>>> recommend the WRF Unified Post-Processor (UPP), anyway. The
bottom
>> line
>>>>>> is don't worry about p_interp.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
>>>>>>>>
>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>> I don't need to run pcp_combine since my files already
contain the
>>>>>>>> precipitation on the interval I want. Also, I don't see this
>> p_interp
>>>>>> tool
>>>>>>>> anywhere in the package or the documentation. What is it and
how do
>> I
>>>>>> use
>>>>>>>> it?
>>>>>>>>
>>>>>>>> Jeff
>>>>>>>>
>>>>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
>>>>>> met_help at ucar.edu>wrote:
>>>>>>>>
>>>>>>>>> Jeff,
>>>>>>>>>
>>>>>>>>> MET is capable of reading gridded NetCDF data, but only for
two
>>>>>> specific
>>>>>>>>> formats. NetCDF files can have arbitrary
>>>>>>>>> structure including the number of dimensions of the
variables
>>>> contained
>>>>>>>>> therein. The two formats read by MET are
>>>>>>>>> 2-dimensional grids that have certain attributes and
dimensions.
>> One
>>>> is
>>>>>>>>> the output of the WRF post-processing tool
>>>>>>>>> p_interp and the other is a format used by MET internally.
>>>>>>>>>
>>>>>>>>> You can see the structure of the MET internal NetCDF file
format by
>>>>>> using
>>>>>>>>> ncdump on the output of the pcp_combine tool
>>>>>>>>> that appears after you run the test scripts. For example,
look at
>>>>>>>>> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
where
>>>>>>>>> $MET_BASE is the installation folder of MET on your
>>>>>>>>> system.
>>>>>>>>>
>>>>>>>>> In general, it is not an easy task to convert data into this
file
>>>>>> format,
>>>>>>>>> but it has been done. In the DTC, we use the
>>>>>>>>> R statistical computing language with the ncdf library to
perform
>>>> this
>>>>>>>>> type of conversion. It is hard to give you more
>>>>>>>>> specific advice without knowing anything about your model
and obs
>>>> data.
>>>>>>>>> If your data consists of one or more
>>>>>>>>> geographical planes with a known map projection, then it
should be
>>>>>>>>> possible to perform this conversion, but it may take
>>>>>>>>> a lot of work to do so.
>>>>>>>>>
>>>>>>>>> Please let me know if you have any questions.
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
>>>>>>>>>>
>>>>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
>>>>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
>>>>>>>>>> Queue: met_help
>>>>>>>>>> Subject: Fwd: problems running MODE v3.1
>>>>>>>>>> Owner: Nobody
>>>>>>>>>> Requestors: jeffduda319 at gmail.com
>>>>>>>>>> Status: new
>>>>>>>>>> Ticket<URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>> I'm trying to run mode from the version 3.1 MET package. I
have to
>>>>>>>>> convert
>>>>>>>>>> some HDF files to netCDF first. I do that, then attempt to
run
>> MODE
>>>>>>>>> using
>>>>>>>>>> a command like
>>>>>>>>>>
>>>>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>> pcp_6hr_OBS.netacc06_108000.01
>>>>>>>>>> MODE_config_6hr -outdir MODE_output
>>>>>>>>>>
>>>>>>>>>> However, I get this output/error:
>>>>>>>>>> DEBUG 1: Default Config File:
>>>>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
>>>>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
>>>>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
>>>>>>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
>>>>>>>>>> NetCDF: Attribute not found
>>>>>>>>>>
>>>>>>>>>> The config file is attached.
>>>>>>>>>> Here is the output from ncdump -h on the forecast and
observed
>> data
>>>>>> sets:
>>>>>>>>>>
>>>>>>>>>> -*FCST*
>>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
>>>> pcp_6hr_ARW_cn.netacc06_108000.01
>>>>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
>>>>>>>>>> dimensions:
>>>>>>>>>> x = 1163 ;
>>>>>>>>>> y = 723 ;
>>>>>>>>>> z = 1 ;
>>>>>>>>>> variables:
>>>>>>>>>> float acc06_(z, y, x) ;
>>>>>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>>>>>> acc06_:long_name = "acc06_" ;
>>>>>>>>>> acc06_:units = "mm" ;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> -*OBS*
>>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
>> pcp_6hr_OBS.netacc06_108000.01
>>>>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
>>>>>>>>>> dimensions:
>>>>>>>>>> x = 1163 ;
>>>>>>>>>> y = 723 ;
>>>>>>>>>> z = 1 ;
>>>>>>>>>> variables:
>>>>>>>>>> float acc06_(z, y, x) ;
>>>>>>>>>> acc06_:standard_name = "acc06_" ;
>>>>>>>>>> acc06_:long_name = "acc06_" ;
>>>>>>>>>> acc06_:units = "mm" ;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Jeff Duda
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Wed Apr 11 15:43:19 2012
Paul,
Heh, that's on my end. Something is wrong with the conversion utility
I'm
using. I'll look into that and get back to you.
Jeff
On Wed, Apr 11, 2012 at 4:19 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> I think part of the problem with the GRIB data you sent me might be
an
> encoding issue. I tried to use the standard NCEP
> wgrib tool to list out the records present in the GRIB file, but it
gives
> the following error:
>
> [pgoldenb at orval 20120411.duda]$ wgrib pcp_6hr_ARW_cn.grbacc06_021600
> Len of grib message is inconsistent.
> missing end section
> 0 0 0 0
>
> Are you able to list the records in that GRIB file using wgrib? If
so,
> perhaps the file is being corrupted by the FTP
> transfer. Would you mind putting all the files you uploaded into a
> tarball and then uploading that? That should
> preserve the binary files and help avoid corruption. Sorry for the
> trouble.
>
> Paul
>
>
> On 04/11/2012 03:00 PM, Jeff Duda via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >
> > Paul,
> > I've uploaded both GRIB and netCDF versions of the fcst and
observed
> precip
> > files, as well as a config file (use MODE_config_6hr). Note that
the
> > config file is set so that the variable to verify is from the GRIB
file.
> > If you want to run it for the netCDF files, you'll have to change
the
> name
> > of the fcst and obs fields in the config file accordingly.
> >
> > Jeff
> >
> > On Wed, Apr 11, 2012 at 3:24 PM, Paul Oldenburg via
RT<met_help at ucar.edu
> >wrote:
> >
> >> Jeff,
> >>
> >> Can you please upload the model data files that you are passing
to mode
> to
> >> the RAL FTP site using the instructions in
> >> the URL below? I'll take a look.
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >>
> >> On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
> >>>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>
> >>> Paul,
> >>> When trying to convert instead to GRIB format, I get this error
when
> >>> running MODE:
> >>>
> >>> [jdduda at sooner2 METv3.1]$ bin/mode
pcp_6hr_ARW_cn.grbacc06_021600
> >>> pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
> >>> ERROR :
> >>> ERROR : VarInfoFactory::new_var_info() -> unsupported gridded
data
> file
> >>> type "0"
> >>> ERROR :
> >>>
> >>> Jeff
> >>>
> >>> On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via RT<
> met_help at ucar.edu
> >>> wrote:
> >>>
> >>>> Jeff,
> >>>>
> >>>> Have you tried using the GRIB version of your data with MET?
That
> >> should
> >>>> be supported by mode. If you still have
> >>>> trouble with GRIB, can you please upload your data file(s) and
config
> >> file
> >>>> to our FTP site using these instructions:
> >>>>
> >>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>>>
> >>>> Thanks for the detailed information about your data. I have
actually
> >>>> verified the CAPS ensemble as part of the HWT
> >>>> project myself using MET, although I have not yet tried to do
so with
> >> this
> >>>> year's data.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Paul
> >>>>
> >>>>
> >>>> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
> >>>>>
> >>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>>>
> >>>>> Paul,
> >>>>> My data is coming from the WRF, but on a very large domain and
at 4
> km
> >>>>> resolution so as to require extreme compression for long-term
> storage.
> >>>>> Specifically I'm verifying precip forecasts from the Center
for
> >> Analysis
> >>>>> and Prediction of Storms' (CAPS at the University of Oklahoma)
storm
> >>>> scale
> >>>>> ensemble forecast system run as a part of the NOAA Hazardous
Weather
> >>>>> Testbed spring experiment conducted annually. Other people at
CAPS
> are
> >>>>> using other tools to convert the WRF history dumps to other
formats
> for
> >>>>> compression and storage, and those compressed files are what I
have
> >>>>> available to me. I have conversion utilities to use, but
apparently
> >> the
> >>>>> structure of the resulting GRIB and netCDF files doesn't
"jive" with
> >> the
> >>>>> structure required by MET.
> >>>>>
> >>>>> I've only heard of and briefly used R and am not at all
proficient
> with
> >>>> it,
> >>>>> nor would I say I have adequate knowledge of it to use it for
the
> >> purpose
> >>>>> you have mentioned. However, I think I can get help from
other CAPS
> >>>>> scientists who have verified the same precip forecasts using
MODE
> >>>>> previously for my purposes. So I will probably revert to
working
> with
> >>>> them
> >>>>> for now.
> >>>>>
> >>>>> I'll make sure to ask if I have any other questions, but you
can
> >> probably
> >>>>> close this ticket. Thanks.
> >>>>>
> >>>>> Jeff
> >>>>>
> >>>>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT<
> >> met_help at ucar.edu
> >>>>> wrote:
> >>>>>
> >>>>>> Jeff,
> >>>>>>
> >>>>>> There is a sample R script that may be of interest to you if
you
> >> decide
> >>>> to
> >>>>>> format your HDF5 data into the MET internal
> >>>>>> NetCDF format. The script is used to convert gridded text
TRMM data
> >>>> with
> >>>>>> a known map projection (lat/long). You can
> >>>>>> download it from the MET website:
> >>>>>>
http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
> >>>>>>
> >>>>>> Also, we are just curious, what is the source of your HDF5
data? If
> >> it
> >>>> is
> >>>>>> an emerging and/or common data format, we may
> >>>>>> consider adding support for it into MET.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Paul
> >>>>>>
> >>>>>>
> >>>>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> >>>>>>> Jeff,
> >>>>>>>
> >>>>>>> I was not suggesting that you process your data using
pcp_combine,
> >> only
> >>>>>> that you look at the format of the NetCDF output
> >>>>>>> from that tool. That format is one of the NetCDF gridded
data
> formats
> >>>>>> understood by mode, and so you must coerce your
> >>>>>>> data into that same format. Does that make sense?
> >>>>>>>
> >>>>>>> I only mentioned p_interp because it is another NetCDF
format of
> >>>> gridded
> >>>>>> data understood by MET tools, but I think it
> >>>>>>> would be easier for you to use the MET internal format,
unless your
> >>>>>> model data is WRF output. In that case, I would
> >>>>>>> recommend the WRF Unified Post-Processor (UPP), anyway. The
bottom
> >> line
> >>>>>> is don't worry about p_interp.
> >>>>>>>
> >>>>>>> Paul
> >>>>>>>
> >>>>>>>
> >>>>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> >>>>>>>>
> >>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> >>>>>>>>
> >>>>>>>> Paul,
> >>>>>>>> I don't need to run pcp_combine since my files already
contain the
> >>>>>>>> precipitation on the interval I want. Also, I don't see
this
> >> p_interp
> >>>>>> tool
> >>>>>>>> anywhere in the package or the documentation. What is it
and how
> do
> >> I
> >>>>>> use
> >>>>>>>> it?
> >>>>>>>>
> >>>>>>>> Jeff
> >>>>>>>>
> >>>>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> >>>>>> met_help at ucar.edu>wrote:
> >>>>>>>>
> >>>>>>>>> Jeff,
> >>>>>>>>>
> >>>>>>>>> MET is capable of reading gridded NetCDF data, but only
for two
> >>>>>> specific
> >>>>>>>>> formats. NetCDF files can have arbitrary
> >>>>>>>>> structure including the number of dimensions of the
variables
> >>>> contained
> >>>>>>>>> therein. The two formats read by MET are
> >>>>>>>>> 2-dimensional grids that have certain attributes and
dimensions.
> >> One
> >>>> is
> >>>>>>>>> the output of the WRF post-processing tool
> >>>>>>>>> p_interp and the other is a format used by MET internally.
> >>>>>>>>>
> >>>>>>>>> You can see the structure of the MET internal NetCDF file
format
> by
> >>>>>> using
> >>>>>>>>> ncdump on the output of the pcp_combine tool
> >>>>>>>>> that appears after you run the test scripts. For example,
look at
> >>>>>>>>>
$MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
> where
> >>>>>>>>> $MET_BASE is the installation folder of MET on your
> >>>>>>>>> system.
> >>>>>>>>>
> >>>>>>>>> In general, it is not an easy task to convert data into
this file
> >>>>>> format,
> >>>>>>>>> but it has been done. In the DTC, we use the
> >>>>>>>>> R statistical computing language with the ncdf library to
perform
> >>>> this
> >>>>>>>>> type of conversion. It is hard to give you more
> >>>>>>>>> specific advice without knowing anything about your model
and obs
> >>>> data.
> >>>>>>>>> If your data consists of one or more
> >>>>>>>>> geographical planes with a known map projection, then it
should
> be
> >>>>>>>>> possible to perform this conversion, but it may take
> >>>>>>>>> a lot of work to do so.
> >>>>>>>>>
> >>>>>>>>> Please let me know if you have any questions.
> >>>>>>>>>
> >>>>>>>>> Paul
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> >>>>>>>>>>
> >>>>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> >>>>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
> >>>>>>>>>> Queue: met_help
> >>>>>>>>>> Subject: Fwd: problems running MODE v3.1
> >>>>>>>>>> Owner: Nobody
> >>>>>>>>>> Requestors: jeffduda319 at gmail.com
> >>>>>>>>>> Status: new
> >>>>>>>>>> Ticket<URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
> >>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Hello,
> >>>>>>>>>> I'm trying to run mode from the version 3.1 MET package.
I have
> to
> >>>>>>>>> convert
> >>>>>>>>>> some HDF files to netCDF first. I do that, then attempt
to run
> >> MODE
> >>>>>>>>> using
> >>>>>>>>>> a command like
> >>>>>>>>>>
> >>>>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>> pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>>>> MODE_config_6hr -outdir MODE_output
> >>>>>>>>>>
> >>>>>>>>>> However, I get this output/error:
> >>>>>>>>>> DEBUG 1: Default Config File:
> >>>>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> >>>>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
> >>>>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
> >>>>>>>>>> DEBUG 1: Forecast File: pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>>>>>> DEBUG 1: Observation File: pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>>>> NetCDF: Attribute not found
> >>>>>>>>>>
> >>>>>>>>>> The config file is attached.
> >>>>>>>>>> Here is the output from ncdump -h on the forecast and
observed
> >> data
> >>>>>> sets:
> >>>>>>>>>>
> >>>>>>>>>> -*FCST*
> >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> >>>> pcp_6hr_ARW_cn.netacc06_108000.01
> >>>>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> >>>>>>>>>> dimensions:
> >>>>>>>>>> x = 1163 ;
> >>>>>>>>>> y = 723 ;
> >>>>>>>>>> z = 1 ;
> >>>>>>>>>> variables:
> >>>>>>>>>> float acc06_(z, y, x) ;
> >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>>>>>> acc06_:long_name = "acc06_" ;
> >>>>>>>>>> acc06_:units = "mm" ;
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>> -*OBS*
> >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> >> pcp_6hr_OBS.netacc06_108000.01
> >>>>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> >>>>>>>>>> dimensions:
> >>>>>>>>>> x = 1163 ;
> >>>>>>>>>> y = 723 ;
> >>>>>>>>>> z = 1 ;
> >>>>>>>>>> variables:
> >>>>>>>>>> float acc06_(z, y, x) ;
> >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> >>>>>>>>>> acc06_:long_name = "acc06_" ;
> >>>>>>>>>> acc06_:units = "mm" ;
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>> Jeff Duda
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >>
> >
> >
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Paul Oldenburg
Time: Tue Apr 24 12:14:34 2012
Jeff,
Do you need any more assistance on this issue at this time, or can I
resolve this ticket? If you want to take up this matter again, just
reply to this email and the ticket will re-open.
Thanks,
Paul
On Wed Apr 11 15:43:19 2012, jeffduda319 at gmail.com wrote:
> Paul,
> Heh, that's on my end. Something is wrong with the conversion
utility
> I'm
> using. I'll look into that and get back to you.
>
> Jeff
>
> On Wed, Apr 11, 2012 at 4:19 PM, Paul Oldenburg via RT
> <met_help at ucar.edu>wrote:
>
> > Jeff,
> >
> > I think part of the problem with the GRIB data you sent me might
be
> an
> > encoding issue. I tried to use the standard NCEP
> > wgrib tool to list out the records present in the GRIB file, but
it
> gives
> > the following error:
> >
> > [pgoldenb at orval 20120411.duda]$ wgrib
pcp_6hr_ARW_cn.grbacc06_021600
> > Len of grib message is inconsistent.
> > missing end section
> > 0 0 0 0
> >
> > Are you able to list the records in that GRIB file using wgrib?
If
> so,
> > perhaps the file is being corrupted by the FTP
> > transfer. Would you mind putting all the files you uploaded into
a
> > tarball and then uploading that? That should
> > preserve the binary files and help avoid corruption. Sorry for
the
> > trouble.
> >
> > Paul
> >
> >
> > On 04/11/2012 03:00 PM, Jeff Duda via RT wrote:
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > >
> > > Paul,
> > > I've uploaded both GRIB and netCDF versions of the fcst and
> observed
> > precip
> > > files, as well as a config file (use MODE_config_6hr). Note
that
> the
> > > config file is set so that the variable to verify is from the
GRIB
> file.
> > > If you want to run it for the netCDF files, you'll have to
change
> the
> > name
> > > of the fcst and obs fields in the config file accordingly.
> > >
> > > Jeff
> > >
> > > On Wed, Apr 11, 2012 at 3:24 PM, Paul Oldenburg via
> RT<met_help at ucar.edu
> > >wrote:
> > >
> > >> Jeff,
> > >>
> > >> Can you please upload the model data files that you are passing
> to mode
> > to
> > >> the RAL FTP site using the instructions in
> > >> the URL below? I'll take a look.
> > >>
> > >> Thanks,
> > >>
> > >> Paul
> > >>
> > >>
> > >> On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
> > >>>
> > >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > >>>
> > >>> Paul,
> > >>> When trying to convert instead to GRIB format, I get this
error
> when
> > >>> running MODE:
> > >>>
> > >>> [jdduda at sooner2 METv3.1]$ bin/mode
> pcp_6hr_ARW_cn.grbacc06_021600
> > >>> pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
> > >>> ERROR :
> > >>> ERROR : VarInfoFactory::new_var_info() -> unsupported
gridded
> data
> > file
> > >>> type "0"
> > >>> ERROR :
> > >>>
> > >>> Jeff
> > >>>
> > >>> On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via RT<
> > met_help at ucar.edu
> > >>> wrote:
> > >>>
> > >>>> Jeff,
> > >>>>
> > >>>> Have you tried using the GRIB version of your data with MET?
> That
> > >> should
> > >>>> be supported by mode. If you still have
> > >>>> trouble with GRIB, can you please upload your data file(s)
and
> config
> > >> file
> > >>>> to our FTP site using these instructions:
> > >>>>
> > >>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >>>>
> > >>>> Thanks for the detailed information about your data. I have
> actually
> > >>>> verified the CAPS ensemble as part of the HWT
> > >>>> project myself using MET, although I have not yet tried to do
> so with
> > >> this
> > >>>> year's data.
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Paul
> > >>>>
> > >>>>
> > >>>> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
> > >>>>>
> > >>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > >>>>>
> > >>>>> Paul,
> > >>>>> My data is coming from the WRF, but on a very large domain
and
> at 4
> > km
> > >>>>> resolution so as to require extreme compression for long-
term
> > storage.
> > >>>>> Specifically I'm verifying precip forecasts from the Center
> for
> > >> Analysis
> > >>>>> and Prediction of Storms' (CAPS at the University of
Oklahoma)
> storm
> > >>>> scale
> > >>>>> ensemble forecast system run as a part of the NOAA Hazardous
> Weather
> > >>>>> Testbed spring experiment conducted annually. Other people
at
> CAPS
> > are
> > >>>>> using other tools to convert the WRF history dumps to other
> formats
> > for
> > >>>>> compression and storage, and those compressed files are what
I
> have
> > >>>>> available to me. I have conversion utilities to use, but
> apparently
> > >> the
> > >>>>> structure of the resulting GRIB and netCDF files doesn't
> "jive" with
> > >> the
> > >>>>> structure required by MET.
> > >>>>>
> > >>>>> I've only heard of and briefly used R and am not at all
> proficient
> > with
> > >>>> it,
> > >>>>> nor would I say I have adequate knowledge of it to use it
for
> the
> > >> purpose
> > >>>>> you have mentioned. However, I think I can get help from
> other CAPS
> > >>>>> scientists who have verified the same precip forecasts using
> MODE
> > >>>>> previously for my purposes. So I will probably revert to
> working
> > with
> > >>>> them
> > >>>>> for now.
> > >>>>>
> > >>>>> I'll make sure to ask if I have any other questions, but you
> can
> > >> probably
> > >>>>> close this ticket. Thanks.
> > >>>>>
> > >>>>> Jeff
> > >>>>>
> > >>>>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT<
> > >> met_help at ucar.edu
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Jeff,
> > >>>>>>
> > >>>>>> There is a sample R script that may be of interest to you
if
> you
> > >> decide
> > >>>> to
> > >>>>>> format your HDF5 data into the MET internal
> > >>>>>> NetCDF format. The script is used to convert gridded text
> TRMM data
> > >>>> with
> > >>>>>> a known map projection (lat/long). You can
> > >>>>>> download it from the MET website:
> > >>>>>>
> http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
> > >>>>>>
> > >>>>>> Also, we are just curious, what is the source of your HDF5
> data? If
> > >> it
> > >>>> is
> > >>>>>> an emerging and/or common data format, we may
> > >>>>>> consider adding support for it into MET.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Paul
> > >>>>>>
> > >>>>>>
> > >>>>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> > >>>>>>> Jeff,
> > >>>>>>>
> > >>>>>>> I was not suggesting that you process your data using
> pcp_combine,
> > >> only
> > >>>>>> that you look at the format of the NetCDF output
> > >>>>>>> from that tool. That format is one of the NetCDF gridded
> data
> > formats
> > >>>>>> understood by mode, and so you must coerce your
> > >>>>>>> data into that same format. Does that make sense?
> > >>>>>>>
> > >>>>>>> I only mentioned p_interp because it is another NetCDF
> format of
> > >>>> gridded
> > >>>>>> data understood by MET tools, but I think it
> > >>>>>>> would be easier for you to use the MET internal format,
> unless your
> > >>>>>> model data is WRF output. In that case, I would
> > >>>>>>> recommend the WRF Unified Post-Processor (UPP), anyway.
The
> bottom
> > >> line
> > >>>>>> is don't worry about p_interp.
> > >>>>>>>
> > >>>>>>> Paul
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> > >>>>>>>>
> > >>>>>>>> <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > >>>>>>>>
> > >>>>>>>> Paul,
> > >>>>>>>> I don't need to run pcp_combine since my files already
> contain the
> > >>>>>>>> precipitation on the interval I want. Also, I don't see
> this
> > >> p_interp
> > >>>>>> tool
> > >>>>>>>> anywhere in the package or the documentation. What is it
> and how
> > do
> > >> I
> > >>>>>> use
> > >>>>>>>> it?
> > >>>>>>>>
> > >>>>>>>> Jeff
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> > >>>>>> met_help at ucar.edu>wrote:
> > >>>>>>>>
> > >>>>>>>>> Jeff,
> > >>>>>>>>>
> > >>>>>>>>> MET is capable of reading gridded NetCDF data, but only
> for two
> > >>>>>> specific
> > >>>>>>>>> formats. NetCDF files can have arbitrary
> > >>>>>>>>> structure including the number of dimensions of the
> variables
> > >>>> contained
> > >>>>>>>>> therein. The two formats read by MET are
> > >>>>>>>>> 2-dimensional grids that have certain attributes and
> dimensions.
> > >> One
> > >>>> is
> > >>>>>>>>> the output of the WRF post-processing tool
> > >>>>>>>>> p_interp and the other is a format used by MET
internally.
> > >>>>>>>>>
> > >>>>>>>>> You can see the structure of the MET internal NetCDF
file
> format
> > by
> > >>>>>> using
> > >>>>>>>>> ncdump on the output of the pcp_combine tool
> > >>>>>>>>> that appears after you run the test scripts. For
example,
> look at
> > >>>>>>>>>
> $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
> > where
> > >>>>>>>>> $MET_BASE is the installation folder of MET on your
> > >>>>>>>>> system.
> > >>>>>>>>>
> > >>>>>>>>> In general, it is not an easy task to convert data into
> this file
> > >>>>>> format,
> > >>>>>>>>> but it has been done. In the DTC, we use the
> > >>>>>>>>> R statistical computing language with the ncdf library
to
> perform
> > >>>> this
> > >>>>>>>>> type of conversion. It is hard to give you more
> > >>>>>>>>> specific advice without knowing anything about your
model
> and obs
> > >>>> data.
> > >>>>>>>>> If your data consists of one or more
> > >>>>>>>>> geographical planes with a known map projection, then it
> should
> > be
> > >>>>>>>>> possible to perform this conversion, but it may take
> > >>>>>>>>> a lot of work to do so.
> > >>>>>>>>>
> > >>>>>>>>> Please let me know if you have any questions.
> > >>>>>>>>>
> > >>>>>>>>> Paul
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted upon.
> > >>>>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
> > >>>>>>>>>> Queue: met_help
> > >>>>>>>>>> Subject: Fwd: problems running MODE v3.1
> > >>>>>>>>>> Owner: Nobody
> > >>>>>>>>>> Requestors: jeffduda319 at gmail.com
> > >>>>>>>>>> Status: new
> > >>>>>>>>>> Ticket<URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
> > >>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Hello,
> > >>>>>>>>>> I'm trying to run mode from the version 3.1 MET
package.
> I have
> > to
> > >>>>>>>>> convert
> > >>>>>>>>>> some HDF files to netCDF first. I do that, then attempt
> to run
> > >> MODE
> > >>>>>>>>> using
> > >>>>>>>>>> a command like
> > >>>>>>>>>>
> > >>>>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> > >>>>>> pcp_6hr_OBS.netacc06_108000.01
> > >>>>>>>>>> MODE_config_6hr -outdir MODE_output
> > >>>>>>>>>>
> > >>>>>>>>>> However, I get this output/error:
> > >>>>>>>>>> DEBUG 1: Default Config File:
> > >>>>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> > >>>>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
> > >>>>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
> > >>>>>>>>>> DEBUG 1: Forecast File:
pcp_6hr_ARW_cn.netacc06_108000.01
> > >>>>>>>>>> DEBUG 1: Observation File:
pcp_6hr_OBS.netacc06_108000.01
> > >>>>>>>>>> NetCDF: Attribute not found
> > >>>>>>>>>>
> > >>>>>>>>>> The config file is attached.
> > >>>>>>>>>> Here is the output from ncdump -h on the forecast and
> observed
> > >> data
> > >>>>>> sets:
> > >>>>>>>>>>
> > >>>>>>>>>> -*FCST*
> > >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> > >>>> pcp_6hr_ARW_cn.netacc06_108000.01
> > >>>>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> > >>>>>>>>>> dimensions:
> > >>>>>>>>>> x = 1163 ;
> > >>>>>>>>>> y = 723 ;
> > >>>>>>>>>> z = 1 ;
> > >>>>>>>>>> variables:
> > >>>>>>>>>> float acc06_(z, y, x) ;
> > >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> > >>>>>>>>>> acc06_:long_name = "acc06_" ;
> > >>>>>>>>>> acc06_:units = "mm" ;
> > >>>>>>>>>> }
> > >>>>>>>>>>
> > >>>>>>>>>> -*OBS*
> > >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> > >> pcp_6hr_OBS.netacc06_108000.01
> > >>>>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> > >>>>>>>>>> dimensions:
> > >>>>>>>>>> x = 1163 ;
> > >>>>>>>>>> y = 723 ;
> > >>>>>>>>>> z = 1 ;
> > >>>>>>>>>> variables:
> > >>>>>>>>>> float acc06_(z, y, x) ;
> > >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> > >>>>>>>>>> acc06_:long_name = "acc06_" ;
> > >>>>>>>>>> acc06_:units = "mm" ;
> > >>>>>>>>>> }
> > >>>>>>>>>>
> > >>>>>>>>>> Jeff Duda
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >
> > >
> >
> >
> >
>
>
------------------------------------------------
Subject: Fwd: problems running MODE v3.1
From: Jeff Duda
Time: Tue Apr 24 12:16:46 2012
Paul,
Go ahead and resolve it for now. I will most likely reopen it later.
Thanks.
Jeff
On Tue, Apr 24, 2012 at 1:14 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:
> Jeff,
>
> Do you need any more assistance on this issue at this time, or can I
> resolve this ticket? If you want to take up this matter again, just
reply
> to this email and the ticket will re-open.
>
> Thanks,
>
> Paul
>
>
> On Wed Apr 11 15:43:19 2012, jeffduda319 at gmail.com wrote:
> > Paul,
> > Heh, that's on my end. Something is wrong with the conversion
utility
> > I'm
> > using. I'll look into that and get back to you.
> >
> > Jeff
> >
> > On Wed, Apr 11, 2012 at 4:19 PM, Paul Oldenburg via RT
> > <met_help at ucar.edu>wrote:
> >
> > > Jeff,
> > >
> > > I think part of the problem with the GRIB data you sent me might
be
> > an
> > > encoding issue. I tried to use the standard NCEP
> > > wgrib tool to list out the records present in the GRIB file, but
it
> > gives
> > > the following error:
> > >
> > > [pgoldenb at orval 20120411.duda]$ wgrib
pcp_6hr_ARW_cn.grbacc06_021600
> > > Len of grib message is inconsistent.
> > > missing end section
> > > 0 0 0 0
> > >
> > > Are you able to list the records in that GRIB file using wgrib?
If
> > so,
> > > perhaps the file is being corrupted by the FTP
> > > transfer. Would you mind putting all the files you uploaded
into a
> > > tarball and then uploading that? That should
> > > preserve the binary files and help avoid corruption. Sorry for
the
> > > trouble.
> > >
> > > Paul
> > >
> > >
> > > On 04/11/2012 03:00 PM, Jeff Duda via RT wrote:
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > > >
> > > > Paul,
> > > > I've uploaded both GRIB and netCDF versions of the fcst and
> > observed
> > > precip
> > > > files, as well as a config file (use MODE_config_6hr). Note
that
> > the
> > > > config file is set so that the variable to verify is from the
GRIB
> > file.
> > > > If you want to run it for the netCDF files, you'll have to
change
> > the
> > > name
> > > > of the fcst and obs fields in the config file accordingly.
> > > >
> > > > Jeff
> > > >
> > > > On Wed, Apr 11, 2012 at 3:24 PM, Paul Oldenburg via
> > RT<met_help at ucar.edu
> > > >wrote:
> > > >
> > > >> Jeff,
> > > >>
> > > >> Can you please upload the model data files that you are
passing
> > to mode
> > > to
> > > >> the RAL FTP site using the instructions in
> > > >> the URL below? I'll take a look.
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Paul
> > > >>
> > > >>
> > > >> On 04/11/2012 01:57 PM, Jeff Duda via RT wrote:
> > > >>>
> > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > > >>>
> > > >>> Paul,
> > > >>> When trying to convert instead to GRIB format, I get this
error
> > when
> > > >>> running MODE:
> > > >>>
> > > >>> [jdduda at sooner2 METv3.1]$ bin/mode
> > pcp_6hr_ARW_cn.grbacc06_021600
> > > >>> pcp_6hr_OBS.grbacc06_021600 MODE_config_6hr
> > > >>> ERROR :
> > > >>> ERROR : VarInfoFactory::new_var_info() -> unsupported
gridded
> > data
> > > file
> > > >>> type "0"
> > > >>> ERROR :
> > > >>>
> > > >>> Jeff
> > > >>>
> > > >>> On Wed, Apr 11, 2012 at 2:40 PM, Paul Oldenburg via RT<
> > > met_help at ucar.edu
> > > >>> wrote:
> > > >>>
> > > >>>> Jeff,
> > > >>>>
> > > >>>> Have you tried using the GRIB version of your data with
MET?
> > That
> > > >> should
> > > >>>> be supported by mode. If you still have
> > > >>>> trouble with GRIB, can you please upload your data file(s)
and
> > config
> > > >> file
> > > >>>> to our FTP site using these instructions:
> > > >>>>
> > > >>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >>>>
> > > >>>> Thanks for the detailed information about your data. I
have
> > actually
> > > >>>> verified the CAPS ensemble as part of the HWT
> > > >>>> project myself using MET, although I have not yet tried to
do
> > so with
> > > >> this
> > > >>>> year's data.
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Paul
> > > >>>>
> > > >>>>
> > > >>>> On 04/11/2012 01:35 PM, Jeff Duda via RT wrote:
> > > >>>>>
> > > >>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > > >>>>>
> > > >>>>> Paul,
> > > >>>>> My data is coming from the WRF, but on a very large domain
and
> > at 4
> > > km
> > > >>>>> resolution so as to require extreme compression for long-
term
> > > storage.
> > > >>>>> Specifically I'm verifying precip forecasts from the
Center
> > for
> > > >> Analysis
> > > >>>>> and Prediction of Storms' (CAPS at the University of
Oklahoma)
> > storm
> > > >>>> scale
> > > >>>>> ensemble forecast system run as a part of the NOAA
Hazardous
> > Weather
> > > >>>>> Testbed spring experiment conducted annually. Other
people at
> > CAPS
> > > are
> > > >>>>> using other tools to convert the WRF history dumps to
other
> > formats
> > > for
> > > >>>>> compression and storage, and those compressed files are
what I
> > have
> > > >>>>> available to me. I have conversion utilities to use, but
> > apparently
> > > >> the
> > > >>>>> structure of the resulting GRIB and netCDF files doesn't
> > "jive" with
> > > >> the
> > > >>>>> structure required by MET.
> > > >>>>>
> > > >>>>> I've only heard of and briefly used R and am not at all
> > proficient
> > > with
> > > >>>> it,
> > > >>>>> nor would I say I have adequate knowledge of it to use it
for
> > the
> > > >> purpose
> > > >>>>> you have mentioned. However, I think I can get help from
> > other CAPS
> > > >>>>> scientists who have verified the same precip forecasts
using
> > MODE
> > > >>>>> previously for my purposes. So I will probably revert to
> > working
> > > with
> > > >>>> them
> > > >>>>> for now.
> > > >>>>>
> > > >>>>> I'll make sure to ask if I have any other questions, but
you
> > can
> > > >> probably
> > > >>>>> close this ticket. Thanks.
> > > >>>>>
> > > >>>>> Jeff
> > > >>>>>
> > > >>>>> On Wed, Apr 11, 2012 at 2:19 PM, Paul Oldenburg via RT<
> > > >> met_help at ucar.edu
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> Jeff,
> > > >>>>>>
> > > >>>>>> There is a sample R script that may be of interest to you
if
> > you
> > > >> decide
> > > >>>> to
> > > >>>>>> format your HDF5 data into the MET internal
> > > >>>>>> NetCDF format. The script is used to convert gridded
text
> > TRMM data
> > > >>>> with
> > > >>>>>> a known map projection (lat/long). You can
> > > >>>>>> download it from the MET website:
> > > >>>>>>
> > http://www.dtcenter.org/met/users/downloads/Rscripts/trmm2nc.R
> > > >>>>>>
> > > >>>>>> Also, we are just curious, what is the source of your
HDF5
> > data? If
> > > >> it
> > > >>>> is
> > > >>>>>> an emerging and/or common data format, we may
> > > >>>>>> consider adding support for it into MET.
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>>
> > > >>>>>> Paul
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On 04/11/2012 10:57 AM, Paul Oldenburg wrote:
> > > >>>>>>> Jeff,
> > > >>>>>>>
> > > >>>>>>> I was not suggesting that you process your data using
> > pcp_combine,
> > > >> only
> > > >>>>>> that you look at the format of the NetCDF output
> > > >>>>>>> from that tool. That format is one of the NetCDF gridded
> > data
> > > formats
> > > >>>>>> understood by mode, and so you must coerce your
> > > >>>>>>> data into that same format. Does that make sense?
> > > >>>>>>>
> > > >>>>>>> I only mentioned p_interp because it is another NetCDF
> > format of
> > > >>>> gridded
> > > >>>>>> data understood by MET tools, but I think it
> > > >>>>>>> would be easier for you to use the MET internal format,
> > unless your
> > > >>>>>> model data is WRF output. In that case, I would
> > > >>>>>>> recommend the WRF Unified Post-Processor (UPP), anyway.
The
> > bottom
> > > >> line
> > > >>>>>> is don't worry about p_interp.
> > > >>>>>>>
> > > >>>>>>> Paul
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On 04/11/2012 10:44 AM, Jeff Duda via RT wrote:
> > > >>>>>>>>
> > > >>>>>>>> <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783>
> > > >>>>>>>>
> > > >>>>>>>> Paul,
> > > >>>>>>>> I don't need to run pcp_combine since my files already
> > contain the
> > > >>>>>>>> precipitation on the interval I want. Also, I don't see
> > this
> > > >> p_interp
> > > >>>>>> tool
> > > >>>>>>>> anywhere in the package or the documentation. What is
it
> > and how
> > > do
> > > >> I
> > > >>>>>> use
> > > >>>>>>>> it?
> > > >>>>>>>>
> > > >>>>>>>> Jeff
> > > >>>>>>>>
> > > >>>>>>>> On Tue, Apr 10, 2012 at 5:57 PM, Paul Oldenburg via RT<
> > > >>>>>> met_help at ucar.edu>wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Jeff,
> > > >>>>>>>>>
> > > >>>>>>>>> MET is capable of reading gridded NetCDF data, but
only
> > for two
> > > >>>>>> specific
> > > >>>>>>>>> formats. NetCDF files can have arbitrary
> > > >>>>>>>>> structure including the number of dimensions of the
> > variables
> > > >>>> contained
> > > >>>>>>>>> therein. The two formats read by MET are
> > > >>>>>>>>> 2-dimensional grids that have certain attributes and
> > dimensions.
> > > >> One
> > > >>>> is
> > > >>>>>>>>> the output of the WRF post-processing tool
> > > >>>>>>>>> p_interp and the other is a format used by MET
internally.
> > > >>>>>>>>>
> > > >>>>>>>>> You can see the structure of the MET internal NetCDF
file
> > format
> > > by
> > > >>>>>> using
> > > >>>>>>>>> ncdump on the output of the pcp_combine tool
> > > >>>>>>>>> that appears after you run the test scripts. For
example,
> > look at
> > > >>>>>>>>>
> > $MET_BASE/out/pcp_combine/sample_fcst_24L_2005080800V_24A.nc
> > > where
> > > >>>>>>>>> $MET_BASE is the installation folder of MET on your
> > > >>>>>>>>> system.
> > > >>>>>>>>>
> > > >>>>>>>>> In general, it is not an easy task to convert data
into
> > this file
> > > >>>>>> format,
> > > >>>>>>>>> but it has been done. In the DTC, we use the
> > > >>>>>>>>> R statistical computing language with the ncdf library
to
> > perform
> > > >>>> this
> > > >>>>>>>>> type of conversion. It is hard to give you more
> > > >>>>>>>>> specific advice without knowing anything about your
model
> > and obs
> > > >>>> data.
> > > >>>>>>>>> If your data consists of one or more
> > > >>>>>>>>> geographical planes with a known map projection, then
it
> > should
> > > be
> > > >>>>>>>>> possible to perform this conversion, but it may take
> > > >>>>>>>>> a lot of work to do so.
> > > >>>>>>>>>
> > > >>>>>>>>> Please let me know if you have any questions.
> > > >>>>>>>>>
> > > >>>>>>>>> Paul
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On 04/10/2012 03:51 PM, Jeff Duda via RT wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Tue Apr 10 15:51:46 2012: Request 55783 was acted
upon.
> > > >>>>>>>>>> Transaction: Ticket created by jeffduda319 at gmail.com
> > > >>>>>>>>>> Queue: met_help
> > > >>>>>>>>>> Subject: Fwd: problems running MODE v3.1
> > > >>>>>>>>>> Owner: Nobody
> > > >>>>>>>>>> Requestors: jeffduda319 at gmail.com
> > > >>>>>>>>>> Status: new
> > > >>>>>>>>>> Ticket<URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55783
> > > >>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hello,
> > > >>>>>>>>>> I'm trying to run mode from the version 3.1 MET
package.
> > I have
> > > to
> > > >>>>>>>>> convert
> > > >>>>>>>>>> some HDF files to netCDF first. I do that, then
attempt
> > to run
> > > >> MODE
> > > >>>>>>>>> using
> > > >>>>>>>>>> a command like
> > > >>>>>>>>>>
> > > >>>>>>>>>> bin/mode pcp_6hr_ARW_cn.netacc06_108000.01
> > > >>>>>> pcp_6hr_OBS.netacc06_108000.01
> > > >>>>>>>>>> MODE_config_6hr -outdir MODE_output
> > > >>>>>>>>>>
> > > >>>>>>>>>> However, I get this output/error:
> > > >>>>>>>>>> DEBUG 1: Default Config File:
> > > >>>>>>>>>> /home/jdduda/METv3.1/data/config/MODEConfig_default
> > > >>>>>>>>>> DEBUG 1: Match Config File: MODE_config_6hr
> > > >>>>>>>>>> DEBUG 1: Merge Config File: MODE_config_6hr
> > > >>>>>>>>>> DEBUG 1: Forecast File:
pcp_6hr_ARW_cn.netacc06_108000.01
> > > >>>>>>>>>> DEBUG 1: Observation File:
pcp_6hr_OBS.netacc06_108000.01
> > > >>>>>>>>>> NetCDF: Attribute not found
> > > >>>>>>>>>>
> > > >>>>>>>>>> The config file is attached.
> > > >>>>>>>>>> Here is the output from ncdump -h on the forecast and
> > observed
> > > >> data
> > > >>>>>> sets:
> > > >>>>>>>>>>
> > > >>>>>>>>>> -*FCST*
> > > >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> > > >>>> pcp_6hr_ARW_cn.netacc06_108000.01
> > > >>>>>>>>>> netcdf pcp_6hr_ARW_cn.netacc06_108000 {
> > > >>>>>>>>>> dimensions:
> > > >>>>>>>>>> x = 1163 ;
> > > >>>>>>>>>> y = 723 ;
> > > >>>>>>>>>> z = 1 ;
> > > >>>>>>>>>> variables:
> > > >>>>>>>>>> float acc06_(z, y, x) ;
> > > >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> > > >>>>>>>>>> acc06_:long_name = "acc06_" ;
> > > >>>>>>>>>> acc06_:units = "mm" ;
> > > >>>>>>>>>> }
> > > >>>>>>>>>>
> > > >>>>>>>>>> -*OBS*
> > > >>>>>>>>>> [jdduda at sooner2 METv3.1]$ ncdump -h
> > > >> pcp_6hr_OBS.netacc06_108000.01
> > > >>>>>>>>>> netcdf pcp_6hr_OBS.netacc06_108000 {
> > > >>>>>>>>>> dimensions:
> > > >>>>>>>>>> x = 1163 ;
> > > >>>>>>>>>> y = 723 ;
> > > >>>>>>>>>> z = 1 ;
> > > >>>>>>>>>> variables:
> > > >>>>>>>>>> float acc06_(z, y, x) ;
> > > >>>>>>>>>> acc06_:standard_name = "acc06_" ;
> > > >>>>>>>>>> acc06_:long_name = "acc06_" ;
> > > >>>>>>>>>> acc06_:units = "mm" ;
> > > >>>>>>>>>> }
> > > >>>>>>>>>>
> > > >>>>>>>>>> Jeff Duda
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> > >
> >
> >
>
>
>
>
--
Jeff Duda
Graduate research assistant
University of Oklahoma School of Meteorology
Center for Analysis and Prediction of Storms
------------------------------------------------
More information about the Met_help
mailing list