[Met_help] [rt.rap.ucar.edu #60404] History for Point_stat tool
John Halley Gotway via RT
met_help at ucar.edu
Fri Mar 29 09:22:45 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I have tried several things, but after each time I run point_stat it errors
with the following output.
DEBUG 1: Default Config File:
/admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
DEBUG 1: User Config File: iCpointStatConfigFile
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_NcMet".
NetCDF: Attribute not found
Please find my configuration and met-netcdf observation files attached to
this email. Question, is it having difficulty finding the attribute in
the observation-netcdf of model-netcdf file?
Thanks for your time,
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Feb 25 08:48:44 2013
Ronald,
I see that you've sent me the NetCDF point observation file, but not
the gridded model file you're trying to verify. Can you please send
me the gridded model file and the command line you're using to
call Point-Stat? That'll help me debug your situation. If it's too
big to send over email, you can post it to our anonymous ftp site
following these instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Please write me back to let me know if/when you've posted data to the
ftp site for me to grab.
Thanks,
John
On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
> Transaction: Ticket created by ronald.leeper at noaa.gov
> Queue: met_help
> Subject: Point_stat tool
> Owner: Nobody
> Requestors: ronald.leeper at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
>
> I have tried several things, but after each time I run point_stat it
errors
> with the following output.
>
> DEBUG 1: Default Config File:
> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
> DEBUG 1: User Config File: iCpointStatConfigFile
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Please find my configuration and met-netcdf observation files
attached to
> this email. Question, is it having difficulty finding the
attribute in
> the observation-netcdf of model-netcdf file?
>
> Thanks for your time,
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Mon Feb 25 11:19:37 2013
I have loaded the model netcdf file to the help-ftp under leeper_data.
Let
me know if you need anything else.
Thanks again for your help,
On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> I see that you've sent me the NetCDF point observation file, but not
the
> gridded model file you're trying to verify. Can you please send me
the
> gridded model file and the command line you're using to
> call Point-Stat? That'll help me debug your situation. If it's too
big
> to send over email, you can post it to our anonymous ftp site
following
> these instructions:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Please write me back to let me know if/when you've posted data to
the ftp
> site for me to grab.
>
> Thanks,
> John
>
> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
> >
> > Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
> > Transaction: Ticket created by ronald.leeper at noaa.gov
> > Queue: met_help
> > Subject: Point_stat tool
> > Owner: Nobody
> > Requestors: ronald.leeper at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >
> >
> > I have tried several things, but after each time I run point_stat
it
> errors
> > with the following output.
> >
> > DEBUG 1: Default Config File:
> > /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
> > DEBUG 1: User Config File: iCpointStatConfigFile
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > NetCDF: Attribute not found
> >
> > Please find my configuration and met-netcdf observation files
attached to
> > this email. Question, is it having difficulty finding the
attribute in
> > the observation-netcdf of model-netcdf file?
> >
> > Thanks for your time,
> >
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Feb 25 11:35:20 2013
Ronald,
The file you sent me is the NetCDF output directly from the WRF model,
as indicated in the NetCDF "TITLE" attribute:
:TITLE = " OUTPUT FROM WRF V3.0.1.1 MODEL" ;
MET does not verify the raw output of WRF. Instead, the WRF output
must first be post-processed using either the pinterp utility for ARW
or the Unified Post-Processor (UPP) for both ARW and NMM:
http://www.mmm.ucar.edu/wrf/OnLineTutorial/Tools/p_interp.htm
http://www.dtcenter.org/wrf-nmm/users/overview/upp_overview.php
There are two reasons we require the output to be post-processed:
(1) The WRF vertical levels are defined on native hybrid
coordinates, whereas point observations are defined at pressure
levels. The post processing tools interpolate from the native WRF
vertical
levels to pressure levels so that the forecasts and observations can
be compared.
(2) The horizontal grid used in WRF is staggered. MET only supports
regular grids. UPP regrids the data onto a regular grid. The pinterp
utility does not - so MET is only able to verify pinterp
fields defined on the non-staggered dimensions. That means, MET can't
be used to verify winds from pinterp output.
We strongly recommend using UPP whose output format is GRIB which MET
supports very well.
Hope that helps clarify.
Thanks,
John
On 02/25/2013 11:19 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> I have loaded the model netcdf file to the help-ftp under
leeper_data. Let
> me know if you need anything else.
>
> Thanks again for your help,
>
> On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> I see that you've sent me the NetCDF point observation file, but
not the
>> gridded model file you're trying to verify. Can you please send me
the
>> gridded model file and the command line you're using to
>> call Point-Stat? That'll help me debug your situation. If it's
too big
>> to send over email, you can post it to our anonymous ftp site
following
>> these instructions:
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> Please write me back to let me know if/when you've posted data to
the ftp
>> site for me to grab.
>>
>> Thanks,
>> John
>>
>> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
>>> Transaction: Ticket created by ronald.leeper at noaa.gov
>>> Queue: met_help
>>> Subject: Point_stat tool
>>> Owner: Nobody
>>> Requestors: ronald.leeper at noaa.gov
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>>
>>> I have tried several things, but after each time I run point_stat
it
>> errors
>>> with the following output.
>>>
>>> DEBUG 1: Default Config File:
>>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
>>> DEBUG 1: User Config File: iCpointStatConfigFile
>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>> Met2dDataFile object of type "FileType_NcMet".
>>> NetCDF: Attribute not found
>>>
>>> Please find my configuration and met-netcdf observation files
attached to
>>> this email. Question, is it having difficulty finding the
attribute in
>>> the observation-netcdf of model-netcdf file?
>>>
>>> Thanks for your time,
>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Thu Feb 28 05:55:52 2013
Good morning,
I have some additional challenges concerning UPP. It did not compile
the
unipost executable? In this regard, I have a couple of questions. It
seems like UPP references the locally installed WRF when compiling,
but I
don't have the same wrf-setup as my wrfout data any more. My question
is,
will this fact be an issue regardless if I have the unipost
executable?
Along the same lines, If I reinstall the older version of WRF do I
also
need the same compiler (g95 versus pgi) and compiler settings (dmpar
versus
smpar)?
With that in mind, I have provided a copy of the unipost portion of
the UPP
make-log. In bold italics is where I think the issue reside, which
seems
to be related to my local WRF settings.
===== Making all in
/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs
=====
make[2]: Entering directory
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs'
gfortran -fconvert=big-endian -fno-second-underscore -frecord-marker=4
-DLINUX -DUPPLITTLEENDIAN -fno-range-check -O3 -c
-I/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/include
mpi_fortran.f
gcc -DLINUX -DUPPLITTLEENDIAN -D_OPENMP -O3 -c
-I/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/include
mpi_c.c
ar ru libmpi.a mpi_fortran.o mpi_c.o
ar: creating libmpi.a
/bin/cp libmpi.a
/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/lib
if [ wrfmpi_stubs != "" ] ; then \
ln -sf ../lib/wrfmpi_stubs/mpif.h ../../unipost/mpif.h
; \
fi
make[2]: Leaving directory
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs'
make[1]: Leaving directory
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib'
Making all in
/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost
make[1]: Entering directory
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost'
makefile:115: warning: overriding commands for target `.F.o'
../../configure.upp:125: warning: ignoring old commands for target
`.F.o'
makefile:119: warning: overriding commands for target `.f.o'
../../configure.upp:119: warning: ignoring old commands for target
`.f.o'
*make[1]: *** No rule to make target
`../../WRFV3/frame/module_internal_header_util.mod', needed by
`wrflink'.
Stop.*
make[1]: Leaving directory
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost'
On Mon, Feb 25, 2013 at 1:35 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> The file you sent me is the NetCDF output directly from the WRF
model, as
> indicated in the NetCDF "TITLE" attribute:
> :TITLE = " OUTPUT FROM WRF V3.0.1.1 MODEL" ;
>
> MET does not verify the raw output of WRF. Instead, the WRF output
must
> first be post-processed using either the pinterp utility for ARW or
the
> Unified Post-Processor (UPP) for both ARW and NMM:
> http://www.mmm.ucar.edu/wrf/OnLineTutorial/Tools/p_interp.htm
> http://www.dtcenter.org/wrf-nmm/users/overview/upp_overview.php
>
> There are two reasons we require the output to be post-processed:
> (1) The WRF vertical levels are defined on native hybrid
coordinates,
> whereas point observations are defined at pressure levels. The post
> processing tools interpolate from the native WRF vertical
> levels to pressure levels so that the forecasts and observations can
be
> compared.
> (2) The horizontal grid used in WRF is staggered. MET only
supports
> regular grids. UPP regrids the data onto a regular grid. The
pinterp
> utility does not - so MET is only able to verify pinterp
> fields defined on the non-staggered dimensions. That means, MET
can't be
> used to verify winds from pinterp output.
>
> We strongly recommend using UPP whose output format is GRIB which
MET
> supports very well.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On 02/25/2013 11:19 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >
> > I have loaded the model netcdf file to the help-ftp under
leeper_data.
> Let
> > me know if you need anything else.
> >
> > Thanks again for your help,
> >
> > On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Ronald,
> >>
> >> I see that you've sent me the NetCDF point observation file, but
not the
> >> gridded model file you're trying to verify. Can you please send
me the
> >> gridded model file and the command line you're using to
> >> call Point-Stat? That'll help me debug your situation. If it's
too big
> >> to send over email, you can post it to our anonymous ftp site
following
> >> these instructions:
> >> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>
> >> Please write me back to let me know if/when you've posted data to
the
> ftp
> >> site for me to grab.
> >>
> >> Thanks,
> >> John
> >>
> >> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
> >>>
> >>> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
> >>> Transaction: Ticket created by ronald.leeper at noaa.gov
> >>> Queue: met_help
> >>> Subject: Point_stat tool
> >>> Owner: Nobody
> >>> Requestors: ronald.leeper at noaa.gov
> >>> Status: new
> >>> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >>>
> >>>
> >>> I have tried several things, but after each time I run
point_stat it
> >> errors
> >>> with the following output.
> >>>
> >>> DEBUG 1: Default Config File:
> >>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
> >>> DEBUG 1: User Config File: iCpointStatConfigFile
> >>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> >>> Met2dDataFile object of type "FileType_NcMet".
> >>> NetCDF: Attribute not found
> >>>
> >>> Please find my configuration and met-netcdf observation files
attached
> to
> >>> this email. Question, is it having difficulty finding the
attribute
> in
> >>> the observation-netcdf of model-netcdf file?
> >>>
> >>> Thanks for your time,
> >>>
> >>
> >>
> >
> >
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Thu Feb 28 08:32:23 2013
Ronald,
I am not equipped to answer your questions about UPP. UPP is
supported via wrfhelp at ucar.edu. Please resend your question to
wrfhelp, and hopefully then can help you out.
Thanks,
John
On 02/28/2013 05:55 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> Good morning,
>
> I have some additional challenges concerning UPP. It did not
compile the
> unipost executable? In this regard, I have a couple of questions.
It
> seems like UPP references the locally installed WRF when compiling,
but I
> don't have the same wrf-setup as my wrfout data any more. My
question is,
> will this fact be an issue regardless if I have the unipost
executable?
> Along the same lines, If I reinstall the older version of WRF do I
also
> need the same compiler (g95 versus pgi) and compiler settings (dmpar
versus
> smpar)?
>
> With that in mind, I have provided a copy of the unipost portion of
the UPP
> make-log. In bold italics is where I think the issue reside, which
seems
> to be related to my local WRF settings.
>
> ===== Making all in
>
/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs
> =====
> make[2]: Entering directory
>
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs'
> gfortran -fconvert=big-endian -fno-second-underscore
-frecord-marker=4
> -DLINUX -DUPPLITTLEENDIAN -fno-range-check -O3 -c
> -I/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/include
mpi_fortran.f
> gcc -DLINUX -DUPPLITTLEENDIAN -D_OPENMP -O3 -c
> -I/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/include
mpi_c.c
> ar ru libmpi.a mpi_fortran.o mpi_c.o
> ar: creating libmpi.a
> /bin/cp libmpi.a
/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/lib
> if [ wrfmpi_stubs != "" ] ; then \
> ln -sf ../lib/wrfmpi_stubs/mpif.h
../../unipost/mpif.h ; \
> fi
> make[2]: Leaving directory
>
`/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib/wrfmpi_stubs'
> make[1]: Leaving directory
> `/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/lib'
> Making all in
> /home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost
> make[1]: Entering directory
> `/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost'
> makefile:115: warning: overriding commands for target `.F.o'
> ../../configure.upp:125: warning: ignoring old commands for target
`.F.o'
> makefile:119: warning: overriding commands for target `.f.o'
> ../../configure.upp:119: warning: ignoring old commands for target
`.f.o'
> *make[1]: *** No rule to make target
> `../../WRFV3/frame/module_internal_header_util.mod', needed by
`wrflink'.
> Stop.*
> make[1]: Leaving directory
> `/home/ronnieleeper/initLatalBundryLyr_study/UPPV1.1/src/unipost'
>
>
>
> On Mon, Feb 25, 2013 at 1:35 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> The file you sent me is the NetCDF output directly from the WRF
model, as
>> indicated in the NetCDF "TITLE" attribute:
>> :TITLE = " OUTPUT FROM WRF V3.0.1.1 MODEL" ;
>>
>> MET does not verify the raw output of WRF. Instead, the WRF output
must
>> first be post-processed using either the pinterp utility for ARW or
the
>> Unified Post-Processor (UPP) for both ARW and NMM:
>> http://www.mmm.ucar.edu/wrf/OnLineTutorial/Tools/p_interp.htm
>> http://www.dtcenter.org/wrf-
nmm/users/overview/upp_overview.php
>>
>> There are two reasons we require the output to be post-processed:
>> (1) The WRF vertical levels are defined on native hybrid
coordinates,
>> whereas point observations are defined at pressure levels. The
post
>> processing tools interpolate from the native WRF vertical
>> levels to pressure levels so that the forecasts and observations
can be
>> compared.
>> (2) The horizontal grid used in WRF is staggered. MET only
supports
>> regular grids. UPP regrids the data onto a regular grid. The
pinterp
>> utility does not - so MET is only able to verify pinterp
>> fields defined on the non-staggered dimensions. That means, MET
can't be
>> used to verify winds from pinterp output.
>>
>> We strongly recommend using UPP whose output format is GRIB which
MET
>> supports very well.
>>
>> Hope that helps clarify.
>>
>> Thanks,
>> John
>>
>> On 02/25/2013 11:19 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>> I have loaded the model netcdf file to the help-ftp under
leeper_data.
>> Let
>>> me know if you need anything else.
>>>
>>> Thanks again for your help,
>>>
>>> On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>> Ronald,
>>>>
>>>> I see that you've sent me the NetCDF point observation file, but
not the
>>>> gridded model file you're trying to verify. Can you please send
me the
>>>> gridded model file and the command line you're using to
>>>> call Point-Stat? That'll help me debug your situation. If it's
too big
>>>> to send over email, you can post it to our anonymous ftp site
following
>>>> these instructions:
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> Please write me back to let me know if/when you've posted data to
the
>> ftp
>>>> site for me to grab.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>>>
>>>>> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
>>>>> Transaction: Ticket created by ronald.leeper at noaa.gov
>>>>> Queue: met_help
>>>>> Subject: Point_stat tool
>>>>> Owner: Nobody
>>>>> Requestors: ronald.leeper at noaa.gov
>>>>> Status: new
>>>>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>>>
>>>>>
>>>>> I have tried several things, but after each time I run
point_stat it
>>>> errors
>>>>> with the following output.
>>>>>
>>>>> DEBUG 1: Default Config File:
>>>>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
>>>>> DEBUG 1: User Config File: iCpointStatConfigFile
>>>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>>>> Met2dDataFile object of type "FileType_NcMet".
>>>>> NetCDF: Attribute not found
>>>>>
>>>>> Please find my configuration and met-netcdf observation files
attached
>> to
>>>>> this email. Question, is it having difficulty finding the
attribute
>> in
>>>>> the observation-netcdf of model-netcdf file?
>>>>>
>>>>> Thanks for your time,
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Thu Mar 07 12:13:14 2013
So I have been able to install and run UPP against my model data.
With
that said, I have a question.
UPP generates a series of output files. Can you specify a series of
files
on the command line for point_stat? If not, what is the best way to
combine the UPP output files?
Thanks,
On Mon, Feb 25, 2013 at 1:35 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> The file you sent me is the NetCDF output directly from the WRF
model, as
> indicated in the NetCDF "TITLE" attribute:
> :TITLE = " OUTPUT FROM WRF V3.0.1.1 MODEL" ;
>
> MET does not verify the raw output of WRF. Instead, the WRF output
must
> first be post-processed using either the pinterp utility for ARW or
the
> Unified Post-Processor (UPP) for both ARW and NMM:
> http://www.mmm.ucar.edu/wrf/OnLineTutorial/Tools/p_interp.htm
> http://www.dtcenter.org/wrf-nmm/users/overview/upp_overview.php
>
> There are two reasons we require the output to be post-processed:
> (1) The WRF vertical levels are defined on native hybrid
coordinates,
> whereas point observations are defined at pressure levels. The post
> processing tools interpolate from the native WRF vertical
> levels to pressure levels so that the forecasts and observations can
be
> compared.
> (2) The horizontal grid used in WRF is staggered. MET only
supports
> regular grids. UPP regrids the data onto a regular grid. The
pinterp
> utility does not - so MET is only able to verify pinterp
> fields defined on the non-staggered dimensions. That means, MET
can't be
> used to verify winds from pinterp output.
>
> We strongly recommend using UPP whose output format is GRIB which
MET
> supports very well.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On 02/25/2013 11:19 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >
> > I have loaded the model netcdf file to the help-ftp under
leeper_data.
> Let
> > me know if you need anything else.
> >
> > Thanks again for your help,
> >
> > On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Ronald,
> >>
> >> I see that you've sent me the NetCDF point observation file, but
not the
> >> gridded model file you're trying to verify. Can you please send
me the
> >> gridded model file and the command line you're using to
> >> call Point-Stat? That'll help me debug your situation. If it's
too big
> >> to send over email, you can post it to our anonymous ftp site
following
> >> these instructions:
> >> http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>
> >> Please write me back to let me know if/when you've posted data to
the
> ftp
> >> site for me to grab.
> >>
> >> Thanks,
> >> John
> >>
> >> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
> >>>
> >>> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
> >>> Transaction: Ticket created by ronald.leeper at noaa.gov
> >>> Queue: met_help
> >>> Subject: Point_stat tool
> >>> Owner: Nobody
> >>> Requestors: ronald.leeper at noaa.gov
> >>> Status: new
> >>> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >>>
> >>>
> >>> I have tried several things, but after each time I run
point_stat it
> >> errors
> >>> with the following output.
> >>>
> >>> DEBUG 1: Default Config File:
> >>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
> >>> DEBUG 1: User Config File: iCpointStatConfigFile
> >>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> >>> Met2dDataFile object of type "FileType_NcMet".
> >>> NetCDF: Attribute not found
> >>>
> >>> Please find my configuration and met-netcdf observation files
attached
> to
> >>> this email. Question, is it having difficulty finding the
attribute
> in
> >>> the observation-netcdf of model-netcdf file?
> >>>
> >>> Thanks for your time,
> >>>
> >>
> >>
> >
> >
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Thu Mar 07 12:55:10 2013
Ronald,
I'm not sure what you mean by a "series of files". But I'm guessing
you have one output file for each lead time in your simulation.
If you find it preferable, you could group all the output times into a
single GRIB file. But then you'd need to set up the configuration
files in MET to accommodate that. Since each GRIB record is
self-contained, you can literally cat multiple GRIB files together to
combine them:
cat file1.grb file2.grb > file3.grb
However, I'd suggest you keep them all separate and run each through
the MET tools to compute verification statistics for each output time.
Just let me know if you have more questions.
Thanks,
John
On 03/07/2013 12:13 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> So I have been able to install and run UPP against my model data.
With
> that said, I have a question.
>
> UPP generates a series of output files. Can you specify a series of
files
> on the command line for point_stat? If not, what is the best way to
> combine the UPP output files?
>
> Thanks,
>
> On Mon, Feb 25, 2013 at 1:35 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> The file you sent me is the NetCDF output directly from the WRF
model, as
>> indicated in the NetCDF "TITLE" attribute:
>> :TITLE = " OUTPUT FROM WRF V3.0.1.1 MODEL" ;
>>
>> MET does not verify the raw output of WRF. Instead, the WRF output
must
>> first be post-processed using either the pinterp utility for ARW or
the
>> Unified Post-Processor (UPP) for both ARW and NMM:
>> http://www.mmm.ucar.edu/wrf/OnLineTutorial/Tools/p_interp.htm
>> http://www.dtcenter.org/wrf-
nmm/users/overview/upp_overview.php
>>
>> There are two reasons we require the output to be post-processed:
>> (1) The WRF vertical levels are defined on native hybrid
coordinates,
>> whereas point observations are defined at pressure levels. The
post
>> processing tools interpolate from the native WRF vertical
>> levels to pressure levels so that the forecasts and observations
can be
>> compared.
>> (2) The horizontal grid used in WRF is staggered. MET only
supports
>> regular grids. UPP regrids the data onto a regular grid. The
pinterp
>> utility does not - so MET is only able to verify pinterp
>> fields defined on the non-staggered dimensions. That means, MET
can't be
>> used to verify winds from pinterp output.
>>
>> We strongly recommend using UPP whose output format is GRIB which
MET
>> supports very well.
>>
>> Hope that helps clarify.
>>
>> Thanks,
>> John
>>
>> On 02/25/2013 11:19 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>> I have loaded the model netcdf file to the help-ftp under
leeper_data.
>> Let
>>> me know if you need anything else.
>>>
>>> Thanks again for your help,
>>>
>>> On Mon, Feb 25, 2013 at 10:48 AM, John Halley Gotway via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>> Ronald,
>>>>
>>>> I see that you've sent me the NetCDF point observation file, but
not the
>>>> gridded model file you're trying to verify. Can you please send
me the
>>>> gridded model file and the command line you're using to
>>>> call Point-Stat? That'll help me debug your situation. If it's
too big
>>>> to send over email, you can post it to our anonymous ftp site
following
>>>> these instructions:
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> Please write me back to let me know if/when you've posted data to
the
>> ftp
>>>> site for me to grab.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 02/24/2013 08:58 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>>>
>>>>> Sun Feb 24 20:58:23 2013: Request 60404 was acted upon.
>>>>> Transaction: Ticket created by ronald.leeper at noaa.gov
>>>>> Queue: met_help
>>>>> Subject: Point_stat tool
>>>>> Owner: Nobody
>>>>> Requestors: ronald.leeper at noaa.gov
>>>>> Status: new
>>>>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>>>
>>>>>
>>>>> I have tried several things, but after each time I run
point_stat it
>>>> errors
>>>>> with the following output.
>>>>>
>>>>> DEBUG 1: Default Config File:
>>>>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
>>>>> DEBUG 1: User Config File: iCpointStatConfigFile
>>>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>>>> Met2dDataFile object of type "FileType_NcMet".
>>>>> NetCDF: Attribute not found
>>>>>
>>>>> Please find my configuration and met-netcdf observation files
attached
>> to
>>>>> this email. Question, is it having difficulty finding the
attribute
>> in
>>>>> the observation-netcdf of model-netcdf file?
>>>>>
>>>>> Thanks for your time,
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Mon Mar 11 06:01:22 2013
Good morning,
Still attempting to run point_stat and I get the following error
message.
I believe it has to do with my message_type in the config file; see
error
message below. I thought the message type in Point_stat config file
was
suppose to match the message type provided in ascii2nc tool?
DEBUG 1: Default Config File:
/admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
DEBUG 1: User Config File: ./iCpointStatConfigFile
ERROR :
ERROR : parse_conf_message_type() -> Invalid message type string
provided
(SurfaceTMP).
ERROR :
On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> ou find it preferable, you could group all the output times into a
single
> GRIB file. But then you'd need to set up the configuration files in
MET to
> accommodate t
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Mar 11 10:16:24 2013
Ronald,
Right you are! I'm very surprised this issue hasn't come up in the
past.
In our internal use of ascii2nc, we've always just made use of the
message types used in PREPBUFR files. There is one small
consideration about this I'll explain below -
In MET, the ADPSFC (obs over land) and SFCSHP (obs over water) message
types have special meaning. They are considered to be "surface
observations". When you configure Point-Stat to verify a field
at some vertical level (2-m temperature or 10-m winds for example)
against the ADPSFC or SFCSHP message types, it will match all
observations of that type regardless of the height value. Suppose for
example, in your temperature observations, the 9th column contains
height values of -9999 (missing), 1-m, 2-m, and 5-m. If those
observations have the message type ADPSFC or SFCSHP, all of them will
be used by Point-Stat. To put it succinctly, when verifying forecasts
on vertical levels against the ADPSFC or SFCSHP message types, the
observation height value is ignored.
Now, if you use some other message type, like SurfaceTMP, Point-Stat
doesn't have any special logic for that string. So it will only use
observations where the level value exactly matches the
forecast level value, 2-m in this case.
The fix for this is simple, just remove lines 250-260 from the file
METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix to the
MET website with this change. That will enable you to use
message types different from the ones used in PREPBUFR
Does that all make sense?
Thanks,
John Halley Gotway
met_help at ucar.edu
On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> Good morning,
>
> Still attempting to run point_stat and I get the following error
message.
> I believe it has to do with my message_type in the config file;
see error
> message below. I thought the message type in Point_stat config file
was
> suppose to match the message type provided in ascii2nc tool?
>
> DEBUG 1: Default Config File:
> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default
> DEBUG 1: User Config File: ./iCpointStatConfigFile
> ERROR :
> ERROR : parse_conf_message_type() -> Invalid message type string
provided
> (SurfaceTMP).
> ERROR :
>
> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT
<met_help at ucar.edu
>> wrote:
>
>> ou find it preferable, you could group all the output times into a
single
>> GRIB file. But then you'd need to set up the configuration files
in MET to
>> accommodate t
>
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Mar 11 11:06:52 2013
Ronald,
OK, I've posted a new bugfix for METv4.0. The error checking for
message type values is now removed:
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
Just follow the instructions at the top of the page to apply the
patches.
Just let us know if more questions or problems arise in your use of
MET.
Thanks,
John
On 03/11/2013 10:16 AM, John Halley Gotway wrote:
> Ronald,
>
> Right you are! I'm very surprised this issue hasn't come up in the
past.
>
> In our internal use of ascii2nc, we've always just made use of the
message types used in PREPBUFR files. There is one small
consideration about this I'll explain below -
>
> In MET, the ADPSFC (obs over land) and SFCSHP (obs over water)
message types have special meaning. They are considered to be
"surface observations". When you configure Point-Stat to verify a
> field at some vertical level (2-m temperature or 10-m winds for
example) against the ADPSFC or SFCSHP message types, it will match all
observations of that type regardless of the height value.
> Suppose for example, in your temperature observations, the 9th
column contains height values of -9999 (missing), 1-m, 2-m, and 5-m.
If those observations have the message type ADPSFC or SFCSHP,
> all of them will be used by Point-Stat. To put it succinctly, when
verifying forecasts on vertical levels against the ADPSFC or SFCSHP
message types, the observation height value is ignored.
>
> Now, if you use some other message type, like SurfaceTMP, Point-Stat
doesn't have any special logic for that string. So it will only use
observations where the level value exactly matches the
> forecast level value, 2-m in this case.
>
> The fix for this is simple, just remove lines 250-260 from the file
METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix to the
MET website with this change. That will enable you to use
> message types different from the ones used in PREPBUFR
>
> Does that all make sense?
>
> Thanks, John Halley Gotway met_help at ucar.edu
>
> On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>
>> Good morning,
>>
>> Still attempting to run point_stat and I get the following error
message. I believe it has to do with my message_type in the config
file; see error message below. I thought the message type in
>> Point_stat config file was suppose to match the message type
provided in ascii2nc tool?
>>
>> DEBUG 1: Default Config File:
/admin/pkgs/met/METv4.0/data/config/PointStatConfig_default DEBUG 1:
User Config File: ./iCpointStatConfigFile ERROR : ERROR :
parse_conf_message_type() -> Invalid
>> message type string provided (SurfaceTMP). ERROR :
>>
>> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT
<met_help at ucar.edu
>>> wrote:
>>
>>> ou find it preferable, you could group all the output times into a
single GRIB file. But then you'd need to set up the configuration
files in MET to accommodate t
>>
>>
>>
>>
>>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Wed Mar 13 13:55:28 2013
I have worked my way through several issues and the important thing
here is
that point_stat is accepting my data or at least I think. My latest
issue
results from running point_stat (see error message below).
DEBUG 1: Forecast File: ./Aug2008NARR6_4km/WRFPRS_d01.000_00
DEBUG 1: Climatology File: none
DEBUG 1: Observation File: ./KYmesoBwgTmp.nc
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 48 observations from 48 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
SurfaceTMP,
over region FULL, for interpolation method UW_MEAN(16), using 4 pairs.
DEBUG 2: Computing Categorical Statistics.
ERROR :
ERROR : SingleThresh::check() -> unexpected threshold indicator value
of 0.
ERROR :
I don't have a threshold set so I am not sure what this error is
suggesting. Google searches indicate that the error means I don't
have
matching observation points? I have attached my config file.
Let me know if you need anything else,
On Mon, Mar 11, 2013 at 1:06 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> OK, I've posted a new bugfix for METv4.0. The error checking for
message
> type values is now removed:
>
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Just follow the instructions at the top of the page to apply the
patches.
>
> Just let us know if more questions or problems arise in your use of
MET.
>
> Thanks,
> John
>
> On 03/11/2013 10:16 AM, John Halley Gotway wrote:
> > Ronald,
> >
> > Right you are! I'm very surprised this issue hasn't come up in
the past.
> >
> > In our internal use of ascii2nc, we've always just made use of the
> message types used in PREPBUFR files. There is one small
consideration
> about this I'll explain below -
> >
> > In MET, the ADPSFC (obs over land) and SFCSHP (obs over water)
message
> types have special meaning. They are considered to be "surface
> observations". When you configure Point-Stat to verify a
> > field at some vertical level (2-m temperature or 10-m winds for
example)
> against the ADPSFC or SFCSHP message types, it will match all
observations
> of that type regardless of the height value.
> > Suppose for example, in your temperature observations, the 9th
column
> contains height values of -9999 (missing), 1-m, 2-m, and 5-m. If
those
> observations have the message type ADPSFC or SFCSHP,
> > all of them will be used by Point-Stat. To put it succinctly,
when
> verifying forecasts on vertical levels against the ADPSFC or SFCSHP
message
> types, the observation height value is ignored.
> >
> > Now, if you use some other message type, like SurfaceTMP, Point-
Stat
> doesn't have any special logic for that string. So it will only use
> observations where the level value exactly matches the
> > forecast level value, 2-m in this case.
> >
> > The fix for this is simple, just remove lines 250-260 from the
file
> METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix to
the MET
> website with this change. That will enable you to use
> > message types different from the ones used in PREPBUFR
> >
> > Does that all make sense?
> >
> > Thanks, John Halley Gotway met_help at ucar.edu
> >
> > On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >>
> >> Good morning,
> >>
> >> Still attempting to run point_stat and I get the following error
> message. I believe it has to do with my message_type in the config
file;
> see error message below. I thought the message type in
> >> Point_stat config file was suppose to match the message type
provided
> in ascii2nc tool?
> >>
> >> DEBUG 1: Default Config File:
> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default DEBUG 1:
User
> Config File: ./iCpointStatConfigFile ERROR : ERROR :
> parse_conf_message_type() -> Invalid
> >> message type string provided (SurfaceTMP). ERROR :
> >>
> >> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT <
> met_help at ucar.edu
> >>> wrote:
> >>
> >>> ou find it preferable, you could group all the output times into
a
> single GRIB file. But then you'd need to set up the configuration
files in
> MET to accommodate t
> >>
> >>
> >>
> >>
> >>
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Wed Mar 13 14:57:26 2013
Ronald,
In your configuration file, you are requested the FHO output line:
fho = BOTH;
The FHO line (for Forecast, Hit, and Observation Rate) is categorical
output computed by thresholding the matched pairs. It's basically
equivalent to the 2x2 contingency table output contained in the
CTC line.
Since your config file requests that categorical computations be done,
Point-Stat is trying to retrieve the categorical thresholds that
should be used:
cat_thresh = [ NA ];
So the error is coming from the NA threshold.
If you don't really want categorical stats to be computed, just set
"fho = NONE;". If you do, you'll need to choose a better threshold,
perhaps:
cat_thresh = [ >=273.15 ];
Hope that helps.
John
On 03/13/2013 01:55 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> I have worked my way through several issues and the important thing
here is
> that point_stat is accepting my data or at least I think. My latest
issue
> results from running point_stat (see error message below).
>
> DEBUG 1: Forecast File: ./Aug2008NARR6_4km/WRFPRS_d01.000_00
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File: ./KYmesoBwgTmp.nc
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for TMP/Z2.
> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 48 observations from 48 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
SurfaceTMP,
> over region FULL, for interpolation method UW_MEAN(16), using 4
pairs.
> DEBUG 2: Computing Categorical Statistics.
> ERROR :
> ERROR : SingleThresh::check() -> unexpected threshold indicator
value of 0.
> ERROR :
>
> I don't have a threshold set so I am not sure what this error is
> suggesting. Google searches indicate that the error means I don't
have
> matching observation points? I have attached my config file.
>
> Let me know if you need anything else,
>
>
> On Mon, Mar 11, 2013 at 1:06 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> OK, I've posted a new bugfix for METv4.0. The error checking for
message
>> type values is now removed:
>>
>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>
>> Just follow the instructions at the top of the page to apply the
patches.
>>
>> Just let us know if more questions or problems arise in your use of
MET.
>>
>> Thanks,
>> John
>>
>> On 03/11/2013 10:16 AM, John Halley Gotway wrote:
>>> Ronald,
>>>
>>> Right you are! I'm very surprised this issue hasn't come up in
the past.
>>>
>>> In our internal use of ascii2nc, we've always just made use of the
>> message types used in PREPBUFR files. There is one small
consideration
>> about this I'll explain below -
>>>
>>> In MET, the ADPSFC (obs over land) and SFCSHP (obs over water)
message
>> types have special meaning. They are considered to be "surface
>> observations". When you configure Point-Stat to verify a
>>> field at some vertical level (2-m temperature or 10-m winds for
example)
>> against the ADPSFC or SFCSHP message types, it will match all
observations
>> of that type regardless of the height value.
>>> Suppose for example, in your temperature observations, the 9th
column
>> contains height values of -9999 (missing), 1-m, 2-m, and 5-m. If
those
>> observations have the message type ADPSFC or SFCSHP,
>>> all of them will be used by Point-Stat. To put it succinctly,
when
>> verifying forecasts on vertical levels against the ADPSFC or SFCSHP
message
>> types, the observation height value is ignored.
>>>
>>> Now, if you use some other message type, like SurfaceTMP, Point-
Stat
>> doesn't have any special logic for that string. So it will only
use
>> observations where the level value exactly matches the
>>> forecast level value, 2-m in this case.
>>>
>>> The fix for this is simple, just remove lines 250-260 from the
file
>> METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix to
the MET
>> website with this change. That will enable you to use
>>> message types different from the ones used in PREPBUFR
>>>
>>> Does that all make sense?
>>>
>>> Thanks, John Halley Gotway met_help at ucar.edu
>>>
>>> On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>>
>>>> Good morning,
>>>>
>>>> Still attempting to run point_stat and I get the following error
>> message. I believe it has to do with my message_type in the config
file;
>> see error message below. I thought the message type in
>>>> Point_stat config file was suppose to match the message type
provided
>> in ascii2nc tool?
>>>>
>>>> DEBUG 1: Default Config File:
>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default DEBUG
1: User
>> Config File: ./iCpointStatConfigFile ERROR : ERROR :
>> parse_conf_message_type() -> Invalid
>>>> message type string provided (SurfaceTMP). ERROR :
>>>>
>>>> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT <
>> met_help at ucar.edu
>>>>> wrote:
>>>>
>>>>> ou find it preferable, you could group all the output times into
a
>> single GRIB file. But then you'd need to set up the configuration
files in
>> MET to accommodate t
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Thu Mar 14 12:19:46 2013
That did the trick. The program is running and producing output.
I have one more question related to this topic. In an earlier email
you
told me I could cat separate unipost grib files. I would like to do
this
for the set files I have for a single simulation (one for every 30
minutes), but when I run point_stat on the concatenated file (*cat
file1
file2 ... > catFile*) it doesn't process the different timesteps of
the
model output.
See the attached .mpr file and you'll find that the observation times
iterate over time, but the modeled date time stays the same? I've
used the
command line arguments -obs_valid_beg 20080824 -obs_valid_end
20080825,
but I am curious if there are some additional arguments I need to
specify
in the configuration file?
On Wed, Mar 13, 2013 at 4:57 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> In your configuration file, you are requested the FHO output line:
> fho = BOTH;
>
> The FHO line (for Forecast, Hit, and Observation Rate) is
categorical
> output computed by thresholding the matched pairs. It's basically
> equivalent to the 2x2 contingency table output contained in the
> CTC line.
>
> Since your config file requests that categorical computations be
done,
> Point-Stat is trying to retrieve the categorical thresholds that
should be
> used:
> cat_thresh = [ NA ];
>
> So the error is coming from the NA threshold.
>
> If you don't really want categorical stats to be computed, just set
"fho =
> NONE;". If you do, you'll need to choose a better threshold,
perhaps:
> cat_thresh = [ >=273.15 ];
>
> Hope that helps.
>
> John
>
> On 03/13/2013 01:55 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >
> > I have worked my way through several issues and the important
thing here
> is
> > that point_stat is accepting my data or at least I think. My
latest
> issue
> > results from running point_stat (see error message below).
> >
> > DEBUG 1: Forecast File: ./Aug2008NARR6_4km/WRFPRS_d01.000_00
> > DEBUG 1: Climatology File: none
> > DEBUG 1: Observation File: ./KYmesoBwgTmp.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for TMP/Z2.
> > DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 48 observations from 48 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
> SurfaceTMP,
> > over region FULL, for interpolation method UW_MEAN(16), using 4
pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > ERROR :
> > ERROR : SingleThresh::check() -> unexpected threshold indicator
value
> of 0.
> > ERROR :
> >
> > I don't have a threshold set so I am not sure what this error is
> > suggesting. Google searches indicate that the error means I don't
have
> > matching observation points? I have attached my config file.
> >
> > Let me know if you need anything else,
> >
> >
> > On Mon, Mar 11, 2013 at 1:06 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Ronald,
> >>
> >> OK, I've posted a new bugfix for METv4.0. The error checking for
> message
> >> type values is now removed:
> >>
> >>
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
> >>
> >> Just follow the instructions at the top of the page to apply the
> patches.
> >>
> >> Just let us know if more questions or problems arise in your use
of MET.
> >>
> >> Thanks,
> >> John
> >>
> >> On 03/11/2013 10:16 AM, John Halley Gotway wrote:
> >>> Ronald,
> >>>
> >>> Right you are! I'm very surprised this issue hasn't come up in
the
> past.
> >>>
> >>> In our internal use of ascii2nc, we've always just made use of
the
> >> message types used in PREPBUFR files. There is one small
consideration
> >> about this I'll explain below -
> >>>
> >>> In MET, the ADPSFC (obs over land) and SFCSHP (obs over water)
message
> >> types have special meaning. They are considered to be "surface
> >> observations". When you configure Point-Stat to verify a
> >>> field at some vertical level (2-m temperature or 10-m winds for
> example)
> >> against the ADPSFC or SFCSHP message types, it will match all
> observations
> >> of that type regardless of the height value.
> >>> Suppose for example, in your temperature observations, the 9th
column
> >> contains height values of -9999 (missing), 1-m, 2-m, and 5-m. If
those
> >> observations have the message type ADPSFC or SFCSHP,
> >>> all of them will be used by Point-Stat. To put it succinctly,
when
> >> verifying forecasts on vertical levels against the ADPSFC or
SFCSHP
> message
> >> types, the observation height value is ignored.
> >>>
> >>> Now, if you use some other message type, like SurfaceTMP, Point-
Stat
> >> doesn't have any special logic for that string. So it will only
use
> >> observations where the level value exactly matches the
> >>> forecast level value, 2-m in this case.
> >>>
> >>> The fix for this is simple, just remove lines 250-260 from the
file
> >> METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix
to the
> MET
> >> website with this change. That will enable you to use
> >>> message types different from the ones used in PREPBUFR
> >>>
> >>> Does that all make sense?
> >>>
> >>> Thanks, John Halley Gotway met_help at ucar.edu
> >>>
> >>> On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
> >>>>
> >>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >>>>
> >>>> Good morning,
> >>>>
> >>>> Still attempting to run point_stat and I get the following
error
> >> message. I believe it has to do with my message_type in the
config file;
> >> see error message below. I thought the message type in
> >>>> Point_stat config file was suppose to match the message type
provided
> >> in ascii2nc tool?
> >>>>
> >>>> DEBUG 1: Default Config File:
> >> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default DEBUG
1:
> User
> >> Config File: ./iCpointStatConfigFile ERROR : ERROR :
> >> parse_conf_message_type() -> Invalid
> >>>> message type string provided (SurfaceTMP). ERROR :
> >>>>
> >>>> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT <
> >> met_help at ucar.edu
> >>>>> wrote:
> >>>>
> >>>>> ou find it preferable, you could group all the output times
into a
> >> single GRIB file. But then you'd need to set up the
configuration
> files in
> >> MET to accommodate t
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>
> >>
> >
> >
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Thu Mar 14 12:19:47 2013
VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
COV_THRESH ALPHA LINE_TYPE TOTAL INDEX OBS_SID OBS_LAT OBS_LON
OBS_LVL OBS_ELV FCST OBS CLIMO
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_060000 20080824_060000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 1 12 36.93000 -86.47000 NA 2.00000
295.89194 293.60001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_063000 20080824_063000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 2 12 36.93000 -86.47000 NA 2.00000
295.89194 293.87000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_070000 20080824_070000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 3 12 36.93000 -86.47000 NA 2.00000
295.89194 294.10001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_073000 20080824_073000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 4 12 36.93000 -86.47000 NA 2.00000
295.89194 294.10001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_080000 20080824_080000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 5 12 36.93000 -86.47000 NA 2.00000
295.89194 293.97000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_083000 20080824_083000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 6 12 36.93000 -86.47000 NA 2.00000
295.89194 294.42001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_090000 20080824_090000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 7 12 36.93000 -86.47000 NA 2.00000
295.89194 293.89001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_093000 20080824_093000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 8 12 36.93000 -86.47000 NA 2.00000
295.89194 293.91000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_100000 20080824_100000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 9 12 36.93000 -86.47000 NA 2.00000
295.89194 293.44000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_103000 20080824_103000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 10 12 36.93000 -86.47000 NA 2.00000
295.89194 293.73999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_110000 20080824_110000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 11 12 36.93000 -86.47000 NA 2.00000
295.89194 293.54999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_113000 20080824_113000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 12 12 36.93000 -86.47000 NA 2.00000
295.89194 293.38000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_120000 20080824_120000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 13 12 36.93000 -86.47000 NA 2.00000
295.89194 294.04999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_123000 20080824_123000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 14 12 36.93000 -86.47000 NA 2.00000
295.89194 294.85001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_130000 20080824_130000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 15 12 36.93000 -86.47000 NA 2.00000
295.89194 295.23999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_133000 20080824_133000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 16 12 36.93000 -86.47000 NA 2.00000
295.89194 295.35001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_140000 20080824_140000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 17 12 36.93000 -86.47000 NA 2.00000
295.89194 294.54001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_143000 20080824_143000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 18 12 36.93000 -86.47000 NA 2.00000
295.89194 292.66000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_150000 20080824_150000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 19 12 36.93000 -86.47000 NA 2.00000
295.89194 292.70999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_153000 20080824_153000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 20 12 36.93000 -86.47000 NA 2.00000
295.89194 291.75000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_160000 20080824_160000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 21 12 36.93000 -86.47000 NA 2.00000
295.89194 291.54999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_163000 20080824_163000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 22 12 36.93000 -86.47000 NA 2.00000
295.89194 291.94000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_170000 20080824_170000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 23 12 36.93000 -86.47000 NA 2.00000
295.89194 291.57999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_173000 20080824_173000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 24 12 36.93000 -86.47000 NA 2.00000
295.89194 291.54001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_180000 20080824_180000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 25 12 36.93000 -86.47000 NA 2.00000
295.89194 292.51001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_183000 20080824_183000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 26 12 36.93000 -86.47000 NA 2.00000
295.89194 293.32999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_190000 20080824_190000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 27 12 36.93000 -86.47000 NA 2.00000
295.89194 293.51999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_193000 20080824_193000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 28 12 36.93000 -86.47000 NA 2.00000
295.89194 292.72000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_200000 20080824_200000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 29 12 36.93000 -86.47000 NA 2.00000
295.89194 292.16000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_203000 20080824_203000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 30 12 36.93000 -86.47000 NA 2.00000
295.89194 292.17999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_210000 20080824_210000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 31 12 36.93000 -86.47000 NA 2.00000
295.89194 292.23001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_213000 20080824_213000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 32 12 36.93000 -86.47000 NA 2.00000
295.89194 292.09000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_220000 20080824_220000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 33 12 36.93000 -86.47000 NA 2.00000
295.89194 292.72000 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_223000 20080824_223000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 34 12 36.93000 -86.47000 NA 2.00000
295.89194 293.01001 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_230000 20080824_230000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 35 12 36.93000 -86.47000 NA 2.00000
295.89194 292.57999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080824_233000 20080824_233000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 36 12 36.93000 -86.47000 NA 2.00000
295.89194 292.79999 NA
V4.0 WRF 000000 20080824_060000 20080824_060000 000000
20080825_000000 20080825_000000 TMP Z2 TMP Z2
SurfaceTMP FULL UW_MEAN 16 NA NA NA
NA MPR 37 37 12 36.93000 -86.47000 NA 2.00000
295.89194 293.54001 NA
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Thu Mar 14 14:29:47 2013
Ronald,
You can configure MET to verify multiple times from a single file, but
it'll take an extra step. Suppose for example, you have a GRIB file
consisting of 5 records, where each is 2-meter temperature
but for 5 different output times.
Then suppose your configuration file contains the following:
fcst = {
field = [ { name = "TMP"; level = [ "Z2" ]; } ];
...
}
Point-Stat will search your input GRIB file for matching record and
then verify it. Since all the records contain 2-m temp, it'll match
only the first one and then verify it.
So how do you get around this? How can you have it verify multiple
output times in the same file?
You use additional filtering criteria concerning time. The timing
information you can specify in the config file includes "init_time",
"valid_time", and "lead_time". Since all of your output for one
simulation is contained in a single GRIB file, I'd suggest using the
lead time in HHMMSS format. Something like the following:
fcst = {
field = [ { name = "TMP"; level = [ "Z2" ]; lead_time = "003000";
},
{ name = "TMP"; level = [ "Z2" ]; lead_time = "010000";
},
{ name = "TMP"; level = [ "Z2" ]; lead_time = "013000";
},
{ name = "TMP"; level = [ "Z2" ]; lead_time = "020000";
},
{ name = "TMP"; level = [ "Z2" ]; lead_time = "023000"; }
];
...
}
The setting above will verify 2-meter temperature for all 5 GRIB
records in our example, where the lead times are every 30 minutes.
Do *NOT* use the obs_valid_beg and obs_valid_end settings on the
command line. Those explicitly set the observation matching time
window to what you specify. Instead, you want the observation time
window to be relative to the valid time for each record. The
obs_window config file settings defines a time window in seconds
relative to the valid time for each field. If you wanted to look at
observations +/- 5 minutes around the forecast valid time, you'd set
it like this:
obs_window = {
beg = -600;
end = 600;
}
Once you've set this up, I'd suggest running Point-Stat at verbosity
level 3 or 4. Then inspect the status messages. It should tell you
which GRIB record it's using for each task. Just make sure
that it's doing what you expect.
Hope that helps clarify.
Thanks,
John
On 03/14/2013 12:19 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> That did the trick. The program is running and producing output.
>
> I have one more question related to this topic. In an earlier email
you
> told me I could cat separate unipost grib files. I would like to do
this
> for the set files I have for a single simulation (one for every 30
> minutes), but when I run point_stat on the concatenated file (*cat
file1
> file2 ... > catFile*) it doesn't process the different timesteps of
the
> model output.
>
> See the attached .mpr file and you'll find that the observation
times
> iterate over time, but the modeled date time stays the same? I've
used the
> command line arguments -obs_valid_beg 20080824 -obs_valid_end
20080825,
> but I am curious if there are some additional arguments I need to
specify
> in the configuration file?
>
>
>
> On Wed, Mar 13, 2013 at 4:57 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> In your configuration file, you are requested the FHO output line:
>> fho = BOTH;
>>
>> The FHO line (for Forecast, Hit, and Observation Rate) is
categorical
>> output computed by thresholding the matched pairs. It's basically
>> equivalent to the 2x2 contingency table output contained in the
>> CTC line.
>>
>> Since your config file requests that categorical computations be
done,
>> Point-Stat is trying to retrieve the categorical thresholds that
should be
>> used:
>> cat_thresh = [ NA ];
>>
>> So the error is coming from the NA threshold.
>>
>> If you don't really want categorical stats to be computed, just set
"fho =
>> NONE;". If you do, you'll need to choose a better threshold,
perhaps:
>> cat_thresh = [ >=273.15 ];
>>
>> Hope that helps.
>>
>> John
>>
>> On 03/13/2013 01:55 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>> I have worked my way through several issues and the important
thing here
>> is
>>> that point_stat is accepting my data or at least I think. My
latest
>> issue
>>> results from running point_stat (see error message below).
>>>
>>> DEBUG 1: Forecast File: ./Aug2008NARR6_4km/WRFPRS_d01.000_00
>>> DEBUG 1: Climatology File: none
>>> DEBUG 1: Observation File: ./KYmesoBwgTmp.nc
>>> DEBUG 2:
>>> DEBUG 2:
>>>
>>
--------------------------------------------------------------------------------
>>> DEBUG 2:
>>> DEBUG 2: Reading data for TMP/Z2.
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
>>
--------------------------------------------------------------------------------
>>> DEBUG 2:
>>> DEBUG 2: Searching 48 observations from 48 messages.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
>>
--------------------------------------------------------------------------------
>>> DEBUG 2:
>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
>> SurfaceTMP,
>>> over region FULL, for interpolation method UW_MEAN(16), using 4
pairs.
>>> DEBUG 2: Computing Categorical Statistics.
>>> ERROR :
>>> ERROR : SingleThresh::check() -> unexpected threshold indicator
value
>> of 0.
>>> ERROR :
>>>
>>> I don't have a threshold set so I am not sure what this error is
>>> suggesting. Google searches indicate that the error means I don't
have
>>> matching observation points? I have attached my config file.
>>>
>>> Let me know if you need anything else,
>>>
>>>
>>> On Mon, Mar 11, 2013 at 1:06 PM, John Halley Gotway via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>> Ronald,
>>>>
>>>> OK, I've posted a new bugfix for METv4.0. The error checking for
>> message
>>>> type values is now removed:
>>>>
>>>>
>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>>>
>>>> Just follow the instructions at the top of the page to apply the
>> patches.
>>>>
>>>> Just let us know if more questions or problems arise in your use
of MET.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 03/11/2013 10:16 AM, John Halley Gotway wrote:
>>>>> Ronald,
>>>>>
>>>>> Right you are! I'm very surprised this issue hasn't come up in
the
>> past.
>>>>>
>>>>> In our internal use of ascii2nc, we've always just made use of
the
>>>> message types used in PREPBUFR files. There is one small
consideration
>>>> about this I'll explain below -
>>>>>
>>>>> In MET, the ADPSFC (obs over land) and SFCSHP (obs over water)
message
>>>> types have special meaning. They are considered to be "surface
>>>> observations". When you configure Point-Stat to verify a
>>>>> field at some vertical level (2-m temperature or 10-m winds for
>> example)
>>>> against the ADPSFC or SFCSHP message types, it will match all
>> observations
>>>> of that type regardless of the height value.
>>>>> Suppose for example, in your temperature observations, the 9th
column
>>>> contains height values of -9999 (missing), 1-m, 2-m, and 5-m. If
those
>>>> observations have the message type ADPSFC or SFCSHP,
>>>>> all of them will be used by Point-Stat. To put it succinctly,
when
>>>> verifying forecasts on vertical levels against the ADPSFC or
SFCSHP
>> message
>>>> types, the observation height value is ignored.
>>>>>
>>>>> Now, if you use some other message type, like SurfaceTMP, Point-
Stat
>>>> doesn't have any special logic for that string. So it will only
use
>>>> observations where the level value exactly matches the
>>>>> forecast level value, 2-m in this case.
>>>>>
>>>>> The fix for this is simple, just remove lines 250-260 from the
file
>>>> METv4.0/src/basic/vx_config/config_util.cc. I'll post a bugfix
to the
>> MET
>>>> website with this change. That will enable you to use
>>>>> message types different from the ones used in PREPBUFR
>>>>>
>>>>> Does that all make sense?
>>>>>
>>>>> Thanks, John Halley Gotway met_help at ucar.edu
>>>>>
>>>>> On 03/11/2013 06:01 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>>>>
>>>>>> Good morning,
>>>>>>
>>>>>> Still attempting to run point_stat and I get the following
error
>>>> message. I believe it has to do with my message_type in the
config file;
>>>> see error message below. I thought the message type in
>>>>>> Point_stat config file was suppose to match the message type
provided
>>>> in ascii2nc tool?
>>>>>>
>>>>>> DEBUG 1: Default Config File:
>>>> /admin/pkgs/met/METv4.0/data/config/PointStatConfig_default DEBUG
1:
>> User
>>>> Config File: ./iCpointStatConfigFile ERROR : ERROR :
>>>> parse_conf_message_type() -> Invalid
>>>>>> message type string provided (SurfaceTMP). ERROR :
>>>>>>
>>>>>> On Thu, Mar 7, 2013 at 2:55 PM, John Halley Gotway via RT <
>>>> met_help at ucar.edu
>>>>>>> wrote:
>>>>>>
>>>>>>> ou find it preferable, you could group all the output times
into a
>>>> single GRIB file. But then you'd need to set up the
configuration
>> files in
>>>> MET to accommodate t
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Fri Mar 15 06:46:37 2013
I do believe I have the script running and producing reasonable
results!!
It is too bad I couldn't have ran into these types of errors at the
MET
workshop and worked through them face to face. Regardless, I really
do
appreciate your help and dedication over the past several weeks and
look
forward to using this great product.
I am sure you shall hear from me again,
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Fri Mar 15 09:36:26 2013
Ronald,
Great, glad to hear it.
Thanks,
John
On 03/15/2013 06:46 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> I do believe I have the script running and producing reasonable
results!!
> It is too bad I couldn't have ran into these types of errors at
the MET
> workshop and worked through them face to face. Regardless, I really
do
> appreciate your help and dedication over the past several weeks and
look
> forward to using this great product.
>
> I am sure you shall hear from me again,
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Mon Mar 25 12:26:06 2013
So I added some additional observational variables (RH, PRCP, and
dewpoint)
for MET to include in the comparison with model data. When I reran
ASCII2NC I got the following error message:
NetCDF: NC_UNLIMITED size already in use
Is there an environment variable I need to set here?
On Fri, Mar 15, 2013 at 11:36 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ronald,
>
> Great, glad to hear it.
>
> Thanks,
> John
>
> On 03/15/2013 06:46 AM, Ronald Leeper - NOAA Affiliate via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
> >
> > I do believe I have the script running and producing reasonable
results!!
> > It is too bad I couldn't have ran into these types of errors at
the MET
> > workshop and worked through them face to face. Regardless, I
really do
> > appreciate your help and dedication over the past several weeks
and look
> > forward to using this great product.
> >
> > I am sure you shall hear from me again,
> >
>
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Mar 25 12:33:32 2013
Ronald,
Not sure what's going on there. Can you please send me an ascii file,
the command line you're using to run ascii2nc, and specify which
version of MET you're running?
I'll try to reproduce the error here and debug.
If the file is small, you can send it via email. Otherwise, you could
post it to our anonymous ftp site.
See section titled "How to send us data":
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Thanks,
John
On 03/25/2013 12:26 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> So I added some additional observational variables (RH, PRCP, and
dewpoint)
> for MET to include in the comparison with model data. When I reran
> ASCII2NC I got the following error message:
>
> NetCDF: NC_UNLIMITED size already in use
>
> Is there an environment variable I need to set here?
>
> On Fri, Mar 15, 2013 at 11:36 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ronald,
>>
>> Great, glad to hear it.
>>
>> Thanks,
>> John
>>
>> On 03/15/2013 06:46 AM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>> I do believe I have the script running and producing reasonable
results!!
>>> It is too bad I couldn't have ran into these types of errors at
the MET
>>> workshop and worked through them face to face. Regardless, I
really do
>>> appreciate your help and dedication over the past several weeks
and look
>>> forward to using this great product.
>>>
>>> I am sure you shall hear from me again,
>>>
>>
>>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Mon Mar 25 12:41:16 2013
Hey,
I am using Met version 4.0. Please find the attached observation
ascii
file. The command line is:
/usr/local/met/bin/ascii2nc \
~/initLatalBundryLyr_study/metAnalysis/asciiFiles/Aug2008_BWGDataBG_test2.csv
\ ~/initLatalBundryLyr_study/metAnalysis/KYmesoAug08_BWG.nc -v 2
On Mon, Mar 25, 2013 at 2:33 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> I'll try to reproduce the error here and debug.
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Mon Mar 25 13:47:17 2013
Ronald,
The problem is in the newline character in your ascii file. If you
open it up in 'vi' you'll see what I mean. It's using the Mac ^M
character instead of a unix newline.
On my linux machine, there's a utility called "mac2unix". After
running the Aug2008_BWGDataBG_test2.csv file through that, ascii2nc
was able to read it. But I got this error:
DEBUG 1: Creating NetCDF Observation file: Aug2008_BWGDataBG_test2.nc
DEBUG 1: Reading ASCII Observation file: Aug2008_BWGDataBG_test2.csv
DEBUG 2: Processing 195 observations.
ERROR :
ERROR : timestring_to_sec(const char *) -> can't parse time string "-
9999"
ERROR :
I ran it through the debugger and found that that's coming from lines
148 to 196 - where you have observations of GRIB code 61, which is
accumulated precipitation. For APCP, the level value is
supposed to contain an accumulation interval in HH[MMSS] format - so
01 for 1 hour accumulation or 053000 for a 5.5 hour accumulation.
ASCII2NC is complaining about the -9999 value in there.
Hope that helps.
Thanks,
John
On 03/25/2013 12:41 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> Hey,
>
> I am using Met version 4.0. Please find the attached observation
ascii
> file. The command line is:
>
> /usr/local/met/bin/ascii2nc \
>
~/initLatalBundryLyr_study/metAnalysis/asciiFiles/Aug2008_BWGDataBG_test2.csv
> \ ~/initLatalBundryLyr_study/metAnalysis/KYmesoAug08_BWG.nc -v 2
>
> On Mon, Mar 25, 2013 at 2:33 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> I'll try to reproduce the error here and debug.
>
>
>
>
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Mon Mar 25 14:38:45 2013
Thanks,
Ohhh ok I forgot to save it as a windows cvs. This is easily fix
able
and another thanks for catching the -9999 for interval. Easily
fixable!!
Have a great day,
On Mon, Mar 25, 2013 at 3:47 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> problem is in the newline character in your ascii file
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Wed Mar 27 09:52:31 2013
Just as a comment.
I noticed that when concatenating various WRF-lead-time (the converted
wrf-netcdf to grib) files that the point_stat tool would crash on the
lead-time file that marked a new date.
For instance, if I had a 24 hour simulation that started at 12 UTC on
1/1/2012 and ended 12 UTC on 1/2/2012 and a grib file for every hour I
would have to concatenate the files into two groups one for each
calendar
date (12 grib files for 1/1/2012 and 12 files for 1/2/2012).
Otherwise
point_stat will crash when it attempts to match the 13th grib file
which is
for date-time 00 UTC on 1/2/2012.
My work around works for me and I could have something set that is
causing
this, but I am curious if this is normal behavior?
Thanks,
On Mon, Mar 25, 2013 at 4:38 PM, Ronald Leeper - NOAA Affiliate <
ronald.leeper at noaa.gov> wrote:
> Thanks,
>
> Ohhh ok I forgot to save it as a windows cvs. This is easily fix
able
> and another thanks for catching the -9999 for interval. Easily
fixable!!
>
> Have a great day,
>
>
> On Mon, Mar 25, 2013 at 3:47 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> problem is in the newline character in your ascii file
>
>
>
>
>
> --
> Ronald David Leeper
> Research Associate, CICS-NC
> Quality Assurance Specialist, USCRN
> National Climatic Data Center (NCDC)
> 151 Patton Ave.
> Asheville, NC 28801-5001
> ronald.leeper at noaa.gov
> leeperd at gmail.com
> Office:(828)257-3185
> Fax:(828)271-4022
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Wed Mar 27 10:53:44 2013
Ronald,
Would you be able to send me some sample data to demonstrate this
behavior?
I'd need:
- GRIB file which results in a crash
- Observation file file for Point-Stat
- Point-Stat configuration file
- The command line you're using to run Point-Stat
It doesn't need to be a large amount of data - just big enough to
demonstrate the problematic behavior.
You could post it to our anonymous ftp site, following these
instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Hopefully I can replicate the problematic behavior and resolve it for
the
next release and/or post a bugfix.
Thanks for letting us know!
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> Just as a comment.
>
> I noticed that when concatenating various WRF-lead-time (the
converted
> wrf-netcdf to grib) files that the point_stat tool would crash on
the
> lead-time file that marked a new date.
>
> For instance, if I had a 24 hour simulation that started at 12 UTC
on
> 1/1/2012 and ended 12 UTC on 1/2/2012 and a grib file for every hour
I
> would have to concatenate the files into two groups one for each
calendar
> date (12 grib files for 1/1/2012 and 12 files for 1/2/2012).
Otherwise
> point_stat will crash when it attempts to match the 13th grib file
which
> is
> for date-time 00 UTC on 1/2/2012.
>
> My work around works for me and I could have something set that is
causing
> this, but I am curious if this is normal behavior?
>
> Thanks,
>
> On Mon, Mar 25, 2013 at 4:38 PM, Ronald Leeper - NOAA Affiliate <
> ronald.leeper at noaa.gov> wrote:
>
>> Thanks,
>>
>> Ohhh ok I forgot to save it as a windows cvs. This is easily fix
able
>> and another thanks for catching the -9999 for interval. Easily
>> fixable!!
>>
>> Have a great day,
>>
>>
>> On Mon, Mar 25, 2013 at 3:47 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> problem is in the newline character in your ascii file
>>
>>
>>
>>
>>
>> --
>> Ronald David Leeper
>> Research Associate, CICS-NC
>> Quality Assurance Specialist, USCRN
>> National Climatic Data Center (NCDC)
>> 151 Patton Ave.
>> Asheville, NC 28801-5001
>> ronald.leeper at noaa.gov
>> leeperd at gmail.com
>> Office:(828)257-3185
>> Fax:(828)271-4022
>>
>
>
>
> --
> Ronald David Leeper
> Research Associate, CICS-NC
> Quality Assurance Specialist, USCRN
> National Climatic Data Center (NCDC)
> 151 Patton Ave.
> Asheville, NC 28801-5001
> ronald.leeper at noaa.gov
> leeperd at gmail.com
> Office:(828)257-3185
> Fax:(828)271-4022
>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Wed Mar 27 14:26:36 2013
I have uploaded a tarball in the incoming dir under
leeper_pointStatDataset.
I have included two concatenated grib files clearly named to indicate
all-gribfile and gribfiles-for-individual-calendar-dates; three total.
In
addition, all the separate gribs were included as well. Along with
those
are config-files for each of the concatenated grib-files; also clearly
titled. Finally, there is also a observation-based netcdf file so you
can
test these things out.
To replicate my error, you just need to run the all-grib file case.
The
other two were provided that it can work if you break the
concatenation
process down by calendar dates.
Hope this makes sense,
On Wed, Mar 27, 2013 at 12:53 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Wed Mar 27 16:05:41 2013
Ronald,
Thanks for sending that data. It made reproducing and debugging the
issue much easier. The problem we're seeing stems from the overall
file size rather than the timestamp. When I tried to run that
test case, it errored out on the 30-minute lead time of 2-m
temperature. That GRIB record is #16944 of 17021 in file
Aug2008_allGribFiles.
However, the MET software thinks that that file only has 12627
records. There's likely a problem in how we're storing the byte
offset for the records. We're overflowing one of the variables being
used. I'll look into it and try to resolve the issue.
Thanks for finding it!
John
On 03/27/2013 02:26 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>
> I have uploaded a tarball in the incoming dir under
leeper_pointStatDataset.
> I have included two concatenated grib files clearly named to
indicate
> all-gribfile and gribfiles-for-individual-calendar-dates; three
total. In
> addition, all the separate gribs were included as well. Along with
those
> are config-files for each of the concatenated grib-files; also
clearly
> titled. Finally, there is also a observation-based netcdf file so
you can
> test these things out.
>
> To replicate my error, you just need to run the all-grib file case.
The
> other two were provided that it can work if you break the
concatenation
> process down by calendar dates.
>
> Hope this makes sense,
>
>
> On Wed, Mar 27, 2013 at 12:53 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Wed Mar 27 17:09:02 2013
Ronald,
So the fix is using the machine-specific off_t type in place of the
integer type for storing byte offsets in the GRIB code. I've made the
change, verified that it now allows you to run all the GRIB
records in a single pass, and posted a METv4.0 bugfix for this:
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
Please just follow the instructions at the top of the page to apply
all of the latest bug fixes for METv4.0.
Then try rerunning your test case and make sure that it completes
without error.
Please let me know how it goes.
Thanks,
John
On 03/27/2013 04:05 PM, John Halley Gotway wrote:
> Ronald,
>
> Thanks for sending that data. It made reproducing and debugging the
issue much easier. The problem we're seeing stems from the overall
file size rather than the timestamp. When I tried to run that
> test case, it errored out on the 30-minute lead time of 2-m
temperature. That GRIB record is #16944 of 17021 in file
Aug2008_allGribFiles.
>
> However, the MET software thinks that that file only has 12627
records. There's likely a problem in how we're storing the byte
offset for the records. We're overflowing one of the variables being
> used. I'll look into it and try to resolve the issue.
>
> Thanks for finding it!
>
> John
>
> On 03/27/2013 02:26 PM, Ronald Leeper - NOAA Affiliate via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>
>> I have uploaded a tarball in the incoming dir under
leeper_pointStatDataset.
>> I have included two concatenated grib files clearly named to
indicate
>> all-gribfile and gribfiles-for-individual-calendar-dates; three
total. In
>> addition, all the separate gribs were included as well. Along with
those
>> are config-files for each of the concatenated grib-files; also
clearly
>> titled. Finally, there is also a observation-based netcdf file so
you can
>> test these things out.
>>
>> To replicate my error, you just need to run the all-grib file case.
The
>> other two were provided that it can work if you break the
concatenation
>> process down by calendar dates.
>>
>> Hope this makes sense,
>>
>>
>> On Wed, Mar 27, 2013 at 12:53 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>>
>>
>>
>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60404] Point_stat tool
From: John Halley Gotway
Time: Wed Mar 27 17:20:56 2013
Ronald,
You may also find it helpful to use the plot_data_plane utility to
visualize your data to make sure it looks good:
METv4.0/bin/plot_data_plane \
Aug2008_allGribFiles \
TMP_Z2_30min.ps \
'name="TMP"; level="Z2"; lead_time="003000";' \
-plot_range 285 300 \
-v 3
Then open up the resulting image (TMP_Z2_30min.ps). Looks like there
some noise in the data. Is that what you would expect?
Thanks,
John
On 03/27/2013 05:09 PM, John Halley Gotway wrote:
> Ronald,
>
> So the fix is using the machine-specific off_t type in place of the
integer type for storing byte offsets in the GRIB code. I've made the
change, verified that it now allows you to run all the GRIB
> records in a single pass, and posted a METv4.0 bugfix for this:
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Please just follow the instructions at the top of the page to apply
all of the latest bug fixes for METv4.0.
>
> Then try rerunning your test case and make sure that it completes
without error.
>
> Please let me know how it goes.
>
> Thanks,
> John
>
> On 03/27/2013 04:05 PM, John Halley Gotway wrote:
>> Ronald,
>>
>> Thanks for sending that data. It made reproducing and debugging
the issue much easier. The problem we're seeing stems from the
overall file size rather than the timestamp. When I tried to run that
>> test case, it errored out on the 30-minute lead time of 2-m
temperature. That GRIB record is #16944 of 17021 in file
Aug2008_allGribFiles.
>>
>> However, the MET software thinks that that file only has 12627
records. There's likely a problem in how we're storing the byte
offset for the records. We're overflowing one of the variables being
>> used. I'll look into it and try to resolve the issue.
>>
>> Thanks for finding it!
>>
>> John
>>
>> On 03/27/2013 02:26 PM, Ronald Leeper - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60404 >
>>>
>>> I have uploaded a tarball in the incoming dir under
leeper_pointStatDataset.
>>> I have included two concatenated grib files clearly named to
indicate
>>> all-gribfile and gribfiles-for-individual-calendar-dates; three
total. In
>>> addition, all the separate gribs were included as well. Along
with those
>>> are config-files for each of the concatenated grib-files; also
clearly
>>> titled. Finally, there is also a observation-based netcdf file so
you can
>>> test these things out.
>>>
>>> To replicate my error, you just need to run the all-grib file
case. The
>>> other two were provided that it can work if you break the
concatenation
>>> process down by calendar dates.
>>>
>>> Hope this makes sense,
>>>
>>>
>>> On Wed, Mar 27, 2013 at 12:53 PM, John Halley Gotway via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>>
>>>
>>>
>>>
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Thu Mar 28 05:36:49 2013
Great news! Plot_data_plane sounds interesting. After I apply the
update
I'll give it a try.
Thanks,
On Wed, Mar 27, 2013 at 7:20 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> also find it helpful to use the plot_data_plane utility to visualize
your d
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
Subject: Point_stat tool
From: Ronald Leeper - NOAA Affiliate
Time: Fri Mar 29 09:07:04 2013
I had the patch applied yesterday and this morning it was able to
process
the entire period.
Thanks yet again for your help!
On Thu, Mar 28, 2013 at 7:36 AM, Ronald Leeper - NOAA Affiliate <
ronald.leeper at noaa.gov> wrote:
> Great news! Plot_data_plane sounds interesting. After I apply the
update
> I'll give it a try.
>
> Thanks,
>
>
> On Wed, Mar 27, 2013 at 7:20 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> also find it helpful to use the plot_data_plane utility to
visualize your
>> d
>
>
>
>
>
> --
> Ronald David Leeper
> Research Associate, CICS-NC
> Quality Assurance Specialist, USCRN
> National Climatic Data Center (NCDC)
> 151 Patton Ave.
> Asheville, NC 28801-5001
> ronald.leeper at noaa.gov
> leeperd at gmail.com
> Office:(828)257-3185
> Fax:(828)271-4022
>
--
Ronald David Leeper
Research Associate, CICS-NC
Quality Assurance Specialist, USCRN
National Climatic Data Center (NCDC)
151 Patton Ave.
Asheville, NC 28801-5001
ronald.leeper at noaa.gov
leeperd at gmail.com
Office:(828)257-3185
Fax:(828)271-4022
------------------------------------------------
More information about the Met_help
mailing list