[Met_help] [rt.rap.ucar.edu #47370] History for point stat questions

RAL HelpDesk {for Paul Oldenburg} met_help at ucar.edu
Wed Jun 15 09:57:07 MDT 2011


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,
I have some results from WRF evaluations against prepbufr data and some concerns.

1.STDEV problem

I ran pb2nc code with options 

obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD", "PRMSL", "MIXR" ];

In point stat config I match

fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
obs_field[] = ["MIXR/Z2"] 

and in a cutout of  *cnt.txt file I get

FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
20100811_120000,Q2,P-9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-0.00013
20100811_130000,Q2,P-9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-0.00020

so that FSTDEV=0 and OSTDEV=0

Same happens for 3d QVAPOR

fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
obs_field[] = ["MIXR/P850"] 


QVAPOR_P850_cnt.txt
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-0.00082
20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-99999.9,0.00597,-99999.9,-99999.9,0.00069
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060

standard deviations = 0 (works fine for other variables, problem only for mixing ratio)
Am I doing something wrong?

2. PR_CORR
It seems that your threshold wrt the number of observations for calculating correlations is > 1. Is it possible to change it to a higher number?
Are these  spatial  between obs and fcst for a given time?

3. My domain is roughly continental US without Alaska. Is it normal that I get about 60-70  upper air (ANYSFC) standard measurements (HGT, T, Q) at 12Z.
ANYAIR is usually 1-3 measurements. Seems small. 
For surface observations the number is about 1000 every hour. 
Are these numbers reasonable?

Thanks for helping,
Mariusz




----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #47370] point stat questions
From: Paul Oldenburg
Time: Tue Jun 07 15:01:37 2011

Mariusz,

First of all, can you please send me the following information?

1.  PrepBUFR file
2.  Model data file
3.  pb2nc config file
4.  point_stat config file

If it is too large to send via email, please follow the instructions
to FTP to our site here:

http://www.dtcenter.org/met/users/support/met_help.php#ftp

When using our FTP site, we have had best results when all of the
files to be uploaded are packed into a gzipped tar
bundle using a command like this:

$ tar czvf bundle.tar.gz prepBUFR_file model_data_file ...

Regarding your question:

> 2. PR_CORR
> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to
change it to a higher number?

It is not possible to configure this threshold in point_stat.
However, here is a one-liner that you could use to print
the total and pr_corr values for all CNT lines whose TOTAL is greater
than 10.  Note that this assumes you are using
METv3.x.x:

$ cat *_cnt.txt | grep '^V3' | awk '{print $22, $43}' | \
  perl -e 'while(<>){($t,$p)=/(\d+) (.*)/;print "$t\t$p\n" if $t>10;}'


> Are these  spatial  between obs and fcst for a given time?

No, the pr_corr is simply the simple correlation between the forecast
value and obs value for all matched pairs.

Please let me know if you have any questions.

Paul



On 06/07/2011 02:07 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>
> Tue Jun 07 14:07:08 2011: Request 47370 was acted upon.
> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>        Queue: met_help
>      Subject: point stat questions
>        Owner: Nobody
>   Requestors: Mariusz.Pagowski at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>
>
> Hello,
> I have some results from WRF evaluations against prepbufr data and
some concerns.
>
> 1.STDEV problem
>
> I ran pb2nc code with options
>
> obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD", "PRMSL",
"MIXR" ];
>
> In point stat config I match
>
> fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
> obs_field[] = ["MIXR/Z2"]
>
> and in a cutout of  *cnt.txt file I get
>
>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
> 20100811_120000,Q2,P-
9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-
0.00013
> 20100811_130000,Q2,P-
9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-
0.00020
>
> so that FSTDEV=0 and OSTDEV=0
>
> Same happens for 3d QVAPOR
>
> fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
> obs_field[] = ["MIXR/P850"]
>
>
> QVAPOR_P850_cnt.txt
>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-
0.00082
> 20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-
99999.9,0.00597,-99999.9,-99999.9,0.00069
>
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060
>
> standard deviations = 0 (works fine for other variables, problem
only for mixing ratio)
> Am I doing something wrong?
>
> 2. PR_CORR
> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to change it to a
higher number?
> Are these  spatial  between obs and fcst for a given time?
>
> 3. My domain is roughly continental US without Alaska. Is it normal
that I get about 60-70  upper air (ANYSFC) standard measurements (HGT,
T, Q) at 12Z.
> ANYAIR is usually 1-3 measurements. Seems small.
> For surface observations the number is about 1000 every hour.
> Are these numbers reasonable?
>
> Thanks for helping,
> Mariusz
>
>


------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47370] point stat questions
From: Mariusz Pagowski
Time: Tue Jun 07 15:27:13 2011

Paul,
I have put my files as instructed on your website. The file is a
little bit large - altogether 12 hour of forecasts plus corresponding
prepbufrs
Config files are created in the scripts that are also included in the
tar file.
Thanks,
Mariusz


On Jun 7, 2011, at 3:01 PM, RAL HelpDesk {for Paul Oldenburg} wrote:

> Mariusz,
>
> First of all, can you please send me the following information?
>
> 1.  PrepBUFR file
> 2.  Model data file
> 3.  pb2nc config file
> 4.  point_stat config file
>
> If it is too large to send via email, please follow the instructions
to FTP to our site here:
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> When using our FTP site, we have had best results when all of the
files to be uploaded are packed into a gzipped tar
> bundle using a command like this:
>
> $ tar czvf bundle.tar.gz prepBUFR_file model_data_file ...
>
> Regarding your question:
>
>> 2. PR_CORR
>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to
> change it to a higher number?
>
> It is not possible to configure this threshold in point_stat.
However, here is a one-liner that you could use to print
> the total and pr_corr values for all CNT lines whose TOTAL is
greater than 10.  Note that this assumes you are using
> METv3.x.x:
>
> $ cat *_cnt.txt | grep '^V3' | awk '{print $22, $43}' | \
>  perl -e 'while(<>){($t,$p)=/(\d+) (.*)/;print "$t\t$p\n" if
$t>10;}'
>
>
>> Are these  spatial  between obs and fcst for a given time?
>
> No, the pr_corr is simply the simple correlation between the
forecast value and obs value for all matched pairs.
>
> Please let me know if you have any questions.
>
> Paul
>
>
>
> On 06/07/2011 02:07 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>
>> Tue Jun 07 14:07:08 2011: Request 47370 was acted upon.
>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>       Queue: met_help
>>     Subject: point stat questions
>>       Owner: Nobody
>>  Requestors: Mariusz.Pagowski at noaa.gov
>>      Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>
>>
>> Hello,
>> I have some results from WRF evaluations against prepbufr data and
some concerns.
>>
>> 1.STDEV problem
>>
>> I ran pb2nc code with options
>>
>> obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD",
"PRMSL", "MIXR" ];
>>
>> In point stat config I match
>>
>> fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
>> obs_field[] = ["MIXR/Z2"]
>>
>> and in a cutout of  *cnt.txt file I get
>>
>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>> 20100811_120000,Q2,P-
9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-
0.00013
>> 20100811_130000,Q2,P-
9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-
0.00020
>>
>> so that FSTDEV=0 and OSTDEV=0
>>
>> Same happens for 3d QVAPOR
>>
>> fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
>> obs_field[] = ["MIXR/P850"]
>>
>>
>> QVAPOR_P850_cnt.txt
>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-
0.00082
>> 20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-
99999.9,0.00597,-99999.9,-99999.9,0.00069
>>
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060
>>
>> standard deviations = 0 (works fine for other variables, problem
only for mixing ratio)
>> Am I doing something wrong?
>>
>> 2. PR_CORR
>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to change it to a
higher number?
>> Are these  spatial  between obs and fcst for a given time?
>>
>> 3. My domain is roughly continental US without Alaska. Is it normal
that I get about 60-70  upper air (ANYSFC) standard measurements (HGT,
T, Q) at 12Z.
>> ANYAIR is usually 1-3 measurements. Seems small.
>> For surface observations the number is about 1000 every hour.
>> Are these numbers reasonable?
>>
>> Thanks for helping,
>> Mariusz
>>
>>
>
>


------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47370] point stat questions
From: Paul Oldenburg
Time: Wed Jun 08 11:30:48 2011

Mariusz,

Can you please send me one of the point_stat config files for Q2/MIXR
generated by your script enkf_point_stat.sh?  I
don't have time to reverse engineer that script to create a config
file, and it may be error prone if I try.

Thanks,

Paul


On 06/07/2011 03:27 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>
> Paul,
> I have put my files as instructed on your website. The file is a
little bit large - altogether 12 hour of forecasts plus corresponding
prepbufrs
> Config files are created in the scripts that are also included in
the tar file.
> Thanks,
> Mariusz
>
>
> On Jun 7, 2011, at 3:01 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>
>> Mariusz,
>>
>> First of all, can you please send me the following information?
>>
>> 1.  PrepBUFR file
>> 2.  Model data file
>> 3.  pb2nc config file
>> 4.  point_stat config file
>>
>> If it is too large to send via email, please follow the
instructions to FTP to our site here:
>>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> When using our FTP site, we have had best results when all of the
files to be uploaded are packed into a gzipped tar
>> bundle using a command like this:
>>
>> $ tar czvf bundle.tar.gz prepBUFR_file model_data_file ...
>>
>> Regarding your question:
>>
>>> 2. PR_CORR
>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to
>> change it to a higher number?
>>
>> It is not possible to configure this threshold in point_stat.
However, here is a one-liner that you could use to print
>> the total and pr_corr values for all CNT lines whose TOTAL is
greater than 10.  Note that this assumes you are using
>> METv3.x.x:
>>
>> $ cat *_cnt.txt | grep '^V3' | awk '{print $22, $43}' | \
>>  perl -e 'while(<>){($t,$p)=/(\d+) (.*)/;print "$t\t$p\n" if
$t>10;}'
>>
>>
>>> Are these  spatial  between obs and fcst for a given time?
>>
>> No, the pr_corr is simply the simple correlation between the
forecast value and obs value for all matched pairs.
>>
>> Please let me know if you have any questions.
>>
>> Paul
>>
>>
>>
>> On 06/07/2011 02:07 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>>
>>> Tue Jun 07 14:07:08 2011: Request 47370 was acted upon.
>>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>>       Queue: met_help
>>>     Subject: point stat questions
>>>       Owner: Nobody
>>>  Requestors: Mariusz.Pagowski at noaa.gov
>>>      Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>>
>>>
>>> Hello,
>>> I have some results from WRF evaluations against prepbufr data and
some concerns.
>>>
>>> 1.STDEV problem
>>>
>>> I ran pb2nc code with options
>>>
>>> obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD",
"PRMSL", "MIXR" ];
>>>
>>> In point stat config I match
>>>
>>> fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
>>> obs_field[] = ["MIXR/Z2"]
>>>
>>> and in a cutout of  *cnt.txt file I get
>>>
>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>> 20100811_120000,Q2,P-
9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-
0.00013
>>> 20100811_130000,Q2,P-
9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-
0.00020
>>>
>>> so that FSTDEV=0 and OSTDEV=0
>>>
>>> Same happens for 3d QVAPOR
>>>
>>> fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
>>> obs_field[] = ["MIXR/P850"]
>>>
>>>
>>> QVAPOR_P850_cnt.txt
>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>>
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-
0.00082
>>> 20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-
99999.9,0.00597,-99999.9,-99999.9,0.00069
>>>
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060
>>>
>>> standard deviations = 0 (works fine for other variables, problem
only for mixing ratio)
>>> Am I doing something wrong?
>>>
>>> 2. PR_CORR
>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to change it to a
higher number?
>>> Are these  spatial  between obs and fcst for a given time?
>>>
>>> 3. My domain is roughly continental US without Alaska. Is it
normal that I get about 60-70  upper air (ANYSFC) standard
measurements (HGT, T, Q) at 12Z.
>>> ANYAIR is usually 1-3 measurements. Seems small.
>>> For surface observations the number is about 1000 every hour.
>>> Are these numbers reasonable?
>>>
>>> Thanks for helping,
>>> Mariusz
>>>
>>>
>>
>>
>


------------------------------------------------
Subject: point stat questions
From: Mariusz Pagowski
Time: Wed Jun 08 11:57:18 2011

Paul,
sorry about that. Attached are configs for Q@ and Qvapor,
Ma

------------------------------------------------
Subject: point stat questions
From: Mariusz Pagowski
Time: Wed Jun 08 11:57:18 2011

riusz

On Jun 8, 2011, at 11:30 AM, RAL HelpDesk {for Paul Oldenburg} wrote:

> Mariusz,
>
> Can you please send me one of the point_stat config files for
Q2/MIXR generated by your script enkf_point_stat.sh?  I
> don't have time to reverse engineer that script to create a config
file, and it may be error prone if I try.
>
> Thanks,
>
> Paul
>
>
> On 06/07/2011 03:27 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>
>> Paul,
>> I have put my files as instructed on your website. The file is a
little bit large - altogether 12 hour of forecasts plus corresponding
prepbufrs
>> Config files are created in the scripts that are also included in
the tar file.
>> Thanks,
>> Mariusz
>>
>>
>> On Jun 7, 2011, at 3:01 PM, RAL HelpDesk {for Paul Oldenburg}
wrote:
>>
>>> Mariusz,
>>>
>>> First of all, can you please send me the following information?
>>>
>>> 1.  PrepBUFR file
>>> 2.  Model data file
>>> 3.  pb2nc config file
>>> 4.  point_stat config file
>>>
>>> If it is too large to send via email, please follow the
instructions to FTP to our site here:
>>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> When using our FTP site, we have had best results when all of the
files to be uploaded are packed into a gzipped tar
>>> bundle using a command like this:
>>>
>>> $ tar czvf bundle.tar.gz prepBUFR_file model_data_file ...
>>>
>>> Regarding your question:
>>>
>>>> 2. PR_CORR
>>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to
>>> change it to a higher number?
>>>
>>> It is not possible to configure this threshold in point_stat.
However, here is a one-liner that you could use to print
>>> the total and pr_corr values for all CNT lines whose TOTAL is
greater than 10.  Note that this assumes you are using
>>> METv3.x.x:
>>>
>>> $ cat *_cnt.txt | grep '^V3' | awk '{print $22, $43}' | \
>>> perl -e 'while(<>){($t,$p)=/(\d+) (.*)/;print "$t\t$p\n" if
$t>10;}'
>>>
>>>
>>>> Are these  spatial  between obs and fcst for a given time?
>>>
>>> No, the pr_corr is simply the simple correlation between the
forecast value and obs value for all matched pairs.
>>>
>>> Please let me know if you have any questions.
>>>
>>> Paul
>>>
>>>
>>>
>>> On 06/07/2011 02:07 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>>>
>>>> Tue Jun 07 14:07:08 2011: Request 47370 was acted upon.
>>>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>>>      Queue: met_help
>>>>    Subject: point stat questions
>>>>      Owner: Nobody
>>>> Requestors: Mariusz.Pagowski at noaa.gov
>>>>     Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>>>
>>>>
>>>> Hello,
>>>> I have some results from WRF evaluations against prepbufr data
and some concerns.
>>>>
>>>> 1.STDEV problem
>>>>
>>>> I ran pb2nc code with options
>>>>
>>>> obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD",
"PRMSL", "MIXR" ];
>>>>
>>>> In point stat config I match
>>>>
>>>> fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
>>>> obs_field[] = ["MIXR/Z2"]
>>>>
>>>> and in a cutout of  *cnt.txt file I get
>>>>
>>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>>> 20100811_120000,Q2,P-
9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-
0.00013
>>>> 20100811_130000,Q2,P-
9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-
0.00020
>>>>
>>>> so that FSTDEV=0 and OSTDEV=0
>>>>
>>>> Same happens for 3d QVAPOR
>>>>
>>>> fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
>>>> obs_field[] = ["MIXR/P850"]
>>>>
>>>>
>>>> QVAPOR_P850_cnt.txt
>>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>>>
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-
0.00082
>>>> 20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-
99999.9,0.00597,-99999.9,-99999.9,0.00069
>>>>
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060
>>>>
>>>> standard deviations = 0 (works fine for other variables, problem
only for mixing ratio)
>>>> Am I doing something wrong?
>>>>
>>>> 2. PR_CORR
>>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to change it to a
higher number?
>>>> Are these  spatial  between obs and fcst for a given time?
>>>>
>>>> 3. My domain is roughly continental US without Alaska. Is it
normal that I get about 60-70  upper air (ANYSFC) standard
measurements (HGT, T, Q) at 12Z.
>>>> ANYAIR is usually 1-3 measurements. Seems small.
>>>> For surface observations the number is about 1000 every hour.
>>>> Are these numbers reasonable?
>>>>
>>>> Thanks for helping,
>>>> Mariusz
>>>>
>>>>
>>>
>>>
>>
>
>


------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47370] point stat questions
From: Paul Oldenburg
Time: Fri Jun 10 08:19:48 2011

Mariusz,

Your report of 0 FSTDEV and OSTDEV led to us uncovering a bug in MET.
A bug fix has been posted on the following two
web pages, depending on the version of MET that you are using:

  http://www.dtcenter.org/met/users/support/known_issues/METv3.0.1/index.php
  http://www.dtcenter.org/met/users/support/known_issues/METv3.0/index.php

Please apply the bug fix to your instance of MET and that should fix
the problem.  If you notice otherwise, please let
us know.

Thanks,

Paul


On 06/08/2011 11:57 AM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>
> Paul,
> sorry about that. Attached are configs for Q@ and Qvapor,
> Ma
>
>
>
>
> riusz
>
> On Jun 8, 2011, at 11:30 AM, RAL HelpDesk {for Paul Oldenburg}
wrote:
>
>> Mariusz,
>>
>> Can you please send me one of the point_stat config files for
Q2/MIXR generated by your script enkf_point_stat.sh?  I
>> don't have time to reverse engineer that script to create a config
file, and it may be error prone if I try.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 06/07/2011 03:27 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>>
>>> Paul,
>>> I have put my files as instructed on your website. The file is a
little bit large - altogether 12 hour of forecasts plus corresponding
prepbufrs
>>> Config files are created in the scripts that are also included in
the tar file.
>>> Thanks,
>>> Mariusz
>>>
>>>
>>> On Jun 7, 2011, at 3:01 PM, RAL HelpDesk {for Paul Oldenburg}
wrote:
>>>
>>>> Mariusz,
>>>>
>>>> First of all, can you please send me the following information?
>>>>
>>>> 1.  PrepBUFR file
>>>> 2.  Model data file
>>>> 3.  pb2nc config file
>>>> 4.  point_stat config file
>>>>
>>>> If it is too large to send via email, please follow the
instructions to FTP to our site here:
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> When using our FTP site, we have had best results when all of the
files to be uploaded are packed into a gzipped tar
>>>> bundle using a command like this:
>>>>
>>>> $ tar czvf bundle.tar.gz prepBUFR_file model_data_file ...
>>>>
>>>> Regarding your question:
>>>>
>>>>> 2. PR_CORR
>>>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to
>>>> change it to a higher number?
>>>>
>>>> It is not possible to configure this threshold in point_stat.
However, here is a one-liner that you could use to print
>>>> the total and pr_corr values for all CNT lines whose TOTAL is
greater than 10.  Note that this assumes you are using
>>>> METv3.x.x:
>>>>
>>>> $ cat *_cnt.txt | grep '^V3' | awk '{print $22, $43}' | \
>>>> perl -e 'while(<>){($t,$p)=/(\d+) (.*)/;print "$t\t$p\n" if
$t>10;}'
>>>>
>>>>
>>>>> Are these  spatial  between obs and fcst for a given time?
>>>>
>>>> No, the pr_corr is simply the simple correlation between the
forecast value and obs value for all matched pairs.
>>>>
>>>> Please let me know if you have any questions.
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>> On 06/07/2011 02:07 PM, RAL HelpDesk {for Mariusz Pagowski}
wrote:
>>>>>
>>>>> Tue Jun 07 14:07:08 2011: Request 47370 was acted upon.
>>>>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>>>>      Queue: met_help
>>>>>    Subject: point stat questions
>>>>>      Owner: Nobody
>>>>> Requestors: Mariusz.Pagowski at noaa.gov
>>>>>     Status: new
>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47370 >
>>>>>
>>>>>
>>>>> Hello,
>>>>> I have some results from WRF evaluations against prepbufr data
and some concerns.
>>>>>
>>>>> 1.STDEV problem
>>>>>
>>>>> I ran pb2nc code with options
>>>>>
>>>>> obs_grib_code[] = [ "SPFH", "TMP",  "HGT",  "UGRD", "VGRD",
"PRMSL", "MIXR" ];
>>>>>
>>>>> In point stat config I match
>>>>>
>>>>> fcst_field[] = [ "Q2(${itime}-${itime},*,*)" ] ;
>>>>> obs_field[] = ["MIXR/Z2"]
>>>>>
>>>>> and in a cutout of  *cnt.txt file I get
>>>>>
>>>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>>>> 20100811_120000,Q2,P-
9999,MIXR,Z2,ANYSFC,1238,0.01330,0.00000,0.01343,0.00000,0.93174,-
0.00013
>>>>> 20100811_130000,Q2,P-
9999,MIXR,Z2,ANYSFC,992,0.01408,0.00000,0.01428,0.00000,0.92839,-
0.00020
>>>>>
>>>>> so that FSTDEV=0 and OSTDEV=0
>>>>>
>>>>> Same happens for 3d QVAPOR
>>>>>
>>>>> fcst_field[] = [ "QVAPO(${itime}-${itime},3,*,*)" ] ;
>>>>> obs_field[] = ["MIXR/P850"]
>>>>>
>>>>>
>>>>> QVAPOR_P850_cnt.txt
>>>>>
FCST_VALID_BEG,FCST_VAR,FCST_LEV,OBS_VAR,OBS_LEV,OBTYPE,TOTAL,FBAR,FSTDEV,OBAR,OSTDEV,PR_CORR,ME
>>>>>
20100811_120000,QVAPOR,P850,MIXR,P850,ANYSFC,65,0.00976,0.00000,0.01058,0.00000,0.94893,-
0.00082
>>>>> 20100811_150000,QVAPOR,P850,MIXR,P850,ANYSFC,1,0.00665,-
99999.9,0.00597,-99999.9,-99999.9,0.00069
>>>>>
20100812_000000,QVAPOR,P850,MIXR,P850,ANYSFC,61,0.01093,0.00000,0.01033,0.00000,0.89522,0.00060
>>>>>
>>>>> standard deviations = 0 (works fine for other variables, problem
only for mixing ratio)
>>>>> Am I doing something wrong?
>>>>>
>>>>> 2. PR_CORR
>>>>> It seems that your threshold wrt the number of observations for
calculating correlations is > 1. Is it possible to change it to a
higher number?
>>>>> Are these  spatial  between obs and fcst for a given time?
>>>>>
>>>>> 3. My domain is roughly continental US without Alaska. Is it
normal that I get about 60-70  upper air (ANYSFC) standard
measurements (HGT, T, Q) at 12Z.
>>>>> ANYAIR is usually 1-3 measurements. Seems small.
>>>>> For surface observations the number is about 1000 every hour.
>>>>> Are these numbers reasonable?
>>>>>
>>>>> Thanks for helping,
>>>>> Mariusz
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>


------------------------------------------------


More information about the Met_help mailing list