[Met_help] [rt.rap.ucar.edu #59797] History for ensemble stat question
John Halley Gotway via RT
met_help at ucar.edu
Tue Mar 26 09:57:56 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I used version 3.0 before and have problem figuring out how to convert the old config files to the new ones.
I am trying to output separate histogram files for different surface and 3d variables. Can you help to convert from the old to the new type?
Thanks,
Mariusz
${MET}/bin/ensemble_stat $nens $wrffiles $configfile \
-point_obs $obsfile \
-outdir $STATSDIR -v 2
var=T2
itime=0
code=TMP/Z2
model = "WRF";
ens_field[] = [ "${var}(${itime},*,*)" ] ;
ens_thresh[] = [ "lt500.0" ] ;
vld_ens_thresh = 1.0;
vld_data_thresh = 1.0;
fcst_field[] = [ "${var}(${itime},*,*)" ] ;
obs_field[] = ["${code}"] ;
beg_ds = -300;
end_ds = 300;
message_type[] = [ "ANYSFC" ];
mask_grid[] = [ "FULL" ];
mask_poly[] = [ ] ;
mask_sid = "";
interp_method[] = [ "DW_MEAN" ];
interp_width[] = [ 2 ];
interp_flag = 1;
interp_thresh = 1.0;
output_flag[] = [ 2, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0 ];
//output_flag[] = [ 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12 ];
rng_type = "mt19937";
rng_seed = "";
grib_ptv = 129;
output_prefix = "${var}";
version = "V3.0.1";
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: ensemble stat question
From: John Halley Gotway
Time: Mon Jan 07 12:46:34 2013
Mariusz,
You can try using the attached ensemble-stat config file for METv4.0.
However, there's a slight problem in how the $code environment
variable is set in your example. For METv4.0, you'll need to
break it out into 2 pieces. So I'm assuming the following environment
variables are set:
var=T2
itime=0
obs_var=TMP
obs_lev=Z2
Please give the attached config file a try. This is just my best
guess, but you'll need to run it to make sure it does what you expect.
Please let me know how if goes. If you run into problems, I'm happy
to help.
Thanks,
John Halley Gotway
met_help at ucar.edu
On 01/07/2013 12:21 PM, Mariusz Pagowski via RT wrote:
>
> Mon Jan 07 12:21:26 2013: Request 59797 was acted upon.
> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
> Queue: met_help
> Subject: ensemble stat question
> Owner: Nobody
> Requestors: Mariusz.Pagowski at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59797 >
>
>
> Hello,
> I used version 3.0 before and have problem figuring out how to
convert the old config files to the new ones.
> I am trying to output separate histogram files for different surface
and 3d variables. Can you help to convert from the old to the new
type?
> Thanks,
> Mariusz
>
> ${MET}/bin/ensemble_stat $nens $wrffiles $configfile \
> -point_obs $obsfile \
> -outdir $STATSDIR -v 2
>
> var=T2
> itime=0
> code=TMP/Z2
>
> model = "WRF";
> ens_field[] = [ "${var}(${itime},*,*)" ] ;
> ens_thresh[] = [ "lt500.0" ] ;
> vld_ens_thresh = 1.0;
> vld_data_thresh = 1.0;
> fcst_field[] = [ "${var}(${itime},*,*)" ] ;
> obs_field[] = ["${code}"] ;
> beg_ds = -300;
> end_ds = 300;
> message_type[] = [ "ANYSFC" ];
> mask_grid[] = [ "FULL" ];
> mask_poly[] = [ ] ;
> mask_sid = "";
> interp_method[] = [ "DW_MEAN" ];
> interp_width[] = [ 2 ];
> interp_flag = 1;
> interp_thresh = 1.0;
> output_flag[] = [ 2, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0 ];
> //output_flag[] = [ 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12 ];
> rng_type = "mt19937";
> rng_seed = "";
> grib_ptv = 129;
> output_prefix = "${var}";
> version = "V3.0.1";
>
------------------------------------------------
Subject: ensemble stat question
From: John Halley Gotway
Time: Mon Jan 07 12:46:34 2013
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble-Stat configuration file.
//
// For additional information, see the MET_BASE/data/config/README
file.
//
////////////////////////////////////////////////////////////////////////////////
//
// Output model name to be written
//
model = "WRF";
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product fields to be processed
//
ens = {
ens_thresh = 1.0;
vld_thresh = 1.0;
field = [
{
name = "${var}";
level = "(${itime},*,*)";
cat_thresh = [ <500.0 ];
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
field = [
{
name = "${var}";
level = "(${itime},*,*)";
}
];
}
obs = {
message_type = [ "ANYSFC" ];
GRIB1_ptv = 129;
field = [
{
name = "${obs_var}";
level = "${obs_lev}";
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Point observation time window
//
obs_window = {
beg = -300;
end = 300;
}
////////////////////////////////////////////////////////////////////////////////
//
// Verification masking regions
//
mask = {
grid = [ "FULL" ];
poly = [];
sid = "";
};
////////////////////////////////////////////////////////////////////////////////
//
// Interpolation methods
//
interp = {
field = FCST;
vld_thresh = 1.0;
type = [
{
method = DW_MEAN;
width = 2;
}
];
};
////////////////////////////////////////////////////////////////////////////////
//
// Statistical output types
//
output_flag = {
rhist = BOTH;
orank = NONE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product output types
//
ensemble_flag = {
mean = TRUE;
stdev = TRUE;
minus = FALSE;
plus = FALSE;
min = FALSE;
max = FALSE;
range = TRUE;
vld_count = FALSE;
frequency = FALSE;
rank = FALSE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Random number generator
//
rng = {
type = "mt19937";
seed = "";
}
////////////////////////////////////////////////////////////////////////////////
duplicate_flag = NONE;
output_prefix = "${var}";
version = "V4.0";
////////////////////////////////////////////////////////////////////////////////
------------------------------------------------
Subject: ensemble stat question
From: Mariusz Pagowski
Time: Mon Jan 07 15:20:05 2013
John,
thanks, the config file works fine with the surface data but has
problems for pressure level data. Not sure about ANYSFC?
CAn you help what is wrong?
Thanks,
Mariusz
------------------------------------------------
Subject: ensemble stat question
From: Mariusz Pagowski
Time: Mon Jan 07 15:20:05 2013
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble-Stat configuration file.
//
// For additional information, see the MET_BASE/data/config/README
file.
//
////////////////////////////////////////////////////////////////////////////////
//
// Output model name to be written
//
model = "WRF";
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product fields to be processed
//
ens = {
ens_thresh = 1.0;
vld_thresh = 1.0;
field = [
{
name = "TT";
level = "(0,0,*,*)";
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
field = [
{
name = "TT";
level = "(0,0,*,*)";
}
];
}
obs = {
message_type = [ "ANYSFC" ];
GRIB1_ptv = 129;
field = [
{
name = "TMP";
level = "P925"
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Point observation time window
//
obs_window = {
beg = -300;
end = 300;
}
////////////////////////////////////////////////////////////////////////////////
//
// Verification masking regions
//
mask = {
grid = [ "FULL" ];
poly = [];
sid = "";
};
////////////////////////////////////////////////////////////////////////////////
//
// Interpolation methods
//
interp = {
field = FCST;
vld_thresh = 1.0;
type = [
{
method = DW_MEAN;
width = 2;
}
];
};
////////////////////////////////////////////////////////////////////////////////
//
// Statistical output types
//
output_flag = {
rhist = BOTH;
orank = NONE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product output types
//
ensemble_flag = {
mean = TRUE;
stdev = TRUE;
minus = FALSE;
plus = FALSE;
min = FALSE;
max = FALSE;
range = TRUE;
vld_count = FALSE;
frequency = FALSE;
rank = FALSE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Random number generator
//
rng = {
type = "mt19937";
seed = "";
}
////////////////////////////////////////////////////////////////////////////////
duplicate_flag = NONE;
output_prefix = "TT";
version = "V4.0";
////////////////////////////////////////////////////////////////////////////////
------------------------------------------------
Subject: ensemble stat question
From: Mariusz Pagowski
Time: Mon Jan 07 15:20:05 2013
On Jan 7, 2013, at 12:46 PM, John Halley Gotway via RT wrote:
> Mariusz,
>
> You can try using the attached ensemble-stat config file for
METv4.0. However, there's a slight problem in how the $code
environment variable is set in your example. For METv4.0, you'll need
to
> break it out into 2 pieces. So I'm assuming the following
environment variables are set:
>
> var=T2
> itime=0
> obs_var=TMP
> obs_lev=Z2
>
> Please give the attached config file a try. This is just my best
guess, but you'll need to run it to make sure it does what you expect.
>
> Please let me know how if goes. If you run into problems, I'm happy
to help.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> On 01/07/2013 12:21 PM, Mariusz Pagowski via RT wrote:
>>
>> Mon Jan 07 12:21:26 2013: Request 59797 was acted upon.
>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>> Queue: met_help
>> Subject: ensemble stat question
>> Owner: Nobody
>> Requestors: Mariusz.Pagowski at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59797 >
>>
>>
>> Hello,
>> I used version 3.0 before and have problem figuring out how to
convert the old config files to the new ones.
>> I am trying to output separate histogram files for different
surface and 3d variables. Can you help to convert from the old to the
new type?
>> Thanks,
>> Mariusz
>>
>> ${MET}/bin/ensemble_stat $nens $wrffiles $configfile \
>> -point_obs $obsfile \
>> -outdir $STATSDIR -v 2
>>
>> var=T2
>> itime=0
>> code=TMP/Z2
>>
>> model = "WRF";
>> ens_field[] = [ "${var}(${itime},*,*)" ] ;
>> ens_thresh[] = [ "lt500.0" ] ;
>> vld_ens_thresh = 1.0;
>> vld_data_thresh = 1.0;
>> fcst_field[] = [ "${var}(${itime},*,*)" ] ;
>> obs_field[] = ["${code}"] ;
>> beg_ds = -300;
>> end_ds = 300;
>> message_type[] = [ "ANYSFC" ];
>> mask_grid[] = [ "FULL" ];
>> mask_poly[] = [ ] ;
>> mask_sid = "";
>> interp_method[] = [ "DW_MEAN" ];
>> interp_width[] = [ 2 ];
>> interp_flag = 1;
>> interp_thresh = 1.0;
>> output_flag[] = [ 2, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0 ];
>> //output_flag[] = [ 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12 ];
>> rng_type = "mt19937";
>> rng_seed = "";
>> grib_ptv = 129;
>> output_prefix = "${var}";
>> version = "V3.0.1";
>>
>
> <EnsembleStatConfig.txt>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59797] ensemble stat question
From: John Halley Gotway
Time: Mon Jan 07 15:45:49 2013
Mariusz,
Yes, I suspect it has to do with the message type. ANYSFC is for
observations at the surface, both over land (ADPSFC) and water
(SFCSHP).
Please try rerunning with the verbosity level turned higher. For
example,
try "-v 4". For each verification task it performs, I believe it'll
give
you counts for why observations were or were not used. That's often
the
easiest way to figure out why you're not finding matched pairs when
you
expect to.
If you're still stuck after that, I'll have you send me data and try
to
debug.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59797 >
>
> John,
> thanks, the config file works fine with the surface data but has
problems
> for pressure level data. Not sure about ANYSFC?
> CAn you help what is wrong?
> Thanks,
> Mariusz
>
>
>
>
> On Jan 7, 2013, at 12:46 PM, John Halley Gotway via RT wrote:
>
>> Mariusz,
>>
>> You can try using the attached ensemble-stat config file for
METv4.0.
>> However, there's a slight problem in how the $code environment
variable
>> is set in your example. For METv4.0, you'll need to
>> break it out into 2 pieces. So I'm assuming the following
environment
>> variables are set:
>>
>> var=T2
>> itime=0
>> obs_var=TMP
>> obs_lev=Z2
>>
>> Please give the attached config file a try. This is just my best
guess,
>> but you'll need to run it to make sure it does what you expect.
>>
>> Please let me know how if goes. If you run into problems, I'm
happy to
>> help.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>> On 01/07/2013 12:21 PM, Mariusz Pagowski via RT wrote:
>>>
>>> Mon Jan 07 12:21:26 2013: Request 59797 was acted upon.
>>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>> Queue: met_help
>>> Subject: ensemble stat question
>>> Owner: Nobody
>>> Requestors: Mariusz.Pagowski at noaa.gov
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59797 >
>>>
>>>
>>> Hello,
>>> I used version 3.0 before and have problem figuring out how to
convert
>>> the old config files to the new ones.
>>> I am trying to output separate histogram files for different
surface
>>> and 3d variables. Can you help to convert from the old to the new
type?
>>> Thanks,
>>> Mariusz
>>>
>>> ${MET}/bin/ensemble_stat $nens $wrffiles $configfile \
>>> -point_obs $obsfile \
>>> -outdir $STATSDIR -v 2
>>>
>>> var=T2
>>> itime=0
>>> code=TMP/Z2
>>>
>>> model = "WRF";
>>> ens_field[] = [ "${var}(${itime},*,*)" ] ;
>>> ens_thresh[] = [ "lt500.0" ] ;
>>> vld_ens_thresh = 1.0;
>>> vld_data_thresh = 1.0;
>>> fcst_field[] = [ "${var}(${itime},*,*)" ] ;
>>> obs_field[] = ["${code}"] ;
>>> beg_ds = -300;
>>> end_ds = 300;
>>> message_type[] = [ "ANYSFC" ];
>>> mask_grid[] = [ "FULL" ];
>>> mask_poly[] = [ ] ;
>>> mask_sid = "";
>>> interp_method[] = [ "DW_MEAN" ];
>>> interp_width[] = [ 2 ];
>>> interp_flag = 1;
>>> interp_thresh = 1.0;
>>> output_flag[] = [ 2, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0 ];
>>> //output_flag[] = [ 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12 ];
>>> rng_type = "mt19937";
>>> rng_seed = "";
>>> grib_ptv = 129;
>>> output_prefix = "${var}";
>>> version = "V3.0.1";
>>>
>>
>> <EnsembleStatConfig.txt>
>
>
------------------------------------------------
Subject: ensemble stat question
From: Mariusz Pagowski
Time: Mon Feb 04 09:25:26 2013
Not really, I sent an e-mail but did not get any response.
Thakns,
MAriusz
Subject: Re: [rt.rap.ucar.edu #59920] ensemble stat question
From: Mariusz Pagowski - NOAA Affiliate <Mariusz.Pagowski at noaa.gov>
Date: January 22, 2013 1:22:25 PM MST
To: met_help at ucar.edu
John,
please forward this question to Tressa.
Is it possible to calculate ensemble spread in obs space or some proxy
for it from available output statistics of any of the MET programs?
I am looking for non-categorical statistics such as e.g. ensemble
spread of surface wind, temperature, or spread e.g. at 850 mb
Thanks,
Mariusz
On Feb 4, 2013, at 9:08 AM, John Halley Gotway via RT wrote:
> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59797] Resolved: ensemble stat question
From: John Halley Gotway
Time: Mon Feb 04 10:12:34 2013
Mariusz,
Sorry for the confusion. We actually had two tickets open in your
name.
I resolved the software-related one that was assigned to me.
There still is another ticket assigned to Tressa.
I'll ping her on it.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59797 >
>
> Not really, I sent an e-mail but did not get any response.
> Thakns,
> MAriusz
>
> Subject: Re: [rt.rap.ucar.edu #59920] ensemble stat question
> From: Mariusz Pagowski - NOAA Affiliate
<Mariusz.Pagowski at noaa.gov>
> Date: January 22, 2013 1:22:25 PM MST
> To: met_help at ucar.edu
>
> John,
> please forward this question to Tressa.
> Is it possible to calculate ensemble spread in obs space or some
proxy for
> it from available output statistics of any of the MET programs?
> I am looking for non-categorical statistics such as e.g. ensemble
spread
> of surface wind, temperature, or spread e.g. at 850 mb
> Thanks,
> Mariusz
>
>
> On Feb 4, 2013, at 9:08 AM, John Halley Gotway via RT wrote:
>
>> According to our records, your request has been resolved. If you
have
>> any
>> further questions or concerns, please respond to this message.
>
>
------------------------------------------------
More information about the Met_help
mailing list