[Met_help] [rt.rap.ucar.edu #96270] History for Is there a way to treat different heights as one layer in config file?
John Halley Gotway via RT
met_help at ucar.edu
Thu Aug 27 14:31:15 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am using "ensemble_stat" for volcano ash verification and I got a
problem: my model forecast is in grib2 format with only one layer. The obs.
data is converted from "ascii2nc", there is only lat/lon, height, and
volcano ash inside. I want to treat all height as one layer as the forecast
data does. How to set that in the config file? I tried :
level=[ "L6000-9000" ] ;
and
level=[ "*,*" ] ;
but none of them work. Should I have to give a fake SAME height for all
sites and regenerate the *nc files?
My script is:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
and the config file is:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
Thank you.
Binyu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Mon Aug 17 09:47:16 2020
Binyu,
I see you have a question about specifying level information in
Ensemble-Stat. I hear that your GRIB2 model output has a single record
for
volcanic ash, whereas you have point observations at multiple heights.
So
you want to extract a single forecast record and match it to multiple
observation heights. Please correct me if that description isn't
accurate.
The good news is that this is easily done via the Ensemble-Stat
configuration file. Notice that that config file contains separate
dictionaries for the forecast and observation settings, named "fcst"
and
"obs", respectively. In the default config file, we just copy the fcst
settings over to the obs by saying "obs = fcst". But for this case,
we'll
want to specify a full obs dictionary with a different "level"
setting. By
way of example, here's how you'd compare forecasts of TMP at 500mb to
TMP
observations at heights between 0 and 10000. Note that the 'Z'
specifies a
height level whereas 'P' specifies a pressure level:
fcst = { field = [
{
name = "TMP";
level = [ "P500" ];
}
];
}
obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; } ]; }
Hopefully you can adapt this example to your case. You just need to be
sure
that you specify the same number of forecast and observation fields.
Please try reconfiguring Ensemble-Stat and rerunning and let me know
how it
goes.
Thanks,
John Halley Gotway
On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> Transaction: Ticket created by binyu.wang at noaa.gov
> Queue: met_help
> Subject: Is there a way to treat different heights as one layer
in
> config file?
> Owner: Nobody
> Requestors: binyu.wang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
>
> Hello,
>
> I am using "ensemble_stat" for volcano ash verification and I got a
> problem: my model forecast is in grib2 format with only one layer.
The obs.
> data is converted from "ascii2nc", there is only lat/lon, height,
and
> volcano ash inside. I want to treat all height as one layer as the
forecast
> data does. How to set that in the config file? I tried :
> level=[ "L6000-9000" ] ;
> and
> level=[ "*,*" ] ;
> but none of them work. Should I have to give a fake SAME height for
all
> sites and regenerate the *nc files?
>
> My script is:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> and the config file is:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
>
> Thank you.
> Binyu
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Mon Aug 17 10:41:47 2020
John,
I did try that. Unfortunately, it doesn't work.
Thank you.
Binyu
On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Binyu,
>
> I see you have a question about specifying level information in
> Ensemble-Stat. I hear that your GRIB2 model output has a single
record for
> volcanic ash, whereas you have point observations at multiple
heights. So
> you want to extract a single forecast record and match it to
multiple
> observation heights. Please correct me if that description isn't
accurate.
>
> The good news is that this is easily done via the Ensemble-Stat
> configuration file. Notice that that config file contains separate
> dictionaries for the forecast and observation settings, named "fcst"
and
> "obs", respectively. In the default config file, we just copy the
fcst
> settings over to the obs by saying "obs = fcst". But for this case,
we'll
> want to specify a full obs dictionary with a different "level"
setting. By
> way of example, here's how you'd compare forecasts of TMP at 500mb
to TMP
> observations at heights between 0 and 10000. Note that the 'Z'
specifies a
> height level whereas 'P' specifies a pressure level:
>
> fcst = { field = [
> {
> name = "TMP";
> level = [ "P500" ];
> }
> ];
> }
> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; } ]; }
> Hopefully you can adapt this example to your case. You just need to
be sure
> that you specify the same number of forecast and observation fields.
>
> Please try reconfiguring Ensemble-Stat and rerunning and let me know
how it
> goes.
>
> Thanks,
> John Halley Gotway
>
>
> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> > Transaction: Ticket created by binyu.wang at noaa.gov
> > Queue: met_help
> > Subject: Is there a way to treat different heights as one
layer in
> > config file?
> > Owner: Nobody
> > Requestors: binyu.wang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >
> >
> > Hello,
> >
> > I am using "ensemble_stat" for volcano ash verification and I got
a
> > problem: my model forecast is in grib2 format with only one layer.
The
> obs.
> > data is converted from "ascii2nc", there is only lat/lon, height,
and
> > volcano ash inside. I want to treat all height as one layer as the
> forecast
> > data does. How to set that in the config file? I tried :
> > level=[ "L6000-9000" ] ;
> > and
> > level=[ "*,*" ] ;
> > but none of them work. Should I have to give a fake SAME height
for all
> > sites and regenerate the *nc files?
> >
> > My script is:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> > and the config file is:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> >
> > Thank you.
> > Binyu
> >
> >
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Mon Aug 17 10:56:12 2020
I know WCOSS/dev is down today, so I tried to upload the config file
and
one of model (grib2 file) and obs data funder my last name "wang"
following
the rule:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp.
Let me know if further information is needed.
Thank you
Binyu
On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate <
binyu.wang at noaa.gov> wrote:
> John,
>
> I did try that. Unfortunately, it doesn't work.
>
> Thank you.
> Binyu
>
> On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binyu,
>>
>> I see you have a question about specifying level information in
>> Ensemble-Stat. I hear that your GRIB2 model output has a single
record for
>> volcanic ash, whereas you have point observations at multiple
heights. So
>> you want to extract a single forecast record and match it to
multiple
>> observation heights. Please correct me if that description isn't
accurate.
>>
>> The good news is that this is easily done via the Ensemble-Stat
>> configuration file. Notice that that config file contains separate
>> dictionaries for the forecast and observation settings, named
"fcst" and
>> "obs", respectively. In the default config file, we just copy the
fcst
>> settings over to the obs by saying "obs = fcst". But for this case,
we'll
>> want to specify a full obs dictionary with a different "level"
setting. By
>> way of example, here's how you'd compare forecasts of TMP at 500mb
to TMP
>> observations at heights between 0 and 10000. Note that the 'Z'
specifies a
>> height level whereas 'P' specifies a pressure level:
>>
>> fcst = { field = [
>> {
>> name = "TMP";
>> level = [ "P500" ];
>> }
>> ];
>> }
>> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; } ]; }
>> Hopefully you can adapt this example to your case. You just need to
be
>> sure
>> that you specify the same number of forecast and observation
fields.
>>
>> Please try reconfiguring Ensemble-Stat and rerunning and let me
know how
>> it
>> goes.
>>
>> Thanks,
>> John Halley Gotway
>>
>>
>> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
>> > Transaction: Ticket created by binyu.wang at noaa.gov
>> > Queue: met_help
>> > Subject: Is there a way to treat different heights as one
layer in
>> > config file?
>> > Owner: Nobody
>> > Requestors: binyu.wang at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>> >
>> >
>> > Hello,
>> >
>> > I am using "ensemble_stat" for volcano ash verification and I got
a
>> > problem: my model forecast is in grib2 format with only one
layer. The
>> obs.
>> > data is converted from "ascii2nc", there is only lat/lon, height,
and
>> > volcano ash inside. I want to treat all height as one layer as
the
>> forecast
>> > data does. How to set that in the config file? I tried :
>> > level=[ "L6000-9000" ] ;
>> > and
>> > level=[ "*,*" ] ;
>> > but none of them work. Should I have to give a fake SAME height
for all
>> > sites and regenerate the *nc files?
>> >
>> > My script is:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
>> > and the config file is:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
>> >
>> > Thank you.
>> > Binyu
>> >
>> >
>>
>>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Mon Aug 17 15:40:05 2020
Binyu,
Thanks for posting some sample data for me to take a look at. Looking
at
the "log" file, I see you've run Ensemble-Stat 5 times for hours 18,
19,
20, 21, and 22. You sent the sample files for 18 hour output time, and
I
see that Ensemble-Stat produced 91 matched pairs for that time:
egrep 'matched pairs|\.stat' log
DEBUG 2: Processing point verification VAFTD/L14000-0 versus
ash/Z10000-0,
for observation type ADPUPA, over region FULL, for interpolation
method
NEAREST(1), using 91 matched pairs.
I checked the NetCDF file that you sent me and see that it contains
exactly
91 point observations for that time:
ncdump -h goes16_ash_20190225-180000.nc
netcdf goes16_ash_20190225-180000 {
dimensions:
...
nhdr = 91 ;
So Ensemble-Stat is using all available point observations you are
passing
to it.
Looks to me like it's working as expected. Is there some problem I'm
not
understanding?
John
On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
> I know WCOSS/dev is down today, so I tried to upload the config file
and
> one of model (grib2 file) and obs data funder my last name "wang"
following
> the rule:
>
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> .
> Let me know if further information is needed.
>
> Thank you
> Binyu
>
> On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate <
> binyu.wang at noaa.gov> wrote:
>
> > John,
> >
> > I did try that. Unfortunately, it doesn't work.
> >
> > Thank you.
> > Binyu
> >
> > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binyu,
> >>
> >> I see you have a question about specifying level information in
> >> Ensemble-Stat. I hear that your GRIB2 model output has a single
record
> for
> >> volcanic ash, whereas you have point observations at multiple
heights.
> So
> >> you want to extract a single forecast record and match it to
multiple
> >> observation heights. Please correct me if that description isn't
> accurate.
> >>
> >> The good news is that this is easily done via the Ensemble-Stat
> >> configuration file. Notice that that config file contains
separate
> >> dictionaries for the forecast and observation settings, named
"fcst" and
> >> "obs", respectively. In the default config file, we just copy the
fcst
> >> settings over to the obs by saying "obs = fcst". But for this
case,
> we'll
> >> want to specify a full obs dictionary with a different "level"
setting.
> By
> >> way of example, here's how you'd compare forecasts of TMP at
500mb to
> TMP
> >> observations at heights between 0 and 10000. Note that the 'Z'
> specifies a
> >> height level whereas 'P' specifies a pressure level:
> >>
> >> fcst = { field = [
> >> {
> >> name = "TMP";
> >> level = [ "P500" ];
> >> }
> >> ];
> >> }
> >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; } ]; }
> >> Hopefully you can adapt this example to your case. You just need
to be
> >> sure
> >> that you specify the same number of forecast and observation
fields.
> >>
> >> Please try reconfiguring Ensemble-Stat and rerunning and let me
know how
> >> it
> >> goes.
> >>
> >> Thanks,
> >> John Halley Gotway
> >>
> >>
> >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> >> > Transaction: Ticket created by binyu.wang at noaa.gov
> >> > Queue: met_help
> >> > Subject: Is there a way to treat different heights as one
layer
> in
> >> > config file?
> >> > Owner: Nobody
> >> > Requestors: binyu.wang at noaa.gov
> >> > Status: new
> >> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> >
> >> >
> >> >
> >> > Hello,
> >> >
> >> > I am using "ensemble_stat" for volcano ash verification and I
got a
> >> > problem: my model forecast is in grib2 format with only one
layer. The
> >> obs.
> >> > data is converted from "ascii2nc", there is only lat/lon,
height, and
> >> > volcano ash inside. I want to treat all height as one layer as
the
> >> forecast
> >> > data does. How to set that in the config file? I tried :
> >> > level=[ "L6000-9000" ] ;
> >> > and
> >> > level=[ "*,*" ] ;
> >> > but none of them work. Should I have to give a fake SAME
height for
> all
> >> > sites and regenerate the *nc files?
> >> >
> >> > My script is:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> >> > and the config file is:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> >> >
> >> > Thank you.
> >> > Binyu
> >> >
> >> >
> >>
> >>
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Mon Aug 17 16:25:11 2020
But why I got something weird in the log file, e.g:
WARNING: check_hdr_str() -> null string!
WARNING:
WARNING: MetGrib2DataFile::data_plane() -> No matching record found
for
'VAFTD/L14000-0'
WARNING:
WARNING:
WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0" not found
in
file
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
Binyu
On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> Thanks for posting some sample data for me to take a look at.
Looking at
> the "log" file, I see you've run Ensemble-Stat 5 times for hours 18,
19,
> 20, 21, and 22. You sent the sample files for 18 hour output time,
and I
> see that Ensemble-Stat produced 91 matched pairs for that time:
>
> egrep 'matched pairs|\.stat' log
>
> DEBUG 2: Processing point verification VAFTD/L14000-0 versus
ash/Z10000-0,
> for observation type ADPUPA, over region FULL, for interpolation
method
> NEAREST(1), using 91 matched pairs.
>
> I checked the NetCDF file that you sent me and see that it contains
exactly
> 91 point observations for that time:
>
> ncdump -h goes16_ash_20190225-180000.nc
>
> netcdf goes16_ash_20190225-180000 {
>
> dimensions:
>
> ...
>
> nhdr = 91 ;
>
>
> So Ensemble-Stat is using all available point observations you are
passing
> to it.
>
> Looks to me like it's working as expected. Is there some problem I'm
not
> understanding?
>
> John
>
> On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >
> > I know WCOSS/dev is down today, so I tried to upload the config
file and
> > one of model (grib2 file) and obs data funder my last name "wang"
> following
> > the rule:
> >
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > .
> > Let me know if further information is needed.
> >
> > Thank you
> > Binyu
> >
> > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate <
> > binyu.wang at noaa.gov> wrote:
> >
> > > John,
> > >
> > > I did try that. Unfortunately, it doesn't work.
> > >
> > > Thank you.
> > > Binyu
> > >
> > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binyu,
> > >>
> > >> I see you have a question about specifying level information in
> > >> Ensemble-Stat. I hear that your GRIB2 model output has a single
record
> > for
> > >> volcanic ash, whereas you have point observations at multiple
heights.
> > So
> > >> you want to extract a single forecast record and match it to
multiple
> > >> observation heights. Please correct me if that description
isn't
> > accurate.
> > >>
> > >> The good news is that this is easily done via the Ensemble-Stat
> > >> configuration file. Notice that that config file contains
separate
> > >> dictionaries for the forecast and observation settings, named
"fcst"
> and
> > >> "obs", respectively. In the default config file, we just copy
the fcst
> > >> settings over to the obs by saying "obs = fcst". But for this
case,
> > we'll
> > >> want to specify a full obs dictionary with a different "level"
> setting.
> > By
> > >> way of example, here's how you'd compare forecasts of TMP at
500mb to
> > TMP
> > >> observations at heights between 0 and 10000. Note that the 'Z'
> > specifies a
> > >> height level whereas 'P' specifies a pressure level:
> > >>
> > >> fcst = { field = [
> > >> {
> > >> name = "TMP";
> > >> level = [ "P500" ];
> > >> }
> > >> ];
> > >> }
> > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; } ];
}
> > >> Hopefully you can adapt this example to your case. You just
need to be
> > >> sure
> > >> that you specify the same number of forecast and observation
fields.
> > >>
> > >> Please try reconfiguring Ensemble-Stat and rerunning and let me
know
> how
> > >> it
> > >> goes.
> > >>
> > >> Thanks,
> > >> John Halley Gotway
> > >>
> > >>
> > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> > >> > Queue: met_help
> > >> > Subject: Is there a way to treat different heights as
one layer
> > in
> > >> > config file?
> > >> > Owner: Nobody
> > >> > Requestors: binyu.wang at noaa.gov
> > >> > Status: new
> > >> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> > >
> > >> >
> > >> >
> > >> > Hello,
> > >> >
> > >> > I am using "ensemble_stat" for volcano ash verification and I
got a
> > >> > problem: my model forecast is in grib2 format with only one
layer.
> The
> > >> obs.
> > >> > data is converted from "ascii2nc", there is only lat/lon,
height,
> and
> > >> > volcano ash inside. I want to treat all height as one layer
as the
> > >> forecast
> > >> > data does. How to set that in the config file? I tried :
> > >> > level=[ "L6000-9000" ] ;
> > >> > and
> > >> > level=[ "*,*" ] ;
> > >> > but none of them work. Should I have to give a fake SAME
height for
> > all
> > >> > sites and regenerate the *nc files?
> > >> >
> > >> > My script is:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> > >> > and the config file is:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> > >> >
> > >> > Thank you.
> > >> > Binyu
> > >> >
> > >> >
> > >>
> > >>
> >
> >
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Mon Aug 17 17:14:43 2020
Binyu,
You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7. However,
the
GRIB file you sent does not have output for 1 hour:
wgrib2 T_cdump1.001.grib2
1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour ave
fcst:
2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3 hour ave
fcst:
3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4 hour ave
fcst:
4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5 hour ave
fcst:
5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6 hour ave
fcst:
6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7 hour ave
fcst:
Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So that's
why
you're getting no output for a forecast hour of 1.
Next, for that warning:
WARNING: check_hdr_str() -> null string!
I'm not exactly sure. Please look in one of your output files and
check to
see if one of the first 21 header columns contains an empty string. I
wonder if its perhaps the "OBS_UNITS" column? And I suspect that
Ensemble-Stat is replacing the empty string with "NA".
I see that you're using met-9.1_beta2, and I'd recommend switching to
the
released version of met-9.1 as soon as it's available on the machine
you're
using.
John
On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
> But why I got something weird in the log file, e.g:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> WARNING: MetGrib2DataFile::data_plane() -> No matching record found
for
> 'VAFTD/L14000-0'
>
> WARNING:
>
> WARNING:
>
> WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0" not
found in
> file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
>
>
> Binyu
>
> On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > Thanks for posting some sample data for me to take a look at.
Looking at
> > the "log" file, I see you've run Ensemble-Stat 5 times for hours
18, 19,
> > 20, 21, and 22. You sent the sample files for 18 hour output time,
and I
> > see that Ensemble-Stat produced 91 matched pairs for that time:
> >
> > egrep 'matched pairs|\.stat' log
> >
> > DEBUG 2: Processing point verification VAFTD/L14000-0 versus
> ash/Z10000-0,
> > for observation type ADPUPA, over region FULL, for interpolation
method
> > NEAREST(1), using 91 matched pairs.
> >
> > I checked the NetCDF file that you sent me and see that it
contains
> exactly
> > 91 point observations for that time:
> >
> > ncdump -h goes16_ash_20190225-180000.nc
> >
> > netcdf goes16_ash_20190225-180000 {
> >
> > dimensions:
> >
> > ...
> >
> > nhdr = 91 ;
> >
> >
> > So Ensemble-Stat is using all available point observations you are
> passing
> > to it.
> >
> > Looks to me like it's working as expected. Is there some problem
I'm not
> > understanding?
> >
> > John
> >
> > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> > >
> > > I know WCOSS/dev is down today, so I tried to upload the config
file
> and
> > > one of model (grib2 file) and obs data funder my last name
"wang"
> > following
> > > the rule:
> > >
> > >
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > > .
> > > Let me know if further information is needed.
> > >
> > > Thank you
> > > Binyu
> > >
> > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate <
> > > binyu.wang at noaa.gov> wrote:
> > >
> > > > John,
> > > >
> > > > I did try that. Unfortunately, it doesn't work.
> > > >
> > > > Thank you.
> > > > Binyu
> > > >
> > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Binyu,
> > > >>
> > > >> I see you have a question about specifying level information
in
> > > >> Ensemble-Stat. I hear that your GRIB2 model output has a
single
> record
> > > for
> > > >> volcanic ash, whereas you have point observations at multiple
> heights.
> > > So
> > > >> you want to extract a single forecast record and match it to
> multiple
> > > >> observation heights. Please correct me if that description
isn't
> > > accurate.
> > > >>
> > > >> The good news is that this is easily done via the Ensemble-
Stat
> > > >> configuration file. Notice that that config file contains
separate
> > > >> dictionaries for the forecast and observation settings, named
"fcst"
> > and
> > > >> "obs", respectively. In the default config file, we just copy
the
> fcst
> > > >> settings over to the obs by saying "obs = fcst". But for this
case,
> > > we'll
> > > >> want to specify a full obs dictionary with a different
"level"
> > setting.
> > > By
> > > >> way of example, here's how you'd compare forecasts of TMP at
500mb
> to
> > > TMP
> > > >> observations at heights between 0 and 10000. Note that the
'Z'
> > > specifies a
> > > >> height level whereas 'P' specifies a pressure level:
> > > >>
> > > >> fcst = { field = [
> > > >> {
> > > >> name = "TMP";
> > > >> level = [ "P500" ];
> > > >> }
> > > >> ];
> > > >> }
> > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; }
]; }
> > > >> Hopefully you can adapt this example to your case. You just
need to
> be
> > > >> sure
> > > >> that you specify the same number of forecast and observation
fields.
> > > >>
> > > >> Please try reconfiguring Ensemble-Stat and rerunning and let
me know
> > how
> > > >> it
> > > >> goes.
> > > >>
> > > >> Thanks,
> > > >> John Halley Gotway
> > > >>
> > > >>
> > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> > > >> > Queue: met_help
> > > >> > Subject: Is there a way to treat different heights as
one
> layer
> > > in
> > > >> > config file?
> > > >> > Owner: Nobody
> > > >> > Requestors: binyu.wang at noaa.gov
> > > >> > Status: new
> > > >> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> > > >
> > > >> >
> > > >> >
> > > >> > Hello,
> > > >> >
> > > >> > I am using "ensemble_stat" for volcano ash verification and
I got
> a
> > > >> > problem: my model forecast is in grib2 format with only one
layer.
> > The
> > > >> obs.
> > > >> > data is converted from "ascii2nc", there is only lat/lon,
height,
> > and
> > > >> > volcano ash inside. I want to treat all height as one layer
as the
> > > >> forecast
> > > >> > data does. How to set that in the config file? I tried :
> > > >> > level=[ "L6000-9000" ] ;
> > > >> > and
> > > >> > level=[ "*,*" ] ;
> > > >> > but none of them work. Should I have to give a fake SAME
height
> for
> > > all
> > > >> > sites and regenerate the *nc files?
> > > >> >
> > > >> > My script is:
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> > > >> > and the config file is:
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> > > >> >
> > > >> > Thank you.
> > > >> > Binyu
> > > >> >
> > > >> >
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Mon Aug 17 18:09:30 2020
Well, the OBS_UNIT has "NA" for the whole column. See th attachment.
However I recall that I have OBS_UNIT as "NA" for other runnings, but
I
didn't get this warning.
Thank you for the suggestion for the version switch from met-9.1_beta2
to
met-9.1. Just curious: is it because the beta2 is still under test?
Binyu
On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7. However,
the
> GRIB file you sent does not have output for 1 hour:
>
> wgrib2 T_cdump1.001.grib2
>
> 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour ave
fcst:
>
> 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3 hour ave
fcst:
>
> 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4 hour
ave fcst:
>
> 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5 hour
ave fcst:
>
> 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6 hour
ave fcst:
>
> 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7 hour
ave fcst:
>
>
> Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So that's
why
> you're getting no output for a forecast hour of 1.
>
> Next, for that warning:
>
> WARNING: check_hdr_str() -> null string!
>
>
> I'm not exactly sure. Please look in one of your output files and
check to
> see if one of the first 21 header columns contains an empty string.
I
> wonder if its perhaps the "OBS_UNITS" column? And I suspect that
> Ensemble-Stat is replacing the empty string with "NA".
>
> I see that you're using met-9.1_beta2, and I'd recommend switching
to the
> released version of met-9.1 as soon as it's available on the machine
you're
> using.
>
> John
>
>
>
> On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >
> > But why I got something weird in the log file, e.g:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > WARNING: MetGrib2DataFile::data_plane() -> No matching record
found for
> > 'VAFTD/L14000-0'
> >
> > WARNING:
> >
> > WARNING:
> >
> > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0" not
found in
> > file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
> >
> >
> > Binyu
> >
> > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > >
> > > Thanks for posting some sample data for me to take a look at.
Looking
> at
> > > the "log" file, I see you've run Ensemble-Stat 5 times for hours
18,
> 19,
> > > 20, 21, and 22. You sent the sample files for 18 hour output
time, and
> I
> > > see that Ensemble-Stat produced 91 matched pairs for that time:
> > >
> > > egrep 'matched pairs|\.stat' log
> > >
> > > DEBUG 2: Processing point verification VAFTD/L14000-0 versus
> > ash/Z10000-0,
> > > for observation type ADPUPA, over region FULL, for interpolation
method
> > > NEAREST(1), using 91 matched pairs.
> > >
> > > I checked the NetCDF file that you sent me and see that it
contains
> > exactly
> > > 91 point observations for that time:
> > >
> > > ncdump -h goes16_ash_20190225-180000.nc
> > >
> > > netcdf goes16_ash_20190225-180000 {
> > >
> > > dimensions:
> > >
> > > ...
> > >
> > > nhdr = 91 ;
> > >
> > >
> > > So Ensemble-Stat is using all available point observations you
are
> > passing
> > > to it.
> > >
> > > Looks to me like it's working as expected. Is there some problem
I'm
> not
> > > understanding?
> > >
> > > John
> > >
> > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
>
> > > >
> > > > I know WCOSS/dev is down today, so I tried to upload the
config file
> > and
> > > > one of model (grib2 file) and obs data funder my last name
"wang"
> > > following
> > > > the rule:
> > > >
> > > >
> > >
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > > > .
> > > > Let me know if further information is needed.
> > > >
> > > > Thank you
> > > > Binyu
> > > >
> > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate <
> > > > binyu.wang at noaa.gov> wrote:
> > > >
> > > > > John,
> > > > >
> > > > > I did try that. Unfortunately, it doesn't work.
> > > > >
> > > > > Thank you.
> > > > > Binyu
> > > > >
> > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Binyu,
> > > > >>
> > > > >> I see you have a question about specifying level
information in
> > > > >> Ensemble-Stat. I hear that your GRIB2 model output has a
single
> > record
> > > > for
> > > > >> volcanic ash, whereas you have point observations at
multiple
> > heights.
> > > > So
> > > > >> you want to extract a single forecast record and match it
to
> > multiple
> > > > >> observation heights. Please correct me if that description
isn't
> > > > accurate.
> > > > >>
> > > > >> The good news is that this is easily done via the Ensemble-
Stat
> > > > >> configuration file. Notice that that config file contains
separate
> > > > >> dictionaries for the forecast and observation settings,
named
> "fcst"
> > > and
> > > > >> "obs", respectively. In the default config file, we just
copy the
> > fcst
> > > > >> settings over to the obs by saying "obs = fcst". But for
this
> case,
> > > > we'll
> > > > >> want to specify a full obs dictionary with a different
"level"
> > > setting.
> > > > By
> > > > >> way of example, here's how you'd compare forecasts of TMP
at 500mb
> > to
> > > > TMP
> > > > >> observations at heights between 0 and 10000. Note that the
'Z'
> > > > specifies a
> > > > >> height level whereas 'P' specifies a pressure level:
> > > > >>
> > > > >> fcst = { field = [
> > > > >> {
> > > > >> name = "TMP";
> > > > >> level = [ "P500" ];
> > > > >> }
> > > > >> ];
> > > > >> }
> > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ]; }
]; }
> > > > >> Hopefully you can adapt this example to your case. You just
need
> to
> > be
> > > > >> sure
> > > > >> that you specify the same number of forecast and
observation
> fields.
> > > > >>
> > > > >> Please try reconfiguring Ensemble-Stat and rerunning and
let me
> know
> > > how
> > > > >> it
> > > > >> goes.
> > > > >>
> > > > >> Thanks,
> > > > >> John Halley Gotway
> > > > >>
> > > > >>
> > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
> > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> > > > >> > Queue: met_help
> > > > >> > Subject: Is there a way to treat different heights
as one
> > layer
> > > > in
> > > > >> > config file?
> > > > >> > Owner: Nobody
> > > > >> > Requestors: binyu.wang at noaa.gov
> > > > >> > Status: new
> > > > >> > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> > > > >
> > > > >> >
> > > > >> >
> > > > >> > Hello,
> > > > >> >
> > > > >> > I am using "ensemble_stat" for volcano ash verification
and I
> got
> > a
> > > > >> > problem: my model forecast is in grib2 format with only
one
> layer.
> > > The
> > > > >> obs.
> > > > >> > data is converted from "ascii2nc", there is only lat/lon,
> height,
> > > and
> > > > >> > volcano ash inside. I want to treat all height as one
layer as
> the
> > > > >> forecast
> > > > >> > data does. How to set that in the config file? I tried :
> > > > >> > level=[ "L6000-9000" ] ;
> > > > >> > and
> > > > >> > level=[ "*,*" ] ;
> > > > >> > but none of them work. Should I have to give a fake SAME
height
> > for
> > > > all
> > > > >> > sites and regenerate the *nc files?
> > > > >> >
> > > > >> > My script is:
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> > > > >> > and the config file is:
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> > > > >> >
> > > > >> > Thank you.
> > > > >> > Binyu
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Mon Aug 17 20:04:25 2020
Beta2 was a development version used for testing on our way to the
official
Met-9.1 release... which we officially released on Aug 10th.
John
On Mon, Aug 17, 2020 at 6:10 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
>
>
> Well, the OBS_UNIT has "NA" for the whole column. See th attachment.
>
> However I recall that I have OBS_UNIT as "NA" for other runnings,
but I
>
> didn't get this warning.
>
>
>
> Thank you for the suggestion for the version switch from met-
9.1_beta2 to
>
> met-9.1. Just curious: is it because the beta2 is still under test?
>
>
>
> Binyu
>
>
>
> On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT <
> met_help at ucar.edu>
>
> wrote:
>
>
>
> > Binyu,
>
> >
>
> > You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7.
However, the
>
> > GRIB file you sent does not have output for 1 hour:
>
> >
>
> > wgrib2 T_cdump1.001.grib2
>
> >
>
> > 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour ave
fcst:
>
> >
>
> > 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3 hour
ave
> fcst:
>
> >
>
> > 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4 hour
ave
> fcst:
>
> >
>
> > 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5 hour
ave
> fcst:
>
> >
>
> > 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6 hour
ave
> fcst:
>
> >
>
> > 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7 hour
ave
> fcst:
>
> >
>
> >
>
> > Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So
that's why
>
> > you're getting no output for a forecast hour of 1.
>
> >
>
> > Next, for that warning:
>
> >
>
> > WARNING: check_hdr_str() -> null string!
>
> >
>
> >
>
> > I'm not exactly sure. Please look in one of your output files and
check
> to
>
> > see if one of the first 21 header columns contains an empty
string. I
>
> > wonder if its perhaps the "OBS_UNITS" column? And I suspect that
>
> > Ensemble-Stat is replacing the empty string with "NA".
>
> >
>
> > I see that you're using met-9.1_beta2, and I'd recommend switching
to the
>
> > released version of met-9.1 as soon as it's available on the
machine
> you're
>
> > using.
>
> >
>
> > John
>
> >
>
> >
>
> >
>
> > On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
>
> > met_help at ucar.edu> wrote:
>
> >
>
> > >
>
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
> > >
>
> > > But why I got something weird in the log file, e.g:
>
> > >
>
> > > WARNING: check_hdr_str() -> null string!
>
> > >
>
> > > WARNING:
>
> > >
>
> > > WARNING: MetGrib2DataFile::data_plane() -> No matching record
found for
>
> > > 'VAFTD/L14000-0'
>
> > >
>
> > > WARNING:
>
> > >
>
> > > WARNING:
>
> > >
>
> > > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0" not
found
> in
>
> > > file
>
> > >
>
> > >
>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
>
> > >
>
> > >
>
> > > Binyu
>
> > >
>
> > > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
>
> > > met_help at ucar.edu>
>
> > > wrote:
>
> > >
>
> > > > Binyu,
>
> > > >
>
> > > > Thanks for posting some sample data for me to take a look at.
Looking
>
> > at
>
> > > > the "log" file, I see you've run Ensemble-Stat 5 times for
hours 18,
>
> > 19,
>
> > > > 20, 21, and 22. You sent the sample files for 18 hour output
time,
> and
>
> > I
>
> > > > see that Ensemble-Stat produced 91 matched pairs for that
time:
>
> > > >
>
> > > > egrep 'matched pairs|\.stat' log
>
> > > >
>
> > > > DEBUG 2: Processing point verification VAFTD/L14000-0 versus
>
> > > ash/Z10000-0,
>
> > > > for observation type ADPUPA, over region FULL, for
interpolation
> method
>
> > > > NEAREST(1), using 91 matched pairs.
>
> > > >
>
> > > > I checked the NetCDF file that you sent me and see that it
contains
>
> > > exactly
>
> > > > 91 point observations for that time:
>
> > > >
>
> > > > ncdump -h goes16_ash_20190225-180000.nc
>
> > > >
>
> > > > netcdf goes16_ash_20190225-180000 {
>
> > > >
>
> > > > dimensions:
>
> > > >
>
> > > > ...
>
> > > >
>
> > > > nhdr = 91 ;
>
> > > >
>
> > > >
>
> > > > So Ensemble-Stat is using all available point observations you
are
>
> > > passing
>
> > > > to it.
>
> > > >
>
> > > > Looks to me like it's working as expected. Is there some
problem I'm
>
> > not
>
> > > > understanding?
>
> > > >
>
> > > > John
>
> > > >
>
> > > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
>
> > > > met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
> > > > >
>
> > > > > I know WCOSS/dev is down today, so I tried to upload the
config
> file
>
> > > and
>
> > > > > one of model (grib2 file) and obs data funder my last name
"wang"
>
> > > > following
>
> > > > > the rule:
>
> > > > >
>
> > > > >
>
> > > >
>
> > >
>
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>
> > > > > .
>
> > > > > Let me know if further information is needed.
>
> > > > >
>
> > > > > Thank you
>
> > > > > Binyu
>
> > > > >
>
> > > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA Affiliate
<
>
> > > > > binyu.wang at noaa.gov> wrote:
>
> > > > >
>
> > > > > > John,
>
> > > > > >
>
> > > > > > I did try that. Unfortunately, it doesn't work.
>
> > > > > >
>
> > > > > > Thank you.
>
> > > > > > Binyu
>
> > > > > >
>
> > > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via RT
<
>
> > > > > > met_help at ucar.edu> wrote:
>
> > > > > >
>
> > > > > >> Binyu,
>
> > > > > >>
>
> > > > > >> I see you have a question about specifying level
information in
>
> > > > > >> Ensemble-Stat. I hear that your GRIB2 model output has a
single
>
> > > record
>
> > > > > for
>
> > > > > >> volcanic ash, whereas you have point observations at
multiple
>
> > > heights.
>
> > > > > So
>
> > > > > >> you want to extract a single forecast record and match it
to
>
> > > multiple
>
> > > > > >> observation heights. Please correct me if that
description isn't
>
> > > > > accurate.
>
> > > > > >>
>
> > > > > >> The good news is that this is easily done via the
Ensemble-Stat
>
> > > > > >> configuration file. Notice that that config file contains
> separate
>
> > > > > >> dictionaries for the forecast and observation settings,
named
>
> > "fcst"
>
> > > > and
>
> > > > > >> "obs", respectively. In the default config file, we just
copy
> the
>
> > > fcst
>
> > > > > >> settings over to the obs by saying "obs = fcst". But for
this
>
> > case,
>
> > > > > we'll
>
> > > > > >> want to specify a full obs dictionary with a different
"level"
>
> > > > setting.
>
> > > > > By
>
> > > > > >> way of example, here's how you'd compare forecasts of TMP
at
> 500mb
>
> > > to
>
> > > > > TMP
>
> > > > > >> observations at heights between 0 and 10000. Note that
the 'Z'
>
> > > > > specifies a
>
> > > > > >> height level whereas 'P' specifies a pressure level:
>
> > > > > >>
>
> > > > > >> fcst = { field = [
>
> > > > > >> {
>
> > > > > >> name = "TMP";
>
> > > > > >> level = [ "P500" ];
>
> > > > > >> }
>
> > > > > >> ];
>
> > > > > >> }
>
> > > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000" ];
} ]; }
>
> > > > > >> Hopefully you can adapt this example to your case. You
just need
>
> > to
>
> > > be
>
> > > > > >> sure
>
> > > > > >> that you specify the same number of forecast and
observation
>
> > fields.
>
> > > > > >>
>
> > > > > >> Please try reconfiguring Ensemble-Stat and rerunning and
let me
>
> > know
>
> > > > how
>
> > > > > >> it
>
> > > > > >> goes.
>
> > > > > >>
>
> > > > > >> Thanks,
>
> > > > > >> John Halley Gotway
>
> > > > > >>
>
> > > > > >>
>
> > > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via
RT <
>
> > > > > >> met_help at ucar.edu> wrote:
>
> > > > > >>
>
> > > > > >> >
>
> > > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted upon.
>
> > > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
>
> > > > > >> > Queue: met_help
>
> > > > > >> > Subject: Is there a way to treat different heights
as one
>
> > > layer
>
> > > > > in
>
> > > > > >> > config file?
>
> > > > > >> > Owner: Nobody
>
> > > > > >> > Requestors: binyu.wang at noaa.gov
>
> > > > > >> > Status: new
>
> > > > > >> > Ticket <URL:
>
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
>
> > > > > >
>
> > > > > >> >
>
> > > > > >> >
>
> > > > > >> > Hello,
>
> > > > > >> >
>
> > > > > >> > I am using "ensemble_stat" for volcano ash verification
and I
>
> > got
>
> > > a
>
> > > > > >> > problem: my model forecast is in grib2 format with only
one
>
> > layer.
>
> > > > The
>
> > > > > >> obs.
>
> > > > > >> > data is converted from "ascii2nc", there is only
lat/lon,
>
> > height,
>
> > > > and
>
> > > > > >> > volcano ash inside. I want to treat all height as one
layer as
>
> > the
>
> > > > > >> forecast
>
> > > > > >> > data does. How to set that in the config file? I tried
:
>
> > > > > >> > level=[ "L6000-9000" ] ;
>
> > > > > >> > and
>
> > > > > >> > level=[ "*,*" ] ;
>
> > > > > >> > but none of them work. Should I have to give a fake
SAME
> height
>
> > > for
>
> > > > > all
>
> > > > > >> > sites and regenerate the *nc files?
>
> > > > > >> >
>
> > > > > >> > My script is:
>
> > > > > >> >
>
> > > > > >> >
>
> > > > > >>
>
> > > > >
>
> > > >
>
> > >
>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
>
> > > > > >> > and the config file is:
>
> > > > > >> >
>
> > > > > >> >
>
> > > > > >>
>
> > > > >
>
> > > >
>
> > >
>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
>
> > > > > >> >
>
> > > > > >> > Thank you.
>
> > > > > >> > Binyu
>
> > > > > >> >
>
> > > > > >> >
>
> > > > > >>
>
> > > > > >>
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> >
>
> >
>
>
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Wed Aug 19 09:57:29 2020
Binyu,
When writing output .stat files, the MET tools check to see if any of
the
header columns contain an empty string. If they do, then it prints
that
warning about an empty string, but then replaces the empty string with
"NA". Since the MET output columns are delimited by whitespace, an
empty
string would wreak havoc when trying to read them back in. So that's
why we
designed the code to check for empty strings.
So it's possible that the units are set to an empty string, cause that
warning, and then are replaced by NA.
John
On Mon, Aug 17, 2020 at 8:04 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Beta2 was a development version used for testing on our way to the
> official Met-9.1 release... which we officially released on Aug
10th.
>
> John
>
> On Mon, Aug 17, 2020 at 6:10 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
>>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>>
>>
>>
>> Well, the OBS_UNIT has "NA" for the whole column. See th
attachment.
>>
>> However I recall that I have OBS_UNIT as "NA" for other runnings,
but I
>>
>> didn't get this warning.
>>
>>
>>
>> Thank you for the suggestion for the version switch from met-
9.1_beta2 to
>>
>> met-9.1. Just curious: is it because the beta2 is still under test?
>>
>>
>>
>> Binyu
>>
>>
>>
>> On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT <
>> met_help at ucar.edu>
>>
>> wrote:
>>
>>
>>
>> > Binyu,
>>
>> >
>>
>> > You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7.
However, the
>>
>> > GRIB file you sent does not have output for 1 hour:
>>
>> >
>>
>> > wgrib2 T_cdump1.001.grib2
>>
>> >
>>
>> > 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour
ave fcst:
>>
>> >
>>
>> > 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3 hour
ave
>> fcst:
>>
>> >
>>
>> > 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4 hour
ave
>> fcst:
>>
>> >
>>
>> > 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5 hour
ave
>> fcst:
>>
>> >
>>
>> > 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6 hour
ave
>> fcst:
>>
>> >
>>
>> > 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7 hour
ave
>> fcst:
>>
>> >
>>
>> >
>>
>> > Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So
that's why
>>
>> > you're getting no output for a forecast hour of 1.
>>
>> >
>>
>> > Next, for that warning:
>>
>> >
>>
>> > WARNING: check_hdr_str() -> null string!
>>
>> >
>>
>> >
>>
>> > I'm not exactly sure. Please look in one of your output files and
check
>> to
>>
>> > see if one of the first 21 header columns contains an empty
string. I
>>
>> > wonder if its perhaps the "OBS_UNITS" column? And I suspect that
>>
>> > Ensemble-Stat is replacing the empty string with "NA".
>>
>> >
>>
>> > I see that you're using met-9.1_beta2, and I'd recommend
switching to
>> the
>>
>> > released version of met-9.1 as soon as it's available on the
machine
>> you're
>>
>> > using.
>>
>> >
>>
>> > John
>>
>> >
>>
>> >
>>
>> >
>>
>> > On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
>>
>> > met_help at ucar.edu> wrote:
>>
>> >
>>
>> > >
>>
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>>
>> > >
>>
>> > > But why I got something weird in the log file, e.g:
>>
>> > >
>>
>> > > WARNING: check_hdr_str() -> null string!
>>
>> > >
>>
>> > > WARNING:
>>
>> > >
>>
>> > > WARNING: MetGrib2DataFile::data_plane() -> No matching record
found
>> for
>>
>> > > 'VAFTD/L14000-0'
>>
>> > >
>>
>> > > WARNING:
>>
>> > >
>>
>> > > WARNING:
>>
>> > >
>>
>> > > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0" not
found
>> in
>>
>> > > file
>>
>> > >
>>
>> > >
>>
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
>>
>> > >
>>
>> > >
>>
>> > > Binyu
>>
>> > >
>>
>> > > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
>>
>> > > met_help at ucar.edu>
>>
>> > > wrote:
>>
>> > >
>>
>> > > > Binyu,
>>
>> > > >
>>
>> > > > Thanks for posting some sample data for me to take a look at.
>> Looking
>>
>> > at
>>
>> > > > the "log" file, I see you've run Ensemble-Stat 5 times for
hours 18,
>>
>> > 19,
>>
>> > > > 20, 21, and 22. You sent the sample files for 18 hour output
time,
>> and
>>
>> > I
>>
>> > > > see that Ensemble-Stat produced 91 matched pairs for that
time:
>>
>> > > >
>>
>> > > > egrep 'matched pairs|\.stat' log
>>
>> > > >
>>
>> > > > DEBUG 2: Processing point verification VAFTD/L14000-0 versus
>>
>> > > ash/Z10000-0,
>>
>> > > > for observation type ADPUPA, over region FULL, for
interpolation
>> method
>>
>> > > > NEAREST(1), using 91 matched pairs.
>>
>> > > >
>>
>> > > > I checked the NetCDF file that you sent me and see that it
contains
>>
>> > > exactly
>>
>> > > > 91 point observations for that time:
>>
>> > > >
>>
>> > > > ncdump -h goes16_ash_20190225-180000.nc
>>
>> > > >
>>
>> > > > netcdf goes16_ash_20190225-180000 {
>>
>> > > >
>>
>> > > > dimensions:
>>
>> > > >
>>
>> > > > ...
>>
>> > > >
>>
>> > > > nhdr = 91 ;
>>
>> > > >
>>
>> > > >
>>
>> > > > So Ensemble-Stat is using all available point observations
you are
>>
>> > > passing
>>
>> > > > to it.
>>
>> > > >
>>
>> > > > Looks to me like it's working as expected. Is there some
problem I'm
>>
>> > not
>>
>> > > > understanding?
>>
>> > > >
>>
>> > > > John
>>
>> > > >
>>
>> > > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT <
>>
>> > > > met_help at ucar.edu> wrote:
>>
>> > > >
>>
>> > > > >
>>
>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>>
>> > > > >
>>
>> > > > > I know WCOSS/dev is down today, so I tried to upload the
config
>> file
>>
>> > > and
>>
>> > > > > one of model (grib2 file) and obs data funder my last name
"wang"
>>
>> > > > following
>>
>> > > > > the rule:
>>
>> > > > >
>>
>> > > > >
>>
>> > > >
>>
>> > >
>>
>> >
>> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>>
>> > > > > .
>>
>> > > > > Let me know if further information is needed.
>>
>> > > > >
>>
>> > > > > Thank you
>>
>> > > > > Binyu
>>
>> > > > >
>>
>> > > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA
Affiliate <
>>
>> > > > > binyu.wang at noaa.gov> wrote:
>>
>> > > > >
>>
>> > > > > > John,
>>
>> > > > > >
>>
>> > > > > > I did try that. Unfortunately, it doesn't work.
>>
>> > > > > >
>>
>> > > > > > Thank you.
>>
>> > > > > > Binyu
>>
>> > > > > >
>>
>> > > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via
RT <
>>
>> > > > > > met_help at ucar.edu> wrote:
>>
>> > > > > >
>>
>> > > > > >> Binyu,
>>
>> > > > > >>
>>
>> > > > > >> I see you have a question about specifying level
information in
>>
>> > > > > >> Ensemble-Stat. I hear that your GRIB2 model output has a
single
>>
>> > > record
>>
>> > > > > for
>>
>> > > > > >> volcanic ash, whereas you have point observations at
multiple
>>
>> > > heights.
>>
>> > > > > So
>>
>> > > > > >> you want to extract a single forecast record and match
it to
>>
>> > > multiple
>>
>> > > > > >> observation heights. Please correct me if that
description
>> isn't
>>
>> > > > > accurate.
>>
>> > > > > >>
>>
>> > > > > >> The good news is that this is easily done via the
Ensemble-Stat
>>
>> > > > > >> configuration file. Notice that that config file
contains
>> separate
>>
>> > > > > >> dictionaries for the forecast and observation settings,
named
>>
>> > "fcst"
>>
>> > > > and
>>
>> > > > > >> "obs", respectively. In the default config file, we just
copy
>> the
>>
>> > > fcst
>>
>> > > > > >> settings over to the obs by saying "obs = fcst". But for
this
>>
>> > case,
>>
>> > > > > we'll
>>
>> > > > > >> want to specify a full obs dictionary with a different
"level"
>>
>> > > > setting.
>>
>> > > > > By
>>
>> > > > > >> way of example, here's how you'd compare forecasts of
TMP at
>> 500mb
>>
>> > > to
>>
>> > > > > TMP
>>
>> > > > > >> observations at heights between 0 and 10000. Note that
the 'Z'
>>
>> > > > > specifies a
>>
>> > > > > >> height level whereas 'P' specifies a pressure level:
>>
>> > > > > >>
>>
>> > > > > >> fcst = { field = [
>>
>> > > > > >> {
>>
>> > > > > >> name = "TMP";
>>
>> > > > > >> level = [ "P500" ];
>>
>> > > > > >> }
>>
>> > > > > >> ];
>>
>> > > > > >> }
>>
>> > > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000"
]; } ];
>> }
>>
>> > > > > >> Hopefully you can adapt this example to your case. You
just
>> need
>>
>> > to
>>
>> > > be
>>
>> > > > > >> sure
>>
>> > > > > >> that you specify the same number of forecast and
observation
>>
>> > fields.
>>
>> > > > > >>
>>
>> > > > > >> Please try reconfiguring Ensemble-Stat and rerunning and
let me
>>
>> > know
>>
>> > > > how
>>
>> > > > > >> it
>>
>> > > > > >> goes.
>>
>> > > > > >>
>>
>> > > > > >> Thanks,
>>
>> > > > > >> John Halley Gotway
>>
>> > > > > >>
>>
>> > > > > >>
>>
>> > > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via
RT <
>>
>> > > > > >> met_help at ucar.edu> wrote:
>>
>> > > > > >>
>>
>> > > > > >> >
>>
>> > > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted
upon.
>>
>> > > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
>>
>> > > > > >> > Queue: met_help
>>
>> > > > > >> > Subject: Is there a way to treat different
heights as
>> one
>>
>> > > layer
>>
>> > > > > in
>>
>> > > > > >> > config file?
>>
>> > > > > >> > Owner: Nobody
>>
>> > > > > >> > Requestors: binyu.wang at noaa.gov
>>
>> > > > > >> > Status: new
>>
>> > > > > >> > Ticket <URL:
>>
>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
>>
>> > > > > >
>>
>> > > > > >> >
>>
>> > > > > >> >
>>
>> > > > > >> > Hello,
>>
>> > > > > >> >
>>
>> > > > > >> > I am using "ensemble_stat" for volcano ash
verification and I
>>
>> > got
>>
>> > > a
>>
>> > > > > >> > problem: my model forecast is in grib2 format with
only one
>>
>> > layer.
>>
>> > > > The
>>
>> > > > > >> obs.
>>
>> > > > > >> > data is converted from "ascii2nc", there is only
lat/lon,
>>
>> > height,
>>
>> > > > and
>>
>> > > > > >> > volcano ash inside. I want to treat all height as one
layer
>> as
>>
>> > the
>>
>> > > > > >> forecast
>>
>> > > > > >> > data does. How to set that in the config file? I tried
:
>>
>> > > > > >> > level=[ "L6000-9000" ] ;
>>
>> > > > > >> > and
>>
>> > > > > >> > level=[ "*,*" ] ;
>>
>> > > > > >> > but none of them work. Should I have to give a fake
SAME
>> height
>>
>> > > for
>>
>> > > > > all
>>
>> > > > > >> > sites and regenerate the *nc files?
>>
>> > > > > >> >
>>
>> > > > > >> > My script is:
>>
>> > > > > >> >
>>
>> > > > > >> >
>>
>> > > > > >>
>>
>> > > > >
>>
>> > > >
>>
>> > >
>>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
>>
>> > > > > >> > and the config file is:
>>
>> > > > > >> >
>>
>> > > > > >> >
>>
>> > > > > >>
>>
>> > > > >
>>
>> > > >
>>
>> > >
>>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
>>
>> > > > > >> >
>>
>> > > > > >> > Thank you.
>>
>> > > > > >> > Binyu
>>
>> > > > > >> >
>>
>> > > > > >> >
>>
>> > > > > >>
>>
>> > > > > >>
>>
>> > > > >
>>
>> > > > >
>>
>> > > >
>>
>> > > >
>>
>> > >
>>
>> > >
>>
>> >
>>
>> >
>>
>>
>>
>>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Wed Aug 19 19:51:30 2020
Got it. Thank you very much!
Binyu
On Wed, Aug 19, 2020 at 11:57 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Binyu,
>
> When writing output .stat files, the MET tools check to see if any
of the
> header columns contain an empty string. If they do, then it prints
that
> warning about an empty string, but then replaces the empty string
with
> "NA". Since the MET output columns are delimited by whitespace, an
empty
> string would wreak havoc when trying to read them back in. So that's
why we
> designed the code to check for empty strings.
>
> So it's possible that the units are set to an empty string, cause
that
> warning, and then are replaced by NA.
>
> John
>
> On Mon, Aug 17, 2020 at 8:04 PM John Halley Gotway <johnhg at ucar.edu>
> wrote:
>
> > Beta2 was a development version used for testing on our way to the
> > official Met-9.1 release... which we officially released on Aug
10th.
> >
> > John
> >
> > On Mon, Aug 17, 2020 at 6:10 PM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >>
> >>
> >>
> >> Well, the OBS_UNIT has "NA" for the whole column. See th
attachment.
> >>
> >> However I recall that I have OBS_UNIT as "NA" for other runnings,
but I
> >>
> >> didn't get this warning.
> >>
> >>
> >>
> >> Thank you for the suggestion for the version switch from met-
9.1_beta2
> to
> >>
> >> met-9.1. Just curious: is it because the beta2 is still under
test?
> >>
> >>
> >>
> >> Binyu
> >>
> >>
> >>
> >> On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT <
> >> met_help at ucar.edu>
> >>
> >> wrote:
> >>
> >>
> >>
> >> > Binyu,
> >>
> >> >
> >>
> >> > You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7.
However,
> the
> >>
> >> > GRIB file you sent does not have output for 1 hour:
> >>
> >> >
> >>
> >> > wgrib2 T_cdump1.001.grib2
> >>
> >> >
> >>
> >> > 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour
ave
> fcst:
> >>
> >> >
> >>
> >> > 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3
hour ave
> >> fcst:
> >>
> >> >
> >>
> >> > 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4
hour ave
> >> fcst:
> >>
> >> >
> >>
> >> > 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5
hour ave
> >> fcst:
> >>
> >> >
> >>
> >> > 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6
hour ave
> >> fcst:
> >>
> >> >
> >>
> >> > 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7
hour ave
> >> fcst:
> >>
> >> >
> >>
> >> >
> >>
> >> > Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So
that's
> why
> >>
> >> > you're getting no output for a forecast hour of 1.
> >>
> >> >
> >>
> >> > Next, for that warning:
> >>
> >> >
> >>
> >> > WARNING: check_hdr_str() -> null string!
> >>
> >> >
> >>
> >> >
> >>
> >> > I'm not exactly sure. Please look in one of your output files
and
> check
> >> to
> >>
> >> > see if one of the first 21 header columns contains an empty
string. I
> >>
> >> > wonder if its perhaps the "OBS_UNITS" column? And I suspect
that
> >>
> >> > Ensemble-Stat is replacing the empty string with "NA".
> >>
> >> >
> >>
> >> > I see that you're using met-9.1_beta2, and I'd recommend
switching to
> >> the
> >>
> >> > released version of met-9.1 as soon as it's available on the
machine
> >> you're
> >>
> >> > using.
> >>
> >> >
> >>
> >> > John
> >>
> >> >
> >>
> >> >
> >>
> >> >
> >>
> >> > On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
> >>
> >> > met_help at ucar.edu> wrote:
> >>
> >> >
> >>
> >> > >
> >>
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
>
> >>
> >> > >
> >>
> >> > > But why I got something weird in the log file, e.g:
> >>
> >> > >
> >>
> >> > > WARNING: check_hdr_str() -> null string!
> >>
> >> > >
> >>
> >> > > WARNING:
> >>
> >> > >
> >>
> >> > > WARNING: MetGrib2DataFile::data_plane() -> No matching record
found
> >> for
> >>
> >> > > 'VAFTD/L14000-0'
> >>
> >> > >
> >>
> >> > > WARNING:
> >>
> >> > >
> >>
> >> > > WARNING:
> >>
> >> > >
> >>
> >> > > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0"
not
> found
> >> in
> >>
> >> > > file
> >>
> >> > >
> >>
> >> > >
> >>
> >> >
> >>
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
> >>
> >> > >
> >>
> >> > >
> >>
> >> > > Binyu
> >>
> >> > >
> >>
> >> > > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
> >>
> >> > > met_help at ucar.edu>
> >>
> >> > > wrote:
> >>
> >> > >
> >>
> >> > > > Binyu,
> >>
> >> > > >
> >>
> >> > > > Thanks for posting some sample data for me to take a look
at.
> >> Looking
> >>
> >> > at
> >>
> >> > > > the "log" file, I see you've run Ensemble-Stat 5 times for
hours
> 18,
> >>
> >> > 19,
> >>
> >> > > > 20, 21, and 22. You sent the sample files for 18 hour
output time,
> >> and
> >>
> >> > I
> >>
> >> > > > see that Ensemble-Stat produced 91 matched pairs for that
time:
> >>
> >> > > >
> >>
> >> > > > egrep 'matched pairs|\.stat' log
> >>
> >> > > >
> >>
> >> > > > DEBUG 2: Processing point verification VAFTD/L14000-0
versus
> >>
> >> > > ash/Z10000-0,
> >>
> >> > > > for observation type ADPUPA, over region FULL, for
interpolation
> >> method
> >>
> >> > > > NEAREST(1), using 91 matched pairs.
> >>
> >> > > >
> >>
> >> > > > I checked the NetCDF file that you sent me and see that it
> contains
> >>
> >> > > exactly
> >>
> >> > > > 91 point observations for that time:
> >>
> >> > > >
> >>
> >> > > > ncdump -h goes16_ash_20190225-180000.nc
> >>
> >> > > >
> >>
> >> > > > netcdf goes16_ash_20190225-180000 {
> >>
> >> > > >
> >>
> >> > > > dimensions:
> >>
> >> > > >
> >>
> >> > > > ...
> >>
> >> > > >
> >>
> >> > > > nhdr = 91 ;
> >>
> >> > > >
> >>
> >> > > >
> >>
> >> > > > So Ensemble-Stat is using all available point observations
you are
> >>
> >> > > passing
> >>
> >> > > > to it.
> >>
> >> > > >
> >>
> >> > > > Looks to me like it's working as expected. Is there some
problem
> I'm
> >>
> >> > not
> >>
> >> > > > understanding?
> >>
> >> > > >
> >>
> >> > > > John
> >>
> >> > > >
> >>
> >> > > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT
<
> >>
> >> > > > met_help at ucar.edu> wrote:
> >>
> >> > > >
> >>
> >> > > > >
> >>
> >> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >>
> >> > > > >
> >>
> >> > > > > I know WCOSS/dev is down today, so I tried to upload the
config
> >> file
> >>
> >> > > and
> >>
> >> > > > > one of model (grib2 file) and obs data funder my last
name
> "wang"
> >>
> >> > > > following
> >>
> >> > > > > the rule:
> >>
> >> > > > >
> >>
> >> > > > >
> >>
> >> > > >
> >>
> >> > >
> >>
> >> >
> >>
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> >>
> >> > > > > .
> >>
> >> > > > > Let me know if further information is needed.
> >>
> >> > > > >
> >>
> >> > > > > Thank you
> >>
> >> > > > > Binyu
> >>
> >> > > > >
> >>
> >> > > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA
Affiliate <
> >>
> >> > > > > binyu.wang at noaa.gov> wrote:
> >>
> >> > > > >
> >>
> >> > > > > > John,
> >>
> >> > > > > >
> >>
> >> > > > > > I did try that. Unfortunately, it doesn't work.
> >>
> >> > > > > >
> >>
> >> > > > > > Thank you.
> >>
> >> > > > > > Binyu
> >>
> >> > > > > >
> >>
> >> > > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via
RT <
> >>
> >> > > > > > met_help at ucar.edu> wrote:
> >>
> >> > > > > >
> >>
> >> > > > > >> Binyu,
> >>
> >> > > > > >>
> >>
> >> > > > > >> I see you have a question about specifying level
information
> in
> >>
> >> > > > > >> Ensemble-Stat. I hear that your GRIB2 model output has
a
> single
> >>
> >> > > record
> >>
> >> > > > > for
> >>
> >> > > > > >> volcanic ash, whereas you have point observations at
multiple
> >>
> >> > > heights.
> >>
> >> > > > > So
> >>
> >> > > > > >> you want to extract a single forecast record and match
it to
> >>
> >> > > multiple
> >>
> >> > > > > >> observation heights. Please correct me if that
description
> >> isn't
> >>
> >> > > > > accurate.
> >>
> >> > > > > >>
> >>
> >> > > > > >> The good news is that this is easily done via the
> Ensemble-Stat
> >>
> >> > > > > >> configuration file. Notice that that config file
contains
> >> separate
> >>
> >> > > > > >> dictionaries for the forecast and observation
settings, named
> >>
> >> > "fcst"
> >>
> >> > > > and
> >>
> >> > > > > >> "obs", respectively. In the default config file, we
just copy
> >> the
> >>
> >> > > fcst
> >>
> >> > > > > >> settings over to the obs by saying "obs = fcst". But
for this
> >>
> >> > case,
> >>
> >> > > > > we'll
> >>
> >> > > > > >> want to specify a full obs dictionary with a different
> "level"
> >>
> >> > > > setting.
> >>
> >> > > > > By
> >>
> >> > > > > >> way of example, here's how you'd compare forecasts of
TMP at
> >> 500mb
> >>
> >> > > to
> >>
> >> > > > > TMP
> >>
> >> > > > > >> observations at heights between 0 and 10000. Note that
the
> 'Z'
> >>
> >> > > > > specifies a
> >>
> >> > > > > >> height level whereas 'P' specifies a pressure level:
> >>
> >> > > > > >>
> >>
> >> > > > > >> fcst = { field = [
> >>
> >> > > > > >> {
> >>
> >> > > > > >> name = "TMP";
> >>
> >> > > > > >> level = [ "P500" ];
> >>
> >> > > > > >> }
> >>
> >> > > > > >> ];
> >>
> >> > > > > >> }
> >>
> >> > > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000"
]; }
> ];
> >> }
> >>
> >> > > > > >> Hopefully you can adapt this example to your case. You
just
> >> need
> >>
> >> > to
> >>
> >> > > be
> >>
> >> > > > > >> sure
> >>
> >> > > > > >> that you specify the same number of forecast and
observation
> >>
> >> > fields.
> >>
> >> > > > > >>
> >>
> >> > > > > >> Please try reconfiguring Ensemble-Stat and rerunning
and let
> me
> >>
> >> > know
> >>
> >> > > > how
> >>
> >> > > > > >> it
> >>
> >> > > > > >> goes.
> >>
> >> > > > > >>
> >>
> >> > > > > >> Thanks,
> >>
> >> > > > > >> John Halley Gotway
> >>
> >> > > > > >>
> >>
> >> > > > > >>
> >>
> >> > > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov
via RT <
> >>
> >> > > > > >> met_help at ucar.edu> wrote:
> >>
> >> > > > > >>
> >>
> >> > > > > >> >
> >>
> >> > > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted
upon.
> >>
> >> > > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> >>
> >> > > > > >> > Queue: met_help
> >>
> >> > > > > >> > Subject: Is there a way to treat different
heights as
> >> one
> >>
> >> > > layer
> >>
> >> > > > > in
> >>
> >> > > > > >> > config file?
> >>
> >> > > > > >> > Owner: Nobody
> >>
> >> > > > > >> > Requestors: binyu.wang at noaa.gov
> >>
> >> > > > > >> > Status: new
> >>
> >> > > > > >> > Ticket <URL:
> >>
> >> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> >>
> >> > > > > >
> >>
> >> > > > > >> >
> >>
> >> > > > > >> >
> >>
> >> > > > > >> > Hello,
> >>
> >> > > > > >> >
> >>
> >> > > > > >> > I am using "ensemble_stat" for volcano ash
verification
> and I
> >>
> >> > got
> >>
> >> > > a
> >>
> >> > > > > >> > problem: my model forecast is in grib2 format with
only one
> >>
> >> > layer.
> >>
> >> > > > The
> >>
> >> > > > > >> obs.
> >>
> >> > > > > >> > data is converted from "ascii2nc", there is only
lat/lon,
> >>
> >> > height,
> >>
> >> > > > and
> >>
> >> > > > > >> > volcano ash inside. I want to treat all height as
one layer
> >> as
> >>
> >> > the
> >>
> >> > > > > >> forecast
> >>
> >> > > > > >> > data does. How to set that in the config file? I
tried :
> >>
> >> > > > > >> > level=[ "L6000-9000" ] ;
> >>
> >> > > > > >> > and
> >>
> >> > > > > >> > level=[ "*,*" ] ;
> >>
> >> > > > > >> > but none of them work. Should I have to give a fake
SAME
> >> height
> >>
> >> > > for
> >>
> >> > > > > all
> >>
> >> > > > > >> > sites and regenerate the *nc files?
> >>
> >> > > > > >> >
> >>
> >> > > > > >> > My script is:
> >>
> >> > > > > >> >
> >>
> >> > > > > >> >
> >>
> >> > > > > >>
> >>
> >> > > > >
> >>
> >> > > >
> >>
> >> > >
> >>
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> >>
> >> > > > > >> > and the config file is:
> >>
> >> > > > > >> >
> >>
> >> > > > > >> >
> >>
> >> > > > > >>
> >>
> >> > > > >
> >>
> >> > > >
> >>
> >> > >
> >>
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> >>
> >> > > > > >> >
> >>
> >> > > > > >> > Thank you.
> >>
> >> > > > > >> > Binyu
> >>
> >> > > > > >> >
> >>
> >> > > > > >> >
> >>
> >> > > > > >>
> >>
> >> > > > > >>
> >>
> >> > > > >
> >>
> >> > > > >
> >>
> >> > > >
> >>
> >> > > >
> >>
> >> > >
> >>
> >> > >
> >>
> >> >
> >>
> >> >
> >>
> >>
> >>
> >>
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: binyu.wang at noaa.gov
Time: Tue Aug 25 16:04:40 2020
Hello John,
Back to this question again, I found there is a problem with using
v9.1:
If I used v9.1, I got error when I use "point_stat":
ERROR :
ERROR : process_obs_file() -> for observation index 175, when
computing
VL1L2 and/or VAL1L2 vector winds each UGRD observation must be
followed by
a VGRD observation with the same header and at the same level.
ERROR :
This ERROR will be gone if I switch to 9.1_beta2.
I am not exactly sure what happened. But I found somehow this is
because
the input "nc" files was generated using "ascii2nc" and I have
multiple
input/variables, like ascii2nc indir/UGRD indir/VGRD indir/WDIR out.
The script to generate the nc files is:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/Scripts/MET/asc2grid_uwyo_SKTQ.sh
The script to get the
error
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/test_12z_02.ecf
and the log file:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.log.err
Thank you.
Binyu
On Mon, Aug 17, 2020 at 10:04 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Beta2 was a development version used for testing on our way to the
official
> Met-9.1 release... which we officially released on Aug 10th.
>
> John
>
> On Mon, Aug 17, 2020 at 6:10 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >
> >
> >
> > Well, the OBS_UNIT has "NA" for the whole column. See th
attachment.
> >
> > However I recall that I have OBS_UNIT as "NA" for other runnings,
but I
> >
> > didn't get this warning.
> >
> >
> >
> > Thank you for the suggestion for the version switch from met-
9.1_beta2 to
> >
> > met-9.1. Just curious: is it because the beta2 is still under
test?
> >
> >
> >
> > Binyu
> >
> >
> >
> > On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> >
> > wrote:
> >
> >
> >
> > > Binyu,
> >
> > >
> >
> > > You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7.
However, the
> >
> > > GRIB file you sent does not have output for 1 hour:
> >
> > >
> >
> > > wgrib2 T_cdump1.001.grib2
> >
> > >
> >
> > > 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour
ave
> fcst:
> >
> > >
> >
> > > 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3 hour
ave
> > fcst:
> >
> > >
> >
> > > 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4
hour ave
> > fcst:
> >
> > >
> >
> > > 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5
hour ave
> > fcst:
> >
> > >
> >
> > > 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6
hour ave
> > fcst:
> >
> > >
> >
> > > 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7
hour ave
> > fcst:
> >
> > >
> >
> > >
> >
> > > Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So
that's
> why
> >
> > > you're getting no output for a forecast hour of 1.
> >
> > >
> >
> > > Next, for that warning:
> >
> > >
> >
> > > WARNING: check_hdr_str() -> null string!
> >
> > >
> >
> > >
> >
> > > I'm not exactly sure. Please look in one of your output files
and check
> > to
> >
> > > see if one of the first 21 header columns contains an empty
string. I
> >
> > > wonder if its perhaps the "OBS_UNITS" column? And I suspect that
> >
> > > Ensemble-Stat is replacing the empty string with "NA".
> >
> > >
> >
> > > I see that you're using met-9.1_beta2, and I'd recommend
switching to
> the
> >
> > > released version of met-9.1 as soon as it's available on the
machine
> > you're
> >
> > > using.
> >
> > >
> >
> > > John
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
> >
> > > met_help at ucar.edu> wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
>
> >
> > > >
> >
> > > > But why I got something weird in the log file, e.g:
> >
> > > >
> >
> > > > WARNING: check_hdr_str() -> null string!
> >
> > > >
> >
> > > > WARNING:
> >
> > > >
> >
> > > > WARNING: MetGrib2DataFile::data_plane() -> No matching record
found
> for
> >
> > > > 'VAFTD/L14000-0'
> >
> > > >
> >
> > > > WARNING:
> >
> > > >
> >
> > > > WARNING:
> >
> > > >
> >
> > > > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0"
not found
> > in
> >
> > > > file
> >
> > > >
> >
> > > >
> >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
> >
> > > >
> >
> > > >
> >
> > > > Binyu
> >
> > > >
> >
> > > > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
> >
> > > > met_help at ucar.edu>
> >
> > > > wrote:
> >
> > > >
> >
> > > > > Binyu,
> >
> > > > >
> >
> > > > > Thanks for posting some sample data for me to take a look
at.
> Looking
> >
> > > at
> >
> > > > > the "log" file, I see you've run Ensemble-Stat 5 times for
hours
> 18,
> >
> > > 19,
> >
> > > > > 20, 21, and 22. You sent the sample files for 18 hour output
time,
> > and
> >
> > > I
> >
> > > > > see that Ensemble-Stat produced 91 matched pairs for that
time:
> >
> > > > >
> >
> > > > > egrep 'matched pairs|\.stat' log
> >
> > > > >
> >
> > > > > DEBUG 2: Processing point verification VAFTD/L14000-0 versus
> >
> > > > ash/Z10000-0,
> >
> > > > > for observation type ADPUPA, over region FULL, for
interpolation
> > method
> >
> > > > > NEAREST(1), using 91 matched pairs.
> >
> > > > >
> >
> > > > > I checked the NetCDF file that you sent me and see that it
contains
> >
> > > > exactly
> >
> > > > > 91 point observations for that time:
> >
> > > > >
> >
> > > > > ncdump -h goes16_ash_20190225-180000.nc
> >
> > > > >
> >
> > > > > netcdf goes16_ash_20190225-180000 {
> >
> > > > >
> >
> > > > > dimensions:
> >
> > > > >
> >
> > > > > ...
> >
> > > > >
> >
> > > > > nhdr = 91 ;
> >
> > > > >
> >
> > > > >
> >
> > > > > So Ensemble-Stat is using all available point observations
you are
> >
> > > > passing
> >
> > > > > to it.
> >
> > > > >
> >
> > > > > Looks to me like it's working as expected. Is there some
problem
> I'm
> >
> > > not
> >
> > > > > understanding?
> >
> > > > >
> >
> > > > > John
> >
> > > > >
> >
> > > > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via RT
<
> >
> > > > > met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> >
> > > > > >
> >
> > > > > > I know WCOSS/dev is down today, so I tried to upload the
config
> > file
> >
> > > > and
> >
> > > > > > one of model (grib2 file) and obs data funder my last name
"wang"
> >
> > > > > following
> >
> > > > > > the rule:
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > >
> >
> > >
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> >
> > > > > > .
> >
> > > > > > Let me know if further information is needed.
> >
> > > > > >
> >
> > > > > > Thank you
> >
> > > > > > Binyu
> >
> > > > > >
> >
> > > > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA
Affiliate <
> >
> > > > > > binyu.wang at noaa.gov> wrote:
> >
> > > > > >
> >
> > > > > > > John,
> >
> > > > > > >
> >
> > > > > > > I did try that. Unfortunately, it doesn't work.
> >
> > > > > > >
> >
> > > > > > > Thank you.
> >
> > > > > > > Binyu
> >
> > > > > > >
> >
> > > > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway via
RT <
> >
> > > > > > > met_help at ucar.edu> wrote:
> >
> > > > > > >
> >
> > > > > > >> Binyu,
> >
> > > > > > >>
> >
> > > > > > >> I see you have a question about specifying level
information
> in
> >
> > > > > > >> Ensemble-Stat. I hear that your GRIB2 model output has
a
> single
> >
> > > > record
> >
> > > > > > for
> >
> > > > > > >> volcanic ash, whereas you have point observations at
multiple
> >
> > > > heights.
> >
> > > > > > So
> >
> > > > > > >> you want to extract a single forecast record and match
it to
> >
> > > > multiple
> >
> > > > > > >> observation heights. Please correct me if that
description
> isn't
> >
> > > > > > accurate.
> >
> > > > > > >>
> >
> > > > > > >> The good news is that this is easily done via the
> Ensemble-Stat
> >
> > > > > > >> configuration file. Notice that that config file
contains
> > separate
> >
> > > > > > >> dictionaries for the forecast and observation settings,
named
> >
> > > "fcst"
> >
> > > > > and
> >
> > > > > > >> "obs", respectively. In the default config file, we
just copy
> > the
> >
> > > > fcst
> >
> > > > > > >> settings over to the obs by saying "obs = fcst". But
for this
> >
> > > case,
> >
> > > > > > we'll
> >
> > > > > > >> want to specify a full obs dictionary with a different
"level"
> >
> > > > > setting.
> >
> > > > > > By
> >
> > > > > > >> way of example, here's how you'd compare forecasts of
TMP at
> > 500mb
> >
> > > > to
> >
> > > > > > TMP
> >
> > > > > > >> observations at heights between 0 and 10000. Note that
the 'Z'
> >
> > > > > > specifies a
> >
> > > > > > >> height level whereas 'P' specifies a pressure level:
> >
> > > > > > >>
> >
> > > > > > >> fcst = { field = [
> >
> > > > > > >> {
> >
> > > > > > >> name = "TMP";
> >
> > > > > > >> level = [ "P500" ];
> >
> > > > > > >> }
> >
> > > > > > >> ];
> >
> > > > > > >> }
> >
> > > > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-10000"
]; }
> ]; }
> >
> > > > > > >> Hopefully you can adapt this example to your case. You
just
> need
> >
> > > to
> >
> > > > be
> >
> > > > > > >> sure
> >
> > > > > > >> that you specify the same number of forecast and
observation
> >
> > > fields.
> >
> > > > > > >>
> >
> > > > > > >> Please try reconfiguring Ensemble-Stat and rerunning
and let
> me
> >
> > > know
> >
> > > > > how
> >
> > > > > > >> it
> >
> > > > > > >> goes.
> >
> > > > > > >>
> >
> > > > > > >> Thanks,
> >
> > > > > > >> John Halley Gotway
> >
> > > > > > >>
> >
> > > > > > >>
> >
> > > > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov via
RT <
> >
> > > > > > >> met_help at ucar.edu> wrote:
> >
> > > > > > >>
> >
> > > > > > >> >
> >
> > > > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted
upon.
> >
> > > > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> >
> > > > > > >> > Queue: met_help
> >
> > > > > > >> > Subject: Is there a way to treat different
heights as
> one
> >
> > > > layer
> >
> > > > > > in
> >
> > > > > > >> > config file?
> >
> > > > > > >> > Owner: Nobody
> >
> > > > > > >> > Requestors: binyu.wang at noaa.gov
> >
> > > > > > >> > Status: new
> >
> > > > > > >> > Ticket <URL:
> >
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> >
> > > > > > >
> >
> > > > > > >> >
> >
> > > > > > >> >
> >
> > > > > > >> > Hello,
> >
> > > > > > >> >
> >
> > > > > > >> > I am using "ensemble_stat" for volcano ash
verification and
> I
> >
> > > got
> >
> > > > a
> >
> > > > > > >> > problem: my model forecast is in grib2 format with
only one
> >
> > > layer.
> >
> > > > > The
> >
> > > > > > >> obs.
> >
> > > > > > >> > data is converted from "ascii2nc", there is only
lat/lon,
> >
> > > height,
> >
> > > > > and
> >
> > > > > > >> > volcano ash inside. I want to treat all height as one
layer
> as
> >
> > > the
> >
> > > > > > >> forecast
> >
> > > > > > >> > data does. How to set that in the config file? I
tried :
> >
> > > > > > >> > level=[ "L6000-9000" ] ;
> >
> > > > > > >> > and
> >
> > > > > > >> > level=[ "*,*" ] ;
> >
> > > > > > >> > but none of them work. Should I have to give a fake
SAME
> > height
> >
> > > > for
> >
> > > > > > all
> >
> > > > > > >> > sites and regenerate the *nc files?
> >
> > > > > > >> >
> >
> > > > > > >> > My script is:
> >
> > > > > > >> >
> >
> > > > > > >> >
> >
> > > > > > >>
> >
> > > > > >
> >
> > > > >
> >
> > > >
> >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> >
> > > > > > >> > and the config file is:
> >
> > > > > > >> >
> >
> > > > > > >> >
> >
> > > > > > >>
> >
> > > > > >
> >
> > > > >
> >
> > > >
> >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> >
> > > > > > >> >
> >
> > > > > > >> > Thank you.
> >
> > > > > > >> > Binyu
> >
> > > > > > >> >
> >
> > > > > > >> >
> >
> > > > > > >>
> >
> > > > > > >>
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> >
> >
> >
>
>
------------------------------------------------
Subject: Is there a way to treat different heights as one layer in config file?
From: John Halley Gotway
Time: Tue Aug 25 16:51:04 2020
Binyu,
To follow up on the earlier issue, I wrote up this feature request
here:
https://github.com/dtcenter/MET/issues/999
So it sounds like with MET version 9.1, you're getting a runtime error
but
with MET verison 9.1 beta2, you are not.
I'll need to log on to WCOSS to debug further. I'll let you know what
I
find.
John
On Tue, Aug 25, 2020 at 4:05 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
>
> Hello John,
> Back to this question again, I found there is a problem with using
v9.1:
> If I used v9.1, I got error when I use "point_stat":
>
> ERROR :
> ERROR : process_obs_file() -> for observation index 175, when
computing
> VL1L2 and/or VAL1L2 vector winds each UGRD observation must be
followed by
> a VGRD observation with the same header and at the same level.
> ERROR :
>
> This ERROR will be gone if I switch to 9.1_beta2.
>
> I am not exactly sure what happened. But I found somehow this is
because
> the input "nc" files was generated using "ascii2nc" and I have
multiple
> input/variables, like ascii2nc indir/UGRD indir/VGRD indir/WDIR out.
> The script to generate the nc files is:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/Scripts/MET/asc2grid_uwyo_SKTQ.sh
>
> The script to get the
> error
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/test_12z_02.ecf
> and the log file:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.log.err
>
> Thank you.
> Binyu
>
> On Mon, Aug 17, 2020 at 10:04 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Beta2 was a development version used for testing on our way to the
> official
> > Met-9.1 release... which we officially released on Aug 10th.
> >
> > John
> >
> > On Mon, Aug 17, 2020 at 6:10 PM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> > >
> > >
> > >
> > > Well, the OBS_UNIT has "NA" for the whole column. See th
attachment.
> > >
> > > However I recall that I have OBS_UNIT as "NA" for other
runnings, but I
> > >
> > > didn't get this warning.
> > >
> > >
> > >
> > > Thank you for the suggestion for the version switch from met-
9.1_beta2
> to
> > >
> > > met-9.1. Just curious: is it because the beta2 is still under
test?
> > >
> > >
> > >
> > > Binyu
> > >
> > >
> > >
> > > On Mon, Aug 17, 2020 at 7:14 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > >
> > > wrote:
> > >
> > >
> > >
> > > > Binyu,
> > >
> > > >
> > >
> > > > You're looping over LEAD_HR hours 1, 2, 3, 4, 5, 6, and 7.
However,
> the
> > >
> > > > GRIB file you sent does not have output for 1 hour:
> > >
> > > >
> > >
> > > > wgrib2 T_cdump1.001.grib2
> > >
> > > >
> > >
> > > > 1:0:d=2019022516:VAFTD:14000-0 m above mean sea level:1-2 hour
ave
> > fcst:
> > >
> > > >
> > >
> > > > 2:856:d=2019022516:VAFTD:14000-0 m above mean sea level:2-3
hour ave
> > > fcst:
> > >
> > > >
> > >
> > > > 3:1818:d=2019022516:VAFTD:14000-0 m above mean sea level:3-4
hour ave
> > > fcst:
> > >
> > > >
> > >
> > > > 4:2782:d=2019022516:VAFTD:14000-0 m above mean sea level:4-5
hour ave
> > > fcst:
> > >
> > > >
> > >
> > > > 5:3800:d=2019022516:VAFTD:14000-0 m above mean sea level:5-6
hour ave
> > > fcst:
> > >
> > > >
> > >
> > > > 6:4902:d=2019022516:VAFTD:14000-0 m above mean sea level:6-7
hour ave
> > > fcst:
> > >
> > > >
> > >
> > > >
> > >
> > > > Those records are for forecast hours 2, 3, 4, 5, 6, and 7. So
that's
> > why
> > >
> > > > you're getting no output for a forecast hour of 1.
> > >
> > > >
> > >
> > > > Next, for that warning:
> > >
> > > >
> > >
> > > > WARNING: check_hdr_str() -> null string!
> > >
> > > >
> > >
> > > >
> > >
> > > > I'm not exactly sure. Please look in one of your output files
and
> check
> > > to
> > >
> > > > see if one of the first 21 header columns contains an empty
string. I
> > >
> > > > wonder if its perhaps the "OBS_UNITS" column? And I suspect
that
> > >
> > > > Ensemble-Stat is replacing the empty string with "NA".
> > >
> > > >
> > >
> > > > I see that you're using met-9.1_beta2, and I'd recommend
switching to
> > the
> > >
> > > > released version of met-9.1 as soon as it's available on the
machine
> > > you're
> > >
> > > > using.
> > >
> > > >
> > >
> > > > John
> > >
> > > >
> > >
> > > >
> > >
> > > >
> > >
> > > > On Mon, Aug 17, 2020 at 4:25 PM binyu.wang at noaa.gov via RT <
> > >
> > > > met_help at ucar.edu> wrote:
> > >
> > > >
> > >
> > > > >
> > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270 >
> > >
> > > > >
> > >
> > > > > But why I got something weird in the log file, e.g:
> > >
> > > > >
> > >
> > > > > WARNING: check_hdr_str() -> null string!
> > >
> > > > >
> > >
> > > > > WARNING:
> > >
> > > > >
> > >
> > > > > WARNING: MetGrib2DataFile::data_plane() -> No matching
record found
> > for
> > >
> > > > > 'VAFTD/L14000-0'
> > >
> > > > >
> > >
> > > > > WARNING:
> > >
> > > > >
> > >
> > > > > WARNING:
> > >
> > > > >
> > >
> > > > > WARNING: process_n_vld() -> ensemble field "VAFTD/L14000-0"
not
> found
> > > in
> > >
> > > > > file
> > >
> > > > >
> > >
> > > > >
> > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/Hysplit/Reventador/Revent.21.0p5/T_cdump1.001.grib2"
> > >
> > > > >
> > >
> > > > >
> > >
> > > > > Binyu
> > >
> > > > >
> > >
> > > > > On Mon, Aug 17, 2020 at 5:40 PM John Halley Gotway via RT <
> > >
> > > > > met_help at ucar.edu>
> > >
> > > > > wrote:
> > >
> > > > >
> > >
> > > > > > Binyu,
> > >
> > > > > >
> > >
> > > > > > Thanks for posting some sample data for me to take a look
at.
> > Looking
> > >
> > > > at
> > >
> > > > > > the "log" file, I see you've run Ensemble-Stat 5 times for
hours
> > 18,
> > >
> > > > 19,
> > >
> > > > > > 20, 21, and 22. You sent the sample files for 18 hour
output
> time,
> > > and
> > >
> > > > I
> > >
> > > > > > see that Ensemble-Stat produced 91 matched pairs for that
time:
> > >
> > > > > >
> > >
> > > > > > egrep 'matched pairs|\.stat' log
> > >
> > > > > >
> > >
> > > > > > DEBUG 2: Processing point verification VAFTD/L14000-0
versus
> > >
> > > > > ash/Z10000-0,
> > >
> > > > > > for observation type ADPUPA, over region FULL, for
interpolation
> > > method
> > >
> > > > > > NEAREST(1), using 91 matched pairs.
> > >
> > > > > >
> > >
> > > > > > I checked the NetCDF file that you sent me and see that it
> contains
> > >
> > > > > exactly
> > >
> > > > > > 91 point observations for that time:
> > >
> > > > > >
> > >
> > > > > > ncdump -h goes16_ash_20190225-180000.nc
> > >
> > > > > >
> > >
> > > > > > netcdf goes16_ash_20190225-180000 {
> > >
> > > > > >
> > >
> > > > > > dimensions:
> > >
> > > > > >
> > >
> > > > > > ...
> > >
> > > > > >
> > >
> > > > > > nhdr = 91 ;
> > >
> > > > > >
> > >
> > > > > >
> > >
> > > > > > So Ensemble-Stat is using all available point observations
you
> are
> > >
> > > > > passing
> > >
> > > > > > to it.
> > >
> > > > > >
> > >
> > > > > > Looks to me like it's working as expected. Is there some
problem
> > I'm
> > >
> > > > not
> > >
> > > > > > understanding?
> > >
> > > > > >
> > >
> > > > > > John
> > >
> > > > > >
> > >
> > > > > > On Mon, Aug 17, 2020 at 10:56 AM binyu.wang at noaa.gov via
RT <
> > >
> > > > > > met_help at ucar.edu> wrote:
> > >
> > > > > >
> > >
> > > > > > >
> > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> >
> > >
> > > > > > >
> > >
> > > > > > > I know WCOSS/dev is down today, so I tried to upload the
config
> > > file
> > >
> > > > > and
> > >
> > > > > > > one of model (grib2 file) and obs data funder my last
name
> "wang"
> > >
> > > > > > following
> > >
> > > > > > > the rule:
> > >
> > > > > > >
> > >
> > > > > > >
> > >
> > > > > >
> > >
> > > > >
> > >
> > > >
> > >
> >
> http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > >
> > > > > > > .
> > >
> > > > > > > Let me know if further information is needed.
> > >
> > > > > > >
> > >
> > > > > > > Thank you
> > >
> > > > > > > Binyu
> > >
> > > > > > >
> > >
> > > > > > > On Mon, Aug 17, 2020 at 12:41 PM Binyu Wang - NOAA
Affiliate <
> > >
> > > > > > > binyu.wang at noaa.gov> wrote:
> > >
> > > > > > >
> > >
> > > > > > > > John,
> > >
> > > > > > > >
> > >
> > > > > > > > I did try that. Unfortunately, it doesn't work.
> > >
> > > > > > > >
> > >
> > > > > > > > Thank you.
> > >
> > > > > > > > Binyu
> > >
> > > > > > > >
> > >
> > > > > > > > On Mon, Aug 17, 2020 at 11:47 AM John Halley Gotway
via RT <
> > >
> > > > > > > > met_help at ucar.edu> wrote:
> > >
> > > > > > > >
> > >
> > > > > > > >> Binyu,
> > >
> > > > > > > >>
> > >
> > > > > > > >> I see you have a question about specifying level
information
> > in
> > >
> > > > > > > >> Ensemble-Stat. I hear that your GRIB2 model output
has a
> > single
> > >
> > > > > record
> > >
> > > > > > > for
> > >
> > > > > > > >> volcanic ash, whereas you have point observations at
> multiple
> > >
> > > > > heights.
> > >
> > > > > > > So
> > >
> > > > > > > >> you want to extract a single forecast record and
match it to
> > >
> > > > > multiple
> > >
> > > > > > > >> observation heights. Please correct me if that
description
> > isn't
> > >
> > > > > > > accurate.
> > >
> > > > > > > >>
> > >
> > > > > > > >> The good news is that this is easily done via the
> > Ensemble-Stat
> > >
> > > > > > > >> configuration file. Notice that that config file
contains
> > > separate
> > >
> > > > > > > >> dictionaries for the forecast and observation
settings,
> named
> > >
> > > > "fcst"
> > >
> > > > > > and
> > >
> > > > > > > >> "obs", respectively. In the default config file, we
just
> copy
> > > the
> > >
> > > > > fcst
> > >
> > > > > > > >> settings over to the obs by saying "obs = fcst". But
for
> this
> > >
> > > > case,
> > >
> > > > > > > we'll
> > >
> > > > > > > >> want to specify a full obs dictionary with a
different
> "level"
> > >
> > > > > > setting.
> > >
> > > > > > > By
> > >
> > > > > > > >> way of example, here's how you'd compare forecasts of
TMP at
> > > 500mb
> > >
> > > > > to
> > >
> > > > > > > TMP
> > >
> > > > > > > >> observations at heights between 0 and 10000. Note
that the
> 'Z'
> > >
> > > > > > > specifies a
> > >
> > > > > > > >> height level whereas 'P' specifies a pressure level:
> > >
> > > > > > > >>
> > >
> > > > > > > >> fcst = { field = [
> > >
> > > > > > > >> {
> > >
> > > > > > > >> name = "TMP";
> > >
> > > > > > > >> level = [ "P500" ];
> > >
> > > > > > > >> }
> > >
> > > > > > > >> ];
> > >
> > > > > > > >> }
> > >
> > > > > > > >> obs = { field = [ { name = "TMP"; level = [ "Z0-
10000" ]; }
> > ]; }
> > >
> > > > > > > >> Hopefully you can adapt this example to your case.
You just
> > need
> > >
> > > > to
> > >
> > > > > be
> > >
> > > > > > > >> sure
> > >
> > > > > > > >> that you specify the same number of forecast and
observation
> > >
> > > > fields.
> > >
> > > > > > > >>
> > >
> > > > > > > >> Please try reconfiguring Ensemble-Stat and rerunning
and let
> > me
> > >
> > > > know
> > >
> > > > > > how
> > >
> > > > > > > >> it
> > >
> > > > > > > >> goes.
> > >
> > > > > > > >>
> > >
> > > > > > > >> Thanks,
> > >
> > > > > > > >> John Halley Gotway
> > >
> > > > > > > >>
> > >
> > > > > > > >>
> > >
> > > > > > > >> On Sun, Aug 16, 2020 at 9:00 PM binyu.wang at noaa.gov
via RT
> <
> > >
> > > > > > > >> met_help at ucar.edu> wrote:
> > >
> > > > > > > >>
> > >
> > > > > > > >> >
> > >
> > > > > > > >> > Sun Aug 16 20:59:39 2020: Request 96270 was acted
upon.
> > >
> > > > > > > >> > Transaction: Ticket created by binyu.wang at noaa.gov
> > >
> > > > > > > >> > Queue: met_help
> > >
> > > > > > > >> > Subject: Is there a way to treat different
heights as
> > one
> > >
> > > > > layer
> > >
> > > > > > > in
> > >
> > > > > > > >> > config file?
> > >
> > > > > > > >> > Owner: Nobody
> > >
> > > > > > > >> > Requestors: binyu.wang at noaa.gov
> > >
> > > > > > > >> > Status: new
> > >
> > > > > > > >> > Ticket <URL:
> > >
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96270
> > >
> > > > > > > >
> > >
> > > > > > > >> >
> > >
> > > > > > > >> >
> > >
> > > > > > > >> > Hello,
> > >
> > > > > > > >> >
> > >
> > > > > > > >> > I am using "ensemble_stat" for volcano ash
verification
> and
> > I
> > >
> > > > got
> > >
> > > > > a
> > >
> > > > > > > >> > problem: my model forecast is in grib2 format with
only
> one
> > >
> > > > layer.
> > >
> > > > > > The
> > >
> > > > > > > >> obs.
> > >
> > > > > > > >> > data is converted from "ascii2nc", there is only
lat/lon,
> > >
> > > > height,
> > >
> > > > > > and
> > >
> > > > > > > >> > volcano ash inside. I want to treat all height as
one
> layer
> > as
> > >
> > > > the
> > >
> > > > > > > >> forecast
> > >
> > > > > > > >> > data does. How to set that in the config file? I
tried :
> > >
> > > > > > > >> > level=[ "L6000-9000" ] ;
> > >
> > > > > > > >> > and
> > >
> > > > > > > >> > level=[ "*,*" ] ;
> > >
> > > > > > > >> > but none of them work. Should I have to give a
fake SAME
> > > height
> > >
> > > > > for
> > >
> > > > > > > all
> > >
> > > > > > > >> > sites and regenerate the *nc files?
> > >
> > > > > > > >> >
> > >
> > > > > > > >> > My script is:
> > >
> > > > > > > >> >
> > >
> > > > > > > >> >
> > >
> > > > > > > >>
> > >
> > > > > > >
> > >
> > > > > >
> > >
> > > > >
> > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/ush/verf_g2o_Reventador.sh
> > >
> > > > > > > >> > and the config file is:
> > >
> > > > > > > >> >
> > >
> > > > > > > >> >
> > >
> > > > > > > >>
> > >
> > > > > > >
> > >
> > > > > >
> > >
> > > > >
> > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_ens/parm/verf_g2o_ens_stat_regn_config_Reventador
> > >
> > > > > > > >> >
> > >
> > > > > > > >> > Thank you.
> > >
> > > > > > > >> > Binyu
> > >
> > > > > > > >> >
> > >
> > > > > > > >> >
> > >
> > > > > > > >>
> > >
> > > > > > > >>
> > >
> > > > > > >
> > >
> > > > > > >
> > >
> > > > > >
> > >
> > > > > >
> > >
> > > > >
> > >
> > > > >
> > >
> > > >
> > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list