[Met_help] [rt.rap.ucar.edu #78350] History for Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
Julie Prestopnik via RT
met_help at ucar.edu
Fri Oct 28 11:07:27 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi, Julie,
For the late reply.
I had been working on another issue in my work and just fixed anc come back
on this:).
Thank you so much for your offering for further help.
Seems I can't directly transfer from Theia and I need to first try to
transfer to a local computer. Because I'm on tele-work today, I will send
them to you next week.
Sorry if that is maybe not as you planned.
Have a good weekend,
Ting
On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate <
ting.lei at noaa.gov> wrote:
> Dear Mr. or Mrs,
> I have a problem using Met _grid state to obtain the CTC for the composite
> reflectivity.
>
>
> From the output of met_grid_stat, the numbers for dbz observation >20 are
> always 0 (namely, numbers for ify_Oy and ifn_OY are always 0). By wgrib2
> seeing the observation file, I know there are maximum refc values larger
> than 60dbz. so, there should be other problems with my use of the
> metradar tool.
> In the Met running, there were warning message:
> ^^
> WARNING:
> WARNING: Multiple GRIB2 table entries match lookup criteria (parm_name =
> REFC):
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
> WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> WARNING:
> WARNING:
> WARNING: Multiple GRIB2 table entries match lookup criteria (parm_name =
> REFC):
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
> WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
>
> VV
>
> The MET in the NAMRR is first implemented by Jacob Charley. He has been
> using it smoothly, while some quantities like CTC causing the problem here
> were not calculated by him. He pointed out the differences between his
> and mine. I also attached here.
> ^^
> I'm using :
>
> //
> // Statistical output types
> //
> output_flag = {
> fho = BOTH;
> ctc = BOTH;
> cts = BOTH;
> mctc = BOTH;
> mcts = BOTH;
> cnt = BOTH;
> sl1l2 = BOTH;
> vl1l2 = BOTH;
> pct = BOTH;
> pstd = BOTH;
> pjc = BOTH;
> prc = BOTH;
> nbrctc = BOTH;
> nbrcts = BOTH;
> nbrcnt = BOTH;
> };
>
> where Jacob used, no problem:
>
> //
> // Statistical output types
> //
> output_flag = {
> fho = NONE;
> ctc = NONE;
> cts = NONE;
> mctc = NONE;
> mcts = NONE;
> cnt = NONE;
> sl1l2 = NONE;
> vl1l2 = NONE;
> pct = NONE;
> pstd = NONE;
> pjc = NONE;
> prc = NONE;
> nbrctc = NONE;
> nbrcts = NONE;
> nbrcnt = STAT;
> };
>
> VV
> Your help is appreciated.
> Best Regards,
> Ting
>
>
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Julie Prestopnik
Time: Fri Oct 21 08:34:58 2016
Hi Ting. I have access to Theia and may be able to help there. Could
you
please point me to your files (data and config) and send along the
command
you are running?
Thanks,
Julie
On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> Transaction: Ticket created by ting.lei at noaa.gov
> Queue: met_help
> Subject: Re: the "missed " samples for CTC calculation for the
> composite reflectivity observation.
> Owner: Nobody
> Requestors: ting.lei at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>
>
> Hi, Julie,
> For the late reply.
> I had been working on another issue in my work and just fixed anc
come back
> on this:).
> Thank you so much for your offering for further help.
> Seems I can't directly transfer from Theia and I need to first try
to
> transfer to a local computer. Because I'm on tele-work today, I
will send
> them to you next week.
> Sorry if that is maybe not as you planned.
> Have a good weekend,
> Ting
>
>
> On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate <
> ting.lei at noaa.gov> wrote:
>
> > Dear Mr. or Mrs,
> > I have a problem using Met _grid state to obtain the CTC for the
> composite
> > reflectivity.
> >
> >
> > From the output of met_grid_stat, the numbers for dbz observation
>20
> are
> > always 0 (namely, numbers for ify_Oy and ifn_OY are always 0). By
wgrib2
> > seeing the observation file, I know there are maximum refc values
larger
> > than 60dbz. so, there should be other problems with my use of
the
> > metradar tool.
> > In the Met running, there were warning message:
> > ^^
> > WARNING:
> > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> > REFC):
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > WARNING:
> > WARNING:
> > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> > REFC):
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> >
> > VV
> >
> > The MET in the NAMRR is first implemented by Jacob Charley. He has
been
> > using it smoothly, while some quantities like CTC causing the
problem
> here
> > were not calculated by him. He pointed out the differences
between his
> > and mine. I also attached here.
> > ^^
> > I'm using :
> >
> > //
> > // Statistical output types
> > //
> > output_flag = {
> > fho = BOTH;
> > ctc = BOTH;
> > cts = BOTH;
> > mctc = BOTH;
> > mcts = BOTH;
> > cnt = BOTH;
> > sl1l2 = BOTH;
> > vl1l2 = BOTH;
> > pct = BOTH;
> > pstd = BOTH;
> > pjc = BOTH;
> > prc = BOTH;
> > nbrctc = BOTH;
> > nbrcts = BOTH;
> > nbrcnt = BOTH;
> > };
> >
> > where Jacob used, no problem:
> >
> > //
> > // Statistical output types
> > //
> > output_flag = {
> > fho = NONE;
> > ctc = NONE;
> > cts = NONE;
> > mctc = NONE;
> > mcts = NONE;
> > cnt = NONE;
> > sl1l2 = NONE;
> > vl1l2 = NONE;
> > pct = NONE;
> > pstd = NONE;
> > pjc = NONE;
> > prc = NONE;
> > nbrctc = NONE;
> > nbrcts = NONE;
> > nbrcnt = STAT;
> > };
> >
> > VV
> > Your help is appreciated.
> > Best Regards,
> > Ting
> >
> >
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Ting Lei - NOAA Affiliate
Time: Fri Oct 21 08:45:16 2016
Julie,
Great. So many thanks.
The actual directory is in
/scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
however I put all files in
/scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17.
The command is like:
grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_01h_REFC
-outdir . -v 3.
(Plz be cautioned, I did it through the queue system on theia and I
didn't
do it in the front end ,
I tried and failed to load the correct module for MET 5.1).
For the RGD used in the configure file, it was defined as:
^^
RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
25.000"'
VV
I have the warning message:
^^
WARNING: Multiple GRIB2 table entries match lookup criteria (parm_name
=
REFC):
WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
WARNING:
WARNING:
WARNING: Multiple GRIB2 table entries match lookup criteria (parm_name
=
REFC):
WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
WARNING:
VV
The problem is in the
file
grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_20151030_230000V.stat,
for ctc , you can see the problem I reported (no observed refc was
counted.
The current run was with a new configure, while the parameter name is
REFC
(composite reflectivity ) rather than original REFCclm. I think this
is
because a new grib2 table being used.
In the observation file, by wgrib2 or g2ctl.pl, it could be shown the
refc
showed an normal appearance of a storm as as expected. So, seems
something
wrong that they were not counted in the verification.
It is very likely I missed something.
In case you need to know more, here is what I actually did for this
job.
In
/scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_NDAS_exp/NDAS_exp/trunk/namrr,
run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
Your help is appreciated.
Ting
BTW: I had used "chmod a+r" to process directories I mentioned above.
However, if still the problem, plz let me know and maybe I can try the
original plan (send the tared data to you).
On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT
<met_help at ucar.edu
> wrote:
> Hi Ting. I have access to Theia and may be able to help there.
Could you
> please point me to your files (data and config) and send along the
command
> you are running?
>
> Thanks,
> Julie
>
> On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > Transaction: Ticket created by ting.lei at noaa.gov
> > Queue: met_help
> > Subject: Re: the "missed " samples for CTC calculation for
the
> > composite reflectivity observation.
> > Owner: Nobody
> > Requestors: ting.lei at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> >
> >
> > Hi, Julie,
> > For the late reply.
> > I had been working on another issue in my work and just fixed anc
come
> back
> > on this:).
> > Thank you so much for your offering for further help.
> > Seems I can't directly transfer from Theia and I need to first try
to
> > transfer to a local computer. Because I'm on tele-work today, I
will
> send
> > them to you next week.
> > Sorry if that is maybe not as you planned.
> > Have a good weekend,
> > Ting
> >
> >
> > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate <
> > ting.lei at noaa.gov> wrote:
> >
> > > Dear Mr. or Mrs,
> > > I have a problem using Met _grid state to obtain the CTC for the
> > composite
> > > reflectivity.
> > >
> > >
> > > From the output of met_grid_stat, the numbers for dbz
observation >20
> > are
> > > always 0 (namely, numbers for ify_Oy and ifn_OY are always 0).
By
> wgrib2
> > > seeing the observation file, I know there are maximum refc
values
> larger
> > > than 60dbz. so, there should be other problems with my use of
the
> > > metradar tool.
> > > In the Met running, there were warning message:
> > > ^^
> > > WARNING:
> > > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name
> =
> > > REFC):
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
5
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> = 5
> > > WARNING:
> > > WARNING:
> > > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name
> =
> > > REFC):
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
5
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> = 5
> > >
> > > VV
> > >
> > > The MET in the NAMRR is first implemented by Jacob Charley. He
has been
> > > using it smoothly, while some quantities like CTC causing the
problem
> > here
> > > were not calculated by him. He pointed out the differences
between
> his
> > > and mine. I also attached here.
> > > ^^
> > > I'm using :
> > >
> > > //
> > > // Statistical output types
> > > //
> > > output_flag = {
> > > fho = BOTH;
> > > ctc = BOTH;
> > > cts = BOTH;
> > > mctc = BOTH;
> > > mcts = BOTH;
> > > cnt = BOTH;
> > > sl1l2 = BOTH;
> > > vl1l2 = BOTH;
> > > pct = BOTH;
> > > pstd = BOTH;
> > > pjc = BOTH;
> > > prc = BOTH;
> > > nbrctc = BOTH;
> > > nbrcts = BOTH;
> > > nbrcnt = BOTH;
> > > };
> > >
> > > where Jacob used, no problem:
> > >
> > > //
> > > // Statistical output types
> > > //
> > > output_flag = {
> > > fho = NONE;
> > > ctc = NONE;
> > > cts = NONE;
> > > mctc = NONE;
> > > mcts = NONE;
> > > cnt = NONE;
> > > sl1l2 = NONE;
> > > vl1l2 = NONE;
> > > pct = NONE;
> > > pstd = NONE;
> > > pjc = NONE;
> > > prc = NONE;
> > > nbrctc = NONE;
> > > nbrcts = NONE;
> > > nbrcnt = STAT;
> > > };
> > >
> > > VV
> > > Your help is appreciated.
> > > Best Regards,
> > > Ting
> > >
> > >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: John Halley Gotway
Time: Wed Oct 26 10:30:16 2016
Hello Ting,
We're trying to replicate the behavior you're seeing on theia when
running
Grid-Stat. However, we are not able to reproduce it.
Please take a look in /home/John.H.Gotway/MET_Help/lei_data_20161026
I've copied over your Grid-Stat config file and am pointing to the
fcst/obs
data in your test directory:
/scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17
When I run Grid-Stat on this data and compare to your output, there's
a big
difference in the number of matched pairs. I get 673085 pairs (i.e.
the
TOTAL column of the CTC output file) while you get 229861.
This difference makes me very suspicious of how you're regridding the
data. You indicated that you're using:
RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
25.000"'
Do you have any idea why my matched pair count is so different from
yours
when I'm pointing to exactly the same input files?
Also, I'd recommend that you turn ON the Grid-Stat NetCDF matched
pairs
output file, by setting the following in your config file:
nc_pairs_flag = TRUE;
Then we'll be able to look at the gridded data that Grid-Stat is
processing
and hopefully the problem will become obvious.
Thanks,
John Halley Gotway
On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>
> Julie,
> Great. So many thanks.
> The actual directory is in
> /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> however I put all files in
> /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17.
>
> The command is like:
> grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_01h_REFC
> -outdir . -v 3.
> (Plz be cautioned, I did it through the queue system on theia and I
didn't
> do it in the front end ,
> I tried and failed to load the correct module for MET 5.1).
> For the RGD used in the configure file, it was defined as:
> ^^
> RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
25.000"'
> VV
> I have the warning message:
> ^^
> WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> REFC):
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
> WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> WARNING:
> WARNING:
> WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> REFC):
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 196
> WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> WARNING:
>
>
> VV
>
>
> The problem is in the
> file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> 20151030_230000V.stat,
> for ctc , you can see the problem I reported (no observed refc was
counted.
> The current run was with a new configure, while the parameter name
is REFC
> (composite reflectivity ) rather than original REFCclm. I think
this is
> because a new grib2 table being used.
> In the observation file, by wgrib2 or g2ctl.pl, it could be shown
the refc
> showed an normal appearance of a storm as as expected. So, seems
something
> wrong that they were not counted in the verification.
> It is very likely I missed something.
>
> In case you need to know more, here is what I actually did for this
job.
> In /scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> NDAS_exp/NDAS_exp/trunk/namrr,
>
> run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
>
>
> Your help is appreciated.
> Ting
> BTW: I had used "chmod a+r" to process directories I mentioned
above.
> However, if still the problem, plz let me know and maybe I can try
the
> original plan (send the tared data to you).
>
>
> On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT <
> met_help at ucar.edu
> > wrote:
>
> > Hi Ting. I have access to Theia and may be able to help there.
Could
> you
> > please point me to your files (data and config) and send along the
> command
> > you are running?
> >
> > Thanks,
> > Julie
> >
> > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate via RT
<
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > > Transaction: Ticket created by ting.lei at noaa.gov
> > > Queue: met_help
> > > Subject: Re: the "missed " samples for CTC calculation for
the
> > > composite reflectivity observation.
> > > Owner: Nobody
> > > Requestors: ting.lei at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350
> >
> > >
> > >
> > > Hi, Julie,
> > > For the late reply.
> > > I had been working on another issue in my work and just fixed
anc come
> > back
> > > on this:).
> > > Thank you so much for your offering for further help.
> > > Seems I can't directly transfer from Theia and I need to first
try to
> > > transfer to a local computer. Because I'm on tele-work today,
I will
> > send
> > > them to you next week.
> > > Sorry if that is maybe not as you planned.
> > > Have a good weekend,
> > > Ting
> > >
> > >
> > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate <
> > > ting.lei at noaa.gov> wrote:
> > >
> > > > Dear Mr. or Mrs,
> > > > I have a problem using Met _grid state to obtain the CTC for
the
> > > composite
> > > > reflectivity.
> > > >
> > > >
> > > > From the output of met_grid_stat, the numbers for dbz
observation >20
> > > are
> > > > always 0 (namely, numbers for ify_Oy and ifn_OY are always 0).
By
> > wgrib2
> > > > seeing the observation file, I know there are maximum refc
values
> > larger
> > > > than 60dbz. so, there should be other problems with my use
of the
> > > > metradar tool.
> > > > In the Met running, there were warning message:
> > > > ^^
> > > > WARNING:
> > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> (parm_name
> > =
> > > > REFC):
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 196
> > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> > = 5
> > > > WARNING:
> > > > WARNING:
> > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> (parm_name
> > =
> > > > REFC):
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 196
> > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> > = 5
> > > >
> > > > VV
> > > >
> > > > The MET in the NAMRR is first implemented by Jacob Charley. He
has
> been
> > > > using it smoothly, while some quantities like CTC causing the
problem
> > > here
> > > > were not calculated by him. He pointed out the differences
between
> > his
> > > > and mine. I also attached here.
> > > > ^^
> > > > I'm using :
> > > >
> > > > //
> > > > // Statistical output types
> > > > //
> > > > output_flag = {
> > > > fho = BOTH;
> > > > ctc = BOTH;
> > > > cts = BOTH;
> > > > mctc = BOTH;
> > > > mcts = BOTH;
> > > > cnt = BOTH;
> > > > sl1l2 = BOTH;
> > > > vl1l2 = BOTH;
> > > > pct = BOTH;
> > > > pstd = BOTH;
> > > > pjc = BOTH;
> > > > prc = BOTH;
> > > > nbrctc = BOTH;
> > > > nbrcts = BOTH;
> > > > nbrcnt = BOTH;
> > > > };
> > > >
> > > > where Jacob used, no problem:
> > > >
> > > > //
> > > > // Statistical output types
> > > > //
> > > > output_flag = {
> > > > fho = NONE;
> > > > ctc = NONE;
> > > > cts = NONE;
> > > > mctc = NONE;
> > > > mcts = NONE;
> > > > cnt = NONE;
> > > > sl1l2 = NONE;
> > > > vl1l2 = NONE;
> > > > pct = NONE;
> > > > pstd = NONE;
> > > > pjc = NONE;
> > > > prc = NONE;
> > > > nbrctc = NONE;
> > > > nbrcts = NONE;
> > > > nbrcnt = STAT;
> > > > };
> > > >
> > > > VV
> > > > Your help is appreciated.
> > > > Best Regards,
> > > > Ting
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Ting Lei - NOAA Affiliate
Time: Wed Oct 26 12:50:58 2016
Hi, John,
Thank you and Julie so much for your help.
With your test run as a benchmark , I ran ,compared and digged and
found,
it is because of the different grid_stat I'm using.
We module load the same module as you did in that script: module load
verif/v4.0.0_verif.
However, you are using /scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat.
while, I'm using the on as shown by which command:
^^
0.310 + which grid_stat
/contrib/MET/5.1/bin/grid_stat
V
After I used yours to directly replace the original one, I got the
same
results as yours (that .stat file).
So, I guess there are some inconsistencies in my use.
I will cc this Jacob and maybe he will have more ideas.
For being now, I will just use your grid_stat.
Again, I really appreciate your efforts on this issue, which I guess I
won't find the cause unless I can spend a large amount of time
to test the source codes and,maybe, different libs compilations.
Best Regards,
Ting
Hi, Jacob,
Your attention on this is appreciated. Maybe, it is the problem on my
modification and use of the original scripts.
Especially, if you don't find any problems in your runs.
Regards,
Ting
On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hello Ting,
>
> We're trying to replicate the behavior you're seeing on theia when
running
> Grid-Stat. However, we are not able to reproduce it.
>
> Please take a look in /home/John.H.Gotway/MET_Help/lei_data_20161026
>
> I've copied over your Grid-Stat config file and am pointing to the
fcst/obs
> data in your test directory:
>
> /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17
>
> When I run Grid-Stat on this data and compare to your output,
there's a big
> difference in the number of matched pairs. I get 673085 pairs (i.e.
the
> TOTAL column of the CTC output file) while you get 229861.
>
> This difference makes me very suspicious of how you're regridding
the
> data. You indicated that you're using:
>
> RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
> 25.000"'
>
> Do you have any idea why my matched pair count is so different from
yours
> when I'm pointing to exactly the same input files?
>
> Also, I'd recommend that you turn ON the Grid-Stat NetCDF matched
pairs
> output file, by setting the following in your config file:
>
> nc_pairs_flag = TRUE;
>
> Then we'll be able to look at the gridded data that Grid-Stat is
processing
> and hopefully the problem will become obvious.
>
> Thanks,
> John Halley Gotway
>
> On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> >
> > Julie,
> > Great. So many thanks.
> > The actual directory is in
> > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> > however I put all files in
> > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17.
> >
> > The command is like:
> > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> > gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_01h_REFC
> > -outdir . -v 3.
> > (Plz be cautioned, I did it through the queue system on theia and
I
> didn't
> > do it in the front end ,
> > I tried and failed to load the correct module for MET 5.1).
> > For the RGD used in the configure file, it was defined as:
> > ^^
> > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
25.000"'
> > VV
> > I have the warning message:
> > ^^
> > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> > REFC):
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > WARNING:
> > WARNING:
> > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name =
> > REFC):
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c = 5
> > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > WARNING:
> >
> >
> > VV
> >
> >
> > The problem is in the
> > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> > 20151030_230000V.stat,
> > for ctc , you can see the problem I reported (no observed refc was
> counted.
> > The current run was with a new configure, while the parameter name
is
> REFC
> > (composite reflectivity ) rather than original REFCclm. I think
this is
> > because a new grib2 table being used.
> > In the observation file, by wgrib2 or g2ctl.pl, it could be shown
the
> refc
> > showed an normal appearance of a storm as as expected. So, seems
> something
> > wrong that they were not counted in the verification.
> > It is very likely I missed something.
> >
> > In case you need to know more, here is what I actually did for
this job.
> > In /scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> > NDAS_exp/NDAS_exp/trunk/namrr,
> >
> > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
> >
> >
> > Your help is appreciated.
> > Ting
> > BTW: I had used "chmod a+r" to process directories I mentioned
above.
> > However, if still the problem, plz let me know and maybe I can try
the
> > original plan (send the tared data to you).
> >
> >
> > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > wrote:
> >
> > > Hi Ting. I have access to Theia and may be able to help there.
Could
> > you
> > > please point me to your files (data and config) and send along
the
> > command
> > > you are running?
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > Queue: met_help
> > > > Subject: Re: the "missed " samples for CTC calculation
for the
> > > > composite reflectivity observation.
> > > > Owner: Nobody
> > > > Requestors: ting.lei at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=78350
> > >
> > > >
> > > >
> > > > Hi, Julie,
> > > > For the late reply.
> > > > I had been working on another issue in my work and just fixed
anc
> come
> > > back
> > > > on this:).
> > > > Thank you so much for your offering for further help.
> > > > Seems I can't directly transfer from Theia and I need to first
try to
> > > > transfer to a local computer. Because I'm on tele-work
today, I
> will
> > > send
> > > > them to you next week.
> > > > Sorry if that is maybe not as you planned.
> > > > Have a good weekend,
> > > > Ting
> > > >
> > > >
> > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate <
> > > > ting.lei at noaa.gov> wrote:
> > > >
> > > > > Dear Mr. or Mrs,
> > > > > I have a problem using Met _grid state to obtain the CTC for
the
> > > > composite
> > > > > reflectivity.
> > > > >
> > > > >
> > > > > From the output of met_grid_stat, the numbers for dbz
observation
> >20
> > > > are
> > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are always
0). By
> > > wgrib2
> > > > > seeing the observation file, I know there are maximum refc
values
> > > larger
> > > > > than 60dbz. so, there should be other problems with my use
of the
> > > > > metradar tool.
> > > > > In the Met running, there were warning message:
> > > > > ^^
> > > > > WARNING:
> > > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> > (parm_name
> > > =
> > > > > REFC):
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 196
> > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> index_c
> > > = 5
> > > > > WARNING:
> > > > > WARNING:
> > > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> > (parm_name
> > > =
> > > > > REFC):
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 196
> > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> index_c
> > > = 5
> > > > >
> > > > > VV
> > > > >
> > > > > The MET in the NAMRR is first implemented by Jacob Charley.
He has
> > been
> > > > > using it smoothly, while some quantities like CTC causing
the
> problem
> > > > here
> > > > > were not calculated by him. He pointed out the differences
> between
> > > his
> > > > > and mine. I also attached here.
> > > > > ^^
> > > > > I'm using :
> > > > >
> > > > > //
> > > > > // Statistical output types
> > > > > //
> > > > > output_flag = {
> > > > > fho = BOTH;
> > > > > ctc = BOTH;
> > > > > cts = BOTH;
> > > > > mctc = BOTH;
> > > > > mcts = BOTH;
> > > > > cnt = BOTH;
> > > > > sl1l2 = BOTH;
> > > > > vl1l2 = BOTH;
> > > > > pct = BOTH;
> > > > > pstd = BOTH;
> > > > > pjc = BOTH;
> > > > > prc = BOTH;
> > > > > nbrctc = BOTH;
> > > > > nbrcts = BOTH;
> > > > > nbrcnt = BOTH;
> > > > > };
> > > > >
> > > > > where Jacob used, no problem:
> > > > >
> > > > > //
> > > > > // Statistical output types
> > > > > //
> > > > > output_flag = {
> > > > > fho = NONE;
> > > > > ctc = NONE;
> > > > > cts = NONE;
> > > > > mctc = NONE;
> > > > > mcts = NONE;
> > > > > cnt = NONE;
> > > > > sl1l2 = NONE;
> > > > > vl1l2 = NONE;
> > > > > pct = NONE;
> > > > > pstd = NONE;
> > > > > pjc = NONE;
> > > > > prc = NONE;
> > > > > nbrctc = NONE;
> > > > > nbrcts = NONE;
> > > > > nbrcnt = STAT;
> > > > > };
> > > > >
> > > > > VV
> > > > > Your help is appreciated.
> > > > > Best Regards,
> > > > > Ting
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: John Halley Gotway
Time: Wed Oct 26 13:04:45 2016
Ting,
I logged on to theia to try to test out the differences between the
two
compiled versions of MET. I obviously hope that that isn't the actual
source of the problem!
However, I see your directory structure has changed and my links don't
work
anymore. I tried to copy over your sample forecast file:
/scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/rt5pint_tten-
pertcld1/namrr.20151030/namrr.t06z.hi_impnest.hiresf17.tm00.grib2
But the permissions don't allow me to copy it over. Can you please
place
that namrr file somewhere and open up the permission so that I can
copy it
over for more testing?
Thanks,
John
On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>
> Hi, John,
> Thank you and Julie so much for your help.
> With your test run as a benchmark , I ran ,compared and digged and
found,
> it is because of the different grid_stat I'm using.
> We module load the same module as you did in that script: module
load
> verif/v4.0.0_verif.
> However, you are using /scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat.
> while, I'm using the on as shown by which command:
> ^^
> 0.310 + which grid_stat
> /contrib/MET/5.1/bin/grid_stat
> V
> After I used yours to directly replace the original one, I got the
same
> results as yours (that .stat file).
> So, I guess there are some inconsistencies in my use.
> I will cc this Jacob and maybe he will have more ideas.
> For being now, I will just use your grid_stat.
> Again, I really appreciate your efforts on this issue, which I guess
I
> won't find the cause unless I can spend a large amount of time
> to test the source codes and,maybe, different libs compilations.
> Best Regards,
> Ting
> Hi, Jacob,
> Your attention on this is appreciated. Maybe, it is the problem on
my
> modification and use of the original scripts.
> Especially, if you don't find any problems in your runs.
> Regards,
> Ting
>
>
>
>
>
> On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hello Ting,
> >
> > We're trying to replicate the behavior you're seeing on theia when
> running
> > Grid-Stat. However, we are not able to reproduce it.
> >
> > Please take a look in
/home/John.H.Gotway/MET_Help/lei_data_20161026
> >
> > I've copied over your Grid-Stat config file and am pointing to the
> fcst/obs
> > data in your test directory:
> >
> > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-gridstat_refc_17
> >
> > When I run Grid-Stat on this data and compare to your output,
there's a
> big
> > difference in the number of matched pairs. I get 673085 pairs
(i.e. the
> > TOTAL column of the CTC output file) while you get 229861.
> >
> > This difference makes me very suspicious of how you're regridding
the
> > data. You indicated that you're using:
> >
> > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > 25.000"'
> >
> > Do you have any idea why my matched pair count is so different
from yours
> > when I'm pointing to exactly the same input files?
> >
> > Also, I'd recommend that you turn ON the Grid-Stat NetCDF matched
pairs
> > output file, by setting the following in your config file:
> >
> > nc_pairs_flag = TRUE;
> >
> > Then we'll be able to look at the gridded data that Grid-Stat is
> processing
> > and hopefully the problem will become obvious.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate via RT
<
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> > >
> > > Julie,
> > > Great. So many thanks.
> > > The actual directory is in
> > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> > > however I put all files in
> > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17.
> > >
> > > The command is like:
> > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> > > gridstat_refc_17/refd3d.t23z.grb2f00 parm/GridStatConfig_hi_imp_
> 01h_REFC
> > > -outdir . -v 3.
> > > (Plz be cautioned, I did it through the queue system on theia
and I
> > didn't
> > > do it in the front end ,
> > > I tried and failed to load the correct module for MET 5.1).
> > > For the RGD used in the configure file, it was defined as:
> > > ^^
> > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2 25.000
> 25.000"'
> > > VV
> > > I have the warning message:
> > > ^^
> > > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name
> =
> > > REFC):
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
5
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> = 5
> > > WARNING:
> > > WARNING:
> > > WARNING: Multiple GRIB2 table entries match lookup criteria
(parm_name
> =
> > > REFC):
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
5
> > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c =
196
> > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> = 5
> > > WARNING:
> > >
> > >
> > > VV
> > >
> > >
> > > The problem is in the
> > > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> > > 20151030_230000V.stat,
> > > for ctc , you can see the problem I reported (no observed refc
was
> > counted.
> > > The current run was with a new configure, while the parameter
name is
> > REFC
> > > (composite reflectivity ) rather than original REFCclm. I think
this
> is
> > > because a new grib2 table being used.
> > > In the observation file, by wgrib2 or g2ctl.pl, it could be
shown the
> > refc
> > > showed an normal appearance of a storm as as expected. So,
seems
> > something
> > > wrong that they were not counted in the verification.
> > > It is very likely I missed something.
> > >
> > > In case you need to know more, here is what I actually did for
this
> job.
> > > In /scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> > > NDAS_exp/NDAS_exp/trunk/namrr,
> > >
> > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
> > >
> > >
> > > Your help is appreciated.
> > > Ting
> > > BTW: I had used "chmod a+r" to process directories I mentioned
above.
> > > However, if still the problem, plz let me know and maybe I can
try the
> > > original plan (send the tared data to you).
> > >
> > >
> > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT <
> > > met_help at ucar.edu
> > > > wrote:
> > >
> > > > Hi Ting. I have access to Theia and may be able to help
there.
> Could
> > > you
> > > > please point me to your files (data and config) and send along
the
> > > command
> > > > you are running?
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: Re: the "missed " samples for CTC calculation
for the
> > > > > composite reflectivity observation.
> > > > > Owner: Nobody
> > > > > Requestors: ting.lei at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > Ticket/Display.html?id=78350
> > > >
> > > > >
> > > > >
> > > > > Hi, Julie,
> > > > > For the late reply.
> > > > > I had been working on another issue in my work and just
fixed anc
> > come
> > > > back
> > > > > on this:).
> > > > > Thank you so much for your offering for further help.
> > > > > Seems I can't directly transfer from Theia and I need to
first try
> to
> > > > > transfer to a local computer. Because I'm on tele-work
today, I
> > will
> > > > send
> > > > > them to you next week.
> > > > > Sorry if that is maybe not as you planned.
> > > > > Have a good weekend,
> > > > > Ting
> > > > >
> > > > >
> > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA Affiliate
<
> > > > > ting.lei at noaa.gov> wrote:
> > > > >
> > > > > > Dear Mr. or Mrs,
> > > > > > I have a problem using Met _grid state to obtain the CTC
for the
> > > > > composite
> > > > > > reflectivity.
> > > > > >
> > > > > >
> > > > > > From the output of met_grid_stat, the numbers for dbz
observation
> > >20
> > > > > are
> > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are always
0).
> By
> > > > wgrib2
> > > > > > seeing the observation file, I know there are maximum refc
values
> > > > larger
> > > > > > than 60dbz. so, there should be other problems with my
use of
> the
> > > > > > metradar tool.
> > > > > > In the Met running, there were warning message:
> > > > > > ^^
> > > > > > WARNING:
> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > (parm_name
> > > > =
> > > > > > REFC):
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 5
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> > 196
> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > > = 5
> > > > > > WARNING:
> > > > > > WARNING:
> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > (parm_name
> > > > =
> > > > > > REFC):
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 5
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> > 196
> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > > = 5
> > > > > >
> > > > > > VV
> > > > > >
> > > > > > The MET in the NAMRR is first implemented by Jacob
Charley. He
> has
> > > been
> > > > > > using it smoothly, while some quantities like CTC causing
the
> > problem
> > > > > here
> > > > > > were not calculated by him. He pointed out the
differences
> > between
> > > > his
> > > > > > and mine. I also attached here.
> > > > > > ^^
> > > > > > I'm using :
> > > > > >
> > > > > > //
> > > > > > // Statistical output types
> > > > > > //
> > > > > > output_flag = {
> > > > > > fho = BOTH;
> > > > > > ctc = BOTH;
> > > > > > cts = BOTH;
> > > > > > mctc = BOTH;
> > > > > > mcts = BOTH;
> > > > > > cnt = BOTH;
> > > > > > sl1l2 = BOTH;
> > > > > > vl1l2 = BOTH;
> > > > > > pct = BOTH;
> > > > > > pstd = BOTH;
> > > > > > pjc = BOTH;
> > > > > > prc = BOTH;
> > > > > > nbrctc = BOTH;
> > > > > > nbrcts = BOTH;
> > > > > > nbrcnt = BOTH;
> > > > > > };
> > > > > >
> > > > > > where Jacob used, no problem:
> > > > > >
> > > > > > //
> > > > > > // Statistical output types
> > > > > > //
> > > > > > output_flag = {
> > > > > > fho = NONE;
> > > > > > ctc = NONE;
> > > > > > cts = NONE;
> > > > > > mctc = NONE;
> > > > > > mcts = NONE;
> > > > > > cnt = NONE;
> > > > > > sl1l2 = NONE;
> > > > > > vl1l2 = NONE;
> > > > > > pct = NONE;
> > > > > > pstd = NONE;
> > > > > > pjc = NONE;
> > > > > > prc = NONE;
> > > > > > nbrctc = NONE;
> > > > > > nbrcts = NONE;
> > > > > > nbrcnt = STAT;
> > > > > > };
> > > > > >
> > > > > > VV
> > > > > > Your help is appreciated.
> > > > > > Best Regards,
> > > > > > Ting
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Ting Lei - NOAA Affiliate
Time: Wed Oct 26 14:40:58 2016
John,
Sorry. Those files were auto-deleted when I ran some debug-
experiments.
Now, I re-build it in which you can directly run your debug-
gridstat.sh.
I also chmod it and hope it is what you need.
Thank you.
Ting
On Wed, Oct 26, 2016 at 3:04 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ting,
>
> I logged on to theia to try to test out the differences between the
two
> compiled versions of MET. I obviously hope that that isn't the
actual
> source of the problem!
>
> However, I see your directory structure has changed and my links
don't work
> anymore. I tried to copy over your sample forecast file:
>
> /scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/
> rt5pint_tten-pertcld1/namrr.20151030/namrr.t06z.hi_
> impnest.hiresf17.tm00.grib2
>
> But the permissions don't allow me to copy it over. Can you please
place
> that namrr file somewhere and open up the permission so that I can
copy it
> over for more testing?
>
> Thanks,
> John
>
> On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> >
> > Hi, John,
> > Thank you and Julie so much for your help.
> > With your test run as a benchmark , I ran ,compared and digged and
found,
> > it is because of the different grid_stat I'm using.
> > We module load the same module as you did in that script: module
load
> > verif/v4.0.0_verif.
> > However, you are using /scratch4/BMC/dtc/MET/met-
5.1/bin/grid_stat.
> > while, I'm using the on as shown by which command:
> > ^^
> > 0.310 + which grid_stat
> > /contrib/MET/5.1/bin/grid_stat
> > V
> > After I used yours to directly replace the original one, I got the
same
> > results as yours (that .stat file).
> > So, I guess there are some inconsistencies in my use.
> > I will cc this Jacob and maybe he will have more ideas.
> > For being now, I will just use your grid_stat.
> > Again, I really appreciate your efforts on this issue, which I
guess I
> > won't find the cause unless I can spend a large amount of time
> > to test the source codes and,maybe, different libs compilations.
> > Best Regards,
> > Ting
> > Hi, Jacob,
> > Your attention on this is appreciated. Maybe, it is the problem
on my
> > modification and use of the original scripts.
> > Especially, if you don't find any problems in your runs.
> > Regards,
> > Ting
> >
> >
> >
> >
> >
> > On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hello Ting,
> > >
> > > We're trying to replicate the behavior you're seeing on theia
when
> > running
> > > Grid-Stat. However, we are not able to reproduce it.
> > >
> > > Please take a look in
/home/John.H.Gotway/MET_Help/lei_data_20161026
> > >
> > > I've copied over your Grid-Stat config file and am pointing to
the
> > fcst/obs
> > > data in your test directory:
> > >
> > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17
> > >
> > > When I run Grid-Stat on this data and compare to your output,
there's a
> > big
> > > difference in the number of matched pairs. I get 673085 pairs
(i.e.
> the
> > > TOTAL column of the CTC output file) while you get 229861.
> > >
> > > This difference makes me very suspicious of how you're
regridding the
> > > data. You indicated that you're using:
> > >
> > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > > 25.000"'
> > >
> > > Do you have any idea why my matched pair count is so different
from
> yours
> > > when I'm pointing to exactly the same input files?
> > >
> > > Also, I'd recommend that you turn ON the Grid-Stat NetCDF
matched pairs
> > > output file, by setting the following in your config file:
> > >
> > > nc_pairs_flag = TRUE;
> > >
> > > Then we'll be able to look at the gridded data that Grid-Stat is
> > processing
> > > and hopefully the problem will become obvious.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350
>
> > > >
> > > > Julie,
> > > > Great. So many thanks.
> > > > The actual directory is in
> > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> > > > however I put all files in
> > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17.
> > > >
> > > > The command is like:
> > > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> > > > gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_
> > 01h_REFC
> > > > -outdir . -v 3.
> > > > (Plz be cautioned, I did it through the queue system on theia
and I
> > > didn't
> > > > do it in the front end ,
> > > > I tried and failed to load the correct module for MET 5.1).
> > > > For the RGD used in the configure file, it was defined as:
> > > > ^^
> > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > 25.000"'
> > > > VV
> > > > I have the warning message:
> > > > ^^
> > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> (parm_name
> > =
> > > > REFC):
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 196
> > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> > = 5
> > > > WARNING:
> > > > WARNING:
> > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> (parm_name
> > =
> > > > REFC):
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 5
> > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16, index_c
= 196
> > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> > = 5
> > > > WARNING:
> > > >
> > > >
> > > > VV
> > > >
> > > >
> > > > The problem is in the
> > > > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> > > > 20151030_230000V.stat,
> > > > for ctc , you can see the problem I reported (no observed refc
was
> > > counted.
> > > > The current run was with a new configure, while the parameter
name is
> > > REFC
> > > > (composite reflectivity ) rather than original REFCclm. I
think this
> > is
> > > > because a new grib2 table being used.
> > > > In the observation file, by wgrib2 or g2ctl.pl, it could be
shown
> the
> > > refc
> > > > showed an normal appearance of a storm as as expected. So,
seems
> > > something
> > > > wrong that they were not counted in the verification.
> > > > It is very likely I missed something.
> > > >
> > > > In case you need to know more, here is what I actually did for
this
> > job.
> > > > In /scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> > > > NDAS_exp/NDAS_exp/trunk/namrr,
> > > >
> > > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
> > > >
> > > >
> > > > Your help is appreciated.
> > > > Ting
> > > > BTW: I had used "chmod a+r" to process directories I
mentioned
> above.
> > > > However, if still the problem, plz let me know and maybe I can
try
> the
> > > > original plan (send the tared data to you).
> > > >
> > > >
> > > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT <
> > > > met_help at ucar.edu
> > > > > wrote:
> > > >
> > > > > Hi Ting. I have access to Theia and may be able to help
there.
> > Could
> > > > you
> > > > > please point me to your files (data and config) and send
along the
> > > > command
> > > > > you are running?
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate
via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > > > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: Re: the "missed " samples for CTC
calculation for
> the
> > > > > > composite reflectivity observation.
> > > > > > Owner: Nobody
> > > > > > Requestors: ting.lei at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > Ticket/Display.html?id=78350
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi, Julie,
> > > > > > For the late reply.
> > > > > > I had been working on another issue in my work and just
fixed anc
> > > come
> > > > > back
> > > > > > on this:).
> > > > > > Thank you so much for your offering for further help.
> > > > > > Seems I can't directly transfer from Theia and I need to
first
> try
> > to
> > > > > > transfer to a local computer. Because I'm on tele-work
today, I
> > > will
> > > > > send
> > > > > > them to you next week.
> > > > > > Sorry if that is maybe not as you planned.
> > > > > > Have a good weekend,
> > > > > > Ting
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA
Affiliate <
> > > > > > ting.lei at noaa.gov> wrote:
> > > > > >
> > > > > > > Dear Mr. or Mrs,
> > > > > > > I have a problem using Met _grid state to obtain the CTC
for
> the
> > > > > > composite
> > > > > > > reflectivity.
> > > > > > >
> > > > > > >
> > > > > > > From the output of met_grid_stat, the numbers for dbz
> observation
> > > >20
> > > > > > are
> > > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are
always 0).
> > By
> > > > > wgrib2
> > > > > > > seeing the observation file, I know there are maximum
refc
> values
> > > > > larger
> > > > > > > than 60dbz. so, there should be other problems with my
use of
> > the
> > > > > > > metradar tool.
> > > > > > > In the Met running, there were warning message:
> > > > > > > ^^
> > > > > > > WARNING:
> > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > (parm_name
> > > > > =
> > > > > > > REFC):
> > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> =
> > 5
> > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> =
> > > 196
> > > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b
= 16,
> > > index_c
> > > > > = 5
> > > > > > > WARNING:
> > > > > > > WARNING:
> > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > (parm_name
> > > > > =
> > > > > > > REFC):
> > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> =
> > 5
> > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
> =
> > > 196
> > > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b
= 16,
> > > index_c
> > > > > = 5
> > > > > > >
> > > > > > > VV
> > > > > > >
> > > > > > > The MET in the NAMRR is first implemented by Jacob
Charley. He
> > has
> > > > been
> > > > > > > using it smoothly, while some quantities like CTC
causing the
> > > problem
> > > > > > here
> > > > > > > were not calculated by him. He pointed out the
differences
> > > between
> > > > > his
> > > > > > > and mine. I also attached here.
> > > > > > > ^^
> > > > > > > I'm using :
> > > > > > >
> > > > > > > //
> > > > > > > // Statistical output types
> > > > > > > //
> > > > > > > output_flag = {
> > > > > > > fho = BOTH;
> > > > > > > ctc = BOTH;
> > > > > > > cts = BOTH;
> > > > > > > mctc = BOTH;
> > > > > > > mcts = BOTH;
> > > > > > > cnt = BOTH;
> > > > > > > sl1l2 = BOTH;
> > > > > > > vl1l2 = BOTH;
> > > > > > > pct = BOTH;
> > > > > > > pstd = BOTH;
> > > > > > > pjc = BOTH;
> > > > > > > prc = BOTH;
> > > > > > > nbrctc = BOTH;
> > > > > > > nbrcts = BOTH;
> > > > > > > nbrcnt = BOTH;
> > > > > > > };
> > > > > > >
> > > > > > > where Jacob used, no problem:
> > > > > > >
> > > > > > > //
> > > > > > > // Statistical output types
> > > > > > > //
> > > > > > > output_flag = {
> > > > > > > fho = NONE;
> > > > > > > ctc = NONE;
> > > > > > > cts = NONE;
> > > > > > > mctc = NONE;
> > > > > > > mcts = NONE;
> > > > > > > cnt = NONE;
> > > > > > > sl1l2 = NONE;
> > > > > > > vl1l2 = NONE;
> > > > > > > pct = NONE;
> > > > > > > pstd = NONE;
> > > > > > > pjc = NONE;
> > > > > > > prc = NONE;
> > > > > > > nbrctc = NONE;
> > > > > > > nbrcts = NONE;
> > > > > > > nbrcnt = STAT;
> > > > > > > };
> > > > > > >
> > > > > > > VV
> > > > > > > Your help is appreciated.
> > > > > > > Best Regards,
> > > > > > > Ting
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: John Halley Gotway
Time: Wed Oct 26 16:29:14 2016
Ting,
Thanks for making the forecast file available again. I ran this data
through Grid-Stat using the two different versions, and I turned on
the
NetCDF matched pairs output file. Then I used the MET plot_data_plane
tool
to plot the resulting observation field and the problem is obvious.
The attached images show that regridding using
/scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat works file, while
regridding
using /contrib/MET/5.1/bin/grid_stat produces bad output.
Here are the plotting commands I used:
plot_data_plane
out1/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_230000V_pairs.nc
\
obs1.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20 50
plot_data_plane
out2/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_230000V_pairs.nc
\
obs2.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20 50
Now the question is why is there a difference?
Those two different builds contain different patch levels. The
following
commands reveal the patch level of these versions:
head -3 /contrib/MET/5.1/README
head -3 /scratch4/BMC/dtc/MET/met-5.1/README
Revision 3569 versus 3613.
Unfortunately, looking at the notes on bugfixes, the offending issue
is not
obvious:
http://www.dtcenter.org/met/users/support/known_issues/METv5.1/index.php
But I suspect recompiling /contrib/MET/5.1 with the latest set of
patches
would fix the problem.
Thanks,
John
On Wed, Oct 26, 2016 at 2:40 PM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>
> John,
> Sorry. Those files were auto-deleted when I ran some debug-
experiments.
> Now, I re-build it in which you can directly run your debug-
gridstat.sh.
> I also chmod it and hope it is what you need.
> Thank you.
> Ting
>
> On Wed, Oct 26, 2016 at 3:04 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Ting,
> >
> > I logged on to theia to try to test out the differences between
the two
> > compiled versions of MET. I obviously hope that that isn't the
actual
> > source of the problem!
> >
> > However, I see your directory structure has changed and my links
don't
> work
> > anymore. I tried to copy over your sample forecast file:
> >
> > /scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/
> > rt5pint_tten-pertcld1/namrr.20151030/namrr.t06z.hi_
> > impnest.hiresf17.tm00.grib2
> >
> > But the permissions don't allow me to copy it over. Can you
please place
> > that namrr file somewhere and open up the permission so that I can
copy
> it
> > over for more testing?
> >
> > Thanks,
> > John
> >
> > On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate via RT
<
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> > >
> > > Hi, John,
> > > Thank you and Julie so much for your help.
> > > With your test run as a benchmark , I ran ,compared and digged
and
> found,
> > > it is because of the different grid_stat I'm using.
> > > We module load the same module as you did in that script:
module load
> > > verif/v4.0.0_verif.
> > > However, you are using /scratch4/BMC/dtc/MET/met-
5.1/bin/grid_stat.
> > > while, I'm using the on as shown by which command:
> > > ^^
> > > 0.310 + which grid_stat
> > > /contrib/MET/5.1/bin/grid_stat
> > > V
> > > After I used yours to directly replace the original one, I got
the same
> > > results as yours (that .stat file).
> > > So, I guess there are some inconsistencies in my use.
> > > I will cc this Jacob and maybe he will have more ideas.
> > > For being now, I will just use your grid_stat.
> > > Again, I really appreciate your efforts on this issue, which I
guess I
> > > won't find the cause unless I can spend a large amount of time
> > > to test the source codes and,maybe, different libs compilations.
> > > Best Regards,
> > > Ting
> > > Hi, Jacob,
> > > Your attention on this is appreciated. Maybe, it is the problem
on my
> > > modification and use of the original scripts.
> > > Especially, if you don't find any problems in your runs.
> > > Regards,
> > > Ting
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Hello Ting,
> > > >
> > > > We're trying to replicate the behavior you're seeing on theia
when
> > > running
> > > > Grid-Stat. However, we are not able to reproduce it.
> > > >
> > > > Please take a look in
/home/John.H.Gotway/MET_Help/lei_data_20161026
> > > >
> > > > I've copied over your Grid-Stat config file and am pointing to
the
> > > fcst/obs
> > > > data in your test directory:
> > > >
> > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17
> > > >
> > > > When I run Grid-Stat on this data and compare to your output,
> there's a
> > > big
> > > > difference in the number of matched pairs. I get 673085 pairs
(i.e.
> > the
> > > > TOTAL column of the CTC output file) while you get 229861.
> > > >
> > > > This difference makes me very suspicious of how you're
regridding the
> > > > data. You indicated that you're using:
> > > >
> > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > > > 25.000"'
> > > >
> > > > Do you have any idea why my matched pair count is so different
from
> > yours
> > > > when I'm pointing to exactly the same input files?
> > > >
> > > > Also, I'd recommend that you turn ON the Grid-Stat NetCDF
matched
> pairs
> > > > output file, by setting the following in your config file:
> > > >
> > > > nc_pairs_flag = TRUE;
> > > >
> > > > Then we'll be able to look at the gridded data that Grid-Stat
is
> > > processing
> > > > and hopefully the problem will become obvious.
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> > > > >
> > > > > Julie,
> > > > > Great. So many thanks.
> > > > > The actual directory is in
> > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> > > > > however I put all files in
> > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17.
> > > > >
> > > > > The command is like:
> > > > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> > > > > gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_
> > > 01h_REFC
> > > > > -outdir . -v 3.
> > > > > (Plz be cautioned, I did it through the queue system on
theia and I
> > > > didn't
> > > > > do it in the front end ,
> > > > > I tried and failed to load the correct module for MET 5.1).
> > > > > For the RGD used in the configure file, it was defined as:
> > > > > ^^
> > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > > 25.000"'
> > > > > VV
> > > > > I have the warning message:
> > > > > ^^
> > > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> > (parm_name
> > > =
> > > > > REFC):
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 196
> > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> index_c
> > > = 5
> > > > > WARNING:
> > > > > WARNING:
> > > > > WARNING: Multiple GRIB2 table entries match lookup criteria
> > (parm_name
> > > =
> > > > > REFC):
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c = 5
> > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 196
> > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> index_c
> > > = 5
> > > > > WARNING:
> > > > >
> > > > >
> > > > > VV
> > > > >
> > > > >
> > > > > The problem is in the
> > > > > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> > > > > 20151030_230000V.stat,
> > > > > for ctc , you can see the problem I reported (no observed
refc was
> > > > counted.
> > > > > The current run was with a new configure, while the
parameter name
> is
> > > > REFC
> > > > > (composite reflectivity ) rather than original REFCclm. I
think
> this
> > > is
> > > > > because a new grib2 table being used.
> > > > > In the observation file, by wgrib2 or g2ctl.pl, it could be
shown
> > the
> > > > refc
> > > > > showed an normal appearance of a storm as as expected. So,
seems
> > > > something
> > > > > wrong that they were not counted in the verification.
> > > > > It is very likely I missed something.
> > > > >
> > > > > In case you need to know more, here is what I actually did
for this
> > > job.
> > > > > In /scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> > > > > NDAS_exp/NDAS_exp/trunk/namrr,
> > > > >
> > > > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
> > > > >
> > > > >
> > > > > Your help is appreciated.
> > > > > Ting
> > > > > BTW: I had used "chmod a+r" to process directories I
mentioned
> > above.
> > > > > However, if still the problem, plz let me know and maybe I
can try
> > the
> > > > > original plan (send the tared data to you).
> > > > >
> > > > >
> > > > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT <
> > > > > met_help at ucar.edu
> > > > > > wrote:
> > > > >
> > > > > > Hi Ting. I have access to Theia and may be able to help
there.
> > > Could
> > > > > you
> > > > > > please point me to your files (data and config) and send
along
> the
> > > > > command
> > > > > > you are running?
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA Affiliate
via
> RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted upon.
> > > > > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: Re: the "missed " samples for CTC
calculation for
> > the
> > > > > > > composite reflectivity observation.
> > > > > > > Owner: Nobody
> > > > > > > Requestors: ting.lei at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > Ticket/Display.html?id=78350
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi, Julie,
> > > > > > > For the late reply.
> > > > > > > I had been working on another issue in my work and just
fixed
> anc
> > > > come
> > > > > > back
> > > > > > > on this:).
> > > > > > > Thank you so much for your offering for further help.
> > > > > > > Seems I can't directly transfer from Theia and I need to
first
> > try
> > > to
> > > > > > > transfer to a local computer. Because I'm on tele-work
> today, I
> > > > will
> > > > > > send
> > > > > > > them to you next week.
> > > > > > > Sorry if that is maybe not as you planned.
> > > > > > > Have a good weekend,
> > > > > > > Ting
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA
Affiliate <
> > > > > > > ting.lei at noaa.gov> wrote:
> > > > > > >
> > > > > > > > Dear Mr. or Mrs,
> > > > > > > > I have a problem using Met _grid state to obtain the
CTC for
> > the
> > > > > > > composite
> > > > > > > > reflectivity.
> > > > > > > >
> > > > > > > >
> > > > > > > > From the output of met_grid_stat, the numbers for dbz
> > observation
> > > > >20
> > > > > > > are
> > > > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are
always
> 0).
> > > By
> > > > > > wgrib2
> > > > > > > > seeing the observation file, I know there are maximum
refc
> > values
> > > > > > larger
> > > > > > > > than 60dbz. so, there should be other problems with
my use
> of
> > > the
> > > > > > > > metradar tool.
> > > > > > > > In the Met running, there were warning message:
> > > > > > > > ^^
> > > > > > > > WARNING:
> > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > > (parm_name
> > > > > > =
> > > > > > > > REFC):
> > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
> index_c
> > =
> > > 5
> > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
> index_c
> > =
> > > > 196
> > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b = 16,
> > > > index_c
> > > > > > = 5
> > > > > > > > WARNING:
> > > > > > > > WARNING:
> > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > > (parm_name
> > > > > > =
> > > > > > > > REFC):
> > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
> index_c
> > =
> > > 5
> > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
> index_c
> > =
> > > > 196
> > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b = 16,
> > > > index_c
> > > > > > = 5
> > > > > > > >
> > > > > > > > VV
> > > > > > > >
> > > > > > > > The MET in the NAMRR is first implemented by Jacob
Charley.
> He
> > > has
> > > > > been
> > > > > > > > using it smoothly, while some quantities like CTC
causing the
> > > > problem
> > > > > > > here
> > > > > > > > were not calculated by him. He pointed out the
differences
> > > > between
> > > > > > his
> > > > > > > > and mine. I also attached here.
> > > > > > > > ^^
> > > > > > > > I'm using :
> > > > > > > >
> > > > > > > > //
> > > > > > > > // Statistical output types
> > > > > > > > //
> > > > > > > > output_flag = {
> > > > > > > > fho = BOTH;
> > > > > > > > ctc = BOTH;
> > > > > > > > cts = BOTH;
> > > > > > > > mctc = BOTH;
> > > > > > > > mcts = BOTH;
> > > > > > > > cnt = BOTH;
> > > > > > > > sl1l2 = BOTH;
> > > > > > > > vl1l2 = BOTH;
> > > > > > > > pct = BOTH;
> > > > > > > > pstd = BOTH;
> > > > > > > > pjc = BOTH;
> > > > > > > > prc = BOTH;
> > > > > > > > nbrctc = BOTH;
> > > > > > > > nbrcts = BOTH;
> > > > > > > > nbrcnt = BOTH;
> > > > > > > > };
> > > > > > > >
> > > > > > > > where Jacob used, no problem:
> > > > > > > >
> > > > > > > > //
> > > > > > > > // Statistical output types
> > > > > > > > //
> > > > > > > > output_flag = {
> > > > > > > > fho = NONE;
> > > > > > > > ctc = NONE;
> > > > > > > > cts = NONE;
> > > > > > > > mctc = NONE;
> > > > > > > > mcts = NONE;
> > > > > > > > cnt = NONE;
> > > > > > > > sl1l2 = NONE;
> > > > > > > > vl1l2 = NONE;
> > > > > > > > pct = NONE;
> > > > > > > > pstd = NONE;
> > > > > > > > pjc = NONE;
> > > > > > > > prc = NONE;
> > > > > > > > nbrctc = NONE;
> > > > > > > > nbrcts = NONE;
> > > > > > > > nbrcnt = STAT;
> > > > > > > > };
> > > > > > > >
> > > > > > > > VV
> > > > > > > > Your help is appreciated.
> > > > > > > > Best Regards,
> > > > > > > > Ting
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Ting Lei - NOAA Affiliate
Time: Wed Oct 26 17:00:56 2016
Hi, John,
Thank you so much for your investigation and clarification.
I will cc this to Jacob and he will do futher if needed.
So many thanks.
Ting
Jacob,
Would you plz have a look into this issue when you have a chance.
Thanks.
On Wed, Oct 26, 2016 at 6:29 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ting,
>
> Thanks for making the forecast file available again. I ran this
data
> through Grid-Stat using the two different versions, and I turned on
the
> NetCDF matched pairs output file. Then I used the MET
plot_data_plane tool
> to plot the resulting observation field and the problem is obvious.
>
> The attached images show that regridding using
> /scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat works file, while
regridding
> using /contrib/MET/5.1/bin/grid_stat produces bad output.
>
> Here are the plotting commands I used:
>
> plot_data_plane
> out1/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_
> 20151030_230000V_pairs.nc
> \
> obs1.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20 50
>
> plot_data_plane
> out2/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_
> 20151030_230000V_pairs.nc
> \
> obs2.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20 50
>
> Now the question is why is there a difference?
>
> Those two different builds contain different patch levels. The
following
> commands reveal the patch level of these versions:
>
> head -3 /contrib/MET/5.1/README
> head -3 /scratch4/BMC/dtc/MET/met-5.1/README
>
> Revision 3569 versus 3613.
>
> Unfortunately, looking at the notes on bugfixes, the offending issue
is not
> obvious:
>
> http://www.dtcenter.org/met/users/support/known_issues/
> METv5.1/index.php
>
> But I suspect recompiling /contrib/MET/5.1 with the latest set of
patches
> would fix the problem.
>
> Thanks,
> John
>
> On Wed, Oct 26, 2016 at 2:40 PM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> >
> > John,
> > Sorry. Those files were auto-deleted when I ran some debug-
experiments.
> > Now, I re-build it in which you can directly run your debug-
gridstat.sh.
> > I also chmod it and hope it is what you need.
> > Thank you.
> > Ting
> >
> > On Wed, Oct 26, 2016 at 3:04 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Ting,
> > >
> > > I logged on to theia to try to test out the differences between
the two
> > > compiled versions of MET. I obviously hope that that isn't the
actual
> > > source of the problem!
> > >
> > > However, I see your directory structure has changed and my links
don't
> > work
> > > anymore. I tried to copy over your sample forecast file:
> > >
> > > /scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/
> > > rt5pint_tten-pertcld1/namrr.20151030/namrr.t06z.hi_
> > > impnest.hiresf17.tm00.grib2
> > >
> > > But the permissions don't allow me to copy it over. Can you
please
> place
> > > that namrr file somewhere and open up the permission so that I
can copy
> > it
> > > over for more testing?
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350
>
> > > >
> > > > Hi, John,
> > > > Thank you and Julie so much for your help.
> > > > With your test run as a benchmark , I ran ,compared and digged
and
> > found,
> > > > it is because of the different grid_stat I'm using.
> > > > We module load the same module as you did in that script:
module
> load
> > > > verif/v4.0.0_verif.
> > > > However, you are using /scratch4/BMC/dtc/MET/met-
5.1/bin/grid_stat.
> > > > while, I'm using the on as shown by which command:
> > > > ^^
> > > > 0.310 + which grid_stat
> > > > /contrib/MET/5.1/bin/grid_stat
> > > > V
> > > > After I used yours to directly replace the original one, I got
the
> same
> > > > results as yours (that .stat file).
> > > > So, I guess there are some inconsistencies in my use.
> > > > I will cc this Jacob and maybe he will have more ideas.
> > > > For being now, I will just use your grid_stat.
> > > > Again, I really appreciate your efforts on this issue, which I
guess
> I
> > > > won't find the cause unless I can spend a large amount of time
> > > > to test the source codes and,maybe, different libs
compilations.
> > > > Best Regards,
> > > > Ting
> > > > Hi, Jacob,
> > > > Your attention on this is appreciated. Maybe, it is the
problem on
> my
> > > > modification and use of the original scripts.
> > > > Especially, if you don't find any problems in your runs.
> > > > Regards,
> > > > Ting
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Hello Ting,
> > > > >
> > > > > We're trying to replicate the behavior you're seeing on
theia when
> > > > running
> > > > > Grid-Stat. However, we are not able to reproduce it.
> > > > >
> > > > > Please take a look in /home/John.H.Gotway/MET_Help/
> lei_data_20161026
> > > > >
> > > > > I've copied over your Grid-Stat config file and am pointing
to the
> > > > fcst/obs
> > > > > data in your test directory:
> > > > >
> > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17
> > > > >
> > > > > When I run Grid-Stat on this data and compare to your
output,
> > there's a
> > > > big
> > > > > difference in the number of matched pairs. I get 673085
pairs
> (i.e.
> > > the
> > > > > TOTAL column of the CTC output file) while you get 229861.
> > > > >
> > > > > This difference makes me very suspicious of how you're
regridding
> the
> > > > > data. You indicated that you're using:
> > > > >
> > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > > > > 25.000"'
> > > > >
> > > > > Do you have any idea why my matched pair count is so
different from
> > > yours
> > > > > when I'm pointing to exactly the same input files?
> > > > >
> > > > > Also, I'd recommend that you turn ON the Grid-Stat NetCDF
matched
> > pairs
> > > > > output file, by setting the following in your config file:
> > > > >
> > > > > nc_pairs_flag = TRUE;
> > > > >
> > > > > Then we'll be able to look at the gridded data that Grid-
Stat is
> > > > processing
> > > > > and hopefully the problem will become obvious.
> > > > >
> > > > > Thanks,
> > > > > John Halley Gotway
> > > > >
> > > > > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate
via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
> > > > > >
> > > > > > Julie,
> > > > > > Great. So many thanks.
> > > > > > The actual directory is in
> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > > >
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
> > > > > > however I put all files in
> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
> > > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
> gridstat_refc_17.
> > > > > >
> > > > > > The command is like:
> > > > > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
> > > > > > gridstat_refc_17/refd3d.t23z.grb2f00
parm/GridStatConfig_hi_imp_
> > > > 01h_REFC
> > > > > > -outdir . -v 3.
> > > > > > (Plz be cautioned, I did it through the queue system on
theia
> and I
> > > > > didn't
> > > > > > do it in the front end ,
> > > > > > I tried and failed to load the correct module for MET
5.1).
> > > > > > For the RGD used in the configure file, it was defined
as:
> > > > > > ^^
> > > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
> > > > 25.000"'
> > > > > > VV
> > > > > > I have the warning message:
> > > > > > ^^
> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > (parm_name
> > > > =
> > > > > > REFC):
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 5
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> > 196
> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > > = 5
> > > > > > WARNING:
> > > > > > WARNING:
> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > (parm_name
> > > > =
> > > > > > REFC):
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> 5
> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
> > 196
> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > > = 5
> > > > > > WARNING:
> > > > > >
> > > > > >
> > > > > > VV
> > > > > >
> > > > > >
> > > > > > The problem is in the
> > > > > > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
> > > > > > 20151030_230000V.stat,
> > > > > > for ctc , you can see the problem I reported (no observed
refc
> was
> > > > > counted.
> > > > > > The current run was with a new configure, while the
parameter
> name
> > is
> > > > > REFC
> > > > > > (composite reflectivity ) rather than original REFCclm. I
think
> > this
> > > > is
> > > > > > because a new grib2 table being used.
> > > > > > In the observation file, by wgrib2 or g2ctl.pl, it could
be
> shown
> > > the
> > > > > refc
> > > > > > showed an normal appearance of a storm as as expected. So,
seems
> > > > > something
> > > > > > wrong that they were not counted in the verification.
> > > > > > It is very likely I missed something.
> > > > > >
> > > > > > In case you need to know more, here is what I actually did
for
> this
> > > > job.
> > > > > > In
/scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
> > > > > > NDAS_exp/NDAS_exp/trunk/namrr,
> > > > > >
> > > > > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
> > > > > >
> > > > > >
> > > > > > Your help is appreciated.
> > > > > > Ting
> > > > > > BTW: I had used "chmod a+r" to process directories I
mentioned
> > > above.
> > > > > > However, if still the problem, plz let me know and maybe I
can
> try
> > > the
> > > > > > original plan (send the tared data to you).
> > > > > >
> > > > > >
> > > > > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT
<
> > > > > > met_help at ucar.edu
> > > > > > > wrote:
> > > > > >
> > > > > > > Hi Ting. I have access to Theia and may be able to help
there.
> > > > Could
> > > > > > you
> > > > > > > please point me to your files (data and config) and send
along
> > the
> > > > > > command
> > > > > > > you are running?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA
Affiliate via
> > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted
upon.
> > > > > > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: Re: the "missed " samples for CTC
calculation
> for
> > > the
> > > > > > > > composite reflectivity observation.
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: ting.lei at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > Ticket/Display.html?id=78350
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi, Julie,
> > > > > > > > For the late reply.
> > > > > > > > I had been working on another issue in my work and
just fixed
> > anc
> > > > > come
> > > > > > > back
> > > > > > > > on this:).
> > > > > > > > Thank you so much for your offering for further help.
> > > > > > > > Seems I can't directly transfer from Theia and I need
to
> first
> > > try
> > > > to
> > > > > > > > transfer to a local computer. Because I'm on tele-
work
> > today, I
> > > > > will
> > > > > > > send
> > > > > > > > them to you next week.
> > > > > > > > Sorry if that is maybe not as you planned.
> > > > > > > > Have a good weekend,
> > > > > > > > Ting
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA
Affiliate <
> > > > > > > > ting.lei at noaa.gov> wrote:
> > > > > > > >
> > > > > > > > > Dear Mr. or Mrs,
> > > > > > > > > I have a problem using Met _grid state to obtain the
CTC
> for
> > > the
> > > > > > > > composite
> > > > > > > > > reflectivity.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > From the output of met_grid_stat, the numbers for
dbz
> > > observation
> > > > > >20
> > > > > > > > are
> > > > > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are
always
> > 0).
> > > > By
> > > > > > > wgrib2
> > > > > > > > > seeing the observation file, I know there are
maximum refc
> > > values
> > > > > > > larger
> > > > > > > > > than 60dbz. so, there should be other problems
with my
> use
> > of
> > > > the
> > > > > > > > > metradar tool.
> > > > > > > > > In the Met running, there were warning message:
> > > > > > > > > ^^
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > > > (parm_name
> > > > > > > =
> > > > > > > > > REFC):
> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > =
> > > > 5
> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > =
> > > > > 196
> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b =
> 16,
> > > > > index_c
> > > > > > > = 5
> > > > > > > > > WARNING:
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
> > > > > > (parm_name
> > > > > > > =
> > > > > > > > > REFC):
> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > =
> > > > 5
> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
> > index_c
> > > =
> > > > > 196
> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b =
> 16,
> > > > > index_c
> > > > > > > = 5
> > > > > > > > >
> > > > > > > > > VV
> > > > > > > > >
> > > > > > > > > The MET in the NAMRR is first implemented by Jacob
Charley.
> > He
> > > > has
> > > > > > been
> > > > > > > > > using it smoothly, while some quantities like CTC
causing
> the
> > > > > problem
> > > > > > > > here
> > > > > > > > > were not calculated by him. He pointed out the
> differences
> > > > > between
> > > > > > > his
> > > > > > > > > and mine. I also attached here.
> > > > > > > > > ^^
> > > > > > > > > I'm using :
> > > > > > > > >
> > > > > > > > > //
> > > > > > > > > // Statistical output types
> > > > > > > > > //
> > > > > > > > > output_flag = {
> > > > > > > > > fho = BOTH;
> > > > > > > > > ctc = BOTH;
> > > > > > > > > cts = BOTH;
> > > > > > > > > mctc = BOTH;
> > > > > > > > > mcts = BOTH;
> > > > > > > > > cnt = BOTH;
> > > > > > > > > sl1l2 = BOTH;
> > > > > > > > > vl1l2 = BOTH;
> > > > > > > > > pct = BOTH;
> > > > > > > > > pstd = BOTH;
> > > > > > > > > pjc = BOTH;
> > > > > > > > > prc = BOTH;
> > > > > > > > > nbrctc = BOTH;
> > > > > > > > > nbrcts = BOTH;
> > > > > > > > > nbrcnt = BOTH;
> > > > > > > > > };
> > > > > > > > >
> > > > > > > > > where Jacob used, no problem:
> > > > > > > > >
> > > > > > > > > //
> > > > > > > > > // Statistical output types
> > > > > > > > > //
> > > > > > > > > output_flag = {
> > > > > > > > > fho = NONE;
> > > > > > > > > ctc = NONE;
> > > > > > > > > cts = NONE;
> > > > > > > > > mctc = NONE;
> > > > > > > > > mcts = NONE;
> > > > > > > > > cnt = NONE;
> > > > > > > > > sl1l2 = NONE;
> > > > > > > > > vl1l2 = NONE;
> > > > > > > > > pct = NONE;
> > > > > > > > > pstd = NONE;
> > > > > > > > > pjc = NONE;
> > > > > > > > > prc = NONE;
> > > > > > > > > nbrctc = NONE;
> > > > > > > > > nbrcts = NONE;
> > > > > > > > > nbrcnt = STAT;
> > > > > > > > > };
> > > > > > > > >
> > > > > > > > > VV
> > > > > > > > > Your help is appreciated.
> > > > > > > > > Best Regards,
> > > > > > > > > Ting
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > Phone: 303.497.8399
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Ting Lei - NOAA Affiliate
Time: Wed Oct 26 17:08:28 2016
Also,
Thank you for your clear descrpition on how you approached this
issue.
It is helpful for my mastering of MET's use.
Regards,
Ting
On Wed, Oct 26, 2016 at 6:52 PM, Ting Lei - NOAA Affiliate <
ting.lei at noaa.gov> wrote:
> Hi, John,
> Thank you so much for your investigation and clarification.
> I will cc this to Jacob and he will do futher if needed.
> So many thanks.
> Ting
> Jacob,
> Would you plz have a look into this issue when you have a chance.
> Thanks.
>
> On Wed, Oct 26, 2016 at 6:29 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ting,
>>
>> Thanks for making the forecast file available again. I ran this
data
>> through Grid-Stat using the two different versions, and I turned on
the
>> NetCDF matched pairs output file. Then I used the MET
plot_data_plane
>> tool
>> to plot the resulting observation field and the problem is obvious.
>>
>> The attached images show that regridding using
>> /scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat works file, while
regridding
>> using /contrib/MET/5.1/bin/grid_stat produces bad output.
>>
>> Here are the plotting commands I used:
>>
>> plot_data_plane
>> out1/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_
>> 230000V_pairs.nc
>> \
>> obs1.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20
>> 50
>>
>> plot_data_plane
>> out2/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_
>> 230000V_pairs.nc
>> \
>> obs2.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";' -plot_range
-20
>> 50
>>
>> Now the question is why is there a difference?
>>
>> Those two different builds contain different patch levels. The
following
>> commands reveal the patch level of these versions:
>>
>> head -3 /contrib/MET/5.1/README
>> head -3 /scratch4/BMC/dtc/MET/met-5.1/README
>>
>> Revision 3569 versus 3613.
>>
>> Unfortunately, looking at the notes on bugfixes, the offending
issue is
>> not
>> obvious:
>>
>> http://www.dtcenter.org/met/users/support/known_issues/METv
>> 5.1/index.php
>>
>> But I suspect recompiling /contrib/MET/5.1 with the latest set of
patches
>> would fix the problem.
>>
>> Thanks,
>> John
>>
>> On Wed, Oct 26, 2016 at 2:40 PM, Ting Lei - NOAA Affiliate via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>> >
>> > John,
>> > Sorry. Those files were auto-deleted when I ran some debug-
experiments.
>> > Now, I re-build it in which you can directly run your debug-
gridstat.sh.
>> > I also chmod it and hope it is what you need.
>> > Thank you.
>> > Ting
>> >
>> > On Wed, Oct 26, 2016 at 3:04 PM, John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > > Ting,
>> > >
>> > > I logged on to theia to try to test out the differences between
the
>> two
>> > > compiled versions of MET. I obviously hope that that isn't the
actual
>> > > source of the problem!
>> > >
>> > > However, I see your directory structure has changed and my
links don't
>> > work
>> > > anymore. I tried to copy over your sample forecast file:
>> > >
>> > > /scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/
>> > > rt5pint_tten-pertcld1/namrr.20151030/namrr.t06z.hi_
>> > > impnest.hiresf17.tm00.grib2
>> > >
>> > > But the permissions don't allow me to copy it over. Can you
please
>> place
>> > > that namrr file somewhere and open up the permission so that I
can
>> copy
>> > it
>> > > over for more testing?
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate via
RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350
>
>> > > >
>> > > > Hi, John,
>> > > > Thank you and Julie so much for your help.
>> > > > With your test run as a benchmark , I ran ,compared and
digged and
>> > found,
>> > > > it is because of the different grid_stat I'm using.
>> > > > We module load the same module as you did in that script:
module
>> load
>> > > > verif/v4.0.0_verif.
>> > > > However, you are using /scratch4/BMC/dtc/MET/met-
5.1/bin/grid_stat.
>> > > > while, I'm using the on as shown by which command:
>> > > > ^^
>> > > > 0.310 + which grid_stat
>> > > > /contrib/MET/5.1/bin/grid_stat
>> > > > V
>> > > > After I used yours to directly replace the original one, I
got the
>> same
>> > > > results as yours (that .stat file).
>> > > > So, I guess there are some inconsistencies in my use.
>> > > > I will cc this Jacob and maybe he will have more ideas.
>> > > > For being now, I will just use your grid_stat.
>> > > > Again, I really appreciate your efforts on this issue, which
I
>> guess I
>> > > > won't find the cause unless I can spend a large amount of
time
>> > > > to test the source codes and,maybe, different libs
compilations.
>> > > > Best Regards,
>> > > > Ting
>> > > > Hi, Jacob,
>> > > > Your attention on this is appreciated. Maybe, it is the
problem on
>> my
>> > > > modification and use of the original scripts.
>> > > > Especially, if you don't find any problems in your runs.
>> > > > Regards,
>> > > > Ting
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT <
>> > > > met_help at ucar.edu> wrote:
>> > > >
>> > > > > Hello Ting,
>> > > > >
>> > > > > We're trying to replicate the behavior you're seeing on
theia when
>> > > > running
>> > > > > Grid-Stat. However, we are not able to reproduce it.
>> > > > >
>> > > > > Please take a look in /home/John.H.Gotway/MET_Help/l
>> ei_data_20161026
>> > > > >
>> > > > > I've copied over your Grid-Stat config file and am pointing
to the
>> > > > fcst/obs
>> > > > > data in your test directory:
>> > > > >
>> > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>> > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_refc_17
>> > > > >
>> > > > > When I run Grid-Stat on this data and compare to your
output,
>> > there's a
>> > > > big
>> > > > > difference in the number of matched pairs. I get 673085
pairs
>> (i.e.
>> > > the
>> > > > > TOTAL column of the CTC output file) while you get 229861.
>> > > > >
>> > > > > This difference makes me very suspicious of how you're
regridding
>> the
>> > > > > data. You indicated that you're using:
>> > > > >
>> > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270
6371.2
>> 25.000
>> > > > > 25.000"'
>> > > > >
>> > > > > Do you have any idea why my matched pair count is so
different
>> from
>> > > yours
>> > > > > when I'm pointing to exactly the same input files?
>> > > > >
>> > > > > Also, I'd recommend that you turn ON the Grid-Stat NetCDF
matched
>> > pairs
>> > > > > output file, by setting the following in your config file:
>> > > > >
>> > > > > nc_pairs_flag = TRUE;
>> > > > >
>> > > > > Then we'll be able to look at the gridded data that Grid-
Stat is
>> > > > processing
>> > > > > and hopefully the problem will become obvious.
>> > > > >
>> > > > > Thanks,
>> > > > > John Halley Gotway
>> > > > >
>> > > > > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate
via RT
>> <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>> > > > > >
>> > > > > > Julie,
>> > > > > > Great. So many thanks.
>> > > > > > The actual directory is in
>> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>> > > > > >
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
>> > > > > > however I put all files in
>> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>> > > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_
>> refc_17.
>> > > > > >
>> > > > > > The command is like:
>> > > > > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
>> > > > > > gridstat_refc_17/refd3d.t23z.grb2f00
>> parm/GridStatConfig_hi_imp_
>> > > > 01h_REFC
>> > > > > > -outdir . -v 3.
>> > > > > > (Plz be cautioned, I did it through the queue system on
theia
>> and I
>> > > > > didn't
>> > > > > > do it in the front end ,
>> > > > > > I tried and failed to load the correct module for MET
5.1).
>> > > > > > For the RGD used in the configure file, it was defined
as:
>> > > > > > ^^
>> > > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270 6371.2
25.000
>> > > > 25.000"'
>> > > > > > VV
>> > > > > > I have the warning message:
>> > > > > > ^^
>> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
>> > > (parm_name
>> > > > =
>> > > > > > REFC):
>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>> = 5
>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
>> > 196
>> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > > = 5
>> > > > > > WARNING:
>> > > > > > WARNING:
>> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
>> > > (parm_name
>> > > > =
>> > > > > > REFC):
>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>> = 5
>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c =
>> > 196
>> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > > = 5
>> > > > > > WARNING:
>> > > > > >
>> > > > > >
>> > > > > > VV
>> > > > > >
>> > > > > >
>> > > > > > The problem is in the
>> > > > > > file grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
>> > > > > > 20151030_230000V.stat,
>> > > > > > for ctc , you can see the problem I reported (no observed
refc
>> was
>> > > > > counted.
>> > > > > > The current run was with a new configure, while the
parameter
>> name
>> > is
>> > > > > REFC
>> > > > > > (composite reflectivity ) rather than original REFCclm.
I think
>> > this
>> > > > is
>> > > > > > because a new grib2 table being used.
>> > > > > > In the observation file, by wgrib2 or g2ctl.pl, it could
be
>> shown
>> > > the
>> > > > > refc
>> > > > > > showed an normal appearance of a storm as as expected.
So,
>> seems
>> > > > > something
>> > > > > > wrong that they were not counted in the verification.
>> > > > > > It is very likely I missed something.
>> > > > > >
>> > > > > > In case you need to know more, here is what I actually
did for
>> this
>> > > > job.
>> > > > > > In
/scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
>> > > > > > NDAS_exp/NDAS_exp/trunk/namrr,
>> > > > > >
>> > > > > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
>> > > > > >
>> > > > > >
>> > > > > > Your help is appreciated.
>> > > > > > Ting
>> > > > > > BTW: I had used "chmod a+r" to process directories I
mentioned
>> > > above.
>> > > > > > However, if still the problem, plz let me know and maybe
I can
>> try
>> > > the
>> > > > > > original plan (send the tared data to you).
>> > > > > >
>> > > > > >
>> > > > > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via RT
<
>> > > > > > met_help at ucar.edu
>> > > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Ting. I have access to Theia and may be able to
help
>> there.
>> > > > Could
>> > > > > > you
>> > > > > > > please point me to your files (data and config) and
send along
>> > the
>> > > > > > command
>> > > > > > > you are running?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Julie
>> > > > > > >
>> > > > > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA
Affiliate via
>> > RT <
>> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted
upon.
>> > > > > > > > Transaction: Ticket created by ting.lei at noaa.gov
>> > > > > > > > Queue: met_help
>> > > > > > > > Subject: Re: the "missed " samples for CTC
calculation
>> for
>> > > the
>> > > > > > > > composite reflectivity observation.
>> > > > > > > > Owner: Nobody
>> > > > > > > > Requestors: ting.lei at noaa.gov
>> > > > > > > > Status: new
>> > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
>> > > > > Ticket/Display.html?id=78350
>> > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Hi, Julie,
>> > > > > > > > For the late reply.
>> > > > > > > > I had been working on another issue in my work and
just
>> fixed
>> > anc
>> > > > > come
>> > > > > > > back
>> > > > > > > > on this:).
>> > > > > > > > Thank you so much for your offering for further help.
>> > > > > > > > Seems I can't directly transfer from Theia and I need
to
>> first
>> > > try
>> > > > to
>> > > > > > > > transfer to a local computer. Because I'm on tele-
work
>> > today, I
>> > > > > will
>> > > > > > > send
>> > > > > > > > them to you next week.
>> > > > > > > > Sorry if that is maybe not as you planned.
>> > > > > > > > Have a good weekend,
>> > > > > > > > Ting
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA
Affiliate
>> <
>> > > > > > > > ting.lei at noaa.gov> wrote:
>> > > > > > > >
>> > > > > > > > > Dear Mr. or Mrs,
>> > > > > > > > > I have a problem using Met _grid state to obtain
the CTC
>> for
>> > > the
>> > > > > > > > composite
>> > > > > > > > > reflectivity.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > From the output of met_grid_stat, the numbers for
dbz
>> > > observation
>> > > > > >20
>> > > > > > > > are
>> > > > > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY are
always
>> > 0).
>> > > > By
>> > > > > > > wgrib2
>> > > > > > > > > seeing the observation file, I know there are
maximum refc
>> > > values
>> > > > > > > larger
>> > > > > > > > > than 60dbz. so, there should be other problems
with my
>> use
>> > of
>> > > > the
>> > > > > > > > > metradar tool.
>> > > > > > > > > In the Met running, there were warning message:
>> > > > > > > > > ^^
>> > > > > > > > > WARNING:
>> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
>> criteria
>> > > > > > (parm_name
>> > > > > > > =
>> > > > > > > > > REFC):
>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > =
>> > > > 5
>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > =
>> > > > > 196
>> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b =
>> 16,
>> > > > > index_c
>> > > > > > > = 5
>> > > > > > > > > WARNING:
>> > > > > > > > > WARNING:
>> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
>> criteria
>> > > > > > (parm_name
>> > > > > > > =
>> > > > > > > > > REFC):
>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > =
>> > > > 5
>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>> > index_c
>> > > =
>> > > > > 196
>> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b =
>> 16,
>> > > > > index_c
>> > > > > > > = 5
>> > > > > > > > >
>> > > > > > > > > VV
>> > > > > > > > >
>> > > > > > > > > The MET in the NAMRR is first implemented by Jacob
>> Charley.
>> > He
>> > > > has
>> > > > > > been
>> > > > > > > > > using it smoothly, while some quantities like CTC
causing
>> the
>> > > > > problem
>> > > > > > > > here
>> > > > > > > > > were not calculated by him. He pointed out the
>> differences
>> > > > > between
>> > > > > > > his
>> > > > > > > > > and mine. I also attached here.
>> > > > > > > > > ^^
>> > > > > > > > > I'm using :
>> > > > > > > > >
>> > > > > > > > > //
>> > > > > > > > > // Statistical output types
>> > > > > > > > > //
>> > > > > > > > > output_flag = {
>> > > > > > > > > fho = BOTH;
>> > > > > > > > > ctc = BOTH;
>> > > > > > > > > cts = BOTH;
>> > > > > > > > > mctc = BOTH;
>> > > > > > > > > mcts = BOTH;
>> > > > > > > > > cnt = BOTH;
>> > > > > > > > > sl1l2 = BOTH;
>> > > > > > > > > vl1l2 = BOTH;
>> > > > > > > > > pct = BOTH;
>> > > > > > > > > pstd = BOTH;
>> > > > > > > > > pjc = BOTH;
>> > > > > > > > > prc = BOTH;
>> > > > > > > > > nbrctc = BOTH;
>> > > > > > > > > nbrcts = BOTH;
>> > > > > > > > > nbrcnt = BOTH;
>> > > > > > > > > };
>> > > > > > > > >
>> > > > > > > > > where Jacob used, no problem:
>> > > > > > > > >
>> > > > > > > > > //
>> > > > > > > > > // Statistical output types
>> > > > > > > > > //
>> > > > > > > > > output_flag = {
>> > > > > > > > > fho = NONE;
>> > > > > > > > > ctc = NONE;
>> > > > > > > > > cts = NONE;
>> > > > > > > > > mctc = NONE;
>> > > > > > > > > mcts = NONE;
>> > > > > > > > > cnt = NONE;
>> > > > > > > > > sl1l2 = NONE;
>> > > > > > > > > vl1l2 = NONE;
>> > > > > > > > > pct = NONE;
>> > > > > > > > > pstd = NONE;
>> > > > > > > > > pjc = NONE;
>> > > > > > > > > prc = NONE;
>> > > > > > > > > nbrctc = NONE;
>> > > > > > > > > nbrcts = NONE;
>> > > > > > > > > nbrcnt = STAT;
>> > > > > > > > > };
>> > > > > > > > >
>> > > > > > > > > VV
>> > > > > > > > > Your help is appreciated.
>> > > > > > > > > Best Regards,
>> > > > > > > > > Ting
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Julie Prestopnik
>> > > > > > > National Center for Atmospheric Research
>> > > > > > > Research Applications Laboratory
>> > > > > > > Phone: 303.497.8399
>> > > > > > > Email: jpresto at ucar.edu
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
>
------------------------------------------------
Subject: Re: the "missed " samples for CTC calculation for the composite reflectivity observation.
From: Jacob Carley - NOAA Affiliate
Time: Thu Oct 27 04:29:44 2016
HI Ting,
Thanks for letting me know about this (and thanks John for your
help!).
Ting: Could you try running this on Tide/Gyre to see if you have the
same
problem? The same version of MET (v5.1) is installed over there as
well
and I'm curious to see if the problem exists in both places before
patching
the install on Theia.
We should probably test the Cray environment as well, although that
has the
newest version of MET installed (as of late August, 2016).
Thanks,
Jacob
On Wed, Oct 26, 2016 at 7:00 PM, Ting Lei - NOAA Affiliate <
ting.lei at noaa.gov> wrote:
> Also,
> Thank you for your clear descrpition on how you approached this
issue.
> It is helpful for my mastering of MET's use.
> Regards,
> Ting
>
>
> On Wed, Oct 26, 2016 at 6:52 PM, Ting Lei - NOAA Affiliate <
> ting.lei at noaa.gov> wrote:
>
>> Hi, John,
>> Thank you so much for your investigation and clarification.
>> I will cc this to Jacob and he will do futher if needed.
>> So many thanks.
>> Ting
>> Jacob,
>> Would you plz have a look into this issue when you have a chance.
>> Thanks.
>>
>> On Wed, Oct 26, 2016 at 6:29 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Ting,
>>>
>>> Thanks for making the forecast file available again. I ran this
data
>>> through Grid-Stat using the two different versions, and I turned
on the
>>> NetCDF matched pairs output file. Then I used the MET
plot_data_plane
>>> tool
>>> to plot the resulting observation field and the problem is
obvious.
>>>
>>> The attached images show that regridding using
>>> /scratch4/BMC/dtc/MET/met-5.1/bin/grid_stat works file, while
regridding
>>> using /contrib/MET/5.1/bin/grid_stat produces bad output.
>>>
>>> Here are the plotting commands I used:
>>>
>>> plot_data_plane
>>> out1/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_
>>> 230000V_pairs.nc
>>> \
>>> obs1.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";'
-plot_range -20
>>> 50
>>>
>>> plot_data_plane
>>> out2/grid_stat_MODEL_DOMAIN_FNONE_01h_REFC_170000L_20151030_
>>> 230000V_pairs.nc
>>> \
>>> obs2.ps 'name="OBS_REFC_L0_FULL"; level="(*,*)";'
-plot_range -20
>>> 50
>>>
>>> Now the question is why is there a difference?
>>>
>>> Those two different builds contain different patch levels. The
following
>>> commands reveal the patch level of these versions:
>>>
>>> head -3 /contrib/MET/5.1/README
>>> head -3 /scratch4/BMC/dtc/MET/met-5.1/README
>>>
>>> Revision 3569 versus 3613.
>>>
>>> Unfortunately, looking at the notes on bugfixes, the offending
issue is
>>> not
>>> obvious:
>>>
>>> http://www.dtcenter.org/met/users/support/known_issues/METv
>>> 5.1/index.php
>>>
>>> But I suspect recompiling /contrib/MET/5.1 with the latest set of
patches
>>> would fix the problem.
>>>
>>> Thanks,
>>> John
>>>
>>> On Wed, Oct 26, 2016 at 2:40 PM, Ting Lei - NOAA Affiliate via RT
<
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>>> >
>>> > John,
>>> > Sorry. Those files were auto-deleted when I ran some debug-
experiments.
>>> > Now, I re-build it in which you can directly run your
>>> debug-gridstat.sh.
>>> > I also chmod it and hope it is what you need.
>>> > Thank you.
>>> > Ting
>>> >
>>> > On Wed, Oct 26, 2016 at 3:04 PM, John Halley Gotway via RT <
>>> > met_help at ucar.edu> wrote:
>>> >
>>> > > Ting,
>>> > >
>>> > > I logged on to theia to try to test out the differences
between the
>>> two
>>> > > compiled versions of MET. I obviously hope that that isn't
the
>>> actual
>>> > > source of the problem!
>>> > >
>>> > > However, I see your directory structure has changed and my
links
>>> don't
>>> > work
>>> > > anymore. I tried to copy over your sample forecast file:
>>> > >
>>> > > /scratch4/NCEPDEV/meso/noscrub/Ting.Lei/com/namrr/
>>> > > rt5pint_tten-pertcld1/namrr.20151030/namrr.t06z.hi_
>>> > > impnest.hiresf17.tm00.grib2
>>> > >
>>> > > But the permissions don't allow me to copy it over. Can you
please
>>> place
>>> > > that namrr file somewhere and open up the permission so that I
can
>>> copy
>>> > it
>>> > > over for more testing?
>>> > >
>>> > > Thanks,
>>> > > John
>>> > >
>>> > > On Wed, Oct 26, 2016 at 12:50 PM, Ting Lei - NOAA Affiliate
via RT <
>>> > > met_help at ucar.edu> wrote:
>>> > >
>>> > > >
>>> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350 >
>>> > > >
>>> > > > Hi, John,
>>> > > > Thank you and Julie so much for your help.
>>> > > > With your test run as a benchmark , I ran ,compared and
digged and
>>> > found,
>>> > > > it is because of the different grid_stat I'm using.
>>> > > > We module load the same module as you did in that script:
module
>>> load
>>> > > > verif/v4.0.0_verif.
>>> > > > However, you are using /scratch4/BMC/dtc/MET/met-5.1/
>>> bin/grid_stat.
>>> > > > while, I'm using the on as shown by which command:
>>> > > > ^^
>>> > > > 0.310 + which grid_stat
>>> > > > /contrib/MET/5.1/bin/grid_stat
>>> > > > V
>>> > > > After I used yours to directly replace the original one, I
got the
>>> same
>>> > > > results as yours (that .stat file).
>>> > > > So, I guess there are some inconsistencies in my use.
>>> > > > I will cc this Jacob and maybe he will have more ideas.
>>> > > > For being now, I will just use your grid_stat.
>>> > > > Again, I really appreciate your efforts on this issue, which
I
>>> guess I
>>> > > > won't find the cause unless I can spend a large amount of
time
>>> > > > to test the source codes and,maybe, different libs
compilations.
>>> > > > Best Regards,
>>> > > > Ting
>>> > > > Hi, Jacob,
>>> > > > Your attention on this is appreciated. Maybe, it is the
problem
>>> on my
>>> > > > modification and use of the original scripts.
>>> > > > Especially, if you don't find any problems in your runs.
>>> > > > Regards,
>>> > > > Ting
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Wed, Oct 26, 2016 at 12:30 PM, John Halley Gotway via RT
<
>>> > > > met_help at ucar.edu> wrote:
>>> > > >
>>> > > > > Hello Ting,
>>> > > > >
>>> > > > > We're trying to replicate the behavior you're seeing on
theia
>>> when
>>> > > > running
>>> > > > > Grid-Stat. However, we are not able to reproduce it.
>>> > > > >
>>> > > > > Please take a look in /home/John.H.Gotway/MET_Help/l
>>> ei_data_20161026
>>> > > > >
>>> > > > > I've copied over your Grid-Stat config file and am
pointing to
>>> the
>>> > > > fcst/obs
>>> > > > > data in your test directory:
>>> > > > >
>>> > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>>> > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_re
>>> fc_17
>>> > > > >
>>> > > > > When I run Grid-Stat on this data and compare to your
output,
>>> > there's a
>>> > > > big
>>> > > > > difference in the number of matched pairs. I get 673085
pairs
>>> (i.e.
>>> > > the
>>> > > > > TOTAL column of the CTC output file) while you get 229861.
>>> > > > >
>>> > > > > This difference makes me very suspicious of how you're
>>> regridding the
>>> > > > > data. You indicated that you're using:
>>> > > > >
>>> > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270
6371.2
>>> 25.000
>>> > > > > 25.000"'
>>> > > > >
>>> > > > > Do you have any idea why my matched pair count is so
different
>>> from
>>> > > yours
>>> > > > > when I'm pointing to exactly the same input files?
>>> > > > >
>>> > > > > Also, I'd recommend that you turn ON the Grid-Stat NetCDF
matched
>>> > pairs
>>> > > > > output file, by setting the following in your config file:
>>> > > > >
>>> > > > > nc_pairs_flag = TRUE;
>>> > > > >
>>> > > > > Then we'll be able to look at the gridded data that Grid-
Stat is
>>> > > > processing
>>> > > > > and hopefully the problem will become obvious.
>>> > > > >
>>> > > > > Thanks,
>>> > > > > John Halley Gotway
>>> > > > >
>>> > > > > On Fri, Oct 21, 2016 at 8:45 AM, Ting Lei - NOAA Affiliate
via
>>> RT <
>>> > > > > met_help at ucar.edu> wrote:
>>> > > > >
>>> > > > > >
>>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78350
>>> >
>>> > > > > >
>>> > > > > > Julie,
>>> > > > > > Great. So many thanks.
>>> > > > > > The actual directory is in
>>> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>>> > > > > >
pertcld1/namrr_metradar_hi_imp_f17.tm00_06/gridstat_refc_17,
>>> > > > > > however I put all files in
>>> > > > > > /scratch4/NCEPDEV/stmp4/Ting.Lei/tmpnwprd/rt5pint_tten-
>>> > > > > > pertcld1/namrr_metradar_hi_imp_f17.tm00_06/debug-
gridstat_re
>>> fc_17.
>>> > > > > >
>>> > > > > > The command is like:
>>> > > > > > grid_stat namrr.t06z.hi_impnest.hiresf17.tm00.grib2
>>> > > > > > gridstat_refc_17/refd3d.t23z.grb2f00
>>> parm/GridStatConfig_hi_imp_
>>> > > > 01h_REFC
>>> > > > > > -outdir . -v 3.
>>> > > > > > (Plz be cautioned, I did it through the queue system on
theia
>>> and I
>>> > > > > didn't
>>> > > > > > do it in the front end ,
>>> > > > > > I tried and failed to load the correct module for MET
5.1).
>>> > > > > > For the RGD used in the configure file, it was defined
as:
>>> > > > > > ^^
>>> > > > > > RGD='"lambert 959 867 25.819 256.193 265.000 1.270
6371.2
>>> 25.000
>>> > > > 25.000"'
>>> > > > > > VV
>>> > > > > > I have the warning message:
>>> > > > > > ^^
>>> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
>>> > > (parm_name
>>> > > > =
>>> > > > > > REFC):
>>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>>> = 5
>>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>>> =
>>> > 196
>>> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b
= 16,
>>> > index_c
>>> > > > = 5
>>> > > > > > WARNING:
>>> > > > > > WARNING:
>>> > > > > > WARNING: Multiple GRIB2 table entries match lookup
criteria
>>> > > (parm_name
>>> > > > =
>>> > > > > > REFC):
>>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>>> = 5
>>> > > > > > WARNING: parm_name: REFC, index_a = 0, index_b = 16,
index_c
>>> =
>>> > 196
>>> > > > > > WARNING: Using: parm_name: REFC, index_a = 0, index_b
= 16,
>>> > index_c
>>> > > > = 5
>>> > > > > > WARNING:
>>> > > > > >
>>> > > > > >
>>> > > > > > VV
>>> > > > > >
>>> > > > > >
>>> > > > > > The problem is in the
>>> > > > > > file
grid_stat_namrr_hi_impnest_F17_01h_REFC_NBR_170000L_
>>> > > > > > 20151030_230000V.stat,
>>> > > > > > for ctc , you can see the problem I reported (no
observed refc
>>> was
>>> > > > > counted.
>>> > > > > > The current run was with a new configure, while the
parameter
>>> name
>>> > is
>>> > > > > REFC
>>> > > > > > (composite reflectivity ) rather than original REFCclm.
I
>>> think
>>> > this
>>> > > > is
>>> > > > > > because a new grib2 table being used.
>>> > > > > > In the observation file, by wgrib2 or g2ctl.pl, it could
be
>>> shown
>>> > > the
>>> > > > > refc
>>> > > > > > showed an normal appearance of a storm as as expected.
So,
>>> seems
>>> > > > > something
>>> > > > > > wrong that they were not counted in the verification.
>>> > > > > > It is very likely I missed something.
>>> > > > > >
>>> > > > > > In case you need to know more, here is what I actually
did for
>>> this
>>> > > > job.
>>> > > > > > In
/scratch4/NCEPDEV/meso/save/Ting.Lei/hybssda_namrr/tlei_
>>> > > > > > NDAS_exp/NDAS_exp/trunk/namrr,
>>> > > > > >
>>> > > > > > run dmet-rt5pint_tten-pertcld1.sh to invoke the rocoto.
>>> > > > > >
>>> > > > > >
>>> > > > > > Your help is appreciated.
>>> > > > > > Ting
>>> > > > > > BTW: I had used "chmod a+r" to process directories I
mentioned
>>> > > above.
>>> > > > > > However, if still the problem, plz let me know and maybe
I can
>>> try
>>> > > the
>>> > > > > > original plan (send the tared data to you).
>>> > > > > >
>>> > > > > >
>>> > > > > > On Fri, Oct 21, 2016 at 10:34 AM, Julie Prestopnik via
RT <
>>> > > > > > met_help at ucar.edu
>>> > > > > > > wrote:
>>> > > > > >
>>> > > > > > > Hi Ting. I have access to Theia and may be able to
help
>>> there.
>>> > > > Could
>>> > > > > > you
>>> > > > > > > please point me to your files (data and config) and
send
>>> along
>>> > the
>>> > > > > > command
>>> > > > > > > you are running?
>>> > > > > > >
>>> > > > > > > Thanks,
>>> > > > > > > Julie
>>> > > > > > >
>>> > > > > > > On Fri, Oct 21, 2016 at 6:01 AM, Ting Lei - NOAA
Affiliate
>>> via
>>> > RT <
>>> > > > > > > met_help at ucar.edu> wrote:
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > Fri Oct 21 06:01:01 2016: Request 78350 was acted
upon.
>>> > > > > > > > Transaction: Ticket created by ting.lei at noaa.gov
>>> > > > > > > > Queue: met_help
>>> > > > > > > > Subject: Re: the "missed " samples for CTC
>>> calculation for
>>> > > the
>>> > > > > > > > composite reflectivity observation.
>>> > > > > > > > Owner: Nobody
>>> > > > > > > > Requestors: ting.lei at noaa.gov
>>> > > > > > > > Status: new
>>> > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
>>> > > > > Ticket/Display.html?id=78350
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Hi, Julie,
>>> > > > > > > > For the late reply.
>>> > > > > > > > I had been working on another issue in my work and
just
>>> fixed
>>> > anc
>>> > > > > come
>>> > > > > > > back
>>> > > > > > > > on this:).
>>> > > > > > > > Thank you so much for your offering for further
help.
>>> > > > > > > > Seems I can't directly transfer from Theia and I
need to
>>> first
>>> > > try
>>> > > > to
>>> > > > > > > > transfer to a local computer. Because I'm on tele-
work
>>> > today, I
>>> > > > > will
>>> > > > > > > send
>>> > > > > > > > them to you next week.
>>> > > > > > > > Sorry if that is maybe not as you planned.
>>> > > > > > > > Have a good weekend,
>>> > > > > > > > Ting
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On Wed, Oct 19, 2016 at 12:58 PM, Ting Lei - NOAA
>>> Affiliate <
>>> > > > > > > > ting.lei at noaa.gov> wrote:
>>> > > > > > > >
>>> > > > > > > > > Dear Mr. or Mrs,
>>> > > > > > > > > I have a problem using Met _grid state to obtain
the CTC
>>> for
>>> > > the
>>> > > > > > > > composite
>>> > > > > > > > > reflectivity.
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > From the output of met_grid_stat, the numbers for
dbz
>>> > > observation
>>> > > > > >20
>>> > > > > > > > are
>>> > > > > > > > > always 0 (namely, numbers for ify_Oy and ifn_OY
are
>>> always
>>> > 0).
>>> > > > By
>>> > > > > > > wgrib2
>>> > > > > > > > > seeing the observation file, I know there are
maximum
>>> refc
>>> > > values
>>> > > > > > > larger
>>> > > > > > > > > than 60dbz. so, there should be other problems
with my
>>> use
>>> > of
>>> > > > the
>>> > > > > > > > > metradar tool.
>>> > > > > > > > > In the Met running, there were warning message:
>>> > > > > > > > > ^^
>>> > > > > > > > > WARNING:
>>> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
>>> criteria
>>> > > > > > (parm_name
>>> > > > > > > =
>>> > > > > > > > > REFC):
>>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>>> > index_c
>>> > > =
>>> > > > 5
>>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>>> > index_c
>>> > > =
>>> > > > > 196
>>> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b
>>> = 16,
>>> > > > > index_c
>>> > > > > > > = 5
>>> > > > > > > > > WARNING:
>>> > > > > > > > > WARNING:
>>> > > > > > > > > WARNING: Multiple GRIB2 table entries match lookup
>>> criteria
>>> > > > > > (parm_name
>>> > > > > > > =
>>> > > > > > > > > REFC):
>>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>>> > index_c
>>> > > =
>>> > > > 5
>>> > > > > > > > > WARNING: parm_name: REFC, index_a = 0, index_b =
16,
>>> > index_c
>>> > > =
>>> > > > > 196
>>> > > > > > > > > WARNING: Using: parm_name: REFC, index_a = 0,
index_b
>>> = 16,
>>> > > > > index_c
>>> > > > > > > = 5
>>> > > > > > > > >
>>> > > > > > > > > VV
>>> > > > > > > > >
>>> > > > > > > > > The MET in the NAMRR is first implemented by Jacob
>>> Charley.
>>> > He
>>> > > > has
>>> > > > > > been
>>> > > > > > > > > using it smoothly, while some quantities like CTC
>>> causing the
>>> > > > > problem
>>> > > > > > > > here
>>> > > > > > > > > were not calculated by him. He pointed out the
>>> differences
>>> > > > > between
>>> > > > > > > his
>>> > > > > > > > > and mine. I also attached here.
>>> > > > > > > > > ^^
>>> > > > > > > > > I'm using :
>>> > > > > > > > >
>>> > > > > > > > > //
>>> > > > > > > > > // Statistical output types
>>> > > > > > > > > //
>>> > > > > > > > > output_flag = {
>>> > > > > > > > > fho = BOTH;
>>> > > > > > > > > ctc = BOTH;
>>> > > > > > > > > cts = BOTH;
>>> > > > > > > > > mctc = BOTH;
>>> > > > > > > > > mcts = BOTH;
>>> > > > > > > > > cnt = BOTH;
>>> > > > > > > > > sl1l2 = BOTH;
>>> > > > > > > > > vl1l2 = BOTH;
>>> > > > > > > > > pct = BOTH;
>>> > > > > > > > > pstd = BOTH;
>>> > > > > > > > > pjc = BOTH;
>>> > > > > > > > > prc = BOTH;
>>> > > > > > > > > nbrctc = BOTH;
>>> > > > > > > > > nbrcts = BOTH;
>>> > > > > > > > > nbrcnt = BOTH;
>>> > > > > > > > > };
>>> > > > > > > > >
>>> > > > > > > > > where Jacob used, no problem:
>>> > > > > > > > >
>>> > > > > > > > > //
>>> > > > > > > > > // Statistical output types
>>> > > > > > > > > //
>>> > > > > > > > > output_flag = {
>>> > > > > > > > > fho = NONE;
>>> > > > > > > > > ctc = NONE;
>>> > > > > > > > > cts = NONE;
>>> > > > > > > > > mctc = NONE;
>>> > > > > > > > > mcts = NONE;
>>> > > > > > > > > cnt = NONE;
>>> > > > > > > > > sl1l2 = NONE;
>>> > > > > > > > > vl1l2 = NONE;
>>> > > > > > > > > pct = NONE;
>>> > > > > > > > > pstd = NONE;
>>> > > > > > > > > pjc = NONE;
>>> > > > > > > > > prc = NONE;
>>> > > > > > > > > nbrctc = NONE;
>>> > > > > > > > > nbrcts = NONE;
>>> > > > > > > > > nbrcnt = STAT;
>>> > > > > > > > > };
>>> > > > > > > > >
>>> > > > > > > > > VV
>>> > > > > > > > > Your help is appreciated.
>>> > > > > > > > > Best Regards,
>>> > > > > > > > > Ting
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > --
>>> > > > > > > Julie Prestopnik
>>> > > > > > > National Center for Atmospheric Research
>>> > > > > > > Research Applications Laboratory
>>> > > > > > > Phone: 303.497.8399
>>> > > > > > > Email: jpresto at ucar.edu
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> >
>>> >
>>>
>>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list