[Met_help] [rt.rap.ucar.edu #86560] History for unresonable results of CTS and PRC from stat_ananlysis

John Halley Gotway via RT met_help at ucar.edu
Tue Aug 14 13:31:33 MDT 2018


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Zhang, Mandy has shared OneDrive for Business files with you. To view them, click the links below.


<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A>
[https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A>

PRC_A1.txt<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A>

<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg>
[https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg>

CTSAPCP_A1.txt<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg>

<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ>
[https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ>

grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ>

<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw>
[https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw>

grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw>




Hi All,

     I use stat_analysis to get CTS and PRC,

********* stat_analysis of CTS

 -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd NEAREST -line_type CTC -dump_row /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat -out_line_type CTS -out_alpha 0.05000

**************************

but from result of CTS, it seems like missing the thresolds of GSS .

and

*** stat_analysis of PRC

 -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT -by FCST_VAR -dump_row /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat -out_line_type PRC
*******
the results of PRC looks weird, total number is negative.
Do you know what's the issue?
I also attach the single run results of both CTS and PRC for reference, which are resonable.
Thank you!
Mandy


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: unresonable results of CTS and PRC from stat_ananlysis
From: John Halley Gotway
Time: Fri Aug 10 17:18:58 2018

Mandy,

Let me address your first question about stat_analysis.  I think I see
the
source of the confusion.

You sent me 4 files:
- grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
- grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
- CTSAPCP_A1.txt
- PRC_A1.txt

The grid_stat...cts.txt file contains contingency table statistics
which
include the GSS column.  You get separate statistics for each
combination
of masking region and threshold, as seen here:
> cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk '{print
$10,$11,$15,$18,$67}'
FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
APCP A1 FULL >=0.254 0.43707
APCP A1 FULL >=2.54 0.39237
APCP A1 FULL >=6.35 0.26286
APCP A1 FULL >=12.7 0.20748
APCP A1 FULL >=25.4 0.087387
APCP A1 MASK >=0.254 0.48104
APCP A1 MASK >=2.54 0.35857
APCP A1 MASK >=6.35 0.24307
APCP A1 MASK >=12.7 0.18594
APCP A1 MASK >=25.4 0.15918

If you run a STAT-Analysis job and tell it to aggregate together all
the
CTC lines together, it'll do exactly that... grouping together
contingency
tables across multiple masking regions (FULL and MASK) and multiple
thresholds (the 5 you listed).  That's probably *NOT* what you
intended.

Note that you're using met-6.0.  If you were running met-7.0, that job
would have resulted in the following debug and warning messages:
DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
>=0.254,>=2.54,>=6.35,>=12.7,>=25.4
WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
>=0.254,>=2.54,>=6.35,>=12.7,>=25.4

We decided that combining results across multiple thresholds should
result
in a warning!

Please try rerunning, your job but add this option "-by
VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
for each unique combination of VX_MASK and FCST_THRESH columns it
encounters.  Does that make sense?

As for the odd numbers in the PRC_A1.txt file, I'd really need to see
the
input you passed to STAT-Analysis to debug.  Can you please send me
this
file:
   grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
or even simpler:
   grid_stat_ensmr01_010000L_20180515_060000V.stat

Thanks,
John




On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: unresonable results of CTS and PRC from stat_ananlysis
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
>
> PRC_A1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
>
> CTSAPCP_A1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
>
>
>
>
> Hi All,
>
>      I use stat_analysis to get CTS and PRC,
>
> ********* stat_analysis of CTS
>
>  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> -line_type CTC -dump_row
> /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
-out_line_type
> CTS -out_alpha 0.05000
>
> **************************
>
> but from result of CTS, it seems like missing the thresolds of GSS .
>
> and
>
> *** stat_analysis of PRC
>
>  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> FCST_VAR -dump_row
> /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type PRC
> *******
> the results of PRC looks weird, total number is negative.
> Do you know what's the issue?
> I also attach the single run results of both CTS and PRC for
reference,
> which are resonable.
> Thank you!
> Mandy
>
>

------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Mon Aug 13 09:25:19 2018

Zhang, Mandy has shared a OneDrive for Business file with you. To view
it, click the link below.


<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ>
[https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<https://sooners-
my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ>

grid_stat_ensmr01_010000L_20180515_060000V.stat<https://sooners-
my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ>




Hi John,

    Please find the attachment of stat file.

Thank you for your suggestion, I'll try it.

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Friday, August 10, 2018 6:18:59 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

Let me address your first question about stat_analysis.  I think I see
the
source of the confusion.

You sent me 4 files:
- grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
- grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
- CTSAPCP_A1.txt
- PRC_A1.txt

The grid_stat...cts.txt file contains contingency table statistics
which
include the GSS column.  You get separate statistics for each
combination
of masking region and threshold, as seen here:
> cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk '{print
$10,$11,$15,$18,$67}'
FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
APCP A1 FULL >=0.254 0.43707
APCP A1 FULL >=2.54 0.39237
APCP A1 FULL >=6.35 0.26286
APCP A1 FULL >=12.7 0.20748
APCP A1 FULL >=25.4 0.087387
APCP A1 MASK >=0.254 0.48104
APCP A1 MASK >=2.54 0.35857
APCP A1 MASK >=6.35 0.24307
APCP A1 MASK >=12.7 0.18594
APCP A1 MASK >=25.4 0.15918

If you run a STAT-Analysis job and tell it to aggregate together all
the
CTC lines together, it'll do exactly that... grouping together
contingency
tables across multiple masking regions (FULL and MASK) and multiple
thresholds (the 5 you listed).  That's probably *NOT* what you
intended.

Note that you're using met-6.0.  If you were running met-7.0, that job
would have resulted in the following debug and warning messages:
DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
>=0.254,>=2.54,>=6.35,>=12.7,>=25.4
WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
>=0.254,>=2.54,>=6.35,>=12.7,>=25.4

We decided that combining results across multiple thresholds should
result
in a warning!

Please try rerunning, your job but add this option "-by
VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
for each unique combination of VX_MASK and FCST_THRESH columns it
encounters.  Does that make sense?

As for the odd numbers in the PRC_A1.txt file, I'd really need to see
the
input you passed to STAT-Analysis to debug.  Can you please send me
this
file:
   grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
or even simpler:
   grid_stat_ensmr01_010000L_20180515_060000V.stat

Thanks,
John




On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: unresonable results of CTS and PRC from stat_ananlysis
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
>
> PRC_A1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
>
> CTSAPCP_A1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> >
>
>
>
>
> Hi All,
>
>      I use stat_analysis to get CTS and PRC,
>
> ********* stat_analysis of CTS
>
>  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> -line_type CTC -dump_row
> /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
-out_line_type
> CTS -out_alpha 0.05000
>
> **************************
>
> but from result of CTS, it seems like missing the thresolds of GSS .
>
> and
>
> *** stat_analysis of PRC
>
>  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> FCST_VAR -dump_row
> /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type PRC
> *******
> the results of PRC looks weird, total number is negative.
> Do you know what's the issue?
> I also attach the single run results of both CTS and PRC for
reference,
> which are resonable.
> Thank you!
> Mandy
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: John Halley Gotway
Time: Mon Aug 13 09:32:25 2018

Mandy,

Great, thanks for sending that .stat file.  I ran the following
command but
am not able to replicate the behavior you describe... getting negative
numbers in the TOTAL column:

 /usr/local/met-6.1/bin/stat_analysis \
-lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
FCST_VAR -out_line_type PRC \
-out agg_stat_PCT_to_PRC.out

I've attached the output file I got.

If you are still seeing those odd PRC output lines.  Please send me
the
exact command you used to generate them.

Thanks,
John


On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V.stat<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
>
>
>
> Hi John,
>
>     Please find the attachment of stat file.
>
> Thank you for your suggestion, I'll try it.
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, August 10, 2018 6:18:59 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Let me address your first question about stat_analysis.  I think I
see the
> source of the confusion.
>
> You sent me 4 files:
> - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> - CTSAPCP_A1.txt
> - PRC_A1.txt
>
> The grid_stat...cts.txt file contains contingency table statistics
which
> include the GSS column.  You get separate statistics for each
combination
> of masking region and threshold, as seen here:
> > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> $10,$11,$15,$18,$67}'
> FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> APCP A1 FULL >=0.254 0.43707
> APCP A1 FULL >=2.54 0.39237
> APCP A1 FULL >=6.35 0.26286
> APCP A1 FULL >=12.7 0.20748
> APCP A1 FULL >=25.4 0.087387
> APCP A1 MASK >=0.254 0.48104
> APCP A1 MASK >=2.54 0.35857
> APCP A1 MASK >=6.35 0.24307
> APCP A1 MASK >=12.7 0.18594
> APCP A1 MASK >=25.4 0.15918
>
> If you run a STAT-Analysis job and tell it to aggregate together all
the
> CTC lines together, it'll do exactly that... grouping together
contingency
> tables across multiple masking regions (FULL and MASK) and multiple
> thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
>
> Note that you're using met-6.0.  If you were running met-7.0, that
job
> would have resulted in the following debug and warning messages:
> DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
> WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
>
> We decided that combining results across multiple thresholds should
result
> in a warning!
>
> Please try rerunning, your job but add this option "-by
> VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> for each unique combination of VX_MASK and FCST_THRESH columns it
> encounters.  Does that make sense?
>
> As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> input you passed to STAT-Analysis to debug.  Can you please send me
this
> file:
>    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> or even simpler:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> Thanks,
> John
>
>
>
>
> On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > PRC_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > CTSAPCP_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> >
> >
> >
> > Hi All,
> >
> >      I use stat_analysis to get CTS and PRC,
> >
> > ********* stat_analysis of CTS
> >
> >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > -line_type CTC -dump_row
> > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> -out_line_type
> > CTS -out_alpha 0.05000
> >
> > **************************
> >
> > but from result of CTS, it seems like missing the thresolds of GSS
.
> >
> > and
> >
> > *** stat_analysis of PRC
> >
> >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -dump_row
> > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type
> PRC
> > *******
> > the results of PRC looks weird, total number is negative.
> > Do you know what's the issue?
> > I also attach the single run results of both CTS and PRC for
reference,
> > which are resonable.
> > Thank you!
> > Mandy
> >
> >
>
>
>

------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Mon Aug 13 10:08:53 2018

Hi John,

     Now I'm try the job_command with adding thresh,

##
#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
####################################

the difference from my previous job command is "-fcst_thresh ge0.254",
I'll let you know once I get result.

Thank you

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 13, 2018 10:32:25 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

Great, thanks for sending that .stat file.  I ran the following
command but
am not able to replicate the behavior you describe... getting negative
numbers in the TOTAL column:

 /usr/local/met-6.1/bin/stat_analysis \
-lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
FCST_VAR -out_line_type PRC \
-out agg_stat_PCT_to_PRC.out

I've attached the output file I got.

If you are still seeing those odd PRC output lines.  Please send me
the
exact command you used to generate them.

Thanks,
John


On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V.stat<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
>
>
>
> Hi John,
>
>     Please find the attachment of stat file.
>
> Thank you for your suggestion, I'll try it.
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, August 10, 2018 6:18:59 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Let me address your first question about stat_analysis.  I think I
see the
> source of the confusion.
>
> You sent me 4 files:
> - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> - CTSAPCP_A1.txt
> - PRC_A1.txt
>
> The grid_stat...cts.txt file contains contingency table statistics
which
> include the GSS column.  You get separate statistics for each
combination
> of masking region and threshold, as seen here:
> > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> $10,$11,$15,$18,$67}'
> FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> APCP A1 FULL >=0.254 0.43707
> APCP A1 FULL >=2.54 0.39237
> APCP A1 FULL >=6.35 0.26286
> APCP A1 FULL >=12.7 0.20748
> APCP A1 FULL >=25.4 0.087387
> APCP A1 MASK >=0.254 0.48104
> APCP A1 MASK >=2.54 0.35857
> APCP A1 MASK >=6.35 0.24307
> APCP A1 MASK >=12.7 0.18594
> APCP A1 MASK >=25.4 0.15918
>
> If you run a STAT-Analysis job and tell it to aggregate together all
the
> CTC lines together, it'll do exactly that... grouping together
contingency
> tables across multiple masking regions (FULL and MASK) and multiple
> thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
>
> Note that you're using met-6.0.  If you were running met-7.0, that
job
> would have resulted in the following debug and warning messages:
> DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
> WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
>
> We decided that combining results across multiple thresholds should
result
> in a warning!
>
> Please try rerunning, your job but add this option "-by
> VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> for each unique combination of VX_MASK and FCST_THRESH columns it
> encounters.  Does that make sense?
>
> As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> input you passed to STAT-Analysis to debug.  Can you please send me
this
> file:
>    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> or even simpler:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> Thanks,
> John
>
>
>
>
> On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > PRC_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > CTSAPCP_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> >
> >
> >
> > Hi All,
> >
> >      I use stat_analysis to get CTS and PRC,
> >
> > ********* stat_analysis of CTS
> >
> >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > -line_type CTC -dump_row
> > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> -out_line_type
> > CTS -out_alpha 0.05000
> >
> > **************************
> >
> > but from result of CTS, it seems like missing the thresolds of GSS
.
> >
> > and
> >
> > *** stat_analysis of PRC
> >
> >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -dump_row
> > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type
> PRC
> > *******
> > the results of PRC looks weird, total number is negative.
> > Do you know what's the issue?
> > I also attach the single run results of both CTS and PRC for
reference,
> > which are resonable.
> > Thank you!
> > Mandy
> >
> >
>
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Mon Aug 13 10:29:58 2018

Hi John,

     the error information of command :

#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
***********************
is :
*** Running STAT-Analysis ***
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744073469325659
WARNING:
WARNING: do_job_aggr_stat() -> no matching STAT lines found for job:
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254 -line_type PCT -by FCST_VAR -out_line_type PRC
WARNING:

###################
Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
Thank you!
Mandy


________________________________
From: Zhang, Mandy
Sent: Monday, August 13, 2018 11:08:48 AM
To: met_help at ucar.edu
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis


Hi John,

     Now I'm try the job_command with adding thresh,

##
#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
####################################

the difference from my previous job command is "-fcst_thresh ge0.254",
I'll let you know once I get result.

Thank you

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 13, 2018 10:32:25 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

Great, thanks for sending that .stat file.  I ran the following
command but
am not able to replicate the behavior you describe... getting negative
numbers in the TOTAL column:

 /usr/local/met-6.1/bin/stat_analysis \
-lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
FCST_VAR -out_line_type PRC \
-out agg_stat_PCT_to_PRC.out

I've attached the output file I got.

If you are still seeing those odd PRC output lines.  Please send me
the
exact command you used to generate them.

Thanks,
John


On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V.stat<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
>
>
>
> Hi John,
>
>     Please find the attachment of stat file.
>
> Thank you for your suggestion, I'll try it.
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, August 10, 2018 6:18:59 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Let me address your first question about stat_analysis.  I think I
see the
> source of the confusion.
>
> You sent me 4 files:
> - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> - CTSAPCP_A1.txt
> - PRC_A1.txt
>
> The grid_stat...cts.txt file contains contingency table statistics
which
> include the GSS column.  You get separate statistics for each
combination
> of masking region and threshold, as seen here:
> > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> $10,$11,$15,$18,$67}'
> FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> APCP A1 FULL >=0.254 0.43707
> APCP A1 FULL >=2.54 0.39237
> APCP A1 FULL >=6.35 0.26286
> APCP A1 FULL >=12.7 0.20748
> APCP A1 FULL >=25.4 0.087387
> APCP A1 MASK >=0.254 0.48104
> APCP A1 MASK >=2.54 0.35857
> APCP A1 MASK >=6.35 0.24307
> APCP A1 MASK >=12.7 0.18594
> APCP A1 MASK >=25.4 0.15918
>
> If you run a STAT-Analysis job and tell it to aggregate together all
the
> CTC lines together, it'll do exactly that... grouping together
contingency
> tables across multiple masking regions (FULL and MASK) and multiple
> thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
>
> Note that you're using met-6.0.  If you were running met-7.0, that
job
> would have resulted in the following debug and warning messages:
> DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
> WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
>
> We decided that combining results across multiple thresholds should
result
> in a warning!
>
> Please try rerunning, your job but add this option "-by
> VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> for each unique combination of VX_MASK and FCST_THRESH columns it
> encounters.  Does that make sense?
>
> As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> input you passed to STAT-Analysis to debug.  Can you please send me
this
> file:
>    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> or even simpler:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> Thanks,
> John
>
>
>
>
> On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > PRC_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > CTSAPCP_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> >
> >
> >
> > Hi All,
> >
> >      I use stat_analysis to get CTS and PRC,
> >
> > ********* stat_analysis of CTS
> >
> >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > -line_type CTC -dump_row
> > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> -out_line_type
> > CTS -out_alpha 0.05000
> >
> > **************************
> >
> > but from result of CTS, it seems like missing the thresolds of GSS
.
> >
> > and
> >
> > *** stat_analysis of PRC
> >
> >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -dump_row
> > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type
> PRC
> > *******
> > the results of PRC looks weird, total number is negative.
> > Do you know what's the issue?
> > I also attach the single run results of both CTS and PRC for
reference,
> > which are resonable.
> > Thank you!
> > Mandy
> >
> >
>
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Mon Aug 13 10:46:31 2018

Zhang, Mandy has shared a OneDrive for Business file with you. To view
it, click the link below.


<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA>
[https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<https://sooners-
my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EdmAep-
diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA>

PRC_A1_thresh1.txt<https://sooners-
my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EdmAep-
diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA>




Hi John,

     The total number is still negative when I use job_command with
MET6.0

****

#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
******
    please find attachment for details.
     Thank you!
Mandy


________________________________
From: Zhang, Mandy
Sent: Monday, August 13, 2018 11:29:49 AM
To: met_help at ucar.edu
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis


Hi John,

     the error information of command :

#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
***********************
is :
*** Running STAT-Analysis ***
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744073469325659
WARNING:
WARNING: do_job_aggr_stat() -> no matching STAT lines found for job:
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254 -line_type PCT -by FCST_VAR -out_line_type PRC
WARNING:

###################
Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
Thank you!
Mandy


________________________________
From: Zhang, Mandy
Sent: Monday, August 13, 2018 11:08:48 AM
To: met_help at ucar.edu
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis


Hi John,

     Now I'm try the job_command with adding thresh,

##
#!/bin/sh
echo
echo "*** Running STAT-Analysis ***"
/home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -v 2
####################################

the difference from my previous job command is "-fcst_thresh ge0.254",
I'll let you know once I get result.

Thank you

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 13, 2018 10:32:25 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

Great, thanks for sending that .stat file.  I ran the following
command but
am not able to replicate the behavior you describe... getting negative
numbers in the TOTAL column:

 /usr/local/met-6.1/bin/stat_analysis \
-lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
-job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
FCST_VAR -out_line_type PRC \
-out agg_stat_PCT_to_PRC.out

I've attached the output file I got.

If you are still seeing those odd PRC output lines.  Please send me
the
exact command you used to generate them.

Thanks,
John


On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
> grid_stat_ensmr01_010000L_20180515_060000V.stat<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> >
>
>
>
>
> Hi John,
>
>     Please find the attachment of stat file.
>
> Thank you for your suggestion, I'll try it.
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, August 10, 2018 6:18:59 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Let me address your first question about stat_analysis.  I think I
see the
> source of the confusion.
>
> You sent me 4 files:
> - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> - CTSAPCP_A1.txt
> - PRC_A1.txt
>
> The grid_stat...cts.txt file contains contingency table statistics
which
> include the GSS column.  You get separate statistics for each
combination
> of masking region and threshold, as seen here:
> > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> $10,$11,$15,$18,$67}'
> FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> APCP A1 FULL >=0.254 0.43707
> APCP A1 FULL >=2.54 0.39237
> APCP A1 FULL >=6.35 0.26286
> APCP A1 FULL >=12.7 0.20748
> APCP A1 FULL >=25.4 0.087387
> APCP A1 MASK >=0.254 0.48104
> APCP A1 MASK >=2.54 0.35857
> APCP A1 MASK >=6.35 0.24307
> APCP A1 MASK >=12.7 0.18594
> APCP A1 MASK >=25.4 0.15918
>
> If you run a STAT-Analysis job and tell it to aggregate together all
the
> CTC lines together, it'll do exactly that... grouping together
contingency
> tables across multiple masking regions (FULL and MASK) and multiple
> thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
>
> Note that you're using met-6.0.  If you were running met-7.0, that
job
> would have resulted in the following debug and warning messages:
> DEBUG 2: For case "(nul)", found 2 unique VX_MASK values: FULL,MASK
> WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
>
> We decided that combining results across multiple thresholds should
result
> in a warning!
>
> Please try rerunning, your job but add this option "-by
> VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> for each unique combination of VX_MASK and FCST_THRESH columns it
> encounters.  Does that make sense?
>
> As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> input you passed to STAT-Analysis to debug.  Can you please send me
this
> file:
>    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> or even simpler:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> Thanks,
> John
>
>
>
>
> On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > PRC_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > CTSAPCP_A1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > >
> >
> >
> >
> >
> > Hi All,
> >
> >      I use stat_analysis to get CTS and PRC,
> >
> > ********* stat_analysis of CTS
> >
> >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > -line_type CTC -dump_row
> > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> -out_line_type
> > CTS -out_alpha 0.05000
> >
> > **************************
> >
> > but from result of CTS, it seems like missing the thresolds of GSS
.
> >
> > and
> >
> > *** stat_analysis of PRC
> >
> >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -dump_row
> > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
-out_line_type
> PRC
> > *******
> > the results of PRC looks weird, total number is negative.
> > Do you know what's the issue?
> > I also attach the single run results of both CTS and PRC for
reference,
> > which are resonable.
> > Thank you!
> > Mandy
> >
> >
>
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: John Halley Gotway
Time: Mon Aug 13 11:17:42 2018

Mandy,

I see the result you've sent and see that it has a large negative
value in
the TOTAL column, but I'm not able to replicate that same behavior
using
the .stat file you sent to me:
   grid_stat_ensmr01_010000L_20180515_060000V.stat

One option would be having you use the "-dump_row" option in
STAT-Analysis.  That tells STAT-Analysis to write out any of the input
lines that were used for the job to an output file:

   /home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -dump_row
/cnproj1/mzhang/met_test/met/2018/PCT_to_PRC_A1_thresh1_dump.txt
\
   -v 2

If you send that file "PCT_to_PRC_A1_thresh1_dump.txt" to me, I should
be
able to rerun your job and get the same bad result.  And then I'd have
a
better idea as to what's causing the problem.

Thanks,
John

On Mon, Aug 13, 2018 at 10:46 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
>
> PRC_A1_thresh1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
>
>
>
>
> Hi John,
>
>      The total number is still negative when I use job_command with
MET6.0
>
> ****
>
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ******
>     please find attachment for details.
>      Thank you!
> Mandy
>
>
> ________________________________
> From: Zhang, Mandy
> Sent: Monday, August 13, 2018 11:29:49 AM
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
>
> Hi John,
>
>      the error information of command :
>
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ***********************
> is :
> *** Running STAT-Analysis ***
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744073469325659
> WARNING:
> WARNING: do_job_aggr_stat() -> no matching STAT lines found for job:
-job
> aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254
> -line_type PCT -by FCST_VAR -out_line_type PRC
> WARNING:
>
> ###################
> Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
> Thank you!
> Mandy
>
>
> ________________________________
> From: Zhang, Mandy
> Sent: Monday, August 13, 2018 11:08:48 AM
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
>
> Hi John,
>
>      Now I'm try the job_command with adding thresh,
>
> ##
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ####################################
>
> the difference from my previous job command is "-fcst_thresh
ge0.254",
> I'll let you know once I get result.
>
> Thank you
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, August 13, 2018 10:32:25 AM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Great, thanks for sending that .stat file.  I ran the following
command but
> am not able to replicate the behavior you describe... getting
negative
> numbers in the TOTAL column:
>
>  /usr/local/met-6.1/bin/stat_analysis \
> -lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
> -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
> FCST_VAR -out_line_type PRC \
> -out agg_stat_PCT_to_PRC.out
>
> I've attached the output file I got.
>
> If you are still seeing those odd PRC output lines.  Please send me
the
> exact command you used to generate them.
>
> Thanks,
> John
>
>
> On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> > Zhang, Mandy has shared a OneDrive for Business file with you. To
view
> it,
> > click the link below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V.stat<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> >
> >
> >
> >
> > Hi John,
> >
> >     Please find the attachment of stat file.
> >
> > Thank you for your suggestion, I'll try it.
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Friday, August 10, 2018 6:18:59 PM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> > Mandy,
> >
> > Let me address your first question about stat_analysis.  I think I
see
> the
> > source of the confusion.
> >
> > You sent me 4 files:
> > - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> > - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> > - CTSAPCP_A1.txt
> > - PRC_A1.txt
> >
> > The grid_stat...cts.txt file contains contingency table statistics
which
> > include the GSS column.  You get separate statistics for each
combination
> > of masking region and threshold, as seen here:
> > > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> > $10,$11,$15,$18,$67}'
> > FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> > APCP A1 FULL >=0.254 0.43707
> > APCP A1 FULL >=2.54 0.39237
> > APCP A1 FULL >=6.35 0.26286
> > APCP A1 FULL >=12.7 0.20748
> > APCP A1 FULL >=25.4 0.087387
> > APCP A1 MASK >=0.254 0.48104
> > APCP A1 MASK >=2.54 0.35857
> > APCP A1 MASK >=6.35 0.24307
> > APCP A1 MASK >=12.7 0.18594
> > APCP A1 MASK >=25.4 0.15918
> >
> > If you run a STAT-Analysis job and tell it to aggregate together
all the
> > CTC lines together, it'll do exactly that... grouping together
> contingency
> > tables across multiple masking regions (FULL and MASK) and
multiple
> > thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
> >
> > Note that you're using met-6.0.  If you were running met-7.0, that
job
> > would have resulted in the following debug and warning messages:
> > DEBUG 2: For case "(nul)", found 2 unique VX_MASK values:
FULL,MASK
> > WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> >
> > We decided that combining results across multiple thresholds
should
> result
> > in a warning!
> >
> > Please try rerunning, your job but add this option "-by
> > VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> > for each unique combination of VX_MASK and FCST_THRESH columns it
> > encounters.  Does that make sense?
> >
> > As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> > input you passed to STAT-Analysis to debug.  Can you please send
me this
> > file:
> >    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> > or even simpler:
> >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> >
> > Thanks,
> > John
> >
> >
> >
> >
> > On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > >        Queue: met_help
> > >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> > >        Owner: Nobody
> > >   Requestors: Man.Zhang-1 at ou.edu
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
> >
> > >
> > >
> > > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > > them, click the links below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > >
> > > PRC_A1.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > >
> > > CTSAPCP_A1.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > >
> > >
> > >
> > >
> > > Hi All,
> > >
> > >      I use stat_analysis to get CTS and PRC,
> > >
> > > ********* stat_analysis of CTS
> > >
> > >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > > -line_type CTC -dump_row
> > > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> > -out_line_type
> > > CTS -out_alpha 0.05000
> > >
> > > **************************
> > >
> > > but from result of CTS, it seems like missing the thresolds of
GSS .
> > >
> > > and
> > >
> > > *** stat_analysis of PRC
> > >
> > >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST
-line_type PCT
> -by
> > > FCST_VAR -dump_row
> > > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
> -out_line_type
> > PRC
> > > *******
> > > the results of PRC looks weird, total number is negative.
> > > Do you know what's the issue?
> > > I also attach the single run results of both CTS and PRC for
reference,
> > > which are resonable.
> > > Thank you!
> > > Mandy
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Mon Aug 13 12:51:43 2018

Zhang, Mandy has shared a OneDrive for Business file with you. To view
it, click the link below.


<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ>
[https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<https://sooners-
my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ>

PCT_to_PRC_A1_thresh1_dump.txt<https://sooners-
my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ>




Hi John,

     I attached the PCT_to_PRC file, please find it!

Thank you!

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 13, 2018 12:17:42 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

I see the result you've sent and see that it has a large negative
value in
the TOTAL column, but I'm not able to replicate that same behavior
using
the .stat file you sent to me:
   grid_stat_ensmr01_010000L_20180515_060000V.stat

One option would be having you use the "-dump_row" option in
STAT-Analysis.  That tells STAT-Analysis to write out any of the input
lines that were used for the job to an output file:

   /home/mzhang/tool/met/bin/stat_analysis \
   -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
   -job aggregate_stat -line_type PCT -out_line_type PRC -by FCST_VAR
-fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
   -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
   -dump_row
/cnproj1/mzhang/met_test/met/2018/PCT_to_PRC_A1_thresh1_dump.txt
\
   -v 2

If you send that file "PCT_to_PRC_A1_thresh1_dump.txt" to me, I should
be
able to rerun your job and get the same bad result.  And then I'd have
a
better idea as to what's causing the problem.

Thanks,
John

On Mon, Aug 13, 2018 at 10:46 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
>
> PRC_A1_thresh1.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> >
>
>
>
>
> Hi John,
>
>      The total number is still negative when I use job_command with
MET6.0
>
> ****
>
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ******
>     please find attachment for details.
>      Thank you!
> Mandy
>
>
> ________________________________
> From: Zhang, Mandy
> Sent: Monday, August 13, 2018 11:29:49 AM
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
>
> Hi John,
>
>      the error information of command :
>
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ***********************
> is :
> *** Running STAT-Analysis ***
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744073469325659
> WARNING:
> WARNING: do_job_aggr_stat() -> no matching STAT lines found for job:
-job
> aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254
> -line_type PCT -by FCST_VAR -out_line_type PRC
> WARNING:
>
> ###################
> Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
> Thank you!
> Mandy
>
>
> ________________________________
> From: Zhang, Mandy
> Sent: Monday, August 13, 2018 11:08:48 AM
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
>
> Hi John,
>
>      Now I'm try the job_command with adding thresh,
>
> ##
> #!/bin/sh
> echo
> echo "*** Running STAT-Analysis ***"
> /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -v 2
> ####################################
>
> the difference from my previous job command is "-fcst_thresh
ge0.254",
> I'll let you know once I get result.
>
> Thank you
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, August 13, 2018 10:32:25 AM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Great, thanks for sending that .stat file.  I ran the following
command but
> am not able to replicate the behavior you describe... getting
negative
> numbers in the TOTAL column:
>
>  /usr/local/met-6.1/bin/stat_analysis \
> -lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
> -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type PCT
-by
> FCST_VAR -out_line_type PRC \
> -out agg_stat_PCT_to_PRC.out
>
> I've attached the output file I got.
>
> If you are still seeing those odd PRC output lines.  Please send me
the
> exact command you used to generate them.
>
> Thanks,
> John
>
>
> On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> > Zhang, Mandy has shared a OneDrive for Business file with you. To
view
> it,
> > click the link below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> >
> > grid_stat_ensmr01_010000L_20180515_060000V.stat<
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > >
> >
> >
> >
> >
> > Hi John,
> >
> >     Please find the attachment of stat file.
> >
> > Thank you for your suggestion, I'll try it.
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Friday, August 10, 2018 6:18:59 PM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> > Mandy,
> >
> > Let me address your first question about stat_analysis.  I think I
see
> the
> > source of the confusion.
> >
> > You sent me 4 files:
> > - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> > - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> > - CTSAPCP_A1.txt
> > - PRC_A1.txt
> >
> > The grid_stat...cts.txt file contains contingency table statistics
which
> > include the GSS column.  You get separate statistics for each
combination
> > of masking region and threshold, as seen here:
> > > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> > $10,$11,$15,$18,$67}'
> > FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> > APCP A1 FULL >=0.254 0.43707
> > APCP A1 FULL >=2.54 0.39237
> > APCP A1 FULL >=6.35 0.26286
> > APCP A1 FULL >=12.7 0.20748
> > APCP A1 FULL >=25.4 0.087387
> > APCP A1 MASK >=0.254 0.48104
> > APCP A1 MASK >=2.54 0.35857
> > APCP A1 MASK >=6.35 0.24307
> > APCP A1 MASK >=12.7 0.18594
> > APCP A1 MASK >=25.4 0.15918
> >
> > If you run a STAT-Analysis job and tell it to aggregate together
all the
> > CTC lines together, it'll do exactly that... grouping together
> contingency
> > tables across multiple masking regions (FULL and MASK) and
multiple
> > thresholds (the 5 you listed).  That's probably *NOT* what you
intended.
> >
> > Note that you're using met-6.0.  If you were running met-7.0, that
job
> > would have resulted in the following debug and warning messages:
> > DEBUG 2: For case "(nul)", found 2 unique VX_MASK values:
FULL,MASK
> > WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> >
> > We decided that combining results across multiple thresholds
should
> result
> > in a warning!
> >
> > Please try rerunning, your job but add this option "-by
> > VX_MASK,FCST_THRESH".  That'll run the same job but write separate
output
> > for each unique combination of VX_MASK and FCST_THRESH columns it
> > encounters.  Does that make sense?
> >
> > As for the odd numbers in the PRC_A1.txt file, I'd really need to
see the
> > input you passed to STAT-Analysis to debug.  Can you please send
me this
> > file:
> >    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> > or even simpler:
> >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> >
> > Thanks,
> > John
> >
> >
> >
> >
> > On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > >        Queue: met_help
> > >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> > >        Owner: Nobody
> > >   Requestors: Man.Zhang-1 at ou.edu
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
> >
> > >
> > >
> > > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > > them, click the links below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > >
> > > PRC_A1.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > >
> > > CTSAPCP_A1.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > >
> > >
> > >
> > >
> > >
> > > Hi All,
> > >
> > >      I use stat_analysis to get CTS and PRC,
> > >
> > > ********* stat_analysis of CTS
> > >
> > >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > > -line_type CTC -dump_row
> > > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> > -out_line_type
> > > CTS -out_alpha 0.05000
> > >
> > > **************************
> > >
> > > but from result of CTS, it seems like missing the thresolds of
GSS .
> > >
> > > and
> > >
> > > *** stat_analysis of PRC
> > >
> > >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST
-line_type PCT
> -by
> > > FCST_VAR -dump_row
> > > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
> -out_line_type
> > PRC
> > > *******
> > > the results of PRC looks weird, total number is negative.
> > > Do you know what's the issue?
> > > I also attach the single run results of both CTS and PRC for
reference,
> > > which are resonable.
> > > Thank you!
> > > Mandy
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: John Halley Gotway
Time: Mon Aug 13 15:35:29 2018

Mandy,

Ah, OK, I understand what's going on now.  Thanks for sending that
file.
The problem here is integer overflow.  You can see this by running the
following 2 jobs:

(1) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
-line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
-fcst_valid_end 20180524_000000

(2) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
-line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
-fcst_valid_end 20180525_000000

The first one uses 1177 input lines and the resulting TOTAL column is
2145234333.
The second one uses 1179 and the TOTAL columns is -2146087705.

When an integer exceeds a value of 2,147,483,647 it gets too big and
integer overflow occurs.  The same thing happens in the latest
development
version of the code as well.  To fix this, we'll need to change the
way
we're storing these values.

Thanks,
John



On Mon, Aug 13, 2018 at 12:52 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
>
> PCT_to_PRC_A1_thresh1_dump.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
>
>
>
>
> Hi John,
>
>      I attached the PCT_to_PRC file, please find it!
>
> Thank you!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, August 13, 2018 12:17:42 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> I see the result you've sent and see that it has a large negative
value in
> the TOTAL column, but I'm not able to replicate that same behavior
using
> the .stat file you sent to me:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> One option would be having you use the "-dump_row" option in
> STAT-Analysis.  That tells STAT-Analysis to write out any of the
input
> lines that were used for the job to an output file:
>
>    /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -dump_row
> /cnproj1/mzhang/met_test/met/2018/PCT_to_PRC_A1_thresh1_dump.txt
> \
>    -v 2
>
> If you send that file "PCT_to_PRC_A1_thresh1_dump.txt" to me, I
should be
> able to rerun your job and get the same bad result.  And then I'd
have a
> better idea as to what's causing the problem.
>
> Thanks,
> John
>
> On Mon, Aug 13, 2018 at 10:46 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> > Zhang, Mandy has shared a OneDrive for Business file with you. To
view
> it,
> > click the link below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> >
> > PRC_A1_thresh1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> >
> >
> >
> >
> > Hi John,
> >
> >      The total number is still negative when I use job_command
with
> MET6.0
> >
> > ****
> >
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ******
> >     please find attachment for details.
> >      Thank you!
> > Mandy
> >
> >
> > ________________________________
> > From: Zhang, Mandy
> > Sent: Monday, August 13, 2018 11:29:49 AM
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> >
> > Hi John,
> >
> >      the error information of command :
> >
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ***********************
> > is :
> > *** Running STAT-Analysis ***
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744073469325659
> > WARNING:
> > WARNING: do_job_aggr_stat() -> no matching STAT lines found for
job: -job
> > aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254
> > -line_type PCT -by FCST_VAR -out_line_type PRC
> > WARNING:
> >
> > ###################
> > Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
> > Thank you!
> > Mandy
> >
> >
> > ________________________________
> > From: Zhang, Mandy
> > Sent: Monday, August 13, 2018 11:08:48 AM
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> >
> > Hi John,
> >
> >      Now I'm try the job_command with adding thresh,
> >
> > ##
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ####################################
> >
> > the difference from my previous job command is "-fcst_thresh
ge0.254",
> > I'll let you know once I get result.
> >
> > Thank you
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, August 13, 2018 10:32:25 AM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> > Mandy,
> >
> > Great, thanks for sending that .stat file.  I ran the following
command
> but
> > am not able to replicate the behavior you describe... getting
negative
> > numbers in the TOTAL column:
> >
> >  /usr/local/met-6.1/bin/stat_analysis \
> > -lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
> > -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -out_line_type PRC \
> > -out agg_stat_PCT_to_PRC.out
> >
> > I've attached the output file I got.
> >
> > If you are still seeing those odd PRC output lines.  Please send
me the
> > exact command you used to generate them.
> >
> > Thanks,
> > John
> >
> >
> > On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> > >
> > > Zhang, Mandy has shared a OneDrive for Business file with you.
To view
> > it,
> > > click the link below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V.stat<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > >
> > >
> > >
> > >
> > > Hi John,
> > >
> > >     Please find the attachment of stat file.
> > >
> > > Thank you for your suggestion, I'll try it.
> > >
> > > Mandy
> > >
> > > ________________________________
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Friday, August 10, 2018 6:18:59 PM
> > > To: Zhang, Mandy
> > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and
> PRC
> > > from stat_ananlysis
> > >
> > > Mandy,
> > >
> > > Let me address your first question about stat_analysis.  I think
I see
> > the
> > > source of the confusion.
> > >
> > > You sent me 4 files:
> > > - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> > > - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> > > - CTSAPCP_A1.txt
> > > - PRC_A1.txt
> > >
> > > The grid_stat...cts.txt file contains contingency table
statistics
> which
> > > include the GSS column.  You get separate statistics for each
> combination
> > > of masking region and threshold, as seen here:
> > > > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> > > $10,$11,$15,$18,$67}'
> > > FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> > > APCP A1 FULL >=0.254 0.43707
> > > APCP A1 FULL >=2.54 0.39237
> > > APCP A1 FULL >=6.35 0.26286
> > > APCP A1 FULL >=12.7 0.20748
> > > APCP A1 FULL >=25.4 0.087387
> > > APCP A1 MASK >=0.254 0.48104
> > > APCP A1 MASK >=2.54 0.35857
> > > APCP A1 MASK >=6.35 0.24307
> > > APCP A1 MASK >=12.7 0.18594
> > > APCP A1 MASK >=25.4 0.15918
> > >
> > > If you run a STAT-Analysis job and tell it to aggregate together
all
> the
> > > CTC lines together, it'll do exactly that... grouping together
> > contingency
> > > tables across multiple masking regions (FULL and MASK) and
multiple
> > > thresholds (the 5 you listed).  That's probably *NOT* what you
> intended.
> > >
> > > Note that you're using met-6.0.  If you were running met-7.0,
that job
> > > would have resulted in the following debug and warning messages:
> > > DEBUG 2: For case "(nul)", found 2 unique VX_MASK values:
FULL,MASK
> > > WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > > WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > >
> > > We decided that combining results across multiple thresholds
should
> > result
> > > in a warning!
> > >
> > > Please try rerunning, your job but add this option "-by
> > > VX_MASK,FCST_THRESH".  That'll run the same job but write
separate
> output
> > > for each unique combination of VX_MASK and FCST_THRESH columns
it
> > > encounters.  Does that make sense?
> > >
> > > As for the odd numbers in the PRC_A1.txt file, I'd really need
to see
> the
> > > input you passed to STAT-Analysis to debug.  Can you please send
me
> this
> > > file:
> > >    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> > > or even simpler:
> > >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > >
> > > On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > > >        Queue: met_help
> > > >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> > > >        Owner: Nobody
> > > >   Requestors: Man.Zhang-1 at ou.edu
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
> > >
> > > >
> > > >
> > > > Zhang, Mandy has shared OneDrive for Business files with you.
To view
> > > > them, click the links below.
> > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > >
> > > > PRC_A1.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > >
> > > > CTSAPCP_A1.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > >
> > > > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > >
> > > > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi All,
> > > >
> > > >      I use stat_analysis to get CTS and PRC,
> > > >
> > > > ********* stat_analysis of CTS
> > > >
> > > >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > > > -line_type CTC -dump_row
> > > > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> > > -out_line_type
> > > > CTS -out_alpha 0.05000
> > > >
> > > > **************************
> > > >
> > > > but from result of CTS, it seems like missing the thresolds of
GSS .
> > > >
> > > > and
> > > >
> > > > *** stat_analysis of PRC
> > > >
> > > >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST
-line_type PCT
> > -by
> > > > FCST_VAR -dump_row
> > > > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
> > -out_line_type
> > > PRC
> > > > *******
> > > > the results of PRC looks weird, total number is negative.
> > > > Do you know what's the issue?
> > > > I also attach the single run results of both CTS and PRC for
> reference,
> > > > which are resonable.
> > > > Thank you!
> > > > Mandy
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: Zhang, Mandy
Time: Tue Aug 14 09:36:14 2018

Hi John,

     Thank you very much! All of runs look reasonable now.

Mandy


________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 13, 2018 4:35:29 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC from stat_ananlysis

Mandy,

Ah, OK, I understand what's going on now.  Thanks for sending that
file.
The problem here is integer overflow.  You can see this by running the
following 2 jobs:

(1) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
-line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
-fcst_valid_end 20180524_000000

(2) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
-line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
-fcst_valid_end 20180525_000000

The first one uses 1177 input lines and the resulting TOTAL column is
2145234333.
The second one uses 1179 and the TOTAL columns is -2146087705.

When an integer exceeds a value of 2,147,483,647 it gets too big and
integer overflow occurs.  The same thing happens in the latest
development
version of the code as well.  To fix this, we'll need to change the
way
we're storing these values.

Thanks,
John



On Mon, Aug 13, 2018 at 12:52 PM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Zhang, Mandy has shared a OneDrive for Business file with you. To
view it,
> click the link below.
>
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
>
> PCT_to_PRC_A1_thresh1_dump.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> >
>
>
>
>
> Hi John,
>
>      I attached the PCT_to_PRC file, please find it!
>
> Thank you!
>
> Mandy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, August 13, 2018 12:17:42 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> I see the result you've sent and see that it has a large negative
value in
> the TOTAL column, but I'm not able to replicate that same behavior
using
> the .stat file you sent to me:
>    grid_stat_ensmr01_010000L_20180515_060000V.stat
>
> One option would be having you use the "-dump_row" option in
> STAT-Analysis.  That tells STAT-Analysis to write out any of the
input
> lines that were used for the job to an output file:
>
>    /home/mzhang/tool/met/bin/stat_analysis \
>    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
>    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
>    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
>    -dump_row
> /cnproj1/mzhang/met_test/met/2018/PCT_to_PRC_A1_thresh1_dump.txt
> \
>    -v 2
>
> If you send that file "PCT_to_PRC_A1_thresh1_dump.txt" to me, I
should be
> able to rerun your job and get the same bad result.  And then I'd
have a
> better idea as to what's causing the problem.
>
> Thanks,
> John
>
> On Mon, Aug 13, 2018 at 10:46 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> > Zhang, Mandy has shared a OneDrive for Business file with you. To
view
> it,
> > click the link below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> >
> > PRC_A1_thresh1.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > >
> >
> >
> >
> >
> > Hi John,
> >
> >      The total number is still negative when I use job_command
with
> MET6.0
> >
> > ****
> >
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ******
> >     please find attachment for details.
> >      Thank you!
> > Mandy
> >
> >
> > ________________________________
> > From: Zhang, Mandy
> > Sent: Monday, August 13, 2018 11:29:49 AM
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> >
> > Hi John,
> >
> >      the error information of command :
> >
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ***********************
> > is :
> > *** Running STAT-Analysis ***
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744073469325659
> > WARNING:
> > WARNING: do_job_aggr_stat() -> no matching STAT lines found for
job: -job
> > aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254
> > -line_type PCT -by FCST_VAR -out_line_type PRC
> > WARNING:
> >
> > ###################
> > Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
> > Thank you!
> > Mandy
> >
> >
> > ________________________________
> > From: Zhang, Mandy
> > Sent: Monday, August 13, 2018 11:08:48 AM
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> >
> > Hi John,
> >
> >      Now I'm try the job_command with adding thresh,
> >
> > ##
> > #!/bin/sh
> > echo
> > echo "*** Running STAT-Analysis ***"
> > /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -v 2
> > ####################################
> >
> > the difference from my previous job command is "-fcst_thresh
ge0.254",
> > I'll let you know once I get result.
> >
> > Thank you
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, August 13, 2018 10:32:25 AM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> > Mandy,
> >
> > Great, thanks for sending that .stat file.  I ran the following
command
> but
> > am not able to replicate the behavior you describe... getting
negative
> > numbers in the TOTAL column:
> >
> >  /usr/local/met-6.1/bin/stat_analysis \
> > -lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
> > -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT -by
> > FCST_VAR -out_line_type PRC \
> > -out agg_stat_PCT_to_PRC.out
> >
> > I've attached the output file I got.
> >
> > If you are still seeing those odd PRC output lines.  Please send
me the
> > exact command you used to generate them.
> >
> > Thanks,
> > John
> >
> >
> > On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> > >
> > > Zhang, Mandy has shared a OneDrive for Business file with you.
To view
> > it,
> > > click the link below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > >
> > > grid_stat_ensmr01_010000L_20180515_060000V.stat<
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > >
> > >
> > >
> > >
> > >
> > > Hi John,
> > >
> > >     Please find the attachment of stat file.
> > >
> > > Thank you for your suggestion, I'll try it.
> > >
> > > Mandy
> > >
> > > ________________________________
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Friday, August 10, 2018 6:18:59 PM
> > > To: Zhang, Mandy
> > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and
> PRC
> > > from stat_ananlysis
> > >
> > > Mandy,
> > >
> > > Let me address your first question about stat_analysis.  I think
I see
> > the
> > > source of the confusion.
> > >
> > > You sent me 4 files:
> > > - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> > > - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> > > - CTSAPCP_A1.txt
> > > - PRC_A1.txt
> > >
> > > The grid_stat...cts.txt file contains contingency table
statistics
> which
> > > include the GSS column.  You get separate statistics for each
> combination
> > > of masking region and threshold, as seen here:
> > > > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
'{print
> > > $10,$11,$15,$18,$67}'
> > > FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> > > APCP A1 FULL >=0.254 0.43707
> > > APCP A1 FULL >=2.54 0.39237
> > > APCP A1 FULL >=6.35 0.26286
> > > APCP A1 FULL >=12.7 0.20748
> > > APCP A1 FULL >=25.4 0.087387
> > > APCP A1 MASK >=0.254 0.48104
> > > APCP A1 MASK >=2.54 0.35857
> > > APCP A1 MASK >=6.35 0.24307
> > > APCP A1 MASK >=12.7 0.18594
> > > APCP A1 MASK >=25.4 0.15918
> > >
> > > If you run a STAT-Analysis job and tell it to aggregate together
all
> the
> > > CTC lines together, it'll do exactly that... grouping together
> > contingency
> > > tables across multiple masking regions (FULL and MASK) and
multiple
> > > thresholds (the 5 you listed).  That's probably *NOT* what you
> intended.
> > >
> > > Note that you're using met-6.0.  If you were running met-7.0,
that job
> > > would have resulted in the following debug and warning messages:
> > > DEBUG 2: For case "(nul)", found 2 unique VX_MASK values:
FULL,MASK
> > > WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > > WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > >
> > > We decided that combining results across multiple thresholds
should
> > result
> > > in a warning!
> > >
> > > Please try rerunning, your job but add this option "-by
> > > VX_MASK,FCST_THRESH".  That'll run the same job but write
separate
> output
> > > for each unique combination of VX_MASK and FCST_THRESH columns
it
> > > encounters.  Does that make sense?
> > >
> > > As for the odd numbers in the PRC_A1.txt file, I'd really need
to see
> the
> > > input you passed to STAT-Analysis to debug.  Can you please send
me
> this
> > > file:
> > >    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> > > or even simpler:
> > >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > >
> > > On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > > >        Queue: met_help
> > > >      Subject: unresonable results of CTS and PRC from
stat_ananlysis
> > > >        Owner: Nobody
> > > >   Requestors: Man.Zhang-1 at ou.edu
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
> > >
> > > >
> > > >
> > > > Zhang, Mandy has shared OneDrive for Business files with you.
To view
> > > > them, click the links below.
> > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > >
> > > > PRC_A1.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > >
> > > > CTSAPCP_A1.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > >
> > > > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > >
> > > > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi All,
> > > >
> > > >      I use stat_analysis to get CTS and PRC,
> > > >
> > > > ********* stat_analysis of CTS
> > > >
> > > >  -job aggregate_stat -fcst_var APCP -fcst_lev A1 -interp_mthd
NEAREST
> > > > -line_type CTC -dump_row
> > > > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> > > -out_line_type
> > > > CTS -out_alpha 0.05000
> > > >
> > > > **************************
> > > >
> > > > but from result of CTS, it seems like missing the thresolds of
GSS .
> > > >
> > > > and
> > > >
> > > > *** stat_analysis of PRC
> > > >
> > > >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST
-line_type PCT
> > -by
> > > > FCST_VAR -dump_row
> > > > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
> > -out_line_type
> > > PRC
> > > > *******
> > > > the results of PRC looks weird, total number is negative.
> > > > Do you know what's the issue?
> > > > I also attach the single run results of both CTS and PRC for
> reference,
> > > > which are resonable.
> > > > Thank you!
> > > > Mandy
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: unresonable results of CTS and PRC from stat_ananlysis
From: John Halley Gotway
Time: Tue Aug 14 13:31:08 2018

Mandy,

I created a development ticket to capture this request for switching
from
storing contingency table counts as integers to storing them as long
long's... which would support much larger values.

For now, I'll go ahead and resolve this support ticket.

Thanks,
John

On Tue, Aug 14, 2018 at 9:36 AM Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
>
> Hi John,
>
>      Thank you very much! All of runs look reasonable now.
>
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, August 13, 2018 4:35:29 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS and
PRC
> from stat_ananlysis
>
> Mandy,
>
> Ah, OK, I understand what's going on now.  Thanks for sending that
file.
> The problem here is integer overflow.  You can see this by running
the
> following 2 jobs:
>
> (1) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
> -line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
> -fcst_valid_end 20180524_000000
>
> (2) met-6.0/bin/stat_analysis -lookin test.jhg -job aggregate_stat
> -line_type PCT -out_line_type PRC -by FCST_VAR -vx_mask FULL
> -fcst_valid_end 20180525_000000
>
> The first one uses 1177 input lines and the resulting TOTAL column
is
> 2145234333.
> The second one uses 1179 and the TOTAL columns is -2146087705.
>
> When an integer exceeds a value of 2,147,483,647 it gets too big and
> integer overflow occurs.  The same thing happens in the latest
development
> version of the code as well.  To fix this, we'll need to change the
way
> we're storing these values.
>
> Thanks,
> John
>
>
>
> On Mon, Aug 13, 2018 at 12:52 PM Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> >
> > Zhang, Mandy has shared a OneDrive for Business file with you. To
view
> it,
> > click the link below.
> >
> >
> > <
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> > >
> > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> > >
> >
> > PCT_to_PRC_A1_thresh1_dump.txt<
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EVa88YqfWr5PhfflYmT9PscBQAJ7xOd149U3HdutL52eqQ
> > >
> >
> >
> >
> >
> > Hi John,
> >
> >      I attached the PCT_to_PRC file, please find it!
> >
> > Thank you!
> >
> > Mandy
> >
> > ________________________________
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, August 13, 2018 12:17:42 PM
> > To: Zhang, Mandy
> > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and PRC
> > from stat_ananlysis
> >
> > Mandy,
> >
> > I see the result you've sent and see that it has a large negative
value
> in
> > the TOTAL column, but I'm not able to replicate that same behavior
using
> > the .stat file you sent to me:
> >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> >
> > One option would be having you use the "-dump_row" option in
> > STAT-Analysis.  That tells STAT-Analysis to write out any of the
input
> > lines that were used for the job to an output file:
> >
> >    /home/mzhang/tool/met/bin/stat_analysis \
> >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
> >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> >    -dump_row
> > /cnproj1/mzhang/met_test/met/2018/PCT_to_PRC_A1_thresh1_dump.txt
> > \
> >    -v 2
> >
> > If you send that file "PCT_to_PRC_A1_thresh1_dump.txt" to me, I
should be
> > able to rerun your job and get the same bad result.  And then I'd
have a
> > better idea as to what's causing the problem.
> >
> > Thanks,
> > John
> >
> > On Mon, Aug 13, 2018 at 10:46 AM Zhang, Mandy via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560 >
> > >
> > > Zhang, Mandy has shared a OneDrive for Business file with you.
To view
> > it,
> > > click the link below.
> > >
> > >
> > > <
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > > >
> > > [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > > >
> > >
> > > PRC_A1_thresh1.txt<
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EdmAep-diKVIqyxffT_eA48BW1TxtBG875exKFZtCTUwSA
> > > >
> > >
> > >
> > >
> > >
> > > Hi John,
> > >
> > >      The total number is still negative when I use job_command
with
> > MET6.0
> > >
> > > ****
> > >
> > > #!/bin/sh
> > > echo
> > > echo "*** Running STAT-Analysis ***"
> > > /home/mzhang/tool/met/bin/stat_analysis \
> > >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> > >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > > -fcst_lev A1 -interp_mthd NEAREST -obs_thresh ge0.254 \
> > >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> > >    -v 2
> > > ******
> > >     please find attachment for details.
> > >      Thank you!
> > > Mandy
> > >
> > >
> > > ________________________________
> > > From: Zhang, Mandy
> > > Sent: Monday, August 13, 2018 11:29:49 AM
> > > To: met_help at ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and
> PRC
> > > from stat_ananlysis
> > >
> > >
> > > Hi John,
> > >
> > >      the error information of command :
> > >
> > > #!/bin/sh
> > > echo
> > > echo "*** Running STAT-Analysis ***"
> > > /home/mzhang/tool/met/bin/stat_analysis \
> > >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> > >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> > >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> > >    -v 2
> > > ***********************
> > > is :
> > > *** Running STAT-Analysis ***
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=18446744073469325659
> > > WARNING:
> > > WARNING: do_job_aggr_stat() -> no matching STAT lines found for
job:
> -job
> > > aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh
ge.0.254
> > > -line_type PCT -by FCST_VAR -out_line_type PRC
> > > WARNING:
> > >
> > > ###################
> > > Can I set -obs_thresh ge.0.254 in place of -fcst_thresh?
> > > Thank you!
> > > Mandy
> > >
> > >
> > > ________________________________
> > > From: Zhang, Mandy
> > > Sent: Monday, August 13, 2018 11:08:48 AM
> > > To: met_help at ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and
> PRC
> > > from stat_ananlysis
> > >
> > >
> > > Hi John,
> > >
> > >      Now I'm try the job_command with adding thresh,
> > >
> > > ##
> > > #!/bin/sh
> > > echo
> > > echo "*** Running STAT-Analysis ***"
> > > /home/mzhang/tool/met/bin/stat_analysis \
> > >    -lookin /cnproj1/mzhang/met_test/met/2018*/ensm \
> > >    -job aggregate_stat -line_type PCT -out_line_type PRC -by
FCST_VAR
> > > -fcst_lev A1 -interp_mthd NEAREST -fcst_thresh ge0.254 \
> > >    -out /cnproj1/mzhang/met_test/met/2018/PRC_A1_thresh1.txt \
> > >    -v 2
> > > ####################################
> > >
> > > the difference from my previous job command is "-fcst_thresh
ge0.254",
> > > I'll let you know once I get result.
> > >
> > > Thank you
> > >
> > > Mandy
> > >
> > > ________________________________
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Monday, August 13, 2018 10:32:25 AM
> > > To: Zhang, Mandy
> > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of CTS
and
> PRC
> > > from stat_ananlysis
> > >
> > > Mandy,
> > >
> > > Great, thanks for sending that .stat file.  I ran the following
command
> > but
> > > am not able to replicate the behavior you describe... getting
negative
> > > numbers in the TOTAL column:
> > >
> > >  /usr/local/met-6.1/bin/stat_analysis \
> > > -lookin grid_stat_ensmr01_010000L_20180515_060000V.stat \
> > > -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST -line_type
PCT
> -by
> > > FCST_VAR -out_line_type PRC \
> > > -out agg_stat_PCT_to_PRC.out
> > >
> > > I've attached the output file I got.
> > >
> > > If you are still seeing those odd PRC output lines.  Please send
me the
> > > exact command you used to generate them.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Mon, Aug 13, 2018 at 9:25 AM Zhang, Mandy via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
>
> > > >
> > > > Zhang, Mandy has shared a OneDrive for Business file with you.
To
> view
> > > it,
> > > > click the link below.
> > > >
> > > >
> > > > <
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > > >
> > > > [https://r1.res.office365.com/owa/prem/images/dc-
generic_20.png]<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > > >
> > > >
> > > > grid_stat_ensmr01_010000L_20180515_060000V.stat<
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EZwxeQqiTfhKtLvOm2RQRS0B24fGH0ADSYcQgKlb467WDQ
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi John,
> > > >
> > > >     Please find the attachment of stat file.
> > > >
> > > > Thank you for your suggestion, I'll try it.
> > > >
> > > > Mandy
> > > >
> > > > ________________________________
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Friday, August 10, 2018 6:18:59 PM
> > > > To: Zhang, Mandy
> > > > Subject: Re: [rt.rap.ucar.edu #86560] unresonable results of
CTS and
> > PRC
> > > > from stat_ananlysis
> > > >
> > > > Mandy,
> > > >
> > > > Let me address your first question about stat_analysis.  I
think I
> see
> > > the
> > > > source of the confusion.
> > > >
> > > > You sent me 4 files:
> > > > - grid_stat_ensmr01_010000L_20180515_060000V_cts.txt
> > > > - grid_stat_ensmr01_010000L_20180515_060000V_prc.txt
> > > > - CTSAPCP_A1.txt
> > > > - PRC_A1.txt
> > > >
> > > > The grid_stat...cts.txt file contains contingency table
statistics
> > which
> > > > include the GSS column.  You get separate statistics for each
> > combination
> > > > of masking region and threshold, as seen here:
> > > > > cat grid_stat_ensmr01_010000L_20180515_060000V_cts.txt | awk
> '{print
> > > > $10,$11,$15,$18,$67}'
> > > > FCST_VAR FCST_LEV VX_MASK FCST_THRESH GSS
> > > > APCP A1 FULL >=0.254 0.43707
> > > > APCP A1 FULL >=2.54 0.39237
> > > > APCP A1 FULL >=6.35 0.26286
> > > > APCP A1 FULL >=12.7 0.20748
> > > > APCP A1 FULL >=25.4 0.087387
> > > > APCP A1 MASK >=0.254 0.48104
> > > > APCP A1 MASK >=2.54 0.35857
> > > > APCP A1 MASK >=6.35 0.24307
> > > > APCP A1 MASK >=12.7 0.18594
> > > > APCP A1 MASK >=25.4 0.15918
> > > >
> > > > If you run a STAT-Analysis job and tell it to aggregate
together all
> > the
> > > > CTC lines together, it'll do exactly that... grouping together
> > > contingency
> > > > tables across multiple masking regions (FULL and MASK) and
multiple
> > > > thresholds (the 5 you listed).  That's probably *NOT* what you
> > intended.
> > > >
> > > > Note that you're using met-6.0.  If you were running met-7.0,
that
> job
> > > > would have resulted in the following debug and warning
messages:
> > > > DEBUG 2: For case "(nul)", found 2 unique VX_MASK values:
FULL,MASK
> > > > WARNING: For case "(nul)", found 5 unique FCST_THRESH values:
> > > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > > > WARNING: For case "(nul)", found 5 unique OBS_THRESH values:
> > > > >=0.254,>=2.54,>=6.35,>=12.7,>=25.4
> > > >
> > > > We decided that combining results across multiple thresholds
should
> > > result
> > > > in a warning!
> > > >
> > > > Please try rerunning, your job but add this option "-by
> > > > VX_MASK,FCST_THRESH".  That'll run the same job but write
separate
> > output
> > > > for each unique combination of VX_MASK and FCST_THRESH columns
it
> > > > encounters.  Does that make sense?
> > > >
> > > > As for the odd numbers in the PRC_A1.txt file, I'd really need
to see
> > the
> > > > input you passed to STAT-Analysis to debug.  Can you please
send me
> > this
> > > > file:
> > > >    grid_stat_ensmr01_010000L_20180515_060000V_pct.txt
> > > > or even simpler:
> > > >    grid_stat_ensmr01_010000L_20180515_060000V.stat
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 10, 2018 at 3:18 PM Zhang, Mandy via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > Fri Aug 10 15:18:22 2018: Request 86560 was acted upon.
> > > > > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > > > >        Queue: met_help
> > > > >      Subject: unresonable results of CTS and PRC from
> stat_ananlysis
> > > > >        Owner: Nobody
> > > > >   Requestors: Man.Zhang-1 at ou.edu
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86560
> > > >
> > > > >
> > > > >
> > > > > Zhang, Mandy has shared OneDrive for Business files with
you. To
> view
> > > > > them, click the links below.
> > > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > > >
> > > > > [https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > > >
> > > > >
> > > > > PRC_A1.txt<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUwxsnuxsi1Dn5YB03eDISYBvQNSE8aBq33jTnxVgmTx6A
> > > > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > > >
> > > > > [https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > > >
> > > > >
> > > > > CTSAPCP_A1.txt<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/ER9NLpKxORlNuReNaSHCZbAB66FLx5Z8Yl7LfGbBg0ndcg
> > > > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > > >
> > > > > [https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > > >
> > > > >
> > > > > grid_stat_ensmr01_010000L_20180515_060000V_cts.txt<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EYaX0IyRM41KnbjdnTQCPXgBCWz4zAD2_S-h74K-t6oKEQ
> > > > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > > >
> > > > > [https://r1.res.office365.com/owa/prem/images/dc-
txt_20.png]<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > > >
> > > > >
> > > > > grid_stat_ensmr01_010000L_20180515_060000V_prc.txt<
> > > > >
> > > >
> > >
> >
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EUeW1NRN-cJOpZ_vCtkqfH4BnC89h7AuaedNhc9BwDWnMw
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi All,
> > > > >
> > > > >      I use stat_analysis to get CTS and PRC,
> > > > >
> > > > > ********* stat_analysis of CTS
> > > > >
> > > > >  -job aggregate_stat -fcst_var APCP -fcst_lev A1
-interp_mthd
> NEAREST
> > > > > -line_type CTC -dump_row
> > > > > /cnproj1/mzhang/met_test/met/2018/01/job_aggregate_stat.stat
> > > > -out_line_type
> > > > > CTS -out_alpha 0.05000
> > > > >
> > > > > **************************
> > > > >
> > > > > but from result of CTS, it seems like missing the thresolds
of GSS
> .
> > > > >
> > > > > and
> > > > >
> > > > > *** stat_analysis of PRC
> > > > >
> > > > >  -job aggregate_stat -fcst_lev A1 -interp_mthd NEAREST
-line_type
> PCT
> > > -by
> > > > > FCST_VAR -dump_row
> > > > > /cnproj1/mzhang/met_test/met/2018/job_aggregate_stat.stat
> > > -out_line_type
> > > > PRC
> > > > > *******
> > > > > the results of PRC looks weird, total number is negative.
> > > > > Do you know what's the issue?
> > > > > I also attach the single run results of both CTS and PRC for
> > reference,
> > > > > which are resonable.
> > > > > Thank you!
> > > > > Mandy
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------


More information about the Met_help mailing list