[Met_help] [rt.rap.ucar.edu #83268] History for question on re-running data with new thresholds
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:04:00 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I've run a series of days with POINT_STAT with one set of thresholds
indicated in the configuration file. I need to change these thresholds,
and re-run my data. Do I need to start again with the original files I
used (GFS hourly output and ASCAT from the prepbufr), or is there a way to
re-run data with the *stat files?
Also, can I use floating numbers as thresholds, or do the thresholds have
to be integers?
Thanks in advance!
Roz
--
Rosalyn MacCracken
Support Scientist
Ocean Applilcations Branch
NOAA/NWS Ocean Prediction Center
NCWCP
5830 University Research Ct
College Park, MD 20740-3818
(p) 301-683-1551
rosalyn.maccracken at noaa.gov
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Mon Dec 11 11:24:14 2017
Roz,
Good question. Please take a look at the output you generated during
each
run of Point-Stat. If the output contains the matched pair (MPR)
output
lines, then the answer is no, you do not need to rerun Point-Stat.
Instead, you can just run STAT-Analysis to apply a new set of
thresholds.
If you do not have MPR output lines, then yes, you'd need to rerun
Point-Stat with your updated set of thresholds.
Yes, thresholds can be defined using floating points. They certainly
don't
need to be integers.
Thanks,
John
On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA Affiliate
via
RT <met_help at ucar.edu> wrote:
>
> Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> Queue: met_help
> Subject: question on re-running data with new thresholds
> Owner: Nobody
> Requestors: rosalyn.maccracken at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>
>
> Hi,
>
> I've run a series of days with POINT_STAT with one set of thresholds
> indicated in the configuration file. I need to change these
thresholds,
> and re-run my data. Do I need to start again with the original
files I
> used (GFS hourly output and ASCAT from the prepbufr), or is there a
way to
> re-run data with the *stat files?
>
> Also, can I use floating numbers as thresholds, or do the thresholds
have
> to be integers?
>
> Thanks in advance!
>
> Roz
>
> --
> Rosalyn MacCracken
> Support Scientist
>
> Ocean Applilcations Branch
> NOAA/NWS Ocean Prediction Center
> NCWCP
> 5830 University Research Ct
> College Park, MD 20740-3818
>
> (p) 301-683-1551
> rosalyn.maccracken at noaa.gov
>
>
------------------------------------------------
Subject: question on re-running data with new thresholds
From: Rosalyn MacCracken - NOAA Affiliate
Time: Mon Dec 11 11:30:59 2017
Hi John,
So, I have a file that has matched pairs, the *mpr file. But, for
each
run, there is also the *.stat file. But, I'm guessing that contrains
the
matched pairs as well. I thought all output contains matched pairs,
no?
If I re-ran STAT_Analysis, what do I need as the input to create new
thresholds? The original files?
Roz
On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Roz,
>
> Good question. Please take a look at the output you generated
during each
> run of Point-Stat. If the output contains the matched pair (MPR)
output
> lines, then the answer is no, you do not need to rerun Point-Stat.
> Instead, you can just run STAT-Analysis to apply a new set of
thresholds.
>
> If you do not have MPR output lines, then yes, you'd need to rerun
> Point-Stat with your updated set of thresholds.
>
> Yes, thresholds can be defined using floating points. They
certainly don't
> need to be integers.
>
> Thanks,
> John
>
> On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> > Queue: met_help
> > Subject: question on re-running data with new thresholds
> > Owner: Nobody
> > Requestors: rosalyn.maccracken at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> >
> >
> > Hi,
> >
> > I've run a series of days with POINT_STAT with one set of
thresholds
> > indicated in the configuration file. I need to change these
thresholds,
> > and re-run my data. Do I need to start again with the original
files I
> > used (GFS hourly output and ASCAT from the prepbufr), or is there
a way
> to
> > re-run data with the *stat files?
> >
> > Also, can I use floating numbers as thresholds, or do the
thresholds have
> > to be integers?
> >
> > Thanks in advance!
> >
> > Roz
> >
> > --
> > Rosalyn MacCracken
> > Support Scientist
> >
> > Ocean Applilcations Branch
> > NOAA/NWS Ocean Prediction Center
> > NCWCP
> > 5830 University Research Ct
> > College Park, MD 20740-3818
> >
> > (p) 301-683-1551
> > rosalyn.maccracken at noaa.gov
> >
> >
>
>
--
Rosalyn MacCracken
Support Scientist
Ocean Applilcations Branch
NOAA/NWS Ocean Prediction Center
NCWCP
5830 University Research Ct
College Park, MD 20740-3818
(p) 301-683-1551
rosalyn.maccracken at noaa.gov
------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Mon Dec 11 14:45:55 2017
Roz,
Each time your run Point-Stat, it always writes out a .stat file. The
line
types that are written to that file are controlled by the settings in
the
"output_flag" config file option. Take a look there are the mpr
setting:
mpr = BOTH;
When mpr is set to STAT, it is written to the .stat output file.
When mpr is set to BOTH, it is written to **both** the .stat output
file
and an ascii output file named *_mpr.txt.
So the MPR data in the .stat and _mpr.txt output files are duplicates.
I'd
suggest setting "mpr = STAT" to avoid writing duplicate data,
especially
since the MPR data is so copious.
You have MPR data and you'd like to run it through STAT-Analysis to
generate stats. But the question is how are you running
stat_analysis?
For example, perhaps you're applying categorical thresholds to define
a 2x2
contingency table and computing stats. You can define the output
thresholds like this (assuming 2 meter temperature):
stat_analysis -lookin mydata.stat -job aggregate_stat -line_type MPR
-out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh gt273
The -out_thresh option defines both the forecast and observation
thresholds. Or you can specify -out_fcst_thresh and -out_obs_thresh
separately.
Please open up the file met-6.0/data/config/README and search for
"out_thresh" for some info on this.
Thanks,
John
On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA Affiliate
via
RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>
> Hi John,
>
> So, I have a file that has matched pairs, the *mpr file. But, for
each
> run, there is also the *.stat file. But, I'm guessing that
contrains the
> matched pairs as well. I thought all output contains matched pairs,
no?
>
> If I re-ran STAT_Analysis, what do I need as the input to create new
> thresholds? The original files?
>
> Roz
>
> On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Roz,
> >
> > Good question. Please take a look at the output you generated
during
> each
> > run of Point-Stat. If the output contains the matched pair (MPR)
output
> > lines, then the answer is no, you do not need to rerun Point-Stat.
> > Instead, you can just run STAT-Analysis to apply a new set of
thresholds.
> >
> > If you do not have MPR output lines, then yes, you'd need to rerun
> > Point-Stat with your updated set of thresholds.
> >
> > Yes, thresholds can be defined using floating points. They
certainly
> don't
> > need to be integers.
> >
> > Thanks,
> > John
> >
> > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
Affiliate via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> > > Queue: met_help
> > > Subject: question on re-running data with new thresholds
> > > Owner: Nobody
> > > Requestors: rosalyn.maccracken at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268
> >
> > >
> > >
> > > Hi,
> > >
> > > I've run a series of days with POINT_STAT with one set of
thresholds
> > > indicated in the configuration file. I need to change these
> thresholds,
> > > and re-run my data. Do I need to start again with the original
files I
> > > used (GFS hourly output and ASCAT from the prepbufr), or is
there a way
> > to
> > > re-run data with the *stat files?
> > >
> > > Also, can I use floating numbers as thresholds, or do the
thresholds
> have
> > > to be integers?
> > >
> > > Thanks in advance!
> > >
> > > Roz
> > >
> > > --
> > > Rosalyn MacCracken
> > > Support Scientist
> > >
> > > Ocean Applilcations Branch
> > > NOAA/NWS Ocean Prediction Center
> > > NCWCP
> > > 5830 University Research Ct
> > > College Park, MD 20740-3818
> > >
> > > (p) 301-683-1551
> > > rosalyn.maccracken at noaa.gov
> > >
> > >
> >
> >
>
>
> --
> Rosalyn MacCracken
> Support Scientist
>
> Ocean Applilcations Branch
> NOAA/NWS Ocean Prediction Center
> NCWCP
> 5830 University Research Ct
> College Park, MD 20740-3818
>
> (p) 301-683-1551
> rosalyn.maccracken at noaa.gov
>
>
------------------------------------------------
Subject: question on re-running data with new thresholds
From: Rosalyn MacCracken - NOAA Affiliate
Time: Wed Dec 13 13:31:42 2017
Hi John,
Thanks for the explanation. I'll have to take a look at the README
file as
soon as I solve this other issue. I know you know the answer to this,
so,
I will ask you (since I asked Julie a similar question, and she said
you
would know better than her). :-)
I have threshold ranges in my PointStat config file (right, I already
said
I did). I didn't make them correct in the beginning. I had created
my
threshold in knots, instead of m/s. Anyway, I'm in the process of
correcting this. I need ranges in m/s to match the Beaufort Scale,
which
is:
0.0-10.99
11.0-16.99
17.0-21.99
etc
in other words, if I wrote this out in an equation, it would be:
>=0.0 && < 11.0
>=11.0 && < 17.0
>= 17.0 && < 22.0
etc.
I tried to do this in the config file (remember, this is m/s, since
there
is nowhere to have it convert automatically to knots in the config
file...at least I didn't think so...):
wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
>=8.74556&&<=11.3126,...
but, I got this error:
ERROR : check_mctc_thresh() -> when verifying using multi-category
contingency tables, the thresholds must be monotonically increasing
and be
of the same inequality type (lt, le, gt, or ge).
So, how do I do a range in the config file, without having the
categories
overlap? I was doing it this way:
wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0
];
but, that would cause the categories to overlap like:
0-8
8-17
17-28
etc, right?
Roz
On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Roz,
>
> Each time your run Point-Stat, it always writes out a .stat file.
The line
> types that are written to that file are controlled by the settings
in the
> "output_flag" config file option. Take a look there are the mpr
setting:
> mpr = BOTH;
>
> When mpr is set to STAT, it is written to the .stat output file.
> When mpr is set to BOTH, it is written to **both** the .stat output
file
> and an ascii output file named *_mpr.txt.
>
> So the MPR data in the .stat and _mpr.txt output files are
duplicates. I'd
> suggest setting "mpr = STAT" to avoid writing duplicate data,
especially
> since the MPR data is so copious.
>
> You have MPR data and you'd like to run it through STAT-Analysis to
> generate stats. But the question is how are you running
stat_analysis?
>
> For example, perhaps you're applying categorical thresholds to
define a 2x2
> contingency table and computing stats. You can define the output
> thresholds like this (assuming 2 meter temperature):
>
> stat_analysis -lookin mydata.stat -job aggregate_stat -line_type MPR
> -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh gt273
>
> The -out_thresh option defines both the forecast and observation
> thresholds. Or you can specify -out_fcst_thresh and -out_obs_thresh
> separately.
>
> Please open up the file met-6.0/data/config/README and search for
> "out_thresh" for some info on this.
>
> Thanks,
> John
>
>
> On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> >
> > Hi John,
> >
> > So, I have a file that has matched pairs, the *mpr file. But, for
each
> > run, there is also the *.stat file. But, I'm guessing that
contrains the
> > matched pairs as well. I thought all output contains matched
pairs, no?
> >
> > If I re-ran STAT_Analysis, what do I need as the input to create
new
> > thresholds? The original files?
> >
> > Roz
> >
> > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Roz,
> > >
> > > Good question. Please take a look at the output you generated
during
> > each
> > > run of Point-Stat. If the output contains the matched pair
(MPR)
> output
> > > lines, then the answer is no, you do not need to rerun Point-
Stat.
> > > Instead, you can just run STAT-Analysis to apply a new set of
> thresholds.
> > >
> > > If you do not have MPR output lines, then yes, you'd need to
rerun
> > > Point-Stat with your updated set of thresholds.
> > >
> > > Yes, thresholds can be defined using floating points. They
certainly
> > don't
> > > need to be integers.
> > >
> > > Thanks,
> > > John
> > >
> > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
Affiliate
> via
> > > RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > > > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> > > > Queue: met_help
> > > > Subject: question on re-running data with new thresholds
> > > > Owner: Nobody
> > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=83268
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I've run a series of days with POINT_STAT with one set of
thresholds
> > > > indicated in the configuration file. I need to change these
> > thresholds,
> > > > and re-run my data. Do I need to start again with the
original
> files I
> > > > used (GFS hourly output and ASCAT from the prepbufr), or is
there a
> way
> > > to
> > > > re-run data with the *stat files?
> > > >
> > > > Also, can I use floating numbers as thresholds, or do the
thresholds
> > have
> > > > to be integers?
> > > >
> > > > Thanks in advance!
> > > >
> > > > Roz
> > > >
> > > > --
> > > > Rosalyn MacCracken
> > > > Support Scientist
> > > >
> > > > Ocean Applilcations Branch
> > > > NOAA/NWS Ocean Prediction Center
> > > > NCWCP
> > > > 5830 University Research Ct
> > > > College Park, MD 20740-3818
> > > >
> > > > (p) 301-683-1551
> > > > rosalyn.maccracken at noaa.gov
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Rosalyn MacCracken
> > Support Scientist
> >
> > Ocean Applilcations Branch
> > NOAA/NWS Ocean Prediction Center
> > NCWCP
> > 5830 University Research Ct
> > College Park, MD 20740-3818
> >
> > (p) 301-683-1551
> > rosalyn.maccracken at noaa.gov
> >
> >
>
>
--
Rosalyn MacCracken
Support Scientist
Ocean Applilcations Branch
NOAA/NWS Ocean Prediction Center
NCWCP
5830 University Research Ct
College Park, MD 20740-3818
(p) 301-683-1551
rosalyn.maccracken at noaa.gov
------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Wed Dec 13 14:24:59 2017
Roz,
Two points to make here.
First, we released met-6.1 last week and it now includes a new feature
named "convert(x)". This enables users to define a conversion
function for
their data. It sounds like you'd like to convert m/s to kts, so you'd
use:
convert(x) = 1.94384449 * x;
So for example...
fcst = {
field = [ { name = "WIND"; level="Z10"; convert(x) = 1.94384449 *
x; } ]
}
If you upgrade to met-6.1, you'd have this new feature available to
you.
Second, if you want to use MCTC and MCTS output lines from MET, that
error
message is correct. If you don't care about MCTC/MCTS output, just
turn
them off in the config file (mctc = NONE; mcts = NONE;).
If you do want MCTC and MCTS output, then by convention, the
cat_thresh
array must be defined as non-overlapping thresholds using a consistent
threshold type:
cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0
];
... or ...
cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
When applying the multi-category logic, Point-Stat and Grid-Stat use
these
thresholds to define which bin each point falls into.
Hope that helps clarify.
Thanks,
John
On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>
> Hi John,
>
> Thanks for the explanation. I'll have to take a look at the README
file as
> soon as I solve this other issue. I know you know the answer to
this, so,
> I will ask you (since I asked Julie a similar question, and she said
you
> would know better than her). :-)
>
> I have threshold ranges in my PointStat config file (right, I
already said
> I did). I didn't make them correct in the beginning. I had created
my
> threshold in knots, instead of m/s. Anyway, I'm in the process of
> correcting this. I need ranges in m/s to match the Beaufort Scale,
which
> is:
> 0.0-10.99
> 11.0-16.99
> 17.0-21.99
> etc
>
> in other words, if I wrote this out in an equation, it would be:
> >=0.0 && < 11.0
> >=11.0 && < 17.0
> >= 17.0 && < 22.0
> etc.
>
> I tried to do this in the config file (remember, this is m/s, since
there
> is nowhere to have it convert automatically to knots in the config
> file...at least I didn't think so...):
> wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
> >=8.74556&&<=11.3126,...
>
> but, I got this error:
> ERROR : check_mctc_thresh() -> when verifying using multi-category
> contingency tables, the thresholds must be monotonically increasing
and be
> of the same inequality type (lt, le, gt, or ge).
>
> So, how do I do a range in the config file, without having the
categories
> overlap? I was doing it this way:
> wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0
];
> but, that would cause the categories to overlap like:
> 0-8
> 8-17
> 17-28
> etc, right?
>
> Roz
>
> On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Roz,
> >
> > Each time your run Point-Stat, it always writes out a .stat file.
The
> line
> > types that are written to that file are controlled by the settings
in the
> > "output_flag" config file option. Take a look there are the mpr
setting:
> > mpr = BOTH;
> >
> > When mpr is set to STAT, it is written to the .stat output file.
> > When mpr is set to BOTH, it is written to **both** the .stat
output file
> > and an ascii output file named *_mpr.txt.
> >
> > So the MPR data in the .stat and _mpr.txt output files are
duplicates.
> I'd
> > suggest setting "mpr = STAT" to avoid writing duplicate data,
especially
> > since the MPR data is so copious.
> >
> > You have MPR data and you'd like to run it through STAT-Analysis
to
> > generate stats. But the question is how are you running
stat_analysis?
> >
> > For example, perhaps you're applying categorical thresholds to
define a
> 2x2
> > contingency table and computing stats. You can define the output
> > thresholds like this (assuming 2 meter temperature):
> >
> > stat_analysis -lookin mydata.stat -job aggregate_stat -line_type
MPR
> > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh gt273
> >
> > The -out_thresh option defines both the forecast and observation
> > thresholds. Or you can specify -out_fcst_thresh and
-out_obs_thresh
> > separately.
> >
> > Please open up the file met-6.0/data/config/README and search for
> > "out_thresh" for some info on this.
> >
> > Thanks,
> > John
> >
> >
> > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
Affiliate via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > >
> > > Hi John,
> > >
> > > So, I have a file that has matched pairs, the *mpr file. But,
for each
> > > run, there is also the *.stat file. But, I'm guessing that
contrains
> the
> > > matched pairs as well. I thought all output contains matched
pairs,
> no?
> > >
> > > If I re-ran STAT_Analysis, what do I need as the input to create
new
> > > thresholds? The original files?
> > >
> > > Roz
> > >
> > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Roz,
> > > >
> > > > Good question. Please take a look at the output you generated
during
> > > each
> > > > run of Point-Stat. If the output contains the matched pair
(MPR)
> > output
> > > > lines, then the answer is no, you do not need to rerun Point-
Stat.
> > > > Instead, you can just run STAT-Analysis to apply a new set of
> > thresholds.
> > > >
> > > > If you do not have MPR output lines, then yes, you'd need to
rerun
> > > > Point-Stat with your updated set of thresholds.
> > > >
> > > > Yes, thresholds can be defined using floating points. They
certainly
> > > don't
> > > > need to be integers.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
Affiliate
> > via
> > > > RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > > > > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: question on re-running data with new
thresholds
> > > > > Owner: Nobody
> > > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > Ticket/Display.html?id=83268
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I've run a series of days with POINT_STAT with one set of
> thresholds
> > > > > indicated in the configuration file. I need to change these
> > > thresholds,
> > > > > and re-run my data. Do I need to start again with the
original
> > files I
> > > > > used (GFS hourly output and ASCAT from the prepbufr), or is
there a
> > way
> > > > to
> > > > > re-run data with the *stat files?
> > > > >
> > > > > Also, can I use floating numbers as thresholds, or do the
> thresholds
> > > have
> > > > > to be integers?
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > > Roz
> > > > >
> > > > > --
> > > > > Rosalyn MacCracken
> > > > > Support Scientist
> > > > >
> > > > > Ocean Applilcations Branch
> > > > > NOAA/NWS Ocean Prediction Center
> > > > > NCWCP
> > > > > 5830 University Research Ct
> > > > > College Park, MD 20740-3818
> > > > >
> > > > > (p) 301-683-1551
> > > > > rosalyn.maccracken at noaa.gov
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Rosalyn MacCracken
> > > Support Scientist
> > >
> > > Ocean Applilcations Branch
> > > NOAA/NWS Ocean Prediction Center
> > > NCWCP
> > > 5830 University Research Ct
> > > College Park, MD 20740-3818
> > >
> > > (p) 301-683-1551
> > > rosalyn.maccracken at noaa.gov
> > >
> > >
> >
> >
>
>
> --
> Rosalyn MacCracken
> Support Scientist
>
> Ocean Applilcations Branch
> NOAA/NWS Ocean Prediction Center
> NCWCP
> 5830 University Research Ct
> College Park, MD 20740-3818
>
> (p) 301-683-1551
> rosalyn.maccracken at noaa.gov
>
>
------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Wed Dec 13 14:44:00 2017
Roz,
Please take a look at the met-6.1 release notes here:
https://dtcenter.org/met/users/support/release_notes/METv6.1_release_notes.php
There's a link at the top of that page to a PDF which shows some nice
pictures.
You may find the "Unit Conversion Functions" on slide 13 useful, as
well
perhaps as the "Censoring Data" on slide 14.
Thanks,
John
On Wed, Dec 13, 2017 at 2:24 PM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Roz,
>
> Two points to make here.
>
> First, we released met-6.1 last week and it now includes a new
feature
> named "convert(x)". This enables users to define a conversion
function for
> their data. It sounds like you'd like to convert m/s to kts, so
you'd use:
> convert(x) = 1.94384449 * x;
>
> So for example...
>
> fcst = {
> field = [ { name = "WIND"; level="Z10"; convert(x) = 1.94384449 *
x; } ]
> }
>
> If you upgrade to met-6.1, you'd have this new feature available to
you.
>
> Second, if you want to use MCTC and MCTS output lines from MET, that
error
> message is correct. If you don't care about MCTC/MCTS output, just
turn
> them off in the config file (mctc = NONE; mcts = NONE;).
>
> If you do want MCTC and MCTS output, then by convention, the
cat_thresh
> array must be defined as non-overlapping thresholds using a
consistent
> threshold type:
>
> cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> ... or ...
> cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
>
> When applying the multi-category logic, Point-Stat and Grid-Stat use
these
> thresholds to define which bin each point falls into.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA Affiliate
via
> RT <met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>>
>> Hi John,
>>
>> Thanks for the explanation. I'll have to take a look at the README
file
>> as
>> soon as I solve this other issue. I know you know the answer to
this, so,
>> I will ask you (since I asked Julie a similar question, and she
said you
>> would know better than her). :-)
>>
>> I have threshold ranges in my PointStat config file (right, I
already said
>> I did). I didn't make them correct in the beginning. I had
created my
>> threshold in knots, instead of m/s. Anyway, I'm in the process of
>> correcting this. I need ranges in m/s to match the Beaufort Scale,
which
>> is:
>> 0.0-10.99
>> 11.0-16.99
>> 17.0-21.99
>> etc
>>
>> in other words, if I wrote this out in an equation, it would be:
>> >=0.0 && < 11.0
>> >=11.0 && < 17.0
>> >= 17.0 && < 22.0
>> etc.
>>
>> I tried to do this in the config file (remember, this is m/s, since
there
>> is nowhere to have it convert automatically to knots in the config
>> file...at least I didn't think so...):
>> wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
>> >=8.74556&&<=11.3126,...
>>
>> but, I got this error:
>> ERROR : check_mctc_thresh() -> when verifying using multi-category
>> contingency tables, the thresholds must be monotonically increasing
and be
>> of the same inequality type (lt, le, gt, or ge).
>>
>> So, how do I do a range in the config file, without having the
categories
>> overlap? I was doing it this way:
>> wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
>> but, that would cause the categories to overlap like:
>> 0-8
>> 8-17
>> 17-28
>> etc, right?
>>
>> Roz
>>
>> On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Roz,
>> >
>> > Each time your run Point-Stat, it always writes out a .stat file.
The
>> line
>> > types that are written to that file are controlled by the
settings in
>> the
>> > "output_flag" config file option. Take a look there are the mpr
>> setting:
>> > mpr = BOTH;
>> >
>> > When mpr is set to STAT, it is written to the .stat output file.
>> > When mpr is set to BOTH, it is written to **both** the .stat
output file
>> > and an ascii output file named *_mpr.txt.
>> >
>> > So the MPR data in the .stat and _mpr.txt output files are
duplicates.
>> I'd
>> > suggest setting "mpr = STAT" to avoid writing duplicate data,
especially
>> > since the MPR data is so copious.
>> >
>> > You have MPR data and you'd like to run it through STAT-Analysis
to
>> > generate stats. But the question is how are you running
stat_analysis?
>> >
>> > For example, perhaps you're applying categorical thresholds to
define a
>> 2x2
>> > contingency table and computing stats. You can define the output
>> > thresholds like this (assuming 2 meter temperature):
>> >
>> > stat_analysis -lookin mydata.stat -job aggregate_stat -line_type
MPR
>> > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh gt273
>> >
>> > The -out_thresh option defines both the forecast and observation
>> > thresholds. Or you can specify -out_fcst_thresh and
-out_obs_thresh
>> > separately.
>> >
>> > Please open up the file met-6.0/data/config/README and search for
>> > "out_thresh" for some info on this.
>> >
>> > Thanks,
>> > John
>> >
>> >
>> > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
Affiliate
>> via
>> > RT <met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>> > >
>> > > Hi John,
>> > >
>> > > So, I have a file that has matched pairs, the *mpr file. But,
for
>> each
>> > > run, there is also the *.stat file. But, I'm guessing that
contrains
>> the
>> > > matched pairs as well. I thought all output contains matched
pairs,
>> no?
>> > >
>> > > If I re-ran STAT_Analysis, what do I need as the input to
create new
>> > > thresholds? The original files?
>> > >
>> > > Roz
>> > >
>> > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > > Roz,
>> > > >
>> > > > Good question. Please take a look at the output you
generated
>> during
>> > > each
>> > > > run of Point-Stat. If the output contains the matched pair
(MPR)
>> > output
>> > > > lines, then the answer is no, you do not need to rerun Point-
Stat.
>> > > > Instead, you can just run STAT-Analysis to apply a new set of
>> > thresholds.
>> > > >
>> > > > If you do not have MPR output lines, then yes, you'd need to
rerun
>> > > > Point-Stat with your updated set of thresholds.
>> > > >
>> > > > Yes, thresholds can be defined using floating points. They
>> certainly
>> > > don't
>> > > > need to be integers.
>> > > >
>> > > > Thanks,
>> > > > John
>> > > >
>> > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
>> Affiliate
>> > via
>> > > > RT <met_help at ucar.edu> wrote:
>> > > >
>> > > > >
>> > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
>> > > > > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
>> > > > > Queue: met_help
>> > > > > Subject: question on re-running data with new
thresholds
>> > > > > Owner: Nobody
>> > > > > Requestors: rosalyn.maccracken at noaa.gov
>> > > > > Status: new
>> > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
>> > Ticket/Display.html?id=83268
>> > > >
>> > > > >
>> > > > >
>> > > > > Hi,
>> > > > >
>> > > > > I've run a series of days with POINT_STAT with one set of
>> thresholds
>> > > > > indicated in the configuration file. I need to change
these
>> > > thresholds,
>> > > > > and re-run my data. Do I need to start again with the
original
>> > files I
>> > > > > used (GFS hourly output and ASCAT from the prepbufr), or is
there
>> a
>> > way
>> > > > to
>> > > > > re-run data with the *stat files?
>> > > > >
>> > > > > Also, can I use floating numbers as thresholds, or do the
>> thresholds
>> > > have
>> > > > > to be integers?
>> > > > >
>> > > > > Thanks in advance!
>> > > > >
>> > > > > Roz
>> > > > >
>> > > > > --
>> > > > > Rosalyn MacCracken
>> > > > > Support Scientist
>> > > > >
>> > > > > Ocean Applilcations Branch
>> > > > > NOAA/NWS Ocean Prediction Center
>> > > > > NCWCP
>> > > > > 5830 University Research Ct
>> > > > > College Park, MD 20740-3818
>> > > > >
>> > > > > (p) 301-683-1551
>> > > > > rosalyn.maccracken at noaa.gov
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Rosalyn MacCracken
>> > > Support Scientist
>> > >
>> > > Ocean Applilcations Branch
>> > > NOAA/NWS Ocean Prediction Center
>> > > NCWCP
>> > > 5830 University Research Ct
>> > > College Park, MD 20740-3818
>> > >
>> > > (p) 301-683-1551
>> > > rosalyn.maccracken at noaa.gov
>> > >
>> > >
>> >
>> >
>>
>>
>> --
>> Rosalyn MacCracken
>> Support Scientist
>>
>> Ocean Applilcations Branch
>> NOAA/NWS Ocean Prediction Center
>> NCWCP
>> 5830 University Research Ct
>> College Park, MD 20740-3818
>>
>> (p) 301-683-1551
>> rosalyn.maccracken at noaa.gov
>>
>>
>
------------------------------------------------
Subject: question on re-running data with new thresholds
From: Rosalyn MacCracken - NOAA Affiliate
Time: Thu Dec 14 06:35:49 2017
hi John,
Thanks for the explanation. I have no idea if they will upgrade MET.
I'm
still waiting on them fixing that international dateline issue. In
the
meantime, I've found a work-around for that, and the conversion thing.
Ok, so, when I do either:
cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0
];
... or ...
cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
the bins don't overlap, correct? In other words, if I use either of
these
cat_thresh statements, I have a value of 8.0, the point will only be
put in
the bin 0.0-8.0, and MET is smart enough not to put the point in both
0.0-8.0 and 8.0-17.0, correct?
So, how what do the bins look like if I use either of these
statements?
Something like:
for the top, with >=
0-8.0
8.01 - 17.0
17.01 - 28.0
etc
and for the bottom, with >
0-7.999
8.0-16.999
17-27.999
etc
Or, something different?
Thanks,
Roz
On Wed, Dec 13, 2017 at 9:24 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Roz,
>
> Two points to make here.
>
> First, we released met-6.1 last week and it now includes a new
feature
> named "convert(x)". This enables users to define a conversion
function for
> their data. It sounds like you'd like to convert m/s to kts, so
you'd use:
> convert(x) = 1.94384449 * x;
>
> So for example...
>
> fcst = {
> field = [ { name = "WIND"; level="Z10"; convert(x) = 1.94384449 *
x; } ]
> }
>
> If you upgrade to met-6.1, you'd have this new feature available to
you.
>
> Second, if you want to use MCTC and MCTS output lines from MET, that
error
> message is correct. If you don't care about MCTC/MCTS output, just
turn
> them off in the config file (mctc = NONE; mcts = NONE;).
>
> If you do want MCTC and MCTS output, then by convention, the
cat_thresh
> array must be defined as non-overlapping thresholds using a
consistent
> threshold type:
>
> cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> ... or ...
> cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
>
> When applying the multi-category logic, Point-Stat and Grid-Stat use
these
> thresholds to define which bin each point falls into.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA Affiliate
via RT
> <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> >
> > Hi John,
> >
> > Thanks for the explanation. I'll have to take a look at the
README file
> as
> > soon as I solve this other issue. I know you know the answer to
this,
> so,
> > I will ask you (since I asked Julie a similar question, and she
said you
> > would know better than her). :-)
> >
> > I have threshold ranges in my PointStat config file (right, I
already
> said
> > I did). I didn't make them correct in the beginning. I had
created my
> > threshold in knots, instead of m/s. Anyway, I'm in the process of
> > correcting this. I need ranges in m/s to match the Beaufort
Scale, which
> > is:
> > 0.0-10.99
> > 11.0-16.99
> > 17.0-21.99
> > etc
> >
> > in other words, if I wrote this out in an equation, it would be:
> > >=0.0 && < 11.0
> > >=11.0 && < 17.0
> > >= 17.0 && < 22.0
> > etc.
> >
> > I tried to do this in the config file (remember, this is m/s,
since there
> > is nowhere to have it convert automatically to knots in the config
> > file...at least I didn't think so...):
> > wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
> > >=8.74556&&<=11.3126,...
> >
> > but, I got this error:
> > ERROR : check_mctc_thresh() -> when verifying using multi-
category
> > contingency tables, the thresholds must be monotonically
increasing and
> be
> > of the same inequality type (lt, le, gt, or ge).
> >
> > So, how do I do a range in the config file, without having the
categories
> > overlap? I was doing it this way:
> > wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> > but, that would cause the categories to overlap like:
> > 0-8
> > 8-17
> > 17-28
> > etc, right?
> >
> > Roz
> >
> > On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Roz,
> > >
> > > Each time your run Point-Stat, it always writes out a .stat
file. The
> > line
> > > types that are written to that file are controlled by the
settings in
> the
> > > "output_flag" config file option. Take a look there are the mpr
> setting:
> > > mpr = BOTH;
> > >
> > > When mpr is set to STAT, it is written to the .stat output file.
> > > When mpr is set to BOTH, it is written to **both** the .stat
output
> file
> > > and an ascii output file named *_mpr.txt.
> > >
> > > So the MPR data in the .stat and _mpr.txt output files are
duplicates.
> > I'd
> > > suggest setting "mpr = STAT" to avoid writing duplicate data,
> especially
> > > since the MPR data is so copious.
> > >
> > > You have MPR data and you'd like to run it through STAT-Analysis
to
> > > generate stats. But the question is how are you running
stat_analysis?
> > >
> > > For example, perhaps you're applying categorical thresholds to
define a
> > 2x2
> > > contingency table and computing stats. You can define the
output
> > > thresholds like this (assuming 2 meter temperature):
> > >
> > > stat_analysis -lookin mydata.stat -job aggregate_stat -line_type
MPR
> > > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh gt273
> > >
> > > The -out_thresh option defines both the forecast and observation
> > > thresholds. Or you can specify -out_fcst_thresh and
-out_obs_thresh
> > > separately.
> > >
> > > Please open up the file met-6.0/data/config/README and search
for
> > > "out_thresh" for some info on this.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
Affiliate
> via
> > > RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268
>
> > > >
> > > > Hi John,
> > > >
> > > > So, I have a file that has matched pairs, the *mpr file. But,
for
> each
> > > > run, there is also the *.stat file. But, I'm guessing that
contrains
> > the
> > > > matched pairs as well. I thought all output contains matched
pairs,
> > no?
> > > >
> > > > If I re-ran STAT_Analysis, what do I need as the input to
create new
> > > > thresholds? The original files?
> > > >
> > > > Roz
> > > >
> > > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Roz,
> > > > >
> > > > > Good question. Please take a look at the output you
generated
> during
> > > > each
> > > > > run of Point-Stat. If the output contains the matched pair
(MPR)
> > > output
> > > > > lines, then the answer is no, you do not need to rerun
Point-Stat.
> > > > > Instead, you can just run STAT-Analysis to apply a new set
of
> > > thresholds.
> > > > >
> > > > > If you do not have MPR output lines, then yes, you'd need to
rerun
> > > > > Point-Stat with your updated set of thresholds.
> > > > >
> > > > > Yes, thresholds can be defined using floating points. They
> certainly
> > > > don't
> > > > > need to be integers.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken - NOAA
> Affiliate
> > > via
> > > > > RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > > > > > Transaction: Ticket created by rosalyn.maccracken at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: question on re-running data with new
thresholds
> > > > > > Owner: Nobody
> > > > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > Ticket/Display.html?id=83268
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I've run a series of days with POINT_STAT with one set of
> > thresholds
> > > > > > indicated in the configuration file. I need to change
these
> > > > thresholds,
> > > > > > and re-run my data. Do I need to start again with the
original
> > > files I
> > > > > > used (GFS hourly output and ASCAT from the prepbufr), or
is
> there a
> > > way
> > > > > to
> > > > > > re-run data with the *stat files?
> > > > > >
> > > > > > Also, can I use floating numbers as thresholds, or do the
> > thresholds
> > > > have
> > > > > > to be integers?
> > > > > >
> > > > > > Thanks in advance!
> > > > > >
> > > > > > Roz
> > > > > >
> > > > > > --
> > > > > > Rosalyn MacCracken
> > > > > > Support Scientist
> > > > > >
> > > > > > Ocean Applilcations Branch
> > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > NCWCP
> > > > > > 5830 University Research Ct
> > > > > > College Park, MD 20740-3818
> > > > > >
> > > > > > (p) 301-683-1551
> > > > > > rosalyn.maccracken at noaa.gov
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Rosalyn MacCracken
> > > > Support Scientist
> > > >
> > > > Ocean Applilcations Branch
> > > > NOAA/NWS Ocean Prediction Center
> > > > NCWCP
> > > > 5830 University Research Ct
> > > > College Park, MD 20740-3818
> > > >
> > > > (p) 301-683-1551
> > > > rosalyn.maccracken at noaa.gov
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Rosalyn MacCracken
> > Support Scientist
> >
> > Ocean Applilcations Branch
> > NOAA/NWS Ocean Prediction Center
> > NCWCP
> > 5830 University Research Ct
> > College Park, MD 20740-3818
> >
> > (p) 301-683-1551
> > rosalyn.maccracken at noaa.gov
> >
> >
>
>
--
Rosalyn MacCracken
Support Scientist
Ocean Applilcations Branch
NOAA/NWS Ocean Prediction Center
NCWCP
5830 University Research Ct
College Park, MD 20740-3818
(p) 301-683-1551
rosalyn.maccracken at noaa.gov
------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Thu Dec 14 09:18:52 2017
Roz,
Let's say that you have the following settings in your config file:
++++++++++++++++++
cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0 ];
...
output_flag = {
ctc = STAT;
mctc = STAT;
}
++++++++++++++++++
So you've defined 7 categorical thresholds and have both CTC and MCTC
output turned on.
In the output .stat file you'll see7 output lines for CTC, and 1
output
line to MCTC.
The 7 CTC lines are for the 2x2 contingency tables you get when
applying
the 7 thresholds listed separately to the data. Let's say you have
10,000
matched pairs. Those 10,000 matched pairs will be processed into a
CTC
line 7 times, once for each threshold.
The 1 MCTC line represents the 8x8 contingency table you get when
applying
all 7 of these thresholds as the same time. The 7 thresholds define
the
following 8 bins:
[-infinity, 0), [0, 8), [8, 17), [17, 28), [28, 34), [34, 48), [48,
64),
[64, +infinity]
The ']' versus ')' notation means that the endpoint is/is not included
in
the interval. For > or >= type thresholds, MET knows to check the
thresholds left-to-right. It goes into the bin corresponding to the
first
thresholds that evaluates to true. You don't need to worry about 8
versus
8.01... instead just choose your thresholds as type '>' or '>=' to
handle
the end-points.
Thanks,
John
On Thu, Dec 14, 2017 at 6:35 AM, Rosalyn MacCracken - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>
> hi John,
>
> Thanks for the explanation. I have no idea if they will upgrade
MET. I'm
> still waiting on them fixing that international dateline issue. In
the
> meantime, I've found a work-around for that, and the conversion
thing.
>
> Ok, so, when I do either:
> cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> ... or ...
> cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
>
> the bins don't overlap, correct? In other words, if I use either of
these
> cat_thresh statements, I have a value of 8.0, the point will only be
put in
> the bin 0.0-8.0, and MET is smart enough not to put the point in
both
> 0.0-8.0 and 8.0-17.0, correct?
>
> So, how what do the bins look like if I use either of these
statements?
> Something like:
>
> for the top, with >=
> 0-8.0
> 8.01 - 17.0
> 17.01 - 28.0
> etc
>
> and for the bottom, with >
> 0-7.999
> 8.0-16.999
> 17-27.999
> etc
>
> Or, something different?
>
> Thanks,
> Roz
>
> On Wed, Dec 13, 2017 at 9:24 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Roz,
> >
> > Two points to make here.
> >
> > First, we released met-6.1 last week and it now includes a new
feature
> > named "convert(x)". This enables users to define a conversion
function
> for
> > their data. It sounds like you'd like to convert m/s to kts, so
you'd
> use:
> > convert(x) = 1.94384449 * x;
> >
> > So for example...
> >
> > fcst = {
> > field = [ { name = "WIND"; level="Z10"; convert(x) = 1.94384449
* x;
> } ]
> > }
> >
> > If you upgrade to met-6.1, you'd have this new feature available
to you.
> >
> > Second, if you want to use MCTC and MCTS output lines from MET,
that
> error
> > message is correct. If you don't care about MCTC/MCTS output,
just turn
> > them off in the config file (mctc = NONE; mcts = NONE;).
> >
> > If you do want MCTC and MCTS output, then by convention, the
cat_thresh
> > array must be defined as non-overlapping thresholds using a
consistent
> > threshold type:
> >
> > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> > ... or ...
> > cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
> >
> > When applying the multi-category logic, Point-Stat and Grid-Stat
use
> these
> > thresholds to define which bin each point falls into.
> >
> > Hope that helps clarify.
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA
Affiliate via
> RT
> > <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > >
> > > Hi John,
> > >
> > > Thanks for the explanation. I'll have to take a look at the
README
> file
> > as
> > > soon as I solve this other issue. I know you know the answer to
this,
> > so,
> > > I will ask you (since I asked Julie a similar question, and she
said
> you
> > > would know better than her). :-)
> > >
> > > I have threshold ranges in my PointStat config file (right, I
already
> > said
> > > I did). I didn't make them correct in the beginning. I had
created my
> > > threshold in knots, instead of m/s. Anyway, I'm in the process
of
> > > correcting this. I need ranges in m/s to match the Beaufort
Scale,
> which
> > > is:
> > > 0.0-10.99
> > > 11.0-16.99
> > > 17.0-21.99
> > > etc
> > >
> > > in other words, if I wrote this out in an equation, it would be:
> > > >=0.0 && < 11.0
> > > >=11.0 && < 17.0
> > > >= 17.0 && < 22.0
> > > etc.
> > >
> > > I tried to do this in the config file (remember, this is m/s,
since
> there
> > > is nowhere to have it convert automatically to knots in the
config
> > > file...at least I didn't think so...):
> > > wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
> > > >=8.74556&&<=11.3126,...
> > >
> > > but, I got this error:
> > > ERROR : check_mctc_thresh() -> when verifying using multi-
category
> > > contingency tables, the thresholds must be monotonically
increasing and
> > be
> > > of the same inequality type (lt, le, gt, or ge).
> > >
> > > So, how do I do a range in the config file, without having the
> categories
> > > overlap? I was doing it this way:
> > > wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> > > but, that would cause the categories to overlap like:
> > > 0-8
> > > 8-17
> > > 17-28
> > > etc, right?
> > >
> > > Roz
> > >
> > > On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Roz,
> > > >
> > > > Each time your run Point-Stat, it always writes out a .stat
file.
> The
> > > line
> > > > types that are written to that file are controlled by the
settings in
> > the
> > > > "output_flag" config file option. Take a look there are the
mpr
> > setting:
> > > > mpr = BOTH;
> > > >
> > > > When mpr is set to STAT, it is written to the .stat output
file.
> > > > When mpr is set to BOTH, it is written to **both** the .stat
output
> > file
> > > > and an ascii output file named *_mpr.txt.
> > > >
> > > > So the MPR data in the .stat and _mpr.txt output files are
> duplicates.
> > > I'd
> > > > suggest setting "mpr = STAT" to avoid writing duplicate data,
> > especially
> > > > since the MPR data is so copious.
> > > >
> > > > You have MPR data and you'd like to run it through STAT-
Analysis to
> > > > generate stats. But the question is how are you running
> stat_analysis?
> > > >
> > > > For example, perhaps you're applying categorical thresholds to
> define a
> > > 2x2
> > > > contingency table and computing stats. You can define the
output
> > > > thresholds like this (assuming 2 meter temperature):
> > > >
> > > > stat_analysis -lookin mydata.stat -job aggregate_stat
-line_type MPR
> > > > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh
gt273
> > > >
> > > > The -out_thresh option defines both the forecast and
observation
> > > > thresholds. Or you can specify -out_fcst_thresh and
-out_obs_thresh
> > > > separately.
> > > >
> > > > Please open up the file met-6.0/data/config/README and search
for
> > > > "out_thresh" for some info on this.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
Affiliate
> > via
> > > > RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > > So, I have a file that has matched pairs, the *mpr file.
But, for
> > each
> > > > > run, there is also the *.stat file. But, I'm guessing that
> contrains
> > > the
> > > > > matched pairs as well. I thought all output contains
matched
> pairs,
> > > no?
> > > > >
> > > > > If I re-ran STAT_Analysis, what do I need as the input to
create
> new
> > > > > thresholds? The original files?
> > > > >
> > > > > Roz
> > > > >
> > > > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Roz,
> > > > > >
> > > > > > Good question. Please take a look at the output you
generated
> > during
> > > > > each
> > > > > > run of Point-Stat. If the output contains the matched
pair (MPR)
> > > > output
> > > > > > lines, then the answer is no, you do not need to rerun
> Point-Stat.
> > > > > > Instead, you can just run STAT-Analysis to apply a new set
of
> > > > thresholds.
> > > > > >
> > > > > > If you do not have MPR output lines, then yes, you'd need
to
> rerun
> > > > > > Point-Stat with your updated set of thresholds.
> > > > > >
> > > > > > Yes, thresholds can be defined using floating points.
They
> > certainly
> > > > > don't
> > > > > > need to be integers.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken -
NOAA
> > Affiliate
> > > > via
> > > > > > RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted upon.
> > > > > > > Transaction: Ticket created by
rosalyn.maccracken at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: question on re-running data with new
thresholds
> > > > > > > Owner: Nobody
> > > > > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > Ticket/Display.html?id=83268
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I've run a series of days with POINT_STAT with one set
of
> > > thresholds
> > > > > > > indicated in the configuration file. I need to change
these
> > > > > thresholds,
> > > > > > > and re-run my data. Do I need to start again with the
original
> > > > files I
> > > > > > > used (GFS hourly output and ASCAT from the prepbufr), or
is
> > there a
> > > > way
> > > > > > to
> > > > > > > re-run data with the *stat files?
> > > > > > >
> > > > > > > Also, can I use floating numbers as thresholds, or do
the
> > > thresholds
> > > > > have
> > > > > > > to be integers?
> > > > > > >
> > > > > > > Thanks in advance!
> > > > > > >
> > > > > > > Roz
> > > > > > >
> > > > > > > --
> > > > > > > Rosalyn MacCracken
> > > > > > > Support Scientist
> > > > > > >
> > > > > > > Ocean Applilcations Branch
> > > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > > NCWCP
> > > > > > > 5830 University Research Ct
> > > > > > > College Park, MD 20740-3818
> > > > > > >
> > > > > > > (p) 301-683-1551
> > > > > > > rosalyn.maccracken at noaa.gov
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Rosalyn MacCracken
> > > > > Support Scientist
> > > > >
> > > > > Ocean Applilcations Branch
> > > > > NOAA/NWS Ocean Prediction Center
> > > > > NCWCP
> > > > > 5830 University Research Ct
> > > > > College Park, MD 20740-3818
> > > > >
> > > > > (p) 301-683-1551
> > > > > rosalyn.maccracken at noaa.gov
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Rosalyn MacCracken
> > > Support Scientist
> > >
> > > Ocean Applilcations Branch
> > > NOAA/NWS Ocean Prediction Center
> > > NCWCP
> > > 5830 University Research Ct
> > > College Park, MD 20740-3818
> > >
> > > (p) 301-683-1551
> > > rosalyn.maccracken at noaa.gov
> > >
> > >
> >
> >
>
>
> --
> Rosalyn MacCracken
> Support Scientist
>
> Ocean Applilcations Branch
> NOAA/NWS Ocean Prediction Center
> NCWCP
> 5830 University Research Ct
> College Park, MD 20740-3818
>
> (p) 301-683-1551
> rosalyn.maccracken at noaa.gov
>
>
------------------------------------------------
Subject: question on re-running data with new thresholds
From: Rosalyn MacCracken - NOAA Affiliate
Time: Thu Dec 14 09:41:28 2017
Ah, got it! So, I do want to use the >= like the example you gave me,
so,
I don't include the endpoint in the interval. Good, then I did it
right
originally.
Oh, so I think they are putting in a helpticket (on my end) to upgrade
to
6.1. If they do upgrade, that will take care of the international
dateline
issue, correct? So, there will be no need to patch that, correct?
Thanks for your help!
Roz
On Thu, Dec 14, 2017 at 4:18 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Roz,
>
> Let's say that you have the following settings in your config file:
> ++++++++++++++++++
> cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0, >=64.0
];
> ...
> output_flag = {
> ctc = STAT;
> mctc = STAT;
> }
> ++++++++++++++++++
>
> So you've defined 7 categorical thresholds and have both CTC and
MCTC
> output turned on.
>
> In the output .stat file you'll see7 output lines for CTC, and 1
output
> line to MCTC.
>
> The 7 CTC lines are for the 2x2 contingency tables you get when
applying
> the 7 thresholds listed separately to the data. Let's say you have
10,000
> matched pairs. Those 10,000 matched pairs will be processed into a
CTC
> line 7 times, once for each threshold.
>
> The 1 MCTC line represents the 8x8 contingency table you get when
applying
> all 7 of these thresholds as the same time. The 7 thresholds define
the
> following 8 bins:
> [-infinity, 0), [0, 8), [8, 17), [17, 28), [28, 34), [34, 48),
[48, 64),
> [64, +infinity]
>
> The ']' versus ')' notation means that the endpoint is/is not
included in
> the interval. For > or >= type thresholds, MET knows to check the
> thresholds left-to-right. It goes into the bin corresponding to the
first
> thresholds that evaluates to true. You don't need to worry about 8
versus
> 8.01... instead just choose your thresholds as type '>' or '>=' to
handle
> the end-points.
>
> Thanks,
> John
>
>
> On Thu, Dec 14, 2017 at 6:35 AM, Rosalyn MacCracken - NOAA Affiliate
via RT
> <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> >
> > hi John,
> >
> > Thanks for the explanation. I have no idea if they will upgrade
MET.
> I'm
> > still waiting on them fixing that international dateline issue.
In the
> > meantime, I've found a work-around for that, and the conversion
thing.
> >
> > Ok, so, when I do either:
> > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> > ... or ...
> > cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0 ];
> >
> > the bins don't overlap, correct? In other words, if I use either
of
> these
> > cat_thresh statements, I have a value of 8.0, the point will only
be put
> in
> > the bin 0.0-8.0, and MET is smart enough not to put the point in
both
> > 0.0-8.0 and 8.0-17.0, correct?
> >
> > So, how what do the bins look like if I use either of these
statements?
> > Something like:
> >
> > for the top, with >=
> > 0-8.0
> > 8.01 - 17.0
> > 17.01 - 28.0
> > etc
> >
> > and for the bottom, with >
> > 0-7.999
> > 8.0-16.999
> > 17-27.999
> > etc
> >
> > Or, something different?
> >
> > Thanks,
> > Roz
> >
> > On Wed, Dec 13, 2017 at 9:24 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Roz,
> > >
> > > Two points to make here.
> > >
> > > First, we released met-6.1 last week and it now includes a new
feature
> > > named "convert(x)". This enables users to define a conversion
function
> > for
> > > their data. It sounds like you'd like to convert m/s to kts, so
you'd
> > use:
> > > convert(x) = 1.94384449 * x;
> > >
> > > So for example...
> > >
> > > fcst = {
> > > field = [ { name = "WIND"; level="Z10"; convert(x) =
1.94384449 * x;
> > } ]
> > > }
> > >
> > > If you upgrade to met-6.1, you'd have this new feature available
to
> you.
> > >
> > > Second, if you want to use MCTC and MCTS output lines from MET,
that
> > error
> > > message is correct. If you don't care about MCTC/MCTS output,
just
> turn
> > > them off in the config file (mctc = NONE; mcts = NONE;).
> > >
> > > If you do want MCTC and MCTS output, then by convention, the
cat_thresh
> > > array must be defined as non-overlapping thresholds using a
consistent
> > > threshold type:
> > >
> > > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0
> ];
> > > ... or ...
> > > cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0
];
> > >
> > > When applying the multi-category logic, Point-Stat and Grid-Stat
use
> > these
> > > thresholds to define which bin each point falls into.
> > >
> > > Hope that helps clarify.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA
Affiliate
> via
> > RT
> > > <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268
>
> > > >
> > > > Hi John,
> > > >
> > > > Thanks for the explanation. I'll have to take a look at the
README
> > file
> > > as
> > > > soon as I solve this other issue. I know you know the answer
to
> this,
> > > so,
> > > > I will ask you (since I asked Julie a similar question, and
she said
> > you
> > > > would know better than her). :-)
> > > >
> > > > I have threshold ranges in my PointStat config file (right, I
already
> > > said
> > > > I did). I didn't make them correct in the beginning. I had
created
> my
> > > > threshold in knots, instead of m/s. Anyway, I'm in the
process of
> > > > correcting this. I need ranges in m/s to match the Beaufort
Scale,
> > which
> > > > is:
> > > > 0.0-10.99
> > > > 11.0-16.99
> > > > 17.0-21.99
> > > > etc
> > > >
> > > > in other words, if I wrote this out in an equation, it would
be:
> > > > >=0.0 && < 11.0
> > > > >=11.0 && < 17.0
> > > > >= 17.0 && < 22.0
> > > > etc.
> > > >
> > > > I tried to do this in the config file (remember, this is m/s,
since
> > there
> > > > is nowhere to have it convert automatically to knots in the
config
> > > > file...at least I didn't think so...):
> > > > wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
> > > > >=8.74556&&<=11.3126,...
> > > >
> > > > but, I got this error:
> > > > ERROR : check_mctc_thresh() -> when verifying using multi-
category
> > > > contingency tables, the thresholds must be monotonically
increasing
> and
> > > be
> > > > of the same inequality type (lt, le, gt, or ge).
> > > >
> > > > So, how do I do a range in the config file, without having the
> > categories
> > > > overlap? I was doing it this way:
> > > > wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0
> ];
> > > > but, that would cause the categories to overlap like:
> > > > 0-8
> > > > 8-17
> > > > 17-28
> > > > etc, right?
> > > >
> > > > Roz
> > > >
> > > > On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Roz,
> > > > >
> > > > > Each time your run Point-Stat, it always writes out a .stat
file.
> > The
> > > > line
> > > > > types that are written to that file are controlled by the
settings
> in
> > > the
> > > > > "output_flag" config file option. Take a look there are the
mpr
> > > setting:
> > > > > mpr = BOTH;
> > > > >
> > > > > When mpr is set to STAT, it is written to the .stat output
file.
> > > > > When mpr is set to BOTH, it is written to **both** the .stat
output
> > > file
> > > > > and an ascii output file named *_mpr.txt.
> > > > >
> > > > > So the MPR data in the .stat and _mpr.txt output files are
> > duplicates.
> > > > I'd
> > > > > suggest setting "mpr = STAT" to avoid writing duplicate
data,
> > > especially
> > > > > since the MPR data is so copious.
> > > > >
> > > > > You have MPR data and you'd like to run it through STAT-
Analysis to
> > > > > generate stats. But the question is how are you running
> > stat_analysis?
> > > > >
> > > > > For example, perhaps you're applying categorical thresholds
to
> > define a
> > > > 2x2
> > > > > contingency table and computing stats. You can define the
output
> > > > > thresholds like this (assuming 2 meter temperature):
> > > > >
> > > > > stat_analysis -lookin mydata.stat -job aggregate_stat
-line_type
> MPR
> > > > > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh
gt273
> > > > >
> > > > > The -out_thresh option defines both the forecast and
observation
> > > > > thresholds. Or you can specify -out_fcst_thresh and
> -out_obs_thresh
> > > > > separately.
> > > > >
> > > > > Please open up the file met-6.0/data/config/README and
search for
> > > > > "out_thresh" for some info on this.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken - NOAA
> Affiliate
> > > via
> > > > > RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > > > > >
> > > > > > Hi John,
> > > > > >
> > > > > > So, I have a file that has matched pairs, the *mpr file.
But,
> for
> > > each
> > > > > > run, there is also the *.stat file. But, I'm guessing
that
> > contrains
> > > > the
> > > > > > matched pairs as well. I thought all output contains
matched
> > pairs,
> > > > no?
> > > > > >
> > > > > > If I re-ran STAT_Analysis, what do I need as the input to
create
> > new
> > > > > > thresholds? The original files?
> > > > > >
> > > > > > Roz
> > > > > >
> > > > > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > > Roz,
> > > > > > >
> > > > > > > Good question. Please take a look at the output you
generated
> > > during
> > > > > > each
> > > > > > > run of Point-Stat. If the output contains the matched
pair
> (MPR)
> > > > > output
> > > > > > > lines, then the answer is no, you do not need to rerun
> > Point-Stat.
> > > > > > > Instead, you can just run STAT-Analysis to apply a new
set of
> > > > > thresholds.
> > > > > > >
> > > > > > > If you do not have MPR output lines, then yes, you'd
need to
> > rerun
> > > > > > > Point-Stat with your updated set of thresholds.
> > > > > > >
> > > > > > > Yes, thresholds can be defined using floating points.
They
> > > certainly
> > > > > > don't
> > > > > > > need to be integers.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken -
NOAA
> > > Affiliate
> > > > > via
> > > > > > > RT <met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted
upon.
> > > > > > > > Transaction: Ticket created by
rosalyn.maccracken at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: question on re-running data with new
thresholds
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > Ticket/Display.html?id=83268
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I've run a series of days with POINT_STAT with one set
of
> > > > thresholds
> > > > > > > > indicated in the configuration file. I need to change
these
> > > > > > thresholds,
> > > > > > > > and re-run my data. Do I need to start again with the
> original
> > > > > files I
> > > > > > > > used (GFS hourly output and ASCAT from the prepbufr),
or is
> > > there a
> > > > > way
> > > > > > > to
> > > > > > > > re-run data with the *stat files?
> > > > > > > >
> > > > > > > > Also, can I use floating numbers as thresholds, or do
the
> > > > thresholds
> > > > > > have
> > > > > > > > to be integers?
> > > > > > > >
> > > > > > > > Thanks in advance!
> > > > > > > >
> > > > > > > > Roz
> > > > > > > >
> > > > > > > > --
> > > > > > > > Rosalyn MacCracken
> > > > > > > > Support Scientist
> > > > > > > >
> > > > > > > > Ocean Applilcations Branch
> > > > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > > > NCWCP
> > > > > > > > 5830 University Research Ct
> > > > > > > > College Park, MD 20740-3818
> > > > > > > >
> > > > > > > > (p) 301-683-1551
> > > > > > > > rosalyn.maccracken at noaa.gov
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Rosalyn MacCracken
> > > > > > Support Scientist
> > > > > >
> > > > > > Ocean Applilcations Branch
> > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > NCWCP
> > > > > > 5830 University Research Ct
> > > > > > College Park, MD 20740-3818
> > > > > >
> > > > > > (p) 301-683-1551
> > > > > > rosalyn.maccracken at noaa.gov
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Rosalyn MacCracken
> > > > Support Scientist
> > > >
> > > > Ocean Applilcations Branch
> > > > NOAA/NWS Ocean Prediction Center
> > > > NCWCP
> > > > 5830 University Research Ct
> > > > College Park, MD 20740-3818
> > > >
> > > > (p) 301-683-1551
> > > > rosalyn.maccracken at noaa.gov
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Rosalyn MacCracken
> > Support Scientist
> >
> > Ocean Applilcations Branch
> > NOAA/NWS Ocean Prediction Center
> > NCWCP
> > 5830 University Research Ct
> > College Park, MD 20740-3818
> >
> > (p) 301-683-1551
> > rosalyn.maccracken at noaa.gov
> >
> >
>
>
--
Rosalyn MacCracken
Support Scientist
Ocean Applilcations Branch
NOAA/NWS Ocean Prediction Center
NCWCP
5830 University Research Ct
College Park, MD 20740-3818
(p) 301-683-1551
rosalyn.maccracken at noaa.gov
------------------------------------------------
Subject: question on re-running data with new thresholds
From: John Halley Gotway
Time: Thu Dec 14 10:06:51 2017
Roz,
All of the patches for previous releases are included in the current
release. So *assuming* the patch actually fixes the problem you were
seeing, then yes, met-6.1 will fix it.
But please let me know if you find otherwise.
Thanks,
John
On Thu, Dec 14, 2017 at 9:41 AM, Rosalyn MacCracken - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
>
> Ah, got it! So, I do want to use the >= like the example you gave
me, so,
> I don't include the endpoint in the interval. Good, then I did it
right
> originally.
>
> Oh, so I think they are putting in a helpticket (on my end) to
upgrade to
> 6.1. If they do upgrade, that will take care of the international
dateline
> issue, correct? So, there will be no need to patch that, correct?
>
> Thanks for your help!
>
> Roz
>
> On Thu, Dec 14, 2017 at 4:18 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Roz,
> >
> > Let's say that you have the following settings in your config
file:
> > ++++++++++++++++++
> > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0 ];
> > ...
> > output_flag = {
> > ctc = STAT;
> > mctc = STAT;
> > }
> > ++++++++++++++++++
> >
> > So you've defined 7 categorical thresholds and have both CTC and
MCTC
> > output turned on.
> >
> > In the output .stat file you'll see7 output lines for CTC, and 1
output
> > line to MCTC.
> >
> > The 7 CTC lines are for the 2x2 contingency tables you get when
applying
> > the 7 thresholds listed separately to the data. Let's say you
have
> 10,000
> > matched pairs. Those 10,000 matched pairs will be processed into
a CTC
> > line 7 times, once for each threshold.
> >
> > The 1 MCTC line represents the 8x8 contingency table you get when
> applying
> > all 7 of these thresholds as the same time. The 7 thresholds
define the
> > following 8 bins:
> > [-infinity, 0), [0, 8), [8, 17), [17, 28), [28, 34), [34, 48),
[48,
> 64),
> > [64, +infinity]
> >
> > The ']' versus ')' notation means that the endpoint is/is not
included in
> > the interval. For > or >= type thresholds, MET knows to check the
> > thresholds left-to-right. It goes into the bin corresponding to
the
> first
> > thresholds that evaluates to true. You don't need to worry about
8
> versus
> > 8.01... instead just choose your thresholds as type '>' or '>=' to
handle
> > the end-points.
> >
> > Thanks,
> > John
> >
> >
> > On Thu, Dec 14, 2017 at 6:35 AM, Rosalyn MacCracken - NOAA
Affiliate via
> RT
> > <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > >
> > > hi John,
> > >
> > > Thanks for the explanation. I have no idea if they will upgrade
MET.
> > I'm
> > > still waiting on them fixing that international dateline issue.
In the
> > > meantime, I've found a work-around for that, and the conversion
thing.
> > >
> > > Ok, so, when I do either:
> > > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0, >=48.0,
>=64.0
> ];
> > > ... or ...
> > > cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0, >64.0
];
> > >
> > > the bins don't overlap, correct? In other words, if I use
either of
> > these
> > > cat_thresh statements, I have a value of 8.0, the point will
only be
> put
> > in
> > > the bin 0.0-8.0, and MET is smart enough not to put the point in
both
> > > 0.0-8.0 and 8.0-17.0, correct?
> > >
> > > So, how what do the bins look like if I use either of these
statements?
> > > Something like:
> > >
> > > for the top, with >=
> > > 0-8.0
> > > 8.01 - 17.0
> > > 17.01 - 28.0
> > > etc
> > >
> > > and for the bottom, with >
> > > 0-7.999
> > > 8.0-16.999
> > > 17-27.999
> > > etc
> > >
> > > Or, something different?
> > >
> > > Thanks,
> > > Roz
> > >
> > > On Wed, Dec 13, 2017 at 9:24 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Roz,
> > > >
> > > > Two points to make here.
> > > >
> > > > First, we released met-6.1 last week and it now includes a new
> feature
> > > > named "convert(x)". This enables users to define a conversion
> function
> > > for
> > > > their data. It sounds like you'd like to convert m/s to kts,
so
> you'd
> > > use:
> > > > convert(x) = 1.94384449 * x;
> > > >
> > > > So for example...
> > > >
> > > > fcst = {
> > > > field = [ { name = "WIND"; level="Z10"; convert(x) =
1.94384449 *
> x;
> > > } ]
> > > > }
> > > >
> > > > If you upgrade to met-6.1, you'd have this new feature
available to
> > you.
> > > >
> > > > Second, if you want to use MCTC and MCTS output lines from
MET, that
> > > error
> > > > message is correct. If you don't care about MCTC/MCTS output,
just
> > turn
> > > > them off in the config file (mctc = NONE; mcts = NONE;).
> > > >
> > > > If you do want MCTC and MCTS output, then by convention, the
> cat_thresh
> > > > array must be defined as non-overlapping thresholds using a
> consistent
> > > > threshold type:
> > > >
> > > > cat_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0,
>=48.0,
> >=64.0
> > ];
> > > > ... or ...
> > > > cat_thresh = [ >0.0, >8.0, >17.0, >28.0, >34.0, >48.0,
>64.0 ];
> > > >
> > > > When applying the multi-category logic, Point-Stat and Grid-
Stat use
> > > these
> > > > thresholds to define which bin each point falls into.
> > > >
> > > > Hope that helps clarify.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Wed, Dec 13, 2017 at 1:31 PM, Rosalyn MacCracken - NOAA
Affiliate
> > via
> > > RT
> > > > <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > > Thanks for the explanation. I'll have to take a look at the
README
> > > file
> > > > as
> > > > > soon as I solve this other issue. I know you know the
answer to
> > this,
> > > > so,
> > > > > I will ask you (since I asked Julie a similar question, and
she
> said
> > > you
> > > > > would know better than her). :-)
> > > > >
> > > > > I have threshold ranges in my PointStat config file (right,
I
> already
> > > > said
> > > > > I did). I didn't make them correct in the beginning. I had
> created
> > my
> > > > > threshold in knots, instead of m/s. Anyway, I'm in the
process of
> > > > > correcting this. I need ranges in m/s to match the Beaufort
Scale,
> > > which
> > > > > is:
> > > > > 0.0-10.99
> > > > > 11.0-16.99
> > > > > 17.0-21.99
> > > > > etc
> > > > >
> > > > > in other words, if I wrote this out in an equation, it would
be:
> > > > > >=0.0 && < 11.0
> > > > > >=11.0 && < 17.0
> > > > > >= 17.0 && < 22.0
> > > > > etc.
> > > > >
> > > > > I tried to do this in the config file (remember, this is
m/s, since
> > > there
> > > > > is nowhere to have it convert automatically to knots in the
config
> > > > > file...at least I didn't think so...):
> > > > > wind_thresh = [ >=0.0&&<=5.65374,>=5.6589&&<=8.7404,
> > > > > >=8.74556&&<=11.3126,...
> > > > >
> > > > > but, I got this error:
> > > > > ERROR : check_mctc_thresh() -> when verifying using multi-
category
> > > > > contingency tables, the thresholds must be monotonically
increasing
> > and
> > > > be
> > > > > of the same inequality type (lt, le, gt, or ge).
> > > > >
> > > > > So, how do I do a range in the config file, without having
the
> > > categories
> > > > > overlap? I was doing it this way:
> > > > > wind_thresh = [ >=0.0, >=8.0, >=17.0, >=28.0, >=34.0,
>=48.0,
> >=64.0
> > ];
> > > > > but, that would cause the categories to overlap like:
> > > > > 0-8
> > > > > 8-17
> > > > > 17-28
> > > > > etc, right?
> > > > >
> > > > > Roz
> > > > >
> > > > > On Mon, Dec 11, 2017 at 9:45 PM, John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Roz,
> > > > > >
> > > > > > Each time your run Point-Stat, it always writes out a
.stat file.
> > > The
> > > > > line
> > > > > > types that are written to that file are controlled by the
> settings
> > in
> > > > the
> > > > > > "output_flag" config file option. Take a look there are
the mpr
> > > > setting:
> > > > > > mpr = BOTH;
> > > > > >
> > > > > > When mpr is set to STAT, it is written to the .stat output
file.
> > > > > > When mpr is set to BOTH, it is written to **both** the
.stat
> output
> > > > file
> > > > > > and an ascii output file named *_mpr.txt.
> > > > > >
> > > > > > So the MPR data in the .stat and _mpr.txt output files are
> > > duplicates.
> > > > > I'd
> > > > > > suggest setting "mpr = STAT" to avoid writing duplicate
data,
> > > > especially
> > > > > > since the MPR data is so copious.
> > > > > >
> > > > > > You have MPR data and you'd like to run it through STAT-
Analysis
> to
> > > > > > generate stats. But the question is how are you running
> > > stat_analysis?
> > > > > >
> > > > > > For example, perhaps you're applying categorical
thresholds to
> > > define a
> > > > > 2x2
> > > > > > contingency table and computing stats. You can define the
output
> > > > > > thresholds like this (assuming 2 meter temperature):
> > > > > >
> > > > > > stat_analysis -lookin mydata.stat -job aggregate_stat
-line_type
> > MPR
> > > > > > -out_line_type CTC -fcst_var TMP -fcst_lev Z2 -out_thresh
gt273
> > > > > >
> > > > > > The -out_thresh option defines both the forecast and
observation
> > > > > > thresholds. Or you can specify -out_fcst_thresh and
> > -out_obs_thresh
> > > > > > separately.
> > > > > >
> > > > > > Please open up the file met-6.0/data/config/README and
search for
> > > > > > "out_thresh" for some info on this.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > > On Mon, Dec 11, 2017 at 11:30 AM, Rosalyn MacCracken -
NOAA
> > Affiliate
> > > > via
> > > > > > RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83268
> >
> > > > > > >
> > > > > > > Hi John,
> > > > > > >
> > > > > > > So, I have a file that has matched pairs, the *mpr file.
But,
> > for
> > > > each
> > > > > > > run, there is also the *.stat file. But, I'm guessing
that
> > > contrains
> > > > > the
> > > > > > > matched pairs as well. I thought all output contains
matched
> > > pairs,
> > > > > no?
> > > > > > >
> > > > > > > If I re-ran STAT_Analysis, what do I need as the input
to
> create
> > > new
> > > > > > > thresholds? The original files?
> > > > > > >
> > > > > > > Roz
> > > > > > >
> > > > > > > On Mon, Dec 11, 2017 at 6:24 PM, John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > > Roz,
> > > > > > > >
> > > > > > > > Good question. Please take a look at the output you
> generated
> > > > during
> > > > > > > each
> > > > > > > > run of Point-Stat. If the output contains the matched
pair
> > (MPR)
> > > > > > output
> > > > > > > > lines, then the answer is no, you do not need to rerun
> > > Point-Stat.
> > > > > > > > Instead, you can just run STAT-Analysis to apply a new
set of
> > > > > > thresholds.
> > > > > > > >
> > > > > > > > If you do not have MPR output lines, then yes, you'd
need to
> > > rerun
> > > > > > > > Point-Stat with your updated set of thresholds.
> > > > > > > >
> > > > > > > > Yes, thresholds can be defined using floating points.
They
> > > > certainly
> > > > > > > don't
> > > > > > > > need to be integers.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > > On Mon, Dec 11, 2017 at 11:16 AM, Rosalyn MacCracken -
NOAA
> > > > Affiliate
> > > > > > via
> > > > > > > > RT <met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Mon Dec 11 11:16:29 2017: Request 83268 was acted
upon.
> > > > > > > > > Transaction: Ticket created by
rosalyn.maccracken at noaa.gov
> > > > > > > > > Queue: met_help
> > > > > > > > > Subject: question on re-running data with new
> thresholds
> > > > > > > > > Owner: Nobody
> > > > > > > > > Requestors: rosalyn.maccracken at noaa.gov
> > > > > > > > > Status: new
> > > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > > Ticket/Display.html?id=83268
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I've run a series of days with POINT_STAT with one
set of
> > > > > thresholds
> > > > > > > > > indicated in the configuration file. I need to
change
> these
> > > > > > > thresholds,
> > > > > > > > > and re-run my data. Do I need to start again with
the
> > original
> > > > > > files I
> > > > > > > > > used (GFS hourly output and ASCAT from the
prepbufr), or is
> > > > there a
> > > > > > way
> > > > > > > > to
> > > > > > > > > re-run data with the *stat files?
> > > > > > > > >
> > > > > > > > > Also, can I use floating numbers as thresholds, or
do the
> > > > > thresholds
> > > > > > > have
> > > > > > > > > to be integers?
> > > > > > > > >
> > > > > > > > > Thanks in advance!
> > > > > > > > >
> > > > > > > > > Roz
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Rosalyn MacCracken
> > > > > > > > > Support Scientist
> > > > > > > > >
> > > > > > > > > Ocean Applilcations Branch
> > > > > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > > > > NCWCP
> > > > > > > > > 5830 University Research Ct
> > > > > > > > > College Park, MD 20740-3818
> > > > > > > > >
> > > > > > > > > (p) 301-683-1551
> > > > > > > > > rosalyn.maccracken at noaa.gov
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Rosalyn MacCracken
> > > > > > > Support Scientist
> > > > > > >
> > > > > > > Ocean Applilcations Branch
> > > > > > > NOAA/NWS Ocean Prediction Center
> > > > > > > NCWCP
> > > > > > > 5830 University Research Ct
> > > > > > > College Park, MD 20740-3818
> > > > > > >
> > > > > > > (p) 301-683-1551
> > > > > > > rosalyn.maccracken at noaa.gov
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Rosalyn MacCracken
> > > > > Support Scientist
> > > > >
> > > > > Ocean Applilcations Branch
> > > > > NOAA/NWS Ocean Prediction Center
> > > > > NCWCP
> > > > > 5830 University Research Ct
> > > > > College Park, MD 20740-3818
> > > > >
> > > > > (p) 301-683-1551
> > > > > rosalyn.maccracken at noaa.gov
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Rosalyn MacCracken
> > > Support Scientist
> > >
> > > Ocean Applilcations Branch
> > > NOAA/NWS Ocean Prediction Center
> > > NCWCP
> > > 5830 University Research Ct
> > > College Park, MD 20740-3818
> > >
> > > (p) 301-683-1551
> > > rosalyn.maccracken at noaa.gov
> > >
> > >
> >
> >
>
>
> --
> Rosalyn MacCracken
> Support Scientist
>
> Ocean Applilcations Branch
> NOAA/NWS Ocean Prediction Center
> NCWCP
> 5830 University Research Ct
> College Park, MD 20740-3818
>
> (p) 301-683-1551
> rosalyn.maccracken at noaa.gov
>
>
------------------------------------------------
More information about the Met_help
mailing list