[Met_help] [rt.rap.ucar.edu #97309] History for Mode output interpretation
John Halley Gotway via RT
met_help at ucar.edu
Tue Nov 17 10:47:32 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello there,
I have tried to use MODE to evaluate precipitation from WRF output with
Stage-IV. I have a few confusions about the outputs stated below:
1. In the ASCII contingency table statistics file, what is the
difference between N_Valid and Totals? Aren't both of these supposed to be
the total number of grids?
2. for different conv_thres, why the totals are different? to be more
specific why the total increased with an increased threshold?
3. How do they define the object field? I am guessing by convolution
radius and threshold. In that case are the raw in cts table and postscripts
files are different?
It would be really great if you could help me with the answer to my
confusion. Please let me know if you need anything else to understand my
confusion.
Bests,
Ummul
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Mode output interpretation
From: John Halley Gotway
Time: Tue Nov 03 09:24:50 2020
Ummul,
I see you have some questions about the MODE output file. Please see
answers inline below.
> 1. In the ASCII contingency table statistics file, what is the
> difference between N_Valid and Totals? Aren't both of these
supposed to
> be
> the total number of grids?
>
The output file format is defined here:
https://dtcenter.github.io/MET/latest/Users_Guide/mode.html#id4
Comparing N_VALID to TOTAL in the CTS output line, there is no
difference.
Those should always be the same number. N_VALID is the count of grid
points
at which both the forecast and observation fields contain valid data
values. TOTAL is the number of grid points whose values are included
in the
computed contingency table. Those should result in the same counts.
We chose to write it twice because N_VALID is included in the common
set of
MODE header columns that is used for the CTS and object output files.
And
TOTAL is a column that appears in the CTC output line type from the
other
MET tools.
> 2. for different conv_thres, why the totals are different? to be
more
> specific why the total increased with an increased threshold?
>
That should not be happening. This is apparently a bug in MODE. When
applying multiple radii and threshold, MODE is failing to reset the
contingency table counts back to zero each time as it should. Instead,
it
appears that the TOTAL column is being populated with multiples of the
original TOTAL counts. I'll write up a bugfix issue for this and get
the
logic patched.
> 3. How do they define the object field? I am guessing by
convolution
> radius and threshold. In that case are the raw in cts table and
> postscripts
> files are different?
>
There are 2 main output files from MODE that should be used. The
PostScript
file shows a visual depiction of the objects defined and how well the
forecast and observation objects compare to each other. When getting
started with MODE, I'd recommend adjusting the convolution radius and
threshold to see what impact it has on the objects defined. It's
really up
to the user to define the scale of the features they're trying to
evaluate.
Generally, if you hold the radius fixed, increasing the threshold will
make
objects smaller, while decreasing it makes objects bigger. Or holding
the
threshold fixed, increasing the radius makes objects smaller (more
smoothing), while decreasing it makes objects bigger (less smoothing).
So try playing around with those settings and view the objects in the
PostScript file to see the result.
The ASCII files ending in "_obj.txt" ASCII files contain all the
individual
and pair object attributes. Lots of good details in there.
The NetCDF output file provides gridded object fields.
And the "_cts.txt" file is really a secondary output. It provides the
contingency table scores that you would have gotten by doing a
tradition
grid point to grid point scoring of the data. It's meant as a
reference.
The whole idea of MODE is that it provides a much more diagnostic
evaluation of the data that a traditional grid-point-by-grid-point
comparison provides.
>
> It would be really great if you could help me with the answer to my
> confusion. Please let me know if you need anything else to
understand my
> confusion.
>
> Bests,
> Ummul
>
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon
> >
> Virus-free.
> www.avast.com
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link
> >
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
>
------------------------------------------------
Subject: Mode output interpretation
From: Ummul Khaira
Time: Wed Nov 04 09:00:06 2020
Thank you for the detailed explanation. So, would you please inform me
when
the bug is fixed for the "_cts.txt" file? In the meantime what do you
suggest? Should I use MODE every time for each convolution threshold?
Bests,
Ummul
<https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On Tue, Nov 3, 2020 at 11:24 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> *Message sent from a system outside of UConn.*
>
>
> Ummul,
>
> I see you have some questions about the MODE output file. Please see
> answers inline below.
>
>
> > 1. In the ASCII contingency table statistics file, what is the
> > difference between N_Valid and Totals? Aren't both of these
supposed
> to
> > be
> > the total number of grids?
> >
>
> The output file format is defined here:
> https://dtcenter.github.io/MET/latest/Users_Guide/mode.html#id4
>
> Comparing N_VALID to TOTAL in the CTS output line, there is no
difference.
> Those should always be the same number. N_VALID is the count of grid
points
> at which both the forecast and observation fields contain valid data
> values. TOTAL is the number of grid points whose values are included
in the
> computed contingency table. Those should result in the same counts.
>
> We chose to write it twice because N_VALID is included in the common
set of
> MODE header columns that is used for the CTS and object output
files. And
> TOTAL is a column that appears in the CTC output line type from the
other
> MET tools.
>
>
> > 2. for different conv_thres, why the totals are different? to
be more
> > specific why the total increased with an increased threshold?
> >
>
> That should not be happening. This is apparently a bug in MODE. When
> applying multiple radii and threshold, MODE is failing to reset the
> contingency table counts back to zero each time as it should.
Instead, it
> appears that the TOTAL column is being populated with multiples of
the
> original TOTAL counts. I'll write up a bugfix issue for this and get
the
> logic patched.
>
>
> > 3. How do they define the object field? I am guessing by
convolution
> > radius and threshold. In that case are the raw in cts table and
> > postscripts
> > files are different?
> >
>
> There are 2 main output files from MODE that should be used. The
PostScript
> file shows a visual depiction of the objects defined and how well
the
> forecast and observation objects compare to each other. When getting
> started with MODE, I'd recommend adjusting the convolution radius
and
> threshold to see what impact it has on the objects defined. It's
really up
> to the user to define the scale of the features they're trying to
evaluate.
> Generally, if you hold the radius fixed, increasing the threshold
will make
> objects smaller, while decreasing it makes objects bigger. Or
holding the
> threshold fixed, increasing the radius makes objects smaller (more
> smoothing), while decreasing it makes objects bigger (less
smoothing).
>
> So try playing around with those settings and view the objects in
the
> PostScript file to see the result.
>
> The ASCII files ending in "_obj.txt" ASCII files contain all the
individual
> and pair object attributes. Lots of good details in there.
>
> The NetCDF output file provides gridded object fields.
>
> And the "_cts.txt" file is really a secondary output. It provides
the
> contingency table scores that you would have gotten by doing a
tradition
> grid point to grid point scoring of the data. It's meant as a
reference.
> The whole idea of MODE is that it provides a much more diagnostic
> evaluation of the data that a traditional grid-point-by-grid-point
> comparison provides.
>
>
> >
> > It would be really great if you could help me with the answer to
my
> > confusion. Please let me know if you need anything else to
understand my
> > confusion.
> >
> > Bests,
> > Ummul
> >
> > <
> >
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon
> > >
> > Virus-free.
> > www.avast.com
> > <
> >
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link
> > >
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
> >
>
>
<https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
------------------------------------------------
Subject: Mode output interpretation
From: John Halley Gotway
Time: Wed Nov 04 09:23:55 2020
Ummul,
FYI, here's the issue describing the problem:
https://github.com/dtcenter/MET/issues/1547
Here's the pull requests to fix for MET version 9.1:
https://github.com/dtcenter/MET/pull/1549
I'm waiting for my colleague to review/approve the change. But this is
literally a one line fix to mode_exec.cc:
https://github.com/dtcenter/MET/pull/1549/files
Your options are:
(1) Do nothing and don't use the CTS output from MODE at all.
(2) Wait for the met-9.1.1 bugfix release that will include this fix.
(3) Manually patch the source code of your local version of met-9.1
and
recompile it to include this fix.
(4) Reconfigure your calls to MODE to process a single convolution
radius
and threshold in each call.
It's up to you to decide how you'd like to proceed.
Thanks,
John
On Wed, Nov 4, 2020 at 9:00 AM Ummul Khaira via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97309 >
>
> Thank you for the detailed explanation. So, would you please inform
me when
> the bug is fixed for the "_cts.txt" file? In the meantime what do
you
> suggest? Should I use MODE every time for each convolution
threshold?
>
> Bests,
> Ummul
>
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon
> >
> Virus-free.
> www.avast.com
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link
> >
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> On Tue, Nov 3, 2020 at 11:24 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > *Message sent from a system outside of UConn.*
> >
> >
> > Ummul,
> >
> > I see you have some questions about the MODE output file. Please
see
> > answers inline below.
> >
> >
> > > 1. In the ASCII contingency table statistics file, what is
the
> > > difference between N_Valid and Totals? Aren't both of these
supposed
> > to
> > > be
> > > the total number of grids?
> > >
> >
> > The output file format is defined here:
> > https://dtcenter.github.io/MET/latest/Users_Guide/mode.html#id4
> >
> > Comparing N_VALID to TOTAL in the CTS output line, there is no
> difference.
> > Those should always be the same number. N_VALID is the count of
grid
> points
> > at which both the forecast and observation fields contain valid
data
> > values. TOTAL is the number of grid points whose values are
included in
> the
> > computed contingency table. Those should result in the same
counts.
> >
> > We chose to write it twice because N_VALID is included in the
common set
> of
> > MODE header columns that is used for the CTS and object output
files. And
> > TOTAL is a column that appears in the CTC output line type from
the other
> > MET tools.
> >
> >
> > > 2. for different conv_thres, why the totals are different? to
be
> more
> > > specific why the total increased with an increased threshold?
> > >
> >
> > That should not be happening. This is apparently a bug in MODE.
When
> > applying multiple radii and threshold, MODE is failing to reset
the
> > contingency table counts back to zero each time as it should.
Instead, it
> > appears that the TOTAL column is being populated with multiples of
the
> > original TOTAL counts. I'll write up a bugfix issue for this and
get the
> > logic patched.
> >
> >
> > > 3. How do they define the object field? I am guessing by
convolution
> > > radius and threshold. In that case are the raw in cts table
and
> > > postscripts
> > > files are different?
> > >
> >
> > There are 2 main output files from MODE that should be used. The
> PostScript
> > file shows a visual depiction of the objects defined and how well
the
> > forecast and observation objects compare to each other. When
getting
> > started with MODE, I'd recommend adjusting the convolution radius
and
> > threshold to see what impact it has on the objects defined. It's
really
> up
> > to the user to define the scale of the features they're trying to
> evaluate.
> > Generally, if you hold the radius fixed, increasing the threshold
will
> make
> > objects smaller, while decreasing it makes objects bigger. Or
holding the
> > threshold fixed, increasing the radius makes objects smaller (more
> > smoothing), while decreasing it makes objects bigger (less
smoothing).
> >
> > So try playing around with those settings and view the objects in
the
> > PostScript file to see the result.
> >
> > The ASCII files ending in "_obj.txt" ASCII files contain all the
> individual
> > and pair object attributes. Lots of good details in there.
> >
> > The NetCDF output file provides gridded object fields.
> >
> > And the "_cts.txt" file is really a secondary output. It provides
the
> > contingency table scores that you would have gotten by doing a
tradition
> > grid point to grid point scoring of the data. It's meant as a
reference.
> > The whole idea of MODE is that it provides a much more diagnostic
> > evaluation of the data that a traditional grid-point-by-grid-point
> > comparison provides.
> >
> >
> > >
> > > It would be really great if you could help me with the answer to
my
> > > confusion. Please let me know if you need anything else to
understand
> my
> > > confusion.
> > >
> > > Bests,
> > > Ummul
> > >
> > > <
> > >
> >
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon
> > > >
> > > Virus-free.
> > > www.avast.com
> > > <
> > >
> >
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link
> > > >
> > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> > >
> > >
> >
> >
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=icon
> >
> Virus-free.
> www.avast.com
> <
> https://www.avast.com/sig-
email?utm_medium=email&utm_source=link&utm_campaign=sig-
email&utm_content=webmail&utm_term=link
> >
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
>
------------------------------------------------
More information about the Met_help
mailing list