[Met_help] [rt.rap.ucar.edu #97136] History for innumerable masks!

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Good morning,

I've been exploring ways to create various masks with some success.  Right
now I produce the following masks each day:
-dominant land use type (24 files - manageable)
-dominant land use type by region (sum of the number of land use types for
each region (e.g. SWC, WEST, etc. - approaching unmanageable)
-cloud fraction (10 * fh * run - very unmanageable).

The last point prompted me to send this email.  Right now, for each
experiment and forecast hour, I generate a cloud mask for clear skies, if
the grid is between 0 to 10 % occupied, if the grid square is greater than
10% occupied but less than 20% occupied, all the way to 100%.  This brings
us to about 10 files for each forecast hour.  However, since I'm running 6
experiments out to 72 hours, you can now see that I would easily be
generating over 1000 files each time (:-/).

I'm trying to use the cloud fraction field to do the following:  x-axis is
the bin cloud fraction (i.e. cloud fraction = 0; cloud fraction between 0
and 10, etc.) while y-axis would be the bias of a field (e.g. PM2.5 or
ozone) collocated with obs.  I'd do this for all geographical regions
ideally, realizing that not every bin would be populated.  In some
instances an entire region might be under clear skies.  This may be a bit
of a pipe dream, but can anyone see the feasibility of attempting this, or
is this beyond the scope of what can currently be done in an efficient
manner?

-In short, I'm wondering what I can do to cut down on these files (lump
each fraction into one file for all forecast hours?).
-Would I be able to limit the point stat_config to just 10 pathway lines?
Right now I have things stored in the following way each day:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
Here, para6 ($model) is the model while 20201010 is the date ($DATE).
In PointStat_AIRNOW - the file I use stored in my met_config - I would look
something like this:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
However, as indicated by the pasted example, you can see the files and how
extensive they could be:










*cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
 cfrac_12_15_gt0p4_le0p5.nc <http://cfrac_12_15_gt0p4_le0p5.nc>
 cfrac_12_62_gt0p8_le0p9.nc
<http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
<http://cfrac_06_39_gt0p1_le0p2.nc>  cfrac_12_15_gt0p5_le0p6.nc
<http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
<http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
<http://cfrac_06_39_gt0p2_le0p3.nc>  cfrac_12_15_gt0p6_le0p7.nc
<http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
<http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
<http://cfrac_06_39_gt0p3_le0p4.nc>  cfrac_12_15_gt0p7_le0p8.nc
<http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
<http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
<http://cfrac_06_39_gt0p4_le0p5.nc>  cfrac_12_15_gt0p8_le0p9.nc
<http://cfrac_12_15_gt0p8_le0p9.nc>  cfrac_12_63_gt0p1_le0p2.nc
<http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
<http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
<http://cfrac_12_15_gt0p9.nc>        cfrac_12_63_gt0p2_le0p3.nc
<http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
<http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
<http://cfrac_12_16_0.nc>            cfrac_12_63_gt0p3_le0p4.nc
<http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
<http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
<http://cfrac_12_16_gt0_le0p1.nc>    cfrac_12_63_gt0p4_le0p5.nc
<http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
<http://cfrac_06_39_gt0p8_le0p9.nc>  cfrac_12_16_gt0p1_le0p2.nc
<http://cfrac_12_16_gt0p1_le0p2.nc>  cfrac_12_63_gt0p5_le0p6.nc
<http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
<http://cfrac_06_39_gt0p9.nc>        cfrac_12_16_gt0p2_le0p3.nc
<http://cfrac_12_16_gt0p2_le0p3.nc>  cfrac_12_63_gt0p6_le0p7.nc
<http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
<http://cfrac_06_40_0.nc>            cfrac_12_16_gt0p3_le0p4.nc
<http://cfrac_12_16_gt0p3_le0p4.nc>  cfrac_12_63_gt0p7_le0p8.nc
<http://cfrac_12_63_gt0p7_le0p8.nc>*

The naming convention of the file would then be something like this in the
config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc

In the past, when I tried to process 3D fields stored in netcdf files, I
found multiple dimensional arrays greater than 2 (horizontal dimensions) to
be a problem.  If I attempted to consolidate these files then I would have
a 4D matrix that would resemble the following:  (cloud mask, time, lon,
lat).  I'm not sure that MET would like that. Am I right that this could be
a problem?

Thanks in advance to any insight you can provide.

-- 
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: innumerable masks!
From: Julie Prestopnik
Time: Tue Oct 20 09:13:32 2020

Hi Edward.

I have assigned this ticket to John HG.  Please allow a few business
days
for a response.

Julie

On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> Transaction: Ticket created by edward.strobach at noaa.gov
>        Queue: met_help
>      Subject: innumerable masks!
>        Owner: Nobody
>   Requestors: edward.strobach at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>
>
> Good morning,
>
> I've been exploring ways to create various masks with some success.
Right
> now I produce the following masks each day:
> -dominant land use type (24 files - manageable)
> -dominant land use type by region (sum of the number of land use
types for
> each region (e.g. SWC, WEST, etc. - approaching unmanageable)
> -cloud fraction (10 * fh * run - very unmanageable).
>
> The last point prompted me to send this email.  Right now, for each
> experiment and forecast hour, I generate a cloud mask for clear
skies, if
> the grid is between 0 to 10 % occupied, if the grid square is
greater than
> 10% occupied but less than 20% occupied, all the way to 100%.  This
brings
> us to about 10 files for each forecast hour.  However, since I'm
running 6
> experiments out to 72 hours, you can now see that I would easily be
> generating over 1000 files each time (:-/).
>
> I'm trying to use the cloud fraction field to do the following:  x-
axis is
> the bin cloud fraction (i.e. cloud fraction = 0; cloud fraction
between 0
> and 10, etc.) while y-axis would be the bias of a field (e.g. PM2.5
or
> ozone) collocated with obs.  I'd do this for all geographical
regions
> ideally, realizing that not every bin would be populated.  In some
> instances an entire region might be under clear skies.  This may be
a bit
> of a pipe dream, but can anyone see the feasibility of attempting
this, or
> is this beyond the scope of what can currently be done in an
efficient
> manner?
>
> -In short, I'm wondering what I can do to cut down on these files
(lump
> each fraction into one file for all forecast hours?).
> -Would I be able to limit the point stat_config to just 10 pathway
lines?
> Right now I have things stored in the following way each day:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> Here, para6 ($model) is the model while 20201010 is the date
($DATE).
> In PointStat_AIRNOW - the file I use stored in my met_config - I
would look
> something like this:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> However, as indicated by the pasted example, you can see the files
and how
> extensive they could be:
>
>
>
>
>
>
>
>
>
>
> *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
>  cfrac_12_15_gt0p4_le0p5.nc <http://cfrac_12_15_gt0p4_le0p5.nc>
>  cfrac_12_62_gt0p8_le0p9.nc
> <http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> <http://cfrac_06_39_gt0p1_le0p2.nc>  cfrac_12_15_gt0p5_le0p6.nc
> <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> <http://cfrac_06_39_gt0p2_le0p3.nc>  cfrac_12_15_gt0p6_le0p7.nc
> <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> <http://cfrac_06_39_gt0p3_le0p4.nc>  cfrac_12_15_gt0p7_le0p8.nc
> <http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
> <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> <http://cfrac_06_39_gt0p4_le0p5.nc>  cfrac_12_15_gt0p8_le0p9.nc
> <http://cfrac_12_15_gt0p8_le0p9.nc>  cfrac_12_63_gt0p1_le0p2.nc
> <http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> <http://cfrac_12_15_gt0p9.nc>        cfrac_12_63_gt0p2_le0p3.nc
> <http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> <http://cfrac_12_16_0.nc>            cfrac_12_63_gt0p3_le0p4.nc
> <http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> <http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
> <http://cfrac_12_16_gt0_le0p1.nc>    cfrac_12_63_gt0p4_le0p5.nc
> <http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> <http://cfrac_06_39_gt0p8_le0p9.nc>  cfrac_12_16_gt0p1_le0p2.nc
> <http://cfrac_12_16_gt0p1_le0p2.nc>  cfrac_12_63_gt0p5_le0p6.nc
> <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> <http://cfrac_06_39_gt0p9.nc>        cfrac_12_16_gt0p2_le0p3.nc
> <http://cfrac_12_16_gt0p2_le0p3.nc>  cfrac_12_63_gt0p6_le0p7.nc
> <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> <http://cfrac_06_40_0.nc>            cfrac_12_16_gt0p3_le0p4.nc
> <http://cfrac_12_16_gt0p3_le0p4.nc>  cfrac_12_63_gt0p7_le0p8.nc
> <http://cfrac_12_63_gt0p7_le0p8.nc>*
>
> The naming convention of the file would then be something like this
in the
> config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
>
> In the past, when I tried to process 3D fields stored in netcdf
files, I
> found multiple dimensional arrays greater than 2 (horizontal
dimensions) to
> be a problem.  If I attempted to consolidate these files then I
would have
> a 4D matrix that would resemble the following:  (cloud mask, time,
lon,
> lat).  I'm not sure that MET would like that. Am I right that this
could be
> a problem?
>
> Thanks in advance to any insight you can provide.
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>

--
Julie Prestopnik (she/her/hers)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: innumerable masks!
From: John Halley Gotway
Time: Tue Oct 20 10:46:36 2020

Hi Ed,

I see you have some questions about masking. I’m answering on my phone
right now so I can’t provide specific details. But let me know if
anything
isn’t clear and I can follow up with specific examples later.

I read the description of the plot you’d like to create and believe
that it
can be accomplished without the use of mask regions.

But before proceeding to that, let me point out that you can apply
data
masks in point_stat WITHOUT running gen_vx_mask first. However for
“complex” masks (e.g. combining 2 masks, like SWD with land use type)
you
do need to run gen_vx_mask first. We could conceivably enhance MET to
support the generation of complex masks on the fly in the config file
to
avoid the need for running gen_vx_mask. So let me know if you think
that’s
worth documenting as a feature request.

But the next part depends on what type of “bias” you’re looking for.
Either
multiplicative bias... ie avg forecast value - avg observed value from
the
CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
from the CTS line? Which are you looking for?

Thanks,
John

On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>
> Hi Edward.
>
> I have assigned this ticket to John HG.  Please allow a few business
days
> for a response.
>
> Julie
>
> On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > Transaction: Ticket created by edward.strobach at noaa.gov
> >        Queue: met_help
> >      Subject: innumerable masks!
> >        Owner: Nobody
> >   Requestors: edward.strobach at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >
> >
> > Good morning,
> >
> > I've been exploring ways to create various masks with some
success.
> Right
> > now I produce the following masks each day:
> > -dominant land use type (24 files - manageable)
> > -dominant land use type by region (sum of the number of land use
types
> for
> > each region (e.g. SWC, WEST, etc. - approaching unmanageable)
> > -cloud fraction (10 * fh * run - very unmanageable).
> >
> > The last point prompted me to send this email.  Right now, for
each
> > experiment and forecast hour, I generate a cloud mask for clear
skies, if
> > the grid is between 0 to 10 % occupied, if the grid square is
greater
> than
> > 10% occupied but less than 20% occupied, all the way to 100%.
This
> brings
> > us to about 10 files for each forecast hour.  However, since I'm
running
> 6
> > experiments out to 72 hours, you can now see that I would easily
be
> > generating over 1000 files each time (:-/).
> >
> > I'm trying to use the cloud fraction field to do the following:
x-axis
> is
> > the bin cloud fraction (i.e. cloud fraction = 0; cloud fraction
between 0
> > and 10, etc.) while y-axis would be the bias of a field (e.g.
PM2.5 or
> > ozone) collocated with obs.  I'd do this for all geographical
regions
> > ideally, realizing that not every bin would be populated.  In some
> > instances an entire region might be under clear skies.  This may
be a bit
> > of a pipe dream, but can anyone see the feasibility of attempting
this,
> or
> > is this beyond the scope of what can currently be done in an
efficient
> > manner?
> >
> > -In short, I'm wondering what I can do to cut down on these files
(lump
> > each fraction into one file for all forecast hours?).
> > -Would I be able to limit the point stat_config to just 10 pathway
lines?
> > Right now I have things stored in the following way each day:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > Here, para6 ($model) is the model while 20201010 is the date
($DATE).
> > In PointStat_AIRNOW - the file I use stored in my met_config - I
would
> look
> > something like this:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > However, as indicated by the pasted example, you can see the files
and
> how
> > extensive they could be:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
> >  cfrac_12_15_gt0p4_le0p5.nc <http://cfrac_12_15_gt0p4_le0p5.nc>
> >  cfrac_12_62_gt0p8_le0p9.nc
> > <http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > <http://cfrac_06_39_gt0p1_le0p2.nc>  cfrac_12_15_gt0p5_le0p6.nc
> > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > <http://cfrac_06_39_gt0p2_le0p3.nc>  cfrac_12_15_gt0p6_le0p7.nc
> > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > <http://cfrac_06_39_gt0p3_le0p4.nc>  cfrac_12_15_gt0p7_le0p8.nc
> > <http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
> > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > <http://cfrac_06_39_gt0p4_le0p5.nc>  cfrac_12_15_gt0p8_le0p9.nc
> > <http://cfrac_12_15_gt0p8_le0p9.nc>  cfrac_12_63_gt0p1_le0p2.nc
> > <http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> > <http://cfrac_12_15_gt0p9.nc>        cfrac_12_63_gt0p2_le0p3.nc
> > <http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > <http://cfrac_12_16_0.nc>            cfrac_12_63_gt0p3_le0p4.nc
> > <http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > <http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
> > <http://cfrac_12_16_gt0_le0p1.nc>    cfrac_12_63_gt0p4_le0p5.nc
> > <http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > <http://cfrac_06_39_gt0p8_le0p9.nc>  cfrac_12_16_gt0p1_le0p2.nc
> > <http://cfrac_12_16_gt0p1_le0p2.nc>  cfrac_12_63_gt0p5_le0p6.nc
> > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > <http://cfrac_06_39_gt0p9.nc>        cfrac_12_16_gt0p2_le0p3.nc
> > <http://cfrac_12_16_gt0p2_le0p3.nc>  cfrac_12_63_gt0p6_le0p7.nc
> > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > <http://cfrac_06_40_0.nc>            cfrac_12_16_gt0p3_le0p4.nc
> > <http://cfrac_12_16_gt0p3_le0p4.nc>  cfrac_12_63_gt0p7_le0p8.nc
> > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> >
> > The naming convention of the file would then be something like
this in
> the
> > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> >
> > In the past, when I tried to process 3D fields stored in netcdf
files, I
> > found multiple dimensional arrays greater than 2 (horizontal
dimensions)
> to
> > be a problem.  If I attempted to consolidate these files then I
would
> have
> > a 4D matrix that would resemble the following:  (cloud mask, time,
lon,
> > lat).  I'm not sure that MET would like that. Am I right that this
could
> be
> > a problem?
> >
> > Thanks in advance to any insight you can provide.
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
> --
> Julie Prestopnik (she/her/hers)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day.  Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: innumerable masks!
From: Edward Strobach - NOAA Affiliate
Time: Tue Oct 20 11:59:41 2020

Hi John,

See my responses below

Hi Ed,

I see you have some questions about masking. I’m answering on my phone
right now so I can’t provide specific details. But let me know if
anything
isn’t clear and I can follow up with specific examples later.

I read the description of the plot you’d like to create and believe
that it
can be accomplished without the use of mask regions.

That would save a lot of space and complexity if I could do that

But before proceeding to that, let me point out that you can apply
data
masks in point_stat WITHOUT running gen_vx_mask first. However for
“complex” masks (e.g. combining 2 masks, like SWD with land use type)
you
do need to run gen_vx_mask first. We could conceivably enhance MET to
support the generation of complex masks on the fly in the config file
to
avoid the need for running gen_vx_mask. So let me know if you think
that’s
worth documenting as a feature request.

The application of masks without gen_vx_mask sounds vaguely familiar.
Maybe it was mentioned in an old email.  I'll try to look for that.  I
have
used the intersection tool to do this.  I found that I had to run
gen_vx_mask twice:  one to generate a netcdf file of the cropped out
region
and another to identify land use type in that region.  Yes, I think
anything that can reduce specifying numerous pathways in the PointStat
file
under the mask definition would be a great alleviation.  I support
making a
note of that.  Thanks.

But the next part depends on what type of “bias” you’re looking for.
Either
multiplicative bias... ie avg forecast value - avg observed value from
the
CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
from the CTS line? Which are you looking for?

I was leaning toward multiplicative bias.  The idea would be to bin
out a
range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2, and so
on)
on the X-axis with a multiplicative bias on the y-axis.  It might be
more
visually appealing if I used the bar graph option for that which I
know MET
can do.   Let me know if there is more information that you need from
me so
I can help with this.  Thanks

On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Ed,
>
> I see you have some questions about masking. I’m answering on my
phone
> right now so I can’t provide specific details. But let me know if
anything
> isn’t clear and I can follow up with specific examples later.
>
> I read the description of the plot you’d like to create and believe
that it
> can be accomplished without the use of mask regions.
>
> But before proceeding to that, let me point out that you can apply
data
> masks in point_stat WITHOUT running gen_vx_mask first. However for
> “complex” masks (e.g. combining 2 masks, like SWD with land use
type) you
> do need to run gen_vx_mask first. We could conceivably enhance MET
to
> support the generation of complex masks on the fly in the config
file to
> avoid the need for running gen_vx_mask. So let me know if you think
that’s
> worth documenting as a feature request.
>
> But the next part depends on what type of “bias” you’re looking for.
Either
> multiplicative bias... ie avg forecast value - avg observed value
from the
> CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
> from the CTS line? Which are you looking for?
>
> Thanks,
> John
>
> On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >
> > Hi Edward.
> >
> > I have assigned this ticket to John HG.  Please allow a few
business days
> > for a response.
> >
> > Julie
> >
> > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > > Transaction: Ticket created by edward.strobach at noaa.gov
> > >        Queue: met_help
> > >      Subject: innumerable masks!
> > >        Owner: Nobody
> > >   Requestors: edward.strobach at noaa.gov
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> >
> > >
> > >
> > > Good morning,
> > >
> > > I've been exploring ways to create various masks with some
success.
> > Right
> > > now I produce the following masks each day:
> > > -dominant land use type (24 files - manageable)
> > > -dominant land use type by region (sum of the number of land use
types
> > for
> > > each region (e.g. SWC, WEST, etc. - approaching unmanageable)
> > > -cloud fraction (10 * fh * run - very unmanageable).
> > >
> > > The last point prompted me to send this email.  Right now, for
each
> > > experiment and forecast hour, I generate a cloud mask for clear
skies,
> if
> > > the grid is between 0 to 10 % occupied, if the grid square is
greater
> > than
> > > 10% occupied but less than 20% occupied, all the way to 100%.
This
> > brings
> > > us to about 10 files for each forecast hour.  However, since I'm
> running
> > 6
> > > experiments out to 72 hours, you can now see that I would easily
be
> > > generating over 1000 files each time (:-/).
> > >
> > > I'm trying to use the cloud fraction field to do the following:
x-axis
> > is
> > > the bin cloud fraction (i.e. cloud fraction = 0; cloud fraction
> between 0
> > > and 10, etc.) while y-axis would be the bias of a field (e.g.
PM2.5 or
> > > ozone) collocated with obs.  I'd do this for all geographical
regions
> > > ideally, realizing that not every bin would be populated.  In
some
> > > instances an entire region might be under clear skies.  This may
be a
> bit
> > > of a pipe dream, but can anyone see the feasibility of
attempting this,
> > or
> > > is this beyond the scope of what can currently be done in an
efficient
> > > manner?
> > >
> > > -In short, I'm wondering what I can do to cut down on these
files (lump
> > > each fraction into one file for all forecast hours?).
> > > -Would I be able to limit the point stat_config to just 10
pathway
> lines?
> > > Right now I have things stored in the following way each day:
> > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > Here, para6 ($model) is the model while 20201010 is the date
($DATE).
> > > In PointStat_AIRNOW - the file I use stored in my met_config - I
would
> > look
> > > something like this:
> > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > However, as indicated by the pasted example, you can see the
files and
> > how
> > > extensive they could be:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
> > >  cfrac_12_15_gt0p4_le0p5.nc <http://cfrac_12_15_gt0p4_le0p5.nc>
> > >  cfrac_12_62_gt0p8_le0p9.nc
> > > <http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > > <http://cfrac_06_39_gt0p1_le0p2.nc>  cfrac_12_15_gt0p5_le0p6.nc
> > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > <http://cfrac_06_39_gt0p2_le0p3.nc>  cfrac_12_15_gt0p6_le0p7.nc
> > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > <http://cfrac_06_39_gt0p3_le0p4.nc>  cfrac_12_15_gt0p7_le0p8.nc
> > > <http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
> > > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > > <http://cfrac_06_39_gt0p4_le0p5.nc>  cfrac_12_15_gt0p8_le0p9.nc
> > > <http://cfrac_12_15_gt0p8_le0p9.nc>  cfrac_12_63_gt0p1_le0p2.nc
> > > <http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> > > <http://cfrac_12_15_gt0p9.nc>        cfrac_12_63_gt0p2_le0p3.nc
> > > <http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > > <http://cfrac_12_16_0.nc>            cfrac_12_63_gt0p3_le0p4.nc
> > > <http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > > <http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
> > > <http://cfrac_12_16_gt0_le0p1.nc>    cfrac_12_63_gt0p4_le0p5.nc
> > > <http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > > <http://cfrac_06_39_gt0p8_le0p9.nc>  cfrac_12_16_gt0p1_le0p2.nc
> > > <http://cfrac_12_16_gt0p1_le0p2.nc>  cfrac_12_63_gt0p5_le0p6.nc
> > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > <http://cfrac_06_39_gt0p9.nc>        cfrac_12_16_gt0p2_le0p3.nc
> > > <http://cfrac_12_16_gt0p2_le0p3.nc>  cfrac_12_63_gt0p6_le0p7.nc
> > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > <http://cfrac_06_40_0.nc>            cfrac_12_16_gt0p3_le0p4.nc
> > > <http://cfrac_12_16_gt0p3_le0p4.nc>  cfrac_12_63_gt0p7_le0p8.nc
> > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > >
> > > The naming convention of the file would then be something like
this in
> > the
> > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > >
> > > In the past, when I tried to process 3D fields stored in netcdf
files,
> I
> > > found multiple dimensional arrays greater than 2 (horizontal
> dimensions)
> > to
> > > be a problem.  If I attempted to consolidate these files then I
would
> > have
> > > a 4D matrix that would resemble the following:  (cloud mask,
time, lon,
> > > lat).  I'm not sure that MET would like that. Am I right that
this
> could
> > be
> > > a problem?
> > >
> > > Thanks in advance to any insight you can provide.
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her/hers)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day.  Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717

------------------------------------------------
Subject: innumerable masks!
From: John Halley Gotway
Time: Tue Oct 20 13:03:28 2020

Ed,

The multiplicative bias statistic is called the mean error, or ME, in
the
CNT column. It's just the avg fcst value - avg obs value. But what you
want
to do is called "conditional continuous" verification. Rather than
computing stats over all the points, you want to condition the
verification
based on the value, whereas masking conditions based on the location.
You
control this using the "cnt_thresh" and "cnt_logic" config options.
You can
condition based on the forecast value, observation value, or both. For
simplicity, I'd recommend starting with the observation value.

So in the obs dictionary, define:
   cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3, >0.3&&<=0.4,
>0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8, >0.8&&<=0.9, >0.9
];

For each 1 CNT line from the output, you should now see 11: 1 with
OBS_THRESH = NA including all the points, and 10 more with OBS_THRESH
= the
bins listed above.

Now I haven't actually done this... but hopefully you can use
METviewer to
make the plot you described.

For frequency bias, the contents of the CTC, CTS, MCTC, and MCTS line
types
are determined by the cat_thresh setting. It could be interesting to
define
cat_thresh as shown below and look at the MCTS output line type:
   cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8, >.9 ];

The MCTS output is described here:
https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13

Hope that helps.

John

On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>
> Hi John,
>
> See my responses below
>
> Hi Ed,
>
> I see you have some questions about masking. I’m answering on my
phone
> right now so I can’t provide specific details. But let me know if
anything
> isn’t clear and I can follow up with specific examples later.
>
> I read the description of the plot you’d like to create and believe
that it
> can be accomplished without the use of mask regions.
>
> That would save a lot of space and complexity if I could do that
>
> But before proceeding to that, let me point out that you can apply
data
> masks in point_stat WITHOUT running gen_vx_mask first. However for
> “complex” masks (e.g. combining 2 masks, like SWD with land use
type) you
> do need to run gen_vx_mask first. We could conceivably enhance MET
to
> support the generation of complex masks on the fly in the config
file to
> avoid the need for running gen_vx_mask. So let me know if you think
that’s
> worth documenting as a feature request.
>
> The application of masks without gen_vx_mask sounds vaguely
familiar.
> Maybe it was mentioned in an old email.  I'll try to look for that.
I have
> used the intersection tool to do this.  I found that I had to run
> gen_vx_mask twice:  one to generate a netcdf file of the cropped out
region
> and another to identify land use type in that region.  Yes, I think
> anything that can reduce specifying numerous pathways in the
PointStat file
> under the mask definition would be a great alleviation.  I support
making a
> note of that.  Thanks.
>
> But the next part depends on what type of “bias” you’re looking for.
Either
> multiplicative bias... ie avg forecast value - avg observed value
from the
> CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
> from the CTS line? Which are you looking for?
>
> I was leaning toward multiplicative bias.  The idea would be to bin
out a
> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2, and
so on)
> on the X-axis with a multiplicative bias on the y-axis.  It might be
more
> visually appealing if I used the bar graph option for that which I
know MET
> can do.   Let me know if there is more information that you need
from me so
> I can help with this.  Thanks
>
> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Ed,
> >
> > I see you have some questions about masking. I’m answering on my
phone
> > right now so I can’t provide specific details. But let me know if
> anything
> > isn’t clear and I can follow up with specific examples later.
> >
> > I read the description of the plot you’d like to create and
believe that
> it
> > can be accomplished without the use of mask regions.
> >
> > But before proceeding to that, let me point out that you can apply
data
> > masks in point_stat WITHOUT running gen_vx_mask first. However for
> > “complex” masks (e.g. combining 2 masks, like SWD with land use
type) you
> > do need to run gen_vx_mask first. We could conceivably enhance MET
to
> > support the generation of complex masks on the fly in the config
file to
> > avoid the need for running gen_vx_mask. So let me know if you
think
> that’s
> > worth documenting as a feature request.
> >
> > But the next part depends on what type of “bias” you’re looking
for.
> Either
> > multiplicative bias... ie avg forecast value - avg observed value
from
> the
> > CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
> > from the CTS line? Which are you looking for?
> >
> > Thanks,
> > John
> >
> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > >
> > > Hi Edward.
> > >
> > > I have assigned this ticket to John HG.  Please allow a few
business
> days
> > > for a response.
> > >
> > > Julie
> > >
> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA Affiliate
via
> RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > > > Transaction: Ticket created by edward.strobach at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: innumerable masks!
> > > >        Owner: Nobody
> > > >   Requestors: edward.strobach at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > >
> > > >
> > > >
> > > > Good morning,
> > > >
> > > > I've been exploring ways to create various masks with some
success.
> > > Right
> > > > now I produce the following masks each day:
> > > > -dominant land use type (24 files - manageable)
> > > > -dominant land use type by region (sum of the number of land
use
> types
> > > for
> > > > each region (e.g. SWC, WEST, etc. - approaching unmanageable)
> > > > -cloud fraction (10 * fh * run - very unmanageable).
> > > >
> > > > The last point prompted me to send this email.  Right now, for
each
> > > > experiment and forecast hour, I generate a cloud mask for
clear
> skies,
> > if
> > > > the grid is between 0 to 10 % occupied, if the grid square is
greater
> > > than
> > > > 10% occupied but less than 20% occupied, all the way to 100%.
This
> > > brings
> > > > us to about 10 files for each forecast hour.  However, since
I'm
> > running
> > > 6
> > > > experiments out to 72 hours, you can now see that I would
easily be
> > > > generating over 1000 files each time (:-/).
> > > >
> > > > I'm trying to use the cloud fraction field to do the
following:
> x-axis
> > > is
> > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
fraction
> > between 0
> > > > and 10, etc.) while y-axis would be the bias of a field (e.g.
PM2.5
> or
> > > > ozone) collocated with obs.  I'd do this for all geographical
regions
> > > > ideally, realizing that not every bin would be populated.  In
some
> > > > instances an entire region might be under clear skies.  This
may be a
> > bit
> > > > of a pipe dream, but can anyone see the feasibility of
attempting
> this,
> > > or
> > > > is this beyond the scope of what can currently be done in an
> efficient
> > > > manner?
> > > >
> > > > -In short, I'm wondering what I can do to cut down on these
files
> (lump
> > > > each fraction into one file for all forecast hours?).
> > > > -Would I be able to limit the point stat_config to just 10
pathway
> > lines?
> > > > Right now I have things stored in the following way each day:
> > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > > Here, para6 ($model) is the model while 20201010 is the date
($DATE).
> > > > In PointStat_AIRNOW - the file I use stored in my met_config -
I
> would
> > > look
> > > > something like this:
> > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > > However, as indicated by the pasted example, you can see the
files
> and
> > > how
> > > > extensive they could be:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
> > > >  cfrac_12_15_gt0p4_le0p5.nc
<http://cfrac_12_15_gt0p4_le0p5.nc>
> > > >  cfrac_12_62_gt0p8_le0p9.nc
> > > > <http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
cfrac_12_15_gt0p5_le0p6.nc
> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
cfrac_12_15_gt0p6_le0p7.nc
> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
cfrac_12_15_gt0p7_le0p8.nc
> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
> > > > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
cfrac_12_15_gt0p8_le0p9.nc
> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
cfrac_12_63_gt0p1_le0p2.nc
> > > > <http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> > > > <http://cfrac_12_15_gt0p9.nc>
cfrac_12_63_gt0p2_le0p3.nc
> > > > <http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > > > <http://cfrac_12_16_0.nc>
cfrac_12_63_gt0p3_le0p4.nc
> > > > <http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
> > > > <http://cfrac_12_16_gt0_le0p1.nc>
cfrac_12_63_gt0p4_le0p5.nc
> > > > <http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
cfrac_12_16_gt0p1_le0p2.nc
> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
cfrac_12_63_gt0p5_le0p6.nc
> > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > > <http://cfrac_06_39_gt0p9.nc>
cfrac_12_16_gt0p2_le0p3.nc
> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
cfrac_12_63_gt0p6_le0p7.nc
> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > > <http://cfrac_06_40_0.nc>
cfrac_12_16_gt0p3_le0p4.nc
> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
cfrac_12_63_gt0p7_le0p8.nc
> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > > >
> > > > The naming convention of the file would then be something like
this
> in
> > > the
> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > > >
> > > > In the past, when I tried to process 3D fields stored in
netcdf
> files,
> > I
> > > > found multiple dimensional arrays greater than 2 (horizontal
> > dimensions)
> > > to
> > > > be a problem.  If I attempted to consolidate these files then
I would
> > > have
> > > > a 4D matrix that would resemble the following:  (cloud mask,
time,
> lon,
> > > > lat).  I'm not sure that MET would like that. Am I right that
this
> > could
> > > be
> > > > a problem?
> > > >
> > > > Thanks in advance to any insight you can provide.
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik (she/her/hers)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day.  Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>

------------------------------------------------
Subject: innumerable masks!
From: John Halley Gotway
Time: Tue Oct 20 13:15:01 2020

Ed,

FYI, I wrote up this GitHub issue:
https://github.com/dtcenter/MET/issues/1530

Please take a look and let me know what you think. I'm not sure what's
reasonable to be done here. I suppose in an ideal world, all the
functionality of gen_vx_mask would be available via the MET config
files,
and the only reason to define them one place versus another would be
efficiency. It's quicker to define it once and use it many times than
re-define each time. But that would be a pretty significant change.

John

On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway <johnhg at ucar.edu>
wrote:

> Ed,
>
> The multiplicative bias statistic is called the mean error, or ME,
in the
> CNT column. It's just the avg fcst value - avg obs value. But what
you want
> to do is called "conditional continuous" verification. Rather than
> computing stats over all the points, you want to condition the
verification
> based on the value, whereas masking conditions based on the
location.  You
> control this using the "cnt_thresh" and "cnt_logic" config options.
You can
> condition based on the forecast value, observation value, or both.
For
> simplicity, I'd recommend starting with the observation value.
>
> So in the obs dictionary, define:
>    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3, >0.3&&<=0.4,
> >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8, >0.8&&<=0.9,
>0.9 ];
>
> For each 1 CNT line from the output, you should now see 11: 1 with
> OBS_THRESH = NA including all the points, and 10 more with
OBS_THRESH = the
> bins listed above.
>
> Now I haven't actually done this... but hopefully you can use
METviewer to
> make the plot you described.
>
> For frequency bias, the contents of the CTC, CTS, MCTC, and MCTS
line
> types are determined by the cat_thresh setting. It could be
interesting to
> define cat_thresh as shown below and look at the MCTS output line
type:
>    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8, >.9 ];
>
> The MCTS output is described here:
>
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
>
> Hope that helps.
>
> John
>
> On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>>
>> Hi John,
>>
>> See my responses below
>>
>> Hi Ed,
>>
>> I see you have some questions about masking. I’m answering on my
phone
>> right now so I can’t provide specific details. But let me know if
anything
>> isn’t clear and I can follow up with specific examples later.
>>
>> I read the description of the plot you’d like to create and believe
that
>> it
>> can be accomplished without the use of mask regions.
>>
>> That would save a lot of space and complexity if I could do that
>>
>> But before proceeding to that, let me point out that you can apply
data
>> masks in point_stat WITHOUT running gen_vx_mask first. However for
>> “complex” masks (e.g. combining 2 masks, like SWD with land use
type) you
>> do need to run gen_vx_mask first. We could conceivably enhance MET
to
>> support the generation of complex masks on the fly in the config
file to
>> avoid the need for running gen_vx_mask. So let me know if you think
that’s
>> worth documenting as a feature request.
>>
>> The application of masks without gen_vx_mask sounds vaguely
familiar.
>> Maybe it was mentioned in an old email.  I'll try to look for that.
I
>> have
>> used the intersection tool to do this.  I found that I had to run
>> gen_vx_mask twice:  one to generate a netcdf file of the cropped
out
>> region
>> and another to identify land use type in that region.  Yes, I think
>> anything that can reduce specifying numerous pathways in the
PointStat
>> file
>> under the mask definition would be a great alleviation.  I support
making
>> a
>> note of that.  Thanks.
>>
>> But the next part depends on what type of “bias” you’re looking
for.
>> Either
>> multiplicative bias... ie avg forecast value - avg observed value
from the
>> CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
>> from the CTS line? Which are you looking for?
>>
>> I was leaning toward multiplicative bias.  The idea would be to bin
out a
>> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2, and
so on)
>> on the X-axis with a multiplicative bias on the y-axis.  It might
be more
>> visually appealing if I used the bar graph option for that which I
know
>> MET
>> can do.   Let me know if there is more information that you need
from me
>> so
>> I can help with this.  Thanks
>>
>> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Hi Ed,
>> >
>> > I see you have some questions about masking. I’m answering on my
phone
>> > right now so I can’t provide specific details. But let me know if
>> anything
>> > isn’t clear and I can follow up with specific examples later.
>> >
>> > I read the description of the plot you’d like to create and
believe
>> that it
>> > can be accomplished without the use of mask regions.
>> >
>> > But before proceeding to that, let me point out that you can
apply data
>> > masks in point_stat WITHOUT running gen_vx_mask first. However
for
>> > “complex” masks (e.g. combining 2 masks, like SWD with land use
type)
>> you
>> > do need to run gen_vx_mask first. We could conceivably enhance
MET to
>> > support the generation of complex masks on the fly in the config
file to
>> > avoid the need for running gen_vx_mask. So let me know if you
think
>> that’s
>> > worth documenting as a feature request.
>> >
>> > But the next part depends on what type of “bias” you’re looking
for.
>> Either
>> > multiplicative bias... ie avg forecast value - avg observed value
from
>> the
>> > CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
>> > from the CTS line? Which are you looking for?
>> >
>> > Thanks,
>> > John
>> >
>> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
>> met_help at ucar.edu
>> > >
>> > wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>> > >
>> > > Hi Edward.
>> > >
>> > > I have assigned this ticket to John HG.  Please allow a few
business
>> days
>> > > for a response.
>> > >
>> > > Julie
>> > >
>> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
Affiliate via
>> RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
>> > > > Transaction: Ticket created by edward.strobach at noaa.gov
>> > > >        Queue: met_help
>> > > >      Subject: innumerable masks!
>> > > >        Owner: Nobody
>> > > >   Requestors: edward.strobach at noaa.gov
>> > > >       Status: new
>> > > >  Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
>> > >
>> > > >
>> > > >
>> > > > Good morning,
>> > > >
>> > > > I've been exploring ways to create various masks with some
success.
>> > > Right
>> > > > now I produce the following masks each day:
>> > > > -dominant land use type (24 files - manageable)
>> > > > -dominant land use type by region (sum of the number of land
use
>> types
>> > > for
>> > > > each region (e.g. SWC, WEST, etc. - approaching unmanageable)
>> > > > -cloud fraction (10 * fh * run - very unmanageable).
>> > > >
>> > > > The last point prompted me to send this email.  Right now,
for each
>> > > > experiment and forecast hour, I generate a cloud mask for
clear
>> skies,
>> > if
>> > > > the grid is between 0 to 10 % occupied, if the grid square is
>> greater
>> > > than
>> > > > 10% occupied but less than 20% occupied, all the way to 100%.
This
>> > > brings
>> > > > us to about 10 files for each forecast hour.  However, since
I'm
>> > running
>> > > 6
>> > > > experiments out to 72 hours, you can now see that I would
easily be
>> > > > generating over 1000 files each time (:-/).
>> > > >
>> > > > I'm trying to use the cloud fraction field to do the
following:
>> x-axis
>> > > is
>> > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
fraction
>> > between 0
>> > > > and 10, etc.) while y-axis would be the bias of a field (e.g.
PM2.5
>> or
>> > > > ozone) collocated with obs.  I'd do this for all geographical
>> regions
>> > > > ideally, realizing that not every bin would be populated.  In
some
>> > > > instances an entire region might be under clear skies.  This
may be
>> a
>> > bit
>> > > > of a pipe dream, but can anyone see the feasibility of
attempting
>> this,
>> > > or
>> > > > is this beyond the scope of what can currently be done in an
>> efficient
>> > > > manner?
>> > > >
>> > > > -In short, I'm wondering what I can do to cut down on these
files
>> (lump
>> > > > each fraction into one file for all forecast hours?).
>> > > > -Would I be able to limit the point stat_config to just 10
pathway
>> > lines?
>> > > > Right now I have things stored in the following way each day:
>> > > >
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
>> > > > Here, para6 ($model) is the model while 20201010 is the date
>> ($DATE).
>> > > > In PointStat_AIRNOW - the file I use stored in my met_config
- I
>> would
>> > > look
>> > > > something like this:
>> > > >
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
>> > > > However, as indicated by the pasted example, you can see the
files
>> and
>> > > how
>> > > > extensive they could be:
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
>> > > >  cfrac_12_15_gt0p4_le0p5.nc
<http://cfrac_12_15_gt0p4_le0p5.nc>
>> > > >  cfrac_12_62_gt0p8_le0p9.nc
>> > > > <http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
>> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
cfrac_12_15_gt0p5_le0p6.nc
>> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
>> > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
>> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
cfrac_12_15_gt0p6_le0p7.nc
>> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
>> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
>> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
cfrac_12_15_gt0p7_le0p8.nc
>> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>  cfrac_12_63_gt0_le0p1.nc
>> > > > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
>> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
cfrac_12_15_gt0p8_le0p9.nc
>> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
cfrac_12_63_gt0p1_le0p2.nc
>> > > > <http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
>> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
>> > > > <http://cfrac_12_15_gt0p9.nc>
cfrac_12_63_gt0p2_le0p3.nc
>> > > > <http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
>> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
>> > > > <http://cfrac_12_16_0.nc>
cfrac_12_63_gt0p3_le0p4.nc
>> > > > <http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
>> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>  cfrac_12_16_gt0_le0p1.nc
>> > > > <http://cfrac_12_16_gt0_le0p1.nc>
cfrac_12_63_gt0p4_le0p5.nc
>> > > > <http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
>> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
cfrac_12_16_gt0p1_le0p2.nc
>> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
cfrac_12_63_gt0p5_le0p6.nc
>> > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
>> > > > <http://cfrac_06_39_gt0p9.nc>
cfrac_12_16_gt0p2_le0p3.nc
>> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
cfrac_12_63_gt0p6_le0p7.nc
>> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
>> > > > <http://cfrac_06_40_0.nc>
cfrac_12_16_gt0p3_le0p4.nc
>> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
cfrac_12_63_gt0p7_le0p8.nc
>> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
>> > > >
>> > > > The naming convention of the file would then be something
like this
>> in
>> > > the
>> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
>> > > >
>> > > > In the past, when I tried to process 3D fields stored in
netcdf
>> files,
>> > I
>> > > > found multiple dimensional arrays greater than 2 (horizontal
>> > dimensions)
>> > > to
>> > > > be a problem.  If I attempted to consolidate these files then
I
>> would
>> > > have
>> > > > a 4D matrix that would resemble the following:  (cloud mask,
time,
>> lon,
>> > > > lat).  I'm not sure that MET would like that. Am I right that
this
>> > could
>> > > be
>> > > > a problem?
>> > > >
>> > > > Thanks in advance to any insight you can provide.
>> > > >
>> > > > --
>> > > > Edward Strobach
>> > > > EMC/NCEP/NWS/
>> > > > IMSG Contractor
>> > > > Cubicle#: 2029
>> > > > 301-683-3717
>> > > >
>> > > >
>> > >
>> > > --
>> > > Julie Prestopnik (she/her/hers)
>> > > Software Engineer
>> > > National Center for Atmospheric Research
>> > > Research Applications Laboratory
>> > > Email: jpresto at ucar.edu
>> > >
>> > > My working day may not be your working day.  Please do not feel
>> obliged
>> > to
>> > > reply to this email outside of your normal working hours.
>> > >
>> > >
>> >
>> >
>>
>> --
>> Edward Strobach
>> EMC/NCEP/NWS/
>> IMSG Contractor
>> Cubicle#: 2029
>> 301-683-3717
>>
>>

------------------------------------------------
Subject: innumerable masks!
From: Edward Strobach - NOAA Affiliate
Time: Tue Oct 20 13:19:11 2020

Hi John,

I actually don't have a way to verify cloud fraction at this point.
This
thresholding would be done using the cloud fraction from the model
output
and then just to evaluate the multiplicative bias of particular fields
at
each threshold. In other words, I would have all the PM2.5 data
gathered at
all sites, identify, according to cloud fraction thresholds from the
model
output, the biases in that field as a function of cloud fraction
binning. I
should be able to do that.  The one thing I've never done, however, is
use
model output that is not being verified to generate these sorts of
stats.
It sounds like I would want to use FCST_THRESH instead. Is there a
fcst_dict analog to the obs_dict?

On Tue, Oct 20, 2020 at 3:03 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Ed,
>
> The multiplicative bias statistic is called the mean error, or ME,
in the
> CNT column. It's just the avg fcst value - avg obs value. But what
you want
> to do is called "conditional continuous" verification. Rather than
> computing stats over all the points, you want to condition the
verification
> based on the value, whereas masking conditions based on the
location.  You
> control this using the "cnt_thresh" and "cnt_logic" config options.
You can
> condition based on the forecast value, observation value, or both.
For
> simplicity, I'd recommend starting with the observation value.
>
> So in the obs dictionary, define:
>    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3, >0.3&&<=0.4,
> >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8, >0.8&&<=0.9,
>0.9 ];
>
> For each 1 CNT line from the output, you should now see 11: 1 with
> OBS_THRESH = NA including all the points, and 10 more with
OBS_THRESH = the
> bins listed above.
>
> Now I haven't actually done this... but hopefully you can use
METviewer to
> make the plot you described.
>
> For frequency bias, the contents of the CTC, CTS, MCTC, and MCTS
line types
> are determined by the cat_thresh setting. It could be interesting to
define
> cat_thresh as shown below and look at the MCTS output line type:
>    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8, >.9 ];
>
> The MCTS output is described here:
>
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
>
> Hope that helps.
>
> John
>
> On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >
> > Hi John,
> >
> > See my responses below
> >
> > Hi Ed,
> >
> > I see you have some questions about masking. I’m answering on my
phone
> > right now so I can’t provide specific details. But let me know if
> anything
> > isn’t clear and I can follow up with specific examples later.
> >
> > I read the description of the plot you’d like to create and
believe that
> it
> > can be accomplished without the use of mask regions.
> >
> > That would save a lot of space and complexity if I could do that
> >
> > But before proceeding to that, let me point out that you can apply
data
> > masks in point_stat WITHOUT running gen_vx_mask first. However for
> > “complex” masks (e.g. combining 2 masks, like SWD with land use
type) you
> > do need to run gen_vx_mask first. We could conceivably enhance MET
to
> > support the generation of complex masks on the fly in the config
file to
> > avoid the need for running gen_vx_mask. So let me know if you
think
> that’s
> > worth documenting as a feature request.
> >
> > The application of masks without gen_vx_mask sounds vaguely
familiar.
> > Maybe it was mentioned in an old email.  I'll try to look for
that.  I
> have
> > used the intersection tool to do this.  I found that I had to run
> > gen_vx_mask twice:  one to generate a netcdf file of the cropped
out
> region
> > and another to identify land use type in that region.  Yes, I
think
> > anything that can reduce specifying numerous pathways in the
PointStat
> file
> > under the mask definition would be a great alleviation.  I support
> making a
> > note of that.  Thanks.
> >
> > But the next part depends on what type of “bias” you’re looking
for.
> Either
> > multiplicative bias... ie avg forecast value - avg observed value
from
> the
> > CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
> > from the CTS line? Which are you looking for?
> >
> > I was leaning toward multiplicative bias.  The idea would be to
bin out a
> > range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2, and
so on)
> > on the X-axis with a multiplicative bias on the y-axis.  It might
be more
> > visually appealing if I used the bar graph option for that which I
know
> MET
> > can do.   Let me know if there is more information that you need
from me
> so
> > I can help with this.  Thanks
> >
> > On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hi Ed,
> > >
> > > I see you have some questions about masking. I’m answering on my
phone
> > > right now so I can’t provide specific details. But let me know
if
> > anything
> > > isn’t clear and I can follow up with specific examples later.
> > >
> > > I read the description of the plot you’d like to create and
believe
> that
> > it
> > > can be accomplished without the use of mask regions.
> > >
> > > But before proceeding to that, let me point out that you can
apply data
> > > masks in point_stat WITHOUT running gen_vx_mask first. However
for
> > > “complex” masks (e.g. combining 2 masks, like SWD with land use
type)
> you
> > > do need to run gen_vx_mask first. We could conceivably enhance
MET to
> > > support the generation of complex masks on the fly in the config
file
> to
> > > avoid the need for running gen_vx_mask. So let me know if you
think
> > that’s
> > > worth documenting as a feature request.
> > >
> > > But the next part depends on what type of “bias” you’re looking
for.
> > Either
> > > multiplicative bias... ie avg forecast value - avg observed
value from
> > the
> > > CNT line? Or categorical frequency bias... ie #fcst events /
#obs
> events
> > > from the CTS line? Which are you looking for?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
>
> > > >
> > > > Hi Edward.
> > > >
> > > > I have assigned this ticket to John HG.  Please allow a few
business
> > days
> > > > for a response.
> > > >
> > > > Julie
> > > >
> > > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
Affiliate via
> > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > > > > Transaction: Ticket created by edward.strobach at noaa.gov
> > > > >        Queue: met_help
> > > > >      Subject: innumerable masks!
> > > > >        Owner: Nobody
> > > > >   Requestors: edward.strobach at noaa.gov
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > > >
> > > > >
> > > > >
> > > > > Good morning,
> > > > >
> > > > > I've been exploring ways to create various masks with some
success.
> > > > Right
> > > > > now I produce the following masks each day:
> > > > > -dominant land use type (24 files - manageable)
> > > > > -dominant land use type by region (sum of the number of land
use
> > types
> > > > for
> > > > > each region (e.g. SWC, WEST, etc. - approaching
unmanageable)
> > > > > -cloud fraction (10 * fh * run - very unmanageable).
> > > > >
> > > > > The last point prompted me to send this email.  Right now,
for each
> > > > > experiment and forecast hour, I generate a cloud mask for
clear
> > skies,
> > > if
> > > > > the grid is between 0 to 10 % occupied, if the grid square
is
> greater
> > > > than
> > > > > 10% occupied but less than 20% occupied, all the way to
100%.  This
> > > > brings
> > > > > us to about 10 files for each forecast hour.  However, since
I'm
> > > running
> > > > 6
> > > > > experiments out to 72 hours, you can now see that I would
easily be
> > > > > generating over 1000 files each time (:-/).
> > > > >
> > > > > I'm trying to use the cloud fraction field to do the
following:
> > x-axis
> > > > is
> > > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
fraction
> > > between 0
> > > > > and 10, etc.) while y-axis would be the bias of a field
(e.g. PM2.5
> > or
> > > > > ozone) collocated with obs.  I'd do this for all
geographical
> regions
> > > > > ideally, realizing that not every bin would be populated.
In some
> > > > > instances an entire region might be under clear skies.  This
may
> be a
> > > bit
> > > > > of a pipe dream, but can anyone see the feasibility of
attempting
> > this,
> > > > or
> > > > > is this beyond the scope of what can currently be done in an
> > efficient
> > > > > manner?
> > > > >
> > > > > -In short, I'm wondering what I can do to cut down on these
files
> > (lump
> > > > > each fraction into one file for all forecast hours?).
> > > > > -Would I be able to limit the point stat_config to just 10
pathway
> > > lines?
> > > > > Right now I have things stored in the following way each
day:
> > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > > > Here, para6 ($model) is the model while 20201010 is the date
> ($DATE).
> > > > > In PointStat_AIRNOW - the file I use stored in my met_config
- I
> > would
> > > > look
> > > > > something like this:
> > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > > > However, as indicated by the pasted example, you can see the
files
> > and
> > > > how
> > > > > extensive they could be:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
> > > > >  cfrac_12_15_gt0p4_le0p5.nc
<http://cfrac_12_15_gt0p4_le0p5.nc>
> > > > >  cfrac_12_62_gt0p8_le0p9.nc
> > > > >
<http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
cfrac_12_15_gt0p5_le0p6.nc
> > > > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> > > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
cfrac_12_15_gt0p6_le0p7.nc
> > > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
cfrac_12_15_gt0p7_le0p8.nc
> > > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
cfrac_12_63_gt0_le0p1.nc
> > > > > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
cfrac_12_15_gt0p8_le0p9.nc
> > > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
cfrac_12_63_gt0p1_le0p2.nc
> > > > >
<http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > > > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> > > > > <http://cfrac_12_15_gt0p9.nc>
cfrac_12_63_gt0p2_le0p3.nc
> > > > >
<http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > > > > <http://cfrac_12_16_0.nc>
cfrac_12_63_gt0p3_le0p4.nc
> > > > >
<http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
cfrac_12_16_gt0_le0p1.nc
> > > > > <http://cfrac_12_16_gt0_le0p1.nc>
cfrac_12_63_gt0p4_le0p5.nc
> > > > >
<http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
cfrac_12_16_gt0p1_le0p2.nc
> > > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
cfrac_12_63_gt0p5_le0p6.nc
> > > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > > > <http://cfrac_06_39_gt0p9.nc>
cfrac_12_16_gt0p2_le0p3.nc
> > > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
cfrac_12_63_gt0p6_le0p7.nc
> > > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > > > <http://cfrac_06_40_0.nc>
cfrac_12_16_gt0p3_le0p4.nc
> > > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
cfrac_12_63_gt0p7_le0p8.nc
> > > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > > > >
> > > > > The naming convention of the file would then be something
like this
> > in
> > > > the
> > > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > > > >
> > > > > In the past, when I tried to process 3D fields stored in
netcdf
> > files,
> > > I
> > > > > found multiple dimensional arrays greater than 2 (horizontal
> > > dimensions)
> > > > to
> > > > > be a problem.  If I attempted to consolidate these files
then I
> would
> > > > have
> > > > > a 4D matrix that would resemble the following:  (cloud mask,
time,
> > lon,
> > > > > lat).  I'm not sure that MET would like that. Am I right
that this
> > > could
> > > > be
> > > > > a problem?
> > > > >
> > > > > Thanks in advance to any insight you can provide.
> > > > >
> > > > > --
> > > > > Edward Strobach
> > > > > EMC/NCEP/NWS/
> > > > > IMSG Contractor
> > > > > Cubicle#: 2029
> > > > > 301-683-3717
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik (she/her/hers)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day.  Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>

--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717

------------------------------------------------
Subject: innumerable masks!
From: Edward Strobach - NOAA Affiliate
Time: Tue Oct 20 13:20:31 2020

Thanks I'll take a look.  Of course some masks are static (land type)
while
others are more dynamic (cloud fraction).  I imagine dynamic masks
require
more of a delicate touch.

On Tue, Oct 20, 2020 at 3:15 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Ed,
>
> FYI, I wrote up this GitHub issue:
> https://github.com/dtcenter/MET/issues/1530
>
> Please take a look and let me know what you think. I'm not sure
what's
> reasonable to be done here. I suppose in an ideal world, all the
> functionality of gen_vx_mask would be available via the MET config
files,
> and the only reason to define them one place versus another would be
> efficiency. It's quicker to define it once and use it many times
than
> re-define each time. But that would be a pretty significant change.
>
> John
>
> On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway <johnhg at ucar.edu>
> wrote:
>
> > Ed,
> >
> > The multiplicative bias statistic is called the mean error, or ME,
in the
> > CNT column. It's just the avg fcst value - avg obs value. But what
you
> want
> > to do is called "conditional continuous" verification. Rather than
> > computing stats over all the points, you want to condition the
> verification
> > based on the value, whereas masking conditions based on the
location.
> You
> > control this using the "cnt_thresh" and "cnt_logic" config
options. You
> can
> > condition based on the forecast value, observation value, or both.
For
> > simplicity, I'd recommend starting with the observation value.
> >
> > So in the obs dictionary, define:
> >    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3,
>0.3&&<=0.4,
> > >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8, >0.8&&<=0.9,
>0.9 ];
> >
> > For each 1 CNT line from the output, you should now see 11: 1 with
> > OBS_THRESH = NA including all the points, and 10 more with
OBS_THRESH =
> the
> > bins listed above.
> >
> > Now I haven't actually done this... but hopefully you can use
METviewer
> to
> > make the plot you described.
> >
> > For frequency bias, the contents of the CTC, CTS, MCTC, and MCTS
line
> > types are determined by the cat_thresh setting. It could be
interesting
> to
> > define cat_thresh as shown below and look at the MCTS output line
type:
> >    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8, >.9
];
> >
> > The MCTS output is described here:
> >
> >
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
> >
> > Hope that helps.
> >
> > John
> >
> > On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >>
> >> Hi John,
> >>
> >> See my responses below
> >>
> >> Hi Ed,
> >>
> >> I see you have some questions about masking. I’m answering on my
phone
> >> right now so I can’t provide specific details. But let me know if
> anything
> >> isn’t clear and I can follow up with specific examples later.
> >>
> >> I read the description of the plot you’d like to create and
believe that
> >> it
> >> can be accomplished without the use of mask regions.
> >>
> >> That would save a lot of space and complexity if I could do that
> >>
> >> But before proceeding to that, let me point out that you can
apply data
> >> masks in point_stat WITHOUT running gen_vx_mask first. However
for
> >> “complex” masks (e.g. combining 2 masks, like SWD with land use
type)
> you
> >> do need to run gen_vx_mask first. We could conceivably enhance
MET to
> >> support the generation of complex masks on the fly in the config
file to
> >> avoid the need for running gen_vx_mask. So let me know if you
think
> that’s
> >> worth documenting as a feature request.
> >>
> >> The application of masks without gen_vx_mask sounds vaguely
familiar.
> >> Maybe it was mentioned in an old email.  I'll try to look for
that.  I
> >> have
> >> used the intersection tool to do this.  I found that I had to run
> >> gen_vx_mask twice:  one to generate a netcdf file of the cropped
out
> >> region
> >> and another to identify land use type in that region.  Yes, I
think
> >> anything that can reduce specifying numerous pathways in the
PointStat
> >> file
> >> under the mask definition would be a great alleviation.  I
support
> making
> >> a
> >> note of that.  Thanks.
> >>
> >> But the next part depends on what type of “bias” you’re looking
for.
> >> Either
> >> multiplicative bias... ie avg forecast value - avg observed value
from
> the
> >> CNT line? Or categorical frequency bias... ie #fcst events / #obs
events
> >> from the CTS line? Which are you looking for?
> >>
> >> I was leaning toward multiplicative bias.  The idea would be to
bin out
> a
> >> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2,
and so
> on)
> >> on the X-axis with a multiplicative bias on the y-axis.  It might
be
> more
> >> visually appealing if I used the bar graph option for that which
I know
> >> MET
> >> can do.   Let me know if there is more information that you need
from me
> >> so
> >> I can help with this.  Thanks
> >>
> >> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Hi Ed,
> >> >
> >> > I see you have some questions about masking. I’m answering on
my phone
> >> > right now so I can’t provide specific details. But let me know
if
> >> anything
> >> > isn’t clear and I can follow up with specific examples later.
> >> >
> >> > I read the description of the plot you’d like to create and
believe
> >> that it
> >> > can be accomplished without the use of mask regions.
> >> >
> >> > But before proceeding to that, let me point out that you can
apply
> data
> >> > masks in point_stat WITHOUT running gen_vx_mask first. However
for
> >> > “complex” masks (e.g. combining 2 masks, like SWD with land use
type)
> >> you
> >> > do need to run gen_vx_mask first. We could conceivably enhance
MET to
> >> > support the generation of complex masks on the fly in the
config file
> to
> >> > avoid the need for running gen_vx_mask. So let me know if you
think
> >> that’s
> >> > worth documenting as a feature request.
> >> >
> >> > But the next part depends on what type of “bias” you’re looking
for.
> >> Either
> >> > multiplicative bias... ie avg forecast value - avg observed
value from
> >> the
> >> > CNT line? Or categorical frequency bias... ie #fcst events /
#obs
> events
> >> > from the CTS line? Which are you looking for?
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> >> met_help at ucar.edu
> >> > >
> >> > wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
>
> >> > >
> >> > > Hi Edward.
> >> > >
> >> > > I have assigned this ticket to John HG.  Please allow a few
business
> >> days
> >> > > for a response.
> >> > >
> >> > > Julie
> >> > >
> >> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
Affiliate via
> >> RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> >> > > > Transaction: Ticket created by edward.strobach at noaa.gov
> >> > > >        Queue: met_help
> >> > > >      Subject: innumerable masks!
> >> > > >        Owner: Nobody
> >> > > >   Requestors: edward.strobach at noaa.gov
> >> > > >       Status: new
> >> > > >  Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> >> > >
> >> > > >
> >> > > >
> >> > > > Good morning,
> >> > > >
> >> > > > I've been exploring ways to create various masks with some
> success.
> >> > > Right
> >> > > > now I produce the following masks each day:
> >> > > > -dominant land use type (24 files - manageable)
> >> > > > -dominant land use type by region (sum of the number of
land use
> >> types
> >> > > for
> >> > > > each region (e.g. SWC, WEST, etc. - approaching
unmanageable)
> >> > > > -cloud fraction (10 * fh * run - very unmanageable).
> >> > > >
> >> > > > The last point prompted me to send this email.  Right now,
for
> each
> >> > > > experiment and forecast hour, I generate a cloud mask for
clear
> >> skies,
> >> > if
> >> > > > the grid is between 0 to 10 % occupied, if the grid square
is
> >> greater
> >> > > than
> >> > > > 10% occupied but less than 20% occupied, all the way to
100%.
> This
> >> > > brings
> >> > > > us to about 10 files for each forecast hour.  However,
since I'm
> >> > running
> >> > > 6
> >> > > > experiments out to 72 hours, you can now see that I would
easily
> be
> >> > > > generating over 1000 files each time (:-/).
> >> > > >
> >> > > > I'm trying to use the cloud fraction field to do the
following:
> >> x-axis
> >> > > is
> >> > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
fraction
> >> > between 0
> >> > > > and 10, etc.) while y-axis would be the bias of a field
(e.g.
> PM2.5
> >> or
> >> > > > ozone) collocated with obs.  I'd do this for all
geographical
> >> regions
> >> > > > ideally, realizing that not every bin would be populated.
In some
> >> > > > instances an entire region might be under clear skies.
This may
> be
> >> a
> >> > bit
> >> > > > of a pipe dream, but can anyone see the feasibility of
attempting
> >> this,
> >> > > or
> >> > > > is this beyond the scope of what can currently be done in
an
> >> efficient
> >> > > > manner?
> >> > > >
> >> > > > -In short, I'm wondering what I can do to cut down on these
files
> >> (lump
> >> > > > each fraction into one file for all forecast hours?).
> >> > > > -Would I be able to limit the point stat_config to just 10
pathway
> >> > lines?
> >> > > > Right now I have things stored in the following way each
day:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> >> > > > Here, para6 ($model) is the model while 20201010 is the
date
> >> ($DATE).
> >> > > > In PointStat_AIRNOW - the file I use stored in my
met_config - I
> >> would
> >> > > look
> >> > > > something like this:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> >> > > > However, as indicated by the pasted example, you can see
the files
> >> and
> >> > > how
> >> > > > extensive they could be:
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > *cfrac_06_39_gt0_le0p1.nc <http://cfrac_06_39_gt0_le0p1.nc>
> >> > > >  cfrac_12_15_gt0p4_le0p5.nc
<http://cfrac_12_15_gt0p4_le0p5.nc>
> >> > > >  cfrac_12_62_gt0p8_le0p9.nc
> >> > > >
<http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> >> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
cfrac_12_15_gt0p5_le0p6.nc
> >> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> >> > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> >> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
cfrac_12_15_gt0p6_le0p7.nc
> >> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> >> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> >> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
cfrac_12_15_gt0p7_le0p8.nc
> >> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
cfrac_12_63_gt0_le0p1.nc
> >> > > > <http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> >> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
cfrac_12_15_gt0p8_le0p9.nc
> >> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
cfrac_12_63_gt0p1_le0p2.nc
> >> > > >
<http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> >> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> >> > > > <http://cfrac_12_15_gt0p9.nc>
cfrac_12_63_gt0p2_le0p3.nc
> >> > > >
<http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> >> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> >> > > > <http://cfrac_12_16_0.nc>
cfrac_12_63_gt0p3_le0p4.nc
> >> > > >
<http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> >> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
cfrac_12_16_gt0_le0p1.nc
> >> > > > <http://cfrac_12_16_gt0_le0p1.nc>
cfrac_12_63_gt0p4_le0p5.nc
> >> > > >
<http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> >> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
cfrac_12_16_gt0p1_le0p2.nc
> >> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
cfrac_12_63_gt0p5_le0p6.nc
> >> > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> >> > > > <http://cfrac_06_39_gt0p9.nc>
cfrac_12_16_gt0p2_le0p3.nc
> >> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
cfrac_12_63_gt0p6_le0p7.nc
> >> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> >> > > > <http://cfrac_06_40_0.nc>
cfrac_12_16_gt0p3_le0p4.nc
> >> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
cfrac_12_63_gt0p7_le0p8.nc
> >> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> >> > > >
> >> > > > The naming convention of the file would then be something
like
> this
> >> in
> >> > > the
> >> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> >> > > >
> >> > > > In the past, when I tried to process 3D fields stored in
netcdf
> >> files,
> >> > I
> >> > > > found multiple dimensional arrays greater than 2
(horizontal
> >> > dimensions)
> >> > > to
> >> > > > be a problem.  If I attempted to consolidate these files
then I
> >> would
> >> > > have
> >> > > > a 4D matrix that would resemble the following:  (cloud
mask, time,
> >> lon,
> >> > > > lat).  I'm not sure that MET would like that. Am I right
that this
> >> > could
> >> > > be
> >> > > > a problem?
> >> > > >
> >> > > > Thanks in advance to any insight you can provide.
> >> > > >
> >> > > > --
> >> > > > Edward Strobach
> >> > > > EMC/NCEP/NWS/
> >> > > > IMSG Contractor
> >> > > > Cubicle#: 2029
> >> > > > 301-683-3717
> >> > > >
> >> > > >
> >> > >
> >> > > --
> >> > > Julie Prestopnik (she/her/hers)
> >> > > Software Engineer
> >> > > National Center for Atmospheric Research
> >> > > Research Applications Laboratory
> >> > > Email: jpresto at ucar.edu
> >> > >
> >> > > My working day may not be your working day.  Please do not
feel
> >> obliged
> >> > to
> >> > > reply to this email outside of your normal working hours.
> >> > >
> >> > >
> >> >
> >> >
> >>
> >> --
> >> Edward Strobach
> >> EMC/NCEP/NWS/
> >> IMSG Contractor
> >> Cubicle#: 2029
> >> 301-683-3717
> >>
> >>
>
>

--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717

------------------------------------------------
Subject: innumerable masks!
From: John Halley Gotway
Time: Tue Oct 20 13:39:40 2020

Ed,

Ah sorry, I missed that detail.  Then you're right. The only way to
have
one field impact the verification of another is through the use of
masking
regions. So you'd use the cloud fraction to define the data mask based
on
the forecast cloud fraction amount. And the only way to INTERSECT that
mask
with another (e.g. cloud fraction from 0.5 to 0.6 AND CONUS) is
through gen_vx_mask.

For now, if possible, I'd recommend:
(1) Mask spatially using the output from gen_vx_mask (e.g. CONUS, SWD,
...)
(2) Do data masking of land use type directly in the point_stat config
file
(3) Do cloud fraction masking directly in the point_stat config file

Do NOT try to intersect the masks from (1) with the data masks from
(2).
And we can figure out if/how to do so based on that GitHub issue. But
of
course it's up to you to decide if running gen_vx_mask all those times
is
reasonable.

Another thought is running the relatively new grid_diag tool. That is
used
to compute more diagnostics instead of verification. It can be used to
bin
the values from one variable (e.g. PM2.5) relative to another (e.g.
cloud fraction). It won't solve these specific issues but may be
interesting.

John

On Tue, Oct 20, 2020 at 1:21 PM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>
> Thanks I'll take a look.  Of course some masks are static (land
type) while
> others are more dynamic (cloud fraction).  I imagine dynamic masks
require
> more of a delicate touch.
>
> On Tue, Oct 20, 2020 at 3:15 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ed,
> >
> > FYI, I wrote up this GitHub issue:
> > https://github.com/dtcenter/MET/issues/1530
> >
> > Please take a look and let me know what you think. I'm not sure
what's
> > reasonable to be done here. I suppose in an ideal world, all the
> > functionality of gen_vx_mask would be available via the MET config
files,
> > and the only reason to define them one place versus another would
be
> > efficiency. It's quicker to define it once and use it many times
than
> > re-define each time. But that would be a pretty significant
change.
> >
> > John
> >
> > On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > Ed,
> > >
> > > The multiplicative bias statistic is called the mean error, or
ME, in
> the
> > > CNT column. It's just the avg fcst value - avg obs value. But
what you
> > want
> > > to do is called "conditional continuous" verification. Rather
than
> > > computing stats over all the points, you want to condition the
> > verification
> > > based on the value, whereas masking conditions based on the
location.
> > You
> > > control this using the "cnt_thresh" and "cnt_logic" config
options. You
> > can
> > > condition based on the forecast value, observation value, or
both. For
> > > simplicity, I'd recommend starting with the observation value.
> > >
> > > So in the obs dictionary, define:
> > >    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3,
>0.3&&<=0.4,
> > > >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8, >0.8&&<=0.9,
>0.9
> ];
> > >
> > > For each 1 CNT line from the output, you should now see 11: 1
with
> > > OBS_THRESH = NA including all the points, and 10 more with
OBS_THRESH =
> > the
> > > bins listed above.
> > >
> > > Now I haven't actually done this... but hopefully you can use
METviewer
> > to
> > > make the plot you described.
> > >
> > > For frequency bias, the contents of the CTC, CTS, MCTC, and MCTS
line
> > > types are determined by the cat_thresh setting. It could be
interesting
> > to
> > > define cat_thresh as shown below and look at the MCTS output
line type:
> > >    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8,
>.9 ];
> > >
> > > The MCTS output is described here:
> > >
> > >
> >
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
> > >
> > > Hope that helps.
> > >
> > > John
> > >
> > > On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA
Affiliate via
> RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > >>
> > >> Hi John,
> > >>
> > >> See my responses below
> > >>
> > >> Hi Ed,
> > >>
> > >> I see you have some questions about masking. I’m answering on
my phone
> > >> right now so I can’t provide specific details. But let me know
if
> > anything
> > >> isn’t clear and I can follow up with specific examples later.
> > >>
> > >> I read the description of the plot you’d like to create and
believe
> that
> > >> it
> > >> can be accomplished without the use of mask regions.
> > >>
> > >> That would save a lot of space and complexity if I could do
that
> > >>
> > >> But before proceeding to that, let me point out that you can
apply
> data
> > >> masks in point_stat WITHOUT running gen_vx_mask first. However
for
> > >> “complex” masks (e.g. combining 2 masks, like SWD with land use
type)
> > you
> > >> do need to run gen_vx_mask first. We could conceivably enhance
MET to
> > >> support the generation of complex masks on the fly in the
config file
> to
> > >> avoid the need for running gen_vx_mask. So let me know if you
think
> > that’s
> > >> worth documenting as a feature request.
> > >>
> > >> The application of masks without gen_vx_mask sounds vaguely
familiar.
> > >> Maybe it was mentioned in an old email.  I'll try to look for
that.  I
> > >> have
> > >> used the intersection tool to do this.  I found that I had to
run
> > >> gen_vx_mask twice:  one to generate a netcdf file of the
cropped out
> > >> region
> > >> and another to identify land use type in that region.  Yes, I
think
> > >> anything that can reduce specifying numerous pathways in the
PointStat
> > >> file
> > >> under the mask definition would be a great alleviation.  I
support
> > making
> > >> a
> > >> note of that.  Thanks.
> > >>
> > >> But the next part depends on what type of “bias” you’re looking
for.
> > >> Either
> > >> multiplicative bias... ie avg forecast value - avg observed
value from
> > the
> > >> CNT line? Or categorical frequency bias... ie #fcst events /
#obs
> events
> > >> from the CTS line? Which are you looking for?
> > >>
> > >> I was leaning toward multiplicative bias.  The idea would be to
bin
> out
> > a
> > >> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to .2,
and so
> > on)
> > >> on the X-axis with a multiplicative bias on the y-axis.  It
might be
> > more
> > >> visually appealing if I used the bar graph option for that
which I
> know
> > >> MET
> > >> can do.   Let me know if there is more information that you
need from
> me
> > >> so
> > >> I can help with this.  Thanks
> > >>
> > >> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> > Hi Ed,
> > >> >
> > >> > I see you have some questions about masking. I’m answering on
my
> phone
> > >> > right now so I can’t provide specific details. But let me
know if
> > >> anything
> > >> > isn’t clear and I can follow up with specific examples later.
> > >> >
> > >> > I read the description of the plot you’d like to create and
believe
> > >> that it
> > >> > can be accomplished without the use of mask regions.
> > >> >
> > >> > But before proceeding to that, let me point out that you can
apply
> > data
> > >> > masks in point_stat WITHOUT running gen_vx_mask first.
However for
> > >> > “complex” masks (e.g. combining 2 masks, like SWD with land
use
> type)
> > >> you
> > >> > do need to run gen_vx_mask first. We could conceivably
enhance MET
> to
> > >> > support the generation of complex masks on the fly in the
config
> file
> > to
> > >> > avoid the need for running gen_vx_mask. So let me know if you
think
> > >> that’s
> > >> > worth documenting as a feature request.
> > >> >
> > >> > But the next part depends on what type of “bias” you’re
looking for.
> > >> Either
> > >> > multiplicative bias... ie avg forecast value - avg observed
value
> from
> > >> the
> > >> > CNT line? Or categorical frequency bias... ie #fcst events /
#obs
> > events
> > >> > from the CTS line? Which are you looking for?
> > >> >
> > >> > Thanks,
> > >> > John
> > >> >
> > >> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> > >> met_help at ucar.edu
> > >> > >
> > >> > wrote:
> > >> >
> > >> > >
> > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > >> > >
> > >> > > Hi Edward.
> > >> > >
> > >> > > I have assigned this ticket to John HG.  Please allow a few
> business
> > >> days
> > >> > > for a response.
> > >> > >
> > >> > > Julie
> > >> > >
> > >> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
Affiliate
> via
> > >> RT <
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > >> > > > Transaction: Ticket created by edward.strobach at noaa.gov
> > >> > > >        Queue: met_help
> > >> > > >      Subject: innumerable masks!
> > >> > > >        Owner: Nobody
> > >> > > >   Requestors: edward.strobach at noaa.gov
> > >> > > >       Status: new
> > >> > > >  Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > >> > >
> > >> > > >
> > >> > > >
> > >> > > > Good morning,
> > >> > > >
> > >> > > > I've been exploring ways to create various masks with
some
> > success.
> > >> > > Right
> > >> > > > now I produce the following masks each day:
> > >> > > > -dominant land use type (24 files - manageable)
> > >> > > > -dominant land use type by region (sum of the number of
land use
> > >> types
> > >> > > for
> > >> > > > each region (e.g. SWC, WEST, etc. - approaching
unmanageable)
> > >> > > > -cloud fraction (10 * fh * run - very unmanageable).
> > >> > > >
> > >> > > > The last point prompted me to send this email.  Right
now, for
> > each
> > >> > > > experiment and forecast hour, I generate a cloud mask for
clear
> > >> skies,
> > >> > if
> > >> > > > the grid is between 0 to 10 % occupied, if the grid
square is
> > >> greater
> > >> > > than
> > >> > > > 10% occupied but less than 20% occupied, all the way to
100%.
> > This
> > >> > > brings
> > >> > > > us to about 10 files for each forecast hour.  However,
since I'm
> > >> > running
> > >> > > 6
> > >> > > > experiments out to 72 hours, you can now see that I would
easily
> > be
> > >> > > > generating over 1000 files each time (:-/).
> > >> > > >
> > >> > > > I'm trying to use the cloud fraction field to do the
following:
> > >> x-axis
> > >> > > is
> > >> > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
fraction
> > >> > between 0
> > >> > > > and 10, etc.) while y-axis would be the bias of a field
(e.g.
> > PM2.5
> > >> or
> > >> > > > ozone) collocated with obs.  I'd do this for all
geographical
> > >> regions
> > >> > > > ideally, realizing that not every bin would be populated.
In
> some
> > >> > > > instances an entire region might be under clear skies.
This may
> > be
> > >> a
> > >> > bit
> > >> > > > of a pipe dream, but can anyone see the feasibility of
> attempting
> > >> this,
> > >> > > or
> > >> > > > is this beyond the scope of what can currently be done in
an
> > >> efficient
> > >> > > > manner?
> > >> > > >
> > >> > > > -In short, I'm wondering what I can do to cut down on
these
> files
> > >> (lump
> > >> > > > each fraction into one file for all forecast hours?).
> > >> > > > -Would I be able to limit the point stat_config to just
10
> pathway
> > >> > lines?
> > >> > > > Right now I have things stored in the following way each
day:
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > >> > > > Here, para6 ($model) is the model while 20201010 is the
date
> > >> ($DATE).
> > >> > > > In PointStat_AIRNOW - the file I use stored in my
met_config - I
> > >> would
> > >> > > look
> > >> > > > something like this:
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > >> > > > However, as indicated by the pasted example, you can see
the
> files
> > >> and
> > >> > > how
> > >> > > > extensive they could be:
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > *cfrac_06_39_gt0_le0p1.nc
<http://cfrac_06_39_gt0_le0p1.nc>
> > >> > > >  cfrac_12_15_gt0p4_le0p5.nc
<http://cfrac_12_15_gt0p4_le0p5.nc>
> > >> > > >  cfrac_12_62_gt0p8_le0p9.nc
> > >> > > >
<http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > >> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
cfrac_12_15_gt0p5_le0p6.nc
> > >> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>  cfrac_12_62_gt0p9.nc
> > >> > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > >> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
cfrac_12_15_gt0p6_le0p7.nc
> > >> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > >> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > >> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
cfrac_12_15_gt0p7_le0p8.nc
> > >> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
cfrac_12_63_gt0_le0p1.nc
> > >> > > >
<http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > >> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
cfrac_12_15_gt0p8_le0p9.nc
> > >> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
cfrac_12_63_gt0p1_le0p2.nc
> > >> > > >
<http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > >> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>  cfrac_12_15_gt0p9.nc
> > >> > > > <http://cfrac_12_15_gt0p9.nc>
cfrac_12_63_gt0p2_le0p3.nc
> > >> > > >
<http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > >> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > >> > > > <http://cfrac_12_16_0.nc>
cfrac_12_63_gt0p3_le0p4.nc
> > >> > > >
<http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > >> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
cfrac_12_16_gt0_le0p1.nc
> > >> > > > <http://cfrac_12_16_gt0_le0p1.nc>
cfrac_12_63_gt0p4_le0p5.nc
> > >> > > >
<http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > >> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
cfrac_12_16_gt0p1_le0p2.nc
> > >> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
cfrac_12_63_gt0p5_le0p6.nc
> > >> > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > >> > > > <http://cfrac_06_39_gt0p9.nc>
cfrac_12_16_gt0p2_le0p3.nc
> > >> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
cfrac_12_63_gt0p6_le0p7.nc
> > >> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > >> > > > <http://cfrac_06_40_0.nc>
cfrac_12_16_gt0p3_le0p4.nc
> > >> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
cfrac_12_63_gt0p7_le0p8.nc
> > >> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > >> > > >
> > >> > > > The naming convention of the file would then be something
like
> > this
> > >> in
> > >> > > the
> > >> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > >> > > >
> > >> > > > In the past, when I tried to process 3D fields stored in
netcdf
> > >> files,
> > >> > I
> > >> > > > found multiple dimensional arrays greater than 2
(horizontal
> > >> > dimensions)
> > >> > > to
> > >> > > > be a problem.  If I attempted to consolidate these files
then I
> > >> would
> > >> > > have
> > >> > > > a 4D matrix that would resemble the following:  (cloud
mask,
> time,
> > >> lon,
> > >> > > > lat).  I'm not sure that MET would like that. Am I right
that
> this
> > >> > could
> > >> > > be
> > >> > > > a problem?
> > >> > > >
> > >> > > > Thanks in advance to any insight you can provide.
> > >> > > >
> > >> > > > --
> > >> > > > Edward Strobach
> > >> > > > EMC/NCEP/NWS/
> > >> > > > IMSG Contractor
> > >> > > > Cubicle#: 2029
> > >> > > > 301-683-3717
> > >> > > >
> > >> > > >
> > >> > >
> > >> > > --
> > >> > > Julie Prestopnik (she/her/hers)
> > >> > > Software Engineer
> > >> > > National Center for Atmospheric Research
> > >> > > Research Applications Laboratory
> > >> > > Email: jpresto at ucar.edu
> > >> > >
> > >> > > My working day may not be your working day.  Please do not
feel
> > >> obliged
> > >> > to
> > >> > > reply to this email outside of your normal working hours.
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >> --
> > >> Edward Strobach
> > >> EMC/NCEP/NWS/
> > >> IMSG Contractor
> > >> Cubicle#: 2029
> > >> 301-683-3717
> > >>
> > >>
> >
> >
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>

------------------------------------------------
Subject: innumerable masks!
From: Edward Strobach - NOAA Affiliate
Time: Wed Oct 21 08:49:31 2020

Hi John,

I think I'll want to avoid creating any cloud fraction masks.  It's
too
much at this point.  The only time I've used the intersection option
is to
group land use categories for different geographical regions.  It
sounds
like I should avoid that though based on what you are saying.  The
grid_diag tool does sound appealing.  It sounds like I might be able
to
bypass the need to generate cloud fraction masks if this option is
used
instead.  I'd like to know more about this tool.

On Tue, Oct 20, 2020 at 3:39 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Ed,
>
> Ah sorry, I missed that detail.  Then you're right. The only way to
have
> one field impact the verification of another is through the use of
masking
> regions. So you'd use the cloud fraction to define the data mask
based on
> the forecast cloud fraction amount. And the only way to INTERSECT
that mask
> with another (e.g. cloud fraction from 0.5 to 0.6 AND CONUS) is
> through gen_vx_mask.
>
> For now, if possible, I'd recommend:
> (1) Mask spatially using the output from gen_vx_mask (e.g. CONUS,
SWD, ...)
> (2) Do data masking of land use type directly in the point_stat
config file
> (3) Do cloud fraction masking directly in the point_stat config file
>
> Do NOT try to intersect the masks from (1) with the data masks from
(2).
> And we can figure out if/how to do so based on that GitHub issue.
But of
> course it's up to you to decide if running gen_vx_mask all those
times is
> reasonable.
>
> Another thought is running the relatively new grid_diag tool. That
is used
> to compute more diagnostics instead of verification. It can be used
to bin
> the values from one variable (e.g. PM2.5) relative to another (e.g.
> cloud fraction). It won't solve these specific issues but may be
> interesting.
>
> John
>
> On Tue, Oct 20, 2020 at 1:21 PM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >
> > Thanks I'll take a look.  Of course some masks are static (land
type)
> while
> > others are more dynamic (cloud fraction).  I imagine dynamic masks
> require
> > more of a delicate touch.
> >
> > On Tue, Oct 20, 2020 at 3:15 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Ed,
> > >
> > > FYI, I wrote up this GitHub issue:
> > > https://github.com/dtcenter/MET/issues/1530
> > >
> > > Please take a look and let me know what you think. I'm not sure
what's
> > > reasonable to be done here. I suppose in an ideal world, all the
> > > functionality of gen_vx_mask would be available via the MET
config
> files,
> > > and the only reason to define them one place versus another
would be
> > > efficiency. It's quicker to define it once and use it many times
than
> > > re-define each time. But that would be a pretty significant
change.
> > >
> > > John
> > >
> > > On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > >
> > > > Ed,
> > > >
> > > > The multiplicative bias statistic is called the mean error, or
ME, in
> > the
> > > > CNT column. It's just the avg fcst value - avg obs value. But
what
> you
> > > want
> > > > to do is called "conditional continuous" verification. Rather
than
> > > > computing stats over all the points, you want to condition the
> > > verification
> > > > based on the value, whereas masking conditions based on the
location.
> > > You
> > > > control this using the "cnt_thresh" and "cnt_logic" config
options.
> You
> > > can
> > > > condition based on the forecast value, observation value, or
both.
> For
> > > > simplicity, I'd recommend starting with the observation value.
> > > >
> > > > So in the obs dictionary, define:
> > > >    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3,
>0.3&&<=0.4,
> > > > >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8,
>0.8&&<=0.9, >0.9
> > ];
> > > >
> > > > For each 1 CNT line from the output, you should now see 11: 1
with
> > > > OBS_THRESH = NA including all the points, and 10 more with
> OBS_THRESH =
> > > the
> > > > bins listed above.
> > > >
> > > > Now I haven't actually done this... but hopefully you can use
> METviewer
> > > to
> > > > make the plot you described.
> > > >
> > > > For frequency bias, the contents of the CTC, CTS, MCTC, and
MCTS line
> > > > types are determined by the cat_thresh setting. It could be
> interesting
> > > to
> > > > define cat_thresh as shown below and look at the MCTS output
line
> type:
> > > >    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7, >.8,
>.9 ];
> > > >
> > > > The MCTS output is described here:
> > > >
> > > >
> > >
> >
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
> > > >
> > > > Hope that helps.
> > > >
> > > > John
> > > >
> > > > On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA
Affiliate via
> > RT
> > > <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
>
> > > >>
> > > >> Hi John,
> > > >>
> > > >> See my responses below
> > > >>
> > > >> Hi Ed,
> > > >>
> > > >> I see you have some questions about masking. I’m answering on
my
> phone
> > > >> right now so I can’t provide specific details. But let me
know if
> > > anything
> > > >> isn’t clear and I can follow up with specific examples later.
> > > >>
> > > >> I read the description of the plot you’d like to create and
believe
> > that
> > > >> it
> > > >> can be accomplished without the use of mask regions.
> > > >>
> > > >> That would save a lot of space and complexity if I could do
that
> > > >>
> > > >> But before proceeding to that, let me point out that you can
apply
> > data
> > > >> masks in point_stat WITHOUT running gen_vx_mask first.
However for
> > > >> “complex” masks (e.g. combining 2 masks, like SWD with land
use
> type)
> > > you
> > > >> do need to run gen_vx_mask first. We could conceivably
enhance MET
> to
> > > >> support the generation of complex masks on the fly in the
config
> file
> > to
> > > >> avoid the need for running gen_vx_mask. So let me know if you
think
> > > that’s
> > > >> worth documenting as a feature request.
> > > >>
> > > >> The application of masks without gen_vx_mask sounds vaguely
> familiar.
> > > >> Maybe it was mentioned in an old email.  I'll try to look for
> that.  I
> > > >> have
> > > >> used the intersection tool to do this.  I found that I had to
run
> > > >> gen_vx_mask twice:  one to generate a netcdf file of the
cropped out
> > > >> region
> > > >> and another to identify land use type in that region.  Yes, I
think
> > > >> anything that can reduce specifying numerous pathways in the
> PointStat
> > > >> file
> > > >> under the mask definition would be a great alleviation.  I
support
> > > making
> > > >> a
> > > >> note of that.  Thanks.
> > > >>
> > > >> But the next part depends on what type of “bias” you’re
looking for.
> > > >> Either
> > > >> multiplicative bias... ie avg forecast value - avg observed
value
> from
> > > the
> > > >> CNT line? Or categorical frequency bias... ie #fcst events /
#obs
> > events
> > > >> from the CTS line? Which are you looking for?
> > > >>
> > > >> I was leaning toward multiplicative bias.  The idea would be
to bin
> > out
> > > a
> > > >> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to
.2, and
> so
> > > on)
> > > >> on the X-axis with a multiplicative bias on the y-axis.  It
might be
> > > more
> > > >> visually appealing if I used the bar graph option for that
which I
> > know
> > > >> MET
> > > >> can do.   Let me know if there is more information that you
need
> from
> > me
> > > >> so
> > > >> I can help with this.  Thanks
> > > >>
> > > >> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> > Hi Ed,
> > > >> >
> > > >> > I see you have some questions about masking. I’m answering
on my
> > phone
> > > >> > right now so I can’t provide specific details. But let me
know if
> > > >> anything
> > > >> > isn’t clear and I can follow up with specific examples
later.
> > > >> >
> > > >> > I read the description of the plot you’d like to create and
> believe
> > > >> that it
> > > >> > can be accomplished without the use of mask regions.
> > > >> >
> > > >> > But before proceeding to that, let me point out that you
can apply
> > > data
> > > >> > masks in point_stat WITHOUT running gen_vx_mask first.
However for
> > > >> > “complex” masks (e.g. combining 2 masks, like SWD with land
use
> > type)
> > > >> you
> > > >> > do need to run gen_vx_mask first. We could conceivably
enhance MET
> > to
> > > >> > support the generation of complex masks on the fly in the
config
> > file
> > > to
> > > >> > avoid the need for running gen_vx_mask. So let me know if
you
> think
> > > >> that’s
> > > >> > worth documenting as a feature request.
> > > >> >
> > > >> > But the next part depends on what type of “bias” you’re
looking
> for.
> > > >> Either
> > > >> > multiplicative bias... ie avg forecast value - avg observed
value
> > from
> > > >> the
> > > >> > CNT line? Or categorical frequency bias... ie #fcst events
/ #obs
> > > events
> > > >> > from the CTS line? Which are you looking for?
> > > >> >
> > > >> > Thanks,
> > > >> > John
> > > >> >
> > > >> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> > > >> met_help at ucar.edu
> > > >> > >
> > > >> > wrote:
> > > >> >
> > > >> > >
> > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > > >> > >
> > > >> > > Hi Edward.
> > > >> > >
> > > >> > > I have assigned this ticket to John HG.  Please allow a
few
> > business
> > > >> days
> > > >> > > for a response.
> > > >> > >
> > > >> > > Julie
> > > >> > >
> > > >> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
Affiliate
> > via
> > > >> RT <
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted upon.
> > > >> > > > Transaction: Ticket created by edward.strobach at noaa.gov
> > > >> > > >        Queue: met_help
> > > >> > > >      Subject: innumerable masks!
> > > >> > > >        Owner: Nobody
> > > >> > > >   Requestors: edward.strobach at noaa.gov
> > > >> > > >       Status: new
> > > >> > > >  Ticket <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > > >> > >
> > > >> > > >
> > > >> > > >
> > > >> > > > Good morning,
> > > >> > > >
> > > >> > > > I've been exploring ways to create various masks with
some
> > > success.
> > > >> > > Right
> > > >> > > > now I produce the following masks each day:
> > > >> > > > -dominant land use type (24 files - manageable)
> > > >> > > > -dominant land use type by region (sum of the number of
land
> use
> > > >> types
> > > >> > > for
> > > >> > > > each region (e.g. SWC, WEST, etc. - approaching
unmanageable)
> > > >> > > > -cloud fraction (10 * fh * run - very unmanageable).
> > > >> > > >
> > > >> > > > The last point prompted me to send this email.  Right
now, for
> > > each
> > > >> > > > experiment and forecast hour, I generate a cloud mask
for
> clear
> > > >> skies,
> > > >> > if
> > > >> > > > the grid is between 0 to 10 % occupied, if the grid
square is
> > > >> greater
> > > >> > > than
> > > >> > > > 10% occupied but less than 20% occupied, all the way to
100%.
> > > This
> > > >> > > brings
> > > >> > > > us to about 10 files for each forecast hour.  However,
since
> I'm
> > > >> > running
> > > >> > > 6
> > > >> > > > experiments out to 72 hours, you can now see that I
would
> easily
> > > be
> > > >> > > > generating over 1000 files each time (:-/).
> > > >> > > >
> > > >> > > > I'm trying to use the cloud fraction field to do the
> following:
> > > >> x-axis
> > > >> > > is
> > > >> > > > the bin cloud fraction (i.e. cloud fraction = 0; cloud
> fraction
> > > >> > between 0
> > > >> > > > and 10, etc.) while y-axis would be the bias of a field
(e.g.
> > > PM2.5
> > > >> or
> > > >> > > > ozone) collocated with obs.  I'd do this for all
geographical
> > > >> regions
> > > >> > > > ideally, realizing that not every bin would be
populated.  In
> > some
> > > >> > > > instances an entire region might be under clear skies.
This
> may
> > > be
> > > >> a
> > > >> > bit
> > > >> > > > of a pipe dream, but can anyone see the feasibility of
> > attempting
> > > >> this,
> > > >> > > or
> > > >> > > > is this beyond the scope of what can currently be done
in an
> > > >> efficient
> > > >> > > > manner?
> > > >> > > >
> > > >> > > > -In short, I'm wondering what I can do to cut down on
these
> > files
> > > >> (lump
> > > >> > > > each fraction into one file for all forecast hours?).
> > > >> > > > -Would I be able to limit the point stat_config to just
10
> > pathway
> > > >> > lines?
> > > >> > > > Right now I have things stored in the following way
each day:
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > >> > > > Here, para6 ($model) is the model while 20201010 is the
date
> > > >> ($DATE).
> > > >> > > > In PointStat_AIRNOW - the file I use stored in my
met_config
> - I
> > > >> would
> > > >> > > look
> > > >> > > > something like this:
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > >> > > > However, as indicated by the pasted example, you can
see the
> > files
> > > >> and
> > > >> > > how
> > > >> > > > extensive they could be:
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > *cfrac_06_39_gt0_le0p1.nc
<http://cfrac_06_39_gt0_le0p1.nc>
> > > >> > > >  cfrac_12_15_gt0p4_le0p5.nc <
> http://cfrac_12_15_gt0p4_le0p5.nc>
> > > >> > > >  cfrac_12_62_gt0p8_le0p9.nc
> > > >> > > >
<http://cfrac_12_62_gt0p8_le0p9.nc>cfrac_06_39_gt0p1_le0p2.nc
> > > >> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
> cfrac_12_15_gt0p5_le0p6.nc
> > > >> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>
cfrac_12_62_gt0p9.nc
> > > >> > > > <http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > >> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
> cfrac_12_15_gt0p6_le0p7.nc
> > > >> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > > >> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > >> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
> cfrac_12_15_gt0p7_le0p8.nc
> > > >> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
cfrac_12_63_gt0_le0p1.nc
> > > >> > > >
<http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > > >> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
> cfrac_12_15_gt0p8_le0p9.nc
> > > >> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
> cfrac_12_63_gt0p1_le0p2.nc
> > > >> > > >
<http://cfrac_12_63_gt0p1_le0p2.nc>cfrac_06_39_gt0p5_le0p6.nc
> > > >> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>
cfrac_12_15_gt0p9.nc
> > > >> > > > <http://cfrac_12_15_gt0p9.nc>
> cfrac_12_63_gt0p2_le0p3.nc
> > > >> > > >
<http://cfrac_12_63_gt0p2_le0p3.nc>cfrac_06_39_gt0p6_le0p7.nc
> > > >> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > > >> > > > <http://cfrac_12_16_0.nc>
> cfrac_12_63_gt0p3_le0p4.nc
> > > >> > > >
<http://cfrac_12_63_gt0p3_le0p4.nc>cfrac_06_39_gt0p7_le0p8.nc
> > > >> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
cfrac_12_16_gt0_le0p1.nc
> > > >> > > > <http://cfrac_12_16_gt0_le0p1.nc>
> cfrac_12_63_gt0p4_le0p5.nc
> > > >> > > >
<http://cfrac_12_63_gt0p4_le0p5.nc>cfrac_06_39_gt0p8_le0p9.nc
> > > >> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
> cfrac_12_16_gt0p1_le0p2.nc
> > > >> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
> cfrac_12_63_gt0p5_le0p6.nc
> > > >> > > > <http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > >> > > > <http://cfrac_06_39_gt0p9.nc>
> cfrac_12_16_gt0p2_le0p3.nc
> > > >> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
> cfrac_12_63_gt0p6_le0p7.nc
> > > >> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > >> > > > <http://cfrac_06_40_0.nc>
> cfrac_12_16_gt0p3_le0p4.nc
> > > >> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
> cfrac_12_63_gt0p7_le0p8.nc
> > > >> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > > >> > > >
> > > >> > > > The naming convention of the file would then be
something like
> > > this
> > > >> in
> > > >> > > the
> > > >> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > > >> > > >
> > > >> > > > In the past, when I tried to process 3D fields stored
in
> netcdf
> > > >> files,
> > > >> > I
> > > >> > > > found multiple dimensional arrays greater than 2
(horizontal
> > > >> > dimensions)
> > > >> > > to
> > > >> > > > be a problem.  If I attempted to consolidate these
files then
> I
> > > >> would
> > > >> > > have
> > > >> > > > a 4D matrix that would resemble the following:  (cloud
mask,
> > time,
> > > >> lon,
> > > >> > > > lat).  I'm not sure that MET would like that. Am I
right that
> > this
> > > >> > could
> > > >> > > be
> > > >> > > > a problem?
> > > >> > > >
> > > >> > > > Thanks in advance to any insight you can provide.
> > > >> > > >
> > > >> > > > --
> > > >> > > > Edward Strobach
> > > >> > > > EMC/NCEP/NWS/
> > > >> > > > IMSG Contractor
> > > >> > > > Cubicle#: 2029
> > > >> > > > 301-683-3717
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > > --
> > > >> > > Julie Prestopnik (she/her/hers)
> > > >> > > Software Engineer
> > > >> > > National Center for Atmospheric Research
> > > >> > > Research Applications Laboratory
> > > >> > > Email: jpresto at ucar.edu
> > > >> > >
> > > >> > > My working day may not be your working day.  Please do
not feel
> > > >> obliged
> > > >> > to
> > > >> > > reply to this email outside of your normal working hours.
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >> --
> > > >> Edward Strobach
> > > >> EMC/NCEP/NWS/
> > > >> IMSG Contractor
> > > >> Cubicle#: 2029
> > > >> 301-683-3717
> > > >>
> > > >>
> > >
> > >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>

--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717

------------------------------------------------
Subject: innumerable masks!
From: John Halley Gotway
Time: Fri Oct 23 10:42:42 2020

Ed,

Sounds good. Here's the existing documentation for grid_diag:
https://dtcenter.github.io/MET/latest/Users_Guide/grid-
diag.html?highlight=grid_diag#

Can I go ahead and resolve this ticket? Or do you have additional
questions/issues?

Thanks,
John

On Wed, Oct 21, 2020 at 8:49 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
>
> Hi John,
>
> I think I'll want to avoid creating any cloud fraction masks.  It's
too
> much at this point.  The only time I've used the intersection option
is to
> group land use categories for different geographical regions.  It
sounds
> like I should avoid that though based on what you are saying.  The
> grid_diag tool does sound appealing.  It sounds like I might be able
to
> bypass the need to generate cloud fraction masks if this option is
used
> instead.  I'd like to know more about this tool.
>
> On Tue, Oct 20, 2020 at 3:39 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ed,
> >
> > Ah sorry, I missed that detail.  Then you're right. The only way
to have
> > one field impact the verification of another is through the use of
> masking
> > regions. So you'd use the cloud fraction to define the data mask
based on
> > the forecast cloud fraction amount. And the only way to INTERSECT
that
> mask
> > with another (e.g. cloud fraction from 0.5 to 0.6 AND CONUS) is
> > through gen_vx_mask.
> >
> > For now, if possible, I'd recommend:
> > (1) Mask spatially using the output from gen_vx_mask (e.g. CONUS,
SWD,
> ...)
> > (2) Do data masking of land use type directly in the point_stat
config
> file
> > (3) Do cloud fraction masking directly in the point_stat config
file
> >
> > Do NOT try to intersect the masks from (1) with the data masks
from (2).
> > And we can figure out if/how to do so based on that GitHub issue.
But of
> > course it's up to you to decide if running gen_vx_mask all those
times is
> > reasonable.
> >
> > Another thought is running the relatively new grid_diag tool. That
is
> used
> > to compute more diagnostics instead of verification. It can be
used to
> bin
> > the values from one variable (e.g. PM2.5) relative to another
(e.g.
> > cloud fraction). It won't solve these specific issues but may be
> > interesting.
> >
> > John
> >
> > On Tue, Oct 20, 2020 at 1:21 PM Edward Strobach - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > >
> > > Thanks I'll take a look.  Of course some masks are static (land
type)
> > while
> > > others are more dynamic (cloud fraction).  I imagine dynamic
masks
> > require
> > > more of a delicate touch.
> > >
> > > On Tue, Oct 20, 2020 at 3:15 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Ed,
> > > >
> > > > FYI, I wrote up this GitHub issue:
> > > > https://github.com/dtcenter/MET/issues/1530
> > > >
> > > > Please take a look and let me know what you think. I'm not
sure
> what's
> > > > reasonable to be done here. I suppose in an ideal world, all
the
> > > > functionality of gen_vx_mask would be available via the MET
config
> > files,
> > > > and the only reason to define them one place versus another
would be
> > > > efficiency. It's quicker to define it once and use it many
times than
> > > > re-define each time. But that would be a pretty significant
change.
> > > >
> > > > John
> > > >
> > > > On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway
<johnhg at ucar.edu>
> > > > wrote:
> > > >
> > > > > Ed,
> > > > >
> > > > > The multiplicative bias statistic is called the mean error,
or ME,
> in
> > > the
> > > > > CNT column. It's just the avg fcst value - avg obs value.
But what
> > you
> > > > want
> > > > > to do is called "conditional continuous" verification.
Rather than
> > > > > computing stats over all the points, you want to condition
the
> > > > verification
> > > > > based on the value, whereas masking conditions based on the
> location.
> > > > You
> > > > > control this using the "cnt_thresh" and "cnt_logic" config
options.
> > You
> > > > can
> > > > > condition based on the forecast value, observation value, or
both.
> > For
> > > > > simplicity, I'd recommend starting with the observation
value.
> > > > >
> > > > > So in the obs dictionary, define:
> > > > >    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3,
>0.3&&<=0.4,
> > > > > >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8,
>0.8&&<=0.9,
> >0.9
> > > ];
> > > > >
> > > > > For each 1 CNT line from the output, you should now see 11:
1 with
> > > > > OBS_THRESH = NA including all the points, and 10 more with
> > OBS_THRESH =
> > > > the
> > > > > bins listed above.
> > > > >
> > > > > Now I haven't actually done this... but hopefully you can
use
> > METviewer
> > > > to
> > > > > make the plot you described.
> > > > >
> > > > > For frequency bias, the contents of the CTC, CTS, MCTC, and
MCTS
> line
> > > > > types are determined by the cat_thresh setting. It could be
> > interesting
> > > > to
> > > > > define cat_thresh as shown below and look at the MCTS output
line
> > type:
> > > > >    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7,
>.8, >.9
> ];
> > > > >
> > > > > The MCTS output is described here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
> > > > >
> > > > > Hope that helps.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA
Affiliate
> via
> > > RT
> > > > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > > > >>
> > > > >> Hi John,
> > > > >>
> > > > >> See my responses below
> > > > >>
> > > > >> Hi Ed,
> > > > >>
> > > > >> I see you have some questions about masking. I’m answering
on my
> > phone
> > > > >> right now so I can’t provide specific details. But let me
know if
> > > > anything
> > > > >> isn’t clear and I can follow up with specific examples
later.
> > > > >>
> > > > >> I read the description of the plot you’d like to create and
> believe
> > > that
> > > > >> it
> > > > >> can be accomplished without the use of mask regions.
> > > > >>
> > > > >> That would save a lot of space and complexity if I could do
that
> > > > >>
> > > > >> But before proceeding to that, let me point out that you
can apply
> > > data
> > > > >> masks in point_stat WITHOUT running gen_vx_mask first.
However for
> > > > >> “complex” masks (e.g. combining 2 masks, like SWD with land
use
> > type)
> > > > you
> > > > >> do need to run gen_vx_mask first. We could conceivably
enhance MET
> > to
> > > > >> support the generation of complex masks on the fly in the
config
> > file
> > > to
> > > > >> avoid the need for running gen_vx_mask. So let me know if
you
> think
> > > > that’s
> > > > >> worth documenting as a feature request.
> > > > >>
> > > > >> The application of masks without gen_vx_mask sounds vaguely
> > familiar.
> > > > >> Maybe it was mentioned in an old email.  I'll try to look
for
> > that.  I
> > > > >> have
> > > > >> used the intersection tool to do this.  I found that I had
to run
> > > > >> gen_vx_mask twice:  one to generate a netcdf file of the
cropped
> out
> > > > >> region
> > > > >> and another to identify land use type in that region.  Yes,
I
> think
> > > > >> anything that can reduce specifying numerous pathways in
the
> > PointStat
> > > > >> file
> > > > >> under the mask definition would be a great alleviation.  I
support
> > > > making
> > > > >> a
> > > > >> note of that.  Thanks.
> > > > >>
> > > > >> But the next part depends on what type of “bias” you’re
looking
> for.
> > > > >> Either
> > > > >> multiplicative bias... ie avg forecast value - avg observed
value
> > from
> > > > the
> > > > >> CNT line? Or categorical frequency bias... ie #fcst events
/ #obs
> > > events
> > > > >> from the CTS line? Which are you looking for?
> > > > >>
> > > > >> I was leaning toward multiplicative bias.  The idea would
be to
> bin
> > > out
> > > > a
> > > > >> range of cloud fractions in steps of 10 (0; 0 to .1, .1 to
.2, and
> > so
> > > > on)
> > > > >> on the X-axis with a multiplicative bias on the y-axis.  It
might
> be
> > > > more
> > > > >> visually appealing if I used the bar graph option for that
which I
> > > know
> > > > >> MET
> > > > >> can do.   Let me know if there is more information that you
need
> > from
> > > me
> > > > >> so
> > > > >> I can help with this.  Thanks
> > > > >>
> > > > >> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> > Hi Ed,
> > > > >> >
> > > > >> > I see you have some questions about masking. I’m
answering on my
> > > phone
> > > > >> > right now so I can’t provide specific details. But let me
know
> if
> > > > >> anything
> > > > >> > isn’t clear and I can follow up with specific examples
later.
> > > > >> >
> > > > >> > I read the description of the plot you’d like to create
and
> > believe
> > > > >> that it
> > > > >> > can be accomplished without the use of mask regions.
> > > > >> >
> > > > >> > But before proceeding to that, let me point out that you
can
> apply
> > > > data
> > > > >> > masks in point_stat WITHOUT running gen_vx_mask first.
However
> for
> > > > >> > “complex” masks (e.g. combining 2 masks, like SWD with
land use
> > > type)
> > > > >> you
> > > > >> > do need to run gen_vx_mask first. We could conceivably
enhance
> MET
> > > to
> > > > >> > support the generation of complex masks on the fly in the
config
> > > file
> > > > to
> > > > >> > avoid the need for running gen_vx_mask. So let me know if
you
> > think
> > > > >> that’s
> > > > >> > worth documenting as a feature request.
> > > > >> >
> > > > >> > But the next part depends on what type of “bias” you’re
looking
> > for.
> > > > >> Either
> > > > >> > multiplicative bias... ie avg forecast value - avg
observed
> value
> > > from
> > > > >> the
> > > > >> > CNT line? Or categorical frequency bias... ie #fcst
events /
> #obs
> > > > events
> > > > >> > from the CTS line? Which are you looking for?
> > > > >> >
> > > > >> > Thanks,
> > > > >> > John
> > > > >> >
> > > > >> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT <
> > > > >> met_help at ucar.edu
> > > > >> > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> >
> > > > >> > >
> > > > >> > > Hi Edward.
> > > > >> > >
> > > > >> > > I have assigned this ticket to John HG.  Please allow a
few
> > > business
> > > > >> days
> > > > >> > > for a response.
> > > > >> > >
> > > > >> > > Julie
> > > > >> > >
> > > > >> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach - NOAA
> Affiliate
> > > via
> > > > >> RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted
upon.
> > > > >> > > > Transaction: Ticket created by
edward.strobach at noaa.gov
> > > > >> > > >        Queue: met_help
> > > > >> > > >      Subject: innumerable masks!
> > > > >> > > >        Owner: Nobody
> > > > >> > > >   Requestors: edward.strobach at noaa.gov
> > > > >> > > >       Status: new
> > > > >> > > >  Ticket <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > > > >> > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > Good morning,
> > > > >> > > >
> > > > >> > > > I've been exploring ways to create various masks with
some
> > > > success.
> > > > >> > > Right
> > > > >> > > > now I produce the following masks each day:
> > > > >> > > > -dominant land use type (24 files - manageable)
> > > > >> > > > -dominant land use type by region (sum of the number
of land
> > use
> > > > >> types
> > > > >> > > for
> > > > >> > > > each region (e.g. SWC, WEST, etc. - approaching
> unmanageable)
> > > > >> > > > -cloud fraction (10 * fh * run - very unmanageable).
> > > > >> > > >
> > > > >> > > > The last point prompted me to send this email.  Right
now,
> for
> > > > each
> > > > >> > > > experiment and forecast hour, I generate a cloud mask
for
> > clear
> > > > >> skies,
> > > > >> > if
> > > > >> > > > the grid is between 0 to 10 % occupied, if the grid
square
> is
> > > > >> greater
> > > > >> > > than
> > > > >> > > > 10% occupied but less than 20% occupied, all the way
to
> 100%.
> > > > This
> > > > >> > > brings
> > > > >> > > > us to about 10 files for each forecast hour.
However, since
> > I'm
> > > > >> > running
> > > > >> > > 6
> > > > >> > > > experiments out to 72 hours, you can now see that I
would
> > easily
> > > > be
> > > > >> > > > generating over 1000 files each time (:-/).
> > > > >> > > >
> > > > >> > > > I'm trying to use the cloud fraction field to do the
> > following:
> > > > >> x-axis
> > > > >> > > is
> > > > >> > > > the bin cloud fraction (i.e. cloud fraction = 0;
cloud
> > fraction
> > > > >> > between 0
> > > > >> > > > and 10, etc.) while y-axis would be the bias of a
field
> (e.g.
> > > > PM2.5
> > > > >> or
> > > > >> > > > ozone) collocated with obs.  I'd do this for all
> geographical
> > > > >> regions
> > > > >> > > > ideally, realizing that not every bin would be
populated.
> In
> > > some
> > > > >> > > > instances an entire region might be under clear
skies.  This
> > may
> > > > be
> > > > >> a
> > > > >> > bit
> > > > >> > > > of a pipe dream, but can anyone see the feasibility
of
> > > attempting
> > > > >> this,
> > > > >> > > or
> > > > >> > > > is this beyond the scope of what can currently be
done in an
> > > > >> efficient
> > > > >> > > > manner?
> > > > >> > > >
> > > > >> > > > -In short, I'm wondering what I can do to cut down on
these
> > > files
> > > > >> (lump
> > > > >> > > > each fraction into one file for all forecast hours?).
> > > > >> > > > -Would I be able to limit the point stat_config to
just 10
> > > pathway
> > > > >> > lines?
> > > > >> > > > Right now I have things stored in the following way
each
> day:
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > > >> > > > Here, para6 ($model) is the model while 20201010 is
the date
> > > > >> ($DATE).
> > > > >> > > > In PointStat_AIRNOW - the file I use stored in my
met_config
> > - I
> > > > >> would
> > > > >> > > look
> > > > >> > > > something like this:
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > > >> > > > However, as indicated by the pasted example, you can
see the
> > > files
> > > > >> and
> > > > >> > > how
> > > > >> > > > extensive they could be:
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > *cfrac_06_39_gt0_le0p1.nc
<http://cfrac_06_39_gt0_le0p1.nc>
> > > > >> > > >  cfrac_12_15_gt0p4_le0p5.nc <
> > http://cfrac_12_15_gt0p4_le0p5.nc>
> > > > >> > > >  cfrac_12_62_gt0p8_le0p9.nc
> > > > >> > > > <http://cfrac_12_62_gt0p8_le0p9.nc>
> cfrac_06_39_gt0p1_le0p2.nc
> > > > >> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
> > cfrac_12_15_gt0p5_le0p6.nc
> > > > >> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>
cfrac_12_62_gt0p9.nc
> > > > >> > > >
<http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > > >> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
> > cfrac_12_15_gt0p6_le0p7.nc
> > > > >> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>  cfrac_12_63_0.nc
> > > > >> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > > >> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
> > cfrac_12_15_gt0p7_le0p8.nc
> > > > >> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
> cfrac_12_63_gt0_le0p1.nc
> > > > >> > > >
<http://cfrac_12_63_gt0_le0p1.nc>cfrac_06_39_gt0p4_le0p5.nc
> > > > >> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
> > cfrac_12_15_gt0p8_le0p9.nc
> > > > >> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
> > cfrac_12_63_gt0p1_le0p2.nc
> > > > >> > > > <http://cfrac_12_63_gt0p1_le0p2.nc>
> cfrac_06_39_gt0p5_le0p6.nc
> > > > >> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>
cfrac_12_15_gt0p9.nc
> > > > >> > > > <http://cfrac_12_15_gt0p9.nc>
> > cfrac_12_63_gt0p2_le0p3.nc
> > > > >> > > > <http://cfrac_12_63_gt0p2_le0p3.nc>
> cfrac_06_39_gt0p6_le0p7.nc
> > > > >> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>  cfrac_12_16_0.nc
> > > > >> > > > <http://cfrac_12_16_0.nc>
> > cfrac_12_63_gt0p3_le0p4.nc
> > > > >> > > > <http://cfrac_12_63_gt0p3_le0p4.nc>
> cfrac_06_39_gt0p7_le0p8.nc
> > > > >> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
> cfrac_12_16_gt0_le0p1.nc
> > > > >> > > > <http://cfrac_12_16_gt0_le0p1.nc>
> > cfrac_12_63_gt0p4_le0p5.nc
> > > > >> > > > <http://cfrac_12_63_gt0p4_le0p5.nc>
> cfrac_06_39_gt0p8_le0p9.nc
> > > > >> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
> > cfrac_12_16_gt0p1_le0p2.nc
> > > > >> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
> > cfrac_12_63_gt0p5_le0p6.nc
> > > > >> > > >
<http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > > >> > > > <http://cfrac_06_39_gt0p9.nc>
> > cfrac_12_16_gt0p2_le0p3.nc
> > > > >> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
> > cfrac_12_63_gt0p6_le0p7.nc
> > > > >> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > > >> > > > <http://cfrac_06_40_0.nc>
> > cfrac_12_16_gt0p3_le0p4.nc
> > > > >> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
> > cfrac_12_63_gt0p7_le0p8.nc
> > > > >> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > > > >> > > >
> > > > >> > > > The naming convention of the file would then be
something
> like
> > > > this
> > > > >> in
> > > > >> > > the
> > > > >> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > > > >> > > >
> > > > >> > > > In the past, when I tried to process 3D fields stored
in
> > netcdf
> > > > >> files,
> > > > >> > I
> > > > >> > > > found multiple dimensional arrays greater than 2
(horizontal
> > > > >> > dimensions)
> > > > >> > > to
> > > > >> > > > be a problem.  If I attempted to consolidate these
files
> then
> > I
> > > > >> would
> > > > >> > > have
> > > > >> > > > a 4D matrix that would resemble the following:
(cloud mask,
> > > time,
> > > > >> lon,
> > > > >> > > > lat).  I'm not sure that MET would like that. Am I
right
> that
> > > this
> > > > >> > could
> > > > >> > > be
> > > > >> > > > a problem?
> > > > >> > > >
> > > > >> > > > Thanks in advance to any insight you can provide.
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Edward Strobach
> > > > >> > > > EMC/NCEP/NWS/
> > > > >> > > > IMSG Contractor
> > > > >> > > > Cubicle#: 2029
> > > > >> > > > 301-683-3717
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > > --
> > > > >> > > Julie Prestopnik (she/her/hers)
> > > > >> > > Software Engineer
> > > > >> > > National Center for Atmospheric Research
> > > > >> > > Research Applications Laboratory
> > > > >> > > Email: jpresto at ucar.edu
> > > > >> > >
> > > > >> > > My working day may not be your working day.  Please do
not
> feel
> > > > >> obliged
> > > > >> > to
> > > > >> > > reply to this email outside of your normal working
hours.
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >> --
> > > > >> Edward Strobach
> > > > >> EMC/NCEP/NWS/
> > > > >> IMSG Contractor
> > > > >> Cubicle#: 2029
> > > > >> 301-683-3717
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> > >
> >
> >
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>

------------------------------------------------
Subject: innumerable masks!
From: Edward Strobach - NOAA Affiliate
Time: Fri Oct 23 10:44:28 2020

You can mark this as resolved. Thanks

On Fri, Oct 23, 2020 at 12:42 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Ed,
>
> Sounds good. Here's the existing documentation for grid_diag:
>
> https://dtcenter.github.io/MET/latest/Users_Guide/grid-
diag.html?highlight=grid_diag#
>
> Can I go ahead and resolve this ticket? Or do you have additional
> questions/issues?
>
> Thanks,
> John
>
> On Wed, Oct 21, 2020 at 8:49 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> >
> > Hi John,
> >
> > I think I'll want to avoid creating any cloud fraction masks.
It's too
> > much at this point.  The only time I've used the intersection
option is
> to
> > group land use categories for different geographical regions.  It
sounds
> > like I should avoid that though based on what you are saying.  The
> > grid_diag tool does sound appealing.  It sounds like I might be
able to
> > bypass the need to generate cloud fraction masks if this option is
used
> > instead.  I'd like to know more about this tool.
> >
> > On Tue, Oct 20, 2020 at 3:39 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Ed,
> > >
> > > Ah sorry, I missed that detail.  Then you're right. The only way
to
> have
> > > one field impact the verification of another is through the use
of
> > masking
> > > regions. So you'd use the cloud fraction to define the data mask
based
> on
> > > the forecast cloud fraction amount. And the only way to
INTERSECT that
> > mask
> > > with another (e.g. cloud fraction from 0.5 to 0.6 AND CONUS) is
> > > through gen_vx_mask.
> > >
> > > For now, if possible, I'd recommend:
> > > (1) Mask spatially using the output from gen_vx_mask (e.g.
CONUS, SWD,
> > ...)
> > > (2) Do data masking of land use type directly in the point_stat
config
> > file
> > > (3) Do cloud fraction masking directly in the point_stat config
file
> > >
> > > Do NOT try to intersect the masks from (1) with the data masks
from
> (2).
> > > And we can figure out if/how to do so based on that GitHub
issue. But
> of
> > > course it's up to you to decide if running gen_vx_mask all those
times
> is
> > > reasonable.
> > >
> > > Another thought is running the relatively new grid_diag tool.
That is
> > used
> > > to compute more diagnostics instead of verification. It can be
used to
> > bin
> > > the values from one variable (e.g. PM2.5) relative to another
(e.g.
> > > cloud fraction). It won't solve these specific issues but may be
> > > interesting.
> > >
> > > John
> > >
> > > On Tue, Oct 20, 2020 at 1:21 PM Edward Strobach - NOAA Affiliate
via
> RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
>
> > > >
> > > > Thanks I'll take a look.  Of course some masks are static
(land type)
> > > while
> > > > others are more dynamic (cloud fraction).  I imagine dynamic
masks
> > > require
> > > > more of a delicate touch.
> > > >
> > > > On Tue, Oct 20, 2020 at 3:15 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Ed,
> > > > >
> > > > > FYI, I wrote up this GitHub issue:
> > > > > https://github.com/dtcenter/MET/issues/1530
> > > > >
> > > > > Please take a look and let me know what you think. I'm not
sure
> > what's
> > > > > reasonable to be done here. I suppose in an ideal world, all
the
> > > > > functionality of gen_vx_mask would be available via the MET
config
> > > files,
> > > > > and the only reason to define them one place versus another
would
> be
> > > > > efficiency. It's quicker to define it once and use it many
times
> than
> > > > > re-define each time. But that would be a pretty significant
change.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Oct 20, 2020 at 1:03 PM John Halley Gotway <
> johnhg at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Ed,
> > > > > >
> > > > > > The multiplicative bias statistic is called the mean
error, or
> ME,
> > in
> > > > the
> > > > > > CNT column. It's just the avg fcst value - avg obs value.
But
> what
> > > you
> > > > > want
> > > > > > to do is called "conditional continuous" verification.
Rather
> than
> > > > > > computing stats over all the points, you want to condition
the
> > > > > verification
> > > > > > based on the value, whereas masking conditions based on
the
> > location.
> > > > > You
> > > > > > control this using the "cnt_thresh" and "cnt_logic" config
> options.
> > > You
> > > > > can
> > > > > > condition based on the forecast value, observation value,
or
> both.
> > > For
> > > > > > simplicity, I'd recommend starting with the observation
value.
> > > > > >
> > > > > > So in the obs dictionary, define:
> > > > > >    cnt_thresh = [ NA, <=0.1, >0.1&&<=0.2, >0.2&&<=0.3,
> >0.3&&<=0.4,
> > > > > > >0.4&&<=0.5, >0.5&&<=0.6, >0.6&&<=0.7, >0.7&&<=0.8,
>0.8&&<=0.9,
> > >0.9
> > > > ];
> > > > > >
> > > > > > For each 1 CNT line from the output, you should now see
11: 1
> with
> > > > > > OBS_THRESH = NA including all the points, and 10 more with
> > > OBS_THRESH =
> > > > > the
> > > > > > bins listed above.
> > > > > >
> > > > > > Now I haven't actually done this... but hopefully you can
use
> > > METviewer
> > > > > to
> > > > > > make the plot you described.
> > > > > >
> > > > > > For frequency bias, the contents of the CTC, CTS, MCTC,
and MCTS
> > line
> > > > > > types are determined by the cat_thresh setting. It could
be
> > > interesting
> > > > > to
> > > > > > define cat_thresh as shown below and look at the MCTS
output line
> > > type:
> > > > > >    cat_thresh = [ >0, >.1, >.2, >.3, >.4, >.5, >.6, >.7,
>.8, >.9
> > ];
> > > > > >
> > > > > > The MCTS output is described here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html?highlight=mcts#id13
> > > > > >
> > > > > > Hope that helps.
> > > > > >
> > > > > > John
> > > > > >
> > > > > > On Tue, Oct 20, 2020 at 11:59 AM Edward Strobach - NOAA
Affiliate
> > via
> > > > RT
> > > > > <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > >>
> > > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136 >
> > > > > >>
> > > > > >> Hi John,
> > > > > >>
> > > > > >> See my responses below
> > > > > >>
> > > > > >> Hi Ed,
> > > > > >>
> > > > > >> I see you have some questions about masking. I’m
answering on my
> > > phone
> > > > > >> right now so I can’t provide specific details. But let me
know
> if
> > > > > anything
> > > > > >> isn’t clear and I can follow up with specific examples
later.
> > > > > >>
> > > > > >> I read the description of the plot you’d like to create
and
> > believe
> > > > that
> > > > > >> it
> > > > > >> can be accomplished without the use of mask regions.
> > > > > >>
> > > > > >> That would save a lot of space and complexity if I could
do that
> > > > > >>
> > > > > >> But before proceeding to that, let me point out that you
can
> apply
> > > > data
> > > > > >> masks in point_stat WITHOUT running gen_vx_mask first.
However
> for
> > > > > >> “complex” masks (e.g. combining 2 masks, like SWD with
land use
> > > type)
> > > > > you
> > > > > >> do need to run gen_vx_mask first. We could conceivably
enhance
> MET
> > > to
> > > > > >> support the generation of complex masks on the fly in the
config
> > > file
> > > > to
> > > > > >> avoid the need for running gen_vx_mask. So let me know if
you
> > think
> > > > > that’s
> > > > > >> worth documenting as a feature request.
> > > > > >>
> > > > > >> The application of masks without gen_vx_mask sounds
vaguely
> > > familiar.
> > > > > >> Maybe it was mentioned in an old email.  I'll try to look
for
> > > that.  I
> > > > > >> have
> > > > > >> used the intersection tool to do this.  I found that I
had to
> run
> > > > > >> gen_vx_mask twice:  one to generate a netcdf file of the
cropped
> > out
> > > > > >> region
> > > > > >> and another to identify land use type in that region.
Yes, I
> > think
> > > > > >> anything that can reduce specifying numerous pathways in
the
> > > PointStat
> > > > > >> file
> > > > > >> under the mask definition would be a great alleviation.
I
> support
> > > > > making
> > > > > >> a
> > > > > >> note of that.  Thanks.
> > > > > >>
> > > > > >> But the next part depends on what type of “bias” you’re
looking
> > for.
> > > > > >> Either
> > > > > >> multiplicative bias... ie avg forecast value - avg
observed
> value
> > > from
> > > > > the
> > > > > >> CNT line? Or categorical frequency bias... ie #fcst
events /
> #obs
> > > > events
> > > > > >> from the CTS line? Which are you looking for?
> > > > > >>
> > > > > >> I was leaning toward multiplicative bias.  The idea would
be to
> > bin
> > > > out
> > > > > a
> > > > > >> range of cloud fractions in steps of 10 (0; 0 to .1, .1
to .2,
> and
> > > so
> > > > > on)
> > > > > >> on the X-axis with a multiplicative bias on the y-axis.
It
> might
> > be
> > > > > more
> > > > > >> visually appealing if I used the bar graph option for
that
> which I
> > > > know
> > > > > >> MET
> > > > > >> can do.   Let me know if there is more information that
you need
> > > from
> > > > me
> > > > > >> so
> > > > > >> I can help with this.  Thanks
> > > > > >>
> > > > > >> On Tue, Oct 20, 2020 at 12:46 PM John Halley Gotway via
RT <
> > > > > >> met_help at ucar.edu> wrote:
> > > > > >>
> > > > > >> > Hi Ed,
> > > > > >> >
> > > > > >> > I see you have some questions about masking. I’m
answering on
> my
> > > > phone
> > > > > >> > right now so I can’t provide specific details. But let
me know
> > if
> > > > > >> anything
> > > > > >> > isn’t clear and I can follow up with specific examples
later.
> > > > > >> >
> > > > > >> > I read the description of the plot you’d like to create
and
> > > believe
> > > > > >> that it
> > > > > >> > can be accomplished without the use of mask regions.
> > > > > >> >
> > > > > >> > But before proceeding to that, let me point out that
you can
> > apply
> > > > > data
> > > > > >> > masks in point_stat WITHOUT running gen_vx_mask first.
However
> > for
> > > > > >> > “complex” masks (e.g. combining 2 masks, like SWD with
land
> use
> > > > type)
> > > > > >> you
> > > > > >> > do need to run gen_vx_mask first. We could conceivably
enhance
> > MET
> > > > to
> > > > > >> > support the generation of complex masks on the fly in
the
> config
> > > > file
> > > > > to
> > > > > >> > avoid the need for running gen_vx_mask. So let me know
if you
> > > think
> > > > > >> that’s
> > > > > >> > worth documenting as a feature request.
> > > > > >> >
> > > > > >> > But the next part depends on what type of “bias” you’re
> looking
> > > for.
> > > > > >> Either
> > > > > >> > multiplicative bias... ie avg forecast value - avg
observed
> > value
> > > > from
> > > > > >> the
> > > > > >> > CNT line? Or categorical frequency bias... ie #fcst
events /
> > #obs
> > > > > events
> > > > > >> > from the CTS line? Which are you looking for?
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > John
> > > > > >> >
> > > > > >> > On Tue, Oct 20, 2020 at 9:14 AM Julie Prestopnik via RT
<
> > > > > >> met_help at ucar.edu
> > > > > >> > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > >
> > > > > >> > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > >
> > > > > >> > >
> > > > > >> > > Hi Edward.
> > > > > >> > >
> > > > > >> > > I have assigned this ticket to John HG.  Please allow
a few
> > > > business
> > > > > >> days
> > > > > >> > > for a response.
> > > > > >> > >
> > > > > >> > > Julie
> > > > > >> > >
> > > > > >> > > On Tue, Oct 20, 2020 at 7:08 AM Edward Strobach -
NOAA
> > Affiliate
> > > > via
> > > > > >> RT <
> > > > > >> > > met_help at ucar.edu> wrote:
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > Tue Oct 20 07:07:45 2020: Request 97136 was acted
upon.
> > > > > >> > > > Transaction: Ticket created by
edward.strobach at noaa.gov
> > > > > >> > > >        Queue: met_help
> > > > > >> > > >      Subject: innumerable masks!
> > > > > >> > > >        Owner: Nobody
> > > > > >> > > >   Requestors: edward.strobach at noaa.gov
> > > > > >> > > >       Status: new
> > > > > >> > > >  Ticket <URL:
> > > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97136
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > Good morning,
> > > > > >> > > >
> > > > > >> > > > I've been exploring ways to create various masks
with some
> > > > > success.
> > > > > >> > > Right
> > > > > >> > > > now I produce the following masks each day:
> > > > > >> > > > -dominant land use type (24 files - manageable)
> > > > > >> > > > -dominant land use type by region (sum of the
number of
> land
> > > use
> > > > > >> types
> > > > > >> > > for
> > > > > >> > > > each region (e.g. SWC, WEST, etc. - approaching
> > unmanageable)
> > > > > >> > > > -cloud fraction (10 * fh * run - very
unmanageable).
> > > > > >> > > >
> > > > > >> > > > The last point prompted me to send this email.
Right now,
> > for
> > > > > each
> > > > > >> > > > experiment and forecast hour, I generate a cloud
mask for
> > > clear
> > > > > >> skies,
> > > > > >> > if
> > > > > >> > > > the grid is between 0 to 10 % occupied, if the grid
square
> > is
> > > > > >> greater
> > > > > >> > > than
> > > > > >> > > > 10% occupied but less than 20% occupied, all the
way to
> > 100%.
> > > > > This
> > > > > >> > > brings
> > > > > >> > > > us to about 10 files for each forecast hour.
However,
> since
> > > I'm
> > > > > >> > running
> > > > > >> > > 6
> > > > > >> > > > experiments out to 72 hours, you can now see that I
would
> > > easily
> > > > > be
> > > > > >> > > > generating over 1000 files each time (:-/).
> > > > > >> > > >
> > > > > >> > > > I'm trying to use the cloud fraction field to do
the
> > > following:
> > > > > >> x-axis
> > > > > >> > > is
> > > > > >> > > > the bin cloud fraction (i.e. cloud fraction = 0;
cloud
> > > fraction
> > > > > >> > between 0
> > > > > >> > > > and 10, etc.) while y-axis would be the bias of a
field
> > (e.g.
> > > > > PM2.5
> > > > > >> or
> > > > > >> > > > ozone) collocated with obs.  I'd do this for all
> > geographical
> > > > > >> regions
> > > > > >> > > > ideally, realizing that not every bin would be
populated.
> > In
> > > > some
> > > > > >> > > > instances an entire region might be under clear
skies.
> This
> > > may
> > > > > be
> > > > > >> a
> > > > > >> > bit
> > > > > >> > > > of a pipe dream, but can anyone see the feasibility
of
> > > > attempting
> > > > > >> this,
> > > > > >> > > or
> > > > > >> > > > is this beyond the scope of what can currently be
done in
> an
> > > > > >> efficient
> > > > > >> > > > manner?
> > > > > >> > > >
> > > > > >> > > > -In short, I'm wondering what I can do to cut down
on
> these
> > > > files
> > > > > >> (lump
> > > > > >> > > > each fraction into one file for all forecast
hours?).
> > > > > >> > > > -Would I be able to limit the point stat_config to
just 10
> > > > pathway
> > > > > >> > lines?
> > > > > >> > > > Right now I have things stored in the following way
each
> > day:
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/para6/20201010.
> > > > > >> > > > Here, para6 ($model) is the model while 20201010 is
the
> date
> > > > > >> ($DATE).
> > > > > >> > > > In PointStat_AIRNOW - the file I use stored in my
> met_config
> > > - I
> > > > > >> would
> > > > > >> > > look
> > > > > >> > > > something like this:
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/masks/cloud_fraction/${model}/${DATE}.
> > > > > >> > > > However, as indicated by the pasted example, you
can see
> the
> > > > files
> > > > > >> and
> > > > > >> > > how
> > > > > >> > > > extensive they could be:
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > *cfrac_06_39_gt0_le0p1.nc <
> http://cfrac_06_39_gt0_le0p1.nc>
> > > > > >> > > >  cfrac_12_15_gt0p4_le0p5.nc <
> > > http://cfrac_12_15_gt0p4_le0p5.nc>
> > > > > >> > > >  cfrac_12_62_gt0p8_le0p9.nc
> > > > > >> > > > <http://cfrac_12_62_gt0p8_le0p9.nc>
> > cfrac_06_39_gt0p1_le0p2.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p1_le0p2.nc>
> > > cfrac_12_15_gt0p5_le0p6.nc
> > > > > >> > > > <http://cfrac_12_15_gt0p5_le0p6.nc>
cfrac_12_62_gt0p9.nc
> > > > > >> > > >
<http://cfrac_12_62_gt0p9.nc>cfrac_06_39_gt0p2_le0p3.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p2_le0p3.nc>
> > > cfrac_12_15_gt0p6_le0p7.nc
> > > > > >> > > > <http://cfrac_12_15_gt0p6_le0p7.nc>
cfrac_12_63_0.nc
> > > > > >> > > > <http://cfrac_12_63_0.nc>cfrac_06_39_gt0p3_le0p4.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p3_le0p4.nc>
> > > cfrac_12_15_gt0p7_le0p8.nc
> > > > > >> > > > <http://cfrac_12_15_gt0p7_le0p8.nc>
> > cfrac_12_63_gt0_le0p1.nc
> > > > > >> > > > <http://cfrac_12_63_gt0_le0p1.nc>
> cfrac_06_39_gt0p4_le0p5.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p4_le0p5.nc>
> > > cfrac_12_15_gt0p8_le0p9.nc
> > > > > >> > > > <http://cfrac_12_15_gt0p8_le0p9.nc>
> > > cfrac_12_63_gt0p1_le0p2.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p1_le0p2.nc>
> > cfrac_06_39_gt0p5_le0p6.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p5_le0p6.nc>
cfrac_12_15_gt0p9.nc
> > > > > >> > > > <http://cfrac_12_15_gt0p9.nc>
> > > cfrac_12_63_gt0p2_le0p3.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p2_le0p3.nc>
> > cfrac_06_39_gt0p6_le0p7.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p6_le0p7.nc>
cfrac_12_16_0.nc
> > > > > >> > > > <http://cfrac_12_16_0.nc>
> > > cfrac_12_63_gt0p3_le0p4.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p3_le0p4.nc>
> > cfrac_06_39_gt0p7_le0p8.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p7_le0p8.nc>
> > cfrac_12_16_gt0_le0p1.nc
> > > > > >> > > > <http://cfrac_12_16_gt0_le0p1.nc>
> > > cfrac_12_63_gt0p4_le0p5.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p4_le0p5.nc>
> > cfrac_06_39_gt0p8_le0p9.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p8_le0p9.nc>
> > > cfrac_12_16_gt0p1_le0p2.nc
> > > > > >> > > > <http://cfrac_12_16_gt0p1_le0p2.nc>
> > > cfrac_12_63_gt0p5_le0p6.nc
> > > > > >> > > >
<http://cfrac_12_63_gt0p5_le0p6.nc>cfrac_06_39_gt0p9.nc
> > > > > >> > > > <http://cfrac_06_39_gt0p9.nc>
> > > cfrac_12_16_gt0p2_le0p3.nc
> > > > > >> > > > <http://cfrac_12_16_gt0p2_le0p3.nc>
> > > cfrac_12_63_gt0p6_le0p7.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p6_le0p7.nc>cfrac_06_40_0.nc
> > > > > >> > > > <http://cfrac_06_40_0.nc>
> > > cfrac_12_16_gt0p3_le0p4.nc
> > > > > >> > > > <http://cfrac_12_16_gt0p3_le0p4.nc>
> > > cfrac_12_63_gt0p7_le0p8.nc
> > > > > >> > > > <http://cfrac_12_63_gt0p7_le0p8.nc>*
> > > > > >> > > >
> > > > > >> > > > The naming convention of the file would then be
something
> > like
> > > > > this
> > > > > >> in
> > > > > >> > > the
> > > > > >> > > > config file:  cfrac_${INIT}_${FH}_gt0p7_le0p8.nc
> > > > > >> > > >
> > > > > >> > > > In the past, when I tried to process 3D fields
stored in
> > > netcdf
> > > > > >> files,
> > > > > >> > I
> > > > > >> > > > found multiple dimensional arrays greater than 2
> (horizontal
> > > > > >> > dimensions)
> > > > > >> > > to
> > > > > >> > > > be a problem.  If I attempted to consolidate these
files
> > then
> > > I
> > > > > >> would
> > > > > >> > > have
> > > > > >> > > > a 4D matrix that would resemble the following:
(cloud
> mask,
> > > > time,
> > > > > >> lon,
> > > > > >> > > > lat).  I'm not sure that MET would like that. Am I
right
> > that
> > > > this
> > > > > >> > could
> > > > > >> > > be
> > > > > >> > > > a problem?
> > > > > >> > > >
> > > > > >> > > > Thanks in advance to any insight you can provide.
> > > > > >> > > >
> > > > > >> > > > --
> > > > > >> > > > Edward Strobach
> > > > > >> > > > EMC/NCEP/NWS/
> > > > > >> > > > IMSG Contractor
> > > > > >> > > > Cubicle#: 2029
> > > > > >> > > > 301-683-3717
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > > --
> > > > > >> > > Julie Prestopnik (she/her/hers)
> > > > > >> > > Software Engineer
> > > > > >> > > National Center for Atmospheric Research
> > > > > >> > > Research Applications Laboratory
> > > > > >> > > Email: jpresto at ucar.edu
> > > > > >> > >
> > > > > >> > > My working day may not be your working day.  Please
do not
> > feel
> > > > > >> obliged
> > > > > >> > to
> > > > > >> > > reply to this email outside of your normal working
hours.
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >> Edward Strobach
> > > > > >> EMC/NCEP/NWS/
> > > > > >> IMSG Contractor
> > > > > >> Cubicle#: 2029
> > > > > >> 301-683-3717
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > > >
> > >
> > >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>

--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717

------------------------------------------------