[Met_help] [rt.rap.ucar.edu #94684] History for Missing Values in gen_vx_mask
John Halley Gotway via RT
met_help at ucar.edu
Fri Mar 27 11:19:52 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I'm not seeing anything in the UG for how to handle missing values with gen_vx_mask, but basically I have a field (ceiling hgt), where points are assigned -9999 (the missing value). I want a mask of where there are clouds, so not including the areas of -9999.
gen_vx_mask seems to include missing vals in the mask, so a value of 1.
Is there another way around this? I can't think of one, but it is early on a Monday morning!
Thanks,
Tracy
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Missing Values in gen_vx_mask
From: Tracy Hertneky
Time: Thu Mar 26 12:57:06 2020
Along with what I previously said, this it the command I am using for
APCP
and REFC, where non-events are zero (but with CEIL, non-event's are
missing
values)
Example APCP (running on dakota)
/usr/local/met-8.1.1/bin/gen_vx_mask \
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_pairs.nc
\
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_pairs.nc
\
apcp_2018010800_f12.nc \
-type data \
-input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' \
-mask_field 'name="OBS_APCP_01_A1_FULL";level="L0";' \
-union -thresh gt0 \
-name apcp_2018010800_f12
This successfully produces a mask where either model has a precip
event
(>0) - image attached
This command was not working for ceiling where non-events are missing
values -9999 - the entire grid was 1-filled. I couldn't figure out a
way to
deal with missing values to get a similar mask for cloud ceiling (mask
of
where CIEL>0).
Thanks,
Tracy
On Mon, Mar 23, 2020 at 12:36 PM met_help at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> Greetings,
>
> This message has been automatically generated in response to the
creation
> of a trouble ticket regarding:
> "Missing Values in gen_vx_mask",
> a summary of which appears below.
>
> There is no need to reply to this message right now. Your ticket
has been
> assigned an ID of [rt.rap.ucar.edu #94684].
>
> Please include the string:
>
> [rt.rap.ucar.edu #94684]
>
> in the subject line of all future correspondence about this issue.
To do
> so, you may reply to this message.
>
> For more information, please see:
>
> MET Online Tutorial:
>
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
>
> MET Users Guide:
> https://www.dtcenter.org/met/users/docs/overview.php
>
> MET FAQs:
> https://www.dtcenter.org/met/users/support/faqs/index.php
>
> MET-Help Email Archive:
> http://mailman.ucar.edu/pipermail/met_help
>
> Thank you,
> met_help at ucar.edu
>
>
-------------------------------------------------------------------------
> I'm not seeing anything in the UG for how to handle missing values
with
> gen_vx_mask, but basically I have a field (ceiling hgt), where
points are
> assigned -9999 (the missing value). I want a mask of where there are
> clouds, so not including the areas of -9999.
>
> gen_vx_mask seems to include missing vals in the mask, so a value of
1.
>
> Is there another way around this? I can't think of one, but it is
early on
> a Monday morning!
>
> Thanks,
> Tracy
>
------------------------------------------------
Subject: Missing Values in gen_vx_mask
From: John Halley Gotway
Time: Thu Mar 26 14:19:59 2020
Tracy,
Based on the details you sent and doing some hunting around, I surmise
that.
This is the command that works well for APCP:
*/usr/local/met-8.1.1/bin/gen_vx_mask
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_pairs.nc
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_pairs.nc
apcp_2018010800_f12.nc <http://apcp_2018010800_f12.nc> -type data
-input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' -mask_field
'name="OBS_APCP_01_A1_FULL";level="L0";' -union -thresh gt0 -name
apcp_2018010800_f12 -v 3*
DEBUG 3: Data Masking: 158383 of 336000 points inside
This is the command that does NOT work well for CEIL:
*/usr/local/met-8.1.1/bin/gen_vx_mask
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_pairs.nc
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_pairs.nc
ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
-input_field 'name="FCST_CEIL_L0_FULL"; level="L0";' -mask_field
'name="OBS_CEIL_L0_FULL"; level="L0";' -union -thresh gt0 -name
apcp_2018010800_f12 -v 3*
DEBUG 3: Mask UNION: 336000 of 336000 points inside
It sounds like you want to create a mask of the grid points where the
forecast OR observation ceiling value is >0. You could do this in
multiple
calls to gen_vx_mask, but you could also do this using the
censor_thresh
and censor_val settings:
*/usr/local/met-8.1.1/bin/gen_vx_mask
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_pairs.nc
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_pairs.nc
ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
-input_field 'name="FCST_CEIL_L0_FULL"; level="L0";
censor_thresh=[eq-9999,gt0]; censor_val=[0,1];' -mask_field
'name="OBS_CEIL_L0_FULL"; level="L0"; censor_thresh=[eq-9999,gt0];
censor_val=[0,1];' -union -thresh gt0 -name ceil_2018010800_f12 -v 3*
DEBUG 3: Data Masking: 254162 of 336000 points inside
And that produces the attached making region.
Here's how the censor logic is working:
censor_thresh=[eq-9999,gt0]; censor_val=[0,1];
(1) Replace any data value ==-9999 with a value of 0.
(2) Replace any data value >0 with a value of 1.
Hope this approach helps.
Thanks,
John
On Thu, Mar 26, 2020 at 1:06 PM Tracy Hertneky via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684 >
>
> Along with what I previously said, this it the command I am using
for APCP
> and REFC, where non-events are zero (but with CEIL, non-event's are
missing
> values)
>
> Example APCP (running on dakota)
>
> /usr/local/met-8.1.1/bin/gen_vx_mask \
>
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> pairs.nc
> \
>
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> pairs.nc
> \
>
> apcp_2018010800_f12.nc \
>
> -type data \
>
> -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' \
>
> -mask_field 'name="OBS_APCP_01_A1_FULL";level="L0";' \
>
> -union -thresh gt0 \
>
> -name apcp_2018010800_f12
>
>
> This successfully produces a mask where either model has a precip
event
> (>0) - image attached
>
>
> This command was not working for ceiling where non-events are
missing
> values -9999 - the entire grid was 1-filled. I couldn't figure out a
way to
> deal with missing values to get a similar mask for cloud ceiling
(mask of
> where CIEL>0).
>
>
> Thanks,
>
> Tracy
>
> On Mon, Mar 23, 2020 at 12:36 PM met_help at ucar.edu via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Greetings,
> >
> > This message has been automatically generated in response to the
creation
> > of a trouble ticket regarding:
> > "Missing Values in gen_vx_mask",
> > a summary of which appears below.
> >
> > There is no need to reply to this message right now. Your ticket
has
> been
> > assigned an ID of [rt.rap.ucar.edu #94684].
> >
> > Please include the string:
> >
> > [rt.rap.ucar.edu #94684]
> >
> > in the subject line of all future correspondence about this issue.
To do
> > so, you may reply to this message.
> >
> > For more information, please see:
> >
> > MET Online Tutorial:
> >
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> >
> > MET Users Guide:
> > https://www.dtcenter.org/met/users/docs/overview.php
> >
> > MET FAQs:
> > https://www.dtcenter.org/met/users/support/faqs/index.php
> >
> > MET-Help Email Archive:
> > http://mailman.ucar.edu/pipermail/met_help
> >
> > Thank you,
> > met_help at ucar.edu
> >
> >
-------------------------------------------------------------------------
> > I'm not seeing anything in the UG for how to handle missing values
with
> > gen_vx_mask, but basically I have a field (ceiling hgt), where
points are
> > assigned -9999 (the missing value). I want a mask of where there
are
> > clouds, so not including the areas of -9999.
> >
> > gen_vx_mask seems to include missing vals in the mask, so a value
of 1.
> >
> > Is there another way around this? I can't think of one, but it is
early
> on
> > a Monday morning!
> >
> > Thanks,
> > Tracy
> >
>
>
------------------------------------------------
Subject: Missing Values in gen_vx_mask
From: Tracy Hertneky
Time: Thu Mar 26 16:49:19 2020
Thanks John, this is exactly what I was looking for. I wouldn't have
thought to use censor_thresh for changing the missing values.
I removed the values from your #(2) and still got the same thing since
I
had -thresh gt0 in that call already, which seems to basically do the
same
thing, but please correct me if I am wrong. Is there a reason you had
it
that way?
Thanks,
Tracy
On Thu, Mar 26, 2020 at 2:26 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Tracy,
>
> Based on the details you sent and doing some hunting around, I
surmise
> that.
>
> This is the command that works well for APCP:
>
> */usr/local/met-8.1.1/bin/gen_vx_mask
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> pairs.nc
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> pairs.nc
> apcp_2018010800_f12.nc <http://apcp_2018010800_f12.nc> -type data
> -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' -mask_field
> 'name="OBS_APCP_01_A1_FULL";level="L0";' -union -thresh gt0 -name
> apcp_2018010800_f12 -v 3*
> DEBUG 3: Data Masking: 158383 of 336000 points inside
>
> This is the command that does NOT work well for CEIL:
>
> */usr/local/met-8.1.1/bin/gen_vx_mask
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> pairs.nc
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> pairs.nc
> ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
> -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";' -mask_field
> 'name="OBS_CEIL_L0_FULL"; level="L0";' -union -thresh gt0 -name
> apcp_2018010800_f12 -v 3*
> DEBUG 3: Mask UNION: 336000 of 336000 points inside
>
> It sounds like you want to create a mask of the grid points where
the
> forecast OR observation ceiling value is >0. You could do this in
multiple
> calls to gen_vx_mask, but you could also do this using the
censor_thresh
> and censor_val settings:
>
> */usr/local/met-8.1.1/bin/gen_vx_mask
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> pairs.nc
>
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> pairs.nc
> ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
> -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";
> censor_thresh=[eq-9999,gt0]; censor_val=[0,1];' -mask_field
> 'name="OBS_CEIL_L0_FULL"; level="L0"; censor_thresh=[eq-9999,gt0];
> censor_val=[0,1];' -union -thresh gt0 -name ceil_2018010800_f12 -v
3*
> DEBUG 3: Data Masking: 254162 of 336000 points inside
>
> And that produces the attached making region.
>
> Here's how the censor logic is working:
> censor_thresh=[eq-9999,gt0]; censor_val=[0,1];
> (1) Replace any data value ==-9999 with a value of 0.
> (2) Replace any data value >0 with a value of 1.
>
> Hope this approach helps.
>
> Thanks,
> John
>
> On Thu, Mar 26, 2020 at 1:06 PM Tracy Hertneky via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684 >
> >
> > Along with what I previously said, this it the command I am using
for
> APCP
> > and REFC, where non-events are zero (but with CEIL, non-event's
are
> missing
> > values)
> >
> > Example APCP (running on dakota)
> >
> > /usr/local/met-8.1.1/bin/gen_vx_mask \
> >
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > pairs.nc
> > \
> >
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > pairs.nc
> > \
> >
> > apcp_2018010800_f12.nc \
> >
> > -type data \
> >
> > -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' \
> >
> > -mask_field 'name="OBS_APCP_01_A1_FULL";level="L0";' \
> >
> > -union -thresh gt0 \
> >
> > -name apcp_2018010800_f12
> >
> >
> > This successfully produces a mask where either model has a precip
event
> > (>0) - image attached
> >
> >
> > This command was not working for ceiling where non-events are
missing
> > values -9999 - the entire grid was 1-filled. I couldn't figure out
a way
> to
> > deal with missing values to get a similar mask for cloud ceiling
(mask of
> > where CIEL>0).
> >
> >
> > Thanks,
> >
> > Tracy
> >
> > On Mon, Mar 23, 2020 at 12:36 PM met_help at ucar.edu via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Greetings,
> > >
> > > This message has been automatically generated in response to the
> creation
> > > of a trouble ticket regarding:
> > > "Missing Values in gen_vx_mask",
> > > a summary of which appears below.
> > >
> > > There is no need to reply to this message right now. Your
ticket has
> > been
> > > assigned an ID of [rt.rap.ucar.edu #94684].
> > >
> > > Please include the string:
> > >
> > > [rt.rap.ucar.edu #94684]
> > >
> > > in the subject line of all future correspondence about this
issue. To
> do
> > > so, you may reply to this message.
> > >
> > > For more information, please see:
> > >
> > > MET Online Tutorial:
> > >
> https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> > >
> > > MET Users Guide:
> > > https://www.dtcenter.org/met/users/docs/overview.php
> > >
> > > MET FAQs:
> > > https://www.dtcenter.org/met/users/support/faqs/index.php
> > >
> > > MET-Help Email Archive:
> > > http://mailman.ucar.edu/pipermail/met_help
> > >
> > > Thank you,
> > > met_help at ucar.edu
> > >
> > >
>
-------------------------------------------------------------------------
> > > I'm not seeing anything in the UG for how to handle missing
values with
> > > gen_vx_mask, but basically I have a field (ceiling hgt), where
points
> are
> > > assigned -9999 (the missing value). I want a mask of where there
are
> > > clouds, so not including the areas of -9999.
> > >
> > > gen_vx_mask seems to include missing vals in the mask, so a
value of 1.
> > >
> > > Is there another way around this? I can't think of one, but it
is early
> > on
> > > a Monday morning!
> > >
> > > Thanks,
> > > Tracy
> > >
> >
> >
>
>
------------------------------------------------
Subject: Missing Values in gen_vx_mask
From: John Halley Gotway
Time: Thu Mar 26 17:16:18 2020
Tracy,
I'd recommend doing both (1) and (2), but it's possible that (2) isn't
necessary.
gen_vx_mask reads in the "input_field" and considers any value of 0 to
mean
the mask is off. And any value that's not 0 to mean the mask is on.
As long as all your height values really are >=0 or bad data, then
this
logic works fine. But a value of -1 would be interpreted as having
the
mask be ON. This seems safer to me, but as long as you understand
what's
happening, it's totally up to you. In addition, this would make it
very
easy to define a mask for a higher threshold... e.g. a mask where fcst
ceiling is >100 or obs ceiling >100.
Thanks,
John
On Thu, Mar 26, 2020 at 4:49 PM Tracy Hertneky via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684 >
>
> Thanks John, this is exactly what I was looking for. I wouldn't have
> thought to use censor_thresh for changing the missing values.
>
> I removed the values from your #(2) and still got the same thing
since I
> had -thresh gt0 in that call already, which seems to basically do
the same
> thing, but please correct me if I am wrong. Is there a reason you
had it
> that way?
>
> Thanks,
> Tracy
>
> On Thu, Mar 26, 2020 at 2:26 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Tracy,
> >
> > Based on the details you sent and doing some hunting around, I
surmise
> > that.
> >
> > This is the command that works well for APCP:
> >
> > */usr/local/met-8.1.1/bin/gen_vx_mask
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > pairs.nc
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > pairs.nc
> > apcp_2018010800_f12.nc <http://apcp_2018010800_f12.nc> -type data
> > -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' -mask_field
> > 'name="OBS_APCP_01_A1_FULL";level="L0";' -union -thresh gt0 -name
> > apcp_2018010800_f12 -v 3*
> > DEBUG 3: Data Masking: 158383 of 336000 points inside
> >
> > This is the command that does NOT work well for CEIL:
> >
> > */usr/local/met-8.1.1/bin/gen_vx_mask
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > pairs.nc
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > pairs.nc
> > ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
> > -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";' -mask_field
> > 'name="OBS_CEIL_L0_FULL"; level="L0";' -union -thresh gt0 -name
> > apcp_2018010800_f12 -v 3*
> > DEBUG 3: Mask UNION: 336000 of 336000 points inside
> >
> > It sounds like you want to create a mask of the grid points where
the
> > forecast OR observation ceiling value is >0. You could do this in
> multiple
> > calls to gen_vx_mask, but you could also do this using the
censor_thresh
> > and censor_val settings:
> >
> > */usr/local/met-8.1.1/bin/gen_vx_mask
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > pairs.nc
> >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > pairs.nc
> > ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type data
> > -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";
> > censor_thresh=[eq-9999,gt0]; censor_val=[0,1];' -mask_field
> > 'name="OBS_CEIL_L0_FULL"; level="L0"; censor_thresh=[eq-9999,gt0];
> > censor_val=[0,1];' -union -thresh gt0 -name ceil_2018010800_f12 -v
3*
> > DEBUG 3: Data Masking: 254162 of 336000 points inside
> >
> > And that produces the attached making region.
> >
> > Here's how the censor logic is working:
> > censor_thresh=[eq-9999,gt0]; censor_val=[0,1];
> > (1) Replace any data value ==-9999 with a value of 0.
> > (2) Replace any data value >0 with a value of 1.
> >
> > Hope this approach helps.
> >
> > Thanks,
> > John
> >
> > On Thu, Mar 26, 2020 at 1:06 PM Tracy Hertneky via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684 >
> > >
> > > Along with what I previously said, this it the command I am
using for
> > APCP
> > > and REFC, where non-events are zero (but with CEIL, non-event's
are
> > missing
> > > values)
> > >
> > > Example APCP (running on dakota)
> > >
> > > /usr/local/met-8.1.1/bin/gen_vx_mask \
> > >
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > pairs.nc
> > > \
> > >
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > pairs.nc
> > > \
> > >
> > > apcp_2018010800_f12.nc \
> > >
> > > -type data \
> > >
> > > -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' \
> > >
> > > -mask_field 'name="OBS_APCP_01_A1_FULL";level="L0";' \
> > >
> > > -union -thresh gt0 \
> > >
> > > -name apcp_2018010800_f12
> > >
> > >
> > > This successfully produces a mask where either model has a
precip event
> > > (>0) - image attached
> > >
> > >
> > > This command was not working for ceiling where non-events are
missing
> > > values -9999 - the entire grid was 1-filled. I couldn't figure
out a
> way
> > to
> > > deal with missing values to get a similar mask for cloud ceiling
(mask
> of
> > > where CIEL>0).
> > >
> > >
> > > Thanks,
> > >
> > > Tracy
> > >
> > > On Mon, Mar 23, 2020 at 12:36 PM met_help at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Greetings,
> > > >
> > > > This message has been automatically generated in response to
the
> > creation
> > > > of a trouble ticket regarding:
> > > > "Missing Values in gen_vx_mask",
> > > > a summary of which appears below.
> > > >
> > > > There is no need to reply to this message right now. Your
ticket has
> > > been
> > > > assigned an ID of [rt.rap.ucar.edu #94684].
> > > >
> > > > Please include the string:
> > > >
> > > > [rt.rap.ucar.edu #94684]
> > > >
> > > > in the subject line of all future correspondence about this
issue. To
> > do
> > > > so, you may reply to this message.
> > > >
> > > > For more information, please see:
> > > >
> > > > MET Online Tutorial:
> > > >
> >
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> > > >
> > > > MET Users Guide:
> > > > https://www.dtcenter.org/met/users/docs/overview.php
> > > >
> > > > MET FAQs:
> > > > https://www.dtcenter.org/met/users/support/faqs/index.php
> > > >
> > > > MET-Help Email Archive:
> > > > http://mailman.ucar.edu/pipermail/met_help
> > > >
> > > > Thank you,
> > > > met_help at ucar.edu
> > > >
> > > >
> >
-------------------------------------------------------------------------
> > > > I'm not seeing anything in the UG for how to handle missing
values
> with
> > > > gen_vx_mask, but basically I have a field (ceiling hgt), where
points
> > are
> > > > assigned -9999 (the missing value). I want a mask of where
there are
> > > > clouds, so not including the areas of -9999.
> > > >
> > > > gen_vx_mask seems to include missing vals in the mask, so a
value of
> 1.
> > > >
> > > > Is there another way around this? I can't think of one, but it
is
> early
> > > on
> > > > a Monday morning!
> > > >
> > > > Thanks,
> > > > Tracy
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Missing Values in gen_vx_mask
From: Tracy Hertneky
Time: Thu Mar 26 18:25:33 2020
Thanks for the explanation!
On Thu, Mar 26, 2020 at 5:16 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Tracy,
>
> I'd recommend doing both (1) and (2), but it's possible that (2)
isn't
> necessary.
> gen_vx_mask reads in the "input_field" and considers any value of 0
to mean
> the mask is off. And any value that's not 0 to mean the mask is on.
>
> As long as all your height values really are >=0 or bad data, then
this
> logic works fine. But a value of -1 would be interpreted as having
the
> mask be ON. This seems safer to me, but as long as you understand
what's
> happening, it's totally up to you. In addition, this would make it
very
> easy to define a mask for a higher threshold... e.g. a mask where
fcst
> ceiling is >100 or obs ceiling >100.
>
> Thanks,
> John
>
> On Thu, Mar 26, 2020 at 4:49 PM Tracy Hertneky via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684 >
> >
> > Thanks John, this is exactly what I was looking for. I wouldn't
have
> > thought to use censor_thresh for changing the missing values.
> >
> > I removed the values from your #(2) and still got the same thing
since I
> > had -thresh gt0 in that call already, which seems to basically do
the
> same
> > thing, but please correct me if I am wrong. Is there a reason you
had it
> > that way?
> >
> > Thanks,
> > Tracy
> >
> > On Thu, Mar 26, 2020 at 2:26 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Tracy,
> > >
> > > Based on the details you sent and doing some hunting around, I
surmise
> > > that.
> > >
> > > This is the command that works well for APCP:
> > >
> > > */usr/local/met-8.1.1/bin/gen_vx_mask
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > pairs.nc
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > pairs.nc
> > > apcp_2018010800_f12.nc <http://apcp_2018010800_f12.nc> -type
data
> > > -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";'
-mask_field
> > > 'name="OBS_APCP_01_A1_FULL";level="L0";' -union -thresh gt0
-name
> > > apcp_2018010800_f12 -v 3*
> > > DEBUG 3: Data Masking: 158383 of 336000 points inside
> > >
> > > This is the command that does NOT work well for CEIL:
> > >
> > > */usr/local/met-8.1.1/bin/gen_vx_mask
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > > pairs.nc
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > > pairs.nc
> > > ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type
data
> > > -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";' -mask_field
> > > 'name="OBS_CEIL_L0_FULL"; level="L0";' -union -thresh gt0 -name
> > > apcp_2018010800_f12 -v 3*
> > > DEBUG 3: Mask UNION: 336000 of 336000 points inside
> > >
> > > It sounds like you want to create a mask of the grid points
where the
> > > forecast OR observation ceiling value is >0. You could do this
in
> > multiple
> > > calls to gen_vx_mask, but you could also do this using the
> censor_thresh
> > > and censor_val settings:
> > >
> > > */usr/local/met-8.1.1/bin/gen_vx_mask
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > > pairs.nc
> > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_nearest_120000L_20180108_120000V_
> > > pairs.nc
> > > ceil_2018010800_f12.nc <http://ceil_2018010800_f12.nc> -type
data
> > > -input_field 'name="FCST_CEIL_L0_FULL"; level="L0";
> > > censor_thresh=[eq-9999,gt0]; censor_val=[0,1];' -mask_field
> > > 'name="OBS_CEIL_L0_FULL"; level="L0"; censor_thresh=[eq-
9999,gt0];
> > > censor_val=[0,1];' -union -thresh gt0 -name ceil_2018010800_f12
-v 3*
> > > DEBUG 3: Data Masking: 254162 of 336000 points inside
> > >
> > > And that produces the attached making region.
> > >
> > > Here's how the censor logic is working:
> > > censor_thresh=[eq-9999,gt0]; censor_val=[0,1];
> > > (1) Replace any data value ==-9999 with a value of 0.
> > > (2) Replace any data value >0 with a value of 1.
> > >
> > > Hope this approach helps.
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Mar 26, 2020 at 1:06 PM Tracy Hertneky via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94684
>
> > > >
> > > > Along with what I previously said, this it the command I am
using for
> > > APCP
> > > > and REFC, where non-events are zero (but with CEIL, non-
event's are
> > > missing
> > > > values)
> > > >
> > > > Example APCP (running on dakota)
> > > >
> > > > /usr/local/met-8.1.1/bin/gen_vx_mask \
> > > >
> > > >
> > > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > > pairs.nc
> > > > \
> > > >
> > > >
> > > >
> > >
> >
>
/d4/projects/AF/CSM/equivalence/met_out/grid_stat/downscale/grid_stat_budget_120000L_20180108_120000V_
> > > > pairs.nc
> > > > \
> > > >
> > > > apcp_2018010800_f12.nc \
> > > >
> > > > -type data \
> > > >
> > > > -input_field 'name="FCST_APCP_01_A1_FULL";level="L0";' \
> > > >
> > > > -mask_field 'name="OBS_APCP_01_A1_FULL";level="L0";' \
> > > >
> > > > -union -thresh gt0 \
> > > >
> > > > -name apcp_2018010800_f12
> > > >
> > > >
> > > > This successfully produces a mask where either model has a
precip
> event
> > > > (>0) - image attached
> > > >
> > > >
> > > > This command was not working for ceiling where non-events are
missing
> > > > values -9999 - the entire grid was 1-filled. I couldn't figure
out a
> > way
> > > to
> > > > deal with missing values to get a similar mask for cloud
ceiling
> (mask
> > of
> > > > where CIEL>0).
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Tracy
> > > >
> > > > On Mon, Mar 23, 2020 at 12:36 PM met_help at ucar.edu via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Greetings,
> > > > >
> > > > > This message has been automatically generated in response to
the
> > > creation
> > > > > of a trouble ticket regarding:
> > > > > "Missing Values in gen_vx_mask",
> > > > > a summary of which appears below.
> > > > >
> > > > > There is no need to reply to this message right now. Your
ticket
> has
> > > > been
> > > > > assigned an ID of [rt.rap.ucar.edu #94684].
> > > > >
> > > > > Please include the string:
> > > > >
> > > > > [rt.rap.ucar.edu #94684]
> > > > >
> > > > > in the subject line of all future correspondence about this
issue.
> To
> > > do
> > > > > so, you may reply to this message.
> > > > >
> > > > > For more information, please see:
> > > > >
> > > > > MET Online Tutorial:
> > > > >
> > >
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> > > > >
> > > > > MET Users Guide:
> > > > > https://www.dtcenter.org/met/users/docs/overview.php
> > > > >
> > > > > MET FAQs:
> > > > > https://www.dtcenter.org/met/users/support/faqs/index.php
> > > > >
> > > > > MET-Help Email Archive:
> > > > > http://mailman.ucar.edu/pipermail/met_help
> > > > >
> > > > > Thank you,
> > > > > met_help at ucar.edu
> > > > >
> > > > >
> > >
>
-------------------------------------------------------------------------
> > > > > I'm not seeing anything in the UG for how to handle missing
values
> > with
> > > > > gen_vx_mask, but basically I have a field (ceiling hgt),
where
> points
> > > are
> > > > > assigned -9999 (the missing value). I want a mask of where
there
> are
> > > > > clouds, so not including the areas of -9999.
> > > > >
> > > > > gen_vx_mask seems to include missing vals in the mask, so a
value
> of
> > 1.
> > > > >
> > > > > Is there another way around this? I can't think of one, but
it is
> > early
> > > > on
> > > > > a Monday morning!
> > > > >
> > > > > Thanks,
> > > > > Tracy
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list