[Met_help] [rt.rap.ucar.edu #97566] History for Mask setup in METplus conf

Fri Nov 20 14:05:19 MST 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I have a question on how to setup mask file in METplus conf
for cloud verification against satellite data.
In the cloud satellite data, the missing grids are indicated by -99.
These grids should be ignored in verification. I try to use poly [ ] in mask entry in GridStat config file:  
mask = {
   grid = [] ;
   poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];
}; 

Then in METplus conf file, how to setup the variable OBSV_FILE?
The OBS template and format variables have setup in the conf file,
OBS_FILE = {OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE} ?

Thanks!
Binbin

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Mask setup in METplus conf
From: John Halley Gotway
Time: Fri Nov 20 11:45:58 2020

Hi Binbin,

I moved this question over to the met_help ticketing system. I think
George would be best suited to answer it.

George, Binbin is using METplus to run Grid-Stat and in his Grid-Stat
config file, he's referencing an environment variable ${OBS_FILE}
which is the currently observation file being processed.

poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];

How does he configure METplus to make sure the ${OBS_FILE} environment
variable is exported and defined so that Grid-Stat can lookup it's
value?

Thanks,
John

------------------------------------------------
Subject: Mask setup in METplus conf
From: George McCabe
Time: Fri Nov 20 12:00:41 2020

Hi Binbin,

If you are referencing ${OBS_FILE} in your GridStat config file, you
can
use the [user_env_vars] section of the METplus config file to set that
variable. Add this to the bottom of your METplus config file:

[user_env_vars]
OBS_FILE={OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE}

The values in the filename template will be substituted for each run
time.
This should give you the result you need.

On a related note, in the default METplus wrapped GridStat config file
(found in the METplus repository at
parm/met_config/GridStatConfig_wrapped), an environment variable
${VERIF_MASK} is generally used to set the poly value of the mask
dictionary:

https://github.com/dtcenter/METplus/blob/e9a4406cc7ff40cd9770382b318e00c94495e8cb/parm/met_config/GridStatConfig_wrapped#L106

This value is set by the GRID_STAT_VERIFICATION_MASK_TEMPLATE variable
in
the METplus config file. However, the string substitution code does
not
like the curly braces that are used to define the name/level of the
field.
You *should* be able to set:

GRID_STAT_VERIFICATION_MASK_TEMPLATE =
{OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE} {name=
\"TCDC\";
level = \"L0\";} >=0

however, this does not currently work. Modifications to the code could
be
made to simply skip processing text between curly braces that do not
correspond to METplus variables or filename template tags instead of
causing an error.

Please let me know if setting OBS_FILE in the [user_env_vars] section
does
not work or if you have any other questions.

Thanks,
George

On Fri, Nov 20, 2020 at 11:46 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Binbin,
>
> I moved this question over to the met_help ticketing system. I think
> George would be best suited to answer it.
>
> George, Binbin is using METplus to run Grid-Stat and in his Grid-
Stat
> config file, he's referencing an environment variable ${OBS_FILE}
which is
> the currently observation file being processed.
>
> poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];
>
> How does he configure METplus to make sure the ${OBS_FILE}
environment
> variable is exported and defined so that Grid-Stat can lookup it's
value?
>
> Thanks,
> John
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Mask setup in METplus conf
From: Binbin.Zhou at noaa.gov
Time: Fri Nov 20 12:36:14 2020

George,

  Sure I will test it.

Thanks!

Binbin

On Fri, Nov 20, 2020 at 2:00 PM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Binbin,
>
> If you are referencing ${OBS_FILE} in your GridStat config file, you
can
> use the [user_env_vars] section of the METplus config file to set
that
> variable. Add this to the bottom of your METplus config file:
>
> [user_env_vars]
> OBS_FILE={OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE}
>
> The values in the filename template will be substituted for each run
time.
> This should give you the result you need.
>
> On a related note, in the default METplus wrapped GridStat config
file
> (found in the METplus repository at
> parm/met_config/GridStatConfig_wrapped), an environment variable
> ${VERIF_MASK} is generally used to set the poly value of the mask
> dictionary:
>
>
>
https://github.com/dtcenter/METplus/blob/e9a4406cc7ff40cd9770382b318e00c94495e8cb/parm/met_config/GridStatConfig_wrapped#L106
>
> This value is set by the GRID_STAT_VERIFICATION_MASK_TEMPLATE
variable in
> the METplus config file. However, the string substitution code does
not
> like the curly braces that are used to define the name/level of the
field.
> You *should* be able to set:
>
> GRID_STAT_VERIFICATION_MASK_TEMPLATE =
> {OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE} {name=
\"TCDC\";
> level = \"L0\";} >=0
>
> however, this does not currently work. Modifications to the code
could be
> made to simply skip processing text between curly braces that do not
> correspond to METplus variables or filename template tags instead of
> causing an error.
>
> Please let me know if setting OBS_FILE in the [user_env_vars]
section does
> not work or if you have any other questions.
>
> Thanks,
> George
>
>
>
> On Fri, Nov 20, 2020 at 11:46 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Binbin,
> >
> > I moved this question over to the met_help ticketing system. I
think
> > George would be best suited to answer it.
> >
> > George, Binbin is using METplus to run Grid-Stat and in his Grid-
Stat
> > config file, he's referencing an environment variable ${OBS_FILE}
which
> is
> > the currently observation file being processed.
> >
> > poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];
> >
> > How does he configure METplus to make sure the ${OBS_FILE}
environment
> > variable is exported and defined so that Grid-Stat can lookup it's
value?
> >
> > Thanks,
> > John
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

--

Binbin Zhou

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct.

College Park, MD 20740

Binbin.Zhou at noaa.gov

301-683-3683

------------------------------------------------
Subject: Mask setup in METplus conf
From: Binbin.Zhou at noaa.gov
Time: Fri Nov 20 12:48:40 2020

George,

  Yes, it works well!

Thanks!

Binbin

On Fri, Nov 20, 2020 at 2:36 PM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:

> George,
>
>   Sure I will test it.
>
> Thanks!
>
> Binbin
>
> On Fri, Nov 20, 2020 at 2:00 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Binbin,
>>
>> If you are referencing ${OBS_FILE} in your GridStat config file,
you can
>> use the [user_env_vars] section of the METplus config file to set
that
>> variable. Add this to the bottom of your METplus config file:
>>
>> [user_env_vars]
>> OBS_FILE={OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE}
>>
>> The values in the filename template will be substituted for each
run time.
>> This should give you the result you need.
>>
>> On a related note, in the default METplus wrapped GridStat config
file
>> (found in the METplus repository at
>> parm/met_config/GridStatConfig_wrapped), an environment variable
>> ${VERIF_MASK} is generally used to set the poly value of the mask
>> dictionary:
>>
>>
>>
https://github.com/dtcenter/METplus/blob/e9a4406cc7ff40cd9770382b318e00c94495e8cb/parm/met_config/GridStatConfig_wrapped#L106
>>
>> This value is set by the GRID_STAT_VERIFICATION_MASK_TEMPLATE
variable in
>> the METplus config file. However, the string substitution code does
not
>> like the curly braces that are used to define the name/level of the
field.
>> You *should* be able to set:
>>
>> GRID_STAT_VERIFICATION_MASK_TEMPLATE =
>> {OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE} {name=
\"TCDC\";
>> level = \"L0\";} >=0
>>
>> however, this does not currently work. Modifications to the code
could be
>> made to simply skip processing text between curly braces that do
not
>> correspond to METplus variables or filename template tags instead
of
>> causing an error.
>>
>> Please let me know if setting OBS_FILE in the [user_env_vars]
section does
>> not work or if you have any other questions.
>>
>> Thanks,
>> George
>>
>>
>>
>> On Fri, Nov 20, 2020 at 11:46 AM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Hi Binbin,
>> >
>> > I moved this question over to the met_help ticketing system. I
think
>> > George would be best suited to answer it.
>> >
>> > George, Binbin is using METplus to run Grid-Stat and in his Grid-
Stat
>> > config file, he's referencing an environment variable ${OBS_FILE}
which
>> is
>> > the currently observation file being processed.
>> >
>> > poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];
>> >
>> > How does he configure METplus to make sure the ${OBS_FILE}
environment
>> > variable is exported and defined so that Grid-Stat can lookup
it's
>> value?
>> >
>> > Thanks,
>> > John
>> >
>>
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>

--

Binbin Zhou

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct.

College Park, MD 20740

Binbin.Zhou at noaa.gov

301-683-3683

------------------------------------------------
Subject: Mask setup in METplus conf
From: George McCabe
Time: Fri Nov 20 13:53:32 2020

Great to hear!

- George

On Fri, Nov 20, 2020 at 12:49 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97566 >
>
> George,
>
>   Yes, it works well!
>
> Thanks!
>
> Binbin
>
> On Fri, Nov 20, 2020 at 2:36 PM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > George,
> >
> >   Sure I will test it.
> >
> > Thanks!
> >
> > Binbin
> >
> > On Fri, Nov 20, 2020 at 2:00 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> Hi Binbin,
> >>
> >> If you are referencing ${OBS_FILE} in your GridStat config file,
you can
> >> use the [user_env_vars] section of the METplus config file to set
that
> >> variable. Add this to the bottom of your METplus config file:
> >>
> >> [user_env_vars]
> >> OBS_FILE={OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE}
> >>
> >> The values in the filename template will be substituted for each
run
> time.
> >> This should give you the result you need.
> >>
> >> On a related note, in the default METplus wrapped GridStat config
file
> >> (found in the METplus repository at
> >> parm/met_config/GridStatConfig_wrapped), an environment variable
> >> ${VERIF_MASK} is generally used to set the poly value of the mask
> >> dictionary:
> >>
> >>
> >>
>
https://github.com/dtcenter/METplus/blob/e9a4406cc7ff40cd9770382b318e00c94495e8cb/parm/met_config/GridStatConfig_wrapped#L106
> >>
> >> This value is set by the GRID_STAT_VERIFICATION_MASK_TEMPLATE
variable
> in
> >> the METplus config file. However, the string substitution code
does not
> >> like the curly braces that are used to define the name/level of
the
> field.
> >> You *should* be able to set:
> >>
> >> GRID_STAT_VERIFICATION_MASK_TEMPLATE =
> >> {OBS_GRID_STAT_INPUT_DIR}/{OBS_GRID_STAT_INPUT_TEMPLATE} {name=
> \"TCDC\";
> >> level = \"L0\";} >=0
> >>
> >> however, this does not currently work. Modifications to the code
could
> be
> >> made to simply skip processing text between curly braces that do
not
> >> correspond to METplus variables or filename template tags instead
of
> >> causing an error.
> >>
> >> Please let me know if setting OBS_FILE in the [user_env_vars]
section
> does
> >> not work or if you have any other questions.
> >>
> >> Thanks,
> >> George
> >>
> >>
> >>
> >> On Fri, Nov 20, 2020 at 11:46 AM John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Hi Binbin,
> >> >
> >> > I moved this question over to the met_help ticketing system. I
think
> >> > George would be best suited to answer it.
> >> >
> >> > George, Binbin is using METplus to run Grid-Stat and in his
Grid-Stat
> >> > config file, he's referencing an environment variable
${OBS_FILE}
> which
> >> is
> >> > the currently observation file being processed.
> >> >
> >> > poly = [ "${OBS_FILE} {name= \"TCDC\"; level = \"L0\";} >=0" ];
> >> >
> >> > How does he configure METplus to make sure the ${OBS_FILE}
environment
> >> > variable is exported and defined so that Grid-Stat can lookup
it's
> >> value?
> >> >
> >> > Thanks,
> >> > John
> >> >
> >>
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not feel
obliged
> to
> >> reply to this email outside of your normal working hours.
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------