[Met_help] [rt.rap.ucar.edu #100324] History for MET update and questions
George McCabe via RT
met_help at ucar.edu
Mon Jul 19 09:35:13 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi all,
I used MET/metpy to calculate the moist static energy for the Dorian (2019)
use-case. MSE may be a very useful metric in general that others may like
to try (as applied in the papers listed below). In the case of Dorian, MSE
may have had some bearing in the development of rainfall over the NE.
https://journals.ametsoc.org/view/journals/atsc/71/11/jas-d-14-0052.1.xml
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019GL083667
https://journals.ametsoc.org/view/journals/clim/32/18/jcli-d-18-0599.1.xml
I have a couple of questions regarding MET. First, for the series
analysis, do I really need to run a separate python script for each
diagnostic? This looks to be the case by default when looking at PV and
IVT, in which each script outputs only one variable for MET to then read.
Can I combine multiple scripts into one that outputs multiple variables?
Second, is it possible for MET to read-in variables directly from wrfout
netcdf files? For example, say I have PBL height or other model variables
on model sigma levels. Would MET know how to read these when specified in
the config files? My understanding is that MET is particular when it comes
to the level specified. It can handle pressure levels ("PXXX"), but what
about sigma levels ("???")? Thank you.
-Nick
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET update and questions
From: Minna Win
Time: Thu Jun 24 11:31:51 2021
Hi Nick,
I will need to find someone who can answer your second question
regarding
netCDF files. I need some clarification on your first question. When
you
are asking whether you need to run multiple python scripts, are you
referring to the Python embedding scripts that MET supports?
Regards,
Minna
---------------
Minna Win
Pronouns: she/her
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
On Thu, Jun 24, 2021 at 10:24 AM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> Thu Jun 24 10:24:25 2021: Request 100324 was acted upon.
> Transaction: Ticket created by nleonardo87 at gmail.com
> Queue: met_help
> Subject: MET update and questions
> Owner: Nobody
> Requestors: nleonardo87 at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
>
> Hi all,
> I used MET/metpy to calculate the moist static energy for the Dorian
(2019)
> use-case. MSE may be a very useful metric in general that others
may like
> to try (as applied in the papers listed below). In the case of
Dorian, MSE
> may have had some bearing in the development of rainfall over the
NE.
> https://journals.ametsoc.org/view/journals/atsc/71/11/jas-d-14-
0052.1.xml
>
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019GL083667
> https://journals.ametsoc.org/view/journals/clim/32/18/jcli-d-18-
0599.1.xml
>
> I have a couple of questions regarding MET. First, for the series
> analysis, do I really need to run a separate python script for each
> diagnostic? This looks to be the case by default when looking at PV
and
> IVT, in which each script outputs only one variable for MET to then
read.
> Can I combine multiple scripts into one that outputs multiple
variables?
>
> Second, is it possible for MET to read-in variables directly from
wrfout
> netcdf files? For example, say I have PBL height or other model
variables
> on model sigma levels. Would MET know how to read these when
specified in
> the config files? My understanding is that MET is particular when
it comes
> to the level specified. It can handle pressure levels ("PXXX"), but
what
> about sigma levels ("???")? Thank you.
>
> -Nick
>
>
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Thu Jun 24 13:06:27 2021
Hello Mina,
Thank you and yes, I was referring to the Python embedding scripts
(e.g.,
from the Dorian feature-relative use case, condensing
gfs_pv_analysis.py,
gfs_ivt_analysis.py, gfs_pv_fcst.py, gfs_ivt_fcst.py into one).
-Nick
On Thu, Jun 24, 2021 at 1:31 PM Minna Win via RT <met_help at ucar.edu>
wrote:
> Hi Nick,
>
> I will need to find someone who can answer your second question
regarding
> netCDF files. I need some clarification on your first question.
When you
> are asking whether you need to run multiple python scripts, are you
> referring to the Python embedding scripts that MET supports?
>
> Regards,
> Minna
> ---------------
> Minna Win
> Pronouns: she/her
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
>
>
>
> On Thu, Jun 24, 2021 at 10:24 AM Nicholas Leonardo via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Thu Jun 24 10:24:25 2021: Request 100324 was acted upon.
> > Transaction: Ticket created by nleonardo87 at gmail.com
> > Queue: met_help
> > Subject: MET update and questions
> > Owner: Nobody
> > Requestors: nleonardo87 at gmail.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> >
> >
> > Hi all,
> > I used MET/metpy to calculate the moist static energy for the
Dorian
> (2019)
> > use-case. MSE may be a very useful metric in general that others
may
> like
> > to try (as applied in the papers listed below). In the case of
Dorian,
> MSE
> > may have had some bearing in the development of rainfall over the
NE.
> >
> https://journals.ametsoc.org/view/journals/atsc/71/11/jas-d-14-
0052.1.xml
> >
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019GL083667
> >
> https://journals.ametsoc.org/view/journals/clim/32/18/jcli-d-18-
0599.1.xml
> >
> > I have a couple of questions regarding MET. First, for the series
> > analysis, do I really need to run a separate python script for
each
> > diagnostic? This looks to be the case by default when looking at
PV and
> > IVT, in which each script outputs only one variable for MET to
then read.
> > Can I combine multiple scripts into one that outputs multiple
variables?
> >
> > Second, is it possible for MET to read-in variables directly from
wrfout
> > netcdf files? For example, say I have PBL height or other model
> variables
> > on model sigma levels. Would MET know how to read these when
specified
> in
> > the config files? My understanding is that MET is particular when
it
> comes
> > to the level specified. It can handle pressure levels ("PXXX"),
but what
> > about sigma levels ("???")? Thank you.
> >
> > -Nick
> >
> >
>
>
------------------------------------------------
Subject: MET update and questions
From: Minna Win
Time: Thu Jun 24 13:10:47 2021
Hi Nick,
I found a response to your second question:
"We do not currently support reading variables on sigma levels in
WRFout
directly. We finally have funding to actively work towards that
capability
and are hoping it will be available in our METplus 4.1.0 beta_3
release at
the end of September."
I will ask George McCabe to answer your first question, now that I
know
that you were referring to Python embedding scripts.
Regards,
Minna
---------------
Minna Win
Pronouns: she/her
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
On Thu, Jun 24, 2021 at 10:24 AM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> Thu Jun 24 10:24:25 2021: Request 100324 was acted upon.
> Transaction: Ticket created by nleonardo87 at gmail.com
> Queue: met_help
> Subject: MET update and questions
> Owner: Nobody
> Requestors: nleonardo87 at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
>
> Hi all,
> I used MET/metpy to calculate the moist static energy for the Dorian
(2019)
> use-case. MSE may be a very useful metric in general that others
may like
> to try (as applied in the papers listed below). In the case of
Dorian, MSE
> may have had some bearing in the development of rainfall over the
NE.
> https://journals.ametsoc.org/view/journals/atsc/71/11/jas-d-14-
0052.1.xml
>
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019GL083667
> https://journals.ametsoc.org/view/journals/clim/32/18/jcli-d-18-
0599.1.xml
>
> I have a couple of questions regarding MET. First, for the series
> analysis, do I really need to run a separate python script for each
> diagnostic? This looks to be the case by default when looking at PV
and
> IVT, in which each script outputs only one variable for MET to then
read.
> Can I combine multiple scripts into one that outputs multiple
variables?
>
> Second, is it possible for MET to read-in variables directly from
wrfout
> netcdf files? For example, say I have PBL height or other model
variables
> on model sigma levels. Would MET know how to read these when
specified in
> the config files? My understanding is that MET is particular when
it comes
> to the level specified. It can handle pressure levels ("PXXX"), but
what
> about sigma levels ("???")? Thank you.
>
> -Nick
>
>
------------------------------------------------
Subject: MET update and questions
From: George McCabe
Time: Mon Jun 28 10:34:10 2021
Hi Nick,
To address this question:
* First, for the series analysis, do I really need to run a separate
python
script for each diagnostic? This looks to be the case by default when
looking at PV and IVT, in which each script outputs only one variable
for
MET to then read. Can I combine multiple scripts into one that outputs
multiple variables? *
Currently, yes, you will need to call a script for each variable. The
MET
tools only read a single 2D field as gridded input and the Python
Embedding
logic was designed to use a single script call to produce a single
input to
the MET tools.
- George
On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> Transaction: Given to mccabe (George McCabe) by minnawin
> Queue: met_help
> Subject: MET update and questions
> Owner: mccabe
> Requestors: nleonardo87 at gmail.com
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
>
> This transaction appears to have no content
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Mon Jun 28 11:17:29 2021
I see. At least regarding the ingestion of WRF data, I figure that
the
embedded python script functionality can be used to first read-in and
convert/interpolate the data into something MET can use. Thank you
all
very much for checking.
-Nick
On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Nick,
>
> To address this question:
>
> * First, for the series analysis, do I really need to run a separate
python
> script for each diagnostic? This looks to be the case by default
when
> looking at PV and IVT, in which each script outputs only one
variable for
> MET to then read. Can I combine multiple scripts into one that
outputs
> multiple variables? *
>
> Currently, yes, you will need to call a script for each variable.
The MET
> tools only read a single 2D field as gridded input and the Python
Embedding
> logic was designed to use a single script call to produce a single
input to
> the MET tools.
>
> - George
>
> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> >
> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> > Transaction: Given to mccabe (George McCabe) by minnawin
> > Queue: met_help
> > Subject: MET update and questions
> > Owner: mccabe
> > Requestors: nleonardo87 at gmail.com
> > Status: open
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> >
> >
> > This transaction appears to have no content
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Tue Jul 13 00:06:37 2021
Hi all,
I have another question on the subject of directly processing WRF for
MET's
feature-relative use-case. Say I want to extract the PBL height
variable
from a wrfout file, which is on just one layer/surface. However,
WRF's
horizontal grid is Lambert Conformal with a resolution of X-km. From
my
understanding, MET expects the grid to be fixed lat-lon in degrees?
Is
there a way that I can have MET read the data knowing it's not on a
fixed
lat-lon grid (having it refer to the 2D lat/lon variables in WRF)?
Then,
if so, can I extract the data normally (in the .conf, setting
BOTH_VAR1_LEVELS = Surface)?
If none of this is true, then I assume others have done this via first
post-processing the wrfout files (maybe through the embedded python
scripts)?
Thank you.
-Nick
On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo
<nleonardo87 at gmail.com>
wrote:
> I see. At least regarding the ingestion of WRF data, I figure that
the
> embedded python script functionality can be used to first read-in
and
> convert/interpolate the data into something MET can use. Thank you
all
> very much for checking.
> -Nick
>
> On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Nick,
>>
>> To address this question:
>>
>> * First, for the series analysis, do I really need to run a
separate
>> python
>> script for each diagnostic? This looks to be the case by default
when
>> looking at PV and IVT, in which each script outputs only one
variable for
>> MET to then read. Can I combine multiple scripts into one that
outputs
>> multiple variables? *
>>
>> Currently, yes, you will need to call a script for each variable.
The MET
>> tools only read a single 2D field as gridded input and the Python
>> Embedding
>> logic was designed to use a single script call to produce a single
input
>> to
>> the MET tools.
>>
>> - George
>>
>> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT
<met_help at ucar.edu>
>> wrote:
>>
>> >
>> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
>> > Transaction: Given to mccabe (George McCabe) by minnawin
>> > Queue: met_help
>> > Subject: MET update and questions
>> > Owner: mccabe
>> > Requestors: nleonardo87 at gmail.com
>> > Status: open
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
>> >
>> >
>> >
>> > This transaction appears to have no content
>> >
>>
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>>
------------------------------------------------
Subject: MET update and questions
From: George McCabe
Time: Tue Jul 13 09:11:47 2021
Hi Nick,
The MET tools support a few different grid projections including
Lambert
Conformal. This section of the MET User's Guide describes what
information
needs to be defined in a Python Embedding script to read that grid:
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
In the Python Embedding logic, the data is passed in as a 2D array and
the
metadata (attrs) defined in the script tell MET how to map the data to
a
projection. The recommended way to test a Python Embedding script is
to use
the plot_data_plane application to create a plot and ensure that the
data
is properly mapped in relation to the map outline.
I hope that helps and let me know if you have any issues getting this
set
up.
Thanks,
George
On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
> Hi all,
> I have another question on the subject of directly processing WRF
for MET's
> feature-relative use-case. Say I want to extract the PBL height
variable
> from a wrfout file, which is on just one layer/surface. However,
WRF's
> horizontal grid is Lambert Conformal with a resolution of X-km.
>From my
> understanding, MET expects the grid to be fixed lat-lon in degrees?
Is
> there a way that I can have MET read the data knowing it's not on a
fixed
> lat-lon grid (having it refer to the 2D lat/lon variables in WRF)?
Then,
> if so, can I extract the data normally (in the .conf, setting
> BOTH_VAR1_LEVELS = Surface)?
>
> If none of this is true, then I assume others have done this via
first
> post-processing the wrfout files (maybe through the embedded python
> scripts)?
>
> Thank you.
> -Nick
>
>
> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo
<nleonardo87 at gmail.com>
> wrote:
>
> > I see. At least regarding the ingestion of WRF data, I figure
that the
> > embedded python script functionality can be used to first read-in
and
> > convert/interpolate the data into something MET can use. Thank
you all
> > very much for checking.
> > -Nick
> >
> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> >> Hi Nick,
> >>
> >> To address this question:
> >>
> >> * First, for the series analysis, do I really need to run a
separate
> >> python
> >> script for each diagnostic? This looks to be the case by default
when
> >> looking at PV and IVT, in which each script outputs only one
variable
> for
> >> MET to then read. Can I combine multiple scripts into one that
outputs
> >> multiple variables? *
> >>
> >> Currently, yes, you will need to call a script for each variable.
The
> MET
> >> tools only read a single 2D field as gridded input and the Python
> >> Embedding
> >> logic was designed to use a single script call to produce a
single input
> >> to
> >> the MET tools.
> >>
> >> - George
> >>
> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT
<met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> >> > Queue: met_help
> >> > Subject: MET update and questions
> >> > Owner: mccabe
> >> > Requestors: nleonardo87 at gmail.com
> >> > Status: open
> >> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> >> >
> >> >
> >> >
> >> > This transaction appears to have no content
> >> >
> >>
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not feel
obliged
> to
> >> reply to this email outside of your normal working hours.
> >>
> >>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MET update and questions
From: George McCabe
Time: Tue Jul 13 09:22:04 2021
Hi Nick,
FYI:
We switched from providing support through this email address to
providing
support through GitHub Discussions. In the future, you will need to
create
a free GitHub account if you don’t have one already and post your
questions
to the METplus Components Discussion Forum:
https://github.com/dtcenter/METplus/discussions
Thanks,
George
On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu> wrote:
> Hi Nick,
>
> The MET tools support a few different grid projections including
Lambert
> Conformal. This section of the MET User's Guide describes what
information
> needs to be defined in a Python Embedding script to read that grid:
>
>
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
>
> In the Python Embedding logic, the data is passed in as a 2D array
and the
> metadata (attrs) defined in the script tell MET how to map the data
to a
> projection. The recommended way to test a Python Embedding script is
to use
> the plot_data_plane application to create a plot and ensure that the
data
> is properly mapped in relation to the map outline.
>
> I hope that helps and let me know if you have any issues getting
this set
> up.
>
> Thanks,
> George
>
> On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>>
>> Hi all,
>> I have another question on the subject of directly processing WRF
for
>> MET's
>> feature-relative use-case. Say I want to extract the PBL height
variable
>> from a wrfout file, which is on just one layer/surface. However,
WRF's
>> horizontal grid is Lambert Conformal with a resolution of X-km.
>From my
>> understanding, MET expects the grid to be fixed lat-lon in degrees?
Is
>> there a way that I can have MET read the data knowing it's not on a
fixed
>> lat-lon grid (having it refer to the 2D lat/lon variables in WRF)?
Then,
>> if so, can I extract the data normally (in the .conf, setting
>> BOTH_VAR1_LEVELS = Surface)?
>>
>> If none of this is true, then I assume others have done this via
first
>> post-processing the wrfout files (maybe through the embedded python
>> scripts)?
>>
>> Thank you.
>> -Nick
>>
>>
>> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo
<nleonardo87 at gmail.com>
>> wrote:
>>
>> > I see. At least regarding the ingestion of WRF data, I figure
that the
>> > embedded python script functionality can be used to first read-in
and
>> > convert/interpolate the data into something MET can use. Thank
you all
>> > very much for checking.
>> > -Nick
>> >
>> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
>> met_help at ucar.edu>
>> > wrote:
>> >
>> >> Hi Nick,
>> >>
>> >> To address this question:
>> >>
>> >> * First, for the series analysis, do I really need to run a
separate
>> >> python
>> >> script for each diagnostic? This looks to be the case by
default when
>> >> looking at PV and IVT, in which each script outputs only one
variable
>> for
>> >> MET to then read. Can I combine multiple scripts into one that
outputs
>> >> multiple variables? *
>> >>
>> >> Currently, yes, you will need to call a script for each
variable. The
>> MET
>> >> tools only read a single 2D field as gridded input and the
Python
>> >> Embedding
>> >> logic was designed to use a single script call to produce a
single
>> input
>> >> to
>> >> the MET tools.
>> >>
>> >> - George
>> >>
>> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT
<met_help at ucar.edu>
>> >> wrote:
>> >>
>> >> >
>> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
>> >> > Transaction: Given to mccabe (George McCabe) by minnawin
>> >> > Queue: met_help
>> >> > Subject: MET update and questions
>> >> > Owner: mccabe
>> >> > Requestors: nleonardo87 at gmail.com
>> >> > Status: open
>> >> > Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
>> >> >
>> >> >
>> >> >
>> >> > This transaction appears to have no content
>> >> >
>> >>
>> >>
>> >> --
>> >> George McCabe - Software Engineer III
>> >> National Center for Atmospheric Research
>> >> Research Applications Laboratory
>> >> 303-497-2768
>> >> ---
>> >> My working day may not be your working day. Please do not feel
obliged
>> to
>> >> reply to this email outside of your normal working hours.
>> >>
>> >>
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Thu Jul 15 14:53:44 2021
Hi George,
Thank you for your help. I have another question regarding a problem
I
reached while trying this out. I just set up a GitHub account and can
post
it there if preferred (I assume it's safe to share files that reveal
my
paths/directories?).
Otherwise, I try to run everything after following the instructions
(e.g.,
run the master_metplus.py) and get an error message that does not
make sense. It says the following: *ERROR : parse_grid_mask() ->
can't
open file
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
even though the file and path given do (no typos). I can actually run
my
embedded python scripts (WRF_PBLhgt_....py) on the data directly with
no
problem, so the error is a little puzzling. I assume it has to do
with
something else I must edit in the .conf file? I've attached these
python
scripts, as well as the .conf file and error log. If indeed preferred
and
safe, I'll post these and more info on GitHub too.
Thank you.
-Nick
On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Nick,
>
> FYI:
>
> We switched from providing support through this email address to
providing
> support through GitHub Discussions. In the future, you will need to
create
> a free GitHub account if you don’t have one already and post your
questions
> to the METplus Components Discussion Forum:
> https://github.com/dtcenter/METplus/discussions
>
> Thanks,
> George
>
> On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Nick,
> >
> > The MET tools support a few different grid projections including
Lambert
> > Conformal. This section of the MET User's Guide describes what
> information
> > needs to be defined in a Python Embedding script to read that
grid:
> >
> >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> >
> > In the Python Embedding logic, the data is passed in as a 2D array
and
> the
> > metadata (attrs) defined in the script tell MET how to map the
data to a
> > projection. The recommended way to test a Python Embedding script
is to
> use
> > the plot_data_plane application to create a plot and ensure that
the data
> > is properly mapped in relation to the map outline.
> >
> > I hope that helps and let me know if you have any issues getting
this set
> > up.
> >
> > Thanks,
> > George
> >
> > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> >>
> >> Hi all,
> >> I have another question on the subject of directly processing WRF
for
> >> MET's
> >> feature-relative use-case. Say I want to extract the PBL height
> variable
> >> from a wrfout file, which is on just one layer/surface. However,
WRF's
> >> horizontal grid is Lambert Conformal with a resolution of X-km.
>From my
> >> understanding, MET expects the grid to be fixed lat-lon in
degrees? Is
> >> there a way that I can have MET read the data knowing it's not on
a
> fixed
> >> lat-lon grid (having it refer to the 2D lat/lon variables in
WRF)?
> Then,
> >> if so, can I extract the data normally (in the .conf, setting
> >> BOTH_VAR1_LEVELS = Surface)?
> >>
> >> If none of this is true, then I assume others have done this via
first
> >> post-processing the wrfout files (maybe through the embedded
python
> >> scripts)?
> >>
> >> Thank you.
> >> -Nick
> >>
> >>
> >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> nleonardo87 at gmail.com>
> >> wrote:
> >>
> >> > I see. At least regarding the ingestion of WRF data, I figure
that
> the
> >> > embedded python script functionality can be used to first read-
in and
> >> > convert/interpolate the data into something MET can use. Thank
you
> all
> >> > very much for checking.
> >> > -Nick
> >> >
> >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> >> met_help at ucar.edu>
> >> > wrote:
> >> >
> >> >> Hi Nick,
> >> >>
> >> >> To address this question:
> >> >>
> >> >> * First, for the series analysis, do I really need to run a
separate
> >> >> python
> >> >> script for each diagnostic? This looks to be the case by
default
> when
> >> >> looking at PV and IVT, in which each script outputs only one
variable
> >> for
> >> >> MET to then read. Can I combine multiple scripts into one that
> outputs
> >> >> multiple variables? *
> >> >>
> >> >> Currently, yes, you will need to call a script for each
variable. The
> >> MET
> >> >> tools only read a single 2D field as gridded input and the
Python
> >> >> Embedding
> >> >> logic was designed to use a single script call to produce a
single
> >> input
> >> >> to
> >> >> the MET tools.
> >> >>
> >> >> - George
> >> >>
> >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT
<met_help at ucar.edu>
> >> >> wrote:
> >> >>
> >> >> >
> >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> >> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> >> >> > Queue: met_help
> >> >> > Subject: MET update and questions
> >> >> > Owner: mccabe
> >> >> > Requestors: nleonardo87 at gmail.com
> >> >> > Status: open
> >> >> > Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> >> >> >
> >> >> >
> >> >> >
> >> >> > This transaction appears to have no content
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >> George McCabe - Software Engineer III
> >> >> National Center for Atmospheric Research
> >> >> Research Applications Laboratory
> >> >> 303-497-2768
> >> >> ---
> >> >> My working day may not be your working day. Please do not feel
> obliged
> >> to
> >> >> reply to this email outside of your normal working hours.
> >> >>
> >> >>
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: MET update and questions
From: John Halley Gotway
Time: Thu Jul 15 17:58:23 2021
Hi Nick,
This is John Halley Gotway. I work with George. In an attempt to
replicate
the error message you're getting, I logged on to cheyenne and ran the
following commands:
*module use /glade/p/ral/jntp/MET/MET_releases/modulefilesmodule load
met/10.0.0*
*module use /glade/p/ral/jntp/MET/METplus/modulefiles*
*module load metplus/4.0.0ncar_pylib*
* setenv MET_PYTHON_EXE `which python3`*
*/glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5/bin/regrid_data_plane
-v 2
PYTHON_NUMPY -field
'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
-name
PBLH
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_1200_000.nc*
And that yields this error message:
ERROR : parse_grid_mask() -> can't open file
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"
It's a little unclear to me what you're trying to do here. Are you
just
trying to have MET write this reformatted data to a NetCDF file? If
so, the
pcp_combine command may be a better bet:
*/glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5/bin/pcp_combine
**-add
PYTHON_NUMPY
**'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
-name
PBLH *
*/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_1200_000.nc*
This is basically running pcp_combine as a pass-through reformatter.
It
just reads the data you are serving up via the python embedding script
and
writing it to an output NetCDF file formatted in a way that MET can
read.
Is that what you're trying to do?
Thanks,
John
On Thu, Jul 15, 2021 at 2:53 PM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
> Hi George,
> Thank you for your help. I have another question regarding a
problem I
> reached while trying this out. I just set up a GitHub account and
can post
> it there if preferred (I assume it's safe to share files that reveal
my
> paths/directories?).
>
> Otherwise, I try to run everything after following the instructions
(e.g.,
> run the master_metplus.py) and get an error message that does not
> make sense. It says the following: *ERROR : parse_grid_mask() ->
can't
> open file
>
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> even though the file and path given do (no typos). I can actually
run my
> embedded python scripts (WRF_PBLhgt_....py) on the data directly
with no
> problem, so the error is a little puzzling. I assume it has to do
with
> something else I must edit in the .conf file? I've attached these
python
> scripts, as well as the .conf file and error log. If indeed
preferred and
> safe, I'll post these and more info on GitHub too.
>
> Thank you.
> -Nick
>
> On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Nick,
> >
> > FYI:
> >
> > We switched from providing support through this email address to
> providing
> > support through GitHub Discussions. In the future, you will need
to
> create
> > a free GitHub account if you don’t have one already and post your
> questions
> > to the METplus Components Discussion Forum:
> > https://github.com/dtcenter/METplus/discussions
> >
> > Thanks,
> > George
> >
> > On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu>
wrote:
> >
> > > Hi Nick,
> > >
> > > The MET tools support a few different grid projections including
> Lambert
> > > Conformal. This section of the MET User's Guide describes what
> > information
> > > needs to be defined in a Python Embedding script to read that
grid:
> > >
> > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > >
> > > In the Python Embedding logic, the data is passed in as a 2D
array and
> > the
> > > metadata (attrs) defined in the script tell MET how to map the
data to
> a
> > > projection. The recommended way to test a Python Embedding
script is to
> > use
> > > the plot_data_plane application to create a plot and ensure that
the
> data
> > > is properly mapped in relation to the map outline.
> > >
> > > I hope that helps and let me know if you have any issues getting
this
> set
> > > up.
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
>
> > >>
> > >> Hi all,
> > >> I have another question on the subject of directly processing
WRF for
> > >> MET's
> > >> feature-relative use-case. Say I want to extract the PBL
height
> > variable
> > >> from a wrfout file, which is on just one layer/surface.
However,
> WRF's
> > >> horizontal grid is Lambert Conformal with a resolution of X-km.
From
> my
> > >> understanding, MET expects the grid to be fixed lat-lon in
degrees?
> Is
> > >> there a way that I can have MET read the data knowing it's not
on a
> > fixed
> > >> lat-lon grid (having it refer to the 2D lat/lon variables in
WRF)?
> > Then,
> > >> if so, can I extract the data normally (in the .conf, setting
> > >> BOTH_VAR1_LEVELS = Surface)?
> > >>
> > >> If none of this is true, then I assume others have done this
via first
> > >> post-processing the wrfout files (maybe through the embedded
python
> > >> scripts)?
> > >>
> > >> Thank you.
> > >> -Nick
> > >>
> > >>
> > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > nleonardo87 at gmail.com>
> > >> wrote:
> > >>
> > >> > I see. At least regarding the ingestion of WRF data, I
figure that
> > the
> > >> > embedded python script functionality can be used to first
read-in
> and
> > >> > convert/interpolate the data into something MET can use.
Thank you
> > all
> > >> > very much for checking.
> > >> > -Nick
> > >> >
> > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > >> met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> >> Hi Nick,
> > >> >>
> > >> >> To address this question:
> > >> >>
> > >> >> * First, for the series analysis, do I really need to run a
> separate
> > >> >> python
> > >> >> script for each diagnostic? This looks to be the case by
default
> > when
> > >> >> looking at PV and IVT, in which each script outputs only one
> variable
> > >> for
> > >> >> MET to then read. Can I combine multiple scripts into one
that
> > outputs
> > >> >> multiple variables? *
> > >> >>
> > >> >> Currently, yes, you will need to call a script for each
variable.
> The
> > >> MET
> > >> >> tools only read a single 2D field as gridded input and the
Python
> > >> >> Embedding
> > >> >> logic was designed to use a single script call to produce a
single
> > >> input
> > >> >> to
> > >> >> the MET tools.
> > >> >>
> > >> >> - George
> > >> >>
> > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> met_help at ucar.edu>
> > >> >> wrote:
> > >> >>
> > >> >> >
> > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> > >> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> > >> >> > Queue: met_help
> > >> >> > Subject: MET update and questions
> > >> >> > Owner: mccabe
> > >> >> > Requestors: nleonardo87 at gmail.com
> > >> >> > Status: open
> > >> >> > Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> > This transaction appears to have no content
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >> George McCabe - Software Engineer III
> > >> >> National Center for Atmospheric Research
> > >> >> Research Applications Laboratory
> > >> >> 303-497-2768
> > >> >> ---
> > >> >> My working day may not be your working day. Please do not
feel
> > obliged
> > >> to
> > >> >> reply to this email outside of your normal working hours.
> > >> >>
> > >> >>
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Thu Jul 15 18:22:28 2021
Hello John,
What I'm trying to do is run the feature-relative use-case, though
with WRF
data instead of the default GFS. From my understanding, it's supposed
to
center the model grids/tiles onto the TC position (in this test, it's
Elsa
(2021)) to get a TC-relative comparison. The command I used to run
everything was: master_metplus.py -c
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/TCStat_SeriesAnalysis_wrfPBLH.conf
-c
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/tutorial.conf
.
Thank you.
-Nick
On Thu, Jul 15, 2021 at 7:58 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Nick,
>
> This is John Halley Gotway. I work with George. In an attempt to
replicate
> the error message you're getting, I logged on to cheyenne and ran
the
> following commands:
>
>
>
> *module use /glade/p/ral/jntp/MET/MET_releases/modulefilesmodule
load
> met/10.0.0*
>
> *module use /glade/p/ral/jntp/MET/METplus/modulefiles*
>
>
> *module load metplus/4.0.0ncar_pylib*
>
> * setenv MET_PYTHON_EXE `which python3`*
> */glade/p/ral/jntp/MET/MET_releases/10.0.0-
beta5/bin/regrid_data_plane -v 2
> PYTHON_NUMPY -field
>
>
'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
>
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
> -name
> PBLH
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
>
>
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_
> 1200_000.nc*
>
> And that yields this error message:
> ERROR : parse_grid_mask() -> can't open file
> "/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"
>
> It's a little unclear to me what you're trying to do here. Are you
just
> trying to have MET write this reformatted data to a NetCDF file? If
so, the
> pcp_combine command may be a better bet:
>
> */glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5/bin/pcp_combine
**-add
> PYTHON_NUMPY
>
**'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
>
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
> -name
> PBLH *
>
>
*/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_
> 1200_000.nc*
>
> This is basically running pcp_combine as a pass-through reformatter.
It
> just reads the data you are serving up via the python embedding
script and
> writing it to an output NetCDF file formatted in a way that MET can
read.
>
> Is that what you're trying to do?
>
> Thanks,
> John
>
> On Thu, Jul 15, 2021 at 2:53 PM Nicholas Leonardo via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> >
> > Hi George,
> > Thank you for your help. I have another question regarding a
problem I
> > reached while trying this out. I just set up a GitHub account and
can
> post
> > it there if preferred (I assume it's safe to share files that
reveal my
> > paths/directories?).
> >
> > Otherwise, I try to run everything after following the
instructions
> (e.g.,
> > run the master_metplus.py) and get an error message that does not
> > make sense. It says the following: *ERROR : parse_grid_mask()
-> can't
> > open file
> >
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> > even though the file and path given do (no typos). I can actually
run my
> > embedded python scripts (WRF_PBLhgt_....py) on the data directly
with no
> > problem, so the error is a little puzzling. I assume it has to do
with
> > something else I must edit in the .conf file? I've attached these
python
> > scripts, as well as the .conf file and error log. If indeed
preferred
> and
> > safe, I'll post these and more info on GitHub too.
> >
> > Thank you.
> > -Nick
> >
> > On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Nick,
> > >
> > > FYI:
> > >
> > > We switched from providing support through this email address to
> > providing
> > > support through GitHub Discussions. In the future, you will need
to
> > create
> > > a free GitHub account if you don’t have one already and post
your
> > questions
> > > to the METplus Components Discussion Forum:
> > > https://github.com/dtcenter/METplus/discussions
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu>
wrote:
> > >
> > > > Hi Nick,
> > > >
> > > > The MET tools support a few different grid projections
including
> > Lambert
> > > > Conformal. This section of the MET User's Guide describes what
> > > information
> > > > needs to be defined in a Python Embedding script to read that
grid:
> > > >
> > > >
> > > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > > >
> > > > In the Python Embedding logic, the data is passed in as a 2D
array
> and
> > > the
> > > > metadata (attrs) defined in the script tell MET how to map the
data
> to
> > a
> > > > projection. The recommended way to test a Python Embedding
script is
> to
> > > use
> > > > the plot_data_plane application to create a plot and ensure
that the
> > data
> > > > is properly mapped in relation to the map outline.
> > > >
> > > > I hope that helps and let me know if you have any issues
getting this
> > set
> > > > up.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > > >>
> > > >> Hi all,
> > > >> I have another question on the subject of directly processing
WRF
> for
> > > >> MET's
> > > >> feature-relative use-case. Say I want to extract the PBL
height
> > > variable
> > > >> from a wrfout file, which is on just one layer/surface.
However,
> > WRF's
> > > >> horizontal grid is Lambert Conformal with a resolution of X-
km.
> From
> > my
> > > >> understanding, MET expects the grid to be fixed lat-lon in
degrees?
> > Is
> > > >> there a way that I can have MET read the data knowing it's
not on a
> > > fixed
> > > >> lat-lon grid (having it refer to the 2D lat/lon variables in
WRF)?
> > > Then,
> > > >> if so, can I extract the data normally (in the .conf, setting
> > > >> BOTH_VAR1_LEVELS = Surface)?
> > > >>
> > > >> If none of this is true, then I assume others have done this
via
> first
> > > >> post-processing the wrfout files (maybe through the embedded
python
> > > >> scripts)?
> > > >>
> > > >> Thank you.
> > > >> -Nick
> > > >>
> > > >>
> > > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > > nleonardo87 at gmail.com>
> > > >> wrote:
> > > >>
> > > >> > I see. At least regarding the ingestion of WRF data, I
figure
> that
> > > the
> > > >> > embedded python script functionality can be used to first
read-in
> > and
> > > >> > convert/interpolate the data into something MET can use.
Thank
> you
> > > all
> > > >> > very much for checking.
> > > >> > -Nick
> > > >> >
> > > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > > >> met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Nick,
> > > >> >>
> > > >> >> To address this question:
> > > >> >>
> > > >> >> * First, for the series analysis, do I really need to run
a
> > separate
> > > >> >> python
> > > >> >> script for each diagnostic? This looks to be the case by
default
> > > when
> > > >> >> looking at PV and IVT, in which each script outputs only
one
> > variable
> > > >> for
> > > >> >> MET to then read. Can I combine multiple scripts into one
that
> > > outputs
> > > >> >> multiple variables? *
> > > >> >>
> > > >> >> Currently, yes, you will need to call a script for each
variable.
> > The
> > > >> MET
> > > >> >> tools only read a single 2D field as gridded input and the
Python
> > > >> >> Embedding
> > > >> >> logic was designed to use a single script call to produce
a
> single
> > > >> input
> > > >> >> to
> > > >> >> the MET tools.
> > > >> >>
> > > >> >> - George
> > > >> >>
> > > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> > met_help at ucar.edu>
> > > >> >> wrote:
> > > >> >>
> > > >> >> >
> > > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> > > >> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> > > >> >> > Queue: met_help
> > > >> >> > Subject: MET update and questions
> > > >> >> > Owner: mccabe
> > > >> >> > Requestors: nleonardo87 at gmail.com
> > > >> >> > Status: open
> > > >> >> > Ticket <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> > This transaction appears to have no content
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >> George McCabe - Software Engineer III
> > > >> >> National Center for Atmospheric Research
> > > >> >> Research Applications Laboratory
> > > >> >> 303-497-2768
> > > >> >> ---
> > > >> >> My working day may not be your working day. Please do not
feel
> > > obliged
> > > >> to
> > > >> >> reply to this email outside of your normal working hours.
> > > >> >>
> > > >> >>
> > > >>
> > > >>
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: MET update and questions
From: John Halley Gotway
Time: Thu Jul 15 19:07:02 2021
Ok, the specific reason why regrid_data_plane can’t extract the grid
definition from that NetCDF file is that it needs additional
information to
do so (specifying file_type = NETCDF_PINT;) and there is no mechanism
to
specify that via regrid_data_plane in this context.
So swapping in WRF (NetCDF) files in place of the GFS (GRIB) files
will
require additional changes to the feature relative use case, and I’ll
defer
back to George to advise on that. Hopefully it’ll only require a
change to
the METplus configuration rather than changes to the python wrappers.
John
On Thu, Jul 15, 2021 at 6:22 PM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
> Hello John,
> What I'm trying to do is run the feature-relative use-case, though
with WRF
> data instead of the default GFS. From my understanding, it's
supposed to
> center the model grids/tiles onto the TC position (in this test,
it's Elsa
> (2021)) to get a TC-relative comparison. The command I used to run
> everything was: master_metplus.py -c
>
>
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/TCStat_SeriesAnalysis_wrfPBLH.conf
> -c
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/tutorial.conf
> .
>
> Thank you.
> -Nick
>
> On Thu, Jul 15, 2021 at 7:58 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Nick,
> >
> > This is John Halley Gotway. I work with George. In an attempt to
> replicate
> > the error message you're getting, I logged on to cheyenne and ran
the
> > following commands:
> >
> >
> >
> > *module use /glade/p/ral/jntp/MET/MET_releases/modulefilesmodule
load
> > met/10.0.0*
> >
> > *module use /glade/p/ral/jntp/MET/METplus/modulefiles*
> >
> >
> > *module load metplus/4.0.0ncar_pylib*
> >
> > * setenv MET_PYTHON_EXE `which python3`*
> > */glade/p/ral/jntp/MET/MET_releases/10.0.0-
beta5/bin/regrid_data_plane
> -v 2
> > PYTHON_NUMPY -field
> >
> >
>
'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
> >
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
> > -name
> > PBLH
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
> >
> >
>
/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_
> > 1200_000.nc*
> >
> > And that yields this error message:
> > ERROR : parse_grid_mask() -> can't open file
> >
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"
> >
> > It's a little unclear to me what you're trying to do here. Are you
just
> > trying to have MET write this reformatted data to a NetCDF file?
If so,
> the
> > pcp_combine command may be a better bet:
> >
> > */glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5/bin/pcp_combine
> **-add
> > PYTHON_NUMPY
> >
>
**'name="/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/WRF_PBLhgt_fcst.py
> >
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc";'
> > -name
> > PBLH *
> >
> >
>
*/glade/u/home/nleonardo/METplus_Tutorial/MULTIparm_test001/output/py_embed_out/20210709/WRFgfs4_20210709_
> > 1200_000.nc*
> >
> > This is basically running pcp_combine as a pass-through
reformatter. It
> > just reads the data you are serving up via the python embedding
script
> and
> > writing it to an output NetCDF file formatted in a way that MET
can read.
> >
> > Is that what you're trying to do?
> >
> > Thanks,
> > John
> >
> > On Thu, Jul 15, 2021 at 2:53 PM Nicholas Leonardo via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > >
> > > Hi George,
> > > Thank you for your help. I have another question regarding a
problem I
> > > reached while trying this out. I just set up a GitHub account
and can
> > post
> > > it there if preferred (I assume it's safe to share files that
reveal my
> > > paths/directories?).
> > >
> > > Otherwise, I try to run everything after following the
instructions
> > (e.g.,
> > > run the master_metplus.py) and get an error message that does
not
> > > make sense. It says the following: *ERROR : parse_grid_mask()
->
> can't
> > > open file
> > >
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> > > even though the file and path given do (no typos). I can
actually run
> my
> > > embedded python scripts (WRF_PBLhgt_....py) on the data directly
with
> no
> > > problem, so the error is a little puzzling. I assume it has to
do with
> > > something else I must edit in the .conf file? I've attached
these
> python
> > > scripts, as well as the .conf file and error log. If indeed
preferred
> > and
> > > safe, I'll post these and more info on GitHub too.
> > >
> > > Thank you.
> > > -Nick
> > >
> > > On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Nick,
> > > >
> > > > FYI:
> > > >
> > > > We switched from providing support through this email address
to
> > > providing
> > > > support through GitHub Discussions. In the future, you will
need to
> > > create
> > > > a free GitHub account if you don’t have one already and post
your
> > > questions
> > > > to the METplus Components Discussion Forum:
> > > > https://github.com/dtcenter/METplus/discussions
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, Jul 13, 2021 at 9:04 AM George McCabe
<mccabe at ucar.edu>
> wrote:
> > > >
> > > > > Hi Nick,
> > > > >
> > > > > The MET tools support a few different grid projections
including
> > > Lambert
> > > > > Conformal. This section of the MET User's Guide describes
what
> > > > information
> > > > > needs to be defined in a Python Embedding script to read
that grid:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > > > >
> > > > > In the Python Embedding logic, the data is passed in as a 2D
array
> > and
> > > > the
> > > > > metadata (attrs) defined in the script tell MET how to map
the data
> > to
> > > a
> > > > > projection. The recommended way to test a Python Embedding
script
> is
> > to
> > > > use
> > > > > the plot_data_plane application to create a plot and ensure
that
> the
> > > data
> > > > > is properly mapped in relation to the map outline.
> > > > >
> > > > > I hope that helps and let me know if you have any issues
getting
> this
> > > set
> > > > > up.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > > > >>
> > > > >> Hi all,
> > > > >> I have another question on the subject of directly
processing WRF
> > for
> > > > >> MET's
> > > > >> feature-relative use-case. Say I want to extract the PBL
height
> > > > variable
> > > > >> from a wrfout file, which is on just one layer/surface.
However,
> > > WRF's
> > > > >> horizontal grid is Lambert Conformal with a resolution of
X-km.
> > From
> > > my
> > > > >> understanding, MET expects the grid to be fixed lat-lon in
> degrees?
> > > Is
> > > > >> there a way that I can have MET read the data knowing it's
not on
> a
> > > > fixed
> > > > >> lat-lon grid (having it refer to the 2D lat/lon variables
in WRF)?
> > > > Then,
> > > > >> if so, can I extract the data normally (in the .conf,
setting
> > > > >> BOTH_VAR1_LEVELS = Surface)?
> > > > >>
> > > > >> If none of this is true, then I assume others have done
this via
> > first
> > > > >> post-processing the wrfout files (maybe through the
embedded
> python
> > > > >> scripts)?
> > > > >>
> > > > >> Thank you.
> > > > >> -Nick
> > > > >>
> > > > >>
> > > > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > > > nleonardo87 at gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > I see. At least regarding the ingestion of WRF data, I
figure
> > that
> > > > the
> > > > >> > embedded python script functionality can be used to first
> read-in
> > > and
> > > > >> > convert/interpolate the data into something MET can use.
Thank
> > you
> > > > all
> > > > >> > very much for checking.
> > > > >> > -Nick
> > > > >> >
> > > > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > > > >> met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Hi Nick,
> > > > >> >>
> > > > >> >> To address this question:
> > > > >> >>
> > > > >> >> * First, for the series analysis, do I really need to
run a
> > > separate
> > > > >> >> python
> > > > >> >> script for each diagnostic? This looks to be the case
by
> default
> > > > when
> > > > >> >> looking at PV and IVT, in which each script outputs only
one
> > > variable
> > > > >> for
> > > > >> >> MET to then read. Can I combine multiple scripts into
one that
> > > > outputs
> > > > >> >> multiple variables? *
> > > > >> >>
> > > > >> >> Currently, yes, you will need to call a script for each
> variable.
> > > The
> > > > >> MET
> > > > >> >> tools only read a single 2D field as gridded input and
the
> Python
> > > > >> >> Embedding
> > > > >> >> logic was designed to use a single script call to
produce a
> > single
> > > > >> input
> > > > >> >> to
> > > > >> >> the MET tools.
> > > > >> >>
> > > > >> >> - George
> > > > >> >>
> > > > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> > > met_help at ucar.edu>
> > > > >> >> wrote:
> > > > >> >>
> > > > >> >> >
> > > > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted
upon.
> > > > >> >> > Transaction: Given to mccabe (George McCabe) by
minnawin
> > > > >> >> > Queue: met_help
> > > > >> >> > Subject: MET update and questions
> > > > >> >> > Owner: mccabe
> > > > >> >> > Requestors: nleonardo87 at gmail.com
> > > > >> >> > Status: open
> > > > >> >> > Ticket <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > > > >> >> >
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > This transaction appears to have no content
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> George McCabe - Software Engineer III
> > > > >> >> National Center for Atmospheric Research
> > > > >> >> Research Applications Laboratory
> > > > >> >> 303-497-2768
> > > > >> >> ---
> > > > >> >> My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > >> to
> > > > >> >> reply to this email outside of your normal working
hours.
> > > > >> >>
> > > > >> >>
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: MET update and questions
From: George McCabe
Time: Fri Jul 16 11:25:46 2021
Hi Nick,
I apologize for the lengthy response.
Looking at your Python embedding scripts, it looks like the fields are
read
in with being modified. The reason Python Embedding was used in the
IVT use
case is the IVT field needed to be computed from other fields inside
the
input file. If the field you are trying to read into MET is found
inside
the input file and the file is one of the supported NetCDF formats,
then
using Python Embedding is not necessary for this step. It looks like
the
file matches NETCDF_PINT which is created by running the p_interp or
wrf_interp utility on WRF output (see
https://met.readthedocs.io/en/latest/Users_Guide/config_options.html?highlight=NETCDF_PINT#settings-
common-to-multiple-tools
and search for NETCDF_PINT). I checked this by running plot_data_plane
and
setting the file_type:
/glade/p/ral/jntp/MET/MET_releases/10.0.0/bin/plot_data_plane
/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
out.ps
'name="PBLH"; level="(0,*,*)"; file_type=NETCDF_PINT;'
convert -rotate 90 out.ps out.png
display out.png
The plot_data_plane command succeeded and the output image appeared to
have
the correct map outlines, which shows that MET can read the data.
If my assumption that there are no derivations needed to the data,
then you
can read these files directly into ExtractTiles wrapper and access the
fields by specifying the NetCDF field info:
BOTH_VAR1_NAME = PBLH
BOTH_VAR1_LEVELS = "(0,*,*)"
BOTH_VAR1_OPTIONS = file_type = NETCDF_PINT
FCST_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
FCST_EXTRACT_TILES_INPUT_TEMPLATE =
{init?fmt=%Y%m%d}/WRFgfs4_{init?fmt=%Y%m%d}_{init?fmt=%H}00_{lead?fmt=%3H}.nc
OBS_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
OBS_EXTRACT_TILES_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/WRFnam4_{valid?fmt=%Y%m%d}_{valid?fmt=%H}00_000.nc
I copied your config files and made some modifications. Starting in
METplus
v4.0.0, the [filename_templates] and [dir] sections are no longer
used, so
all of those variables can go under [config] unless they are specific
to a
defined instance in the PROCESS_LIST i.e. TCStat(for_series_analysis).
I
rearranged the variables so that they are easier to read including
pairing
the corresponding _DIR and _TEMPLATE variables and putting all file
I/O
variables together. You can copy my file from Cheyenne here:
/glade/u/home/mccabe/100324/TCStat_SeriesAnalysis_wrfPBLH.conf
I removed the PyEmbedIngest processing, updated the field info to read
the
data directly from the NetCDF files, and updated the dir/template
variables
to read the NetCDF files directly into ExtractTiles. This removed the
error
you saw, however there are no pairs found from TCPairs:
DEBUG 1: [Source 1 of 1] ADECK Source:
/glade/u/home/nleonardo/ELSA/A_Bdecks/aal052021.dat, Model Suffix:
(nul)
DEBUG 1: [Source 1 of 1] BDECK Source:
/glade/u/home/nleonardo/ELSA/A_Bdecks/bal052021.dat, Model Suffix:
(nul)
DEBUG 1: Config File Default:
/glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/config/TCPairsConfig_default
DEBUG 1: Config File User:
/glade/p/ral/jntp/MET/METplus/METplus-
4.0.0/parm/met_config/TCPairsConfig_wrapped
DEBUG 1: Distance to land file:
/glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/tc_data/
dland_global_tenth_degree.nc
DEBUG 2: Processing 1 BDECK file(s).
DEBUG 2: Found 1 BDECK track(s).
DEBUG 2: Processing 1 ADECK file(s).
DEBUG 1: When reading ATCF track data, all instances of "AVN" are
automatically replaced with "GFS" (ATCF ID = AVNO).
WARNING:
WARNING: int ATCFTrackLine::read_line(LineDataFile * ldf) -> found
fewer
than the expected number of elements (6) in ATCF track line:
WARNING: AL, 05, 2021063018, 03, AP16, 03
WARNING:
In the log output from this run, I noticed that the METPLUS_INIT_BEG
and
METPLUS_INIT_END environment variables were set to filter the init
time:
07/16 10:58:26.004 metplus INFO: METPLUS_INIT_BEG=init_beg =
"2021070912";
07/16 10:58:26.004 metplus INFO: METPLUS_INIT_END=init_end =
"2021070912";
These values are read from INIT_BEG and INIT_END because
TC_PAIRS_INIT_BEG
and TC_PAIRS_INIT_END are not set. INIT_BEG and INIT_END are typically
used
for looping over time and processing multiple run times, so it is
recommended to also set the TC_PAIRS_ variables as well. I set these
to an
empty value to prevent restricting the init times that are processed.
This next run produces output for TCPairs and TCStat, however
ExtractTiles
failed to find the corresponding forecast and analysis data because it
looks like there are only files in your directory for 20210709 12Z 0
hr
forecast. You will either have to change the criteria that goes into
creating the storm track files to match the available data or make
more
forecast/analysis data available to be read.
Note this message in the TCPairs log file:
DEBUG 1: When reading ATCF track data, all instances of "AVN" are
automatically replaced with "GFS" (ATCF ID = AVNO).
The MODEL value in your config file is set to GFSO, so this message is
saying that it is processing ADECK data that match AVNO. Changing the
MODEL
value can change what is read from these files.
An unrelated thing I noticed is your tutorial.conf file sets
MET_INSTALL_DIR = /glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5. The
stable 10.0.0 version of MET is available and will automatically be
used by
the instance of METplus 4.0.0 that is installed on Cheyenne, so you
don't
need to set this variable here.
I know this is a lot of info but I hope this helps you set up this use
case. Let me know if you have any other questions about this. Also,
posting
paths on GitHub Discussions shouldn't be an issue because other users
likely don't have access to that machine. There is actually a archive
of
MET Help tickets online that while not as easy to access as the
dicussions
board is still available to the public. Any sensitive information can
be
emailed to a METplus team member if necessary.
Thanks,
George
On Thu, Jul 15, 2021 at 2:54 PM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
> Hi George,
> Thank you for your help. I have another question regarding a
problem I
> reached while trying this out. I just set up a GitHub account and
can post
> it there if preferred (I assume it's safe to share files that reveal
my
> paths/directories?).
>
> Otherwise, I try to run everything after following the instructions
(e.g.,
> run the master_metplus.py) and get an error message that does not
> make sense. It says the following: *ERROR : parse_grid_mask() ->
can't
> open file
>
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> even though the file and path given do (no typos). I can actually
run my
> embedded python scripts (WRF_PBLhgt_....py) on the data directly
with no
> problem, so the error is a little puzzling. I assume it has to do
with
> something else I must edit in the .conf file? I've attached these
python
> scripts, as well as the .conf file and error log. If indeed
preferred and
> safe, I'll post these and more info on GitHub too.
>
> Thank you.
> -Nick
>
> On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Nick,
> >
> > FYI:
> >
> > We switched from providing support through this email address to
> providing
> > support through GitHub Discussions. In the future, you will need
to
> create
> > a free GitHub account if you don’t have one already and post your
> questions
> > to the METplus Components Discussion Forum:
> > https://github.com/dtcenter/METplus/discussions
> >
> > Thanks,
> > George
> >
> > On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu>
wrote:
> >
> > > Hi Nick,
> > >
> > > The MET tools support a few different grid projections including
> Lambert
> > > Conformal. This section of the MET User's Guide describes what
> > information
> > > needs to be defined in a Python Embedding script to read that
grid:
> > >
> > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > >
> > > In the Python Embedding logic, the data is passed in as a 2D
array and
> > the
> > > metadata (attrs) defined in the script tell MET how to map the
data to
> a
> > > projection. The recommended way to test a Python Embedding
script is to
> > use
> > > the plot_data_plane application to create a plot and ensure that
the
> data
> > > is properly mapped in relation to the map outline.
> > >
> > > I hope that helps and let me know if you have any issues getting
this
> set
> > > up.
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
>
> > >>
> > >> Hi all,
> > >> I have another question on the subject of directly processing
WRF for
> > >> MET's
> > >> feature-relative use-case. Say I want to extract the PBL
height
> > variable
> > >> from a wrfout file, which is on just one layer/surface.
However,
> WRF's
> > >> horizontal grid is Lambert Conformal with a resolution of X-km.
From
> my
> > >> understanding, MET expects the grid to be fixed lat-lon in
degrees?
> Is
> > >> there a way that I can have MET read the data knowing it's not
on a
> > fixed
> > >> lat-lon grid (having it refer to the 2D lat/lon variables in
WRF)?
> > Then,
> > >> if so, can I extract the data normally (in the .conf, setting
> > >> BOTH_VAR1_LEVELS = Surface)?
> > >>
> > >> If none of this is true, then I assume others have done this
via first
> > >> post-processing the wrfout files (maybe through the embedded
python
> > >> scripts)?
> > >>
> > >> Thank you.
> > >> -Nick
> > >>
> > >>
> > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > nleonardo87 at gmail.com>
> > >> wrote:
> > >>
> > >> > I see. At least regarding the ingestion of WRF data, I
figure that
> > the
> > >> > embedded python script functionality can be used to first
read-in
> and
> > >> > convert/interpolate the data into something MET can use.
Thank you
> > all
> > >> > very much for checking.
> > >> > -Nick
> > >> >
> > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > >> met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> >> Hi Nick,
> > >> >>
> > >> >> To address this question:
> > >> >>
> > >> >> * First, for the series analysis, do I really need to run a
> separate
> > >> >> python
> > >> >> script for each diagnostic? This looks to be the case by
default
> > when
> > >> >> looking at PV and IVT, in which each script outputs only one
> variable
> > >> for
> > >> >> MET to then read. Can I combine multiple scripts into one
that
> > outputs
> > >> >> multiple variables? *
> > >> >>
> > >> >> Currently, yes, you will need to call a script for each
variable.
> The
> > >> MET
> > >> >> tools only read a single 2D field as gridded input and the
Python
> > >> >> Embedding
> > >> >> logic was designed to use a single script call to produce a
single
> > >> input
> > >> >> to
> > >> >> the MET tools.
> > >> >>
> > >> >> - George
> > >> >>
> > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> met_help at ucar.edu>
> > >> >> wrote:
> > >> >>
> > >> >> >
> > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> > >> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> > >> >> > Queue: met_help
> > >> >> > Subject: MET update and questions
> > >> >> > Owner: mccabe
> > >> >> > Requestors: nleonardo87 at gmail.com
> > >> >> > Status: open
> > >> >> > Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> > This transaction appears to have no content
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >> George McCabe - Software Engineer III
> > >> >> National Center for Atmospheric Research
> > >> >> Research Applications Laboratory
> > >> >> 303-497-2768
> > >> >> ---
> > >> >> My working day may not be your working day. Please do not
feel
> > obliged
> > >> to
> > >> >> reply to this email outside of your normal working hours.
> > >> >>
> > >> >>
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MET update and questions
From: Nicholas Leonardo
Time: Fri Jul 16 12:01:42 2021
Hi George,
Thank you very much for your help. This was all mostly to see if WRF
fields could be ingested/processed in the feature-relative program
(derivations would be for another day). If I'm interpreting this all
correctly, I *don't *need any python embedded scripts to first read-in
the
data.
For the sake of a quick test, I was only looking at the WRF analysis
fields
centered on the GFS analysis position of the same forecast cycle
(20210709
12z). I do not have tracker data for this WRF run, but figured it
would
not differ too much from the GFS at 0 h. However, if I need more
times for
this to work, I'll upload more data, make the edits to my .conf file,
and
give it another try .
-Nick
On Fri, Jul 16, 2021 at 1:25 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Nick,
>
> I apologize for the lengthy response.
>
> Looking at your Python embedding scripts, it looks like the fields
are read
> in with being modified. The reason Python Embedding was used in the
IVT use
> case is the IVT field needed to be computed from other fields inside
the
> input file. If the field you are trying to read into MET is found
inside
> the input file and the file is one of the supported NetCDF formats,
then
> using Python Embedding is not necessary for this step. It looks
like the
> file matches NETCDF_PINT which is created by running the p_interp or
> wrf_interp utility on WRF output (see
>
>
https://met.readthedocs.io/en/latest/Users_Guide/config_options.html?highlight=NETCDF_PINT#settings-
common-to-multiple-tools
> and search for NETCDF_PINT). I checked this by running
plot_data_plane and
> setting the file_type:
>
> /glade/p/ral/jntp/MET/MET_releases/10.0.0/bin/plot_data_plane
> /glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
out.ps
> 'name="PBLH"; level="(0,*,*)"; file_type=NETCDF_PINT;'
> convert -rotate 90 out.ps out.png
> display out.png
>
> The plot_data_plane command succeeded and the output image appeared
to have
> the correct map outlines, which shows that MET can read the data.
>
> If my assumption that there are no derivations needed to the data,
then you
> can read these files directly into ExtractTiles wrapper and access
the
> fields by specifying the NetCDF field info:
>
> BOTH_VAR1_NAME = PBLH
> BOTH_VAR1_LEVELS = "(0,*,*)"
> BOTH_VAR1_OPTIONS = file_type = NETCDF_PINT
>
> FCST_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
> FCST_EXTRACT_TILES_INPUT_TEMPLATE =
>
>
{init?fmt=%Y%m%d}/WRFgfs4_{init?fmt=%Y%m%d}_{init?fmt=%H}00_{lead?fmt=%3H}.nc
>
> OBS_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
> OBS_EXTRACT_TILES_INPUT_TEMPLATE =
>
{valid?fmt=%Y%m%d}/WRFnam4_{valid?fmt=%Y%m%d}_{valid?fmt=%H}00_000.nc
>
> I copied your config files and made some modifications. Starting in
METplus
> v4.0.0, the [filename_templates] and [dir] sections are no longer
used, so
> all of those variables can go under [config] unless they are
specific to a
> defined instance in the PROCESS_LIST i.e.
TCStat(for_series_analysis). I
> rearranged the variables so that they are easier to read including
pairing
> the corresponding _DIR and _TEMPLATE variables and putting all file
I/O
> variables together. You can copy my file from Cheyenne here:
>
> /glade/u/home/mccabe/100324/TCStat_SeriesAnalysis_wrfPBLH.conf
>
> I removed the PyEmbedIngest processing, updated the field info to
read the
> data directly from the NetCDF files, and updated the dir/template
variables
> to read the NetCDF files directly into ExtractTiles. This removed
the error
> you saw, however there are no pairs found from TCPairs:
>
>
> DEBUG 1: [Source 1 of 1] ADECK Source:
> /glade/u/home/nleonardo/ELSA/A_Bdecks/aal052021.dat, Model Suffix:
(nul)
> DEBUG 1: [Source 1 of 1] BDECK Source:
> /glade/u/home/nleonardo/ELSA/A_Bdecks/bal052021.dat, Model Suffix:
(nul)
> DEBUG 1: Config File Default:
>
>
/glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/config/TCPairsConfig_default
> DEBUG 1: Config File User:
>
> /glade/p/ral/jntp/MET/METplus/METplus-
4.0.0/parm/met_config/TCPairsConfig_wrapped
> DEBUG 1: Distance to land file:
> /glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/tc_data/
> dland_global_tenth_degree.nc
> DEBUG 2: Processing 1 BDECK file(s).
> DEBUG 2: Found 1 BDECK track(s).
> DEBUG 2: Processing 1 ADECK file(s).
> DEBUG 1: When reading ATCF track data, all instances of "AVN" are
> automatically replaced with "GFS" (ATCF ID = AVNO).
> WARNING:
> WARNING: int ATCFTrackLine::read_line(LineDataFile * ldf) -> found
fewer
> than the expected number of elements (6) in ATCF track line:
> WARNING: AL, 05, 2021063018, 03, AP16, 03
> WARNING:
>
> In the log output from this run, I noticed that the METPLUS_INIT_BEG
and
> METPLUS_INIT_END environment variables were set to filter the init
time:
>
> 07/16 10:58:26.004 metplus INFO: METPLUS_INIT_BEG=init_beg =
"2021070912";
> 07/16 10:58:26.004 metplus INFO: METPLUS_INIT_END=init_end =
"2021070912";
>
> These values are read from INIT_BEG and INIT_END because
TC_PAIRS_INIT_BEG
> and TC_PAIRS_INIT_END are not set. INIT_BEG and INIT_END are
typically used
> for looping over time and processing multiple run times, so it is
> recommended to also set the TC_PAIRS_ variables as well. I set these
to an
> empty value to prevent restricting the init times that are
processed.
>
> This next run produces output for TCPairs and TCStat, however
ExtractTiles
> failed to find the corresponding forecast and analysis data because
it
> looks like there are only files in your directory for 20210709 12Z 0
hr
> forecast. You will either have to change the criteria that goes into
> creating the storm track files to match the available data or make
more
> forecast/analysis data available to be read.
>
> Note this message in the TCPairs log file:
>
> DEBUG 1: When reading ATCF track data, all instances of "AVN" are
> automatically replaced with "GFS" (ATCF ID = AVNO).
>
> The MODEL value in your config file is set to GFSO, so this message
is
> saying that it is processing ADECK data that match AVNO. Changing
the MODEL
> value can change what is read from these files.
>
> An unrelated thing I noticed is your tutorial.conf file sets
> MET_INSTALL_DIR = /glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5.
The
> stable 10.0.0 version of MET is available and will automatically be
used by
> the instance of METplus 4.0.0 that is installed on Cheyenne, so you
don't
> need to set this variable here.
>
> I know this is a lot of info but I hope this helps you set up this
use
> case. Let me know if you have any other questions about this. Also,
posting
> paths on GitHub Discussions shouldn't be an issue because other
users
> likely don't have access to that machine. There is actually a
archive of
> MET Help tickets online that while not as easy to access as the
dicussions
> board is still available to the public. Any sensitive information
can be
> emailed to a METplus team member if necessary.
>
> Thanks,
> George
>
> On Thu, Jul 15, 2021 at 2:54 PM Nicholas Leonardo via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> >
> > Hi George,
> > Thank you for your help. I have another question regarding a
problem I
> > reached while trying this out. I just set up a GitHub account and
can
> post
> > it there if preferred (I assume it's safe to share files that
reveal my
> > paths/directories?).
> >
> > Otherwise, I try to run everything after following the
instructions
> (e.g.,
> > run the master_metplus.py) and get an error message that does not
> > make sense. It says the following: *ERROR : parse_grid_mask()
-> can't
> > open file
> >
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> > even though the file and path given do (no typos). I can actually
run my
> > embedded python scripts (WRF_PBLhgt_....py) on the data directly
with no
> > problem, so the error is a little puzzling. I assume it has to do
with
> > something else I must edit in the .conf file? I've attached these
python
> > scripts, as well as the .conf file and error log. If indeed
preferred
> and
> > safe, I'll post these and more info on GitHub too.
> >
> > Thank you.
> > -Nick
> >
> > On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Nick,
> > >
> > > FYI:
> > >
> > > We switched from providing support through this email address to
> > providing
> > > support through GitHub Discussions. In the future, you will need
to
> > create
> > > a free GitHub account if you don’t have one already and post
your
> > questions
> > > to the METplus Components Discussion Forum:
> > > https://github.com/dtcenter/METplus/discussions
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, Jul 13, 2021 at 9:04 AM George McCabe <mccabe at ucar.edu>
wrote:
> > >
> > > > Hi Nick,
> > > >
> > > > The MET tools support a few different grid projections
including
> > Lambert
> > > > Conformal. This section of the MET User's Guide describes what
> > > information
> > > > needs to be defined in a Python Embedding script to read that
grid:
> > > >
> > > >
> > > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > > >
> > > > In the Python Embedding logic, the data is passed in as a 2D
array
> and
> > > the
> > > > metadata (attrs) defined in the script tell MET how to map the
data
> to
> > a
> > > > projection. The recommended way to test a Python Embedding
script is
> to
> > > use
> > > > the plot_data_plane application to create a plot and ensure
that the
> > data
> > > > is properly mapped in relation to the map outline.
> > > >
> > > > I hope that helps and let me know if you have any issues
getting this
> > set
> > > > up.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > > >>
> > > >> Hi all,
> > > >> I have another question on the subject of directly processing
WRF
> for
> > > >> MET's
> > > >> feature-relative use-case. Say I want to extract the PBL
height
> > > variable
> > > >> from a wrfout file, which is on just one layer/surface.
However,
> > WRF's
> > > >> horizontal grid is Lambert Conformal with a resolution of X-
km.
> From
> > my
> > > >> understanding, MET expects the grid to be fixed lat-lon in
degrees?
> > Is
> > > >> there a way that I can have MET read the data knowing it's
not on a
> > > fixed
> > > >> lat-lon grid (having it refer to the 2D lat/lon variables in
WRF)?
> > > Then,
> > > >> if so, can I extract the data normally (in the .conf, setting
> > > >> BOTH_VAR1_LEVELS = Surface)?
> > > >>
> > > >> If none of this is true, then I assume others have done this
via
> first
> > > >> post-processing the wrfout files (maybe through the embedded
python
> > > >> scripts)?
> > > >>
> > > >> Thank you.
> > > >> -Nick
> > > >>
> > > >>
> > > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > > nleonardo87 at gmail.com>
> > > >> wrote:
> > > >>
> > > >> > I see. At least regarding the ingestion of WRF data, I
figure
> that
> > > the
> > > >> > embedded python script functionality can be used to first
read-in
> > and
> > > >> > convert/interpolate the data into something MET can use.
Thank
> you
> > > all
> > > >> > very much for checking.
> > > >> > -Nick
> > > >> >
> > > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > > >> met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Nick,
> > > >> >>
> > > >> >> To address this question:
> > > >> >>
> > > >> >> * First, for the series analysis, do I really need to run
a
> > separate
> > > >> >> python
> > > >> >> script for each diagnostic? This looks to be the case by
default
> > > when
> > > >> >> looking at PV and IVT, in which each script outputs only
one
> > variable
> > > >> for
> > > >> >> MET to then read. Can I combine multiple scripts into one
that
> > > outputs
> > > >> >> multiple variables? *
> > > >> >>
> > > >> >> Currently, yes, you will need to call a script for each
variable.
> > The
> > > >> MET
> > > >> >> tools only read a single 2D field as gridded input and the
Python
> > > >> >> Embedding
> > > >> >> logic was designed to use a single script call to produce
a
> single
> > > >> input
> > > >> >> to
> > > >> >> the MET tools.
> > > >> >>
> > > >> >> - George
> > > >> >>
> > > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> > met_help at ucar.edu>
> > > >> >> wrote:
> > > >> >>
> > > >> >> >
> > > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted upon.
> > > >> >> > Transaction: Given to mccabe (George McCabe) by minnawin
> > > >> >> > Queue: met_help
> > > >> >> > Subject: MET update and questions
> > > >> >> > Owner: mccabe
> > > >> >> > Requestors: nleonardo87 at gmail.com
> > > >> >> > Status: open
> > > >> >> > Ticket <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> > This transaction appears to have no content
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >> George McCabe - Software Engineer III
> > > >> >> National Center for Atmospheric Research
> > > >> >> Research Applications Laboratory
> > > >> >> 303-497-2768
> > > >> >> ---
> > > >> >> My working day may not be your working day. Please do not
feel
> > > obliged
> > > >> to
> > > >> >> reply to this email outside of your normal working hours.
> > > >> >>
> > > >> >>
> > > >>
> > > >>
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: MET update and questions
From: George McCabe
Time: Mon Jul 19 09:35:11 2021
Hi Nick,
Yes, it appears that you don't need Python embedding scripts to read
in the
data. Your own Python script is only needed if the data you are trying
to
read is in a format that MET cannot read natively (i.e. NetCDF files
that
are not CF compliant or generated by MET itself) OR if you need to
compute
a field from the input data (i.e. IVT). A good first test is to use
plot_data_plane to see if you can get the file/field read and plotted
on
the correct grid (aligns with the map outlines). If you are able to
generate a good plot, then you can safely assume that the other MET
tools
can read the data as well.
I'm going to close this ticket now. Feel free to open a new GitHub
Discussions topic if you run into any other issues.
Thanks,
George
On Fri, Jul 16, 2021 at 12:02 PM Nicholas Leonardo via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
>
> Hi George,
> Thank you very much for your help. This was all mostly to see if
WRF
> fields could be ingested/processed in the feature-relative program
> (derivations would be for another day). If I'm interpreting this
all
> correctly, I *don't *need any python embedded scripts to first read-
in the
> data.
>
> For the sake of a quick test, I was only looking at the WRF analysis
fields
> centered on the GFS analysis position of the same forecast cycle
(20210709
> 12z). I do not have tracker data for this WRF run, but figured it
would
> not differ too much from the GFS at 0 h. However, if I need more
times for
> this to work, I'll upload more data, make the edits to my .conf
file, and
> give it another try .
>
> -Nick
>
> On Fri, Jul 16, 2021 at 1:25 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Nick,
> >
> > I apologize for the lengthy response.
> >
> > Looking at your Python embedding scripts, it looks like the fields
are
> read
> > in with being modified. The reason Python Embedding was used in
the IVT
> use
> > case is the IVT field needed to be computed from other fields
inside the
> > input file. If the field you are trying to read into MET is found
inside
> > the input file and the file is one of the supported NetCDF
formats, then
> > using Python Embedding is not necessary for this step. It looks
like the
> > file matches NETCDF_PINT which is created by running the p_interp
or
> > wrf_interp utility on WRF output (see
> >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/config_options.html?highlight=NETCDF_PINT#settings-
common-to-multiple-tools
> > and search for NETCDF_PINT). I checked this by running
plot_data_plane
> and
> > setting the file_type:
> >
> > /glade/p/ral/jntp/MET/MET_releases/10.0.0/bin/plot_data_plane
> > /glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc
> out.ps
> > 'name="PBLH"; level="(0,*,*)"; file_type=NETCDF_PINT;'
> > convert -rotate 90 out.ps out.png
> > display out.png
> >
> > The plot_data_plane command succeeded and the output image
appeared to
> have
> > the correct map outlines, which shows that MET can read the data.
> >
> > If my assumption that there are no derivations needed to the data,
then
> you
> > can read these files directly into ExtractTiles wrapper and access
the
> > fields by specifying the NetCDF field info:
> >
> > BOTH_VAR1_NAME = PBLH
> > BOTH_VAR1_LEVELS = "(0,*,*)"
> > BOTH_VAR1_OPTIONS = file_type = NETCDF_PINT
> >
> > FCST_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
> > FCST_EXTRACT_TILES_INPUT_TEMPLATE =
> >
> >
>
{init?fmt=%Y%m%d}/WRFgfs4_{init?fmt=%Y%m%d}_{init?fmt=%H}00_{lead?fmt=%3H}.nc
> >
> > OBS_EXTRACT_TILES_INPUT_DIR = {INPUT_BASE}
> > OBS_EXTRACT_TILES_INPUT_TEMPLATE =
> >
{valid?fmt=%Y%m%d}/WRFnam4_{valid?fmt=%Y%m%d}_{valid?fmt=%H}00_000.nc
> >
> > I copied your config files and made some modifications. Starting
in
> METplus
> > v4.0.0, the [filename_templates] and [dir] sections are no longer
used,
> so
> > all of those variables can go under [config] unless they are
specific to
> a
> > defined instance in the PROCESS_LIST i.e.
TCStat(for_series_analysis). I
> > rearranged the variables so that they are easier to read including
> pairing
> > the corresponding _DIR and _TEMPLATE variables and putting all
file I/O
> > variables together. You can copy my file from Cheyenne here:
> >
> > /glade/u/home/mccabe/100324/TCStat_SeriesAnalysis_wrfPBLH.conf
> >
> > I removed the PyEmbedIngest processing, updated the field info to
read
> the
> > data directly from the NetCDF files, and updated the dir/template
> variables
> > to read the NetCDF files directly into ExtractTiles. This removed
the
> error
> > you saw, however there are no pairs found from TCPairs:
> >
> >
> > DEBUG 1: [Source 1 of 1] ADECK Source:
> > /glade/u/home/nleonardo/ELSA/A_Bdecks/aal052021.dat, Model Suffix:
(nul)
> > DEBUG 1: [Source 1 of 1] BDECK Source:
> > /glade/u/home/nleonardo/ELSA/A_Bdecks/bal052021.dat, Model Suffix:
(nul)
> > DEBUG 1: Config File Default:
> >
> >
>
/glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/config/TCPairsConfig_default
> > DEBUG 1: Config File User:
> >
> >
> /glade/p/ral/jntp/MET/METplus/METplus-
4.0.0/parm/met_config/TCPairsConfig_wrapped
> > DEBUG 1: Distance to land file:
> > /glade/p/ral/jntp/MET/MET_releases/10.0.0/share/met/tc_data/
> > dland_global_tenth_degree.nc
> > DEBUG 2: Processing 1 BDECK file(s).
> > DEBUG 2: Found 1 BDECK track(s).
> > DEBUG 2: Processing 1 ADECK file(s).
> > DEBUG 1: When reading ATCF track data, all instances of "AVN" are
> > automatically replaced with "GFS" (ATCF ID = AVNO).
> > WARNING:
> > WARNING: int ATCFTrackLine::read_line(LineDataFile * ldf) -> found
fewer
> > than the expected number of elements (6) in ATCF track line:
> > WARNING: AL, 05, 2021063018, 03, AP16, 03
> > WARNING:
> >
> > In the log output from this run, I noticed that the
METPLUS_INIT_BEG and
> > METPLUS_INIT_END environment variables were set to filter the init
time:
> >
> > 07/16 10:58:26.004 metplus INFO: METPLUS_INIT_BEG=init_beg =
> "2021070912";
> > 07/16 10:58:26.004 metplus INFO: METPLUS_INIT_END=init_end =
> "2021070912";
> >
> > These values are read from INIT_BEG and INIT_END because
> TC_PAIRS_INIT_BEG
> > and TC_PAIRS_INIT_END are not set. INIT_BEG and INIT_END are
typically
> used
> > for looping over time and processing multiple run times, so it is
> > recommended to also set the TC_PAIRS_ variables as well. I set
these to
> an
> > empty value to prevent restricting the init times that are
processed.
> >
> > This next run produces output for TCPairs and TCStat, however
> ExtractTiles
> > failed to find the corresponding forecast and analysis data
because it
> > looks like there are only files in your directory for 20210709 12Z
0 hr
> > forecast. You will either have to change the criteria that goes
into
> > creating the storm track files to match the available data or make
more
> > forecast/analysis data available to be read.
> >
> > Note this message in the TCPairs log file:
> >
> > DEBUG 1: When reading ATCF track data, all instances of "AVN" are
> > automatically replaced with "GFS" (ATCF ID = AVNO).
> >
> > The MODEL value in your config file is set to GFSO, so this
message is
> > saying that it is processing ADECK data that match AVNO. Changing
the
> MODEL
> > value can change what is read from these files.
> >
> > An unrelated thing I noticed is your tutorial.conf file sets
> > MET_INSTALL_DIR = /glade/p/ral/jntp/MET/MET_releases/10.0.0-beta5.
The
> > stable 10.0.0 version of MET is available and will automatically
be used
> by
> > the instance of METplus 4.0.0 that is installed on Cheyenne, so
you don't
> > need to set this variable here.
> >
> > I know this is a lot of info but I hope this helps you set up this
use
> > case. Let me know if you have any other questions about this.
Also,
> posting
> > paths on GitHub Discussions shouldn't be an issue because other
users
> > likely don't have access to that machine. There is actually a
archive of
> > MET Help tickets online that while not as easy to access as the
> dicussions
> > board is still available to the public. Any sensitive information
can be
> > emailed to a METplus team member if necessary.
> >
> > Thanks,
> > George
> >
> > On Thu, Jul 15, 2021 at 2:54 PM Nicholas Leonardo via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > >
> > > Hi George,
> > > Thank you for your help. I have another question regarding a
problem I
> > > reached while trying this out. I just set up a GitHub account
and can
> > post
> > > it there if preferred (I assume it's safe to share files that
reveal my
> > > paths/directories?).
> > >
> > > Otherwise, I try to run everything after following the
instructions
> > (e.g.,
> > > run the master_metplus.py) and get an error message that does
not
> > > make sense. It says the following: *ERROR : parse_grid_mask()
->
> can't
> > > open file
> > >
"/glade/u/home/nleonardo/ELSA/20210709/WRFgfs4_20210709_1200_000.nc"*
> > > even though the file and path given do (no typos). I can
actually run
> my
> > > embedded python scripts (WRF_PBLhgt_....py) on the data directly
with
> no
> > > problem, so the error is a little puzzling. I assume it has to
do with
> > > something else I must edit in the .conf file? I've attached
these
> python
> > > scripts, as well as the .conf file and error log. If indeed
preferred
> > and
> > > safe, I'll post these and more info on GitHub too.
> > >
> > > Thank you.
> > > -Nick
> > >
> > > On Tue, Jul 13, 2021 at 11:22 AM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Nick,
> > > >
> > > > FYI:
> > > >
> > > > We switched from providing support through this email address
to
> > > providing
> > > > support through GitHub Discussions. In the future, you will
need to
> > > create
> > > > a free GitHub account if you don’t have one already and post
your
> > > questions
> > > > to the METplus Components Discussion Forum:
> > > > https://github.com/dtcenter/METplus/discussions
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, Jul 13, 2021 at 9:04 AM George McCabe
<mccabe at ucar.edu>
> wrote:
> > > >
> > > > > Hi Nick,
> > > > >
> > > > > The MET tools support a few different grid projections
including
> > > Lambert
> > > > > Conformal. This section of the MET User's Guide describes
what
> > > > information
> > > > > needs to be defined in a Python Embedding script to read
that grid:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#python-
embedding-for-2d-data
> > > > >
> > > > > In the Python Embedding logic, the data is passed in as a 2D
array
> > and
> > > > the
> > > > > metadata (attrs) defined in the script tell MET how to map
the data
> > to
> > > a
> > > > > projection. The recommended way to test a Python Embedding
script
> is
> > to
> > > > use
> > > > > the plot_data_plane application to create a plot and ensure
that
> the
> > > data
> > > > > is properly mapped in relation to the map outline.
> > > > >
> > > > > I hope that helps and let me know if you have any issues
getting
> this
> > > set
> > > > > up.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, Jul 13, 2021 at 12:06 AM Nicholas Leonardo via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324 >
> > > > >>
> > > > >> Hi all,
> > > > >> I have another question on the subject of directly
processing WRF
> > for
> > > > >> MET's
> > > > >> feature-relative use-case. Say I want to extract the PBL
height
> > > > variable
> > > > >> from a wrfout file, which is on just one layer/surface.
However,
> > > WRF's
> > > > >> horizontal grid is Lambert Conformal with a resolution of
X-km.
> > From
> > > my
> > > > >> understanding, MET expects the grid to be fixed lat-lon in
> degrees?
> > > Is
> > > > >> there a way that I can have MET read the data knowing it's
not on
> a
> > > > fixed
> > > > >> lat-lon grid (having it refer to the 2D lat/lon variables
in WRF)?
> > > > Then,
> > > > >> if so, can I extract the data normally (in the .conf,
setting
> > > > >> BOTH_VAR1_LEVELS = Surface)?
> > > > >>
> > > > >> If none of this is true, then I assume others have done
this via
> > first
> > > > >> post-processing the wrfout files (maybe through the
embedded
> python
> > > > >> scripts)?
> > > > >>
> > > > >> Thank you.
> > > > >> -Nick
> > > > >>
> > > > >>
> > > > >> On Mon, Jun 28, 2021 at 1:17 PM Nicholas Leonardo <
> > > > nleonardo87 at gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > I see. At least regarding the ingestion of WRF data, I
figure
> > that
> > > > the
> > > > >> > embedded python script functionality can be used to first
> read-in
> > > and
> > > > >> > convert/interpolate the data into something MET can use.
Thank
> > you
> > > > all
> > > > >> > very much for checking.
> > > > >> > -Nick
> > > > >> >
> > > > >> > On Mon, Jun 28, 2021 at 12:47 PM George McCabe via RT <
> > > > >> met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Hi Nick,
> > > > >> >>
> > > > >> >> To address this question:
> > > > >> >>
> > > > >> >> * First, for the series analysis, do I really need to
run a
> > > separate
> > > > >> >> python
> > > > >> >> script for each diagnostic? This looks to be the case
by
> default
> > > > when
> > > > >> >> looking at PV and IVT, in which each script outputs only
one
> > > variable
> > > > >> for
> > > > >> >> MET to then read. Can I combine multiple scripts into
one that
> > > > outputs
> > > > >> >> multiple variables? *
> > > > >> >>
> > > > >> >> Currently, yes, you will need to call a script for each
> variable.
> > > The
> > > > >> MET
> > > > >> >> tools only read a single 2D field as gridded input and
the
> Python
> > > > >> >> Embedding
> > > > >> >> logic was designed to use a single script call to
produce a
> > single
> > > > >> input
> > > > >> >> to
> > > > >> >> the MET tools.
> > > > >> >>
> > > > >> >> - George
> > > > >> >>
> > > > >> >> On Thu, Jun 24, 2021 at 1:26 PM Minna Win via RT <
> > > met_help at ucar.edu>
> > > > >> >> wrote:
> > > > >> >>
> > > > >> >> >
> > > > >> >> > Thu Jun 24 13:26:02 2021: Request 100324 was acted
upon.
> > > > >> >> > Transaction: Given to mccabe (George McCabe) by
minnawin
> > > > >> >> > Queue: met_help
> > > > >> >> > Subject: MET update and questions
> > > > >> >> > Owner: mccabe
> > > > >> >> > Requestors: nleonardo87 at gmail.com
> > > > >> >> > Status: open
> > > > >> >> > Ticket <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100324
> > > > >> >> >
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > This transaction appears to have no content
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> George McCabe - Software Engineer III
> > > > >> >> National Center for Atmospheric Research
> > > > >> >> Research Applications Laboratory
> > > > >> >> 303-497-2768
> > > > >> >> ---
> > > > >> >> My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > >> to
> > > > >> >> reply to this email outside of your normal working
hours.
> > > > >> >>
> > > > >> >>
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list