[Met_help] [rt.rap.ucar.edu #97508] History for Questions about METplus python embedding
George McCabe via RT
met_help at ucar.edu
Wed Nov 18 12:18:07 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi MET support,
I have been reading through the METplus-3.1/MET-9.1 documentation
regarding python embedding. I see a list of examples here
https://dtcenter.org/community-code/model-evaluation-tools-met/sample-analysis-scripts
but they are all using the same plot_data_plane command. Do you have
some examples of python embedding using other tools such as grid_stat
and point_stat which handle multiple input files? Also, is there a way
these tools can return the results in memory in the form of some data
structures (e.g. pandas) with specific names? I might be a little
ignorant on these questions since I am just learning the METplus interface.
Thanks,
Scott
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Questions about METplus python embedding
From: George McCabe
Time: Wed Nov 18 11:33:20 2020
Hi Scott,
The following link lists all of the existing METplus use cases that
utilize
Python embedding:
https://dtcenter.github.io/METplus/v3.1/search.html?q=PythonEmbeddingFileUseCase&check_keywords=yes&area=default
There is a METplus use case that uses Python embedding with GridStat
that
is available in the 4.0-beta1 release of the METplus wrappers. The
release
is available here:
https://github.com/dtcenter/METplus/releases/tag/v4.0-beta1. Please
note
that this is a beta release and not an official stable release.
Information about this use case can be found here:
https://dtcenter.github.io/METplus/develop/generated/met_tool_wrapper/GridStat/GridStat_python_embedding.html#metplus-
configuration
Running this use case will generate a call to grid_stat that looks
something like this:
grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
/my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
/my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700
The METplus wrapper sets environment variables that set values in the
MET
config file (GridStatConfig_wrapped). Here is an example of what is
set in
that file:
fcst = {
field = [ {
name="/input/base/met_test/scripts/python/read_ascii_numpy.py
/input/base/met_test/data/python/fcst.txt FCST"; } ];
}
The app point_stat does support Python embedding. However, there are
currently no METplus use cases that demonstrate running PointStat in
this
way yet. The method to setting this up is similar to the approach used
for
GridStat.
Currently, the MET tools cannot return data in memory that can be used
in
Python scripts. This has been discussed as a feature that will
eventually
be added, but there are many details that need to be worked out before
it
can be implemented. The tools write a NetCDF file to disk that could
be
read into a Python script.
Please let me know if you have any other questions.
Thanks,
George
On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
> Transaction: Ticket created by scott.rabenhorst at gmail.com
> Queue: met_help
> Subject: Questions about METplus python embedding
> Owner: Nobody
> Requestors: scott.rabenhorst at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97508 >
>
>
> Hi MET support,
>
> I have been reading through the METplus-3.1/MET-9.1 documentation
> regarding python embedding. I see a list of examples here
>
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> but they are all using the same plot_data_plane command. Do you have
> some examples of python embedding using other tools such as
grid_stat
> and point_stat which handle multiple input files? Also, is there a
way
> these tools can return the results in memory in the form of some
data
> structures (e.g. pandas) with specific names? I might be a little
> ignorant on these questions since I am just learning the METplus
interface.
>
> Thanks,
> Scott
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #97508] Questions about METplus python embedding
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Wed Nov 18 12:11:30 2020
Hi George,
Thanks for your prompt response. These links are very helpful - I
didn't see that beforehand. So it sounds like the grid/point_stat
wrappers for embedding are only in v4.0-beta1. I'll look forward to
when some results can live in python memory. For now I will try to
familiarize myself with the current capabilities. Thanks for your
help.
Thanks,
Scott
On 11/18/20, 1:33 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
The following link lists all of the existing METplus use cases
that utilize
Python embedding:
https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_v3.1_search.html-3Fq-
3DPythonEmbeddingFileUseCase-26check-5Fkeywords-3Dyes-26area-
3Ddefault&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=wn0YbuwWDh93QlHeaq7SxTQJsCPN9Sm8xV7nDDSD3Ns&e=
There is a METplus use case that uses Python embedding with
GridStat that
is available in the 4.0-beta1 release of the METplus wrappers. The
release
is available here:
https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_dtcenter_METplus_releases_tag_v4.0-
2Dbeta1&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=labgZpL3SlT-
MfOiOsQK4nbmy5JrfQVQjIudbND7R4A&e= . Please note
that this is a beta release and not an official stable release.
Information about this use case can be found here:
https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_develop_generated_met-5Ftool-
5Fwrapper_GridStat_GridStat-5Fpython-5Fembedding.html-23metplus-
2Dconfiguration&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=oqCH6rwuwlSjbEI4qauLTuiuQ34h556Adn2XTOaL-
Y0&e=
Running this use case will generate a call to grid_stat that looks
something like this:
grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
/my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
/my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700
The METplus wrapper sets environment variables that set values in
the MET
config file (GridStatConfig_wrapped). Here is an example of what
is set in
that file:
fcst = {
field = [ {
name="/input/base/met_test/scripts/python/read_ascii_numpy.py
/input/base/met_test/data/python/fcst.txt FCST"; } ];
}
The app point_stat does support Python embedding. However, there
are
currently no METplus use cases that demonstrate running PointStat
in this
way yet. The method to setting this up is similar to the approach
used for
GridStat.
Currently, the MET tools cannot return data in memory that can be
used in
Python scripts. This has been discussed as a feature that will
eventually
be added, but there are many details that need to be worked out
before it
can be implemented. The tools write a NetCDF file to disk that
could be
read into a Python script.
Please let me know if you have any other questions.
Thanks,
George
On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
> Transaction: Ticket created by scott.rabenhorst at gmail.com
> Queue: met_help
> Subject: Questions about METplus python embedding
> Owner: Nobody
> Requestors: scott.rabenhorst at gmail.com
> Status: new
> Ticket <URL: https://urldefense.proofpoint.com/v2/url?u=https-
3A__rt.rap.ucar.edu_rt_Ticket_Display.html-3Fid-
3D97508&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=tbJm8XKW4diQF4bguVsOJyMXrEc6OpmHWoO2RDHPtSY&e=
>
>
>
> Hi MET support,
>
> I have been reading through the METplus-3.1/MET-9.1
documentation
> regarding python embedding. I see a list of examples here
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.org_community-2Dcode_model-2Devaluation-2Dtools-
2Dmet_sample-2Danalysis-
2Dscripts&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=keuM-
x1n0AMqs4LYe25jafy4kq5TiP7AyL2KgzLOvRA&e=
> but they are all using the same plot_data_plane command. Do you
have
> some examples of python embedding using other tools such as
grid_stat
> and point_stat which handle multiple input files? Also, is there
a way
> these tools can return the results in memory in the form of some
data
> structures (e.g. pandas) with specific names? I might be a
little
> ignorant on these questions since I am just learning the METplus
interface.
>
> Thanks,
> Scott
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Questions about METplus python embedding
From: George McCabe
Time: Wed Nov 18 12:18:02 2020
Hi Scott,
To clarify, it is possible to configure the METplus wrappers to
utilize
Python embedding with these tools. What is missing is an example
demonstrating this configuration.
- George
On Wed, Nov 18, 2020 at 12:12 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97508 >
>
> Hi George,
>
> Thanks for your prompt response. These links are very helpful - I
didn't
> see that beforehand. So it sounds like the grid/point_stat wrappers
for
> embedding are only in v4.0-beta1. I'll look forward to when some
results
> can live in python memory. For now I will try to familiarize myself
with
> the current capabilities. Thanks for your help.
>
> Thanks,
> Scott
>
>
>
> On 11/18/20, 1:33 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> The following link lists all of the existing METplus use cases
that
> utilize
> Python embedding:
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_v3.1_search.html-3Fq-
3DPythonEmbeddingFileUseCase-26check-5Fkeywords-3Dyes-26area-
3Ddefault&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=wn0YbuwWDh93QlHeaq7SxTQJsCPN9Sm8xV7nDDSD3Ns&e=
>
> There is a METplus use case that uses Python embedding with
GridStat
> that
> is available in the 4.0-beta1 release of the METplus wrappers.
The
> release
> is available here:
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_dtcenter_METplus_releases_tag_v4.0-
2Dbeta1&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=labgZpL3SlT-
MfOiOsQK4nbmy5JrfQVQjIudbND7R4A&e=
> . Please note
> that this is a beta release and not an official stable release.
>
> Information about this use case can be found here:
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_develop_generated_met-5Ftool-
5Fwrapper_GridStat_GridStat-5Fpython-5Fembedding.html-23metplus-
2Dconfiguration&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=oqCH6rwuwlSjbEI4qauLTuiuQ34h556Adn2XTOaL-
Y0&e=
>
> Running this use case will generate a call to grid_stat that
looks
> something like this:
>
> grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
> /my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
>
/my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700
>
> The METplus wrapper sets environment variables that set values
in the
> MET
> config file (GridStatConfig_wrapped). Here is an example of what
is
> set in
> that file:
>
> fcst = {
> field = [ {
> name="/input/base/met_test/scripts/python/read_ascii_numpy.py
> /input/base/met_test/data/python/fcst.txt FCST"; } ];
> }
>
> The app point_stat does support Python embedding. However, there
are
> currently no METplus use cases that demonstrate running
PointStat in
> this
> way yet. The method to setting this up is similar to the
approach used
> for
> GridStat.
>
> Currently, the MET tools cannot return data in memory that can
be used
> in
> Python scripts. This has been discussed as a feature that will
> eventually
> be added, but there are many details that need to be worked out
before
> it
> can be implemented. The tools write a NetCDF file to disk that
could be
> read into a Python script.
>
> Please let me know if you have any other questions.
>
> Thanks,
> George
>
> On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
> > Transaction: Ticket created by scott.rabenhorst at gmail.com
> > Queue: met_help
> > Subject: Questions about METplus python embedding
> > Owner: Nobody
> > Requestors: scott.rabenhorst at gmail.com
> > Status: new
> > Ticket <URL:
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__rt.rap.ucar.edu_rt_Ticket_Display.html-3Fid-
3D97508&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=tbJm8XKW4diQF4bguVsOJyMXrEc6OpmHWoO2RDHPtSY&e=
> >
> >
> >
> > Hi MET support,
> >
> > I have been reading through the METplus-3.1/MET-9.1
documentation
> > regarding python embedding. I see a list of examples here
> >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.org_community-2Dcode_model-2Devaluation-2Dtools-
2Dmet_sample-2Danalysis-
2Dscripts&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=keuM-
x1n0AMqs4LYe25jafy4kq5TiP7AyL2KgzLOvRA&e=
> > but they are all using the same plot_data_plane command. Do
you have
> > some examples of python embedding using other tools such as
grid_stat
> > and point_stat which handle multiple input files? Also, is
there a
> way
> > these tools can return the results in memory in the form of
some data
> > structures (e.g. pandas) with specific names? I might be a
little
> > ignorant on these questions since I am just learning the
METplus
> interface.
> >
> > Thanks,
> > Scott
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list