[Met_help] [rt.rap.ucar.edu #97508] History for Questions about METplus python embedding

George McCabe via RT met_help at ucar.edu
Wed Nov 18 12:18:07 MST 2020


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi MET support,

I have been reading through the METplus-3.1/MET-9.1 documentation 
regarding python embedding. I see a list of examples here 
https://dtcenter.org/community-code/model-evaluation-tools-met/sample-analysis-scripts 
but they are all using the same plot_data_plane command. Do you have 
some examples of python embedding using other tools such as grid_stat 
and point_stat which handle multiple input files? Also, is there a way 
these tools can return the results in memory in the form of some data 
structures (e.g. pandas) with specific names? I might be a little 
ignorant on these questions since I am just learning the METplus interface.

Thanks,
Scott



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Questions about METplus python embedding
From: George McCabe
Time: Wed Nov 18 11:33:20 2020

Hi Scott,

The following link lists all of the existing METplus use cases that
utilize
Python embedding:
https://dtcenter.github.io/METplus/v3.1/search.html?q=PythonEmbeddingFileUseCase&check_keywords=yes&area=default

There is a METplus use case that uses Python embedding with GridStat
that
is available in the 4.0-beta1 release of the METplus wrappers. The
release
is available here:
https://github.com/dtcenter/METplus/releases/tag/v4.0-beta1. Please
note
that this is a beta release and not an official stable release.

Information about this use case can be found here:

https://dtcenter.github.io/METplus/develop/generated/met_tool_wrapper/GridStat/GridStat_python_embedding.html#metplus-
configuration

Running this use case will generate a call to grid_stat that looks
something like this:

grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
/my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
/my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700

The METplus wrapper sets environment variables that set values in the
MET
config file (GridStatConfig_wrapped). Here is an example of what is
set in
that file:

fcst = {
   field = [ {
name="/input/base/met_test/scripts/python/read_ascii_numpy.py
/input/base/met_test/data/python/fcst.txt FCST"; } ];
}

The app point_stat does support Python embedding. However, there are
currently no METplus use cases that demonstrate running PointStat in
this
way yet. The method to setting this up is similar to the approach used
for
GridStat.

Currently, the MET tools cannot return data in memory that can be used
in
Python scripts. This has been discussed as a feature that will
eventually
be added, but there are many details that need to be worked out before
it
can be implemented. The tools write a NetCDF file to disk that could
be
read into a Python script.

Please let me know if you have any other questions.

Thanks,
George

On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT
<met_help at ucar.edu>
wrote:

>
> Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
> Transaction: Ticket created by scott.rabenhorst at gmail.com
>        Queue: met_help
>      Subject: Questions about METplus python embedding
>        Owner: Nobody
>   Requestors: scott.rabenhorst at gmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97508 >
>
>
> Hi MET support,
>
> I have been reading through the METplus-3.1/MET-9.1 documentation
> regarding python embedding. I see a list of examples here
>
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> but they are all using the same plot_data_plane command. Do you have
> some examples of python embedding using other tools such as
grid_stat
> and point_stat which handle multiple input files? Also, is there a
way
> these tools can return the results in memory in the form of some
data
> structures (e.g. pandas) with specific names? I might be a little
> ignorant on these questions since I am just learning the METplus
interface.
>
> Thanks,
> Scott
>
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #97508] Questions about METplus python embedding
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Wed Nov 18 12:11:30 2020

Hi George,

Thanks for your prompt response. These links are very helpful - I
didn't see that beforehand. So it sounds like the grid/point_stat
wrappers for embedding are only in v4.0-beta1. I'll look forward to
when some results can live in python memory. For now I will try to
familiarize myself with the current capabilities. Thanks for your
help.

Thanks,
Scott



On 11/18/20, 1:33 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:

    Hi Scott,

    The following link lists all of the existing METplus use cases
that utilize
    Python embedding:
    https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_v3.1_search.html-3Fq-
3DPythonEmbeddingFileUseCase-26check-5Fkeywords-3Dyes-26area-
3Ddefault&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=wn0YbuwWDh93QlHeaq7SxTQJsCPN9Sm8xV7nDDSD3Ns&e=

    There is a METplus use case that uses Python embedding with
GridStat that
    is available in the 4.0-beta1 release of the METplus wrappers. The
release
    is available here:
    https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_dtcenter_METplus_releases_tag_v4.0-
2Dbeta1&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=labgZpL3SlT-
MfOiOsQK4nbmy5JrfQVQjIudbND7R4A&e= . Please note
    that this is a beta release and not an official stable release.

    Information about this use case can be found here:

    https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_develop_generated_met-5Ftool-
5Fwrapper_GridStat_GridStat-5Fpython-5Fembedding.html-23metplus-
2Dconfiguration&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=oqCH6rwuwlSjbEI4qauLTuiuQ34h556Adn2XTOaL-
Y0&e=

    Running this use case will generate a call to grid_stat that looks
    something like this:

    grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
    /my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
    /my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700

    The METplus wrapper sets environment variables that set values in
the MET
    config file (GridStatConfig_wrapped). Here is an example of what
is set in
    that file:

    fcst = {
       field = [ {
    name="/input/base/met_test/scripts/python/read_ascii_numpy.py
    /input/base/met_test/data/python/fcst.txt FCST"; } ];
    }

    The app point_stat does support Python embedding. However, there
are
    currently no METplus use cases that demonstrate running PointStat
in this
    way yet. The method to setting this up is similar to the approach
used for
    GridStat.

    Currently, the MET tools cannot return data in memory that can be
used in
    Python scripts. This has been discussed as a feature that will
eventually
    be added, but there are many details that need to be worked out
before it
    can be implemented. The tools write a NetCDF file to disk that
could be
    read into a Python script.

    Please let me know if you have any other questions.

    Thanks,
    George

    On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT
<met_help at ucar.edu>
    wrote:

    >
    > Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
    > Transaction: Ticket created by scott.rabenhorst at gmail.com
    >        Queue: met_help
    >      Subject: Questions about METplus python embedding
    >        Owner: Nobody
    >   Requestors: scott.rabenhorst at gmail.com
    >       Status: new
    >  Ticket <URL: https://urldefense.proofpoint.com/v2/url?u=https-
3A__rt.rap.ucar.edu_rt_Ticket_Display.html-3Fid-
3D97508&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=tbJm8XKW4diQF4bguVsOJyMXrEc6OpmHWoO2RDHPtSY&e=
>
    >
    >
    > Hi MET support,
    >
    > I have been reading through the METplus-3.1/MET-9.1
documentation
    > regarding python embedding. I see a list of examples here
    >
    > https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.org_community-2Dcode_model-2Devaluation-2Dtools-
2Dmet_sample-2Danalysis-
2Dscripts&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=keuM-
x1n0AMqs4LYe25jafy4kq5TiP7AyL2KgzLOvRA&e=
    > but they are all using the same plot_data_plane command. Do you
have
    > some examples of python embedding using other tools such as
grid_stat
    > and point_stat which handle multiple input files? Also, is there
a way
    > these tools can return the results in memory in the form of some
data
    > structures (e.g. pandas) with specific names? I might be a
little
    > ignorant on these questions since I am just learning the METplus
interface.
    >
    > Thanks,
    > Scott
    >
    >
    >

    --
    George McCabe - Software Engineer III
    National Center for Atmospheric Research
    Research Applications Laboratory
    303-497-2768
    ---
    My working day may not be your working day. Please do not feel
obliged to
    reply to this email outside of your normal working hours.




------------------------------------------------
Subject: Questions about METplus python embedding
From: George McCabe
Time: Wed Nov 18 12:18:02 2020

Hi Scott,

To clarify, it is possible to configure the METplus wrappers to
utilize
Python embedding with these tools. What is missing is an example
demonstrating this configuration.

- George

On Wed, Nov 18, 2020 at 12:12 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97508 >
>
> Hi George,
>
> Thanks for your prompt response. These links are very helpful - I
didn't
> see that beforehand. So it sounds like the grid/point_stat wrappers
for
> embedding are only in v4.0-beta1. I'll look forward to when some
results
> can live in python memory. For now I will try to familiarize myself
with
> the current capabilities. Thanks for your help.
>
> Thanks,
> Scott
>
>
>
> On 11/18/20, 1:33 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
>     Hi Scott,
>
>     The following link lists all of the existing METplus use cases
that
> utilize
>     Python embedding:
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_v3.1_search.html-3Fq-
3DPythonEmbeddingFileUseCase-26check-5Fkeywords-3Dyes-26area-
3Ddefault&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=wn0YbuwWDh93QlHeaq7SxTQJsCPN9Sm8xV7nDDSD3Ns&e=
>
>     There is a METplus use case that uses Python embedding with
GridStat
> that
>     is available in the 4.0-beta1 release of the METplus wrappers.
The
> release
>     is available here:
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_dtcenter_METplus_releases_tag_v4.0-
2Dbeta1&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=labgZpL3SlT-
MfOiOsQK4nbmy5JrfQVQjIudbND7R4A&e=
> . Please note
>     that this is a beta release and not an official stable release.
>
>     Information about this use case can be found here:
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.github.io_METplus_develop_generated_met-5Ftool-
5Fwrapper_GridStat_GridStat-5Fpython-5Fembedding.html-23metplus-
2Dconfiguration&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=oqCH6rwuwlSjbEI4qauLTuiuQ34h556Adn2XTOaL-
Y0&e=
>
>     Running this use case will generate a call to grid_stat that
looks
>     something like this:
>
>     grid_stat -v 2 PYTHON_NUMPY PYTHON_NUMPY
>     /my/path/METplus/parm/met_config/GridStatConfig_wrapped -outdir
>
/my/path/out/met_tool_wrapper/GridStat_python_embedding/2005080700
>
>     The METplus wrapper sets environment variables that set values
in the
> MET
>     config file (GridStatConfig_wrapped). Here is an example of what
is
> set in
>     that file:
>
>     fcst = {
>        field = [ {
>     name="/input/base/met_test/scripts/python/read_ascii_numpy.py
>     /input/base/met_test/data/python/fcst.txt FCST"; } ];
>     }
>
>     The app point_stat does support Python embedding. However, there
are
>     currently no METplus use cases that demonstrate running
PointStat in
> this
>     way yet. The method to setting this up is similar to the
approach used
> for
>     GridStat.
>
>     Currently, the MET tools cannot return data in memory that can
be used
> in
>     Python scripts. This has been discussed as a feature that will
> eventually
>     be added, but there are many details that need to be worked out
before
> it
>     can be implemented. The tools write a NetCDF file to disk that
could be
>     read into a Python script.
>
>     Please let me know if you have any other questions.
>
>     Thanks,
>     George
>
>     On Wed, Nov 18, 2020 at 9:27 AM Scott Rabenhorst via RT <
> met_help at ucar.edu>
>     wrote:
>
>     >
>     > Wed Nov 18 09:27:12 2020: Request 97508 was acted upon.
>     > Transaction: Ticket created by scott.rabenhorst at gmail.com
>     >        Queue: met_help
>     >      Subject: Questions about METplus python embedding
>     >        Owner: Nobody
>     >   Requestors: scott.rabenhorst at gmail.com
>     >       Status: new
>     >  Ticket <URL:
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__rt.rap.ucar.edu_rt_Ticket_Display.html-3Fid-
3D97508&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=tbJm8XKW4diQF4bguVsOJyMXrEc6OpmHWoO2RDHPtSY&e=
> >
>     >
>     >
>     > Hi MET support,
>     >
>     > I have been reading through the METplus-3.1/MET-9.1
documentation
>     > regarding python embedding. I see a list of examples here
>     >
>     >
> https://urldefense.proofpoint.com/v2/url?u=https-
3A__dtcenter.org_community-2Dcode_model-2Devaluation-2Dtools-
2Dmet_sample-2Danalysis-
2Dscripts&d=DwIDaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=ZkPjbDkprOw2gq9vZ6HyXOFfdueZFMiKMGetb2nDmVk&m=0JSNuCIEtOsRjVCi9jjA712aA_7g1SypgQrzqTGGCpw&s=keuM-
x1n0AMqs4LYe25jafy4kq5TiP7AyL2KgzLOvRA&e=
>     > but they are all using the same plot_data_plane command. Do
you have
>     > some examples of python embedding using other tools such as
grid_stat
>     > and point_stat which handle multiple input files? Also, is
there a
> way
>     > these tools can return the results in memory in the form of
some data
>     > structures (e.g. pandas) with specific names? I might be a
little
>     > ignorant on these questions since I am just learning the
METplus
> interface.
>     >
>     > Thanks,
>     > Scott
>     >
>     >
>     >
>
>     --
>     George McCabe - Software Engineer III
>     National Center for Atmospheric Research
>     Research Applications Laboratory
>     303-497-2768
>     ---
>     My working day may not be your working day. Please do not feel
obliged
> to
>     reply to this email outside of your normal working hours.
>
>
>
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------


More information about the Met_help mailing list