[Met_help] [rt.rap.ucar.edu #95615] History for python embedding
George McCabe via RT
met_help at ucar.edu
Tue Jun 16 10:06:24 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi John,
I'm starting to work with MET version 9. Python embedding for point obs
will help us a lot!
I just want to make sure I understand correctly. I see that the point obs
need to be fed into the ascii2nc routine. We are trying to work with h5
data and have python routines to read the h5 files - can we repurpose those
routines to read the h5 directly and pass to ascii2nc? E.g. we just need to
read in data (any format) and store it in a Pandas data frame to be fed into
ascii2nc?
Thanks,
Liz
Elizabeth Satterfield
Editor, Monthly Weather Review
Chair, AMS Committee on Probability and Statistics
Meteorologist, Code 7531
Marine Meteorology Division
U.S. Naval Research Laboratory
T 831.656.4056 F 831.656.4026
<http://www.nrl.navy.mil/> www.nrl.navy.mil
__________________________________________________________________
FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
Any misuse or unauthorized disclosure may result in both civil and criminal
penalties.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: python embedding
From: Minna Win
Time: Mon Jun 15 14:28:58 2020
Hello,
It looks like you have questions on Python embedding. I've assigned
this
ticket to John Halley Gotway, who is currently out of the office.
Please
allow a business week for a complete response.
Thank you for your patience.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Mon, Jun 15, 2020 at 1:46 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 15 13:45:33 2020: Request 95615 was acted upon.
> Transaction: Ticket created by Elizabeth.Satterfield at nrlmry.navy.mil
> Queue: met_help
> Subject: python embedding
> Owner: Nobody
> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95615 >
>
>
> Hi John,
>
>
>
> I'm starting to work with MET version 9. Python embedding for point
obs
> will help us a lot!
>
>
>
> I just want to make sure I understand correctly. I see that the
point obs
> need to be fed into the ascii2nc routine. We are trying to work
with h5
> data and have python routines to read the h5 files - can we
repurpose those
> routines to read the h5 directly and pass to ascii2nc? E.g. we just
need
> to
> read in data (any format) and store it in a Pandas data frame to be
fed
> into
> ascii2nc?
>
>
>
> Thanks,
>
> Liz
>
>
>
>
>
>
>
>
>
>
> Elizabeth Satterfield
> Editor, Monthly Weather Review
>
> Chair, AMS Committee on Probability and Statistics
>
> Meteorologist, Code 7531
> Marine Meteorology Division
> U.S. Naval Research Laboratory
> T 831.656.4056 F 831.656.4026
> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>
> __________________________________________________________________
> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
> Any misuse or unauthorized disclosure may result in both civil and
criminal
> penalties.
>
>
>
>
>
>
>
>
>
------------------------------------------------
Subject: python embedding
From: George McCabe
Time: Tue Jun 16 07:03:37 2020
Hi Liz,
I hope you are doing well! Yes, you are understanding correctly. You
can
utilize your existing python script that reads the H5 data and put it
into
a pandas dataframe to be read by ASCII2NC. You need to make sure the
data
matches the 11 column format that ASCII2NC expects.
# (1) string: Message_Type
# (2) string: Station_ID
# (3) string: Valid_Time(YYYYMMDD_HHMMSS)
# (4) numeric: Lat(Deg North)
# (5) numeric: Lon(Deg East)
# (6) numeric: Elevation(msl)
# (7) string: Var_Name(or GRIB_Code)
# (8) numeric: Level
# (9) numeric: Height(msl or agl)
# (10) string: QC_String
# (11) numeric: Observation_Value
The python script must name the variable that holds the pandas data
'point_data' and the app knows to pull data from that variable. There
is an
example script that does this in the MET installation area here:
share/met/python/read_ascii_point.py
The script reads an ASCII file and puts the data into pandas. Here is
an
example call to ASCII2NC utilizing python embedding that was generated
from
one of the METplus wrapper use cases:
/usr/local/met-9.0.1/bin/ascii2nc
"/usr/local/met-9.0.1/share/met/python/read_ascii_point.py
/d1/projects/METplus/METplus_Data/met_test/data/sample_obs/ascii/sample_ascii_obs.txt"
/d1/personal/mccabe/out/met_tool_wrapper/ASCII2NC/ascii2nc_python.nc
-format python -v 2
The 1st argument is the python script to call with any command line
arguments following it (within the quotes). The 2nd argument is the
output
filename. You will also need to set the '-format' flag to 'python.'
Please
let me know if you have any questions or issues getting this set up
and I
would be happy to help you work out any kinks.
Thanks,
George
On Mon, Jun 15, 2020 at 1:46 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 15 13:45:33 2020: Request 95615 was acted upon.
> Transaction: Ticket created by Elizabeth.Satterfield at nrlmry.navy.mil
> Queue: met_help
> Subject: python embedding
> Owner: Nobody
> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95615 >
>
>
> Hi John,
>
>
>
> I'm starting to work with MET version 9. Python embedding for point
obs
> will help us a lot!
>
>
>
> I just want to make sure I understand correctly. I see that the
point obs
> need to be fed into the ascii2nc routine. We are trying to work
with h5
> data and have python routines to read the h5 files - can we
repurpose those
> routines to read the h5 directly and pass to ascii2nc? E.g. we just
need
> to
> read in data (any format) and store it in a Pandas data frame to be
fed
> into
> ascii2nc?
>
>
>
> Thanks,
>
> Liz
>
>
>
>
>
>
>
>
>
>
> Elizabeth Satterfield
> Editor, Monthly Weather Review
>
> Chair, AMS Committee on Probability and Statistics
>
> Meteorologist, Code 7531
> Marine Meteorology Division
> U.S. Naval Research Laboratory
> T 831.656.4056 F 831.656.4026
> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>
> __________________________________________________________________
> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
> Any misuse or unauthorized disclosure may result in both civil and
criminal
> penalties.
>
>
>
>
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: python embedding
From: Satterfield, Dr. Elizabeth
Time: Tue Jun 16 10:02:04 2020
Thanks, George. This is very helpful.
Hope you can make it back to Monterey again soon!
Best,
Liz
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: Tuesday, June 16, 2020 6:04 AM
To: Satterfield, Dr. Elizabeth <Elizabeth.Satterfield at nrlmry.navy.mil>
Cc: Campbell, Dr. Bill <Bill.Campbell at nrlmry.navy.mil>; Tsu, Mr.
Justin
<justin.tsu at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95615] python embedding
Hi Liz,
I hope you are doing well! Yes, you are understanding correctly. You
can
utilize your existing python script that reads the H5 data and put it
into a
pandas dataframe to be read by ASCII2NC. You need to make sure the
data
matches the 11 column format that ASCII2NC expects.
# (1) string: Message_Type
# (2) string: Station_ID
# (3) string: Valid_Time(YYYYMMDD_HHMMSS)
# (4) numeric: Lat(Deg North)
# (5) numeric: Lon(Deg East)
# (6) numeric: Elevation(msl)
# (7) string: Var_Name(or GRIB_Code)
# (8) numeric: Level
# (9) numeric: Height(msl or agl)
# (10) string: QC_String
# (11) numeric: Observation_Value
The python script must name the variable that holds the pandas data
'point_data' and the app knows to pull data from that variable. There
is an
example script that does this in the MET installation area here:
share/met/python/read_ascii_point.py
The script reads an ASCII file and puts the data into pandas. Here is
an
example call to ASCII2NC utilizing python embedding that was generated
from
one of the METplus wrapper use cases:
/usr/local/met-9.0.1/bin/ascii2nc
"/usr/local/met-9.0.1/share/met/python/read_ascii_point.py
/d1/projects/METplus/METplus_Data/met_test/data/sample_obs/ascii/sample_ascii_obs.txt"
/d1/personal/mccabe/out/met_tool_wrapper/ASCII2NC/ascii2nc_python.nc
-format python -v 2
The 1st argument is the python script to call with any command line
arguments
following it (within the quotes). The 2nd argument is the output
filename. You
will also need to set the '-format' flag to 'python.' Please let me
know if
you have any questions or issues getting this set up and I would be
happy to
help you work out any kinks.
Thanks,
George
On Mon, Jun 15, 2020 at 1:46 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 15 13:45:33 2020: Request 95615 was acted upon.
> Transaction: Ticket created by Elizabeth.Satterfield at nrlmry.navy.mil
> Queue: met_help
> Subject: python embedding
> Owner: Nobody
> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95615
> >
>
>
> Hi John,
>
>
>
> I'm starting to work with MET version 9. Python embedding for point
> obs will help us a lot!
>
>
>
> I just want to make sure I understand correctly. I see that the
point
> obs need to be fed into the ascii2nc routine. We are trying to work
> with h5 data and have python routines to read the h5 files - can we
> repurpose those routines to read the h5 directly and pass to
ascii2nc?
> E.g. we just need to read in data (any format) and store it in a
> Pandas data frame to be fed into ascii2nc?
>
>
>
> Thanks,
>
> Liz
>
>
>
>
>
>
>
>
>
>
> Elizabeth Satterfield
> Editor, Monthly Weather Review
>
> Chair, AMS Committee on Probability and Statistics
>
> Meteorologist, Code 7531
> Marine Meteorology Division
> U.S. Naval Research Laboratory
> T 831.656.4056 F 831.656.4026
> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>
> __________________________________________________________________
> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE Any misuse or unauthorized
> disclosure may result in both civil and criminal penalties.
>
>
>
>
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: python embedding
From: George McCabe
Time: Tue Jun 16 10:06:22 2020
I really hope so, too! I am going to close this ticket, but feel free
to
open a new one if you run into issues getting this set up.
- George
On Tue, Jun 16, 2020 at 10:02 AM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95615 >
>
> Thanks, George. This is very helpful.
>
> Hope you can make it back to Monterey again soon!
>
> Best,
> Liz
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: Tuesday, June 16, 2020 6:04 AM
> To: Satterfield, Dr. Elizabeth
<Elizabeth.Satterfield at nrlmry.navy.mil>
> Cc: Campbell, Dr. Bill <Bill.Campbell at nrlmry.navy.mil>; Tsu, Mr.
Justin
> <justin.tsu at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95615] python embedding
>
> Hi Liz,
>
> I hope you are doing well! Yes, you are understanding correctly. You
can
> utilize your existing python script that reads the H5 data and put
it into
> a
> pandas dataframe to be read by ASCII2NC. You need to make sure the
data
> matches the 11 column format that ASCII2NC expects.
>
> # (1) string: Message_Type
> # (2) string: Station_ID
> # (3) string: Valid_Time(YYYYMMDD_HHMMSS)
> # (4) numeric: Lat(Deg North)
> # (5) numeric: Lon(Deg East)
> # (6) numeric: Elevation(msl)
> # (7) string: Var_Name(or GRIB_Code)
> # (8) numeric: Level
> # (9) numeric: Height(msl or agl)
> # (10) string: QC_String
> # (11) numeric: Observation_Value
>
> The python script must name the variable that holds the pandas data
> 'point_data' and the app knows to pull data from that variable.
There is
> an
> example script that does this in the MET installation area here:
>
> share/met/python/read_ascii_point.py
>
> The script reads an ASCII file and puts the data into pandas. Here
is an
> example call to ASCII2NC utilizing python embedding that was
generated
> from
> one of the METplus wrapper use cases:
>
> /usr/local/met-9.0.1/bin/ascii2nc
> "/usr/local/met-9.0.1/share/met/python/read_ascii_point.py
>
>
/d1/projects/METplus/METplus_Data/met_test/data/sample_obs/ascii/sample_ascii_obs.txt"
> /d1/personal/mccabe/out/met_tool_wrapper/ASCII2NC/ascii2nc_python.nc
> -format python -v 2
>
> The 1st argument is the python script to call with any command line
> arguments
> following it (within the quotes). The 2nd argument is the output
filename.
> You
> will also need to set the '-format' flag to 'python.' Please let me
know
> if
> you have any questions or issues getting this set up and I would be
happy
> to
> help you work out any kinks.
>
> Thanks,
> George
>
> On Mon, Jun 15, 2020 at 1:46 PM Satterfield, Dr. Elizabeth via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jun 15 13:45:33 2020: Request 95615 was acted upon.
> > Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
> > Queue: met_help
> > Subject: python embedding
> > Owner: Nobody
> > Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95615
> > >
> >
> >
> > Hi John,
> >
> >
> >
> > I'm starting to work with MET version 9. Python embedding for
point
> > obs will help us a lot!
> >
> >
> >
> > I just want to make sure I understand correctly. I see that the
point
> > obs need to be fed into the ascii2nc routine. We are trying to
work
> > with h5 data and have python routines to read the h5 files - can
we
> > repurpose those routines to read the h5 directly and pass to
ascii2nc?
> > E.g. we just need to read in data (any format) and store it in a
> > Pandas data frame to be fed into ascii2nc?
> >
> >
> >
> > Thanks,
> >
> > Liz
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Elizabeth Satterfield
> > Editor, Monthly Weather Review
> >
> > Chair, AMS Committee on Probability and Statistics
> >
> > Meteorologist, Code 7531
> > Marine Meteorology Division
> > U.S. Naval Research Laboratory
> > T 831.656.4056 F 831.656.4026
> > <http://www.nrl.navy.mil/> www.nrl.navy.mil
> >
> > __________________________________________________________________
> > FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE Any misuse or
unauthorized
> > disclosure may result in both civil and criminal penalties.
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list