[pyngl-talk] how to use the new MRMS grib codetable

Root, Benjamin broot at aer.com
Thu Jul 2 15:45:15 MDT 2015


Understood. I would be more than happy with PyNIO being able to read in an un-gzipped version of the MRMS data file (I was planning on doing that at the LDM level, just strip them all of the gzip containers. The space savings is negligible anyway.

By the way, if anybody will be at the SciPy 2015 conference next week, we can certainly chat up about PyNIO and anything else.

Cheers!
Ben Root
________________________________________
From: David Brown [dbrown at ucar.edu]
Sent: Thursday, July 02, 2015 5:29 PM
To: Root, Benjamin
Cc: pyngl-talk at ucar.edu; Nick Guy
Subject: Re: [pyngl-talk] how to use the new MRMS grib codetable

OK, this is a pretty different issue, but I would definitely like to
be able to handle file-like objects as you propose. We will definitely
put this on the to-do list.
 -dave


On Thu, Jul 2, 2015 at 3:19 PM, Root, Benjamin <broot at aer.com> wrote:
> That's great to hear! I would just like to point out one particular
> pitfall that is tripping me up. The MRMS data from NCEP's LDM feed are
> all gzipped (yeah, I know, craziness...). Both PyNIO and pygrib expect a
> filepath object to a grib2 file as opposed to a file-like object. I have
> dug down into PyNIO's Grib2 reader, and I see that this requirement
> seems to be fairly artificial, as the relevant C-code just simply opens
> up a standard C file object.
>
> matplotlib has code that can produce a C file object from a Python
> file-like object that you guys can use. We use it to interface with
> libfreetype. It originally came from Numpy so that that library could
> interface with datafile readers that expect a FILE pointer.
>
> https://github.com/matplotlib/matplotlib/blob/master/src/file_compat.h
>
> I don't know yet if this would work for file-like objects such as
> objects from a TarFile or GzipFile, but it might be a step forward. This
> code works on Windows, Linux, and Macs, and is py3k compatible.
>
> Cheers!
> Ben Root
>
> On 07/02/2015 03:51 PM, David Brown wrote:
>> Hi Ben,
>> Coincidentally, there is someone also asking about MRMS data support
>> for NCL. I am not sure at this point whether it is just a matter of
>> creating the right tables or whether some source code changes will be
>> needed as well. But given the amount of interest I will look into this
>> now.
>>  -dave
>>
>> On Thu, Jul 2, 2015 at 1:25 PM, Root, Benjamin <broot at aer.com> wrote:
>>> ...unless one is having difficulties compiling the new wgrib2 package
>>> (grrr...).
>>>
>>> I guess what I am trying to get at is (1) is there a utility that can
>>> help convert a wgrib2 codetable into a form recognizable by the ecmwf
>>> grib api and (2) how does one get custom codetables recognized by PyNIO?
>>>
>>> Ben Root
>>>
>>> On 07/02/2015 03:12 PM, David Brown wrote:
>>>> Hi Ben and Nick,
>>>> Thanks Nick for the pointer to MMM-Py. From there I can get to
>>>> information about the MRMS data, plus some sample data sets.
>>>> The key point, relative to PyNIO from this documentation is the following:
>>>>
>>>> "The GRIB2 IDs used by MRMS are not officially sanctioned at this
>>>> time.  Instead, MRMS uses a local discipline number (209) and it’s own
>>>> set of category and parameter values.  This means that most widely
>>>> available GRIB2 decoders will require customization."
>>>>
>>>> MMM-Py relies on a customized version of wgrib2 to do the actual
>>>> reading. To make this data readable by PyNIO it will probably be
>>>> easiest to work from the modified wgrib2 code. This may be pretty
>>>> simple, but it will take a bit of research.
>>>>
>>>> So for now, I would agree that MMM-Py is probably the best bet.
>>>>  -dave
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jul 2, 2015 at 12:14 PM, Nick Guy <Nick.Guy at uwyo.edu> wrote:
>>>>> Ben,
>>>>>
>>>>> Not sure what your use is, but this package works to read in MRMS and is
>>>>> pretty nice to work with:
>>>>> https://github.com/nasa/MMM-Py
>>>>>
>>>>> Nick
>>>>>
>>>>> On Jul 2, 2015, at 12:00 PM, pyngl-talk-request at ucar.edu wrote:
>>>>>
>>>>> Send pyngl-talk mailing list submissions to
>>>>> pyngl-talk at ucar.edu
>>>>>
>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>>>>> or, via email, send a message with subject or body 'help' to
>>>>> pyngl-talk-request at ucar.edu
>>>>>
>>>>> You can reach the person managing the list at
>>>>> pyngl-talk-owner at ucar.edu
>>>>>
>>>>> When replying, please edit your Subject line so it is more specific
>>>>> than "Re: Contents of pyngl-talk digest..."
>>>>>
>>>>>
>>>>> Today's Topics:
>>>>>
>>>>>   1. how to use the new MRMS grib codetable (Root, Benjamin)
>>>>>
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Message: 1
>>>>> Date: Thu, 2 Jul 2015 16:43:27 +0000
>>>>> From: "Root, Benjamin" <broot at aer.com>
>>>>> Subject: [pyngl-talk] how to use the new MRMS grib codetable
>>>>> To: "pyngl-talk at ucar.edu" <pyngl-talk at ucar.edu>
>>>>> Message-ID:
>>>>> <47A91F4E74556440877A9E87A31CE9D20952D420 at VRSKUTMB2.veriskdom.com>
>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>
>>>>> I am working with the (somewhat) new MRMS datafeed from NCEP. I managed
>>>>> to find a codetable for it in the wgrib2 source code:
>>>>> http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/user_tables/MRMS_usertable,
>>>>> but I am stuck in trying to get PyNIO to utilize it. I am not even sure
>>>>> if it is in the right format for use in PyNIO.
>>>>>
>>>>> Any tips would be appreciated.
>>>>>
>>>>> Cheers!
>>>>> Ben Root
>>>>>
>>>>> ________________________________
>>>>>
>>>>> This email is intended solely for the recipient. It may contain privileged,
>>>>> proprietary or confidential information or material. If you are not the
>>>>> intended recipient, please delete this email and any attachments and notify
>>>>> the sender of the error.
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> pyngl-talk mailing list
>>>>> pyngl-talk at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>>>>>
>>>>>
>>>>> End of pyngl-talk Digest, Vol 125, Issue 2
>>>>> ******************************************
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> pyngl-talk mailing list
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>>>>>
>>>
>>> ________________________________
>>>
>>> This email is intended solely for the recipient. It may contain privileged, proprietary or confidential information or material. If you are not the intended recipient, please delete this email and any attachments and notify the sender of the error.
>>>
>
>



More information about the pyngl-talk mailing list