[ncl-talk] Unable to read JPEG-packed GRIB2 data with NCL

[Mike Kay]

Hi Rick -

The reason I mentioned GDAL is that the path (/vsimem/...) appears to
be a GDAL virtual filesystem path.

I ran the code through ltrace to see what else I could learn about the
calls being made and here we go:

<snip>
[pid 13820] fopen64("2mtemp-jpeg.grib2", "r")    = 0x7425540
[pid 13820] getpagesize()                        = 4096
[pid 13820] malloc(16384)                        = 0x743b920
[pid 13820] getpagesize()                        = 4096
[pid 13820] setvbuf(0x7425540, 0x743b920, 0, 16384) = 0
[pid 13820] malloc(32)                           = 0x6f614b0
[pid 13820] malloc(4152960)                      = 0x7fd4196f2010
[pid 13820] malloc(416)                          = 0x6f31690
[pid 13820] malloc(848604)                       = 0x743f930
[pid 13820] fseek(0x7425540, 0, 0, 0x743f930)    = 0
[pid 13820] fread(0x743f930, 1, 848604, 0x7425540) = 848604
[pid 13820] g2_getfld(0x743f930, 1, 1, 1ERROR 4:
`/vsimem/work_grib_0x743f9e1.jpc' not recognized as a supported file
format.
dec_jpeg2000: Unable to open JPEG2000 image within GRIB file.
Is the JPEG2000 driver available?)        = 0
[pid 13820] malloc(4)                            = 0x6a97930
[pid 13820] free(0x743f930)                      = <void>
[pid 13820] g2_free(0x702e6c0, 4, 1, 1)          = 193
</snip>

(Sorry for the jumble on the call in question; everything's written to
stderr there)

So, it's thankfully calling g2_getfld but something else is getting
clobbered along the way.

An additional data point - a co-worker just tested the NCAR conda
packages on an Ubuntu release (16.04) and experienced the same
inability to read the GRIB2 file.

thanks,
-mike

Mike Kay

Global Validation Model

Spire Global, Inc.

1050 Walnut Street, Suite 402, Boulder, CO 80302 USA

+1-303-562-9585


On Tue, Oct 17, 2017 at 9:55 AM, Rick Brownrigg <brownrig at ucar.edu> wrote:
> Hi Mike,
>
> Something odd is going on here.  I was able to read that file just fine.
> The error you are getting seems to be unrelated to the file you are trying
> to open, referencing a completely different filename (?)   And I'm not
> certain how gdal might be involved in this?
>
> Rick
>
> On Tue, Oct 17, 2017 at 9:43 AM, Mike Kay <mike.kay at spire.com> wrote:
>>
>> Hi -
>>
>> We are unable to read grids from GRIB2 files using JPEG 2000 packing
>> using NCL (6.4.0). These data files are readable via a number of other
>> libraries/tools (PyNIO, ecCodes, NetCDF-Java-based tools (e.g.,
>> THREDDS, PanoplyJ)).
>>
>> The NCL reads always seem to fail within GDAL:
>>
>> Example:
>> ncl 0> nc = addfile("2mtemp-jpeg.grib2","r")
>> ncl 1> data = nc->TMP_P0_L103_GLL0
>> ERROR 4: `/vsimem/work_grib_0x8d2f761.jpc' not recognized as a
>> supported file format.
>> dec_jpeg2000: Unable to open JPEG2000 image within GRIB file.
>>
>> This was tested on several machines (RHEL-like OSs (CentOS 7.3,
>> Scientific Linux 7.2)) with identical results using a conda-installed
>> ncl via either:
>>
>> conda create -n ncl_stable -c conda-forge ncl=6.4.0
>>
>> or
>>
>> conda create -n ncl_stable -c NCAR -c conda-forge -c defaults ncl
>>
>> We tested gdal by itself by creating a conda environment using the
>> same libgdal as installed in the ncl environment (2.2.2) and running a
>> simple gdal_translate on the data:
>>
>> > gdal_translate 2mtemp-jpeg.grib2 2mtemp.tiff
>>
>> gdal_translate successfully reads the grid and creates a GeoTIFF.
>>
>> I have placed a sample file (2mtemp-jpeg.grib2) in the incoming dir of
>> the ftp site for you to test against.
>>
>> Please let me know if we're missing something obvious here.
>>
>> thanks a lot,
>> -mike
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>