[ncl-talk] HDF5 reading issues

[David Brown]

Hi Alex,

Just a note to let you know that the string issue for HDF5 has been
fixed in the latest development version of NCL. Your "string_test.ncl"
script now prints the correct results:

ncl string_test.ncl
....

Variable: date_time
Type: string
Total Size: 8 bytes
            1 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_000 | 1]
Coordinates:
(0) 2015-08-26T16-27-44.858999

On Wed, Jun 29, 2016 at 12:35 PM, David Brown <dbrown at ucar.edu> wrote:
> Hi Alex,
> Work has been ongoing on the HDF file I/O code since 6.3.0 was
> released. I can report that the read_var_test.ncl now works as
> expected using the development version, but there do still seem to be
> issues with reading the string variable in the file. That is a bit of
> a surprise since we have spent some time improving the string
> handling. Apparently there is still work to be done. So thanks for
> providing this file. It will be a good test case.
>
> Here is the output of your read_var_test.ncl with the dev version of NCL:
>
> Variable: nlats
> Type: integer
> Total Size: 4 bytes
>             1 values
> Number of Dimensions: 1
> Dimensions and sizes: [1]
> Coordinates:
>
> (0) 56476
> (0) The number of non-zero lats: 56476
> (0) The number of non-zero lons: 56476
>
> Variable: backscatter_dims
> Type: integer
> Total Size: 8 bytes
>             2 values
> Number of Dimensions: 1
> Dimensions and sizes: [2]
> Coordinates:
> (0) 56476
> (1) 533
> (0) 30101708
>
> string_test.ncl produces a different set of error messages now from
> what showed up in 6.3.0, but there are still issues. Regarding a
> string being a scalar in HDF5, that is actually to be expected. (A
> single string is considered to be a scalar in NCL as well.) Here is
> what h5dump says about this variable:
>
> h5dump -d /metadata_parameters/Granule_Start_DateTime
> CATS-ISS_L1B_D-M7.2-V2-07.2015-08-26T16-27-44T17-14-51UTC.hdf5
> HDF5 "CATS-ISS_L1B_D-M7.2-V2-07.2015-08-26T16-27-44T17-14-51UTC.hdf5" {
> DATASET "/metadata_parameters/Granule_Start_DateTime" {
>    DATATYPE  H5T_STRING {
>       STRSIZE 27;
>       STRPAD H5T_STR_NULLTERM;
>       CSET H5T_CSET_ASCII;
>       CTYPE H5T_C_S1;
>    }
>    DATASPACE  SCALAR
>    DATA {
>    (0): "2015-08-26T16-27-44.858999"
>    }
>  -dave
>
>
>
> On Wed, Jun 29, 2016 at 8:16 AM, Alexander Schaefer <aschae11 at uncc.edu> wrote:
>> Hi All-
>>
>> I am working on reading some LIDAR data and running into some issues with
>> how NCL is handling some of the commands.
>>
>> The first is with regards to strings.  I try to read a string variable and
>> it spits out an HDF error.  I know the variable is a single string that is
>> supposed to be read in; this was checked with hdfview.  Google and the HDF
>> forums were of no help.
>>
>> HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
>>
>>   #000: H5Shyper.c line 6595 in H5Sselect_hyperslab(): hyperslab doesn't
>> support H5S_SCALAR space
>>
>>     major: Invalid arguments to routine
>>
>>     minor: Inappropriate type
>>
>> HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
>>
>>   #000: H5Shyper.c line 6595 in H5Sselect_hyperslab(): hyperslab doesn't
>> support H5S_SCALAR space
>>
>>     major: Invalid arguments to routine
>>
>>     minor: Inappropriate type
>>
>>
>> Why is NCL trying to read a string as a scalar?  Any ideas on whether this
>> is a library issue or an NCL coding issue?
>>
>> My second question is with respect to how NCL is reading in float variables.
>> I am reading in a variable that I wouldn’t expect NCL to struggle with but
>> it is.  The variable being read should be about 57K x 533 point.  It reads
>> in with no errors returned but checking the dimensions tells me that NCL has
>> made it a ~57k x ~57k variable.  I thought maybe NCL was assuming a lat x
>> lon grid; checking the data in the returned array gives me all 0s but
>> printing the lats and lons I know there is data. Any ideas on what is going
>> on?  Does the square array have to do with the hyperslab routines?
>>
>>
>> I am running NCL 6.3.0 on OS X 10.11.5.
>>
>> I uploaded to the ftp the file for the tests:
>> CATS-ISS_L1B_D-M7.2-V2-07.2015-08-26T16-27-44T17-14-51UTC.hdf5
>>
>> Attached are the tests that produce the string error and the variable issue.
>>
>> Thanks for any insight,
>> -Alex
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>