[ncl-talk] h5 dataset
Dennis Shea
shea at ucar.edu
Fri Apr 1 12:03:38 MDT 2016
Are you sure your h5 file was not truncated?
Perhaps, when you ftp'd it to NCAR or the original download. The file in
ftp has the following size:
ftp 420623450 Mar 31 23:46 3DIMG_23APR2015_1200_LIB_STD.h5
If your file is that size, then likely the original download was corrupted.
Please check
==
AN NCL developer did a quick look and commented internally:
Unfortunately, the error arises deep within the hdf5 library itself,
trying to traverse the variable "/Longitude_VIS". There are 16-odd (even?)
utility programs that come with the library, and several of those fail
outright with little more than "h5dump error: internal error (file
h5dump.c:line 1471)" or "h5stat warning: Unable to traverse objects/links
in file "3DIMG_23APR2015_1200_LIB_STD.h5"
Other utilities are able to print aspects of the file, such as the list of
variables. /Longitude_VIS is not one listed, and yet I can see it come up
at the point of failure while tracing the code.
Finally, I strongly suspect this file has been truncated, although I can't
prove that. *All* the other .h5 files in that /glade directory conclude
with what looks like metadata; this file concludes abruptly with what
looks like actual data. That's not necessarily definitive, but...
[snip]
On Thu, Mar 31, 2016 at 8:21 AM, Dennis Shea <shea at ucar.edu> wrote:
> The error message says the file does not exist.
> f = addfile("./3DIMG_23APR2015_1200_L1B_STD.h5", "r")
> Are you sure that the file is in the current directory?
>
> ===
> There have been *significant* upgrades tp NCL's H5 capabilities since
> 6.3.0.
>
> They will be in the (as yet, unreleased 6.4.0).
>
> ===
> Please make the file available. You can ftp the file via
>
> ftp ftp.cgd.ucar.edu
> anonymous
> email
> cd incoming
> put 3DIMG_23APR2015_1200_L1B_STD.h5
> quit
>
> Then, after a successful transfer, send ncl-talk a message
>
>
> On Thu, Mar 31, 2016 at 1:40 AM, Geeta Geeta <geetag54 at yahoo.com> wrote:
>
>> Hi all.
>> I am trying to read a satellite data file which has h5 extension. when I
>> do ncl_filesdump, I am able to see the variables and other details of the
>> file but addfile function is not able to read it.
>> Copyright (C) 1995-2015 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.3.0
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>>
>> Variable: f
>> Type: file
>> filename: 3DIMG_23APR2015_1200_LIB_STD
>> path: 3DIMG_23APR2015_1200_LIB_STD.h5
>> file global attributes:
>> Acquisition_Date : 23APR2015
>> Acquisition_End_Time : 23-APR-2015T12:26:52
>> Acquisition_Start_Time : 23-APR-2015T12:00:02
>> Acquisition_Time_in_GMT : 1200PR-2015T12:00:02
>> Attitude_Source : STARPR-2015T12:00:02
>> Datum : WGS84R-2015T12:00:02
>> Ellipsoid : WGS84R-2015T12:00:02
>> FastScan_Linearity_Enabled : noS84R-2015T12:00:02
>> Field_of_View(degrees) : 17.973925
>> Ground_Station : IMD,New Delhi,INDIA.
>> HDF_Product_File_Name : 3DIMG_23APR2015_1200_L1B_STD.h5
>> Imaging_Mode : FULL FRAMER2015_1200_L1B_STD.h5
>> MCD_FS_Enabled : noLL FRAMER2015_1200_L1B_STD.h5
>> MCD_SS_Enabled : noLL FRAMER2015_1200_L1B_STD.h5
>> MIR_Acquisition_Mode : MAIN FRAMER2015_1200_L1B_STD.h5
>> MIR_Gain_Mode : 2
>> Nominal_Altitude(km) : 36000
>> Nominal_Central_Point_Coordinates(degrees)_Latitude_Longitude :
>> ( 0, 82 )
>> Observed_Altitude(km) : 35786.819452
>> Output_Format : hdf5-1.8.8R2015_1200_L1B_STD.h5
>> Processing_Level : L1B5-1.8.8R2015_1200_L1B_STD.h5
>> Product_Creation_Time : 2015-04-23T17:44:300_L1B_STD.h5
>> Product_Type : STANDARD(FULL DISK)0_L1B_STD.h5
>> Radiometric_Calibration_Type : LAB CALIBRATEDDISK)0_L1B_STD.h5
>> SWIR_Acquisition_Mode : MAINCALIBRATEDDISK)0_L1B_STD.h5
>> SWIR_Gain_Mode : 3
>> Sat_Azimuth(Degrees) : 290.400818
>> Sat_Elevation(Degrees) : 89.8
>> Sensor_Id : IMGAT-3DBRATEDDISK)0_L1B_STD.h5
>> Sensor_Name : IMAGER3DBRATEDDISK)0_L1B_STD.h5
>> SlowScan_Linearity_Enabled : noAGER3DBRATEDDISK)0_L1B_STD.h5
>> Software_Version : 1.0GER3DBRATEDDISK)0_L1B_STD.h5
>> Station_Id : IMDGER3DBRATEDDISK)0_L1B_STD.h5
>> Sun_Azimuth(Degrees) : 282.568512
>> Sun_Elevation(Degrees) : 4.370282
>> TIR1_Acquisition_Mode : MAINER3DBRATEDDISK)0_L1B_STD.h5
>> TIR1_Gain_Mode : 2
>> TIR2_Acquisition_Mode : MAINER3DBRATEDDISK)0_L1B_STD.h5
>> TIR2_Gain_Mode : 2
>> Unique_Id : 3DIMG_23APR2015_1200_L1B_STD.h5
>> VIS_Acquisition_Mode : MAING_23APR2015_1200_L1B_STD.h5
>> VIS_Gain_Mode : 2
>> WV_Acquisition_Mode : MAING_23APR2015_1200_L1B_STD.h5
>> WV_Gain_Mode : 2
>> Yaw_Flip_Flag : YAING_23APR2015_1200_L1B_STD.h5
>> conventions : CF-1.623APR2015_1200_L1B_STD.h5
>> institute : IMD,New Delhi,INDIA._L1B_STD.h5
>> left_longitude : 0.8432964
>> lower_latitude : -81.04153
>> right_longitude : 163.1567
>> source : IMAGER (IMG)i,INDIA._L1B_STD.h5
>> title : 3DIMG_23APR2015_1200_L1B_STD.h5
>> upper_latitude : 81.04153
>> dimensions:
>> = 2816
>> time = 1
>> Latitude_WV = 1408
>> Longitude_WVISmperature = 1402
>> Latitude_VIS = 11264
>> Longitude_VISmperature = 11220
>> Latitude = 2816
>> Longitudetness = 2805
>> Temperature = 0
>> Longitudeance_VISerature = 2805
>> Longitude_VISerature = 11220
>> Longitude = 2805
>> DIM_012 = 0
>> --More--
>> DIM_013 = 0
>> DIM_014 = 0
>> DIM_015 = 0
>> DIM_016 = 0
>> DIM_017 = 0
>> DIM_018 = 0
>> variables:
>> integer GeoX ( Longitude )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GeoX1 ( Longitude_WVISmperature )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GeoX2 ( Longitude_VISerature )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GeoY ( Latitude )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GeoY1 ( Latitude_WV )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GeoY2 ( Latitude_VIS )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> integer GreyCount ( )
>> CLASS : DIMENSION_SCALE
>> REFERENCE_LIST : <ARRAY of 0 elements>
>>
>> ushort IMG_MIR ( time, Latitude, Longitude )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 1023
>> bandwidth : 0.2
>> bits_per_pixel : 10
>> central_wavelength : 3.9313
>> coordinates : time Latitude Longitude
>> invert : true Latitude Longitude
>> lab_radiance_add_offset : -0.00530575
>> lab_radiance_quad : 0
>> lab_radiance_scale_factor : 0.000316155
>> long_name : Middle Infrared Countde
>> online_radiance_add_offset : -0.00531338
>> online_radiance_quad : -2.68784e-14
>> online_radiance_scale_factor : 0.000318073
>> radiance_units : mW.cm-2.sr-1.micron-1de
>> resolution : 4
>> resolution_unit : km.cm-2.sr-1.micron-1de
>> wavelength_unit : micron2.sr-1.micron-1de
>>
>> float IMG_MIR_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : trueon2.sr-1.micron-1de
>> long_name : Middle Infrared Radiance
>> units : mW.cm-2.sr-1.micron-1nce
>>
>> float IMG_MIR_TEMP ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : truem-2.sr-1.micron-1nce
>> long_name : Middle Infrared Brightness Temperature
>> units : Kiddle Infrared Brightness Temperature
>>
>> ushort IMG_SWIR ( time, Latitude_VIS, Longitude_VISerature )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 0
>> bandwidth : 0.15
>> bits_per_pixel : 10
>> central_wavelength : 1.625
>> coordinates : time Latitude_VIS Longitude_VISerature
>> invert : falseLatitude_VIS Longitude_VISerature
>> --More--
>> invert : falseLatitude_VIS Longitude_VISerature
>> lab_radiance_add_offset : -0.172166
>> lab_radiance_quad : 0
>> lab_radiance_scale_factor : 0.007835
>> long_name : Shortwave Infrared Countude_VISerature
>> online_radiance_add_offset : -0.172166
>> online_radiance_quad : 0
>> online_radiance_scale_factor : 0.007835
>> radiance_units : mW.cm-2.sr-1.micron-1untude_VISerature
>> resolution : 1
>> resolution_unit : km.cm-2.sr-1.micron-1untude_VISerature
>> wavelength_unit : micron2.sr-1.micron-1untude_VISerature
>>
>> float IMG_SWIR_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : falsen2.sr-1.micron-1untude_VISerature
>> long_name : Shortwave Infrared Radiance_VISerature
>> units : mW.cm-2.sr-1.micron-1diance_VISerature
>>
>> ushort IMG_TIR1 ( time, Latitude, Longitude )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 1023
>> bandwidth : 1
>> bits_per_pixel : 10
>> central_wavelength : 10.8288
>> coordinates : time Latitude Longitudeance_VISerature
>> invert : true Latitude Longitudeance_VISerature
>> lab_radiance_add_offset : -0.166517
>> lab_radiance_quad : -4.4237e-07
>> lab_radiance_scale_factor : 0.00232158
>> long_name : Thermal Infrared1 Countance_VISerature
>> online_radiance_add_offset : -0.160677
>> online_radiance_quad : -4.29137e-07
>> online_radiance_scale_factor : 0.00232855
>> radiance_units : mW.cm-2.sr-1.micron-1ntance_VISerature
>> resolution : 4
>> resolution_unit : km.cm-2.sr-1.micron-1ntance_VISerature
>> wavelength_unit : micron2.sr-1.micron-1ntance_VISerature
>> --More--
>> float IMG_TIR1_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : trueon2.sr-1.micron-1ntance_VISerature
>> long_name : Thermal Infrared1 Radiancee_VISerature
>> units : mW.cm-2.sr-1.micron-1iancee_VISerature
>>
>> float IMG_TIR1_TEMP ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : truem-2.sr-1.micron-1iancee_VISerature
>> long_name : Thermal Infrared1 Brightness Temperature
>> units : Khermal Infrared1 Brightness Temperature
>>
>> ushort IMG_TIR2 ( time, Latitude, Longitude )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 1023
>> bandwidth : 1
>> bits_per_pixel : 10
>> central_wavelength : 11.9593
>> coordinates : time Latitude Longitudetness Temperature
>> invert : true Latitude Longitudetness Temperature
>> lab_radiance_add_offset : -0.131206
>> lab_radiance_quad : -3.73059e-07
>> lab_radiance_scale_factor : 0.00203049
>> long_name : Thermal Infrared2 Counttness Temperature
>> online_radiance_add_offset : -0.12046
>> online_radiance_quad : -3.29754e-07
>> online_radiance_scale_factor : 0.00201267
>> radiance_units : mW.cm-2.sr-1.micron-1nttness Temperature
>> resolution : 4
>> resolution_unit : km.cm-2.sr-1.micron-1nttness Temperature
>> wavelength_unit : micron2.sr-1.micron-1nttness Temperature
>>
>> float IMG_TIR2_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : trueon2.sr-1.micron-1nttness Temperature
>> long_name : Thermal Infrared2 Radiancess Temperature
>> units : mW.cm-2.sr-1.micron-1iancess Temperature
>>
>> float IMG_TIR2_TEMP ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : truem-2.sr-1.micron-1iancess Temperature
>> long_name : Thermal Infrared2 Brightness Temperature
>> units : Khermal Infrared2 Brightness Temperature
>>
>> ushort IMG_VIS ( time, Latitude_VIS, Longitude_VISerature )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 0
>> bandwidth : 0.25
>> bits_per_pixel : 10
>> central_wavelength : 0.65
>> coordinates : time Latitude_VIS Longitude_VISmperature
>> invert : falseLatitude_VIS Longitude_VISmperature
>> lab_radiance_add_offset : -1.85891
>> lab_radiance_quad : 0
>> lab_radiance_scale_factor : 0.0630857
>> long_name : Visible Count_VIS Longitude_VISmperature
>> online_radiance_add_offset : -1.85891
>> online_radiance_quad : 0
>> online_radiance_scale_factor : 0.0630857
>> radiance_units : mW.cm-2.sr-1.micron-1gitude_VISmperature
>> resolution : 1
>> resolution_unit : km.cm-2.sr-1.micron-1gitude_VISmperature
>> wavelength_unit : micron2.sr-1.micron-1gitude_VISmperature
>>
>> float IMG_VIS_ALBEDO ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> invert : falsen2.sr-1.micron-1gitude_VISmperature
>> long_name : Visible Albedoicron-1gitude_VISmperature
>> units : %isible Albedoicron-1gitude_VISmperature
>>
>> float IMG_VIS_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> --More--
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : falsele Albedoicron-1gitude_VISmperature
>> long_name : Visible Radianceron-1gitude_VISmperature
>> units : mW.cm-2.sr-1.micron-1gitude_VISmperature
>>
>> ushort IMG_WV ( time, Latitude_WV, Longitude_WVISmperature )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 1023
>> bandwidth : 0.6
>> bits_per_pixel : 10
>> central_wavelength : 6.8841
>> coordinates : time Latitude_WV Longitude_WVISmperature
>> invert : true Latitude_WV Longitude_WVISmperature
>> lab_radiance_add_offset : -0.0406098
>> lab_radiance_quad : -2.38628e-07
>> lab_radiance_scale_factor : 0.00121532
>> long_name : Water Vapor CountLongitude_WVISmperature
>> online_radiance_add_offset : -0.0629506
>> online_radiance_quad : -2.13838e-07
>> online_radiance_scale_factor : 0.00124276
>> radiance_units : mW.cm-2.sr-1.micron-1itude_WVISmperature
>> resolution : 8
>> resolution_unit : km.cm-2.sr-1.micron-1itude_WVISmperature
>> wavelength_unit : micron2.sr-1.micron-1itude_WVISmperature
>>
>> float IMG_WV_RADIANCE ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : trueon2.sr-1.micron-1itude_WVISmperature
>> long_name : Water Vapor Radiance1itude_WVISmperature
>> units : mW.cm-2.sr-1.micron-1itude_WVISmperature
>>
>> float IMG_WV_TEMP ( )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 999
>> invert : truem-2.sr-1.micron-1itude_WVISmperature
>> long_name : Water Vapor Brightness Temperaturerature
>> units : Kater Vapor Brightness Temperaturerature
>> --More--
>> long_name : Water Vapor Brightness Temperaturerature
>> units : Kater Vapor Brightness Temperaturerature
>>
>> short Latitude ( Latitude, Longitude )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 32767
>> add_offset : 0
>> long_name : latitudepor Brightness Temperaturerature
>> scale_factor : 0.01
>> units : degrees_northrightness Temperaturerature
>>
>> integer Latitude_VIS ( Latitude_VIS, Longitude_VISerature )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 327670
>> add_offset : 0
>> long_name : latitudenorthrightness Temperaturerature
>> scale_factor : 0.001
>> units : degrees_northrightness Temperaturerature
>>
>> short Latitude_WV ( Latitude_WV, Longitude_WVISmperature )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 32767
>> add_offset : 0
>> long_name : latitudenorthrightness Temperaturerature
>> scale_factor : 0.01
>> units : degrees_northrightness Temperaturerature
>>
>> short Longitude ( Latitude, Longitude )
>> DIMENSION_LIST : <ARRAY of 0 elements>
>> _FillValue : 32767
>> add_offset : 0
>> long_name : longitudeorthrightness Temperaturerature
>> scale_factor : 0.01
>> units : degrees_easthrightness Temperaturerature
>>
>>
>>
>> [mcbng at localhost IMD-h5files]$ ncl test_imgl1b.ncl
>> Copyright (C) 1995-2015 - All Rights Reserved
>> University Corporation for Atmospheric Research
>> NCAR Command Language Version 6.3.0
>> The use of this software is governed by a License Agreement.
>> See http://www.ncl.ucar.edu/ for more details.
>> fatal:["FileSupport.c":3731]:_NclFindFileExt: Requested file
>> <./3DIMG_23APR2015_1200_L1B_STD.h5> or <./3DIMG_23APR2015_1200_L1B_STD>
>> does not exist
>>
>> fatal:["FileSupport.c":4159]:(./3DIMG_23APR2015_1200_L1B_STD.h5) has no
>> file extension, can't determine type of file to open
>> fatal:file (f) isn't defined
>> fatal:["Execute.c":8575]:Execute: Error occurred at or near line 8 in
>> file test_imgl1b.ncl
>>
>> [mcbng at localhost IMD-h5files]$ vi test_imgl1b.ncl
>> [mcbng at localhost IMD-h5files]$
>>
>> Geeta.
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160401/2020ff16/attachment.html
More information about the ncl-talk
mailing list