[ncl-talk] CNL read CALIPSO v4.51 data

Thu Sep 4 18:43:07 MDT 2025

Rick, that is more good info.  It looks like NCL 6.6.2 can read this file
directly, by providing one or more explicit dimension sizes as needed.  It
would be good to examine the array data and metadata (attributes)
carefully, before using these data for other work.

NCO utilities says it can easily convert HDF4 files to Netcdf.  Please see
section *1.6 netCDF2/3/4 and HDF4/5 Support* in the current NCO manual.
This might be the simplest and safest way -- if it works in this case.
Again, verify the results.

https://nco.sourceforge.net/nco.html

On Thu, Sep 4, 2025 at 4:26 PM Rick Brownrigg <brownrig at ucar.edu> wrote:

> Thanks for that tip about hdp, Dave.  Using NCL to
> print(f->Feature_Classification_Flags) indicates it has dimensions
>
> Dimensions and sizes: [fakeDim18 | 63360] x [fakeDim19 | 5515]
>
> But hdp shows:
>
> Variable Name = Feature_Classification_Flags
> Index = 9
> Type= 16-bit unsigned integer
> Ref. = 11
> Compression method = NONE
> Rank = 2
> Number of attributes = 3
> Dim0: Name=fakeDim18
> Size = UNLIMITED (currently 4224)
> Scale Type = number-type not set
> Number of attributes = 0
> Dim1: Name=fakeDim19
> Size = 5515
> Scale Type = number-type not set
> Number of attributes = 0
>
> So if I have NCL read the variables as:
>
>   foo = f->Feature_Classification_Flags(:4223, :)
>
> It seems to load fine.  If I say
>
>   foo = f->Feature_Classification_Flags(:4224, :)
>
> it errs out.  The same is true of some of the other variables, ex.
> Latitude's 1st dimension is only 4224, not 63360.
>
> Rick
>
>
> On Thu, Sep 4, 2025 at 4:06 PM Dave Allured - NOAA Affiliate via ncl-talk <
> ncl-talk at mailman.ucar.edu> wrote:
>
>> Where is the documentation for CALIPSO files?
>>
>>
>> On Thu, Sep 4, 2025 at 4:03 PM Dave Allured - NOAA Affiliate <
>> dave.allured at noaa.gov> wrote:
>>
>>> HDF4, that is a good clue.  The dump utility for HDF4 is "hdp".  Do you
>>> have it?   There are many options.  To start, try "hdp -H list" and go from
>>> there.
>>> https://support.hdfgroup.org/hdf4/
>>>
>>> Note that HDF4 tools include an HDF4 to HDF5 converter, which they
>>> recommend.
>>>
>>> NCL addfile might be identifying this file as HDF-EOS, which is based on
>>> HDF4.  See addfile documentation.  Try switching the filename extension
>>> from .hdf to .he4 and see if that improves NCL readability.
>>>
>>>
>>> On Thu, Sep 4, 2025 at 3:39 PM Rick Brownrigg <brownrig at ucar.edu> wrote:
>>>
>>>> h5dump can't open it.  Running "strings" on that filename indicates its
>>>> an HDF4.2 file?
>>>>
>>>>
>>>> On Thu, Sep 4, 2025 at 3:30 PM Dave Allured - NOAA Affiliate via
>>>> ncl-talk <ncl-talk at mailman.ucar.edu> wrote:
>>>>
>>>>> More diagnostics needed.  Someone please show output from:
>>>>>
>>>>> NCL:
>>>>> fh = addfile("CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-
>>>>> 00-11ZD.hdf","r")
>>>>> print (fh)
>>>>>
>>>>> Linux:
>>>>> h5dump -BH CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf
>>>>>
>>>>>
>>>>> On Thu, Sep 4, 2025 at 3:23 PM Rick Brownrigg via ncl-talk <
>>>>> ncl-talk at mailman.ucar.edu> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As that file resides on /glade, I was able to access it and duplicate
>>>>>> the problem. I can print() that variable, but I can't load it into an NCL
>>>>>> variable. Its rather large -- 699MB. Perhaps that's the problem?
>>>>>> Interestingly, if I run "ncdump" on that file, I get:
>>>>>>
>>>>>>  ncdump
>>>>>> /glade/derecho/scratch/zhifengy/data/experiment_input/doe_cloud_africa/CALIPSO/CAL_LID_L2_VFM-Standard-V4-51/test/CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf
>>>>>>
>>>>>> ncdump:
>>>>>> /glade/derecho/scratch/zhifengy/data/experiment_input/doe_cloud_africa/CALIPSO/CAL_LID_L2_VFM-Standard-V4-51/test/CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf:
>>>>>> NetCDF: Attempt to use feature that was not turned on when netCDF was built.
>>>>>>
>>>>>> Rick
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 4, 2025 at 3:16 PM Ehsan Taghizadeh via ncl-talk <
>>>>>> ncl-talk at mailman.ucar.edu> wrote:
>>>>>>
>>>>>>> Hi Zhifeng,
>>>>>>>
>>>>>>> Could you please add the following commands before your addfile
>>>>>>> command:
>>>>>>>
>>>>>>> exists =
>>>>>>> isfilepresent("CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf")
>>>>>>> print (exists)
>>>>>>>
>>>>>>> I wonder if your ncl script and the file are in the same place.
>>>>>>> In case it returns True, could you please add the following
>>>>>>> commands just after reading your file (after addfile command):
>>>>>>>
>>>>>>> names = getfilevarnames(
>>>>>>> "CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf")
>>>>>>> print (names)
>>>>>>>
>>>>>>>
>>>>>>> *-----------------------------------------------------------------------------*
>>>>>>> *Best regards,*
>>>>>>> *Ehsan Taghizadeh*
>>>>>>>
>>>>>>> On Thursday, September 4, 2025 at 01:37:05 PM PDT, Zhifeng Yang via
>>>>>>> ncl-talk <ncl-talk at mailman.ucar.edu> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am using NCL 6.6.2 to read CALIPSO latitude, longitude, and
>>>>>>> Feature_Classification_Flags variable from CAL_LID_L2_VFM-Standard-V4-51.
>>>>>>>
>>>>>>>   fh =
>>>>>>> addfile("CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf","r")
>>>>>>>    lath      = fh->Latitude(:, 0)
>>>>>>>    lonh      = fh->Longitude(:, 0)
>>>>>>>    vfmh      = fh->Feature_Classification_Flags
>>>>>>>
>>>>>>> However, I got the error below
>>>>>>> fatal:HDF: An error occurred while attempting to read variable
>>>>>>> (Feature_Classification_Flags) from file
>>>>>>> (/glade/derecho/scratch/zhifengy/data/experiment_input/doe_cloud_africa/CALIPSO/CAL_LID_L2_VFM-Standard-V4-51/test/CAL_LID_L2_VFM-Standard-V4-51.2017-08-21T14-00-11ZD.hdf)
>>>>>>>
>>>>>>> Do you have any idea about this? Thanks,
>>>>>>>
>>>>>>> Best,
>>>>>>> Zhifeng
>>>>>>>
>>>>>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at mailman.ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20250904/4f1e9214/attachment.htm>