[ncl-talk] hdf_to_netcdf
Kunal Bali
kunal.bali9 at gmail.com
Wed Aug 15 00:02:45 MDT 2018
yes sir/ma'am, I did look at the file summary. and that's what I want know
why it's giving two different variable names.
whats the difference between .hdf and .he2. if we can get the variable name
from .hdf then why are we using .he2? what is the use of .he2 over .hdf?
could you please provide some information on that.
regards
Kunal Bali
On Wed, Aug 15, 2018 at 2:28 AM Dennis Shea <shea at ucar.edu> wrote:
> Did you look at the script?
>
> I added the following specifically to help you. Using .hdf and .he2 yields
> slightly different variable names. The .he2 appends _mod04 to each variable
> name.
>
>
> *;*vName= "Deep_Blue_Aerosol_Optical_Depth_550_Land_mod04" ; *".hdf.he2"
> *
> vName= "Deep_Blue_Aerosol_Optical_Depth_550_Land" ; "
> *.hdf*"
> ....
> pthi = diri + fili
> *;;*pthi = pthi + ".he2" ; use .he2 extension
>
> Please do a
>
> *%> ncl_filedump* MOD04_L2.A2017060.0510.006.2017060225118*.hdf *| less
>
> The *.hdf *variable name is: Deep_Blue_Aerosol_Optical_Depth_550_Land
>
> and
>
> *%> ncl_filedump* MOD04_L2.A2017060.0510.006.2017060225118*.hdf.he2* |
> less
>
> The *.hdf.he2* variable name is:
> Deep_Blue_Aerosol_Optical_Depth_550_Land_mod04
>
> ====
>
>
> On Tue, Aug 14, 2018 at 12:07 AM, Kunal Bali <kunal.bali9 at gmail.com>
> wrote:
>
>> First of all thanks a lot, Sir. You have been spending so much time on my
>> work. I appreciate your time.
>>
>> ........
>>
>> I have one query regarding this script.
>> Here you use the variable name
>> vName= "Deep_Blue_Aerosol_Optical_Depth_550_Land"
>>
>> not
>>
>> vName= "Deep_Blue_Aerosol_Optical_Depth_550_Land_*mod04*"
>>
>> why is it so?
>>
>> Also, the output is coming with the values -9999
>>
>> AOT at 0.55 micron for land with all quality data (Quality flag=1,2,3)
>> (None) : min=-9999 max=-9999
>>
>>
>>
>>
>> ---
>> Kunal Bali
>>
>>
>>
>>
>>
>>
>> On Tue, Aug 14, 2018 at 5:42 AM Dennis Shea <shea at ucar.edu> wrote:
>>
>>> You must do more debugging on your own. It takes time to look at your
>>> issues.
>>>
>>> ===
>>> re: "But the files have the given variable then what would be the
>>> reason for this warning?"
>>>
>>> NCL is telling you the issue:
>>> Warning:File ./MOD04_L2.A2017087.0450.006.2017087134018.hdf *dimension
>>> sizes do not conform to others in list;* skipping file
>>>
>>> What should you do?
>>>
>>> Well, of course, look at the *dimension sizes* on each file.
>>> ncl-talk always tells people to "look at the data" ... assume nothing."
>>>
>>> Please see: the attachments:
>>>
>>> *%> ncl* MOD04_dimsizes.ncl >&! out.MOD04_dimsizes
>>>
>>> Some files have soil_moisture with dimension sizes: (/*203*,135/)
>>> while others are (/*204*,135/)
>>>
>>> *addfiles
>>> <https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml>*
>>> expects *consistent *dimension sizes. *addfiles* ["*join*"] across 60
>>> files would expect *all*
>>>
>>> *all* (60, 203, 135)
>>> or
>>> *all* (60, 204, 135)
>>>
>>> It can *not* do
>>>
>>> (60, *203 or 204*, 135)
>>>
>>>
>>> =======================
>>> I am attaching a script that makes netCDF files. One netCDF per input
>>> HDF file.
>>>
>>> %> ncl MOD04_to_netCDF.ncl
>>>
>>>
>>>
>>>
>>> On Mon, Aug 13, 2018 at 1:56 AM, Kunal Bali <kunal.bali9 at gmail.com>
>>> wrote:
>>>
>>>> Sir,
>>>>
>>>> sorry for sending this offline message.
>>>>
>>>>
>>>> After running your script for the attached file list. I still gets the
>>>> same warning
>>>>
>>>> warning:File ./MOD04_L2.A2017087.0450.006.2017087134018.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:File ./MOD04_L2.A2017082.0435.006.2017082135321.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:File ./MOD04_L2.A2017078.0500.006.2017081195531.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:File ./MOD04_L2.A2017074.0520.006.2017074182316.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:File ./MOD04_L2.A2017069.0505.006.2017069135117.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:File ./MOD04_L2.A2017060.0510.006.2017060225118.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:*A** valid instance of variable
>>>> Deep_Blue_Aerosol_Optical_Depth_550_Land_mod04 was not found in one or more
>>>> elements of the file list*
>>>>
>>>> But the files have the given variable then what would be the reason for
>>>> this warning?
>>>>
>>>> and also, when I run the same script for the given skipping file
>>>> (separately in the new directory) then it it gives warning with one
>>>> skipping file.
>>>>
>>>> warning:File ./MOD04_L2.A2017073.0440.006.2017073134018.hdf dimension
>>>> sizes do not conform to others in list; skipping file
>>>> warning:A valid instance of variable
>>>> Deep_Blue_Aerosol_Optical_Depth_550_Land_mod04 was not found in one or more
>>>> elements of the file list
>>>>
>>>>
>>>> Could you please provide some thoughts on that as well.
>>>>
>>>>
>>>> and sorry for sending this offline mail.
>>>>
>>>>
>>>>
>>>> ---
>>>> Kunal Bali
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> [image: Mailtrack]
>>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality1&> Sent
>>>> with Mailtrack
>>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality1&>
>>>>
>>>> On Thu, Aug 9, 2018 at 10:14 AM Kunal Bali <kunal.bali9 at gmail.com>
>>>> wrote:
>>>>
>>>>> Dear Sir,
>>>>> Thanks for the explanation and spending some time on this query. Thank
>>>>> You so much.
>>>>>
>>>>>
>>>>>
>>>>> ---
>>>>> Kunal Bali
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 9, 2018 at 6:00 AM Dennis Shea <shea at ucar.edu> wrote:
>>>>>
>>>>>> *This is offline from ncl-talk*
>>>>>>
>>>>>> I had a few MOD04_L2 files so I created a basic script with many
>>>>>> print statements.
>>>>>> It shows multiple ways to define 'time.'
>>>>>>
>>>>>> Please run on your MOD04_L2.
>>>>>> Look carefully at the output. In particular, the various 'time'
>>>>>> variables.
>>>>>> I suppose you use 'time_file' used rather that 'time_scan'
>>>>>>
>>>>>> This is *swath* data so while the latitude/longitude *dimension
>>>>>> sizes *on each file are the same, it is not likely that the actual
>>>>>> latitude and longitude values are the same. Hence, there is a 'time'
>>>>>> associated with the lat/lon arrays.
>>>>>>
>>>>>> ============
>>>>>>
>>>>>> Good Luck
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 8, 2018 at 9:20 AM, Dennis Shea <shea at ucar.edu> wrote:
>>>>>>
>>>>>>> Your 'summary.txt' contains:
>>>>>>>
>>>>>>> double* Scan_Start_Time* ( Cell_Along_Swath_mod04,
>>>>>>> Cell_Across_Swath_mod04 )
>>>>>>> valid_range : ( 0, 3155800000 )
>>>>>>> _FillValue : -999
>>>>>>> long_name : TAI Time at Start of Scan replicated across
>>>>>>> the swath
>>>>>>> units : Seconds since 1993-1-1 00:00:00.0 0
>>>>>>> scale_factor : 1
>>>>>>> add_offset : 0
>>>>>>> Parameter_Type : MODIS Input
>>>>>>> Cell_Along_Swath_Sampling : ( 1, 2021, 10 )
>>>>>>> Cell_Across_Swath_Sampling : ( 1, 1345, 10 )
>>>>>>> Geolocation_Pointer : Internal geolocation arrays
>>>>>>> hdf_name : Scan_Start_Time
>>>>>>>
>>>>>>> This 'time' and it is *two-dimensional*.
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> The 'time' you are trying to create is *one-dimensional *and is
>>>>>>> based strictly on a file name.
>>>>>>>
>>>>>>> ----
>>>>>>> Also, the following may be if interest
>>>>>>>
>>>>>>> https://www.ncl.ucar.edu/Applications/HDF.shtml
>>>>>>> hdf4sds_5.ncl
>>>>>>> <https://www.ncl.ucar.edu/Applications/Scripts/hdf4sds_5.ncl>
>>>>>>>
>>>>>>> ---
>>>>>>> These files are HDF-EOS format:
>>>>>>>
>>>>>>> *HDFEOSVersion : HDFEOS_V2.19*
>>>>>>> For fun, append a *.he2* file extension:
>>>>>>>
>>>>>>> %>* ncl_filedump *MOD04_L2.A2017202.0705.061.2017316182248.hdf*.he2*
>>>>>>> | less
>>>>>>>
>>>>>>> It makes the file look like a netCDF file dump.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Aug 8, 2018 at 3:50 AM, Kunal Bali <kunal.bali9 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Dear NCL user,
>>>>>>>>
>>>>>>>> I am converting hdf to netcdf using attached ncl script (extracting
>>>>>>>> aerosol product). But it's showing some errors and values are also coming
>>>>>>>> from +01. to -0.1.
>>>>>>>> Summary of the file is also attached herewith.
>>>>>>>>
>>>>>>>>
>>>>>>>> the errors/warning are:
>>>>>>>>
>>>>>>>> warning:File ./MOD04_L2.A2017209.0710.061.2017317003637.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017209.0525.061.2017317002757.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017205.0550.061.2017316194801.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017200.0355.061.2017316192232.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017196.0600.061.2017316164649.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017195.0340.061.2017316165535.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017192.0625.061.2017316024320.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017191.0405.061.2017316013818.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017188.0645.061.2017316002323.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017187.0610.061.2017315234444.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017187.0425.061.2017315234452.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017184.0710.061.2017315214958.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:File ./MOD04_L2.A2017183.0450.061.2017315215216.hdf
>>>>>>>> dimension sizes do not conform to others in list; skipping file
>>>>>>>> warning:A valid instance of variable
>>>>>>>> Deep_Blue_Aerosol_Optical_Depth_550_Land_mod04 was not found in one or more
>>>>>>>> elements of the file list
>>>>>>>> fatal:Coordinate variables must be the same dimension as their
>>>>>>>> dimension
>>>>>>>> fatal:No coordinate variable exists for dimension (time) in
>>>>>>>> variable (var)
>>>>>>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line 35
>>>>>>>> in file script_01.ncl
>>>>>>>>
>>>>>>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line 69
>>>>>>>> in file script_01.ncl
>>>>>>>>
>>>>>>>>
>>>>>>>> Variable: od55_1km
>>>>>>>> Type: short
>>>>>>>> Total Size: 6303150 bytes
>>>>>>>> 3151575 values
>>>>>>>> Number of Dimensions: 3
>>>>>>>> Dimensions and sizes: [time | 115] x [Cell_Along_Swath_mod04 |
>>>>>>>> 203] x [Cell_Across_Swath_mod04 | 135]
>>>>>>>> Coordinates:
>>>>>>>> Number Of Attributes: 12
>>>>>>>> coordinates : Latitude_mod04, Longitude_mod04
>>>>>>>> hdfeos_name : Deep_Blue_Aerosol_Optical_Depth_550_Land
>>>>>>>> _FillValue : -9999
>>>>>>>> Geolocation_Pointer : Internal geolocation arrays
>>>>>>>> Cell_Across_Swath_Sampling : ( 1, 1354, 10 )
>>>>>>>> Cell_Along_Swath_Sampling : ( 1, 2021, 10 )
>>>>>>>> Parameter_Type : Output
>>>>>>>> add_offset : 0
>>>>>>>> scale_factor : 0.001000000047497451
>>>>>>>> units : None
>>>>>>>> long_name : AOT at 0.55 micron for land with all quality data
>>>>>>>> (Quality flag=1,2,3)
>>>>>>>> valid_range : ( 0, 5000 )
>>>>>>>> fatal:Dimension sizes of left hand side do not match right hand side
>>>>>>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line
>>>>>>>> 110 in file script_01.ncl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> could you please provide some information on this.
>>>>>>>>
>>>>>>>> Thank You
>>>>>>>>
>>>>>>>>
>>>>>>>> ---
>>>>>>>> Kunal Bali
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [image: Mailtrack]
>>>>>>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality1&> Sent
>>>>>>>> with Mailtrack
>>>>>>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality1&>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ncl-talk mailing list
>>>>>>>> ncl-talk at ucar.edu
>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180815/fbc9572f/attachment.html>
More information about the ncl-talk
mailing list