[ncl-talk] Problems converting hdf to netcdf using NCL
Dennis Shea
shea at ucar.edu
Sat May 23 13:30:03 MDT 2015
I have looked at this and there are *lots* of issues.
"... taking several hdf files (200-400) and trying to generate one NetCDF
file."
So ... **Conceptually**, you want
AOT(time, Cell_Along_Swath, Cell_Across_Swath)
Thing is your geographic coordinates are 2-dimensional. see below
[1]
The files contain swath data. From your dump:
path: MOD04_L2.A2014306.2330.006.2015077230656.pscs_000500937810.hdf
file global attributes:
HDFEOSVersion : HDFEOS_V2.17
StructMetadata_0 : GROUP=SwathStructure
GROUP=SWATH_1
SwathName="mod04"
[2] Are the dimension sizes
(Cell_Along_Swath,Cell_Across_Swath) => (203,135)
exactly the same for *all* files?
[3] If [2] is satisfied .... Are the *values* contained within the
following variables
Latitude ( Cell_Along_Swath, Cell_Across_Swath)
Longitude ( Cell_Along_Swath, Cell_Across_Swath )
*exactly* the same for *all* files? ie: homogeneous space!
My (limited) experience with satellite swath data is that the answer
is "no"
[4] If [2] *and* [3] are not satisfied, I see no advantage of creating a
single netCDF file.
You would end up with 200-400 sets of dimension sizes *and* 200-400
variables
associated 200-400two-dimensional spatial variables.
[5] I have no idea why you are sub-sampling [decimating] the array by 5:
(::5,::5) the resolution from (203,135) to ??(41,27)??. This seems
drastic!
Sub-sampling (array decimation) is commonly used used when the
resolution is (say) (2500,4000) and the data are reasonably smooth.
[6] In any case you should be using
ListSetType(a,"join")
to add a record dimension ... which you should fill in with 'time'
(parse file name) or some sequential number.
Please read
https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml
Example 3 illustrates what "join" will do.
[7] I do not have your ?MODIS? data file. Based upon your dump, I created a
script that creates a netCDF file for each HDF file and (I hope) a sample
graphic.
[8] I am not sure of ultimate use of the data but from an NCL perspective,
there is no reason to convert to netCDF.
===========================
Please upgrade to 6.3.0
http://www.ncl.ucar.edu/Download/
============================
On Fri, May 22, 2015 at 9:59 AM, BLIUJUS, STEVEN D CTR USAF AFMC
AFLCMC/HBAW-OL <steven.bliujus.3.ctr at us.af.mil> wrote:
> I wanted to provide an example of what is in the hdf file. See attached.
>
> Steve
>
> -----Original Message-----
> From: ncl-talk-bounces at ucar.edu [mailto:ncl-talk-bounces at ucar.edu] On
> Behalf Of BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL
> Sent: Friday, May 22, 2015 10:36 AM
> To: ncl-talk at ucar.edu
> Subject: [ncl-talk] Problems converting hdf to netcdf using NCL
>
> I am taking several hdf files (200-400) and trying to generate one NetCDF
> file. I am able to plot it on a grid, but when I try to write to a NetCDF
> file, I get the following error:
>
> fatal:Number of dimensions in parameter (2) of (filedimdef) is (2), (1)
> dimensions were expected
> fatal:["Execute.c":8128]:Execute: Error occurred at or near line 82 in
> file File.ncl
>
>
> The data I am creating is AOD with two dimensions (lat/lon). I am
> uncertain as to why it is expecting one dimension. Below is my script. FYI,
> the original files had to be manipulated in order to make AOD have the
> lat/lon dimensions. Any help would be greatly appreciated.
>
>
> load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "/home/bliujuss/ncl/lib/ncarg/nclscripts/wrf/WRFUserARW.ncl"
> load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/shea_util.ncl"
> begin
>
> diri = "/home/bliujuss/Verification_data/Test/"
> fili = systemfunc("ls "+diri+"*.hdf")
> a = addfiles(fili,"r")
>
> ;ListSetType(a,"cat")
>
> var1=a[:]->Optical_Depth_Land_And_Ocean(:,:)
> Optical_Depth_Land_And_Ocean = short2flt(var1)
>
> var2=a[:]->Longitude(:,:)
> Longitude=var2
>
>
> var3 = a[:]->Latitude(:,:)
> Latitude=var3
>
> dims = dimsizes(Optical_Depth_Land_And_Ocean)
> nrows = dims(0) ;203
> npixels = dims(1) ;135
>
> lat2d = new((/nrows,npixels/),float)
> lon2d = new((/nrows,npixels/),float)
>
> do i=0,npixels-1
> lat2d(:,i) = Latitude(:,i)
> end do
>
> do i=0,nrows-1
> lon2d(i,:) = Longitude(i,:)
> end do
>
> Optical_Depth_Land_And_Ocean at lat2d = lat2d(::5,::5)
> Optical_Depth_Land_And_Ocean at lon2d = lon2d(::5,::5)
>
>
> wks = gsn_open_wks("x11","plot_aod")
> gsn_define_colormap(wks,"NCV_banded")
> setvalues NhlGetWorkspaceObjectId()
> "wsMaximumSize": 300000000
> end setvalues
>
> res = True ; plot mods desired
> res at cnLinesOn = False ; turn off contour lines
> res at cnFillOn = True ; color plot desired
> res at cnLineLabelsOn = False ; turn off contour lines
> res at gsnAddCyclic = False ; non-global data
> res at gsnSpreadColors = True ; use full range of colormap
> ;res at cnFillMode = "RasterFill" ; turn on raster mode
> res at lbLabelAutoStride = True ; nice spacing of lables
> res at gsnMaximize = True ; blow up plot as much as poss
> ;print(min(lat2d))
> ;print(max(lat2d))
> ;print(min(lon2d))
> ;print(max(lon2d))
>
> plot =
> gsn_csm_contour_map_ce(wks,Optical_Depth_Land_And_Ocean(::5,::5),res)
> print(max(Optical_Depth_Land_And_Ocean(::5,::5)))
>
> nlat = dimsizes(lat2d(::5,::5))
> nlon = dimsizes(lon2d(::5,::5))
>
> diro = "/home/bliujuss/Plots/"
> filo = "example.nc"
> system("rm -f" + diro + filo)
> fout = addfile (diro + filo, "c")
>
> setfileoption(fout,"DefineMode",True)
>
> dim_names = (/"lat", "lon"/)
> dim_sizes = (/nlat, nlon/)
> dimUnlim = (/False, False/)
> filedimdef(fout,dim_names,dim_sizes,dimUnlim)
>
> filevardef(fout, "lat", typeof(lat2d(::5,::5)),getvardims(lat2d(::5,::5)))
> filevardef(fout, "lon", typeof(lon2d(::5,::5)),getvardims(lon2d(::5,::5)))
> filevardef(fout, "AOT",
> typeof(Optical_Depth_Land_And_Ocean(::5,::5)),getvardims(Optical_Depth_Land_And_Ocean(::5,::5)))
>
> filevarattdef(fout,"AOT", Optical_Depth_Land_And_Ocean(::5,::5))
> filevarattdef(fout,"lat", lat2d(::5,::5)) filevarattdef(fout,"lon",
> lon2d(::5,::5))
>
> setfileoption(fout,"DefineMode",False)
>
> fout->Optical_Depth_Land_And_Ocean(::5,::5) =
> fout->(/Optical_Depth_Land_And_Ocean(::5,::5)/)
> fout->lat2d(::5,::5) = (/lat2d(::5,::5)/)
> fout->lon2d(::5,::5) = (/lon2d(::5,::5)/)
>
> end
>
>
> Steven Bliujus
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150523/77a71563/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bliujus.ncl
Type: application/octet-stream
Size: 4056 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150523/77a71563/attachment.obj
More information about the ncl-talk
mailing list