<div dir="ltr"><div><div><div>I just looked at the 5 sample files you sent .... <br><br>The dimension sizes are *not* the same size across all files<br></div><br></div>The 1st 4files are:<br><br><br>MOD04_L2.A2014305.0000.006.2015077213911.pscs_000500937810.hdf<br> dimensions:<br> Cell_Along_Swath = 203 <=====<br> Cell_Across_Swath = 135<br><br>The last file is:<br><br>MOD04_L2.A2014305.0500.006.2015077215106.pscs_000500937810<br> dimensions:<br> Cell_Along_Swath = 204 <====<br> Cell_Across_Swath= 135<br><br></div><div>That can be 'programmed around'.<br><br></div><div>See attached<br></div><div><br></div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 26, 2015 at 6:34 AM, BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL <span dir="ltr"><<a href="mailto:steven.bliujus.3.ctr@us.af.mil" target="_blank">steven.bliujus.3.ctr@us.af.mil</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dennis,<br>
<br>
#2 and #3: Cell_Along_Swath, and Cell_Across_Swath are exactly the same for every file as far as actual dimension size, the variables are not though.<br>
#4: I am trying to create a mosaic of all the files and that is why I am wanting to generate one NetCDF file. So when I run my script the image above is what comes out. This image is what I want to create in a NetCDF file (Globe.PNG).<br>
#5: I did this because I was following a script online that I found where I needed to give the main variable(Optical Depth) the latitude/longitude dimensions. It used (::5) which what I just followed. When I try plotting using just (:,:), the plot take an extremely long time to plot(though there is much more data prevalent)...If it is one file it doesn’t take long, but when you add them all in the mosaic form, it takes a much more substantial amount of time to generate the image.<br>
#6: I'm not sure exactly what you mean by this as there is not time field in the file.<br>
#7: See 5 attached MODIS files<br>
#8: I am trying to create a NetCDF file for a coworker to use for his verification software. He only reads in NetCDF files or text files (though he would prefer NetCDF files)<br>
<br>
I also attached the script (with all previous commented lines removed) that plots the files and also creates a text table of the data.<br>
<br>
Any help would be appreciated.<br>
<br>
Steve<br>
<span class="im HOEnZb"><br>
<br>
<br>
-----Original Message-----<br>
From: Dennis Shea [mailto:<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>]<br>
</span><div class="HOEnZb"><div class="h5">Sent: Saturday, May 23, 2015 2:30 PM<br>
To: BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL<br>
Cc: <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
Subject: Re: [ncl-talk] Problems converting hdf to netcdf using NCL<br>
<br>
I have looked at this and there are *lots* of issues.<br>
<br>
"... taking several hdf files (200-400) and trying to generate one NetCDF file."<br>
<br>
<br>
So ... **Conceptually**, you want<br>
<br>
<br>
AOT(time, Cell_Along_Swath, Cell_Across_Swath)<br>
<br>
<br>
Thing is your geographic coordinates are 2-dimensional. see below<br>
<br>
<br>
<br>
<br>
[1]<br>
<br>
The files contain swath data. From your dump:<br>
<br>
path: MOD04_L2.A2014306.2330.006.2015077230656.pscs_000500937810.hdf<br>
file global attributes:<br>
HDFEOSVersion : HDFEOS_V2.17<br>
StructMetadata_0 : GROUP=SwathStructure<br>
GROUP=SWATH_1<br>
SwathName="mod04"<br>
<br>
<br>
[2] Are the dimension sizes<br>
<br>
(Cell_Along_Swath,Cell_Across_Swath) => (203,135)<br>
<br>
<br>
exactly the same for *all* files?<br>
<br>
<br>
[3] If [2] is satisfied .... Are the *values* contained within the following variables<br>
<br>
Latitude ( Cell_Along_Swath, Cell_Across_Swath)<br>
<br>
Longitude ( Cell_Along_Swath, Cell_Across_Swath )<br>
<br>
<br>
*exactly* the same for *all* files? ie: homogeneous space!<br>
<br>
<br>
My (limited) experience with satellite swath data is that the answer is "no"<br>
<br>
<br>
[4] If [2] *and* [3] are not satisfied, I see no advantage of creating a single netCDF file.<br>
<br>
You would end up with 200-400 sets of dimension sizes *and* 200-400 variables<br>
<br>
associated 200-400two-dimensional spatial variables.<br>
<br>
<br>
[5] I have no idea why you are sub-sampling [decimating] the array by 5:<br>
(::5,::5) the resolution from (203,135) to ??(41,27)??. This seems drastic!<br>
<br>
Sub-sampling (array decimation) is commonly used used when the resolution is (say) (2500,4000) and the data are reasonably smooth.<br>
<br>
<br>
<br>
[6] In any case you should be using<br>
<br>
ListSetType(a,"join")<br>
<br>
<br>
to add a record dimension ... which you should fill in with 'time'<br>
<br>
(parse file name) or some sequential number.<br>
<br>
<br>
Please read<br>
<a href="https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml" target="_blank">https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml</a><br>
<br>
Example 3 illustrates what "join" will do.<br>
<br>
<br>
<br>
[7] I do not have your ?MODIS? data file. Based upon your dump, I created a script that creates a netCDF file for each HDF file and (I hope) a sample graphic.<br>
<br>
<br>
[8] I am not sure of ultimate use of the data but from an NCL perspective, there is no reason to convert to netCDF.<br>
<br>
<br>
===========================<br>
<br>
<br>
Please upgrade to 6.3.0<br>
<br>
<a href="http://www.ncl.ucar.edu/Download/" target="_blank">http://www.ncl.ucar.edu/Download/</a><br>
<br>
============================<br>
<br>
<br>
<br>
<br>
<br>
On Fri, May 22, 2015 at 9:59 AM, BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL <<a href="mailto:steven.bliujus.3.ctr@us.af.mil">steven.bliujus.3.ctr@us.af.mil</a>> wrote:<br>
<br>
<br>
I wanted to provide an example of what is in the hdf file. See attached.<br>
<br>
Steve<br>
<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:ncl-talk-bounces@ucar.edu">ncl-talk-bounces@ucar.edu</a> [mailto:<a href="mailto:ncl-talk-bounces@ucar.edu">ncl-talk-bounces@ucar.edu</a>] On Behalf Of BLIUJUS, STEVEN D CTR USAF AFMC AFLCMC/HBAW-OL<br>
Sent: Friday, May 22, 2015 10:36 AM<br>
To: <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
Subject: [ncl-talk] Problems converting hdf to netcdf using NCL<br>
<br>
I am taking several hdf files (200-400) and trying to generate one NetCDF file. I am able to plot it on a grid, but when I try to write to a NetCDF file, I get the following error:<br>
<br>
fatal:Number of dimensions in parameter (2) of (filedimdef) is (2), (1) dimensions were expected<br>
fatal:["Execute.c":8128]:Execute: Error occurred at or near line 82 in file File.ncl<br>
<br>
<br>
The data I am creating is AOD with two dimensions (lat/lon). I am uncertain as to why it is expecting one dimension. Below is my script. FYI, the original files had to be manipulated in order to make AOD have the lat/lon dimensions. Any help would be greatly appreciated.<br>
<br>
<br>
load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/gsn_code.ncl"<br>
load "/home/bliujuss/ncl/lib/ncarg/nclscripts/wrf/WRFUserARW.ncl"<br>
load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/gsn_csm.ncl"<br>
load "/home/bliujuss/ncl/lib/ncarg/nclscripts/csm/shea_util.ncl"<br>
begin<br>
<br>
diri = "/home/bliujuss/Verification_data/Test/"<br>
fili = systemfunc("ls "+diri+"*.hdf")<br>
a = addfiles(fili,"r")<br>
<br>
;ListSetType(a,"cat")<br>
<br>
var1=a[:]->Optical_Depth_Land_And_Ocean(:,:)<br>
Optical_Depth_Land_And_Ocean = short2flt(var1)<br>
<br>
var2=a[:]->Longitude(:,:)<br>
Longitude=var2<br>
<br>
<br>
var3 = a[:]->Latitude(:,:)<br>
Latitude=var3<br>
<br>
dims = dimsizes(Optical_Depth_Land_And_Ocean)<br>
nrows = dims(0) ;203<br>
npixels = dims(1) ;135<br>
<br>
lat2d = new((/nrows,npixels/),float)<br>
lon2d = new((/nrows,npixels/),float)<br>
<br>
do i=0,npixels-1<br>
lat2d(:,i) = Latitude(:,i)<br>
end do<br>
<br>
do i=0,nrows-1<br>
lon2d(i,:) = Longitude(i,:)<br>
end do<br>
<br>
Optical_Depth_Land_And_Ocean@lat2d = lat2d(::5,::5) Optical_Depth_Land_And_Ocean@lon2d = lon2d(::5,::5)<br>
<br>
<br>
wks = gsn_open_wks("x11","plot_aod")<br>
gsn_define_colormap(wks,"NCV_banded")<br>
setvalues NhlGetWorkspaceObjectId()<br>
"wsMaximumSize": 300000000<br>
end setvalues<br>
<br>
res = True ; plot mods desired<br>
res@cnLinesOn = False ; turn off contour lines<br>
res@cnFillOn = True ; color plot desired<br>
res@cnLineLabelsOn = False ; turn off contour lines<br>
res@gsnAddCyclic = False ; non-global data<br>
res@gsnSpreadColors = True ; use full range of colormap<br>
;res@cnFillMode = "RasterFill" ; turn on raster mode<br>
res@lbLabelAutoStride = True ; nice spacing of lables<br>
res@gsnMaximize = True ; blow up plot as much as poss<br>
;print(min(lat2d))<br>
;print(max(lat2d))<br>
;print(min(lon2d))<br>
;print(max(lon2d))<br>
<br>
plot = gsn_csm_contour_map_ce(wks,Optical_Depth_Land_And_Ocean(::5,::5),res)<br>
print(max(Optical_Depth_Land_And_Ocean(::5,::5)))<br>
<br>
nlat = dimsizes(lat2d(::5,::5))<br>
nlon = dimsizes(lon2d(::5,::5))<br>
<br>
diro = "/home/bliujuss/Plots/"<br>
filo = "<a href="http://example.nc" target="_blank">example.nc</a>"<br>
system("rm -f" + diro + filo)<br>
fout = addfile (diro + filo, "c")<br>
<br>
setfileoption(fout,"DefineMode",True)<br>
<br>
dim_names = (/"lat", "lon"/)<br>
dim_sizes = (/nlat, nlon/)<br>
dimUnlim = (/False, False/)<br>
filedimdef(fout,dim_names,dim_sizes,dimUnlim)<br>
<br>
filevardef(fout, "lat", typeof(lat2d(::5,::5)),getvardims(lat2d(::5,::5)))<br>
filevardef(fout, "lon", typeof(lon2d(::5,::5)),getvardims(lon2d(::5,::5)))<br>
filevardef(fout, "AOT", typeof(Optical_Depth_Land_And_Ocean(::5,::5)),getvardims(Optical_Depth_Land_And_Ocean(::5,::5)))<br>
<br>
filevarattdef(fout,"AOT", Optical_Depth_Land_And_Ocean(::5,::5))<br>
filevarattdef(fout,"lat", lat2d(::5,::5)) filevarattdef(fout,"lon", lon2d(::5,::5))<br>
<br>
setfileoption(fout,"DefineMode",False)<br>
<br>
fout->Optical_Depth_Land_And_Ocean(::5,::5) =<br>
<br>
fout->(/Optical_Depth_Land_And_Ocean(::5,::5)/)<br>
<br>
fout->lat2d(::5,::5) = (/lat2d(::5,::5)/)<br>
fout->lon2d(::5,::5) = (/lon2d(::5,::5)/)<br>
<br>
end<br>
<br>
<br>
Steven Bliujus<br>
<br>
<br>
_______________________________________________<br>
ncl-talk mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br>
<br>
_______________________________________________<br>
ncl-talk mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br>
<br>
<br>
<br>
</div></div></blockquote></div><br></div>