[ncl-talk] iteration over files
Steeven Paul Yerraguntla
steevenpaul at gmail.com
Tue Feb 24 02:21:27 MST 2015
Dear NCL users
Thanks to Griffin for good suggestions and i could convert the hdf to
nc with the script provided by Dennis long ago and the problems is the
iteration is done only to the first file in the loop and the error
occuring with the message "segmentation fault (core dumped)". I run
the script again using the command ncl -x filename.ncl and the output
as follows. Please let me know how can i get the iteration completed.
***********************************************************************************
Copyright (C) 1995-2012 - All Rights Reserved
University Corporation for Atmospheric Research
NCAR Command Language Version 6.1.0
The use of this software is governed by a License Agreement.
See http://www.ncl.ucar.edu/ for more details.
+ ;************************************************
+ ; Allways examine file contents:
+ ; %> ncl_filedump foo.h5 | less
+ ;************************************************
Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
+
Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
+
Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
+
+ ;************************************************
+ begin
+ diri = "./"
+ diroxx = "./"
+
+ fili = systemfunc("cd "+diri+" ; ls *.h5")
+ nfil = dimsizes(fili)
+ print("nfil="+nfil)
+
+ do nf=0,nfil-1
+ print("===========================")
+ f = addfile(diri+fili(nf), "r")
+ Latitude = f->/OLR/OLR_Dataset.Latitude
+ Longitude = f->/OLR/OLR_Dataset.Longitude
+
+ print("---")
+ printVarSummary(Latitude) ; [DIM_000 | 250601]
+ print("Latitude: min="+min(Latitude)+" max="+max(Latitude))
+ print("---")
+ printVarSummary(Longitude) ; [DIM_000 | 250601]
+ print("Longitude: min="+min(Longitude)+" max="+max(Longitude))
+ print("---")
+ ;print(Latitude+" "+Longitude)
+ print("---")
+
+ gp = f->GP_PARAM_INFO ; read group
+ print(gp)
+ print("---")
+
+ latS = gp at ValidBottomLat
+ latN = gp at ValidTopLat
+ lonL = gp at ValidLeftLon
+ lonR = gp at ValidRightLon
+
+ dlat = gp at LatInterval
+ nlat = toint((latN-latS)/dlat)+1
+ lat = fspan(latS, latN, nlat)
+ lat!0 = "lat"
+ lat at units = "degrees_north"
+
+ dlon = gp at LonInterval
+ mlon = toint((lonR-lonL)/dlon)+1
+ lon = fspan(lonL, lonR, mlon)
+ lon!0 = "lon"
+ lon at units = "degrees_east"
+
+ ;OLR = f->/OLR/OLR_Dataset.OLR
+ ;printVarSummary(OLR) ; [DIM_000 | 250601]
+
+ olr = onedtond( f->/OLR/OLR_Dataset.OLR, (/nlat,mlon/) )
+ olr!0 = "lat"
+ olr!1 = "lon"
+ olr&lat = lat
+ olr&lon = lon
+ ;;olr at long_name = gp at GP_PARAM_NAME
+ ;;olr at units = gp at OLR_Unit
+ olr at long_name = "OLR"
+ olr at units = "W/m^2"
+
+ print("typeof(gp at MissingValueInProduct="+typeof(gp at MissingValueInProduct))
; type double
+
+ olr at _FillValue= tofloat(gp at MissingValueInProduct) ; match olr type
+
+ printVarSummary(olr)
+ print("olr: min="+min(olr)+" max="+max(olr))
+ print("---")
+
+ ;************************************************
+ ; Examine distribution
+ ;************************************************
+
+ opt = True
+ opt at PrintStat = True
+ stat = stat_dispersion(olr, opt )
+
+ print("---")
+ fbase = str_get_field(fili(nf),1,".")
+ print("fbase="+fbase)
+
+ diro = "./"
+ filo = fbase+".nc"
+ ptho = diro+filo
+
+ system("/bin/rm -f "+ptho) ; remove any pre-existing file
+ ncdf = addfile(ptho ,"c") ; open output netCDF file
+
+ ; create global attributes
of the file
+ fAtt = True ; assign file attributes
+ fAtt at title = "NCL: H5 to netCDF"
+ fAtt at source = "MOSDAC: Meteorological & Oceanographic
Satellite Data Archival Center (India)"
+ fAtt at source_file = fili(nf)
+ fAtt at Conventions = "None"
+ fAtt at creation_date = systemfunc ("date")
+ fileattdef( ncdf, fAtt ) ; copy file attributes
+
+ ncdf->OLR = olr
+ end do
+ end
(0) nfil=31
(0) ===========================
(0) ---
Variable: Latitude
Type: float
Total Size: 1002404 bytes
250601 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_000 | 250601]
Coordinates:
Number Of Attributes: 0
(0) Latitude: min=-60 max=60
(0) ---
Variable: Longitude
Type: float
Total Size: 1002404 bytes
250601 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_000 | 250601]
Coordinates:
Number Of Attributes: 0
(0) Longitude: min=10 max=140
(0) ---
(0) ---
Variable: gp
Type: none
Total Size: 0 bytes
1 values
Number of Dimensions: 0
Dimensions and sizes:
Coordinates:
Number Of Attributes: 59
BinningInput_01 : K1VHR_05OCT2013_0000
.
.
.
BinningInput_44 : K1VHR_05OCT2013_2330
BinningTime : 2330R_05OCT2013_2330
GP_PARAM_DESCRIPTION : Daily_05OCT2013_2330
GP_PARAM_NAME : Outgoing Longwave Radiation (OLR)
Input_Channels : TIR
LatInterval : 0.25
Latitude_Unit : Degrees
LonInterval : 0.25
Longitude_Unit : Degrees
MissingValueInProduct : 999
NoOfBinningInputs : 44
OLR_Unit : Watts/sq. met.
ValidBottomLat : -60
ValidLeftLon : 10
ValidRightLon : 140
ValidTopLat : 60
(0) ---
(0) typeof(gp at MissingValueInProduct=double
Variable: olr
Type: float
Total Size: 1002404 bytes
250601 values
Number of Dimensions: 2
Dimensions and sizes: [lat | 481] x [lon | 521]
Coordinates:
lat: [ -60.. 60]
lon: [ 10.. 140]
Number Of Attributes: 3
_FillValue : 999
units : W/m^2
long_name : OLR
(0) olr: min=110.786 max=343.597
(0) ---
(0)
(0) ===> Robust Dispersion Statistics: OLR <===
(0) [0] Mean=254.263
.
.
.
(0) [29] Kurtosis=-0.392987
(0)
(0) ---
(0) fbase=K1VHR_05OCT2013_DAILY_L03_OLR
(0) ===========================
(0) ---
Variable: Latitude
Type: float
Total Size: 1002404 bytes
250601 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_000 | 250601]
Coordinates:
Number Of Attributes: 0
(0) Latitude: min=-60 max=60
(0) ---
Variable: Longitude
Type: float
Total Size: 1002404 bytes
250601 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_000 | 250601]
Coordinates:
Number Of Attributes: 0
(0) Longitude: min=10 max=140
(0) ---
(0) ---
Segmentation fault (core dumped)
**************************************************************************
regards
Paul.
On 2/24/15, Kyle Griffin <ksgriffin2 at wisc.edu> wrote:
> Look at what the error message is telling you. In this case, it says
> "variable OLR1 is undefined" - which is your problem. OLR1 does not exist
> anywhere else in your script! It appears you are trying to write it out to
> the output file correctly, but there is no data to output because you never
> read that variable in!
>
> Some info on reading (and writing, but that's not relevant for you at the
> moment) HDF5 files is available here:
>
> http://www.ncl.ucar.edu/Applications/hdf5.shtml
>
> It is not the most organized of pages, but you can get the general gist of
> things if you have a more complicated HDF5 file with groups. Otherwise,
> note that the typical notation for reading in a variable from file(s) is
> like this, using your code from above:
>
> f = addfile(diri+fili(nf), "r") ;same as above
> OLR1 = f->OLR_var_name ;change to match the variable name desired from
> your file
>
> You may need/want other coordinate variables from the file as well. I don't
> have much experience with HDF5 myself, but if you have some issues, you can
> look into resolving them with some tips from the general file I/O page as
> well:
>
> http://www.ncl.ucar.edu/Applications/list_io.shtml
>
> Please respond back to the list if you have more questions!
>
>
> Kyle
>
> ----------------------------------------
> Kyle S. Griffin
> Department of Atmospheric and Oceanic Sciences
> University of Wisconsin - Madison
> Room 1421
> 1225 W Dayton St, Madison, WI 53706
> Email: ksgriffin2 at wisc.edu
>
> On Mon, Feb 23, 2015 at 10:56 PM, Steeven Paul Yerraguntla <
> steevenpaul at gmail.com> wrote:
>
>> Dear NCL users,
>> I wanted to convert number of OLR hdf(.h5) files to netcdf
>> files
>> and to get it done i used the following script and facing error at the
>> line
>> marked in the following script. I could not make out how to solve it.
>> Please help me in solving out it. Thanks in advance.
>>
>> =================================================================
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>> ;************************************************
>> begin
>> diri = "./"
>> diro = "./"
>>
>> fili = systemfunc("cd "+diri+" ; ls *.h5")
>> nfil = dimsizes(fili)
>> print("nfil="+nfil)
>>
>> do nf=0,nfil-1
>> print("===========================")
>> f = addfile(diri+fili(nf), "r")
>>
>> print("---")
>> fbase = str_get_field(fili(nf),1,".")
>> print("fbase="+fbase)
>>
>> diro = "./"
>> filo = fbase+".nc"
>> ptho = diro+filo
>>
>> system("/bin/rm -f "+ptho) ; remove any pre-existing file
>> ncdf = addfile(ptho ,"c") ; open output netCDF file
>>
>> ; create global attributes of
>> the file
>> fAtt = True ; assign file attributes
>> fAtt at title = "NCL: H5 to netCDF"
>> fAtt at source = "MOSDAC: Meteorological & Oceanographic
>> Satellite Data Archival Center (India)"
>> fAtt at source_file = fili(nf)
>> fAtt at Conventions = "None"
>> fAtt at creation_date = systemfunc ("date")
>> fileattdef( ncdf, fAtt ) ; copy file attributes
>>
>> *ncdf->OLR = OLR1 <============== This is where error is
>> occuring*
>> end do
>> end
>> =====================================
>> *Error message as follows:*
>>
>> (0) nfil=31
>> (0) ===========================
>> (0) ---
>> (0) fbase=K1VHR_05OCT2013_DAILY_L03_OLR
>> fatal:Variable (OLR1) is undefined
>> fatal:["Execute.c":7743]:Execute: Error occurred at or near line 41 in
>> file nfiles_conversion_h5_to_nc.ncl
>>
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
--
*
***
More information about the ncl-talk
mailing list