[ncl-talk] iteration over files
Mary Haley
haley at ucar.edu
Wed Feb 25 07:58:39 MST 2015
Paul,
The output from print(gp) says that "gp" has 0 bytes. This means that NCL
is failing when trying to read this variable:
Variable: gp
Type: none
Total Size: 0 bytes
1 values
Number of Dimensions: 0
Dimensions and sizes:
Coordinates:
Number Of Attributes: 59
I believe we have fixed this bug in the upcoming release of NCL. Would you
be able to provide us with the file so we can test it?
--Mary
On Tue, Feb 24, 2015 at 2:21 AM, Steeven Paul Yerraguntla <
steevenpaul at gmail.com> wrote:
> Dear NCL users
> Thanks to Griffin for good suggestions and i could convert the hdf to
> nc with the script provided by Dennis long ago and the problems is the
> iteration is done only to the first file in the loop and the error
> occuring with the message "segmentation fault (core dumped)". I run
> the script again using the command ncl -x filename.ncl and the output
> as follows. Please let me know how can i get the iteration completed.
>
>
> ***********************************************************************************
> Copyright (C) 1995-2012 - All Rights Reserved
> University Corporation for Atmospheric Research
> NCAR Command Language Version 6.1.0
> The use of this software is governed by a License Agreement.
> See http://www.ncl.ucar.edu/ for more details.
> + ;************************************************
> + ; Allways examine file contents:
> + ; %> ncl_filedump foo.h5 | less
> + ;************************************************
> Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> +
> Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> +
> Loading file "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> +
> + ;************************************************
> + begin
> + diri = "./"
> + diroxx = "./"
> +
> + fili = systemfunc("cd "+diri+" ; ls *.h5")
> + nfil = dimsizes(fili)
> + print("nfil="+nfil)
> +
> + do nf=0,nfil-1
> + print("===========================")
> + f = addfile(diri+fili(nf), "r")
> + Latitude = f->/OLR/OLR_Dataset.Latitude
> + Longitude = f->/OLR/OLR_Dataset.Longitude
> +
> + print("---")
> + printVarSummary(Latitude) ; [DIM_000 |
> 250601]
> + print("Latitude: min="+min(Latitude)+" max="+max(Latitude))
> + print("---")
> + printVarSummary(Longitude) ; [DIM_000 |
> 250601]
> + print("Longitude: min="+min(Longitude)+" max="+max(Longitude))
> + print("---")
> + ;print(Latitude+" "+Longitude)
> + print("---")
> +
> + gp = f->GP_PARAM_INFO ; read group
> + print(gp)
> + print("---")
> +
> + latS = gp at ValidBottomLat
> + latN = gp at ValidTopLat
> + lonL = gp at ValidLeftLon
> + lonR = gp at ValidRightLon
> +
> + dlat = gp at LatInterval
> + nlat = toint((latN-latS)/dlat)+1
> + lat = fspan(latS, latN, nlat)
> + lat!0 = "lat"
> + lat at units = "degrees_north"
> +
> + dlon = gp at LonInterval
> + mlon = toint((lonR-lonL)/dlon)+1
> + lon = fspan(lonL, lonR, mlon)
> + lon!0 = "lon"
> + lon at units = "degrees_east"
> +
> + ;OLR = f->/OLR/OLR_Dataset.OLR
> + ;printVarSummary(OLR) ; [DIM_000 | 250601]
> +
> + olr = onedtond( f->/OLR/OLR_Dataset.OLR, (/nlat,mlon/) )
> + olr!0 = "lat"
> + olr!1 = "lon"
> + olr&lat = lat
> + olr&lon = lon
> + ;;olr at long_name = gp at GP_PARAM_NAME
> + ;;olr at units = gp at OLR_Unit
> + olr at long_name = "OLR"
> + olr at units = "W/m^2"
> +
> + print("typeof(gp at MissingValueInProduct
> ="+typeof(gp at MissingValueInProduct))
> ; type double
> +
> + olr at _FillValue= tofloat(gp at MissingValueInProduct) ; match olr type
> +
> + printVarSummary(olr)
> + print("olr: min="+min(olr)+" max="+max(olr))
> + print("---")
> +
> + ;************************************************
> + ; Examine distribution
> + ;************************************************
> +
> + opt = True
> + opt at PrintStat = True
> + stat = stat_dispersion(olr, opt )
> +
> + print("---")
> + fbase = str_get_field(fili(nf),1,".")
> + print("fbase="+fbase)
> +
> + diro = "./"
> + filo = fbase+".nc"
> + ptho = diro+filo
> +
> + system("/bin/rm -f "+ptho) ; remove any pre-existing file
> + ncdf = addfile(ptho ,"c") ; open output netCDF file
> +
> + ; create global attributes
> of the file
> + fAtt = True ; assign file attributes
> + fAtt at title = "NCL: H5 to netCDF"
> + fAtt at source = "MOSDAC: Meteorological & Oceanographic
> Satellite Data Archival Center (India)"
> + fAtt at source_file = fili(nf)
> + fAtt at Conventions = "None"
> + fAtt at creation_date = systemfunc ("date")
> + fileattdef( ncdf, fAtt ) ; copy file attributes
> +
> + ncdf->OLR = olr
> + end do
> + end
> (0) nfil=31
> (0) ===========================
> (0) ---
>
> Variable: Latitude
> Type: float
> Total Size: 1002404 bytes
> 250601 values
> Number of Dimensions: 1
> Dimensions and sizes: [DIM_000 | 250601]
> Coordinates:
> Number Of Attributes: 0
> (0) Latitude: min=-60 max=60
> (0) ---
>
> Variable: Longitude
> Type: float
> Total Size: 1002404 bytes
> 250601 values
> Number of Dimensions: 1
> Dimensions and sizes: [DIM_000 | 250601]
> Coordinates:
> Number Of Attributes: 0
> (0) Longitude: min=10 max=140
> (0) ---
> (0) ---
>
>
> Variable: gp
> Type: none
> Total Size: 0 bytes
> 1 values
> Number of Dimensions: 0
> Dimensions and sizes:
> Coordinates:
> Number Of Attributes: 59
> BinningInput_01 : K1VHR_05OCT2013_0000
> .
> .
> .
> BinningInput_44 : K1VHR_05OCT2013_2330
> BinningTime : 2330R_05OCT2013_2330
> GP_PARAM_DESCRIPTION : Daily_05OCT2013_2330
> GP_PARAM_NAME : Outgoing Longwave Radiation (OLR)
> Input_Channels : TIR
> LatInterval : 0.25
> Latitude_Unit : Degrees
> LonInterval : 0.25
> Longitude_Unit : Degrees
> MissingValueInProduct : 999
> NoOfBinningInputs : 44
> OLR_Unit : Watts/sq. met.
> ValidBottomLat : -60
> ValidLeftLon : 10
> ValidRightLon : 140
> ValidTopLat : 60
> (0) ---
> (0) typeof(gp at MissingValueInProduct=double
>
> Variable: olr
> Type: float
> Total Size: 1002404 bytes
> 250601 values
> Number of Dimensions: 2
> Dimensions and sizes: [lat | 481] x [lon | 521]
> Coordinates:
> lat: [ -60.. 60]
> lon: [ 10.. 140]
> Number Of Attributes: 3
> _FillValue : 999
> units : W/m^2
> long_name : OLR
> (0) olr: min=110.786 max=343.597
> (0) ---
> (0)
> (0) ===> Robust Dispersion Statistics: OLR <===
> (0) [0] Mean=254.263
> .
> .
> .
> (0) [29] Kurtosis=-0.392987
> (0)
> (0) ---
> (0) fbase=K1VHR_05OCT2013_DAILY_L03_OLR
> (0) ===========================
> (0) ---
>
> Variable: Latitude
> Type: float
> Total Size: 1002404 bytes
> 250601 values
> Number of Dimensions: 1
> Dimensions and sizes: [DIM_000 | 250601]
> Coordinates:
> Number Of Attributes: 0
> (0) Latitude: min=-60 max=60
> (0) ---
>
> Variable: Longitude
> Type: float
> Total Size: 1002404 bytes
> 250601 values
> Number of Dimensions: 1
> Dimensions and sizes: [DIM_000 | 250601]
> Coordinates:
> Number Of Attributes: 0
> (0) Longitude: min=10 max=140
> (0) ---
> (0) ---
> Segmentation fault (core dumped)
> **************************************************************************
>
> regards
> Paul.
>
> On 2/24/15, Kyle Griffin <ksgriffin2 at wisc.edu> wrote:
> > Look at what the error message is telling you. In this case, it says
> > "variable OLR1 is undefined" - which is your problem. OLR1 does not exist
> > anywhere else in your script! It appears you are trying to write it out
> to
> > the output file correctly, but there is no data to output because you
> never
> > read that variable in!
> >
> > Some info on reading (and writing, but that's not relevant for you at the
> > moment) HDF5 files is available here:
> >
> > http://www.ncl.ucar.edu/Applications/hdf5.shtml
> >
> > It is not the most organized of pages, but you can get the general gist
> of
> > things if you have a more complicated HDF5 file with groups. Otherwise,
> > note that the typical notation for reading in a variable from file(s) is
> > like this, using your code from above:
> >
> > f = addfile(diri+fili(nf), "r") ;same as above
> > OLR1 = f->OLR_var_name ;change to match the variable name desired from
> > your file
> >
> > You may need/want other coordinate variables from the file as well. I
> don't
> > have much experience with HDF5 myself, but if you have some issues, you
> can
> > look into resolving them with some tips from the general file I/O page as
> > well:
> >
> > http://www.ncl.ucar.edu/Applications/list_io.shtml
> >
> > Please respond back to the list if you have more questions!
> >
> >
> > Kyle
> >
> > ----------------------------------------
> > Kyle S. Griffin
> > Department of Atmospheric and Oceanic Sciences
> > University of Wisconsin - Madison
> > Room 1421
> > 1225 W Dayton St, Madison, WI 53706
> > Email: ksgriffin2 at wisc.edu
> >
> > On Mon, Feb 23, 2015 at 10:56 PM, Steeven Paul Yerraguntla <
> > steevenpaul at gmail.com> wrote:
> >
> >> Dear NCL users,
> >> I wanted to convert number of OLR hdf(.h5) files to netcdf
> >> files
> >> and to get it done i used the following script and facing error at the
> >> line
> >> marked in the following script. I could not make out how to solve it.
> >> Please help me in solving out it. Thanks in advance.
> >>
> >> =================================================================
> >> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> >> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> >> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> >> ;************************************************
> >> begin
> >> diri = "./"
> >> diro = "./"
> >>
> >> fili = systemfunc("cd "+diri+" ; ls *.h5")
> >> nfil = dimsizes(fili)
> >> print("nfil="+nfil)
> >>
> >> do nf=0,nfil-1
> >> print("===========================")
> >> f = addfile(diri+fili(nf), "r")
> >>
> >> print("---")
> >> fbase = str_get_field(fili(nf),1,".")
> >> print("fbase="+fbase)
> >>
> >> diro = "./"
> >> filo = fbase+".nc"
> >> ptho = diro+filo
> >>
> >> system("/bin/rm -f "+ptho) ; remove any pre-existing file
> >> ncdf = addfile(ptho ,"c") ; open output netCDF file
> >>
> >> ; create global attributes of
> >> the file
> >> fAtt = True ; assign file attributes
> >> fAtt at title = "NCL: H5 to netCDF"
> >> fAtt at source = "MOSDAC: Meteorological & Oceanographic
> >> Satellite Data Archival Center (India)"
> >> fAtt at source_file = fili(nf)
> >> fAtt at Conventions = "None"
> >> fAtt at creation_date = systemfunc ("date")
> >> fileattdef( ncdf, fAtt ) ; copy file attributes
> >>
> >> *ncdf->OLR = OLR1 <============== This is where error is
> >> occuring*
> >> end do
> >> end
> >> =====================================
> >> *Error message as follows:*
> >>
> >> (0) nfil=31
> >> (0) ===========================
> >> (0) ---
> >> (0) fbase=K1VHR_05OCT2013_DAILY_L03_OLR
> >> fatal:Variable (OLR1) is undefined
> >> fatal:["Execute.c":7743]:Execute: Error occurred at or near line 41 in
> >> file nfiles_conversion_h5_to_nc.ncl
> >>
> >> _______________________________________________
> >> ncl-talk mailing list
> >> List instructions, subscriber options, unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>
> >>
> >
>
>
> --
>
>
>
>
> *
> ***
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150225/47e943b4/attachment.html
More information about the ncl-talk
mailing list