[ncl-talk] How to resolve SVD warning: column contains constant
Dennis Shea
shea at ucar.edu
Thu Dec 17 09:19:37 MST 2015
FAQ: http://www.ncl.ucar.edu/FAQ/
Error messages
When I run my script, I'm getting a "fatal:NclMalloc Failed" error message.
When I run NCL, I get an error message about being unable to load the
"System Resource File".
When I run NCL, I get an error message about the X driver, like "error
opening display".
What's the deal with the error message "error: line -1 before or near..."?
How can I turn off NCL warning messages?
===
How can I turn off NCL warning messages?
FAQ: http://www.ncl.ucar.edu/FAQ/
Error messages
You can add the following code near the top of your NCL script:
err = NhlGetErrorObjectId()
setvalues err
"errLevel" : "Fatal" ; only report Fatal errors
end setvalues
This will turn off all "warning" type messages, so only do this if you
are feeling pretty confident about your code.
For more information about error reporting, see the documentation for
NhlGetErrorObjectId.
On Thu, Dec 17, 2015 at 7:00 AM, Bithi De <bde at purdue.edu> wrote:
> Hi All,
>
> Thanks for the reply Will!
>
> But is there any way to suppress the warning and let the code run? I'm
> having too many warning.
>
>
> I'm copying the 1st part of the code (before doing SVD) here, if anybody
> could help to clean it up! There is no error message but probably I did
> something wrong .
>
>
> Thanks in advance!
>
> Bithi
>
>
>
> ;********************************************************
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
> ;*********************************************************
> begin
>
> ;********************************
> ;read binary files
> ;********************************
> data_path = "/scratch/lustreD/b/bde/seaice_bin/"
> ;data_filename= "bt_197812_n07_v02_n.bin"
> nrows = 448
> ncols = 304
>
>
> ;************* read sic data
> *************************************************************************
> ; documentation of the data at
> http://nsidc.org/data/docs/daac/nsidc0079_bootstrap_seaice.gd.html
> ; 2-byte integer is NCL type "short"
> ; divide stored value by 10 to get the percent (SIC values stored as
> 0-1000)
> ; land is registered as 1200 andnorth pole as 1100
>
> ;***************************************************************************************************
>
> nrec=0
> Idims1 = (/448,304/)
>
>
> fils = systemfunc("ls /scratch/lustreD/b/bde/seaice_bin/*.bin")
>
> SIC = new((/dimsizes(fils),nrows,ncols/),”short") ; predefine array
> for data files
>
> do gg =0,dimsizes(fils)-1
> setfileoption("bin","ReadByteOrder","LittleEndian")
>
> SIC(gg,:,:) = fbindirread(fils(gg),nrec,dims1,"short")
> end do
>
> printVarSummary(SIC)
> printMinMax(SIC,True)
>
>
> SIC_f= short2flt(SIC)
> SIC_f at _FillValue = -999
> SIC_f = SIC_f/10
> printVarSummary(SIC_f)
> printMinMax(SIC_f,True)
>
>
> SIC_f = where(SIC_f .gt. 100,SIC_f at _FillValue,SIC_f)
> SIC_f at _FillValue = -999
> printVarSummary(SIC_f)
> printMinMax(SIC_f,True)
>
>
>
> ;*********************** read lat-lon for 25x25 km grid points in polar
> stereographic projection *
> ; documentation for the grid
> ;http://nsidc.org/data/polar_stereo/tools_geo_pixel.html#psn25_pss25_lats
> ; Grids that determine the latitude of a given pixel for the 25 km grids
> for either hemisphere
> ; 4-byte integer(little endian), scaled by 100,000
> ; Each array location (i, j) contains the latitude/longitude value at the
> center of the ;corresponding data grid cells
> ; 304 columns x 448 rows, range = [31.102, 89.836] for lats ; 304 columns
> x 448 rows, range = ;[-180.000, 179.814] for lons
>
> ;****************************************************************************************************
>
> setfileoption("bin","ReadByteOrder","LittleEndian")
> lat =
> fbindirread("/scratch/lustreD/b/bde/seaice_bin/psn25lats_v3.dat",0,(/nrows,ncols/),
> "integer") /100000.
> lon =
> fbindirread("/scratch/lustreD/b/bde/seaice_bin/psn25lons_v3.dat",0,(/nrows,ncols/),
> "integer") /100000.
>
> lat at units = "degrees_north"
> lon at units = "degrees_east"
> printVarSummary(lon)
> printVarSummary(lat)
>
> print(max(lon)+" " +min(lon))
> print(max(lat)+" " +min(lat))
>
> SIC_f at lat2d = lat
> SIC_f at lon2d = lon
>
> printVarSummary(SIC_f)
> printMinMax(SIC_f,True)
>
> ;************* remove mean ****************8
> ice_ano= dim_rmvmean_n_Wrap(SIC_f,0) ;timexlatxlon
> printVarSummary(ice_ano)
> printMinMax(ice_ano,True)
>
> if(any(ismissing(ice_ano))) then
> print("Your data contains some missing values. Beware.")
> end if
>
>
> ;; SVD will not contain any data with missingvalue so change fillvalue to
> 0.0
> ;; http://www.ncl.ucar.edu/Support/talk_archives/2013/3756.html
>
>
> ice_ano at _FillValue=0.0
> delete(ice_ano at _FillValue)
> printVarSummary(ice_ano)
>
> ;**************************************************
> ; split into month and year
> ;********************************************************
> ice_ym = new((/12,35,448,304/),"float")
>
> m = 0
> y = 0
>
> do ii = 2,421
>
> ; print(ii)
> ; print(m)
> ; print(y)
>
> ice_ym(m,y,:,:) = ice_ano(ii,:,:)
>
> m = m+1
>
> if (m .eq. 12)
> y = y+1
> m = 0
> end if
>
> end do
> printVarSummary(ice_ym)
>
>
> ice_Jan=ice_ym(0,:,:,:)
> copy_VarCoords(ice_ym(0,0,:,:),ice_Jan)
> printVarSummary(ice_Jan)
>
> ice_Feb=ice_ym(1,:,:,:)
> copy_VarCoords(ice_ym(0,0,:,:),ice_Feb)
> printVarSummary(ice_Feb)
>
> ice_Dec=ice_ym(11,:,:,:)
> copy_VarCoords(ice_ym(0,0,:,:),ice_Dec)
> printVarSummary(ice_Dec)
>
>
> ice_Wint = new((/3,35,448,304/),"float")
> ice_Wint(0,:,:,:)=ice_Dec
> ice_Wint(1,:,:,:)=ice_Jan
> ice_Wint(2,:,:,:)=ice_Feb
> printVarSummary(ice_Wint)
> printMinMax(ice_Wint,True)
>
> ice_djf=dim_avg_n_Wrap(ice_Wint,0)
> printVarSummary(ice_djf)
> printMinMax(ice_djf,True)
>
> if(any(ismissing(ice_djf))) then
> print("Your data contains some missing values. Beware.")
> end if
>
>
>
> ;***********************************************************************************************************
> ; read Z500 data and remove climatology for each winter month
>
> ;***********************************************************************************************************
> in =
> addfile("/scratch/lustreD/b/bde/reanalysis/ERA_data/ERA_z500_lowreso.nc","r")
>
> lat1 = in->latitude
> lon1 = in->longitude
> printVarSummary(lat1)
> printVarSummary(lon1)
>
> latz=lat1({90:20})
> l1 = dimsizes(latz)
>
>
> z500 = (short2flt(in->z)) ;m2/s2 geopotential
> printVarSummary(z500)
> printMinMax(z500,True)
> z5 = (z500(0,0,:)/9.8) ;geopotential height
>
> copy_VarCoords(z500,z5)
> printVarSummary(z5)
> printMinMax(z5,True)
>
> z_35 =z5(2:421,:,:); ;timesteps for 1979-2013, 35 years
> printVarSummary(z_35)
> printMinMax(z_35,True)
>
> zClm = clmMonTLL( z_35 ) ; monthly climatology (12xlatxlon)
> printVarSummary(zClm)
> printMinMax(zClm,True)
>
> ;************* remove monthly climatology and calculate anomaly for each
> month *************
> zAno= calcMonAnomTLL (z_35,zClm)
> printVarSummary(zAno)
> printMinMax(zAno,True) ;time(420=35*12)xlatxlon for zanomaly
>
>
> if(any(ismissing(zAno))) then
> print("Your data contains some missing values. Beware.")
> end if
>
>
> ;; SVD will not contain any data with missingvalue so change fillvalue to
> 0.0
> zAno at _FillValue=0.0
> delete(zAno at _FillValue)
> printVarSummary(zAno)
> printMinMax(zAno,True)
>
> if(any(ismissing(zAno))) then
> print("Your data contains some missing values. Beware.")
> end if
>
>
> ;********************* split in monthsxyear of the z-anomaly
> ***********************************
> z_ym = new((/12,35,dimsizes(lat1),dimsizes(lon1)/),"float")
>
> m = 0
> y = 0
>
> do ii = 0,419
>
> z_ym(m,y,:,:) = zAno(ii,:,:)
>
> m = m+1
>
> if (m .eq. 12)
> y = y+1
> m = 0
> end if
> end do
>
> printVarSummary(z_ym)
> printMinMax(z_ym,True)
>
>
> z_Jan=z_ym(0,:,:,:)
> printVarSummary(z_Jan)
>
> z_Feb=z_ym(1,:,:,:)
> printVarSummary(z_Feb)
>
> z_Dec=z_ym(11,:,:,:)
> printVarSummary(z_Dec)
>
>
> z_Wint = new((/3,35,dimsizes(latz),dimsizes(lon1)/),"float")
> z_Wint(0,:,:,:)=z_Dec(:,{90:20},:)
> z_Wint(1,:,:,:)=z_Jan(:,{90:20},:)
> z_Wint(2,:,:,:)=z_Feb(:,{90:20},:)
> printVarSummary(z_Wint)
> printMinMax(z_Wint,True)
>
> z_djf=dim_avg_n_Wrap(z_Wint,0)
> z_djf!0="time"
> printVarSummary(z_djf)
> printMinMax(z_djf,True)
>
> if(any(ismissing(z_djf))) then
> print("Your data contains some missing values. Beware.")
> end if
>
>
> ;***********************************************************
> ; calculate SVD
> ;***************************************** *******
>
> ;posted before
>
>
> end
>
>
> ________________________________
> From: Will Hobbs <will.hobbs at utas.edu.au>
> Sent: Wednesday, December 16, 2015 6:29 PM
> To: Bithi De; ncl-talk at ucar.edu
> Subject: Re: [ncl-talk] How to resolve SVD warning: column contains constant
>
> Bithi
>
> You will have constant values in the SIC array anywhere that is a) land, b)
> perenially ice covered or c) never ice covered. Any of these conditions are
> physically plausible for some data points in either the Arctic or Southern
> Ocean, and you would get the same warning with a regular cross-correlation
> using escort()
>
> In short, it's not (necessarily) a problem and is encountered in almost all
> gridded ocean data.
>
> Will
>
> From: Bithi De <bde at purdue.edu>
> Date: Thursday, 17 December 2015 10:19 AM
> To: "ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
> Subject: [ncl-talk] How to resolve SVD warning: column contains constant
>
> Hi All,
>
> I am calculating SVD between seaice concentration anomaly (documentation of
> the data at
> http://nsidc.org/data/docs/daac/nsidc0079_bootstrap_seaice.gd.html ) and
> geopotential height anomaly from ERA-interim data, for winter(35years). I am
> not getting any error until the step for calculating SVD.
>
> The part of my code is as follows:
>
> ;***********************************************************
> ; calculate SVD
> ;**************************************************
> ; Z-anomaly (timexlatxlon)
>
> nxy = dimsizes(lon1)*dimsizes(latz) ;size(space)
> ntim = 35 ;size(time)
>
> ; reorder so time varying fast
> ; create spacextime matrix for Z_ano
>
>
> Z = z_djf(longitude|:,latitude|:,time|:)
> printVarSummary(Z)
> Z2 = onedtond(ndtooned(Z),(/nxy,ntim/)) ;convert into spacextime matrix
> printVarSummary(Z2)
> printMinMax(Z2,True)
>
> Z2!0="space"
> Z2!1="time"
>
>
> Z2 at _FillValue=0.0
> delete(Z2 at _FillValue)
> printVarSummary(Z2)
> printMinMax(Z2,True)
>
> if(any(ismissing(Z2))) then
> print("Your data contains some missing values. Beware.")
> end if ; did not get any error
>
> ;********* SIC anomaly *********************
> ; create spacextime matrix for sic_ano(timex448x304) ; 448rows
> x304columns in 25kmx25km grid
>
>
> xy=448*304
> si = ice_djf(::-1,::-1,::-1) ;dimension
> reordering -->448x304xtime
> printVarSummary(si)
>
>
> Si2 = onedtond(ndtooned(si),(/xy,ntim/)) ;convert into
> (spacextime)
> printVarSummary(Si2)
> printMinMax(Si2,True)
>
> Si2!0="space"
> Si2!1="time"
>
> Si2 at _FillValue=0.0
> delete(Si2 at _FillValue)
> printVarSummary(Si2)
> printMinMax(Si2,True)
>
> if(any(ismissing(Si2))) then
> print("Your data contains some missing values. Beware.")
> end if
> ; did not get any error
>
>
> ; check if some columns has constant value
> p2d=Si2(time|:,space|:)
> printVarSummary(p2d)
>
> p2d_dims = dimsizes(p2d) ; I assume this is time x p1d
>
> ntim = p2d_dims(0)
> np1d = p2d_dims(1)
>
> ;do i=0,np1d-1
> ;pmin = min(p2d(:,i))
> ; pmax = max(p2d(:,i))
> ;if(pmin.eq.pmax) then
> ; print("p2d contains all constant values at np1d = " + i)
> ;end if
> ;end do
> exit
>
> ; delete([/Z,Zano,SIC,si/])
>
> ;***************************************************
> ;spatial plot of 1st mode of left and right singular vector
> ;*************************************************************
>
> nsvd = nxy
> svLeft = new((/nsvd,nxy/),float) ; pre-allocate space
> svRight = new((/nsvd,xy/),float)
>
> pc = svdcov_sv(Z2,Si2,nsvd,svLeft,svRight)
>
> printVarSummary(pc)
>
> end
>
> ;; giving warning ==> all Y values in columns are missing or constant
>
> *********
>
> I checked for Si2 following the
> http://www.ncl.ucar.edu/Support/talk_archives/2011/2803.html and it is shows
> it has constant value.
>
> Could you suggest how I can solve it?
>
>
> Thanks in advance!
>
> Bithi De
>
> Purdue University
>
>
>
>
>
>
> University of Tasmania Electronic Communications Policy (December, 2014).
> This email is confidential, and is for the intended recipient only. Access,
> disclosure, copying, distribution, or reliance on any of it by anyone
> outside the intended recipient organisation is prohibited and may be a
> criminal offence. Please delete if obtained in error and email confirmation
> to the sender. The views expressed in this email are not necessarily the
> views of the University of Tasmania, unless clearly intended otherwise.
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
More information about the ncl-talk
mailing list