<div dir="ltr">Hi Sri,<div>The error messages you are receiving are very likely due to equiv_sample_size returning sample sizes less than 3 (as the message states). It is your choice whether to use equiv_sample_size or not. I personally always use it, as you generally do not want to include points with very low sample sizes as part of your t-test plots. Did you examine your probt array to see what the range of values is? Are there any values greater than 95%? You can use <a href="https://www.ncl.ucar.edu/Document/Functions/Contributed/stat_dispersion.shtml">stat_dispersion</a> to help you with this. You are only attempting to plot those values higher than 99%, so there's a chance that no point reaches that threshold.</div><div><br></div><div>For the record: I don't see anything amiss with your coding. </div><div>If you have any further questions please respond to ncl-talk.</div><div>Adam </div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jun 18, 2018 at 3:27 AM Sri Nandini <<a href="mailto:snandini@marum.de" target="_blank">snandini@marum.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hello All<br><br>I am using the ttest routine on monthly climatology, and plotting only those
values that are significant to the 95% level but the error i get is <br><br>warning:ttest: encountered 4692 cases where s1 and/or s2 were less than 2. Output set to missing values in these cases<br><br>Upon various searches online i came to understand it means: <br><br>This message just indicates that you have spatial locations in your
input arrays which have no non-missing values, and the 'ttest' function
will set the output to missing as well.
<br>Does this mean performing ttest on monthly climatologies or getting the equiv_sample_size is not a good idea?<br><br>In the end my plot doesn't contain any stippling dots for ttest values.<br><br>my coding looks something
like this:<br><br><br>f= addfile("Total_eva_PI.nc", "r") ;<br>T41   = f->QFLUX(:,:,:)<br>T41@_FillValue = -9.96921e+36                       ; (12,lat,lon) <br>printVarSummary(T41)                <br>                             <br>aveX   = dim_avg_n_Wrap(T41,0)                     ; (lat,lon)<br>printVarSummary(aveX)                               <br><br>varX = dim_variance_n_Wrap(T41,0)                   ; compute variance the 0th dim<br>printVarSummary(varX)                               ; (lat,lon) <br><br>;=================================================================================<br> yrStrt = 0950<br> yrLast = 1000<br><br>f1 = addfile("Total_eva_9ka.nc", "r") ;<br>TIME1 = f1->time<br>TIME1 = TIME1-31<br><br> YYYY1  = cd_calendar(TIME1,-1)/100                ; entire file<br> iYYYY1 = ind(YYYY1.ge.yrStrt .and. YYYY1.le.yrLast)<br><br>air22  = f1->QFLUX(iYYYY1,:,:)<br>air22@_FillValue = -9.96921e+36  <br>                                   <br>T2M_4 = clmMonLLT(air22(lat|:,lon|:,time|:))           ; monthly climatology<br>printVarSummary(T2M_4)                                 ; (12,lat,lon) <br><br>aveY   = dim_avg_n_Wrap(T2M_4,2) <br>printVarSummary(aveY)                                  ; (lat,lon)<br><br>varY = dim_variance_n_Wrap(T2M_4,2) ; compute variance<br>printVarSummary(varY)                                  ; (lat,lon) <br><br><br><br><br><br>;================================================================================<br>;equiv_sample_size: Estimates the number of independent values of a series of correlated observations. Specify a critical significance level to test the lag-one auto-correlation coefficien. Use annual means.<br>;(Generally, there is no significant year-to-year autocorrelation of monthly data [e.g. successive Januaries].) So here, xX and sY are scalars indicating the number of years used.<br>;=================================================================================<br> sigr = 0.05                                       ; critical sig lvl for r<br><br> xEqv = new(dimsizes(aveY),typeof(aveY),aveY@_FillValue)<br>printVarSummary(xEqv)                                ; (lat,lon) <br><br> xEqv = equiv_sample_size (T41(lat|:,lon|:,time|:), sigr,0)  <br> copy_VarMeta(aveY,xEqv)<br>printVarSummary(xEqv)                                ; (lat,lon) <br><br> yEqv = new(dimsizes(aveY),typeof(aveY),aveY@_FillValue)<br>printVarSummary(yEqv) <br>                             ; (lat,lon) <br> yEqv = equiv_sample_size (T2M_4(lat|:,lon|:,month|:), sigr,0)<br>copy_VarMeta(aveY,yEqv)<br>printVarSummary(yEqv)                               ; (lat,lon) <br><br>;==================================================================================<br>;Use "ttest" to compute the probabilities <br>;==================================================================================<br> probt = new(dimsizes(varY),typeof(varY),varY@_FillValue)<br><br>       iflag= False                                ; population variance similar?<br><br>probt = 100.*(1. -ttest(aveY,varY,xEqv,aveX,varX,yEqv,iflag,False)) ; all areas whose value > 95 are significant<br>;printVarSummary(probt)                            ; (lat,lon) <br><br>copy_VarMeta(varY,probt)<br>printVarSummary(probt)                             ; (lat,lon) <br> printMinMax(probt,0)                              ; min=0.0001486018726404303  max=100<br><br>;====================================================================<br>;calculate the T2M difference <br>;==================================================================== <br>T2diff = new(dimsizes(aveY),typeof(aveY),aveY@_FillValue)<br><br>copy_VarMeta(aveY,T2diff)<br><br>T2diff=aveY-aveX<br><br><br>outfile = addfile("E_9_Diff.nc","c")   ;<br>outfile->T1diff = T2diff<br><br>;====================================================================<br>;Here is a section of my code that draws a color fill plot, and then<br>;overlays contours of statistical significance.<br>;====================================================================<br> wks = gsn_open_wks("pdf","QFLUX_ttest_9ka_PI99%1")<br><br><br> res = True<br> res@cnFillOn            = True               ; turn on color<br> res@gsnSpreadColors     = True               ; use full colormap<br><br> res@cnLinesOn           = False              ; turn off contour lines<br><br> res@cnLevelSelectionMode = "ManualLevels" ; set manual contour levels<br> res@cnMinLevelValF      = -4           ; set min contour level<br> res@cnMaxLevelValF      = 4           ; set max contour level<br> res@cnLevelSpacingF     = .5           ; set contour spacing<br><br> res@gsnDraw             = False          ; Do not draw plot<br> res@gsnFrame            = False          ; Do not advance frome<br><br> res@tiMainString        = "QFLUX: 9ka-PI"<br> res@tiMainOffsetYF         = 0.02  ; Move title up a little<br> res@gsnCenterString     = " "<br> res@gsnLeftString       = " "<br> res@gsnRightString       = " "<br><br> res@cnRasterModeOn             = True              ; Raster mode shows grid cells<br><br> res@lbBoxEndCapStyle    = "TriangleBothEnds"       ; Added in NCL V6.4.0<br> res@lbTitlePosition     = "bottom"             ; title position<br> res@lbTitleFontHeightF  = .01               ; make title smaller<br> res@lbTitleDirection    = "Across"            ; title direction<br> res@lbLabelFontHeightF = 0.01<br><br> minlat = 30.                         ; min lat to mask<br> maxlat = 80.                         ; max lat to mask<br> minlon = -70.                         ; min lon to mask<br> maxlon = 63.                         ; max lon to mask<br><br> res@mpProjection = "LambertConformal"           ; choose projection<br><br>;---masked plot<br> res@gsnAddCyclic = True               ; regional plot<br><br> res@mpMinLatF = minlat             ; min lat to mask<br> res@mpMaxLatF = maxlat             ; max lat to mask<br> res@mpMinLonF = minlon             ; min lon to mask<br> res@mpMaxLonF = maxlon             ; max lon to mask<br><br> res@gsnMaskLambertConformal = True               ; turn on lc masking<br><br> plot = gsn_csm_contour_map(wks,T2diff,res)<br><br> add_lc_labels(wks,plot,minlat,maxlat,minlon,maxlon) ; attach latitude labels<br><br>;=====================================================================<br>;Second plot resources and overlay onto first<br>;=====================================================================<br> res2 = True                           ; res2 probability plots<br> res2@gsnDraw            = False      ; Do not draw plot<br> res2@gsnFrame           = False      ; Do not advance frome<br><br> res2@cnLevelSelectionMode = "ExplicitLevels"      ; set explicit cnlev<br> res2@cnLevels  = (/99./)             ; only have 1 contour level<br> res2@cnInfoLabelOn      = False<br> res2@cnLinesOn          = False      ; do not draw contour lines<br> res2@cnLineLabelsOn     = False      ; do not draw contour labels<br> res2@cnFillScaleF       = 0.6        ; add extra density<br><br> res2@gsnRightString           = ""               ; Turn off subtitles<br> res2@gsnLeftString            = ""                                <br> res@gsnAddCyclic               = True            ; add cyclic point<br> plot2  = gsn_csm_contour(wks,probt(:,:),res2) <br><br> opt = True                                        ; set up parameters for pattern fill<br> opt@gsnShadeFillType = "pattern"                  ; specify pattern fill<br> opt@gsnShadeHigh     = 17                        ; stipple pattern<br> opt@gsnShadeDotSizeF = 1                          ; make dots larger<br><br>;=====================================Attach the polylines<br>ADD_SHAPEFILE_OUTLINES = True<br> sname = "Caspian_Basin.shp"<br><br>  pres            = True<br>  pres@gsLineColor = "black"<br>  pres@gsLineThicknessF = 2.0           ; 3x thickness<br>   shp1 = gsn_add_shapefile_polylines(wks,plot,sname,pres) <br> printVarSummary(shp1)<br>;=====================================<br><br> plot2  = gsn_contour_shade(plot2,1,99., opt)     ; stipple all areas >= 90 contour      <br> overlay (plot, plot2)<br><br> draw (plot)<br> frame(wks)<br><br>end<br></div>
_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_1902559926005768331gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div><div><span><font color="#888888">Adam Phillips <br></font></span></div><span><font color="#888888">Associate Scientist, </font></span><span><font color="#888888">Climate and Global Dynamics Laboratory, NCAR<br></font></span></div></div><div><span><font color="#888888"><a href="http://www.cgd.ucar.edu/staff/asphilli/" target="_blank">www.cgd.ucar.edu/staff/asphilli/</a>  </font></span><span><font color="#888888">303-497-1726 </font></span></div><span><font color="#888888"></font></span><div><div><span><font color="#888888"><br></font></span><div><span><font color="#888888"><a href="http://www.cgd.ucar.edu/staff/asphilli" target="_blank"></a></font></span></div></div></div></div></div></div></div></div></div></div></div>