<div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Hi Anahita,<br><br></div>THX for ftp'ing the stability file.<br><br></div>I've attached two simple scripts. <br></div> (a) reg_simple.ncl_1: <br> Create area weighted (30N-90N) time series of 'CLDLOW' and 'theta_ANN'. These are input to the 'reg_multlin_stats' function. <br></div> (b) reg_simple.ncl_:<br></div> Same as (a) but masks out all non-land locations.<br><br></div><div>So, input to the reg_multlin_stats' function are just two time series with no missing data.<br></div><div><br>For fun: I have added a very basic 2-dimensional PDF to each. <br><br>===<br></div>Re: Your offline question about your use of the ttest within 'SDE_significance.ncl'<br><br></div>That is a rather large script so I skimmed throgh it. I think you use of 'ttest' is correct.<br><br>===<br></div>A word of caution:<br><br></div>The 4-dimensional model data are on hybrid levels: (time,lev,lat,lon)<br></div>The '<a href="http://omega.nc">omega.nc</a>' variable is on isobaric levels: (time,lev,lat,lon)<br><br></div>Just to be explicitly clear about the vertical coordinate, even though the dimension names are the same the are very different variables.<br><br></div>Hybrid levels are a function of surface pressure at each grid point. At a particular location:<br>
<i>p(k) = a(k)*p0 + b(k)*psfc<br><br></i></div>The 'lev' coordinate variable for the model variables is a 'nominal' level. It is the pressure level **if** the surface pressure was 1000 hPa. It is just a placeholder. Think if psfc was say 732 hPa. <br><br>===<br></div>Good Luck<br></div>D<br><div><div><i></i><div><div><div><br><br><div><div><div><div><div><div><div><div><div><h2 id="gmail-:11t" class="gmail-hP" tabindex="-1"><br></h2><div><div><br><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 24, 2018 at 10:44 PM, Dennis Shea <span dir="ltr"><<a href="mailto:shea@ucar.edu" target="_blank">shea@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hello,<br><br></div>You did ftp most of the files. However, the 'stability_E_total.nc' file is not present.<br><br>---<br></div>Furher, offline you sent two scripts. One was a "simple script"; the other was (let's say) not so simple'.<br><br><br>====<br>[1]<br></div><span class="">"my code is very simple:"<br><br></span></div>I edited your "simple code" to make it as simple as possible<br><div><div> <wbr> ; **edited code**<br></div><div><span class=""><div>ana=addfile("clouds_E_total.nc<wbr>","r")</div></span><div>CLDLOW=ana->CLDLOW <wbr> ; (180,96,144)<br></div><div>printVarSummary(CLDLOW)<br></div></div><div><br></div><div><div>ana=addfile("stability_E_<a href="http://total.nc" target="_blank">total<wbr>.nc</a>","r") ; missing file ... not on ftp site<br></div><div>theta_ANN=ana->theta_ANN<br>printVarSummary(theta_ANN) <wbr> ; (???, ?96?,?144?) <=== missing file</div><div><br></div></div><div><div>CLDLOW_1d=ndtooned(CLDLOW)</div><div>theta_ANN_1d=ndtooned(theta_AN<wbr>N) ; is this the same shape & size as CLDLOW????<br><br>print("nmsg_CLDLOW="+num(<wbr>ismissing(CLDLOW_1d))) <wbr> ; =0<br>print("nmsg_theta_ANN="+num(<wbr>ismissing(theta_ANN_1d)))<br><br></div><div>; the following inputs 180*96*180 values for each variable.<br></div><div>; Is that what you want to do???<br></div><span class=""><div><br></div><div>opt = True</div><div>opt@print_anova = True ; optional</div><div>opt@print_data = True</div><div>b = reg_multlin_stats(CLDLOW_1d,th<wbr>eta_ANN_1d,opt) ; partial regression coef</div></span></div><div><br> <br><div>Not sure what to say. Does variable theta_ANN have 180 time steps? <br>The _ANN (?Annual?) suffix would imply to me that these are annual values rather than monthly values.<br></div><div>If this is the case you would have to make the CLDLOW variables also be Annual values.<br></div><div>The dimensionalities must match.<br><br></div><div>You said that the stability has missing values. The dimensionalites MUST match . If so, <br></div><div>you can use the following two step approacj<br><br></div><div>(a) <br>CLDLOW@_FillValue = theta_ANN@_FillValue<br>CLDLOW = where(ismissing(theta_ANN), CLDLOW@_FillValue, CLDLOW)<br><br></div><div>(b) <br></div><div>The way you have the independent and dependent variables as 1D would weight grid points near the poles the same as values at low latitudes. I do *not* think this is appropriate. High latitude grid points represent much smaller areas.<br></div><div><br></div><div>After step (a) you should compute weighted areal averages<br></div><div><br></div><div><a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/wgt_areaave_Wrap.shtml" target="_blank">http://www.ncl.ucar.edu/<wbr>Document/Functions/<wbr>Contributed/wgt_areaave_Wrap.<wbr>shtml</a><br><br></div><div>The files you sent do not have any gaussian weights [ gwt ] so you could the cos(rad*lat) <br></div><div>or<br></div><div><a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/latGauWgt.shtml" target="_blank">http://www.ncl.ucar.edu/<wbr>Document/Functions/<wbr>Contributed/latGauWgt.shtml</a><br><br></div><div>for the wgty.<br><br></div><div>The 'wgt_areaave' function allows missing values.<br></div><div><br></div><div>This will yield one global time series which is what I think you want.<br><br>=====================<br><br></div><div>Your 2nd offline script is quite a bit more complicated. Also, it looks like you only want latitudes north of 30N. Correct? I'd suggest the following approach. It will make for much smaller arrays.<br><br>diri = "./"<br>diri = "/project/cas/shea/netCDF_<wbr>Misc/"<br><br>latStrt = 30.0<br>latLast = 90.0<br><br>ana=addfile(diri+"clouds_E_<wbr><a href="http://total.nc">total.nc</a>","r") ; *hybrid* levels<br>CLDLOW=ana->CLDLOW(:,{latStrt:<wbr>latLast},:) <wbr> <br>printVarSummary(CLDLOW) <wbr> ; [time | 180] x [lat | 32] x [lon | 144]<br><br></div><div>print(CLDLOW&lat)<br><br>---<br></div>Your 2nd script is treating variable RH's hybrid levels to be mixed with omega's isobaric levels. In my opinion, this is not appropriate.<br><br></div><div>This script also mask out ocean areas. <br><br></div><div>You can use an approach like (b)<br><br></div><div> lsm = .... ; land-sea mask<br> <br></div><div>Then, use an approach where CLDLOW , theta_ANN are masked via (say)<br><br><a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/mask.shtml" target="_blank">http://www.ncl.ucar.edu/<wbr>Document/Functions/Built-in/<wbr>mask.shtml</a><br><br></div><div>Then do the global averges; then the regression.<br><br>----<br></div><div>Please when you send code to ncl-talk, it shoul be clean and structured. Please do not include numerous commented lines. It is tedious to read code that has been heavily commented.<br><br></div><div>Good luck<span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888"><div>D<br></div><div><br><br></div><div><br></div><div><div><br></div></div></font></span></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 10, 2018 at 7:55 PM, Dennis Shea <span dir="ltr"><<a href="mailto:shea@ucar.edu" target="_blank">shea@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>The "reg_multlin_stats" documentation <br> <a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/reg_multlin_stats.shtml" target="_blank">http://www.ncl.ucar.edu/Docume<wbr>nt/Functions/Contributed/reg_<wbr>multlin_stats.shtml</a><br>does state that but the code (located in contributed.ncl) has:<br></div><br>undef ("reg_multlin_stats")<br>function reg_multlin_stats(Y[*]:numeric<wbr>, XP:numeric, opt) ; should be float or double<br>;<br>; Nomenclature:<br>; Y - dependent variable (size NY)<br>; missing values (_FillValue) are not allowed. <=============<br>; XP - one [*] or more [*][*] independent variables.<br>; Missing values (_FillValue) are not allowed. <=============<br><br></div><div>[snip]<br><br> if (isatt(Y,"_FillValue") .and. any(ismissing(Y))) then<br> print("reg_multlin_stats: Y has missing values. Not allowed!")<br> exit<br> end if<br><br> if (isatt(XP,"_FillValue") .and. any(ismissing(XP))) then<br> print("reg_multlin_stats: XP has missing values. Not allowed!")<br> exit<br> end if<br></div><div>[snip]<br></div><div>==========<br><br></div><div>Still, there might be a way to use the function. My thought is to manually loop ove ?grid points? skipping any points that have missing values.<br></div><div><br></div><div>How are you using it? Please be clear and concise.<br><br></div><div>I am gone through 22 Jan 2018 so maybe someone else can help.<br></div><div><br><br><br></div><div> <br></div><div><br></div><div> </div><div><br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_72368654761911000h5">On Wed, Jan 10, 2018 at 4:00 PM, Anahita Amiri Farahani via ncl-talk <span dir="ltr"><<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_72368654761911000h5"><div dir="ltr">Dear all,<div><br></div><div>I wanted to use reg_multlin_stats to calculate linear regression between low cloud and stability, the low cloud and stability are my dependent and independent variables. Stability is calculated based on the potential temperature at the surface and 700 hPa. So we have missing values for some areas. When I read the description of this function, it says <span style="color:rgb(51,51,51);font-family:verdana,sans-serif;font-size:13.3333px">While missing values are allowed, it is recommended that users not input any missing independent values. It just confuses the results. Missing values should be indicated by the </span><a href="https://www.ncl.ucar.edu/Document/Language/fillval.shtml" style="margin:0px;padding:0px;text-decoration-line:none;color:rgb(133,45,133);font-family:verdana,sans-serif;font-size:13.3333px" target="_blank">_FillValue</a><span style="color:rgb(51,51,51);font-family:verdana,sans-serif;font-size:13.3333px"> attribute.</span></div><div><br></div><div><br></div><div>But when I use this function I got this error: <span style="font-family:Courier">reg_multlin_stats: XP has missing values. Not allowed!</span></div><div><span style="font-family:Courier"><br></span></div><div>and it popped out from NCL, Any suggestions? <br></div><div><br></div><div>Thanks,</div><div>Ana</div>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:12pt;font-family:Cambria"><span style="font-size:19pt"><span></span></span></p>
</div>
<br></div></div>______________________________<wbr>_________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailma<wbr>n/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>