<div dir="ltr">just to keep anyone from looking into this, I've found a solution that I'm happy with. If I don't remove the zero's, I actually get a cleaner signal:<div><br></div><div><div style="font-size:12.8px">;;;DON'T REMOVE ZEROS<br class="gmail-Apple-interchange-newline">;;;print("remove zeros")</div><div style="font-size:12.8px">;;;Xp@_FillValue = default_fillvalue("float")</div><div style="font-size:12.8px">;;;Yp@_FillValue = default_fillvalue("float")</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">;;;remove fillvalue attribute</div><div style="font-size:12.8px"> delete(Xp@_FillValue)</div><div style="font-size:12.8px"><span style="font-size:12.8px"> delete(Yp@_FillValue)</span></div><div style="font-size:12.8px"><br></div></div><div style="font-size:12.8px"><span style="font-size:12.8px"> lapack = svdcov_sv(Xp,Yp,nsvd,L,R)</span><br></div><div style="font-size:12.8px"><span style="font-size:12.8px"><br></span></div><div style="font-size:12.8px"><span style="font-size:12.8px">Gives me the same number of warnings as when I had missing values, because the same warning applies to a bunch of zeros as it does to a bunch of missing values....however, t</span><span style="font-size:12.8px">he results look a little cleaner when I don't use missing values.</span></div><div style="font-size:12.8px"><span style="font-size:12.8px"><br></span></div><div style="font-size:12.8px"><span style="font-size:12.8px">Adam</span></div><div style="font-size:12.8px"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, May 27, 2017 at 12:21 PM, Adam Herrington <span dir="ltr"><<a href="mailto:adam.herrington@stonybrook.edu" target="_blank">adam.herrington@stonybrook.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">the svdcov_sv documentation (<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/svdcov_sv.shtml" target="_blank">http://www.ncl.ucar.edu/<wbr>Document/Functions/Built-in/<wbr>svdcov_sv.shtml</a>) states that "This function does not allow for missing data." However, the function will not exit, but rather gives you a warning. For example, <div><br></div><div><div><div>print("remove zeros")</div><div> Xp@_FillValue = default_fillvalue("float")</div><div> Yp@_FillValue = default_fillvalue("float")</div><div><br></div><div> Xp = where(Xp.ne.0.,Xp,Xp@_<wbr>FillValue)</div><div> Yp = where(Yp.ne.0.,Yp,Yp@_<wbr>FillValue)</div><div> </div><div> lapack = svdcov_sv(Xp,Yp,nsvd,L,R)</div></div><div><br></div><div>I get about 4000 of these warnings: </div><div><div> "SVD: Warning all X values in column are missing or are constant"</div><div><br></div><div>Followed by about 1000 of these warnings</div><div> "SVD: Warning all Y values in column are missing or are constant"<br></div><div><br></div><div>I'm trying to figure out how much of the data are being ignored, because I have reason to believe the L and R singular vectors that come out of this example are using quite abit of the data still.</div><div><br></div><div>In the example, Xp and Yp are dimensioned (/ntime,ncols/), where ntime = 980 and ncols = 30*48602. Since the number of warnings is much larger than ntime, I don't know how to interpret what the warning is referring to when they say "column".</div></div></div><div><br></div><div>In trying to track down the code, I've scanned the entire directory $NCARG_ROOT/lib/ncarg/<wbr>nclscripts/csm/ and can't find any references to the wrappers used to create svdcov_sv.</div><div><br></div><div>The "eofunc" also uses lapack and the documentation (<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/eofunc.shtml" target="_blank">http://www.ncl.ucar.edu/<wbr>Document/Functions/Built-in/<wbr>eofunc.shtml</a>) states:</div><div><br></div><div>"Missing values are ignored when computing the covariance or correlation matrix"</div><div><br></div><div>I've actually had some success in passing missing values using this function. Is the handling of missing values the same for the svdcov_sv function?</div><div><br></div><div>In contrast, the function "dgeevx_lapack" is very sensitive to missing values. In fact, the function will exit even if a missing value attribute is set, and there are no actual missing values.</div><div><br></div><div>So I'm hoping to get some clarity on how missing values are handled. Is it possible to get the FORTRAN lapack code to understand the error messages and potentially make modifications?</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Adam</div></font></span></div>
</blockquote></div><br></div>