[ncl-talk] clarity on missing values in svdcov_sv and other lapack rountines

[Adam Herrington]

the svdcov_sv documentation (
http://www.ncl.ucar.edu/Document/Functions/Built-in/svdcov_sv.shtml) states
that "This function does not allow for missing data." However, the function
will not exit, but rather gives you a warning. For example,

print("remove zeros")
  Xp at _FillValue = default_fillvalue("float")
  Yp at _FillValue = default_fillvalue("float")

  Xp = where(Xp.ne.0.,Xp,Xp at _FillValue)
  Yp = where(Yp.ne.0.,Yp,Yp at _FillValue)

  lapack = svdcov_sv(Xp,Yp,nsvd,L,R)

I get about 4000 of these warnings:
 "SVD: Warning all X values in column are missing or are constant"

Followed by about 1000 of these warnings
 "SVD: Warning all Y values in column are missing or are constant"

I'm trying to figure out how much of the data are being ignored, because I
have reason to believe the L and R singular vectors that come out of this
example are using quite abit of the data still.

In the example, Xp and Yp are dimensioned (/ntime,ncols/), where ntime =
980 and ncols = 30*48602. Since the number of warnings is much larger than
ntime, I don't know how to interpret what the warning is referring to when
they say "column".

In trying to track down the code, I've scanned the entire directory
$NCARG_ROOT/lib/ncarg/nclscripts/csm/ and can't find any references to the
wrappers used to create svdcov_sv.

The "eofunc" also uses lapack and the documentation (
http://www.ncl.ucar.edu/Document/Functions/Built-in/eofunc.shtml) states:

"Missing values are ignored when computing the covariance or correlation
matrix"

I've actually had some success in passing missing values using this
function. Is the handling of missing values the same for the svdcov_sv
function?

In contrast, the function "dgeevx_lapack" is very sensitive to missing
values. In fact, the function will exit even if a missing value attribute
is set, and there are no actual missing values.

So I'm hoping to get some clarity on how missing values are handled. Is it
possible to get the FORTRAN lapack code to understand the error messages
and potentially make modifications?

Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170527/f9b5ec01/attachment.html