[ncl-talk] eof function

Mon Sep 7 08:21:28 MDT 2015

Attached is one of the test scripts used to test NCL's 'eofunc' function.

It uses data from

; John C Davis
; Statistics and Data Analysis in Geology
; Wiley, 2nd Edition, 1986
; Source Data: page 524 , EOF results: page537

To provide a geophysical context:
    There are 25 rows which you can consider 'time'   (25 time steps)
    There are 7 columns which you can consider 'grid points' or 'stations'

NCL matches the book results *exactly*. The script performs the following tasks:

[1] Read and 'pretty print' the input data

[2] (for convenience) it assigns the named dimensions 'time' and 'sta'
(station) to the row & columns

[3] It uses the named dimensions to conform to the input expected by eofunc

[4] It 'pretty prints'  all the eigenvectors (evevc) and principal
components (PCs; evecv_ts)

[5] It calculates and prints out the 'sum of squares' of each
eigenvector [ sumsqr = dim_sum(evecv^2)]. These sum to one (1.0) so
this shows the eigenvectors are normalized.

[6] It reconstructs the original input array from the evevc and
evevc_ts arrays:  'eof2data'.
Since NCL's 'eofunc' works with anomalies ... the column/station menas
are added prior to reconstruction.

[7] it computes the correlations between each of the time series (PCs)
and verifies the there is no correlation between each PC. This means
they are orthogonal.

====
In short, NCL's 'eofunc' is doing what it should do.

====

As noted by Maria and me,

[a] maybe the ferret and fortran are not normalizing
[b] NCL uses a covariance matrix containing covariances (symmetric
storage mode). Even if you do not input anomalies, under the hood,
covariances are used.  This could be another source of discrepancy.
Note: if you are using tropical Pacific SSTs, you should be using
anomalies.

=====

Regards
D

On Fri, Sep 4, 2015 at 1:02 PM, Maria Gehne - NOAA Affiliate
<maria.gehne at noaa.gov> wrote:
> Very likely that's where the difference comes from. Have you checked the
> Ferret documentation to see how the EOFs are normalized there? The variance
> is different as well. How does Ferret compute the singular value
> decomposition?
>
> Again, have you looked at the time series? Do those have a different
> amplitude?
>
> In any case for both Ferret and NCL you should be able to reconstruct the
> whole data set using the EOFs, PCs, and singular values. Have you tried
> doing that?
>
> Maria
>
> On Fri, Sep 4, 2015 at 7:49 AM, Vanúcia Schumacher
> <vanucia-schumacher at hotmail.com> wrote:
>>
>> The difference between results generated by NCL and Fortran or ferret is
>> in the amplitude! Using the same data set!
>> I wonder if this difference in amplitude (very low values for NCL) is due
>> to normalization inserted in eofunc_wrap function, would it?
>>
>>
>>
>>
>> ________________________________
>> From: maria.gehne at noaa.gov
>> Date: Thu, 3 Sep 2015 16:55:09 -0600
>> Subject: Re: [ncl-talk] eof function
>> To: vanucia-schumacher at hotmail.com
>> CC: ncl-talk at ucar.edu
>>
>>
>> I don't think you attached the figure you meant to attach.
>>
>> EOFs have no fixed normalization. The value of the patterns (EOFs) and
>> principal components (the time series associated with the patterns) depends
>> on how you normalize. The conventions on how to normalize differ between
>> applications.The only value that is fixed coming out of an EOF analysis are
>> the singular values (the percentage of variance of the EOFs).
>>
>> Are the spatial patterns the same between Fortran, Ferret and NCL, even if
>> they don't have the same amplitude? What about the time series? If both
>> patterns and time series have a different amplitude maybe that's where the
>> normalization is different.
>>
>> For NCL the website for the EOF function states:
>> The returned values are normalized such that the sum of squares for each
>> EOF pattern equals one. To denormalize the returned EOFs multiply by the
>> square root of the associated eigenvalue (aka, the singular value).
>>
>>
>> Check if the Fortran and Ferret EOF patterns are normalized in the same
>> way, or not.
>>
>> Hope this helps,
>> Maria
>>
>> On Thu, Sep 3, 2015 at 2:59 PM, Vanúcia Schumacher
>> <vanucia-schumacher at hotmail.com> wrote:
>>
>>
>> The values of EOF (output) are different, as attached (figure)
>>
>> ________________________________
>> From: vanucia-schumacher at hotmail.com
>> To: shea at ucar.edu
>> Subject: RE: [ncl-talk] eof function
>> Date: Thu, 3 Sep 2015 17:57:45 -0300
>>
>> The values of EOF output are different, as attached (figure)
>>
>>
>> > Date: Thu, 3 Sep 2015 14:51:19 -0600
>> > Subject: Re: [ncl-talk] eof function
>> > From: shea at ucar.edu
>> > To: vanucia-schumacher at hotmail.com
>> > CC: ncl-talk at ucar.edu
>> >
>> > Different normalizations by the EOFs?
>> >
>> > On Thu, Sep 3, 2015 at 1:44 PM, Vanúcia Schumacher
>> > <vanucia-schumacher at hotmail.com> wrote:
>> > > Dear NCL users,
>> > >
>> > > I calculated the EOF for the Pacific region ({-20:20},{140:290}) using
>> > > the
>> > > function of NCL (eofunc_Wrap) and tested with other programs (ferret
>> > > and
>> > > fortran ) and the values of EOF using the NCL are very small compared
>> > > with
>> > > the other tested outputs.
>> > >
>> > > Using Fortran and Ferret - values between 0.6 to -0.2
>> > > Using NCL function - values between 0.14 to -0.10
>> > >
>> > >
>> > > Could someone explain to me why this happens?
>> > >
>> > > _______________________________________________
>> > > ncl-talk mailing list
>> > > ncl-talk at ucar.edu
>> > > List instructions, subscriber options, unsubscribe:
>> > > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> > >
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eof_0.ncl
Type: application/octet-stream
Size: 3713 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150907/52c2d836/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out.eof_0
Type: application/octet-stream
Size: 18733 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150907/52c2d836/attachment-0001.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eoftest.davis_data
Type: application/octet-stream
Size: 1249 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150907/52c2d836/attachment-0002.obj 