[ncl-talk] Crash with eofunc_varimax_Wrap function
Dennis Shea
shea at ucar.edu
Tue Aug 2 15:20:29 MDT 2016
Hi Allison
I have run (see below) the script on my old MAC using NCL 6.3.0 and the
beta 6.4.0 with no problems.
---------------------------
%> ncl ncl-talk_reof.ncl
Specifically:
neval = 6
optEOF = True
ev = eofunc_Wrap(av,neval,optEOF)
printVarSummary(ev)
printMinMax(ev,0)
print("===")
;***** VARIMAX ROTATION *****
ev_rot = eofunc_varimax_Wrap(ev,-1)
printVarSummary(ev_rot)
printMinMax(ev_rot,0)
============ OUTPUT ===========
Variable: ev
Type: float
Total Size: 1555200 bytes
388800 values
Number of Dimensions: 3
Dimensions and sizes: [evn | 6] x [latitude | 180] x [longitude | 360]
Coordinates:
evn: [1..6]
latitude: [89.5..-89.5]
longitude: [-179.5..179.5]
Number Of Attributes: 7
eval_transpose : ( 4.416919, 2.388321, 0.7132505, 0.5452091,
0.4846498, 0.388064 )
eval : ( 1492.919, 807.2526, 241.0787, 184.2807, 163.8116, 131.1656 )
pcvar : ( 34.8554, 18.84705, 5.6285, 4.302428, 3.824535, 3.062344 )
matrix : covariance
method : transpose
_FillValue : -1000
long_name : EOF: sst
(0) EOF: sst : min=-0.0294479 max=0.0344466
(0) ===
Variable: ev_rot
Type: float
Total Size: 1555200 bytes
388800 values
Number of Dimensions: 3
Dimensions and sizes: [evn | 6] x [latitude | 180] x [longitude | 360]
Coordinates:
evn: [1..6]
latitude: [89.5..-89.5]
longitude: [-179.5..179.5]
Number Of Attributes: 4
pcvar_varimax : ( 20.40459, 19.06581, 13.1998, 9.32391, 4.929915,
3.596234 )
variance_varimax : ( 873.9649, 816.6224, 565.3708, 399.3596, 211.157,
154.0331 )
_FillValue : -1000
op : Kaiser Varimax Rotation: opt=-1
(0) min=-0.527019 max=0.743203
(0) ===
==============
What do the following return?
%> uname -a
%> gcc --version
D
On Mon, Aug 1, 2016 at 12:48 PM, Allison Serakos <sera0021 at umn.edu> wrote:
> Hi Dennis,
>
> Neither 'eofunc_varimax' or 'eofunc_varimax_Wrap' will work.
>
> Let me know if you are able to take another look.
>
> Thanks for your help,
> Allison
>
> On Wed, Jul 27, 2016 at 11:55 AM, Dennis Shea <shea at ucar.edu> wrote:
>
>> I'm am not sure why 'eofunc_varimax' would work and 'eofunc_varimax_Wrap'
>> would not. The *_Wrap invokes the function and, where appropriate, adds
>> attributes and coordinates. Here is the code used:
>>
>> undef ("eofunc_varimax_Wrap")
>> function eofunc_varimax_Wrap (eof:numeric, optEVX:integer)
>> local eofEVX
>> begin
>> eofEVX = eofunc_varimax(eof, optEVX) ; invoke built-in function
>> eofEVX at op = "Kaiser Varimax Rotation: opt="+optEVX
>> copy_VarCoords(eof, eofEVX)
>>
>> return (eofEVX) ; return
>> end
>>
>> ===
>>
>> I am busy and will look later.
>>
>> On Tue, Jul 26, 2016 at 5:04 PM, Allison Serakos <sera0021 at umn.edu>
>> wrote:
>>
>>> Hi Dennis,
>>>
>>> Thank you again for your assistance with my issue of the
>>> eofunc_varimax_Wrap function stalling. I was able to produce REOF plots of
>>> the annually averaged data. However, I am running into the same issue of
>>> the eofunc_varimax_Wrap function stalling when I use the original monthly
>>> data as well as using a weighted seasonal average instead of a weighted
>>> annual average.
>>>
>>> I am wondering if you could explain your modifications to the script I
>>> sent you. I am also wondering if you have an idea as to why now the
>>> original monthly dataset and the weighted seasonally averaged dataset stall
>>> with the eofunc_varimax_Wrap function.
>>>
>>> Thanks again for all your help.
>>>
>>> Kind regards,
>>> Allison
>>>
>>> On Thu, Jun 30, 2016 at 8:09 AM, Dennis Shea <shea at ucar.edu> wrote:
>>>
>>>> offline from ncl-talk
>>>>
>>>> re my comment:
>>>>
>>>> I am not sure why you want to rotate.
>>>> The 1st two modes look very distinct.
>>>>
>>>> ===
>>>>
>>>> Deriving a "global" eof is rather unusual for ocean grids. The
>>>> atmosphere is 'continuous'. The oceans are 'discontinuous' due to the
>>>> continents. The EOFs may be though of as orthogonal structures that result
>>>> from covariances. Deriving a 'global' EOF is implicitly 'saying' that SST
>>>> variations in the Kuroshio may be related to the Gulf Stream.
>>>>
>>>> An Example of using an EOF analysis over the ocean is:
>>>>
>>>> http://www.ncl.ucar.edu/Applications/eof.shtml
>>>> Example 3
>>>>
>>>> Good Luck
>>>>
>>>>
>>>>
>>>> On Wed, Jun 29, 2016 at 4:17 PM, Dennis Shea <shea at ucar.edu> wrote:
>>>>
>>>>> Hi Allison,
>>>>>
>>>>> Can you please try the attached?
>>>>>
>>>>> I am not sure why you want to rotate.
>>>>> The 1st two modes look very distinct.
>>>>>
>>>>> I did not plot anything. I'll let that to you
>>>>>
>>>>>
>>>>> THX
>>>>> D
>>>>>
>>>>> On Tue, Jun 28, 2016 at 8:07 AM, Allison Serakos <sera0021 at umn.edu>
>>>>> wrote:
>>>>>
>>>>>> script name: ncl-talk_reof.ncl
>>>>>> data file name: HadISST_sst.nc.gz
>>>>>>
>>>>>> Thank you,
>>>>>> Allison
>>>>>>
>>>>>> On Mon, Jun 27, 2016 at 1:49 PM, Allison Serakos <sera0021 at umn.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am using the eofunc_varimax_Wrap function to analyze global
>>>>>>> rotated EOFs produced from the eofunc_Wrap function. However, when I run my
>>>>>>> script, it hangs when it reaches the eofunc_varimax_Wrap function. The
>>>>>>> script does not finish and I get no error message.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The dataset that I am using is the HadISST 1.1 monthly average sea
>>>>>>> surface temperature dataset. It is a 3-dimensional (time, latitude,
>>>>>>> longitude) global 1° x 1° grid dataset with 1755 time values, 180
>>>>>>> latitude, and 360 longitude. I produce annually averaged values and
>>>>>>> spatially weighted it prior to the EOF analysis.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I have been using a Mac machine with NCL version 6.3.0 and a Linux
>>>>>>> machine with NCL version 6.2.1 and the program hangs when conducting the
>>>>>>> rotation.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I tried running the script with global data for different time
>>>>>>> periods (all between 113 and 102 years of global data) on both the Mac and
>>>>>>> Linux machines and found that the function works only for some time
>>>>>>> periods and crashes with others. These periods differ between the two Mac
>>>>>>> and Linux machines that I used. I also tried reshaping the data
>>>>>>> from a 3D matrix to a 2D matrix using the reshape function and combining
>>>>>>> the spatial components. In this situation, the rotation output was still
>>>>>>> the same as above for the respective time periods and machines.
>>>>>>> Additionally, I tried reducing the spatial domain and found that when I
>>>>>>> reduce the latitude range (anything smaller than 69S-90N) for any
>>>>>>> time period the script runs on both machines and the function does not
>>>>>>> crash.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I used ‘ncl –x’ to see if there is an error in my script, but the
>>>>>>> script only crashes before the eofunc_varimax_Warp function with no
>>>>>>> messages. When I run the script, NCL uses about 592.5MB of the 12GB of
>>>>>>> memory on the Mac machine, so I do not believe this an issue of lack of
>>>>>>> memory space.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am wondering if there is a known problem with the
>>>>>>> eofunc_varimax_Wrap function? I am also wondering if you have any thoughts
>>>>>>> as to why the function may be crashing under certain situations and not
>>>>>>> others?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Here is a copy of my EOF and Varimax Rotation script:
>>>>>>>
>>>>>>> ;----------------------------------------------------------------
>>>>>>>
>>>>>>> ;***** EOF *****
>>>>>>>
>>>>>>> neval = 6
>>>>>>>
>>>>>>> optEOF = True
>>>>>>>
>>>>>>> optEOF at method = "transpose"
>>>>>>>
>>>>>>> optEOF at jopt = 0
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> eof = eofunc_Wrap(av,neval,optEOF)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ;***** VARIMAX ROTATION *****
>>>>>>>
>>>>>>> eof_rot = eofunc_varimax_Wrap(eof,-1)
>>>>>>>
>>>>>>> eof_varimax_reorder(eof_rot)
>>>>>>>
>>>>>>> ;----------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks for your help in advance.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Allison
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Allison Serakos
>>>>>>> Graduate Research Assistant
>>>>>>> Department of Soil, Water, and Climate
>>>>>>> University of Minnesota
>>>>>>> 1991 Upper Buford Circle, Rm S535
>>>>>>> St. Paul, MN 55108
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Allison Serakos
>>>>>> Graduate Research Assistant
>>>>>> Department of Soil, Water, and Climate
>>>>>> University of Minnesota
>>>>>> 1991 Upper Buford Circle, Rm S535
>>>>>> St. Paul, MN 55108
>>>>>>
>>>>>> _______________________________________________
>>>>>> ncl-talk mailing list
>>>>>> ncl-talk at ucar.edu
>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Allison Serakos
>>> Graduate Research Assistant
>>> Department of Soil, Water, and Climate
>>> University of Minnesota
>>> 1991 Upper Buford Circle, Rm S535
>>> St. Paul, MN 55108
>>>
>>
>>
>
>
> --
> Allison Serakos
> Graduate Research Assistant
> Department of Soil, Water, and Climate
> University of Minnesota
> 1991 Upper Buford Circle, Rm S535
> St. Paul, MN 55108
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160802/5d1d700b/attachment.html
More information about the ncl-talk
mailing list