[ncl-talk] Help in saving values in looping
Andreas Chrysanthou
eeac at leeds.ac.uk
Sun Jun 11 07:46:19 MDT 2017
Hi NCL users,
Following up on my last email when I try to draw that array to a contour plot, the negative values (criterion in the else statement) are not plotted at all though whereas the positive ones are plotted and I don’t understand why.
Applying only the second criterion to be plotted works normally, the problem arises when I try to plot both of them, saved in the same array.
Tested with the printed values, the calculations and the saved output is ok.
The snippet of my code follows:
finarr_nies = new((/42,42,1/),typeof(value_test))
finarr_nies!0 = "startyear"
finarr_nies&startyear = ispan(1960,2001,1)
finarr_nies!1 = "endyear"
finarr_nies&endyear = ispan(1969,2010,1)
do i=1960,2001,1
do j=1969,2010,1
ntimes3 = ispan(i,j,1)
start_date = i
end_date = j
if (j-i .ge. 2) then
test_rc_nies = regline_stats(ntimes3,value_test_nies({i:j}))
if (j-i .ge. 9) then
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .gt. 0 .and. test_rc_nies at b95(0) .gt. 0 , test_rc_nies at b(1)*10, value_test at _FillValue )
else
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .lt. 0 .and. test_rc_nies at b95(1) .lt. 0 ,test_rc_nies at b(1)*10, value_test at _FillValue )
end if
end if
delete(ntimes3)
end do
end do
nies = finarr_nies(:,:,0)
copy_VarCoords(finarr_nies, nies)
nies_ = nies(endyear|:,startyear|:)
wks = gsn_open_wks("eps”,”ccsrnies")
res = True
res at cnFillOn = True
res at cnFillPalette = "BlWhRe"
res at cnLineLabelsOn = True
res at cnLinesOn = True
res at tiYAxisString = "End Year"
res at tiXAxisString = "Start Year"
plot = gsn_csm_contour(wks,nies_,res)
Andreas
On 9 Jun 2017, at 17:48, Adam Phillips <asphilli at ucar.edu<mailto:asphilli at ucar.edu>> wrote:
Hi Andreas,
I think it would be good to send the current state of your script to ncl-talk for all to see, otherwise folks will be hard-pressed to be able to assist.
Adam
On Thu, Jun 8, 2017 at 8:08 AM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Thanks for the help and support provided.
Adam, your last suggestion seems to be working properly, calculating the correct values and applying both the desired criteria, saving the output to the array.
I’ve tested that with the printed values.
When I go and draw that array to a contour plot, the negative values (criterion in the else statement) are not plotted at all though whereas the positive ones are and I don’t understand why.
Applying only the second criterion to be plotted works normally, the problem arises when I try to plot both of them, saved in the same array.
The colormap automatically has negative values in the label but they are not being drawn. Any idea?
Cheers,
Andreas
On 7 Jun 2017, at 21:30, Adam Phillips <asphilli at ucar.edu<mailto:asphilli at ucar.edu>> wrote:
Hi Andreas,
Based on that snippet of code the only line I see where you may be dividing by zero is this line:
test_rc_nies = regline_stats(ntimes3,value_test_nies({i:j}))
Following through your do loops you will have iterations when both input arrays are of size 1 (when say i and j = 1989).
Quick testing here shows that each of the input arrays need to be of size 3 or greater to avoid triggering that error message of dividing by 0. To avoid this situation you can change your code to this to make sure more than 2 values are input into regline_stats:
if (j-i.ge.2) then
test_rc_nies = regline_stats(ntimes3,value_test_nies({i:j}))
if (j-i .ge. 9) then
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .gt. 0 .and. test_rc_nies at b95(0) .gt. 0 , finarr_nies({i},{j},0), value_test at _FillValue )
else
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .lt. 0 .and. test_rc_nies at b95(1) .lt. 0 , finarr_nies({i},{j},0), value_test at _FillValue )
end if
end if
Hope that helps. If not, or if you have further questions please respond to ncl-talk...
Adam
On Wed, Jun 7, 2017 at 1:16 PM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Following up on the loop I created, when I tried to add some criteria based on the slope of the linear trend and the regression coefficients in order to save them to a new array I got the error:
fatal:divide: Division by 0, Can't continue
fatal:Div: operator failed, can't continue
I want to store the calculated value only when it meets one of the following criteria.
Any ideas on how to improve the syntax? Or ways to bypass the problem?
I’ve read on the NCL website in the lazy evaluation section that: "If the left operand is an array, you cannot depend on a left-hand side test to guard against possible error conditions resulting from evaluating an expression on the right side. For example, you cannot avoid division by 0 and the consequences of an error result…”
The snippet of the code is:
finarr_nies = new((/22,22,1/),typeof(value_test))
finarr_nies!0 = "startyear"
finarr_nies&startyear = ispan(1980,2001,1)
finarr_nies!1 = "endyear"
finarr_nies&endyear = ispan(1989,2010,1)
do i=1980,2001,1
do j=1989,2010,1
ntimes3 = ispan(i,j,1)
start_date = i
end_date = j
test_rc_nies = regline_stats(ntimes3,value_test_nies({i:j}))
if (j-i .ge. 9) then
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .gt. 0 .and. test_rc_nies at b95(0) .gt. 0 , finarr_nies({i},{j},0), value_test at _FillValue )
else
finarr_nies({i},{j},0) = where(test_rc_nies at b(1) .lt. 0 .and. test_rc_nies at b95(1) .lt. 0 , finarr_nies({i},{j},0), value_test at _FillValue )
end if
delete(ntimes3)
end do
end do
Cheers,
Andreas
On 6 Jun 2017, at 13:22, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
I really appreciate your help Adam and Dennis.
I could manage to extract the one value I wanted from the slope and the regression coefficients, pass it on to the new array outside of the loop and improved the code by writing the loop with variables instead of constants.
Thanks so much.
On 6 Jun 2017, at 01:29, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:
I am not sure what is meant by: "There was one not insignificant thing I didn’t mention"
---
As you note:
"the regline_stats function for the b calculates both the b(0) +b(1*)x of the linear regression, while the b95 calculates both the 2.5% and 97.5% regression coefficient confidence intervals. So when it tries to save to that cell, there are two values for the @b and @b95."
Change:
trend_slope=test_rc at b*10 ; prints slope of the linear trend for the selected range of years
reg_coeff=test_rc at b95 ; prints 2.5% and 97.5% regression coefficient confidence intervals
To extract only one value
trend_slope=test_rc at b(1) *10 ; prints slope of the linear trend for the selected range of years
reg_coeff=test_rc at b95(1) ; prints 2.5% and 97.5% regression coefficient confidence intervals
---
"4 different columns (start_date, end_date,trend_slope,reg_coeff)
---
The following create 4 (well 3) different columns. I may give you an idea how to proceed.
Also, good programming practice is to use variables and not constants.
nyrSpan = 10
nyrCrit = 10
nyrStrt1 = 1980
nyrStrt2 = 2001
nyrLast1 = nyrStrt1+nyrSpan-1
nyrLast2 = 2010
NTIM = 1000 ; arbitrary max possible # elements
strtYear = new( NTIM, "integer")
lastYear = new( NTIM, "integer")
trend = new( NTIM, "float")
rc = new( NTIM, "float")
nt = -1
do nyrStrt=nyrStrt1,nyrStrt2
do nyrLast=nyrLast1,nyrLast2
nyrs = nyrLast-nyrStrt+1
if (nyrs.ge.nyrCrit) then
test_rc = regline_stats(tofloat(ispan(1,nyrSpan,1)),random_normal(-1,1,nyrSpan))
nt = nt+1
strtYear(nt) = nyrStrt
lastYear(nt) = nyrLast
spanYear = nyrLast-nyrStrt+1
rc(nt) = (/ test_rc /) ; (/.../) no meta data ; same as b(1)
;trend(nt) = ?????
print(strtYear(nt)+" "+lastYear(nt)+" "+spanYear+" "+rc(nt))
end if
end do
end do
print("=================================================")
ntim = nt+1 ; # of elements
print("ntim="+ntim)
On Mon, Jun 5, 2017 at 11:10 AM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
There was one not insignificant thing I didn’t mention.
the regline_stats function for the b calculates both the b(0) +b(1*)x of the linear regression, while the b95 calculates both the 2.5% and 97.5% regression coefficient confidence intervals.
So when it tries to save to that cell, there are two values for the @b and @b95.
That’s why I got the error of : "fatal:Dimension sizes on right hand side of assignment do not match dimension sizes of left hand side"
I only want the slope of the liner trend (b(1)) and my criterion if I need to save these values is that the confidence intervals should not be negative (that would probably need to be included in the if loop).
But I don’t know how to keep only one value of those pair of calculated stats.
Cheers,
Andreas
On 5 Jun 2017, at 17:17, Adam Phillips <asphilli at ucar.edu<mailto:asphilli at ucar.edu>> wrote:
Hi Andreas,
As there are singular values for reg_coef and trend_slope for each start and end time, I think it would be best to simply create a 3D array dimensioned (start_date, end_date,2), with the last dimension (0) = trend_slope and (1) representing reg_coeff. If you must have a 4-dimensional array then you should create an array that is dimensioned 22 x 22 x 2 x 2, but (:,:,1,0) and (:,:,0,1) will be missing. In the example below I show how to create both types of arrays:
finarr = new((/22,22,2/),typeof(value_test))
finarr!0 = "startyear"
finarr&startyear = ispan(1980,2001,1)
finarr!1 = "endyear"
finarr&endyear = ispan(1989,2010,1)
finarrZ = new((/22,22,2,2/),typeof(value_test))
finarrZ!0 = "startyear"
finarrZ&startyear = ispan(1980,2001,1)
finarrZ!1 = "endyear"
finarrZ&endyear = ispan(1989,2010,1)
do i=1980,2001,1
do j=1989,2010,1
ntimes=ispan(i,j,1)
if (j-i .ge. 9 ) then
test_rc=regline_stats(ntimes,value_test({i:j}))
print(test_rc)
start_date=i
end_date=j
finarr({i},{j},0) = test_rc at b*10 ; prints slope of the linear trend for the selected range of years
finarr({i},{j},1) = test_rc at b95 ; prints 2.5% and 97.5% regression coefficient confidence intervals
finarrZ({i},{j},0,0) = test_rc at b*10
finarrZ({i},{j},1,1) = test_rc at b95
else
print(“this age range has not been selected, since its less than 9 years")
end if
delete(ntimes)
end do
end do
Hope that helps. If you have any further questions please respond to the ncl-talk email list.
Adam
On Mon, Jun 5, 2017 at 9:48 AM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Hi NCL users,
I created a loop for calculating a linear trend sensitivity for different start and end dates.
My timeseries span over 1980-2010 and I want to calculate the linear trend of the timeseries for when the start - end date are more than >= 9years.
(1980-1989, 1980-1990, 1980-1991,..., 1980-2010
1981-1990, 1981-1991, ..., 1981-2010
...
...
...
2000-2009, 2001-2010
2001-2010)
I’ve created the loops that calculate those values but I need to save them in an array as 4 different columns (start_date, end_date,trend_slope,reg_coeff)
My aim is to plot those as a a contour box plot with x axis the start date and y axis the end date, and the calculated (i’m gonna filter out those values with some criteria) trend values based on a colormap.
Can you help me in order to save the values to that new array so I can plot it?
A snippet of the code follows:
do i=1980,2001,1
do j=1989,2010,1
ntimes=ispan(i,j,1)
if (j-i .ge. 9 ) then
test_rc=regline_stats(ntimes,value_test({i:j}))
print(test_rc)
start_date=i
end_date=j
trend_slope=test_rc at b*10 ; prints slope of the linear trend for the selected range of years
reg_coeff=test_rc at b95 ; prints 2.5% and 97.5% regression coefficient confidence intervals
else print(“this age range has not been selected, since its less than 9 years")
end if
delete(ntimes)
end do
end do
Cheers,
Andreas
Andreas
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
--
Adam Phillips
Associate Scientist, Climate and Global Dynamics Laboratory, NCAR
www.cgd.ucar.edu/staff/asphilli/<http://www.cgd.ucar.edu/staff/asphilli/> 303-497-1726<tel:(303)%20497-1726>
<http://www.cgd.ucar.edu/staff/asphilli>
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
--
Adam Phillips
Associate Scientist, Climate and Global Dynamics Laboratory, NCAR
www.cgd.ucar.edu/staff/asphilli/<http://www.cgd.ucar.edu/staff/asphilli/> 303-497-1726<tel:(303)%20497-1726>
<http://www.cgd.ucar.edu/staff/asphilli>
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
--
Adam Phillips
Associate Scientist, Climate and Global Dynamics Laboratory, NCAR
www.cgd.ucar.edu/staff/asphilli/<http://www.cgd.ucar.edu/staff/asphilli/> 303-497-1726
<http://www.cgd.ucar.edu/staff/asphilli>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170611/48fd43e6/attachment.html
More information about the ncl-talk
mailing list