<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Thank you, Dennis, for all the time you put into this</p>
<p>and your helpful advice. And good catch about the time steps.</p>
<p>Michael<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
Michael Notaro<br>
<div>Associate Director<br>
</div>
<div>Nelson Institute Center for Climatic Research<br>
</div>
<div>University of Wisconsin-Madison<br>
</div>
<div>Phone: (608) 261-1503<br>
</div>
<div>Email: mnotaro@wisc.edu<br>
</div>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Dennis Shea <shea@ucar.edu><br>
<b>Sent:</b> Friday, January 15, 2016 4:42 PM<br>
<b>To:</b> ncl-talk@ucar.edu<br>
<b>Cc:</b> Michael Notaro; Appo derbetini<br>
<b>Subject:</b> Re: [ncl-talk] Slow code</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>This is a long and rather detailed post which I have worked upon on-and-off over the past day. The basic message is: Don't use tools (operators) without looking at your data.<br>
-----<br>
Appolinaire's suggestion follows the general approach I mentioned in a previous ncl-talk response on this thread:<br>
<br>
</div>
[1] Process each year-month file containing hourly values: <br>
<a id="LPlnk390845" href="http://ACCESS_SRF.1980010100.nc" target="_blank">
ACCESS_SRF.1980010100.nc</a>; <a href="http://ACCESS_SRF.1980020100.nc" target="_blank">
ACCESS_SRF.1980020100.nc</a>, .....<br>
</div>
[2] Write a netCDF file containing daily means( with the time dimension written as 'unlimited') for each year-month file<br>
</div>
[3] After [2], concatenate the daily mean files ('cdo mergetime' or the NCO 'ncrcat' operator)<br>
<br>
</div>
<div>I wrote a sample NCL driver script to do this sequence (see attached: tst.cdo_daymean.ncl). Prior to running the NCL driver script, I looked at the number of time steps on the two sample year-month files that Mike sent.<br>
<br>
The <a href="http://ACCESS_SRF.1980010100.nc" target="_blank">ACCESS_SRF.1980010100.nc</a> file has 744 time (hourly) values:
<br>
744/24= 31 <br>
which is expected for January.<br>
<br>
The <a href="http://ACCESS_SRF.1980010100.nc" target="_blank">ACCESS_SRF.1980020100.nc</a> file has 696 time (hourly) values:
<br>
696/24= 29 <br>
which is expected for February in a leap year.<br>
<br>
</div>
<div>The total number of daily means should be 60 (=31+29). However, when I ran the tst.cdo_daymean.ncl script on my two sample files I got:<br>
<br>
========================<br>
%> ncl tst.cdo_daymean.ncl <br>
<br>
Copyright (C) 1995-2015 - All Rights Reserved<br>
University Corporation for Atmospheric Research<br>
NCAR Command Language Version 6.3.0<br>
The use of this software is governed by a License Agreement.<br>
See <a href="http://www.ncl.ucar.edu/" target="_blank">http://www.ncl.ucar.edu/</a> for more details.<br>
<br>
<br>
Variable: fili<br>
Type: string<br>
Total Size: 16 bytes<br>
2 values<br>
Number of Dimensions: 1<br>
Dimensions and sizes: [2]<br>
Coordinates: <br>
(0) <a href="http://ACCESS_SRF.1980010100.nc" target="_blank">ACCESS_SRF.1980010100.nc</a><br>
(1) <a href="http://ACCESS_SRF.1980020100.nc" target="_blank">ACCESS_SRF.1980020100.nc</a><br>
(0) ---------<br>
(0) cdo daymean ./<a href="http://ACCESS_SRF.1980010100.nc" target="_blank">ACCESS_SRF.1980010100.nc</a> ./daily/<a href="http://ACCESS_SRF.19800101.nc" target="_blank">ACCESS_SRF.19800101.nc</a><br>
Warning (cdfSetVar) : Inconsistent variable definition for xlat!<br>
Warning (cdfSetVar) : Inconsistent variable definition for xlon!<br>
cdo daymean: Processed 614693730 values from 27 variables over 744 timesteps ( 8.13s )<br>
<br>
(0) cdo daymean ./<a href="http://ACCESS_SRF.1980020100.nc" target="_blank">ACCESS_SRF.1980020100.nc</a> ./daily/<a href="http://ACCESS_SRF.19800201.nc" target="_blank">ACCESS_SRF.19800201.nc</a><br>
Warning (cdfSetVar) : Inconsistent variable definition for xlat!<br>
Warning (cdfSetVar) : Inconsistent variable definition for xlon!<br>
cdo daymean: Processed 575040018 values from 27 variables over 696 timesteps ( 7.57s )<br>
<br>
(0) =========<br>
(0) PATHO=./daily/<a href="http://ACCESS_SRF.19800101-19800201.nc" target="_blank">ACCESS_SRF.19800101-19800201.nc</a><br>
(0) cdo_mergetim: cdo mergetime ./daily/ACCESS_SRF*nc ./daily/<a href="http://ACCESS_SRF.19800101-19800201.nc" target="_blank">ACCESS_SRF.19800101-19800201.nc</a><br>
cdo mergetime: Processed 51341766 values from 54 variables over 62 timesteps ( 3.19s )<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
======<br>
</div>
<div>Note: ** 62 ** time steps when 60 are expected. <br>
(FYI: NCL's 6.3.1 'calculate_daily_values' function returns 62 also!).<br>
<br>
</div>
<div>Are both CDO *and* NCL 'wrong' ? The short answer is "no" ! Pourquoi?? Why??<br>
</div>
<div><br>
Unfortunately, the way the source files were created presents a problem for this generic one month at a time approach. :-(</div>
</div>
<div>It assumes that each day on the file has 24 hourly values for each day. <br>
=================<br>
As is frequently mentioned on ncl-talk, the golden rule of data processing is *look at your data*<br>
<br>
diri = "./"<br>
fili = "<a href="http://ACCESS_SRF.1980010100.nc" target="_blank">ACCESS_SRF.1980010100.nc</a>" ; hourly<br>
f = addfile(diri+fili, "r")<br>
time = f->time<br>
date = cd_calendar(time, -3) ; yyyymmddhh<br>
print(date)<br>
<br>
===<br>
Variable: date<br>
Type: integer<br>
Total Size: 2976 bytes<br>
744 values<br>
Number of Dimensions: 1<br>
Dimensions and sizes: [744]<br>
Coordinates: <br>
Number Of Attributes: 1<br>
calendar : gregorian<br>
<br>
</div>
<div>(0) 1980010101 <=== starts at hour 1 (hh=1) ... not hour 00 as one might expect<br>
</div>
<div>(1) 1980010102<br>
(2) 1980010103<br>
</div>
<div>[snip]<br>
(21) 1980010122<br>
</div>
<div>(22) 1980010123 <=== day 01 ends here ... *** 23 hours *** for day 1<br>
</div>
<div>(23) 1980010200 <=== day 02 starts at 00<br>
</div>
<div>(24) 1980010201<br>
</div>
<div>[snip]<br>
(45) 1980010222<br>
</div>
<div>(46) 1980010223 <=== day 02 end at 23 *** 24 hours *** for day 2<br>
</div>
<div>(47) 1980010300 <=== day 02 starts at 00<br>
(48) 1980010301<br>
</div>
<div>[SNIP]<br>
</div>
<div> days 2-31 have 24 value<br>
</div>
<div>[SNIP]<br>
(719) 1980013100<br>
(720) 1980013101<br>
(721) 1980013102<br>
</div>
<div>[snip]<br>
(741) 1980013122<br>
(742) 1980013123<br>
</div>
<div>(Jan & Feb)<br>
</div>
<div>(743) 1980020100 *** This is where the 32nd day come from!! 1 hour at February 1 at 00Z<br>
^^^^^^^^^^^^^^^^^^^<br>
<br>
</div>
<div>So, 32 daily means with the daily mean for Jan 1, calculated from 23 values; The last daily mean is Feb 1 at 00Z with one value. All other values would have 24 hourly values used.<br>
<br>
===========<br>
</div>
<div>The resultant merged file will have ***two*** Feb 1 means<br>
<br>
</div>
<div>[snip]<br>
(29) 1980013023<br>
(30) 1980013123<br>
</div>
<div>(31) 1980020100 <======== one value at 00Z from the 1st file<br>
</div>
<div>(32) 1980020123 <======== derived from 23 hourly values on the 2nd file<br>
</div>
<div>[snip]<br>
</div>
<div><br>
(741) 1980013122<br>
(742) 1980013123<br>
</div>
<div>(743) 1980020100 <======== one value<br>
</div>
<div><br>
</div>
<div>Punch line: This will likely create issues with subsequent post-processing.<br>
<br>
<br>
</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jan 12, 2016 at 11:44 PM, Appo derbetini <span dir="ltr">
<<a href="mailto:appopson4@gmail.com" target="_blank">appopson4@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Hi,<br>
</div>
To convert hourly to daily data, I'm using CDO<br>
</div>
<div><br>
</div>
<div>For january 1980 for example, <br>
<br>
</div>
<span style="color:rgb(255,0,0)"><span style="background-color:rgb(255,255,255)">cdo daymean
<a href="http://hourly_01.1980.nc" target="_blank">hourly_01.1980.nc</a> <a href="http://daily_01.1980.nc" target="_blank">
daily_01.1980.nc</a> </span></span><br>
<br>
</div>
Then I merge monthly files to have yearly data <br>
<br>
<br>
</div>
with <br>
<br>
<span style="background-color:rgb(255,255,255)"><span></span></span><br>
</div>
<span style="color:rgb(255,0,0)"><span style="background-color:rgb(255,255,255)">cdo mergetime daily_*.<a href="http://1980.nc" target="_blank">1980.nc</a>
<a href="http://daily.1980.nc" target="_blank">daily.1980.nc</a></span></span><br>
<br>
<br>
<br>
</div>
<div>Merci<br>
<br>
</div>
<div>Appolinaire<br>
</div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2016-01-12 21:47 GMT+01:00 Dennis Shea <span dir="ltr"><<a href="mailto:shea@ucar.edu" target="_blank">shea@ucar.edu</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>At my request, Mike sent me two sample files. I will look when I get a chance.<br>
<br>
</div>
'My' strategy when dealing with multiple files is<br>
<br>
</div>
[1] <br>
Will use of 'addfiles' be advantageous for the problem.? if hourly arrays are 20GB ... the answer is likely 'no'. Memory allocation, especially on multi user systems) can be slow.<br>
<br>
[2]<br>
</div>
<div>Most commonly <br>
<br>
[a] I will process one year at a time, converting year, month, day, hr to (say) 'hours since ...'.<br>
</div>
<div> [b] Compute the daily averages; write each year or month to netCDF with 'time' unlimited<br>
</div>
<div> [c] use the 'ncks' or 'ncrcat' operators to combine the files if that is desirable.<br>
<br>
---<br>
</div>
<div>I am attaching the 'contrbuted.ncl' functions that are used to calculate monthly or daily quantities: avg, sum, min, max<br>
</div>
<div><br>
</div>
<div><br>
<br>
<a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/calculate_daily_values.shtml" target="_blank">http://www.ncl.ucar.edu/Document/Functions/Contributed/calculate_daily_values.shtml</a><br>
<a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/calculate_monthly_values.shtml" target="_blank">http://www.ncl.ucar.edu/Document/Functions/Contributed/calculate_monthly_values.shtml</a><span><font color="#888888"><br>
<br>
</font></span></div>
<span><font color="#888888">
<div>D<br>
</div>
</font></span></div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jan 12, 2016 at 12:48 PM, Michael Notaro <span dir="ltr">
<<a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt; color:#000000; background-color:#ffffff; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Thanks, Mary, for your further suggestions.</p>
<p><br>
</p>
<p>After I got Alan's first email which helped me reassess my code, </p>
<p>I modified my code <span style="font-size:12pt">to remove the year dimension from most variables</span></p>
<p>to make them more manageable. Now the code</p>
<p>runs in 1 hour, rather than 1 day+. </p>
<p><br>
</p>
<p>Michael</p>
<span>
<p><br>
</p>
<p></p>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
begin</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
</span><span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
mns=(/"01","02","03","04","05","06","07","08","09","10","11","12"/)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
ndays=(/31,28,31,30,31,30,31,31,30,31,30,31/)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
</span><span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily365=new((/365,20,141,217/),float) ; 365 day calendar data</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily365@_FillValue=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily365=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do iyr=0,19</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
</span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
data=new((/141,217,12,744/),float) ; hourly data</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
data@_FillValue=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
data=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=0</div>
<span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do im=0,11</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
prefix=(1980+iyr)+""+mns(im)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
b=addfile("/volumes/data1/yafang/Downscaling/ACCESS1-0/historical/output/ACCESS_SRF."+(1980+iyr)+""+mns(im)+"<a href="http://0100.nc" target="_blank">0100.nc</a>","r") ; read hourly SRF data</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
iy=b->iy</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
jx=b->jx</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
xlat=b->xlat</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
xlon=b->xlon</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
snow=b->snv ; liquid equiv of snow on ground</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
dims=dimsizes(snow)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
nt=dims(0)</div>
</span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
data(:,:,im,0:nt-1)=snow(iy|:,jx|:,time|:)</div>
<span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(snow)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(b)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(dims)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=cnt+1</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
</span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
data@_FillValue=1e+20</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily=new((/141,217,12,31/),float) ; daily data per month</div>
<span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily@_FillValue=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=0</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do id=0,30</div>
</span><span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily(:,:,:,id)=dim_avg(data(:,:,:,cnt:cnt+23)) ; convert hourly to daily data</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=cnt+24</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(data)</div>
<br style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=0</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do im=0,11</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do id=0,ndays(im)-1</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily365(cnt,iyr,:,:)=daily(:,:,im,id) ; convert daily data per month to 365 day calendar</div>
</span><span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
cnt=cnt+1</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
</span>
<div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(daily)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212=new((/19,212,141,217/),float) ; 212 day calendar data for Sep-Mar</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212@_FillValue=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212=1e+35</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
do iyr=0,18</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212(iyr,0:121,:,:)=daily365(243:364,iyr,:,:) ; retrieve Sep-Mar</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212(iyr,122:211,:,:)=daily365(0:89,iyr+1,:,:)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
end do</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
delete(daily365)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
year=ispan(0,18,1)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
year!0="year"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
year&year=year</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
time=ispan(0,211,1)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
time!0="time"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
time&time=time</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212!0="year"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212!1="time"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212!2="iy"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212!3="jx"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212&year=year</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212&time=time</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212&iy=iy</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212&jx=jx</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212@long_name = "liquid snow water on ground"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212@units = "kg m-2"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212@coordinates="xlat xlon"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
daily212@grid_mapping = "rcm_map"</div>
<br style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
system("rm <a href="http://save_daily212_actual_snv_access_late20_faster.nc" target="_blank">
save_daily212_actual_snv_access_late20_faster.nc</a>")</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out=addfile("<a href="http://save_daily212_actual_snv_access_late20_faster.nc" target="_blank">save_daily212_actual_snv_access_late20_faster.nc</a>","c")</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out->daily212=daily212</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out->xlat=xlat</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out->xlon=xlon</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out->iy=iy</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
out->jx=jx</div>
<br style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; margin-top:0px; margin-bottom:0px">
</div>
<br>
<p></p>
<p><br>
</p>
<p><br>
</p>
</div>
</div>
<span>
<div>
<div style="font-size:12pt; color:#000000; background-color:#ffffff; font-family:Calibri,Arial,Helvetica,sans-serif">
Michael Notaro<br>
<div>Associate Director<br>
</div>
<div>Nelson Institute Center for Climatic Research<br>
</div>
<div>University of Wisconsin-Madison<br>
</div>
<div>Phone: <a href="tel:%28608%29%20261-1503" value="+16082611503" target="_blank">
(608) 261-1503</a><br>
</div>
<div>Email: <a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a><br>
</div>
</div>
</div>
<br>
<br>
</span>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block; width:98%">
<div dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Mary Haley <<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>><br>
<b>Sent:</b> Tuesday, January 12, 2016 1:34 PM<br>
<b>To:</b> Alan Brammer<br>
<b>Cc:</b> Michael Notaro; <a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a>
<div>
<div><br>
<b>Subject:</b> Re: [ncl-talk] Slow code</div>
</div>
</font>
<div> </div>
</div>
<div>
<div>
<div>
<div dir="ltr">
<div class="gmail_default"><font color="#000000">Hi folks,</font></div>
<div class="gmail_default"><font color="#000000"><br>
</font></div>
<div class="gmail_default"><font color="#000000">These are all good suggestions.</font></div>
<div class="gmail_default"><font color="#000000"><br>
</font></div>
<div class="gmail_default"><font color="#000000">Another thing that is expensive in NCL is reordering arrays with syntax like:</font></div>
<div class="gmail_default"><font color="#000000"><br>
</font></div>
<div class="gmail_default"><font color="#000000"><span style="font-family:Calibri,Arial,Helvetica,sans-serif">snow(iy|:,jx|:,time|:),</span><br>
</font></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000"><br>
</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000">NCL makes a copy of the array when you do this, and it has to swap all the dimensions every time in the loop.</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000"><br>
</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000">Isf reordering the array is absolutely necessary? I see that you are reordering and then calling "dim_avg_n". Since you are already using dim_avg_n,
why not leave the array as is and just change the dimension you do the average on?</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000"><br>
</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000">--Mary</font></span></div>
<div class="gmail_default"><span style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#000000"><br>
</font></span></div>
<div class="gmail_extra"><font color="#000000"><br>
</font>
<div class="gmail_quote"><font color="#000000">On Tue, Jan 12, 2016 at 8:30 AM, Alan Brammer
<span dir="ltr"><<a href="mailto:abrammer@albany.edu" target="_blank">abrammer@albany.edu</a>></span> wrote:<br>
</font>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div style="word-wrap:break-word"><span style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)"><font color="#000000">Hi Michael, </font></span>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font>
<div><font color="#000000"><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)">I was going to suggest reshape that data array but it’s 20GB and is going to be unnecessarily slow whatever. Do you actually need
to store all the hourly data? the below edits suggest that you don’t. The below uses less than a 1GB of memory rather than 20+GB. </span></font><br>
</font>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000"><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)"> This is </span></font><span style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)">obviously untested
so may need editing. </span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)">(requires 6.1.1 or newer. )</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font color="#000000"><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)"><span>
<div>mns=(/"01","02","03","04","05","06","07","08","09","10","11","12"/)</div>
<div>ndays=(/31,28,31,30,31,30,31,31,30,31,30,31/)</div>
<div><br>
</div>
</span>
<div>daily=new((/141,217,20,12,31/),float) ; hourly data</div>
<div>daily@_FillValue=1e+35</div>
<div>daily=1e+35</div>
<span>
<div><br>
</div>
<div>cnt=0</div>
<div>do iyr=0,19</div>
<div> do im=0,11</div>
<div> prefix=(1980+iyr)+""+mns(im)</div>
<div> b=addfile("/volumes/data1/yafang/Downscaling/ACCESS1-0/historical/output/ACCESS_SRF."+(1980+iyr)+""+mns(im)+"<a href="http://0100.nc" target="_blank">0100.nc</a>","r") ; read hourly SRF data</div>
<div> iy=b->iy</div>
<div> jx=b->jx</div>
</span>
<div> xlat=b->xlat <span style="white-space:pre-wrap"> </span>; These aren’t doing anything?</div>
<div> xlon=b->xlon <span style="white-space:pre-wrap"></span>; These aren’t doing anything?</div>
</span></font><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)">
<div> snow =b->snv ; liquid equiv of snow on ground</div>
<div> dims=dimsizes(snow)</div>
<div> nt=dims(0)</div>
<div><br>
</div>
<div> snow4d := reshape( snow(iy|:,jx|:,time|:), (/dims(1), dims(2), ndays(im), 24/) ) ; I assume snow is originally (time|:,iy|:,ix|:)</div>
<div> daily(:,:,iyr,im,:ndays(im)-1)=dim_avg_n(snow4d, 3)</div>
<span>
<div><br>
</div>
<div> delete(snow)</div>
<div> delete(b)</div>
<div> delete(dims)</div>
<div> cnt=cnt+1</div>
<div> end do</div>
<div>end do</div>
<div><br>
</div>
</span>
<div>daily@_FillValue=1e+20</div>
<div><br>
</div>
</span></font></font></div>
<div><font color="#000000"><br>
</font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)">Good luck, </span></font></div>
<span><font color="#000000">
<div><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif"><span style="background-color:rgb(255,255,255)">Alan Brammer.</span></font></div>
</font></span>
<div>
<div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font color="#000000"><br>
</font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font></div>
<div><font face="Calibri, Arial, Helvetica, sans-serif" color="#000000"><span style="background-color:rgb(255,255,255)"><br>
</span></font>
<div>
<blockquote type="cite">
<div><font color="#000000">On 12 Jan 2016, at 10:00, Michael Notaro <<a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a>> wrote:</font></div>
<font color="#000000"><br>
</font>
<div>
<div style="font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)">
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Thanks for your email.</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Actually, this is the main part slowing me down, not the top part of the code with the addfiles.</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<p style="margin-top:0px; margin-bottom:0px"></p>
<blockquote type="cite">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif">
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">cnt=0</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">do id=0,30</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> daily(:,:,:,:,id)=dim_avg(data(:,:,:,:,cnt:cnt+23)) ; convert hourly to daily data ***** THIS PART IS SLOW *****</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> cnt=cnt+24</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">end do</font></div>
</div>
</div>
</blockquote>
<div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif">
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Any way to perform this task quicker?</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Michael</font></div>
</div>
</div>
</div>
<font color="#000000"><br>
</font>
<p style="margin-top:0px; margin-bottom:0px"></p>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)">
<font color="#000000">Michael Notaro<br>
</font>
<div><font color="#000000">Associate Director<br>
</font></div>
<div><font color="#000000">Nelson Institute Center for Climatic Research<br>
</font></div>
<div><font color="#000000">University of Wisconsin-Madison<br>
</font></div>
<div><font color="#000000">Phone: <a href="tel:%28608%29%20261-1503" value="+16082611503" target="_blank">
(608) 261-1503</a><br>
</font></div>
<div><font color="#000000">Email:<span> </span><a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a><br>
</font></div>
</div>
</div>
<font color="#000000"><br>
<br>
</font>
<div><font color="#000000">
<hr style="display:inline-block; width:1120.125px">
</font>
<div dir="ltr"><font face="Calibri, sans-serif" color="#000000"><b>From:</b><span> </span>Guido Cioni <<a href="mailto:guidocioni@gmail.com" target="_blank">guidocioni@gmail.com</a>><br>
<b>Sent:</b><span> </span>Tuesday, January 12, 2016 8:57 AM<br>
<b>To:</b><span> </span>Michael Notaro<br>
<b>Cc:</b><span> </span><a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
<b>Subject:</b><span> </span>Re: [ncl-talk] Slow code</font>
<div><font color="#000000"> </font></div>
</div>
<div><font color="#000000">Everyone here will tell you that using loops in NCL it’s not efficient :)</font>
<div><font color="#000000">But from my experience I think that the main thing slowing you down is that you are using addfile at every iteration. </font></div>
<div><font color="#000000">Does creating a whole file and reading that in the beginning would change what you are trying to compute? <br>
</font>
<div>
<div style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<font color="#000000"><br>
Guido Cioni</font></div>
<div style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<font color="#000000"><a href="http://guidocioni.altervista.org/" target="_blank">http://guidocioni.altervista.org</a> </font></div>
<div style="margin-bottom:20px; overflow:auto; width:1143px">
<table style="width:1028px; overflow:auto; padding-top:20px; padding-bottom:20px; margin-top:20px; border-top-width:1px; border-top-style:dotted; border-top-color:rgb(200,200,200); border-bottom-width:1px; border-bottom-style:dotted; border-bottom-color:rgb(200,200,200); background-color:rgb(255,255,255)" cellspacing="0">
<tbody>
<tr style="border-spacing:0px" valign="top">
<td colspan="1" style="width:250px; display:table-cell; padding-right:20px">
<div style="min-height:145px; margin:auto; display:table; width:250px; background-color:rgb(255,255,255)">
<a href="http://guidocioni.altervista.org/" target="_blank" style="display:table-cell; text-align:center"><font color="#000000"><img style="display: inline-block; margin-left: auto; margin-right: auto; max-width: 250px; max-height: 250px; min-height: 145px; width: 250px; border-width: 0px; vertical-align: bottom; -moz-user-select: none;" height="145" width="250" src="http://guidocioni.altervista.org/nuovosito/wp-content/uploads/2015/10/Screenshot.png"></font></a></div>
</td>
<td colspan="2" style="vertical-align:top; padding:0px; display:table-cell">
<div></div>
<div style="font-weight:normal; font-family:wf_segoe-ui_light,'Segoe UI Light','Segoe WP Light','Segoe UI','Segoe WP',Tahoma,Arial,sans-serif; line-height:21px">
<a href="http://guidocioni.altervista.org/" target="_blank" style="text-decoration:none"><font color="#000000">Guido Cioni</font></a></div>
<div style="margin:10px 0px 16px; font-weight:normal; font-family:wf_segoe-ui_normal,'Segoe UI','Segoe WP',Tahoma,Arial,sans-serif; line-height:14px">
<a href="http://guidocioni.altervista.org/" target="_blank"><font color="#000000">guidocioni.altervista.org</font></a></div>
<div style="display:block; font-weight:normal; font-family:wf_segoe-ui_normal,'Segoe UI','Segoe WP',Tahoma,Arial,sans-serif; line-height:20px; max-height:100px; overflow:hidden">
<font color="#000000">Le stazioni sono state riparate ed i dati vengono nuovamente aggiornati in tempo reale. Purtroppo a causa di numerosi malfunzionamenti i dati pluviometrici di Pisa e ...</font></div>
</td>
</tr>
</tbody>
</table>
</div>
<font color="#000000"><br>
</font></div>
<font color="#000000"><br>
</font>
<div>
<blockquote type="cite">
<div><font color="#000000">On 12 Jan 2016, at 15:35, Michael Notaro <<a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a>> wrote:</font></div>
<font color="#000000"><br>
</font>
<div>
<div style="font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)">
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Does anyone have a recommendation to speed up my code?</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">It's been running for a day now. I put asterisks next to the real slow loop.</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Basically, that part is converting a large array of hourly data into daily data.</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">Thanks, Michael</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<p style="margin-top:0px; margin-bottom:0px"></p>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">mns=(/"01","02","03","04","05","06","07","08","09","10","11","12"/)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">ndays=(/31,28,31,30,31,30,31,31,30,31,30,31/)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">data=new((/141,217,20,12,744/),float) ; hourly data</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">data@_FillValue=1e+35</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">data=1e+35</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">cnt=0</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">do iyr=0,19</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> do im=0,11</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> prefix=(1980+iyr)+""+mns(im)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> b=addfile("/volumes/data1/yafang/Downscaling/ACCESS1-0/historical/output/ACCESS_SRF."+(1980+iyr)+""+mns(im)+"<a href="http://0100.nc" target="_blank">0100.nc</a>","r") ; read hourly SRF
data</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> iy=b->iy</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> jx=b->jx</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> xlat=b->xlat</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> xlon=b->xlon</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> snow=b->snv ; liquid equiv of snow on ground</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> dims=dimsizes(snow)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> nt=dims(0)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> data(:,:,iyr,im,0:nt-1)=snow(iy|:,jx|:,time|:)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> delete(snow)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> delete(b)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> delete(dims)</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> cnt=cnt+1</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> end do</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">end do</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">data@_FillValue=1e+20</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">daily=new((/141,217,20,12,31/),float) ; daily data per month</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">daily@_FillValue=1e+35</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">daily=1e+35</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">cnt=0</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">do id=0,30</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> daily(:,:,:,:,id)=dim_avg(data(:,:,:,:,cnt:cnt+23)) ; convert hourly to daily data ***** THIS PART IS SLOW *****</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"> cnt=cnt+24</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">end do</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000">delete(data)</font></div>
<font color="#000000"><br>
</font>
<p style="margin-top:0px; margin-bottom:0px"></p>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div style="margin-top:0px; margin-bottom:0px"><font color="#000000"><br>
</font></div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)">
<font color="#000000">Michael Notaro<br>
</font>
<div><font color="#000000">Associate Director<br>
</font></div>
<div><font color="#000000">Nelson Institute Center for Climatic Research<br>
</font></div>
<div><font color="#000000">University of Wisconsin-Madison<br>
</font></div>
<div><font color="#000000">Phone: <a href="tel:%28608%29%20261-1503" value="+16082611503" target="_blank">
(608) 261-1503</a><br>
</font></div>
<div><font color="#000000">Email:<span> </span><a href="mailto:mnotaro@wisc.edu" target="_blank">mnotaro@wisc.edu</a><br>
</font></div>
</div>
</div>
</div>
<font color="#000000"><span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">_______________________________________________</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">ncl-talk
mailing list</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<a href="mailto:ncl-talk@ucar.edu" target="_blank" style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">ncl-talk@ucar.edu</a><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">List instructions,
subscriber options, unsubscribe:</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a></font></div>
</blockquote>
</div>
<font color="#000000"><br>
</font></div>
</div>
</div>
</div>
<font color="#000000"><span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">_______________________________________________</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">ncl-talk
mailing list</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<a href="mailto:ncl-talk@ucar.edu" target="_blank" style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">ncl-talk@ucar.edu</a><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<span style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; float:none; display:inline!important">List instructions,
subscriber options, unsubscribe:</span><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br style="font-family:Helvetica; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px">
</font></div>
</blockquote>
</div>
<font color="#000000"><br>
</font></div>
</div>
</div>
</div>
</div>
</div>
<font color="#000000"><br>
_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br>
</font></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</body>
</html>