[ncl-talk] optimization problem

[Marston Johnston]

Hi Mary,

Thanks for looking at this. Unfortunately the idea you put forth doesn't
work.
I played around with it and it was my thought to play around with the
"where" statement, but the problem is that I need to find all the
points in mhs5_2d
= a single value in char AND all the points tb5m4_2d = to a single value in
darr. The way you did it doesn't work for the intended purpose, which is to
produce an output similar to PDFXY, but returning the counts (sum) instead
of a procent or fraction.
Originally, mhm(4,:,:) doesn't have the same dimensions as darr and char,
but I can easily change this. However, this doesn't solve the problem.
If we take PDFXY as a template, how do I get PDFXY to return the counts?

/M


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Marston S. Johnston, PhD
Department of Earth Sciences
University of Gothenburg, Sweden
Email: marston.johnston at gu.se
Phone: +46-31-7864901
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Only the fruitful thing is true!

On Tue, Jul 26, 2016 at 6:55 PM, Mary Haley <haley at ucar.edu> wrote:

> ​Marston,
>
> In general, I recommend trying to use array arithmetic over looping, even
> if it means using extra memory to create 2D versions of 1D arrays.
>
> If you have enough memory to create 2D arrays of your various 1D arrays,
> then I recommend replacing all of the code after the last "copy_VarMeta"
> call with the following (UNTESTED) code:
>
> I'm not exactly sure I have all the dimensions of your various arrays
> correct, so this code may need some correction. For example, I don't know
> what the dimensions of "mhs" are, so I'm not sure that I have that part
> right.  If mhs is already dimensioned ndarr x nchar, then the mhsx_2d lines
> should look simply like this:
>
>   mhs5_2d := round(mhs(4,:,:),0)
>
>
> ; Create 2D versions of these arrays for faster (?) calculations later
>
> dims_2d = (/ndarr,nchar/)
> mhs5_2d = conform_dims(dims_2d,round(ndtooned(mhs(4,:,:)),0),1)
> mhs4_2d = conform_dims(dims_2d,round(ndtooned(mhs(3,:,:)),0),1)
> mhs3_2d = conform_dims(dims_2d,round(ndtooned(mhs(2,:,:)),0),1)
>
> tb5m4_2d = conform_dims(dims_2d,round(ndtooned(tb5m4),0),0)
> tb5m3_2d = conform_dims(dims_2d,round(ndtooned(tb5m3),0),0)
> tb4m3_2d = conform_dims(dims_2d,round(ndtooned(tb4m3),0),0)
>
> darr_2d = conform_dims(dims_2d,darr,0)
> char_2d = conform_dims(dims_2d,char,1)
>
> ; This creates a 2D logical array of Trues/Falses
> mhs5_eq_char = mhs5_2d .eq. char_2d
>
> ; Create 2D arrays containing 1s and 0s
> tmp5 = where(mhs5_eq_char .and. tb5m4_2d .eq. darr_2d, 1, 0)
> tmp4 = where(mhs5_eq_char .and. tb5m3_2d .eq. darr_2d, 1, 0)
> tmp3 = where(mhs5_eq_char .and. tb4m3_2d .eq. darr_2d, 1, 0)
>
> hist5m4 = hist5m4 + tmp5
> hist5m3 = hist5m3 + tmp4
> hist4m3 = hist4m3 + tmp3​
>
> delete([/mhs5_eq_char,
> mhs5_2d,mhs4_2d,mhs3_2d,darr_2d,char_2d,tmp5,tmp4,tmp3/])
>
> On Tue, Jul 26, 2016 at 6:05 AM, Marston Johnston <shejo284 at gmail.com>
> wrote:
>
>> Hi,
>>
>> I'm wondering if there is a way to speed up the do loops in the code
>> below?
>> I'm not sure it can be done, but I'm not an expert.
>>
>> ;***** 2D histogram ******
>> ; x and y axis
>> darr = ispan(-80,80,1)
>> sarr = new(dimsizes(darr),string)
>> sarr = " "
>> char = ispan(150,300,1)
>>
>> ; histogram bins
>> hist5m4 = new((/dimsizes(darr),dimsizes(char)/),float)
>> hist5m4 at _FillValue = default_fillvalue("float")
>> hist5m4!0 = "diff"
>> hist5m4!1 = "Tb"
>> hist5m4&diff = darr
>> hist5m4&Tb = char
>> ndarr = dimsizes(darr)
>> nchar = dimsizes(char)
>> hist5m4 = 0.0
>> hist5m3 = hist5m4
>> hist5m3 = 0.0
>> hist4m3 = hist5m4
>> hist4m3 = 0.0
>>
>> tb5m4 = mhs(4,:,:) - mhs(3,:,:)
>> copy_VarMeta(mhs(3,:,:),tb5m4)
>> tb5m3 = mhs(4,:,:) - mhs(2,:,:)
>> copy_VarMeta(mhs(2,:,:),tb5m3)
>> tb4m3 = mhs(3,:,:) - mhs(2,:,:)
>> copy_VarMeta(mhs(2,:,:),tb5m3)
>>
>> mhs5 := round(ndtooned(mhs(4,:,:)),0)
>> mhs4 := round(ndtooned(mhs(3,:,:)),0)
>> mhs3 := round(ndtooned(mhs(2,:,:)),0)
>>
>> tb5m4 := round(ndtooned(tb5m4),0)
>> tb5m3 := round(ndtooned(tb5m3),0)
>> tb4m3 := round(ndtooned(tb4m3),0)
>>
>> ; print(darr(y)+" <= DARR:CH5AR => "+ch5ar(x))
>> ; Fill the bins
>> do x = 0, nchar - 1
>>    do y = 0, ndarr - 1
>>       tmp := ind(mhs5.eq.char(x) .and. tb5m4.eq.darr(y))
>>       if(dimsizes(tmp).gt.1) then
>>         hist5m4(y,x) = hist5m4(y,x) + (dimsizes(tmp) - 1)
>>       end if
>>       tmp := ind(mhs5.eq.char(x) .and. tb5m3.eq.darr(y))
>>       if(dimsizes(tmp).gt.1) then
>>         hist5m3(y,x) = hist5m3(y,x) + (dimsizes(tmp) - 1)
>>       end if
>>       tmp := ind(mhs4.eq.char(x) .and. tb4m3.eq.darr(y))
>>       if(dimsizes(tmp).gt.1) then
>>         hist4m3(y,x) = hist4m3(y,x) + (dimsizes(tmp) - 1)
>>       end if
>>   end do
>> end do
>>
>> Appreciate your help with this,
>> /M
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Marston S. Johnston, PhD
>> Department of Earth Sciences
>> University of Gothenburg, Sweden
>> Email: marston.johnston at gu.se
>> Phone: +46-31-7864901
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Only the fruitful thing is true!
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160726/df5f5988/attachment.html