[ncl-talk] sign_f90 slow down work flow

[Marston Johnston]

Where to begin….

It is not easy for anyone to simple solve your problem for you. You have to do a lot of the work yourself. The issue here has nothing to do with NCL but rather your programming skills.
The speed is a function of many things such as how much memory you have available, the size of the files/arrays, and the way you write your script.
You are doing many heavy-duty things in a single script. NCL is rather high-level so it will not be as fast as low-level languages. But even in low-level languages it is wise to do things in 
chunks that uses memory efficiently.  

   1.)	One trick you can do is the read in the files in the order you are joining them, do your necessary calculations and then write them to a netcdf in the same order. This would be 1 separate script, for example. This way has many advantages: you can examine the data in stages so that when you make mistakes it is easier to find. 
   2.)	So, you need to redesign you script and break it into self-contained pieces. Do this on paper before you write the scripts.	

3.) This is a bottleneck and would slow down any language. CDO can do this very efficiently using the “expr” feature.

do ll=1,84
  do k=2,61
    do i=1,387
      do j=1,387               
        pterm = 0.5 + sign_f90(0.5, (tdf(ll,i,j,k)-260.0))
        tema = pterm*17.2694 + (1.-pterm)*21.87456
        temb = pterm*35.86   + (1.-pterm)*7.66
        SWVP = 610.78*exp(tema*(tdf-273.16)/(tdf(ll,i,j,k)-temb))
  end do
  end do
  end do
  end do
        printVarSummary(SWVP)           ; [*]   

Hope this helps,
/M

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Marston S. Ward, PhD
Department of Earth Sciences
University of Gothenburg, Sweden
Email: marston.johnston at gu.se
SkypeID: marston.johnston 
Phone: +46-31-7864901 
Only the fruitful thing is true!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

On 2017-10-19, 11:10, "ncl-talk on behalf of Paolina Bongioannini Cerlini" <ncl-talk-bounces at ucar.edu on behalf of paolina.cerlini at unipg.it> wrote:

    Hi NCL people,
    
    I am working with this script attached that reads 81 netcdf files-and extracting a 
    variable from them is already slow-
    then I added the sign_f90 function to compute another variable, and it is lasting forever.
    I am sure that ther is a better way to apply this function without entering this
    4 indexes loop, but I don't see how. 
    Can you help me?
    Thank you.
    
    Paolina
    
    -- 
    Paolina Bongioannini Cerlini Ph.D. 
    University of Perugia 
    CIRIAF -room 8 
    06125 - Perugia 
    
    Tel.+39 075 5853576 
    e-mail: paolina.cerlini at unipg.it 
    skype: pcerlini 
    --
    _______________________________________________
    ncl-talk mailing list
    ncl-talk at ucar.edu
    List instructions, subscriber options, unsubscribe:
    http://mailman.ucar.edu/mailman/listinfo/ncl-talk