[ncl-talk] "graphic" array cannot store many figures?

[music piano]

Hi,  everyone

Sorry for bothering you. Thanks for Alan's detail suggestions. I like your
idea that process the data before plotting. I didn't show my original code
which is very lengthy, in which I actually process the data inside ncl
since it has convenient functions to deal with WRF data, such as
interpolating to pressure levels.

The code I show here is just a structure to demonstrate the problem, in
which I made a typo error(should be level, not k). I actually use panel
plot to organize my figure, not just drawing one figure as I showed in this
email. Thus, drawing before the panel plot is not suitable. That is why I
need an "graphic array" to store all the figures.

The code actually stops before the panel plot. So I think it is not the
problem of panel plot. I am still interested to know why the script stops
working when I increase the number of loops.

Anybody has more suggestions?

Best regards
Richard


On Fri, Nov 10, 2017 at 2:41 PM, Alan Brammer <abrammer at albany.edu> wrote:

> There might be some copy/paste errors in your email and I don't know about
> limits on graphic array sizes but a couple comments before someone from the
> NCL team can reply.
>
>
> Possibly most importantly, given that you're drawing individual plots, why
> not just plot in the first loop and drop the dependence on the array.
>
> Plots have a bunch of attributes connected to them I feel like using the
> (/ /) notation will likely disrupt how NCL handles graphics and tickmarks
> etc.  Although if this works on smaller loops maybe it doesn't matter.
>
> Why use gsn_panel() for a single plot? Why not just draw() and frame(),
> and why frame() after gsn_panel() instead of just letting gsn_panel() do
> it.
>
> Your 2nd nested loop uses level as the iterator but you use k as the index
> in the gsn_panel call.  (copy paste error or actual typo?)
>
> Finally, if you're processing 650GB of data, it's probably best to process
> the data and save the processed output to a netcdf. Then have a second
> script that plots the processed data. This way you can play with the
> plotting criteria without having to run through all of the data again.
>
> Or if you're just digging through raw data which based on the loops you
> may well be doing, panoply, ncview or similar would allow you to just
> visualise the data without having to process it at all. Are you sure you
> want 972 pages in a single pdf?
>
>
>
>
>
> Good luck,
>
> Alan
>
>
> On Fri, Nov 10, 2017 at 2:17 PM, music piano <musicpianoljy at gmail.com>
> wrote:
>
>> Hi, everyone
>>
>> I produced many figures and passed to panel plot. I define a "graphic"
>> array to store my figures.
>>
>> The following script is the essential part to produce the figure. There
>> are many files, the code read one file at a time. It loops over many times,
>> many variables and many vertical levels.
>>
>> After all the figures were stored in plot_1_3d array. the panel plot
>> plot them. *This code works fine* when plot_1_3d is a smaller array. For
>> instance, ntimes=4, num_var=2, nlevels=18.
>>
>>  However, when I loop over many files, such as ntimes=27, the program
>> stops without any error message when it loops to it=12 or some other
>> number. Each of the file the program read is 25GB, but I delete old
>> variables in every time loop. The only array exists over the whole period
>> is the "graphic" array. I am not sure it is the memory issue. I run this
>> code on a supercomputer node, which has  32GB memory. The output pdf file
>> for the figure is 43MB if ntimes=4, nlevels=18, num_var=2.
>>
>> I will be very appreciated if anybody can help me.
>>
>> My scripts:
>> plot_1_3d=new((/ntimes,num_var,nlevels/),"graphic")
>>
>> do it=0, ntimes-1
>>     do var_id=num_var-1
>>               do level=nlevels-1
>>                          ........................... ; I skip my code here
>>                         plot_1_3d(it,var_id,level) =
>> contour_data_1_3d(var_id,level)
>>                end do
>>      end do
>> end do
>>
>> do it=0, ntimes-1
>>     do var_id=num_var-1
>>               do level=nlevels-1
>>                       gsn_panel(wks,(/plot_1_3d(it,var_id,k)/),(/1,1/),resP
>>     )
>>                        frame(wks)
>>       end do
>>      end do
>> end do
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171110/c7cd4111/attachment.html>