[ncl-talk] Why could Killed happen?

[Rick Brownrigg]

Hi Harry,

A simple "killed" message is Linux's very unhelpful manner of saying it has
direly run out  out of memory (google that for plenty of
references/discussion). It is neither an NCL or a Python issue.

Without better understanding of your script, I'm not sure what to suggest.
Perhaps process your files in smaller batches? You are aware that
"delete"ing no-longer-used variables frees their memory?

Rick

On Fri, Jun 14, 2019 at 4:11 AM Ian Harris (ENV - Staff) <I.Harris at uea.ac.uk>
wrote:

> Hi,
>
> I’m running a long and slightly cumbersome NCL script - with callouts to
> WRAPITted Fortran subs. It takes about 8 days to complete, as it’s reading
> a lot of high res gridded files (HARMONIE/V1 reanalysis), regridding them,
> and producing an aligned-with-CRU TS forcing dataset 1901-2018. It also
> sends each file via scp as it’s produced.
>
> However, I’ve now had two instances of it just.. dying. The most recent
> was this morning (UK time):
>
> (0) Fri 14 Jun 2019 04:47:42 BST: top of monthly loop with iy=2005 and
> im=8
> (0) Fri 14 Jun 2019 04:47:42 BST: top of variable loop with
> var=TMP_P0_L103_GLC0
> cruhar_v3_2005_08_3hourly_Tairalign.nc
>                         100%  162MB  10.8MB/s   00:15
> (0) /Volumes/VERIFY2/output_v3/cruhar_v3_2005_08_3hourly_Tairalign.nc
> successfully uploaded
> (0) Fri 14 Jun 2019 04:48:48 BST: top of variable loop with
> var=PRES_P0_L1_GLC0
> cruhar_v3_2005_08_3hourly_Psurfnoalign.nc
>                          100%  162MB   9.5MB/s   00:17
> (0) /Volumes/VERIFY2/output_v3/cruhar_v3_2005_08_3hourly_Psurfnoalign.nc
> successfully uploaded
> (0) Fri 14 Jun 2019 04:49:55 BST: top of variable loop with
> var=WDIR_P0_L103_GLC0
> (0) Fri 14 Jun 2019 04:50:39 BST: top of variable loop with
> var=WIND_P0_L103_GLC0
> Killed
> [imac:~/WORKING/VERIFY] ianharris%
>
> Now where does that 'Killed' come from? Is the system killing it? I see
> nothing in the system logs. I have plenty of RAM (double what I had the
> first time it Killed), so I can’t see it being that.
>
> Any suggestions would be helpful!
>
> Cheers
>
> Harry
> PS Yes I’m sure it won’t happen with Python :)
>
> Ian 'Harry'Harris | NCAS-Climate | Climatic Research Unit |School
> of Environmental Sciences | 01.02 Hubert Lamb Building | Faculty of
> Science | University of East Anglia | Norwich Research Park | Norwich |NR4
> 7TJ
> Tel: +44 1603 593 818   Fax: +44 1603 591 327
>
> I work 80%FTE, normally Tuesday-Friday
>
>
> Gold (Teaching Excellence Framework 2017-2020)
> UK Top 15 (The Times/Sunday Times 2019 and Complete University Guide 2019)
> World Top 200 (Times Higher Education World University Rankings 2019)
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 6210 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 1179 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 1726 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 1799 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 1733 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 1732 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 1784 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.png
Type: image/png
Size: 2063 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.png
Type: image/png
Size: 936 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190614/fbdeb190/attachment-0007.png>