<div dir="ltr">I have opened a JIRA ticket on this. Will try for 6.3.0 but may not make it in for that release.<br><div><br>NCL-2174: readAsciiTable: make thread safe<br><br></div><div>THX <br></div><div>D<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 12, 2015 at 4:59 AM, <span dir="ltr"><<a href="mailto:Daniel.Leuenberger@meteoswiss.ch" target="_blank">Daniel.Leuenberger@meteoswiss.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="DE-CH">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Kyle,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Thanks for this very helpful solution. I implemented and tested your suggestion and it works like a charm! Additionally I was made aware of the
unix utility “mktemp” (default available on most linux distributions) which also is intended to generate unique file names and which could also be a good solution.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Best regards<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Daniel<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US"> <a href="mailto:windrunnerxc@gmail.com" target="_blank">windrunnerxc@gmail.com</a> [mailto:<a href="mailto:windrunnerxc@gmail.com" target="_blank">windrunnerxc@gmail.com</a>]
<b>On Behalf Of </b>Kyle Griffin<br>
<b>Sent:</b> Dienstag, 10. März 2015 16:38<br>
<b>To:</b> Leuenberger Daniel<br>
<b>Cc:</b> ncl-talk<br>
<b>Subject:</b> Re: [ncl-talk] readAsciiTable not thread safe<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi Daniel,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">That's an excellent point. It looks like the function is designed to try and call a random file name with the 'echo tmp$$' call, but that returns a single number for each shell. I assume if multiple sub-processes are being run from a single
shell, this would cause the files to overlap.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">However, since this function is in the contributed.ncl file, you can change it on your system as a temporary fix. Although NCL does have a rand() function, it is always seeded with the same numbers upon initialization of a new NCL process
and therefore can be expected to produce predictable numbers with each call. You could try to use srand(x) or random_setallseed(x,y) with some component of the time, but for the sake of producing consistently unique numbers, I would recommend using a component
of the date as the actual random number. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I've used the nanosecond return from the UNIX date function, systemfunc("date +%N") in NCL, to get the job done. Producing a very large number of random numbers a day and up to five simultaneously and I have yet to have any issues with
duplicate numbers. If you are using NCL on Mac OS X, you can get the same result by installing the GNU-compatible version of date, typically referenced as "gdate" in MacPort or Homebrew or similar UNIX-ish package manager.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Anyway, the fix you'd be looking for is below. It's not particularly robust, but it should fix your problem while the developers might look to address this in the future. This can be used in place of the line<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">tmpfile = systemfunc("echo tmp$$") <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">near the end of the readAsciiTable function. This is line 9062 in <span style="font-size:10.0pt;font-family:Courier;color:black">$NCARG_ROOT/lib/ncarg/nclscripts/csm/</span><a href="https://www.ncl.ucar.edu/Document/Functions/Contributed/contrib.shtml" target="_blank"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#852d85;text-decoration:none">contributed.ncl</span></a> if
you're using version 6.2.1 and have write permissions in the directory where NCL is installed. If not, find the readAsciiTable function in the contributed.ncl file, copy it into a new file and make the changes. Then you can "load" the new file at the top of
your script after your loading of the contributed.ncl file.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> tmpnum = systemfunc("date +%N")</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> if(tmpnum.eq."N")then</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> delete(tmpnum)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> srand(toint(systemfunc("date +%s")))</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> tmpnum = rand()</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> end if</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> tmpfile = "tmp"+tmpnum</span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif"">Hope this has made sense. If not, feel free to reply back and include the list with questions. I'm sure the developers will look at
this a bit closer in the future, as making functions parallel-safe appears to be one of the things they are working on alongside the ongoing development.</span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Kyle<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">----------------------------------------<u></u><u></u></p>
<div>
<p class="MsoNormal">Kyle S. Griffin<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Department of Atmospheric and Oceanic Sciences<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">University of Wisconsin - Madison<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Room 1421<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1225 W Dayton St, Madison, WI 53706<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Email: <a href="mailto:ksgriffin2@wisc.edu" target="_blank">
ksgriffin2@wisc.edu</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Mar 10, 2015 at 9:46 AM, <<a href="mailto:Daniel.Leuenberger@meteoswiss.ch" target="_blank">Daniel.Leuenberger@meteoswiss.ch</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Dear NCL Team,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">When running a large number of parallel NCL jobs using the readAsciiTable function (contributed.ncl) I realized that the function is not thread safe, e.g. may
fail. The reason is that it writes a temporary ascii file and deletes it again at the end of the function. If now two parallel jobs want to write to the temporary file at exactly the same time, one is not allowed and will crash. One method to circumvent the
problem would be to use a random number and/or the process ID in the file name of the temporary file.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Best regards</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">Daniel Leuenberger</span></b><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">Federal Department of Home Affairs FDHA</span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">Federal Office of Meteorology and Climatology MeteoSwiss</span></b><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">All about weather and climate, please visit our new website</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif""><a href="http://www.meteoschweiz.admin.ch/web/en.html" title="Web" target="_blank"><span lang="EN-US">www.meteoswiss.ch</span></a></span><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">
and </span><span style="font-size:10.0pt;font-family:"Arial","sans-serif""><a href="http://www.meteoswiss.admin.ch/home/services-and-publications/beratung-und-service/meteoswiss-app.html" title="Web" target="_blank"><span lang="EN-US">MeteoSwiss App</span></a></span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
ncl-talk mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
ncl-talk mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div>