<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:10pt;color:#000000;background-color:#FFFFFF;font-family:Mangal,Sans-Serif;">
<p>Hi, </p>
<p><br>
</p>
<p><span style="font-size: 13.3333px;">This is embarrassing - </span><span style="font-size: 10pt;">the speed problem was solved</span><span style="font-size: 10pt;">.
</span><span style="font-size: 10pt;">The -d option was on when I built WRF months ago!</span></p>
<p><span style="font-size: 10pt;"></span></p>
<p><span style="font-size: 10pt;"><br>
</span></p>
<p><span style="font-size: 10pt;">The segfault problem with OpenMP seems hopeless. The "ulimit -s unlimited" command did not help. I also debugged another error, but it did not help either. I am giving it up for now. </span></p>
<p><span style="font-size: 10pt;"><br>
</span></p>
<p><span style="font-size: 10pt;">For completeness, the compile-time error when building with openmp was: </span></p>
<p><span style="font-size: 10pt;"><br>
</span></p>
<p><span style="font-size: 10pt;"></p>
<div>../../frame/module_wrf_error.o: In function `MODULE_WRF_ERROR':</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:4: undefined reference to `__kmpc_global_thread_num'</div>
<div>../../frame/module_wrf_error.o: In function `wrf_message':</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:166: undefined reference to `__kmpc_global_thread_num'</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:174: undefined reference to `__kmpc_master'</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:174: undefined reference to `__kmpc_end_master'</div>
<div>../../frame/module_wrf_error.o: In function `wrf_message2':</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:192: undefined reference to `__kmpc_global_thread_num'</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:195: undefined reference to `__kmpc_master'</div>
<div>/fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:195: undefined reference to `__kmpc_end_master'</div>
<div>make[2]: [diffwrf] Error 1 (ignored)</div>
<div><br>
</div>
And was solved by adding "-openmp" into <span>FCBASEOPTS_NO_G</span></span>
<p></p>
<p><br>
</p>
<p>Thank you,</p>
<p>Yaoping Wang</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Reginaldo Ventura de Sá <regis@lamma.ufrj.br><br>
<b>Sent:</b> Saturday, September 3, 2016 8:47:00 AM<br>
<b>To:</b> Wang, Yaoping<br>
<b>Subject:</b> Re: [Wrf-users] What is a reasonable speed for WRF / how to increase it?</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div><br>
</div>
hello, <br>
<br>
</div>
try the command:<br>
<br>
</div>
ulimit -s unlimited <br>
<br>
<br>
</div>
Regis<br>
<div><br>
<br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Sep 2, 2016 at 7:33 PM, Wang, Yaoping <span dir="ltr">
<<a href="mailto:wang.3866@buckeyemail.osu.edu" target="_blank">wang.3866@buckeyemail.osu.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div style="font-size:10pt;color:#000000;background-color:#ffffff;font-family:Mangal,Sans-Serif">
<p>Hi, </p>
<p><br>
</p>
<p><span>Could you explain more <span>about the "using 2 process in one core" and how to find out/address it? I am not very familiar with the technical aspect of supercomputing. I thought one core in supercomputing means one CPU, but did you mean that each
CPU is further made up of multiple "little-cores" themselves, or 2 cores can be on the same CPU?</span></span></p>
<p><br>
</p>
<p><span><span>I am compiling MPI using the intel compiler</span></span><span style="font-size:10pt">. But here is another problem. OpenMP</span><span style="font-size:10pt"> does not work at all on my system. The compiler finishes successfully, but then, when
I run "./real.exe", it segfaults without even creating an rsl file. I tried using "./configure -d", and updating my WRF version from 3.8 to 3.8.1, </span><span style="font-size:10pt">but the segfault was the same.
</span><span style="font-size:10pt">Do you </span><span style="font-size:10pt">know</span><span style="font-size:10pt">
</span><span style="font-size:10pt">what else</span><span style="font-size:10pt"> I might </span><span style="font-size:10pt">do</span><span style="font-size:10pt">? I attached my "configure.wrf" and "configure.wps" files. </span></p>
<p><span style="font-size:10pt"></span></p>
<p><br>
</p>
<p><span><span>I also tested 44 cores with no luck. Increasing the number of nodes from 4 to 6 only increased the throughput marginally. </span></span></p>
<p><span><span><br>
</span></span></p>
<p><span><span>Thank you, </span></span></p>
<p><span><span>Yaoping</span></span></p>
</div>
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
<a href="mailto:wrf-users-bounces@ucar.edu" target="_blank">wrf-users-bounces@ucar.edu</a> <<a href="mailto:wrf-users-bounces@ucar.edu" target="_blank">wrf-users-bounces@ucar.edu</a>> on behalf of Mike Dvorak <<a href="mailto:mike@sailtactics.com" target="_blank">mike@sailtactics.com</a>><br>
<b>Sent:</b> Thursday, September 1, 2016 5:05:28 PM<br>
<b>To:</b> <a href="mailto:wrf-users@ucar.edu" target="_blank">wrf-users@ucar.edu</a><br>
<b>Subject:</b> Re: [Wrf-users] What is a reasonable speed for WRF / how to increase it?</font>
<div> </div>
</div>
<div>
<div class="h5">
<div>Hi Yaoping,<br>
<br>
What parallelization option did you compile WRF with (e.g. MPI only)? Also, I've found the Intel compilers to be 3 times faster than the GNU compilers on some WRF configurations (unfortunately). What compiler did you use?<br>
<br>
You may also want to experiment using less than the number of total cores on the machine. For example, you could try using 44 cores instead of 48. I think WRF EMS is set to do this by default. I've verified on some of my multi-core machines that this does indeed
reduce the runtime.<br>
<br>
Cheers,<br>
Mike<br>
<br>
<br>
<div>On 09/01/2016 03:15 PM, Carlos Ross wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I think it should be faster, <span style="color:rgb(0,0,0);font-family:mangal,sans-serif;font-size:13.3333px">Xeon x5650 CPUs are 6 cores and 12 threads, so you maybe using 2 process in one core and that is slowing it down.</span></div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2016-08-31 18:36 GMT-03:00 Wang, Yaoping <span dir="ltr">
<<a href="mailto:wang.3866@buckeyemail.osu.edu" target="_blank">wang.3866@buckeyemail.osu.edu</a><wbr>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-size:10pt;color:#000000;background-color:#ffffff;font-family:Mangal,Sans-Serif">
<p>Hi All, </p>
<p><br>
</p>
<p>I am running WRF on a ~6km resolution, <span style="font-size:10pt">91 x 121 domain in the eastern United States. I am using an adaptive time step which makes it mostly 72 sec increments. </span><span style="font-size:10pt">There are 34 vertical levels. I
use 4 x 12 cores on a Intel Xeon x5650 CPUs machine. </span><span style="font-size:10pt">The throughput is about 1.2 hour wall time per 24 hours model time. </span></p>
<p><span style="font-size:10pt"></span></p>
<p><span style="font-size:10pt"><br>
</span></p>
<p><span style="font-size:10pt">Is this a reasonable speed? I found some information here (<a href="http://www.ecmwf.int/sites/default/files/elibrary/2014/13662-performance-analysis-operational-implementaion-wrf.pdf" target="_blank">http://www.ecmwf.int/sites/de<wbr>fault/files/elibrary/2014/1366<wbr>2-performance-analysis-operati<wbr>onal-implementaion-wrf.pdf</a>)
and after considering the domain difference, my run still seems a touch slow. And i</span><span style="font-size:10pt">s there anyway I could figure how to make the model run faster?</span></p>
<p><span style="font-size:10pt"><br>
</span></p>
<p><span style="font-size:10pt">Thank you, </span></p>
<span><font color="#888888">
<p><span style="font-size:10pt">Yaoping Wang</span></p>
</font></span></div>
</div>
<br>
______________________________<wbr>_________________<br>
Wrf-users mailing list<br>
<a href="mailto:Wrf-users@ucar.edu" target="_blank">Wrf-users@ucar.edu</a><br>
<a href="http://mailman.ucar.edu/mailman/listinfo/wrf-users" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailma<wbr>n/listinfo/wrf-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset> <br>
<pre>______________________________<wbr>_________________
Wrf-users mailing list
<a href="mailto:Wrf-users@ucar.edu" target="_blank">Wrf-users@ucar.edu</a>
<a href="http://mailman.ucar.edu/mailman/listinfo/wrf-users" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/wrf-users</a>
</pre>
</blockquote>
<br>
<div>-- <br>
<table height="126" border="0" cellpadding="2" cellspacing="2" width="15%">
<tbody>
<tr>
<td valign="center"><img alt="Sail Tactics logo" height="118" width="90" src="cid:part5.FD4A0143.10B55550@sailtactics.com"><br>
</td>
<td valign="top">Mike Dvorak, PhD<br>
Founder<br>
Sail Tactics, LLC<br>
Corpus Christi, TX<br>
<a href="tel:%2B1%20650-454-5243" value="+16504545243" target="_blank">+1 650-454-5243</a><br>
<a href="http://sailtactics.com" target="_blank">http://sailtactics.com</a><br>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Wrf-users mailing list<br>
<a href="mailto:Wrf-users@ucar.edu">Wrf-users@ucar.edu</a><br>
<a href="http://mailman.ucar.edu/mailman/listinfo/wrf-users" rel="noreferrer" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/wrf-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>