[Wrf-users] What is a reasonable speed for WRF / how to increase it?

Tue Sep 6 11:24:23 MDT 2016

Hi Yaoping,

> The segfault problem with OpenMP seems hopeless. The "ulimit -s unlimited" command did not help. I also debugged another error, but it did not help either. I am giving it up for now. 

did you try to set/increase the OMP stacksize? You can set the limit using the environment variable OMP_STACKSIZE (KMP_STACKSIZE for Intel compilers).

Cheers

Dom

> On 3/09/2016, at 7:26 AM, Wang, Yaoping <wang.3866 at buckeyemail.osu.edu> wrote:
> 
> Hi, 
> 
> This is embarrassing - the speed problem was solved. The -d option was on when I built WRF months ago!
> 
> 
> For completeness, the compile-time error when building with openmp was: 
> 
> ../../frame/module_wrf_error.o: In function `MODULE_WRF_ERROR':
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:4: undefined reference to `__kmpc_global_thread_num'
> ../../frame/module_wrf_error.o: In function `wrf_message':
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:166: undefined reference to `__kmpc_global_thread_num'
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:174: undefined reference to `__kmpc_master'
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:174: undefined reference to `__kmpc_end_master'
> ../../frame/module_wrf_error.o: In function `wrf_message2':
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:192: undefined reference to `__kmpc_global_thread_num'
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:195: undefined reference to `__kmpc_master'
> /fs/project/PAS0661/WRF3.8_3/WRFV3/frame/module_wrf_error.f90:195: undefined reference to `__kmpc_end_master'
> make[2]: [diffwrf] Error 1 (ignored)
> 
> And was solved by adding "-openmp" into FCBASEOPTS_NO_G
> 
> Thank you,
> Yaoping Wang
> From: Reginaldo Ventura de Sá <regis at lamma.ufrj.br <mailto:regis at lamma.ufrj.br>>
> Sent: Saturday, September 3, 2016 8:47:00 AM
> To: Wang, Yaoping
> Subject: Re: [Wrf-users] What is a reasonable speed for WRF / how to increase it?
>  
> 
> hello, 
> 
> try the command:
> 
> ulimit -s unlimited  
> 
> 
> Regis
> 
> 
> 
> On Fri, Sep 2, 2016 at 7:33 PM, Wang, Yaoping <wang.3866 at buckeyemail.osu.edu <mailto:wang.3866 at buckeyemail.osu.edu>> wrote:
> Hi, 
> 
> Could you explain more about the "using 2 process in one core" and how to find out/address it? I am not very familiar with the technical aspect of supercomputing. I thought one core in supercomputing means one CPU, but did you mean that each CPU is further made up of multiple "little-cores" themselves, or 2 cores can be on the same CPU?
> 
> I am compiling MPI using the intel compiler. But here is another problem. OpenMP does not work at all on my system. The compiler finishes successfully, but then, when I run "./real.exe", it segfaults without even creating an rsl file. I tried using "./configure -d", and updating my WRF version from 3.8 to 3.8.1, but the segfault was the same. Do you know what else I might do? I attached my "configure.wrf" and "configure.wps" files. 
> 
> I also tested 44 cores with no luck. Increasing the number of nodes from 4 to 6 only increased the throughput marginally. 
> 
> Thank you, 
> Yaoping
> From: wrf-users-bounces at ucar.edu <mailto:wrf-users-bounces at ucar.edu> <wrf-users-bounces at ucar.edu <mailto:wrf-users-bounces at ucar.edu>> on behalf of Mike Dvorak <mike at sailtactics.com <mailto:mike at sailtactics.com>>
> Sent: Thursday, September 1, 2016 5:05:28 PM
> To: wrf-users at ucar.edu <mailto:wrf-users at ucar.edu>
> Subject: Re: [Wrf-users] What is a reasonable speed for WRF / how to increase it?
>  
> Hi Yaoping,
> 
> What parallelization option did you compile WRF with (e.g. MPI only)? Also, I've found the Intel compilers to be 3 times faster than the GNU compilers on some WRF configurations (unfortunately). What compiler did you use?
> 
> You may also want to experiment using less than the number of total cores on the machine. For example, you could try using 44 cores instead of 48. I think WRF EMS is set to do this by default. I've verified on some of my multi-core machines that this does indeed reduce the runtime.
> 
> Cheers,
> Mike
> 
> 
> On 09/01/2016 03:15 PM, Carlos Ross wrote:
>> I think it should be faster, Xeon x5650 CPUs are 6 cores and 12 threads, so you maybe using 2 process in one core and that is slowing it down.
>> 
>> 2016-08-31 18:36 GMT-03:00 Wang, Yaoping <wang.3866 at buckeyemail.osu.edu <mailto:wang.3866 at buckeyemail.osu.edu>>:
>> Hi All, 
>> 
>> I am running WRF on a ~6km resolution, 91 x 121 domain in the eastern United States. I am using an adaptive time step which makes it mostly 72 sec increments. There are 34 vertical levels. I use 4 x 12 cores on a Intel Xeon x5650 CPUs machine. The throughput is about 1.2 hour wall time per 24 hours model time. 
>> 
>> Is this a reasonable speed? I found some information here (http://www.ecmwf.int/sites/default/files/elibrary/2014/13662-performance-analysis-operational-implementaion-wrf.pdf <http://www.ecmwf.int/sites/default/files/elibrary/2014/13662-performance-analysis-operational-implementaion-wrf.pdf>) and after considering the domain difference, my run still seems a touch slow. And is there anyway I could figure how to make the model run faster?
>> 
>> Thank you, 
>> Yaoping Wang
>> 
>> _______________________________________________
>> Wrf-users mailing list
>> Wrf-users at ucar.edu <mailto:Wrf-users at ucar.edu>
>> http://mailman.ucar.edu/mailman/listinfo/wrf-users <http://mailman.ucar.edu/mailman/listinfo/wrf-users>
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Wrf-users mailing list
>> Wrf-users at ucar.edu <mailto:Wrf-users at ucar.edu>
>> http://mailman.ucar.edu/mailman/listinfo/wrf-users <http://mailman.ucar.edu/mailman/listinfo/wrf-users>
> 
> -- 
> <logo.png>
> Mike Dvorak, PhD
> Founder
> Sail Tactics, LLC
> Corpus Christi, TX
> +1 650-454-5243 <tel:%2B1%20650-454-5243>
> http://sailtactics.com <http://sailtactics.com/>
> 
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu <mailto:Wrf-users at ucar.edu>
> http://mailman.ucar.edu/mailman/listinfo/wrf-users <http://mailman.ucar.edu/mailman/listinfo/wrf-users>
> 
> 
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu <mailto:Wrf-users at ucar.edu>
> http://mailman.ucar.edu/mailman/listinfo/wrf-users <http://mailman.ucar.edu/mailman/listinfo/wrf-users>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/wrf-users/attachments/20160906/ae0807e3/attachment-0001.html 