<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Laura (and Rick),</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I think there are issues with the Intel version of ESMF_RegridWeightGen, which generates the weights file. We've had other issues trying to build this application using the Intel compiler, and we've been in touch with the ESMF group about it.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Meanwhile, things seem to work if I use the GNU version of NCL on cheyenne:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">module load gnu</div><div class="gmail_default" style="font-size:small">module load ncl</div><div class="gmail_default" style="font-size:small">which ncl<br>/glade/u/apps/ch/opt/ncl/6.4.0/gnu/6.2.0/bin/ncl</div><div class="gmail_default"><p class="gmail-p1" style="font-size:small"><span class="gmail-s1">I just ran your script, and it completed with the following output:</span></p><p class="gmail-p1">(0)<span style="white-space:pre"> </span>ESMF_regrid_with_weights: retrieving interpolation weights ...<br>(0)<span style="white-space:pre"> </span>ESMF_regrid_with_weights: calling sparse_matrix_mult to apply weights...<br>(0)<span style="white-space:pre"> </span>ESMF_regrid_with_weights: dstData<br>(0)<span style="white-space:pre"> </span> Dimensions: 1798 3598<br>(0)<span style="white-space:pre"> </span> minSrcData: 1.5258789063e-05<br>(0)<span style="white-space:pre"> </span> maxSrcData: 0.999069673989844<br>(0)<span style="white-space:pre"> </span> minDstData: 1.5258789063e-05<br>(0)<span style="white-space:pre"> </span> maxDstData: 0.9990587636447192</p><p class="gmail-p1">Variable: mpas_regrid<br>Type: double<br>Total Size: 51753632 bytes<br>. . .<br>Number of Dimensions: 2<br>Dimensions and sizes:<span style="white-space:pre"> </span>[lat | 1798] x [lon | 3598]<br>Coordinates: <br> lat: [-89.875..89.875]<br> lon: [0.125..359.875]</p><p class="gmail-p1">Number Of Attributes: 3<br> missing_value :<span style="white-space:pre"> </span>9.969209968386869e+36<br> remap :<span style="white-space:pre"> </span>remapped via ESMF_regrid_with_weights: Bilinear<br> _FillValue :<span style="white-space:pre"> </span>9.969209968386869e+36</p></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_default" style="font-size:small">--Mary</div><div class="gmail_default" style="font-size:small"></div><br><div class="gmail_quote">On Tue, Oct 24, 2017 at 5:14 PM, Laura Fowler <span dir="ltr"><<a href="mailto:laura@ucar.edu" target="_blank">laura@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Rick:<br>
<br>
Thank you for looking into this. The newer mesh I am trying to regrid<br>
has 6488066 cells while the mesh I regridded a couple of years ago had<br>
more cells (6848514). So I did not think that it was the size of the<br>
mesh that was the culprit.<br>
<br>
Here is what I found:<br>
<br>
1. Using the largest mesh (6848514 cells) and on yellowstone, I tested<br>
the regrid script I used a couple of years ago using ncl/6.3.0 and<br>
ncl/<a href="http://6.4.0." rel="noreferrer" target="_blank">6.4.0.</a> I did not have any issue. You can look at the source,<br>
destination, and weight files in the directories<br>
/glade2/scratch2/laura/ncl/<wbr>yellowstone.ncl-6.3.0 and<br>
/glade2/scratch2/laura/ncl/<wbr>yellowstone.ncl-6.4.0. The ncl script is<br>
regrid.to_CMORPHdata.ncl<br>
<br>
<br>
2. Then, I tested the same script but on cheyenne and I got the same<br>
SIGSEGV using ncl/6.4.0 or ncl/<a href="http://6.3.0." rel="noreferrer" target="_blank">6.3.0.</a> So it seems that it may have to<br>
do with ncl on cheyenne only.<br>
<br>
<br>
3. Unfortunately, I cannot use yellowstone to regrid my newest mesh<br>
since it is uses the cdf5 format which I cannot read on yellowstone.<br>
<br>
<br>
Hope that this helps to resolve this issue.<br>
<span class="HOEnZb"><font color="#888888">Laura<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
<br>
<br>
<br>
On Tue, Oct 24, 2017 at 4:19 PM, Rick Brownrigg <<a href="mailto:brownrig@ucar.edu">brownrig@ucar.edu</a>> wrote:<br>
> Just to follow up on this, the message regarding max-value-size is related<br>
> to the debugger, so fixed limits within the ESMF software are likely not the<br>
> issue.<br>
><br>
> Nonetheless, the SEGV is occuring in the ESMF software and appears to be<br>
> happening in the NetCDF library, function<br>
> netcdf_expanded.f90::nf90_get_<wbr>var_2d_fourbyteint(). For anyone else<br>
> looking into this, the line number is 1960, and a link to the current source<br>
> is:<br>
><br>
> <a href="https://github.com/Unidata/netcdf-fortran/blob/master/fortran/netcdf_expanded.f90" rel="noreferrer" target="_blank">https://github.com/Unidata/<wbr>netcdf-fortran/blob/master/<wbr>fortran/netcdf_expanded.f90</a><br>
><br>
> (I don't know what version of NetCDF ESMF may be linked against, but that<br>
> line number is in the right function).<br>
><br>
><br>
><br>
> On Tue, Oct 24, 2017 at 3:48 PM, Rick Brownrigg <<a href="mailto:brownrig@ucar.edu">brownrig@ucar.edu</a>> wrote:<br>
>><br>
>> Hi Laura,<br>
>><br>
>> I don't really know much about the regridding process, but what I have<br>
>> been able to surmise running the script:<br>
>><br>
>> i) NCL reads the MPAS, and creates source*.nc and destination*.nc files.<br>
>> These appear to reflect the geometry of the src/dest grids<br>
>><br>
>> ii) The actual regridding is is done by ESMF software, with a command:<br>
>><br>
>> ESMF_RegridWeightGen --source <a href="http://source_grid_file.nc" rel="noreferrer" target="_blank">source_grid_file.nc</a> --destination<br>
>> <a href="http://destination_grid_file.nc" rel="noreferrer" target="_blank">destination_grid_file.nc</a> --weight<br>
>> <a href="http://weights_onCells.15-3Mesh_to_0.15rectangular.nc" rel="noreferrer" target="_blank">weights_onCells.15-3Mesh_to_0.<wbr>15rectangular.nc</a> --src_type ESMF -i<br>
>><br>
>> This program SEGVs almost immediately, with a message:<br>
>><br>
>> "values=<error reading variable: value requires 155713536 bytes, which is<br>
>> more than max-value-size>...."<br>
>><br>
>> That value is exactly the size of one of the variables in the source*.nc<br>
>> file. So it looks like some internal limit is being exceeded in the ESMF<br>
>> software.<br>
>><br>
>> Is this one of the larger MPAS files you've attempted to regrid? I wonder<br>
>> if anyone else can comment on this? Those on the glade file system can see<br>
>> all the relevant files under /glade/scratch/brownrig<br>
>><br>
>> I'm not sure what to tell you as a work-around. Without a debug version of<br>
>> the code, its nearly impossible for me to tell much more or to detemine<br>
>> what the limits might be. Wish I had a better answer.<br>
>><br>
>> Rick<br>
>><br>
>><br>
>><br>
>> On Tue, Oct 24, 2017 at 10:13 AM, Laura Fowler <<a href="mailto:laura@ucar.edu">laura@ucar.edu</a>> wrote:<br>
>>><br>
>>> Hi:<br>
>>><br>
>>> I am trying to regrid an MPAS unstructured mesh to a rectangular mesh<br>
>>> on cheyenne using ncl/<a href="http://6.4.0." rel="noreferrer" target="_blank">6.4.0.</a> My script crashes with a SIGSEGV and I am<br>
>>> not understanding where this comes from. I have done this successfully<br>
>>> in the past but do not see what I am doing wrong right now, but I<br>
>>> recall that it was with an ealier version of ncl. I am attaching the<br>
>>> output of my script in regrid.to_rectMesh.out.<br>
>>><br>
>>> The script itself can be found in<br>
>>><br>
>>> /glade2/scratch2/laura/MPAS.<wbr>PacificOcean/initialization.<wbr>centeredPacificOceanMesh.15-<wbr>3km/regrid.to_rectMesh.ncl.<br>
>>> I also tried to regrid the same MPAS mesh to an other unstructured<br>
>>> mesh and got a similar SIGSEGV (see regrid.to_2621442Mesh.ncl), so I<br>
>>> assume that the errors are the same.<br>
>>><br>
>>> Hope you can help me figure this one out.<br>
>>> Thanks,<br>
>>> Laura<br>
>>><br>
>>><br>
>>> --<br>
>>><br>
>>> !-----------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------------------<br>
>>> Laura D. Fowler<br>
>>> Mesoscale and Microscale Meteorology Division (MMM)<br>
>>> National Center for Atmospheric Research<br>
>>> P.O. Box 3000, Boulder CO 80307-3000<br>
>>><br>
>>> e-mail: <a href="mailto:laura@ucar.edu">laura@ucar.edu</a><br>
>>> phone: <a href="tel:303-497-1628" value="+13034971628">303-497-1628</a><br>
>>><br>
>>><br>
>>> !-----------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------------------<br>
>>><br>
>>> ______________________________<wbr>_________________<br>
>>> ncl-talk mailing list<br>
>>> <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
>>> List instructions, subscriber options, unsubscribe:<br>
>>> <a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
>>><br>
>><br>
><br>
<br>
<br>
<br>
--<br>
!-----------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------------------<br>
Laura D. Fowler<br>
Mesoscale and Microscale Meteorology Division (MMM)<br>
National Center for Atmospheric Research<br>
P.O. Box 3000, Boulder CO 80307-3000<br>
<br>
e-mail: <a href="mailto:laura@ucar.edu">laura@ucar.edu</a><br>
phone: <a href="tel:303-497-1628" value="+13034971628">303-497-1628</a><br>
<br>
!-----------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------------------<br>
______________________________<wbr>_________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
</div></div></blockquote></div><br></div></div>