[mpas-developers] Fix for isolated cells
Michael Duda
duda at ucar.edu
Thu Oct 14 11:27:39 MDT 2010
Hi, Mark.
Excellent work in tracking this down, and thanks for the very
concise description of the problem. The fix you proposed looks
good to me; unless anyone else has comments, I'd agree with
committing this whenever you're ready.
Cheers,
Michael
On Thu, Oct 14, 2010 at 08:32:17AM -0600, Mark Petersen wrote:
> mpas developers,
>
> We have had trouble with our global domains with land, and I have a
> bug fix that solves the problem. The error occurs on runs with more
> than one processor, where a processor has an isolated ocean cell. For
> example:
>
> 1 1 1 L L L 2 2 2
> 1 1 1 L L 1 2 2 2
> 1 1 1 L L L 2 2 2
>
> These are cells owned by processor 1 and 2, and land cells L are shown
> but are not really cells.
>
> In module block_decomp, subroutine block_decomp_add_halo, the section
>
> do i=1,local_graph_info % nVertices
> do j=1,local_graph_info % nAdjacent(i)
> if (local_graph_info % adjacencyList(j,i) /= 0) then
> if (.not. hash_search(h, local_graph_info % adjacencyList(j,i))) then
> call hash_insert(h, local_graph_info % adjacencyList(j,i))
> end if
> end if
> end do
> end do
>
> only adds cells to the hash table if they have neighbors owned by that
> processor. Thus isolated cells are not added,
>
> local_graph_with_halo % nVerticesTotal = hash_size(h)
>
> is too small, the warning
>
> block_decomp_add_halo: Somehow we don't have the right number of total
> cells.
>
> appears, and we end up reading outside of memory bounds on arrays like
> local_graph_with_halo % vertexID.
>
> My fix is to add all cells to the hash table first, and then add
> neighbors of cells:
>
> lo1-fe> pwd
> /usr/projects/climate/mpeterse/coy/mpas_100603/trunk/mpas/src/framework
> lo1-fe> svn diff module_block_decomp.F
> Index: module_block_decomp.F
> ===================================================================
> --- module_block_decomp.F (revision 528)
> +++ module_block_decomp.F (working copy)
> @@ -236,6 +236,10 @@
> call hash_init(h)
>
> do i=1,local_graph_info % nVertices
> + call hash_insert(h, local_graph_info % vertexID(i))
> + end do
> +
> + do i=1,local_graph_info % nVertices
> do j=1,local_graph_info % nAdjacent(i)
> if (local_graph_info % adjacencyList(j,i) /= 0) then
> if (.not. hash_search(h, local_graph_info % adjacencyList(j,i))) then
>
> A more elegant solution would be for the domain decomposition to not
> have isolated cells, but kmetis handles this and I don't know how to
> change it. Also, users should be able to choose any grid and
> decomposition they like without unnecessary errors. A warning would be
> more appropriate.
>
> I will make the above change tomorrow, unless I hear other feedback.
>
> Mark
>
>
>
> _______________________________________________
> mpas-developers mailing list
> mpas-developers at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/mpas-developers
More information about the mpas-developers
mailing list