[ncl-talk] Regridding data on unstructured grid with ESMF_Regrid
Mary Haley
haley at ucar.edu
Wed Nov 19 15:43:21 MST 2014
Hi Peter,
That's certainly unpleasant to have everything look like it's going well
and the seg fault near the end. All the ESMF debug output looks good, and
usually the "sparse_matrix_mult" part is the part that goes quickly and
always works.
My guess is that it ran out of memory somewhere.
Would it be possible for you to provide the full script and necessary files
so I can run it here? You can provide this information offline if
preferred.
You can use ftp, Google Drive, or dropbox. The ftp information is here:
http://www.ncl.ucar.edu/report_bug.shtml#HowToFTP
Thanks,
--Mary
On Wed, Nov 19, 2014 at 5:34 AM, Peter Watson <watson at atm.ox.ac.uk> wrote:
> Hi again all,
>
> This is a follow-on from a previous conversation I began when I was having
> trouble regridding data from an irregular grid (a reduced Gaussian grid) to
> a lat-lon grid. I waited for the next version of NCL to be released, but
> then I suddenly had to do work for another project and have only been able
> to pick this up again now. The problem does not seem to have been fixed in
> NCL 6.2.1 - I get a segmentation fault. Is there anything I can do to get
> this to work, or does NCL just not have the capacity to do this for now?
>
> My input data on an irregular grid is in a file called "regrid_cons_rg.nc",
> and an ncdump -h on this file gives:
>
> netcdf regrid_cons_rg {
> dimensions:
> space = 35718 ;
> corner = 4 ;
> variables:
> double data(space) ;
> double lon1d(space) ;
> double lat1d(space) ;
> double corner_lons(space, corner) ;
> double corner_lats(space, corner) ;
> }
>
> corner_lons and corner_lats give the positions of the corners of each grid
> cell.
>
> I tried running the following NCL code to do regridding to a 5x5 lat-lon
> grid (the same as I posted before):
>
> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
> load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"
> file_in=addfile("regrid_cons_rg.nc","r")
> data = file_in->data
> lats = file_in->lat1d
> lons = file_in->lon1d
> corner_lats = file_in->corner_lats
> corner_lons = file_in->corner_lons
> Opt=True
> Opt at SrcGridLat = lats
> Opt at SrcGridLon = lons
> Opt at SrcGridCornerLat = corner_lats
> Opt at SrcGridCornerLon = corner_lons
> Opt at DstGridType = "5x5"
> Opt at ForceOverwrite = True
> Opt at InterpMethod="conserve"
> Opt at DstLLCorner = (/-85.0d, 0.0d /)
> Opt at DstURCorner = (/ 85.0d, 355.0d /)
>
> Opt at Debug = True
>
> data_latlon=ESMF_regrid(data,Opt)
>
>
> This gives the following output, ending in a segmentation fault:
>
> (0) get_src_grid_info: source lat dims = (35718)
> (0) get_src_grid_info: source lon dims = (35718)
> (0) get_src_grid_info: source grid type is 'unstructured'
> (0) unstructured_to_ESMF: triangulating the data ... this can be slow
> (0) min/max ElementVertices = 0/35717
> (0) unstructured_to_ESMF: total number of elements created: 71432
> (0) unstructured_to_ESMF: Element Area: min:4.629831710012411e-06
> max:0.0002461602913053085
> (0) curvilinear_to_SCRIP: calculating grid corners...
> (0) curvilinear_to_SCRIP: no lat values are at the poles, so
> (0) calculating grid corners using
> (0) calc_SCRIP_corners_noboundaries...
> (0) calc_SCRIP_corners_noboundaries
> (0) min/max original lat: -85/85
> (0) min/max original lon: 0/355
> (0) calc_SCRIP_corners_noboundaries
> (0) min/max Extlat2d: -90/90
> (0) min/max Extlon2d: -5/360
> (0) calc_SCRIP_corners_noboundaries
> (0) min/max ExtGridCenter_lat: -87.5/87.5
> (0) min/max ExtGridCenter_lon: -2.5/357.5
> (0) ESMF_regrid_gen_weights: number of processors used: 1
> (0) --------------------------------------------------
> (0) ESMF_regrid_gen_weights: the following command is about to be
> executed on the system:
> (0) 'ESMF_RegridWeightGen --source source_grid_file.nc --destination
> destination_grid_file.nc --weight weights_file.nc --method conserve
> --src_type ESMF -i --norm_type fracarea'
> (0) --------------------------------------------------
> (0) ESMF_regrid_gen_weights: output from 'ESMF_RegridWeightGen':
> (0) Starting weight generation with these inputs:
> (1) Source File: source_grid_file.nc
> (2) Destination File: destination_grid_file.nc
> (3) Weight File: weights_file.nc
> (4) Source File is in ESMF format
> (5) Source Grid is a global grid
> (6) Source Grid is an unstructured grid
> (7) Destination File is in SCRIP format
> (8) Destination Grid is a global grid
> (9) Destination Grid is a logically rectangular grid
> (10) Regrid Method: conserve
> (11) Pole option: NONE
> (12) Ignore unmapped destination points
> (13) Norm Type: fracarea
> (14)
> (15) Completed weight generation successfully.
> (16)
> (0) --------------------------------------------------
> (0) ESMF_regrid_gen_weights: 'ESMF_RegridWeightGen' was successful.
> (0) ESMF_regrid_with_weights: regridding using interpolation weights
> ...
> (0) ESMF_regrid_with_weights: warning: destination grid is not
> (0) completely covered by the source grid. This is not an error.
> (0) It just means your destination grid covers a larger area
> (0) than your source grid.
> (0) ESMF_regrid_with_weights: Source Grid:
> (0) rank: 1
> (0) dimensions: 71432
> (0) original source rank: 1
> (0) latitude min/max:
> -89.14151942646112/89.14151942646112
> (0) longitude min/max:0.375/358.5
> (0) ESMF_regrid_with_weights: Destination Grid:
> (0) dimensions: 35 72
> (0) latitude min/max: -85/85
> (0) longitude min/max:0/355
> (0) ESMF_regrid_with_weights: retrieving interpolation weights ...
> (0) ESMF_regrid_with_weights: calling sparse_matrix_mult to apply
> weights...
> Segmentation fault
>
>
> The code works if I specify "patch" as the regridding method.
>
>
> Kind regards,
>
> Peter Watson
>
> On Tue, Aug 26, 2014 at 3:35 AM, Mary Haley <haley at ucar.edu> wrote:
>
>> I'm hoping next week for a release of V6.2.1. I want to get to get it
>> out by Tuesday after labor day.
>>
>> --Mary
>>
>>
>>
>> On Mon, Aug 25, 2014 at 11:06 AM, Peter Watson <watson at atm.ox.ac.uk>
>> wrote:
>>
>>> Thanks Mary. Trying that gave me a segmentation fault, though. When
>>> will the next version of NCL be released?
>>>
>>> Kind regards,
>>>
>>> Peter
>>>
>>>
>>> On Mon, Aug 25, 2014 at 5:58 PM, Mary Haley <haley at ucar.edu> wrote:
>>>
>>>> Hi Peter,
>>>>
>>>> You may have run into a bug that we fixed in the upcoming release of
>>>> NCL. I've attached a new "ESMF_regridding.ncl" script. Try saving it to
>>>> the same directory as your script, and then comment out this line:
>>>>
>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"
>>>>
>>>> and add this line after or before it:
>>>>
>>>> load "./ESMF_regridding.ncl"
>>>>
>>>> I'm not entirely sure this will work, because you may need to have
>>>> the latest version of NCL to use this, but it's worth a try.
>>>>
>>>> --Mary
>>>>
>>>>
>>>> On Mon, Aug 25, 2014 at 10:37 AM, Peter Watson <watson at atm.ox.ac.uk>
>>>> wrote:
>>>>
>>>>> Thanks for your help Dennis. I tried this again, this time
>>>>> providing the corner information as shown below, but got the same error
>>>>> message. My input netCDF file regrid_cons_rg.nc (note I changed the
>>>>> filename) is now structured as follows:
>>>>>
>>>>> netcdf regrid_cons_rg {
>>>>> dimensions:
>>>>> space = 35718 ;
>>>>> corner = 4 ;
>>>>>
>>>>> variables:
>>>>> double data(space) ;
>>>>> double lon1d(space) ;
>>>>> double lat1d(space) ;
>>>>> double corner_lons(space, corner) ;
>>>>> double corner_lats(space, corner) ;
>>>>> }
>>>>>
>>>>> I added the corner_lons and corner_lats arrays, which define
>>>>> tessellating rectangular cells centred on each grid point (except at the
>>>>> northernmost and southernost latitudes, where the cells stretch to the
>>>>> pole). I transferred the file regrid_cons_rg.nc by ftp as before.
>>>>>
>>>>> The following lines of ncl:
>>>>>
>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"
>>>>> file_in=addfile("regrid_cons_rg.nc","r")
>>>>>
>>>>> data = file_in->data
>>>>> lats = file_in->lat1d
>>>>> lons = file_in->lon1d
>>>>> corner_lats = file_in->corner_lats
>>>>> corner_lons = file_in->corner_lons
>>>>>
>>>>> Opt=True
>>>>> Opt at SrcGridLat = lats
>>>>> Opt at SrcGridLon = lons
>>>>> Opt at SrcGridCornerLat = corner_lats
>>>>> Opt at SrcGridCornerLon = corner_lons
>>>>>
>>>>> Opt at DstGridType = "5x5"
>>>>> Opt at ForceOverwrite = True
>>>>> Opt at InterpMethod="conserve"
>>>>>
>>>>> Opt at DstLLCorner = (/-85.0d, 0.0d /)
>>>>> Opt at DstURCorner = (/ 85.0d, 355.0d /)
>>>>>
>>>>> Opt at Debug = True
>>>>>
>>>>> data_latlon=ESMF_regrid(data,Opt)
>>>>>
>>>>>
>>>>> give the error:
>>>>>
>>>>> (0) ESMF_regrid_gen_weights: 'ESMF_RegridWeightGen' was successful.
>>>>> (0) ESMF_regrid_with_weights: regridding using interpolation
>>>>> weights ...
>>>>> (0) ESMF_regrid_with_weights: warning: destination grid is not
>>>>> (0) completely covered by the source grid. This is not an error.
>>>>> (0) It just means your destination grid covers a larger area
>>>>> (0) than your source grid.
>>>>> (0) ESMF_regrid_with_weights: Source Grid:
>>>>> (0) rank: 1
>>>>> (0) dimensions: 71432
>>>>> (0) original source rank: 1
>>>>> (0) latitude min/max: -89.1415/89.1415
>>>>> (0) longitude min/max:0.375/358.5
>>>>> (0) ESMF_regrid_with_weights: Destination Grid:
>>>>> (0) dimensions: 35 72
>>>>> (0) latitude min/max: -85/85
>>>>> (0) longitude min/max:0/355
>>>>>
>>>>> (0) ESMF_regrid_with_weights: retrieving interpolation weights ...
>>>>> (0) ESMF_regrid_with_weights: error: source data on the description
>>>>> (0) file does not have proper dimensions.
>>>>>
>>>>>
>>>>> Is it possible to tell what is going wrong here? Or is there another
>>>>> way of doing this?
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>>>> On Sat, Aug 23, 2014 at 3:28 AM, Dennis Shea <shea at ucar.edu> wrote:
>>>>>
>>>>>> The attached works with the bilinear and patch methods. The
>>>>>> conserve method requires 'corner' information. NCL's implementation tries
>>>>>> to derive the corner info but is not always successful. I think that is
>>>>>> what is happening in your case. This will be looked at more when we get a
>>>>>> chance.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 21, 2014 at 1:52 PM, Peter Watson <watson at atm.ox.ac.uk>
>>>>>> wrote:
>>>>>>
>>>>>>> Dear fellow NCL users,
>>>>>>>
>>>>>>> I am trying to do conservative regridding of precipitation data
>>>>>>> from a reduced Gaussian grid to a lat-lon grid using the ESMF_Regrid
>>>>>>> function in NCL 6.1.2, but I get an error that I do not understand. My
>>>>>>> input data is in a NetCDF file, with the precip data and the latitudes and
>>>>>>> longitudes of the grid points stored as 1D arrays. An ncdump -h of the
>>>>>>> input file gives:
>>>>>>>
>>>>>>> dimensions:
>>>>>>> space = 35718 ;
>>>>>>> variables:
>>>>>>> double data(space) ;
>>>>>>> double lon1d(space) ;
>>>>>>> double lat1d(space) ;
>>>>>>>
>>>>>>>
>>>>>>> I use the following lines of NCL, based on the example script at
>>>>>>> http://www.ncl.ucar.edu/Applications/Templates/ESMF_unstruct_to_0.25deg.ncl
>>>>>>> :
>>>>>>>
>>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
>>>>>>> load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"
>>>>>>>
>>>>>>> file_in=addfile("regrid_cons_irreg.nc","r")
>>>>>>> data = file_in->data
>>>>>>> lats = file_in->lat1d
>>>>>>> lons = file_in->lon1d
>>>>>>>
>>>>>>> Opt=True
>>>>>>> Opt at SrcGridLat = lats
>>>>>>> Opt at SrcGridLon = lons
>>>>>>> Opt at DstGridType = "5x5"
>>>>>>> Opt at ForceOverwrite = True
>>>>>>> Opt at InterpMethod = "conserve"
>>>>>>> Opt at DstLLCorner = (/-85.0d, 0.0d /)
>>>>>>> Opt at DstURCorner = (/ 85.0d, 355.0d /)
>>>>>>>
>>>>>>> Opt at Debug = True
>>>>>>>
>>>>>>> data_latlon=ESMF_regrid(data,Opt)
>>>>>>>
>>>>>>> This gives the error:
>>>>>>>
>>>>>>> (0) ESMF_regrid_with_weights: retrieving interpolation weights ...
>>>>>>> (0) ESMF_regrid_with_weights: error: source data on the
>>>>>>> description
>>>>>>> (0) file does not have proper dimensions.
>>>>>>>
>>>>>>> I don't understand why this fails. Is there something wrong with
>>>>>>> how I am reading in the data and coordinates? Are there any conditions that
>>>>>>> need to be satisfied, like the coordinates increasing monotonically (mine
>>>>>>> do not)? If anyone has a script and NetCDF file for which this sort of
>>>>>>> regridding succeeds that they could let me see, then that would be very
>>>>>>> helpful.
>>>>>>>
>>>>>>> Thanks very much for any help you can give.
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Peter Watson
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ncl-talk mailing list
>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20141119/c770f5f2/attachment.html
More information about the ncl-talk
mailing list