[ncl-talk] ESMF-Regrid Large Files (> 4 Gb)

[Dennis Shea]

Bonjour,

For testing, I would suggest that you operate on a decimated (sub-sampled)
array.
This will make for quick turn around and verify that the options you are
using are correct

   f= addfile("...","r")

   jy = 10
   ix = 10
   x= f->X(::jy,::ix)        ; subsampled array
   lat = f->LAT(::jy,::ix)
   lon = .....
etc

---
Currently, NCL's implementation of the ESMF regrid software uses a single
processor. The underlying code is capable of parallel operation but (to my
knowledge) this has not been implemented.

---
Also, when sending to ncl-talk, it is nice to have a 'printVarSummary(...)'
of the variable(s) you are using.
Or a ncl_filedump ...

Bon Chance!



On Thu, Jul 16, 2015 at 8:44 AM, Gabriele Arduini <
Gabriele.Arduini at legi.grenoble-inp.fr> wrote:

> Dear ncl-support,
>
> I am trying to regrid a large high-resolution dataset (7200 lat x 7200
> lon) on a coarser resolution grid using
> the ESMF_regrid tool.
> Everything is going well, but during the weight generation (that takes a
> huge amount of time...)
> I receive this error::
>
> (0)     --------------------------------------------------
> (0)     ESMF_regrid_gen_weights: the following command is about to be
> executed on the system:
> (0)     'ESMF_RegridWeightGen --source source_grid.nc --destination
> dest_grid.nc --weight weight_file --method patch --src_regional
> --dst_regional -i'
> (0)     --------------------------------------------------
> (0)     ESMF_regrid_gen_weights: output from 'ESMF_RegridWeightGen':
> (0)           Starting weight generation with these inputs:
> (1)             Source File: source_grid.nc
> (2)             Destination File: dest_grid.nc
> (3)             Weight File: weight_file
> (4)             Source File is in SCRIP format
> (5)             Source Grid is a regional grid
> (6)             Source Grid is a logically rectangular grid
> (7)             Destination File is in SCRIP format
> (8)             Destination Grid is a regional grid
> (9)             Destination Grid is a logically rectangular grid
> (10)            Regrid Method: patch
> (11)            Pole option: NONE
> (12)            Ignore unmapped destination points
> (13)            Norm Type: dstarea
> (14)
> (15)         NetCDF Error: weight_file : NetCDF: One or more variable sizes
> violate format constraints
> (16)          ERROR: Problem on processor            0 . Please see the
> PET*.RegridWeightGen.Log files for a traceback.
> (0)     --------------------------------------------------
> (0)     ESMF_regrid_gen_weights: 'ESMF_RegridWeightGen' was not successful.
>
>
> I understand the error ( my dataset is too big being written in NetCDF-3
> file --> 7200 * 7200 * 8 > 4Gb), but I don't know how I can resolve it.
> Looking on the web, I saw that you had similar problems in the past.
> Does a solution exist?
>
> Thank you in advance for your precious help.
> Regards,
> Gabriele Arduini
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150716/9d66ade8/attachment.html