[ncl-talk] ESMF

Wed May 22 12:31:01 MDT 2019

Hello Dennis,

I wanted to give you and ncl community an update about the solution that you kindly offered for ESMF error that I was getting. I wanted to test the suggestion before suddenly jumping into a conclusion. Just like you said, ESMF regridding page mentions the solution

SrcNetCDFType / DstNetCDFType / WgtNetCDFType (default = "netcdf3") setting it to “netcdf4”

which I tried but did not work. I also set

FileType = Large

along with these options. That didn’t help either. However, when I commented out Src, Dst and Wgt file type setting and leaving only FileType to large allowed me to generate the files that I needed. I am guessing maybe the netcdf libraries that I have on our system is not compiled with netcdf4 option.

In any case, thank you for your suggestion. I can move on to the next task now. With regards,

Kemal.

From: Dennis Shea <shea at ucar.edu>
Sent: Monday, May 20, 2019 11:27 AM
To: Gurer, Kemal at ARB <kemal.gurer at arb.ca.gov>
Cc: ncl-talk at ucar.edu
Subject: Re: [ncl-talk] ESMF

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

The error messages are not directly from NCL.
Rather, they are from the Fortran-90 code used by the underlying ESMF software invoked by NCL.

---
20190519 172019.181 ERROR            PET0 ESMF_IOScrip.F90:1322 ESMF_OutputScripWeightFile netCDF Status Return Error
20190519 172019.181 ERROR            PET0 ESMF_RegridWeightGen.F90:1395 ESMF_RegridWeightGenFile Failure - Internal subroutine call returned Error
---

I *speculate* that the weight file may exceed a size limit.

esmf_regrid<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FFunctions%2FESMF%2FESMF_regrid.shtml&data=02%7C01%7Ckemal.gurer%40arb.ca.gov%7C8ea54f8fde26468ae62308d6dd50cd01%7C9de5aaee778840b1a438c0ccc98c87cc%7C0%7C0%7C636939736453320284&sdata=h4sCX5l%2F7F34YjwHPgJ%2BemH%2FQ%2BYFEIJRTqbQ%2FdtUJF8%3D&reserved=0>

As noted in the function documentation:

SrcNetCDFType / DstNetCDFType / WgtNetCDFType (default = "netcdf3")

Set these options or "NetCDFType" to "netcdf4" to force a NetCDF-4 file to be written for the source, destination, and/or weights files. This will be necessary if your source and/or destination grids or meshes are large, and require more than 2 GB to store them and/or the weights file in the NetCDF file.  Try:

   opt at WgtFileName = "weights.nc<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweights.nc&data=02%7C01%7Ckemal.gurer%40arb.ca.gov%7C8ea54f8fde26468ae62308d6dd50cd01%7C9de5aaee778840b1a438c0ccc98c87cc%7C0%7C0%7C636939736453330281&sdata=d5xcguEGZcvehQ3OynkKV3zG5PZBNhSiuQM%2F5uEHNIk%3D&reserved=0>"

   opt at WgtNetCDFType = "netcdf4"    ; <=======

Good Luck

=== quote
The NetCDF files created by this function are written with "large file support" turned on. This creates a "64-bit offset" NetCDF file under the hood and allows you to write variables > 2 GB but less than 4 GB.

If you are working with large data files that require more than 4 GB, you may need to set one or more of the NetCDFType<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FFunctions%2FESMF%2FESMF_regrid.shtml%23NetCDFType&data=02%7C01%7Ckemal.gurer%40arb.ca.gov%7C8ea54f8fde26468ae62308d6dd50cd01%7C9de5aaee778840b1a438c0ccc98c87cc%7C0%7C0%7C636939736453330281&sdata=68ADOjTrl64KJBExatI5izetUVubU5GdyXqOR2QQkZs%3D&reserved=0> options to "netcdf4". This will force a NetCDF4 file to be written under the hood and will allow you to write variables that are > 4 GB.

=== end quote

Try

setfileoption("nc","Format","NetCDF4")
prior to invoking the ESMF regrid function.

On Mon, May 20, 2019 at 8:45 AM Gurer, Kemal at ARB <kemal.gurer at arb.ca.gov<mailto:kemal.gurer at arb.ca.gov>> wrote:
Dear ncl community,

I can regrid unstructured MPAS output that has nearly 800,000 nCells and 55 vertical levels to a 0.1 degree resolution rectangular grid using ESMF, but cannot regrid the data for any higher resolution than 0.1 degree such as 0.05 degree. Source and destination netcdf files are written but weights file is not generated with “ESMF_RegridWeightGenFile” error. I attached the ncl script that I use for regridding as well as PET0.*.log file and error output dumped to the screen to this email. I am running the script on Centos Linux 6.2 with 128 Gb of RAM. I am using ncl 6.4.0. What is the highest resolution that I can regrid this data set to? Thank you for any information that you can give.

Kemal.

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ucar.edu%2Fmailman%2Flistinfo%2Fncl-talk&data=02%7C01%7Ckemal.gurer%40arb.ca.gov%7C8ea54f8fde26468ae62308d6dd50cd01%7C9de5aaee778840b1a438c0ccc98c87cc%7C0%7C0%7C636939736453340282&sdata=waErq8qe3wqQQua2eGtj3bPQwiZZVKMZjRghoSwwY%2B4%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190522/4ff8145f/attachment.html>