[ncl-talk] ESMF regrid for large array data
Dennis Shea
shea at ucar.edu
Wed Apr 6 14:47:23 MDT 2022
Sorry, I do not have any answer.
---
Your initial posting stated:
"It is taking a long time and CPU time terminates the script and even
job_script takes more than 8 hours and time limitation issues with
job_scripts.
Any suggestions to resolve this issue? "
As you know, the underlying regrid/interpolation routines are from the ESMF
project. NCL interfaces to that software. Hence, NCL developers are not
knowledgeable about coding 'work arounds.'
---
You were/are using:
*Opt at InterpMethod = "neareststod"*
Have you tried *"bilinear" *?
Others can chime in with suggestions.
On Wed, Apr 6, 2022 at 11:14 AM Anil Kumar - NOAA Affiliate <
anil.kumar at noaa.gov> wrote:
> Dennis,
> I am still not able to mask out the WRF selected region from a large
> covered region on unstructured data.
> any advice or suggestion to try. Thanks Anil
>
> *********************************
> src_file = "../fort.63.nc"
> sfile = addfile(src_file,"r")
>
> ;printVarSummary(sfile)
> lat1D = sfile->y
> lon1D = sfile->x
> var = sfile->zeta(89,:)
> dst_file = "../geo_em.d01.nc" //WRF grid
> dfile = addfile(dst_file,"r")
>
>
> dst_lat = dfile->XLAT_M(0,:,:) ; Be sure to use
> appropriate names
> dst_lon = dfile->XLONG_M(0,:,:) ; here.
>
> Opt at SrcGridLat = lat1D
> Opt at SrcGridLon = lon1D
> Opt at SrcRegional = True ;;--Change (maybe)
> Opt at SrcInputFileName = src_file ; optional, but good idea
>
> Opt at SrcGridCornerLat = (/ min(dst_lat), max(dst_lat)/)
> ;sfile->lat_vertices ; corners are necessary
> Opt at SrcGridCornerLon = (/ min(dst_lon), max(dst_lon)/)
> ;sfile->lon_vertices ; for "conserve" method
>
>
>
>
> On Fri, Apr 1, 2022 at 11:04 AM Anil Kumar - NOAA Affiliate <
> anil.kumar at noaa.gov> wrote:
>
>> Hi Dennis,
>> Thanks for getting back to me.
>> Opt at SrcGridCornerLat and Opt at SrcGridCornerLon points provided through
>> WRF max and min coordinates.
>> I don't know Is this way we can do it.
>> Src grid (Unstructured) is covering the Gulf and Atlantic and I want to
>> mask out the region where my wrf grid is located. For example WRF grid is
>> set up over the Houston Coastal region.
>> Is this correct?
>> Thanks
>>
>> dst_file = "../geo_em.d01.nc" //WRF grid
>> dfile = addfile(dst_file,"r")
>>
>>
>> dst_lat = dfile->XLAT_M(0,:,:) ; Be sure to use
>> appropriate names
>> dst_lon = dfile->XLONG_M(0,:,:) ; here.
>>
>> Opt at SrcGridLat = lat1D
>> Opt at SrcGridLon = lon1D
>> Opt at SrcRegional = True ;;--Change (maybe)
>> Opt at SrcInputFileName = src_file ; optional, but good idea
>>
>> Opt at SrcGridCornerLat = (/ min(dst_lat), max(dst_lat)/)
>> ;sfile->lat_vertices ; corners are necessary
>> Opt at SrcGridCornerLon = (/ min(dst_lon), max(dst_lon)/)
>> ;sfile->lon_vertices ; for "conserve" method
>>
>>
>> On Mon, Mar 28, 2022 at 9:22 PM Dennis Shea <shea at ucar.edu> wrote:
>>
>>> Given no responses, I speculate no user knows how to generate the edge
>>> vertices.
>>>
>>> *https://www.ncl.ucar.edu/Applications/ESMF.shtml*
>>> <https://www.ncl.ucar.edu/Applications/ESMF.shtml>
>>>
>>> Example 6 reads the vertices from a file.
>>>
>>> ESMF_regrid_6.ncl: Opt at SrcGridCornerLat = sfile->lat_vertices ;
>>> corners are necessary
>>> ESMF_regrid_6.ncl: Opt at SrcGridCornerLon = sfile->lon_vertices ;
>>> for "conserve" method
>>>
>>> ===
>>> The text above the examples contains:
>>>
>>> " You can (and are highly recommended to) provide your own grid corners
>>> via the special Src/Dst/GridCornerLat/Lon resources. They must be provided
>>> as an *N* x 4 array, where *N* represents the dimensionality of the
>>> center lat/lon grid. For example, if the center lat/lon grid is dimensioned
>>> 256 x 220, then the corner arrays must be 256 x 220 x 4. Sometimes the
>>> corner grids are provided on the file along with the center grids, with
>>> names like "lat_vertices"/"lon_vertices" or "lat_bounds"/"lon_bounds"."
>>>
>>> Good Luck
>>>
>>> On Wed, Mar 23, 2022 at 7:17 AM Anil Kumar - NOAA Affiliate via ncl-talk
>>> <ncl-talk at mailman.ucar.edu> wrote:
>>>
>>>> Hi All,
>>>> I am using ncl script for esmf regrid and converting unstructured onto
>>>> WRF grid.
>>>> There is nothing wrong with the script, it was working fine with the
>>>> previous unstructured grid nodes ( node = 1813443).
>>>> Now I am facing issues running these scripts for new unstructured data
>>>> nodes ( node = 6057307). It is taking a long time and CPU time terminates
>>>> the script and even job_script takes more than 8 hours and time limitation
>>>> issues with job_scripts.
>>>> Any suggestions to resolve this issue? Below is my script
>>>>
>>>> However, when I run script, it says
>>>> This can be VERY slow if you have a large array.
>>>> (0) To speed up code, provide your own edge
>>>> vertices via the
>>>> (0) 'ElementVertices' option.
>>>> How to use ElementVertices?
>>>>
>>>> *******************************************************
>>>> src_file = "../fort.63.nc"
>>>> sfile = addfile(src_file,"r")
>>>> lat1D = sfile->y
>>>> lon1D = sfile->x
>>>> var = sfile->zeta(89,:)
>>>>
>>>> dst_file = "../geo_em.d01.nc"
>>>> dfile = addfile(dst_file,"r")
>>>> Opt = True
>>>>
>>>> ;---"bilinear" is the default. "patch" and "conserve" are other options.
>>>> Opt at InterpMethod = "neareststod" ;;---Change (maybe)
>>>> Opt at WgtFileName = "unstruct_to_rect_2906_D1.nc"
>>>> Opt at SrcGridLat = lat1D
>>>> Opt at SrcGridLon = lon1D
>>>> Opt at SrcRegional = False ;;--Change (maybe)
>>>> Opt at SrcInputFileName = src_file ; optional, but good idea
>>>>
>>>> Opt at SrcMask2D = where(.not.ismissing(var),1,0) ; Necessary
>>>> if has
>>>> ; missing
>>>> values.
>>>>
>>>> dst_lat = dfile->XLAT_M(0,:,:) ; Be sure to use
>>>> appropriate names
>>>> dst_lon = dfile->XLONG_M(0,:,:) ; here.
>>>> Opt at DstGridLon = dst_lon
>>>> Opt at DstGridLat = dst_lat
>>>> Opt at DstRegional = True ;;--Change (maybe)
>>>> Opt at ForceOverwrite = True
>>>> Opt at Debug = True
>>>> Opt at PrintTimings = True
>>>> var_regrid = ESMF_regrid(var,Opt) ; Do the regridding
>>>> printVarSummary(var_regrid)
>>>>
>>>> --
>>>> Dr. Anil Kumar
>>>> Environmental Modeling Center/NCEP/NWS/NOAA
>>>> College Park, MD, USA
>>>>
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at mailman.ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>> --
>> Dr. Anil Kumar
>> Environmental Modeling Center/NCEP/NWS/NOAA
>> College Park, MD, USA
>>
>>
>>
>
> --
> Dr. Anil Kumar
> Environmental Modeling Center/NCEP/NWS/NOAA
> College Park, MD, USA
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20220406/5f602b72/attachment.html>
More information about the ncl-talk
mailing list