[ncl-talk] ESMF, was: WRAPIT
Ian Harris (ENV - Staff)
I.Harris at uea.ac.uk
Thu Jul 30 08:11:47 MDT 2020
Call off the dogs!
I’ve taken delivery of a new Linux box, with twice as much memory. NCL installed with condo, runs fine. No more Mac hacking!
Cheers
Harry
From: ncl-talk <ncl-talk-bounces at mailman.ucar.edu> on behalf of "Ian Harris (ENV - Staff) via ncl-talk" <ncl-talk at mailman.ucar.edu>
Reply to: "Ian Harris (ENV - Staff)" <I.Harris at uea.ac.uk>
Date: Wednesday, 29 July 2020 at 19:33
To: "ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] ESMF, was: WRAPIT
Hi Dave,
No, latest I have is 7.1.0. Before that it was 6.3.0.
Cheers
Harry
From: Dave Allured - NOAA Affiliate <dave.allured at noaa.gov>
Date: Wednesday, 29 July 2020 at 19:18
To: "Ian Harris (ENV - Staff)" <I.Harris at uea.ac.uk>
Cc: "ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
Subject: ESMF, was: WRAPIT
Warning: This email is from outside the UEA system. Do not click on links or attachments unless you expect them from the sender and know the content is safe.
Harry, I see that you are using Macports. They updated ESMF from version 7.* to 8_0_1 about four days ago. I wonder whether you picked up that update, and whether that needs some adjustment inside NCL.
On Wed, Jul 29, 2020 at 11:05 AM Ian Harris (ENV - Staff) via ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>> wrote:
Hi,
I’m still at it. I had to abandon the efforts to write NetCDF in Fortran via WRAPIT – in the end it just refused to compile, even when I hard-coded the Includes into the WRAPIT script.
[as a complete aside: Mac users! Do not, I repeat, do NOT upgrade your system to 10.14 or higher. The system security is full-on paranoia, and editing files in (eg) /usr or /opt requires a lot of tomfoolery. I know I mentioned it last time but it’s worth mentioning again.]
So I’ve split the task into two scripts, working on two sections of the year range I need, and with handovers of key settings and data in files.
New problem! The ESMF gridding isn’t working any more:
Fortran runtime error: shape and target do not conform
fatal:The result of the conditional expression yields a missing value. NCL can not determine branch, see ismissing function
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 2977 in file $NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 4052 in file $NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 484 in file verifyforcev3.2a.ncl
The relevant lines in my script are:
Opt = True
Opt at DstGridLat = dlats;
Opt at DstGridLon = dlons;
Opt at SrcRegional = True;
Opt at DstRegional = True;
Opt at SrcGridLat = crulats;
Opt at SrcGridLon = crulons;
Opt at InterpMethod = "neareststod"; we just want to replicate CRU TS into the higher-resolution grid
Opt at WgtFileName = "weights_cruts_subset_neareststod.nc<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweights_cruts_subset_neareststod.nc%2F&data=02%7C01%7Ci.harris%40uea.ac.uk%7C4dde2b00e3e44b9ae07108d833edefe9%7Cc65f8795ba3d43518a070865e5d8f090%7C0%7C0%7C637316444318317333&sdata=9kPfBZyhT7cJ0h3HHKk5A0WiydyAz3ImN2XMw%2FOXQJg%3D&reserved=0>";
crutmp125 = ESMF_regrid(crutmp,Opt); copy CRUTMP into the high-resolution grid
That last line is, of course, #484.
None of the gridding code has changed, and I can’t work out what the error messages are trying to tell me. Previous versions of the script, ie, ones that haven’t been fiddled with in the latest shenanigans, give the same error. This leads me to suspect the new install of NCL.
Finally, because I know I’ll be asked for it, here’s the regridding with Debug = True:
(0) get_src_grid_info: source lat dims = (76)
(0) get_src_grid_info: source lon dims = (140)
(0) get_src_grid_info: source grid type is 'rectilinear'
(0) curvilinear_to_SCRIP: calculating grid corners...
(0) curvilinear_to_SCRIP: no lat values are at the poles, so
(0) calculating grid corners using
(0) calc_SCRIP_corners_noboundaries...
(0) calc_SCRIP_corners_noboundaries
(0) min/max original lat: 35.25/72.75
(0) min/max original lon: -24.75/44.75
(0) calc_SCRIP_corners_noboundaries
(0) min/max Extlat2d: 34.75/73.25
(0) min/max Extlon2d: -25.25/45.25
(0) calc_SCRIP_corners_noboundaries
(0) min/max ExtGridCenter_lat: 35/73
(0) min/max ExtGridCenter_lon: -25/45
(0) get_dst_grid_info: destination lat dims = (304)
(0) get_dst_grid_info: destination lon dims = (560)
(0) curvilinear_to_SCRIP: calculating grid corners...
(0) curvilinear_to_SCRIP: no lat values are at the poles, so
(0) calculating grid corners using
(0) calc_SCRIP_corners_noboundaries...
(0) calc_SCRIP_corners_noboundaries
(0) min/max original lat: 35.0625/72.9375
(0) min/max original lon: -24.9375/44.9375
(0) calc_SCRIP_corners_noboundaries
(0) min/max Extlat2d: 34.9375/73.0625
(0) min/max Extlon2d: -25.0625/45.0625
(0) calc_SCRIP_corners_noboundaries
(0) min/max ExtGridCenter_lat: 35/73
(0) min/max ExtGridCenter_lon: -25/45
(0) ESMF_regrid_gen_weights: number of processors used: 1
(0) --------------------------------------------------
(0) ESMF_regrid_gen_weights: the following command is about to be executed on the system:
(0) 'ESMF_RegridWeightGen --ignore_degenerate --source source_grid_file.nc<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsource_grid_file.nc%2F&data=02%7C01%7Ci.harris%40uea.ac.uk%7C4dde2b00e3e44b9ae07108d833edefe9%7Cc65f8795ba3d43518a070865e5d8f090%7C0%7C0%7C637316444318327329&sdata=3IDX8wIFtPJj18jF25kV82z72hGLVx3BSQaM8c8O0B0%3D&reserved=0> --destination destination_grid_file.nc<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdestination_grid_file.nc%2F&data=02%7C01%7Ci.harris%40uea.ac.uk%7C4dde2b00e3e44b9ae07108d833edefe9%7Cc65f8795ba3d43518a070865e5d8f090%7C0%7C0%7C637316444318337314&sdata=xtA2tTYZtqRiGASvCYr2WLxOupkQ%2BGHA8pQTlDHYPh8%3D&reserved=0> --weight weights_cruts_subset_neareststod.nc<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweights_cruts_subset_neareststod.nc%2F&data=02%7C01%7Ci.harris%40uea.ac.uk%7C4dde2b00e3e44b9ae07108d833edefe9%7Cc65f8795ba3d43518a070865e5d8f090%7C0%7C0%7C637316444318337314&sdata=0Z%2FplPMpXsp0tZpU0V6FkQl7BFU2ywP1fUEAzyRgSyg%3D&reserved=0> --method neareststod --src_regional --dst_regional -i'
(0) --------------------------------------------------
Fortran runtime error: shape and target do not conform
(0) ESMF_regrid_gen_weights: output from 'ESMF_RegridWeightGen':
(0) missing
(0) --------------------------------------------------
fatal:The result of the conditional expression yields a missing value. NCL can not determine branch, see ismissing function
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 2977 in file $NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 4052 in file $NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl
fatal:["Execute.c":8637]:Execute: Error occurred at or near line 485 in file verifyforcev3.2a.ncl
Cheers for any insights,
Harry
From: ncl-talk <ncl-talk-bounces at mailman.ucar.edu<mailto:ncl-talk-bounces at mailman.ucar.edu>> on behalf of "Ian Harris (ENV - Staff) via ncl-talk" <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Reply to: "Ian Harris (ENV - Staff)" <I.Harris at uea.ac.uk<mailto:I.Harris at uea.ac.uk>>
Date: Sunday, 19 July 2020 at 01:05
To: "ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>" <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Subject: Re: [ncl-talk] Using NetCDF with WRAPIT
Okay, I’ve moved things on.
1. I located and edited the WRAPIT script. This has a ’set compiler options’ section, and within that a 'Darwin' subsection. Here I was able to add the minimum includes and libraries for NetCDF to be incorporated. Easy. This meant I didn’t need to spend two days upgrading everything, which is a shame, because of point 2.
2. Because I upgraded the Mac to 10.14 ('Mojave'), a security feature called FDA (Full Disk Access) prevents me from running WRAPIT! It’s very restrictive if you want to do things outside of the approved areas (ie, buying Apple Music, joining Apple TV, etc). You can add apps to FDA in the system Prefs, but not scripts. I tried writing a dummy Fortran proglet to run the script, but I don’t get any outputs, so that’s not working.
It doesn’t get easier!
Harry
From: ncl-talk <ncl-talk-bounces at mailman.ucar.edu<mailto:ncl-talk-bounces at mailman.ucar.edu>> on behalf of "Ian Harris (ENV - Staff) via ncl-talk" <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Reply-To: Ian Harris <i.harris at uea.ac.uk<mailto:i.harris at uea.ac.uk>>
Date: Friday, 17 July 2020 at 23:57
To: ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Subject: Re: [ncl-talk] Using NetCDF with WRAPIT
Hi Rick,
Nice suggestion, sadly no difference.
One of the reasons I went for a manual ncl install (“You have chosen Slow and Painful”) was in case I could mod WRAPIT to recognise Includes (or even just hard code NetCDF into it). I haven’t attempted that yet..
Cheers
Harry
From: Rick Brownrigg <brownrig at ucar.edu<mailto:brownrig at ucar.edu>>
Date: Friday, 17 July 2020 at 18:41
To: Ian Harris <i.harris at uea.ac.uk<mailto:i.harris at uea.ac.uk>>
Cc: ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Subject: Re: [ncl-talk] Using NetCDF with WRAPIT
Hi Harry,
I don't know for certain. It really looks like the compiler is not finding the netcdf header, and needs the equivalent of a -I/usr/local/include option. As you point out, WRAPIT doesn't accept that (seems like an oversight to me).
I presume you are using gfortran? It manual states: "The gfortran compiler currently does not make use of any environment variables to control its operation above and beyond those that affect the operation of gcc"
With that, you might try setting CPATH:
export CPATH=/usr/local/include
Other than that, you might try running WRAPIT with the -d switch. This will show the build commands and not delete temporary outputs. From there you might be able to manually fix-up and run the appropriate build commands.
Wish I had a better answer...
Rick
On Fri, Jul 17, 2020 at 10:32 AM Ian Harris (ENV - Staff) via ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>> wrote:
Hi Rick,
WRAPIT Version: 120209
A syntax error occurred while parsing: include
COMPILING /Users/ianharris/Programs/Fortran/forverify3.for
/Users/ianharris/Programs/Fortran/forverify3.for:443: Error: Can't open included file 'netcdf.inc'
LINKING
gcc: error: forverify3.o: No such file or directory
rm: forverify3.o: No such file or directory
END WRAPIT
This is the gfortran equivalent that works:
% gfortran -m64 -I/usr/local/include/ -L/usr/local/lib -lnetcdf /usr/local/lib/libnetcdff.a ~/Programs/Fortran/forverify3.for
also, new tree from macports:
% gfortran -m64 -I/opt/local/include/ -L/opt/local/lib -lnetcdf /opt/local/lib/libnetcdff.a ~/Programs/Fortran/forverify3.for
Now WRAPIT doesn't like Includes, and you have to put spaces after the -L and -l linkages, but:
% WRAPIT -m64 -L /opt/local/lib -l netcdf /opt/local/lib/libnetcdff.a ~/Programs/Fortran/forverify3.for
WRAPIT Version: 120209
A syntax error occurred while parsing: include
COMPILING /Users/ianharris/Programs/Fortran/forverify3.for
/Users/ianharris/Programs/Fortran/forverify3.for:442: Error: Can't open included file 'netcdf.inc'
LINKING
gcc: error: forverify3.o: No such file or directory
rm: forverify3.o: No such file or directory
END WRAPIT
Cheers
Harry
________________________________
From: Rick Brownrigg <brownrig at ucar.edu<mailto:brownrig at ucar.edu>>
Sent: 17 July 2020 17:15
To: Ian Harris (ENV - Staff) <I.Harris at uea.ac.uk<mailto:I.Harris at uea.ac.uk>>
Cc: Dennis Shea via ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>>
Subject: Re: [ncl-talk] Using NetCDF with WRAPIT
Hi,
What do you mean by "cannot get WRAPIT to recognise the "include 'netcdf.inc'""? Are you getting a compiler error, or an error from WRAPIT itself, or? I see you have -L directives pointing to where the netcdf library resides; do you need a similar -I directive to point to where the header is?
Rick
On Fri, Jul 17, 2020 at 10:03 AM Ian Harris (ENV - Staff) via ncl-talk <ncl-talk at mailman.ucar.edu<mailto:ncl-talk at mailman.ucar.edu>> wrote:
Hi,
I need to write NetCDF files with a Fortran subroutine, because there seems to be a memory leak in addfile(). I know that was supposed to be fixed years back, but I have memory usage graphs that indicate otherwise. And yes, I have comprehensive deletion strategies.
I've tried a lot of things, including reinstalling ncl, (oh, yes..), NetCDF, etc, etc, but I cannot get WRAPIT to recognise the "include 'netcdf.inc'" line in the subroutine. Has anyone managed this? How did you link to the libraries?
Here are a couple examples that don't work:
WRAPIT -L/usr/local/lib -l netcdf -L /usr/local /usr/local/lib/libnetcdff.a /usr/local/lib/libnetcdf.a ~/Programs/Fortran/forverify3.for
WRAPIT -L /opt/local/lib -l netcdf -L /opt/local -l libnetcdff.a ~/Programs/Fortran/forverify3.for
..I just chose two from dozens.
Is it actually *possible* to have a NetCDF-aware Fortran subroutine under WRAPIT?
Cheers
Harry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20200730/4d1cf1c4/attachment.html>
More information about the ncl-talk
mailing list