[ncl-talk] ESMF bilinear regridding - error: Segmentation fault (core dumped)
Barry Lynn
barry.h.lynn at gmail.com
Sun Nov 18 23:52:56 MST 2018
Hello Mirce:
Is there some number of loops that the program will pass through without
giving a segmentation fault? If so, it may mean that you are running out of
memory.
Or, you could find the loop it crashes in and see if you can just run that
loop by itself. Or, perhaps, you can run other loops around it, but not
that one indicating a data problem.
Just some things to check before trying to figure out if the program itself
is broken.
Barry
On Mon, Nov 19, 2018 at 2:14 AM Mirce Morales <mirce.morales at gmail.com>
wrote:
> Hi all,
>
> I am trying to do regrid many variables on ERA5 data using the script
> below. Each file contains many times in it, so I am looping through the
> time and the variables to apply bilinear interpolation. However, my script
> is failing in the time loop.
>
> I tested it without going into the time loop by just defining t = 1, and
> everything worked well...however when I run all my script it
> crashes....This is the error I am getting:
>
> *Segmentation fault (core dumped)*
>
> My script:
>
> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>
> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"*
> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"*
> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"*
> *load "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/ut_string.ncl"*
> *load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"*
>
> *begin*
>
> *;----------------------------------------------------------------------*
> *; User settings*
> *;----------------------------------------------------------------------*
>
> * ;---------------------------------------------------------------;*
> * ; Set the domain number to agree with the geogrid file and ;*
> * ; the IGRID option in the hydro.namelist for WRF-Hydro ;*
> * ;---------------------------------------------------------------;*
> * domain = "DOMAIN1"*
>
> * ;---------------------------------------------------------------;*
> * ; Set input and output file directories. ;*
> * ;---------------------------------------------------------------;*
> * dirm = "/glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/"
> ; directory where source forcing data resides*
> * outdir = "./output_regriddes_files/" ; directory where regridded
> forcing data will be placed. set to dirm for overwriting the original file*
>
> * ;---------------------------------------------------------------;*
> * ; Set a variable to indicate the timestep of source data ;*
> * ;---------------------------------------------------------------;*
> * dt=3600.0 ; time over which precipitation is accumulated in the
> source dataset in units of seconds*
> * ;(currently set for ERA5 forcing)*
>
> * ;---------------------------------------------------------------;*
> * ; Weight filenames for regridding ;*
> * ;---------------------------------------------------------------;*
> * ;wgtFileName_conserve = "./NLDAS2WRFHydro_weight_conserve.nc"*
> * wgtFileName_bilinear = "./ERA52WRFHydro_weight_bilinear.nc"*
>
> * ;---------------------------------------------------------------;*
> * ; Data field names from the source file to be regridded ;*
> * ;---------------------------------------------------------------;*
> * P_varname = "sp"*
> * T_varname = "t2m"*
> * U_varname = "u10"*
> * V_varname = "v10"*
> * ;Q_varname = "SPF_H_110_HTGL"*
> * PCP_varname = "tp"*
> * DSWR_varname = "msdwswrf"*
> * DLWR_varname = "msdwlwrf"*
>
> *;----------------------------------------------------------------------*
> *; read in source and destination files*
> *;----------------------------------------------------------------------*
>
> *dstGridName="/glade/work/mirce/DOMAINS/DOMAIN_Files_250m_200/geo_em.d04.nc
> <http://geo_em.d04.nc>"*
>
> * if ( .not.isfilepresent( dstGridName ) ) then*
> * print( " ... source grid file not found : "+ dstGridName )*
> * exit*
> * end if*
>
> *;--- destination grid data*
> * dstfile = addfile( dstGridName ,"r")*
> * dlon2d=dstfile->XLONG_M(0,:,:)*
> * dlat2d=dstfile->XLAT_M(0,:,:)*
> * dims=dimsizes(dlat2d)*
>
> *; flag=0 ; flag for removing accum precip... (should be set to 0 for
> NLDAS2 forcing data)*
>
> *;----------------------------------------------------------------------*
> *; Open source data files to be regridded...*
> *;----------------------------------------------------------------------*
> * system("mkdir "+outdir)*
>
> * srcFileName="download_*"*
>
> * datfils = systemfunc ("ls "+dirm+srcFileName) ; list of file names*
>
> * num_datfils = dimsizes(datfils)*
>
>
> * do ifil = 0,num_datfils-1,1 ; loop through datafiles one at a time*
>
> * datfile = addfile( datfils(ifil), "r")*
> * print(datfile)*
>
> * print( " ... Open input file : "+ datfils(ifil) )*
>
> * ;Getting variables for time loop*
> * time_1 = datfile->time*
> * ntime = 100000.d ;Defining the time as double*
> * ntime = dimsizes(time_1)*
>
> * do t = 0,ntime-1,1 ;Loop through the time per file*
>
> * ; It does another do loop to read the variables and do some process
> with them*
>
>
> * end do ; end for time loop*
>
> * end do ; end do for file loop*
>
> *end*
>
>
> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> The file it is processing has de following attributes:
> Variable: datfile
> Type: file
> filename: download_200901_alltimesdays
> path: /glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/
> download_200901_alltimesdays.nc
> file global attributes:
> Conventions : CF-1.6
> history : 2018-11-17 21:33:45 GMT by grib_to_netcdf-2.7.0:
> /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data2/
> adaptor.mars.internal-1542489985.6343107-12063-21-3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454.nc
> /cache/tmp/3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454-adaptor.mars.internal-1542489985.6345837-12063-5-tmp.grib
> dimensions:
> longitude = 1440
> latitude = 721
> time = 744
> variables:
>
> I am sorry that this is quite long, if it's better, the path to the data
> and scripts is the following on Cheyenne:
> */glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5*
>
> I will appreciate any help about this error, in case more information is
> necessary you can tell me and I will add it.
>
> Thanks!,
> Mirce.
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
--
Barry H. Lynn, Ph.D
Senior Associate Scientist, Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581
C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181119/f8d54550/attachment.html>
More information about the ncl-talk
mailing list