[ncl-talk] ESMF bilinear regridding - error: Segmentation fault (core dumped)
Mary Haley
haley at ucar.edu
Mon Nov 19 11:03:40 MST 2018
Mirce,
I don't think the problem is with the regridding per se.
The issue is with one of your "do" loops which is using a double as a loop
index, and the use of ">" and "<" instead of ".gt." and ".lt".
You have:
ntime = 100000.d ;Defining the time as double
ntime = dimsizes(time_1)
do t = 0,ntime-1,1 ;Loop through the time per file
NCL apparently doesn't like the double as a loop argument, which kind of
surprises me. If you really need t to be double, you could do this:
ntime = dimsizes(time_1)
do it = 0,ntime-1,1 ;Loop through the time per file
t = todouble(it)
To fix the other problem, change these lines:
if (t < 24)
if (t>24)
to:
if (t .lt. 24)
if (t.gt.24)
These changes still don't fix your script, because "var" is undefined when
the "do v=1,7" loop gets to "v=2", causing an error. I assume you can fix
this, since it looks like you have some commented code to deal with this.
Good luck,
--Mary
On Mon, Nov 19, 2018 at 10:41 AM Mary Haley <haley at ucar.edu> wrote:
> Hi Mirce,
>
> Since the files are on cheyenne, I'll have a look. I'll respond back to
> ncl-talk if I have any suggestions.
>
> --Mary
>
>
> On Sun, Nov 18, 2018 at 11:53 PM Barry Lynn <barry.h.lynn at gmail.com>
> wrote:
>
>> Hello Mirce:
>>
>> Is there some number of loops that the program will pass through without
>> giving a segmentation fault? If so, it may mean that you are running out of
>> memory.
>>
>> Or, you could find the loop it crashes in and see if you can just run
>> that loop by itself. Or, perhaps, you can run other loops around it, but
>> not that one indicating a data problem.
>>
>> Just some things to check before trying to figure out if the program
>> itself is broken.
>>
>> Barry
>>
>> On Mon, Nov 19, 2018 at 2:14 AM Mirce Morales <mirce.morales at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I am trying to do regrid many variables on ERA5 data using the script
>>> below. Each file contains many times in it, so I am looping through the
>>> time and the variables to apply bilinear interpolation. However, my script
>>> is failing in the time loop.
>>>
>>> I tested it without going into the time loop by just defining t = 1, and
>>> everything worked well...however when I run all my script it
>>> crashes....This is the error I am getting:
>>>
>>> *Segmentation fault (core dumped)*
>>>
>>> My script:
>>>
>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>>
>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"*
>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"*
>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"*
>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/ut_string.ncl"*
>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"*
>>>
>>> *begin*
>>>
>>> *;----------------------------------------------------------------------*
>>> *; User settings*
>>> *;----------------------------------------------------------------------*
>>>
>>> * ;---------------------------------------------------------------;*
>>> * ; Set the domain number to agree with the geogrid file and ;*
>>> * ; the IGRID option in the hydro.namelist for WRF-Hydro ;*
>>> * ;---------------------------------------------------------------;*
>>> * domain = "DOMAIN1"*
>>>
>>> * ;---------------------------------------------------------------;*
>>> * ; Set input and output file directories. ;*
>>> * ;---------------------------------------------------------------;*
>>> * dirm = "/glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/"
>>> ; directory where source forcing data resides*
>>> * outdir = "./output_regriddes_files/" ; directory where
>>> regridded forcing data will be placed. set to dirm for overwriting the
>>> original file*
>>>
>>> * ;---------------------------------------------------------------;*
>>> * ; Set a variable to indicate the timestep of source data ;*
>>> * ;---------------------------------------------------------------;*
>>> * dt=3600.0 ; time over which precipitation is accumulated in the
>>> source dataset in units of seconds*
>>> * ;(currently set for ERA5 forcing)*
>>>
>>> * ;---------------------------------------------------------------;*
>>> * ; Weight filenames for regridding ;*
>>> * ;---------------------------------------------------------------;*
>>> * ;wgtFileName_conserve = "./NLDAS2WRFHydro_weight_conserve.nc"*
>>> * wgtFileName_bilinear = "./ERA52WRFHydro_weight_bilinear.nc"*
>>>
>>> * ;---------------------------------------------------------------;*
>>> * ; Data field names from the source file to be regridded ;*
>>> * ;---------------------------------------------------------------;*
>>> * P_varname = "sp"*
>>> * T_varname = "t2m"*
>>> * U_varname = "u10"*
>>> * V_varname = "v10"*
>>> * ;Q_varname = "SPF_H_110_HTGL"*
>>> * PCP_varname = "tp"*
>>> * DSWR_varname = "msdwswrf"*
>>> * DLWR_varname = "msdwlwrf"*
>>>
>>> *;----------------------------------------------------------------------*
>>> *; read in source and destination files*
>>> *;----------------------------------------------------------------------*
>>>
>>> *dstGridName="/glade/work/mirce/DOMAINS/DOMAIN_Files_250m_200/geo_em.d04.nc
>>> <http://geo_em.d04.nc>"*
>>>
>>> * if ( .not.isfilepresent( dstGridName ) ) then*
>>> * print( " ... source grid file not found : "+ dstGridName )*
>>> * exit*
>>> * end if*
>>>
>>> *;--- destination grid data*
>>> * dstfile = addfile( dstGridName ,"r")*
>>> * dlon2d=dstfile->XLONG_M(0,:,:)*
>>> * dlat2d=dstfile->XLAT_M(0,:,:)*
>>> * dims=dimsizes(dlat2d)*
>>>
>>> *; flag=0 ; flag for removing accum precip... (should be set to 0
>>> for NLDAS2 forcing data)*
>>>
>>> *;----------------------------------------------------------------------*
>>> *; Open source data files to be regridded...*
>>> *;----------------------------------------------------------------------*
>>> * system("mkdir "+outdir)*
>>>
>>> * srcFileName="download_*"*
>>>
>>> * datfils = systemfunc ("ls "+dirm+srcFileName) ; list of file names*
>>>
>>> * num_datfils = dimsizes(datfils)*
>>>
>>>
>>> * do ifil = 0,num_datfils-1,1 ; loop through datafiles one at a time*
>>>
>>> * datfile = addfile( datfils(ifil), "r")*
>>> * print(datfile)*
>>>
>>> * print( " ... Open input file : "+ datfils(ifil) )*
>>>
>>> * ;Getting variables for time loop*
>>> * time_1 = datfile->time*
>>> * ntime = 100000.d ;Defining the time as double*
>>> * ntime = dimsizes(time_1)*
>>>
>>> * do t = 0,ntime-1,1 ;Loop through the time per file*
>>>
>>> * ; It does another do loop to read the variables and do some process
>>> with them*
>>>
>>>
>>> * end do ; end for time loop*
>>>
>>> * end do ; end do for file loop*
>>>
>>> *end*
>>>
>>>
>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>> The file it is processing has de following attributes:
>>> Variable: datfile
>>> Type: file
>>> filename: download_200901_alltimesdays
>>> path: /glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/
>>> download_200901_alltimesdays.nc
>>> file global attributes:
>>> Conventions : CF-1.6
>>> history : 2018-11-17 21:33:45 GMT by grib_to_netcdf-2.7.0:
>>> /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data2/
>>> adaptor.mars.internal-1542489985.6343107-12063-21-3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454.nc
>>> /cache/tmp/3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454-adaptor.mars.internal-1542489985.6345837-12063-5-tmp.grib
>>> dimensions:
>>> longitude = 1440
>>> latitude = 721
>>> time = 744
>>> variables:
>>>
>>> I am sorry that this is quite long, if it's better, the path to the data
>>> and scripts is the following on Cheyenne:
>>> */glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5*
>>>
>>> I will appreciate any help about this error, in case more information is
>>> necessary you can tell me and I will add it.
>>>
>>> Thanks!,
>>> Mirce.
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>
>>
>> --
>> Barry H. Lynn, Ph.D
>> Senior Associate Scientist, Lecturer,
>> The Institute of the Earth Science,
>> The Hebrew University of Jerusalem,
>> Givat Ram, Jerusalem 91904, Israel
>> Tel: 972 547 231 170
>> Fax: (972)-25662581
>>
>> C.E.O, Weather It Is, LTD
>> Weather and Climate Focus
>> http://weather-it-is.com
>> Jerusalem, Israel
>> Local: 02 930 9525
>> Cell: 054 7 231 170
>> Int-IS: x972 2 930 9525
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181119/898bdc5f/attachment.html>
More information about the ncl-talk
mailing list