[ncl-talk] ESMF bilinear regridding - error: Segmentation fault (core dumped)

Mary Haley haley at ucar.edu
Mon Nov 19 10:41:59 MST 2018


Hi Mirce,

Since the files are on cheyenne, I'll have a look. I'll respond back to
ncl-talk if I have any suggestions.

--Mary


On Sun, Nov 18, 2018 at 11:53 PM Barry Lynn <barry.h.lynn at gmail.com> wrote:

> Hello Mirce:
>
> Is there some number of loops that the program will pass through without
> giving a segmentation fault? If so, it may mean that you are running out of
> memory.
>
> Or, you could find the loop it crashes in and see if you can just run that
> loop by itself.  Or, perhaps, you can run other loops around it, but not
> that one indicating a data problem.
>
> Just some things to check before trying to figure out if the program
> itself is broken.
>
> Barry
>
> On Mon, Nov 19, 2018 at 2:14 AM Mirce Morales <mirce.morales at gmail.com>
> wrote:
>
>> Hi all,
>>
>> I am trying to do regrid many variables on ERA5 data using the script
>> below. Each file contains many times in it, so I am looping through the
>> time and the variables to apply bilinear interpolation. However, my script
>> is failing in the time loop.
>>
>> I tested it without going into the time loop by just defining t = 1, and
>> everything worked well...however when I run all my script it
>> crashes....This is the error I am getting:
>>
>> *Segmentation fault (core dumped)*
>>
>> My script:
>>
>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>
>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"*
>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"*
>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"*
>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/ut_string.ncl"*
>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"*
>>
>> *begin*
>>
>> *;----------------------------------------------------------------------*
>> *; User settings*
>> *;----------------------------------------------------------------------*
>>
>> *  ;---------------------------------------------------------------;*
>> *  ; Set the domain number to agree with the geogrid file and      ;*
>> *  ; the IGRID option in the hydro.namelist for WRF-Hydro          ;*
>> *  ;---------------------------------------------------------------;*
>> *  domain  = "DOMAIN1"*
>>
>> *  ;---------------------------------------------------------------;*
>> *  ; Set input and output file directories.                        ;*
>> *  ;---------------------------------------------------------------;*
>> *  dirm    = "/glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/"
>>  ; directory where source forcing data resides*
>> *  outdir  = "./output_regriddes_files/"      ; directory where regridded
>> forcing data will be placed. set to dirm for overwriting the original file*
>>
>> *  ;---------------------------------------------------------------;*
>> *  ; Set a variable to indicate the timestep of source data        ;*
>> *  ;---------------------------------------------------------------;*
>> *  dt=3600.0   ; time over which precipitation is accumulated in the
>> source dataset in units of seconds*
>> *  ;(currently set for ERA5 forcing)*
>>
>> *  ;---------------------------------------------------------------;*
>> *  ; Weight filenames for regridding                               ;*
>> *  ;---------------------------------------------------------------;*
>> *  ;wgtFileName_conserve = "./NLDAS2WRFHydro_weight_conserve.nc"*
>> *  wgtFileName_bilinear = "./ERA52WRFHydro_weight_bilinear.nc"*
>>
>> *  ;---------------------------------------------------------------;*
>> *  ; Data field names from the source file to be regridded         ;*
>> *  ;---------------------------------------------------------------;*
>> *   P_varname = "sp"*
>> *   T_varname = "t2m"*
>> *   U_varname = "u10"*
>> *   V_varname = "v10"*
>> *   ;Q_varname = "SPF_H_110_HTGL"*
>> *   PCP_varname  = "tp"*
>> *   DSWR_varname = "msdwswrf"*
>> *   DLWR_varname = "msdwlwrf"*
>>
>> *;----------------------------------------------------------------------*
>> *; read in source and destination files*
>> *;----------------------------------------------------------------------*
>>
>> *dstGridName="/glade/work/mirce/DOMAINS/DOMAIN_Files_250m_200/geo_em.d04.nc
>> <http://geo_em.d04.nc>"*
>>
>> *  if ( .not.isfilepresent( dstGridName ) ) then*
>> *     print( " ... source grid file not found : "+ dstGridName )*
>> *     exit*
>> *  end if*
>>
>> *;---  destination grid data*
>> *  dstfile     = addfile( dstGridName ,"r")*
>> *  dlon2d=dstfile->XLONG_M(0,:,:)*
>> *  dlat2d=dstfile->XLAT_M(0,:,:)*
>> *  dims=dimsizes(dlat2d)*
>>
>> *;  flag=0    ; flag for removing accum precip... (should be set to 0 for
>> NLDAS2 forcing data)*
>>
>> *;----------------------------------------------------------------------*
>> *; Open source data files to be regridded...*
>> *;----------------------------------------------------------------------*
>> *  system("mkdir "+outdir)*
>>
>> *  srcFileName="download_*"*
>>
>> *  datfils = systemfunc ("ls "+dirm+srcFileName)    ; list of file names*
>>
>> *  num_datfils     = dimsizes(datfils)*
>>
>>
>> *  do ifil = 0,num_datfils-1,1   ; loop through datafiles one at a time*
>>
>> *      datfile = addfile( datfils(ifil), "r")*
>> *      print(datfile)*
>>
>> *      print( " ... Open input file : "+ datfils(ifil) )*
>>
>> *   ;Getting variables for time loop*
>> *   time_1 = datfile->time*
>> *   ntime = 100000.d     ;Defining the time as double*
>> *   ntime = dimsizes(time_1)*
>>
>> *    do t = 0,ntime-1,1     ;Loop through the time per file*
>>
>> *   ; It does another do loop to read the variables and do some process
>> with them*
>>
>>
>> *     end do   ; end for time loop*
>>
>> *     end do   ; end do for file loop*
>>
>> *end*
>>
>>
>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>> The file it is processing has de following attributes:
>> Variable: datfile
>> Type: file
>> filename:       download_200901_alltimesdays
>> path:   /glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/
>> download_200901_alltimesdays.nc
>>    file global attributes:
>>       Conventions : CF-1.6
>>       history : 2018-11-17 21:33:45 GMT by grib_to_netcdf-2.7.0:
>> /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data2/
>> adaptor.mars.internal-1542489985.6343107-12063-21-3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454.nc
>> /cache/tmp/3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454-adaptor.mars.internal-1542489985.6345837-12063-5-tmp.grib
>>    dimensions:
>>       longitude = 1440
>>       latitude = 721
>>       time = 744
>>    variables:
>>
>> I am sorry that this is quite long, if it's better, the path to the data
>> and scripts is the following on Cheyenne:
>> */glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5*
>>
>> I will appreciate any help about this error, in case more information is
>> necessary you can tell me and I will add it.
>>
>> Thanks!,
>> Mirce.
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
>
> --
> Barry H. Lynn, Ph.D
> Senior Associate Scientist, Lecturer,
> The Institute of the Earth Science,
> The Hebrew University of Jerusalem,
> Givat Ram, Jerusalem 91904, Israel
> Tel: 972 547 231 170
> Fax: (972)-25662581
>
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181119/d35b86d7/attachment.html>


More information about the ncl-talk mailing list