[ncl-talk] ESMF bilinear regridding - error: Segmentation fault (core dumped)
Mirce Morales
mirce.morales at gmail.com
Fri Nov 23 02:38:36 MST 2018
Hello Again Mary,
Regarding my regridding script, I did the changes you told me and fixed the
var problem and the script is working well, thank you very much!!.
Now I am having issues to verify that the regridding is well done...Since I
am regridding global data to a small region, it is not easy to verify at
first look.
Thanks!,
Mirce.
El lun., 19 nov. 2018 a las 12:24, Mirce Morales (<mirce.morales at gmail.com>)
escribió:
> Hi Mary!,
>
> Thank you very much for looking at the script!, I will do the all
> the changes you told me, I totally forgot about the "gt." and ".lt". in my
> loops., thanks, and I'll try to fix the var problem.
> I'll be back in a moment to update you about the error.
>
> Thanks a lot again.
>
> Cheers,
> Mirce.
>
> El lun., 19 nov. 2018 a las 11:04, Mary Haley (<haley at ucar.edu>) escribió:
>
>> Mirce,
>>
>> I don't think the problem is with the regridding per se.
>>
>> The issue is with one of your "do" loops which is using a double as a
>> loop index, and the use of ">" and "<" instead of ".gt." and ".lt".
>> You have:
>>
>> ntime = 100000.d ;Defining the time as double
>> ntime = dimsizes(time_1)
>>
>> do t = 0,ntime-1,1 ;Loop through the time per file
>>
>> NCL apparently doesn't like the double as a loop argument, which kind of
>> surprises me. If you really need t to be double, you could do this:
>>
>> ntime = dimsizes(time_1)
>>
>> do it = 0,ntime-1,1 ;Loop through the time per file
>> t = todouble(it)
>>
>> To fix the other problem, change these lines:
>>
>> if (t < 24)
>> if (t>24)
>>
>> to:
>>
>> if (t .lt. 24)
>> if (t.gt.24)
>>
>> These changes still don't fix your script, because "var" is undefined
>> when the "do v=1,7" loop gets to "v=2", causing an error. I assume you can
>> fix this, since it looks like you have some commented code to deal with
>> this.
>>
>> Good luck,
>>
>> --Mary
>>
>> On Mon, Nov 19, 2018 at 10:41 AM Mary Haley <haley at ucar.edu> wrote:
>>
>>> Hi Mirce,
>>>
>>> Since the files are on cheyenne, I'll have a look. I'll respond back to
>>> ncl-talk if I have any suggestions.
>>>
>>> --Mary
>>>
>>>
>>> On Sun, Nov 18, 2018 at 11:53 PM Barry Lynn <barry.h.lynn at gmail.com>
>>> wrote:
>>>
>>>> Hello Mirce:
>>>>
>>>> Is there some number of loops that the program will pass through
>>>> without giving a segmentation fault? If so, it may mean that you are
>>>> running out of memory.
>>>>
>>>> Or, you could find the loop it crashes in and see if you can just run
>>>> that loop by itself. Or, perhaps, you can run other loops around it, but
>>>> not that one indicating a data problem.
>>>>
>>>> Just some things to check before trying to figure out if the program
>>>> itself is broken.
>>>>
>>>> Barry
>>>>
>>>> On Mon, Nov 19, 2018 at 2:14 AM Mirce Morales <mirce.morales at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am trying to do regrid many variables on ERA5 data using the script
>>>>> below. Each file contains many times in it, so I am looping through the
>>>>> time and the variables to apply bilinear interpolation. However, my script
>>>>> is failing in the time loop.
>>>>>
>>>>> I tested it without going into the time loop by just defining t = 1,
>>>>> and everything worked well...however when I run all my script it
>>>>> crashes....This is the error I am getting:
>>>>>
>>>>> *Segmentation fault (core dumped)*
>>>>>
>>>>> My script:
>>>>>
>>>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>>>>
>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"*
>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"*
>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"*
>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/ut_string.ncl"*
>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"*
>>>>>
>>>>> *begin*
>>>>>
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>> *; User settings*
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>>
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ; Set the domain number to agree with the geogrid file and ;*
>>>>> * ; the IGRID option in the hydro.namelist for WRF-Hydro ;*
>>>>> * ;---------------------------------------------------------------;*
>>>>> * domain = "DOMAIN1"*
>>>>>
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ; Set input and output file directories. ;*
>>>>> * ;---------------------------------------------------------------;*
>>>>> * dirm = "/glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/"
>>>>> ; directory where source forcing data resides*
>>>>> * outdir = "./output_regriddes_files/" ; directory where
>>>>> regridded forcing data will be placed. set to dirm for overwriting the
>>>>> original file*
>>>>>
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ; Set a variable to indicate the timestep of source data ;*
>>>>> * ;---------------------------------------------------------------;*
>>>>> * dt=3600.0 ; time over which precipitation is accumulated in the
>>>>> source dataset in units of seconds*
>>>>> * ;(currently set for ERA5 forcing)*
>>>>>
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ; Weight filenames for regridding ;*
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ;wgtFileName_conserve = "./NLDAS2WRFHydro_weight_conserve.nc"*
>>>>> * wgtFileName_bilinear = "./ERA52WRFHydro_weight_bilinear.nc"*
>>>>>
>>>>> * ;---------------------------------------------------------------;*
>>>>> * ; Data field names from the source file to be regridded ;*
>>>>> * ;---------------------------------------------------------------;*
>>>>> * P_varname = "sp"*
>>>>> * T_varname = "t2m"*
>>>>> * U_varname = "u10"*
>>>>> * V_varname = "v10"*
>>>>> * ;Q_varname = "SPF_H_110_HTGL"*
>>>>> * PCP_varname = "tp"*
>>>>> * DSWR_varname = "msdwswrf"*
>>>>> * DLWR_varname = "msdwlwrf"*
>>>>>
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>> *; read in source and destination files*
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>>
>>>>> *dstGridName="/glade/work/mirce/DOMAINS/DOMAIN_Files_250m_200/geo_em.d04.nc
>>>>> <http://geo_em.d04.nc>"*
>>>>>
>>>>> * if ( .not.isfilepresent( dstGridName ) ) then*
>>>>> * print( " ... source grid file not found : "+ dstGridName )*
>>>>> * exit*
>>>>> * end if*
>>>>>
>>>>> *;--- destination grid data*
>>>>> * dstfile = addfile( dstGridName ,"r")*
>>>>> * dlon2d=dstfile->XLONG_M(0,:,:)*
>>>>> * dlat2d=dstfile->XLAT_M(0,:,:)*
>>>>> * dims=dimsizes(dlat2d)*
>>>>>
>>>>> *; flag=0 ; flag for removing accum precip... (should be set to 0
>>>>> for NLDAS2 forcing data)*
>>>>>
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>> *; Open source data files to be regridded...*
>>>>>
>>>>> *;----------------------------------------------------------------------*
>>>>> * system("mkdir "+outdir)*
>>>>>
>>>>> * srcFileName="download_*"*
>>>>>
>>>>> * datfils = systemfunc ("ls "+dirm+srcFileName) ; list of file
>>>>> names*
>>>>>
>>>>> * num_datfils = dimsizes(datfils)*
>>>>>
>>>>>
>>>>> * do ifil = 0,num_datfils-1,1 ; loop through datafiles one at a
>>>>> time*
>>>>>
>>>>> * datfile = addfile( datfils(ifil), "r")*
>>>>> * print(datfile)*
>>>>>
>>>>> * print( " ... Open input file : "+ datfils(ifil) )*
>>>>>
>>>>> * ;Getting variables for time loop*
>>>>> * time_1 = datfile->time*
>>>>> * ntime = 100000.d ;Defining the time as double*
>>>>> * ntime = dimsizes(time_1)*
>>>>>
>>>>> * do t = 0,ntime-1,1 ;Loop through the time per file*
>>>>>
>>>>> * ; It does another do loop to read the variables and do some
>>>>> process with them*
>>>>>
>>>>>
>>>>> * end do ; end for time loop*
>>>>>
>>>>> * end do ; end do for file loop*
>>>>>
>>>>> *end*
>>>>>
>>>>>
>>>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>>>> The file it is processing has de following attributes:
>>>>> Variable: datfile
>>>>> Type: file
>>>>> filename: download_200901_alltimesdays
>>>>> path: /glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/
>>>>> download_200901_alltimesdays.nc
>>>>> file global attributes:
>>>>> Conventions : CF-1.6
>>>>> history : 2018-11-17 21:33:45 GMT by grib_to_netcdf-2.7.0:
>>>>> /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data2/
>>>>> adaptor.mars.internal-1542489985.6343107-12063-21-3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454.nc
>>>>> /cache/tmp/3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454-adaptor.mars.internal-1542489985.6345837-12063-5-tmp.grib
>>>>> dimensions:
>>>>> longitude = 1440
>>>>> latitude = 721
>>>>> time = 744
>>>>> variables:
>>>>>
>>>>> I am sorry that this is quite long, if it's better, the path to the
>>>>> data and scripts is the following on Cheyenne:
>>>>> */glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5*
>>>>>
>>>>> I will appreciate any help about this error, in case more information
>>>>> is necessary you can tell me and I will add it.
>>>>>
>>>>> Thanks!,
>>>>> Mirce.
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>>
>>>>
>>>> --
>>>> Barry H. Lynn, Ph.D
>>>> Senior Associate Scientist, Lecturer,
>>>> The Institute of the Earth Science,
>>>> The Hebrew University of Jerusalem,
>>>> Givat Ram, Jerusalem 91904, Israel
>>>> Tel: 972 547 231 170
>>>> Fax: (972)-25662581
>>>>
>>>> C.E.O, Weather It Is, LTD
>>>> Weather and Climate Focus
>>>> http://weather-it-is.com
>>>> Jerusalem, Israel
>>>> Local: 02 930 9525
>>>> Cell: 054 7 231 170
>>>> Int-IS: x972 2 930 9525
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181123/8c8bfdef/attachment.html>
More information about the ncl-talk
mailing list