[ncl-talk] ESMF bilinear regridding - error: Segmentation fault (core dumped)
Mirce Morales
mirce.morales at gmail.com
Thu Nov 29 11:36:07 MST 2018
Hello Mary!,
Thanks a lot for this advice! I tried to plot them in matlab but I was
having some issues. I'll definitely try with the scripts you suggest me and
I'll pay attention to the ";;---Change" sections.
As always, I appreciate a lot your help!
Thanks,
Mirce.
El mié., 28 nov. 2018 a las 12:19, Mary Haley (<haley at ucar.edu>) escribió:
> Hi Mirce,
>
> Have you tried plotting it? We have some template scripts that you can
> use for plotting regridded data:
>
> http://www.ncl.ucar.edu/Applications/Templates/#ESMFRegriddingTemplates
>
> You will see a number of scripts, like "curvilinear to curviinear". You
> want to use the one that matches what grid you started with, and what
> grid/mesh you regridded to. Each of these have a plotting section that you
> can cut-and-pasted, and then you will need to modify it for your own data.
>
> I believe your source grid was rectilinear and your destination grid was
> curvilinear, so this might be the one you want:
>
> http://www.ncl.ucar.edu/Applications/Templates/ESMF_rect_to_curv.ncl
>
> The script has several comments of the form ";;---Change", which
> indicates places that you will likely have to change to work with your own
> data.
>
> If you have questions about using this script, then please post back to
> ncl-talk.
>
> Good luck,
>
> --Mary
>
>
>
>
> On Thu, Nov 22, 2018 at 7:39 PM Mirce Morales <mirce.morales at gmail.com>
> wrote:
>
>> Hello Again Mary,
>>
>> Regarding my regridding script, I did the changes you told me and fixed
>> the var problem and the script is working well, thank you very much!!.
>>
>> Now I am having issues to verify that the regridding is well done...Since
>> I am regridding global data to a small region, it is not easy to verify at
>> first look.
>>
>> Thanks!,
>> Mirce.
>>
>> El lun., 19 nov. 2018 a las 12:24, Mirce Morales (<
>> mirce.morales at gmail.com>) escribió:
>>
>>> Hi Mary!,
>>>
>>> Thank you very much for looking at the script!, I will do the all
>>> the changes you told me, I totally forgot about the "gt." and ".lt". in my
>>> loops., thanks, and I'll try to fix the var problem.
>>> I'll be back in a moment to update you about the error.
>>>
>>> Thanks a lot again.
>>>
>>> Cheers,
>>> Mirce.
>>>
>>> El lun., 19 nov. 2018 a las 11:04, Mary Haley (<haley at ucar.edu>)
>>> escribió:
>>>
>>>> Mirce,
>>>>
>>>> I don't think the problem is with the regridding per se.
>>>>
>>>> The issue is with one of your "do" loops which is using a double as a
>>>> loop index, and the use of ">" and "<" instead of ".gt." and ".lt".
>>>> You have:
>>>>
>>>> ntime = 100000.d ;Defining the time as double
>>>> ntime = dimsizes(time_1)
>>>>
>>>> do t = 0,ntime-1,1 ;Loop through the time per file
>>>>
>>>> NCL apparently doesn't like the double as a loop argument, which kind
>>>> of surprises me. If you really need t to be double, you could do this:
>>>>
>>>> ntime = dimsizes(time_1)
>>>>
>>>> do it = 0,ntime-1,1 ;Loop through the time per file
>>>> t = todouble(it)
>>>>
>>>> To fix the other problem, change these lines:
>>>>
>>>> if (t < 24)
>>>> if (t>24)
>>>>
>>>> to:
>>>>
>>>> if (t .lt. 24)
>>>> if (t.gt.24)
>>>>
>>>> These changes still don't fix your script, because "var" is undefined
>>>> when the "do v=1,7" loop gets to "v=2", causing an error. I assume you can
>>>> fix this, since it looks like you have some commented code to deal with
>>>> this.
>>>>
>>>> Good luck,
>>>>
>>>> --Mary
>>>>
>>>> On Mon, Nov 19, 2018 at 10:41 AM Mary Haley <haley at ucar.edu> wrote:
>>>>
>>>>> Hi Mirce,
>>>>>
>>>>> Since the files are on cheyenne, I'll have a look. I'll respond back
>>>>> to ncl-talk if I have any suggestions.
>>>>>
>>>>> --Mary
>>>>>
>>>>>
>>>>> On Sun, Nov 18, 2018 at 11:53 PM Barry Lynn <barry.h.lynn at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Mirce:
>>>>>>
>>>>>> Is there some number of loops that the program will pass through
>>>>>> without giving a segmentation fault? If so, it may mean that you are
>>>>>> running out of memory.
>>>>>>
>>>>>> Or, you could find the loop it crashes in and see if you can just run
>>>>>> that loop by itself. Or, perhaps, you can run other loops around it, but
>>>>>> not that one indicating a data problem.
>>>>>>
>>>>>> Just some things to check before trying to figure out if the program
>>>>>> itself is broken.
>>>>>>
>>>>>> Barry
>>>>>>
>>>>>> On Mon, Nov 19, 2018 at 2:14 AM Mirce Morales <
>>>>>> mirce.morales at gmail.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am trying to do regrid many variables on ERA5 data using the
>>>>>>> script below. Each file contains many times in it, so I am looping through
>>>>>>> the time and the variables to apply bilinear interpolation. However, my
>>>>>>> script is failing in the time loop.
>>>>>>>
>>>>>>> I tested it without going into the time loop by just defining t = 1,
>>>>>>> and everything worked well...however when I run all my script it
>>>>>>> crashes....This is the error I am getting:
>>>>>>>
>>>>>>> *Segmentation fault (core dumped)*
>>>>>>>
>>>>>>> My script:
>>>>>>>
>>>>>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>>>>>>
>>>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"*
>>>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"*
>>>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"*
>>>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/contrib/ut_string.ncl"*
>>>>>>> *load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl"*
>>>>>>>
>>>>>>> *begin*
>>>>>>>
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>> *; User settings*
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>>
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ; Set the domain number to agree with the geogrid file and ;*
>>>>>>> * ; the IGRID option in the hydro.namelist for WRF-Hydro ;*
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * domain = "DOMAIN1"*
>>>>>>>
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ; Set input and output file directories. ;*
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * dirm =
>>>>>>> "/glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/" ; directory
>>>>>>> where source forcing data resides*
>>>>>>> * outdir = "./output_regriddes_files/" ; directory where
>>>>>>> regridded forcing data will be placed. set to dirm for overwriting the
>>>>>>> original file*
>>>>>>>
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ; Set a variable to indicate the timestep of source data ;*
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * dt=3600.0 ; time over which precipitation is accumulated in the
>>>>>>> source dataset in units of seconds*
>>>>>>> * ;(currently set for ERA5 forcing)*
>>>>>>>
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ; Weight filenames for regridding ;*
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ;wgtFileName_conserve = "./NLDAS2WRFHydro_weight_conserve.nc"*
>>>>>>> * wgtFileName_bilinear = "./ERA52WRFHydro_weight_bilinear.nc"*
>>>>>>>
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * ; Data field names from the source file to be regridded ;*
>>>>>>> * ;---------------------------------------------------------------;*
>>>>>>> * P_varname = "sp"*
>>>>>>> * T_varname = "t2m"*
>>>>>>> * U_varname = "u10"*
>>>>>>> * V_varname = "v10"*
>>>>>>> * ;Q_varname = "SPF_H_110_HTGL"*
>>>>>>> * PCP_varname = "tp"*
>>>>>>> * DSWR_varname = "msdwswrf"*
>>>>>>> * DLWR_varname = "msdwlwrf"*
>>>>>>>
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>> *; read in source and destination files*
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>>
>>>>>>> *dstGridName="/glade/work/mirce/DOMAINS/DOMAIN_Files_250m_200/geo_em.d04.nc
>>>>>>> <http://geo_em.d04.nc>"*
>>>>>>>
>>>>>>> * if ( .not.isfilepresent( dstGridName ) ) then*
>>>>>>> * print( " ... source grid file not found : "+ dstGridName )*
>>>>>>> * exit*
>>>>>>> * end if*
>>>>>>>
>>>>>>> *;--- destination grid data*
>>>>>>> * dstfile = addfile( dstGridName ,"r")*
>>>>>>> * dlon2d=dstfile->XLONG_M(0,:,:)*
>>>>>>> * dlat2d=dstfile->XLAT_M(0,:,:)*
>>>>>>> * dims=dimsizes(dlat2d)*
>>>>>>>
>>>>>>> *; flag=0 ; flag for removing accum precip... (should be set to
>>>>>>> 0 for NLDAS2 forcing data)*
>>>>>>>
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>> *; Open source data files to be regridded...*
>>>>>>>
>>>>>>> *;----------------------------------------------------------------------*
>>>>>>> * system("mkdir "+outdir)*
>>>>>>>
>>>>>>> * srcFileName="download_*"*
>>>>>>>
>>>>>>> * datfils = systemfunc ("ls "+dirm+srcFileName) ; list of file
>>>>>>> names*
>>>>>>>
>>>>>>> * num_datfils = dimsizes(datfils)*
>>>>>>>
>>>>>>>
>>>>>>> * do ifil = 0,num_datfils-1,1 ; loop through datafiles one at a
>>>>>>> time*
>>>>>>>
>>>>>>> * datfile = addfile( datfils(ifil), "r")*
>>>>>>> * print(datfile)*
>>>>>>>
>>>>>>> * print( " ... Open input file : "+ datfils(ifil) )*
>>>>>>>
>>>>>>> * ;Getting variables for time loop*
>>>>>>> * time_1 = datfile->time*
>>>>>>> * ntime = 100000.d ;Defining the time as double*
>>>>>>> * ntime = dimsizes(time_1)*
>>>>>>>
>>>>>>> * do t = 0,ntime-1,1 ;Loop through the time per file*
>>>>>>>
>>>>>>> * ; It does another do loop to read the variables and do some
>>>>>>> process with them*
>>>>>>>
>>>>>>>
>>>>>>> * end do ; end for time loop*
>>>>>>>
>>>>>>> * end do ; end do for file loop*
>>>>>>>
>>>>>>> *end*
>>>>>>>
>>>>>>>
>>>>>>> ;::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>>>>>>> The file it is processing has de following attributes:
>>>>>>> Variable: datfile
>>>>>>> Type: file
>>>>>>> filename: download_200901_alltimesdays
>>>>>>> path: /glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5/
>>>>>>> download_200901_alltimesdays.nc
>>>>>>> file global attributes:
>>>>>>> Conventions : CF-1.6
>>>>>>> history : 2018-11-17 21:33:45 GMT by grib_to_netcdf-2.7.0:
>>>>>>> /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data2/
>>>>>>> adaptor.mars.internal-1542489985.6343107-12063-21-3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454.nc
>>>>>>> /cache/tmp/3ffc2cf9-73f5-454d-bc1b-f54dfbfb6454-adaptor.mars.internal-1542489985.6345837-12063-5-tmp.grib
>>>>>>> dimensions:
>>>>>>> longitude = 1440
>>>>>>> latitude = 721
>>>>>>> time = 744
>>>>>>> variables:
>>>>>>>
>>>>>>> I am sorry that this is quite long, if it's better, the path to the
>>>>>>> data and scripts is the following on Cheyenne:
>>>>>>> */glade/scratch/mirce/LaSierra/FullDomainForcing/ERA5*
>>>>>>>
>>>>>>> I will appreciate any help about this error, in case more
>>>>>>> information is necessary you can tell me and I will add it.
>>>>>>>
>>>>>>> Thanks!,
>>>>>>> Mirce.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ncl-talk mailing list
>>>>>>> ncl-talk at ucar.edu
>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Barry H. Lynn, Ph.D
>>>>>> Senior Associate Scientist, Lecturer,
>>>>>> The Institute of the Earth Science,
>>>>>> The Hebrew University of Jerusalem,
>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>> Tel: 972 547 231 170
>>>>>> Fax: (972)-25662581
>>>>>>
>>>>>> C.E.O, Weather It Is, LTD
>>>>>> Weather and Climate Focus
>>>>>> http://weather-it-is.com
>>>>>> Jerusalem, Israel
>>>>>> Local: 02 930 9525
>>>>>> Cell: 054 7 231 170
>>>>>> Int-IS: x972 2 930 9525
>>>>>>
>>>>>> _______________________________________________
>>>>>> ncl-talk mailing list
>>>>>> ncl-talk at ucar.edu
>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181129/b3026f1e/attachment.html>
More information about the ncl-talk
mailing list