[ncl-talk] strange and intermittent error when processing CMIP5 data

[Daniel Barandiaran]

That does seem to have helped, but not entirely. The error still happens,
but not as frequently. To the devs: if you would like some data to
diagnose, let me know what exactly you would like so I can send it along. A
caveat though, my script is processing somewhere around 2000 files, so
whatever I might send will be big.

On Wed, Jul 15, 2015 at 6:35 AM, Alan Brammer <abrammer at albany.edu> wrote:

> This isn't going to be a solution and one of the devs who knows about the
> internals of the cd_ routines may have a better answer.
> That said,  have you tried reading in the times first, then passing the
> array to cd_calendar. e.g.
>
> fin_time = fin[:]->time
> mtime := cd_calendar(fin_time, -1)
>
> It shouldn't really make a difference, but when I've had weird quirks like
> this it's occasionally on the netcdf side of things.
>
> If that still fails, it may help if you then save that time array to a new
> netcdf and send it to the list so devs can recreate the problem and
> diagnose.
>
>
> Alan.
>
>
> On Tue, 14 Jul 2015 at 15:01 Daniel Barandiaran <dbarandiaran at gmail.com>
> wrote:
>
>> Hi all,
>>
>> I have an unusual problem, which I believe is happening when I call the
>> cd_calendar function. What is particularly strange is that the error does
>> not always happen, but it occurs quite frequently, which requires me to run
>> my script many times to get a successful run. The error code I get is:
>>
>> CDMS error: Error on character time conversion: invalid hour = 80
>>
>> The invalid hour changes from time to time, and the file that causes it
>> to trip up changes from run to run as well, although there do seem to be a
>> few files that cause the trip-up more frequently. Here is a code snippet of
>> my usage:
>>
>>       flist := systemfunc("ls " + diri + model(m) + "/" + dirlist(d) +
>> "/pr*_r" + r + "i" + i + "*.nc")
>>       if (.not.any(ismissing(flist))) then
>>
>>          nfiles   := dimsizes(flist)
>>          fin   := addfiles(flist,"r")
>>          ListSetType(fin,"cat")
>>          mtime := cd_calendar(fin[:]->time,-1)
>>          mntime   = dimsizes(mtime)
>>          tind  = ind(TIME.ge.mtime(0) .and. TIME.le.mtime(mntime-1))
>>          mind  := ind(mtime.le.max(TIME))
>>          mpreU(r,tind)  =
>> dim_avg_n(fin[:]->pr(mind,{Uminlat:Umaxlat},{Uminlon:Umaxlon}),(/1,2/))
>>
>> I realize that my usage might be a little weird, but the CMIP5 data files
>> have such variation on dimension sizes, particularly in the time dimension,
>> so I have to make this funky code to get it to be generic enough, and read
>> in file contents to predefined arrays. This code chunk resides within three
>> nested loops (m,d,i), which perhaps is causing this problem, but I don't
>> know how else to tackle what I am trying to do. I would offer the files
>> that the script reads but it is way too much...
>>
>>
>> --
>> Danny Barandiaran
>> Ph.D. Candidate
>> Department of Plants, Soils, and Climate
>> Utah State University
>>
>>
>>  _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>


-- 
Danny Barandiaran
Ph.D. Candidate
Department of Plants, Soils, and Climate
Utah State University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150715/b6705fae/attachment.html