[ncl-talk] Joining large files using "cat" option

[amit timilsina]

Respected Mary,


I am using 6.3.0 version of NCL in 64 bit Mac. Here, I am trying to concatenate at least 30 years files (like 2006 to 2035) or 100 years if possible (each file on yearly basis and has size 168.2 MB). I have attached the syntax I have used.



Thank you




Amit Prasad Timilsina

PhD student

School of Natural Resources,UNL

243 Hardin Hall

3310 Holdrege Street

Lincoln, Nebraska


________________________________
From: Mary Haley <haley at ucar.edu>
Sent: Tuesday, December 1, 2015 10:15 AM
To: amit timilsina
Cc: ncl-talk at ucar.edu
Subject: Re: Joining large files using "cat" option

Amit,

[Sorry, I forgot to cc ncl-talk on the previous message.]

We are a bit puzzled by the "File too large" error. We believe this is a UNIX error that is indicating your file is too large to be written on your system.

Do you have some kind of quota limit on the size of files you can write? Or, are you by any chance running on a 32-bit system?

What is the size of "pr"?  You can do a "printVarSummary" on pr to get this information.

--Mary


On Mon, Nov 30, 2015 at 12:27 PM, amit timilsina <timilsinaamit87 at huskers.unl.edu<mailto:timilsinaamit87 at huskers.unl.edu>> wrote:

Respected Mary,

Thank you very much. I tried to many files using "cat" option. I got error like "File too large: error attempting to write variable (pr) to file (pr_day_CMCC-CM_rcp45_r1i1p1_20060101-20351231.nc)". It seems that there is a limitation on a number of files and files sizes. Is there any way to solve this problem? Thank you very much for your time.


ncl 2> f=addfiles(fils,"r")

ncl 3> ListSetType(f,"cat")

ncl 4> pr=f[:]->pr

ncl 5> ncdf=addfile("pr_day_CMCC-CM_rcp45_r1i1p1_20060101-20351231.nc","c")

ncl 6> filedimdef(ncdf,"time",-1,True)

ncl 7> ncdf->pr=pr

fatal:File too large: error attempting to write variable (pr) to file (pr_day_CMCC-CM_rcp45_r1i1p1_20060101-20351231.nc)

fatal:["Execute.c":8573]:Execute: Error occurred at or near line 7




Amit Prasad Timilsina
PhD student
School of Natural Resources,UNL
243 Hardin Hall
3310 Holdrege Street
Lincoln, Nebraska


________________________________
From: Mary Haley <haley at ucar.edu<mailto:haley at ucar.edu>>
Sent: Monday, November 30, 2015 11:22 AM
To: amit timilsina
Cc: ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] Error in joining files having

Amit,

The error is telling you that your coordinate variable is non-monotonic.  In order to use "cat" mode to join a dimension, which is usually the time dimension, the time values must be monotonic. Also, I believe the time arrays must have the same units.

Please look at your "time" dimension on both files and make sure 1) they are monotonic, 2) they have the same units, and 3) the first time value of the second file is greater than the last time value of the first file.

--Mary




On Fri, Nov 27, 2015 at 4:23 PM, amit timilsina <timilsinaamit87 at huskers.unl.edu<mailto:timilsinaamit87 at huskers.unl.edu>> wrote:
Respected all,

Is the warning " warning:error attempting to fix non-monotonic aggregation variable"  while joining files makes any differences? The first file has dimension size: 366x240X480 and second file have 365x240x480 or first-year data is leap year data. Even with using both non-leap year data, I found the similar problem. What is the reason for getting this problem and how can I solve it?

ncl 0> fils = (/"pr_day_CMCC-CM_decadal2005_r1i2p1_20120101-20121231.nc","pr_day_CMCC-CM_decadal2005_r1i2p1_20130101-20131231.nc"/)ncl 1> f=addfiles(fils,"r")
ncl 2> ListSetType(f,"cat")
ncl 3> pr=f[:]->pr
warning:error attempting to fix non-monotonic aggregation variable


Thank you in advance.

Amit Prasad Timilsina
PhD student
School of Natural Resources,UNL
243 Hardin Hall
3310 Holdrege Street
Lincoln, Nebraska
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20151201/afbc01d6/attachment.html 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: concanating_error.txt
Url: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20151201/afbc01d6/attachment.txt