[ncl-talk] Deriving monthly mean from files of ten years daily data
Dennis Shea
shea at ucar.edu
Mon Jun 19 10:30:47 MDT 2017
I believe we have been through some of this before:
-------------------------------------------------------
On 1 April 2017
There was an email about
"[ncl-talk] Plotting Multiple time series of same variable from 9 different
files (2005-2013)"
Several people replied including me. In fact, I attached two scripts:
(1) read the txt (ascii) files and plotted the desired time series
(2) created a single netCDF file from one test (ascii) file
Requested by me:
"Please: Carefully go through the tutorial at:
http://www.ncl.ucar.edu/Document/Manuals/NCL_User_Guide/
You must put some effort into learning a computer language."
====
Your latest script:
[A]
files = systemfunc("ls " + wkdir +"AOT_550*.txt")
You are getting *txt* [text, ascii] files.
[B]
You use 'addfiles' and 'addfiles_GetVar'. **Neither reads text (ascii)
files.**
https://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml
Further, addfiles_GetVar is "deprected" and the documentation says to
use 'addfiles'
PLEASE read the documentation for the functions you are using.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I have attached a script that will convert each .txt file to a netCDF file.
It also creates 3 different 'time' variables to facilitate usage: time,
yyyyddd, yyyymmdd
%> ncl AOT_txt2nc.ncl
PLEASE examine the netCDF files produced by the above script via
%> ncl_filedump -v time,yyyyddd,yyyymmdd AOT_550_2005.nc
or
%> ncdump -v time,yyyyddd,yyyymmdd AOT_550_2005.nc
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The attached 'najiby_monthly.ncl' (requires NCL 6.4.0) script computes the
monthly means.
NOTE: The most important rule of data processing is: LOOK AT YOUR DATA!
Some of the months have only one day (value). Computing a monthly mean
based uoon one value is
not appropriate. In fact, not one year has complete daily values.
yyyy days
2005 - 266
2006 - 270
2007 - 300
2008 - 216
2009 - 178
2010 - 9
2011 - 89
2012 - 249
2023 - 124
2014 - 156
The driver script [ najiby_monthly.ncl ] that calculates the monthly means
requires 10 values before a monthly mean is calculated.
https://www.ncl.ucar.edu/Document/Functions/Contributed/calculate_monthly_values.shtml
You will need NCL 6.4.0
If you do not have NCL-6.4.0, I have attached the function:
calculate_monthly_values.640.ncl
You will have to explicitily (manually) load it. [you must unzip the file:
gzip -d ...]
===
Actually, I have decided to alos attache the ASCII (.txt) and netCDF (.nc)
files containg the monthly mean data.
Again, **please read the documentation for the functions you are using.**
On Sat, Jun 17, 2017 at 4:22 AM, <najib.yusuf at carnasrda.com> wrote:
> Hi Rick,
>
> Thank you for your response. The asciiread part of the program works
> well, there is no error in there, that is why
> I didn't bother you with it, I only cut part that has the problem from
> the printed output as well as from the program due to length of the
> files. I will correct the addfiles command with ".txt" files and revert.
>
> Thank you.
>
> Najib
>
>
>
>
>
>
>
> On 2017-06-16 13:48, Rick Brownrigg wrote:
> > Hi,
> >
> > There must be more going on to this script than is shown. The printed
> > output does not align with print statements in the script, and I don't
> > believe an addfiles command works with ".txt" files. The error message
> > is coming from asciiread(), which does not appear anywhere in the
> > script you've shown. In any case, the error message is telling you
> > that there is no variable names AOT_550 in the files. Are there other
> > error messages from the script?
> >
> > Rick
> >
> > On Fri, Jun 16, 2017 at 8:41 AM, <najib.yusuf at carnasrda.com> wrote:
> >
> >> -------- Original Message --------
> >> Subject: Deriving monthly mean from files of ten years data
> >> Date: 2017-06-16 10:00
> >> From: najib.yusuf at carnasrda.com
> >> To: ncl-talk at ucar.edu
> >> Cc: shea at ucar.edu
> >>
> >> Dear ncl experts,
> >>
> >> I am trying to derived monthly mean (for example all January,
> >> all
> >> February....all December) mean from 10 files (0-9) years of Julian
> >> date,
> >> so I can plot the mean on the bulk of the data. I tried but I got an
> >> error;
> >>
> >> ; to derive the mean of the data
> >>
> > ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> ;;;;;;;;;;;;;;;;;
> >> wkdir = "./"
> >> fil_dat = "AOT_550"
> >> ; list all AOT files for different years
> >> files = systemfunc("ls " + wkdir +"AOT_550*.txt")
> >> print(files)
> >>
> >> ; read years (files)
> >> files10 = files(0:9)
> >> print(files10)
> >> f = addfiles(files10, "r")
> >> AOT = addfiles_GetVar(f,files10,"AOT_550")
> >> time = addfiles_GetVar(f,files10,"time")
> >>
> >> ; get date
> >> t1=calendar_decode2(time,0)
> >> ; get day of year
> >> printVarSummary(t1)
> >> ndays = 365
> >> doy = ndays
> >> doy(9) = 365
> >> print(doy)
> >>
> >> ; read only monthly mean (not sure?)
> >> AOT_550_all12 = AOT_550_all12(0:11)
> >> print(AOT_550_all12)
> >>
> >> f = addfiles(files12, "r")
> >> AOT = addfiles_GetVar(f,AOT_550_all12,"AOT_550_all")
> >> time = addfiles_GetVar(f,AOT_550_all12,"time")
> >> ; get date
> >> t1=calendar_decode2(time,0)
> >> ; get day of year
> >> printVarSummary(t1)
> >>
> > doy=day_of_year(floattoint(t1(:,0)),floattoint(t1(:,1)),
> floattoint(t1(:,2)))
> >> ; move to mid-months
> >> doy=doy-15
> >> doy(11:9) = 365 ;
> >> print(doy)
> >> AOT_550_all12 = AOT
> >>
> >> printVarSummary(AOT)
> >>
> >> Variable: variables
> >> Type: string
> >> Total Size: 56 bytes
> >> 7 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [7]
> >> Coordinates:
> >> Number Of Attributes: 1
> >> _FillValue : missing
> >> (0) Day
> >> (1) AOT_675
> >> (2) AOT_500
> >> (3) AOT_440
> >> (4) 440-675Angstrom
> >> (5) ╬▒
> >> (6) AOT_550
> >>
> >> Variable: files (subsection)
> >> Type: string
> >> Total Size: 8 bytes
> >> 1 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [1]
> >> Coordinates:
> >> (0) /home/model-user/AOT_550_2014.txt
> >> warning:asciiread: End of file reached and only (157) elements were
> >> read
> >> from the file, filling remaining elements with the default missing
> >> value
> >> for the requested type
> >>
> >> Variable: z1
> >> Type: string
> >> Total Size: 2928 bytes
> >> 366 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [366]
> >> Coordinates:
> >> Number Of Attributes: 1
> >> _FillValue : missing
> >>
> >> Variable: new_str
> >> Type: string
> >> Total Size: 20496 bytes
> >> 2562 values
> >> Number of Dimensions: 2
> >> Dimensions and sizes: [366] x [7]
> >> Coordinates:
> >> Number Of Attributes: 1
> >> _FillValue : missing
> >>
> >> Variable: variables
> >> Type: string
> >> Total Size: 56 bytes
> >> 7 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [7]
> >> Coordinates:
> >> Number Of Attributes: 1
> >> _FillValue : missing
> >> (0) Day
> >> (1) AOT_675
> >> (2) AOT_500
> >> (3) AOT_440
> >> (4) 440-675Angstrom
> >> (5) ╬▒
> >> (6) AOT_550
> >>
> >> Variable: AOT_550_all
> >> Type: float
> >> Total Size: 14600 bytes
> >> 3650 values
> >> Number of Dimensions: 2
> >> Dimensions and sizes: [years | 10] x [days | 365]
> >> Coordinates:
> >> Number Of Attributes: 1
> >> _FillValue : 9.96921e+36
> >>
> >> Variable: files
> >> Type: string
> >> Total Size: 80 bytes
> >> 10 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [10]
> >> Coordinates:
> >> (0) /home/model-user/AOT_550_2005.txt
> >> (1) /home/model-user/AOT_550_2006.txt
> >> (2) /home/model-user/AOT_550_2007.txt
> >> (3) /home/model-user/AOT_550_2008.txt
> >> (4) /home/model-user/AOT_550_2009.txt
> >> (5) /home/model-user/AOT_550_2010.txt
> >> (6) /home/model-user/AOT_550_2011.txt
> >> (7) /home/model-user/AOT_550_2012.txt
> >> (8) /home/model-user/AOT_550_2013.txt
> >> (9) /home/model-user/AOT_550_2014.txt
> >>
> >> Variable: files10
> >> Type: string
> >> Total Size: 80 bytes
> >> 10 values
> >> Number of Dimensions: 1
> >> Dimensions and sizes: [10]
> >> Coordinates:
> >> (0) /home/model-user/AOT_550_2005.txt
> >> (1) /home/model-user/AOT_550_2006.txt
> >> (2) /home/model-user/AOT_550_2007.txt
> >> (3) /home/model-user/AOT_550_2008.txt
> >> (4) /home/model-user/AOT_550_2009.txt
> >> (5) /home/model-user/AOT_550_2010.txt
> >> (6) /home/model-user/AOT_550_2011.txt
> >> (7) /home/model-user/AOT_550_2012.txt
> >> (8) /home/model-user/AOT_550_2013.txt
> >> (9) /home/model-user/AOT_550_2014.txt
> >> fatal:No valid instance of variable AOT_550 found in file list
> >> fatal:["Execute.c":8575]:Execute: Error occurred at or near line
> >> 6095 in
> >> file
> >>
> > /home/model-user/NCL/ncl-6.3.0/lib/ncarg/nclscripts/csm/contributed.ncl
> >>
> >> Please assist to put me through.
> >>
> >> Thank you for your time.
> >>
> >> Najib
> >> _______________________________________________
> >> ncl-talk mailing list
> >> ncl-talk at ucar.edu
> >> List instructions, subscriber options, unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk [1]
> >
> >
> >
> > Links:
> > ------
> > [1] http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AOT_txt2nc.ncl
Type: application/octet-stream
Size: 3819 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: najiby_monthly.ncl
Type: application/octet-stream
Size: 3206 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: calculate_monthly_values.640.ncl.gz
Type: application/x-gzip
Size: 2949 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment.gz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a5a6.2005-2014.nc.gz
Type: application/x-gzip
Size: 1546 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment-0001.gz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a5a6.2005-2014.txt.gz
Type: application/x-gzip
Size: 852 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170619/3494a3e7/attachment-0002.gz
More information about the ncl-talk
mailing list