[ncl-talk] Era-Interim daily precipitation
Ehsan Taghizadeh
ehsantaghizadeh at yahoo.com
Sun Jun 10 16:20:29 MDT 2018
Thank you so much for your helpful comments.However could it be related to do not using statements likesetfileoption("nc", "FileStructure", "Advanced")or setfileoption("nc", "Format", "NetCDF4Classic")Because in the script we don't use these statements while reading input file:[SNIP]
;diri = "/Users/shea/Data/ECMWF/ERAI/"
diri = "/home/taghizade/97_1_17/5_Era-Interim/" fili = "2_ERAI_Prec_2017_Jan_00_12_12.nc" pthi = diri+fili
f = addfile(pthi, "r") tp = short2flt(f->tp)[SNIP]
However I've used them like below:[SNIP] setfileoption("nc", "FileStructure", "Advanced") ;diri = "/Users/shea/Data/ECMWF/ERAI/"
diri = "/home/taghizade/97_1_17/5_Era-Interim/" fili = "2_ERAI_Prec_2017_Jan_00_12_12.nc" pthi = diri+fili
f = addfile(pthi, "r") tp = short2flt(f->tp)setfileoption("nc", "Format", "NetCDF4Classic")[SNIP]
and I didn't see any change, but I'm not expert and I thought may be I don't use correct options.
SincerelyEhsan
On Sunday, June 10, 2018, 6:45:31 PM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
There appears to be an unusual bug with 6.4.0 with this file.
This has not been encountered before.The work-around is to not use the 'unlimited' designation.
Please remove the following line:
filedimdef(ncdf,"time",-1,True ) ; make time UNLIMITED
====
Writing the file as NetCDF-4 with time not set to unlimited results in a 14 MB file
Writing the file as NetCDF-3, both limited and unlimited,results in 14 MB files
====
A detail:
Specifically: The unusual thing about the 474 MB fie is that it seems to think it has chunked variables with a dimension size of 1024:
Variable: TP Type: float Total Size: 3586080 values 14344320 bytes Number of Dimensions: 3 Dimensions and sizes: [ 31 <time | unlimited> x 241 <latitude> x 480 <longitude> ] Chunking Info: [ 1024 <time> x 241 <latitude> x 480 <longitude> ]
The large size is coming from a time dimension that it thinks is size 1024:This has been reported as NCL 2757.
On Sat, Jun 9, 2018 at 12:43 PM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dears,
In the script to write nc format, fn = "3_ERAI_Prec_2017_Jan_00_12_ 24_ncl.nc4",which is created, has size of about 450 MB! However input filefili = "2_ERAI_Prec_2017_Jan_00_12_12.nc"
is less than 10 MB!Also the file which was created with cdo: F_cdo = "2_2_ERAI_Prec_2015_Jan_24_ cdo.nc4", is less than 15 MB!Could I ask why the created file in ncl has this large size?Also how could I have a nc file with smaller size?
SincerelyEhsan
On Saturday, June 9, 2018, 1:23:31 AM GMT+4:30, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Thanks a lot. It works correctly.
SincerelyEhsan
On Saturday, June 9, 2018, 1:12:23 AM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
offline:
Please add the following 'filedimdef' to what I previously sent.
[SNIP]
fileattdef(fo, fAtt)
filedimdef(ncdf,"time",-1, True) ; make time UNLIMITED
var_name = "TP"
[SNIP]
On Fri, Jun 8, 2018 at 2:00 PM, Dennis Shea <shea at ucar.edu> wrote:
Please use the 'simple method' to write netCDF:
https://www.ncl.ucar.edu/ Applications/method_1.shtml
===
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;
;;;Start: writing tp3_ncl in nc file
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;
setfileoption("nc", "Format", "NetCDF4")
dn = "./"
fn = "3_ERAI_Prec_2017_Jan_00_12_ 24_ncl.nc4"
pn = dn + fn ; path name
system("/bin/rm -f " + pn)
fo = addfile(pn, "c")
fAtt = True ; assign file attributes
fAtt at title = "ERAI: two-timestep sum"
fAtt at source_file = fili
fAtt at Conventions = "None"
fAtt at creation_date = systemfunc("date")
fileattdef(fo, fAtt)
;============================= ============================== ========
var_name = "TP"
fo->$var_name$ = tp3_ncl
On Fri, Jun 8, 2018 at 8:31 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dears,I did some editions on previous script, and it seems works better.However the problem about time dimensions. I'll be thankful if discard previous script and for any help about new attached file.
SincerelyEhsan
On Friday, June 8, 2018, 5:54:04 PM GMT+4:30, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dear Dennis,
Thank you so much for your script.In your script, "tp3_ncl" is exactly what I need, as its values are the same as "tp_cdo" values.However could I have any help to write this variable (tp3_ncl) in nc file, just like (tp) in (2_ERAI_Prec_2015_Jan_00_12_12 .nc)?I've added between 185-218, I've attached the script. However it is absolutely wrong.
I'll be thankful for any help.
SincerelyEhsan
On Tuesday, June 5, 2018, 4:26:15 AM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
I am going to add a few more items to a previous response:
The Golden Rules of data processing are:
(a) Look at your data!
(b) Know your data!
(c) Make sure *you* know what a function or tool is doing to your data
---
The source netcDF '2_2_ERAI_Prec_2015_Jan_00_12_ 12.nc' was created from a GRIB file:
history attribute:
2018-05-16 09:15:25 GMT by grib_to_netcdf-2.7.3: grib_to_netcdf /data/data05/scratch/3c/71/_ma rs-atls00-a562cefde8a29a7288fa 0b8b7f9413f7-b4mPzK.grib -o /data/data04/scratch/41/14/_gr ib2netcdf-atls17-a562cefde8a29 a7288fa0b8b7f9413f7-PZaKeY.nc -utime" ;
---
There are 62 time steps on the source ERAI file:
yyyymmddhh tp value
(0) ERAI: 2015010112 0.000330344 <=== only one value for this calendar day
(1) ERAI: 2015010200 0.0051588 ...... two time steps per calendar day ...
(2) ERAI: 2015010212 0.00401913
(3) ERAI: 2015010300 0
(4) ERAI: 2015010312 0
(5) ERAI: 2015010400 0
(6) ERAI: 2015010412 0.000478998
(7) ERAI: 2015010500 0.00344653
[SNIP]
(55) ERAI: 2015012900 0
(56) ERAI: 2015012912 0
(57) ERAI: 2015013000 9.35942e-05
(58) ERAI: 2015013012 7.15703e-05
(59) ERAI: 2015013100 0.00390901
(60) ERAI: 2015013112 0
(61) ERAI: 2015020100 0.00183889 <=== only one value for this calendar day---
To what 'time' does the source ERAI 'time stamp apply"?
That is ***USER RESPONSIBILITY*** to know or find the answer.
Three possibilities:
[1] beginning
[2] end
[3] 'center-of-mass' ... mid temporal time?
---
NCL's 'calculate_daily_values' with 'ncrit=2' requires two values on the same calendar day
https://www.ncl.ucar.edu/ Document/Functions/ Contributed/calculate_daily_ values.shtml
With ncrit=2 the function returns 30 values ... not 31 or 32. Why?
Because it requires 2 values per calendar date. It ignores the 1st and last times because there is only one time step per calendar day
tp2_ncl
NCL_tag : calculate_daily_values: arith=sum
----
The function 'tags' it with the 1st time step of each *complete* calendar day
(0) NCL: 2015010200 0.00917792
(1) NCL: 2015010300 0
(2) NCL: 2015010400 0.000478998
(3) NCL: 2015010500 0.00879252
[SNIP]
(28) NCL: 2015013000 0.000165164
(29) NCL: 2015013100 0.00390901
---
The CDO takes the first two values [0,1], then the next two [2,3], etc.
It 'tags' the time as the 2nd time step
(0) CDO: 2015010200 0.00548913
(1) CDO: 2015010300 0.00401912
(2) CDO: 2015010400 -1.08611e-10
[SNIP]
(29) CDO: 2015013100 0.00398058
(30) CDO: 2015020100 0.00183889
Even though the time 'tags' are the same, they refer to different sums.
---
NCL can be made to handle the time and data any way the user wants.
I'll let you look at the attached script and output
On Fri, Jun 1, 2018 at 4:03 PM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dears,About summation of 12 hours precipitation to obtain daily precipitation, as suggested me I used "cdo" and it seems work correctly:"cdo -b F32 -f nc4c -z zip timselsum,2 2_2_ERAI_Prec_2015_Jan_00_12_1 2.nc 2_2_ERAI_Prec_2015_Jan_24_cdo. nc4"However I like to know how writing nc files with "Geo2D".
SincerelyEhsan
On Thursday, May 31, 2018, 7:12:17 PM GMT+4:30, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Thank you so much for your nice replies.With you comment, it seems work correctly. However I've added some changes to the script to have an output file same as input file, I mean about dimensions, attributes, and so on.However I have some difficulties about it and I'll be thankful if I could have your help, again.For example aboutfout->tp24_1!1 = tp!1fout->tp24_1!2 = tp!2I encountered error. It seems they have conflict with following lines:fout->latitude = fin->latitudefout->longitude = fin->longitude
Beside that I want to have "tp24_1" with dimension "Geo2D" to plot.I attached new version of script, and I'll be thankful if I hear from you.
SincerleyEhsan On Thursday, May 31, 2018, 6:44:48 PM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
The golden rule of data processing is look at your data
You have:
tp=f->tp ; <=== type 'short' [ packed data]
However, your file has:
short tp ( time, latitude, longitude )
scale_factor : 5.505649013956001e-06
add_offset : 0.1803980955369768
_FillValue : -32767
missing_value : -32767
units : m
long_name : Total precipitation
You *MUST* unpack the variable before computing the sums.
tp = short2flt(f->tp)
printVarSummary(tp)
printMinMax(tp,0)
===
https://www.ncl.ucar.edu/ Document/Functions/ Contributed/short2flt.shtml
On Thu, May 31, 2018 at 7:19 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Dears,
I've tried sum 12 hour precipitation to obtain 24 hour precipitation using attached script.However it seems doesn't work correctly!I had to sum tp(0,:,:)+tp(1,:,:), also tp(2,:,;)+tp(3,;,:), and so on.I uploaded "2_2_ERAI_Prec_2015_Jan_00_12_ 12.nc" in ftp.cgd.ucar.edu.I'll be thankful if I hear from you.
SincerelyEhsan
On Wednesday, May 16, 2018, 8:23:32 AM GMT+4:30, Dennis Shea <shea at ucar.edu> wrote:
This is book keeping. Something like:
loop each year
loop each month
loop each day
sum all values occurring between 00:00 and 11:59 inclusive ==>PRC(time,lat,lon,0)
sum all values occurring between 12:00 and 23:59 inclusive ==>PRC(time,lat,lon,1)
end day
end month
end year
On Sun, May 13, 2018 at 9:07 AM, Ehsan Taghizadeh <ehsantaghizadeh at yahoo.com> wrote:
Hi every body;Could I have any help to get daily precipitation from Era-Interim, please?I mean, a way to sum "accumulated precipitation from 00:00 to 12:00" with "accumulated precipitation from 12:00 to 24:00"?
SincerelyEhsan
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailma n/listinfo/ncl-talk
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180610/31a05533/attachment.html>
More information about the ncl-talk
mailing list