[ncl-talk] netcdf file creation and order of dimensions
Dennis Shea
shea at ucar.edu
Thu Oct 22 21:38:02 MDT 2020
You csn use functions/procedures but I prefer using NCL syntax:
See: *https://www.ncl.ucar.edu/Applications/method_1.shtml*
<https://www.ncl.ucar.edu/Applications/method_1.shtml>
zmid(latitude,longitude,levz_mid)
========
work = zmid(levz_mid|:,latitude|:,longitude|:) ; reorder via NCL
syntax: sizes: (klev,nlat,mlon)
printVarSummary(work)
print("=====")
ntim = 1 ; 0 1 2 3
WORK = *conform_dims*
<http://www.ncl.ucar.edu/Document/Functions/Built-in/conform_dims.shtml>((/ntim,klev,nlat,mlon/),
work, (/1,2,3/))
printVarSummary(WORK)
print("=====")
copy_VarMeta(x, WORK(0,:,:,:)) ; coordinates and attributes
WORK!0 = "time" ; name dimension
;; WORK&time = (/...../) ; add time coordinate
printVarSummary(WORK)
print("=====")
*setfileoption*
<http://www.ncl.ucar.edu/Document/Functions/Built-in/setfileoption.shtml>("nc","Format","NetCDF4Classic")
diro = "./"
filo = "SampleFile_KemalGurer.nc"
ptho = doro+filo
fo = *addfile*
<https://www.ncl.ucar.edu/Document/Functions/Built-in/addfile.shtml>(ptho,"c")
;===================================================================
; create global attributes of the file (optional)
;===================================================================
fAtt = True ; assign file attributes
fAtt at title = "NCL netCDF Creation: Kemal Gurer"
fAtt at source_file = "original-file.nc"
fAtt at Conventions = "None"
fAtt at creation_date = *systemfunc*
<https://www.ncl.ucar.edu/Document/Functions/Built-in/systemfunc.shtml>
("date")
*fileattdef* <https://www.ncl.ucar.edu/Document/Functions/Built-in/fileattdef.shtml>(
fo, fAtt ) ; copy file attributes
;===================================================================
; make time an *UNLIMITED* dimension; recommended for most applications
;===================================================================
*filedimdef*
<https://www.ncl.ucar.edu/Document/Functions/Built-in/filedimdef.shtml>(fo,"time",-1,True)
;===================================================================
; output variables directly; NCL will call appropriate functions
; to write the meta data associated with each variable
;===================================================================
fo->ZMID = WORK ; 4D
==========
After file is created, examine file contents
%> *ncdump* -h SampleFile_KemalGurer.nc
or
%> *ncl_filedump* SampleFile_KemalGurer.nc
On Thu, Oct 22, 2020 at 5:28 PM Gurer, Kemal at ARB via ncl-talk <
ncl-talk at mailman.ucar.edu> wrote:
> Hello,
>
>
>
> I am writing some variables into a netcdf file, but for my problem, let me
> show one variable here, zmid, with the netcdf name ZH, using the following
> netcdf commands:
>
>
>
> filevardef(f,"ZH","float",(/"TSTEP","LAY","ROW","COL"/))
>
> ZH_atts = 0.0
>
> ZH_atts at long_name = "ZH"
>
> ZH_atts at var_desc = "mid-layer height above ground"
>
> ZH_atts at units = "M"
>
> ZH_atts at _FillValue = 1e32
>
> filevarattdef(f,"ZH",ZH_atts)
>
> ZH = new((/nlev, nlat, nlon/), "float", ZH_atts at _FillValue)
>
>
>
> do n = 0, ntim-1
>
> f->ZH(n,:,:,:) = (/zmid/)
>
> end do
>
>
>
> Here, TSTEP is Time, LAY is levels, ROW is latitude, and COL is longitude
> with the exact dimensions of the dimensions given in the original data.
>
>
>
> printVarSummary(zmid) gives the following information:
>
>
> -----------------------------------------------------------------------------------
>
> Variable: zmid
>
> Type: float
>
> Total Size: 2328480 bytes
>
> 582120 values
>
> Number of Dimensions: 3
>
> Dimensions and sizes: [latitude | 98] x [longitude | 108] x [levz_mid |
> 55]
>
> Coordinates:
>
> latitude: [31.6178..42.50189]
>
> longitude: [-129.0292..-113.0208]
>
> Number Of Attributes: 2
>
> units : m MSL
>
> long_name : Geometric height of mid-levels
>
> (0)
>
> (0) Geometric height of mid-levels (m MSL) : min=-64.3161 max=29608.8
>
>
> -----------------------------------------------------------------------------------
>
> Since I need to write the variable “zmid” as well as other variables into
> the netcdf file with the following order of dimensions:
>
>
>
> ("Time","nVertLevels","latitude","longitude") with the dimension names
>
>
>
> I reorder “zmid” via:
>
>
>
> zmid_reorder = zmid(levz_mid|:,latitude|:,longitude|:)
>
> printVarSummary(zmid_reorder),
>
>
>
> which gives the following output:
>
>
> -----------------------------------------------------------------------------------
>
> Variable: zmid_reorder
>
> Type: float
>
> Total Size: 2328480 bytes
>
> 582120 values
>
> Number of Dimensions: 3
>
> Dimensions and sizes: [levz_mid | 55] x [latitude | 98] x [longitude |
> 108]
>
> Coordinates:
>
> latitude: [31.6178..42.50189]
>
> longitude: [-129.0292..-113.0208]
>
> (0) min=-64.3161 max=29608.8
>
>
> -----------------------------------------------------------------------------------
>
> However, the reordered zmid output in netcdf file is wrong. The output of
> zmid without reordering of dimensions is:
>
>
>
> float ZH(Time, LAY, ROW, COL) ;
>
> ZH:_FillValue = 1.e+32f ;
>
> string ZH:units = "M" ;
>
> string ZH:var_desc = "mid-layer height above ground" ;
>
> string ZH:long_name = "ZH" ;
>
>
>
> data:
>
>
>
> ZH =
>
> 9.999999, 32, 58.4, 90.09999, 128.1, 173.7, 228.45, 294.15, 373, 467.6,
>
> 581.1, 717.3, 880.75, 1076.9, 1312.3, 1594.75, 1933.7, 2340.45,
> 2828.55,
>
> 3414.3, 4108.8, 4858.8, 5608.799, 6358.8, 7108.8, 7858.8, 8608.8,
> 9358.8,
>
> 10108.8, 10858.8, 11608.8, 12358.8, 13108.8, 13858.8, 14608.8, 15358.8,
>
> 16108.8, 16858.8, 17608.8, 18358.8, 19108.8, 19858.8, 20608.8, 21358.8,
>
> 22108.8, 22858.8, 23608.8, 24358.8, 25108.8, 25858.8, 26608.8, 27358.8,
>
> 28108.8, 28858.8, 29608.8, 10, 32, 58.4, 90.10001, 128.1, 173.7,
> 228.45,
>
> 294.15, 373, 467.6, 581.1, 717.3, 880.75, 1076.9, 1312.3, 1594.75,
>
> 1933.7, 2340.45, 2828.55, 3414.3, 4108.8, 4858.8, 5608.8, 6358.8,
> 7108.8,
>
> 7858.8, 8608.8, 9358.8, 10108.8, 10858.8, 11608.8, 12358.8, 13108.8,
>
> 13858.8, 14608.8, 15358.8, 16108.8, 16858.8, 17608.8, 18358.8, 19108.8,
>
> 19858.8, 20608.8, 21358.8, 22108.8, 22858.8, 23608.8, 24358.8, 25108.8,
>
> 25858.8, 26608.8, 27358.8, 28108.8,
>
> 28858.8, 29608.8,
>
>
>
> ----------------------------------------------------------
>
>
>
> The netcdf output of zmid_reorder with netcdf variable name ZH1 as test is:
>
>
>
> float ZH1(TSTEP, LAY, ROW, COL) ;
>
> ZH1:_FillValue = 1.e+32f ;
>
> string ZH1:units = "M" ;
>
> string ZH1:var_desc = "mid-layer height above ground" ;
>
> string ZH1:long_name = "ZH1" ;
>
>
>
> data:
>
>
>
> ZH1 =
>
> 9.999999, 10, 10, 10, 10, 10, 10, 10, 10, 10, 9.999999, 10, 10, 10, 10,
> 10,
>
> 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
>
> 10, 10, 10, 9.999999, 10, 10, 10, 9.999999, 10, 10, 10, 10, 10, 10, 10,
>
> 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
>
> 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
> 423.6477,
>
> 378.6578, 662.8945, 1097.154, 1152.52, 1072.004, 1299.672, 1005.835,
>
> 592.6567, 270.4732, 318.5765, 13.87211, 13.37675, 10, 10, 49.30841,
>
> 48.51982, 38.2547, 10.43509, 20.66064, 89.90919, 139.163, 114.4501,
>
> 201.582, 276.1719,
>
> 10, 10, 10, 10, 10, 10, 10,
>
>
>
> ---------------------------------------------------------------
>
> As another example of for the impact of variable reordering on the output,
> here is the information about code to read potential temperature and
> reordering it as well as its results:
>
>
>
> tk1 = a->theta
>
> ; copy_VarMeta(a->theta,tk1)
>
> ; tk1 at units = ""
>
> printVarSummary(tk1)
>
> printMinMax(tk1,0)
>
>
>
> tk = tk1(Time|:,nVertLevels|:,latitude|:,longitude|:)
>
> printVarSummary(tk)
>
> printMinMax(tk,0)
>
>
>
> --------------------------------------------------------------------
>
> Variable: tk1
>
> Type: float
>
> Total Size: 55883520 bytes
>
> 13970880 values
>
> Number of Dimensions: 4
>
> Dimensions and sizes: [Time | 24] x [latitude | 98] x [longitude | 108]
> x [nVertLevels | 55]
>
> Coordinates:
>
> latitude: [31.6178..42.50189]
>
> longitude: [-129.0292..-113.0208]
>
> Number Of Attributes: 2
>
> units : K
>
> long_name : Potential temperature
>
> (0) Potential temperature (K) : min=282.766 max=797.725
>
>
>
> data:
>
>
>
> TK1 =
>
> 290.8131, 290.7571, 290.7412, 290.7352, 290.734, 290.7361, 290.7417,
>
> 290.7557, 290.9228, 293.661, 296.6131, 298.7729, 301.4621, 303.3734,
>
> 305.5786, 307.7134, 309.639, 311.1685, 312.9564, 315.1082, 317.7267,
>
> 320.5912, 323.4933, 326.3149, 329.7065, 333.1702, 337.303, 339.9579,
>
> 341.5059, 344.3701, 347.8319, 354.1167, 358.0386, 360.7633, 364.5301,
>
> 372.0264, 381.8416, 392.6732, 410.2569, 427.9963, 446.3387, 469.1274,
>
> 492.7356, 513.7944, 534.12, 553.8484, 575.1737, 600.4743, 628.2712,
>
> 658.1637, 681.6721, 702.9726, 730.4858, 756.6381, 787.6335, 290.8405,
>
> 290.7827, 290.7664, 290.7603, 290.7591, 290.7614, 290.7675, 290.7849,
>
> 290.9932, 293.7723, 296.4947, 298.9371, 301.5804, 303.4898, 305.7006,
>
> 307.7661, 309.706, 311.2367, 313.1123, 315.1071, 317.8082, 320.5843,
>
> 323.5288, 326.2887, 329.711, 333.046, 337.2808, 339.8705, 341.3407,
>
> 344.2988, 348.018, 354.2007, 357.992, 360.691, 364.6261, 371.9153,
>
> 381.6589, 392.6092, 410.5144, 428.3927, 446.4816, 469.0464, 491.7896,
>
> 513.2549, 534.1764, 555.1635, 576.1888, 599.7042, 626.2191, 656.9671,
>
> 681.5657, 704.613, 733.2448,
>
>
> -----------------------------------------------------------------------------------
>
>
>
> Variable: tk
>
> Type: float
>
> Total Size: 55883520 bytes
>
> 13970880 values
>
> Number of Dimensions: 4
>
> Dimensions and sizes: [Time | 24] x [nVertLevels | 55] x [latitude | 98]
> x [longitude | 108]
>
> Coordinates:
>
> latitude: [31.6178..42.50189]
>
> longitude: [-129.0292..-113.0208]
>
> Number Of Attributes: 2
>
> long_name : Reordered potential temperature
>
> units : K
>
> (0) Potential temperature (K) : min=282.766 max=797.725
>
>
>
> data:
>
>
>
> TK =
>
> 290.8131, 290.8405, 290.8532, 290.874, 290.8821, 290.8575, 290.9009,
>
> 290.9425, 290.9517, 290.9512, 290.9521, 290.9454, 290.9456, 290.9441,
>
> 290.9357, 290.9396, 290.9547, 290.9457, 290.9522, 290.9639, 290.9545,
>
> 290.9518, 290.9407, 290.9241, 290.9189, 290.9194, 290.8954, 290.8531,
>
> 290.8095, 290.7653, 290.6684, 290.5908, 290.5103, 290.4197, 290.3355,
>
> 290.2524, 290.0889, 289.9354, 289.8426, 289.769, 289.7196, 289.5693,
>
> 289.5384, 289.5408, 289.3453, 289.1881, 289.1, 289.0808, 289.054,
>
> 289.0443, 289.0854, 288.9913, 289.123, 289.17, 289.3506, 289.4048,
>
> 289.4511, 289.4771, 289.5936, 289.642, 289.163, 289.2242, 289.2056,
>
> 289.2532, 289.3674, 289.5267, 289.6508, 289.8015, 289.9764, 290.2227,
>
> 290.4557, 290.7174, 290.9393, 291.1483, 291.4482, 291.6609, 291.8963,
>
> 292.1499, 292.4067, 292.6108, 292.777, 292.8965, 293.0784, 299.2768,
>
> 299.7407, 305.4052, 313.3188, 313.5712, 314.6627, 313.7572, 313.2654,
>
> 312.6786, 311.9484, 311.588, 311.4529, 308.0678, 307.5142, 307.9211,
>
> 308.1941, 309.1768, 309.8638, 310.4225, 309.4877, 310.032, 311.1737,
>
> 311.9905, 312.4248, 311.6823,
>
>
> ---------------------------------------------------------------------------------------
>
>
>
> I use the following netcdf file definitions:
>
>
>
> theflnm = "./nc4_write.nc"
>
> setfileoption("nc","format","netcdf4")
>
> setfileoption("nc","headerReserveSpace",64000)
>
> setfileoption("nc","preFill",False)
>
> setfileoption("nc","defineMode",True)
>
>
>
> I also tried to use the following chunking definitions, but commented them
> out because of the errors that I get and let the internal default
> definitions take over:
>
>
>
> ;mtim = 1
>
> ;mlev = nlev/10
>
> ;mlat = nlat/10
>
> ;mlon = nlon/10
>
> ;chunk_sizes = (/mtim,mlev,mlat,mlon/)
>
> ;filechunkdimdef(f,dim_names,chunk_sizes,dim_unlimited)
>
>
>
> ;filevarchunkdef(f, "TA", chunk_sizes)
>
> ;filevarcompressleveldef(f, "TA", 2)
>
>
>
> Is my definition of netcdf file and/or variables in netcdf definitions
> wrong? Are my definitions for the order of dimensions using reorder and the
> order of dimensions in netcdf declarations not supposed to be the same?
> Just as a summary, original variable, reordered variable and dimension
> declarations in netcdf attributes are (in order):
>
>
>
> Potential Temperature - [Time | 24] x [latitude |
> 98] x [longitude | 108] x [nVertLevels | 55]
>
> Reordered Potential Temperature - [Time | 24] x [nVertLevels | 55] x
> [latitude | 98] x [longitude | 108]
>
> float TK1(TSTEP, LAY, ROW, COL) ;
>
> TK1:_FillValue = 1.e+32f ;
>
> string TK1:units = "K" ;
>
> string TK1:var_desc = "potential temperature" ;
>
> string TK1:long_name = "TK1" ;
>
> float TK(TSTEP, LAY, ROW, COL) ;
>
> TK:_FillValue = 1.e+32f ;
>
> string TK:units = "K" ;
>
> string TK:var_desc = "reordered potential temperature" ;
>
>
>
> Thank you for your help.
>
>
>
> Kemal.
>
>
>
>
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at mailman.ucar.edu
> List instructions, subscriber options, unsubscribe:
> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20201022/60caa2fb/attachment.html>
More information about the ncl-talk
mailing list