[ncl-talk] Reading in .ieeer8 file and plotting it with NCL
Hughlett, Taylor M
taylor.hughlett at uta.edu
Thu Jan 12 11:16:46 MST 2017
Thank you! We have done what we need to with the binary file now!
Taylor M Hughlett, Ph.D.
UTA Earth and Environmental Sciences
500 Yates St.
Geoscience Bldg Rm 117
Arlington, TX 76013-0049
From: Mary Haley <haley at ucar.edu<mailto:haley at ucar.edu>>
Date: Thursday, January 12, 2017 at 12:14 PM
To: Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>>
Cc: "Hughlett, Taylor M" <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>>, "ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>" <ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>
Subject: Re: [ncl-talk] Reading in .ieeer8 file and plotting it with NCL
Correction: The lat2d and lon2d should be attached to the data variable, not the "res" variable.
flx at lat2d = tlat
flx
@lon2d = ltlon
On Mon, Jan 9, 2017 at 3:39 PM, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:
I was just corrected on
----
then, in your graphics code
res at lat2d = lat2d
res at lon2d = lon2d
;---
Be this (?)
;----
then, in your graphics code
res at lat2d = tlat <=== correct
res at lon2d = ltlon <=== correct
On Mon, Jan 9, 2017 at 3:33 PM, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:
There are 'variables that contain coordinates' and 'coordinate variables'.
Coordinate Variables (CVs) are one-dimensional and contain montonicall {in/de}creasing numerical values. Think 'rectilinear grid'. The 'copy_VarCoords' applies only to this strict subset of coordinates.
===
Looking at the SFWF variable:
float SFWF(time, nlat, nlon) ;
SFWF:long_name = "Virtual Salt Flux in FW Flux formulation" ;
SFWF:units = "kg/m^2/s" ;
SFWF:coordinates = "TLONG TLAT time" ; <*****************
SFWF:grid_loc = "2110" ;
SFWF:cell_methods = "time: mean" ;
SFWF:_FillValue = 9.96921e+36f ;
SFWF:missing_value = 9.96921e+36f ;
===
REPLACE:
************************************************************
; Copy variable attributes to the binary data
;************************************************************
printVarSummary(flx)
copyVarMeta(sfwf,flx)
WITH
tlat = f->TLAT
ltlon = f->TLON
=======================
then, in your graphics code
res at lat2d = lat2d
res at lon2d = lon2d
+++++++++++++++++++++++++
http://www.ncl.ucar.edu/Applications/
Under "Models" ... Click CESM: POP
There are numerous examples of plotting this type of data.
HTH
D
On Mon, Jan 9, 2017 at 9:46 AM, Hughlett, Taylor M <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>> wrote:
Hi Dennis,
I finally got around to doing this. I am attempting to copy over metadata from the lat/lon coordinates in a netCDF file to the flx variable in the binary file, but it tells me the syntax is wrong? I followed the examples on the website…
Here is the full script with the area of concern highlighted:
;************************************************************
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl"
;************************************************************
begin
;************************************************************
; Set file paths
;************************************************************
fileb = "sfwf_mm_PHC2_salx_flxio_20090205.ieeer8"
dirb = "/glade/p/work/hughlett/PETM/paleo_setup/ocn/sfwf/"
pathb = dirb+fileb
reffl = "b.e12.B1850CN.PETM8x.f09_g16.pop.h.0001-01.nc<http://b.e12.B1850CN.PETM8x.f09_g16.pop.h.0001-01.nc>"
refflpath = "/glade/scratch/hughlett/csm/b.e12.B1850CN.PETM8x.f09_g16/ocn/hist/"
refpth = refflpath+reffl
pltnm = "bin_sfwf_mm_PHC2_salx_flxio_20090205"
pltdir = "/glade/scratch/hughlett/plots/b.e12.B1850CN.PETM8x.f09_g16/ocn/horizontal/"
pltpth = pltdir+pltnm
;************************************************************
; Set dimensions
;************************************************************
ntim = 12
nlat = 320
nlon = 384
dims = (/ntim,nlat,nlon/)
;************************************************************
; Open and read binary file
;************************************************************
setfileoption("bin","ReadByteOrder","BigEndian")
sss = fbindirread(pathb, 0, dims, "double")
flx = fbindirread(pathb, 1, dims, "double")
;************************************************************
; Open and read referece file
;************************************************************
f = addfile(refpth,"r")
sfwf = f->SFWF
;************************************************************
; Copy variable attributes to the binary data
;************************************************************
printVarSummary(flx)
copyVarMeta(sfwf,flx)
;************************************************************
; Details of plot
;************************************************************
field = "SFWF"
plt_unit = "kg m ~S~-2~N~ s~S~-1~N~"
fieldname = "Surface Freshwater Forcing"
ContourMode = "ManualLevels"
;ContourMode = "AutomaticLevels"
;ContourMode = "ExplicitLevels"
maxcn = 0.0002
mincn = -0.0002
ci = 0.00005
;************************************************************
; Set workstation and resources
;************************************************************
wks = gsn_open_wks("ps",pltpth) ; open a postscript file
gsn_define_colormap(wks,"MPL_BrBG")
res = True
res at mpProjection = "MOLLWEIDE" ; choose projection
res at mpGridAndLimbOn = True ; turn on lat/lon lines
res at mpPerimOn = False ; turn off box around plot
res at mpGridLatSpacingF = 30. ; spacing for lat lines
res at mpGridLonSpacingF = 30. ; spacing for lon lines
res at cnFillOn = True ; Fill contours
res at mpGridLonSpacingF = 30. ; spacing for lon lines
res at cnFillOn = True ; Fill contours
res at cnSmoothingOn = True
;res at mpMinLatF = 30. ; range to zoom in on
;res at mpMaxLatF = 60.
;res at mpMinLonF = 100.
;res at mpMaxLonF = 180.
cnSmoothingTensionF = -2.5
res at mpFillOn = False
res at mpCenterLonF = 0
res at cnLevelSelectionMode = ContourMode
if(ContourMode.eq."ExplicitLevels")then
res at cnLevels = contours
end if
if(ContourMode.eq."ManualLevels")then
res at cnMaxLevelValF = maxcn
res at cnMinLevelValF = mincn
res at cnLevelSpacingF = ci
end if
res at gsnSpreadColors = True
res at cnLinesOn = True
res at cnLineLabelsOn = True
res at cnLineLabelFontHeightF = 0.006
;res at cnLineLabelFontAspectF = 1.1
res at gsnDraw = False ; Do not draw plot
res at gsnFrame = False ; Do not advance frome
res at tiMainFontHeightF = .03
res at lbLabelFontHeightF = 0.012
res at cnLineLabelFontHeightF = .006
res at cnLineLabelBackgroundColor = "white"
res at txFontHeightF = 0.02
res at gsnRightString = plt_unit
res at gsnLeftString = fieldname
res at mpDataBaseVersion ="MediumRes"
res at txFont = 21
res at tiMainFont = 21
res at tiXAxisFont = 21
res at tiYAxisFont = 21
res at lbLabelFont = 21
res at tmXBLabelFont = 21
res at tmYLLabelFont = 21
;************************************************************
; Plot
;************************************************************
plot = new(1,graphic)
plot(0) = gsn_csm_contour_map(wks,flx(1,:,:),res)
end
Taylor M Hughlett, Ph.D.
UTA Earth and Environmental Sciences
500 Yates St.
Geoscience Bldg Rm 117
Arlington, TX 76013-0049
From: Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>>
Date: Thursday, January 5, 2017 at 12:26 PM
To: "Hughlett, Taylor M" <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>>
Cc: Dave Allured - NOAA Affiliate <dave.allured at noaa.gov<mailto:dave.allured at noaa.gov>>, "ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>" <ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>
Subject: Re: [ncl-talk] Reading in .ieeer8 file and plotting it with NCL
ntim = 12
nlat = 320
mlon = 384
dims = (/ntim,nlat,mlon/) ; dims(3)
filb = "binary_file_name"
dirb = "/dir/where_binary_file/located/"
pthb = dirb+filb
setfileoption("bin","ReadByteOrder","BigEndian")
sss = fbindirread(pthb, 0, dims, "double")
fx = fbindirread(pthb, 1, dims, "double")
On Thu, Jan 5, 2017 at 10:21 AM, Hughlett, Taylor M <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>> wrote:
Hi Dave,
Thank you so much for the information. I was able to track down the file that has all of the descriptions I required, and am now working on reading it in using NCL!
Cheers,
Taylor M Hughlett, Ph.D.
UTA Earth and Environmental Sciences
500 Yates St.
Geoscience Bldg Rm 117
Arlington, TX 76013-0049
From: Dave Allured - NOAA Affiliate <dave.allured at noaa.gov<mailto:dave.allured at noaa.gov>>
Date: Wednesday, January 4, 2017 at 6:52 PM
To: "Hughlett, Taylor M" <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>>
Cc: "ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>" <ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>>
Subject: Re: [ncl-talk] Reading in .ieeer8 file and plotting it with NCL
Taylor,
> 2. How do I know what data is stored on the file prior
> to reading it in (i.e. Variables)? Or is this even possible?
Files called "binary" without additional qualification can only be understood by formal documentation, or by at least a minimal description from the person who created the file. You need to get this description from the source of your data. Typically there is no descriptive metadata stored inside such files, and the assumption is that this description is somewhere external.
The description should include file layout details such as array sizes, positions, dimension order of storage, elemental data type, byte order, fortran write mode if any, and record structure if any. The description should also include what variables and possibly metadata are stored. If the layout details are hard to get, it can sometimes help to find out just what software created the file.
The suffix .ieeer8 says to me, "8-byte reals", because only fortran uses the type name "real" instead of "float" or "floating point". A fuller type description is "8-byte IEEE floating point values", but that is just a guess. If correct, that would be NCL data type "double", and that does not answer any of the other questions about file layout.
--Dave
On Wed, Jan 4, 2017 at 1:47 PM, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:
Were the data records written by fortran or
See: http://www.atmos.washington.edu/~salathe/osx_unix/endian.html
or ?????
[0]
The problem with binary is that someone must describe how the data was written. Fortran's default (sequential) binary has **hidden record separators**. The user could also open a binary with "access=direct". This means 'flat' binary files (no record separators.
[1]
Were the binary files created on a big- or little-endian system? See: https://en.wikipedia.org/wiki/Endianness
[2]
Likely, fbindirread *or* fbinrecread would work. However, without more knowledge not much more can be said.
[3]
One major reason for the creation of HDF and netCDF to allow the files to be 'blindly' examined via a suite of standard function functions.
On Wed, Jan 4, 2017 at 1:30 PM, Hughlett, Taylor M <taylor.hughlett at uta.edu<mailto:taylor.hughlett at uta.edu>> wrote:
Good afternoon everyone,
I am attempting to read in a .ieeer8 file using NCL so that it can be plotted.
I am completely new with binary files, so bear with me.
I understand that this is a binary file, so visualization and reading it is much more complicated than it would be with netCDF’s.
After having looked over the instructions on the NCL site, I am still having some confusion with how to get started.
Specifically, I have these questions:
1. Will the fbindirread work to read in a .ieeer8 file (I am assuming the file is fortran as it is a restart file for CESM1’s POP2)?
2. How do I know what data is stored on the file prior to reading it in (i.e. Variables)? Or is this even possible?
Any help would be greatly appreciated with this, and I am happy to provide more information if it is required.
Thanks,
Taylor M Hughlett, Ph.D.
UTA Earth and Environmental Sciences
500 Yates St.
Geoscience Bldg Rm 117
Arlington, TX 76013-0049
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170112/e2be2139/attachment.html
More information about the ncl-talk
mailing list