[ncl-talk] How to read variables in hdf5 group
David Brown
dbrown at ucar.edu
Thu Sep 14 16:30:49 MDT 2017
Hi Laura,
The variable you are looking at "LMA_170510_210000_600" is a compound
variable. The names "area, ctr_alt, ctr_lat, ctr_lon", etc. are
components of this compound variable.
Assuming you are using NCL 6.4.0, although not yet formally
documented, the usage for working with groups and compound variables
is discussed in the release notes at
http://www.ncl.ucar.edu/current_release.shtml#FileIOImprovements6.4.0.
Since I don't at the moment have access to your file, I'll give an
example of accessing a compound variable for a file I do have (I'm
assuming I already know the groups and variable names I want to
access):
ncl 0> f = addfile("MSG3-SEVI-MSG15-0100-NA-20130521001244.164000000Z-1074164.h5","r")
; here I use string referencing because the group names have
characters that are invalid in NCL symbol names
ncl 1> g = f=>$"U-MARF/MSG/Level1.5/DATA/Channel 07"$
ncl 2> print(g)
Variable: g
Type: group
groupname: /U-MARF/MSG/Level1.5/DATA/Channel 07
filename: MSG3-SEVI-MSG15-0100-NA-20130521001244.164000000Z-1074164
path: MSG3-SEVI-MSG15-0100-NA-20130521001244.164000000Z-1074164.h5
...
Variable: PacketHeader_DESCR
Type: compound
Total Size: 20 values
3200 bytes
Number of Dimensions: 1
Dimensions and sizes: [ 20 <DIM_003> ]
Coordinates:
Number of Attributes: 6
component_names : ( EntryName, Value )
CLASS : TABLE
FIELD_0_NAME : EntryName
FIELD_1_NAME : Value
TITLE : PacketHeader_DESCR
VERSION : 2.0
; there are two ways to get the data from this variable: extracting
each component individually or getting the whole variable into an NCL
list. I will show each:
ncl 3> p = g->PacketHeader_DESCR
ncl 4> print(p)
Variable: p
Type: list
Total Size: 4 bytes
1 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_003 | 1]
Coordinates:
Number Of Attributes: 6
component_names : ( EntryName, Value )
CLASS : TABLE
FIELD_0_NAME : EntryName
FIELD_1_NAME : Value
TITLE : PacketHeader_DESCR
VERSION : 2.0
Type: list <fifo>
Total items: 2
List Item 0: NclVarClass
Variable: EntryName
Type: string
Total Size: 160 bytes
20 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_003 | 20]
Coordinates:
List Item 1: NclVarClass
Variable: Value
Type: string
Total Size: 160 bytes
20 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_003 | 20]
Coordinates:
; Notice there are two items in the list, one for each compound
member.Now look at the values:
ncl 5> print(p[0])
Variable: EntryName
Type: string
Total Size: 160 bytes
20 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_003 | 20]
Coordinates:
(0) HeaderVersionNo
(1) PacketType
(2) SubHeaderType
(3) SourceFacilityId
(4) SourceEnvId
(5) SourceInstanceId
(6) SourceSUId
(7) SourceCPUId
(8) DestfacilityId
(9) DestEnvId
(10) SequenceCount
(11) PacketLength
(12) SubheaderVersionNo
(13) ChecksumFlag
(14) Acknowledgement
(15) ServiceType
(16) ServiceSubType
(17) PacketTime-days
(18) PacketTime-milliseconds
(19) SpacecraftId
ncl 6> print(p[1])
Variable: Value
Type: string
Total Size: 160 bytes
20 values
Number of Dimensions: 1
Dimensions and sizes: [DIM_003 | 20]
Coordinates:
(0) 0
(1) 2
(2) 1
(3) 4
(4) 1
(5) 3
(6) 40024
(7) 169090592
(8) 5
(9) 0
(10) data
(11) 4682
(12) 0
(13) 0
(14) 0
(15) 154
(16) 2
(17) 20229
(18) data
(19) 323
; Getting these values by component:
ncl 8> e = g->/PacketHeader_DESCR.EntryName
ncl 9> print(e)
Variable: e
Type: string
Total Size: 160 bytes
20 values
Number of Dimensions: 2
Dimensions and sizes: [DIM_003 | 20] x [EntryName | 1]
Coordinates:
Number Of Attributes: 6
component_names : ( EntryName, Value )
CLASS : TABLE
FIELD_0_NAME : EntryName
FIELD_1_NAME : Value
TITLE : PacketHeader_DESCR
VERSION : 2.0
(0,0) HeaderVersionNo
(1,0) PacketType
(2,0) SubHeaderType
(3,0) SourceFacilityId
(4,0) SourceEnvId
(5,0) SourceInstanceId
(6,0) SourceSUId
(7,0) SourceCPUId
(8,0) DestfacilityId
(9,0) DestEnvId
(10,0) SequenceCount
(11,0) PacketLength
(12,0) SubheaderVersionNo
(13,0) ChecksumFlag
(14,0) Acknowledgement
(15,0) ServiceType
(16,0) ServiceSubType
(17,0) PacketTime-days
(18,0) PacketTime-milliseconds
(19,0) SpacecraftId
The second dimension here is bogus; this flaw will be fixed in the next release.
One thing to notice is the '/' character after the arrow '->'. This
character is necessary for backwards compatibility as noted in the
release notes. However, the release notes are not
quite complete. The '/' is needed anytime you use the '.' character to
specify a component member of a compound variable, as well as anytime
you specify a variable in a subgroup of the current group. For
example, in your case if you want to get the variable from the
top-level file, you would need to say:
v = f->/flashes/LMA_170510_210000_600
or to get the component "area":
area = f->/flashes/LMA_170510_210000_600.area
If you are referencing the variable from its immediate group then the
'/' may be optional, as in:
v = flashes->LMA_170510_210000_600
works as well as
v = flashes->/LMA_170510_210000_600
However, it is still required to retrieve individual components:
area = flashes->/LMA_170510_210000_600.area
I hope this helps.
-dave
On Thu, Sep 14, 2017 at 2:30 PM, Laura Fowler <laura at ucar.edu> wrote:
> Hello:
>
> I would like to read a hdf5 file using ncl. After using ncdump, I see
> that besides 4 global attributes the file also contains 2 groups. So,
> after using addfile, I read and print the group "flashes":
>
> flashes = f=>flashes
> print(flashes)
>
> The beginning of the output from print is:
>
> Number of Attributes:3
> CLASS : GROUP
> TITLE : Sorted LMA flash data
> VERSION : 1.0
>
> variables:
> Variable: LMA_170510_210000_600
> Type: compound
> Total Size: 2590 values
> 10360 bytes
> Number of Dimensions: 1
> Dimensions and sizes: [ 2590 <(null)> ]
> Coordinates:
> Number of Attributes: 35
>
> component_names : ( area, ctr_alt, ctr_lat, ctr_lon,
> duration, flash_id, init_alt, init_lat, init_lon, init_pts, ... [Total
> of 15 values] )
>
> CLASS : TABLE
> FIELD_0_FILL : 0
> FIELD_0_NAME : area
> FIELD_10_FILL :
> FIELD_10_NAME : n_points
> FIELD_11_FILL : 0
> FIELD_11_NAME : specific_energy
>
>
> Now I would like to read the variables (area, ctr_alt, ...) in the
> group "flashes" but cannot figure out how to access them. I looked
> around the documentation but did not see an obvious example to do so.
> Is there some more detailed documentation somewhere, or can someone
> write me back on how to do this?
>
> Thanks,
> Laura
>
> --
> !-------------------------------------------------------------------------------------------------------------
> Laura D. Fowler
> Mesoscale and Microscale Meteorology Division (MMM)
> National Center for Atmospheric Research
> P.O. Box 3000, Boulder CO 80307-3000
>
> e-mail: laura at ucar.edu
> phone: 303-497-1628
>
> !-------------------------------------------------------------------------------------------------------------
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
More information about the ncl-talk
mailing list