[ncl-talk] hdf confusion
Dennis Shea
shea at ucar.edu
Tue Jun 13 15:29:25 MDT 2017
NASA often concatenates 'bit streams' of flags to minimize file/variable
sizes. Users must 'parse out' (extract) the appropriate bits.
NCL's dim_gbits can be used to perform the task:
https://www.ncl.ucar.edu/Document/Functions/Built-in/dim_gbits.shtml
See: https://www.ncl.ucar.edu/Applications/HDF.shtml
Example 5
=========================
Note about 'short2flt' and 'short2flt_hdf'
If a file is COARDS or CF compliant and the variable (xs) is type short
with 'scale_factor' and 'add_offset' attributes, the unpack formula is
required to be
x = xs*xs at scale_factor + x at add_offset
NASA does not seem to have standard. As a result, there is not standard
names of the 'scale_factor' or 'add_offset' attributes. Further, some
variables may be unpacked via an alternative the the CF formula:
x = (xs + xs at _add_offset)*x at scale_factor
NCL's short2flt_hdf actually searches for:
; names to check
oNames = (/"add_offset", "offset", "OFFSET", "Offset", "_offset" \
,"Intercept", "intercept", "scalingIntercept", "INTERCEPT" \
,"add_off" /)
sNames = (/"scale", "SCALE", "Scale", "_scale", "scale_factor" \
,"Scale_factor", "Slope" , "slope", "ScaleFactor" \
,"Scale_Factor", "scalingSlope", "SCALING_FACTOR" \
,"SCALE_FACTOR", "SLOPE" /)
Good luck
On Tue, Jun 13, 2017 at 3:14 PM, Marston Johnston <shejo284 at gmail.com>
wrote:
> Here’s a NCL page dedicated to the handling of HDF files used by
> satellites such as MODIS.
>
>
>
> https://www.ncl.ucar.edu/Applications/HDF.shtml
>
>
>
> /M
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Marston S. Johnston, PhD
>
> Department of Earth Sciences
>
> University of Gothenburg, Sweden
>
> Email: marston.johnston at gu.se
>
> SkypeID: marston.johnston
>
> Phone: +46-31-7864901 <+46%2031%20786%2049%2001>
>
> Only the fruitful thing is true!
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
>
>
> *From: *Michael Notaro <mnotaro at wisc.edu>
> *Date: *Tuesday, 13 June 2017 at 22:46
> *To: *Marston Johnston <shejo284 at gmail.com>, "ncl-talk at ucar.edu" <
> ncl-talk at ucar.edu>
>
> *Subject: *Re: [ncl-talk] hdf confusion
>
>
>
> Thanks. I tried the following and only end up with a 2d grid of values
> like 2066, 18709, etc.
>
> How can I retrieve slot [11-13] land/water flag with values like 000, 001,
> 010, etc?
>
> Michael
>
>
>
> aa=addfile("h10v04_yellowstone/MOD13Q1.A2012001.
> h10v04.006.2015236172021.hdf.hdfeos","r")
>
> lat2d=doubletofloat(aa->GridLat_MODIS_Grid_16DAY_250m_500m_VI(::-1,:))
>
> lon2d=doubletofloat(aa->GridLon_MODIS_Grid_16DAY_250m_500m_VI(::-1,:))
>
> lat2d!0="nlat"
>
> lat2d!1="nlon"
>
> lon2d!0="nlat"
>
> lon2d!1="nlon"
>
> land=short2flt_hdf(aa->250m_16_days_VI_Quality_MODIS_Grid_
> 16DAY_250m_500m_VI)
>
> print(land)
>
>
>
>
>
> Michael Notaro
>
> Associate Director
>
> Nelson Institute Center for Climatic Research
>
> University of Wisconsin-Madison
>
> Phone: (608) 261-1503
>
> Email: mnotaro at wisc.edu
>
>
> ------------------------------
>
> *From:* Marston Johnston <shejo284 at gmail.com>
> *Sent:* Tuesday, June 13, 2017 3:37 PM
> *To:* Michael Notaro; ncl-talk at ucar.edu
> *Subject:* Re: [ncl-talk] hdf confusion
>
>
>
> Hi,
>
>
>
> Try this function: http://www.ncl.ucar.edu/Document/Functions/
> Contributed/short2flt_hdf.shtml
>
> short2flt_hdf - ncl.ucar.edu
> <http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt_hdf.shtml>
>
> www.ncl.ucar.edu
>
> NCL Home > Documentation > Functions > Type converters short2flt_hdf.
> Converts values of type short to values of type float using the "scale" and
> "offset" attributes ...
>
>
>
>
>
> /M
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Marston S. Johnston, PhD
>
> Department of Earth Sciences
>
> University of Gothenburg, Sweden
>
> Email: marston.johnston at gu.se
>
> SkypeID: marston.johnston
>
> Phone: +46-31-7864901 <+46%2031%20786%2049%2001>
>
> Only the fruitful thing is true!
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
>
>
> *From: *<ncl-talk-bounces at ucar.edu> on behalf of Michael Notaro <
> mnotaro at wisc.edu>
> *Date: *Tuesday, 13 June 2017 at 22:33
> *To: *"ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
> *Subject: *[ncl-talk] hdf confusion
>
>
>
> Sorry for the ignorant question but I am having trouble figuring out
>
> how to properly retrieve the land-lake-ocean mask off this MODIS hdf file.
>
> If I ncl_filedump *hdf, it shows the variable as ushort. If I
> ncl_filedump *hdf.hdfeos,
>
> it shows the variable as short. Which should I use in addfile and how do
> I
>
> retrieve the values of 001=land, 000=ocean, etc?
>
> Michael
>
>
>
>
>
> ncl_filedump h10v04_yellowstone/MOD13Q1.A2012001.h10v04.006.
> 2015236172021.hdf.hdfeos
>
>
>
> short 250m_16_days_VI_Quality_MODIS_Grid_16DAY_250m_500m_VI (
> YDim_MODIS_Grid_16DAY_250m_500m_VI, XDim_MODIS_Grid_16DAY_250m_500m_VI )
>
> coordinates :
>
> GridLat_MODIS_Grid_16DAY_250m_500m_VI, GridLon_MODIS_Grid_16DAY_250m_
> 500m_VI
>
> hdfeos_name :
>
> 250m 16 days VI Quality
>
> projection :
>
> Sinusoidal
>
> unsigned :
>
> True
>
> Legend :
>
> Bit Fields Description (Right to Left):
>
> [0-1] : MODLAND_QA [2 bit range]
>
> 00: VI produced, good quality
>
> 01: VI produced, but check other QA
>
> 10: Pixel produced, but most probably cloudy
>
> 11: Pixel not produced due to other reasons than clouds
>
> [2-5] : VI usefulness [4 bit range]
>
> 0000: Highest quality
>
> 0001: Lower quality
>
> 0010..1010: Decreasing quality
>
> 1100: Lowest quality
>
> 1101: Quality so low that it is not useful
>
> 1110: L1B data faulty
>
> 1111: Not useful for any other reason/not processed
>
> [6-7] : Aerosol quantity [2 bit range]
>
> 00: Climatology
>
> 01: Low
>
> 10: Average
>
> 11: High (11)
>
> [8] : Adjacent cloud detected; [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [9] : Atmosphere BRDF correction performed [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [10] : Mixed clouds [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [11-13] : Land/Water Flag [3 bit range]
>
> 000: Shallow ocean
>
> 001: Land (Nothing else but land)
>
> 010: Ocean coastlines and lake shorelines
>
> 011: Shallow inland water
>
> 100: Ephemeral water
>
> 101: Deep inland water
>
> 110: Moderate or continental ocean
>
> 111: Deep ocean
>
> [14] : Possible snow/ice [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [15] : Possible shadow [1 bit range]
>
> 1: Yes
>
> 0: No
>
> _FillValue :
>
> -1
>
> valid_range :
>
> ( 0, -2 )
>
> units :
>
> bit field
>
> long_name :
>
> 250m 16 days VI Quality
>
>
>
> ncl_filedump h10v04_yellowstone/MOD13Q1.A2012001.h10v04.006.
> 2015236172021.hdf
>
>
>
> ushort 250m_16_days_VI_Quality ( YDim_MODIS_Grid_16DAY_250m_500m_VI,
> XDim_MODIS_Grid_16DAY_250m_500m_VI )
>
> long_name :
>
> 250m 16 days VI Quality
>
> units :
>
> bit field
>
> valid_range :
>
> ( 0, 65534 )
>
> _FillValue :
>
> 65535
>
> Legend :
>
> Bit Fields Description (Right to Left):
>
> [0-1] : MODLAND_QA [2 bit range]
>
> 00: VI produced, good quality
>
> 01: VI produced, but check other QA
>
> 10: Pixel produced, but most probably cloudy
>
> 11: Pixel not produced due to other reasons than clouds
>
> [2-5] : VI usefulness [4 bit range]
>
> 0000: Highest quality
>
> 0001: Lower quality
>
> 0010..1010: Decreasing quality
>
> 1100: Lowest quality
>
> 1101: Quality so low that it is not useful
>
> 1110: L1B data faulty
>
> 1111: Not useful for any other reason/not processed
>
> [6-7] : Aerosol quantity [2 bit range]
>
> 00: Climatology
>
> 01: Low
>
> 10: Average
>
> 11: High (11)
>
> [8] : Adjacent cloud detected; [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [9] : Atmosphere BRDF correction performed [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [10] : Mixed clouds [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [11-13] : Land/Water Flag [3 bit range]
>
> 000: Shallow ocean
>
> 001: Land (Nothing else but land)
>
> 010: Ocean coastlines and lake shorelines
>
> 011: Shallow inland water
>
> 100: Ephemeral water
>
> 101: Deep inland water
>
> 110: Moderate or continental ocean
>
> 111: Deep ocean
>
> [14] : Possible snow/ice [1 bit range]
>
> 1: Yes
>
> 0: No
>
> [15] : Possible shadow [1 bit range]
>
> 1: Yes
>
> 0: No
>
> hdf_name :
>
> 250m 16 days VI Quality
>
>
>
>
>
> Michael Notaro
>
> Associate Director
>
> Nelson Institute Center for Climatic Research
>
> University of Wisconsin-Madison
>
> Phone: (608) 261-1503
>
> Email: mnotaro at wisc.edu
>
> _______________________________________________ ncl-talk mailing list
> ncl-talk at ucar.edu List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170613/f1cd6013/attachment.html
More information about the ncl-talk
mailing list