[Go-essp-tech] NetCDF metadata in THREDDS catalogues
Bryan Lawrence
bryan.lawrence at stfc.ac.uk
Wed Nov 4 13:55:06 MST 2009
Hi Bob
thanks for this, so this means the gateway can only know this stuff and no more.
There is a considerable amount of "cmip5" metadata prescribed in the global
file attributes. Is that propagated to the gateway?
Where then is the tracking id that is in each file? What service would expose that?
I have understood that a user could always go from the tracking identifier
in the file to find the appropriate metadata (e.g. which version it is).
> The database does contain all the metadata in the files, with the
> possible exception of coordinate data if you consider that as metadata.
> However coordinate ranges for aggregate variables are kept in the
> variable table. The postgres database would be the logical place to
> obtain the desired information.
Will it be possible to access the postgres database on a data node from
outside the data node (I would assume not following our earlier conversation
about what is exposed, but I want to be sure :-)
(I guess the gateway can also query the services which expose more infomation
about the data, but to do that, the data node would actually have to read
the data ...)
Bryan
>
> Some of the table definitions will change in the next version, but I
> would expect the tables containing attribute and dimension information
> to remain stable.
>
> Bob
> > For instance:
> >
> > esgcet=> select name, value from file_var_attr limit 20;
> > name |
> > value
> > ---------------+------------------------------------------------------------------
> > axis | Y
> > bounds | lat_bnds
> > getitem | <built-in method getitem of CdunifVariable object at
> > 0x19f8150>
> > getslice | <built-in method getslice of CdunifVariable object at
> > 0x19f8150>
> > long_name | latitude
> > setitem | <built-in method setitem of CdunifVariable object at
> > 0x19f8150>
> > setslice | <built-in method setslice of CdunifVariable object at
> > 0x19f8150>
> > standard_name | latitude
> > units | degrees_north
> > getitem | <built-in method getitem of CdunifVariable object at
> > 0x19f8030>
> > getslice | <built-in method getslice of CdunifVariable object at
> > 0x19f8030>
> > setitem | <built-in method setitem of CdunifVariable object at
> > 0x19f8030>
> > setslice | <built-in method setslice of CdunifVariable object at
> > 0x19f8030>
> > axis | X
> > bounds | lon_bnds
> > getitem | <built-in method getitem of CdunifVariable object at
> > 0x19f8108>
> > getslice | <built-in method getslice of CdunifVariable object at
> > 0x19f8108>
> > long_name | longitude
> > setitem | <built-in method setitem of CdunifVariable object at
> > 0x19f8108>
> > setslice | <built-in method setslice of CdunifVariable object at
> > 0x19f8108>
> > (20 rows)
> >
> > ---
> > Stephen Pascoe +44 (0)1235 445980
> > British Atmospheric Data Centre
> > Rutherford Appleton Laboratory
> >
> >
> > --
> > Scanned by iCritical.
> >
> >
> > ------------------------------------------------------------------------
> >
> > <?xml version='1.0' encoding='UTF-8'?>
> > <catalog xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0" name="TDS configuration file" xsi:schemaLocation="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0 http://www.unidata.ucar.edu/schemas/thredds/InvCatalog.1.0.2.xsd">
> > <service serviceType="OpenDAP" base="/thredds/dodsC/" name="gridded"/>
> > <service serviceType="HTTPServer" base="/thredds/fileServer/" name="HTTPServer"/>
> > <service serviceType="SRM" base="srm://host.sample.gov:6288/srm/v2/server?SFN=/archive.sample.gov/" name="HRMatPCMDI"/>
> > <dataset restrictAccess="esg-user" ID="cmip3" name="cmip3">
> > <property name="project" value="test"/>
> > <documentation type="summary"/>
> > <metadata inherited="true">
> > <variables vocabulary="CF-1.0">
> > <variable name="clivi" vocabulary_name="atmosphere_cloud_ice_content" units="kg m-2">Column Integerated Cloud Ice Content</variable>
> > <variable name="clwvi" vocabulary_name="atmosphere_cloud_condensed_water_content" units="kg m-2">Column Integerated Cloud Water Content</variable>
> > <variable name="mrsos" vocabulary_name="moisture_content_of_soil_layer" units="kg m-2">Moisture in Upper 0.1 m of Soil Column</variable>
> > <variable name="prsn" vocabulary_name="snowfall_flux" units="kg m-2 s-1">Snowfall Flux</variable>
> > <variable name="rlut" vocabulary_name="toa_outgoing_longwave_flux" units="W m-2">Outgoing Longwave Radiation</variable>
> > <variable name="rsftoai_co2" vocabulary_name="toa_instantaneous_shortwave_forcing" units="W m-2">Instantaneous Top of the Atmosphere Shortwave Forcing Due to Anthropogenic Carbon Dioxide</variable>
> > <variable name="rsftoaics_co2" vocabulary_name="Instantaneous Top of the Atmosphere Shortwave Clear-Sky Forcing Due to Anthropogenic Carbon Dioxide" units="W m-2">Instantaneous Top of the Atmosphere Shortwave Clear-Sky Forcing Due to Anthropogenic Carbon Dioxide</variable>
> > <variable name="rsftropacs_co2" vocabulary_name="Adjusted Tropopause Shortwave Clear-Sky Forcing Due to Anthropogenic Carbon Dioxide" units="W m-2">Adjusted Tropopause Shortwave Clear-Sky Forcing Due to Anthropogenic Carbon Dioxide</variable>
> > <variable name="rsntp" vocabulary_name="net_downward_shortwave_flux_in_air" units="W m-2">Net Downward Shortwave Flux at 200 hPa</variable>
> > <variable name="rsuscs" vocabulary_name="surface_upwelling_shortwave_flux_in_air_assuming_clear_sky" units="W m-2">Reflected Surface Clear-Sky Shortwave</variable>
> > <variable name="rsut" vocabulary_name="toa_outgoing_shortwave_flux" units="W m-2">TOA Reflected Shortwave Radiation</variable>
> > <variable name="tauv" vocabulary_name="surface_downward_northward_stress" units="Pa">Meridional Surface Wind Stress</variable>
> > <variable name="ts" vocabulary_name="surface_temperature" units="K">Surface Skin Temperature</variable>
> > <variable name="uas" vocabulary_name="eastward_wind" units="m s-1">Zonal Surface Wind Speed</variable>
> > <variable name="va" vocabulary_name="northward_wind" units="m s-1">Meridional Wind Component</variable>
> > <variable name="sic" vocabulary_name="sea_ice_area_fraction" units="%">Sea Ice Concentration</variable>
> > <variable name="orog" vocabulary_name="surface_altitude" units="m">Surface Altitude</variable>
> > <variable name="sftlf" vocabulary_name="land_area_fraction" units="%">Land Area Fraction</variable>
> > <variable name="mrro" vocabulary_name="runoff_flux" units="kg m-2 s-1">Surface and Subsurface Runoff</variable>
> > <variable name="mrros" vocabulary_name="surface_runoff_flux" units="kg m-2 s-1">Surface Runoff</variable>
> > <variable name="mrso" vocabulary_name="soil_moisture_content" units="kg m-2">Total Soil Moisture Content</variable>
> > <variable name="snm" vocabulary_name="surface_snow_melt_flux_where_land" units="kg m-2 s-1">Snow Melt</variable>
> > <variable name="snw" vocabulary_name="surface_snow_amount_where_land" units="kg m-2">Snow Amount</variable>
> > <variable name="zobt" vocabulary_name="sea_floor_depth_below_geoid" units="m">Sea Floor Depth</variable>
> > <variable name="tos" vocabulary_name="sea_surface_temperature" units="K">Sea Surface Temperature</variable>
> > <variable name="usi" vocabulary_name="eastward_sea_ice_velocity" units="m s-1">Eastward Sea Ice Velocity</variable>
> > <variable name="vsi" vocabulary_name="northward_sea_ice_velocity" units="m s-1">Northward Sea Ice Velocity</variable>
> > </variables>
> > </metadata>
> > <metadata inherited="true">
> > <dataType>Grid</dataType>
> > <dataFormat>NetCDF</dataFormat>
> > </metadata>
> > <dataset ID="cmip3.files" name="cmip3.files">
> > <dataset name="clivi_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.clivi_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685040</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/clivi/ukmo_hadgem1/run1/clivi_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="clwvi_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.clwvi_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685056</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/clwvi/ukmo_hadgem1/run1/clwvi_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="mrsos_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.mrsos_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685352</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/mrsos/ukmo_hadgem1/run1/mrsos_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="prsn_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.prsn_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685008</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/prsn/ukmo_hadgem1/run1/prsn_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rlut_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.rlut_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685028</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rlut/ukmo_hadgem1/run1/rlut_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftoai_co2_A5.nc" ID="cmip3.files.rsftoai_co2_A5.nc">
> > <dataSize units="bytes">680224</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftoai_co2/ukmo_hadcm3/run1/rsftoai_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftoai_co2_A5.nc" ID="cmip3.files.rsftoai_co2_A5.nc">
> > <dataSize units="bytes">2684460</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftoai_co2/ukmo_hadgem1/run1/rsftoai_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftoaics_co2_A5.nc" ID="cmip3.files.rsftoaics_co2_A5.nc">
> > <dataSize units="bytes">680256</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftoaics_co2/ukmo_hadcm3/run1/rsftoaics_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftoaics_co2_A5.nc" ID="cmip3.files.rsftoaics_co2_A5.nc">
> > <dataSize units="bytes">2684496</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftoaics_co2/ukmo_hadgem1/run1/rsftoaics_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftropacs_co2_A5.nc" ID="cmip3.files.rsftropacs_co2_A5.nc">
> > <dataSize units="bytes">680240</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftropacs_co2/ukmo_hadcm3/run1/rsftropacs_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsftropacs_co2_A5.nc" ID="cmip3.files.rsftropacs_co2_A5.nc">
> > <dataSize units="bytes">2684480</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsftropacs_co2/ukmo_hadgem1/run1/rsftropacs_co2_A5.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsntp_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.rsntp_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685260</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsntp/ukmo_hadgem1/run1/rsntp_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsuscs_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.rsuscs_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685076</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsuscs/ukmo_hadgem1/run1/rsuscs_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="rsut_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.rsut_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685036</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/rsut/ukmo_hadgem1/run1/rsut_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="tauv_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.tauv_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93039124</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/tauv/ukmo_hadgem1/run1/tauv_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="ts_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.ts_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685012</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/ts/ukmo_hadgem1/run1/ts_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="uas_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.uas_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93039312</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/uas/ukmo_hadgem1/run1/uas_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="va_A1_2006_Dec_to_2039_Dec.nc" ID="cmip3.files.va_A1_2006_Dec_to_2039_Dec.nc">
> > <dataSize units="bytes">702501312</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/va/ukmo_hadgem1/run1/va_A1_2006_Dec_to_2039_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="va_A1_2040_Jan_to_2076_Dec.nc" ID="cmip3.files.va_A1_2040_Jan_to_2076_Dec.nc">
> > <dataSize units="bytes">785667624</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/atm/mo/va/ukmo_hadgem1/run1/va_A1_2040_Jan_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="sic_O1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.sic_O1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685016</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/ice/mo/sic/ukmo_hadgem1/run1/sic_O1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="orog_A1.nc" ID="cmip3.files.orog_A1.nc">
> > <dataSize units="bytes">122080</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/fixed/orog/ukmo_hadgem1/run1/orog_A1.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="sftlf_A1.nc" ID="cmip3.files.sftlf_A1.nc">
> > <dataSize units="bytes">122068</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/fixed/sftlf/ukmo_hadgem1/run1/sftlf_A1.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="mrro_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.mrro_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685020</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/mo/mrro/ukmo_hadgem1/run1/mrro_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="mrros_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.mrros_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685016</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/mo/mrros/ukmo_hadgem1/run1/mrros_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="mrso_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.mrso_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685024</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/mo/mrso/ukmo_hadgem1/run1/mrso_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="snm_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.snm_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685024</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/mo/snm/ukmo_hadgem1/run1/snm_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="snw_A1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.snw_A1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685016</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/land/mo/snw/ukmo_hadgem1/run1/snw_A1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="zobt_O1.nc" ID="cmip3.files.zobt_O1.nc">
> > <dataSize units="bytes">327560</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/ocn/fixed/zobt/ukmo_hadgem1/run1/zobt_O1.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="tos_O1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.tos_O1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685016</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/ocn/mo/tos/ukmo_hadgem1/run1/tos_O1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="usi_O1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.usi_O1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93685028</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/ocn/mo/usi/ukmo_hadgem1/run1/usi_O1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > <dataset name="vsi_O1_2006_Dec_to_2076_Dec.nc" ID="cmip3.files.vsi_O1_2006_Dec_to_2076_Dec.nc">
> > <dataSize units="bytes">93039116</dataSize>
> > <access urlPath="cmip3_datadir/2xco2/ocn/mo/vsi/ukmo_hadgem1/run1/vsi_O1_2006_Dec_to_2076_Dec.nc" serviceName="HTTPServer"/>
> > </dataset>
> > </dataset>
> > </dataset>
> > </catalog>
> >
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
More information about the GO-ESSP-TECH
mailing list