[Go-essp-tech] output1 vs. output2
Kettleborough, Jamie
jamie.kettleborough at metoffice.gov.uk
Wed Feb 29 03:09:34 MST 2012
Hello Karl,
I think its an error if someone tries to publish an unknown diagnostic
with product=output. *But* do we know the scale of the problem of data
already in the system? How much data has the wrong product assigned
(either in the DRS id, or in the files netCDF attribute)? Is there an
expectation that this data will be reassigned to the correct product?
Or will we just live with it? If its sufficiently large volume/number
of files *and* we are going to 'live with' the legacy problem then I
don't see a strong reason to fix new data - users already have to deal
with the wrong products. (but happy to be wrong on this)
Jamie
________________________________
From: go-essp-tech-bounces at ucar.edu
[mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Karl Taylor
Sent: 28 February 2012 16:51
To: stephen.pascoe at stfc.ac.uk
Cc: Drach, Bob; go-essp-tech at ucar.edu
Subject: Re: [Go-essp-tech] output1 vs. output2
Dear all,
thanks for identifying the likely bug.
While fixing this problem, we should check whether the publisher
correctly treats product="unsolicited" data sets correctly. Is there an
"if test" for this case? What should we do with data where product as
been set by the data writer to "output", but "the product cannot be
determined" by the publisher? Should we set product to "output" as the
publisher apparently now does, or should be throw an error, or should be
set it to "unsolicited"?
Please vote. (and give reasons for your view).
thanks,
Karl
On 2/28/12 3:14 AM, stephen.pascoe at stfc.ac.uk wrote:
Karl, Bob,
I think this is a bug in ESG publisher. I've just taken
a look at the code and it converts the variable to lower-case before
looking it up in an internal table, however the internal table has
"sfcWind" so it never finds the variable. Other variables that may be
affected are tasAdjust, tsAdjust and parasolRefl
### In esgcet.config.cmip5_product
def getProduct(cmor_table, variable, experiment, year1,
year2):
"""Get the DRS product value associated with the
file.
Returns
'output1' for datasets to be replicated,
'output2' for datasets outside the replicated
datasets,
'output' if the product cannot be determined.
"""
cmor_table = cmor_table.lower()
variable = variable.lower()
### In esgcet.config.cmip5_tables
# cmor_variables : cmor_table => { variable =>
(priority, dimension_names)}
cmor_variables = { '3hr': { 'clt': (1, ['longitude',
'latitude', 'time']),
# ...
'amon': { 'ccb': (1, ['longitude', 'latitude',
'time']),
# ...
'sfcWind': (1, ['longitude', 'latitude',
'time', 'height10m']),
'ta': (1, ['longitude', 'latitude',
'plevs', 'time']),
'tas': (1, ['longitude', 'latitude',
'time', 'height2m']),
'tasAdjust': (1, ['longitude',
'latitude', 'time', 'height2m']),
Cheers,
Stephen
---
Stephen Pascoe +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford,
Didcot OX11 0QX, UK
From: go-essp-tech-bounces at ucar.edu
[mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Karl Taylor
Sent: 28 February 2012 00:52
To: go-essp-tech at ucar.edu
Subject: [Go-essp-tech] output1 vs. output2
Hi all,
I have noticed (using the new p2p search engine, which
works much faster, and I'm pretty sure is accurate) that
sfcWind_Amon_historical output is designated as "output 1" for some
models (from IPSL, MOHC, MPI-M, and NOAA-GFDL), but "output2" for the
rest. I think "output1" is correct.
Can anyone explain why the publisher correctly
characterized the output from some groups, but not for the rest?
thanks,
Karl
--
Scanned by iCritical.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120229/c6e00a74/attachment-0001.html
More information about the GO-ESSP-TECH
mailing list