[Go-essp-tech] How does "experiment=volcanic aerosol" come to be in CMIP5?
Drach, Bob
drach1 at llnl.gov
Thu Feb 23 12:33:15 MST 2012
Hi Mike,
Currently the model description is initialized into the database from the
values in esg.ini. Subsequently the 'model' database table is updated when
a new model entry is added to esg.ini, but not when an existing entry (in
esg.ini) is modified. I agree that this is unintuitive - I'll look at ways
to provide a simpler way of correcting model metadata, and include this in
an upcoming release of the publication module.
Regards,
--Bob
On 2/23/12 8:06 AM, "Mike Berkley" <mike.berkley at gmail.com> wrote:
>Thanks, Bob.
>
>If I understand this correctly, it means that unpublish/publish did
>not fix historicalMISC before, because the database's experiment table
>is not updated when esg.ini is changed.
>
>Some of the ESG documentation implies that esg.ini is authoritative, e.g.
>
> "It is critical that the ESG publication client configuration
>(esg.ini) be kept up to date
> with respect to the vocabulary, ..."
>
>http://esg-pcmdi.llnl.gov/internal/esg-data-node-documentation/cmip5-best-
>practices
>
>In reality, it looks like it sometimes doesn't matter what values are
>in esg.ini, because the values are already set in the database.
>
>
>
>
>On Wed, Feb 22, 2012 at 18:43, Drach, Bob <drach1 at llnl.gov> wrote:
>> Hi Mike,
>>
>> Estani's comment is essentially correct. Here are the steps to fix the
>> description:
>>
>> (1) Fix the relevant database entry. In psql, connect to the esgcet
>> database, then:
>>
>>>>> update experiment set description='historical simulation but with
>>>>>other individual forcing agents or combinations of forcings' where
>>>>>name='historicalMisc';
>>
>> (2) Make a list of the historicalMisc datasets to be modified. For
>>example:
>>
>> % esglist_datasets --experiment historicalMisc --select name --no-header
>> cmip5 > histmisc.txt
>>
>> (3) Unpublish and remove the THREDDS catalogs:
>>
>> % esgunpublish --use-list histmisc.txt
>>
>> (4) Republish:
>>
>> % esgpublish --use-list histmisc.txt --project cmip5
>>
>> Estani's suggestion is good: I would run (3-4) on one dataset first to
>> make sure the fix succeeded.
>>
>> --Bob
>>
>> On 2/20/12 10:10 AM, "Estanislao Gonzalez" <gonzalez at dkrz.de> wrote:
>>
>>>That's Bob area...
>>>
>>>But I'm sure you'll have at least to recreate the catalogs and publish
>>>again.
>>>
>>>Go step-by-tep. Just unpublish up to the catalog, generate the catalog
>>>again and check if the description is ok. If it's not, you will have to
>>>remove the data completely from the DB,
>>>
>>>Check step-by-step and with only one dataset. Once you know what's the
>>>minimal set of changes required, you can correct all other datasets
>>>faster.
>>>
>>>Take care though that if oyu have to delete the data from the DB you
>>>will also loose the version number, so you will have to provide it again
>>>afterwards.
>>>Please make sure the version number match the one in the DRS structure!
>>>
>>>Thanks,
>>>Estani
>>>Am 20.02.2012 18:57, schrieb Mike Berkley:
>>>> The description for historicalMisc is indeed "volcanic aerosol",
>>>> instead of "other historical forcing".
>>>>
>>>> esgcet=> select * from experiment where name='historicalMisc';
>>>> name | project | description
>>>> ----------------+---------+------------------
>>>> historicalMisc | cmip5 | volcanic aerosol
>>>> (1 row)
>>>>
>>>> If we update the database by hand, will that update THREDDS and the
>>>> gateway, or will we need to publish/republish?
>>>>
>>>>
>>>>
>>>> On Sat, Feb 18, 2012 at 03:39, Estanislao Gonzalez<gonzalez at dkrz.de>
>>>>wrote:
>>>>> Hi,
>>>>>
>>>>> If the problem was a copy& paste from the historicalVL experiment,
>>>>>then you
>>>>> have just to change the description of the HistoricalMisc. This is
>>>>>done in
>>>>> the esgcet DB, in the table experiment.
>>>>>
>>>>> But we have a different description for it:
>>>>> esgcet=# SELECT * from experiment where name='historicalVL';
>>>>> name | project | description
>>>>> --------------+---------+------------------
>>>>> historicalVL | cmip5 | volcanic aerosol
>>>>> (1 row)
>>>>>
>>>>> esgcet=# SELECT * from experiment where name='historicalMisc';
>>>>> name | project |
>>>>> description
>>>>>
>>>>>----------------+---------+-------------------------------------------
>>>>>--
>>>>>-----------------------------------------------
>>>>> historicalMisc | cmip5 | historical simulation but with other
>>>>>individual
>>>>> forcing agents or combinations of forcings
>>>>> (1 row)
>>>>>
>>>>> If you have the same description as we do, then the information is
>>>>>coming
>>>>> from somewhere else.
>>>>>
>>>>> Hope this helps,
>>>>> Estani
>>>>>
>>>>> Am 17.02.2012 23:44, schrieb Mike Berkley:
>>>>>
>>>>>> Probably, we should have corrected esg.ini before publishing, but we
>>>>>> didn't realize that it had this effect.
>>>>>>
>>>>>> It would help if we knew how to correct the error. The "volcanic
>>>>>> aerosol" description does not appear in the map file, and
>>>>>> esgupdate_metadata didn't think "experiment_options" was a property
>>>>>>to
>>>>>> adjust. Unpublish and republish doesn't resolve the problem, even
>>>>>> with a database delete. I'm not sure what command to use to update
>>>>>> the description in local THREDDS and the gateway.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 17, 2012 at 13:04, Karl Taylor<taylor13 at llnl.gov>
>>>>>>wrote:
>>>>>>> Hi Mike,
>>>>>>>
>>>>>>> Do you want us to do something on our end?
>>>>>>>
>>>>>>> I'm not that familiar with esg.ini. For anyone who publishes an
>>>>>>> historicalMisc dataset, will the experiment description that
>>>>>>>appears
>>>>>>>in
>>>>>>> ESG
>>>>>>> be "volcanic aerosol"? If so, this is generally wrong, so what
>>>>>>>should be
>>>>>>> done to fix this?
>>>>>>>
>>>>>>> best regards,
>>>>>>> Karl
>>>>>>>
>>>>>>> On 2/17/12 10:54 AM, Mike Berkley wrote:
>>>>>>>
>>>>>>> The "volcanic aerosol" description was a surprise to CCCma, since
>>>>>>>it
>>>>>>> appeared nowhere in our netCDF files. We finally found "volcanic
>>>>>>> aerosol" associated with historicalMisc in the default data node
>>>>>>> esg.ini file from PCMDI. We have corrected our copy of esg.ini,
>>>>>>>but
>>>>>>> have so far not found the magic incantation to update the metadata
>>>>>>> description. Unpublishing and republishing does not change the
>>>>>>> metadata, presumably because the data hasn't actually changed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Feb 16, 2012 at 12:21, Karl Taylor<taylor13 at llnl.gov>
>>>>>>>wrote:
>>>>>>>
>>>>>>> Hi Gary,
>>>>>>>
>>>>>>> To be sure, the experiment_id for the CCCMA dataset shown below was
>>>>>>>(if
>>>>>>> correct) "historicalMisc". When they published it, they chose to
>>>>>>>name
>>>>>>> the
>>>>>>> experiment "volcanic aerosol". This is informational but is not
>>>>>>>used by
>>>>>>> the
>>>>>>> ESG search engines or in any other automatic processing that I know
>>>>>>>of.
>>>>>>> (I
>>>>>>> thought whoever was publishing would just put in
>>>>>>> experiment="historicalMisc", so it would be consistent with the
>>>>>>> experiment_id global attribute.)
>>>>>>>
>>>>>>> I don't think we've provided much guidance about this (outside the
>>>>>>>DRS
>>>>>>> document), but for each different combination of forcings in the
>>>>>>> historicalMisc runs, you should assign a different
>>>>>>>"physics_version"
>>>>>>> number
>>>>>>> (i.e., the "p" in the ensemble identifier, which in the example
>>>>>>>below is
>>>>>>> "p3"). You should also record in the global "forcing" attribute in
>>>>>>>the
>>>>>>> netCDF file each of the forcings included in each of your
>>>>>>> historical(Misc)
>>>>>>> runs. If you do an ensemble of runs with the same forcing, they
>>>>>>>should
>>>>>>> all
>>>>>>> have the same "p" value and be distinguished by the "r" values.
>>>>>>>The
>>>>>>> standard abbreviations for each forcing are provided in the last
>>>>>>>appendix
>>>>>>> of
>>>>>>> the DRS document and also can be found at
>>>>>>>
>>>>>>>
>>>>>>>http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_Appendix1
>>>>>>>-2
>>>>>>>.pdf
>>>>>>> [Note that the output_requirements, available at
>>>>>>>
>>>>>>>
>>>>>>>http://cmip-pcmdi.llnl.gov/cmip5/docs/CMIP5_output_metadata_requirem
>>>>>>>en
>>>>>>>ts.pdf
>>>>>>> , describes the "forcing" attribute.]
>>>>>>>
>>>>>>> I would like to put together a table that provides a key showing
>>>>>>>which
>>>>>>> forcings are included for each model and for each value of "p"; I
>>>>>>>just
>>>>>>> haven't gotten to this yet.
>>>>>>>
>>>>>>> Please let me know if you can think of any way to make this clearer
>>>>>>>to
>>>>>>> everyone.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2/16/12 7:31 AM, Gary Strand wrote:
>>>>>>>
>>>>>>>
>>>>>>> At what point (publishing?) does one of an "historicalMisc"
>>>>>>>experiment
>>>>>>> (as
>>>>>>> defined and required by the CMIP5 tables) get displayed as:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I would like our (NCAR) "historicalMisc" datasets have a bit more
>>>>>>> information available than just the standard (as described in the
>>>>>>>DRS
>>>>>>> document) "historical simulation but with other individual forcing
>>>>>>>agents
>>>>>>> or
>>>>>>> combinations of forcings."
>>>>>>>
>>>>>>> We have a whole slew of variations that are not readily
>>>>>>>distinguishable
>>>>>>> (and
>>>>>>> less likely usable) by the above generic description.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> Gary Strand
>>>>>>> strandwg at ucar.edu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> GO-ESSP-TECH mailing list
>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> GO-ESSP-TECH mailing list
>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>>
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>
>>>>>
>>>>> --
>>>>> Estanislao Gonzalez
>>>>>
>>>>> Max-Planck-Institut für Meteorologie (MPI-M)
>>>>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>>>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>>>>
>>>>> Phone: +49 (40) 46 00 94-126
>>>>> E-Mail: gonzalez at dkrz.de
>>>>>
>>>
>>>
>>>--
>>>Estanislao Gonzalez
>>>
>>>Max-Planck-Institut für Meteorologie (MPI-M)
>>>Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>>Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>>
>>>Phone: +49 (40) 46 00 94-126
>>>E-Mail: gonzalez at dkrz.de
>>>
>>
More information about the GO-ESSP-TECH
mailing list