[Go-essp-tech] How does "experiment=volcanic aerosol" come to be in CMIP5?

Mike Berkley mike.berkley at gmail.com
Thu Feb 23 09:06:08 MST 2012 

Thanks, Bob.

If I understand this correctly, it means that unpublish/publish did
not fix historicalMISC before, because the database's experiment table
is not updated when esg.ini is changed.

Some of the ESG documentation implies that esg.ini is authoritative, e.g.

  "It is critical that the ESG publication client configuration
(esg.ini) be kept up to date
   with respect to the vocabulary, ..."
  http://esg-pcmdi.llnl.gov/internal/esg-data-node-documentation/cmip5-best-practices

In reality, it looks like it sometimes doesn't matter what values are
in esg.ini, because the values are already set in the database.




On Wed, Feb 22, 2012 at 18:43, Drach, Bob <drach1 at llnl.gov> wrote:
> Hi Mike,
>
> Estani's comment is essentially correct. Here are the steps to fix the
> description:
>
> (1) Fix the relevant database entry. In psql, connect to the esgcet
> database, then:
>
>>>> update experiment set description='historical simulation but with
>>>>other individual forcing agents or combinations of forcings' where
>>>>name='historicalMisc';
>
> (2) Make a list of the historicalMisc datasets to be modified. For example:
>
> % esglist_datasets --experiment historicalMisc --select name --no-header
> cmip5 > histmisc.txt
>
> (3) Unpublish and remove the THREDDS catalogs:
>
> % esgunpublish --use-list histmisc.txt
>
> (4) Republish:
>
> % esgpublish --use-list histmisc.txt --project cmip5
>
> Estani's suggestion is good: I would run (3-4) on one dataset first to
> make sure the fix succeeded.
>
> --Bob
>
> On 2/20/12 10:10 AM, "Estanislao Gonzalez" <gonzalez at dkrz.de> wrote:
>
>>That's Bob area...
>>
>>But I'm sure you'll have at least to recreate the catalogs and publish
>>again.
>>
>>Go step-by-tep. Just unpublish up to the catalog, generate the catalog
>>again and check if the description is ok. If it's not, you will have to
>>remove the data completely from the DB,
>>
>>Check step-by-step and with only one dataset. Once you know what's the
>>minimal set of changes required, you can correct all other datasets
>>faster.
>>
>>Take care though that if oyu have to delete the data from the DB you
>>will also loose the version number, so you will have to provide it again
>>afterwards.
>>Please make sure the version number match the one in the DRS structure!
>>
>>Thanks,
>>Estani
>>Am 20.02.2012 18:57, schrieb Mike Berkley:
>>> The description for historicalMisc is indeed "volcanic aerosol",
>>> instead of "other historical forcing".
>>>
>>> esgcet=>  select * from experiment where name='historicalMisc';
>>>        name      | project |   description
>>> ----------------+---------+------------------
>>>   historicalMisc | cmip5   | volcanic aerosol
>>> (1 row)
>>>
>>> If we update the database by hand, will that update THREDDS and the
>>> gateway, or will we need to publish/republish?
>>>
>>>
>>>
>>> On Sat, Feb 18, 2012 at 03:39, Estanislao Gonzalez<gonzalez at dkrz.de>
>>>wrote:
>>>> Hi,
>>>>
>>>> If the problem was a copy&  paste from the historicalVL experiment,
>>>>then you
>>>> have just to change the description of the HistoricalMisc. This is
>>>>done in
>>>> the esgcet DB, in the table experiment.
>>>>
>>>> But we have a different description for it:
>>>> esgcet=# SELECT * from experiment where name='historicalVL';
>>>>      name     | project |   description
>>>> --------------+---------+------------------
>>>>   historicalVL | cmip5   | volcanic aerosol
>>>> (1 row)
>>>>
>>>> esgcet=# SELECT * from experiment where name='historicalMisc';
>>>>       name      | project |
>>>>   description
>>>>
>>>>----------------+---------+---------------------------------------------
>>>>-----------------------------------------------
>>>>   historicalMisc | cmip5   | historical simulation but with other
>>>>individual
>>>> forcing agents or combinations of forcings
>>>> (1 row)
>>>>
>>>> If you have the same description as we do, then the information is
>>>>coming
>>>> from somewhere else.
>>>>
>>>> Hope this helps,
>>>> Estani
>>>>
>>>> Am 17.02.2012 23:44, schrieb Mike Berkley:
>>>>
>>>>> Probably, we should have corrected esg.ini before publishing, but we
>>>>> didn't realize that it had this effect.
>>>>>
>>>>> It would help if we knew how to correct the error.  The "volcanic
>>>>> aerosol" description does not appear in the map file, and
>>>>> esgupdate_metadata didn't think "experiment_options" was a property to
>>>>> adjust.  Unpublish and republish doesn't resolve the problem, even
>>>>> with a database delete.  I'm not sure what command to use to update
>>>>> the description in local THREDDS and the gateway.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Feb 17, 2012 at 13:04, Karl Taylor<taylor13 at llnl.gov>
>>>>>wrote:
>>>>>> Hi Mike,
>>>>>>
>>>>>> Do you want us to do something on our end?
>>>>>>
>>>>>> I'm not that familiar with esg.ini.  For anyone who publishes an
>>>>>> historicalMisc dataset, will the experiment description that appears
>>>>>>in
>>>>>>   ESG
>>>>>> be "volcanic aerosol"?  If so, this is generally wrong, so what
>>>>>>should be
>>>>>> done to fix this?
>>>>>>
>>>>>> best regards,
>>>>>> Karl
>>>>>>
>>>>>> On 2/17/12 10:54 AM, Mike Berkley wrote:
>>>>>>
>>>>>> The "volcanic aerosol" description was a  surprise to CCCma, since it
>>>>>> appeared nowhere in our netCDF files.  We finally found "volcanic
>>>>>> aerosol" associated with historicalMisc in the default data node
>>>>>> esg.ini file from PCMDI.  We have corrected our copy of esg.ini, but
>>>>>> have so far not found the magic incantation to update the metadata
>>>>>> description.  Unpublishing and republishing does not change the
>>>>>> metadata, presumably because the data hasn't actually changed.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 16, 2012 at 12:21, Karl Taylor<taylor13 at llnl.gov>
>>>>>>wrote:
>>>>>>
>>>>>> Hi Gary,
>>>>>>
>>>>>> To be sure, the experiment_id for the CCCMA dataset shown below was
>>>>>>(if
>>>>>> correct) "historicalMisc".  When they published it, they chose to
>>>>>>name
>>>>>> the
>>>>>> experiment "volcanic aerosol".  This is informational but is not
>>>>>>used by
>>>>>> the
>>>>>> ESG search engines or in any other automatic processing that I know
>>>>>>of.
>>>>>>   (I
>>>>>> thought whoever was publishing would just put in
>>>>>> experiment="historicalMisc", so it would be consistent with the
>>>>>> experiment_id global attribute.)
>>>>>>
>>>>>> I don't think we've provided much guidance about this (outside the
>>>>>>DRS
>>>>>> document), but for each different combination of forcings in the
>>>>>> historicalMisc runs, you should assign a different "physics_version"
>>>>>> number
>>>>>> (i.e., the "p" in the ensemble identifier, which in the example
>>>>>>below is
>>>>>> "p3").  You should also record in the global "forcing" attribute in
>>>>>>the
>>>>>> netCDF file each of the forcings included in each of your
>>>>>> historical(Misc)
>>>>>> runs.  If you do an ensemble of runs with the same forcing, they
>>>>>>should
>>>>>> all
>>>>>> have the same "p" value and be distinguished by the "r" values.  The
>>>>>> standard abbreviations for each forcing are provided in the last
>>>>>>appendix
>>>>>> of
>>>>>> the DRS document and also can be found at
>>>>>>
>>>>>>
>>>>>>http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_Appendix1-2
>>>>>>.pdf
>>>>>> [Note that the output_requirements, available at
>>>>>>
>>>>>>
>>>>>>http://cmip-pcmdi.llnl.gov/cmip5/docs/CMIP5_output_metadata_requiremen
>>>>>>ts.pdf
>>>>>> , describes the "forcing" attribute.]
>>>>>>
>>>>>> I would like to put together a table that provides a key showing
>>>>>>which
>>>>>> forcings are included for each model and for each value of "p"; I
>>>>>>just
>>>>>> haven't gotten to this yet.
>>>>>>
>>>>>> Please let me know if you can think of any way to make this clearer
>>>>>>to
>>>>>> everyone.
>>>>>>
>>>>>> Best regards,
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2/16/12 7:31 AM, Gary Strand wrote:
>>>>>>
>>>>>>
>>>>>> At what point (publishing?) does one of an "historicalMisc"
>>>>>>experiment
>>>>>> (as
>>>>>> defined and required by the CMIP5 tables) get displayed as:
>>>>>>
>>>>>>
>>>>>>
>>>>>> I would like our (NCAR) "historicalMisc" datasets have a bit more
>>>>>> information available than just the standard (as described in the DRS
>>>>>> document) "historical simulation but with other individual forcing
>>>>>>agents
>>>>>> or
>>>>>> combinations of forcings."
>>>>>>
>>>>>> We have a whole slew of variations that are not readily
>>>>>>distinguishable
>>>>>> (and
>>>>>> less likely usable) by the above generic description.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> Gary Strand
>>>>>> strandwg at ucar.edu
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>
>>>>> _______________________________________________
>>>>> GO-ESSP-TECH mailing list
>>>>> GO-ESSP-TECH at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>
>>>>
>>>> --
>>>> Estanislao Gonzalez
>>>>
>>>> Max-Planck-Institut für Meteorologie (MPI-M)
>>>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>>>
>>>> Phone:   +49 (40) 46 00 94-126
>>>> E-Mail:  gonzalez at dkrz.de
>>>>
>>
>>
>>--
>>Estanislao Gonzalez
>>
>>Max-Planck-Institut für Meteorologie (MPI-M)
>>Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>
>>Phone:   +49 (40) 46 00 94-126
>>E-Mail:  gonzalez at dkrz.de
>>
>

More information about the GO-ESSP-TECH mailing list