[Go-essp-tech] GO-ESSP-TECH Digest, Vol 29, Issue 56
mark collier
mark.collier at csiro.au
Tue Jul 26 16:25:52 MDT 2011
Hi Karl,
On 26/07/2011, at 10:52 AM, go-essp-tech-request at ucar.edu wrote:
> Send GO-ESSP-TECH mailing list submissions to
> go-essp-tech at ucar.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> or, via email, send a message with subject or body 'help' to
> go-essp-tech-request at ucar.edu
>
> You can reach the person managing the list at
> go-essp-tech-owner at ucar.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of GO-ESSP-TECH digest..."
>
>
> Today's Topics:
>
> 1. Re: ESGF Telco This Week 7-26-2011! (Nathan Wilhelmi)
> 2. Re: ESGF Telco This Week 7-26-2011! (Cinquini, Luca (3880))
> 3. Re: ESGF Telco This Week 7-26-2011! (Nathan Wilhelmi)
> 4. Re: discrepancies among gateways (Karl Taylor)
> 5. model names (Karl Taylor)
> 6. Re: discrepancies among gateways (Muhammad Atif)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 25 Jul 2011 14:21:41 -0600
> From: Nathan Wilhelmi <wilhelmi at ucar.edu>
> Subject: Re: [Go-essp-tech] ESGF Telco This Week 7-26-2011!
> To: "Williams, Dean N." <williams13 at llnl.gov>
> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
> Message-ID: <4E2DD055.10706 at ucar.edu>
> Content-Type: text/plain; charset=windows-1252; format=flowed
>
> Hi Dean,
>
> A couple of items I would like to raise:
>
> 1. What is the status of getting the P2P registry protocol documented?
>
> 2. I would like to start the discussion about the protocol we use for
> metadata federation. Like the other API's this will need to be
> reviewed
> and documented. This has come up as out of scope for the search group.
> Is there another logical API group for this or does one need to be
> created? If needed I will volunteer to work on getting this going.
>
> Thanks!
>
> -Nate
>
> Williams, Dean N. wrote:
>> Dear Colleagues,
>>
>> We we have our regularly scheduled telco this week. The dial-in
>> number is: (925) 424-8105 access code 305757#
>>
>> Agenda for this week?s meeting:
>>
>> 1. ESGF/CMIP5 help desk: (getting more of the team involved, FAQ,
>> feedback, who?s handling issues, how can we tell if an issue has
>> been resolved, etc.)
>> 2. Improving federation stability
>> 3. Interface group discussion (possibility discussion topic)
>> 4. Bulk data movement/replication (possible discussion topic)
>> 5. Any other items to discuss?
>>
>>
>>
>> Thanks and best regards,
>> Dean
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 25 Jul 2011 13:24:48 -0700
> From: "Cinquini, Luca (3880)" <Luca.Cinquini at jpl.nasa.gov>
> Subject: Re: [Go-essp-tech] ESGF Telco This Week 7-26-2011!
> To: Nathan Wilhelmi <wilhelmi at ucar.edu>
> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
> Message-ID: <D6FA1161-10E8-4970-A0B2-8D2529E11E55 at jpl.nasa.gov>
> Content-Type: text/plain; charset="Windows-1252"
>
> Hi Nate,
>
> On Jul 25, 2011, at 2:21 PM, Nathan Wilhelmi wrote:
>
>> Hi Dean,
>>
>> A couple of items I would like to raise:
>>
>> 1. What is the status of getting the P2P registry protocol
>> documented?
>>
>> 2. I would like to start the discussion about the protocol we use for
>> metadata federation. Like the other API's this will need to be
>> reviewed
>> and documented. This has come up as out of scope for the search
>> group.
>> Is there another logical API group for this or does one need to be
>> created? If needed I will volunteer to work on getting this going.
>
> I actually think this is an integral part of the search working
> group, especially considering its ramifications into Open Search.
> We should start discussing this at the next call.
>
> thanks, Luca
>
>>
>> Thanks!
>>
>> -Nate
>>
>> Williams, Dean N. wrote:
>>> Dear Colleagues,
>>>
>>> We we have our regularly scheduled telco this week. The dial-in
>>> number is: (925) 424-8105 access code 305757#
>>>
>>> Agenda for this week?s meeting:
>>>
>>> 1. ESGF/CMIP5 help desk: (getting more of the team involved, FAQ,
>>> feedback, who?s handling issues, how can we tell if an issue has
>>> been resolved, etc.)
>>> 2. Improving federation stability
>>> 3. Interface group discussion (possibility discussion topic)
>>> 4. Bulk data movement/replication (possible discussion topic)
>>> 5. Any other items to discuss?
>>>
>>>
>>>
>>> Thanks and best regards,
>>> Dean
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 25 Jul 2011 14:32:44 -0600
> From: Nathan Wilhelmi <wilhelmi at ucar.edu>
> Subject: Re: [Go-essp-tech] ESGF Telco This Week 7-26-2011!
> To: "Cinquini, Luca (3880)" <Luca.Cinquini at jpl.nasa.gov>
> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
> Message-ID: <4E2DD2EC.7070701 at ucar.edu>
> Content-Type: text/plain; charset=windows-1252; format=flowed
>
> Hi,
>
> Next week is great, the search API documentation (last sentence under
> the actor section) lists this as out of scope for search....
>
> -Nate
>
> Cinquini, Luca (3880) wrote:
>> Hi Nate,
>>
>> On Jul 25, 2011, at 2:21 PM, Nathan Wilhelmi wrote:
>>
>>> Hi Dean,
>>>
>>> A couple of items I would like to raise:
>>>
>>> 1. What is the status of getting the P2P registry protocol
>>> documented?
>>>
>>> 2. I would like to start the discussion about the protocol we use
>>> for
>>> metadata federation. Like the other API's this will need to be
>>> reviewed
>>> and documented. This has come up as out of scope for the search
>>> group.
>>> Is there another logical API group for this or does one need to be
>>> created? If needed I will volunteer to work on getting this going.
>>
>> I actually think this is an integral part of the search working
>> group, especially considering its ramifications into Open Search.
>> We should start discussing this at the next call.
>>
>> thanks, Luca
>>
>>> Thanks!
>>>
>>> -Nate
>>>
>>> Williams, Dean N. wrote:
>>>> Dear Colleagues,
>>>>
>>>> We we have our regularly scheduled telco this week. The dial-in
>>>> number is: (925) 424-8105 access code 305757#
>>>>
>>>> Agenda for this week?s meeting:
>>>>
>>>> 1. ESGF/CMIP5 help desk: (getting more of the team involved, FAQ,
>>>> feedback, who?s handling issues, how can we tell if an issue
>>>> has
>>>> been resolved, etc.)
>>>> 2. Improving federation stability
>>>> 3. Interface group discussion (possibility discussion topic)
>>>> 4. Bulk data movement/replication (possible discussion topic)
>>>> 5. Any other items to discuss?
>>>>
>>>>
>>>>
>>>> Thanks and best regards,
>>>> Dean
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 25 Jul 2011 14:37:39 -0700
> From: Karl Taylor <taylor13 at llnl.gov>
> Subject: Re: [Go-essp-tech] discrepancies among gateways
> To: go-essp-tech at ucar.edu
> Message-ID: <4E2DE223.4040204 at llnl.gov>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Dear all,
>
> Another discrepancy:
>
> When limited to the CMIP5 project, the "ensemble" search category is
> missing from all but the BADC, PCMDI, and NCI gateways. Those 3
> gateways all indicate that there are 8033 datasets available, but
> under
> "ensemble" they list different numbers of ensemble members: PCMDI
> displays a list of 38; BADC, 23; and NCI, 13.
>
> Why don't all gateways include this search option, and why isn't the
> same list displayed at all 3 sites that have the option?
>
> Best regards,
> Karl
>
>
> ------------------------------
>
> Message: 5
> Date: Mon, 25 Jul 2011 17:03:23 -0700
> From: Karl Taylor <taylor13 at llnl.gov>
> Subject: [Go-essp-tech] model names
> To: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>, Bryan Lawrence
> <bryan.lawrence at stfc.ac.uk>, Eric Guilyardi
> <Eric.Guilyardi at locean-ipsl.upmc.fr>, "charlotte.pascoe at stfc.ac.uk
> "
> <charlotte.pascoe at stfc.ac.uk>
> Message-ID: <4E2E044B.4010800 at llnl.gov>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
>
> Bob Drach is on vacation, so I'll ask my question to this list.
>
> There is a model_id stored as a global attribute in the CMIP5 netCDF
> files. This name is (typically) passed to CMOR which simply writes it
> to the files. The name is also used by CMOR to construct filenames
> and
> directory names, but it first removes characters forbidden by the
> CMIP5
> specs. For example, the model_id=BCC-CSM1.1 is changed to BCC-
> CSM1-1 in
> file and directory names.
>
> My question is: If a group published a bunch of data with
> model_id=CSIRO-Mk3-6-0 and then published additional data with
> CSIRO-Mk3.6.0, would the ESG publisher or related software get
> confused? Note that the model name appearing in filenames and
> directory
> structures is not changed in going from 3-6-0 to 3.6.0.
>
> Best regards,
> Karl
Hi Karl,
yes we have consistently used CSIRO-Mk3-6-0 for the filenames, but our
group would prefer to use CSIRO-Mk3.6.0 as the reason for the numbers
is to provide versioning information. We are happy to change the
filenames on our disk area where the NCI ESG picks up the data from,
but of course don't want to create any confusion.
As you say, we would be interested in the approach of using CSIRO-
Mk3.6.0 for any new files that we add to the repository. The only
downside to this is that we are trying to serve the same data up
locally on a file-system and the 2 different filenames would create
some confusion and make commands like unix slightly more tricky to
operate on them or even when developing tools to process them.
We would prefer to keep using the convention CSIRO-Mk3-6-0 with the
filenames but for the label of the model (ie. the text that would
appear in publications or be seen on the ESG gateways) to be CSIRO-
Mk3.6.0
Regards,
Mark.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110725/2dfa0ee0/attachment-0001.html
>
> ------------------------------
>
> Message: 6
> Date: Tue, 26 Jul 2011 10:52:45 +1000
> From: Muhammad Atif <muhammad.atif at anu.edu.au>
> Subject: Re: [Go-essp-tech] discrepancies among gateways
> To: Shreyas Cholia <scholia at lbl.gov>
> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>,
> esg-gateway-dev at earthsystemgrid.org
> Message-ID: <4E2E0FDD.6010107 at anu.edu.au>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Karl,
>
> Thanks for this. We will look into the issue and post an update.
>
> Regards,
>
> On 25/07/11 09:55, Shreyas Cholia wrote:
>> Hi Karl
>>
>> We will look into the NERSC discrepency on Monday.
>>
>> -Shreyas
>>
>> On Fri, Jul 22, 2011 at 4:12 PM, Karl Taylor <taylor13 at llnl.gov
>> <mailto:taylor13 at llnl.gov>> wrote:
>>
>> Dear all,
>>
>> I noticed that as of 15:30 PDT on 7/22/11 that different ESG
>> gateways are serving different amounts of CMIP5 data:
>>
>> gateway # of CMIP5 datasets available
>> ---------- -----------------------------------
>> PCMDI 6973
>> BADC 6973
>> DKRZ 6973
>> NCI 6974 (why 1 more than the above????)
>> JPL 6457 (missing CSIRO-MK3-3 model)
>> NCAR 6457 (missing CSIRO-MK3-3 model)
>> NERSC 6458 (missing CSIRO-MK3-3 model)
>> ORNL 0
>>
>> I noticed several other problems:
>>
>> 1. NERSC gateway: the search categories are not correctly
>> populated.
>>
>> 2. NCI gateway: If you click on CMIP5 in search category
>> "project" you get an error message instead of a list of datasets.
>> Error: Failed to convert property value of type [java.lang.String]
>> to required type [int] for property limit; nested exception is
>> java.lang.NumberFormatException: For input string: ""
>> If you then click on CSIRO-Mk3-6-0 in the "model" category, you
>> get 516 datasets returned, which is proper behavior. Curiously,
>> if you then click on "any model" you get the complete CMIP5 list
>> of datasets, as you should have when you previously got the error
>> message.
>>
>> 3. NCAR gateway: CMIP5/IPCC AR5 appears as one of the "projects"
>> (along with "cmip5"). This is confusing. We want users seeking
>> CMIP5 data to go to cmip5 and not get distracted by CMIP5/IPCC
>> AR5. Please remove project CMIP5/IPCC AR5.
>>
>> 4. There are inconsistencies among the gateways in rendering the
>> list of models that appear under the "model" search category. A
>> common problem is that the word "model" appears before some of the
>> model names on some of the gateways (but not others). Also there
>> are other minor differences (e.g., lower case in the place of
>> uppercase acronyms). We want users to get used to the "official"
>> CMIP5 names, which they should use in publications. These
>> official names can differ slightly from the names that appear in
>> the filenames. The official model name is supposed to have been
>> recorded in each file as a global attribute called "model_id".
>> From information sent to me my the groups, following official
>> model names should appear in the search list (or if this is not
>> possible, the name of the model as recorded in each filename
>> should be used (sorry that sometimes more than 1 model appears in
>> a table cell):
>>
>> Official model_id model name appearing in filename
>> ---------------------
>> ------------------------------------------
>> ACCESS ACCESS
>> BCC-CSM1.1 bcc-csm1-1
>> BNU-ESM BNU-ESM
>> CanESM2, CanCM4, CanAM4 CanESM2, CanCM4, CanAM4
>> CCSM4 CCSM4
>> CCSM4(RSMAS) CCSM4-RSMAS
>> CESM1(BGC) CESM1-BGC
>> CESM1(CAM5) CESM1-CAM5
>> CESM1(CHEM,CAM5) CESM1-CHEM-CAM5
>> CESM1(CHEM) CESM1-CHEM
>> CESM1(WACCM) CESM1-WACCM
>> CMCC-CESM CMCC-CESM
>> CMCC-CM CMCC-CM
>> CMCC-CMS CMCC-CMS
>> CNRM-CM5 CNRM-CM5
>> CSIRO-Mk3.6 CSIRO-Mk3-6
>> EC-EARTH EC-EARTH
>> FGOALS-G2.0 FGOALS-G2
>> FGOALS-gl FGOALS-gl
>> FGOALS-S2.0 FGOALS-S2
>> FIO-ESM FIO-ESM
>> GEOS-5 GEOS-5
>> GFDL-CM2.1 GFDL-CM2P1
>> GFDL-CM3 GFDL-CM3
>> GFDL-ESM2G GFDL-ESM2G
>> GFDL-ESM2M GFDL-ESM2M
>> GFDL-HIRAM-C180 GFDL-HIRAM-C180
>> GFDL-HIRAM-C360 GFDL-HIRAM-C360
>> GISS-E2-H GISS-E2-H
>> GISS-E2-H-CC GISS-E2-H-CC
>> GISS-E2-R GISS-E2-R
>> GISS-E2-R-CC GISS-E2-R-CC
>> GISS-E2CS-H GISS-E2CS-H
>> GISS-E2CS-R GISS-E2CS-R
>> HadCM3 HadCM3
>> HadCM3Q HadCM3Q
>> HadGEM2-AO, HadGEM2-A
>> HadGEM2-AO, HadGEM2-A
>> HadGEM2-CC HadGEM2-CC
>> HadGEM2-ES HadGEM2-ES
>> HiGEM1.2 HiGEM1-2
>> INM-CM4 inmcm4
>> IPSL-CM5A-LR IPSL-CM5A-LR
>> IPSL-CM5A-MR IPSL-CM5A-MR
>> IPSL-CM5B IPSL-CM5B
>> MIROC-ESM MIROC-ESM
>> MIROC-ESM-CHEM MIROC-ESM-CHEM
>> MIROC4h MIROC4h
>> MIROC4m MIROC4m
>> MIROC5 MIROC5
>> MPI-ESM-HR MPI-ESM-HR
>> MPI-ESM-LR MPI-ESM-LR
>> MPI-ESM-MR MPI-ESM-MR
>> MRI-AGCM3.2H MRI-AGCM3.2H
>> MRI-AGCM3.2S MRI-AGCM3-2S
>> MRI-CGCM3 MRI-CGCM3
>> MRI-ESM1 MRI-ESM1
>> NorESM-M, NorESM-ME, NorESM1-L NorESM-M, NorESM-ME, NorESM1-L
>>
>>
>> I have not confirmed that model_id is necessarily consistent with
>> the names in the first column, nor that the filenames are
>> consistent with the 2nd column. If there are inconsistencies, I
>> would like to hear about them.
>>
>> Because of the problems above (other than 4), the CMIP5 website
>> now (temporarily) directs researchers to only the PCMDI, BADC, and
>> DKRZ gateways. Only at these sites is both the search engine
>> working o.k. and all the output from CMIP5 can be seen. We don't
>> want to direct users to sites where they will encounter problems
>> or be unable to see some of the CMIP5 output.
>>
>> Best regards,
>> Karl
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>>
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
> --
> Muhammad Atif
> ANU Supercomputer Facility NCI National Facility
> Leonard Huxley, Mills Road Room 318, Bldg # 56
> Australian National University Ph: +61 2 6125 5031
> Canberra, ACT 0200, Australia M: 0430 393863
> CRICOS Provider #00120C http://anusf.anu.edu.au
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110726/cee79561/attachment.html
>
> ------------------------------
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
> End of GO-ESSP-TECH Digest, Vol 29, Issue 56
> ********************************************
More information about the GO-ESSP-TECH
mailing list