[Go-essp-tech] trackback data to metadata working at badc

Nathan Wilhelmi wilhelmi at ucar.edu
Wed Nov 9 07:53:48 MST 2011 

Hi Bryan,

That is correct. In the 2.0 genealogy both local and remote datasets all 
go into the DB. In addition to the Trackback working as noted it also 
means you can generate wget scripts etc across the entire federation.

Thanks!
-Nate

On 11/9/2011 7:50 AM, Bryan Lawrence wrote:
> Hi Nathan
>
> That's very cool ... is that something in the gateway 2.0 genealogy?
>
> (As you can see I am having a metadata day, as part of the kickoff for our new PIMMS project - http://proj.badc.rl.ac.uk/pimms)
>
> Cheers
> Bryan
>
>
>
>> Hi Bryan,
>>
>> Here is an example of the Trackback working with all the datasets, not
>> just local. Technically this instance doesn't have any local datasets.
>>
>> http://search-esg.prototype.ucar.edu/
>>
>> Disclaimer, this code is a little old, but you should see it working.
>> It's missing some recent trackback changes.
>>
>> Thanks!
>>
>> -Nate
>>
>> On 11/9/2011 7:13 AM, Bryan Lawrence wrote:
>>> Hi Cecelia
>>>
>>> I should say that it is working perfectly traversing from data to metadata, without us doing anything specific (GOOD!) As Stephen points out though, there is a disconnect if you try and get from metadata to data that isn't on the local gateway ...
>>>
>>> To see it working well, look at:
>>> http://cmip-gw.badc.rl.ac.uk/trackback/query.htm?id=esg%3amodel_hadgem2-es_picontrol&session=true&fromSearch=true
>>> and navigate from the output tab to the data ... (one has to open the data collections +)
>>>
>>> And coming back there is a box on the summary data page
>>> http://cmip-gw.badc.rl.ac.uk/dataset/cmip5.output1.MOHC.HadGEM2-ES.piControl.6hr.atmos.6hrPlev.r1i1p1.html>
>>>
>>> LIke I say, it doesn' work well if all the data and metadata are not colocated - but it works pretty well when they are! Big progress.
>>>
>>> Cheers
>>> Bryan
>>>
>>>> hi Dean et al
>>>>
>>>> It looks like this now works at PCMDI?
>>>>
>>>> However, at both PCMDI and BADC, I get internal errors from ECHAM simulation descriptions. Do we know what that's about?
>>>>
>>>> Thanks in advance,
>>>> Bryan
>>>>
>>>>> Dear Cecelia,
>>>>>
>>>>> Bob is looking into updating the postgres database...but this is very
>>>>> sensitive work. Under careful consideration, we are afraid that if we
>>>>> update the database, then it may break a number of other things and the
>>>>> entire system may fail. I am told that Bob had to do some special things
>>>>> on our end in order to upgrade to the latest gateway.
>>>>>
>>>>> Let's discuss this at the next go-essp tech call.  WE realize that your
>>>>> team has put a lot of effort into this and we want to make sure it is used
>>>>> by the community.
>>>>>
>>>>> Best regards,
>>>>> 	Dean
>>>>>
>>>>> On 10/27/11 8:23 AM, "Cecelia DeLuca"<Cecelia.DeLuca at noaa.gov>   wrote:
>>>>>
>>>>>> Hi Dean and all,
>>>>>>
>>>>>> Just checking - any update on this?  I have been away for a while and it
>>>>>> looks like it
>>>>>> is still not working at pcmdi.  It's been almost two months now.  This
>>>>>> represents a huge
>>>>>> amount of effort and it would be a shame to not offer it to users
>>>>>> because of the lack of a
>>>>>> software update. Is there a new holdup, or anything our team can do to
>>>>>> help move
>>>>>> this along?
>>>>>>
>>>>>> Thanks,
>>>>>> Cecelia
>>>>>>
>>>>>> On 10/11/2011 8:16 AM, Cecelia DeLuca wrote:
>>>>>>> Thanks Dean.
>>>>>>> - Cecelia
>>>>>>>
>>>>>>> On 10/7/2011 3:04 PM, Williams, Dean N. wrote:
>>>>>>>> Dear Cecelia,
>>>>>>>>
>>>>>>>>        We apologize for not being able to update the postgres database
>>>>>>>> this week. We are on a deadline to get a number of things done for the
>>>>>>>> CMIP5 archive. Not to say that the trackback isn't high on our list. We
>>>>>>>> have got to start moving data to the core repositories sites and
>>>>>>>> replicating. As stated in an earlier message, the team has accomplished
>>>>>>>> the first task to move and replicate data. Mike is currently working to
>>>>>>>> continue the replication effort.
>>>>>>>>
>>>>>>>> I am hopeful that Bob will be able to look into the postgres upgrade
>>>>>>>> next week. This should allow the trackback...but we want to make sure
>>>>>>>> this upgrade doesn't cause something else to break. We need to be
>>>>>>>> careful here.
>>>>>>>>
>>>>>>>> Again, sorry for the delay...
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>         Dean
>>>>>>>> ________________________________________
>>>>>>>> From: go-essp-tech-bounces at ucar.edu [go-essp-tech-bounces at ucar.edu] On
>>>>>>>> Behalf Of Cecelia DeLuca [Cecelia.DeLuca at noaa.gov]
>>>>>>>> Sent: Friday, October 07, 2011 8:39 AM
>>>>>>>> To: go-essp-tech at ucar.edu
>>>>>>>> Subject: Re: [Go-essp-tech] trackback still not working at PCMDI
>>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> Please be aware that the trackback is still not working at PCMDI.
>>>>>>>> I think on the call on Tues we were expecting resolution of the issue
>>>>>>>> by
>>>>>>>> today.
>>>>>>>> Can someone please help fix this?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -- Cecelia
>>>>>>>>
>>>>>>>> On 10/3/2011 8:31 AM, Williams, Dean N. wrote:
>>>>>>>>> Dear Cecelia,
>>>>>>>>>
>>>>>>>>>           We have upgraded to the latest version of the gateway 1.3.2
>>>>>>>>> and I believe
>>>>>>>>> Bob has the latest version Postgres for the latest release of the
>>>>>>>>> gateway.
>>>>>>>>>
>>>>>>>>>           Let's put this on the agenda for tomorrow's telco. Do you and
>>>>>>>>> Sylvia
>>>>>>>>> still want to have a portion of the telco time to demo CoG?
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>           Dean
>>>>>>>>>
>>>>>>>>> On 10/3/11 7:16 AM, "Cecelia DeLuca"<Cecelia.DeLuca at noaa.gov>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi everyone,
>>>>>>>>>>
>>>>>>>>>> Sylvia reported early in September that the trackback was not working
>>>>>>>>>> in the PCMDI gateway.  It is still not working at all for the CMIP5
>>>>>>>>>> runs.
>>>>>>>>>> It is frustrating to not have this available after so much effort
>>>>>>>>>> went
>>>>>>>>>> into it.  It is also making it impossible to report out success on
>>>>>>>>>> milestones
>>>>>>>>>> that we have for getting this running in the production site by the
>>>>>>>>>> end
>>>>>>>>>> of the FY.
>>>>>>>>>>
>>>>>>>>>> Eric has reported that the problem lies with PCMDI not upgrading to
>>>>>>>>>> Postgres
>>>>>>>>>> database version 8.4.  Does everyone agree this is all that is
>>>>>>>>>> needed and
>>>>>>>>>> if so is there a timelines for doing this?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Cecelia
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> ===================================================================
>>>>>>>>>> Cecelia DeLuca
>>>>>>>>>> NESII/CIRES/NOAA Earth System Research Laboratory
>>>>>>>>>> 325 Broadway, Boulder 80305-337
>>>>>>>>>> Email: cecelia.deluca at noaa.gov
>>>>>>>>>> Phone: 303-497-3604
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> GO-ESSP-TECH mailing list
>>>>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>>> --
>>>>>>>> ===================================================================
>>>>>>>> Cecelia DeLuca
>>>>>>>> NESII/CIRES/NOAA Earth System Research Laboratory
>>>>>>>> 325 Broadway, Boulder 80305-337
>>>>>>>> Email: cecelia.deluca at noaa.gov
>>>>>>>> Phone: 303-497-3604
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> GO-ESSP-TECH mailing list
>>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> GO-ESSP-TECH mailing list
>>>>> GO-ESSP-TECH at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>
>>>>
>>>> --
>>>> Bryan Lawrence
>>>> University of Reading:  Professor of Weather and Climate Computing.
>>>> National Centre for Atmospheric Science: Director of Models and Data.
>>>> STFC: Director of the Centre for Environmental Data Archival.
>>>> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>
>>>
>>> --
>>> Bryan Lawrence
>>> University of Reading:  Professor of Weather and Climate Computing.
>>> National Centre for Atmospheric Science: Director of Models and Data.
>>> STFC: Director of the Centre for Environmental Data Archival.
>>> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>
> --
> Bryan Lawrence
> University of Reading:  Professor of Weather and Climate Computing.
> National Centre for Atmospheric Science: Director of Models and Data.
> STFC: Director of the Centre for Environmental Data Archival.
> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence

More information about the GO-ESSP-TECH mailing list