[Go-essp-tech] Publishing dataset with option --update
Nathan Wilhelmi
wilhelmi at ucar.edu
Tue Dec 20 17:06:26 MST 2011
Hi Bob/Mike,
I believe the problem is that when files were added the timestamp on the
dataset wasn't updated.
The triple store will only harvest datasets that have files and an
updated timestamp after the last harvest.
So what likely happened is the dataset was created without files, so it
wasn't initially harvested. Files were subsequently added, but the
timestamp wasn't updated, so it was still not a candidate for harvesting.
Can you update the date_updated timestamp for the dataset in question
and then trigger the RDF harvesting, I believe the dataset will show up
then.
Thanks!
-Nate
On 12/20/2011 11:49 AM, Serguei Nikonov wrote:
> Hi Mike,
>
> I am a member of data publishers group. I have been publishing considerable
> amount of data without such kind of troubles but this one occurred only when I
> tried to add some files to existing dataset. Publishing from scratch works fine
> for me.
>
> Thanks,
> Sergey
>
> On 12/20/2011 01:29 PM, Ganzberger, Michael wrote:
>> Hi Serguei,
>>
>> That task is on a scheduler and will re-run every 10 minutes. If your data does not appear after that time then perhaps there is another issue. One issue could be that publishing to the gateway requires that you have the role of "Data Publisher";
>>
>> "check that the account is member of the proper group and has the special role of Data Publisher"
>>
>> http://esgf.org/wiki/ESGFNode/FAQ
>>
>> Mike
>>
>>
>> -----Original Message-----
>> From: Serguei Nikonov [mailto:serguei.nikonov at noaa.gov]
>> Sent: Tuesday, December 20, 2011 10:12 AM
>> To: Ganzberger, Michael
>> Cc: Stéphane Senesi; Drach, Bob; go-essp-tech at ucar.edu
>> Subject: Re: [Go-essp-tech] Publishing dataset with option --update
>>
>> Hi Mike,
>>
>> thansk for suggestion but I don't have any privileges to do anything on gateway.
>> I am just publishing data on GFDL data node.
>>
>> Regards,
>> Sergey
>>
>> On 12/20/2011 01:05 PM, Ganzberger, Michael wrote:
>>>
>>>
>>> Hi Serguei,
>>>
>>> I'd like to suggest this that may help you from http://esgf.org/wiki/Cmip5Gateway/FAQ
>>>
>>>
>>>
>>> "The search does not reflect the latest DB changes I've made
>>>
>>> You have to manually trigger the 3store harvesting. Logging as root and go to Admin->"Gateway Scheduled Tasks"->"Run tasks" and restart the job named RDFSynchronizationJobDetail"
>>>
>>> Mike Ganzberger
>>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Stéphane Senesi
>>> Sent: Tuesday, December 20, 2011 9:42 AM
>>> To: Serguei Nikonov
>>> Cc: Drach, Bob; go-essp-tech at ucar.edu
>>> Subject: Re: [Go-essp-tech] Publishing dataset with option --update
>>>
>>> Serguei
>>>
>>> We have for some time now experienced similar problems when publishing
>>> to the PCMDI gateway, i.e. not getting a "SUCCESS" message when
>>> publishing . Sometimes, files are actually published (or at least
>>> accessible through the gateway, their status being actually
>>> "START_PUBLISHING", after esg_list_datasets report) , sometimes not. An
>>> hypothesis is that the PCMDI Gateway load do generate the problem. We
>>> havn't yet got a confirmation by Bob.
>>>
>>> In contrast to your case, this happens when publishing a dataset from
>>> scratch (I mean, not an update)
>>>
>>> Best regards (do not expect any feeback from me since early january, yet)
>>>
>>> S
>>>
>>>
>>> Serguei Nikonov wrote, On 20/12/2011 18:11:
>>>> Hi Bob,
>>>>
>>>> I needed to add some missed variables to existing dataset and I found in
>>>> esgpublish command an option --update. When I tried it I've got normal message like
>>>> INFO 2011-12-20 11:21:00,893 Publishing:
>>>> cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1, parent =
>>>> pcmdi.GFDL
>>>> INFO 2011-12-20 11:21:07,564 Result: PROCESSING
>>>> INFO 2011-12-20 11:21:11,209 Result: PROCESSING
>>>> ....
>>>>
>>>> but nothing happened on gateway - new variables are not there. The files
>>>> corresponding to these variables are in database and in THREDDS catalog but
>>>> apparently were not published on gateway.
>>>>
>>>> I used command line
>>>> esgpublish --update --keep-version --map<map_file> --project cmip5 --noscan
>>>> --publish.
>>>>
>>>> Should map file be of some specific format to make it works in mode I need?
>>>>
>>>> Thanks,
>>>> Sergey Nikonov
>>>> GFDL
>>>>
>>>>
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>
>>>>
>>>
>>>
>>
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
More information about the GO-ESSP-TECH
mailing list