[Go-essp-tech] Publishing dataset with option --update

[Nathan Wilhelmi]

Hi Bob/Mike,

I believe the problem is that when files were added the timestamp on the 
dataset wasn't updated.

The triple store will only harvest datasets that have files and an 
updated timestamp after the last harvest.

So what likely happened is the dataset was created without files, so it 
wasn't initially harvested. Files were subsequently added, but the 
timestamp wasn't updated, so it was still not a candidate for harvesting.

Can you update the date_updated timestamp for the dataset in question 
and then trigger the RDF harvesting, I believe the dataset will show up 
then.

Thanks!
-Nate

On 12/20/2011 11:49 AM, Serguei Nikonov wrote:
> Hi Mike,
>
> I am a member of data publishers group. I have been publishing considerable
> amount of data without such kind of troubles but this one occurred only when I
> tried to add some files to existing dataset. Publishing from scratch works fine
> for me.
>
> Thanks,
> Sergey
>
> On 12/20/2011 01:29 PM, Ganzberger, Michael wrote:
>> Hi Serguei,
>>
>> That task is on a scheduler and will re-run every 10 minutes.  If your data does not appear after that time then perhaps there is another issue.  One issue could be that publishing to the gateway requires that you have the role of "Data Publisher";
>>
>> "check that the account is member of the proper group and has the special role of Data Publisher"
>>
>> http://esgf.org/wiki/ESGFNode/FAQ
>>
>> Mike
>>
>>
>> -----Original Message-----
>> From: Serguei Nikonov [mailto:serguei.nikonov at noaa.gov]
>> Sent: Tuesday, December 20, 2011 10:12 AM
>> To: Ganzberger, Michael
>> Cc: Stéphane Senesi; Drach, Bob; go-essp-tech at ucar.edu
>> Subject: Re: [Go-essp-tech] Publishing dataset with option --update
>>
>> Hi Mike,
>>
>> thansk for suggestion but I don't have any privileges to do anything on gateway.
>> I am just publishing data on GFDL data node.
>>
>> Regards,
>> Sergey
>>
>> On 12/20/2011 01:05 PM, Ganzberger, Michael wrote:
>>>
>>>
>>> Hi Serguei,
>>>
>>> I'd like to suggest this that may help you from http://esgf.org/wiki/Cmip5Gateway/FAQ
>>>
>>>
>>>
>>> "The search does not reflect the latest DB changes I've made
>>>
>>> You have to manually trigger the 3store harvesting. Logging as root and go to Admin->"Gateway Scheduled Tasks"->"Run tasks" and restart the job named RDFSynchronizationJobDetail"
>>>
>>> Mike Ganzberger
>>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Stéphane Senesi
>>> Sent: Tuesday, December 20, 2011 9:42 AM
>>> To: Serguei Nikonov
>>> Cc: Drach, Bob; go-essp-tech at ucar.edu
>>> Subject: Re: [Go-essp-tech] Publishing dataset with option --update
>>>
>>> Serguei
>>>
>>> We have for some time now experienced similar problems when publishing
>>> to the PCMDI gateway, i.e. not getting a "SUCCESS" message when
>>> publishing . Sometimes, files are actually published (or at least
>>> accessible through the gateway, their status being actually
>>> "START_PUBLISHING", after esg_list_datasets report) , sometimes not. An
>>> hypothesis is that the PCMDI Gateway load do generate the problem. We
>>> havn't yet got a confirmation by Bob.
>>>
>>> In contrast to your case, this happens when publishing a dataset from
>>> scratch (I mean, not an update)
>>>
>>> Best regards (do not expect any feeback from me since early january, yet)
>>>
>>> S
>>>
>>>
>>> Serguei Nikonov wrote, On 20/12/2011 18:11:
>>>> Hi Bob,
>>>>
>>>> I needed to add some missed variables to existing dataset and I found in
>>>> esgpublish command an option --update. When I tried it I've got normal message like
>>>> INFO       2011-12-20 11:21:00,893 Publishing:
>>>> cmip5.output1.NOAA-GFDL.GFDL-CM3.historical.mon.atmos.Amon.r1i1p1, parent =
>>>> pcmdi.GFDL
>>>> INFO       2011-12-20 11:21:07,564   Result: PROCESSING
>>>> INFO       2011-12-20 11:21:11,209   Result: PROCESSING
>>>> ....
>>>>
>>>> but nothing happened on gateway - new variables are not there. The files
>>>> corresponding to these variables are in database and in THREDDS catalog but
>>>> apparently were not published on gateway.
>>>>
>>>> I used command line
>>>> esgpublish --update --keep-version --map<map_file>     --project cmip5 --noscan
>>>> --publish.
>>>>
>>>> Should map file be of some specific format to make it works in mode I need?
>>>>
>>>> Thanks,
>>>> Sergey Nikonov
>>>> GFDL
>>>>
>>>>
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>
>>>>
>>>
>>>
>>
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech