[Go-essp-tech] search problems with ESG

[Drach, Bob]

Hi Eric,


On 6/2/11 7:13 AM, "Eric Nienhouse" <ejn at ucar.edu> wrote:

> Hi Stephen,
> 
> stephen.pascoe at stfc.ac.uk wrote:
>> Eric,
>> 
>> I'm not sure this is entirely clear either.  Presumably you mean make a
>> hessian call to tell the Gateway to re-retrieve the catalog.
> Your presumption is correct and I like your term "re-retrieve".  I'll
> use this language in the future to avoid confusion :-)
>> From the datanode's perspective this is the final step in publishing.  I
>> think of harvesting as being the exchange of RDF amongst Gateways.
>> 
>> So, to be clear, do we need to do something like this:
>> 
>>  $ esgpublish --publish --noscan --use-list <listing-file>
>>   
> Yes.  This should simply make the publishing hessian call to the Gateway
> to re-retrieve the catalog for each dataset in the list.  No rewriting
> of thredds nor rescanning of data will occur.

This will generate gateway errors I believe, since the datasets already
exist. And the alternative is to unpublish first, which presumably would
just compound the problem ...

--Bob


>> This would make the publishing hessian call for each dataset in
>> <listing-file> without rewriting thredds or rescanning  the data (I hope :-)
>> ).
>> 
>> S.
>> 
>> ---
>> Stephen Pascoe  +44 (0)1235 445980
>> Centre of Environmental Data Archival
>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>> 
>> 
>> -----Original Message-----
>> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On
>> Behalf Of Eric Nienhouse
>> Sent: 02 June 2011 03:44
>> To: Drach, Bob
>> Cc: go-essp-tech at ucar.edu
>> Subject: Re: [Go-essp-tech] search problems with ESG
>> 
>> Hi Bob,
>> 
>> Sorry this was not clear.  By "re-publish all affected dataset catalogs"
>> I mean: cause the gateway to re-harvest any existing catalogs that have
>> been affected by the delete side effect.  In other words, please
>> "re-havest" all affected dataset catalogs.
>> 
>> For example, unpublishing and republishing the INM datasets to add
>> checksum information may have affected other datasets (such as those
>> from BCC) causing the BCC datasets to loose key DRS search components.
>> In this case the BCC catalogs should be re-harvested by the gateway.
>> Note that re-harvesting INM catalogs may be required in this case as well.
>> 
>> I believe the best way to do so is to run 'esgpublish --publish' on a
>> list of datasets.  Is this a reasonable approach?
>> 
>> Please let me know if you need any more details about this.
>> 
>> Thanks,
>> 
>> -Eric
>> 
>> Drach, Bob wrote:
>>   
>>> Hi Eric,
>>> 
>>> 
>>>   
>>>     
>>>> 2)  Please re-publish all affected dataset catalogs.
>>>>     
>>>>       
>>> Not sure what you mean. Presumably republication happens to correct errors
>>> or reflect modified datasets. You don't want to undo that.
>>> 
>>> For example, I unpublished and republished the INM datasets to add checksum
>>> information.
>>> 
>>> I agree it's a good idea to refrain from removing existing datasets until a
>>> solution to the search problem can be distributed.
>>> 
>>> --Bob
>>> 
>>> 
>>>   
>>>     
>> 
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>   
> 
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech