[Go-essp-tech] search problems with ESG

[Eric Nienhouse]

Hi Stephen,

stephen.pascoe at stfc.ac.uk wrote:
> Eric,
>
> I'm not sure this is entirely clear either.  Presumably you mean make a hessian call to tell the Gateway to re-retrieve the catalog.  
Your presumption is correct and I like your term "re-retrieve".  I'll 
use this language in the future to avoid confusion :-)
> From the datanode's perspective this is the final step in publishing.  I think of harvesting as being the exchange of RDF amongst Gateways.
>
> So, to be clear, do we need to do something like this:
>
>  $ esgpublish --publish --noscan --use-list <listing-file>
>   
Yes.  This should simply make the publishing hessian call to the Gateway 
to re-retrieve the catalog for each dataset in the list.  No rewriting 
of thredds nor rescanning of data will occur.
> This would make the publishing hessian call for each dataset in <listing-file> without rewriting thredds or rescanning  the data (I hope :-) ).
>
> S.
>
> ---
> Stephen Pascoe  +44 (0)1235 445980
> Centre of Environmental Data Archival
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Eric Nienhouse
> Sent: 02 June 2011 03:44
> To: Drach, Bob
> Cc: go-essp-tech at ucar.edu
> Subject: Re: [Go-essp-tech] search problems with ESG
>
> Hi Bob,
>
> Sorry this was not clear.  By "re-publish all affected dataset catalogs" 
> I mean: cause the gateway to re-harvest any existing catalogs that have 
> been affected by the delete side effect.  In other words, please 
> "re-havest" all affected dataset catalogs.
>
> For example, unpublishing and republishing the INM datasets to add 
> checksum information may have affected other datasets (such as those 
> from BCC) causing the BCC datasets to loose key DRS search components.  
> In this case the BCC catalogs should be re-harvested by the gateway.  
> Note that re-harvesting INM catalogs may be required in this case as well.
>
> I believe the best way to do so is to run 'esgpublish --publish' on a 
> list of datasets.  Is this a reasonable approach?
>
> Please let me know if you need any more details about this.
>
> Thanks,
>
> -Eric
>
> Drach, Bob wrote:
>   
>> Hi Eric,
>>
>>
>>   
>>     
>>> 2)  Please re-publish all affected dataset catalogs.
>>>     
>>>       
>> Not sure what you mean. Presumably republication happens to correct errors
>> or reflect modified datasets. You don't want to undo that.
>>
>> For example, I unpublished and republished the INM datasets to add checksum
>> information.
>>
>> I agree it's a good idea to refrain from removing existing datasets until a
>> solution to the search problem can be distributed.
>>
>> --Bob
>>
>>
>>   
>>     
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>