[Go-essp-tech] Latest discrepancies among datasets available from different gateways

[Karl Taylor]

Hi Estani,

  I understand that discrepancies can arise because things are only 
synced once a day.  I was a little surprised that you were missing only 
8 datasets, as I assumed that folks would usually publish in larger chunks.

Anyway, thanks for looking into it, and I agree that once a day is fine.

Best regards,
Karl

On 7/29/11 7:49 AM, Estanislao Gonzalez wrote:
> Hi Karl,
>
> I'm checking this, but there will always be discrepancies as the
> harvesting gets triggered once every while and not immediately after
> changes are performed.
>
> Harvesting is not a cheap operation as AFAIK everything is sent and
> then compared to what was already present. This will grow considerably
> (harvesting PCMDI already takes more than a minute).
>
> After re-triggering the harvesting operations we have exactly the same
> values as PCMDI and BADC (which are at this time 8193).
>
> The harvesting operations are triggered once per day in our case. I
> don't think there's a real benefit for increasing this frequency, but we
> could do it if it's required.
>
> Thanks,
> Estani
>
> On Wed, 27 Jul 2011 15:53:53 -0700, Karl Taylor wrote:
>> Dear all,
>>
>>   Currently (as of a few minutes ago) the ESG sites are showing the
>> following number of CMIP5 datasets:
>>
>>   PCMDI 8040
>>   BADC 8040
>>   NCAR 8040
>>   NCI 8041
>>   DKRZ 8032
>>   ORNL 8031
>>   NERSC 7525
>>   JPL 7524
>>
>>   I think we know about the NCI single extra dataset, which they're
>> presumably working on removing. Also we know NERSC and JPL are not
>> seeing data at the NCI gateway (CSIRO model). Can anyone explain
>>
>>   1) the missing 8 datasets at DKRZ and the 9 missing datasets at
>> ORNL?
>>   2) the difference of 1 between NERSC and JPL?
>>
>>   Best regards,
>>   Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110801/c8a40787/attachment.html