[Go-essp-tech] Latest discrepancies among datasets available from different gateways
Karl Taylor
taylor13 at llnl.gov
Mon Aug 1 10:24:48 MDT 2011
Hi Estani,
I understand that discrepancies can arise because things are only
synced once a day. I was a little surprised that you were missing only
8 datasets, as I assumed that folks would usually publish in larger chunks.
Anyway, thanks for looking into it, and I agree that once a day is fine.
Best regards,
Karl
On 7/29/11 7:49 AM, Estanislao Gonzalez wrote:
> Hi Karl,
>
> I'm checking this, but there will always be discrepancies as the
> harvesting gets triggered once every while and not immediately after
> changes are performed.
>
> Harvesting is not a cheap operation as AFAIK everything is sent and
> then compared to what was already present. This will grow considerably
> (harvesting PCMDI already takes more than a minute).
>
> After re-triggering the harvesting operations we have exactly the same
> values as PCMDI and BADC (which are at this time 8193).
>
> The harvesting operations are triggered once per day in our case. I
> don't think there's a real benefit for increasing this frequency, but we
> could do it if it's required.
>
> Thanks,
> Estani
>
> On Wed, 27 Jul 2011 15:53:53 -0700, Karl Taylor wrote:
>> Dear all,
>>
>> Currently (as of a few minutes ago) the ESG sites are showing the
>> following number of CMIP5 datasets:
>>
>> PCMDI 8040
>> BADC 8040
>> NCAR 8040
>> NCI 8041
>> DKRZ 8032
>> ORNL 8031
>> NERSC 7525
>> JPL 7524
>>
>> I think we know about the NCI single extra dataset, which they're
>> presumably working on removing. Also we know NERSC and JPL are not
>> seeing data at the NCI gateway (CSIRO model). Can anyone explain
>>
>> 1) the missing 8 datasets at DKRZ and the 9 missing datasets at
>> ORNL?
>> 2) the difference of 1 between NERSC and JPL?
>>
>> Best regards,
>> Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110801/c8a40787/attachment.html
More information about the GO-ESSP-TECH
mailing list