[Go-essp-tech] Data file gaps
Jennifer Adams
jma at cola.iges.org
Wed Mar 21 15:33:49 MDT 2012
I checked the branch_time from a file in this dataset, cmip5.output1.NASA-GISS.GISS-E2-R.historical.mon.land.Lmon.r1i1p1, and it was 3981. So I guess I understand this one example, but not without help from Ken Lo and the gentle guidance from others here in the forum. I'd say the risk of users mis-understanding the subtleties of how to use this data is pretty high.
Any ideas about the other data gaps I found and the files that went AWOL without a version number change? If I have data that has been de-published, should I assume it shouldn't be used and toss it ?
--Jennifer
> Hi Jennifer and all,
>
> Data writers were instructed to set "branch_time=0." if a run was *not* initialized from another run (or in cases when this was irrelevant ... e.g., some control runs). Perhaps this accounts for the cases you examined?
>
> regards,
> Karl
>
> On 3/21/12 7:03 AM, Jennifer Adams wrote:
>>
>> Ken Lo's explanation makes me want to laugh and cry at the same time. I see a global attribute called "branch_time", but for the handful of random files from I checked from both periods it is set to "0." --Jennifer
>>
>>
>> On Mar 20, 2012, at 7:13 PM, Jeffrey F. Painter wrote:
>>
>>> I can shed some light on one of the gaps: the GISS-E2-R piControl files actually contain TWO control runs! One side of the gap is one control run, and the other side of the gap is the other control run. I cut-and-pasted a message from Ken Lo about this, at the bottom of this message.
>>>
>>> - Jeff
>>>
>>> On 3/20/12 11:55 AM, Karl Taylor wrote:
>>>>
>>>> database problem??
>>>>
>>>> -------- Original Message --------
>>>> Subject: [Go-essp-tech] Data file gaps
>>>> Date: Tue, 20 Mar 2012 11:50:07 -0700
>>>> From: Jennifer Adams <jma at cola.iges.org>
>>>> To: go-essp-tech at ucar.edu <go-essp-tech at ucar.edu>
>>>>
>>>> Dear All,
>>>> I have a GrADS script that creates descriptor files for CMIP5 data, and it has uncovered some gaps in the date strings of files that belong to an atomic data set. I have checked the ESGF gateways for all the datasets listed below and I can confirm that these files are missing for some variables. I haven't given the variable names, because I have only checked the subset of variables that are of interest to me and that may not be a complete list. If the specific var names that I checked would be useful, I can provide them. While checking on these gaps, I noticed that not all variables in an atomic dataset span the same time range … an inconsistency that doesn't quite make sense to me. I also noticed that in some cases, data files that I had grabbed are no longer listed on the gateway, even though the version number is the same. An example is cmip5.output1.NCC.NorESM1-M.piControl.day.land.day.r1i1p1.v20110901, mrsos, date range 11000101-12001231. What does that mean?
>>>>
>>>> The gaps put these data into the "not quite usable" category. I wasn't sure whether to send this to the helpdesk or the forum or both; in the end I am just posting here.
>>>> --Jennifer
>>>>
>>>> 11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.atmos.Amon.r1i1p1 between 141912 and 241001
>>>> 11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.land.Lmon.r1i1p1 between 141912 and 241001
>>>> 4200 times missing in cmip5.output1.NASA-GISS.GISS-E2-R.piControl.mon.land.Lmon.r1i1p1 between 363012 and 398101
>>>> 11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.ocean.Omon.r1i1p1 between 141912 and 241001
>>>> 4200 times missing in cmip5.output1.NASA-GISS.GISS-E2-R.piControl.mon.ocean.Omon.r1i1p1 between 363012 and 398101
>>>> 1 times missing in cmip5.output1.MOHC.HadGEM2-CC.historical.mon.ocean.Omon.r3i1p1 between 200110 and 200112
>>>> 60 times missing in cmip5.output1.NOAA-GFDL.GFDL-ESM2M.piControl.mon.land.Lmon.r1i1p1 between 017012 and 017601
>>>> 60 times missing in cmip5.output1.NOAA-GFDL.GFDL-ESM2M.piControl.mon.land.Lmon.r1i1p1 between 003012 and 003601
>>>> 3650 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1 between 00101231 and 00210101
>>>> 1825 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1 between 00401231 and 00460101
>>>> 7300 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1 between 00751231 and 00960101
>>>> 142350 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1 between 01001231 and 04910101
>>>> 73000 times missing in cmip5.output1.NCC.NorESM1-M.piControl.day.land.day.r1i1p1 between 08991231 and 11000101
>>>> 29220 times missing in cmip5.output1.MPI-M.MPI-ESM-LR.rcp85.day.atmos.day.r1i1p1 between 22001231 and 22810101
>>>>
>>>> --
>>>> Jennifer M. Adams
>>>> IGES/COLA
>>>> 4041 Powder Mill Road, Suite 302
>>>> Calverton, MD 20705
>>>> jma at cola.iges.org
>>>>
>>>>
>>>>
>>>>
>>> On 11/4/11 6:07 AM, Ken Lo wrote:
>>> Dear Jeff Painter,
>>>
>>> Years 3331 to 3630 of the control run are meant to be compared with
>>> historicalMisc (p1), while years 3981 to 4530 are meant to be compared
>>> with historical, historicalMisc (p109), historicalGHG, historicalNat
>>> and rcp runs (all p1 except indicated otherwise). The appropriate years
>>> for comparison are written in the branch time of the metadata of each
>>> file.
>>
>> --
>> Jennifer M. Adams
>> IGES/COLA
>> 4041 Powder Mill Road, Suite 302
>> Calverton, MD 20705
>> jma at cola.iges.org
>>
>>
>>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120321/5923aad6/attachment-0001.html
More information about the GO-ESSP-TECH
mailing list