[Go-essp-tech] Data file gaps

Wed Mar 21 15:33:49 MDT 2012

I checked the branch_time from a file in this dataset, cmip5.output1.NASA-GISS.GISS-E2-R.historical.mon.land.Lmon.r1i1p1, and it was 3981. So I guess I understand this one example, but not without help from Ken Lo and the gentle guidance from others here in the forum. I'd say the risk of users mis-understanding the subtleties of how to use this data is pretty high. 

Any ideas about the other data gaps I found and the files that went AWOL without a version number change? If I have data that has been de-published, should I assume it shouldn't be used and toss it ? 
--Jennifer

> Hi Jennifer and all,
> 
> Data writers were instructed to set "branch_time=0." if a run was *not* initialized from another run (or in cases when this was irrelevant ... e.g., some control runs).  Perhaps this accounts for the cases you examined?
> 
> regards,
> Karl
> 
> On 3/21/12 7:03 AM, Jennifer Adams wrote:
>> 
>> Ken Lo's explanation makes me want to laugh and cry at the same time. I see a global attribute called "branch_time", but for the handful of random files from I checked from both periods it is set to "0."  --Jennifer
>> 
>> 
>> On Mar 20, 2012, at 7:13 PM, Jeffrey F. Painter wrote:
>> 
>>> I can shed some light on one of the gaps: the GISS-E2-R piControl files actually contain TWO control runs!  One side of the gap is one control run, and the other side of the gap is the other control run.  I cut-and-pasted a message from Ken Lo about this, at the bottom of this                   message.
>>> 
>>> - Jeff
>>> 
>>> On 3/20/12 11:55 AM, Karl Taylor wrote:
>>>> 
>>>> database problem??
>>>> 
>>>> -------- Original Message --------
>>>> Subject:	[Go-essp-tech] Data file gaps
>>>> Date:	Tue, 20 Mar 2012 11:50:07 -0700
>>>> From:	Jennifer Adams <jma at cola.iges.org>
>>>> To:	go-essp-tech at ucar.edu <go-essp-tech at ucar.edu>
>>>> 
>>>> Dear All,
>>>> I have a GrADS script that creates descriptor files for CMIP5 data, and it has uncovered some gaps in the date strings of files that belong to an atomic data set. I have checked the ESGF gateways for all the datasets listed below and I can confirm that these files are missing for some variables. I haven't given the variable names, because I have only checked the subset of variables that are of interest to me and that may not be a complete list. If the specific var names that I checked would be useful, I can provide them. While checking on these gaps, I noticed that not all variables in an atomic dataset span the same time range … an inconsistency that doesn't quite make sense to me. I also noticed that in some cases, data files that I had grabbed are no longer listed on the gateway, even though the version number is the same. An example is cmip5.output1.NCC.NorESM1-M.piControl.day.land.day.r1i1p1.v20110901, mrsos, date range 11000101-12001231. What does that mean? 
>>>> 
>>>> The gaps put these data into the "not quite usable" category. I wasn't sure whether to send this to the helpdesk or the forum or both; in the end I am just posting here.
>>>> --Jennifer
>>>> 
>>>>  11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.atmos.Amon.r1i1p1 between 141912 and 241001
>>>>  11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.land.Lmon.r1i1p1  between 141912 and 241001
>>>>   4200 times missing in cmip5.output1.NASA-GISS.GISS-E2-R.piControl.mon.land.Lmon.r1i1p1  between 363012 and 398101 
>>>>  11880 times missing in cmip5.output1.NASA-GISS.GISS-E2-H.piControl.mon.ocean.Omon.r1i1p1 between 141912 and 241001
>>>>   4200 times missing in cmip5.output1.NASA-GISS.GISS-E2-R.piControl.mon.ocean.Omon.r1i1p1 between 363012 and 398101 
>>>>      1 times missing in cmip5.output1.MOHC.HadGEM2-CC.historical.mon.ocean.Omon.r3i1p1    between 200110 and 200112
>>>>     60 times missing in cmip5.output1.NOAA-GFDL.GFDL-ESM2M.piControl.mon.land.Lmon.r1i1p1 between 017012 and 017601
>>>>     60 times missing in cmip5.output1.NOAA-GFDL.GFDL-ESM2M.piControl.mon.land.Lmon.r1i1p1 between 003012 and 003601
>>>>   3650 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1   between 00101231 and 00210101
>>>>   1825 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1   between 00401231 and 00460101
>>>>   7300 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1   between 00751231 and 00960101
>>>> 142350 times missing in cmip5.output1.NOAA-GFDL.GFDL-CM3.piControl.day.atmos.day.r1i1p1   between 01001231 and 04910101
>>>>  73000 times missing in cmip5.output1.NCC.NorESM1-M.piControl.day.land.day.r1i1p1         between 08991231 and 11000101
>>>>  29220 times missing in cmip5.output1.MPI-M.MPI-ESM-LR.rcp85.day.atmos.day.r1i1p1         between 22001231 and 22810101 
>>>> 
>>>> --
>>>> Jennifer M. Adams
>>>> IGES/COLA
>>>> 4041 Powder Mill Road, Suite 302
>>>> Calverton, MD 20705
>>>> jma at cola.iges.org
>>>> 
>>>> 
>>>> 
>>>> 
>>> On 11/4/11 6:07 AM, Ken Lo wrote: 
>>> Dear Jeff Painter, 
>>> 
>>> Years 3331 to 3630 of the control run are meant to be compared with 
>>> historicalMisc (p1), while years 3981 to 4530 are meant to be compared 
>>> with historical, historicalMisc (p109), historicalGHG, historicalNat 
>>> and rcp runs (all p1 except indicated otherwise).  The appropriate years 
>>> for comparison are written in the branch time of the metadata of each 
>>> file.
>> 
>> --
>> Jennifer M. Adams
>> IGES/COLA
>> 4041 Powder Mill Road, Suite 302
>> Calverton, MD 20705
>> jma at cola.iges.org
>> 
>> 
>> 
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120321/5923aad6/attachment-0001.html 