[Rda-users] NCAR Research Data Archive Blog

[NCAR Research Data Archive Blog]

NCAR Research Data Archive Blog

///////////////////////////////////////////
Where's my data?

Posted: 07 Jul 2015 06:29 PM PDT
http://feedproxy.google.com/~r/NcarResearchDataArchiveBlog/~3/8fnGUqoWSgM/wheres-my-data.html?utm_source=feedburner&utm_medium=email

The RDA considers the ecosystem that data resides in when deciding what to  
archive.  For instance, we archive both NCEP GFS and FNL analyses*; they  
are part of the NCEP Global Data Assimilation System (GDAS) system.
List of RDA datasets related to GDASds351.0 NCEP ADP Global Upper Air  
Observational Weather Data, October 1999 - continuingds461.0 NCEP ADP  
Global Surface Observational Weather Data, October 1999 - continuingds735.0  
NCEP GDAS Satellite Radiance DataResearchers may want to examine the data  
assimilated into the analyses.  They could be useful for future reanalyses,  
Observation System Simulation Experiments (OSSE) or regional monitoring.   
Thus, we archive several of the major data sources in the GDAS data flow.   
You can see the data shortfall in the AIRS 2015 total web server inventory  
listing.


We pull data from national centers such as NCEP and monitor the data flow.   
In late June and early July, 2015, we experienced a drop in ds735.0  
expected file sizes of the data.  After some investigation, we traced it to  
planned network maintenance at NCEP that resulted in a reduction of  
satellite data that were ingested in the GDAS update cycle.

The investigation turned up some insights that I would like to share.  Do  
try this at home.
View the ds735.0 Web Server Holdings for the AIRS subset.  Notice that the  
files are smaller than expected for June 22 and July 1, 2015 (all dates and  
times UTC).Click on the smaller tar files to download them one at a time.   
Examination of their contents shows an absence of the 06Z file and a short  
12Z file: 20150701.airs:
total 1031864
314608 gdas.airsev.t00z.20150701.bufr
76416 gdas.airsev.t12z.20150701.bufr
640840 gdas.airsev.t18z.20150701.bufrView the NCEP Real Time Data  
Monitoring System page.Scroll down to 'Model Data Dump Tables' and select  
06z under GDS.You should see a table labeled 'GDS Dump Data Counts Time  
Series Plots'.  Select airsev and you should see this plot showing the data  
drops.Now go back and try 12z and see the missing days for this cycle.Now  
select acars (Aircraft Communications Addressing and Reporting System),  
which is part of ds335.0.  Those with NCAR internal access accounts can  
view the inventory of corresponding  
big_endian/gdas.aircar.tHHz.YYYYMMDD.bufr.be files.

What's causing these data drops?  Leave a comment with your answer.  The  
first correct answer will get a NCAR photo postcard autographed by the RDA  
data specialist team.
* Read What's the difference between GFS and FNL? for background  
information.

///////////////////////////////////////////
NCAR RDA System Outage July 7, 2015

Posted: 07 Jul 2015 05:34 PM PDT
http://feedproxy.google.com/~r/NcarResearchDataArchiveBlog/~3/zyc0k-wPs5w/ncar-rda-system-outage-july-7-2015.html?utm_source=feedburner&utm_medium=email

RDA data services will be down today for system maintenance between 6:00  
and 16:00 18:00 MDT.

For up-to-date information, visit the NCAR CISL Resource Status page.

UPDATE Tue Jul  7 18:28:02 MDT 2015:
At this time Yellowstone, Geyser and Caldera has been released for  
production workload.Data request processing has resumed.  If your job did  
not automatically restart and complete satisfactorily, resubmit a new one.

--
You are subscribed to email updates from "NCAR Research Data Archive Blog."
To stop receiving these emails, you may unsubscribe now:  
https://feedburner.google.com/fb/a/mailunsubscribe?k=J0HJCpZ3cfcfvRzzDcG9piIXHIs

Email delivery powered by Google.
Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, United  
States

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/rda-users/attachments/20150708/0ab34082/attachment.html