<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>NCAR Research Data Archive Blog</title>
</head>
<body>
<style type="text/css">
h1 a:hover {background-color:#888;color:#fff ! important;}
div#emailbody table#itemcontentlist tr td div ul {
list-style-type:square;
padding-left:1em;
}
div#emailbody table#itemcontentlist tr td div blockquote {
padding-left:6px;
border-left: 6px solid #dadada;
margin-left:1em;
}
div#emailbody table#itemcontentlist tr td div li {
margin-bottom:1em;
margin-left:1em;
}
table#itemcontentlist tr td a:link, table#itemcontentlist tr td a:visited, table#itemcontentlist tr td a:active, ul#summarylist li a {
color:#000099;
font-weight:bold;
text-decoration:none;
}
img {border:none;}
</style>
<div xmlns="http://www.w3.org/1999/xhtml" id="emailbody" style="margin:0 2em;font-family:Georgia,Helvetica,Arial,Sans-Serif;line-height:140%;font-size:13px;color:#000000;">
<table style="border:0;padding:0;margin:0;width:100%">
<tr>
<td style="vertical-align:top" width="99%">
<h1 style="margin:0;padding-bottom:6px;">
<a style="color:#888;font-size:22px;font-family:Arial,Helvetica,Sans-Serif;font-weight:normal;text-decoration:none;" href="http://ncarrda.blogspot.com/" title="(http://ncarrda.blogspot.com/)">NCAR Research Data Archive Blog</a>
</h1>
</td>
<td width="1%" />
</tr>
</table>
<hr style="border:1px solid #ccc;padding:0;margin:0" />
<table id="itemcontentlist">
<tr xmlns="">
<td style="margin-bottom:0;line-height:1.4em;">
<p style="margin:1em 0 3px 0;">
<a name="1" style="font-family:Arial,Helvetica,Sans-Serif;font-size:18px;" href="http://feedproxy.google.com/~r/NcarResearchDataArchiveBlog/~3/8fnGUqoWSgM/wheres-my-data.html?utm_source=feedburner&utm_medium=email">Where's my data?</a>
</p>
<p style="font-size:13px;color:#555;margin:9px 0 3px 0;font-family:Georgia,Helvetica,Arial,Sans-Serif;line-height:140%;font-size:13px;">
<span>Posted:</span> 07 Jul 2015 06:29 PM PDT</p>
<div style="margin:0;font-family:Georgia,Helvetica,Arial,Sans-Serif;line-height:140%;font-size:13px;color:#000000;">The RDA considers the ecosystem that data resides in when deciding what to archive. For instance, we archive both <a href="http://www.ncep.noaa.gov/">NCEP</a> <a href="http://rda.ucar.edu/datasets/ds084.1/">GFS</a> and <a href="http://rda.ucar.edu/datasets/ds083.2/">FNL</a> analyses*; they are part of the NCEP Global Data Assimilation System (GDAS) system.<br /><ul><li><a href="http://rda.ucar.edu/index.html?hash=cgi-bin/dssearch?words=gdas">List of RDA datasets related to GDAS</a></li><li><a href="http://rda.ucar.edu/datasets/ds351.0/">ds351.0</a> NCEP ADP Global Upper Air Observational Weather Data, October 1999 - continuing</li><li><a href="http://rda.ucar.edu/datasets/ds461.0/">ds461.0</a> NCEP ADP Global Surface Observational Weather Data, October 1999 - continuing</li><li><a href="http://rda.ucar.edu/datasets/ds735.0/">ds735.0</a> NCEP GDAS Satellite Radiance Data</li></ul>Researchers may want to examine the data assimilated into the analyses. They could be useful for future reanalyses, Observation System Simulation Experiments (OSSE) or regional monitoring. Thus, we archive several of the major data sources in the GDAS data flow. You can see the data shortfall in the <a href="http://rda.ucar.edu/datasets/ds735.0/index.html#sfol-wl-/data/ds735.0?g=615">AIRS 2015 total web server inventory listing</a>.<br /><br /><a name='more'></a><br />We pull data from national centers such as NCEP and monitor the data flow. In late June and early July, 2015, we experienced a drop in ds735.0 expected file sizes of the data. After some investigation, we traced it to planned network maintenance at NCEP that resulted in a reduction of satellite data that were ingested in the GDAS update cycle.<br /><br />The investigation turned up some insights that I would like to share. Do try this at home.<br /><ol><li>View the <a href="http://rda.ucar.edu/datasets/ds735.0/index.html#sfol-wl-/data/ds735.0?g=615">ds735.0 Web Server Holdings for the AIRS subset</a>. Notice that the files are smaller than expected for June 22 and July 1, 2015 (all dates and times UTC).</li><li>Click on the smaller tar files to download them one at a time. Examination of their contents shows an absence of the 06Z file and a short 12Z file: <blockquote>20150701.airs:<br />total 1031864<br />314608 gdas.airsev.t00z.20150701.bufr<br />76416 gdas.airsev.t12z.20150701.bufr<br />640840 gdas.airsev.t18z.20150701.bufr</blockquote></li><li><a href="http://www.nco.ncep.noaa.gov/pmb/nwprod/realtime/">View the NCEP Real Time Data Monitoring System page</a>.</li><li>Scroll down to 'Model Data Dump Tables' and select 06z under GDS.</li><div a="" class="separator" gt="" href="http://2.bp.blogspot.com/-UIDEmoAz6QE/VZxxLgeGNbI/AAAAAAAAAOk/j8JFJRTHhBM/s1600/RealTime.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="http://2.bp.blogspot.com/-UIDEmoAz6QE/VZxxLgeGNbI/AAAAAAAAAOk/j8JFJRTHhBM/s320/RealTime.png" width="320" /></div><li>You should see a table labeled 'GDS Dump Data Counts Time Series Plots'. Select airsev and you should see this plot showing the data drops.</li><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-R5vT8_pu8fw/VZxxMGJ4ONI/AAAAAAAAAOo/46TBGrsu96s/s1600/gfs.072.2015070706.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="255" src="http://1.bp.blogspot.com/-R5vT8_pu8fw/VZxxMGJ4ONI/AAAAAAAAAOo/46TBGrsu96s/s320/gfs.072.2015070706.gif" width="320" /></a></div><li>Now go back and try 12z and see the missing days for this cycle.</li><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-Iu8kA3_ARis/VZxxL8JNM9I/AAAAAAAAAOg/nUxhshE0uBo/s1600/gfs.072.2015070612.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="255" src="http://1.bp.blogspot.com/-Iu8kA3_ARis/VZxxL8JNM9I/AAAAAAAAAOg/nUxhshE0uBo/s320/gfs.072.2015070612.gif" width="320" /></a></div><li>Now select acars (Aircraft Communications Addressing and Reporting System), which is part of ds335.0. Those with NCAR internal access accounts can <a href="http://rda.ucar.edu/datasets/ds351.0/#sfol-gl?g=52015&p=2">view the inventory</a> of corresponding big_endian/gdas.aircar.tHHz.YYYYMMDD.bufr.be files.<br /><br />What's causing these data drops? Leave a comment with your answer. The first correct answer will get a NCAR photo postcard autographed by the <a href="http://rda.ucar.edu/#!about">RDA data specialist team</a>.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-MO87Gnrl8qA/VZxxLs0npJI/AAAAAAAAAOc/raUlHVd3FKg/s1600/gfs.028.acars.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="255" src="http://2.bp.blogspot.com/-MO87Gnrl8qA/VZxxLs0npJI/AAAAAAAAAOc/raUlHVd3FKg/s320/gfs.028.acars.gif" width="320" /></a></div></li></ol>* Read <a href="http://ncarrda.blogspot.com/2015/04/whats-difference-between-gfs-and-fnl.html">What's the difference between GFS and FNL?</a> for background information.<img src="http://feeds.feedburner.com/~r/NcarResearchDataArchiveBlog/~4/8fnGUqoWSgM?utm_source=feedburner&utm_medium=email" height="1" width="1" alt=""/></div>
</td>
</tr>
<tr>
<td style="margin-bottom:0;line-height:1.4em;">
<p style="margin:1em 0 3px 0;">
<a name="2" style="font-family:Arial,Helvetica,Sans-Serif;font-size:18px;" href="http://feedproxy.google.com/~r/NcarResearchDataArchiveBlog/~3/zyc0k-wPs5w/ncar-rda-system-outage-july-7-2015.html?utm_source=feedburner&utm_medium=email">NCAR RDA System Outage July 7, 2015</a>
</p>
<p style="font-size:13px;color:#555;margin:9px 0 3px 0;font-family:Georgia,Helvetica,Arial,Sans-Serif;line-height:140%;font-size:13px;">
<span>Posted:</span> 07 Jul 2015 05:34 PM PDT</p>
<div style="margin:0;font-family:Georgia,Helvetica,Arial,Sans-Serif;line-height:140%;font-size:13px;color:#000000;">RDA data services will be down today for system maintenance between 6:00 and <strike>16:00</strike> 18:00 MDT. <br /><br />For up-to-date information, visit the <a href="https://www2.cisl.ucar.edu/cisl-status">NCAR CISL Resource Status page</a>.<br /><br />UPDATE Tue Jul 7 18:28:02 MDT 2015:<br /><blockquote>At this time Yellowstone, Geyser and Caldera has been released for production workload.</blockquote>Data request processing has resumed. If your job did not automatically restart and complete satisfactorily, resubmit a new one.<img src="http://feeds.feedburner.com/~r/NcarResearchDataArchiveBlog/~4/zyc0k-wPs5w?utm_source=feedburner&utm_medium=email" height="1" width="1" alt=""/></div>
</td>
</tr>
</table>
<table style="border-top:1px solid #999;padding-top:4px;margin-top:1.5em;width:100%" id="footer">
<tr>
<td style="text-align:left;font-family:Helvetica,Arial,Sans-Serif;font-size:11px;margin:0 6px 1.2em 0;color:#333;">You are subscribed to email updates from <a href="http://ncarrda.blogspot.com/">NCAR Research Data Archive Blog</a>
<br />To stop receiving these emails, you may <a href="https://feedburner.google.com/fb/a/mailunsubscribe?k=J0HJCpZ3cfcfvRzzDcG9piIXHIs">unsubscribe now</a>.</td>
<td style="font-family:Helvetica,Arial,Sans-Serif;font-size:11px;margin:0 6px 1.2em 0;color:#333;text-align:right;vertical-align:top">Email delivery powered by Google</td>
</tr>
<tr>
<td colspan="2" style="text-align:left;font-family:Helvetica,Arial,Sans-Serif;font-size:11px;margin:0 6px 1.2em 0;color:#333;">Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States</td>
</tr>
</table>
</div>
</body>
</html>