[icoads_usa] Delayed Mode buoy update for ICOADS
worley at ucar.edu
Fri Nov 22 15:03:27 MST 2013
I have been studying the ftp directory systems at PMEL, reviewed how we accomplished the previous update (circa 2008), and now I am try to layout the strategy to do the same in early 2014.
Thank you for entertaining my questions below and helping me thoroughly understand the options.
The objective is:
Acquire the full period of record for all GTMBA data under PMEL management. Process the high resolution data (probably ascii), both hourly and the 10 min. resolution, selecting fields that exist in the ICOADS Core and Nocn record format.
First, I would like to request your permission to share the PMEL ftp access with Dr. Elizabeth Kent, cc'd here, from the National Oceanographic Centre (NOC) in Southampton, UK. She is a marine data expert and is considering offering assistance to the ICOADS project by converting the GTMBA data into the ICOADS format. May I provide Dr. Kent the password?
There is a wealth of metadata online at the sites listed below and in the README files within the ftp directory structure:
Below are described the files I believe we need. Your comments and further recommendations
are welcomed. Since, ICOADS is most about surface meteorology and near surface oceanographic measurements the "met……….." files are a key starting point.
From ftp.pmel.noaa.gov/high_resolution/ascii/hr we would acquire all "met…" files, e.g.
met0n110w_hr.ascii. These will provide WSPD, WDIR, AIRT, SST, and RH.
Similarly, from ftp.pmel.noaa.gov/high_resolution/ascii/10m we would acquire all the "met… " files.
There are some additional measurements we need that are not in the "met" files.
hourly sea surface salinity, e.g. sss0n0e_hr.ascii
daily barometric pressure, e.g. bp0n110w_dy.ascii
From ftp.pmel.noaa.gov/ascii/sites/daily we would acquire all the "pos…." files, e.g. pos0n0e_dy.ascii.
As we process the "met" files we will attached the daily buoy position, from the "pos" files, when the dates can be matched, otherwise the buoy nominal position will be used.
Of course we will study the documentation in:
Are there any daily "met" files we should gather to ensure we get the earliest start times possible for each time series?
Is it correct to assume we do not need the temperature and salinity files, because SST is included in the "met" files, and SSS has a stand alone file set?
Any other recommendations?
We need to use the WMO buoy identification for eliminating GTS GTMBA data (received from NCEP) in favor of the delayed mode data from PMEL. To do that we have used a table in the form shown below
(only a snippet is shown for illustration).
BUOYLOCS LOWLIMIT UPLIMIT WMONO
000N1470E 0 19990219 52318
000N1470E 19990220 19991030 52079
000N1470E 19991029 20000908 52046
000N1470E 20001027 0 52079
LOWLIMIT and UPLIMIT define the applicable time period for the WMO number.
The same information (without the time LIMITs) is here: http://www.pmel.noaa.gov/tao/proj_over/wmo.html
Do you have a file like the one above - with the date LIMITS included?
On Nov 15, 2013, at 3:46 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:
> Hi Steve -
> You might want to check out our high resolution files on our website.
> I have just updated them.
> I am still in the process of checking everything, so you should
> re-download your files again periodically to make sure you have
> I will send you the password in a separate email for security.
> Instructions for accessing the TAO PMEL FTP site
> Please do not share these details with other users.
> If anyone is interested, please give them our contact
> information, i.e.
> Dai McClurg <dai.c.mcclurg at noaa.gov>
> Paul Freitag <paul.freitag at noaa.gov>
> In a web Browser:
> enter the password into the little pop up window.
> High time-resolution data are located in the
> high_resolution subdirectory, in both ascii
> and netcdf format. All high resolution files are
> compressed using gzip, and can be uncompressed
> using gunzip.
> Below is an example, which should give you a sense
> of how the daily data are organized.
> For daily ASCII ocean velocity data with files arranged
> by mooring site navigate to
> and look for files named cur*.ascii.
> For NetCDF files, you would navigate to
> for 4-byte NetCDF files or
> for more compact NetCDF files.
> If you want all sites in one file, you would
> navigate to, for example,
> and the file named cur*dy.ascii will contain
> all available daily data for all sites in the
> Tropical Pacific. Similarly for PIRATA and RAMA in
> To get files interactively in a script
> ftp ftp.pmel.noaa.gov
> login as taopmelftp
> using the password. Then, for example, to get
> all daily velocity ascii files
> cd ascii/sites/daily/
> mget cur*.ascii
> Similarly for 4-byte netcdf
> cd cdf/sites/daily
> mget cur*.cdf
> On Thu, Nov 14, 2013 at 3:13 PM, Steve Worley <worley at ucar.edu> wrote:
> Thank you for the terrific very complete response. I will get back to you again, I'm in a bit
> of crunched with things right now, but wanted you to know I appreciated your effort.
> On Nov 12, 2013, at 5:23 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:
>> Hello Steve -
>> We are working on including high resolution data in our FTP site, and
>> will let you know when files become available.
>> Please note that the high resolution data offered in our FTP site
>> will be identical to what is offered now through our websites
>> like these
>> JAVA Data Delivery:
>> Non-JAVA Data Delivery:
>> JAVA Data Display and Delivery
>> Non-JAVA Data Display and Delivery
>> Surface Air-Sea Flux Data Display and Delivery
>> The air-sea flux bulk algorithm is described here
>> We don't include high resolution data from NDBC Refresh buoys
>> in our files. Refresh data are provided to us by NDBC, and we get
>> Refresh data from
>> for example for 2N,110W
>> The data are on OceanSites netcdf format, with one file per
>> instrument per deployment. As far as I know all of the data
>> are realtime, and not delayed mode. I don't believe we have
>> ever gotten delayed mode data from NDBC.
>> Originally NDBC based their quality control procedures on ours, but
>> they may have changed what they do over the years, so it would
>> be best to get information directly from them about their QC.
>> I am not familiar with the details of metadata which we have provided
>> you in the past, by my colleagues probably are. It would be helpful
>> if you could provide any details on what was provided before, so
>> I could query them about it.
>> We get hourly TRITON data form JAMSTEC in their so-called
>> CSV ascii format. For you to do so would require contacting them
>> to get permission to access their data server through their
>> firewall. Their data manager
>> Iwao UEKI <uekii at jamstec.go.jp>
>> would be the recommended technical contact, although you may
>> need to more formally request access from Kentaro Ando, and
>> I believe his email is
>> andouk at jamstec.go.jp
>> I believe they have 10-minute resolution data in a format similar
>> to their hourly data. My sense is that the hourly data are more highly
>> quality controlled, and that the 10 minute subsurface data have
>> not been mapped to fixed depths as the hourly data have, so you
>> may find those data more difficult to work with. Iwao would certainly
>> be able to provide you with more complete information.
>> Let me know if you have further questions, and in any case I
>> will let you know when our FTP site contains high resolution data.
>> Dai McClurg
>> NOAA/PMEL/OCRD/JISAO/Univ. of Washington
>> Dai.C.McClurg at noaa.gov, 206-526-6785
>> On Fri, Nov 8, 2013 at 8:46 AM, Steve Worley <worley at ucar.edu> wrote:
>> We wish to use FTP access to acquire all the TAO/TRITON, PIRATA,
>> and RAMA delayed mode data. As in the past, these data are the foundation
>> for the Global Tropical Moored Buoy Array (GTMBA) in the International Comprehensive
>> Ocean-Atmosphere Data Set (ICOADS). We are seeking the highest resolution, quality
>> controlled, long duration file sets. All meteorological parameters (wind, air temperature, RH,
>> alp, etc..) and ocean surface data (temperature, salinity, etc.) are of interest.
>> Metadata is also important. In the past you have provided several files with information about
>> deployment and recovery dates, WMO numbers, nominal locations, etc.. We will need these also.
>> Our data processing removes GTMBA data received via GTS and replaces it with the DM data.
>> - Can you describe any quality or resolution differences, if any, that exist between the PMEL DM GTMBA
>> collection and the NDBC (since 2011) prepared data now being archived at NODC? Is more or different
>> QC being applied by NDBC?
>> - We can search the web and have records where we last collected the JAMSTEC part of the GTMBA. Can
>> you recommend a specific location?
>> Steven Worley, and the ICOADS team
>> Steven Worley / NCAR
>> worley at ucar.edu
>> Wrk: 303.497.1248
>> Mobile: 720.468.1961
> Steven Worley / NCAR
> worley at ucar.edu
> Wrk: 303.497.1248
> Mobile: 720.468.1961
Steven Worley / NCAR
worley at ucar.edu
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the icoads_usa