[icoads_usa] Delayed Mode buoy update for ICOADS

Steve Worley worley at ucar.edu
Mon Nov 25 19:08:38 MST 2013


Thanks Dai,
We will refine our plan and get back to you.
We appreciate and time history of WMO buoy numbers  you can provide.
I did not realize Linda was still in communications about buoy data - extend my greeting to her.
Thanks

On Nov 25, 2013, at 5:29 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:

> 
> And, I forgot to mention, that prior to Next Generation ATLAS
> in May 1996, the primary time resolution for high resolution data 
> was hourly, and so for earlier start times, you will probably also 
> want to see the met*.ascii files here
> 
>   ftp://ftp.pmel.noaa.gov/high_resolution/ascii/hr/
> 
> which I believe begin in April 1990.
> 
> At Current Meter sites, you can find even earlier data
> at a few mainly equator sites at 15-minute, 30-minute, 
> and other resolutions. These data are located here
> 
>   ftp://ftp.pmel.noaa.gov/high_resolution/ascii/rescue/
> 
> Some of these data being in 1977.
> 
> 
> 
> On Mon, Nov 25, 2013 at 4:15 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:
> 
> Hi Steve - 
> 
> Yes, you may provide access information to any of your 
> colleagues who need it, just pass along their contact 
> info. I have added Elizabeth's contact information to 
> our records. 
> 
> In answer to your questions:
> 
> Regarding daily met files: 
> 
> At some sites, yes there will be earlier start times for the 
> daily met files, but that may be simply because realtime
> data are included in the daily files, but not the high 
> resolution files.
> 
> The daily met files include a composite of delayed mode
> and realtime data, and should generally have more complete
> time coverage. They are located here
> 
>     ftp://ftp.pmel.noaa.gov/ascii/sites/daily/
> 
> with names met*.ascii. You can distinguish between realtime
> and delayed mode data using the source codes at the extreme
> right. 
> 
> Regarding salinity and temperature files:
> 
> For daily data, you should find exact agreement between the 
> SST from the sst*.ascii files and the SST from the t*.ascii files. 
> The same should be true for daily salinity (sss*.ascii and s*.ascii). 
> 
> For hourly salinity data, I believe you will find exact agreement
> also. 
> 
> SST is more complicated. 
> 
> There are several different time resolutions for SST, but primarily 
> its 10-minute for data since May 1996 (next generation ATLAS), 
> and hourly prior to that (ATLAS). When TRITON took over the west
> pacific more than a decade ago, they have provided only hourly SSTs.
> 
> There is an additional complication for hourly SST for a few sites. 
> You will find these t*hr.ascii files
> 
> t0n110w_hr.ascii
> t0n137e_hr.ascii
> t0n140w_hr.ascii
> t0n147e_hr.ascii
> t0n154e_hr.ascii
> t0n156e_hr.ascii
> t0n161e_hr.ascii
> t0n165e_hr.ascii
> t0n170e_hr.ascii
> t1.5s90e_hr.ascii
> t2n130e_hr.ascii
> t2n137e_hr.ascii
> t2n147e_hr.ascii
> t2n156e_hr.ascii
> t2n165e_hr.ascii
> t2s156e_hr.ascii
> t2s165e_hr.ascii
> t5n130e_hr.ascii
> t5n137e_hr.ascii
> t5n147e_hr.ascii
> t5n156e_hr.ascii
> t5n165e_hr.ascii
> t5s156e_hr.ascii
> t5s95e_hr.ascii
> t8n130e_hr.ascii
> t8n137e_hr.ascii
> t8n156e_hr.ascii
> t8s95e_hr.ascii
> 
> which contain SST and subsurface data. The corresponding
> SST files contain different SST data, and so you should 
> probably use both sets of files. Let me know if you need
> further details. We had several different buoy systems at 
> different times, including current-meter sites, and ATLAS
> sites, and more recently TRITON. We do not provide hourly
> temperature data from the Next Generation ATLAS buoys.
> 
> I will ask Paul and Linda for information about WMO numbers, 
> and what documentation is available and get back to you.
> Please send further questions as they arise.
> 
>                   Cheers,
> 
>                                      --Dai
> 
> 
> 
> On Fri, Nov 22, 2013 at 2:03 PM, Steve Worley <worley at ucar.edu> wrote:
> Dai,
> 
> I have been studying the ftp directory systems at PMEL, reviewed how we accomplished the previous update (circa 2008), and now I am try to layout the strategy to do the same in early 2014.
> 
> Thank you for entertaining my questions below and helping me thoroughly understand the options. 
> 
> The objective is:
> Acquire the full period of record for all GTMBA data under PMEL management.  Process the high resolution data (probably ascii), both hourly and the 10 min. resolution, selecting fields that exist in the ICOADS Core and Nocn  record format.
> 
> First, I would like to request your permission to share the PMEL ftp access with Dr. Elizabeth Kent, cc'd here, from the National Oceanographic Centre (NOC) in Southampton, UK.  She is a marine data expert and is considering offering assistance to the ICOADS project by converting the GTMBA data into the ICOADS format.  May I provide Dr. Kent the password?
> 
> There is a wealth of metadata online at the sites listed below and in the README files within the ftp directory structure:
> http://www.pmel.noaa.gov/tao/proj_over/mooring.shtml
> http://www.pmel.noaa.gov/tao/proj_over/sensors.shtml
> http://www.pmel.noaa.gov/tao/proj_over/qc.html
> http://www.pmel.noaa.gov/tao/proj_over/gts.html
> 
> Below are described the files I believe we need.  Your comments and further recommendations 
> are welcomed.  Since, ICOADS is most about surface meteorology and near surface oceanographic measurements the "met……….." files are a key starting point.
> 
> From ftp.pmel.noaa.gov/high_resolution/ascii/hr we would acquire  all "met…" files, e.g. 
>  met0n110w_hr.ascii.  These will provide WSPD, WDIR, AIRT, SST, and RH.
> Similarly, from ftp.pmel.noaa.gov/high_resolution/ascii/10m we would acquire all the "met… " files.
> 
> There are some additional measurements we need that are not in the "met" files.
> ftp.pmel.noaa.gov/high_resolution/ascii/hr
>   hourly sea surface salinity, e.g. sss0n0e_hr.ascii
>  ftp.pmel.noaa.gov/ascii/sites/daily/
>   daily barometric pressure, e.g. bp0n110w_dy.ascii
> 
> From ftp.pmel.noaa.gov/ascii/sites/daily we would acquire all the "pos…." files, e.g. pos0n0e_dy.ascii.
> As we process the "met" files we will attached the daily buoy position, from the "pos" files, when the dates can be matched, otherwise the buoy nominal position will be used.
> 
> Of course we will study the documentation in:
> ftp.pmel.noaa.gov/Documentation/README_met.txt
> ftp.pmel.noaa.gov/Documentation/README_pos.txt
> ftp.pmel.noaa.gov/Documentation/README_sss.txt
> ftp.pmel.noaa.gov/Documentation/README_bp.txt
> 
> Questions:
> Are there any daily "met" files we should gather to ensure we get the earliest start times possible for each time series?
> Is it correct to assume we do not need the temperature and salinity files, because SST is included in the "met" files, and SSS has a stand alone file set?
> Any other recommendations?
> 
> We need to use the WMO buoy identification for eliminating GTS GTMBA data (received from NCEP) in favor of the delayed mode data from PMEL.  To do that we have used a table in the form shown below
> (only a snippet is shown for illustration).
> 
> BUOYLOCS  LOWLIMIT UPLIMIT  WMONO
> --------------------------------
> 000N1470E        0 19990219 52318
> 000N1470E 19990220 19991030 52079
> 000N1470E 19991029 20000908 52046
> 000N1470E 20001027        0 52079
> ……………….
> LOWLIMIT and UPLIMIT define the applicable time period for the WMO number.
> The same information (without the time LIMITs)  is here: http://www.pmel.noaa.gov/tao/proj_over/wmo.html
> Do you have a file like the one above - with the date LIMITS included?
> Regards,
> Steve
> 
> On Nov 15, 2013, at 3:46 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:
> 
>> 
>> Hi Steve - 
>> 
>> You might want to check out our high resolution files on our website. 
>> I have just updated them.
>> 
>> I am still in the process of checking everything, so you should 
>> re-download your files again periodically to make sure you have 
>> everything.
>> 
>> I will send you the password in a separate email for security.
>> 
>>                  Regards, 
>> 
>>                                    --Dai
>> 
>> -----------------------------------------------------
>> 
>>   Instructions for accessing the TAO PMEL FTP site
>> 
>> Please do not share these details with other users.
>> If anyone is interested, please give them our contact
>> information, i.e.
>> 
>>  Dai McClurg <dai.c.mcclurg at noaa.gov>
>>  Paul Freitag <paul.freitag at noaa.gov>
>> 
>> In a web Browser:
>> 
>>  ftp://taopmelftp@ftp.pmel.noaa.gov
>> 
>> enter the password into the little pop up window.
>> 
>> High time-resolution data are located in the 
>> high_resolution subdirectory, in both ascii
>> and netcdf format. All high resolution files are
>> compressed using gzip, and can be uncompressed
>> using gunzip.
>> 
>> Below is an example, which should give you a sense
>> of how the daily data are organized.
>> 
>> For daily ASCII ocean velocity data with files arranged
>> by mooring site navigate to
>> 
>>   ascii/sites/daily/
>> 
>> and look for files named cur*.ascii.
>> 
>> For NetCDF files, you would navigate to
>> 
>>   cdf/sites/daily/
>> 
>> for 4-byte NetCDF files or
>> 
>>   cdf_2byte/sites/daily/
>> 
>> for more compact NetCDF files.
>> 
>> If you want all sites in one file, you would
>> navigate to, for example,
>> 
>>   ascii/fields/TAO_TRITON/
>> 
>> and the file named cur*dy.ascii will contain
>> all available daily data for all sites in the
>> Tropical Pacific. Similarly for PIRATA and RAMA in
>> 
>>   ascii/fields/PIRATA/
>> 
>> or
>> 
>>   ascii/fields/RAMA/
>> 
>> To get files interactively in a script
>> 
>>   ftp ftp.pmel.noaa.gov
>> 
>>   login as taopmelftp
>> 
>> using the password. Then, for example, to get
>> all daily velocity ascii files
>> 
>>  cd ascii/sites/daily/
>>  ascii
>>  prompt
>>  mget cur*.ascii
>>  quit
>> 
>> Similarly for 4-byte netcdf
>> 
>>  cd cdf/sites/daily
>>  binary
>>  prompt
>>  mget cur*.cdf
>>  quit
>> 
>> 
>> 
>> On Thu, Nov 14, 2013 at 3:13 PM, Steve Worley <worley at ucar.edu> wrote:
>> Dai,
>> Thank you for the terrific very complete response.  I will get back to you again, I'm in a bit 
>> of crunched with things right now, but wanted you to know I appreciated your effort.
>> Thanks
>> sjw
>> 
>> On Nov 12, 2013, at 5:23 PM, Dai Mcclurg - NOAA Affiliate <dai.c.mcclurg at noaa.gov> wrote:
>> 
>>> 
>>> Hello Steve - 
>>> 
>>> We are working on including high resolution data in our FTP site, and 
>>> will let you know when files become available. 
>>> 
>>> Please note that the high resolution data offered in our FTP site 
>>> will be identical to what is offered now through our websites
>>> like these
>>> 
>>>   JAVA Data Delivery:
>>>     http://www.pmel.noaa.gov/tao/data_deliv/deliv.html
>>> 
>>>   Non-JAVA Data Delivery:
>>>     http://www.pmel.noaa.gov/tao/data_deliv/frames/
>>> 
>>>   JAVA Data Display and Delivery
>>>     http://www.pmel.noaa.gov/tao/disdel/
>>> 
>>>   Non-JAVA Data Display and Delivery
>>>     http://www.pmel.noaa.gov/tao/disdel/frames/
>>> 
>>>   Surface Air-Sea Flux Data Display and Delivery
>>>     http://www.pmel.noaa.gov/tao/oceansites/flux/
>>> 
>>>     The air-sea flux bulk algorithm is described here
>>>        http://www.pmel.noaa.gov/tao/oceansites/flux/documentation.html
>>> 
>>> We don't include high resolution data from NDBC Refresh buoys
>>> in our files. Refresh data are provided to us by NDBC, and we get 
>>> Refresh data from
>>> 
>>>   ftp://data.ndbc.noaa.gov/data/oceansites/DATA/
>>> 
>>> for example for 2N,110W
>>> 
>>>   ftp://data.ndbc.noaa.gov/data/oceansites/DATA/T2N110W/
>>> 
>>> The data are on OceanSites netcdf format, with one file per
>>> instrument per deployment. As far as I know all of the data 
>>> are realtime, and not delayed mode. I don't believe we have 
>>> ever gotten delayed mode data from NDBC. 
>>> 
>>> Originally NDBC based their quality control procedures on ours, but
>>> they may have changed what they do over the years, so it would
>>> be best to get information directly from them about their QC.
>>> 
>>> I am not familiar with the details of metadata which we have provided
>>> you in the past, by my colleagues probably are. It would be helpful
>>> if you could provide any details on what was provided before, so 
>>> I could query them about it.
>>> 
>>> We get hourly TRITON data form JAMSTEC in their so-called 
>>> CSV ascii format. For you to do so would require contacting them
>>> to get permission to access their data server through their
>>> firewall. Their data manager
>>> 
>>>   Iwao UEKI <uekii at jamstec.go.jp>
>>> 
>>> would be the recommended technical contact, although you may
>>> need to more formally request access from Kentaro Ando, and 
>>> I believe his email is 
>>> 
>>>    andouk at jamstec.go.jp
>>> 
>>> I believe they have 10-minute resolution data in a format similar
>>> to their hourly data. My sense is that the hourly data are more highly 
>>> quality controlled, and that the 10 minute subsurface data have 
>>> not been mapped to fixed depths as the hourly data have, so you 
>>> may find those data more difficult to work with. Iwao would certainly 
>>> be able to provide you with more complete information.
>>> 
>>> Let me know if you have further questions, and in any case I 
>>> will let you know when our FTP site contains high resolution data.
>>> 
>>>                 Regards, 
>>> 
>>>                                     --Dai
>>> 
>>> -----------------------------------------------------------
>>> Dai McClurg
>>> NOAA/PMEL/OCRD/JISAO/Univ. of Washington
>>> Dai.C.McClurg at noaa.gov, 206-526-6785
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Nov 8, 2013 at 8:46 AM, Steve Worley <worley at ucar.edu> wrote:
>>> Hi,
>>> 
>>> We wish to use FTP access to acquire all the TAO/TRITON, PIRATA,
>>> and RAMA delayed mode data.  As in the past, these data are the foundation
>>> for the Global Tropical Moored Buoy Array (GTMBA) in the International Comprehensive
>>> Ocean-Atmosphere Data Set (ICOADS).  We are seeking the highest resolution, quality 
>>> controlled, long duration file sets.   All meteorological parameters (wind, air temperature, RH,
>>> alp, etc..) and ocean surface data (temperature, salinity, etc.) are of interest.  
>>> 
>>> Metadata is also important.  In the past you have provided several files with information about 
>>> deployment and recovery dates, WMO numbers, nominal locations, etc..  We will need these also.
>>> 
>>> Our data processing removes GTMBA data received via GTS and replaces it with the DM data.
>>> 
>>> Aside: 
>>>  - Can you describe any quality or resolution differences, if any, that exist between the PMEL DM GTMBA
>>>    collection and the NDBC (since 2011) prepared data now being archived at NODC?  Is more or different
>>>    QC being applied by NDBC?
>>>  - We can search the web and have records where we last collected the JAMSTEC part of the GTMBA.  Can
>>>     you recommend a specific location?
>>> 
>>> Regards,
>>> Steven Worley, and the ICOADS team
>>> 
>>> ---
>>> Steven Worley / NCAR
>>> worley at ucar.edu
>>> rda.ucar.edu
>>> Wrk: 303.497.1248
>>> Mobile: 720.468.1961
>>> ----
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> ---
>> Steven Worley / NCAR
>> worley at ucar.edu
>> rda.ucar.edu
>> Wrk: 303.497.1248
>> Mobile: 720.468.1961
>> ----
>> 
>> 
>> 
>> 
>> 
> 
> ---
> Steven Worley / NCAR
> worley at ucar.edu
> rda.ucar.edu
> Wrk: 303.497.1248
> Mobile: 720.468.1961
> ----
> 
> 
> 
> 
> 
> 

---
Steven Worley / NCAR
worley at ucar.edu
rda.ucar.edu
Wrk: 303.497.1248
Mobile: 720.468.1961
----




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/icoads_usa/attachments/20131125/8dba252e/attachment-0001.html 


More information about the icoads_usa mailing list