[icoads_usa] Need input for IVAD planning

Kent, Elizabeth C. eck at noc.ac.uk
Mon Feb 3 06:17:23 MST 2014


In our 1997 paper we compared anemometer and visual data for 1980-1990: http://dx.doi.org/10.1175/1520-0426(1997)014%3C0228:COABES%3E2.0.CO;2 and the Lindau Scale came out well by our criteria.

Bridget's paper http://onlinelibrary.wiley.com/doi/10.1002/joc.1570/abstract adjusted winds from 1950 to 2002.  This seemed to reveal problems with visual winds increasing in recent times, probably due to the "influence" of an available anemometer reading. I recall that ECMWF height adjust visual winds as well as anemometer winds. In the NOC dataset (http://onlinelibrary.wiley.com/doi/10.1002/joc.2059/abstract) we applied an ad hoc adjustment to visual winds to account for this.

Its probably worth having a look at the papers here as well: http://iprc.soest.hawaii.edu/users/tokinaga/waswind.html


From: Scott Woodruff <Scott.D.Woodruff at noaa.gov<mailto:Scott.D.Woodruff at noaa.gov>>
Date: Saturday, 1 February 2014 19:39
To: "Shawn R. Smith" <smith at coaps.fsu.edu<mailto:smith at coaps.fsu.edu>>, "icoads_usa at mailman.ucar.edu<mailto:icoads_usa at mailman.ucar.edu>" <icoads_usa at mailman.ucar.edu<mailto:icoads_usa at mailman.ucar.edu>>, "Berry, David I." <dyb at noc.ac.uk<mailto:dyb at noc.ac.uk>>
Cc: Mark Bourassa <mbourassa at fsu.edu<mailto:mbourassa at fsu.edu>>, Elizabeth Kent <eck at noc.ac.uk<mailto:eck at noc.ac.uk>>
Subject: Re: [icoads_usa] Need input for IVAD planning

Hi Shawn,

As we're getting into some nitty gritty discussion of the visual/Beaufort wind issues etc., I've cc'd Liz too. Responses interleaved below:

On 1/31/14 1:18 PM, Shawn R. Smith wrote:
Hey all,

I have been working today with one of my grad students to move forward on the IVAD project, specifically the Beaufort correction. After reviewing some papers (including Lindau's work), I have some questions to raise.

1. It is pretty clear that the estimated wind reports have been dropping over the past few decades, but without actually looking at the numbers, I was wondering if anyone had a feel for what year or years we should start with in creating the IVAD attm for Lindau bias adjustment. His paper focused on the 1960-1971 period, so that is one option. It would be nice to have an overlap with the period where Dave can create Tair adjusted IVAD records (in the hope that some records would have both adjustments). The latter would facilitate more interesting "scientific" excersising of the IVAD concept.

I'm not sure how worthwhile this would be, but from 1966 forward you would also have the option of using the WMO Pub. 47 metadata in IMMA (Meta-vos attm; ref. also http://icoads.noaa.gov/e-doc/imma/WMO47IMMA_1966_2007-R2.5.pdf), e.g. whether there is an anemometer height provided (HOB) as a cross check. Related to this possibility, attached is Fig. 9 from Kent et al. (2007; http://journals.ametsoc.org/doi/abs/10.1175/JTECH1949.1).

Eventually of course it seems like it would be highly valuable to our user community to offer re-calibrated estimated winds much farther back (e.g. possible earlier time block starting 1900, when I think steamships became predominant?), but I wonder if something like 1960-2007 (end R2.5) might be a useful and more manageable initial period.

Dave - can you remind me over what period of record you can run the Tair correction?

2. I have downloaded the R2.5.1i from NCAR, but now think I should go back and get just the 2.5.1, so as not to have to work with the duplicates.

Dave - have you worked from 2.5.1 or 2.5.1i?

3. When trying to identify Beaufort winds in the IMMA, we plan to extract records based on the status of the wind indicator (WI). From what I can tell indicators 0, 2, 3, 5, and 6 are all "estimated" winds (implying Beaufort), though only 5 is actually labeled Beaufort. I assume 5 is a rather new useage. Are we "safe" to assume any estimated wind is derived from a Beaufort approach (whether a conversion by ICOADS from an actual Beaufort wind # or a wind value extracted from a Beaufort scale as was WMO convention in the post WWII years)?
While all the estimated winds should be visual observations, WI=6 (as discussed under WI in http://icoads.noaa.gov/e-doc/imma/R2.5-imma_short.pdf) is a bit uncertain, because of unresolvable issues in how the original WI information was stored (e.g. in NCDC's old TDF-11 format). As you may know, this problem is also discussed some in the literature (e.g. Kent et al. 2007).

Correct, WI=5 was a relatively new configuration set (during "preconditioning") for some decks from R2.0/2.1, when we concluded from the documentation that the original data were Beaufort force codes. If you're interested, the PS at the end here provides a fairly complicated example of such a preconditioning rule for one deck.

At present, ICOADS has only done a fairly limited number of conversions from original Beaufort codes to store a m/s value (specifically a WMO Code 1100 "old" midpoint, ref. Release 1, Supp. K, Table K5-5) , probably mostly for newly digitized/available sources.

More info in case useful on the post-WWII coding situation (as you mentioned above): When the GTS SHIP code came into effect (by 1949 according to: http://goos.kishou.go.jp/ws/ETMC/code_task/history/SHIP.html) observers could report knots (and importantly weren't constrained to Beaufort mid points). Similarly when the punched card (IMMPC) precursor to the IMMT keyed logbook format was introduced in the 1950s (e.g. 1957 version here http://goos.kishou.go.jp/ws/ETMC/code_task/history/IMMT.html has "Beaufort force" but then "knots" are coded). Then at some point both GTS/IMMT codes evolved so that whole m/s could be reported instead of whole knots (as flagged by a wind speed indicator).

Also in case helpful, the second attachment provides some 1992 notes that Joe Elms put together on several older historical ICOADS decks (to dig deeper, see also original card deck documentation at: http://icoads.noaa.gov/reclaim/us.html), as to the digital data availability and original Beaufort code reporting procedures (which I may have sent you before, but maybe worth another look now).

4. Similarly, we will be extracting ship only records by looking at the platform type (PT). Looks like PT = 0, 1, 2, 3, 4, and 5 are ships. Any gotcha's with this assumption?

Looks good to me, except PT=missing would be one other possibility, and depending on the time period/deck one might conclude a given report with PT=missing is most likely to be from a ship.

5. Also another history lesson for me. Am I correct that wind speed (W) has always been stored in m/s in ICOADS? Lindau's manuscript implies that it was stored in kts, but looking at the original ICOADS1 documentation indicates that m/s was always used.

Correct, ICOADS has always stored wind speeds in m/s. As noted above, the current SHIP (FM13)/IMMT codes allows VOS to report whole knots or whole m/s. (Looking forward, I'm not up to date with how wind speeds are handled in the latest incarnation of the VOS BUFR template, but Dave could comment.)

Finally, next week I will take a look at the IVAD attm structure as it stands now. Unless Dave has any concerns, I think we should lock this down as the version for all the final prototyping. Hopefully that will allow Steve to move forward with mapping some IVAD records into the database. More next week.
If you think this is workable, I'd like to suggest that we transition to the revised Ivad attm structure as agreed at our Nov. 2013 Boulder IVAD meeting.

Sorry that this has taken much longer than anticipated, but I'm reaching closure (hopefully by the end of next week) on a full new version (v15) of the Proposed IMMA Revisions document, incorporating many decisions we took at the November meeting. Meanwhile, attached as a preview are just Tables C96 (Ivad attm) and C96a (VQC flag configuration, so as agreed we are dropping the IMMT flag scheme for VQC) incorporating changes agreed last Nov.

Also, Sandy recently updated the {rdimma1} software (http://icoads.noaa.gov/software/rdimma1) to handle I/O for the finalized Nocn attm (e.g. for WOD13), and, while I'll need to check with her on the logistics/timing, I believe she was also on the verge of enabling the Ivad (and Error) attms in their currently documented (ref. http://icoads.noaa.gov/ivad/IMMA-Rev-v14.pdf) form, so my suspicion is that we will have both the documentation (note: pending checking proposed new field abbreviations for uniqueness etc.) and software for the revised Ivad attm as attached locked down fairly soon.

Sorry for such a long and complicated e-mail, but I hope it's helpful for getting moving on the scientific IVAD demonstration. -Scott

PS: Example Release 2.0 preconditioning rule for deck 187 (from http://icoads.noaa.gov/e-doc/other/dupelim_1946):
Deck 187: Japanese Whaling Fleet
       Rules:  If both wind speed (W) and direction (D) are missing (and not
       in the error attachment), and if 12-point wind direction is "12", and
       Beaufort force falls in the range 0-9 or is "-" or "+" (fields in the
       supplemental attachment corresponding to original TDF-11 positions 114-
       115 and 116, respectively).  If these conditions are all true, make the
       following settings, ignoring any existing settings of the wind direction
       indicator (DI) and wind speed indicator (WI):
            i) DI to missing.
            ii) D to 360 degrees.
            iii) WI to 5 (Beaufort force).
            iv) Convert Beaufort force to W in meters per second using the
            midpoint of the "old" Beaufort equivalence scale, where 0-9 are
            mapped to meters per second according to Release 1, Table K5-5,
            "-" (equivalent to Beaufort force 10) is mapped to 26.8, and "+"
            (equivalent to Beaufort force 11 or higher) to 30.9.

       Background:  In conversion from the original card format into TDF-11,
       errors appear to have been made in failing to convert any direction
       code 36 (north in the 36-point scale) and its associated wind speed.
       However, a 12-point wind direction code and Beaufort force usually seem
       to be available in the supplemental attachment to provide a correction.
       [NOTE: This correction is approximate because D and W in TDF-11 were
       derived from separate fields, when extant, that appeared only in the
       original card format (36-point direction and speed in knots); the
       relationship between the two different sets of D and W fields in the
       original card format is unclear.  DI is set to missing because we
       currently lack a DI value for indication of the 12-point scale.  In
       addition, the 12-point wind direction code of "12" corresponds to a
       range of 335-4 degrees, whose midpoint is not 360 degrees.  Moreover,
       Beaufort value "+" actually signifies force 11 or greater (up to 17),
       according to the original card deck reference manual; usage of the
       midpoint for force 11 is thus definitely an approximation.  Assuming
       that the original card deck reference manual is correct, NCDC (1968)
       incorrectly documents the Beaufort value "+" as signifying force 11.]

This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/icoads_usa/attachments/20140203/036cde51/attachment-0001.html 

More information about the icoads_usa mailing list