[icoads_usa] Need input for IVAD planning

Mark Bourassa mbourassa at fsu.edu
Tue Feb 4 10:39:20 MST 2014

I agree and thank you for the advice! Part of the strength of what we 
are doing in the modern period is assessing the corrections in 
comparison to satellite data (not without it's own exciting issues). If 
we look at speed, as opposed to wind vectors, we should be able to get 
quite a few collocations.


On 2/4/2014 11:06 AM, Shawn R. Smith wrote:
> Thanks Liz. We will pull these papers and review them. Clearly this 
> could be a serious rats nest, but we are just working on proof of 
> concept at present. I already can envision an entire proposal to 
> address some aspects of the visual winds and their 
>  variations/uncertainties.
> Cheers,
> Shawn
> On Feb 3, 2014, at 8:17 AM, Kent, Elizabeth C. wrote:
>> Hi,
>> In our 1997 paper we compared anemometer and visual data for 
>> 1980-1990: 
>> http://dx.doi.org/10.1175/1520-0426(1997)014%3C0228:COABES%3E2.0.CO;2 
>> <http://dx.doi.org/10.1175/1520-0426%281997%29014%3C0228:COABES%3E2.0.CO;2> and 
>> the Lindau Scale came out well by our criteria.
>> Bridget's paper 
>> http://onlinelibrary.wiley.com/doi/10.1002/joc.1570/abstract adjusted 
>> winds from 1950 to 2002.  This seemed to reveal problems with visual 
>> winds increasing in recent times, probably due to the "influence" of 
>> an available anemometer reading. I recall that ECMWF height adjust 
>> visual winds as well as anemometer winds. In the NOC dataset 
>> (http://onlinelibrary.wiley.com/doi/10.1002/joc.2059/abstract) we 
>> applied an ad hoc adjustment to visual winds to account for this.
>> Its probably worth having a look at the papers here as well: 
>> http://iprc.soest.hawaii.edu/users/tokinaga/waswind.html
>> Liz
>> From: Scott Woodruff <Scott.D.Woodruff at noaa.gov 
>> <mailto:Scott.D.Woodruff at noaa.gov>>
>> Date: Saturday, 1 February 2014 19:39
>> To: "Shawn R. Smith" <smith at coaps.fsu.edu 
>> <mailto:smith at coaps.fsu.edu>>, "icoads_usa at mailman.ucar.edu 
>> <mailto:icoads_usa at mailman.ucar.edu>" <icoads_usa at mailman.ucar.edu 
>> <mailto:icoads_usa at mailman.ucar.edu>>, "Berry, David I." 
>> <dyb at noc.ac.uk <mailto:dyb at noc.ac.uk>>
>> Cc: Mark Bourassa <mbourassa at fsu.edu <mailto:mbourassa at fsu.edu>>, 
>> Elizabeth Kent <eck at noc.ac.uk <mailto:eck at noc.ac.uk>>
>> Subject: Re: [icoads_usa] Need input for IVAD planning
>> Hi Shawn,
>> As we're getting into some nitty gritty discussion of the 
>> visual/Beaufort wind issues etc., I've cc'd Liz too. Responses 
>> interleaved below:
>> On 1/31/14 1:18 PM, Shawn R. Smith wrote:
>>> Hey all,
>>> I have been working today with one of my grad students to move 
>>> forward on the IVAD project, specifically the Beaufort correction. 
>>> After reviewing some papers (including Lindau's work), I have some 
>>> questions to raise.
>>> 1. It is pretty clear that the estimated wind reports have been 
>>> dropping over the past few decades, but without actually looking at 
>>> the numbers, I was wondering if anyone had a feel for what year or 
>>> years we should start with in creating the IVAD attm for Lindau bias 
>>> adjustment. His paper focused on the 1960-1971 period, so that is 
>>> one option. It would be nice to have an overlap with the period 
>>> where Dave can create Tair adjusted IVAD records (in the hope that 
>>> some records would have both adjustments). The latter would 
>>> facilitate more interesting "scientific" excersising of the IVAD 
>>> concept.
>> I'm not sure how worthwhile this would be, but from 1966 forward you 
>> would also have the option of using the WMO Pub. 47 metadata in IMMA 
>> (/Meta-vos/ attm; ref. also 
>> http://icoads.noaa.gov/e-doc/imma/WMO47IMMA_1966_2007-R2.5.pdf), e.g. 
>> whether there is an anemometer height provided (HOB) as a cross 
>> check. Related to this possibility, attached is Fig. 9 from Kent et 
>> al. (2007; http://journals.ametsoc.org/doi/abs/10.1175/JTECH1949.1).
>> Eventually of course it seems like it would be highly valuable to our 
>> user community to offer re-calibrated estimated winds much farther 
>> back (e.g. possible earlier time block starting 1900, when I think 
>> steamships became predominant?), but I wonder if something like 
>> 1960-2007 (end R2.5) might be a useful and more manageable initial 
>> period.
>>> Dave - can you remind me over what period of record you can run the 
>>> Tair correction?
>>> 2. I have downloaded the R2.5.1i from NCAR, but now think I should 
>>> go back and get just the 2.5.1, so as not to have to work with the 
>>> duplicates.
>>> Dave - have you worked from 2.5.1 or 2.5.1i?
>>> 3. When trying to identify Beaufort winds in the IMMA, we plan to 
>>> extract records based on the status of the wind indicator (WI). From 
>>> what I can tell indicators 0, 2, 3, 5, and 6 are all "estimated" 
>>> winds (implying Beaufort), though only 5 is actually labeled 
>>> Beaufort. I assume 5 is a rather new useage. Are we "safe" to assume 
>>> any estimated wind is derived from a Beaufort approach (whether a 
>>> conversion by ICOADS from an actual Beaufort wind # or a wind value 
>>> extracted from a Beaufort scale as was WMO convention in the post 
>>> WWII years)?
>> While all the estimated winds should be visual observations, WI=6 (as 
>> discussed under WI in 
>> http://icoads.noaa.gov/e-doc/imma/R2.5-imma_short.pdf) is a bit 
>> uncertain, because of unresolvable issues in how the original WI 
>> information was stored (e.g. in NCDC's old TDF-11 format). As you may 
>> know, this problem is also discussed some in the literature (e.g. 
>> Kent et al. 2007).
>> Correct, WI=5 was a relatively new configuration set (during 
>> "preconditioning") for some decks from R2.0/2.1, when we concluded 
>> from the documentation that the original data were Beaufort force 
>> codes. If you're interested, the PS at the end here provides a fairly 
>> complicated example of such a preconditioning rule for one deck.
>> At present, ICOADS has only done a fairly limited number of 
>> conversions from original Beaufort codes to store a m/s value 
>> (specifically a WMO Code 1100 "old" midpoint, ref. Release 1, Supp. 
>> K, Table K5-5) , probably mostly for newly digitized/available sources.
>> More info in case useful on the post-WWII coding situation (as you 
>> mentioned above): When the GTS SHIP code came into effect (by 1949 
>> according to: 
>> http://goos.kishou.go.jp/ws/ETMC/code_task/history/SHIP.html) 
>> observers could report knots (and importantly weren't constrained to 
>> Beaufort mid points). Similarly when the punched card (IMMPC) 
>> precursor to the IMMT keyed logbook format was introduced in the 
>> 1950s (e.g. 1957 version here 
>> http://goos.kishou.go.jp/ws/ETMC/code_task/history/IMMT.html has 
>> "Beaufort force" but then "knots" are coded). Then at some point both 
>> GTS/IMMT codes evolved so that whole m/s could be reported instead of 
>> whole knots (as flagged by a wind speed indicator).
>> Also in case helpful, the second attachment provides some 1992 notes 
>> that Joe Elms put together on several older historical ICOADS decks 
>> (to dig deeper, see also original card deck documentation at: 
>> http://icoads.noaa.gov/reclaim/us.html), as to the digital data 
>> availability and original Beaufort code reporting procedures (which I 
>> may have sent you before, but maybe worth another look now).
>>> 4. Similarly, we will be extracting ship only records by looking at 
>>> the platform type (PT). Looks like PT = 0, 1, 2, 3, 4, and 5 are 
>>> ships. Any gotcha's with this assumption?
>> Looks good to me, except PT=missing would be one other possibility, 
>> and depending on the time period/deck one might conclude a given 
>> report with PT=missing is most likely to be from a ship.
>>> 5. Also another history lesson for me. Am I correct that wind speed 
>>> (W) has always been stored in m/s in ICOADS? Lindau's manuscript 
>>> implies that it was stored in kts, but looking at the original 
>>> ICOADS1 documentation indicates that m/s was always used.
>> Correct, ICOADS has always stored wind speeds in m/s. As noted above, 
>> the current SHIP (FM13)/IMMT codes allows VOS to report whole knots 
>> or whole m/s. (Looking forward, I'm not up to date with how wind 
>> speeds are handled in the latest incarnation of the VOS BUFR 
>> template, but Dave could comment.)
>>> Finally, next week I will take a look at the IVAD attm structure as 
>>> it stands now. Unless Dave has any concerns, I think we should lock 
>>> this down as the version for all the final prototyping. Hopefully 
>>> that will allow Steve to move forward with mapping some IVAD records 
>>> into the database. More next week.
>> If you think this is workable, I'd like to suggest that we transition 
>> to the revised /Ivad/ attm structure as agreed at our Nov. 2013 
>> Boulder IVAD meeting.
>> Sorry that this has taken much longer than anticipated, but I'm 
>> reaching closure (hopefully by the end of next week) on a full new 
>> version (v15) of the Proposed IMMA Revisions document, incorporating 
>> many decisions we took at the November meeting. Meanwhile, attached 
>> as a preview are just Tables C96 (/Ivad/ attm) and C96a (/VQC/ flag 
>> configuration, so as agreed we are dropping the IMMT flag scheme for 
>> /VQC/) incorporating changes agreed last Nov.
>> Also, Sandy recently updated the {rdimma1} software 
>> (http://icoads.noaa.gov/software/rdimma1) to handle I/O for the 
>> finalized /Nocn/ attm (e.g. for WOD13), and, while I'll need to check 
>> with her on the logistics/timing, I believe she was also on the verge 
>> of enabling the /Ivad/ (and /Error)/ attms in their currently 
>> documented (ref. http://icoads.noaa.gov/ivad/IMMA-Rev-v14.pdf) form, 
>> so my suspicion is that we will have both the documentation (note: 
>> pending checking proposed new field abbreviations for uniqueness 
>> etc.) and software for the revised /Ivad/ attm as attached locked 
>> down fairly soon.
>> Sorry for such a long and complicated e-mail, but I hope it's helpful 
>> for getting moving on the scientific IVAD demonstration. -Scott
>> PS: Example Release 2.0 preconditioning rule for deck 187 (from 
>> http://icoads.noaa.gov/e-doc/other/dupelim_1946):
>> Deck 187: Japanese Whaling Fleet
>>        Rules:  If both wind speed (W) and direction (D) are missing 
>> (and not
>>        in the error attachment), and if 12-point wind direction is 
>> "12", and
>>        Beaufort force falls in the range 0-9 or is "-" or "+" (fields 
>> in the
>>        supplemental attachment corresponding to original TDF-11 
>> positions 114-
>>        115 and 116, respectively).  If these conditions are all true, 
>> make the
>>        following settings, ignoring any existing settings of the wind 
>> direction
>>        indicator (DI) and wind speed indicator (WI):
>>             i) DI to missing.
>>             ii) D to 360 degrees.
>>             iii) WI to 5 (Beaufort force).
>>             iv) Convert Beaufort force to W in meters per second 
>> using the
>>             midpoint of the "old" Beaufort equivalence scale, where 
>> 0-9 are
>>             mapped to meters per second according to Release 1, Table 
>> K5-5,
>>             "-" (equivalent to Beaufort force 10) is mapped to 26.8, 
>> and "+"
>>             (equivalent to Beaufort force 11 or higher) to 30.9.
>>        Background:  In conversion from the original card format into 
>> TDF-11,
>>        errors appear to have been made in failing to convert any 
>> direction
>>        code 36 (north in the 36-point scale) and its associated wind 
>> speed.
>>        However, a 12-point wind direction code and Beaufort force 
>> usually seem
>>        to be available in the supplemental attachment to provide a 
>> correction.
>>        [NOTE: This correction is approximate because D and W in 
>> TDF-11 were
>>        derived from separate fields, when extant, that appeared only 
>> in the
>>        original card format (36-point direction and speed in knots); the
>>        relationship between the two different sets of D and W fields 
>> in the
>>        original card format is unclear.  DI is set to missing because we
>>        currently lack a DI value for indication of the 12-point 
>> scale.  In
>>        addition, the 12-point wind direction code of "12" corresponds 
>> to a
>>        range of 335-4 degrees, whose midpoint is not 360 degrees.  
>> Moreover,
>>        Beaufort value "+" actually signifies force 11 or greater (up 
>> to 17),
>>        according to the original card deck reference manual; usage of the
>>        midpoint for force 11 is thus definitely an approximation.  
>> Assuming
>>        that the original card deck reference manual is correct, NCDC 
>> (1968)
>>        incorrectly documents the Beaufort value "+" as signifying 
>> force 11.]
>> ------------------------------------------------------------------------
>> This message (and any attachments) is for the recipient only. NERC is 
>> subject to the Freedom of Information Act 2000 and the contents of 
>> this email and any reply you make may be disclosed by NERC unless it 
>> is exempt from release under the Act. Any material supplied to NERC 
>> may be stored in an electronic records management system.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/icoads_usa/attachments/20140204/a4532893/attachment-0001.html 

More information about the icoads_usa mailing list