[Met_help] [rt.rap.ucar.edu #62155] History for Virtual temperature in PREPBUFR data files
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 16 16:31:05 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello MET team,
one of our users recently discovered that the temperature variable in the native NCEP PREPBUFR data files is actually virtual temperature. I did a spot-check of some of the observations in the PREPBUFR files and verified that this is indeed the case. After some digging through the NCEP documentation, I found the following regarding virtual temperature in the PREPBUFR processing:
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/Virtual_Temperature_Processing.htm
I looked at the ASCII output generated by the NCEP PREPBUFR decoder, and the second event in the PREPBUFR processing of the temperature field is coded (in most PREPBUFR messages I've looked at) with program code = 8 and reason code = 1. This coding corresponds to converting the dry bulb temperature to virtual temperature, as described in Table 14 of the PREPBUFR documentation:
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_14.htm
The temperature variable, however, remains coded as dry bulb temperature--labeled as "T" in the PREPBUFR messages--and users who are unaware of this conversion will therefore think they are looking at dry bulb temperature instead of virtual temperature in their research/analysis. I can only speculate why it's done this way at NCEP, but I'm guessing the temperature is converted to virtual temperature to make the PREPBUFR data compatible with the GDAS assimilation programs.
I'm curious if your group is aware of this. I see no mention of virtual temperature in the MET documentation, and the temperature variable is assigned grib code 11 (dry bulb temperature) in the pb2nc NetCDF output, when it actually should be grib code 12 (virtual temperature).
If you'd like to take a look at the same ASCII output I've been looking at, you can request a subset of the PREPBUFR data at our ds337.0 subsetting page, and specify ASCII output in your request.
Thanks in advance for your response.
Best regards,
- Tom
Thomas Cram
NCAR / CISL / Data Support Section
303-497-1217
tcram at ucar.edu
rda.ucar.edu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Virtual temperature in PREPBUFR data files
From: Thomas Cram
Time: Tue Jul 09 12:44:08 2013
Sorry, I forgot to include the link to our ds337.0 subsetting page.
Here it is:
http://rda.ucar.edu/datasets/ds337.0/index.html#forms/337_subset.php?_da=y
- Tom
On Jul 9, 2013, at 12:36 PM, "met_help at ucar.edu via RT"
<met_help at ucar.edu> wrote:
> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
> "Virtual temperature in PREPBUFR data files",
> a summary of which appears below.
>
> There is no need to reply to this message right now. Your ticket
has been
> assigned an ID of [rt.rap.ucar.edu #62155].
>
> Please include the string:
>
> [rt.rap.ucar.edu #62155]
>
> in the subject line of all future correspondence about this issue.
To do so,
> you may reply to this message.
>
> Thank you,
> met_help at ucar.edu
>
>
-------------------------------------------------------------------------
> Hello MET team,
>
> one of our users recently discovered that the temperature variable
in the native NCEP PREPBUFR data files is actually virtual
temperature. I did a spot-check of some of the observations in the
PREPBUFR files and verified that this is indeed the case. After some
digging through the NCEP documentation, I found the following
regarding virtual temperature in the PREPBUFR processing:
>
>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/Virtual_Temperature_Processing.htm
>
> I looked at the ASCII output generated by the NCEP PREPBUFR decoder,
and the second event in the PREPBUFR processing of the temperature
field is coded (in most PREPBUFR messages I've looked at) with program
code = 8 and reason code = 1. This coding corresponds to converting
the dry bulb temperature to virtual temperature, as described in Table
14 of the PREPBUFR documentation:
>
>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_14.htm
>
> The temperature variable, however, remains coded as dry bulb
temperature--labeled as "T" in the PREPBUFR messages--and users who
are unaware of this conversion will therefore think they are looking
at dry bulb temperature instead of virtual temperature in their
research/analysis. I can only speculate why it's done this way at
NCEP, but I'm guessing the temperature is converted to virtual
temperature to make the PREPBUFR data compatible with the GDAS
assimilation programs.
>
> I'm curious if your group is aware of this. I see no mention of
virtual temperature in the MET documentation, and the temperature
variable is assigned grib code 11 (dry bulb temperature) in the pb2nc
NetCDF output, when it actually should be grib code 12 (virtual
temperature).
>
> If you'd like to take a look at the same ASCII output I've been
looking at, you can request a subset of the PREPBUFR data at our
ds337.0 subsetting page, and specify ASCII output in your request.
>
> Thanks in advance for your response.
>
> Best regards,
> - Tom
>
> Thomas Cram
> NCAR / CISL / Data Support Section
> 303-497-1217
> tcram at ucar.edu
> rda.ucar.edu
>
>
>
>
>
>
Thomas Cram
NCAR / CISL / Data Support Section
303-497-1217
tcram at ucar.edu
rda.ucar.edu
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62155] AutoReply: Virtual temperature in PREPBUFR data files
From: John Halley Gotway
Time: Tue Jul 09 15:03:51 2013
Tom,
Thanks for the email. Yes, we became aware of this issue about 2
weeks ago. We've been in touch with some data assimilation folks that
have provided guidance for how to determine if the temperature
values should be interpreted as virtual or sensible temperature.
We're working on an update the pb2nc tool to detect this case and
convert the virtual temperature to sensible temperature before
storing it in the pb2nc output.
Once we've resolved the issue and worked up a fix, we'll send out a
notice to the registered MET user's describing the situation.
Thanks,
John Halley Gotway
met_help at ucar.edu
On 07/09/2013 12:44 PM, Thomas Cram via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62155 >
>
> Sorry, I forgot to include the link to our ds337.0 subsetting page.
Here it is:
http://rda.ucar.edu/datasets/ds337.0/index.html#forms/337_subset.php?_da=y
>
> - Tom
>
> On Jul 9, 2013, at 12:36 PM, "met_help at ucar.edu via RT"
<met_help at ucar.edu> wrote:
>
>> Greetings,
>>
>> This message has been automatically generated in response to the
>> creation of a trouble ticket regarding:
>> "Virtual temperature in PREPBUFR data files",
>> a summary of which appears below.
>>
>> There is no need to reply to this message right now. Your ticket
has been
>> assigned an ID of [rt.rap.ucar.edu #62155].
>>
>> Please include the string:
>>
>> [rt.rap.ucar.edu #62155]
>>
>> in the subject line of all future correspondence about this issue.
To do so,
>> you may reply to this message.
>>
>> Thank you,
>> met_help at ucar.edu
>>
>>
-------------------------------------------------------------------------
>> Hello MET team,
>>
>> one of our users recently discovered that the temperature variable
in the native NCEP PREPBUFR data files is actually virtual
temperature. I did a spot-check of some of the observations in the
PREPBUFR files and verified that this is indeed the case. After some
digging through the NCEP documentation, I found the following
regarding virtual temperature in the PREPBUFR processing:
>>
>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/Virtual_Temperature_Processing.htm
>>
>> I looked at the ASCII output generated by the NCEP PREPBUFR
decoder, and the second event in the PREPBUFR processing of the
temperature field is coded (in most PREPBUFR messages I've looked at)
with program code = 8 and reason code = 1. This coding corresponds to
converting the dry bulb temperature to virtual temperature, as
described in Table 14 of the PREPBUFR documentation:
>>
>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_14.htm
>>
>> The temperature variable, however, remains coded as dry bulb
temperature--labeled as "T" in the PREPBUFR messages--and users who
are unaware of this conversion will therefore think they are looking
at dry bulb temperature instead of virtual temperature in their
research/analysis. I can only speculate why it's done this way at
NCEP, but I'm guessing the temperature is converted to virtual
temperature to make the PREPBUFR data compatible with the GDAS
assimilation programs.
>>
>> I'm curious if your group is aware of this. I see no mention of
virtual temperature in the MET documentation, and the temperature
variable is assigned grib code 11 (dry bulb temperature) in the pb2nc
NetCDF output, when it actually should be grib code 12 (virtual
temperature).
>>
>> If you'd like to take a look at the same ASCII output I've been
looking at, you can request a subset of the PREPBUFR data at our
ds337.0 subsetting page, and specify ASCII output in your request.
>>
>> Thanks in advance for your response.
>>
>> Best regards,
>> - Tom
>>
>> Thomas Cram
>> NCAR / CISL / Data Support Section
>> 303-497-1217
>> tcram at ucar.edu
>> rda.ucar.edu
>>
>>
>>
>>
>>
>>
>
> Thomas Cram
> NCAR / CISL / Data Support Section
> 303-497-1217
> tcram at ucar.edu
> rda.ucar.edu
>
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62155] AutoReply: Virtual temperature in PREPBUFR data files
From: Thomas Cram
Time: Wed Jul 10 11:52:09 2013
Hi John,
I'm glad you're aware of this and working toward a fix. Would you mind
sharing your correspondence with NCEP regarding identifying sensible
temperature vs virtual temperature in the prepbufr messages? I would
like to fix this in our ds337.0 ASCII subsetting tool.
Thanks,
- Tom
Sent from my iPhone
On Jul 9, 2013, at 5:03 PM, "John Halley Gotway via RT"
<met_help at ucar.edu> wrote:
> Tom,
>
> Thanks for the email. Yes, we became aware of this issue about 2
weeks ago. We've been in touch with some data assimilation folks that
have provided guidance for how to determine if the temperature
> values should be interpreted as virtual or sensible temperature.
We're working on an update the pb2nc tool to detect this case and
convert the virtual temperature to sensible temperature before
> storing it in the pb2nc output.
>
> Once we've resolved the issue and worked up a fix, we'll send out a
notice to the registered MET user's describing the situation.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> On 07/09/2013 12:44 PM, Thomas Cram via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62155 >
>>
>> Sorry, I forgot to include the link to our ds337.0 subsetting page.
Here it is:
http://rda.ucar.edu/datasets/ds337.0/index.html#forms/337_subset.php?_da=y
>>
>> - Tom
>>
>> On Jul 9, 2013, at 12:36 PM, "met_help at ucar.edu via RT"
<met_help at ucar.edu> wrote:
>>
>>> Greetings,
>>>
>>> This message has been automatically generated in response to the
>>> creation of a trouble ticket regarding:
>>> "Virtual temperature in PREPBUFR data files",
>>> a summary of which appears below.
>>>
>>> There is no need to reply to this message right now. Your ticket
has been
>>> assigned an ID of [rt.rap.ucar.edu #62155].
>>>
>>> Please include the string:
>>>
>>> [rt.rap.ucar.edu #62155]
>>>
>>> in the subject line of all future correspondence about this issue.
To do so,
>>> you may reply to this message.
>>>
>>> Thank you,
>>> met_help at ucar.edu
>>>
>>>
-------------------------------------------------------------------------
>>> Hello MET team,
>>>
>>> one of our users recently discovered that the temperature variable
in the native NCEP PREPBUFR data files is actually virtual
temperature. I did a spot-check of some of the observations in the
PREPBUFR files and verified that this is indeed the case. After some
digging through the NCEP documentation, I found the following
regarding virtual temperature in the PREPBUFR processing:
>>>
>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/Virtual_Temperature_Processing.htm
>>>
>>> I looked at the ASCII output generated by the NCEP PREPBUFR
decoder, and the second event in the PREPBUFR processing of the
temperature field is coded (in most PREPBUFR messages I've looked at)
with program code = 8 and reason code = 1. This coding corresponds to
converting the dry bulb temperature to virtual temperature, as
described in Table 14 of the PREPBUFR documentation:
>>>
>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_14.htm
>>>
>>> The temperature variable, however, remains coded as dry bulb
temperature--labeled as "T" in the PREPBUFR messages--and users who
are unaware of this conversion will therefore think they are looking
at dry bulb temperature instead of virtual temperature in their
research/analysis. I can only speculate why it's done this way at
NCEP, but I'm guessing the temperature is converted to virtual
temperature to make the PREPBUFR data compatible with the GDAS
assimilation programs.
>>>
>>> I'm curious if your group is aware of this. I see no mention of
virtual temperature in the MET documentation, and the temperature
variable is assigned grib code 11 (dry bulb temperature) in the pb2nc
NetCDF output, when it actually should be grib code 12 (virtual
temperature).
>>>
>>> If you'd like to take a look at the same ASCII output I've been
looking at, you can request a subset of the PREPBUFR data at our
ds337.0 subsetting page, and specify ASCII output in your request.
>>>
>>> Thanks in advance for your response.
>>>
>>> Best regards,
>>> - Tom
>>>
>>> Thomas Cram
>>> NCAR / CISL / Data Support Section
>>> 303-497-1217
>>> tcram at ucar.edu
>>> rda.ucar.edu
>>
>> Thomas Cram
>> NCAR / CISL / Data Support Section
>> 303-497-1217
>> tcram at ucar.edu
>> rda.ucar.edu
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62155] AutoReply: Virtual temperature in PREPBUFR data files
From: John Halley Gotway
Time: Tue Jul 16 16:30:37 2013
Hello Tom,
I wanted to let you know that we just posted a bugfix for the PB2NC
tool to address this virtual/sensible temperature problem in PREPBUFR
files. You can find it here:
http://www.dtcenter.org/met/users/support/known_issues/METv4.1/index.php
Also, here's listed below is correspondence I received from NCEP on
this issue.
Thanks,
John
-------------------------------------------------------------------------------
John,
Yes your logic all sounds correct. Also, all aircraft temperatures
are sensible so you don't have to check those. As you note, RASSDA
temperatures are always virtual from the beginning. There
should never be any temperature reports in SYNDATA so I'm not sure why
you would be seeing TPC=8, TRC=3 in those. The SATEMP (satellite
temperature profile) reports can have TPC=8, TRC=3 (because
their temperatures are virtual) but these are only in the CDAS
PREPBUFR files.
Dennis
On 7/12/2013 4:42 PM, John Halley Gotway wrote:
> Perry,
>
> Thanks so much. That's very helpful. I'll wait for confirmation
from Dennis or Jeff.
>
> Thanks,
> John
>
> On 07/12/2013 02:18 PM, Perry Shafran - NOAA Affiliate wrote:
>> Hi, John,
>>
>> I believe that I use the same logic in my own verification codes.
I did
>> not write that part of the code so I am unsure, though, so just to
make
>> sure I am cc'ing members of our obsproc team, Dennis Keyser and
Jeff
>> Whiting, who can clarify things a little better for you.
>>
>> Thanks!
>>
>> Perry
>>
>>
>> On Fri, Jul 12, 2013 at 4:13 PM, John Halley Gotway
<johnhg at ucar.edu> wrote:
>>
>>> Hello Perry,
>>>
>>> I'm John Halley Gotway. I work at NCAR as a developer for the
Model
>>> Evaluation Tools (MET) software. It's recently come to our
attention that
>>> we're likely handling temperature observations from PREPBUFR
files
>>> incorrectly. We've just been using the temperature observations
from the
>>> top of the event stack, but often those are observations of
virtual
>>> temperature rather than sensible temperature. For verification,
we should
>>> be using sensible temperatures, not virtual ones.
>>>
>>> After reading the PREPBUFR documentation, I believe I have a
handle on the
>>> logic we should be applying. But before distributing a patch to
2000
>>> registered MET users, I'd really like someone at NCEP to review
the logic
>>> and verify that it's sound.
>>>
>>> If you're the right person to do this, that's great. If not,
could you
>>> please point me in the right direction?
>>>
>>> This logic is based on PREPBUFR table 14, which describes the
virtual
>>> temperature processing step (VIRTMP program code == 8):
>>> http://www.emc.ncep.noaa.gov/**mmb/data_processing/prepbufr.**
>>>
doc/table_14.htm<http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_14.htm>
>>>
>>> The valid reason codes associated with the VIRTMP program code
are 0, 1,
>>> 2, 3, 4, 6, and 7. All of those reason codes except for 3
indicate that
>>> the temperature value has been converted from sensible to virtual
>>> temperature. A reason code of 3 indicates that the observation
value was
>>> already virtual temperature. My suggested plan of action is
this:
>>> - For temperature observations, if the program code is 8 with
a reason
>>> code of 3, do not use that observation since it's virtual.
>>> - If the program code is 8 with any reason code other than 3,
step down
>>> the event stack to find a version of the observation with a
program code
>>> that is not 8.
>>>
>>> I looked through some PREPBUFR data and found that the only
message types
>>> with program code 8/reason code 3 are SYNDAT and RASSDA. So we
won't use
>>> any of those temperature observations since they're virtual to
begin with.
>>> For the other ones, by stepping down the event stack, we should
be able to
>>> find a version of that observation that is sensible temperature
before it's
>>> gone through the VIRTMP processing step.
>>>
>>> Is that logic correct? We really want to ensure that we're not
using
>>> virtual temperature for verification, only sensible temperature.
>>>
>>> I really appreciate the help on this.
>>>
>>> Thanks,
>>> John Halley Gotway
>>> johnhg at ucar.edu
------------------------------------------------
More information about the Met_help
mailing list