[Met_help] [rt.rap.ucar.edu #90445] History for FW: Winds in Prepbufr File
John Halley Gotway via RT
met_help at ucar.edu
Tue Jun 4 10:00:23 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, I am having an issue getting point stat to verify 10M winds using
derived wind speed in prepbufr ob file. It throws out the majority of obs
due to "obs type". I think the problem could be in the prepbufr file.
Attached in my PB2NC config file and a prepbufr file I am using. I am
using this file just fine to verify upper level winds but not 10 meter
winds. Using ncdump of the various lvl, elv variables, I don't
specifically see 10 Meter winds. According to the users manual, since I
left most of the selections blank, I should have all levels in the file.
Is 10Meter winds in the file, and if not, what didn't I set correctly.
Thanks
Bob
P.S. If the ob file is too big to go in email, I will place on AMRDEC.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: FW: Winds in Prepbufr File
From: John Halley Gotway
Time: Mon Jun 03 16:20:45 2019
Bob,
I agree. You're NetCDF point observation file does not include 10-m
winds. Here's how I determined that:
(1) I ran an Rscript to convert that NetCDF point observation file to
ascii
to make it easier to inspect:
* Rscript met-8.1/share/met/Rscripts/pntnc2ascii.R 2019053012.nc
<http://2019053012.nc> > 2019053012.txt*
(2) I grepped for UGRD, VGRD, or WIND from that file and looked at the
message type:
* egrep "UGRD|VGRD|WIND" 2019053012.txt | awk '{print $1}' | sort -u
ADPUPA*
They're only ADPUPA, which is upper-air.
Here's one possibility... perhaps you're using global GDAS PrepBUFR
data in
which they set the quality flag for surface obs to 9 to avoid
assimilating
them. Here's a note from the MET website about that:
(https://dtcenter.org/community-code/model-evaluation-tools-met/input-
data)
**** NOTE: The quality flag values for surface observations in GDAS
PrepBufr files are set to a value of 9 to prevent them from being used
in
the data assimilation process. To retain them in the PB2NC tool, the
quality mark threshold in the configuration file must be set to at
least 9.*
You could try configuring PB2NC to use:
*quality_mark_thresh = 9;*
And see if that causes any additional point observations to show up.
Thanks,
John
On Mon, Jun 3, 2019 at 2:41 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 03 14:40:31 2019: Request 90445 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: FW: Winds in Prepbufr File
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90445 >
>
>
> John, I am having an issue getting point stat to verify 10M winds
using
> derived wind speed in prepbufr ob file. It throws out the majority
of obs
> due to "obs type". I think the problem could be in the prepbufr
file.
> Attached in my PB2NC config file and a prepbufr file I am using. I
am
> using this file just fine to verify upper level winds but not 10
meter
> winds. Using ncdump of the various lvl, elv variables, I don't
> specifically see 10 Meter winds. According to the users manual,
since I
> left most of the selections blank, I should have all levels in the
file.
> Is 10Meter winds in the file, and if not, what didn't I set
correctly.
>
>
>
> Thanks
>
> Bob
>
>
>
> P.S. If the ob file is too big to go in email, I will place on
AMRDEC.
>
>
>
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds in Prepbufr File
From: robert.craig.2 at us.af.mil
Time: Tue Jun 04 09:01:41 2019
John, I changed the quality_mark_thresh to 9 and then 10 but I don't
think I got 10 meter wind speed. I don't have Rscript to convert to
the netcdf file to an ASCII file. What variable in the netcdf file
should show the 10 meter level: obs_lvl, obs_elv, or something else?
When I ran point stat against the new netcdf file, I still reject all
obs for wind speed at 10 meters due to obs_type.
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, June 3, 2019 5:21 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds in
Prepbufr File
Bob,
I agree. You're NetCDF point observation file does not include 10-m
winds. Here's how I determined that:
(1) I ran an Rscript to convert that NetCDF point observation file to
ascii to make it easier to inspect:
* Rscript met-8.1/share/met/Rscripts/pntnc2ascii.R 2019053012.nc
<https://usg01.safelinks.protection.office365.us/?url=http%3A%2F%2F2019053012.nc&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=8DfPfa7P1aYNJz47S2NxCK5bHHJ6aYAyxtVZFvVqvHU%3D&reserved=0>
> 2019053012.txt*
(2) I grepped for UGRD, VGRD, or WIND from that file and looked at the
message type:
* egrep "UGRD|VGRD|WIND" 2019053012.txt | awk '{print $1}' | sort -u
ADPUPA*
They're only ADPUPA, which is upper-air.
Here's one possibility... perhaps you're using global GDAS PrepBUFR
data in which they set the quality flag for surface obs to 9 to avoid
assimilating them. Here's a note from the MET website about that:
(https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Finput-
data&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=6PEacxnaH774EeojYV8KsbIGFeSEPTTiJJn3x5u6vDU%3D&reserved=0)
**** NOTE: The quality flag values for surface observations in GDAS
PrepBufr files are set to a value of 9 to prevent them from being used
in the data assimilation process. To retain them in the PB2NC tool,
the quality mark threshold in the configuration file must be set to at
least 9.*
You could try configuring PB2NC to use:
*quality_mark_thresh = 9;*
And see if that causes any additional point observations to show up.
Thanks,
John
On Mon, Jun 3, 2019 at 2:41 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 03 14:40:31 2019: Request 90445 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: FW: Winds in Prepbufr File
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
>
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Frt.
>
rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D90445&data=02%7C0
>
1%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C83
>
31b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=
> j1ZB6fPD%2FR360g8uKJhy%2Bopsd49eyCw%2B4dEZ95iGXzc%3D&reserved=0
>
>
>
> John, I am having an issue getting point stat to verify 10M winds
> using derived wind speed in prepbufr ob file. It throws out the
> majority of obs due to "obs type". I think the problem could be in
the prepbufr file.
> Attached in my PB2NC config file and a prepbufr file I am using. I
am
> using this file just fine to verify upper level winds but not 10
meter
> winds. Using ncdump of the various lvl, elv variables, I don't
> specifically see 10 Meter winds. According to the users manual,
since
> I left most of the selections blank, I should have all levels in the
file.
> Is 10Meter winds in the file, and if not, what didn't I set
correctly.
>
>
>
> Thanks
>
> Bob
>
>
>
> P.S. If the ob file is too big to go in email, I will place on
AMRDEC.
>
>
>
>
>
>
>
------------------------------------------------
Subject: FW: Winds in Prepbufr File
From: John Halley Gotway
Time: Tue Jun 04 09:23:52 2019
Bob,
I believe the culprit here preventing you from retrieving 10-m winds
is
your setting for "level_category":
level_category = [0,1,3];
Changing that to an empty list (level_category=[];) results in 10-m
winds
being retained. Below is a description of that config file option
from the
config README file. I'm not sure exactly how the level category is
set for
10-m winds... perhaps to a value of 4 or a 6?
Thanks,
John
//
// The "level_category" entry is an array of integers specifying which
level
// categories should be retained:
// 0 = Surface level (mass reports only)
// 1 = Mandatory level (upper-air profile reports)
// 2 = Significant temperature level (upper-air profile reports)
// 2 = Significant temperature and winds-by-pressure level
// (future combined mass and wind upper-air reports)
// 3 = Winds-by-pressure level (upper-air profile reports)
// 4 = Winds-by-height level (upper-air profile reports)
// 5 = Tropopause level (upper-air profile reports)
// 6 = Reports on a single level
// (e.g., aircraft, satellite-wind, surface wind,
// precipitable water retrievals, etc.)
// 7 = Auxiliary levels generated via interpolation from spanning
levels
// (upper-air profile reports)
// An empty list indicates that all should be retained.
//
//
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
//
level_category = [];
On Tue, Jun 4, 2019 at 9:01 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90445 >
>
> John, I changed the quality_mark_thresh to 9 and then 10 but I don't
think
> I got 10 meter wind speed. I don't have Rscript to convert to the
netcdf
> file to an ASCII file. What variable in the netcdf file should show
the 10
> meter level: obs_lvl, obs_elv, or something else? When I ran point
stat
> against the new netcdf file, I still reject all obs for wind speed
at 10
> meters due to obs_type.
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, June 3, 2019 5:21 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds in
> Prepbufr File
>
> Bob,
>
> I agree. You're NetCDF point observation file does not include 10-m
> winds. Here's how I determined that:
>
> (1) I ran an Rscript to convert that NetCDF point observation file
to
> ascii to make it easier to inspect:
> * Rscript met-8.1/share/met/Rscripts/pntnc2ascii.R 2019053012.nc
> <
>
https://usg01.safelinks.protection.office365.us/?url=http%3A%2F%2F2019053012.nc&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=8DfPfa7P1aYNJz47S2NxCK5bHHJ6aYAyxtVZFvVqvHU%3D&reserved=0>
> > 2019053012.txt*
>
> (2) I grepped for UGRD, VGRD, or WIND from that file and looked at
the
> message type:
>
> * egrep "UGRD|VGRD|WIND" 2019053012.txt | awk '{print $1}' | sort
-u
> ADPUPA*
>
> They're only ADPUPA, which is upper-air.
>
> Here's one possibility... perhaps you're using global GDAS PrepBUFR
data
> in which they set the quality flag for surface obs to 9 to avoid
> assimilating them. Here's a note from the MET website about that:
> (
>
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Finput-
data&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=6PEacxnaH774EeojYV8KsbIGFeSEPTTiJJn3x5u6vDU%3D&reserved=0
> )
> **** NOTE: The quality flag values for surface observations in GDAS
> PrepBufr files are set to a value of 9 to prevent them from being
used in
> the data assimilation process. To retain them in the PB2NC tool, the
> quality mark threshold in the configuration file must be set to at
least 9.*
>
>
> You could try configuring PB2NC to use:
> *quality_mark_thresh = 9;*
>
> And see if that causes any additional point observations to show up.
>
> Thanks,
> John
>
> On Mon, Jun 3, 2019 at 2:41 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jun 03 14:40:31 2019: Request 90445 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: FW: Winds in Prepbufr File
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> >
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Frt.
> >
rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D90445&data=02%7C0
> >
1%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C83
> >
31b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdata=
> >
j1ZB6fPD%2FR360g8uKJhy%2Bopsd49eyCw%2B4dEZ95iGXzc%3D&reserved=0 >
> >
> >
> > John, I am having an issue getting point stat to verify 10M winds
> > using derived wind speed in prepbufr ob file. It throws out the
> > majority of obs due to "obs type". I think the problem could be
in the
> prepbufr file.
> > Attached in my PB2NC config file and a prepbufr file I am using.
I am
> > using this file just fine to verify upper level winds but not 10
meter
> > winds. Using ncdump of the various lvl, elv variables, I don't
> > specifically see 10 Meter winds. According to the users manual,
since
> > I left most of the selections blank, I should have all levels in
the
> file.
> > Is 10Meter winds in the file, and if not, what didn't I set
correctly.
> >
> >
> >
> > Thanks
> >
> > Bob
> >
> >
> >
> > P.S. If the ob file is too big to go in email, I will place on
AMRDEC.
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds in Prepbufr File
From: robert.craig.2 at us.af.mil
Time: Tue Jun 04 09:52:32 2019
John, that was it - I am now getting 10 meter winds. Thanks
I will be getting back to trying what you recommended for getting MET
to better find the correct model record if several match ->
GRIB2_ipdtmpl_index. Keep that ticket open a little longer.
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, June 4, 2019 10:24 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds
in Prepbufr File
Bob,
I believe the culprit here preventing you from retrieving 10-m winds
is your setting for "level_category":
level_category = [0,1,3];
Changing that to an empty list (level_category=[];) results in 10-m
winds being retained. Below is a description of that config file
option from the config README file. I'm not sure exactly how the
level category is set for 10-m winds... perhaps to a value of 4 or a
6?
Thanks,
John
//
// The "level_category" entry is an array of integers specifying which
level // categories should be retained:
// 0 = Surface level (mass reports only)
// 1 = Mandatory level (upper-air profile reports)
// 2 = Significant temperature level (upper-air profile reports)
// 2 = Significant temperature and winds-by-pressure level
// (future combined mass and wind upper-air reports)
// 3 = Winds-by-pressure level (upper-air profile reports)
// 4 = Winds-by-height level (upper-air profile reports)
// 5 = Tropopause level (upper-air profile reports)
// 6 = Reports on a single level
// (e.g., aircraft, satellite-wind, surface wind,
// precipitable water retrievals, etc.)
// 7 = Auxiliary levels generated via interpolation from spanning
levels
// (upper-air profile reports)
// An empty list indicates that all should be retained.
//
//
https://usg01.safelinks.protection.office365.us/?url=http%3A%2F%2Fwww.emc.ncep.noaa.gov%2Fmmb%2Fdata_processing%2Fprepbufr.doc%2Ftable_1.htm&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C3377d6b35fba4136a88108d6e900bbc1%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C1%7C636952586725563696&sdata=0RXTOPdZTlVySFI7TLBaXnSpal8a9AfvfDdsikT2eSg%3D&reserved=0
//
level_category = [];
On Tue, Jun 4, 2019 at 9:01 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL:
>
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Frt.
>
rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D90445&data=02%7C0
>
1%7Crobert.craig.2%40us.af.mil%7C3377d6b35fba4136a88108d6e900bbc1%7C83
>
31b18d2d8748efa35fac8818ebf9b4%7C1%7C1%7C636952586725563696&sdata=
> fg3gQllS%2BDhvOLTIXb4igadaUMhVPQ1n60HcwSsRel0%3D&reserved=0 >
>
> John, I changed the quality_mark_thresh to 9 and then 10 but I don't
think
> I got 10 meter wind speed. I don't have Rscript to convert to the
netcdf
> file to an ASCII file. What variable in the netcdf file should show
> the 10 meter level: obs_lvl, obs_elv, or something else? When I ran
> point stat against the new netcdf file, I still reject all obs for
> wind speed at 10 meters due to obs_type.
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, June 3, 2019 5:21 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #90445] FW: Winds in
> Prepbufr File
>
> Bob,
>
> I agree. You're NetCDF point observation file does not include 10-m
> winds. Here's how I determined that:
>
> (1) I ran an Rscript to convert that NetCDF point observation file
to
> ascii to make it easier to inspect:
> * Rscript met-8.1/share/met/Rscripts/pntnc2ascii.R 2019053012.nc
> <
>
https://usg01.safelinks.protection.office365.us/?url=http%3A%2F%2F2019
>
053012.nc&data=02%7C01%7Crobert.craig.2%40us.af.mil%7C3377d6b35fba
>
4136a88108d6e900bbc1%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C6369
>
52586725563696&sdata=WzDJxg31khzyyBNaVaIvGUwrOfewJMj3ZqSiV4gH6m0%3
> D&reserved=0>
> > 2019053012.txt*
>
> (2) I grepped for UGRD, VGRD, or WIND from that file and looked at
the
> message type:
>
> * egrep "UGRD|VGRD|WIND" 2019053012.txt | awk '{print $1}' | sort
-u
> ADPUPA*
>
> They're only ADPUPA, which is upper-air.
>
> Here's one possibility... perhaps you're using global GDAS PrepBUFR
> data in which they set the quality flag for surface obs to 9 to
avoid
> assimilating them. Here's a note from the MET website about that:
> (
>
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Fdtc
> enter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Finput-
data&a
>
mp;data=02%7C01%7Crobert.craig.2%40us.af.mil%7C3377d6b35fba4136a88108d
>
6e900bbc1%7C8331b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636952586725563
>
696&sdata=WuzHLxLF5ZVyvdPogPDzXuKwfICBJd5%2FnCPZrRmoTtQ%3D&res
> erved=0
> )
> **** NOTE: The quality flag values for surface observations in GDAS
> PrepBufr files are set to a value of 9 to prevent them from being
used
> in the data assimilation process. To retain them in the PB2NC tool,
> the quality mark threshold in the configuration file must be set to
at
> least 9.*
>
>
> You could try configuring PB2NC to use:
> *quality_mark_thresh = 9;*
>
> And see if that causes any additional point observations to show up.
>
> Thanks,
> John
>
> On Mon, Jun 3, 2019 at 2:41 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jun 03 14:40:31 2019: Request 90445 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: FW: Winds in Prepbufr File
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> >
https://usg01.safelinks.protection.office365.us/?url=https%3A%2F%2Frt.
> >
rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D90445&data=02%7
> > C0
> >
1%7Crobert.craig.2%40us.af.mil%7C6931ababa66e4f94ccb608d6e871c836%7C
> > 83
> >
31b18d2d8748efa35fac8818ebf9b4%7C1%7C0%7C636951972755240363&sdat
> > a=
> >
j1ZB6fPD%2FR360g8uKJhy%2Bopsd49eyCw%2B4dEZ95iGXzc%3D&reserved=0
> > >
> >
> >
> > John, I am having an issue getting point stat to verify 10M winds
> > using derived wind speed in prepbufr ob file. It throws out the
> > majority of obs due to "obs type". I think the problem could be
in
> > the
> prepbufr file.
> > Attached in my PB2NC config file and a prepbufr file I am using.
I am
> > using this file just fine to verify upper level winds but not 10
meter
> > winds. Using ncdump of the various lvl, elv variables, I don't
> > specifically see 10 Meter winds. According to the users manual,
> > since I left most of the selections blank, I should have all
levels
> > in the
> file.
> > Is 10Meter winds in the file, and if not, what didn't I set
correctly.
> >
> >
> >
> > Thanks
> >
> > Bob
> >
> >
> >
> > P.S. If the ob file is too big to go in email, I will place on
AMRDEC.
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list