[Met_help] [rt.rap.ucar.edu #62542] History for Test of ASCII2NC on little_r file
John Halley Gotway via RT
met_help at ucar.edu
Wed Aug 28 09:40:32 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I ran ASCII2NC as follows:
ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000 qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log test7 -v 4
It appears that the LittleRHandler is working to indicate that it recognizes the input file as a little_r file and then generates the NetCDF output file.
The output file attached as the text file " test7_ncdump" appears to contain good data, but the obs_qty information looks odd in places.
I've also attached the log file which contain extensive WARNINGS, but no ERROR.
Can you explain the odd numbers in the obs_qty array? I was expecting numbers like 0, 1, 2, etc but not "512" or "16384". Also any help you can provide on the reasons for the WARNINGS would be appreciated.
Thanks.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file
From: John Halley Gotway
Time: Thu Aug 08 10:29:34 2013
John,
Hmmm, good question. Let me give you some background. We only
recently began providing support for little_r data, which was driven
by a modelling project in which the DTC was involved. Our test
data was limited to the data available for that project. Based on
that data, we defined the following mappings:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
Instances of the little_r input strings on the left are mapped to the
message types used in the ASCII2NC output on the right. Looking at
the data you sent me, I see that your little_r data includes
the following 22 message types:
"FM-132"
"FM-15_AIRNow"
"FM-15_APRSWXNET"
"FM-15_ASOS"
"FM-15_CA-Hydro"
"FM-15_CODOT"
"FM-15_DDMET"
"FM-15_GPSMET"
"FM-15_HADS"
"FM-15_MAP"
"FM-15_MesoWest"
"FM-15_NERRS"
"FM-15_NonFedAWOS"
"FM-15_NOS-NWLON"
"FM-15_NOS-PORTS"
"FM-15_NVDOT"
"FM-15_OTHER-MTR"
"FM-15_RAWS"
"FM-15_SURFRAD"
"FM-35"
"FM-96"
"FM-97_TAMDAR"
None of those match the strings we were expecting. So instead of
mapping them to the PREPBUFR message types, we just wrote the strings
that were input. The impact is that when you use this data in
Point-Stat, you'll need to list those message types separately and get
output for each one.
I see several questions here:
(1) Is it reasonable for us to figure out the super-set of all message
types we can expect to see in little_r data?
(2) If yes, what is that super-set?
(3) If no, is it OK for us to just use in the input string in the
output? Or should we provide a way for users to define a mapping of
input little_r strings to output message types? We could do that
by providing a configuration file for the ASCII2NC tool - which is
already used for ASCII SURFRAD data.
Do you have any perspectives on these issues?
Regarding the quality control values, yes, I agree, they are odd.
Unfortunately, I don't have much information for you. I would suggest
verifying that the values used in the little_r files match the
values that are showing up in the MET output. But making sense of
them is up to you. In Point-Stat, you can provide a list of quality
control strings to use in the obs_quality parameter. For the
DTC project using little_r data, here's the list of strings we used:
obs_quality = [
"3", "4", "5", "6", "7", "8",
"9", "10",
"100003", "100004", "100005", "100006", "100007", "100008",
"100009", "100010",
"200003", "200004", "200005", "200006", "200007", "200008",
"200009", "200010",
"300003", "300004", "300005", "300006", "300007", "300008",
"300009", "300010",
"400003", "400004", "400005", "400006", "400007", "400008",
"400009", "400010",
"500003", "500004", "500005", "500006", "500007", "500008",
"500009", "500010",
"600003", "600004", "600005", "600006", "600007", "600008",
"600009", "600010"
];
But it looks like these are not the same values that are showing up in
your data.
Thanks,
John
On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>
> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Test of ASCII2NC on little_r file
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
>
> I ran ASCII2NC as follows:
>
> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log test7
-v 4
>
> It appears that the LittleRHandler is working to indicate that it
recognizes the input file as a little_r file and then generates the
NetCDF output file.
>
> The output file attached as the text file " test7_ncdump" appears to
contain good data, but the obs_qty information looks odd in places.
>
> I've also attached the log file which contain extensive WARNINGS,
but no ERROR.
>
> Can you explain the odd numbers in the obs_qty array? I was
expecting numbers like 0, 1, 2, etc but not "512" or "16384". Also any
help you can provide on the reasons for the WARNINGS would be
appreciated.
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Thu Aug 08 13:14:55 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
Thanks for this info, your analysis of our results and opening the
opportunity
for a dialog on this topic.
I think of observational data in the following categories:
Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
which I use the verify model forecasts for surface met variables
(2meter and
10meter AGL) using Point-Stat.
Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
variables at various pressure levels using Point-Stat.
My thoughts to follow are not the final word on this and subject to
input from
other ARL colleagues which I will coordinate before giving you the
final
answers.
Lump all the sources of surface observations together to produce error
statistics for surface variables and do the same for upper air
observations
and their error statistics. Then, break out the error statistics by
message
type like Point-Stat does now, so you could isolate a particular type
(observation source/type) which might be weighing heavily in
contributing to a
particularly large overall error stat.
That said, I don't want to generate a big change for the Point-Stat
tool. I
may be forgetting some features of Stat-Analysis which would allow me
to
aggregate/stratify the error stats in order to show the stats I
mentioned
above.
I'm tending towards the latter suggestion you made in (3) about the
user
setting the mapping of input little_r strings to output message types.
That
way I could control the generation of error stats in Point-Stat to
adhere to
the groupings I mentioned above. Then, if the need arose, I could
rerun the
ASCII2NC with the mapping changed to regroup the little_r strings
differently.
The following example describes one such general mapping for surface
verification:
MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
In the above mapping, I'm assuming that the little_r strings I
selected are
all observations of surface met variables (2 and 10 meter AGL). I
didn't
select all the strings which were in the data, but just enough to give
you and
others an idea. I also assume that I obtained MADIS observations for
the
additional strings (synop, ship, metar, buoy) which were not present
in the
iittle_r file we used in this test.
Then for upper air verification, I would use the following:
MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
The above list could be expanded with more types of upper air data,
but I
think you get the idea. What I've done with this is to set up the
generation
of error stats in Point-Stat so that I get overall error stats which
were
derived from all possible sources of upper air observations. As long
as the
source provides measurements of met variables at the various pressure
levels
just as Point-Stat handles now, then this should work for the overall
stats.
For these stats I'm trying to capitalize on acquiring the maximum
number of
observations possible for upper air verification which often suffers
from a
paucity of "ground" truth data.
Then, I would have the option to go back and re-characterize the
various types
of observations into smaller groupings which focus in on trying to
group based
on the specific type of measurement used.
For example break out the ship data and using a different mapping
which
characterizes it as "SFCSHP" so that its stats are calculated
separately in
Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
That way you can evaluate the contribution/weight of those stats
towards the
overall stats. Any number of different mappings are possible by re-
running
ASCII2NC with different configuration file settings.
For surface observations where there are often large numbers of
measurements
from disparate sensor types, we are also interested in trying break
down the
verification and restrict the scoring to subdomains and having the
ability to
break out the stats by sensor type would be of value for these more
in-depth
assessments.
Regard the quality control values, I'm not sure what to think on this
yet.
Hopefully others can chime in here.
I would like the opportunity to discuss this among colleagues here in
ARL so
that I'm not misinterpreting anything here or am making invalid
assumptions.
I'm sure others have some good input to make about the statistical
implications of grouping the observation types the way I did. I have
probably
omitted the kind of stratifications which can be made with Stat-
Analysis which
might factor into this discussion.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, August 08, 2013 10:30 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
(US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
John,
Hmmm, good question. Let me give you some background. We only
recently began
providing support for little_r data, which was driven by a modelling
project
in which the DTC was involved. Our test data was limited to the data
available for that project. Based on that data, we defined the
following
mappings:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
Instances of the little_r input strings on the left are mapped to the
message
types used in the ASCII2NC output on the right. Looking at the data
you sent
me, I see that your little_r data includes the following 22 message
types:
"FM-132"
"FM-15_AIRNow"
"FM-15_APRSWXNET"
"FM-15_ASOS"
"FM-15_CA-Hydro"
"FM-15_CODOT"
"FM-15_DDMET"
"FM-15_GPSMET"
"FM-15_HADS"
"FM-15_MAP"
"FM-15_MesoWest"
"FM-15_NERRS"
"FM-15_NonFedAWOS"
"FM-15_NOS-NWLON"
"FM-15_NOS-PORTS"
"FM-15_NVDOT"
"FM-15_OTHER-MTR"
"FM-15_RAWS"
"FM-15_SURFRAD"
"FM-35"
"FM-96"
"FM-97_TAMDAR"
None of those match the strings we were expecting. So instead of
mapping them
to the PREPBUFR message types, we just wrote the strings that were
input. The
impact is that when you use this data in Point-Stat, you'll need to
list those
message types separately and get output for each one.
I see several questions here:
(1) Is it reasonable for us to figure out the super-set of all message
types
we can expect to see in little_r data?
(2) If yes, what is that super-set?
(3) If no, is it OK for us to just use in the input string in the
output? Or
should we provide a way for users to define a mapping of input
little_r
strings to output message types? We could do that by providing a
configuration file for the ASCII2NC tool - which is already used for
ASCII
SURFRAD data.
Do you have any perspectives on these issues?
Regarding the quality control values, yes, I agree, they are odd.
Unfortunately, I don't have much information for you. I would suggest
verifying that the values used in the little_r files match the values
that are
showing up in the MET output. But making sense of them is up to you.
In
Point-Stat, you can provide a list of quality control strings to use
in the
obs_quality parameter. For the DTC project using little_r data,
here's the
list of strings we used:
obs_quality = [
"3", "4", "5", "6", "7", "8",
"9",
"10",
"100003", "100004", "100005", "100006", "100007", "100008",
"100009",
"100010",
"200003", "200004", "200005", "200006", "200007", "200008",
"200009",
"200010",
"300003", "300004", "300005", "300006", "300007", "300008",
"300009",
"300010",
"400003", "400004", "400005", "400006", "400007", "400008",
"400009",
"400010",
"500003", "500004", "500005", "500006", "500007", "500008",
"500009",
"500010",
"600003", "600004", "600005", "600006", "600007", "600008",
"600009",
"600010"
];
But it looks like these are not the same values that are showing up in
your
data.
Thanks,
John
On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>
> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Test of ASCII2NC on little_r file
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
> >
>
>
> I ran ASCII2NC as follows:
>
> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
> -v 4
>
> It appears that the LittleRHandler is working to indicate that it
recognizes
> the input file as a little_r file and then generates the NetCDF
output file.
>
> The output file attached as the text file " test7_ncdump" appears to
contain
> good data, but the obs_qty information looks odd in places.
>
> I've also attached the log file which contain extensive WARNINGS,
but no
> ERROR.
>
> Can you explain the odd numbers in the obs_qty array? I was
expecting
> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
> provide on the reasons for the WARNINGS would be appreciated.
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: brian.p.reen.civ at mail.mil
Time: Thu Aug 08 13:22:31 2013
Classification: UNCLASSIFIED
Caveats: NONE
Regarding the obs_qty array, these are the quality control flags used
by the little_r format. A brief description can be found under the
heading QCFlags at:
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
A more complete description in regard to output from standard Obsgrid
is:
2 = pressure derived from the first-guess height field (e.g. GFS)
4 = pressure derived from standard atmosphere assumption and height
8 = pressure derived from standard atmosphere assumption and
temperature
16 = Temperature/Dewpoint are zero
32 = Calm wind
64 = Negative wind speed
128 = Wind direction did not fall between 0 and 360 degrees
[129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
256 = Value was vertically interpolated (to get to level with first-
guess field to QC against)
512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
1024 = Temperature appeared to have the wrong sign based on adjacent
temperatures in the vertical profile so OBSGRID flipped the sign of
the temperature and adjusted dewpoint/RH
2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
4096 = Spike in wind (results in OBSGRID marking winds as missing)
8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
16384 = No buddies (nearby observations) were found to do a buddy
check
32768 = Fails error max check (check against first guess field)
65536 = Fails buddy check (check against nearby obs)
131072 = Observation outside of domain
The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
Brian
-----Original Message-----
From: Raby, John W CIV USARMY ARL (US)
Sent: Thursday, August 08, 2013 3:15 PM
To: met_help at ucar.edu
Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
Classification: UNCLASSIFIED
Caveats: NONE
John -
Thanks for this info, your analysis of our results and opening the
opportunity
for a dialog on this topic.
I think of observational data in the following categories:
Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
which I use the verify model forecasts for surface met variables
(2meter and
10meter AGL) using Point-Stat.
Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
variables at various pressure levels using Point-Stat.
My thoughts to follow are not the final word on this and subject to
input from
other ARL colleagues which I will coordinate before giving you the
final
answers.
Lump all the sources of surface observations together to produce error
statistics for surface variables and do the same for upper air
observations
and their error statistics. Then, break out the error statistics by
message
type like Point-Stat does now, so you could isolate a particular type
(observation source/type) which might be weighing heavily in
contributing to a
particularly large overall error stat.
That said, I don't want to generate a big change for the Point-Stat
tool. I
may be forgetting some features of Stat-Analysis which would allow me
to
aggregate/stratify the error stats in order to show the stats I
mentioned
above.
I'm tending towards the latter suggestion you made in (3) about the
user
setting the mapping of input little_r strings to output message types.
That
way I could control the generation of error stats in Point-Stat to
adhere to
the groupings I mentioned above. Then, if the need arose, I could
rerun the
ASCII2NC with the mapping changed to regroup the little_r strings
differently.
The following example describes one such general mapping for surface
verification:
MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
In the above mapping, I'm assuming that the little_r strings I
selected are
all observations of surface met variables (2 and 10 meter AGL). I
didn't
select all the strings which were in the data, but just enough to give
you and
others an idea. I also assume that I obtained MADIS observations for
the
additional strings (synop, ship, metar, buoy) which were not present
in the
iittle_r file we used in this test.
Then for upper air verification, I would use the following:
MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
The above list could be expanded with more types of upper air data,
but I
think you get the idea. What I've done with this is to set up the
generation
of error stats in Point-Stat so that I get overall error stats which
were
derived from all possible sources of upper air observations. As long
as the
source provides measurements of met variables at the various pressure
levels
just as Point-Stat handles now, then this should work for the overall
stats.
For these stats I'm trying to capitalize on acquiring the maximum
number of
observations possible for upper air verification which often suffers
from a
paucity of "ground" truth data.
Then, I would have the option to go back and re-characterize the
various types
of observations into smaller groupings which focus in on trying to
group based
on the specific type of measurement used.
For example break out the ship data and using a different mapping
which
characterizes it as "SFCSHP" so that its stats are calculated
separately in
Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
That way you can evaluate the contribution/weight of those stats
towards the
overall stats. Any number of different mappings are possible by re-
running
ASCII2NC with different configuration file settings.
For surface observations where there are often large numbers of
measurements
from disparate sensor types, we are also interested in trying break
down the
verification and restrict the scoring to subdomains and having the
ability to
break out the stats by sensor type would be of value for these more
in-depth
assessments.
Regard the quality control values, I'm not sure what to think on this
yet.
Hopefully others can chime in here.
I would like the opportunity to discuss this among colleagues here in
ARL so
that I'm not misinterpreting anything here or am making invalid
assumptions.
I'm sure others have some good input to make about the statistical
implications of grouping the observation types the way I did. I have
probably
omitted the kind of stratifications which can be made with Stat-
Analysis which
might factor into this discussion.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, August 08, 2013 10:30 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
(US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
John,
Hmmm, good question. Let me give you some background. We only
recently began
providing support for little_r data, which was driven by a modelling
project
in which the DTC was involved. Our test data was limited to the data
available for that project. Based on that data, we defined the
following
mappings:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
Instances of the little_r input strings on the left are mapped to the
message
types used in the ASCII2NC output on the right. Looking at the data
you sent
me, I see that your little_r data includes the following 22 message
types:
"FM-132"
"FM-15_AIRNow"
"FM-15_APRSWXNET"
"FM-15_ASOS"
"FM-15_CA-Hydro"
"FM-15_CODOT"
"FM-15_DDMET"
"FM-15_GPSMET"
"FM-15_HADS"
"FM-15_MAP"
"FM-15_MesoWest"
"FM-15_NERRS"
"FM-15_NonFedAWOS"
"FM-15_NOS-NWLON"
"FM-15_NOS-PORTS"
"FM-15_NVDOT"
"FM-15_OTHER-MTR"
"FM-15_RAWS"
"FM-15_SURFRAD"
"FM-35"
"FM-96"
"FM-97_TAMDAR"
None of those match the strings we were expecting. So instead of
mapping them
to the PREPBUFR message types, we just wrote the strings that were
input. The
impact is that when you use this data in Point-Stat, you'll need to
list those
message types separately and get output for each one.
I see several questions here:
(1) Is it reasonable for us to figure out the super-set of all message
types
we can expect to see in little_r data?
(2) If yes, what is that super-set?
(3) If no, is it OK for us to just use in the input string in the
output? Or
should we provide a way for users to define a mapping of input
little_r
strings to output message types? We could do that by providing a
configuration file for the ASCII2NC tool - which is already used for
ASCII
SURFRAD data.
Do you have any perspectives on these issues?
Regarding the quality control values, yes, I agree, they are odd.
Unfortunately, I don't have much information for you. I would suggest
verifying that the values used in the little_r files match the values
that are
showing up in the MET output. But making sense of them is up to you.
In
Point-Stat, you can provide a list of quality control strings to use
in the
obs_quality parameter. For the DTC project using little_r data,
here's the
list of strings we used:
obs_quality = [
"3", "4", "5", "6", "7", "8",
"9",
"10",
"100003", "100004", "100005", "100006", "100007", "100008",
"100009",
"100010",
"200003", "200004", "200005", "200006", "200007", "200008",
"200009",
"200010",
"300003", "300004", "300005", "300006", "300007", "300008",
"300009",
"300010",
"400003", "400004", "400005", "400006", "400007", "400008",
"400009",
"400010",
"500003", "500004", "500005", "500006", "500007", "500008",
"500009",
"500010",
"600003", "600004", "600005", "600006", "600007", "600008",
"600009",
"600010"
];
But it looks like these are not the same values that are showing up in
your
data.
Thanks,
John
On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>
> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Test of ASCII2NC on little_r file
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
> >
>
>
> I ran ASCII2NC as follows:
>
> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
> -v 4
>
> It appears that the LittleRHandler is working to indicate that it
recognizes
> the input file as a little_r file and then generates the NetCDF
output file.
>
> The output file attached as the text file " test7_ncdump" appears to
contain
> good data, but the obs_qty information looks odd in places.
>
> I've also attached the log file which contain extensive WARNINGS,
but no
> ERROR.
>
> Can you explain the odd numbers in the obs_qty array? I was
expecting
> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
> provide on the reasons for the WARNINGS would be appreciated.
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>
Classification: UNCLASSIFIED
Caveats: NONE
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Aug 13 08:29:12 2013
John,
One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
We'll let you know what we figure out. Any potential changes wouldn't
be released as a "bugfix" for METv4.1. Instead, they'd show up in the
next release.
For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
At line 80, you can add in the mappings of little_R strings to output
message type names you'd like, and then recompile.
Thanks,
John
On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>
> A more complete description in regard to output from standard
Obsgrid is:
> 2 = pressure derived from the first-guess height field (e.g. GFS)
> 4 = pressure derived from standard atmosphere assumption and height
> 8 = pressure derived from standard atmosphere assumption and
temperature
> 16 = Temperature/Dewpoint are zero
> 32 = Calm wind
> 64 = Negative wind speed
> 128 = Wind direction did not fall between 0 and 360 degrees
> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
> 256 = Value was vertically interpolated (to get to level with first-
guess field to QC against)
> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
> 1024 = Temperature appeared to have the wrong sign based on adjacent
temperatures in the vertical profile so OBSGRID flipped the sign of
the temperature and adjusted dewpoint/RH
> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
> 16384 = No buddies (nearby observations) were found to do a buddy
check
> 32768 = Fails error max check (check against first guess field)
> 65536 = Fails buddy check (check against nearby obs)
> 131072 = Observation outside of domain
>
> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>
> Brian
>
>
> -----Original Message-----
> From: Raby, John W CIV USARMY ARL (US)
> Sent: Thursday, August 08, 2013 3:15 PM
> To: met_help at ucar.edu
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for this info, your analysis of our results and opening the
opportunity
> for a dialog on this topic.
>
> I think of observational data in the following categories:
>
> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
> which I use the verify model forecasts for surface met variables
(2meter and
> 10meter AGL) using Point-Stat.
>
> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
> variables at various pressure levels using Point-Stat.
>
> My thoughts to follow are not the final word on this and subject to
input from
> other ARL colleagues which I will coordinate before giving you the
final
> answers.
>
> Lump all the sources of surface observations together to produce
error
> statistics for surface variables and do the same for upper air
observations
> and their error statistics. Then, break out the error statistics by
message
> type like Point-Stat does now, so you could isolate a particular
type
> (observation source/type) which might be weighing heavily in
contributing to a
> particularly large overall error stat.
>
> That said, I don't want to generate a big change for the Point-Stat
tool. I
> may be forgetting some features of Stat-Analysis which would allow
me to
> aggregate/stratify the error stats in order to show the stats I
mentioned
> above.
>
> I'm tending towards the latter suggestion you made in (3) about the
user
> setting the mapping of input little_r strings to output message
types. That
> way I could control the generation of error stats in Point-Stat to
adhere to
> the groupings I mentioned above. Then, if the need arose, I could
rerun the
> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
> The following example describes one such general mapping for surface
> verification:
>
> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>
>
>
> In the above mapping, I'm assuming that the little_r strings I
selected are
> all observations of surface met variables (2 and 10 meter AGL). I
didn't
> select all the strings which were in the data, but just enough to
give you and
> others an idea. I also assume that I obtained MADIS observations for
the
> additional strings (synop, ship, metar, buoy) which were not present
in the
> iittle_r file we used in this test.
>
> Then for upper air verification, I would use the following:
>
> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>
>
> The above list could be expanded with more types of upper air data,
but I
> think you get the idea. What I've done with this is to set up the
generation
> of error stats in Point-Stat so that I get overall error stats which
were
> derived from all possible sources of upper air observations. As long
as the
> source provides measurements of met variables at the various
pressure levels
> just as Point-Stat handles now, then this should work for the
overall stats.
> For these stats I'm trying to capitalize on acquiring the maximum
number of
> observations possible for upper air verification which often suffers
from a
> paucity of "ground" truth data.
>
> Then, I would have the option to go back and re-characterize the
various types
> of observations into smaller groupings which focus in on trying to
group based
> on the specific type of measurement used.
>
> For example break out the ship data and using a different mapping
which
> characterizes it as "SFCSHP" so that its stats are calculated
separately in
> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
> That way you can evaluate the contribution/weight of those stats
towards the
> overall stats. Any number of different mappings are possible by re-
running
> ASCII2NC with different configuration file settings.
>
> For surface observations where there are often large numbers of
measurements
> from disparate sensor types, we are also interested in trying break
down the
> verification and restrict the scoring to subdomains and having the
ability to
> break out the stats by sensor type would be of value for these more
in-depth
> assessments.
>
> Regard the quality control values, I'm not sure what to think on
this yet.
> Hopefully others can chime in here.
>
> I would like the opportunity to discuss this among colleagues here
in ARL so
> that I'm not misinterpreting anything here or am making invalid
assumptions.
> I'm sure others have some good input to make about the statistical
> implications of grouping the observation types the way I did. I have
probably
> omitted the kind of stratifications which can be made with Stat-
Analysis which
> might factor into this discussion.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, August 08, 2013 10:30 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
> (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>
> John,
>
> Hmmm, good question. Let me give you some background. We only
recently began
> providing support for little_r data, which was driven by a modelling
project
> in which the DTC was involved. Our test data was limited to the
data
> available for that project. Based on that data, we defined the
following
> mappings:
>
> //
> // Load mapping of report types to message types
> //
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>
> Instances of the little_r input strings on the left are mapped to
the message
> types used in the ASCII2NC output on the right. Looking at the data
you sent
> me, I see that your little_r data includes the following 22 message
types:
> "FM-132"
> "FM-15_AIRNow"
> "FM-15_APRSWXNET"
> "FM-15_ASOS"
> "FM-15_CA-Hydro"
> "FM-15_CODOT"
> "FM-15_DDMET"
> "FM-15_GPSMET"
> "FM-15_HADS"
> "FM-15_MAP"
> "FM-15_MesoWest"
> "FM-15_NERRS"
> "FM-15_NonFedAWOS"
> "FM-15_NOS-NWLON"
> "FM-15_NOS-PORTS"
> "FM-15_NVDOT"
> "FM-15_OTHER-MTR"
> "FM-15_RAWS"
> "FM-15_SURFRAD"
> "FM-35"
> "FM-96"
> "FM-97_TAMDAR"
>
> None of those match the strings we were expecting. So instead of
mapping them
> to the PREPBUFR message types, we just wrote the strings that were
input. The
> impact is that when you use this data in Point-Stat, you'll need to
list those
> message types separately and get output for each one.
>
> I see several questions here:
> (1) Is it reasonable for us to figure out the super-set of all
message types
> we can expect to see in little_r data?
> (2) If yes, what is that super-set?
> (3) If no, is it OK for us to just use in the input string in the
output? Or
> should we provide a way for users to define a mapping of input
little_r
> strings to output message types? We could do that by providing a
> configuration file for the ASCII2NC tool - which is already used for
ASCII
> SURFRAD data.
>
> Do you have any perspectives on these issues?
>
> Regarding the quality control values, yes, I agree, they are odd.
> Unfortunately, I don't have much information for you. I would
suggest
> verifying that the values used in the little_r files match the
values that are
> showing up in the MET output. But making sense of them is up to
you. In
> Point-Stat, you can provide a list of quality control strings to use
in the
> obs_quality parameter. For the DTC project using little_r data,
here's the
> list of strings we used:
>
> obs_quality = [
> "3", "4", "5", "6", "7", "8",
"9",
> "10",
> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
> "100010",
> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
> "200010",
> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
> "300010",
> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
> "400010",
> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
> "500010",
> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
> "600010"
> ];
>
> But it looks like these are not the same values that are showing up
in your
> data.
>
> Thanks,
> John
>
> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>
>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Test of ASCII2NC on little_r file
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>
>>
>>
>> I ran ASCII2NC as follows:
>>
>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>> -v 4
>>
>> It appears that the LittleRHandler is working to indicate that it
recognizes
>> the input file as a little_r file and then generates the NetCDF
output file.
>>
>> The output file attached as the text file " test7_ncdump" appears
to contain
>> good data, but the obs_qty information looks odd in places.
>>
>> I've also attached the log file which contain extensive WARNINGS,
but no
>> ERROR.
>>
>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>> provide on the reasons for the WARNINGS would be appreciated.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Aug 13 09:48:06 2013
John -
Thanks for an update on your assessment of how to incorporate all the
report type strings from little_r files. We are pleased to hear of the
direction you are headed to allow the user to control the mappings
either with the config file or the ascii file. In the case of the
ascii file, would this require a recompile for each subsequent
modification of the file?
I believe that we acquire the data which ends up in little_r files
from the MADIS archived/current source, but Dr Brian Reen (cc'd on
this email) is our expert on data acquisition and the generation of
little_r files to support his modeling work.
Dr Reen sent me the forecast file for the same date/time as the
little_r file I sent you and I am in the process of setting up a
Point-Stat run to see how it works using the report string names as
they appear in the little_r file per your instructions last week.
I will work with Bob Flanigan to see if we can set up a test of the
last option you mentioned about editing the little_r_handler.cc file
then recompile.
Will keep you posted on our testing results.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, August 13, 2013 8:29 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
We'll let you know what we figure out. Any potential changes wouldn't
be released as a "bugfix" for METv4.1. Instead, they'd show up in the
next release.
For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
At line 80, you can add in the mappings of little_R strings to output
message type names you'd like, and then recompile.
Thanks,
John
On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>
> A more complete description in regard to output from standard
Obsgrid is:
> 2 = pressure derived from the first-guess height field (e.g. GFS)
> 4 = pressure derived from standard atmosphere assumption and height
> 8 = pressure derived from standard atmosphere assumption and
temperature
> 16 = Temperature/Dewpoint are zero
> 32 = Calm wind
> 64 = Negative wind speed
> 128 = Wind direction did not fall between 0 and 360 degrees
> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
> 256 = Value was vertically interpolated (to get to level with first-
guess field to QC against)
> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
> 1024 = Temperature appeared to have the wrong sign based on adjacent
temperatures in the vertical profile so OBSGRID flipped the sign of
the temperature and adjusted dewpoint/RH
> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
> 16384 = No buddies (nearby observations) were found to do a buddy
check
> 32768 = Fails error max check (check against first guess field)
> 65536 = Fails buddy check (check against nearby obs)
> 131072 = Observation outside of domain
>
> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>
> Brian
>
>
> -----Original Message-----
> From: Raby, John W CIV USARMY ARL (US)
> Sent: Thursday, August 08, 2013 3:15 PM
> To: met_help at ucar.edu
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for this info, your analysis of our results and opening the
opportunity
> for a dialog on this topic.
>
> I think of observational data in the following categories:
>
> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
> which I use the verify model forecasts for surface met variables
(2meter and
> 10meter AGL) using Point-Stat.
>
> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
> variables at various pressure levels using Point-Stat.
>
> My thoughts to follow are not the final word on this and subject to
input from
> other ARL colleagues which I will coordinate before giving you the
final
> answers.
>
> Lump all the sources of surface observations together to produce
error
> statistics for surface variables and do the same for upper air
observations
> and their error statistics. Then, break out the error statistics by
message
> type like Point-Stat does now, so you could isolate a particular
type
> (observation source/type) which might be weighing heavily in
contributing to a
> particularly large overall error stat.
>
> That said, I don't want to generate a big change for the Point-Stat
tool. I
> may be forgetting some features of Stat-Analysis which would allow
me to
> aggregate/stratify the error stats in order to show the stats I
mentioned
> above.
>
> I'm tending towards the latter suggestion you made in (3) about the
user
> setting the mapping of input little_r strings to output message
types. That
> way I could control the generation of error stats in Point-Stat to
adhere to
> the groupings I mentioned above. Then, if the need arose, I could
rerun the
> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
> The following example describes one such general mapping for surface
> verification:
>
> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>
>
>
> In the above mapping, I'm assuming that the little_r strings I
selected are
> all observations of surface met variables (2 and 10 meter AGL). I
didn't
> select all the strings which were in the data, but just enough to
give you and
> others an idea. I also assume that I obtained MADIS observations for
the
> additional strings (synop, ship, metar, buoy) which were not present
in the
> iittle_r file we used in this test.
>
> Then for upper air verification, I would use the following:
>
> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>
>
> The above list could be expanded with more types of upper air data,
but I
> think you get the idea. What I've done with this is to set up the
generation
> of error stats in Point-Stat so that I get overall error stats which
were
> derived from all possible sources of upper air observations. As long
as the
> source provides measurements of met variables at the various
pressure levels
> just as Point-Stat handles now, then this should work for the
overall stats.
> For these stats I'm trying to capitalize on acquiring the maximum
number of
> observations possible for upper air verification which often suffers
from a
> paucity of "ground" truth data.
>
> Then, I would have the option to go back and re-characterize the
various types
> of observations into smaller groupings which focus in on trying to
group based
> on the specific type of measurement used.
>
> For example break out the ship data and using a different mapping
which
> characterizes it as "SFCSHP" so that its stats are calculated
separately in
> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
> That way you can evaluate the contribution/weight of those stats
towards the
> overall stats. Any number of different mappings are possible by re-
running
> ASCII2NC with different configuration file settings.
>
> For surface observations where there are often large numbers of
measurements
> from disparate sensor types, we are also interested in trying break
down the
> verification and restrict the scoring to subdomains and having the
ability to
> break out the stats by sensor type would be of value for these more
in-depth
> assessments.
>
> Regard the quality control values, I'm not sure what to think on
this yet.
> Hopefully others can chime in here.
>
> I would like the opportunity to discuss this among colleagues here
in ARL so
> that I'm not misinterpreting anything here or am making invalid
assumptions.
> I'm sure others have some good input to make about the statistical
> implications of grouping the observation types the way I did. I have
probably
> omitted the kind of stratifications which can be made with Stat-
Analysis which
> might factor into this discussion.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, August 08, 2013 10:30 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
> (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>
> John,
>
> Hmmm, good question. Let me give you some background. We only
recently began
> providing support for little_r data, which was driven by a modelling
project
> in which the DTC was involved. Our test data was limited to the
data
> available for that project. Based on that data, we defined the
following
> mappings:
>
> //
> // Load mapping of report types to message types
> //
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>
> Instances of the little_r input strings on the left are mapped to
the message
> types used in the ASCII2NC output on the right. Looking at the data
you sent
> me, I see that your little_r data includes the following 22 message
types:
> "FM-132"
> "FM-15_AIRNow"
> "FM-15_APRSWXNET"
> "FM-15_ASOS"
> "FM-15_CA-Hydro"
> "FM-15_CODOT"
> "FM-15_DDMET"
> "FM-15_GPSMET"
> "FM-15_HADS"
> "FM-15_MAP"
> "FM-15_MesoWest"
> "FM-15_NERRS"
> "FM-15_NonFedAWOS"
> "FM-15_NOS-NWLON"
> "FM-15_NOS-PORTS"
> "FM-15_NVDOT"
> "FM-15_OTHER-MTR"
> "FM-15_RAWS"
> "FM-15_SURFRAD"
> "FM-35"
> "FM-96"
> "FM-97_TAMDAR"
>
> None of those match the strings we were expecting. So instead of
mapping them
> to the PREPBUFR message types, we just wrote the strings that were
input. The
> impact is that when you use this data in Point-Stat, you'll need to
list those
> message types separately and get output for each one.
>
> I see several questions here:
> (1) Is it reasonable for us to figure out the super-set of all
message types
> we can expect to see in little_r data?
> (2) If yes, what is that super-set?
> (3) If no, is it OK for us to just use in the input string in the
output? Or
> should we provide a way for users to define a mapping of input
little_r
> strings to output message types? We could do that by providing a
> configuration file for the ASCII2NC tool - which is already used for
ASCII
> SURFRAD data.
>
> Do you have any perspectives on these issues?
>
> Regarding the quality control values, yes, I agree, they are odd.
> Unfortunately, I don't have much information for you. I would
suggest
> verifying that the values used in the little_r files match the
values that are
> showing up in the MET output. But making sense of them is up to
you. In
> Point-Stat, you can provide a list of quality control strings to use
in the
> obs_quality parameter. For the DTC project using little_r data,
here's the
> list of strings we used:
>
> obs_quality = [
> "3", "4", "5", "6", "7", "8",
"9",
> "10",
> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
> "100010",
> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
> "200010",
> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
> "300010",
> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
> "400010",
> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
> "500010",
> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
> "600010"
> ];
>
> But it looks like these are not the same values that are showing up
in your
> data.
>
> Thanks,
> John
>
> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>
>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Test of ASCII2NC on little_r file
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>
>>
>>
>> I ran ASCII2NC as follows:
>>
>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>> -v 4
>>
>> It appears that the LittleRHandler is working to indicate that it
recognizes
>> the input file as a little_r file and then generates the NetCDF
output file.
>>
>> The output file attached as the text file " test7_ncdump" appears
to contain
>> good data, but the obs_qty information looks odd in places.
>>
>> I've also attached the log file which contain extensive WARNINGS,
but no
>> ERROR.
>>
>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>> provide on the reasons for the WARNINGS would be appreciated.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Aug 13 13:02:06 2013
John,
The point of storing this information in an ascii data file (either a
config file or something in /data) would be to prevent the need to
recompile. You'd just edit that file as needed and when the
tool runs, it'd read the information it needs from that file. Make
sense?
Thanks,
John
On 08/13/2013 09:48 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Thanks for an update on your assessment of how to incorporate all
the report type strings from little_r files. We are pleased to hear of
the direction you are headed to allow the user to control the mappings
either with the config file or the ascii file. In the case of the
ascii file, would this require a recompile for each subsequent
modification of the file?
>
> I believe that we acquire the data which ends up in little_r files
from the MADIS archived/current source, but Dr Brian Reen (cc'd on
this email) is our expert on data acquisition and the generation of
little_r files to support his modeling work.
>
> Dr Reen sent me the forecast file for the same date/time as the
little_r file I sent you and I am in the process of setting up a
Point-Stat run to see how it works using the report string names as
they appear in the little_r file per your instructions last week.
>
> I will work with Bob Flanigan to see if we can set up a test of the
last option you mentioned about editing the little_r_handler.cc file
then recompile.
>
> Will keep you posted on our testing results.
>
> R/
> John
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Aug 13 14:07:34 2013
John -
I understand. That sounds pretty easy to do. Thanks.
By the way, the Point-Stat test I conducted using the little_r
observation file worked. I used a few of the report strings from the
file and generated stats for all the upper air variables/levels I
specified in the config file. I could not find any stats for the
surface (2meter and 10meter AGL) levels for all variables (TMP, DPT,
UGRD, VGRD, RH). I'm sure they were present in the forecast file based
on my earlier inspection of that file. The log file show 0 pairs for
those variables/levels, so the little_r file must not have any surface
data. I'm gleaning though the little_r file now to confirm this.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, August 13, 2013 1:02 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
The point of storing this information in an ascii data file (either a
config file or something in /data) would be to prevent the need to
recompile. You'd just edit that file as needed and when the
tool runs, it'd read the information it needs from that file. Make
sense?
Thanks,
John
On 08/13/2013 09:48 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Thanks for an update on your assessment of how to incorporate all
the report type strings from little_r files. We are pleased to hear of
the direction you are headed to allow the user to control the mappings
either with the config file or the ascii file. In the case of the
ascii file, would this require a recompile for each subsequent
modification of the file?
>
> I believe that we acquire the data which ends up in little_r files
from the MADIS archived/current source, but Dr Brian Reen (cc'd on
this email) is our expert on data acquisition and the generation of
little_r files to support his modeling work.
>
> Dr Reen sent me the forecast file for the same date/time as the
little_r file I sent you and I am in the process of setting up a
Point-Stat run to see how it works using the report string names as
they appear in the little_r file per your instructions last week.
>
> I will work with Bob Flanigan to see if we can set up a test of the
last option you mentioned about editing the little_r_handler.cc file
then recompile.
>
> Will keep you posted on our testing results.
>
> R/
> John
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Aug 13 14:26:51 2013
John -
I can't see anything obvious in the NetCDF file (little_r) that would
indicate that the data is for the levels "Z2" or "Z10" (2m or 10m
AGL). I have other NetCDF observation files (not little_r) here which
contain these levels and I am trying to compare them with the little_r
NetCDF file, but I can't see anything in either file that would show
the data is for those levels.
Can you show me what to look for?
Thanks.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, August 13, 2013 1:02 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
The point of storing this information in an ascii data file (either a
config file or something in /data) would be to prevent the need to
recompile. You'd just edit that file as needed and when the
tool runs, it'd read the information it needs from that file. Make
sense?
Thanks,
John
On 08/13/2013 09:48 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Thanks for an update on your assessment of how to incorporate all
the report type strings from little_r files. We are pleased to hear of
the direction you are headed to allow the user to control the mappings
either with the config file or the ascii file. In the case of the
ascii file, would this require a recompile for each subsequent
modification of the file?
>
> I believe that we acquire the data which ends up in little_r files
from the MADIS archived/current source, but Dr Brian Reen (cc'd on
this email) is our expert on data acquisition and the generation of
little_r files to support his modeling work.
>
> Dr Reen sent me the forecast file for the same date/time as the
little_r file I sent you and I am in the process of setting up a
Point-Stat run to see how it works using the report string names as
they appear in the little_r file per your instructions last week.
>
> I will work with Bob Flanigan to see if we can set up a test of the
last option you mentioned about editing the little_r_handler.cc file
then recompile.
>
> Will keep you posted on our testing results.
>
> R/
> John
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Tue Aug 20 07:30:16 2013
John -
Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
This is quite different from my first Point-Stat test last week where
the little_r_handler.cc file was the default one. In that case, I used
your suggestion about putting the actual report string names from the
little_r file in the message_type array in the config file. That test
had varying numbers of pairs found depending on the number of
observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
I've attached the little_r_handler.cc file I used for yesterday's test
and the default one (ORIG) used for the first test for ref. I have
also attached my config file.
Do you have any idea why my mapping doesn't seem to be working and why
the two AGL level results are not being generated?
Thanks.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, August 13, 2013 8:29 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
We'll let you know what we figure out. Any potential changes wouldn't
be released as a "bugfix" for METv4.1. Instead, they'd show up in the
next release.
For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
At line 80, you can add in the mappings of little_R strings to output
message type names you'd like, and then recompile.
Thanks,
John
On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>
> A more complete description in regard to output from standard
Obsgrid is:
> 2 = pressure derived from the first-guess height field (e.g. GFS)
> 4 = pressure derived from standard atmosphere assumption and height
> 8 = pressure derived from standard atmosphere assumption and
temperature
> 16 = Temperature/Dewpoint are zero
> 32 = Calm wind
> 64 = Negative wind speed
> 128 = Wind direction did not fall between 0 and 360 degrees
> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
> 256 = Value was vertically interpolated (to get to level with first-
guess field to QC against)
> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
> 1024 = Temperature appeared to have the wrong sign based on adjacent
temperatures in the vertical profile so OBSGRID flipped the sign of
the temperature and adjusted dewpoint/RH
> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
> 16384 = No buddies (nearby observations) were found to do a buddy
check
> 32768 = Fails error max check (check against first guess field)
> 65536 = Fails buddy check (check against nearby obs)
> 131072 = Observation outside of domain
>
> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>
> Brian
>
>
> -----Original Message-----
> From: Raby, John W CIV USARMY ARL (US)
> Sent: Thursday, August 08, 2013 3:15 PM
> To: met_help at ucar.edu
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for this info, your analysis of our results and opening the
opportunity
> for a dialog on this topic.
>
> I think of observational data in the following categories:
>
> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
> which I use the verify model forecasts for surface met variables
(2meter and
> 10meter AGL) using Point-Stat.
>
> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
> variables at various pressure levels using Point-Stat.
>
> My thoughts to follow are not the final word on this and subject to
input from
> other ARL colleagues which I will coordinate before giving you the
final
> answers.
>
> Lump all the sources of surface observations together to produce
error
> statistics for surface variables and do the same for upper air
observations
> and their error statistics. Then, break out the error statistics by
message
> type like Point-Stat does now, so you could isolate a particular
type
> (observation source/type) which might be weighing heavily in
contributing to a
> particularly large overall error stat.
>
> That said, I don't want to generate a big change for the Point-Stat
tool. I
> may be forgetting some features of Stat-Analysis which would allow
me to
> aggregate/stratify the error stats in order to show the stats I
mentioned
> above.
>
> I'm tending towards the latter suggestion you made in (3) about the
user
> setting the mapping of input little_r strings to output message
types. That
> way I could control the generation of error stats in Point-Stat to
adhere to
> the groupings I mentioned above. Then, if the need arose, I could
rerun the
> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
> The following example describes one such general mapping for surface
> verification:
>
> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>
>
>
> In the above mapping, I'm assuming that the little_r strings I
selected are
> all observations of surface met variables (2 and 10 meter AGL). I
didn't
> select all the strings which were in the data, but just enough to
give you and
> others an idea. I also assume that I obtained MADIS observations for
the
> additional strings (synop, ship, metar, buoy) which were not present
in the
> iittle_r file we used in this test.
>
> Then for upper air verification, I would use the following:
>
> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>
>
> The above list could be expanded with more types of upper air data,
but I
> think you get the idea. What I've done with this is to set up the
generation
> of error stats in Point-Stat so that I get overall error stats which
were
> derived from all possible sources of upper air observations. As long
as the
> source provides measurements of met variables at the various
pressure levels
> just as Point-Stat handles now, then this should work for the
overall stats.
> For these stats I'm trying to capitalize on acquiring the maximum
number of
> observations possible for upper air verification which often suffers
from a
> paucity of "ground" truth data.
>
> Then, I would have the option to go back and re-characterize the
various types
> of observations into smaller groupings which focus in on trying to
group based
> on the specific type of measurement used.
>
> For example break out the ship data and using a different mapping
which
> characterizes it as "SFCSHP" so that its stats are calculated
separately in
> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
> That way you can evaluate the contribution/weight of those stats
towards the
> overall stats. Any number of different mappings are possible by re-
running
> ASCII2NC with different configuration file settings.
>
> For surface observations where there are often large numbers of
measurements
> from disparate sensor types, we are also interested in trying break
down the
> verification and restrict the scoring to subdomains and having the
ability to
> break out the stats by sensor type would be of value for these more
in-depth
> assessments.
>
> Regard the quality control values, I'm not sure what to think on
this yet.
> Hopefully others can chime in here.
>
> I would like the opportunity to discuss this among colleagues here
in ARL so
> that I'm not misinterpreting anything here or am making invalid
assumptions.
> I'm sure others have some good input to make about the statistical
> implications of grouping the observation types the way I did. I have
probably
> omitted the kind of stratifications which can be made with Stat-
Analysis which
> might factor into this discussion.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, August 08, 2013 10:30 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
> (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>
> John,
>
> Hmmm, good question. Let me give you some background. We only
recently began
> providing support for little_r data, which was driven by a modelling
project
> in which the DTC was involved. Our test data was limited to the
data
> available for that project. Based on that data, we defined the
following
> mappings:
>
> //
> // Load mapping of report types to message types
> //
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>
> Instances of the little_r input strings on the left are mapped to
the message
> types used in the ASCII2NC output on the right. Looking at the data
you sent
> me, I see that your little_r data includes the following 22 message
types:
> "FM-132"
> "FM-15_AIRNow"
> "FM-15_APRSWXNET"
> "FM-15_ASOS"
> "FM-15_CA-Hydro"
> "FM-15_CODOT"
> "FM-15_DDMET"
> "FM-15_GPSMET"
> "FM-15_HADS"
> "FM-15_MAP"
> "FM-15_MesoWest"
> "FM-15_NERRS"
> "FM-15_NonFedAWOS"
> "FM-15_NOS-NWLON"
> "FM-15_NOS-PORTS"
> "FM-15_NVDOT"
> "FM-15_OTHER-MTR"
> "FM-15_RAWS"
> "FM-15_SURFRAD"
> "FM-35"
> "FM-96"
> "FM-97_TAMDAR"
>
> None of those match the strings we were expecting. So instead of
mapping them
> to the PREPBUFR message types, we just wrote the strings that were
input. The
> impact is that when you use this data in Point-Stat, you'll need to
list those
> message types separately and get output for each one.
>
> I see several questions here:
> (1) Is it reasonable for us to figure out the super-set of all
message types
> we can expect to see in little_r data?
> (2) If yes, what is that super-set?
> (3) If no, is it OK for us to just use in the input string in the
output? Or
> should we provide a way for users to define a mapping of input
little_r
> strings to output message types? We could do that by providing a
> configuration file for the ASCII2NC tool - which is already used for
ASCII
> SURFRAD data.
>
> Do you have any perspectives on these issues?
>
> Regarding the quality control values, yes, I agree, they are odd.
> Unfortunately, I don't have much information for you. I would
suggest
> verifying that the values used in the little_r files match the
values that are
> showing up in the MET output. But making sense of them is up to
you. In
> Point-Stat, you can provide a list of quality control strings to use
in the
> obs_quality parameter. For the DTC project using little_r data,
here's the
> list of strings we used:
>
> obs_quality = [
> "3", "4", "5", "6", "7", "8",
"9",
> "10",
> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
> "100010",
> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
> "200010",
> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
> "300010",
> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
> "400010",
> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
> "500010",
> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
> "600010"
> ];
>
> But it looks like these are not the same values that are showing up
in your
> data.
>
> Thanks,
> John
>
> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>
>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Test of ASCII2NC on little_r file
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>
>>
>>
>> I ran ASCII2NC as follows:
>>
>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>> -v 4
>>
>> It appears that the LittleRHandler is working to indicate that it
recognizes
>> the input file as a little_r file and then generates the NetCDF
output file.
>>
>> The output file attached as the text file " test7_ncdump" appears
to contain
>> good data, but the obs_qty information looks odd in places.
>>
>> I've also attached the log file which contain extensive WARNINGS,
but no
>> ERROR.
>>
>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>> provide on the reasons for the WARNINGS would be appreciated.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Aug 21 16:16:19 2013
John,
I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
Thanks,
John
On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>
> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>
> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>
> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Thu Aug 22 07:18:08 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
Thanks for your reply and offer to do some testing there. Before I
send the file and ask you to test with it, I'd like to do one more
test here. This morning I realized that I had not rerun ASCII2NC on
the little_r file before doing the test with Point-Stat. If I
understand correctly, the mapping specified in the little_r_handler.cc
file gets implemented by ASCII2NC as it converts the little_r file to
NetCDF format. Then Point-Stat should find the data in the little_r
file with the right message_type names which agree with those I put in
the config file. I should have thought of this before! Anyway, let me
know if I'm not on the right track with this latest approach. I will
do the test here this morning and let you know as soon as I get some
output.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, August 21, 2013 4:16 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me a sample
one to work with? I'd run it through ascii2nc and inspect the NetCDF
output to see if it's working as expected.
Thanks,
John
On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>
> This is quite different from my first Point-Stat test last week
where
> the little_r_handler.cc file was the default one. In that case, I
used
> your suggestion about putting the actual report string names from
the
> little_r file in the message_type array in the config file. That
test
> had varying numbers of pairs found depending on the number of
> observations found for a given report type/level and produced stats
> for all report types/levels except for the variables at 2m and 10m
AGL
> I had specified in the config file. I'm still trying to figure out
why
> no results for these AGL levels
>
> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>
> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
> rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
> file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
> little_r observations. We're trying to figure out if we can define
> the super-set of all possible message type strings that may show up
in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
> editing this file:
> METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap
>> 7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
>> temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees [129] =
>> WRF uses this to indicate winds that are earth-relative (instead of
grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
>> first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a
>> first-guess field was available for qc (some variables may have
been
>> adjusted to account for the change in pressure) (this only applies
>> for obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent
>> temperatures in the vertical profile so OBSGRID flipped the sign of
>> the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
>> adjustment (results in OBSGRID marking temperature, dewpoint, and
>> relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
>> superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
>> check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
>> Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
>> Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>> file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
>> opportunity for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
>> reports which I use the verify model forecasts for surface met
>> variables (2meter and 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
>> AIRCRAFT, AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC
>> SOUNDER (SODAR), VAD (NEXRAD) WIND some of which I use to verify
>> model forecasts for upper air variables at various pressure levels
using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
>> input from other ARL colleagues which I will coordinate before
giving
>> you the final answers.
>>
>> Lump all the sources of surface observations together to produce
>> error statistics for surface variables and do the same for upper
air
>> observations and their error statistics. Then, break out the error
>> statistics by message type like Point-Stat does now, so you could
>> isolate a particular type (observation source/type) which might be
>> weighing heavily in contributing to a particularly large overall
error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
>> tool. I may be forgetting some features of Stat-Analysis which
would
>> allow me to aggregate/stratify the error stats in order to show the
>> stats I mentioned above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
>> user setting the mapping of input little_r strings to output
message
>> types. That way I could control the generation of error stats in
>> Point-Stat to adhere to the groupings I mentioned above. Then, if
the
>> need arose, I could rerun the ASCII2NC with the mapping changed to
regroup the little_r strings differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_ASOS
>> " ] = "ADPSFC"; MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_CODOT"
>> ] = "ADPSFC"; MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_GPSMET"
>> ] = "ADPSFC"; MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Thu Aug 22 14:19:33 2013
John -
I checked the path to the MET software and it was pointed to the
edited little_r_handler.cc file I sent you. I ran ASCII2NC and got the
same results which don't seem to produce report type names like I
wanted (ADPSFC, ADPUPA, ...etc). in the NetCDF output file. The report
type names in the NetCDF file are the one straight out of the little_r
file (attached)
Thanks for your help.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Wednesday, August 21, 2013 4:16 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
Thanks,
John
On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>
> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>
> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>
> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Thu Aug 22 14:33:13 2013
John,
You sent me the NetCDF *output* of the ASCII2NC tool. In order for
you to test the functionality of ASCII2NC with the changes you've
made, I'll need the *input* to that tool. It should be a little_r
file in ascii format. Are you able to send me one of those?
Thanks,
John
On 08/22/2013 02:19 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
>
> John -
>
> I checked the path to the MET software and it was pointed to the
edited little_r_handler.cc file I sent you. I ran ASCII2NC and got the
same results which don't seem to produce report type names like I
wanted (ADPSFC, ADPUPA, ...etc). in the NetCDF output file. The report
type names in the NetCDF file are the one straight out of the little_r
file (attached)
>
> Thanks for your help.
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Wednesday, August 21, 2013 4:16 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
> a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
>
> Thanks,
> John
>
> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>
>> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>>
>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>
>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 13, 2013 8:29 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> John,
>>
>> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
>> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
>> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>
>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>
>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>
>> Thanks,
>> John
>>
>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>>
>>> A more complete description in regard to output from standard
Obsgrid is:
>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>> 4 = pressure derived from standard atmosphere assumption and
height
>>> 8 = pressure derived from standard atmosphere assumption and
temperature
>>> 16 = Temperature/Dewpoint are zero
>>> 32 = Calm wind
>>> 64 = Negative wind speed
>>> 128 = Wind direction did not fall between 0 and 360 degrees
>>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>>> 512 = The observation was assigned a nearby pressure where a
first-guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>>> 32768 = Fails error max check (check against first guess field)
>>> 65536 = Fails buddy check (check against nearby obs)
>>> 131072 = Observation outside of domain
>>>
>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>
>>> Brian
>>>
>>>
>>> -----Original Message-----
>>> From: Raby, John W CIV USARMY ARL (US)
>>> Sent: Thursday, August 08, 2013 3:15 PM
>>> To: met_help at ucar.edu
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> Thanks for this info, your analysis of our results and opening the
opportunity
>>> for a dialog on this topic.
>>>
>>> I think of observational data in the following categories:
>>>
>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy reports
>>> which I use the verify model forecasts for surface met variables
(2meter and
>>> 10meter AGL) using Point-Stat.
>>>
>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>>> VAD (NEXRAD) WIND some of which I use to verify model forecasts
for upper air
>>> variables at various pressure levels using Point-Stat.
>>>
>>> My thoughts to follow are not the final word on this and subject
to input from
>>> other ARL colleagues which I will coordinate before giving you the
final
>>> answers.
>>>
>>> Lump all the sources of surface observations together to produce
error
>>> statistics for surface variables and do the same for upper air
observations
>>> and their error statistics. Then, break out the error statistics
by message
>>> type like Point-Stat does now, so you could isolate a particular
type
>>> (observation source/type) which might be weighing heavily in
contributing to a
>>> particularly large overall error stat.
>>>
>>> That said, I don't want to generate a big change for the Point-
Stat tool. I
>>> may be forgetting some features of Stat-Analysis which would allow
me to
>>> aggregate/stratify the error stats in order to show the stats I
mentioned
>>> above.
>>>
>>> I'm tending towards the latter suggestion you made in (3) about
the user
>>> setting the mapping of input little_r strings to output message
types. That
>>> way I could control the generation of error stats in Point-Stat to
adhere to
>>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>>> The following example describes one such general mapping for
surface
>>> verification:
>>>
>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>
>>>
>>>
>>> In the above mapping, I'm assuming that the little_r strings I
selected are
>>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>>> select all the strings which were in the data, but just enough to
give you and
>>> others an idea. I also assume that I obtained MADIS observations
for the
>>> additional strings (synop, ship, metar, buoy) which were not
present in the
>>> iittle_r file we used in this test.
>>>
>>> Then for upper air verification, I would use the following:
>>>
>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>>
>>>
>>> The above list could be expanded with more types of upper air
data, but I
>>> think you get the idea. What I've done with this is to set up the
generation
>>> of error stats in Point-Stat so that I get overall error stats
which were
>>> derived from all possible sources of upper air observations. As
long as the
>>> source provides measurements of met variables at the various
pressure levels
>>> just as Point-Stat handles now, then this should work for the
overall stats.
>>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>>> observations possible for upper air verification which often
suffers from a
>>> paucity of "ground" truth data.
>>>
>>> Then, I would have the option to go back and re-characterize the
various types
>>> of observations into smaller groupings which focus in on trying to
group based
>>> on the specific type of measurement used.
>>>
>>> For example break out the ship data and using a different mapping
which
>>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>>> That way you can evaluate the contribution/weight of those stats
towards the
>>> overall stats. Any number of different mappings are possible by
re-running
>>> ASCII2NC with different configuration file settings.
>>>
>>> For surface observations where there are often large numbers of
measurements
>>> from disparate sensor types, we are also interested in trying
break down the
>>> verification and restrict the scoring to subdomains and having the
ability to
>>> break out the stats by sensor type would be of value for these
more in-depth
>>> assessments.
>>>
>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>> Hopefully others can chime in here.
>>>
>>> I would like the opportunity to discuss this among colleagues here
in ARL so
>>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>>> I'm sure others have some good input to make about the statistical
>>> implications of grouping the observation types the way I did. I
have probably
>>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>>> might factor into this discussion.
>>>
>>> R/
>>> John
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, August 08, 2013 10:30 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>>> (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>>
>>> John,
>>>
>>> Hmmm, good question. Let me give you some background. We only
recently began
>>> providing support for little_r data, which was driven by a
modelling project
>>> in which the DTC was involved. Our test data was limited to the
data
>>> available for that project. Based on that data, we defined the
following
>>> mappings:
>>>
>>> //
>>> // Load mapping of report types to message types
>>> //
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>
>>> Instances of the little_r input strings on the left are mapped to
the message
>>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>>> me, I see that your little_r data includes the following 22
message types:
>>> "FM-132"
>>> "FM-15_AIRNow"
>>> "FM-15_APRSWXNET"
>>> "FM-15_ASOS"
>>> "FM-15_CA-Hydro"
>>> "FM-15_CODOT"
>>> "FM-15_DDMET"
>>> "FM-15_GPSMET"
>>> "FM-15_HADS"
>>> "FM-15_MAP"
>>> "FM-15_MesoWest"
>>> "FM-15_NERRS"
>>> "FM-15_NonFedAWOS"
>>> "FM-15_NOS-NWLON"
>>> "FM-15_NOS-PORTS"
>>> "FM-15_NVDOT"
>>> "FM-15_OTHER-MTR"
>>> "FM-15_RAWS"
>>> "FM-15_SURFRAD"
>>> "FM-35"
>>> "FM-96"
>>> "FM-97_TAMDAR"
>>>
>>> None of those match the strings we were expecting. So instead of
mapping them
>>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>>> impact is that when you use this data in Point-Stat, you'll need
to list those
>>> message types separately and get output for each one.
>>>
>>> I see several questions here:
>>> (1) Is it reasonable for us to figure out the super-set of all
message types
>>> we can expect to see in little_r data?
>>> (2) If yes, what is that super-set?
>>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>>> should we provide a way for users to define a mapping of input
little_r
>>> strings to output message types? We could do that by providing a
>>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>>> SURFRAD data.
>>>
>>> Do you have any perspectives on these issues?
>>>
>>> Regarding the quality control values, yes, I agree, they are odd.
>>> Unfortunately, I don't have much information for you. I would
suggest
>>> verifying that the values used in the little_r files match the
values that are
>>> showing up in the MET output. But making sense of them is up to
you. In
>>> Point-Stat, you can provide a list of quality control strings to
use in the
>>> obs_quality parameter. For the DTC project using little_r data,
here's the
>>> list of strings we used:
>>>
>>> obs_quality = [
>>> "3", "4", "5", "6", "7", "8",
"9",
>>> "10",
>>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>>> "100010",
>>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>>> "200010",
>>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>>> "300010",
>>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>>> "400010",
>>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>>> "500010",
>>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>>> "600010"
>>> ];
>>>
>>> But it looks like these are not the same values that are showing
up in your
>>> data.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>> Queue: met_help
>>>> Subject: Test of ASCII2NC on little_r file
>>>> Owner: Nobody
>>>> Requestors: john.w.raby2.civ at mail.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>
>>>>
>>>>
>>>> I ran ASCII2NC as follows:
>>>>
>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>>> -v 4
>>>>
>>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>>
>>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>>> good data, but the obs_qty information looks odd in places.
>>>>
>>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>>> ERROR.
>>>>
>>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>>> provide on the reasons for the WARNINGS would be appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Thu Aug 22 14:37:55 2013
John -
Sorry about that. I've attached the input little_r file.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, August 22, 2013 2:33 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
You sent me the NetCDF *output* of the ASCII2NC tool. In order for
you to test the functionality of ASCII2NC with the changes you've
made, I'll need the *input* to that tool. It should be a little_r
file in ascii format. Are you able to send me one of those?
Thanks,
John
On 08/22/2013 02:19 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
>
> John -
>
> I checked the path to the MET software and it was pointed to the
edited little_r_handler.cc file I sent you. I ran ASCII2NC and got the
same results which don't seem to produce report type names like I
wanted (ADPSFC, ADPUPA, ...etc). in the NetCDF output file. The report
type names in the NetCDF file are the one straight out of the little_r
file (attached)
>
> Thanks for your help.
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Wednesday, August 21, 2013 4:16 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
> a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
>
> Thanks,
> John
>
> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>
>> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>>
>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>
>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 13, 2013 8:29 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> John,
>>
>> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
>> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
>> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>
>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>
>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>
>> Thanks,
>> John
>>
>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>>
>>> A more complete description in regard to output from standard
Obsgrid is:
>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>> 4 = pressure derived from standard atmosphere assumption and
height
>>> 8 = pressure derived from standard atmosphere assumption and
temperature
>>> 16 = Temperature/Dewpoint are zero
>>> 32 = Calm wind
>>> 64 = Negative wind speed
>>> 128 = Wind direction did not fall between 0 and 360 degrees
>>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>>> 512 = The observation was assigned a nearby pressure where a
first-guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>>> 32768 = Fails error max check (check against first guess field)
>>> 65536 = Fails buddy check (check against nearby obs)
>>> 131072 = Observation outside of domain
>>>
>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>
>>> Brian
>>>
>>>
>>> -----Original Message-----
>>> From: Raby, John W CIV USARMY ARL (US)
>>> Sent: Thursday, August 08, 2013 3:15 PM
>>> To: met_help at ucar.edu
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> Thanks for this info, your analysis of our results and opening the
opportunity
>>> for a dialog on this topic.
>>>
>>> I think of observational data in the following categories:
>>>
>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy reports
>>> which I use the verify model forecasts for surface met variables
(2meter and
>>> 10meter AGL) using Point-Stat.
>>>
>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>>> VAD (NEXRAD) WIND some of which I use to verify model forecasts
for upper air
>>> variables at various pressure levels using Point-Stat.
>>>
>>> My thoughts to follow are not the final word on this and subject
to input from
>>> other ARL colleagues which I will coordinate before giving you the
final
>>> answers.
>>>
>>> Lump all the sources of surface observations together to produce
error
>>> statistics for surface variables and do the same for upper air
observations
>>> and their error statistics. Then, break out the error statistics
by message
>>> type like Point-Stat does now, so you could isolate a particular
type
>>> (observation source/type) which might be weighing heavily in
contributing to a
>>> particularly large overall error stat.
>>>
>>> That said, I don't want to generate a big change for the Point-
Stat tool. I
>>> may be forgetting some features of Stat-Analysis which would allow
me to
>>> aggregate/stratify the error stats in order to show the stats I
mentioned
>>> above.
>>>
>>> I'm tending towards the latter suggestion you made in (3) about
the user
>>> setting the mapping of input little_r strings to output message
types. That
>>> way I could control the generation of error stats in Point-Stat to
adhere to
>>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>>> The following example describes one such general mapping for
surface
>>> verification:
>>>
>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>
>>>
>>>
>>> In the above mapping, I'm assuming that the little_r strings I
selected are
>>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>>> select all the strings which were in the data, but just enough to
give you and
>>> others an idea. I also assume that I obtained MADIS observations
for the
>>> additional strings (synop, ship, metar, buoy) which were not
present in the
>>> iittle_r file we used in this test.
>>>
>>> Then for upper air verification, I would use the following:
>>>
>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>>
>>>
>>> The above list could be expanded with more types of upper air
data, but I
>>> think you get the idea. What I've done with this is to set up the
generation
>>> of error stats in Point-Stat so that I get overall error stats
which were
>>> derived from all possible sources of upper air observations. As
long as the
>>> source provides measurements of met variables at the various
pressure levels
>>> just as Point-Stat handles now, then this should work for the
overall stats.
>>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>>> observations possible for upper air verification which often
suffers from a
>>> paucity of "ground" truth data.
>>>
>>> Then, I would have the option to go back and re-characterize the
various types
>>> of observations into smaller groupings which focus in on trying to
group based
>>> on the specific type of measurement used.
>>>
>>> For example break out the ship data and using a different mapping
which
>>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>>> That way you can evaluate the contribution/weight of those stats
towards the
>>> overall stats. Any number of different mappings are possible by
re-running
>>> ASCII2NC with different configuration file settings.
>>>
>>> For surface observations where there are often large numbers of
measurements
>>> from disparate sensor types, we are also interested in trying
break down the
>>> verification and restrict the scoring to subdomains and having the
ability to
>>> break out the stats by sensor type would be of value for these
more in-depth
>>> assessments.
>>>
>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>> Hopefully others can chime in here.
>>>
>>> I would like the opportunity to discuss this among colleagues here
in ARL so
>>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>>> I'm sure others have some good input to make about the statistical
>>> implications of grouping the observation types the way I did. I
have probably
>>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>>> might factor into this discussion.
>>>
>>> R/
>>> John
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, August 08, 2013 10:30 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>>> (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>>
>>> John,
>>>
>>> Hmmm, good question. Let me give you some background. We only
recently began
>>> providing support for little_r data, which was driven by a
modelling project
>>> in which the DTC was involved. Our test data was limited to the
data
>>> available for that project. Based on that data, we defined the
following
>>> mappings:
>>>
>>> //
>>> // Load mapping of report types to message types
>>> //
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>
>>> Instances of the little_r input strings on the left are mapped to
the message
>>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>>> me, I see that your little_r data includes the following 22
message types:
>>> "FM-132"
>>> "FM-15_AIRNow"
>>> "FM-15_APRSWXNET"
>>> "FM-15_ASOS"
>>> "FM-15_CA-Hydro"
>>> "FM-15_CODOT"
>>> "FM-15_DDMET"
>>> "FM-15_GPSMET"
>>> "FM-15_HADS"
>>> "FM-15_MAP"
>>> "FM-15_MesoWest"
>>> "FM-15_NERRS"
>>> "FM-15_NonFedAWOS"
>>> "FM-15_NOS-NWLON"
>>> "FM-15_NOS-PORTS"
>>> "FM-15_NVDOT"
>>> "FM-15_OTHER-MTR"
>>> "FM-15_RAWS"
>>> "FM-15_SURFRAD"
>>> "FM-35"
>>> "FM-96"
>>> "FM-97_TAMDAR"
>>>
>>> None of those match the strings we were expecting. So instead of
mapping them
>>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>>> impact is that when you use this data in Point-Stat, you'll need
to list those
>>> message types separately and get output for each one.
>>>
>>> I see several questions here:
>>> (1) Is it reasonable for us to figure out the super-set of all
message types
>>> we can expect to see in little_r data?
>>> (2) If yes, what is that super-set?
>>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>>> should we provide a way for users to define a mapping of input
little_r
>>> strings to output message types? We could do that by providing a
>>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>>> SURFRAD data.
>>>
>>> Do you have any perspectives on these issues?
>>>
>>> Regarding the quality control values, yes, I agree, they are odd.
>>> Unfortunately, I don't have much information for you. I would
suggest
>>> verifying that the values used in the little_r files match the
values that are
>>> showing up in the MET output. But making sense of them is up to
you. In
>>> Point-Stat, you can provide a list of quality control strings to
use in the
>>> obs_quality parameter. For the DTC project using little_r data,
here's the
>>> list of strings we used:
>>>
>>> obs_quality = [
>>> "3", "4", "5", "6", "7", "8",
"9",
>>> "10",
>>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>>> "100010",
>>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>>> "200010",
>>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>>> "300010",
>>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>>> "400010",
>>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>>> "500010",
>>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>>> "600010"
>>> ];
>>>
>>> But it looks like these are not the same values that are showing
up in your
>>> data.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>> Queue: met_help
>>>> Subject: Test of ASCII2NC on little_r file
>>>> Owner: Nobody
>>>> Requestors: john.w.raby2.civ at mail.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>
>>>>
>>>>
>>>> I ran ASCII2NC as follows:
>>>>
>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>>> -v 4
>>>>
>>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>>
>>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>>> good data, but the obs_qty information looks odd in places.
>>>>
>>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>>> ERROR.
>>>>
>>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>>> provide on the reasons for the WARNINGS would be appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Aug 22 14:41:14 2013
John -
Update on the test is the following:
I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Wednesday, August 21, 2013 4:16 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
Thanks,
John
On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>
> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>
> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>
> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>
> Thanks.
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 13, 2013 8:29 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>
> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>
> For now, in order to get ASCII2NC to do what you'd like, I'd suggest
editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>
> Thanks,
> John
>
> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>
>> A more complete description in regard to output from standard
Obsgrid is:
>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>> 4 = pressure derived from standard atmosphere assumption and height
>> 8 = pressure derived from standard atmosphere assumption and
temperature
>> 16 = Temperature/Dewpoint are zero
>> 32 = Calm wind
>> 64 = Negative wind speed
>> 128 = Wind direction did not fall between 0 and 360 degrees
>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>> 512 = The observation was assigned a nearby pressure where a first-
guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>> 32768 = Fails error max check (check against first guess field)
>> 65536 = Fails buddy check (check against nearby obs)
>> 131072 = Observation outside of domain
>>
>> The QC codes are powers of 2 to enable the QC flags to be extracted
from a single value.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Thursday, August 08, 2013 3:15 PM
>> To: met_help at ucar.edu
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Thanks for this info, your analysis of our results and opening the
opportunity
>> for a dialog on this topic.
>>
>> I think of observational data in the following categories:
>>
>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and buoy
reports
>> which I use the verify model forecasts for surface met variables
(2meter and
>> 10meter AGL) using Point-Stat.
>>
>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>> VAD (NEXRAD) WIND some of which I use to verify model forecasts for
upper air
>> variables at various pressure levels using Point-Stat.
>>
>> My thoughts to follow are not the final word on this and subject to
input from
>> other ARL colleagues which I will coordinate before giving you the
final
>> answers.
>>
>> Lump all the sources of surface observations together to produce
error
>> statistics for surface variables and do the same for upper air
observations
>> and their error statistics. Then, break out the error statistics by
message
>> type like Point-Stat does now, so you could isolate a particular
type
>> (observation source/type) which might be weighing heavily in
contributing to a
>> particularly large overall error stat.
>>
>> That said, I don't want to generate a big change for the Point-Stat
tool. I
>> may be forgetting some features of Stat-Analysis which would allow
me to
>> aggregate/stratify the error stats in order to show the stats I
mentioned
>> above.
>>
>> I'm tending towards the latter suggestion you made in (3) about the
user
>> setting the mapping of input little_r strings to output message
types. That
>> way I could control the generation of error stats in Point-Stat to
adhere to
>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>> The following example describes one such general mapping for
surface
>> verification:
>>
>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>
>>
>>
>> In the above mapping, I'm assuming that the little_r strings I
selected are
>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>> select all the strings which were in the data, but just enough to
give you and
>> others an idea. I also assume that I obtained MADIS observations
for the
>> additional strings (synop, ship, metar, buoy) which were not
present in the
>> iittle_r file we used in this test.
>>
>> Then for upper air verification, I would use the following:
>>
>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>
>>
>> The above list could be expanded with more types of upper air data,
but I
>> think you get the idea. What I've done with this is to set up the
generation
>> of error stats in Point-Stat so that I get overall error stats
which were
>> derived from all possible sources of upper air observations. As
long as the
>> source provides measurements of met variables at the various
pressure levels
>> just as Point-Stat handles now, then this should work for the
overall stats.
>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>> observations possible for upper air verification which often
suffers from a
>> paucity of "ground" truth data.
>>
>> Then, I would have the option to go back and re-characterize the
various types
>> of observations into smaller groupings which focus in on trying to
group based
>> on the specific type of measurement used.
>>
>> For example break out the ship data and using a different mapping
which
>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>> That way you can evaluate the contribution/weight of those stats
towards the
>> overall stats. Any number of different mappings are possible by re-
running
>> ASCII2NC with different configuration file settings.
>>
>> For surface observations where there are often large numbers of
measurements
>> from disparate sensor types, we are also interested in trying break
down the
>> verification and restrict the scoring to subdomains and having the
ability to
>> break out the stats by sensor type would be of value for these more
in-depth
>> assessments.
>>
>> Regard the quality control values, I'm not sure what to think on
this yet.
>> Hopefully others can chime in here.
>>
>> I would like the opportunity to discuss this among colleagues here
in ARL so
>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>> I'm sure others have some good input to make about the statistical
>> implications of grouping the observation types the way I did. I
have probably
>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>> might factor into this discussion.
>>
>> R/
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, August 08, 2013 10:30 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>> (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>
>> John,
>>
>> Hmmm, good question. Let me give you some background. We only
recently began
>> providing support for little_r data, which was driven by a
modelling project
>> in which the DTC was involved. Our test data was limited to the
data
>> available for that project. Based on that data, we defined the
following
>> mappings:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>
>> Instances of the little_r input strings on the left are mapped to
the message
>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>> me, I see that your little_r data includes the following 22 message
types:
>> "FM-132"
>> "FM-15_AIRNow"
>> "FM-15_APRSWXNET"
>> "FM-15_ASOS"
>> "FM-15_CA-Hydro"
>> "FM-15_CODOT"
>> "FM-15_DDMET"
>> "FM-15_GPSMET"
>> "FM-15_HADS"
>> "FM-15_MAP"
>> "FM-15_MesoWest"
>> "FM-15_NERRS"
>> "FM-15_NonFedAWOS"
>> "FM-15_NOS-NWLON"
>> "FM-15_NOS-PORTS"
>> "FM-15_NVDOT"
>> "FM-15_OTHER-MTR"
>> "FM-15_RAWS"
>> "FM-15_SURFRAD"
>> "FM-35"
>> "FM-96"
>> "FM-97_TAMDAR"
>>
>> None of those match the strings we were expecting. So instead of
mapping them
>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>> impact is that when you use this data in Point-Stat, you'll need to
list those
>> message types separately and get output for each one.
>>
>> I see several questions here:
>> (1) Is it reasonable for us to figure out the super-set of all
message types
>> we can expect to see in little_r data?
>> (2) If yes, what is that super-set?
>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>> should we provide a way for users to define a mapping of input
little_r
>> strings to output message types? We could do that by providing a
>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>> SURFRAD data.
>>
>> Do you have any perspectives on these issues?
>>
>> Regarding the quality control values, yes, I agree, they are odd.
>> Unfortunately, I don't have much information for you. I would
suggest
>> verifying that the values used in the little_r files match the
values that are
>> showing up in the MET output. But making sense of them is up to
you. In
>> Point-Stat, you can provide a list of quality control strings to
use in the
>> obs_quality parameter. For the DTC project using little_r data,
here's the
>> list of strings we used:
>>
>> obs_quality = [
>> "3", "4", "5", "6", "7", "8",
"9",
>> "10",
>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>> "100010",
>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>> "200010",
>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>> "300010",
>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>> "400010",
>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>> "500010",
>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>> "600010"
>> ];
>>
>> But it looks like these are not the same values that are showing up
in your
>> data.
>>
>> Thanks,
>> John
>>
>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Test of ASCII2NC on little_r file
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>
>>>
>>>
>>> I ran ASCII2NC as follows:
>>>
>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>> -v 4
>>>
>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>
>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>> good data, but the obs_qty information looks odd in places.
>>>
>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>> ERROR.
>>>
>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>> provide on the reasons for the WARNINGS would be appreciated.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Aug 27 11:58:18 2013
John,
Sorry for the delay in getting back to you. I ran with your modified
version of ASCII2NC and see the problem. You were using underscores
where there should have been spaces. Here's why...
Take a look at this warning message from ASCII2NC:
WARNING: LittleRHandler::_processObs() -> Storing message type as
"FM-15_APRSWXNET" for unexpected report type "FM-15 APRSWXNET".
This is saying that the tool encountered the report type "FM-15
APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The tool
replaced that space with an underscore because we don't allow
spaces in the message type. Otherwise, it'd mess up the space-
separated columns of output from Point-Stat.
All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
MAP_MSG_TYP["FM-132" ] = "PROFLR";
MAP_MSG_TYP["FM-35" ] = "ADPUPA";
MAP_MSG_TYP["FM-96" ] = "AIRCAR";
MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
Then it should work fine.
Thanks,
John
On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Update on the test is the following:
>
> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>
> R/
> John
>
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Wednesday, August 21, 2013 4:16 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
> a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
>
> Thanks,
> John
>
> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>
>> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>>
>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>
>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 13, 2013 8:29 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> John,
>>
>> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
>> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
>> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>
>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>
>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>
>> Thanks,
>> John
>>
>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>>
>>> A more complete description in regard to output from standard
Obsgrid is:
>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>> 4 = pressure derived from standard atmosphere assumption and
height
>>> 8 = pressure derived from standard atmosphere assumption and
temperature
>>> 16 = Temperature/Dewpoint are zero
>>> 32 = Calm wind
>>> 64 = Negative wind speed
>>> 128 = Wind direction did not fall between 0 and 360 degrees
>>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>>> 512 = The observation was assigned a nearby pressure where a
first-guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>>> 32768 = Fails error max check (check against first guess field)
>>> 65536 = Fails buddy check (check against nearby obs)
>>> 131072 = Observation outside of domain
>>>
>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>
>>> Brian
>>>
>>>
>>> -----Original Message-----
>>> From: Raby, John W CIV USARMY ARL (US)
>>> Sent: Thursday, August 08, 2013 3:15 PM
>>> To: met_help at ucar.edu
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> Thanks for this info, your analysis of our results and opening the
opportunity
>>> for a dialog on this topic.
>>>
>>> I think of observational data in the following categories:
>>>
>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy reports
>>> which I use the verify model forecasts for surface met variables
(2meter and
>>> 10meter AGL) using Point-Stat.
>>>
>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>>> VAD (NEXRAD) WIND some of which I use to verify model forecasts
for upper air
>>> variables at various pressure levels using Point-Stat.
>>>
>>> My thoughts to follow are not the final word on this and subject
to input from
>>> other ARL colleagues which I will coordinate before giving you the
final
>>> answers.
>>>
>>> Lump all the sources of surface observations together to produce
error
>>> statistics for surface variables and do the same for upper air
observations
>>> and their error statistics. Then, break out the error statistics
by message
>>> type like Point-Stat does now, so you could isolate a particular
type
>>> (observation source/type) which might be weighing heavily in
contributing to a
>>> particularly large overall error stat.
>>>
>>> That said, I don't want to generate a big change for the Point-
Stat tool. I
>>> may be forgetting some features of Stat-Analysis which would allow
me to
>>> aggregate/stratify the error stats in order to show the stats I
mentioned
>>> above.
>>>
>>> I'm tending towards the latter suggestion you made in (3) about
the user
>>> setting the mapping of input little_r strings to output message
types. That
>>> way I could control the generation of error stats in Point-Stat to
adhere to
>>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>>> The following example describes one such general mapping for
surface
>>> verification:
>>>
>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>
>>>
>>>
>>> In the above mapping, I'm assuming that the little_r strings I
selected are
>>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>>> select all the strings which were in the data, but just enough to
give you and
>>> others an idea. I also assume that I obtained MADIS observations
for the
>>> additional strings (synop, ship, metar, buoy) which were not
present in the
>>> iittle_r file we used in this test.
>>>
>>> Then for upper air verification, I would use the following:
>>>
>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>>
>>>
>>> The above list could be expanded with more types of upper air
data, but I
>>> think you get the idea. What I've done with this is to set up the
generation
>>> of error stats in Point-Stat so that I get overall error stats
which were
>>> derived from all possible sources of upper air observations. As
long as the
>>> source provides measurements of met variables at the various
pressure levels
>>> just as Point-Stat handles now, then this should work for the
overall stats.
>>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>>> observations possible for upper air verification which often
suffers from a
>>> paucity of "ground" truth data.
>>>
>>> Then, I would have the option to go back and re-characterize the
various types
>>> of observations into smaller groupings which focus in on trying to
group based
>>> on the specific type of measurement used.
>>>
>>> For example break out the ship data and using a different mapping
which
>>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>>> That way you can evaluate the contribution/weight of those stats
towards the
>>> overall stats. Any number of different mappings are possible by
re-running
>>> ASCII2NC with different configuration file settings.
>>>
>>> For surface observations where there are often large numbers of
measurements
>>> from disparate sensor types, we are also interested in trying
break down the
>>> verification and restrict the scoring to subdomains and having the
ability to
>>> break out the stats by sensor type would be of value for these
more in-depth
>>> assessments.
>>>
>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>> Hopefully others can chime in here.
>>>
>>> I would like the opportunity to discuss this among colleagues here
in ARL so
>>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>>> I'm sure others have some good input to make about the statistical
>>> implications of grouping the observation types the way I did. I
have probably
>>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>>> might factor into this discussion.
>>>
>>> R/
>>> John
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, August 08, 2013 10:30 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>>> (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>>
>>> John,
>>>
>>> Hmmm, good question. Let me give you some background. We only
recently began
>>> providing support for little_r data, which was driven by a
modelling project
>>> in which the DTC was involved. Our test data was limited to the
data
>>> available for that project. Based on that data, we defined the
following
>>> mappings:
>>>
>>> //
>>> // Load mapping of report types to message types
>>> //
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>
>>> Instances of the little_r input strings on the left are mapped to
the message
>>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>>> me, I see that your little_r data includes the following 22
message types:
>>> "FM-132"
>>> "FM-15_AIRNow"
>>> "FM-15_APRSWXNET"
>>> "FM-15_ASOS"
>>> "FM-15_CA-Hydro"
>>> "FM-15_CODOT"
>>> "FM-15_DDMET"
>>> "FM-15_GPSMET"
>>> "FM-15_HADS"
>>> "FM-15_MAP"
>>> "FM-15_MesoWest"
>>> "FM-15_NERRS"
>>> "FM-15_NonFedAWOS"
>>> "FM-15_NOS-NWLON"
>>> "FM-15_NOS-PORTS"
>>> "FM-15_NVDOT"
>>> "FM-15_OTHER-MTR"
>>> "FM-15_RAWS"
>>> "FM-15_SURFRAD"
>>> "FM-35"
>>> "FM-96"
>>> "FM-97_TAMDAR"
>>>
>>> None of those match the strings we were expecting. So instead of
mapping them
>>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>>> impact is that when you use this data in Point-Stat, you'll need
to list those
>>> message types separately and get output for each one.
>>>
>>> I see several questions here:
>>> (1) Is it reasonable for us to figure out the super-set of all
message types
>>> we can expect to see in little_r data?
>>> (2) If yes, what is that super-set?
>>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>>> should we provide a way for users to define a mapping of input
little_r
>>> strings to output message types? We could do that by providing a
>>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>>> SURFRAD data.
>>>
>>> Do you have any perspectives on these issues?
>>>
>>> Regarding the quality control values, yes, I agree, they are odd.
>>> Unfortunately, I don't have much information for you. I would
suggest
>>> verifying that the values used in the little_r files match the
values that are
>>> showing up in the MET output. But making sense of them is up to
you. In
>>> Point-Stat, you can provide a list of quality control strings to
use in the
>>> obs_quality parameter. For the DTC project using little_r data,
here's the
>>> list of strings we used:
>>>
>>> obs_quality = [
>>> "3", "4", "5", "6", "7", "8",
"9",
>>> "10",
>>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>>> "100010",
>>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>>> "200010",
>>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>>> "300010",
>>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>>> "400010",
>>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>>> "500010",
>>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>>> "600010"
>>> ];
>>>
>>> But it looks like these are not the same values that are showing
up in your
>>> data.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>> Queue: met_help
>>>> Subject: Test of ASCII2NC on little_r file
>>>> Owner: Nobody
>>>> Requestors: john.w.raby2.civ at mail.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>
>>>>
>>>>
>>>> I ran ASCII2NC as follows:
>>>>
>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>>> -v 4
>>>>
>>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>>
>>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>>> good data, but the obs_qty information looks odd in places.
>>>>
>>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>>> ERROR.
>>>>
>>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>>> provide on the reasons for the WARNINGS would be appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Tue Aug 27 12:27:52 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
No apology necessary. Instead, I owe you one for not spotting this
one!
I'll edit the file per your example, run another test and let you
know.
On our preference for a method to allow users to control the mapping
of little_r report strings to the PrepBUFR message types is per your
earlier suggestion:
(3) "...provide a way for users to define a mapping of
input little_r strings to output message types...by
providing a configuration file for the ASCII2NC tool"
Thanks for your help.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, August 27, 2013 11:58 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
Sorry for the delay in getting back to you. I ran with your modified
version of ASCII2NC and see the problem. You were using underscores
where there should have been spaces. Here's why...
Take a look at this warning message from ASCII2NC:
WARNING: LittleRHandler::_processObs() -> Storing message type as
"FM-15_APRSWXNET" for unexpected report type "FM-15 APRSWXNET".
This is saying that the tool encountered the report type "FM-15
APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The tool
replaced that space with an underscore because we don't allow spaces
in the message type. Otherwise, it'd mess up the space-separated
columns of output from Point-Stat.
All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
MAP_MSG_TYP["FM-132" ] = "PROFLR";
MAP_MSG_TYP["FM-35" ] = "ADPUPA";
MAP_MSG_TYP["FM-96" ] = "AIRCAR";
MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
Then it should work fine.
Thanks,
John
On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Update on the test is the following:
>
> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>
> R/
> John
>
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Wednesday, August 21, 2013 4:16 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
> rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
> file (UNCLASSIFIED)
>
> John,
>
> I'm not sure why your mapping did not work as expected. I could try
> running it here with your updated version of little_r_handler.cc,
but I'd need little_r file you're using. Are you able to send me a
sample one to work with? I'd run it through ascii2nc and inspect the
NetCDF output to see if it's working as expected.
>
> Thanks,
> John
>
> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>
>> This is quite different from my first Point-Stat test last week
where
>> the little_r_handler.cc file was the default one. In that case, I
>> used your suggestion about putting the actual report string names
>> from the little_r file in the message_type array in the config
file.
>> That test had varying numbers of pairs found depending on the
number
>> of observations found for a given report type/level and produced
>> stats for all report types/levels except for the variables at 2m
and
>> 10m AGL I had specified in the config file. I'm still trying to
>> figure out why no results for these AGL levels
>>
>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>
>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 13, 2013 8:29 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>> file (UNCLASSIFIED)
>>
>> John,
>>
>> One of the scientists here is looking more into the data types for
>> little_r observations. We're trying to figure out if we can define
>> the super-set of all possible message type strings that may show up
in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>
>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>
>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest
>> editing this file:
>> METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>
>> Thanks,
>> John
>>
>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_cha
>>> p7.htm#format
>>>
>>> A more complete description in regard to output from standard
Obsgrid is:
>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>> 4 = pressure derived from standard atmosphere assumption and
height
>>> 8 = pressure derived from standard atmosphere assumption and
>>> temperature
>>> 16 = Temperature/Dewpoint are zero
>>> 32 = Calm wind
>>> 64 = Negative wind speed
>>> 128 = Wind direction did not fall between 0 and 360 degrees [129]
=
>>> WRF uses this to indicate winds that are earth-relative (instead
of grid-relative).
>>> 256 = Value was vertically interpolated (to get to level with
>>> first-guess field to QC against)
>>> 512 = The observation was assigned a nearby pressure where a
>>> first-guess field was available for qc (some variables may have
been
>>> adjusted to account for the change in pressure) (this only applies
>>> for obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent
>>> temperatures in the vertical profile so OBSGRID flipped the sign
of
>>> the temperature and adjusted dewpoint/RH
>>> 2048 = A superadiabatic temperature remains after convective
>>> adjustment (results in OBSGRID marking temperature, dewpoint, and
>>> relative humidity as missing)
>>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>>> 8192 = Dry convective adjustment used in attempt to remove
>>> superadiabatic level
>>> 16384 = No buddies (nearby observations) were found to do a buddy
>>> check
>>> 32768 = Fails error max check (check against first guess field)
>>> 65536 = Fails buddy check (check against nearby obs)
>>> 131072 = Observation outside of domain
>>>
>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>
>>> Brian
>>>
>>>
>>> -----Original Message-----
>>> From: Raby, John W CIV USARMY ARL (US)
>>> Sent: Thursday, August 08, 2013 3:15 PM
>>> To: met_help at ucar.edu
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
>>> Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
>>> Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>>> file (UNCLASSIFIED)
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> Thanks for this info, your analysis of our results and opening the
>>> opportunity for a dialog on this topic.
>>>
>>> I think of observational data in the following categories:
>>>
>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy
>>> reports which I use the verify model forecasts for surface met
>>> variables (2meter and 10meter AGL) using Point-Stat.
>>>
>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
>>> AIRCRAFT, AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC
>>> SOUNDER (SODAR), VAD (NEXRAD) WIND some of which I use to verify
>>> model forecasts for upper air variables at various pressure levels
using Point-Stat.
>>>
>>> My thoughts to follow are not the final word on this and subject
to
>>> input from other ARL colleagues which I will coordinate before
>>> giving you the final answers.
>>>
>>> Lump all the sources of surface observations together to produce
>>> error statistics for surface variables and do the same for upper
air
>>> observations and their error statistics. Then, break out the error
>>> statistics by message type like Point-Stat does now, so you could
>>> isolate a particular type (observation source/type) which might be
>>> weighing heavily in contributing to a particularly large overall
error stat.
>>>
>>> That said, I don't want to generate a big change for the Point-
Stat
>>> tool. I may be forgetting some features of Stat-Analysis which
would
>>> allow me to aggregate/stratify the error stats in order to show
the
>>> stats I mentioned above.
>>>
>>> I'm tending towards the latter suggestion you made in (3) about
the
>>> user setting the mapping of input little_r strings to output
message
>>> types. That way I could control the generation of error stats in
>>> Point-Stat to adhere to the groupings I mentioned above. Then, if
>>> the need arose, I could rerun the ASCII2NC with the mapping
changed to regroup the little_r strings differently.
>>> The following example describes one such general mapping for
surface
>>> verification:
>>>
>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_ASOS
>>> " ] = "ADPSFC"; MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_CODOT"
>>> ] = "ADPSFC"; MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>
>>>
>>>
>>> In the above mapping, I'm assuming that the little_r strings I
>>> selected are all observations of surface met variables (2 and 10
>>> meter AGL). I didn't select all the strings which were in the
data,
>>> but just enough to give you and others an idea. I also assume that
I
>>> obtained MADIS observations for the additional strings (synop,
ship,
>>> metar, buoy) which were not present in the iittle_r file we used
in this test.
>>>
>>> Then for upper air verification, I would use the following:
>>>
>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX RAOB"
]
>>> = "ADPUPA"; MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX VADWND"
]
>>> = "ADPUPA";
>>>
>>>
>>> The above list could be expanded with more types of upper air
data,
>>> but I think you get the idea. What I've done with this is to set
up
>>> the generation of error stats in Point-Stat so that I get overall
>>> error stats which were derived from all possible sources of upper
>>> air observations. As long as the source provides measurements of
met
>>> variables at the various pressure levels just as Point-Stat
handles now, then this should work for the overall stats.
>>> For these stats I'm trying to capitalize on acquiring the maximum
>>> number of observations possible for upper air verification which
>>> often suffers from a paucity of "ground" truth data.
>>>
>>> Then, I would have the option to go back and re-characterize the
>>> various types of observations into smaller groupings which focus
in
>>> on trying to group based on the specific type of measurement used.
>>>
>>> For example break out the ship data and using a different mapping
>>> which characterizes it as "SFCSHP" so that its stats are
calculated
>>> separately in Point-Stat or break out the profiler data by
characterizing it as "PROFLR".
>>> That way you can evaluate the contribution/weight of those stats
>>> towards the overall stats. Any number of different mappings are
>>> possible by re-running ASCII2NC with different configuration file
settings.
>>>
>>> For surface observations where there are often large numbers of
>>> measurements from disparate sensor types, we are also interested
in
>>> trying break down the verification and restrict the scoring to
>>> subdomains and having the ability to break out the stats by sensor
>>> type would be of value for these more in-depth assessments.
>>>
>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>> Hopefully others can chime in here.
>>>
>>> I would like the opportunity to discuss this among colleagues here
>>> in ARL so that I'm not misinterpreting anything here or am making
invalid assumptions.
>>> I'm sure others have some good input to make about the statistical
>>> implications of grouping the observation types the way I did. I
have
>>> probably omitted the kind of stratifications which can be made
with
>>> Stat-Analysis which might factor into this discussion.
>>>
>>> R/
>>> John
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, August 08, 2013 10:30 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
>>> Robert T CIV
>>> (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>>> file
>>>
>>> John,
>>>
>>> Hmmm, good question. Let me give you some background. We only
>>> recently began providing support for little_r data, which was
driven
>>> by a modelling project in which the DTC was involved. Our test
data
>>> was limited to the data available for that project. Based on that
>>> data, we defined the following
>>> mappings:
>>>
>>> //
>>> // Load mapping of report types to message types
>>> //
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>
>>> Instances of the little_r input strings on the left are mapped to
>>> the message types used in the ASCII2NC output on the right.
Looking
>>> at the data you sent me, I see that your little_r data includes
the following 22 message types:
>>> "FM-132"
>>> "FM-15_AIRNow"
>>> "FM-15_APRSWXNET"
>>> "FM-15_ASOS"
>>> "FM-15_CA-Hydro"
>>> "FM-15_CODOT"
>>> "FM-15_DDMET"
>>> "FM-15_GPSMET"
>>> "FM-15_HADS"
>>> "FM-15_MAP"
>>> "FM-15_MesoWest"
>>> "FM-15_NERRS"
>>> "FM-15_NonFedAWOS"
>>> "FM-15_NOS-NWLON"
>>> "FM-15_NOS-PORTS"
>>> "FM-15_NVDOT"
>>> "FM-15_OTHER-MTR"
>>> "FM-15_RAWS"
>>> "FM-15_SURFRAD"
>>> "FM-35"
>>> "FM-96"
>>> "FM-97_TAMDAR"
>>>
>>> None of those match the strings we were expecting. So instead of
>>> mapping them to the PREPBUFR message types, we just wrote the
>>> strings that were input. The impact is that when you use this
data
>>> in Point-Stat, you'll need to list those message types separately
and get output for each one.
>>>
>>> I see several questions here:
>>> (1) Is it reasonable for us to figure out the super-set of all
>>> message types we can expect to see in little_r data?
>>> (2) If yes, what is that super-set?
>>> (3) If no, is it OK for us to just use in the input string in the
>>> output? Or should we provide a way for users to define a mapping
of
>>> input little_r strings to output message types? We could do that
by
>>> providing a configuration file for the ASCII2NC tool - which is
>>> already used for ASCII SURFRAD data.
>>>
>>> Do you have any perspectives on these issues?
>>>
>>> Regarding the quality control values, yes, I agree, they are odd.
>>> Unfortunately, I don't have much information for you. I would
>>> suggest verifying that the values used in the little_r files match
>>> the values that are showing up in the MET output. But making
sense
>>> of them is up to you. In Point-Stat, you can provide a list of
>>> quality control strings to use in the obs_quality parameter. For
>>> the DTC project using little_r data, here's the list of strings we
used:
>>>
>>> obs_quality = [
>>> "3", "4", "5", "6", "7", "8",
"9",
>>> "10",
>>> "100003", "100004", "100005", "100006", "100007", "100008",
>>> "100009", "100010",
>>> "200003", "200004", "200005", "200006", "200007", "200008",
>>> "200009", "200010",
>>> "300003", "300004", "300005", "300006", "300007", "300008",
>>> "300009", "300010",
>>> "400003", "400004", "400005", "400006", "400007", "400008",
>>> "400009", "400010",
>>> "500003", "500004", "500005", "500006", "500007", "500008",
>>> "500009", "500010",
>>> "600003", "600004", "600005", "600006", "600007", "600008",
>>> "600009", "600010"
>>> ];
>>>
>>> But it looks like these are not the same values that are showing
up
>>> in your data.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>> Queue: met_help
>>>> Subject: Test of ASCII2NC on little_r file
>>>> Owner: Nobody
>>>> Requestors: john.w.raby2.civ at mail.mil
>>>> Status: new
>>>> Ticket <URL:
>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>
>>>>
>>>>
>>>> I ran ASCII2NC as follows:
>>>>
>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
>>>> test7 -v 4
>>>>
>>>> It appears that the LittleRHandler is working to indicate that it
>>>> recognizes the input file as a little_r file and then generates
the NetCDF output file.
>>>>
>>>> The output file attached as the text file " test7_ncdump" appears
>>>> to contain good data, but the obs_qty information looks odd in
places.
>>>>
>>>> I've also attached the log file which contain extensive WARNINGS,
>>>> but no ERROR.
>>>>
>>>> Can you explain the odd numbers in the obs_qty array? I was
>>>> expecting numbers like 0, 1, 2, etc but not "512" or "16384".
Also
>>>> any help you can provide on the reasons for the WARNINGS would be
appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>
>>
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Tue Aug 27 15:13:22 2013
John -
I edited the little_r_handler.cc file and Bob Flanigan recompiled MET.
I reran ASCII2NC on the test little_r file and the results look much
more encouraging.
The data in the attached ncdmp of the output NetCDF file shows no
hdr_typ names other than the PrepBUFR message type names. I found
observational data for all the message types I had specified in the
little_r_handler.cc where I had mapped every report type string which
was present on the little_r file. I've also attached a copy of the log
file (test10). I noticed that there was only one RAOB report (ADPUPA)
which was surprising with so many observations, but maybe the time
being 1800Z explains that.
The log file had 24 WARNINGS about the number of data lines in the
header not matching the number of lines in the data. I'm not sure what
this problem is.
Does this result look OK to you?
I will set up a Point-Stat test to see how it handles the NetCDF file
and let you know how that comes out.
Thanks.
R/
John
_______________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, August 27, 2013 11:58 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
Sorry for the delay in getting back to you. I ran with your modified
version of ASCII2NC and see the problem. You were using underscores
where there should have been spaces. Here's why...
Take a look at this warning message from ASCII2NC:
WARNING: LittleRHandler::_processObs() -> Storing message type as
"FM-15_APRSWXNET" for unexpected report type "FM-15 APRSWXNET".
This is saying that the tool encountered the report type "FM-15
APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The tool
replaced that space with an underscore because we don't allow
spaces in the message type. Otherwise, it'd mess up the space-
separated columns of output from Point-Stat.
All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
//
// Load mapping of report types to message types
//
MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
MAP_MSG_TYP["FM-132" ] = "PROFLR";
MAP_MSG_TYP["FM-35" ] = "ADPUPA";
MAP_MSG_TYP["FM-96" ] = "AIRCAR";
MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
Then it should work fine.
Thanks,
John
On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> Update on the test is the following:
>
> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>
> R/
> John
>
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Wednesday, August 21, 2013 4:16 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> I'm not sure why your mapping did not work as expected. I could try
running it here with your updated version of little_r_handler.cc, but
I'd need little_r file you're using. Are you able to send me
> a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
>
> Thanks,
> John
>
> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>
>> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>>
>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>
>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 13, 2013 8:29 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> John,
>>
>> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
>> up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
>> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>
>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>
>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>
>> Thanks,
>> John
>>
>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>>
>>> A more complete description in regard to output from standard
Obsgrid is:
>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>> 4 = pressure derived from standard atmosphere assumption and
height
>>> 8 = pressure derived from standard atmosphere assumption and
temperature
>>> 16 = Temperature/Dewpoint are zero
>>> 32 = Calm wind
>>> 64 = Negative wind speed
>>> 128 = Wind direction did not fall between 0 and 360 degrees
>>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>>> 512 = The observation was assigned a nearby pressure where a
first-guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>>> 4096 = Spike in wind (results in OBSGRID marking winds as missing)
>>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>>> 32768 = Fails error max check (check against first guess field)
>>> 65536 = Fails buddy check (check against nearby obs)
>>> 131072 = Observation outside of domain
>>>
>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>
>>> Brian
>>>
>>>
>>> -----Original Message-----
>>> From: Raby, John W CIV USARMY ARL (US)
>>> Sent: Thursday, August 08, 2013 3:15 PM
>>> To: met_help at ucar.edu
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> Thanks for this info, your analysis of our results and opening the
opportunity
>>> for a dialog on this topic.
>>>
>>> I think of observational data in the following categories:
>>>
>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy reports
>>> which I use the verify model forecasts for surface met variables
(2meter and
>>> 10meter AGL) using Point-Stat.
>>>
>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>>> VAD (NEXRAD) WIND some of which I use to verify model forecasts
for upper air
>>> variables at various pressure levels using Point-Stat.
>>>
>>> My thoughts to follow are not the final word on this and subject
to input from
>>> other ARL colleagues which I will coordinate before giving you the
final
>>> answers.
>>>
>>> Lump all the sources of surface observations together to produce
error
>>> statistics for surface variables and do the same for upper air
observations
>>> and their error statistics. Then, break out the error statistics
by message
>>> type like Point-Stat does now, so you could isolate a particular
type
>>> (observation source/type) which might be weighing heavily in
contributing to a
>>> particularly large overall error stat.
>>>
>>> That said, I don't want to generate a big change for the Point-
Stat tool. I
>>> may be forgetting some features of Stat-Analysis which would allow
me to
>>> aggregate/stratify the error stats in order to show the stats I
mentioned
>>> above.
>>>
>>> I'm tending towards the latter suggestion you made in (3) about
the user
>>> setting the mapping of input little_r strings to output message
types. That
>>> way I could control the generation of error stats in Point-Stat to
adhere to
>>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>>> The following example describes one such general mapping for
surface
>>> verification:
>>>
>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>
>>>
>>>
>>> In the above mapping, I'm assuming that the little_r strings I
selected are
>>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>>> select all the strings which were in the data, but just enough to
give you and
>>> others an idea. I also assume that I obtained MADIS observations
for the
>>> additional strings (synop, ship, metar, buoy) which were not
present in the
>>> iittle_r file we used in this test.
>>>
>>> Then for upper air verification, I would use the following:
>>>
>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>>
>>>
>>> The above list could be expanded with more types of upper air
data, but I
>>> think you get the idea. What I've done with this is to set up the
generation
>>> of error stats in Point-Stat so that I get overall error stats
which were
>>> derived from all possible sources of upper air observations. As
long as the
>>> source provides measurements of met variables at the various
pressure levels
>>> just as Point-Stat handles now, then this should work for the
overall stats.
>>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>>> observations possible for upper air verification which often
suffers from a
>>> paucity of "ground" truth data.
>>>
>>> Then, I would have the option to go back and re-characterize the
various types
>>> of observations into smaller groupings which focus in on trying to
group based
>>> on the specific type of measurement used.
>>>
>>> For example break out the ship data and using a different mapping
which
>>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>>> That way you can evaluate the contribution/weight of those stats
towards the
>>> overall stats. Any number of different mappings are possible by
re-running
>>> ASCII2NC with different configuration file settings.
>>>
>>> For surface observations where there are often large numbers of
measurements
>>> from disparate sensor types, we are also interested in trying
break down the
>>> verification and restrict the scoring to subdomains and having the
ability to
>>> break out the stats by sensor type would be of value for these
more in-depth
>>> assessments.
>>>
>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>> Hopefully others can chime in here.
>>>
>>> I would like the opportunity to discuss this among colleagues here
in ARL so
>>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>>> I'm sure others have some good input to make about the statistical
>>> implications of grouping the observation types the way I did. I
have probably
>>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>>> might factor into this discussion.
>>>
>>> R/
>>> John
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, August 08, 2013 10:30 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>>> (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file
>>>
>>> John,
>>>
>>> Hmmm, good question. Let me give you some background. We only
recently began
>>> providing support for little_r data, which was driven by a
modelling project
>>> in which the DTC was involved. Our test data was limited to the
data
>>> available for that project. Based on that data, we defined the
following
>>> mappings:
>>>
>>> //
>>> // Load mapping of report types to message types
>>> //
>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>
>>> Instances of the little_r input strings on the left are mapped to
the message
>>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>>> me, I see that your little_r data includes the following 22
message types:
>>> "FM-132"
>>> "FM-15_AIRNow"
>>> "FM-15_APRSWXNET"
>>> "FM-15_ASOS"
>>> "FM-15_CA-Hydro"
>>> "FM-15_CODOT"
>>> "FM-15_DDMET"
>>> "FM-15_GPSMET"
>>> "FM-15_HADS"
>>> "FM-15_MAP"
>>> "FM-15_MesoWest"
>>> "FM-15_NERRS"
>>> "FM-15_NonFedAWOS"
>>> "FM-15_NOS-NWLON"
>>> "FM-15_NOS-PORTS"
>>> "FM-15_NVDOT"
>>> "FM-15_OTHER-MTR"
>>> "FM-15_RAWS"
>>> "FM-15_SURFRAD"
>>> "FM-35"
>>> "FM-96"
>>> "FM-97_TAMDAR"
>>>
>>> None of those match the strings we were expecting. So instead of
mapping them
>>> to the PREPBUFR message types, we just wrote the strings that were
input. The
>>> impact is that when you use this data in Point-Stat, you'll need
to list those
>>> message types separately and get output for each one.
>>>
>>> I see several questions here:
>>> (1) Is it reasonable for us to figure out the super-set of all
message types
>>> we can expect to see in little_r data?
>>> (2) If yes, what is that super-set?
>>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>>> should we provide a way for users to define a mapping of input
little_r
>>> strings to output message types? We could do that by providing a
>>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>>> SURFRAD data.
>>>
>>> Do you have any perspectives on these issues?
>>>
>>> Regarding the quality control values, yes, I agree, they are odd.
>>> Unfortunately, I don't have much information for you. I would
suggest
>>> verifying that the values used in the little_r files match the
values that are
>>> showing up in the MET output. But making sense of them is up to
you. In
>>> Point-Stat, you can provide a list of quality control strings to
use in the
>>> obs_quality parameter. For the DTC project using little_r data,
here's the
>>> list of strings we used:
>>>
>>> obs_quality = [
>>> "3", "4", "5", "6", "7", "8",
"9",
>>> "10",
>>> "100003", "100004", "100005", "100006", "100007", "100008",
"100009",
>>> "100010",
>>> "200003", "200004", "200005", "200006", "200007", "200008",
"200009",
>>> "200010",
>>> "300003", "300004", "300005", "300006", "300007", "300008",
"300009",
>>> "300010",
>>> "400003", "400004", "400005", "400006", "400007", "400008",
"400009",
>>> "400010",
>>> "500003", "500004", "500005", "500006", "500007", "500008",
"500009",
>>> "500010",
>>> "600003", "600004", "600005", "600006", "600007", "600008",
"600009",
>>> "600010"
>>> ];
>>>
>>> But it looks like these are not the same values that are showing
up in your
>>> data.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>> Queue: met_help
>>>> Subject: Test of ASCII2NC on little_r file
>>>> Owner: Nobody
>>>> Requestors: john.w.raby2.civ at mail.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>
>>>>
>>>>
>>>> I ran ASCII2NC as follows:
>>>>
>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>>> -v 4
>>>>
>>>> It appears that the LittleRHandler is working to indicate that it
recognizes
>>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>>
>>>> The output file attached as the text file " test7_ncdump" appears
to contain
>>>> good data, but the obs_qty information looks odd in places.
>>>>
>>>> I've also attached the log file which contain extensive WARNINGS,
but no
>>>> ERROR.
>>>>
>>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any help
you can
>>>> provide on the reasons for the WARNINGS would be appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Aug 27 16:00:20 2013
John,
I think the warnings are probably fine. There's supposed to be a
number in the header saying how many observation lines are to follow.
ASCII2NC should process each of those lines that follow and
just print a warning message if the number of lines doesn't match what
the header said.
John
On 08/27/2013 03:13 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> I edited the little_r_handler.cc file and Bob Flanigan recompiled
MET.
>
> I reran ASCII2NC on the test little_r file and the results look much
more encouraging.
>
> The data in the attached ncdmp of the output NetCDF file shows no
hdr_typ names other than the PrepBUFR message type names. I found
observational data for all the message types I had specified in the
little_r_handler.cc where I had mapped every report type string which
was present on the little_r file. I've also attached a copy of the log
file (test10). I noticed that there was only one RAOB report (ADPUPA)
which was surprising with so many observations, but maybe the time
being 1800Z explains that.
>
> The log file had 24 WARNINGS about the number of data lines in the
header not matching the number of lines in the data. I'm not sure what
this problem is.
>
> Does this result look OK to you?
>
> I will set up a Point-Stat test to see how it handles the NetCDF
file and let you know how that comes out.
>
> Thanks.
>
> R/
> John
>
> _______________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 27, 2013 11:58 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> Sorry for the delay in getting back to you. I ran with your
modified version of ASCII2NC and see the problem. You were using
underscores where there should have been spaces. Here's why...
>
> Take a look at this warning message from ASCII2NC:
> WARNING: LittleRHandler::_processObs() -> Storing message type
as "FM-15_APRSWXNET" for unexpected report type "FM-15 APRSWXNET".
>
> This is saying that the tool encountered the report type "FM-15
APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The tool
replaced that space with an underscore because we don't allow
> spaces in the message type. Otherwise, it'd mess up the space-
separated columns of output from Point-Stat.
>
> All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
>
> //
> // Load mapping of report types to message types
> //
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
> MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
> MAP_MSG_TYP["FM-132" ] = "PROFLR";
> MAP_MSG_TYP["FM-35" ] = "ADPUPA";
> MAP_MSG_TYP["FM-96" ] = "AIRCAR";
> MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
>
> Then it should work fine.
>
> Thanks,
> John
>
> On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Update on the test is the following:
>>
>> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>>
>> R/
>> John
>>
>>
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Wednesday, August 21, 2013 4:16 PM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>
>> John,
>>
>> I'm not sure why your mapping did not work as expected. I could
try running it here with your updated version of little_r_handler.cc,
but I'd need little_r file you're using. Are you able to send me
>> a sample one to work with? I'd run it through ascii2nc and inspect
the NetCDF output to see if it's working as expected.
>>
>> Thanks,
>> John
>>
>> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> John -
>>>
>>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>>
>>> This is quite different from my first Point-Stat test last week
where the little_r_handler.cc file was the default one. In that case,
I used your suggestion about putting the actual report string names
from the little_r file in the message_type array in the config file.
That test had varying numbers of pairs found depending on the number
of observations found for a given report type/level and produced stats
for all report types/levels except for the variables at 2m and 10m AGL
I had specified in the config file. I'm still trying to figure out why
no results for these AGL levels
>>>
>>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>>
>>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil
>>>
>>> ________________________________________
>>> From: John Halley Gotway via RT [met_help at ucar.edu]
>>> Sent: Tuesday, August 13, 2013 8:29 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>>>
>>> John,
>>>
>>> One of the scientists here is looking more into the data types for
little_r observations. We're trying to figure out if we can define
the super-set of all possible message type strings that may show
>>> up in a little R file. If so, we could update our mapping of
input little R strings to output message types. However, whatever
mapping we use, it seems like it'd be prudent to either move it into a
>>> configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>>
>>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>>
>>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest editing this file:
METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap7.htm#format
>>>>
>>>> A more complete description in regard to output from standard
Obsgrid is:
>>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>>> 4 = pressure derived from standard atmosphere assumption and
height
>>>> 8 = pressure derived from standard atmosphere assumption and
temperature
>>>> 16 = Temperature/Dewpoint are zero
>>>> 32 = Calm wind
>>>> 64 = Negative wind speed
>>>> 128 = Wind direction did not fall between 0 and 360 degrees
>>>> [129] = WRF uses this to indicate winds that are earth-relative
(instead of grid-relative).
>>>> 256 = Value was vertically interpolated (to get to level with
first-guess field to QC against)
>>>> 512 = The observation was assigned a nearby pressure where a
first-guess field was available for qc (some variables may have been
adjusted to account for the change in pressure) (this only applies for
obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88 SATOB’)
>>>> 1024 = Temperature appeared to have the wrong sign based on
adjacent temperatures in the vertical profile so OBSGRID flipped the
sign of the temperature and adjusted dewpoint/RH
>>>> 2048 = A superadiabatic temperature remains after convective
adjustment (results in OBSGRID marking temperature, dewpoint, and
relative humidity as missing)
>>>> 4096 = Spike in wind (results in OBSGRID marking winds as
missing)
>>>> 8192 = Dry convective adjustment used in attempt to remove
superadiabatic level
>>>> 16384 = No buddies (nearby observations) were found to do a buddy
check
>>>> 32768 = Fails error max check (check against first guess field)
>>>> 65536 = Fails buddy check (check against nearby obs)
>>>> 131072 = Observation outside of domain
>>>>
>>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>>
>>>> Brian
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Raby, John W CIV USARMY ARL (US)
>>>> Sent: Thursday, August 08, 2013 3:15 PM
>>>> To: met_help at ucar.edu
>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r file (UNCLASSIFIED)
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>> John -
>>>>
>>>> Thanks for this info, your analysis of our results and opening
the opportunity
>>>> for a dialog on this topic.
>>>>
>>>> I think of observational data in the following categories:
>>>>
>>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy reports
>>>> which I use the verify model forecasts for surface met variables
(2meter and
>>>> 10meter AGL) using Point-Stat.
>>>>
>>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
AIRCRAFT,
>>>> AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC SOUNDER
(SODAR),
>>>> VAD (NEXRAD) WIND some of which I use to verify model forecasts
for upper air
>>>> variables at various pressure levels using Point-Stat.
>>>>
>>>> My thoughts to follow are not the final word on this and subject
to input from
>>>> other ARL colleagues which I will coordinate before giving you
the final
>>>> answers.
>>>>
>>>> Lump all the sources of surface observations together to produce
error
>>>> statistics for surface variables and do the same for upper air
observations
>>>> and their error statistics. Then, break out the error statistics
by message
>>>> type like Point-Stat does now, so you could isolate a particular
type
>>>> (observation source/type) which might be weighing heavily in
contributing to a
>>>> particularly large overall error stat.
>>>>
>>>> That said, I don't want to generate a big change for the Point-
Stat tool. I
>>>> may be forgetting some features of Stat-Analysis which would
allow me to
>>>> aggregate/stratify the error stats in order to show the stats I
mentioned
>>>> above.
>>>>
>>>> I'm tending towards the latter suggestion you made in (3) about
the user
>>>> setting the mapping of input little_r strings to output message
types. That
>>>> way I could control the generation of error stats in Point-Stat
to adhere to
>>>> the groupings I mentioned above. Then, if the need arose, I could
rerun the
>>>> ASCII2NC with the mapping changed to regroup the little_r strings
differently.
>>>> The following example describes one such general mapping for
surface
>>>> verification:
>>>>
>>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_AIRNow" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_DDMET" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>>
>>>>
>>>>
>>>> In the above mapping, I'm assuming that the little_r strings I
selected are
>>>> all observations of surface met variables (2 and 10 meter AGL). I
didn't
>>>> select all the strings which were in the data, but just enough to
give you and
>>>> others an idea. I also assume that I obtained MADIS observations
for the
>>>> additional strings (synop, ship, metar, buoy) which were not
present in the
>>>> iittle_r file we used in this test.
>>>>
>>>> Then for upper air verification, I would use the following:
>>>>
>>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-XX RAOB" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-XX VADWND" ] = "ADPUPA";
>>>>
>>>>
>>>> The above list could be expanded with more types of upper air
data, but I
>>>> think you get the idea. What I've done with this is to set up the
generation
>>>> of error stats in Point-Stat so that I get overall error stats
which were
>>>> derived from all possible sources of upper air observations. As
long as the
>>>> source provides measurements of met variables at the various
pressure levels
>>>> just as Point-Stat handles now, then this should work for the
overall stats.
>>>> For these stats I'm trying to capitalize on acquiring the maximum
number of
>>>> observations possible for upper air verification which often
suffers from a
>>>> paucity of "ground" truth data.
>>>>
>>>> Then, I would have the option to go back and re-characterize the
various types
>>>> of observations into smaller groupings which focus in on trying
to group based
>>>> on the specific type of measurement used.
>>>>
>>>> For example break out the ship data and using a different mapping
which
>>>> characterizes it as "SFCSHP" so that its stats are calculated
separately in
>>>> Point-Stat or break out the profiler data by characterizing it as
"PROFLR".
>>>> That way you can evaluate the contribution/weight of those stats
towards the
>>>> overall stats. Any number of different mappings are possible by
re-running
>>>> ASCII2NC with different configuration file settings.
>>>>
>>>> For surface observations where there are often large numbers of
measurements
>>>> from disparate sensor types, we are also interested in trying
break down the
>>>> verification and restrict the scoring to subdomains and having
the ability to
>>>> break out the stats by sensor type would be of value for these
more in-depth
>>>> assessments.
>>>>
>>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>>> Hopefully others can chime in here.
>>>>
>>>> I would like the opportunity to discuss this among colleagues
here in ARL so
>>>> that I'm not misinterpreting anything here or am making invalid
assumptions.
>>>> I'm sure others have some good input to make about the
statistical
>>>> implications of grouping the observation types the way I did. I
have probably
>>>> omitted the kind of stratifications which can be made with Stat-
Analysis which
>>>> might factor into this discussion.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Thursday, August 08, 2013 10:30 AM
>>>> To: Raby, John W CIV USARMY ARL (US)
>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
Robert T CIV
>>>> (US)
>>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r file
>>>>
>>>> John,
>>>>
>>>> Hmmm, good question. Let me give you some background. We only
recently began
>>>> providing support for little_r data, which was driven by a
modelling project
>>>> in which the DTC was involved. Our test data was limited to the
data
>>>> available for that project. Based on that data, we defined the
following
>>>> mappings:
>>>>
>>>> //
>>>> // Load mapping of report types to message types
>>>> //
>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>>
>>>> Instances of the little_r input strings on the left are mapped to
the message
>>>> types used in the ASCII2NC output on the right. Looking at the
data you sent
>>>> me, I see that your little_r data includes the following 22
message types:
>>>> "FM-132"
>>>> "FM-15_AIRNow"
>>>> "FM-15_APRSWXNET"
>>>> "FM-15_ASOS"
>>>> "FM-15_CA-Hydro"
>>>> "FM-15_CODOT"
>>>> "FM-15_DDMET"
>>>> "FM-15_GPSMET"
>>>> "FM-15_HADS"
>>>> "FM-15_MAP"
>>>> "FM-15_MesoWest"
>>>> "FM-15_NERRS"
>>>> "FM-15_NonFedAWOS"
>>>> "FM-15_NOS-NWLON"
>>>> "FM-15_NOS-PORTS"
>>>> "FM-15_NVDOT"
>>>> "FM-15_OTHER-MTR"
>>>> "FM-15_RAWS"
>>>> "FM-15_SURFRAD"
>>>> "FM-35"
>>>> "FM-96"
>>>> "FM-97_TAMDAR"
>>>>
>>>> None of those match the strings we were expecting. So instead of
mapping them
>>>> to the PREPBUFR message types, we just wrote the strings that
were input. The
>>>> impact is that when you use this data in Point-Stat, you'll need
to list those
>>>> message types separately and get output for each one.
>>>>
>>>> I see several questions here:
>>>> (1) Is it reasonable for us to figure out the super-set of all
message types
>>>> we can expect to see in little_r data?
>>>> (2) If yes, what is that super-set?
>>>> (3) If no, is it OK for us to just use in the input string in the
output? Or
>>>> should we provide a way for users to define a mapping of input
little_r
>>>> strings to output message types? We could do that by providing a
>>>> configuration file for the ASCII2NC tool - which is already used
for ASCII
>>>> SURFRAD data.
>>>>
>>>> Do you have any perspectives on these issues?
>>>>
>>>> Regarding the quality control values, yes, I agree, they are odd.
>>>> Unfortunately, I don't have much information for you. I would
suggest
>>>> verifying that the values used in the little_r files match the
values that are
>>>> showing up in the MET output. But making sense of them is up to
you. In
>>>> Point-Stat, you can provide a list of quality control strings to
use in the
>>>> obs_quality parameter. For the DTC project using little_r data,
here's the
>>>> list of strings we used:
>>>>
>>>> obs_quality = [
>>>> "3", "4", "5", "6", "7",
"8", "9",
>>>> "10",
>>>> "100003", "100004", "100005", "100006", "100007",
"100008", "100009",
>>>> "100010",
>>>> "200003", "200004", "200005", "200006", "200007",
"200008", "200009",
>>>> "200010",
>>>> "300003", "300004", "300005", "300006", "300007",
"300008", "300009",
>>>> "300010",
>>>> "400003", "400004", "400005", "400006", "400007",
"400008", "400009",
>>>> "400010",
>>>> "500003", "500004", "500005", "500006", "500007",
"500008", "500009",
>>>> "500010",
>>>> "600003", "600004", "600005", "600006", "600007",
"600008", "600009",
>>>> "600010"
>>>> ];
>>>>
>>>> But it looks like these are not the same values that are showing
up in your
>>>> data.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>>
>>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>>> Queue: met_help
>>>>> Subject: Test of ASCII2NC on little_r file
>>>>> Owner: Nobody
>>>>> Requestors: john.w.raby2.civ at mail.mil
>>>>> Status: new
>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>>
>>>>>
>>>>>
>>>>> I ran ASCII2NC as follows:
>>>>>
>>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
test7
>>>>> -v 4
>>>>>
>>>>> It appears that the LittleRHandler is working to indicate that
it recognizes
>>>>> the input file as a little_r file and then generates the NetCDF
output file.
>>>>>
>>>>> The output file attached as the text file " test7_ncdump"
appears to contain
>>>>> good data, but the obs_qty information looks odd in places.
>>>>>
>>>>> I've also attached the log file which contain extensive
WARNINGS, but no
>>>>> ERROR.
>>>>>
>>>>> Can you explain the odd numbers in the obs_qty array? I was
expecting
>>>>> numbers like 0, 1, 2, etc but not "512" or "16384". Also any
help you can
>>>>> provide on the reasons for the WARNINGS would be appreciated.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> R/
>>>>> John
>>>>>
>>>>>
>>>>> Mr John W. Raby, Meteorologist
>>>>>
>>>>> U.S. Army Research Laboratory
>>>>>
>>>>> White Sands Missile Range, NM 88002
>>>>>
>>>>> (575) 678-2004 DSN 258-2004
>>>>>
>>>>> FAX (575) 678-1230 DSN 258-1230
>>>>>
>>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>>
>>>>
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>>
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
Subject: Test of ASCII2NC on little_r file
From: Raby, John W USA CIV
Time: Wed Aug 28 07:07:07 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
Thanks for your response.
I ran Point-Stat using the little_r file with config file message_type
settings to match those in the mapping for ASCII2NC and the results
look good. This time I even have the 2m and 10m AGL stats which were
not there before.
Thanks for all your help.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, August 27, 2013 4:00 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
John,
I think the warnings are probably fine. There's supposed to be a
number in the header saying how many observation lines are to follow.
ASCII2NC should process each of those lines that follow and just print
a warning message if the number of lines doesn't match what the header
said.
John
On 08/27/2013 03:13 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> John -
>
> I edited the little_r_handler.cc file and Bob Flanigan recompiled
MET.
>
> I reran ASCII2NC on the test little_r file and the results look much
more encouraging.
>
> The data in the attached ncdmp of the output NetCDF file shows no
hdr_typ names other than the PrepBUFR message type names. I found
observational data for all the message types I had specified in the
little_r_handler.cc where I had mapped every report type string which
was present on the little_r file. I've also attached a copy of the log
file (test10). I noticed that there was only one RAOB report (ADPUPA)
which was surprising with so many observations, but maybe the time
being 1800Z explains that.
>
> The log file had 24 WARNINGS about the number of data lines in the
header not matching the number of lines in the data. I'm not sure what
this problem is.
>
> Does this result look OK to you?
>
> I will set up a Point-Stat test to see how it handles the NetCDF
file and let you know how that comes out.
>
> Thanks.
>
> R/
> John
>
> _______________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, August 27, 2013 11:58 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
> rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
> file (UNCLASSIFIED)
>
> John,
>
> Sorry for the delay in getting back to you. I ran with your
modified version of ASCII2NC and see the problem. You were using
underscores where there should have been spaces. Here's why...
>
> Take a look at this warning message from ASCII2NC:
> WARNING: LittleRHandler::_processObs() -> Storing message type
as "FM-15_APRSWXNET" for unexpected report type "FM-15 APRSWXNET".
>
> This is saying that the tool encountered the report type "FM-15
> APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The
tool replaced that space with an underscore because we don't allow
spaces in the message type. Otherwise, it'd mess up the space-
separated columns of output from Point-Stat.
>
> All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
>
> //
> // Load mapping of report types to message types
> //
> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
> MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
> MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
> MAP_MSG_TYP["FM-132" ] = "PROFLR";
> MAP_MSG_TYP["FM-35" ] = "ADPUPA";
> MAP_MSG_TYP["FM-96" ] = "AIRCAR";
> MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
>
> Then it should work fine.
>
> Thanks,
> John
>
> On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> Update on the test is the following:
>>
>> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>>
>> R/
>> John
>>
>>
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Wednesday, August 21, 2013 4:16 PM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>> file (UNCLASSIFIED)
>>
>> John,
>>
>> I'm not sure why your mapping did not work as expected. I could
try
>> running it here with your updated version of little_r_handler.cc,
but I'd need little_r file you're using. Are you able to send me a
sample one to work with? I'd run it through ascii2nc and inspect the
NetCDF output to see if it's working as expected.
>>
>> Thanks,
>> John
>>
>> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> John -
>>>
>>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>>
>>> This is quite different from my first Point-Stat test last week
>>> where the little_r_handler.cc file was the default one. In that
>>> case, I used your suggestion about putting the actual report
string
>>> names from the little_r file in the message_type array in the
config
>>> file. That test had varying numbers of pairs found depending on
the
>>> number of observations found for a given report type/level and
>>> produced stats for all report types/levels except for the
variables
>>> at 2m and 10m AGL I had specified in the config file. I'm still
>>> trying to figure out why no results for these AGL levels
>>>
>>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>>
>>> Do you have any idea why my mapping doesn't seem to be working and
why the two AGL level results are not being generated?
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>> Mr John W. Raby, Meteorologist
>>>
>>> U.S. Army Research Laboratory
>>>
>>> White Sands Missile Range, NM 88002
>>>
>>> (575) 678-2004 DSN 258-2004
>>>
>>> FAX (575) 678-1230 DSN 258-1230
>>>
>>> Email: john.w.raby2.civ at mail.mil
>>>
>>> ________________________________________
>>> From: John Halley Gotway via RT [met_help at ucar.edu]
>>> Sent: Tuesday, August 13, 2013 8:29 AM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>>> file (UNCLASSIFIED)
>>>
>>> John,
>>>
>>> One of the scientists here is looking more into the data types for
>>> little_r observations. We're trying to figure out if we can
define
>>> the super-set of all possible message type strings that may show
up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>>
>>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>>
>>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest
>>> editing this file:
>>> METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_ch
>>>> ap7.htm#format
>>>>
>>>> A more complete description in regard to output from standard
Obsgrid is:
>>>> 2 = pressure derived from the first-guess height field (e.g. GFS)
>>>> 4 = pressure derived from standard atmosphere assumption and
height
>>>> 8 = pressure derived from standard atmosphere assumption and
>>>> temperature
>>>> 16 = Temperature/Dewpoint are zero
>>>> 32 = Calm wind
>>>> 64 = Negative wind speed
>>>> 128 = Wind direction did not fall between 0 and 360 degrees [129]
=
>>>> WRF uses this to indicate winds that are earth-relative (instead
of grid-relative).
>>>> 256 = Value was vertically interpolated (to get to level with
>>>> first-guess field to QC against)
>>>> 512 = The observation was assigned a nearby pressure where a
>>>> first-guess field was available for qc (some variables may have
>>>> been adjusted to account for the change in pressure) (this only
>>>> applies for obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88
>>>> SATOB’)
>>>> 1024 = Temperature appeared to have the wrong sign based on
>>>> adjacent temperatures in the vertical profile so OBSGRID flipped
>>>> the sign of the temperature and adjusted dewpoint/RH
>>>> 2048 = A superadiabatic temperature remains after convective
>>>> adjustment (results in OBSGRID marking temperature, dewpoint, and
>>>> relative humidity as missing)
>>>> 4096 = Spike in wind (results in OBSGRID marking winds as
missing)
>>>> 8192 = Dry convective adjustment used in attempt to remove
>>>> superadiabatic level
>>>> 16384 = No buddies (nearby observations) were found to do a buddy
>>>> check
>>>> 32768 = Fails error max check (check against first guess field)
>>>> 65536 = Fails buddy check (check against nearby obs)
>>>> 131072 = Observation outside of domain
>>>>
>>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>>
>>>> Brian
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Raby, John W CIV USARMY ARL (US)
>>>> Sent: Thursday, August 08, 2013 3:15 PM
>>>> To: met_help at ucar.edu
>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
>>>> Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
>>>> Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r
>>>> file (UNCLASSIFIED)
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>> John -
>>>>
>>>> Thanks for this info, your analysis of our results and opening
the
>>>> opportunity for a dialog on this topic.
>>>>
>>>> I think of observational data in the following categories:
>>>>
>>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy
>>>> reports which I use the verify model forecasts for surface met
>>>> variables (2meter and 10meter AGL) using Point-Stat.
>>>>
>>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
>>>> AIRCRAFT, AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND ACOUSTIC
>>>> SOUNDER (SODAR), VAD (NEXRAD) WIND some of which I use to verify
>>>> model forecasts for upper air variables at various pressure
levels using Point-Stat.
>>>>
>>>> My thoughts to follow are not the final word on this and subject
to
>>>> input from other ARL colleagues which I will coordinate before
>>>> giving you the final answers.
>>>>
>>>> Lump all the sources of surface observations together to produce
>>>> error statistics for surface variables and do the same for upper
>>>> air observations and their error statistics. Then, break out the
>>>> error statistics by message type like Point-Stat does now, so you
>>>> could isolate a particular type (observation source/type) which
>>>> might be weighing heavily in contributing to a particularly large
overall error stat.
>>>>
>>>> That said, I don't want to generate a big change for the Point-
Stat
>>>> tool. I may be forgetting some features of Stat-Analysis which
>>>> would allow me to aggregate/stratify the error stats in order to
>>>> show the stats I mentioned above.
>>>>
>>>> I'm tending towards the latter suggestion you made in (3) about
the
>>>> user setting the mapping of input little_r strings to output
>>>> message types. That way I could control the generation of error
>>>> stats in Point-Stat to adhere to the groupings I mentioned above.
>>>> Then, if the need arose, I could rerun the ASCII2NC with the
mapping changed to regroup the little_r strings differently.
>>>> The following example describes one such general mapping for
>>>> surface
>>>> verification:
>>>>
>>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_AIRNow"
>>>> ] = "ADPSFC"; MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC"; MAP_MSG_TYP["FM-15_DDMET"
]
>>>> = "ADPSFC"; MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>>
>>>>
>>>>
>>>> In the above mapping, I'm assuming that the little_r strings I
>>>> selected are all observations of surface met variables (2 and 10
>>>> meter AGL). I didn't select all the strings which were in the
data,
>>>> but just enough to give you and others an idea. I also assume
that
>>>> I obtained MADIS observations for the additional strings (synop,
>>>> ship, metar, buoy) which were not present in the iittle_r file we
used in this test.
>>>>
>>>> Then for upper air verification, I would use the following:
>>>>
>>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX RAOB"
]
>>>> = "ADPUPA"; MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX
VADWND"
>>>> ] = "ADPUPA";
>>>>
>>>>
>>>> The above list could be expanded with more types of upper air
data,
>>>> but I think you get the idea. What I've done with this is to set
up
>>>> the generation of error stats in Point-Stat so that I get overall
>>>> error stats which were derived from all possible sources of upper
>>>> air observations. As long as the source provides measurements of
>>>> met variables at the various pressure levels just as Point-Stat
handles now, then this should work for the overall stats.
>>>> For these stats I'm trying to capitalize on acquiring the maximum
>>>> number of observations possible for upper air verification which
>>>> often suffers from a paucity of "ground" truth data.
>>>>
>>>> Then, I would have the option to go back and re-characterize the
>>>> various types of observations into smaller groupings which focus
in
>>>> on trying to group based on the specific type of measurement
used.
>>>>
>>>> For example break out the ship data and using a different mapping
>>>> which characterizes it as "SFCSHP" so that its stats are
calculated
>>>> separately in Point-Stat or break out the profiler data by
characterizing it as "PROFLR".
>>>> That way you can evaluate the contribution/weight of those stats
>>>> towards the overall stats. Any number of different mappings are
>>>> possible by re-running ASCII2NC with different configuration file
settings.
>>>>
>>>> For surface observations where there are often large numbers of
>>>> measurements from disparate sensor types, we are also interested
in
>>>> trying break down the verification and restrict the scoring to
>>>> subdomains and having the ability to break out the stats by
sensor
>>>> type would be of value for these more in-depth assessments.
>>>>
>>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>>> Hopefully others can chime in here.
>>>>
>>>> I would like the opportunity to discuss this among colleagues
here
>>>> in ARL so that I'm not misinterpreting anything here or am making
invalid assumptions.
>>>> I'm sure others have some good input to make about the
statistical
>>>> implications of grouping the observation types the way I did. I
>>>> have probably omitted the kind of stratifications which can be
made
>>>> with Stat-Analysis which might factor into this discussion.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Thursday, August 08, 2013 10:30 AM
>>>> To: Raby, John W CIV USARMY ARL (US)
>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com; Flanigan,
>>>> Robert T CIV
>>>> (US)
>>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r
>>>> file
>>>>
>>>> John,
>>>>
>>>> Hmmm, good question. Let me give you some background. We only
>>>> recently began providing support for little_r data, which was
>>>> driven by a modelling project in which the DTC was involved. Our
>>>> test data was limited to the data available for that project.
>>>> Based on that data, we defined the following
>>>> mappings:
>>>>
>>>> //
>>>> // Load mapping of report types to message types
>>>> //
>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>>
>>>> Instances of the little_r input strings on the left are mapped to
>>>> the message types used in the ASCII2NC output on the right.
>>>> Looking at the data you sent me, I see that your little_r data
includes the following 22 message types:
>>>> "FM-132"
>>>> "FM-15_AIRNow"
>>>> "FM-15_APRSWXNET"
>>>> "FM-15_ASOS"
>>>> "FM-15_CA-Hydro"
>>>> "FM-15_CODOT"
>>>> "FM-15_DDMET"
>>>> "FM-15_GPSMET"
>>>> "FM-15_HADS"
>>>> "FM-15_MAP"
>>>> "FM-15_MesoWest"
>>>> "FM-15_NERRS"
>>>> "FM-15_NonFedAWOS"
>>>> "FM-15_NOS-NWLON"
>>>> "FM-15_NOS-PORTS"
>>>> "FM-15_NVDOT"
>>>> "FM-15_OTHER-MTR"
>>>> "FM-15_RAWS"
>>>> "FM-15_SURFRAD"
>>>> "FM-35"
>>>> "FM-96"
>>>> "FM-97_TAMDAR"
>>>>
>>>> None of those match the strings we were expecting. So instead of
>>>> mapping them to the PREPBUFR message types, we just wrote the
>>>> strings that were input. The impact is that when you use this
data
>>>> in Point-Stat, you'll need to list those message types separately
and get output for each one.
>>>>
>>>> I see several questions here:
>>>> (1) Is it reasonable for us to figure out the super-set of all
>>>> message types we can expect to see in little_r data?
>>>> (2) If yes, what is that super-set?
>>>> (3) If no, is it OK for us to just use in the input string in the
>>>> output? Or should we provide a way for users to define a mapping
>>>> of input little_r strings to output message types? We could do
>>>> that by providing a configuration file for the ASCII2NC tool -
>>>> which is already used for ASCII SURFRAD data.
>>>>
>>>> Do you have any perspectives on these issues?
>>>>
>>>> Regarding the quality control values, yes, I agree, they are odd.
>>>> Unfortunately, I don't have much information for you. I would
>>>> suggest verifying that the values used in the little_r files
match
>>>> the values that are showing up in the MET output. But making
sense
>>>> of them is up to you. In Point-Stat, you can provide a list of
>>>> quality control strings to use in the obs_quality parameter. For
>>>> the DTC project using little_r data, here's the list of strings
we used:
>>>>
>>>> obs_quality = [
>>>> "3", "4", "5", "6", "7",
"8", "9",
>>>> "10",
>>>> "100003", "100004", "100005", "100006", "100007",
"100008",
>>>> "100009", "100010",
>>>> "200003", "200004", "200005", "200006", "200007",
"200008",
>>>> "200009", "200010",
>>>> "300003", "300004", "300005", "300006", "300007",
"300008",
>>>> "300009", "300010",
>>>> "400003", "400004", "400005", "400006", "400007",
"400008",
>>>> "400009", "400010",
>>>> "500003", "500004", "500005", "500006", "500007",
"500008",
>>>> "500009", "500010",
>>>> "600003", "600004", "600005", "600006", "600007",
"600008",
>>>> "600009", "600010"
>>>> ];
>>>>
>>>> But it looks like these are not the same values that are showing
up
>>>> in your data.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>>
>>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>>> Queue: met_help
>>>>> Subject: Test of ASCII2NC on little_r file
>>>>> Owner: Nobody
>>>>> Requestors: john.w.raby2.civ at mail.mil
>>>>> Status: new
>>>>> Ticket <URL:
>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>>
>>>>>
>>>>>
>>>>> I ran ASCII2NC as follows:
>>>>>
>>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000
>>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
>>>>> test7 -v 4
>>>>>
>>>>> It appears that the LittleRHandler is working to indicate that
it
>>>>> recognizes the input file as a little_r file and then generates
the NetCDF output file.
>>>>>
>>>>> The output file attached as the text file " test7_ncdump"
appears
>>>>> to contain good data, but the obs_qty information looks odd in
places.
>>>>>
>>>>> I've also attached the log file which contain extensive
WARNINGS,
>>>>> but no ERROR.
>>>>>
>>>>> Can you explain the odd numbers in the obs_qty array? I was
>>>>> expecting numbers like 0, 1, 2, etc but not "512" or "16384".
Also
>>>>> any help you can provide on the reasons for the WARNINGS would
be appreciated.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> R/
>>>>> John
>>>>>
>>>>>
>>>>> Mr John W. Raby, Meteorologist
>>>>>
>>>>> U.S. Army Research Laboratory
>>>>>
>>>>> White Sands Missile Range, NM 88002
>>>>>
>>>>> (575) 678-2004 DSN 258-2004
>>>>>
>>>>> FAX (575) 678-1230 DSN 258-1230
>>>>>
>>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>>
>>>>
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>>
>>>>
>>>> Classification: UNCLASSIFIED
>>>> Caveats: NONE
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r file (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Aug 28 09:40:28 2013
John,
Sure thing. Glad that did the trick. We have a development task
defined for the next release to enable the user to define that mapping
of input little_r strings to output message types in an ascii
table file. That way you won't need to recompile in the future.
Thanks,
John
On 08/28/2013 07:07 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for your response.
>
> I ran Point-Stat using the little_r file with config file
message_type settings to match those in the mapping for ASCII2NC and
the results look good. This time I even have the 2m and 10m AGL stats
which were not there before.
>
> Thanks for all your help.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, August 27, 2013 4:00 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
rflanigan at q.com; Flanigan, Robert T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
file (UNCLASSIFIED)
>
> John,
>
> I think the warnings are probably fine. There's supposed to be a
number in the header saying how many observation lines are to follow.
ASCII2NC should process each of those lines that follow and just print
a warning message if the number of lines doesn't match what the header
said.
>
> John
>
>
> On 08/27/2013 03:13 PM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>
>> John -
>>
>> I edited the little_r_handler.cc file and Bob Flanigan recompiled
MET.
>>
>> I reran ASCII2NC on the test little_r file and the results look
much more encouraging.
>>
>> The data in the attached ncdmp of the output NetCDF file shows no
hdr_typ names other than the PrepBUFR message type names. I found
observational data for all the message types I had specified in the
little_r_handler.cc where I had mapped every report type string which
was present on the little_r file. I've also attached a copy of the log
file (test10). I noticed that there was only one RAOB report (ADPUPA)
which was surprising with so many observations, but maybe the time
being 1800Z explains that.
>>
>> The log file had 24 WARNINGS about the number of data lines in the
header not matching the number of lines in the data. I'm not sure what
this problem is.
>>
>> Does this result look OK to you?
>>
>> I will set up a Point-Stat test to see how it handles the NetCDF
file and let you know how that comes out.
>>
>> Thanks.
>>
>> R/
>> John
>>
>> _______________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Tuesday, August 27, 2013 11:58 AM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>> file (UNCLASSIFIED)
>>
>> John,
>>
>> Sorry for the delay in getting back to you. I ran with your
modified version of ASCII2NC and see the problem. You were using
underscores where there should have been spaces. Here's why...
>>
>> Take a look at this warning message from ASCII2NC:
>> WARNING: LittleRHandler::_processObs() -> Storing message
type as "FM-15_APRSWXNET" for unexpected report type "FM-15
APRSWXNET".
>>
>> This is saying that the tool encountered the report type "FM-15
>> APRSWXNET" and wrote it in the output as "FM-15_APRSWXNET". The
tool replaced that space with an underscore because we don't allow
spaces in the message type. Otherwise, it'd mess up the space-
separated columns of output from Point-Stat.
>>
>> All you need to do is remove the underscores from your mappings in
"little_r_handler.cc", like this:
>>
>> //
>> // Load mapping of report types to message types
>> //
>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>> MAP_MSG_TYP["FM-15 AIRNow" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 APRSWXNET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 ASOS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 CA-Hydro" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 CODOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 DDMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 GPSMET" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 HADS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 MAP" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 MesoWest" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 NERRS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 NonFedAWOS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 NOS-NWLON" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 NOS-PORTS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 NVDOT" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 OTHER-MTR" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 RAWS" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-15 SURFRAD" ] = "ADPSFC";
>> MAP_MSG_TYP["FM-132" ] = "PROFLR";
>> MAP_MSG_TYP["FM-35" ] = "ADPUPA";
>> MAP_MSG_TYP["FM-96" ] = "AIRCAR";
>> MAP_MSG_TYP["FM-97 TAMDAR" ] = "AIRCFT";
>>
>> Then it should work fine.
>>
>> Thanks,
>> John
>>
>> On 08/22/2013 02:41 PM, Raby, John W USA CIV via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>
>>> John -
>>>
>>> Update on the test is the following:
>>>
>>> I used "diff" on the two text files of the ASCII2NC output (from a
ncdump command) from the earlier test and the one I did today and
found that indeed some of my standard message types are showing up.
Not all, however, ADPSFC and SFCSHP do not show up, but I did find
ADPUPA, PROFLR and AIRCAR. Not sure if the data is ready for Point-
Stat, but there is evidence that it read my edited little_r_handler.cc
file.
>>>
>>> R/
>>> John
>>>
>>>
>>>
>>> ________________________________________
>>> From: John Halley Gotway via RT [met_help at ucar.edu]
>>> Sent: Wednesday, August 21, 2013 4:16 PM
>>> To: Raby, John W CIV USARMY ARL (US)
>>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on little_r
>>> file (UNCLASSIFIED)
>>>
>>> John,
>>>
>>> I'm not sure why your mapping did not work as expected. I could
try
>>> running it here with your updated version of little_r_handler.cc,
but I'd need little_r file you're using. Are you able to send me a
sample one to work with? I'd run it through ascii2nc and inspect the
NetCDF output to see if it's working as expected.
>>>
>>> Thanks,
>>> John
>>>
>>> On 08/20/2013 07:30 AM, Raby, John W USA CIV via RT wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>>
>>>> John -
>>>>
>>>> Bob Flanigan recompiled MET yesterday after I edited the
little_r_handler.cc file to change the mappings according to the
report string names in my little_r observation file. I edited my
config file message_type array to specify the message_type names I
used in the mapping. When I ran Point-Stat, there were no pairs found
for any of the message_type names/levels I specified in the config
file. The normal output files and log file were generated with no
ERROR/WARNING, but there were no stats generated in the CNT file.
>>>>
>>>> This is quite different from my first Point-Stat test last week
>>>> where the little_r_handler.cc file was the default one. In that
>>>> case, I used your suggestion about putting the actual report
string
>>>> names from the little_r file in the message_type array in the
config
>>>> file. That test had varying numbers of pairs found depending on
the
>>>> number of observations found for a given report type/level and
>>>> produced stats for all report types/levels except for the
variables
>>>> at 2m and 10m AGL I had specified in the config file. I'm still
>>>> trying to figure out why no results for these AGL levels
>>>>
>>>> I've attached the little_r_handler.cc file I used for yesterday's
test and the default one (ORIG) used for the first test for ref. I
have also attached my config file.
>>>>
>>>> Do you have any idea why my mapping doesn't seem to be working
and why the two AGL level results are not being generated?
>>>>
>>>> Thanks.
>>>>
>>>> R/
>>>> John
>>>>
>>>>
>>>> Mr John W. Raby, Meteorologist
>>>>
>>>> U.S. Army Research Laboratory
>>>>
>>>> White Sands Missile Range, NM 88002
>>>>
>>>> (575) 678-2004 DSN 258-2004
>>>>
>>>> FAX (575) 678-1230 DSN 258-1230
>>>>
>>>> Email: john.w.raby2.civ at mail.mil
>>>>
>>>> ________________________________________
>>>> From: John Halley Gotway via RT [met_help at ucar.edu]
>>>> Sent: Tuesday, August 13, 2013 8:29 AM
>>>> To: Raby, John W CIV USARMY ARL (US)
>>>> Cc: Reen, Brian P CIV USARMY ARL (US); jensen at ucar.edu;
>>>> rflanigan at q.com; Flanigan, Robert T CIV (US)
>>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r
>>>> file (UNCLASSIFIED)
>>>>
>>>> John,
>>>>
>>>> One of the scientists here is looking more into the data types
for
>>>> little_r observations. We're trying to figure out if we can
define
>>>> the super-set of all possible message type strings that may show
up in a little R file. If so, we could update our mapping of input
little R strings to output message types. However, whatever mapping
we use, it seems like it'd be prudent to either move it into a
configuration file for ASCII2NC or create a data file for it in
METv4.1/data/table_files. That way it'd be easy for a user to modify
the mapping just by editing an ASCII file.
>>>>
>>>> We'll let you know what we figure out. Any potential changes
wouldn't be released as a "bugfix" for METv4.1. Instead, they'd show
up in the next release.
>>>>
>>>> For now, in order to get ASCII2NC to do what you'd like, I'd
suggest
>>>> editing this file:
>>>> METv4.1/src/tools/other/ascii2nc/little_r_handler.cc
>>>> At line 80, you can add in the mappings of little_R strings to
output message type names you'd like, and then recompile.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> On 08/08/2013 01:22 PM, brian.p.reen.civ at mail.mil via RT wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542 >
>>>>>
>>>>> Classification: UNCLASSIFIED
>>>>> Caveats: NONE
>>>>>
>>>>> Regarding the obs_qty array, these are the quality control flags
used by the little_r format. A brief description can be found under
the heading QCFlags at:
>>>>>
http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_ch
>>>>> ap7.htm#format
>>>>>
>>>>> A more complete description in regard to output from standard
Obsgrid is:
>>>>> 2 = pressure derived from the first-guess height field (e.g.
GFS)
>>>>> 4 = pressure derived from standard atmosphere assumption and
height
>>>>> 8 = pressure derived from standard atmosphere assumption and
>>>>> temperature
>>>>> 16 = Temperature/Dewpoint are zero
>>>>> 32 = Calm wind
>>>>> 64 = Negative wind speed
>>>>> 128 = Wind direction did not fall between 0 and 360 degrees
[129] =
>>>>> WRF uses this to indicate winds that are earth-relative (instead
of grid-relative).
>>>>> 256 = Value was vertically interpolated (to get to level with
>>>>> first-guess field to QC against)
>>>>> 512 = The observation was assigned a nearby pressure where a
>>>>> first-guess field was available for qc (some variables may have
>>>>> been adjusted to account for the change in pressure) (this only
>>>>> applies for obs with a platform type of ‘FM-97 AIREP’ OR ‘FM-88
>>>>> SATOB’)
>>>>> 1024 = Temperature appeared to have the wrong sign based on
>>>>> adjacent temperatures in the vertical profile so OBSGRID flipped
>>>>> the sign of the temperature and adjusted dewpoint/RH
>>>>> 2048 = A superadiabatic temperature remains after convective
>>>>> adjustment (results in OBSGRID marking temperature, dewpoint,
and
>>>>> relative humidity as missing)
>>>>> 4096 = Spike in wind (results in OBSGRID marking winds as
missing)
>>>>> 8192 = Dry convective adjustment used in attempt to remove
>>>>> superadiabatic level
>>>>> 16384 = No buddies (nearby observations) were found to do a
buddy
>>>>> check
>>>>> 32768 = Fails error max check (check against first guess field)
>>>>> 65536 = Fails buddy check (check against nearby obs)
>>>>> 131072 = Observation outside of domain
>>>>>
>>>>> The QC codes are powers of 2 to enable the QC flags to be
extracted from a single value.
>>>>>
>>>>> Brian
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Raby, John W CIV USARMY ARL (US)
>>>>> Sent: Thursday, August 08, 2013 3:15 PM
>>>>> To: met_help at ucar.edu
>>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com;
Flanigan,
>>>>> Robert T CIV (US); Smith, Jeffrey A CIV USARMY ARL (US); Dumais,
>>>>> Robert E Jr CIV (US); Raby, John W CIV USARMY ARL (US)
>>>>> Subject: RE: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r
>>>>> file (UNCLASSIFIED)
>>>>>
>>>>> Classification: UNCLASSIFIED
>>>>> Caveats: NONE
>>>>>
>>>>> John -
>>>>>
>>>>> Thanks for this info, your analysis of our results and opening
the
>>>>> opportunity for a dialog on this topic.
>>>>>
>>>>> I think of observational data in the following categories:
>>>>>
>>>>> Surface observations: METAR, SYNOP, Mesonet, Ship reports, and
buoy
>>>>> reports which I use the verify model forecasts for surface met
>>>>> variables (2meter and 10meter AGL) using Point-Stat.
>>>>>
>>>>> Upper air observations: RAOB, PIBAL, RECCO, DROPS, MDCRS, ACARS.
>>>>> AIRCRAFT, AIREP, PIREP, AMDAR, TAMDAR, WIND PROFILER AND
ACOUSTIC
>>>>> SOUNDER (SODAR), VAD (NEXRAD) WIND some of which I use to verify
>>>>> model forecasts for upper air variables at various pressure
levels using Point-Stat.
>>>>>
>>>>> My thoughts to follow are not the final word on this and subject
to
>>>>> input from other ARL colleagues which I will coordinate before
>>>>> giving you the final answers.
>>>>>
>>>>> Lump all the sources of surface observations together to produce
>>>>> error statistics for surface variables and do the same for upper
>>>>> air observations and their error statistics. Then, break out the
>>>>> error statistics by message type like Point-Stat does now, so
you
>>>>> could isolate a particular type (observation source/type) which
>>>>> might be weighing heavily in contributing to a particularly
large overall error stat.
>>>>>
>>>>> That said, I don't want to generate a big change for the Point-
Stat
>>>>> tool. I may be forgetting some features of Stat-Analysis which
>>>>> would allow me to aggregate/stratify the error stats in order to
>>>>> show the stats I mentioned above.
>>>>>
>>>>> I'm tending towards the latter suggestion you made in (3) about
the
>>>>> user setting the mapping of input little_r strings to output
>>>>> message types. That way I could control the generation of error
>>>>> stats in Point-Stat to adhere to the groupings I mentioned
above.
>>>>> Then, if the need arose, I could rerun the ASCII2NC with the
mapping changed to regroup the little_r strings differently.
>>>>> The following example describes one such general mapping for
>>>>> surface
>>>>> verification:
>>>>>
>>>>> MAP_MSG_TYP["FM-15_APRSWXNET " ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15_ASOS " ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_AIRNow"
>>>>> ] = "ADPSFC"; MAP_MSG_TYP["FM-15_CA-Hydro" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15_CODOT" ] = "ADPSFC"; MAP_MSG_TYP["FM-
15_DDMET" ]
>>>>> = "ADPSFC"; MAP_MSG_TYP["FM-15_MesoWest" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15_GPSMET" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15_NonFedAWOS " ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15_RAWS" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "ADPSFC";
>>>>>
>>>>>
>>>>>
>>>>> In the above mapping, I'm assuming that the little_r strings I
>>>>> selected are all observations of surface met variables (2 and 10
>>>>> meter AGL). I didn't select all the strings which were in the
data,
>>>>> but just enough to give you and others an idea. I also assume
that
>>>>> I obtained MADIS observations for the additional strings (synop,
>>>>> ship, metar, buoy) which were not present in the iittle_r file
we used in this test.
>>>>>
>>>>> Then for upper air verification, I would use the following:
>>>>>
>>>>> MAP_MSG_TYP["FM-97_TAMDAR" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX
RAOB" ]
>>>>> = "ADPUPA"; MAP_MSG_TYP["FM-XX Profiler" ] = "ADPUPA";
>>>>> MAP_MSG_TYP["FM-XX AIRCFT" ] = "ADPUPA";
>>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "ADPUPA"; MAP_MSG_TYP["FM-XX
VADWND"
>>>>> ] = "ADPUPA";
>>>>>
>>>>>
>>>>> The above list could be expanded with more types of upper air
data,
>>>>> but I think you get the idea. What I've done with this is to set
up
>>>>> the generation of error stats in Point-Stat so that I get
overall
>>>>> error stats which were derived from all possible sources of
upper
>>>>> air observations. As long as the source provides measurements of
>>>>> met variables at the various pressure levels just as Point-Stat
handles now, then this should work for the overall stats.
>>>>> For these stats I'm trying to capitalize on acquiring the
maximum
>>>>> number of observations possible for upper air verification which
>>>>> often suffers from a paucity of "ground" truth data.
>>>>>
>>>>> Then, I would have the option to go back and re-characterize the
>>>>> various types of observations into smaller groupings which focus
in
>>>>> on trying to group based on the specific type of measurement
used.
>>>>>
>>>>> For example break out the ship data and using a different
mapping
>>>>> which characterizes it as "SFCSHP" so that its stats are
calculated
>>>>> separately in Point-Stat or break out the profiler data by
characterizing it as "PROFLR".
>>>>> That way you can evaluate the contribution/weight of those stats
>>>>> towards the overall stats. Any number of different mappings are
>>>>> possible by re-running ASCII2NC with different configuration
file settings.
>>>>>
>>>>> For surface observations where there are often large numbers of
>>>>> measurements from disparate sensor types, we are also interested
in
>>>>> trying break down the verification and restrict the scoring to
>>>>> subdomains and having the ability to break out the stats by
sensor
>>>>> type would be of value for these more in-depth assessments.
>>>>>
>>>>> Regard the quality control values, I'm not sure what to think on
this yet.
>>>>> Hopefully others can chime in here.
>>>>>
>>>>> I would like the opportunity to discuss this among colleagues
here
>>>>> in ARL so that I'm not misinterpreting anything here or am
making invalid assumptions.
>>>>> I'm sure others have some good input to make about the
statistical
>>>>> implications of grouping the observation types the way I did. I
>>>>> have probably omitted the kind of stratifications which can be
made
>>>>> with Stat-Analysis which might factor into this discussion.
>>>>>
>>>>> R/
>>>>> John
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>>> Sent: Thursday, August 08, 2013 10:30 AM
>>>>> To: Raby, John W CIV USARMY ARL (US)
>>>>> Cc: Reen, Brian P CIV USARMY ARL (US); rflanigan at q.com;
Flanigan,
>>>>> Robert T CIV
>>>>> (US)
>>>>> Subject: Re: [rt.rap.ucar.edu #62542] Test of ASCII2NC on
little_r
>>>>> file
>>>>>
>>>>> John,
>>>>>
>>>>> Hmmm, good question. Let me give you some background. We only
>>>>> recently began providing support for little_r data, which was
>>>>> driven by a modelling project in which the DTC was involved.
Our
>>>>> test data was limited to the data available for that project.
>>>>> Based on that data, we defined the following
>>>>> mappings:
>>>>>
>>>>> //
>>>>> // Load mapping of report types to message types
>>>>> //
>>>>> MAP_MSG_TYP["FM-12 SYNOP" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-13 SHIP" ] = "SFCSHP";
>>>>> MAP_MSG_TYP["FM-15 METAR" ] = "ADPSFC";
>>>>> MAP_MSG_TYP["FM-18 BUOY" ] = "SFCSHP";
>>>>> MAP_MSG_TYP["FM-281 QSCAT"] = "ASCATW";
>>>>> MAP_MSG_TYP["FM-32 PILOT" ] = "ADPUPA";
>>>>> MAP_MSG_TYP["FM-35 TEMP" ] = "ADPUPA";
>>>>> MAP_MSG_TYP["FM-88 SATOB" ] = "SATWND";
>>>>> MAP_MSG_TYP["FM-97 ACARS" ] = "AIRCFT";
>>>>>
>>>>> Instances of the little_r input strings on the left are mapped
to
>>>>> the message types used in the ASCII2NC output on the right.
>>>>> Looking at the data you sent me, I see that your little_r data
includes the following 22 message types:
>>>>> "FM-132"
>>>>> "FM-15_AIRNow"
>>>>> "FM-15_APRSWXNET"
>>>>> "FM-15_ASOS"
>>>>> "FM-15_CA-Hydro"
>>>>> "FM-15_CODOT"
>>>>> "FM-15_DDMET"
>>>>> "FM-15_GPSMET"
>>>>> "FM-15_HADS"
>>>>> "FM-15_MAP"
>>>>> "FM-15_MesoWest"
>>>>> "FM-15_NERRS"
>>>>> "FM-15_NonFedAWOS"
>>>>> "FM-15_NOS-NWLON"
>>>>> "FM-15_NOS-PORTS"
>>>>> "FM-15_NVDOT"
>>>>> "FM-15_OTHER-MTR"
>>>>> "FM-15_RAWS"
>>>>> "FM-15_SURFRAD"
>>>>> "FM-35"
>>>>> "FM-96"
>>>>> "FM-97_TAMDAR"
>>>>>
>>>>> None of those match the strings we were expecting. So instead
of
>>>>> mapping them to the PREPBUFR message types, we just wrote the
>>>>> strings that were input. The impact is that when you use this
data
>>>>> in Point-Stat, you'll need to list those message types
separately and get output for each one.
>>>>>
>>>>> I see several questions here:
>>>>> (1) Is it reasonable for us to figure out the super-set of all
>>>>> message types we can expect to see in little_r data?
>>>>> (2) If yes, what is that super-set?
>>>>> (3) If no, is it OK for us to just use in the input string in
the
>>>>> output? Or should we provide a way for users to define a
mapping
>>>>> of input little_r strings to output message types? We could do
>>>>> that by providing a configuration file for the ASCII2NC tool -
>>>>> which is already used for ASCII SURFRAD data.
>>>>>
>>>>> Do you have any perspectives on these issues?
>>>>>
>>>>> Regarding the quality control values, yes, I agree, they are
odd.
>>>>> Unfortunately, I don't have much information for you. I would
>>>>> suggest verifying that the values used in the little_r files
match
>>>>> the values that are showing up in the MET output. But making
sense
>>>>> of them is up to you. In Point-Stat, you can provide a list of
>>>>> quality control strings to use in the obs_quality parameter.
For
>>>>> the DTC project using little_r data, here's the list of strings
we used:
>>>>>
>>>>> obs_quality = [
>>>>> "3", "4", "5", "6", "7",
"8", "9",
>>>>> "10",
>>>>> "100003", "100004", "100005", "100006", "100007",
"100008",
>>>>> "100009", "100010",
>>>>> "200003", "200004", "200005", "200006", "200007",
"200008",
>>>>> "200009", "200010",
>>>>> "300003", "300004", "300005", "300006", "300007",
"300008",
>>>>> "300009", "300010",
>>>>> "400003", "400004", "400005", "400006", "400007",
"400008",
>>>>> "400009", "400010",
>>>>> "500003", "500004", "500005", "500006", "500007",
"500008",
>>>>> "500009", "500010",
>>>>> "600003", "600004", "600005", "600006", "600007",
"600008",
>>>>> "600009", "600010"
>>>>> ];
>>>>>
>>>>> But it looks like these are not the same values that are showing
up
>>>>> in your data.
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>> On 08/07/2013 09:57 AM, Raby, John W USA CIV via RT wrote:
>>>>>>
>>>>>> Wed Aug 07 09:57:08 2013: Request 62542 was acted upon.
>>>>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>>>>> Queue: met_help
>>>>>> Subject: Test of ASCII2NC on little_r file
>>>>>> Owner: Nobody
>>>>>> Requestors: john.w.raby2.civ at mail.mil
>>>>>> Status: new
>>>>>> Ticket <URL:
>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62542
>>>>>>>
>>>>>>
>>>>>>
>>>>>> I ran ASCII2NC as follows:
>>>>>>
>>>>>> ascii2nc qc_obs_used_earth_relative.d01.2012-03-
01_18_00_00.0000
>>>>>> qc_obs_used_earth_relative.d01.2012-03-01_18_00_00.0000.nc -log
>>>>>> test7 -v 4
>>>>>>
>>>>>> It appears that the LittleRHandler is working to indicate that
it
>>>>>> recognizes the input file as a little_r file and then generates
the NetCDF output file.
>>>>>>
>>>>>> The output file attached as the text file " test7_ncdump"
appears
>>>>>> to contain good data, but the obs_qty information looks odd in
places.
>>>>>>
>>>>>> I've also attached the log file which contain extensive
WARNINGS,
>>>>>> but no ERROR.
>>>>>>
>>>>>> Can you explain the odd numbers in the obs_qty array? I was
>>>>>> expecting numbers like 0, 1, 2, etc but not "512" or "16384".
Also
>>>>>> any help you can provide on the reasons for the WARNINGS would
be appreciated.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> R/
>>>>>> John
>>>>>>
>>>>>>
>>>>>> Mr John W. Raby, Meteorologist
>>>>>>
>>>>>> U.S. Army Research Laboratory
>>>>>>
>>>>>> White Sands Missile Range, NM 88002
>>>>>>
>>>>>> (575) 678-2004 DSN 258-2004
>>>>>>
>>>>>> FAX (575) 678-1230 DSN 258-1230
>>>>>>
>>>>>> Email: john.w.raby2.civ at mail.mil<mailto:john.raby at us.army.mil>
>>>>>>
>>>>>
>>>>>
>>>>> Classification: UNCLASSIFIED
>>>>> Caveats: NONE
>>>>>
>>>>>
>>>>>
>>>>> Classification: UNCLASSIFIED
>>>>> Caveats: NONE
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
More information about the Met_help
mailing list