[Met_help] [rt.rap.ucar.edu #79721] History for Point Stat Obs Not Filtering
John Halley Gotway via RT
met_help at ucar.edu
Tue Mar 7 09:16:22 MST 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, I am running point stat and have included a SID file to have MET only
use the observations in the SID file for verification. The SID file is
attached. I followed the format in the users guide. When I run point stat
with the SID file (see attached config file), I don't get any apparent
filtering. A listing of the output is below:
DEBUG 2:
DEBUG 2: Searching 322840 observations from 77809 messages.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method NEAREST(1), using 33863 pairs.
DEBUG 3: Number of matched pairs = 33863
DEBUG 3: Observations processed = 322840
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: GRIB code = 284544
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 52
DEBUG 3: Rejected: level mismatch = 19
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 4362
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Continuous Statistics.
DEBUG 3: Using 33863 of 33863 pairs for forecast filtering threshold NA,
observation filtering threshold NA, and field logic UNION.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region hira_compare_obs_tt_t44b_met.stations, for interpolation method
NEAREST(1), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 322840
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: GRIB code = 284544
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 52
DEBUG 3: Rejected: level mismatch = 19
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 4362
DEBUG 3: Rejected: masking region = 33863
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2:
The output files are the same size as if I had run them without filtering.
When I search for "t44b" (my filter name) I don't find it in the on the
datalines that it should be on. So, I am at loss to figure out why I am not
getting a filtered file. Am mis-understanding how this is supposed to work?
Thanks
Bob Craig
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Fri Mar 03 17:00:48 2017
Bob,
Short answer ... add a newline to the end of your
"hira_compare_obs_tt_t44b_met.stations"
file.
Long answer ...
I can see why this is confusing. There are two pieces of logic
related to
station ID's that are occurring... one is "sid_exc" and another is
"mask.sid". You are (correctly) using the latter, but not the former.
I took the following from your Point-Stat config file:
mask = {
grid = [ "FULL" ];
poly = [];
sid = ["hira_compare_obs_tt_t44b_met.stations"];
}
You are telling Point-Stat to compute statistics using two masking
regions.
First, with grid = "FULL" you're telling Point-Stat to use all
stations
falling inside you FULL model domain. Looking at the log output you
sent,
that results in 33863 matched pairs.
The second masking region is defined by the list of stations ID's in
the
file named "hira_compare_obs_tt_t44b_met.stations". Looking at the
log
output you sent, that results in *** 0 *** matched pairs:
DEBUG 3: Number of matched pairs = 0
So it makes complete sense that you're getting the same results
with/without mask.sid since mask.sid isn't producing any matched
pairs. I
suspect that you're getting confused by this line in the log output:
DEBUG 3: Rejected: SID exclusion = 0
That line is telling you the number of observations discarded because
of
the "sid_exc" setting... which totally separate from "mask.sid".
That's a
list of stations you always want to ignore because you know they
contain
bad data. You're not using "sid_exc" so that log message should
always be
0 in your output.
The real question here is why are you getting 0 matched pairs from
your
"mask.sid" masking region?
I looked at an example of using the "mask.sid" option from our nightly
build at verbosity level 4 and saw the following lines in the log
output:
DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
DEBUG 4: parse_sid_mask() -> parsing station ID masking file
"/d1/johnhg/MET/MET_development/
svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the "SID_CO"
mask
from file "/d1/johnhg/MET/MET_development/
svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
Verbosity level 4 will tell you how many stations were parsed from
your
file. I'm pretty sure yours will say 0.
I did some testing and think that what's missing is a newline at the
end of
your list of stations. If you add a newline, I'm guessing that'll fix
it.
I'll look in the user's guide and add the instruction to terminate the
line
with a newline.
Hope that does it.
Thanks,
John
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 09:33:12 2017
John, I looked at the file and there is a newline character there. I
took it out and ran it and get the same lack of filtering with or
without it. I tried changing the filename to end in .txt and none of
that changed anything. When I compare your output to mine, there is
one big difference. My code thinks I am providing a "single station
ID mask" where your code correctly said you were providing a "
station ID masking file". See below:
Mine-
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll: -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: parse_sid_mask() -> storing single station ID mask "text.txt"
DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology levels.
Yours -
DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
DEBUG 4: parse_sid_mask() -> parsing station ID masking file
"/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the "SID_CO"
mask from file "/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
So, why the difference. I took the brackets off of the sid line in
the config file and ran but got no difference. Is my code assuming I
am not providing a file but single station id?
Also, notice you have a line “processing sid mask” where I don’t see
that in my listing. So why the difference?
Also, I still can’t process PRMSL.
DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2017022306
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for PRMSL/L0.
WARNING:
WARNING: MetGrib2DataFile::data_plane_array() - No matching records
found for 'PRMSL/L0'
WARNING:
WARNING:
WARNING: process_fcst_climo_files() -> no fields matching PRMSL/L0
found in file: /h/data/global/WXQC/data/umm/1702230006
WARNING:
DEBUG 2:
I stored the config file and model file in the ftp directory.
Thanks for you help
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, March 03, 2017 6:01 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
Short answer ... add a newline to the end of your
"hira_compare_obs_tt_t44b_met.stations"
file.
Long answer ...
I can see why this is confusing. There are two pieces of logic
related to station ID's that are occurring... one is "sid_exc" and
another is "mask.sid". You are (correctly) using the latter, but not
the former.
I took the following from your Point-Stat config file:
mask = {
grid = [ "FULL" ];
poly = [];
sid = ["hira_compare_obs_tt_t44b_met.stations"];
}
You are telling Point-Stat to compute statistics using two masking
regions.
First, with grid = "FULL" you're telling Point-Stat to use all
stations falling inside you FULL model domain. Looking at the log
output you sent, that results in 33863 matched pairs.
The second masking region is defined by the list of stations ID's in
the file named "hira_compare_obs_tt_t44b_met.stations". Looking at
the log output you sent, that results in *** 0 *** matched pairs:
DEBUG 3: Number of matched pairs = 0
So it makes complete sense that you're getting the same results
with/without mask.sid since mask.sid isn't producing any matched
pairs. I suspect that you're getting confused by this line in the log
output:
DEBUG 3: Rejected: SID exclusion = 0
That line is telling you the number of observations discarded because
of the "sid_exc" setting... which totally separate from "mask.sid".
That's a list of stations you always want to ignore because you know
they contain bad data. You're not using "sid_exc" so that log message
should always be
0 in your output.
The real question here is why are you getting 0 matched pairs from
your "mask.sid" masking region?
I looked at an example of using the "mask.sid" option from our nightly
build at verbosity level 4 and saw the following lines in the log
output:
DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
DEBUG 4: parse_sid_mask() -> parsing station ID masking file
"/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the "SID_CO"
mask from file "/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt"
Verbosity level 4 will tell you how many stations were parsed from
your file. I'm pretty sure yours will say 0.
I did some testing and think that what's missing is a newline at the
end of your list of stations. If you add a newline, I'm guessing
that'll fix it.
I'll look in the user's guide and add the instruction to terminate the
line with a newline.
Hope that does it.
Thanks,
John
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 10:45:12 2017
Bob,
Ah OK... here's how the logic works. Nice job identifying that
difference. The "sid.mask" setting can either be set to the name of a
file
or to a single station ID name (which was the behavior from earlier
releases).
Point-Stat first tries to read the string as a filename... but if that
is
unsuccessful, then it interprets it as the name of a station.
Based on the log output you sent, Point-Stat is looking for stations
named
"text.txt"! And obviously, none of your stations are named that.
If you put in the full path to that "text.txt" file, it should start
working. But this is a pretty subtle detail and pretty frustrating to
debug I'm guessing! What do you think about adding the following
logic...
When parsing that string, if it contains a '/' or a '.' character and
trying to interpret it as a filename is unsuccessful, print a WARNING
message indicating that. Basically, the presence of a slash or dot
would
likely indicate the user is trying to specify a file name.
Would that warning message have made this situation a lot easier to
debug?
Thanks,
John
On Mon, Mar 6, 2017 at 9:33 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> John, I looked at the file and there is a newline character there.
I
> took it out and ran it and get the same lack of filtering with or
without
> it. I tried changing the filename to end in .txt and none of that
changed
> anything. When I compare your output to mine, there is one big
> difference. My code thinks I am providing a "single station ID
mask"
> where your code correctly said you were providing a " station ID
masking
> file". See below:
>
>
>
> Mine-
>
> DEBUG 3: Use the matching forecast and observation grids.
>
> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll:
> -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
>
> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>
> DEBUG 4: parse_sid_mask() -> storing single station ID mask
"text.txt"
>
> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
>
>
>
> Yours -
>
> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>
> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/
> trunk/test/config/SID_CO.txt"
>
> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the "SID_CO"
mask
> from file "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/
> trunk/test/config/SID_CO.txt"
>
>
>
> So, why the difference. I took the brackets off of the sid line in
the
> config file and ran but got no difference. Is my code assuming I am
not
> providing a file but single station id?
>
> Also, notice you have a line “processing sid mask” where I don’t see
that
> in my listing. So why the difference?
>
>
>
> Also, I still can’t process PRMSL.
>
>
>
> DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
>
> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2017022306
>
> DEBUG 2:
>
> DEBUG 2:
------------------------------------------------------------
> --------------------
>
> DEBUG 2:
>
> DEBUG 2: Reading data for PRMSL/L0.
>
> WARNING:
>
> WARNING: MetGrib2DataFile::data_plane_array() - No matching records
found
> for 'PRMSL/L0'
>
> WARNING:
>
> WARNING:
>
> WARNING: process_fcst_climo_files() -> no fields matching PRMSL/L0
found
> in file: /h/data/global/WXQC/data/umm/1702230006
>
> WARNING:
>
> DEBUG 2:
>
>
>
> I stored the config file and model file in the ftp directory.
>
>
>
> Thanks for you help
>
> Bob
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, March 03, 2017 6:01 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
>
>
> Bob,
>
>
>
> Short answer ... add a newline to the end of your
> "hira_compare_obs_tt_t44b_met.stations"
>
> file.
>
>
>
> Long answer ...
>
>
>
> I can see why this is confusing. There are two pieces of logic
related to
> station ID's that are occurring... one is "sid_exc" and another is
> "mask.sid". You are (correctly) using the latter, but not the
former.
>
>
>
> I took the following from your Point-Stat config file:
>
> mask = {
>
> grid = [ "FULL" ];
>
> poly = [];
>
> sid = ["hira_compare_obs_tt_t44b_met.stations"];
>
> }
>
>
>
> You are telling Point-Stat to compute statistics using two masking
regions.
>
>
>
> First, with grid = "FULL" you're telling Point-Stat to use all
stations
> falling inside you FULL model domain. Looking at the log output you
sent,
> that results in 33863 matched pairs.
>
>
>
> The second masking region is defined by the list of stations ID's in
the
> file named "hira_compare_obs_tt_t44b_met.stations". Looking at the
log
> output you sent, that results in *** 0 *** matched pairs:
>
> DEBUG 3: Number of matched pairs = 0
>
>
>
> So it makes complete sense that you're getting the same results
> with/without mask.sid since mask.sid isn't producing any matched
pairs. I
> suspect that you're getting confused by this line in the log output:
>
> DEBUG 3: Rejected: SID exclusion = 0
>
>
>
> That line is telling you the number of observations discarded
because of
> the "sid_exc" setting... which totally separate from "mask.sid".
That's a
> list of stations you always want to ignore because you know they
contain
> bad data. You're not using "sid_exc" so that log message should
always be
>
> 0 in your output.
>
>
>
> The real question here is why are you getting 0 matched pairs from
your
> "mask.sid" masking region?
>
>
>
> I looked at an example of using the "mask.sid" option from our
nightly
> build at verbosity level 4 and saw the following lines in the log
output:
>
>
>
> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>
> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/
> trunk/test/config/SID_CO.txt"
>
> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the "SID_CO"
mask
> from file "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/
> trunk/test/config/SID_CO.txt"
>
>
>
> Verbosity level 4 will tell you how many stations were parsed from
your
> file. I'm pretty sure yours will say 0.
>
>
>
> I did some testing and think that what's missing is a newline at the
end
> of your list of stations. If you add a newline, I'm guessing
that'll fix
> it.
>
>
>
> I'll look in the user's guide and add the instruction to terminate
the
> line with a newline.
>
>
>
> Hope that does it.
>
>
>
> Thanks,
>
> John
>
>
>
>
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 11:28:15 2017
Bob,
I found that we're already preventing the use slashes in the name of a
station id:
ERROR :
ERROR : parse_sid_mask() -> masking station ID name can't contain
whitespace or slashes "this/is/a/bad/filename".
ERROR :
But to cover your exact case (i.e. no slashes but with dots), I added
the
following warning:
WARNING:
WARNING: parse_sid_mask() -> unable to process
"this.is.a.bad.filename" as
a file name and processing it as a single station ID mask instead.
WARNING:
I think it should be a warning rather than an error in case users
actually
have stations ID names which include a dot.
Does that look good to you?
Thanks,
John
On Mon, Mar 6, 2017 at 10:44 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Bob,
>
> Ah OK... here's how the logic works. Nice job identifying that
> difference. The "sid.mask" setting can either be set to the name of
a file
> or to a single station ID name (which was the behavior from earlier
> releases).
>
> Point-Stat first tries to read the string as a filename... but if
that is
> unsuccessful, then it interprets it as the name of a station.
>
> Based on the log output you sent, Point-Stat is looking for stations
named
> "text.txt"! And obviously, none of your stations are named that.
>
> If you put in the full path to that "text.txt" file, it should start
> working. But this is a pretty subtle detail and pretty frustrating
to
> debug I'm guessing! What do you think about adding the following
logic...
>
> When parsing that string, if it contains a '/' or a '.' character
and
> trying to interpret it as a filename is unsuccessful, print a
WARNING
> message indicating that. Basically, the presence of a slash or dot
would
> likely indicate the user is trying to specify a file name.
>
> Would that warning message have made this situation a lot easier to
debug?
>
> Thanks,
> John
>
>
>
> On Mon, Mar 6, 2017 at 9:33 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>>
>> John, I looked at the file and there is a newline character there.
I
>> took it out and ran it and get the same lack of filtering with or
without
>> it. I tried changing the filename to end in .txt and none of that
changed
>> anything. When I compare your output to mine, there is one big
>> difference. My code thinks I am providing a "single station ID
mask"
>> where your code correctly said you were providing a " station ID
masking
>> file". See below:
>>
>>
>>
>> Mine-
>>
>> DEBUG 3: Use the matching forecast and observation grids.
>>
>> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll:
>> -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
>>
>> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>>
>> DEBUG 4: parse_sid_mask() -> storing single station ID mask
"text.txt"
>>
>> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
>>
>>
>>
>> Yours -
>>
>> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>>
>> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
>> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO" mask
>> from file "/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>>
>>
>> So, why the difference. I took the brackets off of the sid line in
the
>> config file and ran but got no difference. Is my code assuming I
am not
>> providing a file but single station id?
>>
>> Also, notice you have a line “processing sid mask” where I don’t
see that
>> in my listing. So why the difference?
>>
>>
>>
>> Also, I still can’t process PRMSL.
>>
>>
>>
>> DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
>>
>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2017022306
>>
>> DEBUG 2:
>>
>> DEBUG 2:
------------------------------------------------------------
>> --------------------
>>
>> DEBUG 2:
>>
>> DEBUG 2: Reading data for PRMSL/L0.
>>
>> WARNING:
>>
>> WARNING: MetGrib2DataFile::data_plane_array() - No matching records
>> found for 'PRMSL/L0'
>>
>> WARNING:
>>
>> WARNING:
>>
>> WARNING: process_fcst_climo_files() -> no fields matching PRMSL/L0
found
>> in file: /h/data/global/WXQC/data/umm/1702230006
>>
>> WARNING:
>>
>> DEBUG 2:
>>
>>
>>
>> I stored the config file and model file in the ftp directory.
>>
>>
>>
>> Thanks for you help
>>
>> Bob
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Friday, March 03, 2017 6:01 PM
>> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
>> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>>
>>
>>
>> Bob,
>>
>>
>>
>> Short answer ... add a newline to the end of your
>> "hira_compare_obs_tt_t44b_met.stations"
>>
>> file.
>>
>>
>>
>> Long answer ...
>>
>>
>>
>> I can see why this is confusing. There are two pieces of logic
related
>> to station ID's that are occurring... one is "sid_exc" and another
is
>> "mask.sid". You are (correctly) using the latter, but not the
former.
>>
>>
>>
>> I took the following from your Point-Stat config file:
>>
>> mask = {
>>
>> grid = [ "FULL" ];
>>
>> poly = [];
>>
>> sid = ["hira_compare_obs_tt_t44b_met.stations"];
>>
>> }
>>
>>
>>
>> You are telling Point-Stat to compute statistics using two masking
>> regions.
>>
>>
>>
>> First, with grid = "FULL" you're telling Point-Stat to use all
stations
>> falling inside you FULL model domain. Looking at the log output
you sent,
>> that results in 33863 matched pairs.
>>
>>
>>
>> The second masking region is defined by the list of stations ID's
in the
>> file named "hira_compare_obs_tt_t44b_met.stations". Looking at the
log
>> output you sent, that results in *** 0 *** matched pairs:
>>
>> DEBUG 3: Number of matched pairs = 0
>>
>>
>>
>> So it makes complete sense that you're getting the same results
>> with/without mask.sid since mask.sid isn't producing any matched
pairs. I
>> suspect that you're getting confused by this line in the log
output:
>>
>> DEBUG 3: Rejected: SID exclusion = 0
>>
>>
>>
>> That line is telling you the number of observations discarded
because of
>> the "sid_exc" setting... which totally separate from "mask.sid".
That's a
>> list of stations you always want to ignore because you know they
contain
>> bad data. You're not using "sid_exc" so that log message should
always be
>>
>> 0 in your output.
>>
>>
>>
>> The real question here is why are you getting 0 matched pairs from
your
>> "mask.sid" masking region?
>>
>>
>>
>> I looked at an example of using the "mask.sid" option from our
nightly
>> build at verbosity level 4 and saw the following lines in the log
output:
>>
>>
>>
>> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>>
>> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
>> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO" mask
>> from file "/d1/johnhg/MET/MET_development/ svn-met-
dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>>
>>
>> Verbosity level 4 will tell you how many stations were parsed from
your
>> file. I'm pretty sure yours will say 0.
>>
>>
>>
>> I did some testing and think that what's missing is a newline at
the end
>> of your list of stations. If you add a newline, I'm guessing
that'll fix
>> it.
>>
>>
>>
>> I'll look in the user's guide and add the instruction to terminate
the
>> line with a newline.
>>
>>
>>
>> Hope that does it.
>>
>>
>>
>> Thanks,
>>
>> John
>>
>>
>>
>>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 11:44:28 2017
Yes, that looks good.
Thanks
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 06, 2017 12:28 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
I found that we're already preventing the use slashes in the name of a
station id:
ERROR :
ERROR : parse_sid_mask() -> masking station ID name can't contain
whitespace or slashes "this/is/a/bad/filename".
ERROR :
But to cover your exact case (i.e. no slashes but with dots), I added
the following warning:
WARNING:
WARNING: parse_sid_mask() -> unable to process
"this.is.a.bad.filename" as a file name and processing it as a single
station ID mask instead.
WARNING:
I think it should be a warning rather than an error in case users
actually have stations ID names which include a dot.
Does that look good to you?
Thanks,
John
On Mon, Mar 6, 2017 at 10:44 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Bob,
>
> Ah OK... here's how the logic works. Nice job identifying that
> difference. The "sid.mask" setting can either be set to the name of
a
> file or to a single station ID name (which was the behavior from
> earlier releases).
>
> Point-Stat first tries to read the string as a filename... but if
that
> is unsuccessful, then it interprets it as the name of a station.
>
> Based on the log output you sent, Point-Stat is looking for stations
> named "text.txt"! And obviously, none of your stations are named
that.
>
> If you put in the full path to that "text.txt" file, it should start
> working. But this is a pretty subtle detail and pretty frustrating
to
> debug I'm guessing! What do you think about adding the following
logic...
>
> When parsing that string, if it contains a '/' or a '.' character
and
> trying to interpret it as a filename is unsuccessful, print a
WARNING
> message indicating that. Basically, the presence of a slash or dot
> would likely indicate the user is trying to specify a file name.
>
> Would that warning message have made this situation a lot easier to
debug?
>
> Thanks,
> John
>
>
>
> On Mon, Mar 6, 2017 at 9:33 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>>
>> John, I looked at the file and there is a newline character there.
>> I took it out and ran it and get the same lack of filtering with or
>> without it. I tried changing the filename to end in .txt and none
of that changed
>> anything. When I compare your output to mine, there is one big
>> difference. My code thinks I am providing a "single station ID
mask"
>> where your code correctly said you were providing a " station ID
>> masking file". See below:
>>
>>
>>
>> Mine-
>>
>> DEBUG 3: Use the matching forecast and observation grids.
>>
>> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll:
>> -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
>>
>> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>>
>> DEBUG 4: parse_sid_mask() -> storing single station ID mask
"text.txt"
>>
>> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
>>
>>
>>
>> Yours -
>>
>> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>>
>> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
>> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
>> mask from file "/d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
>>
>>
>>
>> So, why the difference. I took the brackets off of the sid line in
>> the config file and ran but got no difference. Is my code assuming
I
>> am not providing a file but single station id?
>>
>> Also, notice you have a line “processing sid mask” where I don’t
see
>> that in my listing. So why the difference?
>>
>>
>>
>> Also, I still can’t process PRMSL.
>>
>>
>>
>> DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
>>
>> DEBUG 1: Observation File:
>> /h/data/global/WXQC/data/obs_prep/2017022306
>>
>> DEBUG 2:
>>
>> DEBUG 2:
------------------------------------------------------------
>> --------------------
>>
>> DEBUG 2:
>>
>> DEBUG 2: Reading data for PRMSL/L0.
>>
>> WARNING:
>>
>> WARNING: MetGrib2DataFile::data_plane_array() - No matching records
>> found for 'PRMSL/L0'
>>
>> WARNING:
>>
>> WARNING:
>>
>> WARNING: process_fcst_climo_files() -> no fields matching PRMSL/L0
>> found in file: /h/data/global/WXQC/data/umm/1702230006
>>
>> WARNING:
>>
>> DEBUG 2:
>>
>>
>>
>> I stored the config file and model file in the ftp directory.
>>
>>
>>
>> Thanks for you help
>>
>> Bob
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Friday, March 03, 2017 6:01 PM
>> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
>> <robert.craig.2 at us.af.mil>
>> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>>
>>
>>
>> Bob,
>>
>>
>>
>> Short answer ... add a newline to the end of your
>> "hira_compare_obs_tt_t44b_met.stations"
>>
>> file.
>>
>>
>>
>> Long answer ...
>>
>>
>>
>> I can see why this is confusing. There are two pieces of logic
>> related to station ID's that are occurring... one is "sid_exc" and
>> another is "mask.sid". You are (correctly) using the latter, but
not the former.
>>
>>
>>
>> I took the following from your Point-Stat config file:
>>
>> mask = {
>>
>> grid = [ "FULL" ];
>>
>> poly = [];
>>
>> sid = ["hira_compare_obs_tt_t44b_met.stations"];
>>
>> }
>>
>>
>>
>> You are telling Point-Stat to compute statistics using two masking
>> regions.
>>
>>
>>
>> First, with grid = "FULL" you're telling Point-Stat to use all
>> stations falling inside you FULL model domain. Looking at the log
>> output you sent, that results in 33863 matched pairs.
>>
>>
>>
>> The second masking region is defined by the list of stations ID's
in
>> the file named "hira_compare_obs_tt_t44b_met.stations". Looking at
>> the log output you sent, that results in *** 0 *** matched pairs:
>>
>> DEBUG 3: Number of matched pairs = 0
>>
>>
>>
>> So it makes complete sense that you're getting the same results
>> with/without mask.sid since mask.sid isn't producing any matched
>> pairs. I suspect that you're getting confused by this line in the
log output:
>>
>> DEBUG 3: Rejected: SID exclusion = 0
>>
>>
>>
>> That line is telling you the number of observations discarded
because
>> of the "sid_exc" setting... which totally separate from "mask.sid".
>> That's a list of stations you always want to ignore because you
know
>> they contain bad data. You're not using "sid_exc" so that log
>> message should always be
>>
>> 0 in your output.
>>
>>
>>
>> The real question here is why are you getting 0 matched pairs from
>> your "mask.sid" masking region?
>>
>>
>>
>> I looked at an example of using the "mask.sid" option from our
>> nightly build at verbosity level 4 and saw the following lines in
the log output:
>>
>>
>>
>> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
>>
>> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
>> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
>> /test/config/SID_CO.txt"
>>
>> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
>> mask from file "/d1/johnhg/MET/MET_development/
>> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
>>
>>
>>
>> Verbosity level 4 will tell you how many stations were parsed from
>> your file. I'm pretty sure yours will say 0.
>>
>>
>>
>> I did some testing and think that what's missing is a newline at
the
>> end of your list of stations. If you add a newline, I'm guessing
>> that'll fix it.
>>
>>
>>
>> I'll look in the user's guide and add the instruction to terminate
>> the line with a newline.
>>
>>
>>
>> Hope that does it.
>>
>>
>>
>> Thanks,
>>
>> John
>>
>>
>>
>>
>
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 12:01:22 2017
Bob,
Great, and just to confirm, did supplying the full path to your
masking
station ID file work as expected?
John
On Mon, Mar 6, 2017 at 11:44 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> Yes, that looks good.
>
> Thanks
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 12:28 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> I found that we're already preventing the use slashes in the name of
a
> station id:
>
> ERROR :
> ERROR : parse_sid_mask() -> masking station ID name can't contain
> whitespace or slashes "this/is/a/bad/filename".
> ERROR :
>
> But to cover your exact case (i.e. no slashes but with dots), I
added the
> following warning:
>
> WARNING:
> WARNING: parse_sid_mask() -> unable to process
"this.is.a.bad.filename" as
> a file name and processing it as a single station ID mask instead.
> WARNING:
>
> I think it should be a warning rather than an error in case users
actually
> have stations ID names which include a dot.
>
> Does that look good to you?
>
> Thanks,
> John
>
>
> On Mon, Mar 6, 2017 at 10:44 AM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Bob,
> >
> > Ah OK... here's how the logic works. Nice job identifying that
> > difference. The "sid.mask" setting can either be set to the name
of a
> > file or to a single station ID name (which was the behavior from
> > earlier releases).
> >
> > Point-Stat first tries to read the string as a filename... but if
that
> > is unsuccessful, then it interprets it as the name of a station.
> >
> > Based on the log output you sent, Point-Stat is looking for
stations
> > named "text.txt"! And obviously, none of your stations are named
that.
> >
> > If you put in the full path to that "text.txt" file, it should
start
> > working. But this is a pretty subtle detail and pretty
frustrating to
> > debug I'm guessing! What do you think about adding the following
> logic...
> >
> > When parsing that string, if it contains a '/' or a '.' character
and
> > trying to interpret it as a filename is unsuccessful, print a
WARNING
> > message indicating that. Basically, the presence of a slash or
dot
> > would likely indicate the user is trying to specify a file name.
> >
> > Would that warning message have made this situation a lot easier
to
> debug?
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 6, 2017 at 9:33 AM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
> >>
> >> John, I looked at the file and there is a newline character
there.
> >> I took it out and ran it and get the same lack of filtering with
or
> >> without it. I tried changing the filename to end in .txt and
none of
> that changed
> >> anything. When I compare your output to mine, there is one big
> >> difference. My code thinks I am providing a "single station ID
mask"
> >> where your code correctly said you were providing a " station ID
> >> masking file". See below:
> >>
> >>
> >>
> >> Mine-
> >>
> >> DEBUG 3: Use the matching forecast and observation grids.
> >>
> >> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll:
> >> -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
> >>
> >> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
> >>
> >> DEBUG 4: parse_sid_mask() -> storing single station ID mask
"text.txt"
> >>
> >> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
> >>
> >>
> >>
> >> Yours -
> >>
> >> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
> >>
> >> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> >> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
> >> /test/config/SID_CO.txt"
> >>
> >> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
> >> mask from file "/d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
> >>
> >>
> >>
> >> So, why the difference. I took the brackets off of the sid line
in
> >> the config file and ran but got no difference. Is my code
assuming I
> >> am not providing a file but single station id?
> >>
> >> Also, notice you have a line “processing sid mask” where I don’t
see
> >> that in my listing. So why the difference?
> >>
> >>
> >>
> >> Also, I still can’t process PRMSL.
> >>
> >>
> >>
> >> DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
> >>
> >> DEBUG 1: Observation File:
> >> /h/data/global/WXQC/data/obs_prep/2017022306
> >>
> >> DEBUG 2:
> >>
> >> DEBUG 2:
------------------------------------------------------------
> >> --------------------
> >>
> >> DEBUG 2:
> >>
> >> DEBUG 2: Reading data for PRMSL/L0.
> >>
> >> WARNING:
> >>
> >> WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> >> found for 'PRMSL/L0'
> >>
> >> WARNING:
> >>
> >> WARNING:
> >>
> >> WARNING: process_fcst_climo_files() -> no fields matching
PRMSL/L0
> >> found in file: /h/data/global/WXQC/data/umm/1702230006
> >>
> >> WARNING:
> >>
> >> DEBUG 2:
> >>
> >>
> >>
> >> I stored the config file and model file in the ftp directory.
> >>
> >>
> >>
> >> Thanks for you help
> >>
> >> Bob
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> >> Sent: Friday, March 03, 2017 6:01 PM
> >> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> >> <robert.craig.2 at us.af.mil>
> >> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not
Filtering
> >>
> >>
> >>
> >> Bob,
> >>
> >>
> >>
> >> Short answer ... add a newline to the end of your
> >> "hira_compare_obs_tt_t44b_met.stations"
> >>
> >> file.
> >>
> >>
> >>
> >> Long answer ...
> >>
> >>
> >>
> >> I can see why this is confusing. There are two pieces of logic
> >> related to station ID's that are occurring... one is "sid_exc"
and
> >> another is "mask.sid". You are (correctly) using the latter, but
not
> the former.
> >>
> >>
> >>
> >> I took the following from your Point-Stat config file:
> >>
> >> mask = {
> >>
> >> grid = [ "FULL" ];
> >>
> >> poly = [];
> >>
> >> sid = ["hira_compare_obs_tt_t44b_met.stations"];
> >>
> >> }
> >>
> >>
> >>
> >> You are telling Point-Stat to compute statistics using two
masking
> >> regions.
> >>
> >>
> >>
> >> First, with grid = "FULL" you're telling Point-Stat to use all
> >> stations falling inside you FULL model domain. Looking at the
log
> >> output you sent, that results in 33863 matched pairs.
> >>
> >>
> >>
> >> The second masking region is defined by the list of stations ID's
in
> >> the file named "hira_compare_obs_tt_t44b_met.stations". Looking
at
> >> the log output you sent, that results in *** 0 *** matched pairs:
> >>
> >> DEBUG 3: Number of matched pairs = 0
> >>
> >>
> >>
> >> So it makes complete sense that you're getting the same results
> >> with/without mask.sid since mask.sid isn't producing any matched
> >> pairs. I suspect that you're getting confused by this line in
the log
> output:
> >>
> >> DEBUG 3: Rejected: SID exclusion = 0
> >>
> >>
> >>
> >> That line is telling you the number of observations discarded
because
> >> of the "sid_exc" setting... which totally separate from
"mask.sid".
> >> That's a list of stations you always want to ignore because you
know
> >> they contain bad data. You're not using "sid_exc" so that log
> >> message should always be
> >>
> >> 0 in your output.
> >>
> >>
> >>
> >> The real question here is why are you getting 0 matched pairs
from
> >> your "mask.sid" masking region?
> >>
> >>
> >>
> >> I looked at an example of using the "mask.sid" option from our
> >> nightly build at verbosity level 4 and saw the following lines in
the
> log output:
> >>
> >>
> >>
> >> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
> >>
> >> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> >> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
> >> /test/config/SID_CO.txt"
> >>
> >> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
> >> mask from file "/d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
> >>
> >>
> >>
> >> Verbosity level 4 will tell you how many stations were parsed
from
> >> your file. I'm pretty sure yours will say 0.
> >>
> >>
> >>
> >> I did some testing and think that what's missing is a newline at
the
> >> end of your list of stations. If you add a newline, I'm guessing
> >> that'll fix it.
> >>
> >>
> >>
> >> I'll look in the user's guide and add the instruction to
terminate
> >> the line with a newline.
> >>
> >>
> >>
> >> Hope that does it.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> John
> >>
> >>
> >>
> >>
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 12:05:10 2017
Yes. Any idea on PRMSL/L0 problem?
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 06, 2017 1:01 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
Great, and just to confirm, did supplying the full path to your
masking station ID file work as expected?
John
On Mon, Mar 6, 2017 at 11:44 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> Yes, that looks good.
>
> Thanks
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 12:28 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> I found that we're already preventing the use slashes in the name of
a
> station id:
>
> ERROR :
> ERROR : parse_sid_mask() -> masking station ID name can't contain
> whitespace or slashes "this/is/a/bad/filename".
> ERROR :
>
> But to cover your exact case (i.e. no slashes but with dots), I
added
> the following warning:
>
> WARNING:
> WARNING: parse_sid_mask() -> unable to process
> "this.is.a.bad.filename" as a file name and processing it as a
single station ID mask instead.
> WARNING:
>
> I think it should be a warning rather than an error in case users
> actually have stations ID names which include a dot.
>
> Does that look good to you?
>
> Thanks,
> John
>
>
> On Mon, Mar 6, 2017 at 10:44 AM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Bob,
> >
> > Ah OK... here's how the logic works. Nice job identifying that
> > difference. The "sid.mask" setting can either be set to the name
of
> > a file or to a single station ID name (which was the behavior from
> > earlier releases).
> >
> > Point-Stat first tries to read the string as a filename... but if
> > that is unsuccessful, then it interprets it as the name of a
station.
> >
> > Based on the log output you sent, Point-Stat is looking for
stations
> > named "text.txt"! And obviously, none of your stations are named
that.
> >
> > If you put in the full path to that "text.txt" file, it should
start
> > working. But this is a pretty subtle detail and pretty
frustrating
> > to debug I'm guessing! What do you think about adding the
following
> logic...
> >
> > When parsing that string, if it contains a '/' or a '.' character
> > and trying to interpret it as a filename is unsuccessful, print a
> > WARNING message indicating that. Basically, the presence of a
slash
> > or dot would likely indicate the user is trying to specify a file
name.
> >
> > Would that warning message have made this situation a lot easier
to
> debug?
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Mar 6, 2017 at 9:33 AM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
> >>
> >> John, I looked at the file and there is a newline character
there.
> >> I took it out and ran it and get the same lack of filtering with
or
> >> without it. I tried changing the filename to end in .txt and
none
> >> of
> that changed
> >> anything. When I compare your output to mine, there is one big
> >> difference. My code thinks I am providing a "single station ID
mask"
> >> where your code correctly said you were providing a " station ID
> >> masking file". See below:
> >>
> >>
> >>
> >> Mine-
> >>
> >> DEBUG 3: Use the matching forecast and observation grids.
> >>
> >> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll:
> >> -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
> >>
> >> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
> >>
> >> DEBUG 4: parse_sid_mask() -> storing single station ID mask
"text.txt"
> >>
> >> DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology
levels.
> >>
> >>
> >>
> >> Yours -
> >>
> >> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
> >>
> >> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> >> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
> >> /test/config/SID_CO.txt"
> >>
> >> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
> >> mask from file "/d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
> >>
> >>
> >>
> >> So, why the difference. I took the brackets off of the sid line
in
> >> the config file and ran but got no difference. Is my code
assuming
> >> I am not providing a file but single station id?
> >>
> >> Also, notice you have a line “processing sid mask” where I don’t
> >> see that in my listing. So why the difference?
> >>
> >>
> >>
> >> Also, I still can’t process PRMSL.
> >>
> >>
> >>
> >> DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1702230006
> >>
> >> DEBUG 1: Observation File:
> >> /h/data/global/WXQC/data/obs_prep/2017022306
> >>
> >> DEBUG 2:
> >>
> >> DEBUG 2:
> >> ------------------------------------------------------------
> >> --------------------
> >>
> >> DEBUG 2:
> >>
> >> DEBUG 2: Reading data for PRMSL/L0.
> >>
> >> WARNING:
> >>
> >> WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> >> found for 'PRMSL/L0'
> >>
> >> WARNING:
> >>
> >> WARNING:
> >>
> >> WARNING: process_fcst_climo_files() -> no fields matching
PRMSL/L0
> >> found in file: /h/data/global/WXQC/data/umm/1702230006
> >>
> >> WARNING:
> >>
> >> DEBUG 2:
> >>
> >>
> >>
> >> I stored the config file and model file in the ftp directory.
> >>
> >>
> >>
> >> Thanks for you help
> >>
> >> Bob
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> >> Sent: Friday, March 03, 2017 6:01 PM
> >> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> >> <robert.craig.2 at us.af.mil>
> >> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not
Filtering
> >>
> >>
> >>
> >> Bob,
> >>
> >>
> >>
> >> Short answer ... add a newline to the end of your
> >> "hira_compare_obs_tt_t44b_met.stations"
> >>
> >> file.
> >>
> >>
> >>
> >> Long answer ...
> >>
> >>
> >>
> >> I can see why this is confusing. There are two pieces of logic
> >> related to station ID's that are occurring... one is "sid_exc"
and
> >> another is "mask.sid". You are (correctly) using the latter, but
> >> not
> the former.
> >>
> >>
> >>
> >> I took the following from your Point-Stat config file:
> >>
> >> mask = {
> >>
> >> grid = [ "FULL" ];
> >>
> >> poly = [];
> >>
> >> sid = ["hira_compare_obs_tt_t44b_met.stations"];
> >>
> >> }
> >>
> >>
> >>
> >> You are telling Point-Stat to compute statistics using two
masking
> >> regions.
> >>
> >>
> >>
> >> First, with grid = "FULL" you're telling Point-Stat to use all
> >> stations falling inside you FULL model domain. Looking at the
log
> >> output you sent, that results in 33863 matched pairs.
> >>
> >>
> >>
> >> The second masking region is defined by the list of stations ID's
> >> in the file named "hira_compare_obs_tt_t44b_met.stations".
Looking
> >> at the log output you sent, that results in *** 0 *** matched
pairs:
> >>
> >> DEBUG 3: Number of matched pairs = 0
> >>
> >>
> >>
> >> So it makes complete sense that you're getting the same results
> >> with/without mask.sid since mask.sid isn't producing any matched
> >> pairs. I suspect that you're getting confused by this line in
the
> >> log
> output:
> >>
> >> DEBUG 3: Rejected: SID exclusion = 0
> >>
> >>
> >>
> >> That line is telling you the number of observations discarded
> >> because of the "sid_exc" setting... which totally separate from
"mask.sid".
> >> That's a list of stations you always want to ignore because you
> >> know they contain bad data. You're not using "sid_exc" so that
log
> >> message should always be
> >>
> >> 0 in your output.
> >>
> >>
> >>
> >> The real question here is why are you getting 0 matched pairs
from
> >> your "mask.sid" masking region?
> >>
> >>
> >>
> >> I looked at an example of using the "mask.sid" option from our
> >> nightly build at verbosity level 4 and saw the following lines in
> >> the
> log output:
> >>
> >>
> >>
> >> DEBUG 3: Processing sid mask: /d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk/test/config/SID_CO.txt
> >>
> >> DEBUG 4: parse_sid_mask() -> parsing station ID masking file
> >> "/d1/johnhg/MET/MET_development/ svn-met-dev.cgd.ucar.edu/trunk
> >> /test/config/SID_CO.txt"
> >>
> >> DEBUG 4: parse_sid_mask() -> parsed 55 station ID's for the
"SID_CO"
> >> mask from file "/d1/johnhg/MET/MET_development/
> >> svn-met-dev.cgd.ucar.edu/trunk /test/config/SID_CO.txt"
> >>
> >>
> >>
> >> Verbosity level 4 will tell you how many stations were parsed
from
> >> your file. I'm pretty sure yours will say 0.
> >>
> >>
> >>
> >> I did some testing and think that what's missing is a newline at
> >> the end of your list of stations. If you add a newline, I'm
> >> guessing that'll fix it.
> >>
> >>
> >>
> >> I'll look in the user's guide and add the instruction to
terminate
> >> the line with a newline.
> >>
> >>
> >>
> >> Hope that does it.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> John
> >>
> >>
> >>
> >>
> >
>
>
>
>
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 13:08:46 2017
Bob,
Sorry about that... I lost that detail in the shuffle.
Thanks for posting that sample data file. I ran it through the
debugger
and see that indeed, the level value is still being set parsed as "-
2147483647 <%28214%29%20748-3647>".
So this is the same behavior from Matt's prior ticket:
http://mailman.ucar.edu/pipermail/met_help/2016-August/002638.html
Matt was looking at LAND and SKINT whose level types were 1. In your
case,
PRMSL has a level type of 101 and the level value is that very
negative
number.
Not sure what logic to add here. I can add level type 101 as another
"special" one where we automatically set the level values to 0. But I
took
a look at the list of possible level types:
http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-5.shtml
My guess is that many of those level types would result in that very
negative number (7 for topopause or 10 for the entire atmosphere).
Rather
than trying to list all possibilities in the code, another option
would be
to range check the level values. If they're outside of the expected
range,
I could reset them to 0.
Any thoughts or opinions?
Thanks,
John
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 14:00:46 2017
You lost me - your saying that L0 translates to a large negative
number in MET? Are you saying I should place that large negative
number in for 0 after L?
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 06, 2017 2:09 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
Sorry about that... I lost that detail in the shuffle.
Thanks for posting that sample data file. I ran it through the
debugger and see that indeed, the level value is still being set
parsed as "-
2147483647 <%28214%29%20748-3647>".
So this is the same behavior from Matt's prior ticket:
http://mailman.ucar.edu/pipermail/met_help/2016-August/002638.html
Matt was looking at LAND and SKINT whose level types were 1. In your
case, PRMSL has a level type of 101 and the level value is that very
negative number.
Not sure what logic to add here. I can add level type 101 as another
"special" one where we automatically set the level values to 0. But I
took a look at the list of possible level types:
http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-5.shtml
My guess is that many of those level types would result in that very
negative number (7 for topopause or 10 for the entire atmosphere).
Rather than trying to list all possibilities in the code, another
option would be to range check the level values. If they're outside
of the expected range, I could reset them to 0.
Any thoughts or opinions?
Thanks,
John
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 14:09:05 2017
Bob,
Close, but not quite. The library code in MET reads through the GRIB2
records and for each extracts three numbers. The level type and a
lower
and upper level value.
With Matt's data (LAND and SKINT) we found that when the level type is
1,
the level value in your GRIB2 files is stored as -2147483647. With
your
data (PRMSL), we found that the same is true for level type 101.
I'm guessing that very large negative number is basically a fill
value.
One simple option would be that when MET parses a level value from a
GRIB2
file, any value less than -9999 (which is MET's bad data value) would
just
be stored as 0.
Does that make sense? That'd be a one line fix.
John
On Mon, Mar 6, 2017 at 2:00 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> You lost me - your saying that L0 translates to a large negative
number in
> MET? Are you saying I should place that large negative number in for
0
> after L?
>
> Bob
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 2:09 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> Sorry about that... I lost that detail in the shuffle.
>
> Thanks for posting that sample data file. I ran it through the
debugger
> and see that indeed, the level value is still being set parsed as "-
> 2147483647 <%28214%29%20748-3647>".
>
> So this is the same behavior from Matt's prior ticket:
> http://mailman.ucar.edu/pipermail/met_help/2016-
August/002638.html
>
> Matt was looking at LAND and SKINT whose level types were 1. In
your
> case, PRMSL has a level type of 101 and the level value is that very
> negative number.
>
> Not sure what logic to add here. I can add level type 101 as
another
> "special" one where we automatically set the level values to 0. But
I took
> a look at the list of possible level types:
> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-5.shtml
>
> My guess is that many of those level types would result in that very
> negative number (7 for topopause or 10 for the entire atmosphere).
Rather
> than trying to list all possibilities in the code, another option
would be
> to range check the level values. If they're outside of the expected
range,
> I could reset them to 0.
>
> Any thoughts or opinions?
>
> Thanks,
> John
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 14:11:17 2017
Okay, yes I guess that would work.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 06, 2017 3:09 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
Close, but not quite. The library code in MET reads through the GRIB2
records and for each extracts three numbers. The level type and a
lower and upper level value.
With Matt's data (LAND and SKINT) we found that when the level type is
1, the level value in your GRIB2 files is stored as -2147483647. With
your data (PRMSL), we found that the same is true for level type 101.
I'm guessing that very large negative number is basically a fill
value.
One simple option would be that when MET parses a level value from a
GRIB2 file, any value less than -9999 (which is MET's bad data value)
would just be stored as 0.
Does that make sense? That'd be a one line fix.
John
On Mon, Mar 6, 2017 at 2:00 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> You lost me - your saying that L0 translates to a large negative
> number in MET? Are you saying I should place that large negative
> number in for 0 after L?
>
> Bob
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 2:09 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> Sorry about that... I lost that detail in the shuffle.
>
> Thanks for posting that sample data file. I ran it through the
> debugger and see that indeed, the level value is still being set
> parsed as "-
> 2147483647 <%28214%29%20748-3647>".
>
> So this is the same behavior from Matt's prior ticket:
> http://mailman.ucar.edu/pipermail/met_help/2016-
August/002638.html
>
> Matt was looking at LAND and SKINT whose level types were 1. In
your
> case, PRMSL has a level type of 101 and the level value is that very
> negative number.
>
> Not sure what logic to add here. I can add level type 101 as
another
> "special" one where we automatically set the level values to 0. But
I
> took a look at the list of possible level types:
> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-5.shtml
>
> My guess is that many of those level types would result in that very
> negative number (7 for topopause or 10 for the entire atmosphere).
> Rather than trying to list all possibilities in the code, another
> option would be to range check the level values. If they're outside
> of the expected range, I could reset them to 0.
>
> Any thoughts or opinions?
>
> Thanks,
> John
>
>
>
>
------------------------------------------------
Subject: Point Stat Obs Not Filtering
From: John Halley Gotway
Time: Mon Mar 06 15:35:49 2017
Bob,
OK, I added an error check to the development version of MET. Any
level
values less that -1.0e9 are just reset to a value of 0.
I spoke with Randy about this and -2147483647 <%28214%29%20748-3647>
is a
pretty special number. It usually indicates a problem between signed
and
unsigned integers. Whomever is creating your GRIB files may have
intended
to write -1 but wrote is as signed or unsigned... which then led to
this
behavior.
How do you want me to get this change to you?
You can of course just specify the record number right now:
'name="PRMSL";
level="R86";'
Do you need me to post a 5.2 bugfix for this or is updating in version
6.0
sufficient?
Thanks,
John
On Mon, Mar 6, 2017 at 2:11 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> Okay, yes I guess that would work.
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 3:09 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> Close, but not quite. The library code in MET reads through the
GRIB2
> records and for each extracts three numbers. The level type and a
lower
> and upper level value.
>
> With Matt's data (LAND and SKINT) we found that when the level type
is 1,
> the level value in your GRIB2 files is stored as -2147483647. With
your
> data (PRMSL), we found that the same is true for level type 101.
>
> I'm guessing that very large negative number is basically a fill
value.
>
> One simple option would be that when MET parses a level value from a
GRIB2
> file, any value less than -9999 (which is MET's bad data value)
would just
> be stored as 0.
>
> Does that make sense? That'd be a one line fix.
>
> John
>
>
>
> On Mon, Mar 6, 2017 at 2:00 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
> >
> > You lost me - your saying that L0 translates to a large negative
> > number in MET? Are you saying I should place that large negative
> > number in for 0 after L?
> >
> > Bob
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 06, 2017 2:09 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > <robert.craig.2 at us.af.mil>
> > Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
> >
> > Bob,
> >
> > Sorry about that... I lost that detail in the shuffle.
> >
> > Thanks for posting that sample data file. I ran it through the
> > debugger and see that indeed, the level value is still being set
> > parsed as "-
> > 2147483647 <%28214%29%20748-3647>".
> >
> > So this is the same behavior from Matt's prior ticket:
> > http://mailman.ucar.edu/pipermail/met_help/2016-
August/002638.html
> >
> > Matt was looking at LAND and SKINT whose level types were 1. In
your
> > case, PRMSL has a level type of 101 and the level value is that
very
> > negative number.
> >
> > Not sure what logic to add here. I can add level type 101 as
another
> > "special" one where we automatically set the level values to 0.
But I
> > took a look at the list of possible level types:
> > http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-
5.shtml
> >
> > My guess is that many of those level types would result in that
very
> > negative number (7 for topopause or 10 for the entire atmosphere).
> > Rather than trying to list all possibilities in the code, another
> > option would be to range check the level values. If they're
outside
> > of the expected range, I could reset them to 0.
> >
> > Any thoughts or opinions?
> >
> > Thanks,
> > John
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
From: robert.craig.2 at us.af.mil
Time: Mon Mar 06 15:40:05 2017
Since you are planning to deliver 6.0 shortly, I will use the work
around and wait.
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, March 06, 2017 4:36 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
Bob,
OK, I added an error check to the development version of MET. Any
level values less that -1.0e9 are just reset to a value of 0.
I spoke with Randy about this and -2147483647 <%28214%29%20748-3647>
is a pretty special number. It usually indicates a problem between
signed and unsigned integers. Whomever is creating your GRIB files
may have intended to write -1 but wrote is as signed or unsigned...
which then led to this behavior.
How do you want me to get this change to you?
You can of course just specify the record number right now:
'name="PRMSL"; level="R86";'
Do you need me to post a 5.2 bugfix for this or is updating in version
6.0 sufficient?
Thanks,
John
On Mon, Mar 6, 2017 at 2:11 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
>
> Okay, yes I guess that would work.
>
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, March 06, 2017 3:09 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
>
> Bob,
>
> Close, but not quite. The library code in MET reads through the
GRIB2
> records and for each extracts three numbers. The level type and a
> lower and upper level value.
>
> With Matt's data (LAND and SKINT) we found that when the level type
is
> 1, the level value in your GRIB2 files is stored as -2147483647.
With
> your data (PRMSL), we found that the same is true for level type
101.
>
> I'm guessing that very large negative number is basically a fill
value.
>
> One simple option would be that when MET parses a level value from a
> GRIB2 file, any value less than -9999 (which is MET's bad data
value)
> would just be stored as 0.
>
> Does that make sense? That'd be a one line fix.
>
> John
>
>
>
> On Mon, Mar 6, 2017 at 2:00 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79721 >
> >
> > You lost me - your saying that L0 translates to a large negative
> > number in MET? Are you saying I should place that large negative
> > number in for 0 after L?
> >
> > Bob
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, March 06, 2017 2:09 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > <robert.craig.2 at us.af.mil>
> > Subject: Re: [rt.rap.ucar.edu #79721] Point Stat Obs Not Filtering
> >
> > Bob,
> >
> > Sorry about that... I lost that detail in the shuffle.
> >
> > Thanks for posting that sample data file. I ran it through the
> > debugger and see that indeed, the level value is still being set
> > parsed as "-
> > 2147483647 <%28214%29%20748-3647>".
> >
> > So this is the same behavior from Matt's prior ticket:
> >
> > http://mailman.ucar.edu/pipermail/met_help/2016-August/002638.html
> >
> > Matt was looking at LAND and SKINT whose level types were 1. In
> > your case, PRMSL has a level type of 101 and the level value is
that
> > very negative number.
> >
> > Not sure what logic to add here. I can add level type 101 as
> > another "special" one where we automatically set the level values
to
> > 0. But I took a look at the list of possible level types:
> > http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-
5.shtml
> >
> > My guess is that many of those level types would result in that
very
> > negative number (7 for topopause or 10 for the entire atmosphere).
> > Rather than trying to list all possibilities in the code, another
> > option would be to range check the level values. If they're
outside
> > of the expected range, I could reset them to 0.
> >
> > Any thoughts or opinions?
> >
> > Thanks,
> > John
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list