[Met_help] [rt.rap.ucar.edu #75431] History for Fractional Skill Score for Probability Data
John Halley Gotway via RT
met_help at ucar.edu
Thu Mar 10 09:22:12 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, after our conversation, I decided to check the .nc files that are produced by grid stat for neighborhood methods. When I look in the file for precip > .01" (grid_stats_1_APCP etc.), looking in the section for obs values, the actual precip values are listed, instead of a 1 or a 0 representing that the precip value exceeded the threshold (1) or did not (0). When I look in the next section, the differences between the probability and precipitation, it looks like the differences are strictly the fcst probability in percent subtracted from the precip value in hundredths of an inch. I would expect to see the probability subtracted from the event occurrence (.i.e. .6 subtracted from 1 (occurrence) or 0 (nonoccurrence). I was assuming that MET was taking the observation values and assigning either a 1 for occurrence or 0 for non-occurrence. If my assumption is wrong, then I will have to change the precip obs file to be either a 1 or a 0 if precipitation reached the th!
reshold or not before I feed it into MET - not hard to do, just want confirmation that it is required.
I sent an grid_stat stat and .nc file along with a asci version of the nc file for your review.
Thanks
Bob Craig
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Fractional Skill Score for Probability Data
From: John Halley Gotway
Time: Tue Mar 08 10:46:50 2016
Bob,
There is not a problem here. The NetCDF matched pairs output file
that
Grid-Stat produces includes the raw forecast, observation, and
difference
fields for each combination of variable/level/masking region. The
data is
analogous to the matched pair (MPR) output line that Point-Stat
produces
which includes the raw observation value and the forecast value
interpolated to the observation location.
Grid-Stat does not write gridded fields of 0's and 1's for each
threshold
specified in the configuration file. We figured that would be of
limited
use and would waste a whole lot of disk space!
Rest assured though that Grid-Stat really is thresholding the data
you've
passed to it an applying the neighborhood methods.
Please let me know if there is something specific you're trying to
visualize, and I might be able to help you plot it.
We did at one point consider adding a configuration option to Grid-
Stat to
write out the fractional coverage fields computed during the
neighborhood
method computations. This came up because we wanted to plot them for
some
reason. But again, this would take a whole LOT of disk space so you'd
only
want to do it in testing mode rather than writing them routinely.
John
On Tue, Mar 8, 2016 at 10:27 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Tue Mar 08 10:27:01 2016: Request 75431 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Fractional Skill Score for Probability Data
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75431 >
>
>
> John, after our conversation, I decided to check the .nc files that
are
> produced by grid stat for neighborhood methods. When I look in the
file
> for precip > .01" (grid_stats_1_APCP etc.), looking in the section
for obs
> values, the actual precip values are listed, instead of a 1 or a 0
> representing that the precip value exceeded the threshold (1) or did
not
> (0). When I look in the next section, the differences between the
> probability and precipitation, it looks like the differences are
strictly
> the fcst probability in percent subtracted from the precip value in
> hundredths of an inch. I would expect to see the probability
subtracted
> from the event occurrence (.i.e. .6 subtracted from 1 (occurrence)
or 0
> (nonoccurrence). I was assuming that MET was taking the observation
values
> and assigning either a 1 for occurrence or 0 for non-occurrence.
If my
> assumption is wrong, then I will have to change the precip obs file
to be
> either a 1 or a 0 if precipitation reached the th!
> reshold or not before I feed it into MET - not hard to do, just
want
> confirmation that it is required.
>
> I sent an grid_stat stat and .nc file along with a asci version of
the nc
> file for your review.
>
> Thanks
> Bob Craig
>
>
>
>
>
------------------------------------------------
Subject: Fractional Skill Score for Probability Data
From: John Halley Gotway
Time: Tue Mar 08 11:06:40 2016
Bob,
I took a look at the Grid-Stat NetCDF matched pairs file you sent and
have
attached images from running them through plot_data_plane:
plot_data_plane
grid_stat_1_APCP_6_240000L_20160215_000000V_pairs.nc
fcst.ps 'name="FCST_PQP1_A06_FULL"; level="(*,*)";'
plot_data_plane
grid_stat_1_APCP_6_240000L_20160215_000000V_pairs.nc
obs.ps 'name="OBS_APCP_06_A6_FULL"; level="(*,*)";'
I'm curious as to why the observation data (obs.png) looks so funny.
The
gray areas represent missing data and I'm wondering how you ended up
with
the dataset. I thought you were just using StageIV, which shouldn't
have
such large areas of missing data.
Thanks,
John
On Tue, Mar 8, 2016 at 10:46 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Bob,
>
> There is not a problem here. The NetCDF matched pairs output file
that
> Grid-Stat produces includes the raw forecast, observation, and
difference
> fields for each combination of variable/level/masking region. The
data is
> analogous to the matched pair (MPR) output line that Point-Stat
produces
> which includes the raw observation value and the forecast value
> interpolated to the observation location.
>
> Grid-Stat does not write gridded fields of 0's and 1's for each
threshold
> specified in the configuration file. We figured that would be of
limited
> use and would waste a whole lot of disk space!
>
> Rest assured though that Grid-Stat really is thresholding the data
you've
> passed to it an applying the neighborhood methods.
>
> Please let me know if there is something specific you're trying to
> visualize, and I might be able to help you plot it.
>
> We did at one point consider adding a configuration option to Grid-
Stat to
> write out the fractional coverage fields computed during the
neighborhood
> method computations. This came up because we wanted to plot them
for some
> reason. But again, this would take a whole LOT of disk space so
you'd only
> want to do it in testing mode rather than writing them routinely.
>
> John
>
> On Tue, Mar 8, 2016 at 10:27 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Tue Mar 08 10:27:01 2016: Request 75431 was acted upon.
>> Transaction: Ticket created by robert.craig.2 at us.af.mil
>> Queue: met_help
>> Subject: Fractional Skill Score for Probability Data
>> Owner: Nobody
>> Requestors: robert.craig.2 at us.af.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75431 >
>>
>>
>> John, after our conversation, I decided to check the .nc files that
are
>> produced by grid stat for neighborhood methods. When I look in the
file
>> for precip > .01" (grid_stats_1_APCP etc.), looking in the section
for obs
>> values, the actual precip values are listed, instead of a 1 or a 0
>> representing that the precip value exceeded the threshold (1) or
did not
>> (0). When I look in the next section, the differences between the
>> probability and precipitation, it looks like the differences are
strictly
>> the fcst probability in percent subtracted from the precip value in
>> hundredths of an inch. I would expect to see the probability
subtracted
>> from the event occurrence (.i.e. .6 subtracted from 1 (occurrence)
or 0
>> (nonoccurrence). I was assuming that MET was taking the
observation values
>> and assigning either a 1 for occurrence or 0 for non-occurrence.
If my
>> assumption is wrong, then I will have to change the precip obs file
to be
>> either a 1 or a 0 if precipitation reached the th!
>> reshold or not before I feed it into MET - not hard to do, just
want
>> confirmation that it is required.
>>
>> I sent an grid_stat stat and .nc file along with a asci version of
the nc
>> file for your review.
>>
>> Thanks
>> Bob Craig
>>
>>
>>
>>
>>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #75431] Fractional Skill Score for Probability Data
From: robert.craig.2 at us.af.mil
Time: Tue Mar 08 11:28:28 2016
Well, that is a good question. Another guy wrote the routine to
retrieve Stage IV (1hrly data) and pass to pcpcombine. I will check
into that. Thanks for reassuring me the precip thresholding was
occurring under the hood in MET.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, March 08, 2016 12:07 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
Subject: Re: [rt.rap.ucar.edu #75431] Fractional Skill Score for
Probability Data
Bob,
I took a look at the Grid-Stat NetCDF matched pairs file you sent and
have attached images from running them through plot_data_plane:
plot_data_plane
grid_stat_1_APCP_6_240000L_20160215_000000V_pairs.nc
fcst.ps 'name="FCST_PQP1_A06_FULL"; level="(*,*)";'
plot_data_plane
grid_stat_1_APCP_6_240000L_20160215_000000V_pairs.nc
obs.ps 'name="OBS_APCP_06_A6_FULL"; level="(*,*)";'
I'm curious as to why the observation data (obs.png) looks so funny.
The gray areas represent missing data and I'm wondering how you ended
up with the dataset. I thought you were just using StageIV, which
shouldn't have such large areas of missing data.
Thanks,
John
On Tue, Mar 8, 2016 at 10:46 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Bob,
>
> There is not a problem here. The NetCDF matched pairs output file
> that Grid-Stat produces includes the raw forecast, observation, and
> difference fields for each combination of variable/level/masking
> region. The data is analogous to the matched pair (MPR) output line
> that Point-Stat produces which includes the raw observation value
and
> the forecast value interpolated to the observation location.
>
> Grid-Stat does not write gridded fields of 0's and 1's for each
> threshold specified in the configuration file. We figured that
would
> be of limited use and would waste a whole lot of disk space!
>
> Rest assured though that Grid-Stat really is thresholding the data
> you've passed to it an applying the neighborhood methods.
>
> Please let me know if there is something specific you're trying to
> visualize, and I might be able to help you plot it.
>
> We did at one point consider adding a configuration option to
> Grid-Stat to write out the fractional coverage fields computed
during
> the neighborhood method computations. This came up because we
wanted
> to plot them for some reason. But again, this would take a whole
LOT
> of disk space so you'd only want to do it in testing mode rather
than writing them routinely.
>
> John
>
> On Tue, Mar 8, 2016 at 10:27 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Tue Mar 08 10:27:01 2016: Request 75431 was acted upon.
>> Transaction: Ticket created by robert.craig.2 at us.af.mil
>> Queue: met_help
>> Subject: Fractional Skill Score for Probability Data
>> Owner: Nobody
>> Requestors: robert.craig.2 at us.af.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=75431
>> >
>>
>>
>> John, after our conversation, I decided to check the .nc files that
>> are produced by grid stat for neighborhood methods. When I look in
>> the file for precip > .01" (grid_stats_1_APCP etc.), looking in the
>> section for obs values, the actual precip values are listed,
instead
>> of a 1 or a 0 representing that the precip value exceeded the
>> threshold (1) or did not (0). When I look in the next section, the
>> differences between the probability and precipitation, it looks
like
>> the differences are strictly the fcst probability in percent
subtracted from the precip value in
>> hundredths of an inch. I would expect to see the probability
subtracted
>> from the event occurrence (.i.e. .6 subtracted from 1 (occurrence)
or
>> 0 (nonoccurrence). I was assuming that MET was taking the
observation values
>> and assigning either a 1 for occurrence or 0 for non-occurrence.
If my
>> assumption is wrong, then I will have to change the precip obs file
>> to be either a 1 or a 0 if precipitation reached the th!
>> reshold or not before I feed it into MET - not hard to do, just
want
>> confirmation that it is required.
>>
>> I sent an grid_stat stat and .nc file along with a asci version of
>> the nc file for your review.
>>
>> Thanks
>> Bob Craig
>>
>>
>>
>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list