[Met_help] [rt.rap.ucar.edu #90130] History for grid-stat question
John Halley Gotway via RT
met_help at ucar.edu
Fri May 10 13:46:43 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I am using MET "series_analysis" tool to compare Model forecast against another model/observation and using continuous statistics like RMSE, Bias etc with 1 month of data to check how the model of interest errors are "spatially" distributed. I am wondering what I should do if I would like to :
1. see this statistics in terms of temporally, over a domain of interest ? Netcdf output from "series_analysis" averaged them over the domain, but is there a way to store them temporally, so a time series can be drawn ? MODE tool used for temporal information, but wondering if that can be done in Grid-stat as well ?
2. Also, at any grid point how do I get the variable values (in this case Aerosol optical depth form both forecast and observations) to construct any correlation ? I am using Pearson correlation coefficient in series_analysis, but does it store AOD values at each grid point for analysis length that can be used to draw correlation on my own ?
Hope I make this clear, let me know if you have questions.
Thanks for your help
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: grid-stat question
From: John Halley Gotway
Time: Wed May 08 12:12:31 2019
Partha,
I see that you're running the Series-Analysis tool to compare some
model and/or observation data over a 1 month time period. And you'd
like to see summary statistics averaged over one or more spatial areas
for each time during that month.
Yes, you would definitely use the Grid-Stat tool to do this. It's
funny, users often start with Grid-Stat to compute summary statistics
and then move on to Series-Analysis to see how model performance
varies over the domain. You're going in the opposite direction.
By way of example, let's say your time series has length 30. You're
running Series-Analysis once and passing in a list of 30 forecast
files and 30 corresponding observation files. You should now run
Grid-Stat 30 times, once for each of those 30 time steps. Each Grid-
Stat run will create a .stat output file which contains all the
statistics you request in the config file. You could then use the
Stat-Analysis tool to summarize those results if you'd like. Or you
could load them into METviewer and create some nice time series plots
of those statistics.
When you configure Grid-Stat, you could request that it write out the
NetCDF matched pairs output file. That contains the individual
forecast/observation pair values. Unfortunately, to extract a time
series, you'd need to pull values from the 30 different NetCDF output
files. If this is something you need to do routinely, perhaps we
could find a better solution than that?
Hope this helps point you in the right direction.
Thanks,
John
------------------------------------------------
Subject: grid-stat question
From: Partha Bhattacharjee - NOAA Affiliate
Time: Wed May 08 14:15:40 2019
Thanks John. I am trying to use grid-stat to read two model output but
getting error like " mask_bad_data() -> grid dimensions do not
match".
But same data works fine with "series_analysis" without any problem. I
put
data in ftp along with config file and .sh that is used to run MET.
Thanks
in advance.
On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Partha,
>
> I see that you're running the Series-Analysis tool to compare some
model
> and/or observation data over a 1 month time period. And you'd like
to see
> summary statistics averaged over one or more spatial areas for each
time
> during that month.
>
> Yes, you would definitely use the Grid-Stat tool to do this. It's
funny,
> users often start with Grid-Stat to compute summary statistics and
then
> move on to Series-Analysis to see how model performance varies over
the
> domain. You're going in the opposite direction.
>
> By way of example, let's say your time series has length 30. You're
> running Series-Analysis once and passing in a list of 30 forecast
files and
> 30 corresponding observation files. You should now run Grid-Stat 30
times,
> once for each of those 30 time steps. Each Grid-Stat run will
create a
> .stat output file which contains all the statistics you request in
the
> config file. You could then use the Stat-Analysis tool to summarize
those
> results if you'd like. Or you could load them into METviewer and
create
> some nice time series plots of those statistics.
>
> When you configure Grid-Stat, you could request that it write out
the
> NetCDF matched pairs output file. That contains the individual
> forecast/observation pair values. Unfortunately, to extract a time
series,
> you'd need to pull values from the 30 different NetCDF output files.
If
> this is something you need to do routinely, perhaps we could find a
better
> solution than that?
>
> Hope this helps point you in the right direction.
>
> Thanks,
> John
>
>
>
--
Partha S. Bhattacharjee, Ph.D.
I.M. Systems Group
Environmental Modeling Centre
NCWCP
5830 University Research Court.
College Park, MD 20740
------------------------------------------------
Subject: grid-stat question
From: John Halley Gotway
Time: Wed May 08 15:15:46 2019
Partha,
I went to go debug this issue but I think you made a mistake in
uploading
your data to the ftp site. The following file is supposed to be the
Grid-Stat config file you used:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
However, it actually contains a copy of the shell script you're
running.
Can you please send me or repost the Grid-Stat config file?
Thanks,
John
On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
>
> Thanks John. I am trying to use grid-stat to read two model output
but
> getting error like " mask_bad_data() -> grid dimensions do not
match".
> But same data works fine with "series_analysis" without any problem.
I put
> data in ftp along with config file and .sh that is used to run MET.
Thanks
> in advance.
>
>
>
> On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Partha,
> >
> > I see that you're running the Series-Analysis tool to compare some
model
> > and/or observation data over a 1 month time period. And you'd
like to
> see
> > summary statistics averaged over one or more spatial areas for
each time
> > during that month.
> >
> > Yes, you would definitely use the Grid-Stat tool to do this. It's
funny,
> > users often start with Grid-Stat to compute summary statistics and
then
> > move on to Series-Analysis to see how model performance varies
over the
> > domain. You're going in the opposite direction.
> >
> > By way of example, let's say your time series has length 30.
You're
> > running Series-Analysis once and passing in a list of 30 forecast
files
> and
> > 30 corresponding observation files. You should now run Grid-Stat
30
> times,
> > once for each of those 30 time steps. Each Grid-Stat run will
create a
> > .stat output file which contains all the statistics you request in
the
> > config file. You could then use the Stat-Analysis tool to
summarize
> those
> > results if you'd like. Or you could load them into METviewer and
create
> > some nice time series plots of those statistics.
> >
> > When you configure Grid-Stat, you could request that it write out
the
> > NetCDF matched pairs output file. That contains the individual
> > forecast/observation pair values. Unfortunately, to extract a
time
> series,
> > you'd need to pull values from the 30 different NetCDF output
files. If
> > this is something you need to do routinely, perhaps we could find
a
> better
> > solution than that?
> >
> > Hope this helps point you in the right direction.
> >
> > Thanks,
> > John
> >
> >
> >
>
> --
> Partha S. Bhattacharjee, Ph.D.
> I.M. Systems Group
> Environmental Modeling Centre
> NCWCP
> 5830 University Research Court.
> College Park, MD 20740
>
>
------------------------------------------------
Subject: grid-stat question
From: Partha Bhattacharjee - NOAA Affiliate
Time: Wed May 08 16:53:01 2019
John,
Config file should be there now, let me know if you not able to find
it
On Wed, May 8, 2019 at 5:15 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Partha,
>
> I went to go debug this issue but I think you made a mistake in
uploading
> your data to the ftp site. The following file is supposed to be the
> Grid-Stat config file you used:
>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
>
> However, it actually contains a copy of the shell script you're
running.
> Can you please send me or repost the Grid-Stat config file?
>
> Thanks,
> John
>
> On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA Affiliate
via RT
> <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> >
> > Thanks John. I am trying to use grid-stat to read two model output
but
> > getting error like " mask_bad_data() -> grid dimensions do not
match".
> > But same data works fine with "series_analysis" without any
problem. I
> put
> > data in ftp along with config file and .sh that is used to run
MET.
> Thanks
> > in advance.
> >
> >
> >
> > On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Partha,
> > >
> > > I see that you're running the Series-Analysis tool to compare
some
> model
> > > and/or observation data over a 1 month time period. And you'd
like to
> > see
> > > summary statistics averaged over one or more spatial areas for
each
> time
> > > during that month.
> > >
> > > Yes, you would definitely use the Grid-Stat tool to do this.
It's
> funny,
> > > users often start with Grid-Stat to compute summary statistics
and then
> > > move on to Series-Analysis to see how model performance varies
over the
> > > domain. You're going in the opposite direction.
> > >
> > > By way of example, let's say your time series has length 30.
You're
> > > running Series-Analysis once and passing in a list of 30
forecast files
> > and
> > > 30 corresponding observation files. You should now run Grid-
Stat 30
> > times,
> > > once for each of those 30 time steps. Each Grid-Stat run will
create a
> > > .stat output file which contains all the statistics you request
in the
> > > config file. You could then use the Stat-Analysis tool to
summarize
> > those
> > > results if you'd like. Or you could load them into METviewer
and
> create
> > > some nice time series plots of those statistics.
> > >
> > > When you configure Grid-Stat, you could request that it write
out the
> > > NetCDF matched pairs output file. That contains the individual
> > > forecast/observation pair values. Unfortunately, to extract a
time
> > series,
> > > you'd need to pull values from the 30 different NetCDF output
files.
> If
> > > this is something you need to do routinely, perhaps we could
find a
> > better
> > > solution than that?
> > >
> > > Hope this helps point you in the right direction.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> >
> > --
> > Partha S. Bhattacharjee, Ph.D.
> > I.M. Systems Group
> > Environmental Modeling Centre
> > NCWCP
> > 5830 University Research Court.
> > College Park, MD 20740
> >
> >
>
>
--
Partha S. Bhattacharjee, Ph.D.
I.M. Systems Group
Environmental Modeling Centre
NCWCP
5830 University Research Court.
College Park, MD 20740
------------------------------------------------
Subject: grid-stat question
From: John Halley Gotway
Time: Fri May 10 12:58:35 2019
Partha,
Sorry for the delay and thanks for sending the config file. I was
able to
run your case and replicate the error message you're seeing:
ERROR : mask_bad_data() -> grid dimensions do not match
This is a problem in your config file in the "mask" dictionary and is
easy
to fix. Listed below is the mask section from the default Grid-Stat
config
file:
*mask = { grid = [ "FULL" ]; poly = [];}*
And here's what you have in yours:
*mask = { grid = ""; poly = "";};*
The grid and poly settings should be arrays with square brackets
around
it. And you must define at least one masking region to use. Please
just
swap in the default mask setting and it should work fine.
Thanks,
John
On Wed, May 8, 2019 at 4:53 PM Partha Bhattacharjee - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
>
> John,
> Config file should be there now, let me know if you not able to find
it
>
> On Wed, May 8, 2019 at 5:15 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Partha,
> >
> > I went to go debug this issue but I think you made a mistake in
uploading
> > your data to the ftp site. The following file is supposed to be
the
> > Grid-Stat config file you used:
> >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
> >
> > However, it actually contains a copy of the shell script you're
running.
> > Can you please send me or repost the Grid-Stat config file?
> >
> > Thanks,
> > John
> >
> > On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA
Affiliate via
> RT
> > <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> > >
> > > Thanks John. I am trying to use grid-stat to read two model
output but
> > > getting error like " mask_bad_data() -> grid dimensions do not
match".
> > > But same data works fine with "series_analysis" without any
problem. I
> > put
> > > data in ftp along with config file and .sh that is used to run
MET.
> > Thanks
> > > in advance.
> > >
> > >
> > >
> > > On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Partha,
> > > >
> > > > I see that you're running the Series-Analysis tool to compare
some
> > model
> > > > and/or observation data over a 1 month time period. And you'd
like
> to
> > > see
> > > > summary statistics averaged over one or more spatial areas for
each
> > time
> > > > during that month.
> > > >
> > > > Yes, you would definitely use the Grid-Stat tool to do this.
It's
> > funny,
> > > > users often start with Grid-Stat to compute summary statistics
and
> then
> > > > move on to Series-Analysis to see how model performance varies
over
> the
> > > > domain. You're going in the opposite direction.
> > > >
> > > > By way of example, let's say your time series has length 30.
You're
> > > > running Series-Analysis once and passing in a list of 30
forecast
> files
> > > and
> > > > 30 corresponding observation files. You should now run Grid-
Stat 30
> > > times,
> > > > once for each of those 30 time steps. Each Grid-Stat run will
> create a
> > > > .stat output file which contains all the statistics you
request in
> the
> > > > config file. You could then use the Stat-Analysis tool to
summarize
> > > those
> > > > results if you'd like. Or you could load them into METviewer
and
> > create
> > > > some nice time series plots of those statistics.
> > > >
> > > > When you configure Grid-Stat, you could request that it write
out the
> > > > NetCDF matched pairs output file. That contains the
individual
> > > > forecast/observation pair values. Unfortunately, to extract a
time
> > > series,
> > > > you'd need to pull values from the 30 different NetCDF output
files.
> > If
> > > > this is something you need to do routinely, perhaps we could
find a
> > > better
> > > > solution than that?
> > > >
> > > > Hope this helps point you in the right direction.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > >
> > > --
> > > Partha S. Bhattacharjee, Ph.D.
> > > I.M. Systems Group
> > > Environmental Modeling Centre
> > > NCWCP
> > > 5830 University Research Court.
> > > College Park, MD 20740
> > >
> > >
> >
> >
>
> --
> Partha S. Bhattacharjee, Ph.D.
> I.M. Systems Group
> Environmental Modeling Centre
> NCWCP
> 5830 University Research Court.
> College Park, MD 20740
>
>
------------------------------------------------
Subject: grid-stat question
From: Partha Bhattacharjee - NOAA Affiliate
Time: Fri May 10 13:08:34 2019
Thanks John. It did solve the problem. But now I have more than 50
NETCDF
output files generated through Grid-Stat that contain
observation/model
pair values for the time period. Is there any easy way to draw time
series
from these 50 files ?
Thanks
Partha
On Fri, May 10, 2019 at 2:58 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Partha,
>
> Sorry for the delay and thanks for sending the config file. I was
able to
> run your case and replicate the error message you're seeing:
> ERROR : mask_bad_data() -> grid dimensions do not match
>
> This is a problem in your config file in the "mask" dictionary and
is easy
> to fix. Listed below is the mask section from the default Grid-Stat
config
> file:
>
>
>
> *mask = { grid = [ "FULL" ]; poly = [];}*
>
> And here's what you have in yours:
>
>
>
> *mask = { grid = ""; poly = "";};*
>
> The grid and poly settings should be arrays with square brackets
around
> it. And you must define at least one masking region to use. Please
just
> swap in the default mask setting and it should work fine.
>
> Thanks,
> John
>
> On Wed, May 8, 2019 at 4:53 PM Partha Bhattacharjee - NOAA Affiliate
via RT
> <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> >
> > John,
> > Config file should be there now, let me know if you not able to
find it
> >
> > On Wed, May 8, 2019 at 5:15 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Partha,
> > >
> > > I went to go debug this issue but I think you made a mistake in
> uploading
> > > your data to the ftp site. The following file is supposed to be
the
> > > Grid-Stat config file you used:
> > >
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
> > >
> > > However, it actually contains a copy of the shell script you're
> running.
> > > Can you please send me or repost the Grid-Stat config file?
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA
Affiliate
> via
> > RT
> > > <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130
>
> > > >
> > > > Thanks John. I am trying to use grid-stat to read two model
output
> but
> > > > getting error like " mask_bad_data() -> grid dimensions do
not
> match".
> > > > But same data works fine with "series_analysis" without any
problem.
> I
> > > put
> > > > data in ftp along with config file and .sh that is used to run
MET.
> > > Thanks
> > > > in advance.
> > > >
> > > >
> > > >
> > > > On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Partha,
> > > > >
> > > > > I see that you're running the Series-Analysis tool to
compare some
> > > model
> > > > > and/or observation data over a 1 month time period. And
you'd like
> > to
> > > > see
> > > > > summary statistics averaged over one or more spatial areas
for each
> > > time
> > > > > during that month.
> > > > >
> > > > > Yes, you would definitely use the Grid-Stat tool to do this.
It's
> > > funny,
> > > > > users often start with Grid-Stat to compute summary
statistics and
> > then
> > > > > move on to Series-Analysis to see how model performance
varies over
> > the
> > > > > domain. You're going in the opposite direction.
> > > > >
> > > > > By way of example, let's say your time series has length 30.
> You're
> > > > > running Series-Analysis once and passing in a list of 30
forecast
> > files
> > > > and
> > > > > 30 corresponding observation files. You should now run
Grid-Stat
> 30
> > > > times,
> > > > > once for each of those 30 time steps. Each Grid-Stat run
will
> > create a
> > > > > .stat output file which contains all the statistics you
request in
> > the
> > > > > config file. You could then use the Stat-Analysis tool to
> summarize
> > > > those
> > > > > results if you'd like. Or you could load them into
METviewer and
> > > create
> > > > > some nice time series plots of those statistics.
> > > > >
> > > > > When you configure Grid-Stat, you could request that it
write out
> the
> > > > > NetCDF matched pairs output file. That contains the
individual
> > > > > forecast/observation pair values. Unfortunately, to extract
a time
> > > > series,
> > > > > you'd need to pull values from the 30 different NetCDF
output
> files.
> > > If
> > > > > this is something you need to do routinely, perhaps we could
find a
> > > > better
> > > > > solution than that?
> > > > >
> > > > > Hope this helps point you in the right direction.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Partha S. Bhattacharjee, Ph.D.
> > > > I.M. Systems Group
> > > > Environmental Modeling Centre
> > > > NCWCP
> > > > 5830 University Research Court.
> > > > College Park, MD 20740
> > > >
> > > >
> > >
> > >
> >
> > --
> > Partha S. Bhattacharjee, Ph.D.
> > I.M. Systems Group
> > Environmental Modeling Centre
> > NCWCP
> > 5830 University Research Court.
> > College Park, MD 20740
> >
> >
>
>
--
Partha S. Bhattacharjee, Ph.D.
I.M. Systems Group
Environmental Modeling Centre
NCWCP
5830 University Research Court.
College Park, MD 20740
------------------------------------------------
Subject: grid-stat question
From: John Halley Gotway
Time: Fri May 10 13:24:48 2019
Partha,
To be clear, you're interested, for example, in a single grid point.
And
you want to see a list of 50 forecast values and corresponding 50
observation values? Is that right?
If that's the case, I don't have a quick and easy solution for you.
I'd
suggest writing a python script to take as input a list of 50
filenames.
And for each file, grab the point or point(s) of interest and print
them to
the screen.
John
On Fri, May 10, 2019 at 1:09 PM Partha Bhattacharjee - NOAA Affiliate
via
RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
>
> Thanks John. It did solve the problem. But now I have more than 50
NETCDF
> output files generated through Grid-Stat that contain
observation/model
> pair values for the time period. Is there any easy way to draw time
series
> from these 50 files ?
> Thanks
> Partha
>
> On Fri, May 10, 2019 at 2:58 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Partha,
> >
> > Sorry for the delay and thanks for sending the config file. I was
able
> to
> > run your case and replicate the error message you're seeing:
> > ERROR : mask_bad_data() -> grid dimensions do not match
> >
> > This is a problem in your config file in the "mask" dictionary and
is
> easy
> > to fix. Listed below is the mask section from the default Grid-
Stat
> config
> > file:
> >
> >
> >
> > *mask = { grid = [ "FULL" ]; poly = [];}*
> >
> > And here's what you have in yours:
> >
> >
> >
> > *mask = { grid = ""; poly = "";};*
> >
> > The grid and poly settings should be arrays with square brackets
around
> > it. And you must define at least one masking region to use.
Please just
> > swap in the default mask setting and it should work fine.
> >
> > Thanks,
> > John
> >
> > On Wed, May 8, 2019 at 4:53 PM Partha Bhattacharjee - NOAA
Affiliate via
> RT
> > <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> > >
> > > John,
> > > Config file should be there now, let me know if you not able to
find it
> > >
> > > On Wed, May 8, 2019 at 5:15 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Partha,
> > > >
> > > > I went to go debug this issue but I think you made a mistake
in
> > uploading
> > > > your data to the ftp site. The following file is supposed to
be the
> > > > Grid-Stat config file you used:
> > > >
> > > >
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
> > > >
> > > > However, it actually contains a copy of the shell script
you're
> > running.
> > > > Can you please send me or repost the Grid-Stat config file?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA
Affiliate
> > via
> > > RT
> > > > <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> > > > >
> > > > > Thanks John. I am trying to use grid-stat to read two model
output
> > but
> > > > > getting error like " mask_bad_data() -> grid dimensions do
not
> > match".
> > > > > But same data works fine with "series_analysis" without any
> problem.
> > I
> > > > put
> > > > > data in ftp along with config file and .sh that is used to
run MET.
> > > > Thanks
> > > > > in advance.
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Partha,
> > > > > >
> > > > > > I see that you're running the Series-Analysis tool to
compare
> some
> > > > model
> > > > > > and/or observation data over a 1 month time period. And
you'd
> like
> > > to
> > > > > see
> > > > > > summary statistics averaged over one or more spatial areas
for
> each
> > > > time
> > > > > > during that month.
> > > > > >
> > > > > > Yes, you would definitely use the Grid-Stat tool to do
this.
> It's
> > > > funny,
> > > > > > users often start with Grid-Stat to compute summary
statistics
> and
> > > then
> > > > > > move on to Series-Analysis to see how model performance
varies
> over
> > > the
> > > > > > domain. You're going in the opposite direction.
> > > > > >
> > > > > > By way of example, let's say your time series has length
30.
> > You're
> > > > > > running Series-Analysis once and passing in a list of 30
forecast
> > > files
> > > > > and
> > > > > > 30 corresponding observation files. You should now run
Grid-Stat
> > 30
> > > > > times,
> > > > > > once for each of those 30 time steps. Each Grid-Stat run
will
> > > create a
> > > > > > .stat output file which contains all the statistics you
request
> in
> > > the
> > > > > > config file. You could then use the Stat-Analysis tool to
> > summarize
> > > > > those
> > > > > > results if you'd like. Or you could load them into
METviewer and
> > > > create
> > > > > > some nice time series plots of those statistics.
> > > > > >
> > > > > > When you configure Grid-Stat, you could request that it
write out
> > the
> > > > > > NetCDF matched pairs output file. That contains the
individual
> > > > > > forecast/observation pair values. Unfortunately, to
extract a
> time
> > > > > series,
> > > > > > you'd need to pull values from the 30 different NetCDF
output
> > files.
> > > > If
> > > > > > this is something you need to do routinely, perhaps we
could
> find a
> > > > > better
> > > > > > solution than that?
> > > > > >
> > > > > > Hope this helps point you in the right direction.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Partha S. Bhattacharjee, Ph.D.
> > > > > I.M. Systems Group
> > > > > Environmental Modeling Centre
> > > > > NCWCP
> > > > > 5830 University Research Court.
> > > > > College Park, MD 20740
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > Partha S. Bhattacharjee, Ph.D.
> > > I.M. Systems Group
> > > Environmental Modeling Centre
> > > NCWCP
> > > 5830 University Research Court.
> > > College Park, MD 20740
> > >
> > >
> >
> >
>
> --
> Partha S. Bhattacharjee, Ph.D.
> I.M. Systems Group
> Environmental Modeling Centre
> NCWCP
> 5830 University Research Court.
> College Park, MD 20740
>
>
------------------------------------------------
Subject: grid-stat question
From: Partha Bhattacharjee - NOAA Affiliate
Time: Fri May 10 13:45:11 2019
Thanks John.
On Fri, May 10, 2019 at 3:24 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Partha,
>
> To be clear, you're interested, for example, in a single grid point.
And
> you want to see a list of 50 forecast values and corresponding 50
> observation values? Is that right?
>
> If that's the case, I don't have a quick and easy solution for you.
I'd
> suggest writing a python script to take as input a list of 50
filenames.
> And for each file, grab the point or point(s) of interest and print
them to
> the screen.
>
> John
>
> On Fri, May 10, 2019 at 1:09 PM Partha Bhattacharjee - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> >
> > Thanks John. It did solve the problem. But now I have more than 50
NETCDF
> > output files generated through Grid-Stat that contain
observation/model
> > pair values for the time period. Is there any easy way to draw
time
> series
> > from these 50 files ?
> > Thanks
> > Partha
> >
> > On Fri, May 10, 2019 at 2:58 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Partha,
> > >
> > > Sorry for the delay and thanks for sending the config file. I
was able
> > to
> > > run your case and replicate the error message you're seeing:
> > > ERROR : mask_bad_data() -> grid dimensions do not match
> > >
> > > This is a problem in your config file in the "mask" dictionary
and is
> > easy
> > > to fix. Listed below is the mask section from the default Grid-
Stat
> > config
> > > file:
> > >
> > >
> > >
> > > *mask = { grid = [ "FULL" ]; poly = [];}*
> > >
> > > And here's what you have in yours:
> > >
> > >
> > >
> > > *mask = { grid = ""; poly = "";};*
> > >
> > > The grid and poly settings should be arrays with square brackets
around
> > > it. And you must define at least one masking region to use.
Please
> just
> > > swap in the default mask setting and it should work fine.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, May 8, 2019 at 4:53 PM Partha Bhattacharjee - NOAA
Affiliate
> via
> > RT
> > > <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130
>
> > > >
> > > > John,
> > > > Config file should be there now, let me know if you not able
to find
> it
> > > >
> > > > On Wed, May 8, 2019 at 5:15 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Partha,
> > > > >
> > > > > I went to go debug this issue but I think you made a mistake
in
> > > uploading
> > > > > your data to the ftp site. The following file is supposed
to be
> the
> > > > > Grid-Stat config file you used:
> > > > >
> > > > >
> > > >
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/Bhattacharjee/GridStatConfig_May19_fh00z
> > > > >
> > > > > However, it actually contains a copy of the shell script
you're
> > > running.
> > > > > Can you please send me or repost the Grid-Stat config file?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Wed, May 8, 2019 at 2:16 PM Partha Bhattacharjee - NOAA
> Affiliate
> > > via
> > > > RT
> > > > > <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90130 >
> > > > > >
> > > > > > Thanks John. I am trying to use grid-stat to read two
model
> output
> > > but
> > > > > > getting error like " mask_bad_data() -> grid dimensions
do not
> > > match".
> > > > > > But same data works fine with "series_analysis" without
any
> > problem.
> > > I
> > > > > put
> > > > > > data in ftp along with config file and .sh that is used to
run
> MET.
> > > > > Thanks
> > > > > > in advance.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, May 8, 2019 at 2:12 PM John Halley Gotway via RT <
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Partha,
> > > > > > >
> > > > > > > I see that you're running the Series-Analysis tool to
compare
> > some
> > > > > model
> > > > > > > and/or observation data over a 1 month time period. And
you'd
> > like
> > > > to
> > > > > > see
> > > > > > > summary statistics averaged over one or more spatial
areas for
> > each
> > > > > time
> > > > > > > during that month.
> > > > > > >
> > > > > > > Yes, you would definitely use the Grid-Stat tool to do
this.
> > It's
> > > > > funny,
> > > > > > > users often start with Grid-Stat to compute summary
statistics
> > and
> > > > then
> > > > > > > move on to Series-Analysis to see how model performance
varies
> > over
> > > > the
> > > > > > > domain. You're going in the opposite direction.
> > > > > > >
> > > > > > > By way of example, let's say your time series has length
30.
> > > You're
> > > > > > > running Series-Analysis once and passing in a list of 30
> forecast
> > > > files
> > > > > > and
> > > > > > > 30 corresponding observation files. You should now run
> Grid-Stat
> > > 30
> > > > > > times,
> > > > > > > once for each of those 30 time steps. Each Grid-Stat
run will
> > > > create a
> > > > > > > .stat output file which contains all the statistics you
request
> > in
> > > > the
> > > > > > > config file. You could then use the Stat-Analysis tool
to
> > > summarize
> > > > > > those
> > > > > > > results if you'd like. Or you could load them into
METviewer
> and
> > > > > create
> > > > > > > some nice time series plots of those statistics.
> > > > > > >
> > > > > > > When you configure Grid-Stat, you could request that it
write
> out
> > > the
> > > > > > > NetCDF matched pairs output file. That contains the
individual
> > > > > > > forecast/observation pair values. Unfortunately, to
extract a
> > time
> > > > > > series,
> > > > > > > you'd need to pull values from the 30 different NetCDF
output
> > > files.
> > > > > If
> > > > > > > this is something you need to do routinely, perhaps we
could
> > find a
> > > > > > better
> > > > > > > solution than that?
> > > > > > >
> > > > > > > Hope this helps point you in the right direction.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Partha S. Bhattacharjee, Ph.D.
> > > > > > I.M. Systems Group
> > > > > > Environmental Modeling Centre
> > > > > > NCWCP
> > > > > > 5830 University Research Court.
> > > > > > College Park, MD 20740
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Partha S. Bhattacharjee, Ph.D.
> > > > I.M. Systems Group
> > > > Environmental Modeling Centre
> > > > NCWCP
> > > > 5830 University Research Court.
> > > > College Park, MD 20740
> > > >
> > > >
> > >
> > >
> >
> > --
> > Partha S. Bhattacharjee, Ph.D.
> > I.M. Systems Group
> > Environmental Modeling Centre
> > NCWCP
> > 5830 University Research Court.
> > College Park, MD 20740
> >
> >
>
> --
Partha S. Bhattacharjee, Ph.D.
I.M. Systems Group
Environmental Modeling Centre
NCWCP
5830 University Research Court.
College Park, MD 20740
------------------------------------------------
More information about the Met_help
mailing list