[Met_help] [rt.rap.ucar.edu #100090] History for 2 questions regarding how to generate specific stats
George McCabe via RT
met_help at ucar.edu
Thu Jul 1 08:10:20 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi met support,
I have 2 questions regarding how to output specific statistics for my experiment (see files in my attached tarball). I have looked through the online documentation, but I have had a hard time finding a direct answer/solution – perhaps I’ve simply missed something obvious.
QUESTION 1: how to output wind/vector (VL1L2, VCNT) statistics. I have added:
GRID_STAT_OUTPUT_FLAG_CNT = BOTH
GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
… to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file. I have also added:
is_u_wind : True
is_grid_relative: False
… attributes to my PYTHON_NUMPY object through python embedding. However, when I look at the output *_vl1l2.txt files, the only information is the header lines. I have included 2 sample outputs from my embedding scripts when called alone (./scripts/read_*2xr_v2.example.log) which show all the data is as expected. It seems like something else needs to be activated somewhere in the conf files, but I can’t seem to find the correct switch. If you have any ideas for other things to try, please let me know.
QUESTION 2: is there a way to use stat_analysis to recompute/aggregate over my *_pairs.nc files containing this information (example ncdump in RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-PL_000000L_20191223_000000V_pairs.ncdump) and output CNT stats for the total of all the matched pair data? I am aware of the “-job summary -line_type CNT -column RMSE -by FCST_VAR,FCST_LEV -dump_row” but this looks like it only provides basic statistics of all the RMSE values in each *_cnt.txt files, correct? What I really want is to generate actual CNT data from all matched pairs over all times. I’ve tried various combinations of aggregate and aggregate_stat but can’t seem to find a combination that will work. Is this possible? If so, what should my stat_analysis job command look like?
Thanks so much for your time in advance. I have asked a lot of questions lately, but I think I’m almost there. Any help you can provide on filling in my knowledge gaps would be very much appreciated.
Thanks,
Scott
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: Minna Win
Time: Thu Jun 03 13:48:59 2021
Hi Scott,
It looks like you've set your MET config file to correctly identify
which
linetypes you'd like to see. I'm not sure why you are essentially
getting
empty output files. I'll need to find someone who can answer your
python
embedding and grid-stat questions.
Regards,
Minna
---------------
Minna Win
Pronouns: she/her
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> Thu Jun 03 12:04:14 2021: Request 100090 was acted upon.
> Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> Queue: met_help
> Subject: 2 questions regarding how to generate specific stats
> Owner: Nobody
> Requestors: scott.d.rabenhorst at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
>
> Hi met support,
>
> I have 2 questions regarding how to output specific statistics for
my
> experiment (see files in my attached tarball). I have looked through
the
> online documentation, but I have had a hard time finding a direct
> answer/solution – perhaps I’ve simply missed something obvious.
>
> QUESTION 1: how to output wind/vector (VL1L2, VCNT) statistics. I
have
> added:
>
> GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
>
> … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file.
I have
> also added:
>
> is_u_wind : True
> is_grid_relative: False
>
> … attributes to my PYTHON_NUMPY object through python embedding.
However,
> when I look at the output *_vl1l2.txt files, the only information is
the
> header lines. I have included 2 sample outputs from my embedding
scripts
> when called alone (./scripts/read_*2xr_v2.example.log) which show
all the
> data is as expected. It seems like something else needs to be
activated
> somewhere in the conf files, but I can’t seem to find the correct
switch.
> If you have any ideas for other things to try, please let me know.
>
> QUESTION 2: is there a way to use stat_analysis to
recompute/aggregate
> over my *_pairs.nc files containing this information (example ncdump
in
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> and output CNT stats for the total of all the matched pair data? I
am aware
> of the “-job summary -line_type CNT -column RMSE -by
FCST_VAR,FCST_LEV
> -dump_row” but this looks like it only provides basic statistics of
all the
> RMSE values in each *_cnt.txt files, correct? What I really want is
to
> generate actual CNT data from all matched pairs over all times. I’ve
tried
> various combinations of aggregate and aggregate_stat but can’t seem
to find
> a combination that will work. Is this possible? If so, what should
my
> stat_analysis job command look like?
>
> Thanks so much for your time in advance. I have asked a lot of
questions
> lately, but I think I’m almost there. Any help you can provide on
filling
> in my knowledge gaps would be very much appreciated.
>
> Thanks,
> Scott
>
>
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: John Halley Gotway
Time: Thu Jun 03 16:21:51 2021
Hi Scott,
Minna asked me to take a look at your questions.
For the second question, the answer is no. Stat-Analysis reads the
.stat
output files created by the MET tools. It does not read the gridded
NetCDF
matched pairs files generated by the Grid-Stat tool.
But I do have two recommendations to make.
1. Grid-Stat computes statistics and partial sums that represent a
spatial
summary. You can configure Grid-Stat to compute its output over
whatever
spatial masking regions you'd like. Stat-Analysis can read/aggregate
those
spatial summaries through time. The SL1L2 line type contains scalar L1
and
L2 norms. Those are written because they allow you to easily aggregate
stats over multiple runs. Many (but not all) of the continuous
statistics
(like ME, RMSE, MAE, ACC, and so on) can be derived from the SL1L2
line
type. So I'd recommend trying this type of stat_analysis job:
-job aggregate_stat -line_type SL1L2 -out_line_type CNT -by
FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
For each unique combination of variable, level, and masking region,
it'll
aggregate all the SL1L2 lines it finds in the input and report the
corresponding continuous statistics.
2. Another option is running the NetCDF matched pairs produced by
Grid-Stat
through the Series-Analysis tool. That'll compute stats through time
separately for each individual grid point. The result with be a map of
statistics rather than a spatial summary.
For the first question, I see that you're using python embedding and
are
trying to compute vector partial sums output. In general, when you
request
an output line type, Grid-Stat will create the corresponding output
file.
But if it's empty, that means that it didn't have any input data
suitable
for creating that output type. So you've requested vector stats, but
Grid-Stat isn't computing any.
Thanks for sending the sample output. Looking in:
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
I do see several warning that you should take a look at:
WARNING: python_dataplane() -> an error occurred importing module
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
But I do also see that Grid-Stat is processing the U and V data and
reporting output as scalar variables in the CNT.txt file:
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
Now I haven't tested this on your data to confirm that it'll actually
work,
but we want to set these flags in the MET config file:
is_u_wind = true
is_v_wind = true
https://met.readthedocs.io/en/latest/Users_Guide/config_options.html?highlight=is_u_wind#settings-
common-to-multiple-tools
Checking in
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
see that var 4 is for U and var 5 is for V.
Add those via the METplus config file by adding:
FCST_VAR4_OPTIONS = is_u_wind = TRUE;
OBS_VAR4_OPTIONS = is_u_wind = TRUE;
FCST_VAR5_OPTIONS = is_v_wind = TRUE;
OBS_VAR5_OPTIONS = is_v_wind = TRUE;
Please give that a shot and let me know how it goes. The goal is NOT
having
any U and V show up in the CNT output file... and instead have entries
show
up in the VCNT output.
One last detail... if you do want to aggregate results through time,
I'd
recommend writing the SL1L2 and VL1L2 line types to enable that
aggregation
to be done later.
Thanks,
John
On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi Scott,
>
> It looks like you've set your MET config file to correctly identify
which
> linetypes you'd like to see. I'm not sure why you are essentially
getting
> empty output files. I'll need to find someone who can answer your
python
> embedding and grid-stat questions.
>
> Regards,
> Minna
> ---------------
> Minna Win
> Pronouns: she/her
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
>
>
>
> On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > Thu Jun 03 12:04:14 2021: Request 100090 was acted upon.
> > Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> > Queue: met_help
> > Subject: 2 questions regarding how to generate specific stats
> > Owner: Nobody
> > Requestors: scott.d.rabenhorst at nasa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
> >
> >
> > Hi met support,
> >
> > I have 2 questions regarding how to output specific statistics for
my
> > experiment (see files in my attached tarball). I have looked
through the
> > online documentation, but I have had a hard time finding a direct
> > answer/solution – perhaps I’ve simply missed something obvious.
> >
> > QUESTION 1: how to output wind/vector (VL1L2, VCNT) statistics. I
have
> > added:
> >
> > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> >
> > … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
file. I
> have
> > also added:
> >
> > is_u_wind : True
> > is_grid_relative: False
> >
> > … attributes to my PYTHON_NUMPY object through python embedding.
However,
> > when I look at the output *_vl1l2.txt files, the only information
is the
> > header lines. I have included 2 sample outputs from my embedding
scripts
> > when called alone (./scripts/read_*2xr_v2.example.log) which show
all the
> > data is as expected. It seems like something else needs to be
activated
> > somewhere in the conf files, but I can’t seem to find the correct
switch.
> > If you have any ideas for other things to try, please let me know.
> >
> > QUESTION 2: is there a way to use stat_analysis to
recompute/aggregate
> > over my *_pairs.nc files containing this information (example
ncdump in
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > and output CNT stats for the total of all the matched pair data? I
am
> aware
> > of the “-job summary -line_type CNT -column RMSE -by
FCST_VAR,FCST_LEV
> > -dump_row” but this looks like it only provides basic statistics
of all
> the
> > RMSE values in each *_cnt.txt files, correct? What I really want
is to
> > generate actual CNT data from all matched pairs over all times.
I’ve
> tried
> > various combinations of aggregate and aggregate_stat but can’t
seem to
> find
> > a combination that will work. Is this possible? If so, what should
my
> > stat_analysis job command look like?
> >
> > Thanks so much for your time in advance. I have asked a lot of
questions
> > lately, but I think I’m almost there. Any help you can provide on
filling
> > in my knowledge gaps would be very much appreciated.
> >
> > Thanks,
> > Scott
> >
> >
>
>
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Fri Jun 04 10:28:53 2021
Hi George,
Thanks for answering my questions below. So sorry for taking so much
of your time lately. From your description, it sounds like the best
option for my needs is to use Stat-Analysis to aggregate SL1L2/VL1L2
data. My goal is to generate single-value statistics for each
variable/level through all valid times and spatial areas. These will
be plugged into a Green's Function algorithm for our model.
However, the Series-Analysis is also an intriguing approach at least
for looking at spatial errors. I guess Grid-Stat could operate on that
data at the end to generate total stats? I have not tried using
Series-Analysis yet, but what has confused me is the documentation
requires separate "-fcst file_1...file_n" and "-obs file_1...file_n".
In most examples I have seen, these are independent files. However,
you make it sound as though it can run on my *_pairs.nc files
generated by Grid-Stat I presume as long as RAW data is in there? In
this case, I assume the list of files would be identical to both
forecast and observations? Are there any examples of using matched-
pairs for the Series-Analysis online?
Unfortunately, the larger problem right now is that I still can't get
Grid-Stat to process any my winds and output VCNT or VL1L2 data. I
tried your suggestion below and added the options to my
RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file (please see
updated attached tarball). I tried running with and without each of
these options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and
those ASCII files never contains any VL1L2/VCNT data, nor do the
*.stat files contain any of those line_types. I also toggled
outputting those attribute in my PYTHON_NUMPY attrs object, but that
didn't work either. I couldn't find any combinations that would work.
In an effort to narrow the problem, I copy/pasted the first references
to the fcst and obs fields in my logfile and ran my python read
scripts stand-alone and they produced the expected output with correct
values:
scripts/read_era5pl2xr_v2.example.log
scripts/read_g5fcst2xr_v2.example.log
Furthermore, I decided to test PlotDataPlane since I recalled you
saying that was used behind the scenes. It ran just fine and produced
these beautiful U 500hPa images without error:
PlotDataPlane_py-embed_test/plot_data_plane/*.ps
I have combed the log files for any errors or warning, but cannot find
any at all, or anything else that would indicate a processing error.
It is almost like there is some flag that has to be set in order to
trigger wind processing. Do you have any other ideas I can try?
Thanks,
Scott
On 6/3/21, 6:22 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
Minna asked me to take a look at your questions.
For the second question, the answer is no. Stat-Analysis reads the
.stat
output files created by the MET tools. It does not read the
gridded NetCDF
matched pairs files generated by the Grid-Stat tool.
But I do have two recommendations to make.
1. Grid-Stat computes statistics and partial sums that represent a
spatial
summary. You can configure Grid-Stat to compute its output over
whatever
spatial masking regions you'd like. Stat-Analysis can
read/aggregate those
spatial summaries through time. The SL1L2 line type contains
scalar L1 and
L2 norms. Those are written because they allow you to easily
aggregate
stats over multiple runs. Many (but not all) of the continuous
statistics
(like ME, RMSE, MAE, ACC, and so on) can be derived from the SL1L2
line
type. So I'd recommend trying this type of stat_analysis job:
-job aggregate_stat -line_type SL1L2 -out_line_type CNT -by
FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
For each unique combination of variable, level, and masking
region, it'll
aggregate all the SL1L2 lines it finds in the input and report the
corresponding continuous statistics.
2. Another option is running the NetCDF matched pairs produced by
Grid-Stat
through the Series-Analysis tool. That'll compute stats through
time
separately for each individual grid point. The result with be a
map of
statistics rather than a spatial summary.
For the first question, I see that you're using python embedding
and are
trying to compute vector partial sums output. In general, when you
request
an output line type, Grid-Stat will create the corresponding
output file.
But if it's empty, that means that it didn't have any input data
suitable
for creating that output type. So you've requested vector stats,
but
Grid-Stat isn't computing any.
Thanks for sending the sample output. Looking in:
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
I do see several warning that you should take a look at:
WARNING: python_dataplane() -> an error occurred importing module
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
But I do also see that Grid-Stat is processing the U and V data
and
reporting output as scalar variables in the CNT.txt file:
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
Now I haven't tested this on your data to confirm that it'll
actually work,
but we want to set these flags in the MET config file:
is_u_wind = true
is_v_wind = true
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoqT8V5Io58SreCFnbED1DKsfOT6OF11qIdOuvTZnDI%3D&reserved=0
Checking in
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
see that var 4 is for U and var 5 is for V.
Add those via the METplus config file by adding:
FCST_VAR4_OPTIONS = is_u_wind = TRUE;
OBS_VAR4_OPTIONS = is_u_wind = TRUE;
FCST_VAR5_OPTIONS = is_v_wind = TRUE;
OBS_VAR5_OPTIONS = is_v_wind = TRUE;
Please give that a shot and let me know how it goes. The goal is
NOT having
any U and V show up in the CNT output file... and instead have
entries show
up in the VCNT output.
One last detail... if you do want to aggregate results through
time, I'd
recommend writing the SL1L2 and VL1L2 line types to enable that
aggregation
to be done later.
Thanks,
John
On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT
<met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>
>
> Hi Scott,
>
> It looks like you've set your MET config file to correctly
identify which
> linetypes you'd like to see. I'm not sure why you are
essentially getting
> empty output files. I'll need to find someone who can answer
your python
> embedding and grid-stat questions.
>
> Regards,
> Minna
> ---------------
> Minna Win
> Pronouns: she/her
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
>
>
>
> On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > Thu Jun 03 12:04:14 2021: Request 100090 was acted upon.
> > Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> > Queue: met_help
> > Subject: 2 questions regarding how to generate specific
stats
> > Owner: Nobody
> > Requestors: scott.d.rabenhorst at nasa.gov
> > Status: new
> > Ticket <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>
> >
> >
> > Hi met support,
> >
> > I have 2 questions regarding how to output specific statistics
for my
> > experiment (see files in my attached tarball). I have looked
through the
> > online documentation, but I have had a hard time finding a
direct
> > answer/solution – perhaps I’ve simply missed something
obvious.
> >
> > QUESTION 1: how to output wind/vector (VL1L2, VCNT)
statistics. I have
> > added:
> >
> > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> >
> > … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
file. I
> have
> > also added:
> >
> > is_u_wind : True
> > is_grid_relative: False
> >
> > … attributes to my PYTHON_NUMPY object through python
embedding. However,
> > when I look at the output *_vl1l2.txt files, the only
information is the
> > header lines. I have included 2 sample outputs from my
embedding scripts
> > when called alone (./scripts/read_*2xr_v2.example.log) which
show all the
> > data is as expected. It seems like something else needs to be
activated
> > somewhere in the conf files, but I can’t seem to find the
correct switch.
> > If you have any ideas for other things to try, please let me
know.
> >
> > QUESTION 2: is there a way to use stat_analysis to
recompute/aggregate
> > over my *_pairs.nc files containing this information (example
ncdump in
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > and output CNT stats for the total of all the matched pair
data? I am
> aware
> > of the “-job summary -line_type CNT -column RMSE -by
FCST_VAR,FCST_LEV
> > -dump_row” but this looks like it only provides basic
statistics of all
> the
> > RMSE values in each *_cnt.txt files, correct? What I really
want is to
> > generate actual CNT data from all matched pairs over all
times. I’ve
> tried
> > various combinations of aggregate and aggregate_stat but can’t
seem to
> find
> > a combination that will work. Is this possible? If so, what
should my
> > stat_analysis job command look like?
> >
> > Thanks so much for your time in advance. I have asked a lot of
questions
> > lately, but I think I’m almost there. Any help you can provide
on filling
> > in my knowledge gaps would be very much appreciated.
> >
> > Thanks,
> > Scott
> >
> >
>
>
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100090] 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Mon Jun 07 09:08:52 2021
Hi George,
Just as a follow up thought to problem below regarding winds stats,
when I look through the CNT and SL1L2 statistics output, I see U and V
listed with all respective stats. So this issue is not a problem
reading U and V, it is a problem recognizing they are vector
components. Are your sure there is not another flag or option to set
either in the PTHON_NUMPY/attrs objects or another option in the
metplus conf file?
Thanks,
Scott
On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
Hi George,
Thanks for answering my questions below. So sorry for taking so
much of your time lately. From your description, it sounds like the
best option for my needs is to use Stat-Analysis to aggregate
SL1L2/VL1L2 data. My goal is to generate single-value statistics for
each variable/level through all valid times and spatial areas. These
will be plugged into a Green's Function algorithm for our model.
However, the Series-Analysis is also an intriguing approach at
least for looking at spatial errors. I guess Grid-Stat could operate
on that data at the end to generate total stats? I have not tried
using Series-Analysis yet, but what has confused me is the
documentation requires separate "-fcst file_1...file_n" and "-obs
file_1...file_n". In most examples I have seen, these are independent
files. However, you make it sound as though it can run on my
*_pairs.nc files generated by Grid-Stat I presume as long as RAW data
is in there? In this case, I assume the list of files would be
identical to both forecast and observations? Are there any examples of
using matched-pairs for the Series-Analysis online?
Unfortunately, the larger problem right now is that I still can't
get Grid-Stat to process any my winds and output VCNT or VL1L2 data. I
tried your suggestion below and added the options to my
RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file (please see
updated attached tarball). I tried running with and without each of
these options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and
those ASCII files never contains any VL1L2/VCNT data, nor do the
*.stat files contain any of those line_types. I also toggled
outputting those attribute in my PYTHON_NUMPY attrs object, but that
didn't work either. I couldn't find any combinations that would work.
In an effort to narrow the problem, I copy/pasted the first references
to the fcst and obs fields in my logfile and ran my python read
scripts stand-alone and they produced the expected output with correct
values:
scripts/read_era5pl2xr_v2.example.log
scripts/read_g5fcst2xr_v2.example.log
Furthermore, I decided to test PlotDataPlane since I recalled you
saying that was used behind the scenes. It ran just fine and produced
these beautiful U 500hPa images without error:
PlotDataPlane_py-embed_test/plot_data_plane/*.ps
I have combed the log files for any errors or warning, but cannot
find any at all, or anything else that would indicate a processing
error. It is almost like there is some flag that has to be set in
order to trigger wind processing. Do you have any other ideas I can
try?
Thanks,
Scott
On 6/3/21, 6:22 PM, "John Halley Gotway via RT"
<met_help at ucar.edu> wrote:
Hi Scott,
Minna asked me to take a look at your questions.
For the second question, the answer is no. Stat-Analysis reads
the .stat
output files created by the MET tools. It does not read the
gridded NetCDF
matched pairs files generated by the Grid-Stat tool.
But I do have two recommendations to make.
1. Grid-Stat computes statistics and partial sums that
represent a spatial
summary. You can configure Grid-Stat to compute its output
over whatever
spatial masking regions you'd like. Stat-Analysis can
read/aggregate those
spatial summaries through time. The SL1L2 line type contains
scalar L1 and
L2 norms. Those are written because they allow you to easily
aggregate
stats over multiple runs. Many (but not all) of the continuous
statistics
(like ME, RMSE, MAE, ACC, and so on) can be derived from the
SL1L2 line
type. So I'd recommend trying this type of stat_analysis job:
-job aggregate_stat -line_type SL1L2 -out_line_type CNT -by
FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
For each unique combination of variable, level, and masking
region, it'll
aggregate all the SL1L2 lines it finds in the input and report
the
corresponding continuous statistics.
2. Another option is running the NetCDF matched pairs produced
by Grid-Stat
through the Series-Analysis tool. That'll compute stats
through time
separately for each individual grid point. The result with be
a map of
statistics rather than a spatial summary.
For the first question, I see that you're using python
embedding and are
trying to compute vector partial sums output. In general, when
you request
an output line type, Grid-Stat will create the corresponding
output file.
But if it's empty, that means that it didn't have any input
data suitable
for creating that output type. So you've requested vector
stats, but
Grid-Stat isn't computing any.
Thanks for sending the sample output. Looking in:
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
I do see several warning that you should take a look at:
WARNING: python_dataplane() -> an error occurred importing
module
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
But I do also see that Grid-Stat is processing the U and V
data and
reporting output as scalar variables in the CNT.txt file:
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
Now I haven't tested this on your data to confirm that it'll
actually work,
but we want to set these flags in the MET config file:
is_u_wind = true
is_v_wind = true
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoqT8V5Io58SreCFnbED1DKsfOT6OF11qIdOuvTZnDI%3D&reserved=0
Checking in
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
see that var 4 is for U and var 5 is for V.
Add those via the METplus config file by adding:
FCST_VAR4_OPTIONS = is_u_wind = TRUE;
OBS_VAR4_OPTIONS = is_u_wind = TRUE;
FCST_VAR5_OPTIONS = is_v_wind = TRUE;
OBS_VAR5_OPTIONS = is_v_wind = TRUE;
Please give that a shot and let me know how it goes. The goal
is NOT having
any U and V show up in the CNT output file... and instead have
entries show
up in the VCNT output.
One last detail... if you do want to aggregate results through
time, I'd
recommend writing the SL1L2 and VL1L2 line types to enable
that aggregation
to be done later.
Thanks,
John
On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT
<met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>
>
> Hi Scott,
>
> It looks like you've set your MET config file to correctly
identify which
> linetypes you'd like to see. I'm not sure why you are
essentially getting
> empty output files. I'll need to find someone who can
answer your python
> embedding and grid-stat questions.
>
> Regards,
> Minna
> ---------------
> Minna Win
> Pronouns: she/her
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
> ---------------
>
>
>
> On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > Thu Jun 03 12:04:14 2021: Request 100090 was acted upon.
> > Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> > Queue: met_help
> > Subject: 2 questions regarding how to generate
specific stats
> > Owner: Nobody
> > Requestors: scott.d.rabenhorst at nasa.gov
> > Status: new
> > Ticket <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>
> >
> >
> > Hi met support,
> >
> > I have 2 questions regarding how to output specific
statistics for my
> > experiment (see files in my attached tarball). I have
looked through the
> > online documentation, but I have had a hard time finding a
direct
> > answer/solution – perhaps I’ve simply missed something
obvious.
> >
> > QUESTION 1: how to output wind/vector (VL1L2, VCNT)
statistics. I have
> > added:
> >
> > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> >
> > … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf file. I
> have
> > also added:
> >
> > is_u_wind : True
> > is_grid_relative: False
> >
> > … attributes to my PYTHON_NUMPY object through python
embedding. However,
> > when I look at the output *_vl1l2.txt files, the only
information is the
> > header lines. I have included 2 sample outputs from my
embedding scripts
> > when called alone (./scripts/read_*2xr_v2.example.log)
which show all the
> > data is as expected. It seems like something else needs to
be activated
> > somewhere in the conf files, but I can’t seem to find the
correct switch.
> > If you have any ideas for other things to try, please let
me know.
> >
> > QUESTION 2: is there a way to use stat_analysis to
recompute/aggregate
> > over my *_pairs.nc files containing this information
(example ncdump in
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > and output CNT stats for the total of all the matched pair
data? I am
> aware
> > of the “-job summary -line_type CNT -column RMSE -by
FCST_VAR,FCST_LEV
> > -dump_row” but this looks like it only provides basic
statistics of all
> the
> > RMSE values in each *_cnt.txt files, correct? What I
really want is to
> > generate actual CNT data from all matched pairs over all
times. I’ve
> tried
> > various combinations of aggregate and aggregate_stat but
can’t seem to
> find
> > a combination that will work. Is this possible? If so,
what should my
> > stat_analysis job command look like?
> >
> > Thanks so much for your time in advance. I have asked a
lot of questions
> > lately, but I think I’m almost there. Any help you can
provide on filling
> > in my knowledge gaps would be very much appreciated.
> >
> > Thanks,
> > Scott
> >
> >
>
>
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Wed Jun 16 13:07:56 2021
Hi Scott,
Regarding SeriesAnalysis -- there is an argument to the tool called "-
both"
that can be used to set both the fcst and obs lists to use the same
files.
This use case runs SeriesAnalysis on GridStat output:
https://metplus.readthedocs.io/en/v4.0.0/generated/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html?highlight=SeriesAnalysisUseCase%20%20GridStatToolUseCase
This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set the
input
file path and specifies the fcst and obs input fields separately:
FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
pairs.nc
Regarding not being able to produce wind output, I'll take a look at
your
files and see if I can narrow down the issue. I will report back with
more
information.
Thanks,
George
On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> Just as a follow up thought to problem below regarding winds stats,
when I
> look through the CNT and SL1L2 statistics output, I see U and V
listed with
> all respective stats. So this issue is not a problem reading U and
V, it is
> a problem recognizing they are vector components. Are your sure
there is
> not another flag or option to set either in the PTHON_NUMPY/attrs
objects
> or another option in the metplus conf file?
>
> Thanks,
> Scott
>
>
> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS
> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>
> Hi George,
>
> Thanks for answering my questions below. So sorry for taking so
much
> of your time lately. From your description, it sounds like the best
option
> for my needs is to use Stat-Analysis to aggregate SL1L2/VL1L2 data.
My goal
> is to generate single-value statistics for each variable/level
through all
> valid times and spatial areas. These will be plugged into a Green's
> Function algorithm for our model.
>
> However, the Series-Analysis is also an intriguing approach at
least
> for looking at spatial errors. I guess Grid-Stat could operate on
that data
> at the end to generate total stats? I have not tried using Series-
Analysis
> yet, but what has confused me is the documentation requires separate
"-fcst
> file_1...file_n" and "-obs file_1...file_n". In most examples I have
seen,
> these are independent files. However, you make it sound as though it
can
> run on my *_pairs.nc files generated by Grid-Stat I presume as long
as
> RAW data is in there? In this case, I assume the list of files would
be
> identical to both forecast and observations? Are there any examples
of
> using matched-pairs for the Series-Analysis online?
>
> Unfortunately, the larger problem right now is that I still
can't get
> Grid-Stat to process any my winds and output VCNT or VL1L2 data. I
tried
> your suggestion below and added the options to my
> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file (please see
> updated attached tarball). I tried running with and without each of
these
> options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and those
ASCII
> files never contains any VL1L2/VCNT data, nor do the *.stat files
contain
> any of those line_types. I also toggled outputting those attribute
in my
> PYTHON_NUMPY attrs object, but that didn't work either. I couldn't
find any
> combinations that would work. In an effort to narrow the problem, I
> copy/pasted the first references to the fcst and obs fields in my
logfile
> and ran my python read scripts stand-alone and they produced the
expected
> output with correct values:
> scripts/read_era5pl2xr_v2.example.log
> scripts/read_g5fcst2xr_v2.example.log
>
> Furthermore, I decided to test PlotDataPlane since I recalled
you
> saying that was used behind the scenes. It ran just fine and
produced these
> beautiful U 500hPa images without error:
> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
>
> I have combed the log files for any errors or warning, but
cannot find
> any at all, or anything else that would indicate a processing error.
It is
> almost like there is some flag that has to be set in order to
trigger wind
> processing. Do you have any other ideas I can try?
>
> Thanks,
> Scott
>
>
> On 6/3/21, 6:22 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Hi Scott,
>
> Minna asked me to take a look at your questions.
>
> For the second question, the answer is no. Stat-Analysis
reads the
> .stat
> output files created by the MET tools. It does not read the
> gridded NetCDF
> matched pairs files generated by the Grid-Stat tool.
> But I do have two recommendations to make.
>
> 1. Grid-Stat computes statistics and partial sums that
represent a
> spatial
> summary. You can configure Grid-Stat to compute its output
over
> whatever
> spatial masking regions you'd like. Stat-Analysis can
> read/aggregate those
> spatial summaries through time. The SL1L2 line type contains
> scalar L1 and
> L2 norms. Those are written because they allow you to easily
> aggregate
> stats over multiple runs. Many (but not all) of the
continuous
> statistics
> (like ME, RMSE, MAE, ACC, and so on) can be derived from the
SL1L2
> line
> type. So I'd recommend trying this type of stat_analysis
job:
>
> -job aggregate_stat -line_type SL1L2 -out_line_type CNT -by
> FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
>
> For each unique combination of variable, level, and masking
> region, it'll
> aggregate all the SL1L2 lines it finds in the input and
report the
> corresponding continuous statistics.
>
> 2. Another option is running the NetCDF matched pairs
produced by
> Grid-Stat
> through the Series-Analysis tool. That'll compute stats
through
> time
> separately for each individual grid point. The result with
be a
> map of
> statistics rather than a spatial summary.
>
> For the first question, I see that you're using python
embedding
> and are
> trying to compute vector partial sums output. In general,
when you
> request
> an output line type, Grid-Stat will create the corresponding
> output file.
> But if it's empty, that means that it didn't have any input
data
> suitable
> for creating that output type. So you've requested vector
stats,
> but
> Grid-Stat isn't computing any.
>
> Thanks for sending the sample output. Looking in:
>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
>
> I do see several warning that you should take a look at:
> WARNING: python_dataplane() -> an error occurred importing
module
>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
>
> But I do also see that Grid-Stat is processing the U and V
data and
> reporting output as scalar variables in the CNT.txt file:
>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
>
> Now I haven't tested this on your data to confirm that it'll
> actually work,
> but we want to set these flags in the MET config file:
> is_u_wind = true
> is_v_wind = true
>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoqT8V5Io58SreCFnbED1DKsfOT6OF11qIdOuvTZnDI%3D&reserved=0
>
> Checking in
> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> see that var 4 is for U and var 5 is for V.
> Add those via the METplus config file by adding:
> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
>
> Please give that a shot and let me know how it goes. The
goal is
> NOT having
> any U and V show up in the CNT output file... and instead
have
> entries show
> up in the VCNT output.
>
> One last detail... if you do want to aggregate results
through
> time, I'd
> recommend writing the SL1L2 and VL1L2 line types to enable
that
> aggregation
> to be done later.
>
> Thanks,
> John
>
> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
> >
> >
> > Hi Scott,
> >
> > It looks like you've set your MET config file to correctly
> identify which
> > linetypes you'd like to see. I'm not sure why you are
> essentially getting
> > empty output files. I'll need to find someone who can
answer
> your python
> > embedding and grid-stat questions.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > Pronouns: she/her
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> > ---------------
> >
> >
> >
> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >
> > >
> > > Thu Jun 03 12:04:14 2021: Request 100090 was acted upon.
> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
> > > Queue: met_help
> > > Subject: 2 questions regarding how to generate
specific
> stats
> > > Owner: Nobody
> > > Requestors: scott.d.rabenhorst at nasa.gov
> > > Status: new
> > > Ticket <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
> >
> > >
> > >
> > > Hi met support,
> > >
> > > I have 2 questions regarding how to output specific
statistics
> for my
> > > experiment (see files in my attached tarball). I have
looked
> through the
> > > online documentation, but I have had a hard time finding
a
> direct
> > > answer/solution – perhaps I’ve simply missed something
obvious.
> > >
> > > QUESTION 1: how to output wind/vector (VL1L2, VCNT)
> statistics. I have
> > > added:
> > >
> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> > >
> > > … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
> file. I
> > have
> > > also added:
> > >
> > > is_u_wind : True
> > > is_grid_relative: False
> > >
> > > … attributes to my PYTHON_NUMPY object through python
> embedding. However,
> > > when I look at the output *_vl1l2.txt files, the only
> information is the
> > > header lines. I have included 2 sample outputs from my
> embedding scripts
> > > when called alone (./scripts/read_*2xr_v2.example.log)
which
> show all the
> > > data is as expected. It seems like something else needs
to be
> activated
> > > somewhere in the conf files, but I can’t seem to find
the
> correct switch.
> > > If you have any ideas for other things to try, please
let me
> know.
> > >
> > > QUESTION 2: is there a way to use stat_analysis to
> recompute/aggregate
> > > over my *_pairs.nc files containing this information
(example
> ncdump in
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > > and output CNT stats for the total of all the matched
pair
> data? I am
> > aware
> > > of the “-job summary -line_type CNT -column RMSE -by
> FCST_VAR,FCST_LEV
> > > -dump_row” but this looks like it only provides basic
> statistics of all
> > the
> > > RMSE values in each *_cnt.txt files, correct? What I
really
> want is to
> > > generate actual CNT data from all matched pairs over all
> times. I’ve
> > tried
> > > various combinations of aggregate and aggregate_stat but
can’t
> seem to
> > find
> > > a combination that will work. Is this possible? If so,
what
> should my
> > > stat_analysis job command look like?
> > >
> > > Thanks so much for your time in advance. I have asked a
lot of
> questions
> > > lately, but I think I’m almost there. Any help you can
provide
> on filling
> > > in my knowledge gaps would be very much appreciated.
> > >
> > > Thanks,
> > > Scott
> > >
> > >
> >
> >
>
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Wed Jun 16 14:12:59 2021
Hi Scott,
I think I see what is happening with the U/V fields. I checked the
code
where it is checking the U/V fields and it expects the name and level
of
the fcst and obs to match to make sure it it processing the correct
U/V
data. I noticed in the log output from your python embedding call that
era5pl2xr has the name "u" while g5fcst2xr has "U". I would suggest
modifying the python embedding script to ensure that both of the
varname
values are the same case.
If that doesn't work, I would suggest bumping up the log verbosity for
GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are any clues in
the
higher number log output. If you are still having issues, you could
send me
the input files for each and I can play around with the settings to
see
what will work.
Thanks,
George
On Wed, Jun 16, 2021 at 1:07 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Scott,
>
> Regarding SeriesAnalysis -- there is an argument to the tool called
> "-both" that can be used to set both the fcst and obs lists to use
the same
> files. This use case runs SeriesAnalysis on GridStat output:
>
https://metplus.readthedocs.io/en/v4.0.0/generated/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html?highlight=SeriesAnalysisUseCase%20%20GridStatToolUseCase
>
> This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set the
input
> file path and specifies the fcst and obs input fields separately:
>
> FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
>
> OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
>
> BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> pairs.nc
>
> Regarding not being able to produce wind output, I'll take a look at
your
> files and see if I can narrow down the issue. I will report back
with more
> information.
>
> Thanks,
> George
>
> On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>>
>> Hi George,
>>
>> Just as a follow up thought to problem below regarding winds stats,
when
>> I look through the CNT and SL1L2 statistics output, I see U and V
listed
>> with all respective stats. So this issue is not a problem reading U
and V,
>> it is a problem recognizing they are vector components. Are your
sure there
>> is not another flag or option to set either in the
PTHON_NUMPY/attrs
>> objects or another option in the metplus conf file?
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS
>> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>>
>> Hi George,
>>
>> Thanks for answering my questions below. So sorry for taking so
much
>> of your time lately. From your description, it sounds like the best
option
>> for my needs is to use Stat-Analysis to aggregate SL1L2/VL1L2 data.
My goal
>> is to generate single-value statistics for each variable/level
through all
>> valid times and spatial areas. These will be plugged into a Green's
>> Function algorithm for our model.
>>
>> However, the Series-Analysis is also an intriguing approach at
least
>> for looking at spatial errors. I guess Grid-Stat could operate on
that data
>> at the end to generate total stats? I have not tried using Series-
Analysis
>> yet, but what has confused me is the documentation requires
separate "-fcst
>> file_1...file_n" and "-obs file_1...file_n". In most examples I
have seen,
>> these are independent files. However, you make it sound as though
it can
>> run on my *_pairs.nc files generated by Grid-Stat I presume as long
as
>> RAW data is in there? In this case, I assume the list of files
would be
>> identical to both forecast and observations? Are there any examples
of
>> using matched-pairs for the Series-Analysis online?
>>
>> Unfortunately, the larger problem right now is that I still
can't get
>> Grid-Stat to process any my winds and output VCNT or VL1L2 data. I
tried
>> your suggestion below and added the options to my
>> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file (please
see
>> updated attached tarball). I tried running with and without each of
these
>> options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and those
ASCII
>> files never contains any VL1L2/VCNT data, nor do the *.stat files
contain
>> any of those line_types. I also toggled outputting those attribute
in my
>> PYTHON_NUMPY attrs object, but that didn't work either. I couldn't
find any
>> combinations that would work. In an effort to narrow the problem, I
>> copy/pasted the first references to the fcst and obs fields in my
logfile
>> and ran my python read scripts stand-alone and they produced the
expected
>> output with correct values:
>> scripts/read_era5pl2xr_v2.example.log
>> scripts/read_g5fcst2xr_v2.example.log
>>
>> Furthermore, I decided to test PlotDataPlane since I recalled
you
>> saying that was used behind the scenes. It ran just fine and
produced these
>> beautiful U 500hPa images without error:
>> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
>>
>> I have combed the log files for any errors or warning, but
cannot
>> find any at all, or anything else that would indicate a processing
error.
>> It is almost like there is some flag that has to be set in order to
trigger
>> wind processing. Do you have any other ideas I can try?
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/3/21, 6:22 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
>> wrote:
>>
>> Hi Scott,
>>
>> Minna asked me to take a look at your questions.
>>
>> For the second question, the answer is no. Stat-Analysis
reads
>> the .stat
>> output files created by the MET tools. It does not read the
>> gridded NetCDF
>> matched pairs files generated by the Grid-Stat tool.
>> But I do have two recommendations to make.
>>
>> 1. Grid-Stat computes statistics and partial sums that
represent
>> a spatial
>> summary. You can configure Grid-Stat to compute its output
over
>> whatever
>> spatial masking regions you'd like. Stat-Analysis can
>> read/aggregate those
>> spatial summaries through time. The SL1L2 line type
contains
>> scalar L1 and
>> L2 norms. Those are written because they allow you to
easily
>> aggregate
>> stats over multiple runs. Many (but not all) of the
continuous
>> statistics
>> (like ME, RMSE, MAE, ACC, and so on) can be derived from
the
>> SL1L2 line
>> type. So I'd recommend trying this type of stat_analysis
job:
>>
>> -job aggregate_stat -line_type SL1L2 -out_line_type CNT -by
>> FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
>>
>> For each unique combination of variable, level, and masking
>> region, it'll
>> aggregate all the SL1L2 lines it finds in the input and
report the
>> corresponding continuous statistics.
>>
>> 2. Another option is running the NetCDF matched pairs
produced by
>> Grid-Stat
>> through the Series-Analysis tool. That'll compute stats
through
>> time
>> separately for each individual grid point. The result with
be a
>> map of
>> statistics rather than a spatial summary.
>>
>> For the first question, I see that you're using python
embedding
>> and are
>> trying to compute vector partial sums output. In general,
when
>> you request
>> an output line type, Grid-Stat will create the
corresponding
>> output file.
>> But if it's empty, that means that it didn't have any input
data
>> suitable
>> for creating that output type. So you've requested vector
stats,
>> but
>> Grid-Stat isn't computing any.
>>
>> Thanks for sending the sample output. Looking in:
>>
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
>>
>> I do see several warning that you should take a look at:
>> WARNING: python_dataplane() -> an error occurred importing
module
>>
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
>>
>> But I do also see that Grid-Stat is processing the U and V
data
>> and
>> reporting output as scalar variables in the CNT.txt file:
>>
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
>>
>> Now I haven't tested this on your data to confirm that
it'll
>> actually work,
>> but we want to set these flags in the MET config file:
>> is_u_wind = true
>> is_v_wind = true
>>
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BoqT8V5Io58SreCFnbED1DKsfOT6OF11qIdOuvTZnDI%3D&reserved=0
>>
>> Checking in
>> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
>> see that var 4 is for U and var 5 is for V.
>> Add those via the METplus config file by adding:
>> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
>> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
>> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
>> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
>>
>> Please give that a shot and let me know how it goes. The
goal is
>> NOT having
>> any U and V show up in the CNT output file... and instead
have
>> entries show
>> up in the VCNT output.
>>
>> One last detail... if you do want to aggregate results
through
>> time, I'd
>> recommend writing the SL1L2 and VL1L2 line types to enable
that
>> aggregation
>> to be done later.
>>
>> Thanks,
>> John
>>
>> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>> >
>> >
>> > Hi Scott,
>> >
>> > It looks like you've set your MET config file to
correctly
>> identify which
>> > linetypes you'd like to see. I'm not sure why you are
>> essentially getting
>> > empty output files. I'll need to find someone who can
answer
>> your python
>> > embedding and grid-stat questions.
>> >
>> > Regards,
>> > Minna
>> > ---------------
>> > Minna Win
>> > Pronouns: she/her
>> > National Center for Atmospheric Research
>> > Developmental Testbed Center
>> > Phone: 303-497-8423
>> > Fax: 303-497-8401
>> > ---------------
>> >
>> >
>> >
>> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D.
>> (GSFC-610.1)[SCIENCE
>> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>> >
>> > >
>> > > Thu Jun 03 12:04:14 2021: Request 100090 was acted
upon.
>> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
>> > > Queue: met_help
>> > > Subject: 2 questions regarding how to generate
specific
>> stats
>> > > Owner: Nobody
>> > > Requestors: scott.d.rabenhorst at nasa.gov
>> > > Status: new
>> > > Ticket <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C2d81ec2a21954c9b3e1d08d926de00b7%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637583557203754956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=q0%2F5YbCTbCxQLjYokxjg31KP1bf%2BlhFdPhHSrcK7wV4%3D&reserved=0
>> >
>> > >
>> > >
>> > > Hi met support,
>> > >
>> > > I have 2 questions regarding how to output specific
>> statistics for my
>> > > experiment (see files in my attached tarball). I have
looked
>> through the
>> > > online documentation, but I have had a hard time
finding a
>> direct
>> > > answer/solution – perhaps I’ve simply missed something
>> obvious.
>> > >
>> > > QUESTION 1: how to output wind/vector (VL1L2, VCNT)
>> statistics. I have
>> > > added:
>> > >
>> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
>> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
>> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
>> > >
>> > > … to my ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
>> file. I
>> > have
>> > > also added:
>> > >
>> > > is_u_wind : True
>> > > is_grid_relative: False
>> > >
>> > > … attributes to my PYTHON_NUMPY object through python
>> embedding. However,
>> > > when I look at the output *_vl1l2.txt files, the only
>> information is the
>> > > header lines. I have included 2 sample outputs from my
>> embedding scripts
>> > > when called alone (./scripts/read_*2xr_v2.example.log)
which
>> show all the
>> > > data is as expected. It seems like something else needs
to be
>> activated
>> > > somewhere in the conf files, but I can’t seem to find
the
>> correct switch.
>> > > If you have any ideas for other things to try, please
let me
>> know.
>> > >
>> > > QUESTION 2: is there a way to use stat_analysis to
>> recompute/aggregate
>> > > over my *_pairs.nc files containing this information
>> (example ncdump in
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
>> > > and output CNT stats for the total of all the matched
pair
>> data? I am
>> > aware
>> > > of the “-job summary -line_type CNT -column RMSE -by
>> FCST_VAR,FCST_LEV
>> > > -dump_row” but this looks like it only provides basic
>> statistics of all
>> > the
>> > > RMSE values in each *_cnt.txt files, correct? What I
really
>> want is to
>> > > generate actual CNT data from all matched pairs over
all
>> times. I’ve
>> > tried
>> > > various combinations of aggregate and aggregate_stat
but
>> can’t seem to
>> > find
>> > > a combination that will work. Is this possible? If so,
what
>> should my
>> > > stat_analysis job command look like?
>> > >
>> > > Thanks so much for your time in advance. I have asked a
lot
>> of questions
>> > > lately, but I think I’m almost there. Any help you can
>> provide on filling
>> > > in my knowledge gaps would be very much appreciated.
>> > >
>> > > Thanks,
>> > > Scott
>> > >
>> > >
>> >
>> >
>>
>>
>>
>>
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Thu Jun 17 10:10:09 2021
Hi George,
Thanks so much for taking a look at the code once again. You had
another great idea, and something rather easy to change via python
embedding compared to using ncatted/ncrename on so many files. I
modified my scripts to ensure all my attrs={...} values matched for
the U and V fields, but still not luck! I can't believe it. Since
metplus doesn't appear to display the output from my python embedding
scripts, I ran both scripts stand-alone by copying/pasting metplus'
name field onto the command line. The output is in the
scripts/read_*2xr_v2.log files. All attributes match exactly (I even
hardcoded the init time just to be sure). I ran with and without the
updating the is_u/v_wind and is_grid_relative attributes in the python
attrs object (which shouldn't be necessary), but that made no
difference either. The vl1l2 and vcnt files are still just header
information only (see tarball). I increased the verbosity of grid_stat
to 10, but my eye didn't catch anything offhand. Maybe you have a
better idea what to look for. This whole thing is super weird. Do you
have any other ideas I can try?
Thanks,
Scott
On 6/16/21, 4:13 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
I think I see what is happening with the U/V fields. I checked the
code
where it is checking the U/V fields and it expects the name and
level of
the fcst and obs to match to make sure it it processing the
correct U/V
data. I noticed in the log output from your python embedding call
that
era5pl2xr has the name "u" while g5fcst2xr has "U". I would
suggest
modifying the python embedding script to ensure that both of the
varname
values are the same case.
If that doesn't work, I would suggest bumping up the log verbosity
for
GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are any clues
in the
higher number log output. If you are still having issues, you
could send me
the input files for each and I can play around with the settings
to see
what will work.
Thanks,
George
On Wed, Jun 16, 2021 at 1:07 PM George McCabe <mccabe at ucar.edu>
wrote:
> Hi Scott,
>
> Regarding SeriesAnalysis -- there is an argument to the tool
called
> "-both" that can be used to set both the fcst and obs lists to
use the same
> files. This use case runs SeriesAnalysis on GridStat output:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944741564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=V9I9lRS3TfLylHg4nYYu2emLSC7IbrC5%2FOQEWHu2lSk%3D&reserved=0
>
> This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
the input
> file path and specifies the fcst and obs input fields
separately:
>
> FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
>
> OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
>
> BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> pairs.nc
>
> Regarding not being able to produce wind output, I'll take a
look at your
> files and see if I can narrow down the issue. I will report back
with more
> information.
>
> Thanks,
> George
>
> On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
>>
>> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
>
>>
>> Hi George,
>>
>> Just as a follow up thought to problem below regarding winds
stats, when
>> I look through the CNT and SL1L2 statistics output, I see U and
V listed
>> with all respective stats. So this issue is not a problem
reading U and V,
>> it is a problem recognizing they are vector components. Are
your sure there
>> is not another flag or option to set either in the
PTHON_NUMPY/attrs
>> objects or another option in the metplus conf file?
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS
>> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>>
>> Hi George,
>>
>> Thanks for answering my questions below. So sorry for
taking so much
>> of your time lately. From your description, it sounds like the
best option
>> for my needs is to use Stat-Analysis to aggregate SL1L2/VL1L2
data. My goal
>> is to generate single-value statistics for each variable/level
through all
>> valid times and spatial areas. These will be plugged into a
Green's
>> Function algorithm for our model.
>>
>> However, the Series-Analysis is also an intriguing approach
at least
>> for looking at spatial errors. I guess Grid-Stat could operate
on that data
>> at the end to generate total stats? I have not tried using
Series-Analysis
>> yet, but what has confused me is the documentation requires
separate "-fcst
>> file_1...file_n" and "-obs file_1...file_n". In most examples I
have seen,
>> these are independent files. However, you make it sound as
though it can
>> run on my *_pairs.nc files generated by Grid-Stat I presume as
long as
>> RAW data is in there? In this case, I assume the list of files
would be
>> identical to both forecast and observations? Are there any
examples of
>> using matched-pairs for the Series-Analysis online?
>>
>> Unfortunately, the larger problem right now is that I still
can't get
>> Grid-Stat to process any my winds and output VCNT or VL1L2
data. I tried
>> your suggestion below and added the options to my
>> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file
(please see
>> updated attached tarball). I tried running with and without
each of these
>> options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and
those ASCII
>> files never contains any VL1L2/VCNT data, nor do the *.stat
files contain
>> any of those line_types. I also toggled outputting those
attribute in my
>> PYTHON_NUMPY attrs object, but that didn't work either. I
couldn't find any
>> combinations that would work. In an effort to narrow the
problem, I
>> copy/pasted the first references to the fcst and obs fields in
my logfile
>> and ran my python read scripts stand-alone and they produced
the expected
>> output with correct values:
>> scripts/read_era5pl2xr_v2.example.log
>> scripts/read_g5fcst2xr_v2.example.log
>>
>> Furthermore, I decided to test PlotDataPlane since I
recalled you
>> saying that was used behind the scenes. It ran just fine and
produced these
>> beautiful U 500hPa images without error:
>> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
>>
>> I have combed the log files for any errors or warning, but
cannot
>> find any at all, or anything else that would indicate a
processing error.
>> It is almost like there is some flag that has to be set in
order to trigger
>> wind processing. Do you have any other ideas I can try?
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/3/21, 6:22 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
>> wrote:
>>
>> Hi Scott,
>>
>> Minna asked me to take a look at your questions.
>>
>> For the second question, the answer is no. Stat-
Analysis reads
>> the .stat
>> output files created by the MET tools. It does not read
the
>> gridded NetCDF
>> matched pairs files generated by the Grid-Stat tool.
>> But I do have two recommendations to make.
>>
>> 1. Grid-Stat computes statistics and partial sums that
represent
>> a spatial
>> summary. You can configure Grid-Stat to compute its
output over
>> whatever
>> spatial masking regions you'd like. Stat-Analysis can
>> read/aggregate those
>> spatial summaries through time. The SL1L2 line type
contains
>> scalar L1 and
>> L2 norms. Those are written because they allow you to
easily
>> aggregate
>> stats over multiple runs. Many (but not all) of the
continuous
>> statistics
>> (like ME, RMSE, MAE, ACC, and so on) can be derived
from the
>> SL1L2 line
>> type. So I'd recommend trying this type of
stat_analysis job:
>>
>> -job aggregate_stat -line_type SL1L2 -out_line_type CNT
-by
>> FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
>>
>> For each unique combination of variable, level, and
masking
>> region, it'll
>> aggregate all the SL1L2 lines it finds in the input and
report the
>> corresponding continuous statistics.
>>
>> 2. Another option is running the NetCDF matched pairs
produced by
>> Grid-Stat
>> through the Series-Analysis tool. That'll compute stats
through
>> time
>> separately for each individual grid point. The result
with be a
>> map of
>> statistics rather than a spatial summary.
>>
>> For the first question, I see that you're using python
embedding
>> and are
>> trying to compute vector partial sums output. In
general, when
>> you request
>> an output line type, Grid-Stat will create the
corresponding
>> output file.
>> But if it's empty, that means that it didn't have any
input data
>> suitable
>> for creating that output type. So you've requested
vector stats,
>> but
>> Grid-Stat isn't computing any.
>>
>> Thanks for sending the sample output. Looking in:
>>
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
>>
>> I do see several warning that you should take a look
at:
>> WARNING: python_dataplane() -> an error occurred
importing module
>>
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
>>
>> But I do also see that Grid-Stat is processing the U
and V data
>> and
>> reporting output as scalar variables in the CNT.txt
file:
>>
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
>>
>> Now I haven't tested this on your data to confirm that
it'll
>> actually work,
>> but we want to set these flags in the MET config file:
>> is_u_wind = true
>> is_v_wind = true
>>
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VhdOqVLjoWr4eUO68OnpQPgiNYoPEnuo%2F8p5vSYEHLI%3D&reserved=0
>>
>> Checking in
>> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
>> see that var 4 is for U and var 5 is for V.
>> Add those via the METplus config file by adding:
>> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
>> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
>> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
>> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
>>
>> Please give that a shot and let me know how it goes.
The goal is
>> NOT having
>> any U and V show up in the CNT output file... and
instead have
>> entries show
>> up in the VCNT output.
>>
>> One last detail... if you do want to aggregate results
through
>> time, I'd
>> recommend writing the SL1L2 and VL1L2 line types to
enable that
>> aggregation
>> to be done later.
>>
>> Thanks,
>> John
>>
>> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
>> >
>> >
>> > Hi Scott,
>> >
>> > It looks like you've set your MET config file to
correctly
>> identify which
>> > linetypes you'd like to see. I'm not sure why you are
>> essentially getting
>> > empty output files. I'll need to find someone who
can answer
>> your python
>> > embedding and grid-stat questions.
>> >
>> > Regards,
>> > Minna
>> > ---------------
>> > Minna Win
>> > Pronouns: she/her
>> > National Center for Atmospheric Research
>> > Developmental Testbed Center
>> > Phone: 303-497-8423
>> > Fax: 303-497-8401
>> > ---------------
>> >
>> >
>> >
>> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott D.
>> (GSFC-610.1)[SCIENCE
>> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu> wrote:
>> >
>> > >
>> > > Thu Jun 03 12:04:14 2021: Request 100090 was acted
upon.
>> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
>> > > Queue: met_help
>> > > Subject: 2 questions regarding how to generate
specific
>> stats
>> > > Owner: Nobody
>> > > Requestors: scott.d.rabenhorst at nasa.gov
>> > > Status: new
>> > > Ticket <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
>> >
>> > >
>> > >
>> > > Hi met support,
>> > >
>> > > I have 2 questions regarding how to output specific
>> statistics for my
>> > > experiment (see files in my attached tarball). I
have looked
>> through the
>> > > online documentation, but I have had a hard time
finding a
>> direct
>> > > answer/solution – perhaps I’ve simply missed
something
>> obvious.
>> > >
>> > > QUESTION 1: how to output wind/vector (VL1L2, VCNT)
>> statistics. I have
>> > > added:
>> > >
>> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
>> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
>> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
>> > >
>> > > … to my
./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
>> file. I
>> > have
>> > > also added:
>> > >
>> > > is_u_wind : True
>> > > is_grid_relative: False
>> > >
>> > > … attributes to my PYTHON_NUMPY object through
python
>> embedding. However,
>> > > when I look at the output *_vl1l2.txt files, the
only
>> information is the
>> > > header lines. I have included 2 sample outputs from
my
>> embedding scripts
>> > > when called alone
(./scripts/read_*2xr_v2.example.log) which
>> show all the
>> > > data is as expected. It seems like something else
needs to be
>> activated
>> > > somewhere in the conf files, but I can’t seem to
find the
>> correct switch.
>> > > If you have any ideas for other things to try,
please let me
>> know.
>> > >
>> > > QUESTION 2: is there a way to use stat_analysis to
>> recompute/aggregate
>> > > over my *_pairs.nc files containing this
information
>> (example ncdump in
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
>> > > and output CNT stats for the total of all the
matched pair
>> data? I am
>> > aware
>> > > of the “-job summary -line_type CNT -column RMSE
-by
>> FCST_VAR,FCST_LEV
>> > > -dump_row” but this looks like it only provides
basic
>> statistics of all
>> > the
>> > > RMSE values in each *_cnt.txt files, correct? What
I really
>> want is to
>> > > generate actual CNT data from all matched pairs
over all
>> times. I’ve
>> > tried
>> > > various combinations of aggregate and
aggregate_stat but
>> can’t seem to
>> > find
>> > > a combination that will work. Is this possible? If
so, what
>> should my
>> > > stat_analysis job command look like?
>> > >
>> > > Thanks so much for your time in advance. I have
asked a lot
>> of questions
>> > > lately, but I think I’m almost there. Any help you
can
>> provide on filling
>> > > in my knowledge gaps would be very much
appreciated.
>> > >
>> > > Thanks,
>> > > Scott
>> > >
>> > >
>> >
>> >
>>
>>
>>
>>
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Thu Jun 17 11:28:27 2021
Hi Scott,
You could try adding
set_attr_name = "U";
to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5 options to see if
that
helps? It may be using the script as the name, so setting this would
override the input name that is used.
FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
Let me know if that doesn't work and I can continue digging.
Thanks,
George
On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> Thanks so much for taking a look at the code once again. You had
another
> great idea, and something rather easy to change via python embedding
> compared to using ncatted/ncrename on so many files. I modified my
scripts
> to ensure all my attrs={...} values matched for the U and V fields,
but
> still not luck! I can't believe it. Since metplus doesn't appear to
display
> the output from my python embedding scripts, I ran both scripts
stand-alone
> by copying/pasting metplus' name field onto the command line. The
output is
> in the scripts/read_*2xr_v2.log files. All attributes match exactly
(I even
> hardcoded the init time just to be sure). I ran with and without the
> updating the is_u/v_wind and is_grid_relative attributes in the
python
> attrs object (which shouldn't be necessary), but that made no
difference
> either. The vl1l2 and vcnt files are still just header information
only
> (see tarball). I increased the verbosity of grid_stat to 10, but my
eye
> didn't catch anything offhand. Maybe yo!
> u have a better idea what to look for. This whole thing is super
weird.
> Do you have any other ideas I can try?
>
> Thanks,
> Scott
>
>
>
> On 6/16/21, 4:13 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> I think I see what is happening with the U/V fields. I checked
the code
> where it is checking the U/V fields and it expects the name and
level
> of
> the fcst and obs to match to make sure it it processing the
correct U/V
> data. I noticed in the log output from your python embedding
call that
> era5pl2xr has the name "u" while g5fcst2xr has "U". I would
suggest
> modifying the python embedding script to ensure that both of the
> varname
> values are the same case.
>
> If that doesn't work, I would suggest bumping up the log
verbosity for
> GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are any clues
in the
> higher number log output. If you are still having issues, you
could
> send me
> the input files for each and I can play around with the settings
to see
> what will work.
>
> Thanks,
> George
>
> On Wed, Jun 16, 2021 at 1:07 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Scott,
> >
> > Regarding SeriesAnalysis -- there is an argument to the tool
called
> > "-both" that can be used to set both the fcst and obs lists to
use
> the same
> > files. This use case runs SeriesAnalysis on GridStat output:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944741564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=V9I9lRS3TfLylHg4nYYu2emLSC7IbrC5%2FOQEWHu2lSk%3D&reserved=0
> >
> > This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
the
> input
> > file path and specifies the fcst and obs input fields
separately:
> >
> > FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> >
> > OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> >
> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > pairs.nc
> >
> > Regarding not being able to produce wind output, I'll take a
look at
> your
> > files and see if I can narrow down the issue. I will report
back
> with more
> > information.
> >
> > Thanks,
> > George
> >
> > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >
> >>
> >> <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
> >
> >>
> >> Hi George,
> >>
> >> Just as a follow up thought to problem below regarding winds
stats,
> when
> >> I look through the CNT and SL1L2 statistics output, I see U
and V
> listed
> >> with all respective stats. So this issue is not a problem
reading U
> and V,
> >> it is a problem recognizing they are vector components. Are
your
> sure there
> >> is not another flag or option to set either in the
PTHON_NUMPY/attrs
> >> objects or another option in the metplus conf file?
> >>
> >> Thanks,
> >> Scott
> >>
> >>
> >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS
> >> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
> >>
> >> Hi George,
> >>
> >> Thanks for answering my questions below. So sorry for
taking so
> much
> >> of your time lately. From your description, it sounds like
the best
> option
> >> for my needs is to use Stat-Analysis to aggregate SL1L2/VL1L2
data.
> My goal
> >> is to generate single-value statistics for each
variable/level
> through all
> >> valid times and spatial areas. These will be plugged into a
Green's
> >> Function algorithm for our model.
> >>
> >> However, the Series-Analysis is also an intriguing
approach at
> least
> >> for looking at spatial errors. I guess Grid-Stat could
operate on
> that data
> >> at the end to generate total stats? I have not tried using
> Series-Analysis
> >> yet, but what has confused me is the documentation requires
> separate "-fcst
> >> file_1...file_n" and "-obs file_1...file_n". In most examples
I
> have seen,
> >> these are independent files. However, you make it sound as
though
> it can
> >> run on my *_pairs.nc files generated by Grid-Stat I presume
as
> long as
> >> RAW data is in there? In this case, I assume the list of
files
> would be
> >> identical to both forecast and observations? Are there any
examples
> of
> >> using matched-pairs for the Series-Analysis online?
> >>
> >> Unfortunately, the larger problem right now is that I
still
> can't get
> >> Grid-Stat to process any my winds and output VCNT or VL1L2
data. I
> tried
> >> your suggestion below and added the options to my
> >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file
(please see
> >> updated attached tarball). I tried running with and without
each of
> these
> >> options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE" and
those
> ASCII
> >> files never contains any VL1L2/VCNT data, nor do the *.stat
files
> contain
> >> any of those line_types. I also toggled outputting those
attribute
> in my
> >> PYTHON_NUMPY attrs object, but that didn't work either. I
couldn't
> find any
> >> combinations that would work. In an effort to narrow the
problem, I
> >> copy/pasted the first references to the fcst and obs fields
in my
> logfile
> >> and ran my python read scripts stand-alone and they produced
the
> expected
> >> output with correct values:
> >> scripts/read_era5pl2xr_v2.example.log
> >> scripts/read_g5fcst2xr_v2.example.log
> >>
> >> Furthermore, I decided to test PlotDataPlane since I
recalled
> you
> >> saying that was used behind the scenes. It ran just fine and
> produced these
> >> beautiful U 500hPa images without error:
> >> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
> >>
> >> I have combed the log files for any errors or warning,
but
> cannot
> >> find any at all, or anything else that would indicate a
processing
> error.
> >> It is almost like there is some flag that has to be set in
order to
> trigger
> >> wind processing. Do you have any other ideas I can try?
> >>
> >> Thanks,
> >> Scott
> >>
> >>
> >> On 6/3/21, 6:22 PM, "John Halley Gotway via RT" <
> met_help at ucar.edu>
> >> wrote:
> >>
> >> Hi Scott,
> >>
> >> Minna asked me to take a look at your questions.
> >>
> >> For the second question, the answer is no. Stat-
Analysis
> reads
> >> the .stat
> >> output files created by the MET tools. It does not
read the
> >> gridded NetCDF
> >> matched pairs files generated by the Grid-Stat tool.
> >> But I do have two recommendations to make.
> >>
> >> 1. Grid-Stat computes statistics and partial sums
that
> represent
> >> a spatial
> >> summary. You can configure Grid-Stat to compute its
output
> over
> >> whatever
> >> spatial masking regions you'd like. Stat-Analysis can
> >> read/aggregate those
> >> spatial summaries through time. The SL1L2 line type
contains
> >> scalar L1 and
> >> L2 norms. Those are written because they allow you to
easily
> >> aggregate
> >> stats over multiple runs. Many (but not all) of the
> continuous
> >> statistics
> >> (like ME, RMSE, MAE, ACC, and so on) can be derived
from the
> >> SL1L2 line
> >> type. So I'd recommend trying this type of
stat_analysis
> job:
> >>
> >> -job aggregate_stat -line_type SL1L2 -out_line_type
CNT -by
> >> FCST_VAR,FCST_LEV,VX_MASK -out_stat sl1l2_to_cnt.stat
> >>
> >> For each unique combination of variable, level, and
masking
> >> region, it'll
> >> aggregate all the SL1L2 lines it finds in the input
and
> report the
> >> corresponding continuous statistics.
> >>
> >> 2. Another option is running the NetCDF matched pairs
> produced by
> >> Grid-Stat
> >> through the Series-Analysis tool. That'll compute
stats
> through
> >> time
> >> separately for each individual grid point. The result
with
> be a
> >> map of
> >> statistics rather than a spatial summary.
> >>
> >> For the first question, I see that you're using
python
> embedding
> >> and are
> >> trying to compute vector partial sums output. In
general,
> when
> >> you request
> >> an output line type, Grid-Stat will create the
corresponding
> >> output file.
> >> But if it's empty, that means that it didn't have any
input
> data
> >> suitable
> >> for creating that output type. So you've requested
vector
> stats,
> >> but
> >> Grid-Stat isn't computing any.
> >>
> >> Thanks for sending the sample output. Looking in:
> >>
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> >>
> >> I do see several warning that you should take a look
at:
> >> WARNING: python_dataplane() -> an error occurred
importing
> module
> >>
> >>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> >>
> >> But I do also see that Grid-Stat is processing the U
and V
> data
> >> and
> >> reporting output as scalar variables in the CNT.txt
file:
> >>
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> >>
> >> Now I haven't tested this on your data to confirm
that it'll
> >> actually work,
> >> but we want to set these flags in the MET config
file:
> >> is_u_wind = true
> >> is_v_wind = true
> >>
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VhdOqVLjoWr4eUO68OnpQPgiNYoPEnuo%2F8p5vSYEHLI%3D&reserved=0
> >>
> >> Checking in
> >> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> >> see that var 4 is for U and var 5 is for V.
> >> Add those via the METplus config file by adding:
> >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> >>
> >> Please give that a shot and let me know how it goes.
The
> goal is
> >> NOT having
> >> any U and V show up in the CNT output file... and
instead
> have
> >> entries show
> >> up in the VCNT output.
> >>
> >> One last detail... if you do want to aggregate
results
> through
> >> time, I'd
> >> recommend writing the SL1L2 and VL1L2 line types to
enable
> that
> >> aggregation
> >> to be done later.
> >>
> >> Thanks,
> >> John
> >>
> >> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
> >> >
> >> >
> >> > Hi Scott,
> >> >
> >> > It looks like you've set your MET config file to
correctly
> >> identify which
> >> > linetypes you'd like to see. I'm not sure why you
are
> >> essentially getting
> >> > empty output files. I'll need to find someone
who can
> answer
> >> your python
> >> > embedding and grid-stat questions.
> >> >
> >> > Regards,
> >> > Minna
> >> > ---------------
> >> > Minna Win
> >> > Pronouns: she/her
> >> > National Center for Atmospheric Research
> >> > Developmental Testbed Center
> >> > Phone: 303-497-8423
> >> > Fax: 303-497-8401
> >> > ---------------
> >> >
> >> >
> >> >
> >> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst, Scott
D.
> >> (GSFC-610.1)[SCIENCE
> >> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >> >
> >> > >
> >> > > Thu Jun 03 12:04:14 2021: Request 100090 was
acted upon.
> >> > > Transaction: Ticket created by
> scott.d.rabenhorst at nasa.gov
> >> > > Queue: met_help
> >> > > Subject: 2 questions regarding how to
generate
> specific
> >> stats
> >> > > Owner: Nobody
> >> > > Requestors: scott.d.rabenhorst at nasa.gov
> >> > > Status: new
> >> > > Ticket <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C92d4e33b53b84a01680408d931032b33%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637594711944751519%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jFtnqkyMupGghfkMIbQf%2BfsMZ5sj0D76oIYsE7qYri8%3D&reserved=0
> >> >
> >> > >
> >> > >
> >> > > Hi met support,
> >> > >
> >> > > I have 2 questions regarding how to output
specific
> >> statistics for my
> >> > > experiment (see files in my attached tarball). I
have
> looked
> >> through the
> >> > > online documentation, but I have had a hard time
> finding a
> >> direct
> >> > > answer/solution – perhaps I’ve simply missed
something
> >> obvious.
> >> > >
> >> > > QUESTION 1: how to output wind/vector (VL1L2,
VCNT)
> >> statistics. I have
> >> > > added:
> >> > >
> >> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> >> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> >> > >
> >> > > … to my
> ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> >> file. I
> >> > have
> >> > > also added:
> >> > >
> >> > > is_u_wind : True
> >> > > is_grid_relative: False
> >> > >
> >> > > … attributes to my PYTHON_NUMPY object through
python
> >> embedding. However,
> >> > > when I look at the output *_vl1l2.txt files, the
only
> >> information is the
> >> > > header lines. I have included 2 sample outputs
from my
> >> embedding scripts
> >> > > when called alone
(./scripts/read_*2xr_v2.example.log)
> which
> >> show all the
> >> > > data is as expected. It seems like something else
needs
> to be
> >> activated
> >> > > somewhere in the conf files, but I can’t seem to
find
> the
> >> correct switch.
> >> > > If you have any ideas for other things to try,
please
> let me
> >> know.
> >> > >
> >> > > QUESTION 2: is there a way to use stat_analysis
to
> >> recompute/aggregate
> >> > > over my *_pairs.nc files containing this
information
> >> (example ncdump in
> >> > >
> >> >
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> >> > > and output CNT stats for the total of all the
matched
> pair
> >> data? I am
> >> > aware
> >> > > of the “-job summary -line_type CNT -column RMSE
-by
> >> FCST_VAR,FCST_LEV
> >> > > -dump_row” but this looks like it only provides
basic
> >> statistics of all
> >> > the
> >> > > RMSE values in each *_cnt.txt files, correct?
What I
> really
> >> want is to
> >> > > generate actual CNT data from all matched pairs
over all
> >> times. I’ve
> >> > tried
> >> > > various combinations of aggregate and
aggregate_stat but
> >> can’t seem to
> >> > find
> >> > > a combination that will work. Is this possible?
If so,
> what
> >> should my
> >> > > stat_analysis job command look like?
> >> > >
> >> > > Thanks so much for your time in advance. I have
asked a
> lot
> >> of questions
> >> > > lately, but I think I’m almost there. Any help
you can
> >> provide on filling
> >> > > in my knowledge gaps would be very much
appreciated.
> >> > >
> >> > > Thanks,
> >> > > Scott
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >>
> >>
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100090] 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Thu Jun 17 12:29:12 2021
Hi George,
I your suggestion and double checked for typos, but unfortunately,
still nothing different - just headers in the output files. Could
there be a bug in something indirectly related to this, such as the
BOTH option does not work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V
winds need a threshold. Just a WAG.
~Scott
On 6/17/21, 1:28 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
You could try adding
set_attr_name = "U";
to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5 options to see
if that
helps? It may be using the script as the name, so setting this
would
override the input name that is used.
FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
Let me know if that doesn't work and I can continue digging.
Thanks,
George
On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jSEYtG3N0QPNAbVTRh0nSCyY0PQhDLfko4%2Fhp1DWa%2BY%3D&reserved=0
>
>
> Hi George,
>
> Thanks so much for taking a look at the code once again. You had
another
> great idea, and something rather easy to change via python
embedding
> compared to using ncatted/ncrename on so many files. I modified
my scripts
> to ensure all my attrs={...} values matched for the U and V
fields, but
> still not luck! I can't believe it. Since metplus doesn't appear
to display
> the output from my python embedding scripts, I ran both scripts
stand-alone
> by copying/pasting metplus' name field onto the command line.
The output is
> in the scripts/read_*2xr_v2.log files. All attributes match
exactly (I even
> hardcoded the init time just to be sure). I ran with and without
the
> updating the is_u/v_wind and is_grid_relative attributes in the
python
> attrs object (which shouldn't be necessary), but that made no
difference
> either. The vl1l2 and vcnt files are still just header
information only
> (see tarball). I increased the verbosity of grid_stat to 10, but
my eye
> didn't catch anything offhand. Maybe yo!
> u have a better idea what to look for. This whole thing is
super weird.
> Do you have any other ideas I can try?
>
> Thanks,
> Scott
>
>
>
> On 6/16/21, 4:13 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> I think I see what is happening with the U/V fields. I
checked the code
> where it is checking the U/V fields and it expects the name
and level
> of
> the fcst and obs to match to make sure it it processing the
correct U/V
> data. I noticed in the log output from your python embedding
call that
> era5pl2xr has the name "u" while g5fcst2xr has "U". I would
suggest
> modifying the python embedding script to ensure that both of
the
> varname
> values are the same case.
>
> If that doesn't work, I would suggest bumping up the log
verbosity for
> GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are any
clues in the
> higher number log output. If you are still having issues,
you could
> send me
> the input files for each and I can play around with the
settings to see
> what will work.
>
> Thanks,
> George
>
> On Wed, Jun 16, 2021 at 1:07 PM George McCabe
<mccabe at ucar.edu> wrote:
>
> > Hi Scott,
> >
> > Regarding SeriesAnalysis -- there is an argument to the
tool called
> > "-both" that can be used to set both the fcst and obs
lists to use
> the same
> > files. This use case runs SeriesAnalysis on GridStat
output:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xhX9f7KCtD1svZgErYg%2Fn4gruu%2Fmv5Nim7EqXqYARz0%3D&reserved=0
> >
> > This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to
set the
> input
> > file path and specifies the fcst and obs input fields
separately:
> >
> > FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> >
> > OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> >
> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > pairs.nc
> >
> > Regarding not being able to produce wind output, I'll take
a look at
> your
> > files and see if I can narrow down the issue. I will
report back
> with more
> > information.
> >
> > Thanks,
> > George
> >
> > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >
> >>
> >> <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jSEYtG3N0QPNAbVTRh0nSCyY0PQhDLfko4%2Fhp1DWa%2BY%3D&reserved=0
> >
> >>
> >> Hi George,
> >>
> >> Just as a follow up thought to problem below regarding
winds stats,
> when
> >> I look through the CNT and SL1L2 statistics output, I see
U and V
> listed
> >> with all respective stats. So this issue is not a problem
reading U
> and V,
> >> it is a problem recognizing they are vector components.
Are your
> sure there
> >> is not another flag or option to set either in the
PTHON_NUMPY/attrs
> >> objects or another option in the metplus conf file?
> >>
> >> Thanks,
> >> Scott
> >>
> >>
> >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS
> >> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov>
wrote:
> >>
> >> Hi George,
> >>
> >> Thanks for answering my questions below. So sorry for
taking so
> much
> >> of your time lately. From your description, it sounds
like the best
> option
> >> for my needs is to use Stat-Analysis to aggregate
SL1L2/VL1L2 data.
> My goal
> >> is to generate single-value statistics for each
variable/level
> through all
> >> valid times and spatial areas. These will be plugged into
a Green's
> >> Function algorithm for our model.
> >>
> >> However, the Series-Analysis is also an intriguing
approach at
> least
> >> for looking at spatial errors. I guess Grid-Stat could
operate on
> that data
> >> at the end to generate total stats? I have not tried
using
> Series-Analysis
> >> yet, but what has confused me is the documentation
requires
> separate "-fcst
> >> file_1...file_n" and "-obs file_1...file_n". In most
examples I
> have seen,
> >> these are independent files. However, you make it sound
as though
> it can
> >> run on my *_pairs.nc files generated by Grid-Stat I
presume as
> long as
> >> RAW data is in there? In this case, I assume the list of
files
> would be
> >> identical to both forecast and observations? Are there
any examples
> of
> >> using matched-pairs for the Series-Analysis online?
> >>
> >> Unfortunately, the larger problem right now is that I
still
> can't get
> >> Grid-Stat to process any my winds and output VCNT or
VL1L2 data. I
> tried
> >> your suggestion below and added the options to my
> >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf file
(please see
> >> updated attached tarball). I tried running with and
without each of
> these
> >> options "is_u(v)_wind=TRUE" and "is_grid_relative=FALSE"
and those
> ASCII
> >> files never contains any VL1L2/VCNT data, nor do the
*.stat files
> contain
> >> any of those line_types. I also toggled outputting those
attribute
> in my
> >> PYTHON_NUMPY attrs object, but that didn't work either. I
couldn't
> find any
> >> combinations that would work. In an effort to narrow the
problem, I
> >> copy/pasted the first references to the fcst and obs
fields in my
> logfile
> >> and ran my python read scripts stand-alone and they
produced the
> expected
> >> output with correct values:
> >> scripts/read_era5pl2xr_v2.example.log
> >> scripts/read_g5fcst2xr_v2.example.log
> >>
> >> Furthermore, I decided to test PlotDataPlane since I
recalled
> you
> >> saying that was used behind the scenes. It ran just fine
and
> produced these
> >> beautiful U 500hPa images without error:
> >> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
> >>
> >> I have combed the log files for any errors or
warning, but
> cannot
> >> find any at all, or anything else that would indicate a
processing
> error.
> >> It is almost like there is some flag that has to be set
in order to
> trigger
> >> wind processing. Do you have any other ideas I can try?
> >>
> >> Thanks,
> >> Scott
> >>
> >>
> >> On 6/3/21, 6:22 PM, "John Halley Gotway via RT" <
> met_help at ucar.edu>
> >> wrote:
> >>
> >> Hi Scott,
> >>
> >> Minna asked me to take a look at your questions.
> >>
> >> For the second question, the answer is no. Stat-
Analysis
> reads
> >> the .stat
> >> output files created by the MET tools. It does
not read the
> >> gridded NetCDF
> >> matched pairs files generated by the Grid-Stat
tool.
> >> But I do have two recommendations to make.
> >>
> >> 1. Grid-Stat computes statistics and partial sums
that
> represent
> >> a spatial
> >> summary. You can configure Grid-Stat to compute
its output
> over
> >> whatever
> >> spatial masking regions you'd like. Stat-Analysis
can
> >> read/aggregate those
> >> spatial summaries through time. The SL1L2 line
type contains
> >> scalar L1 and
> >> L2 norms. Those are written because they allow
you to easily
> >> aggregate
> >> stats over multiple runs. Many (but not all) of
the
> continuous
> >> statistics
> >> (like ME, RMSE, MAE, ACC, and so on) can be
derived from the
> >> SL1L2 line
> >> type. So I'd recommend trying this type of
stat_analysis
> job:
> >>
> >> -job aggregate_stat -line_type SL1L2
-out_line_type CNT -by
> >> FCST_VAR,FCST_LEV,VX_MASK -out_stat
sl1l2_to_cnt.stat
> >>
> >> For each unique combination of variable, level,
and masking
> >> region, it'll
> >> aggregate all the SL1L2 lines it finds in the
input and
> report the
> >> corresponding continuous statistics.
> >>
> >> 2. Another option is running the NetCDF matched
pairs
> produced by
> >> Grid-Stat
> >> through the Series-Analysis tool. That'll compute
stats
> through
> >> time
> >> separately for each individual grid point. The
result with
> be a
> >> map of
> >> statistics rather than a spatial summary.
> >>
> >> For the first question, I see that you're using
python
> embedding
> >> and are
> >> trying to compute vector partial sums output. In
general,
> when
> >> you request
> >> an output line type, Grid-Stat will create the
corresponding
> >> output file.
> >> But if it's empty, that means that it didn't have
any input
> data
> >> suitable
> >> for creating that output type. So you've
requested vector
> stats,
> >> but
> >> Grid-Stat isn't computing any.
> >>
> >> Thanks for sending the sample output. Looking in:
> >>
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> >>
> >> I do see several warning that you should take a
look at:
> >> WARNING: python_dataplane() -> an error occurred
importing
> module
> >>
> >>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> >>
> >> But I do also see that Grid-Stat is processing
the U and V
> data
> >> and
> >> reporting output as scalar variables in the
CNT.txt file:
> >>
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> >>
> >> Now I haven't tested this on your data to confirm
that it'll
> >> actually work,
> >> but we want to set these flags in the MET config
file:
> >> is_u_wind = true
> >> is_v_wind = true
> >>
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0LFY94EWsSncJkeFp1Pp4fGiRWHB%2BTGCoPQfRWrkLfU%3D&reserved=0
> >>
> >> Checking in
> >>
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> >> see that var 4 is for U and var 5 is for V.
> >> Add those via the METplus config file by adding:
> >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> >>
> >> Please give that a shot and let me know how it
goes. The
> goal is
> >> NOT having
> >> any U and V show up in the CNT output file... and
instead
> have
> >> entries show
> >> up in the VCNT output.
> >>
> >> One last detail... if you do want to aggregate
results
> through
> >> time, I'd
> >> recommend writing the SL1L2 and VL1L2 line types
to enable
> that
> >> aggregation
> >> to be done later.
> >>
> >> Thanks,
> >> John
> >>
> >> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178324842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UjsSIi6kIyovEh7AOUIfTeC9pIVLiFGcn3Wl1lWH6ek%3D&reserved=0
> >> >
> >> >
> >> > Hi Scott,
> >> >
> >> > It looks like you've set your MET config file
to correctly
> >> identify which
> >> > linetypes you'd like to see. I'm not sure why
you are
> >> essentially getting
> >> > empty output files. I'll need to find
someone who can
> answer
> >> your python
> >> > embedding and grid-stat questions.
> >> >
> >> > Regards,
> >> > Minna
> >> > ---------------
> >> > Minna Win
> >> > Pronouns: she/her
> >> > National Center for Atmospheric Research
> >> > Developmental Testbed Center
> >> > Phone: 303-497-8423
> >> > Fax: 303-497-8401
> >> > ---------------
> >> >
> >> >
> >> >
> >> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst,
Scott D.
> >> (GSFC-610.1)[SCIENCE
> >> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >> >
> >> > >
> >> > > Thu Jun 03 12:04:14 2021: Request 100090 was
acted upon.
> >> > > Transaction: Ticket created by
> scott.d.rabenhorst at nasa.gov
> >> > > Queue: met_help
> >> > > Subject: 2 questions regarding how to
generate
> specific
> >> stats
> >> > > Owner: Nobody
> >> > > Requestors: scott.d.rabenhorst at nasa.gov
> >> > > Status: new
> >> > > Ticket <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178324842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UjsSIi6kIyovEh7AOUIfTeC9pIVLiFGcn3Wl1lWH6ek%3D&reserved=0
> >> >
> >> > >
> >> > >
> >> > > Hi met support,
> >> > >
> >> > > I have 2 questions regarding how to output
specific
> >> statistics for my
> >> > > experiment (see files in my attached
tarball). I have
> looked
> >> through the
> >> > > online documentation, but I have had a hard
time
> finding a
> >> direct
> >> > > answer/solution – perhaps I’ve simply missed
something
> >> obvious.
> >> > >
> >> > > QUESTION 1: how to output wind/vector (VL1L2,
VCNT)
> >> statistics. I have
> >> > > added:
> >> > >
> >> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> >> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> >> > >
> >> > > … to my
> ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> >> file. I
> >> > have
> >> > > also added:
> >> > >
> >> > > is_u_wind : True
> >> > > is_grid_relative: False
> >> > >
> >> > > … attributes to my PYTHON_NUMPY object
through python
> >> embedding. However,
> >> > > when I look at the output *_vl1l2.txt files,
the only
> >> information is the
> >> > > header lines. I have included 2 sample
outputs from my
> >> embedding scripts
> >> > > when called alone
(./scripts/read_*2xr_v2.example.log)
> which
> >> show all the
> >> > > data is as expected. It seems like something
else needs
> to be
> >> activated
> >> > > somewhere in the conf files, but I can’t seem
to find
> the
> >> correct switch.
> >> > > If you have any ideas for other things to
try, please
> let me
> >> know.
> >> > >
> >> > > QUESTION 2: is there a way to use
stat_analysis to
> >> recompute/aggregate
> >> > > over my *_pairs.nc files containing this
information
> >> (example ncdump in
> >> > >
> >> >
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> >> > > and output CNT stats for the total of all the
matched
> pair
> >> data? I am
> >> > aware
> >> > > of the “-job summary -line_type CNT -column
RMSE -by
> >> FCST_VAR,FCST_LEV
> >> > > -dump_row” but this looks like it only
provides basic
> >> statistics of all
> >> > the
> >> > > RMSE values in each *_cnt.txt files, correct?
What I
> really
> >> want is to
> >> > > generate actual CNT data from all matched
pairs over all
> >> times. I’ve
> >> > tried
> >> > > various combinations of aggregate and
aggregate_stat but
> >> can’t seem to
> >> > find
> >> > > a combination that will work. Is this
possible? If so,
> what
> >> should my
> >> > > stat_analysis job command look like?
> >> > >
> >> > > Thanks so much for your time in advance. I
have asked a
> lot
> >> of questions
> >> > > lately, but I think I’m almost there. Any
help you can
> >> provide on filling
> >> > > in my knowledge gaps would be very much
appreciated.
> >> > >
> >> > > Thanks,
> >> > > Scott
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >>
> >>
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not
feel
> obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not
feel obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Thu Jun 17 15:01:00 2021
Hi Scott,
The VL1L2 flag should accept BOTH. I'm not familiar enough with the
U/V
calculations to know if it needs a threshold but John didn't mention
that
it was a requirement in his first email. Could you send me a set of
files
that are used as input for this use case? Also, the latest log file
would
be helpful. I noticed that the name of the fields did reflect the
script
name, so I'm curious to see if it was overridden properly in your last
run.
| | | | | | | Dictionary Entry[0] ...
| | | | | | | | Name = *name*
| | | | | | | | Type = StringType
| | | | | | | | *String Value =
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
U10M 20191221_2100 20191223_0000"*
| | | | | | | Dictionary Entry[1] ...
| | | | | | | | Name = is_u_wind
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = true
| | | | | | | Dictionary Entry[2] ...
| | | | | | | | Name = is_grid_relative
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = false
Thanks,
George
On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> I your suggestion and double checked for typos, but unfortunately,
still
> nothing different - just headers in the output files. Could there be
a bug
> in something indirectly related to this, such as the BOTH option
does not
> work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V winds need a
threshold.
> Just a WAG.
>
> ~Scott
>
>
>
> On 6/17/21, 1:28 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> You could try adding
>
> set_attr_name = "U";
>
> to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5 options to
see if
> that
> helps? It may be using the script as the name, so setting this
would
> override the input name that is used.
>
> FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
> OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name = "U";
> FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
> OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name = "V";
>
> Let me know if that doesn't work and I can continue digging.
>
> Thanks,
> George
>
> On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jSEYtG3N0QPNAbVTRh0nSCyY0PQhDLfko4%2Fhp1DWa%2BY%3D&reserved=0
> >
> >
> > Hi George,
> >
> > Thanks so much for taking a look at the code once again. You
had
> another
> > great idea, and something rather easy to change via python
embedding
> > compared to using ncatted/ncrename on so many files. I
modified my
> scripts
> > to ensure all my attrs={...} values matched for the U and V
fields,
> but
> > still not luck! I can't believe it. Since metplus doesn't
appear to
> display
> > the output from my python embedding scripts, I ran both
scripts
> stand-alone
> > by copying/pasting metplus' name field onto the command line.
The
> output is
> > in the scripts/read_*2xr_v2.log files. All attributes match
exactly
> (I even
> > hardcoded the init time just to be sure). I ran with and
without the
> > updating the is_u/v_wind and is_grid_relative attributes in
the
> python
> > attrs object (which shouldn't be necessary), but that made no
> difference
> > either. The vl1l2 and vcnt files are still just header
information
> only
> > (see tarball). I increased the verbosity of grid_stat to 10,
but my
> eye
> > didn't catch anything offhand. Maybe yo!
> > u have a better idea what to look for. This whole thing is
super
> weird.
> > Do you have any other ideas I can try?
> >
> > Thanks,
> > Scott
> >
> >
> >
> > On 6/16/21, 4:13 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > I think I see what is happening with the U/V fields. I
checked
> the code
> > where it is checking the U/V fields and it expects the
name and
> level
> > of
> > the fcst and obs to match to make sure it it processing
the
> correct U/V
> > data. I noticed in the log output from your python
embedding
> call that
> > era5pl2xr has the name "u" while g5fcst2xr has "U". I
would
> suggest
> > modifying the python embedding script to ensure that both
of the
> > varname
> > values are the same case.
> >
> > If that doesn't work, I would suggest bumping up the log
> verbosity for
> > GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are any
clues
> in the
> > higher number log output. If you are still having issues,
you
> could
> > send me
> > the input files for each and I can play around with the
settings
> to see
> > what will work.
> >
> > Thanks,
> > George
> >
> > On Wed, Jun 16, 2021 at 1:07 PM George McCabe
<mccabe at ucar.edu>
> wrote:
> >
> > > Hi Scott,
> > >
> > > Regarding SeriesAnalysis -- there is an argument to the
tool
> called
> > > "-both" that can be used to set both the fcst and obs
lists to
> use
> > the same
> > > files. This use case runs SeriesAnalysis on GridStat
output:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xhX9f7KCtD1svZgErYg%2Fn4gruu%2Fmv5Nim7EqXqYARz0%3D&reserved=0
> > >
> > > This example uses BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE
to set
> the
> > input
> > > file path and specifies the fcst and obs input fields
> separately:
> > >
> > > FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> > > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> > > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > pairs.nc
> > >
> > > Regarding not being able to produce wind output, I'll
take a
> look at
> > your
> > > files and see if I can narrow down the issue. I will
report
> back
> > with more
> > > information.
> > >
> > > Thanks,
> > > George
> > >
> > > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
> wrote:
> > >
> > >>
> > >> <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jSEYtG3N0QPNAbVTRh0nSCyY0PQhDLfko4%2Fhp1DWa%2BY%3D&reserved=0
> > >
> > >>
> > >> Hi George,
> > >>
> > >> Just as a follow up thought to problem below regarding
winds
> stats,
> > when
> > >> I look through the CNT and SL1L2 statistics output, I
see U
> and V
> > listed
> > >> with all respective stats. So this issue is not a
problem
> reading U
> > and V,
> > >> it is a problem recognizing they are vector components.
Are
> your
> > sure there
> > >> is not another flag or option to set either in the
> PTHON_NUMPY/attrs
> > >> objects or another option in the metplus conf file?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS
> > >> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov>
wrote:
> > >>
> > >> Hi George,
> > >>
> > >> Thanks for answering my questions below. So sorry
for
> taking so
> > much
> > >> of your time lately. From your description, it sounds
like
> the best
> > option
> > >> for my needs is to use Stat-Analysis to aggregate
SL1L2/VL1L2
> data.
> > My goal
> > >> is to generate single-value statistics for each
variable/level
> > through all
> > >> valid times and spatial areas. These will be plugged
into a
> Green's
> > >> Function algorithm for our model.
> > >>
> > >> However, the Series-Analysis is also an intriguing
> approach at
> > least
> > >> for looking at spatial errors. I guess Grid-Stat could
> operate on
> > that data
> > >> at the end to generate total stats? I have not tried
using
> > Series-Analysis
> > >> yet, but what has confused me is the documentation
requires
> > separate "-fcst
> > >> file_1...file_n" and "-obs file_1...file_n". In most
examples
> I
> > have seen,
> > >> these are independent files. However, you make it sound
as
> though
> > it can
> > >> run on my *_pairs.nc files generated by Grid-Stat I
presume
> as
> > long as
> > >> RAW data is in there? In this case, I assume the list
of files
> > would be
> > >> identical to both forecast and observations? Are there
any
> examples
> > of
> > >> using matched-pairs for the Series-Analysis online?
> > >>
> > >> Unfortunately, the larger problem right now is that
I
> still
> > can't get
> > >> Grid-Stat to process any my winds and output VCNT or
VL1L2
> data. I
> > tried
> > >> your suggestion below and added the options to my
> > >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
file
> (please see
> > >> updated attached tarball). I tried running with and
without
> each of
> > these
> > >> options "is_u(v)_wind=TRUE" and
"is_grid_relative=FALSE" and
> those
> > ASCII
> > >> files never contains any VL1L2/VCNT data, nor do the
*.stat
> files
> > contain
> > >> any of those line_types. I also toggled outputting
those
> attribute
> > in my
> > >> PYTHON_NUMPY attrs object, but that didn't work either.
I
> couldn't
> > find any
> > >> combinations that would work. In an effort to narrow
the
> problem, I
> > >> copy/pasted the first references to the fcst and obs
fields
> in my
> > logfile
> > >> and ran my python read scripts stand-alone and they
produced
> the
> > expected
> > >> output with correct values:
> > >> scripts/read_era5pl2xr_v2.example.log
> > >> scripts/read_g5fcst2xr_v2.example.log
> > >>
> > >> Furthermore, I decided to test PlotDataPlane since
I
> recalled
> > you
> > >> saying that was used behind the scenes. It ran just
fine and
> > produced these
> > >> beautiful U 500hPa images without error:
> > >> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
> > >>
> > >> I have combed the log files for any errors or
warning, but
> > cannot
> > >> find any at all, or anything else that would indicate a
> processing
> > error.
> > >> It is almost like there is some flag that has to be set
in
> order to
> > trigger
> > >> wind processing. Do you have any other ideas I can try?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/3/21, 6:22 PM, "John Halley Gotway via RT" <
> > met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> Hi Scott,
> > >>
> > >> Minna asked me to take a look at your
questions.
> > >>
> > >> For the second question, the answer is no.
> Stat-Analysis
> > reads
> > >> the .stat
> > >> output files created by the MET tools. It does
not
> read the
> > >> gridded NetCDF
> > >> matched pairs files generated by the Grid-Stat
tool.
> > >> But I do have two recommendations to make.
> > >>
> > >> 1. Grid-Stat computes statistics and partial
sums that
> > represent
> > >> a spatial
> > >> summary. You can configure Grid-Stat to compute
its
> output
> > over
> > >> whatever
> > >> spatial masking regions you'd like. Stat-
Analysis can
> > >> read/aggregate those
> > >> spatial summaries through time. The SL1L2 line
type
> contains
> > >> scalar L1 and
> > >> L2 norms. Those are written because they allow
you to
> easily
> > >> aggregate
> > >> stats over multiple runs. Many (but not all) of
the
> > continuous
> > >> statistics
> > >> (like ME, RMSE, MAE, ACC, and so on) can be
derived
> from the
> > >> SL1L2 line
> > >> type. So I'd recommend trying this type of
> stat_analysis
> > job:
> > >>
> > >> -job aggregate_stat -line_type SL1L2
-out_line_type
> CNT -by
> > >> FCST_VAR,FCST_LEV,VX_MASK -out_stat
sl1l2_to_cnt.stat
> > >>
> > >> For each unique combination of variable, level,
and
> masking
> > >> region, it'll
> > >> aggregate all the SL1L2 lines it finds in the
input
> and
> > report the
> > >> corresponding continuous statistics.
> > >>
> > >> 2. Another option is running the NetCDF matched
pairs
> > produced by
> > >> Grid-Stat
> > >> through the Series-Analysis tool. That'll
compute
> stats
> > through
> > >> time
> > >> separately for each individual grid point. The
result
> with
> > be a
> > >> map of
> > >> statistics rather than a spatial summary.
> > >>
> > >> For the first question, I see that you're using
python
> > embedding
> > >> and are
> > >> trying to compute vector partial sums output.
In
> general,
> > when
> > >> you request
> > >> an output line type, Grid-Stat will create the
> corresponding
> > >> output file.
> > >> But if it's empty, that means that it didn't
have any
> input
> > data
> > >> suitable
> > >> for creating that output type. So you've
requested
> vector
> > stats,
> > >> but
> > >> Grid-Stat isn't computing any.
> > >>
> > >> Thanks for sending the sample output. Looking
in:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > >>
> > >> I do see several warning that you should take a
look
> at:
> > >> WARNING: python_dataplane() -> an error
occurred
> importing
> > module
> > >>
> > >>
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > >>
> > >> But I do also see that Grid-Stat is processing
the U
> and V
> > data
> > >> and
> > >> reporting output as scalar variables in the
CNT.txt
> file:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > >>
> > >> Now I haven't tested this on your data to
confirm
> that it'll
> > >> actually work,
> > >> but we want to set these flags in the MET
config file:
> > >> is_u_wind = true
> > >> is_v_wind = true
> > >>
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178314885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0LFY94EWsSncJkeFp1Pp4fGiRWHB%2BTGCoPQfRWrkLfU%3D&reserved=0
> > >>
> > >> Checking in
> > >>
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > >> see that var 4 is for U and var 5 is for V.
> > >> Add those via the METplus config file by
adding:
> > >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> > >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> > >>
> > >> Please give that a shot and let me know how it
goes.
> The
> > goal is
> > >> NOT having
> > >> any U and V show up in the CNT output file...
and
> instead
> > have
> > >> entries show
> > >> up in the VCNT output.
> > >>
> > >> One last detail... if you do want to aggregate
results
> > through
> > >> time, I'd
> > >> recommend writing the SL1L2 and VL1L2 line
types to
> enable
> > that
> > >> aggregation
> > >> to be done later.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Thu, Jun 3, 2021 at 1:49 PM Minna Win via RT
<
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178324842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UjsSIi6kIyovEh7AOUIfTeC9pIVLiFGcn3Wl1lWH6ek%3D&reserved=0
> > >> >
> > >> >
> > >> > Hi Scott,
> > >> >
> > >> > It looks like you've set your MET config file
to
> correctly
> > >> identify which
> > >> > linetypes you'd like to see. I'm not sure why
you
> are
> > >> essentially getting
> > >> > empty output files. I'll need to find
someone
> who can
> > answer
> > >> your python
> > >> > embedding and grid-stat questions.
> > >> >
> > >> > Regards,
> > >> > Minna
> > >> > ---------------
> > >> > Minna Win
> > >> > Pronouns: she/her
> > >> > National Center for Atmospheric Research
> > >> > Developmental Testbed Center
> > >> > Phone: 303-497-8423
> > >> > Fax: 303-497-8401
> > >> > ---------------
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Jun 3, 2021 at 12:04 PM Rabenhorst,
Scott D.
> > >> (GSFC-610.1)[SCIENCE
> > >> > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> > wrote:
> > >> >
> > >> > >
> > >> > > Thu Jun 03 12:04:14 2021: Request 100090
was
> acted upon.
> > >> > > Transaction: Ticket created by
> > scott.d.rabenhorst at nasa.gov
> > >> > > Queue: met_help
> > >> > > Subject: 2 questions regarding how to
> generate
> > specific
> > >> stats
> > >> > > Owner: Nobody
> > >> > > Requestors: scott.d.rabenhorst at nasa.gov
> > >> > > Status: new
> > >> > > Ticket <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce493f85282be4476996a08d931b55658%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595477178324842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UjsSIi6kIyovEh7AOUIfTeC9pIVLiFGcn3Wl1lWH6ek%3D&reserved=0
> > >> >
> > >> > >
> > >> > >
> > >> > > Hi met support,
> > >> > >
> > >> > > I have 2 questions regarding how to output
> specific
> > >> statistics for my
> > >> > > experiment (see files in my attached
tarball). I
> have
> > looked
> > >> through the
> > >> > > online documentation, but I have had a hard
time
> > finding a
> > >> direct
> > >> > > answer/solution – perhaps I’ve simply
missed
> something
> > >> obvious.
> > >> > >
> > >> > > QUESTION 1: how to output wind/vector
(VL1L2,
> VCNT)
> > >> statistics. I have
> > >> > > added:
> > >> > >
> > >> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> > >> > >
> > >> > > … to my
> > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> > >> file. I
> > >> > have
> > >> > > also added:
> > >> > >
> > >> > > is_u_wind : True
> > >> > > is_grid_relative: False
> > >> > >
> > >> > > … attributes to my PYTHON_NUMPY object
through
> python
> > >> embedding. However,
> > >> > > when I look at the output *_vl1l2.txt
files, the
> only
> > >> information is the
> > >> > > header lines. I have included 2 sample
outputs
> from my
> > >> embedding scripts
> > >> > > when called alone
> (./scripts/read_*2xr_v2.example.log)
> > which
> > >> show all the
> > >> > > data is as expected. It seems like
something else
> needs
> > to be
> > >> activated
> > >> > > somewhere in the conf files, but I can’t
seem to
> find
> > the
> > >> correct switch.
> > >> > > If you have any ideas for other things to
try,
> please
> > let me
> > >> know.
> > >> > >
> > >> > > QUESTION 2: is there a way to use
stat_analysis to
> > >> recompute/aggregate
> > >> > > over my *_pairs.nc files containing this
> information
> > >> (example ncdump in
> > >> > >
> > >> >
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > >> > > and output CNT stats for the total of all
the
> matched
> > pair
> > >> data? I am
> > >> > aware
> > >> > > of the “-job summary -line_type CNT -column
RMSE
> -by
> > >> FCST_VAR,FCST_LEV
> > >> > > -dump_row” but this looks like it only
provides
> basic
> > >> statistics of all
> > >> > the
> > >> > > RMSE values in each *_cnt.txt files,
correct?
> What I
> > really
> > >> want is to
> > >> > > generate actual CNT data from all matched
pairs
> over all
> > >> times. I’ve
> > >> > tried
> > >> > > various combinations of aggregate and
> aggregate_stat but
> > >> can’t seem to
> > >> > find
> > >> > > a combination that will work. Is this
possible?
> If so,
> > what
> > >> should my
> > >> > > stat_analysis job command look like?
> > >> > >
> > >> > > Thanks so much for your time in advance. I
have
> asked a
> > lot
> > >> of questions
> > >> > > lately, but I think I’m almost there. Any
help
> you can
> > >> provide on filling
> > >> > > in my knowledge gaps would be very much
> appreciated.
> > >> > >
> > >> > > Thanks,
> > >> > > Scott
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do
not feel
> > obliged to
> > > reply to this email outside of your normal working
hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not
feel
> obliged
> > to
> > reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100090] 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Thu Jun 17 20:32:26 2021
Hi George,
I have attached the log file - that is easy. However, I'm not sure how
to go about sending you the data. The experiment (forecast) 3D file is
734G and the surface (single) level files are 619M each.
On 6/17/21, 5:01 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
The VL1L2 flag should accept BOTH. I'm not familiar enough with
the U/V
calculations to know if it needs a threshold but John didn't
mention that
it was a requirement in his first email. Could you send me a set
of files
that are used as input for this use case? Also, the latest log
file would
be helpful. I noticed that the name of the fields did reflect the
script
name, so I'm curious to see if it was overridden properly in your
last run.
| | | | | | | Dictionary Entry[0] ...
| | | | | | | | Name = *name*
| | | | | | | | Type = StringType
| | | | | | | | *String Value =
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
U10M 20191221_2100 20191223_0000"*
| | | | | | | Dictionary Entry[1] ...
| | | | | | | | Name = is_u_wind
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = true
| | | | | | | Dictionary Entry[2] ...
| | | | | | | | Name = is_grid_relative
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = false
Thanks,
George
On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
>
>
> Hi George,
>
> I your suggestion and double checked for typos, but
unfortunately, still
> nothing different - just headers in the output files. Could
there be a bug
> in something indirectly related to this, such as the BOTH option
does not
> work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V winds need a
threshold.
> Just a WAG.
>
> ~Scott
>
>
>
> On 6/17/21, 1:28 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> You could try adding
>
> set_attr_name = "U";
>
> to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5 options
to see if
> that
> helps? It may be using the script as the name, so setting
this would
> override the input name that is used.
>
> FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name =
"U";
> OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name =
"U";
> FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name =
"V";
> OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name =
"V";
>
> Let me know if that doesn't work and I can continue digging.
>
> Thanks,
> George
>
> On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> >
> >
> > Hi George,
> >
> > Thanks so much for taking a look at the code once again.
You had
> another
> > great idea, and something rather easy to change via python
embedding
> > compared to using ncatted/ncrename on so many files. I
modified my
> scripts
> > to ensure all my attrs={...} values matched for the U and
V fields,
> but
> > still not luck! I can't believe it. Since metplus doesn't
appear to
> display
> > the output from my python embedding scripts, I ran both
scripts
> stand-alone
> > by copying/pasting metplus' name field onto the command
line. The
> output is
> > in the scripts/read_*2xr_v2.log files. All attributes
match exactly
> (I even
> > hardcoded the init time just to be sure). I ran with and
without the
> > updating the is_u/v_wind and is_grid_relative attributes
in the
> python
> > attrs object (which shouldn't be necessary), but that made
no
> difference
> > either. The vl1l2 and vcnt files are still just header
information
> only
> > (see tarball). I increased the verbosity of grid_stat to
10, but my
> eye
> > didn't catch anything offhand. Maybe yo!
> > u have a better idea what to look for. This whole thing
is super
> weird.
> > Do you have any other ideas I can try?
> >
> > Thanks,
> > Scott
> >
> >
> >
> > On 6/16/21, 4:13 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > I think I see what is happening with the U/V fields. I
checked
> the code
> > where it is checking the U/V fields and it expects the
name and
> level
> > of
> > the fcst and obs to match to make sure it it
processing the
> correct U/V
> > data. I noticed in the log output from your python
embedding
> call that
> > era5pl2xr has the name "u" while g5fcst2xr has "U". I
would
> suggest
> > modifying the python embedding script to ensure that
both of the
> > varname
> > values are the same case.
> >
> > If that doesn't work, I would suggest bumping up the
log
> verbosity for
> > GridStat (LOG_GRID_STAT_VERBOSITY) to see if there are
any clues
> in the
> > higher number log output. If you are still having
issues, you
> could
> > send me
> > the input files for each and I can play around with
the settings
> to see
> > what will work.
> >
> > Thanks,
> > George
> >
> > On Wed, Jun 16, 2021 at 1:07 PM George McCabe
<mccabe at ucar.edu>
> wrote:
> >
> > > Hi Scott,
> > >
> > > Regarding SeriesAnalysis -- there is an argument to
the tool
> called
> > > "-both" that can be used to set both the fcst and
obs lists to
> use
> > the same
> > > files. This use case runs SeriesAnalysis on GridStat
output:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IVziYkcTRDZxiPBggao6ZrZMeCqBPyiL0uYrB%2FmKnGw%3D&reserved=0
> > >
> > > This example uses
BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> the
> > input
> > > file path and specifies the fcst and obs input
fields
> separately:
> > >
> > > FCST_SERIES_ANALYSIS_VAR1_NAME = FCST_precip_FULL
> > > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> > > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > pairs.nc
> > >
> > > Regarding not being able to produce wind output,
I'll take a
> look at
> > your
> > > files and see if I can narrow down the issue. I will
report
> back
> > with more
> > > information.
> > >
> > > Thanks,
> > > George
> > >
> > > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> > >
> > >>
> > >> <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > >
> > >>
> > >> Hi George,
> > >>
> > >> Just as a follow up thought to problem below
regarding winds
> stats,
> > when
> > >> I look through the CNT and SL1L2 statistics output,
I see U
> and V
> > listed
> > >> with all respective stats. So this issue is not a
problem
> reading U
> > and V,
> > >> it is a problem recognizing they are vector
components. Are
> your
> > sure there
> > >> is not another flag or option to set either in the
> PTHON_NUMPY/attrs
> > >> objects or another option in the metplus conf file?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS
> > >> AND APPLICATIONS INC]"
<scott.d.rabenhorst at nasa.gov> wrote:
> > >>
> > >> Hi George,
> > >>
> > >> Thanks for answering my questions below. So
sorry for
> taking so
> > much
> > >> of your time lately. From your description, it
sounds like
> the best
> > option
> > >> for my needs is to use Stat-Analysis to aggregate
SL1L2/VL1L2
> data.
> > My goal
> > >> is to generate single-value statistics for each
variable/level
> > through all
> > >> valid times and spatial areas. These will be
plugged into a
> Green's
> > >> Function algorithm for our model.
> > >>
> > >> However, the Series-Analysis is also an
intriguing
> approach at
> > least
> > >> for looking at spatial errors. I guess Grid-Stat
could
> operate on
> > that data
> > >> at the end to generate total stats? I have not
tried using
> > Series-Analysis
> > >> yet, but what has confused me is the documentation
requires
> > separate "-fcst
> > >> file_1...file_n" and "-obs file_1...file_n". In
most examples
> I
> > have seen,
> > >> these are independent files. However, you make it
sound as
> though
> > it can
> > >> run on my *_pairs.nc files generated by Grid-Stat I
presume
> as
> > long as
> > >> RAW data is in there? In this case, I assume the
list of files
> > would be
> > >> identical to both forecast and observations? Are
there any
> examples
> > of
> > >> using matched-pairs for the Series-Analysis online?
> > >>
> > >> Unfortunately, the larger problem right now is
that I
> still
> > can't get
> > >> Grid-Stat to process any my winds and output VCNT
or VL1L2
> data. I
> > tried
> > >> your suggestion below and added the options to my
> > >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
file
> (please see
> > >> updated attached tarball). I tried running with and
without
> each of
> > these
> > >> options "is_u(v)_wind=TRUE" and
"is_grid_relative=FALSE" and
> those
> > ASCII
> > >> files never contains any VL1L2/VCNT data, nor do
the *.stat
> files
> > contain
> > >> any of those line_types. I also toggled outputting
those
> attribute
> > in my
> > >> PYTHON_NUMPY attrs object, but that didn't work
either. I
> couldn't
> > find any
> > >> combinations that would work. In an effort to
narrow the
> problem, I
> > >> copy/pasted the first references to the fcst and
obs fields
> in my
> > logfile
> > >> and ran my python read scripts stand-alone and they
produced
> the
> > expected
> > >> output with correct values:
> > >> scripts/read_era5pl2xr_v2.example.log
> > >> scripts/read_g5fcst2xr_v2.example.log
> > >>
> > >> Furthermore, I decided to test PlotDataPlane
since I
> recalled
> > you
> > >> saying that was used behind the scenes. It ran just
fine and
> > produced these
> > >> beautiful U 500hPa images without error:
> > >> PlotDataPlane_py-
embed_test/plot_data_plane/*.ps
> > >>
> > >> I have combed the log files for any errors or
warning, but
> > cannot
> > >> find any at all, or anything else that would
indicate a
> processing
> > error.
> > >> It is almost like there is some flag that has to be
set in
> order to
> > trigger
> > >> wind processing. Do you have any other ideas I can
try?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/3/21, 6:22 PM, "John Halley Gotway via RT"
<
> > met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> Hi Scott,
> > >>
> > >> Minna asked me to take a look at your
questions.
> > >>
> > >> For the second question, the answer is no.
> Stat-Analysis
> > reads
> > >> the .stat
> > >> output files created by the MET tools. It
does not
> read the
> > >> gridded NetCDF
> > >> matched pairs files generated by the Grid-
Stat tool.
> > >> But I do have two recommendations to make.
> > >>
> > >> 1. Grid-Stat computes statistics and
partial sums that
> > represent
> > >> a spatial
> > >> summary. You can configure Grid-Stat to
compute its
> output
> > over
> > >> whatever
> > >> spatial masking regions you'd like. Stat-
Analysis can
> > >> read/aggregate those
> > >> spatial summaries through time. The SL1L2
line type
> contains
> > >> scalar L1 and
> > >> L2 norms. Those are written because they
allow you to
> easily
> > >> aggregate
> > >> stats over multiple runs. Many (but not
all) of the
> > continuous
> > >> statistics
> > >> (like ME, RMSE, MAE, ACC, and so on) can be
derived
> from the
> > >> SL1L2 line
> > >> type. So I'd recommend trying this type of
> stat_analysis
> > job:
> > >>
> > >> -job aggregate_stat -line_type SL1L2
-out_line_type
> CNT -by
> > >> FCST_VAR,FCST_LEV,VX_MASK -out_stat
sl1l2_to_cnt.stat
> > >>
> > >> For each unique combination of variable,
level, and
> masking
> > >> region, it'll
> > >> aggregate all the SL1L2 lines it finds in
the input
> and
> > report the
> > >> corresponding continuous statistics.
> > >>
> > >> 2. Another option is running the NetCDF
matched pairs
> > produced by
> > >> Grid-Stat
> > >> through the Series-Analysis tool. That'll
compute
> stats
> > through
> > >> time
> > >> separately for each individual grid point.
The result
> with
> > be a
> > >> map of
> > >> statistics rather than a spatial summary.
> > >>
> > >> For the first question, I see that you're
using python
> > embedding
> > >> and are
> > >> trying to compute vector partial sums
output. In
> general,
> > when
> > >> you request
> > >> an output line type, Grid-Stat will create
the
> corresponding
> > >> output file.
> > >> But if it's empty, that means that it
didn't have any
> input
> > data
> > >> suitable
> > >> for creating that output type. So you've
requested
> vector
> > stats,
> > >> but
> > >> Grid-Stat isn't computing any.
> > >>
> > >> Thanks for sending the sample output.
Looking in:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > >>
> > >> I do see several warning that you should
take a look
> at:
> > >> WARNING: python_dataplane() -> an error
occurred
> importing
> > module
> > >>
> > >>
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > >>
> > >> But I do also see that Grid-Stat is
processing the U
> and V
> > data
> > >> and
> > >> reporting output as scalar variables in the
CNT.txt
> file:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > >>
> > >> Now I haven't tested this on your data to
confirm
> that it'll
> > >> actually work,
> > >> but we want to set these flags in the MET
config file:
> > >> is_u_wind = true
> > >> is_v_wind = true
> > >>
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h5AAyqR2xi8LPCZcMvBJMrVq0MuRtJFmkNwFqKrLDKg%3D&reserved=0
> > >>
> > >> Checking in
> > >>
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > >> see that var 4 is for U and var 5 is for V.
> > >> Add those via the METplus config file by
adding:
> > >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> > >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> > >>
> > >> Please give that a shot and let me know how
it goes.
> The
> > goal is
> > >> NOT having
> > >> any U and V show up in the CNT output
file... and
> instead
> > have
> > >> entries show
> > >> up in the VCNT output.
> > >>
> > >> One last detail... if you do want to
aggregate results
> > through
> > >> time, I'd
> > >> recommend writing the SL1L2 and VL1L2 line
types to
> enable
> > that
> > >> aggregation
> > >> to be done later.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Thu, Jun 3, 2021 at 1:49 PM Minna Win
via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > >> >
> > >> >
> > >> > Hi Scott,
> > >> >
> > >> > It looks like you've set your MET config
file to
> correctly
> > >> identify which
> > >> > linetypes you'd like to see. I'm not sure
why you
> are
> > >> essentially getting
> > >> > empty output files. I'll need to find
someone
> who can
> > answer
> > >> your python
> > >> > embedding and grid-stat questions.
> > >> >
> > >> > Regards,
> > >> > Minna
> > >> > ---------------
> > >> > Minna Win
> > >> > Pronouns: she/her
> > >> > National Center for Atmospheric Research
> > >> > Developmental Testbed Center
> > >> > Phone: 303-497-8423
> > >> > Fax: 303-497-8401
> > >> > ---------------
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Jun 3, 2021 at 12:04 PM
Rabenhorst, Scott D.
> > >> (GSFC-610.1)[SCIENCE
> > >> > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> > wrote:
> > >> >
> > >> > >
> > >> > > Thu Jun 03 12:04:14 2021: Request
100090 was
> acted upon.
> > >> > > Transaction: Ticket created by
> > scott.d.rabenhorst at nasa.gov
> > >> > > Queue: met_help
> > >> > > Subject: 2 questions regarding how
to
> generate
> > specific
> > >> stats
> > >> > > Owner: Nobody
> > >> > > Requestors:
scott.d.rabenhorst at nasa.gov
> > >> > > Status: new
> > >> > > Ticket <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697051977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YloVQ%2BfOmCnoZy7DWZFlnyZJlPR5dxXbBQ71sqI641I%3D&reserved=0
> > >> >
> > >> > >
> > >> > >
> > >> > > Hi met support,
> > >> > >
> > >> > > I have 2 questions regarding how to
output
> specific
> > >> statistics for my
> > >> > > experiment (see files in my attached
tarball). I
> have
> > looked
> > >> through the
> > >> > > online documentation, but I have had a
hard time
> > finding a
> > >> direct
> > >> > > answer/solution – perhaps I’ve simply
missed
> something
> > >> obvious.
> > >> > >
> > >> > > QUESTION 1: how to output wind/vector
(VL1L2,
> VCNT)
> > >> statistics. I have
> > >> > > added:
> > >> > >
> > >> > > GRID_STAT_OUTPUT_FLAG_CNT = BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VCNT = BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 = BOTH
> > >> > >
> > >> > > … to my
> > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> > >> file. I
> > >> > have
> > >> > > also added:
> > >> > >
> > >> > > is_u_wind : True
> > >> > > is_grid_relative: False
> > >> > >
> > >> > > … attributes to my PYTHON_NUMPY object
through
> python
> > >> embedding. However,
> > >> > > when I look at the output *_vl1l2.txt
files, the
> only
> > >> information is the
> > >> > > header lines. I have included 2 sample
outputs
> from my
> > >> embedding scripts
> > >> > > when called alone
> (./scripts/read_*2xr_v2.example.log)
> > which
> > >> show all the
> > >> > > data is as expected. It seems like
something else
> needs
> > to be
> > >> activated
> > >> > > somewhere in the conf files, but I
can’t seem to
> find
> > the
> > >> correct switch.
> > >> > > If you have any ideas for other things
to try,
> please
> > let me
> > >> know.
> > >> > >
> > >> > > QUESTION 2: is there a way to use
stat_analysis to
> > >> recompute/aggregate
> > >> > > over my *_pairs.nc files containing
this
> information
> > >> (example ncdump in
> > >> > >
> > >> >
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > >> > > and output CNT stats for the total of
all the
> matched
> > pair
> > >> data? I am
> > >> > aware
> > >> > > of the “-job summary -line_type CNT
-column RMSE
> -by
> > >> FCST_VAR,FCST_LEV
> > >> > > -dump_row” but this looks like it only
provides
> basic
> > >> statistics of all
> > >> > the
> > >> > > RMSE values in each *_cnt.txt files,
correct?
> What I
> > really
> > >> want is to
> > >> > > generate actual CNT data from all
matched pairs
> over all
> > >> times. I’ve
> > >> > tried
> > >> > > various combinations of aggregate and
> aggregate_stat but
> > >> can’t seem to
> > >> > find
> > >> > > a combination that will work. Is this
possible?
> If so,
> > what
> > >> should my
> > >> > > stat_analysis job command look like?
> > >> > >
> > >> > > Thanks so much for your time in
advance. I have
> asked a
> > lot
> > >> of questions
> > >> > > lately, but I think I’m almost there.
Any help
> you can
> > >> provide on filling
> > >> > > in my knowledge gaps would be very much
> appreciated.
> > >> > >
> > >> > > Thanks,
> > >> > > Scott
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please
do not feel
> > obliged to
> > > reply to this email outside of your normal working
hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do
not feel
> obliged
> > to
> > reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not
feel obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Thu Jun 17 20:50:36 2021
Hi George,
I apologize, my last email sent prematurely for some weird reason. As
I was saying below, our forecast and observation files are very large.
I could ncks a subset, but that really wouldn't be the same file. I
suppose it would contain the same fields, but even a subset would be
too large for an email attachment.
Regarding the output below, you are correct. I did not modify the 10m
U/V winds (U10M, V10M, etc). I only changed the 3D pressure level U/V
winds to have the same name and attributes. I figured if those started
working I would propagate the changes to the surface python scripts
too. So I would recommend to only look at the output pertaining to
pressure level winds from the read_era5pl2xr_v2.py and
read_g5fcst2xr_v2.py scripts. Those have the changes.
Please let me know how to proceed with posting the
forecast/observation files if you think that will help. Do you have an
ftp site?
Thanks,
Scott
On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
Hi George,
I have attached the log file - that is easy. However, I'm not sure
how to go about sending you the data. The experiment (forecast) 3D
file is 734G and the surface (single) level files are 619M each.
On 6/17/21, 5:01 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
The VL1L2 flag should accept BOTH. I'm not familiar enough
with the U/V
calculations to know if it needs a threshold but John didn't
mention that
it was a requirement in his first email. Could you send me a
set of files
that are used as input for this use case? Also, the latest log
file would
be helpful. I noticed that the name of the fields did reflect
the script
name, so I'm curious to see if it was overridden properly in
your last run.
| | | | | | | Dictionary Entry[0] ...
| | | | | | | | Name = *name*
| | | | | | | | Type = StringType
| | | | | | | | *String Value =
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
U10M 20191221_2100 20191223_0000"*
| | | | | | | Dictionary Entry[1] ...
| | | | | | | | Name = is_u_wind
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = true
| | | | | | | Dictionary Entry[2] ...
| | | | | | | | Name =
is_grid_relative
| | | | | | | | Type = BooleanType
| | | | | | | | Boolean Value = false
Thanks,
George
On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
>
>
> Hi George,
>
> I your suggestion and double checked for typos, but
unfortunately, still
> nothing different - just headers in the output files. Could
there be a bug
> in something indirectly related to this, such as the BOTH
option does not
> work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V winds need
a threshold.
> Just a WAG.
>
> ~Scott
>
>
>
> On 6/17/21, 1:28 PM, "George McCabe via RT"
<met_help at ucar.edu> wrote:
>
> Hi Scott,
>
> You could try adding
>
> set_attr_name = "U";
>
> to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5
options to see if
> that
> helps? It may be using the script as the name, so
setting this would
> override the input name that is used.
>
> FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name
= "U";
> OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name
= "U";
> FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name
= "V";
> OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name
= "V";
>
> Let me know if that doesn't work and I can continue
digging.
>
> Thanks,
> George
>
> On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> >
> >
> > Hi George,
> >
> > Thanks so much for taking a look at the code once
again. You had
> another
> > great idea, and something rather easy to change via
python embedding
> > compared to using ncatted/ncrename on so many files. I
modified my
> scripts
> > to ensure all my attrs={...} values matched for the U
and V fields,
> but
> > still not luck! I can't believe it. Since metplus
doesn't appear to
> display
> > the output from my python embedding scripts, I ran
both scripts
> stand-alone
> > by copying/pasting metplus' name field onto the
command line. The
> output is
> > in the scripts/read_*2xr_v2.log files. All attributes
match exactly
> (I even
> > hardcoded the init time just to be sure). I ran with
and without the
> > updating the is_u/v_wind and is_grid_relative
attributes in the
> python
> > attrs object (which shouldn't be necessary), but that
made no
> difference
> > either. The vl1l2 and vcnt files are still just header
information
> only
> > (see tarball). I increased the verbosity of grid_stat
to 10, but my
> eye
> > didn't catch anything offhand. Maybe yo!
> > u have a better idea what to look for. This whole
thing is super
> weird.
> > Do you have any other ideas I can try?
> >
> > Thanks,
> > Scott
> >
> >
> >
> > On 6/16/21, 4:13 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > I think I see what is happening with the U/V
fields. I checked
> the code
> > where it is checking the U/V fields and it expects
the name and
> level
> > of
> > the fcst and obs to match to make sure it it
processing the
> correct U/V
> > data. I noticed in the log output from your python
embedding
> call that
> > era5pl2xr has the name "u" while g5fcst2xr has
"U". I would
> suggest
> > modifying the python embedding script to ensure
that both of the
> > varname
> > values are the same case.
> >
> > If that doesn't work, I would suggest bumping up
the log
> verbosity for
> > GridStat (LOG_GRID_STAT_VERBOSITY) to see if there
are any clues
> in the
> > higher number log output. If you are still having
issues, you
> could
> > send me
> > the input files for each and I can play around
with the settings
> to see
> > what will work.
> >
> > Thanks,
> > George
> >
> > On Wed, Jun 16, 2021 at 1:07 PM George McCabe
<mccabe at ucar.edu>
> wrote:
> >
> > > Hi Scott,
> > >
> > > Regarding SeriesAnalysis -- there is an argument
to the tool
> called
> > > "-both" that can be used to set both the fcst
and obs lists to
> use
> > the same
> > > files. This use case runs SeriesAnalysis on
GridStat output:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IVziYkcTRDZxiPBggao6ZrZMeCqBPyiL0uYrB%2FmKnGw%3D&reserved=0
> > >
> > > This example uses
BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> the
> > input
> > > file path and specifies the fcst and obs input
fields
> separately:
> > >
> > > FCST_SERIES_ANALYSIS_VAR1_NAME =
FCST_precip_FULL
> > > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > OBS_SERIES_ANALYSIS_VAR1_NAME = OBS_precip_FULL
> > > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > >
> > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > pairs.nc
> > >
> > > Regarding not being able to produce wind output,
I'll take a
> look at
> > your
> > > files and see if I can narrow down the issue. I
will report
> back
> > with more
> > > information.
> > >
> > > Thanks,
> > > George
> > >
> > > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst, Scott
D.
> > (GSFC-610.1)[SCIENCE
> > > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> > >
> > >>
> > >> <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > >
> > >>
> > >> Hi George,
> > >>
> > >> Just as a follow up thought to problem below
regarding winds
> stats,
> > when
> > >> I look through the CNT and SL1L2 statistics
output, I see U
> and V
> > listed
> > >> with all respective stats. So this issue is not
a problem
> reading U
> > and V,
> > >> it is a problem recognizing they are vector
components. Are
> your
> > sure there
> > >> is not another flag or option to set either in
the
> PTHON_NUMPY/attrs
> > >> objects or another option in the metplus conf
file?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS
> > >> AND APPLICATIONS INC]"
<scott.d.rabenhorst at nasa.gov> wrote:
> > >>
> > >> Hi George,
> > >>
> > >> Thanks for answering my questions below. So
sorry for
> taking so
> > much
> > >> of your time lately. From your description, it
sounds like
> the best
> > option
> > >> for my needs is to use Stat-Analysis to
aggregate SL1L2/VL1L2
> data.
> > My goal
> > >> is to generate single-value statistics for each
variable/level
> > through all
> > >> valid times and spatial areas. These will be
plugged into a
> Green's
> > >> Function algorithm for our model.
> > >>
> > >> However, the Series-Analysis is also an
intriguing
> approach at
> > least
> > >> for looking at spatial errors. I guess Grid-
Stat could
> operate on
> > that data
> > >> at the end to generate total stats? I have not
tried using
> > Series-Analysis
> > >> yet, but what has confused me is the
documentation requires
> > separate "-fcst
> > >> file_1...file_n" and "-obs file_1...file_n". In
most examples
> I
> > have seen,
> > >> these are independent files. However, you make
it sound as
> though
> > it can
> > >> run on my *_pairs.nc files generated by Grid-
Stat I presume
> as
> > long as
> > >> RAW data is in there? In this case, I assume
the list of files
> > would be
> > >> identical to both forecast and observations?
Are there any
> examples
> > of
> > >> using matched-pairs for the Series-Analysis
online?
> > >>
> > >> Unfortunately, the larger problem right now
is that I
> still
> > can't get
> > >> Grid-Stat to process any my winds and output
VCNT or VL1L2
> data. I
> > tried
> > >> your suggestion below and added the options to
my
> > >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf file
> (please see
> > >> updated attached tarball). I tried running with
and without
> each of
> > these
> > >> options "is_u(v)_wind=TRUE" and
"is_grid_relative=FALSE" and
> those
> > ASCII
> > >> files never contains any VL1L2/VCNT data, nor
do the *.stat
> files
> > contain
> > >> any of those line_types. I also toggled
outputting those
> attribute
> > in my
> > >> PYTHON_NUMPY attrs object, but that didn't work
either. I
> couldn't
> > find any
> > >> combinations that would work. In an effort to
narrow the
> problem, I
> > >> copy/pasted the first references to the fcst
and obs fields
> in my
> > logfile
> > >> and ran my python read scripts stand-alone and
they produced
> the
> > expected
> > >> output with correct values:
> > >> scripts/read_era5pl2xr_v2.example.log
> > >> scripts/read_g5fcst2xr_v2.example.log
> > >>
> > >> Furthermore, I decided to test
PlotDataPlane since I
> recalled
> > you
> > >> saying that was used behind the scenes. It ran
just fine and
> > produced these
> > >> beautiful U 500hPa images without error:
> > >> PlotDataPlane_py-
embed_test/plot_data_plane/*.ps
> > >>
> > >> I have combed the log files for any errors
or warning, but
> > cannot
> > >> find any at all, or anything else that would
indicate a
> processing
> > error.
> > >> It is almost like there is some flag that has
to be set in
> order to
> > trigger
> > >> wind processing. Do you have any other ideas I
can try?
> > >>
> > >> Thanks,
> > >> Scott
> > >>
> > >>
> > >> On 6/3/21, 6:22 PM, "John Halley Gotway via
RT" <
> > met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> Hi Scott,
> > >>
> > >> Minna asked me to take a look at your
questions.
> > >>
> > >> For the second question, the answer is
no.
> Stat-Analysis
> > reads
> > >> the .stat
> > >> output files created by the MET tools.
It does not
> read the
> > >> gridded NetCDF
> > >> matched pairs files generated by the
Grid-Stat tool.
> > >> But I do have two recommendations to
make.
> > >>
> > >> 1. Grid-Stat computes statistics and
partial sums that
> > represent
> > >> a spatial
> > >> summary. You can configure Grid-Stat to
compute its
> output
> > over
> > >> whatever
> > >> spatial masking regions you'd like.
Stat-Analysis can
> > >> read/aggregate those
> > >> spatial summaries through time. The
SL1L2 line type
> contains
> > >> scalar L1 and
> > >> L2 norms. Those are written because
they allow you to
> easily
> > >> aggregate
> > >> stats over multiple runs. Many (but not
all) of the
> > continuous
> > >> statistics
> > >> (like ME, RMSE, MAE, ACC, and so on)
can be derived
> from the
> > >> SL1L2 line
> > >> type. So I'd recommend trying this type
of
> stat_analysis
> > job:
> > >>
> > >> -job aggregate_stat -line_type SL1L2
-out_line_type
> CNT -by
> > >> FCST_VAR,FCST_LEV,VX_MASK -out_stat
sl1l2_to_cnt.stat
> > >>
> > >> For each unique combination of
variable, level, and
> masking
> > >> region, it'll
> > >> aggregate all the SL1L2 lines it finds
in the input
> and
> > report the
> > >> corresponding continuous statistics.
> > >>
> > >> 2. Another option is running the NetCDF
matched pairs
> > produced by
> > >> Grid-Stat
> > >> through the Series-Analysis tool.
That'll compute
> stats
> > through
> > >> time
> > >> separately for each individual grid
point. The result
> with
> > be a
> > >> map of
> > >> statistics rather than a spatial
summary.
> > >>
> > >> For the first question, I see that
you're using python
> > embedding
> > >> and are
> > >> trying to compute vector partial sums
output. In
> general,
> > when
> > >> you request
> > >> an output line type, Grid-Stat will
create the
> corresponding
> > >> output file.
> > >> But if it's empty, that means that it
didn't have any
> input
> > data
> > >> suitable
> > >> for creating that output type. So
you've requested
> vector
> > stats,
> > >> but
> > >> Grid-Stat isn't computing any.
> > >>
> > >> Thanks for sending the sample output.
Looking in:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > >>
> > >> I do see several warning that you
should take a look
> at:
> > >> WARNING: python_dataplane() -> an error
occurred
> importing
> > module
> > >>
> > >>
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > >>
> > >> But I do also see that Grid-Stat is
processing the U
> and V
> > data
> > >> and
> > >> reporting output as scalar variables in
the CNT.txt
> file:
> > >>
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > >>
> > >> Now I haven't tested this on your data
to confirm
> that it'll
> > >> actually work,
> > >> but we want to set these flags in the
MET config file:
> > >> is_u_wind = true
> > >> is_v_wind = true
> > >>
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h5AAyqR2xi8LPCZcMvBJMrVq0MuRtJFmkNwFqKrLDKg%3D&reserved=0
> > >>
> > >> Checking in
> > >>
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > >> see that var 4 is for U and var 5 is
for V.
> > >> Add those via the METplus config file
by adding:
> > >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> > >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> > >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> > >>
> > >> Please give that a shot and let me know
how it goes.
> The
> > goal is
> > >> NOT having
> > >> any U and V show up in the CNT output
file... and
> instead
> > have
> > >> entries show
> > >> up in the VCNT output.
> > >>
> > >> One last detail... if you do want to
aggregate results
> > through
> > >> time, I'd
> > >> recommend writing the SL1L2 and VL1L2
line types to
> enable
> > that
> > >> aggregation
> > >> to be done later.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Thu, Jun 3, 2021 at 1:49 PM Minna
Win via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > >> >
> > >> >
> > >> > Hi Scott,
> > >> >
> > >> > It looks like you've set your MET
config file to
> correctly
> > >> identify which
> > >> > linetypes you'd like to see. I'm not
sure why you
> are
> > >> essentially getting
> > >> > empty output files. I'll need to
find someone
> who can
> > answer
> > >> your python
> > >> > embedding and grid-stat questions.
> > >> >
> > >> > Regards,
> > >> > Minna
> > >> > ---------------
> > >> > Minna Win
> > >> > Pronouns: she/her
> > >> > National Center for Atmospheric
Research
> > >> > Developmental Testbed Center
> > >> > Phone: 303-497-8423
> > >> > Fax: 303-497-8401
> > >> > ---------------
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Jun 3, 2021 at 12:04 PM
Rabenhorst, Scott D.
> > >> (GSFC-610.1)[SCIENCE
> > >> > SYSTEMS AND APPLICATIONS INC] via RT
<
> met_help at ucar.edu>
> > wrote:
> > >> >
> > >> > >
> > >> > > Thu Jun 03 12:04:14 2021: Request
100090 was
> acted upon.
> > >> > > Transaction: Ticket created by
> > scott.d.rabenhorst at nasa.gov
> > >> > > Queue: met_help
> > >> > > Subject: 2 questions regarding
how to
> generate
> > specific
> > >> stats
> > >> > > Owner: Nobody
> > >> > > Requestors:
scott.d.rabenhorst at nasa.gov
> > >> > > Status: new
> > >> > > Ticket <URL:
> > >>
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697051977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YloVQ%2BfOmCnoZy7DWZFlnyZJlPR5dxXbBQ71sqI641I%3D&reserved=0
> > >> >
> > >> > >
> > >> > >
> > >> > > Hi met support,
> > >> > >
> > >> > > I have 2 questions regarding how to
output
> specific
> > >> statistics for my
> > >> > > experiment (see files in my
attached tarball). I
> have
> > looked
> > >> through the
> > >> > > online documentation, but I have
had a hard time
> > finding a
> > >> direct
> > >> > > answer/solution – perhaps I’ve
simply missed
> something
> > >> obvious.
> > >> > >
> > >> > > QUESTION 1: how to output
wind/vector (VL1L2,
> VCNT)
> > >> statistics. I have
> > >> > > added:
> > >> > >
> > >> > > GRID_STAT_OUTPUT_FLAG_CNT =
BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VCNT =
BOTH
> > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 =
BOTH
> > >> > >
> > >> > > … to my
> > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> > >> file. I
> > >> > have
> > >> > > also added:
> > >> > >
> > >> > > is_u_wind : True
> > >> > > is_grid_relative: False
> > >> > >
> > >> > > … attributes to my PYTHON_NUMPY
object through
> python
> > >> embedding. However,
> > >> > > when I look at the output
*_vl1l2.txt files, the
> only
> > >> information is the
> > >> > > header lines. I have included 2
sample outputs
> from my
> > >> embedding scripts
> > >> > > when called alone
> (./scripts/read_*2xr_v2.example.log)
> > which
> > >> show all the
> > >> > > data is as expected. It seems like
something else
> needs
> > to be
> > >> activated
> > >> > > somewhere in the conf files, but I
can’t seem to
> find
> > the
> > >> correct switch.
> > >> > > If you have any ideas for other
things to try,
> please
> > let me
> > >> know.
> > >> > >
> > >> > > QUESTION 2: is there a way to use
stat_analysis to
> > >> recompute/aggregate
> > >> > > over my *_pairs.nc files containing
this
> information
> > >> (example ncdump in
> > >> > >
> > >> >
> > >>
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > >> > > and output CNT stats for the total
of all the
> matched
> > pair
> > >> data? I am
> > >> > aware
> > >> > > of the “-job summary -line_type CNT
-column RMSE
> -by
> > >> FCST_VAR,FCST_LEV
> > >> > > -dump_row” but this looks like it
only provides
> basic
> > >> statistics of all
> > >> > the
> > >> > > RMSE values in each *_cnt.txt
files, correct?
> What I
> > really
> > >> want is to
> > >> > > generate actual CNT data from all
matched pairs
> over all
> > >> times. I’ve
> > >> > tried
> > >> > > various combinations of aggregate
and
> aggregate_stat but
> > >> can’t seem to
> > >> > find
> > >> > > a combination that will work. Is
this possible?
> If so,
> > what
> > >> should my
> > >> > > stat_analysis job command look
like?
> > >> > >
> > >> > > Thanks so much for your time in
advance. I have
> asked a
> > lot
> > >> of questions
> > >> > > lately, but I think I’m almost
there. Any help
> you can
> > >> provide on filling
> > >> > > in my knowledge gaps would be very
much
> appreciated.
> > >> > >
> > >> > > Thanks,
> > >> > > Scott
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day.
Please do not feel
> > obliged to
> > > reply to this email outside of your normal
working hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please
do not feel
> obliged
> > to
> > reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do
not feel obliged
> to
> reply to this email outside of your normal working
hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Fri Jun 18 09:43:16 2021
Hi Scott,
Thanks for sending the new log file. We do have an FTP site that you
can
use to send us data. Information on how is on this webpage under the
"How
To Send Us Data" heading:
https://dtcenter.org/community-code/model-evaluation-tools-met/get-
help
Those files are quite large, but I would really only need the U and V
fields from each data set to debug why that logic is not working
properly.
I did also notice that there are some warnings in your log file saying
that
there is a time mismatch between the forecast and observation data for
some
of the other fields. It will still use the data, but the results may
not
reflect what you are expecting. The valid time that comes from the
forecast
script doesn't match the valid time passed into the script, so there
may be
something going on there. This isn't related to the U/V issue but I
thought
I would point it out.
{
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
DEBUG 3: Running user's python script
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: 0
DEBUG 4: plane max: 0.01865353266
DEBUG 4: valid time: *20191223_003000*
DEBUG 4: lead time: 273000
DEBUG 4: init time: 20191221_210000
DEBUG 4: accum time: 000000
DEBUG 4:
DEBUG 3: Reading forecast data for TTPRCP/Surface.
DEBUG 3: Running user's python script
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: 0
DEBUG 4: plane max: 0.02649974823
DEBUG 4: valid time: *20191223_000000*
DEBUG 4: lead time: 000000
DEBUG 4: init time: 20191223_000000
DEBUG 4: accum time: 010000
DEBUG 4:
DEBUG 3: Reading observation data for tp/Surface.
WARNING:
WARNING: process_scores() -> Forecast and observation valid times do
not
match 20191223_003000 != 20191223_000000 for TTPRCP/Surface versus
tp/Surface.
On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> I apologize, my last email sent prematurely for some weird reason.
As I
> was saying below, our forecast and observation files are very large.
I
> could ncks a subset, but that really wouldn't be the same file. I
suppose
> it would contain the same fields, but even a subset would be too
large for
> an email attachment.
>
> Regarding the output below, you are correct. I did not modify the
10m U/V
> winds (U10M, V10M, etc). I only changed the 3D pressure level U/V
winds to
> have the same name and attributes. I figured if those started
working I
> would propagate the changes to the surface python scripts too. So I
would
> recommend to only look at the output pertaining to pressure level
winds
> from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py scripts.
Those have
> the changes.
>
> Please let me know how to proceed with posting the
forecast/observation
> files if you think that will help. Do you have an ftp site?
>
> Thanks,
> Scott
>
>
>
>
> On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS
> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>
> Hi George,
>
> I have attached the log file - that is easy. However, I'm not
sure how
> to go about sending you the data. The experiment (forecast) 3D file
is 734G
> and the surface (single) level files are 619M each.
>
> On 6/17/21, 5:01 PM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> The VL1L2 flag should accept BOTH. I'm not familiar enough
with
> the U/V
> calculations to know if it needs a threshold but John didn't
> mention that
> it was a requirement in his first email. Could you send me a
set
> of files
> that are used as input for this use case? Also, the latest
log
> file would
> be helpful. I noticed that the name of the fields did
reflect the
> script
> name, so I'm curious to see if it was overridden properly in
your
> last run.
>
> | | | | | | | Dictionary Entry[0] ...
> | | | | | | | | Name = *name*
> | | | | | | | | Type = StringType
> | | | | | | | | *String Value =
>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> U10M 20191221_2100 20191223_0000"*
> | | | | | | | Dictionary Entry[1] ...
> | | | | | | | | Name = is_u_wind
> | | | | | | | | Type = BooleanType
> | | | | | | | | Boolean Value = true
> | | | | | | | Dictionary Entry[2] ...
> | | | | | | | | Name =
is_grid_relative
> | | | | | | | | Type = BooleanType
> | | | | | | | | Boolean Value =
false
>
> Thanks,
> George
>
> On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> >
> >
> > Hi George,
> >
> > I your suggestion and double checked for typos, but
> unfortunately, still
> > nothing different - just headers in the output files.
Could
> there be a bug
> > in something indirectly related to this, such as the BOTH
option
> does not
> > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V winds
need a
> threshold.
> > Just a WAG.
> >
> > ~Scott
> >
> >
> >
> > On 6/17/21, 1:28 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > You could try adding
> >
> > set_attr_name = "U";
> >
> > to the [FCST/OBS]_VAR4_OPTIONS and "V" for the VAR5
options
> to see if
> > that
> > helps? It may be using the script as the name, so
setting
> this would
> > override the input name that is used.
> >
> > FCST_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name
=
> "U";
> > OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name
=
> "U";
> > FCST_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name
=
> "V";
> > OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name
=
> "V";
> >
> > Let me know if that doesn't work and I can continue
digging.
> >
> > Thanks,
> > George
> >
> > On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >
> > >
> > > <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > >
> > >
> > > Hi George,
> > >
> > > Thanks so much for taking a look at the code once
again.
> You had
> > another
> > > great idea, and something rather easy to change via
python
> embedding
> > > compared to using ncatted/ncrename on so many files.
I
> modified my
> > scripts
> > > to ensure all my attrs={...} values matched for the
U and
> V fields,
> > but
> > > still not luck! I can't believe it. Since metplus
doesn't
> appear to
> > display
> > > the output from my python embedding scripts, I ran
both
> scripts
> > stand-alone
> > > by copying/pasting metplus' name field onto the
command
> line. The
> > output is
> > > in the scripts/read_*2xr_v2.log files. All
attributes
> match exactly
> > (I even
> > > hardcoded the init time just to be sure). I ran with
and
> without the
> > > updating the is_u/v_wind and is_grid_relative
attributes
> in the
> > python
> > > attrs object (which shouldn't be necessary), but
that made
> no
> > difference
> > > either. The vl1l2 and vcnt files are still just
header
> information
> > only
> > > (see tarball). I increased the verbosity of
grid_stat to
> 10, but my
> > eye
> > > didn't catch anything offhand. Maybe yo!
> > > u have a better idea what to look for. This whole
thing
> is super
> > weird.
> > > Do you have any other ideas I can try?
> > >
> > > Thanks,
> > > Scott
> > >
> > >
> > >
> > > On 6/16/21, 4:13 PM, "George McCabe via RT" <
> met_help at ucar.edu>
> > wrote:
> > >
> > > Hi Scott,
> > >
> > > I think I see what is happening with the U/V
fields. I
> checked
> > the code
> > > where it is checking the U/V fields and it
expects the
> name and
> > level
> > > of
> > > the fcst and obs to match to make sure it it
> processing the
> > correct U/V
> > > data. I noticed in the log output from your
python
> embedding
> > call that
> > > era5pl2xr has the name "u" while g5fcst2xr has
"U". I
> would
> > suggest
> > > modifying the python embedding script to ensure
that
> both of the
> > > varname
> > > values are the same case.
> > >
> > > If that doesn't work, I would suggest bumping up
the
> log
> > verbosity for
> > > GridStat (LOG_GRID_STAT_VERBOSITY) to see if
there are
> any clues
> > in the
> > > higher number log output. If you are still
having
> issues, you
> > could
> > > send me
> > > the input files for each and I can play around
with
> the settings
> > to see
> > > what will work.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Jun 16, 2021 at 1:07 PM George McCabe <
> mccabe at ucar.edu>
> > wrote:
> > >
> > > > Hi Scott,
> > > >
> > > > Regarding SeriesAnalysis -- there is an
argument to
> the tool
> > called
> > > > "-both" that can be used to set both the fcst
and
> obs lists to
> > use
> > > the same
> > > > files. This use case runs SeriesAnalysis on
GridStat
> output:
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IVziYkcTRDZxiPBggao6ZrZMeCqBPyiL0uYrB%2FmKnGw%3D&reserved=0
> > > >
> > > > This example uses
> BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> > the
> > > input
> > > > file path and specifies the fcst and obs input
fields
> > separately:
> > > >
> > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
FCST_precip_FULL
> > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > > >
> > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
OBS_precip_FULL
> > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > > >
> > > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > > >
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > > pairs.nc
> > > >
> > > > Regarding not being able to produce wind
output,
> I'll take a
> > look at
> > > your
> > > > files and see if I can narrow down the issue.
I will
> report
> > back
> > > with more
> > > > information.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst,
Scott D.
> > > (GSFC-610.1)[SCIENCE
> > > > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> > wrote:
> > > >
> > > >>
> > > >> <URL:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > > >
> > > >>
> > > >> Hi George,
> > > >>
> > > >> Just as a follow up thought to problem below
> regarding winds
> > stats,
> > > when
> > > >> I look through the CNT and SL1L2 statistics
output,
> I see U
> > and V
> > > listed
> > > >> with all respective stats. So this issue is
not a
> problem
> > reading U
> > > and V,
> > > >> it is a problem recognizing they are vector
> components. Are
> > your
> > > sure there
> > > >> is not another flag or option to set either
in the
> > PTHON_NUMPY/attrs
> > > >> objects or another option in the metplus conf
file?
> > > >>
> > > >> Thanks,
> > > >> Scott
> > > >>
> > > >>
> > > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > > SYSTEMS
> > > >> AND APPLICATIONS INC]"
<scott.d.rabenhorst at nasa.gov>
> wrote:
> > > >>
> > > >> Hi George,
> > > >>
> > > >> Thanks for answering my questions below.
So
> sorry for
> > taking so
> > > much
> > > >> of your time lately. From your description,
it
> sounds like
> > the best
> > > option
> > > >> for my needs is to use Stat-Analysis to
aggregate
> SL1L2/VL1L2
> > data.
> > > My goal
> > > >> is to generate single-value statistics for
each
> variable/level
> > > through all
> > > >> valid times and spatial areas. These will be
> plugged into a
> > Green's
> > > >> Function algorithm for our model.
> > > >>
> > > >> However, the Series-Analysis is also an
> intriguing
> > approach at
> > > least
> > > >> for looking at spatial errors. I guess Grid-
Stat
> could
> > operate on
> > > that data
> > > >> at the end to generate total stats? I have
not
> tried using
> > > Series-Analysis
> > > >> yet, but what has confused me is the
documentation
> requires
> > > separate "-fcst
> > > >> file_1...file_n" and "-obs file_1...file_n".
In
> most examples
> > I
> > > have seen,
> > > >> these are independent files. However, you
make it
> sound as
> > though
> > > it can
> > > >> run on my *_pairs.nc files generated by Grid-
Stat
> I presume
> > as
> > > long as
> > > >> RAW data is in there? In this case, I assume
the
> list of files
> > > would be
> > > >> identical to both forecast and observations?
Are
> there any
> > examples
> > > of
> > > >> using matched-pairs for the Series-Analysis
online?
> > > >>
> > > >> Unfortunately, the larger problem right
now is
> that I
> > still
> > > can't get
> > > >> Grid-Stat to process any my winds and output
VCNT
> or VL1L2
> > data. I
> > > tried
> > > >> your suggestion below and added the options
to my
> > > >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
> file
> > (please see
> > > >> updated attached tarball). I tried running
with and
> without
> > each of
> > > these
> > > >> options "is_u(v)_wind=TRUE" and
> "is_grid_relative=FALSE" and
> > those
> > > ASCII
> > > >> files never contains any VL1L2/VCNT data, nor
do
> the *.stat
> > files
> > > contain
> > > >> any of those line_types. I also toggled
outputting
> those
> > attribute
> > > in my
> > > >> PYTHON_NUMPY attrs object, but that didn't
work
> either. I
> > couldn't
> > > find any
> > > >> combinations that would work. In an effort to
> narrow the
> > problem, I
> > > >> copy/pasted the first references to the fcst
and
> obs fields
> > in my
> > > logfile
> > > >> and ran my python read scripts stand-alone
and they
> produced
> > the
> > > expected
> > > >> output with correct values:
> > > >> scripts/read_era5pl2xr_v2.example.log
> > > >> scripts/read_g5fcst2xr_v2.example.log
> > > >>
> > > >> Furthermore, I decided to test
PlotDataPlane
> since I
> > recalled
> > > you
> > > >> saying that was used behind the scenes. It
ran just
> fine and
> > > produced these
> > > >> beautiful U 500hPa images without error:
> > > >> PlotDataPlane_py-
embed_test/plot_data_plane/*.ps
> > > >>
> > > >> I have combed the log files for any
errors or
> warning, but
> > > cannot
> > > >> find any at all, or anything else that would
> indicate a
> > processing
> > > error.
> > > >> It is almost like there is some flag that has
to be
> set in
> > order to
> > > trigger
> > > >> wind processing. Do you have any other ideas
I can
> try?
> > > >>
> > > >> Thanks,
> > > >> Scott
> > > >>
> > > >>
> > > >> On 6/3/21, 6:22 PM, "John Halley Gotway
via RT"
> <
> > > met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> Hi Scott,
> > > >>
> > > >> Minna asked me to take a look at your
> questions.
> > > >>
> > > >> For the second question, the answer
is no.
> > Stat-Analysis
> > > reads
> > > >> the .stat
> > > >> output files created by the MET
tools. It
> does not
> > read the
> > > >> gridded NetCDF
> > > >> matched pairs files generated by the
> Grid-Stat tool.
> > > >> But I do have two recommendations to
make.
> > > >>
> > > >> 1. Grid-Stat computes statistics and
> partial sums that
> > > represent
> > > >> a spatial
> > > >> summary. You can configure Grid-Stat
to
> compute its
> > output
> > > over
> > > >> whatever
> > > >> spatial masking regions you'd like.
> Stat-Analysis can
> > > >> read/aggregate those
> > > >> spatial summaries through time. The
SL1L2
> line type
> > contains
> > > >> scalar L1 and
> > > >> L2 norms. Those are written because
they
> allow you to
> > easily
> > > >> aggregate
> > > >> stats over multiple runs. Many (but
not
> all) of the
> > > continuous
> > > >> statistics
> > > >> (like ME, RMSE, MAE, ACC, and so on)
can be
> derived
> > from the
> > > >> SL1L2 line
> > > >> type. So I'd recommend trying this
type of
> > stat_analysis
> > > job:
> > > >>
> > > >> -job aggregate_stat -line_type SL1L2
> -out_line_type
> > CNT -by
> > > >> FCST_VAR,FCST_LEV,VX_MASK -out_stat
> sl1l2_to_cnt.stat
> > > >>
> > > >> For each unique combination of
variable,
> level, and
> > masking
> > > >> region, it'll
> > > >> aggregate all the SL1L2 lines it
finds in
> the input
> > and
> > > report the
> > > >> corresponding continuous statistics.
> > > >>
> > > >> 2. Another option is running the
NetCDF
> matched pairs
> > > produced by
> > > >> Grid-Stat
> > > >> through the Series-Analysis tool.
That'll
> compute
> > stats
> > > through
> > > >> time
> > > >> separately for each individual grid
point.
> The result
> > with
> > > be a
> > > >> map of
> > > >> statistics rather than a spatial
summary.
> > > >>
> > > >> For the first question, I see that
you're
> using python
> > > embedding
> > > >> and are
> > > >> trying to compute vector partial sums
> output. In
> > general,
> > > when
> > > >> you request
> > > >> an output line type, Grid-Stat will
create
> the
> > corresponding
> > > >> output file.
> > > >> But if it's empty, that means that it
> didn't have any
> > input
> > > data
> > > >> suitable
> > > >> for creating that output type. So
you've
> requested
> > vector
> > > stats,
> > > >> but
> > > >> Grid-Stat isn't computing any.
> > > >>
> > > >> Thanks for sending the sample output.
> Looking in:
> > > >>
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > > >>
> > > >> I do see several warning that you
should
> take a look
> > at:
> > > >> WARNING: python_dataplane() -> an
error
> occurred
> > importing
> > > module
> > > >>
> > > >>
> > >
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > > >>
> > > >> But I do also see that Grid-Stat is
> processing the U
> > and V
> > > data
> > > >> and
> > > >> reporting output as scalar variables
in the
> CNT.txt
> > file:
> > > >>
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > > >>
> > > >> Now I haven't tested this on your
data to
> confirm
> > that it'll
> > > >> actually work,
> > > >> but we want to set these flags in the
MET
> config file:
> > > >> is_u_wind = true
> > > >> is_v_wind = true
> > > >>
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h5AAyqR2xi8LPCZcMvBJMrVq0MuRtJFmkNwFqKrLDKg%3D&reserved=0
> > > >>
> > > >> Checking in
> > > >>
> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > > >> see that var 4 is for U and var 5 is
for V.
> > > >> Add those via the METplus config file
by
> adding:
> > > >> FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> > > >> OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> > > >> FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> > > >> OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> > > >>
> > > >> Please give that a shot and let me
know how
> it goes.
> > The
> > > goal is
> > > >> NOT having
> > > >> any U and V show up in the CNT output
> file... and
> > instead
> > > have
> > > >> entries show
> > > >> up in the VCNT output.
> > > >>
> > > >> One last detail... if you do want to
> aggregate results
> > > through
> > > >> time, I'd
> > > >> recommend writing the SL1L2 and VL1L2
line
> types to
> > enable
> > > that
> > > >> aggregation
> > > >> to be done later.
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >> On Thu, Jun 3, 2021 at 1:49 PM Minna
Win
> via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697042020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5KXo21rU%2B%2FVW9FXoA%2F%2B7McqC4dkwKu7peK80HTinHJY%3D&reserved=0
> > > >> >
> > > >> >
> > > >> > Hi Scott,
> > > >> >
> > > >> > It looks like you've set your MET
config
> file to
> > correctly
> > > >> identify which
> > > >> > linetypes you'd like to see. I'm
not sure
> why you
> > are
> > > >> essentially getting
> > > >> > empty output files. I'll need to
find
> someone
> > who can
> > > answer
> > > >> your python
> > > >> > embedding and grid-stat questions.
> > > >> >
> > > >> > Regards,
> > > >> > Minna
> > > >> > ---------------
> > > >> > Minna Win
> > > >> > Pronouns: she/her
> > > >> > National Center for Atmospheric
Research
> > > >> > Developmental Testbed Center
> > > >> > Phone: 303-497-8423
> > > >> > Fax: 303-497-8401
> > > >> > ---------------
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Jun 3, 2021 at 12:04 PM
> Rabenhorst, Scott D.
> > > >> (GSFC-610.1)[SCIENCE
> > > >> > SYSTEMS AND APPLICATIONS INC] via
RT <
> > met_help at ucar.edu>
> > > wrote:
> > > >> >
> > > >> > >
> > > >> > > Thu Jun 03 12:04:14 2021: Request
> 100090 was
> > acted upon.
> > > >> > > Transaction: Ticket created by
> > > scott.d.rabenhorst at nasa.gov
> > > >> > > Queue: met_help
> > > >> > > Subject: 2 questions
regarding how
> to
> > generate
> > > specific
> > > >> stats
> > > >> > > Owner: Nobody
> > > >> > > Requestors:
> scott.d.rabenhorst at nasa.gov
> > > >> > > Status: new
> > > >> > > Ticket <URL:
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Ce7bd34434e72485415ed08d931d30738%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637595604697051977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YloVQ%2BfOmCnoZy7DWZFlnyZJlPR5dxXbBQ71sqI641I%3D&reserved=0
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > > Hi met support,
> > > >> > >
> > > >> > > I have 2 questions regarding how
to
> output
> > specific
> > > >> statistics for my
> > > >> > > experiment (see files in my
attached
> tarball). I
> > have
> > > looked
> > > >> through the
> > > >> > > online documentation, but I have
had a
> hard time
> > > finding a
> > > >> direct
> > > >> > > answer/solution – perhaps I’ve
simply
> missed
> > something
> > > >> obvious.
> > > >> > >
> > > >> > > QUESTION 1: how to output
wind/vector
> (VL1L2,
> > VCNT)
> > > >> statistics. I have
> > > >> > > added:
> > > >> > >
> > > >> > > GRID_STAT_OUTPUT_FLAG_CNT =
BOTH
> > > >> > > GRID_STAT_OUTPUT_FLAG_VCNT =
BOTH
> > > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2 =
BOTH
> > > >> > >
> > > >> > > … to my
> > > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
> > > >> file. I
> > > >> > have
> > > >> > > also added:
> > > >> > >
> > > >> > > is_u_wind : True
> > > >> > > is_grid_relative: False
> > > >> > >
> > > >> > > … attributes to my PYTHON_NUMPY
object
> through
> > python
> > > >> embedding. However,
> > > >> > > when I look at the output
*_vl1l2.txt
> files, the
> > only
> > > >> information is the
> > > >> > > header lines. I have included 2
sample
> outputs
> > from my
> > > >> embedding scripts
> > > >> > > when called alone
> > (./scripts/read_*2xr_v2.example.log)
> > > which
> > > >> show all the
> > > >> > > data is as expected. It seems
like
> something else
> > needs
> > > to be
> > > >> activated
> > > >> > > somewhere in the conf files, but
I
> can’t seem to
> > find
> > > the
> > > >> correct switch.
> > > >> > > If you have any ideas for other
things
> to try,
> > please
> > > let me
> > > >> know.
> > > >> > >
> > > >> > > QUESTION 2: is there a way to use
> stat_analysis to
> > > >> recompute/aggregate
> > > >> > > over my *_pairs.nc files
containing
> this
> > information
> > > >> (example ncdump in
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > > >> > > and output CNT stats for the
total of
> all the
> > matched
> > > pair
> > > >> data? I am
> > > >> > aware
> > > >> > > of the “-job summary -line_type
CNT
> -column RMSE
> > -by
> > > >> FCST_VAR,FCST_LEV
> > > >> > > -dump_row” but this looks like it
only
> provides
> > basic
> > > >> statistics of all
> > > >> > the
> > > >> > > RMSE values in each *_cnt.txt
files,
> correct?
> > What I
> > > really
> > > >> want is to
> > > >> > > generate actual CNT data from all
> matched pairs
> > over all
> > > >> times. I’ve
> > > >> > tried
> > > >> > > various combinations of aggregate
and
> > aggregate_stat but
> > > >> can’t seem to
> > > >> > find
> > > >> > > a combination that will work. Is
this
> possible?
> > If so,
> > > what
> > > >> should my
> > > >> > > stat_analysis job command look
like?
> > > >> > >
> > > >> > > Thanks so much for your time in
> advance. I have
> > asked a
> > > lot
> > > >> of questions
> > > >> > > lately, but I think I’m almost
there.
> Any help
> > you can
> > > >> provide on filling
> > > >> > > in my knowledge gaps would be
very much
> > appreciated.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Scott
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day.
Please
> do not feel
> > > obliged to
> > > > reply to this email outside of your normal
working
> hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day.
Please do
> not feel
> > obliged
> > > to
> > > reply to this email outside of your normal
working
> hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do
not
> feel obliged
> > to
> > reply to this email outside of your normal working
hours.
> >
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not
feel
> obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100090] 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Mon Jun 21 10:16:29 2021
Hi George,
Thanks for getting back in touch with me so soon. Sorry for the delay.
I'm back at work today.
This morning I simplified my grid_stat config file so it is only
reading a few select fields on pressure levels. Hopefully, this will
avoid the complications of the surface data with time offsets and all
the other non-conventional comparisons with masking that you have
noticed. This latest run demonstrates the problem - still no vector
stats. I uploaded 3 files to the ucar ftp site:
1. geosgcm_pres.20191223_0000z.nc4.gz (forecast data time sliced to
index 26 the observation comparison time)
2. era5_pl-allvar.20191223_00z.nc4.gz (observation data)
3. expfiles.5.tar.gz (logs/scripts from my latest run)
Just as a reminder, my python read scripts search through a directory
hierarchy to locate the proper forecast and observation files for
comparison. This will not work for you as is. You will have to change
and hard-code the file paths to make this work on your side.
Please let me know if you have any issues or need any further
information. I am eager and hopeful we can finally solve this issue.
Looking forward to hearing from you.
Thanks,
Scott
On 6/18/21, 11:43 AM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
Thanks for sending the new log file. We do have an FTP site that
you can
use to send us data. Information on how is on this webpage under
the "How
To Send Us Data" heading:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fget-
help&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074114075%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yQDC8isdwK7buTn0M8Hk%2F0Bb0QoI7Y6lw81ppYhBKuw%3D&reserved=0
Those files are quite large, but I would really only need the U
and V
fields from each data set to debug why that logic is not working
properly.
I did also notice that there are some warnings in your log file
saying that
there is a time mismatch between the forecast and observation data
for some
of the other fields. It will still use the data, but the results
may not
reflect what you are expecting. The valid time that comes from the
forecast
script doesn't match the valid time passed into the script, so
there may be
something going on there. This isn't related to the U/V issue but
I thought
I would point it out.
{
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
DEBUG 3: Running user's python script
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: 0
DEBUG 4: plane max: 0.01865353266
DEBUG 4: valid time: *20191223_003000*
DEBUG 4: lead time: 273000
DEBUG 4: init time: 20191221_210000
DEBUG 4: accum time: 000000
DEBUG 4:
DEBUG 3: Reading forecast data for TTPRCP/Surface.
DEBUG 3: Running user's python script
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: 0
DEBUG 4: plane max: 0.02649974823
DEBUG 4: valid time: *20191223_000000*
DEBUG 4: lead time: 000000
DEBUG 4: init time: 20191223_000000
DEBUG 4: accum time: 010000
DEBUG 4:
DEBUG 3: Reading observation data for tp/Surface.
WARNING:
WARNING: process_scores() -> Forecast and observation valid times
do not
match 20191223_003000 != 20191223_000000 for TTPRCP/Surface versus
tp/Surface.
On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>
>
> Hi George,
>
> I apologize, my last email sent prematurely for some weird
reason. As I
> was saying below, our forecast and observation files are very
large. I
> could ncks a subset, but that really wouldn't be the same file.
I suppose
> it would contain the same fields, but even a subset would be too
large for
> an email attachment.
>
> Regarding the output below, you are correct. I did not modify
the 10m U/V
> winds (U10M, V10M, etc). I only changed the 3D pressure level
U/V winds to
> have the same name and attributes. I figured if those started
working I
> would propagate the changes to the surface python scripts too.
So I would
> recommend to only look at the output pertaining to pressure
level winds
> from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py scripts.
Those have
> the changes.
>
> Please let me know how to proceed with posting the
forecast/observation
> files if you think that will help. Do you have an ftp site?
>
> Thanks,
> Scott
>
>
>
>
> On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE
SYSTEMS
> AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>
> Hi George,
>
> I have attached the log file - that is easy. However, I'm
not sure how
> to go about sending you the data. The experiment (forecast) 3D
file is 734G
> and the surface (single) level files are 619M each.
>
> On 6/17/21, 5:01 PM, "George McCabe via RT"
<met_help at ucar.edu> wrote:
>
> Hi Scott,
>
> The VL1L2 flag should accept BOTH. I'm not familiar
enough with
> the U/V
> calculations to know if it needs a threshold but John
didn't
> mention that
> it was a requirement in his first email. Could you send
me a set
> of files
> that are used as input for this use case? Also, the
latest log
> file would
> be helpful. I noticed that the name of the fields did
reflect the
> script
> name, so I'm curious to see if it was overridden
properly in your
> last run.
>
> | | | | | | | Dictionary Entry[0]
...
> | | | | | | | | Name = *name*
> | | | | | | | | Type =
StringType
> | | | | | | | | *String Value =
>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> U10M 20191221_2100 20191223_0000"*
> | | | | | | | Dictionary Entry[1]
...
> | | | | | | | | Name =
is_u_wind
> | | | | | | | | Type =
BooleanType
> | | | | | | | | Boolean Value =
true
> | | | | | | | Dictionary Entry[2]
...
> | | | | | | | | Name =
is_grid_relative
> | | | | | | | | Type =
BooleanType
> | | | | | | | | Boolean Value =
false
>
> Thanks,
> George
>
> On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> >
> >
> > Hi George,
> >
> > I your suggestion and double checked for typos, but
> unfortunately, still
> > nothing different - just headers in the output files.
Could
> there be a bug
> > in something indirectly related to this, such as the
BOTH option
> does not
> > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V
winds need a
> threshold.
> > Just a WAG.
> >
> > ~Scott
> >
> >
> >
> > On 6/17/21, 1:28 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > You could try adding
> >
> > set_attr_name = "U";
> >
> > to the [FCST/OBS]_VAR4_OPTIONS and "V" for the
VAR5 options
> to see if
> > that
> > helps? It may be using the script as the name, so
setting
> this would
> > override the input name that is used.
> >
> > FCST_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name =
> "U";
> > OBS_VAR4_OPTIONS = is_u_wind = TRUE; set_attr_name
=
> "U";
> > FCST_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name =
> "V";
> > OBS_VAR5_OPTIONS = is_v_wind = TRUE; set_attr_name
=
> "V";
> >
> > Let me know if that doesn't work and I can
continue digging.
> >
> > Thanks,
> > George
> >
> > On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst, Scott
D.
> > (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >
> > >
> > > <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > >
> > >
> > > Hi George,
> > >
> > > Thanks so much for taking a look at the code
once again.
> You had
> > another
> > > great idea, and something rather easy to change
via python
> embedding
> > > compared to using ncatted/ncrename on so many
files. I
> modified my
> > scripts
> > > to ensure all my attrs={...} values matched for
the U and
> V fields,
> > but
> > > still not luck! I can't believe it. Since
metplus doesn't
> appear to
> > display
> > > the output from my python embedding scripts, I
ran both
> scripts
> > stand-alone
> > > by copying/pasting metplus' name field onto the
command
> line. The
> > output is
> > > in the scripts/read_*2xr_v2.log files. All
attributes
> match exactly
> > (I even
> > > hardcoded the init time just to be sure). I ran
with and
> without the
> > > updating the is_u/v_wind and is_grid_relative
attributes
> in the
> > python
> > > attrs object (which shouldn't be necessary), but
that made
> no
> > difference
> > > either. The vl1l2 and vcnt files are still just
header
> information
> > only
> > > (see tarball). I increased the verbosity of
grid_stat to
> 10, but my
> > eye
> > > didn't catch anything offhand. Maybe yo!
> > > u have a better idea what to look for. This
whole thing
> is super
> > weird.
> > > Do you have any other ideas I can try?
> > >
> > > Thanks,
> > > Scott
> > >
> > >
> > >
> > > On 6/16/21, 4:13 PM, "George McCabe via RT" <
> met_help at ucar.edu>
> > wrote:
> > >
> > > Hi Scott,
> > >
> > > I think I see what is happening with the U/V
fields. I
> checked
> > the code
> > > where it is checking the U/V fields and it
expects the
> name and
> > level
> > > of
> > > the fcst and obs to match to make sure it it
> processing the
> > correct U/V
> > > data. I noticed in the log output from your
python
> embedding
> > call that
> > > era5pl2xr has the name "u" while g5fcst2xr
has "U". I
> would
> > suggest
> > > modifying the python embedding script to
ensure that
> both of the
> > > varname
> > > values are the same case.
> > >
> > > If that doesn't work, I would suggest
bumping up the
> log
> > verbosity for
> > > GridStat (LOG_GRID_STAT_VERBOSITY) to see if
there are
> any clues
> > in the
> > > higher number log output. If you are still
having
> issues, you
> > could
> > > send me
> > > the input files for each and I can play
around with
> the settings
> > to see
> > > what will work.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Jun 16, 2021 at 1:07 PM George
McCabe <
> mccabe at ucar.edu>
> > wrote:
> > >
> > > > Hi Scott,
> > > >
> > > > Regarding SeriesAnalysis -- there is an
argument to
> the tool
> > called
> > > > "-both" that can be used to set both the
fcst and
> obs lists to
> > use
> > > the same
> > > > files. This use case runs SeriesAnalysis
on GridStat
> output:
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CaEvxnPTd%2FXCcR5PZGjT4reGDAdbN0110%2BSAqw%2BXXQI%3D&reserved=0
> > > >
> > > > This example uses
> BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> > the
> > > input
> > > > file path and specifies the fcst and obs
input fields
> > separately:
> > > >
> > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
FCST_precip_FULL
> > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > > >
> > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
OBS_precip_FULL
> > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS = "(*,*)"
> > > >
> > > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > > >
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > > pairs.nc
> > > >
> > > > Regarding not being able to produce wind
output,
> I'll take a
> > look at
> > > your
> > > > files and see if I can narrow down the
issue. I will
> report
> > back
> > > with more
> > > > information.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Mon, Jun 7, 2021 at 9:09 AM Rabenhorst,
Scott D.
> > > (GSFC-610.1)[SCIENCE
> > > > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> > wrote:
> > > >
> > > >>
> > > >> <URL:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > >
> > > >>
> > > >> Hi George,
> > > >>
> > > >> Just as a follow up thought to problem
below
> regarding winds
> > stats,
> > > when
> > > >> I look through the CNT and SL1L2
statistics output,
> I see U
> > and V
> > > listed
> > > >> with all respective stats. So this issue
is not a
> problem
> > reading U
> > > and V,
> > > >> it is a problem recognizing they are
vector
> components. Are
> > your
> > > sure there
> > > >> is not another flag or option to set
either in the
> > PTHON_NUMPY/attrs
> > > >> objects or another option in the metplus
conf file?
> > > >>
> > > >> Thanks,
> > > >> Scott
> > > >>
> > > >>
> > > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott
D.
> > (GSFC-610.1)[SCIENCE
> > > SYSTEMS
> > > >> AND APPLICATIONS INC]"
<scott.d.rabenhorst at nasa.gov>
> wrote:
> > > >>
> > > >> Hi George,
> > > >>
> > > >> Thanks for answering my questions
below. So
> sorry for
> > taking so
> > > much
> > > >> of your time lately. From your
description, it
> sounds like
> > the best
> > > option
> > > >> for my needs is to use Stat-Analysis to
aggregate
> SL1L2/VL1L2
> > data.
> > > My goal
> > > >> is to generate single-value statistics
for each
> variable/level
> > > through all
> > > >> valid times and spatial areas. These will
be
> plugged into a
> > Green's
> > > >> Function algorithm for our model.
> > > >>
> > > >> However, the Series-Analysis is also
an
> intriguing
> > approach at
> > > least
> > > >> for looking at spatial errors. I guess
Grid-Stat
> could
> > operate on
> > > that data
> > > >> at the end to generate total stats? I
have not
> tried using
> > > Series-Analysis
> > > >> yet, but what has confused me is the
documentation
> requires
> > > separate "-fcst
> > > >> file_1...file_n" and "-obs
file_1...file_n". In
> most examples
> > I
> > > have seen,
> > > >> these are independent files. However, you
make it
> sound as
> > though
> > > it can
> > > >> run on my *_pairs.nc files generated by
Grid-Stat
> I presume
> > as
> > > long as
> > > >> RAW data is in there? In this case, I
assume the
> list of files
> > > would be
> > > >> identical to both forecast and
observations? Are
> there any
> > examples
> > > of
> > > >> using matched-pairs for the Series-
Analysis online?
> > > >>
> > > >> Unfortunately, the larger problem
right now is
> that I
> > still
> > > can't get
> > > >> Grid-Stat to process any my winds and
output VCNT
> or VL1L2
> > data. I
> > > tried
> > > >> your suggestion below and added the
options to my
> > > >>
RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> file
> > (please see
> > > >> updated attached tarball). I tried
running with and
> without
> > each of
> > > these
> > > >> options "is_u(v)_wind=TRUE" and
> "is_grid_relative=FALSE" and
> > those
> > > ASCII
> > > >> files never contains any VL1L2/VCNT data,
nor do
> the *.stat
> > files
> > > contain
> > > >> any of those line_types. I also toggled
outputting
> those
> > attribute
> > > in my
> > > >> PYTHON_NUMPY attrs object, but that
didn't work
> either. I
> > couldn't
> > > find any
> > > >> combinations that would work. In an
effort to
> narrow the
> > problem, I
> > > >> copy/pasted the first references to the
fcst and
> obs fields
> > in my
> > > logfile
> > > >> and ran my python read scripts stand-
alone and they
> produced
> > the
> > > expected
> > > >> output with correct values:
> > > >> scripts/read_era5pl2xr_v2.example.log
> > > >> scripts/read_g5fcst2xr_v2.example.log
> > > >>
> > > >> Furthermore, I decided to test
PlotDataPlane
> since I
> > recalled
> > > you
> > > >> saying that was used behind the scenes.
It ran just
> fine and
> > > produced these
> > > >> beautiful U 500hPa images without error:
> > > >> PlotDataPlane_py-
embed_test/plot_data_plane/*.ps
> > > >>
> > > >> I have combed the log files for any
errors or
> warning, but
> > > cannot
> > > >> find any at all, or anything else that
would
> indicate a
> > processing
> > > error.
> > > >> It is almost like there is some flag that
has to be
> set in
> > order to
> > > trigger
> > > >> wind processing. Do you have any other
ideas I can
> try?
> > > >>
> > > >> Thanks,
> > > >> Scott
> > > >>
> > > >>
> > > >> On 6/3/21, 6:22 PM, "John Halley
Gotway via RT"
> <
> > > met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> Hi Scott,
> > > >>
> > > >> Minna asked me to take a look at
your
> questions.
> > > >>
> > > >> For the second question, the
answer is no.
> > Stat-Analysis
> > > reads
> > > >> the .stat
> > > >> output files created by the MET
tools. It
> does not
> > read the
> > > >> gridded NetCDF
> > > >> matched pairs files generated by
the
> Grid-Stat tool.
> > > >> But I do have two recommendations
to make.
> > > >>
> > > >> 1. Grid-Stat computes statistics
and
> partial sums that
> > > represent
> > > >> a spatial
> > > >> summary. You can configure Grid-
Stat to
> compute its
> > output
> > > over
> > > >> whatever
> > > >> spatial masking regions you'd
like.
> Stat-Analysis can
> > > >> read/aggregate those
> > > >> spatial summaries through time.
The SL1L2
> line type
> > contains
> > > >> scalar L1 and
> > > >> L2 norms. Those are written
because they
> allow you to
> > easily
> > > >> aggregate
> > > >> stats over multiple runs. Many
(but not
> all) of the
> > > continuous
> > > >> statistics
> > > >> (like ME, RMSE, MAE, ACC, and so
on) can be
> derived
> > from the
> > > >> SL1L2 line
> > > >> type. So I'd recommend trying
this type of
> > stat_analysis
> > > job:
> > > >>
> > > >> -job aggregate_stat -line_type
SL1L2
> -out_line_type
> > CNT -by
> > > >> FCST_VAR,FCST_LEV,VX_MASK
-out_stat
> sl1l2_to_cnt.stat
> > > >>
> > > >> For each unique combination of
variable,
> level, and
> > masking
> > > >> region, it'll
> > > >> aggregate all the SL1L2 lines it
finds in
> the input
> > and
> > > report the
> > > >> corresponding continuous
statistics.
> > > >>
> > > >> 2. Another option is running the
NetCDF
> matched pairs
> > > produced by
> > > >> Grid-Stat
> > > >> through the Series-Analysis tool.
That'll
> compute
> > stats
> > > through
> > > >> time
> > > >> separately for each individual
grid point.
> The result
> > with
> > > be a
> > > >> map of
> > > >> statistics rather than a spatial
summary.
> > > >>
> > > >> For the first question, I see
that you're
> using python
> > > embedding
> > > >> and are
> > > >> trying to compute vector partial
sums
> output. In
> > general,
> > > when
> > > >> you request
> > > >> an output line type, Grid-Stat
will create
> the
> > corresponding
> > > >> output file.
> > > >> But if it's empty, that means
that it
> didn't have any
> > input
> > > data
> > > >> suitable
> > > >> for creating that output type. So
you've
> requested
> > vector
> > > stats,
> > > >> but
> > > >> Grid-Stat isn't computing any.
> > > >>
> > > >> Thanks for sending the sample
output.
> Looking in:
> > > >>
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > > >>
> > > >> I do see several warning that you
should
> take a look
> > at:
> > > >> WARNING: python_dataplane() -> an
error
> occurred
> > importing
> > > module
> > > >>
> > > >>
> > >
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > > >>
> > > >> But I do also see that Grid-Stat
is
> processing the U
> > and V
> > > data
> > > >> and
> > > >> reporting output as scalar
variables in the
> CNT.txt
> > file:
> > > >>
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > > >>
> > > >> Now I haven't tested this on your
data to
> confirm
> > that it'll
> > > >> actually work,
> > > >> but we want to set these flags in
the MET
> config file:
> > > >> is_u_wind = true
> > > >> is_v_wind = true
> > > >>
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VnwsgB2i2RQwVAIUEfnO3xZLhClxh%2Bqb0q7b78HQ0YU%3D&reserved=0
> > > >>
> > > >> Checking in
> > > >>
> RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > > >> see that var 4 is for U and var 5
is for V.
> > > >> Add those via the METplus config
file by
> adding:
> > > >> FCST_VAR4_OPTIONS = is_u_wind =
TRUE;
> > > >> OBS_VAR4_OPTIONS = is_u_wind =
TRUE;
> > > >> FCST_VAR5_OPTIONS = is_v_wind =
TRUE;
> > > >> OBS_VAR5_OPTIONS = is_v_wind =
TRUE;
> > > >>
> > > >> Please give that a shot and let
me know how
> it goes.
> > The
> > > goal is
> > > >> NOT having
> > > >> any U and V show up in the CNT
output
> file... and
> > instead
> > > have
> > > >> entries show
> > > >> up in the VCNT output.
> > > >>
> > > >> One last detail... if you do want
to
> aggregate results
> > > through
> > > >> time, I'd
> > > >> recommend writing the SL1L2 and
VL1L2 line
> types to
> > enable
> > > that
> > > >> aggregation
> > > >> to be done later.
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >> On Thu, Jun 3, 2021 at 1:49 PM
Minna Win
> via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > >> >
> > > >> >
> > > >> > Hi Scott,
> > > >> >
> > > >> > It looks like you've set your
MET config
> file to
> > correctly
> > > >> identify which
> > > >> > linetypes you'd like to see.
I'm not sure
> why you
> > are
> > > >> essentially getting
> > > >> > empty output files. I'll
need to find
> someone
> > who can
> > > answer
> > > >> your python
> > > >> > embedding and grid-stat
questions.
> > > >> >
> > > >> > Regards,
> > > >> > Minna
> > > >> > ---------------
> > > >> > Minna Win
> > > >> > Pronouns: she/her
> > > >> > National Center for Atmospheric
Research
> > > >> > Developmental Testbed Center
> > > >> > Phone: 303-497-8423
> > > >> > Fax: 303-497-8401
> > > >> > ---------------
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Jun 3, 2021 at 12:04 PM
> Rabenhorst, Scott D.
> > > >> (GSFC-610.1)[SCIENCE
> > > >> > SYSTEMS AND APPLICATIONS INC]
via RT <
> > met_help at ucar.edu>
> > > wrote:
> > > >> >
> > > >> > >
> > > >> > > Thu Jun 03 12:04:14 2021:
Request
> 100090 was
> > acted upon.
> > > >> > > Transaction: Ticket created
by
> > > scott.d.rabenhorst at nasa.gov
> > > >> > > Queue: met_help
> > > >> > > Subject: 2 questions
regarding how
> to
> > generate
> > > specific
> > > >> stats
> > > >> > > Owner: Nobody
> > > >> > > Requestors:
> scott.d.rabenhorst at nasa.gov
> > > >> > > Status: new
> > > >> > > Ticket <URL:
> > > >>
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > > Hi met support,
> > > >> > >
> > > >> > > I have 2 questions regarding
how to
> output
> > specific
> > > >> statistics for my
> > > >> > > experiment (see files in my
attached
> tarball). I
> > have
> > > looked
> > > >> through the
> > > >> > > online documentation, but I
have had a
> hard time
> > > finding a
> > > >> direct
> > > >> > > answer/solution – perhaps
I’ve simply
> missed
> > something
> > > >> obvious.
> > > >> > >
> > > >> > > QUESTION 1: how to output
wind/vector
> (VL1L2,
> > VCNT)
> > > >> statistics. I have
> > > >> > > added:
> > > >> > >
> > > >> > > GRID_STAT_OUTPUT_FLAG_CNT
= BOTH
> > > >> > > GRID_STAT_OUTPUT_FLAG_VCNT
= BOTH
> > > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2
= BOTH
> > > >> > >
> > > >> > > … to my
> > > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
> > > >> file. I
> > > >> > have
> > > >> > > also added:
> > > >> > >
> > > >> > > is_u_wind : True
> > > >> > > is_grid_relative: False
> > > >> > >
> > > >> > > … attributes to my
PYTHON_NUMPY object
> through
> > python
> > > >> embedding. However,
> > > >> > > when I look at the output
*_vl1l2.txt
> files, the
> > only
> > > >> information is the
> > > >> > > header lines. I have included
2 sample
> outputs
> > from my
> > > >> embedding scripts
> > > >> > > when called alone
> > (./scripts/read_*2xr_v2.example.log)
> > > which
> > > >> show all the
> > > >> > > data is as expected. It seems
like
> something else
> > needs
> > > to be
> > > >> activated
> > > >> > > somewhere in the conf files,
but I
> can’t seem to
> > find
> > > the
> > > >> correct switch.
> > > >> > > If you have any ideas for
other things
> to try,
> > please
> > > let me
> > > >> know.
> > > >> > >
> > > >> > > QUESTION 2: is there a way to
use
> stat_analysis to
> > > >> recompute/aggregate
> > > >> > > over my *_pairs.nc files
containing
> this
> > information
> > > >> (example ncdump in
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > > >> > > and output CNT stats for the
total of
> all the
> > matched
> > > pair
> > > >> data? I am
> > > >> > aware
> > > >> > > of the “-job summary
-line_type CNT
> -column RMSE
> > -by
> > > >> FCST_VAR,FCST_LEV
> > > >> > > -dump_row” but this looks
like it only
> provides
> > basic
> > > >> statistics of all
> > > >> > the
> > > >> > > RMSE values in each *_cnt.txt
files,
> correct?
> > What I
> > > really
> > > >> want is to
> > > >> > > generate actual CNT data from
all
> matched pairs
> > over all
> > > >> times. I’ve
> > > >> > tried
> > > >> > > various combinations of
aggregate and
> > aggregate_stat but
> > > >> can’t seem to
> > > >> > find
> > > >> > > a combination that will work.
Is this
> possible?
> > If so,
> > > what
> > > >> should my
> > > >> > > stat_analysis job command
look like?
> > > >> > >
> > > >> > > Thanks so much for your time
in
> advance. I have
> > asked a
> > > lot
> > > >> of questions
> > > >> > > lately, but I think I’m
almost there.
> Any help
> > you can
> > > >> provide on filling
> > > >> > > in my knowledge gaps would be
very much
> > appreciated.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Scott
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working
day. Please
> do not feel
> > > obliged to
> > > > reply to this email outside of your normal
working
> hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day.
Please do
> not feel
> > obliged
> > > to
> > > reply to this email outside of your normal
working
> hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please
do not
> feel obliged
> > to
> > reply to this email outside of your normal working
hours.
> >
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do
not feel
> obliged to
> reply to this email outside of your normal working
hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Mon Jun 21 18:51:06 2021
Hi Scott,
Thanks for sending the input files. I did some digging and I believe
this
may require an enhancement to the logic for handling Python embedding.
The
value for level is typically not needed for Python embedding because
all of
the information is provided to the name value. However, it looks like
the
logic to compute U/V vector partial sums uses the level value to match
up
the U and V fields. The developer(s) who are more familiar with this
logic
in the C++ code are currently on vacation, but I will create a GitHub
issue
to investigate this and create a fix.
Thanks,
George
On Mon, Jun 21, 2021 at 10:17 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> Thanks for getting back in touch with me so soon. Sorry for the
delay. I'm
> back at work today.
>
> This morning I simplified my grid_stat config file so it is only
reading a
> few select fields on pressure levels. Hopefully, this will avoid the
> complications of the surface data with time offsets and all the
other
> non-conventional comparisons with masking that you have noticed.
This
> latest run demonstrates the problem - still no vector stats. I
uploaded 3
> files to the ucar ftp site:
>
> 1. geosgcm_pres.20191223_0000z.nc4.gz (forecast data time sliced to
index
> 26 the observation comparison time)
> 2. era5_pl-allvar.20191223_00z.nc4.gz (observation data)
> 3. expfiles.5.tar.gz (logs/scripts from my latest run)
>
> Just as a reminder, my python read scripts search through a
directory
> hierarchy to locate the proper forecast and observation files for
> comparison. This will not work for you as is. You will have to
change and
> hard-code the file paths to make this work on your side.
>
> Please let me know if you have any issues or need any further
information.
> I am eager and hopeful we can finally solve this issue. Looking
forward to
> hearing from you.
>
> Thanks,
> Scott
>
>
> On 6/18/21, 11:43 AM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> Thanks for sending the new log file. We do have an FTP site that
you
> can
> use to send us data. Information on how is on this webpage under
the
> "How
> To Send Us Data" heading:
>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fget-
help&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074114075%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yQDC8isdwK7buTn0M8Hk%2F0Bb0QoI7Y6lw81ppYhBKuw%3D&reserved=0
> Those files are quite large, but I would really only need the U
and V
> fields from each data set to debug why that logic is not working
> properly.
>
> I did also notice that there are some warnings in your log file
saying
> that
> there is a time mismatch between the forecast and observation
data for
> some
> of the other fields. It will still use the data, but the results
may
> not
> reflect what you are expecting. The valid time that comes from
the
> forecast
> script doesn't match the valid time passed into the script, so
there
> may be
> something going on there. This isn't related to the U/V issue
but I
> thought
> I would point it out.
>
>
> {
>
>
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
> gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
>
> DEBUG 3: Running user's python script
>
>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
> DEBUG 4:
> DEBUG 4: Data plane information:
> DEBUG 4: plane min: 0
> DEBUG 4: plane max: 0.01865353266
> DEBUG 4: valid time: *20191223_003000*
> DEBUG 4: lead time: 273000
> DEBUG 4: init time: 20191221_210000
> DEBUG 4: accum time: 000000
> DEBUG 4:
> DEBUG 3: Reading forecast data for TTPRCP/Surface.
> DEBUG 3: Running user's python script
>
>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
> DEBUG 4:
> DEBUG 4: Data plane information:
> DEBUG 4: plane min: 0
> DEBUG 4: plane max: 0.02649974823
> DEBUG 4: valid time: *20191223_000000*
> DEBUG 4: lead time: 000000
> DEBUG 4: init time: 20191223_000000
> DEBUG 4: accum time: 010000
> DEBUG 4:
> DEBUG 3: Reading observation data for tp/Surface.
> WARNING:
> WARNING: process_scores() -> Forecast and observation valid
times do
> not
> match 20191223_003000 != 20191223_000000 for TTPRCP/Surface
versus
> tp/Surface.
>
> On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> >
> >
> > Hi George,
> >
> > I apologize, my last email sent prematurely for some weird
reason.
> As I
> > was saying below, our forecast and observation files are very
large.
> I
> > could ncks a subset, but that really wouldn't be the same
file. I
> suppose
> > it would contain the same fields, but even a subset would be
too
> large for
> > an email attachment.
> >
> > Regarding the output below, you are correct. I did not modify
the
> 10m U/V
> > winds (U10M, V10M, etc). I only changed the 3D pressure level
U/V
> winds to
> > have the same name and attributes. I figured if those started
> working I
> > would propagate the changes to the surface python scripts too.
So I
> would
> > recommend to only look at the output pertaining to pressure
level
> winds
> > from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py
scripts.
> Those have
> > the changes.
> >
> > Please let me know how to proceed with posting the
> forecast/observation
> > files if you think that will help. Do you have an ftp site?
> >
> > Thanks,
> > Scott
> >
> >
> >
> >
> > On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS
> > AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
> >
> > Hi George,
> >
> > I have attached the log file - that is easy. However, I'm
not
> sure how
> > to go about sending you the data. The experiment (forecast) 3D
file
> is 734G
> > and the surface (single) level files are 619M each.
> >
> > On 6/17/21, 5:01 PM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >
> > Hi Scott,
> >
> > The VL1L2 flag should accept BOTH. I'm not familiar
enough
> with
> > the U/V
> > calculations to know if it needs a threshold but John
didn't
> > mention that
> > it was a requirement in his first email. Could you
send me a
> set
> > of files
> > that are used as input for this use case? Also, the
latest
> log
> > file would
> > be helpful. I noticed that the name of the fields did
> reflect the
> > script
> > name, so I'm curious to see if it was overridden
properly in
> your
> > last run.
> >
> > | | | | | | | Dictionary Entry[0]
...
> > | | | | | | | | Name = *name*
> > | | | | | | | | Type =
StringType
> > | | | | | | | | *String Value
=
> >
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
> >
> >
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> > U10M 20191221_2100 20191223_0000"*
> > | | | | | | | Dictionary Entry[1]
...
> > | | | | | | | | Name =
is_u_wind
> > | | | | | | | | Type =
BooleanType
> > | | | | | | | | Boolean Value
= true
> > | | | | | | | Dictionary Entry[2]
...
> > | | | | | | | | Name =
> is_grid_relative
> > | | | | | | | | Type =
BooleanType
> > | | | | | | | | Boolean Value
= false
> >
> > Thanks,
> > George
> >
> > On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >
> > >
> > > <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > >
> > >
> > > Hi George,
> > >
> > > I your suggestion and double checked for typos, but
> > unfortunately, still
> > > nothing different - just headers in the output
files. Could
> > there be a bug
> > > in something indirectly related to this, such as the
BOTH
> option
> > does not
> > > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V
winds
> need a
> > threshold.
> > > Just a WAG.
> > >
> > > ~Scott
> > >
> > >
> > >
> > > On 6/17/21, 1:28 PM, "George McCabe via RT" <
> met_help at ucar.edu>
> > wrote:
> > >
> > > Hi Scott,
> > >
> > > You could try adding
> > >
> > > set_attr_name = "U";
> > >
> > > to the [FCST/OBS]_VAR4_OPTIONS and "V" for the
VAR5
> options
> > to see if
> > > that
> > > helps? It may be using the script as the name,
so
> setting
> > this would
> > > override the input name that is used.
> > >
> > > FCST_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
> =
> > "U";
> > > OBS_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
> =
> > "U";
> > > FCST_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
> =
> > "V";
> > > OBS_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
> =
> > "V";
> > >
> > > Let me know if that doesn't work and I can
continue
> digging.
> > >
> > > Thanks,
> > > George
> > >
> > > On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst,
Scott D.
> > > (GSFC-610.1)[SCIENCE
> > > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> > wrote:
> > >
> > > >
> > > > <URL:
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > >
> > > >
> > > > Hi George,
> > > >
> > > > Thanks so much for taking a look at the code
once
> again.
> > You had
> > > another
> > > > great idea, and something rather easy to
change via
> python
> > embedding
> > > > compared to using ncatted/ncrename on so many
files.
> I
> > modified my
> > > scripts
> > > > to ensure all my attrs={...} values matched
for the
> U and
> > V fields,
> > > but
> > > > still not luck! I can't believe it. Since
metplus
> doesn't
> > appear to
> > > display
> > > > the output from my python embedding scripts, I
ran
> both
> > scripts
> > > stand-alone
> > > > by copying/pasting metplus' name field onto
the
> command
> > line. The
> > > output is
> > > > in the scripts/read_*2xr_v2.log files. All
attributes
> > match exactly
> > > (I even
> > > > hardcoded the init time just to be sure). I
ran with
> and
> > without the
> > > > updating the is_u/v_wind and is_grid_relative
> attributes
> > in the
> > > python
> > > > attrs object (which shouldn't be necessary),
but
> that made
> > no
> > > difference
> > > > either. The vl1l2 and vcnt files are still
just
> header
> > information
> > > only
> > > > (see tarball). I increased the verbosity of
> grid_stat to
> > 10, but my
> > > eye
> > > > didn't catch anything offhand. Maybe yo!
> > > > u have a better idea what to look for. This
whole
> thing
> > is super
> > > weird.
> > > > Do you have any other ideas I can try?
> > > >
> > > > Thanks,
> > > > Scott
> > > >
> > > >
> > > >
> > > > On 6/16/21, 4:13 PM, "George McCabe via RT" <
> > met_help at ucar.edu>
> > > wrote:
> > > >
> > > > Hi Scott,
> > > >
> > > > I think I see what is happening with the
U/V
> fields. I
> > checked
> > > the code
> > > > where it is checking the U/V fields and it
> expects the
> > name and
> > > level
> > > > of
> > > > the fcst and obs to match to make sure it
it
> > processing the
> > > correct U/V
> > > > data. I noticed in the log output from
your
> python
> > embedding
> > > call that
> > > > era5pl2xr has the name "u" while g5fcst2xr
has
> "U". I
> > would
> > > suggest
> > > > modifying the python embedding script to
ensure
> that
> > both of the
> > > > varname
> > > > values are the same case.
> > > >
> > > > If that doesn't work, I would suggest
bumping up
> the
> > log
> > > verbosity for
> > > > GridStat (LOG_GRID_STAT_VERBOSITY) to see
if
> there are
> > any clues
> > > in the
> > > > higher number log output. If you are still
having
> > issues, you
> > > could
> > > > send me
> > > > the input files for each and I can play
around
> with
> > the settings
> > > to see
> > > > what will work.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Jun 16, 2021 at 1:07 PM George
McCabe <
> > mccabe at ucar.edu>
> > > wrote:
> > > >
> > > > > Hi Scott,
> > > > >
> > > > > Regarding SeriesAnalysis -- there is an
> argument to
> > the tool
> > > called
> > > > > "-both" that can be used to set both the
fcst
> and
> > obs lists to
> > > use
> > > > the same
> > > > > files. This use case runs SeriesAnalysis
on
> GridStat
> > output:
> > > > >
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CaEvxnPTd%2FXCcR5PZGjT4reGDAdbN0110%2BSAqw%2BXXQI%3D&reserved=0
> > > > >
> > > > > This example uses
> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> > > the
> > > > input
> > > > > file path and specifies the fcst and obs
input
> fields
> > > separately:
> > > > >
> > > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
> FCST_precip_FULL
> > > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
> > > > >
> > > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
OBS_precip_FULL
> > > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
> > > > >
> > > > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> > > > >
> > > >
> > >
> >
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> > > > > pairs.nc
> > > > >
> > > > > Regarding not being able to produce wind
> output,
> > I'll take a
> > > look at
> > > > your
> > > > > files and see if I can narrow down the
issue.
> I will
> > report
> > > back
> > > > with more
> > > > > information.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Mon, Jun 7, 2021 at 9:09 AM
Rabenhorst,
> Scott D.
> > > > (GSFC-610.1)[SCIENCE
> > > > > SYSTEMS AND APPLICATIONS INC] via RT <
> > met_help at ucar.edu>
> > > wrote:
> > > > >
> > > > >>
> > > > >> <URL:
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > > >
> > > > >>
> > > > >> Hi George,
> > > > >>
> > > > >> Just as a follow up thought to problem
below
> > regarding winds
> > > stats,
> > > > when
> > > > >> I look through the CNT and SL1L2
statistics
> output,
> > I see U
> > > and V
> > > > listed
> > > > >> with all respective stats. So this
issue is
> not a
> > problem
> > > reading U
> > > > and V,
> > > > >> it is a problem recognizing they are
vector
> > components. Are
> > > your
> > > > sure there
> > > > >> is not another flag or option to set
either
> in the
> > > PTHON_NUMPY/attrs
> > > > >> objects or another option in the
metplus conf
> file?
> > > > >>
> > > > >> Thanks,
> > > > >> Scott
> > > > >>
> > > > >>
> > > > >> On 6/4/21, 12:28 PM, "Rabenhorst, Scott
D.
> > > (GSFC-610.1)[SCIENCE
> > > > SYSTEMS
> > > > >> AND APPLICATIONS INC]" <
> scott.d.rabenhorst at nasa.gov>
> > wrote:
> > > > >>
> > > > >> Hi George,
> > > > >>
> > > > >> Thanks for answering my questions
below.
> So
> > sorry for
> > > taking so
> > > > much
> > > > >> of your time lately. From your
description, it
> > sounds like
> > > the best
> > > > option
> > > > >> for my needs is to use Stat-Analysis to
> aggregate
> > SL1L2/VL1L2
> > > data.
> > > > My goal
> > > > >> is to generate single-value statistics
for
> each
> > variable/level
> > > > through all
> > > > >> valid times and spatial areas. These
will be
> > plugged into a
> > > Green's
> > > > >> Function algorithm for our model.
> > > > >>
> > > > >> However, the Series-Analysis is
also an
> > intriguing
> > > approach at
> > > > least
> > > > >> for looking at spatial errors. I guess
> Grid-Stat
> > could
> > > operate on
> > > > that data
> > > > >> at the end to generate total stats? I
have not
> > tried using
> > > > Series-Analysis
> > > > >> yet, but what has confused me is the
> documentation
> > requires
> > > > separate "-fcst
> > > > >> file_1...file_n" and "-obs
file_1...file_n".
> In
> > most examples
> > > I
> > > > have seen,
> > > > >> these are independent files. However,
you
> make it
> > sound as
> > > though
> > > > it can
> > > > >> run on my *_pairs.nc files generated by
> Grid-Stat
> > I presume
> > > as
> > > > long as
> > > > >> RAW data is in there? In this case, I
assume
> the
> > list of files
> > > > would be
> > > > >> identical to both forecast and
observations?
> Are
> > there any
> > > examples
> > > > of
> > > > >> using matched-pairs for the Series-
Analysis
> online?
> > > > >>
> > > > >> Unfortunately, the larger problem
right
> now is
> > that I
> > > still
> > > > can't get
> > > > >> Grid-Stat to process any my winds and
output
> VCNT
> > or VL1L2
> > > data. I
> > > > tried
> > > > >> your suggestion below and added the
options
> to my
> > > > >>
> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> > file
> > > (please see
> > > > >> updated attached tarball). I tried
running
> with and
> > without
> > > each of
> > > > these
> > > > >> options "is_u(v)_wind=TRUE" and
> > "is_grid_relative=FALSE" and
> > > those
> > > > ASCII
> > > > >> files never contains any VL1L2/VCNT
data, nor
> do
> > the *.stat
> > > files
> > > > contain
> > > > >> any of those line_types. I also toggled
> outputting
> > those
> > > attribute
> > > > in my
> > > > >> PYTHON_NUMPY attrs object, but that
didn't
> work
> > either. I
> > > couldn't
> > > > find any
> > > > >> combinations that would work. In an
effort to
> > narrow the
> > > problem, I
> > > > >> copy/pasted the first references to the
fcst
> and
> > obs fields
> > > in my
> > > > logfile
> > > > >> and ran my python read scripts stand-
alone
> and they
> > produced
> > > the
> > > > expected
> > > > >> output with correct values:
> > > > >>
scripts/read_era5pl2xr_v2.example.log
> > > > >>
scripts/read_g5fcst2xr_v2.example.log
> > > > >>
> > > > >> Furthermore, I decided to test
> PlotDataPlane
> > since I
> > > recalled
> > > > you
> > > > >> saying that was used behind the scenes.
It
> ran just
> > fine and
> > > > produced these
> > > > >> beautiful U 500hPa images without
error:
> > > > >>
> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
> > > > >>
> > > > >> I have combed the log files for any
> errors or
> > warning, but
> > > > cannot
> > > > >> find any at all, or anything else that
would
> > indicate a
> > > processing
> > > > error.
> > > > >> It is almost like there is some flag
that has
> to be
> > set in
> > > order to
> > > > trigger
> > > > >> wind processing. Do you have any other
ideas
> I can
> > try?
> > > > >>
> > > > >> Thanks,
> > > > >> Scott
> > > > >>
> > > > >>
> > > > >> On 6/3/21, 6:22 PM, "John Halley
Gotway
> via RT"
> > <
> > > > met_help at ucar.edu>
> > > > >> wrote:
> > > > >>
> > > > >> Hi Scott,
> > > > >>
> > > > >> Minna asked me to take a look
at your
> > questions.
> > > > >>
> > > > >> For the second question, the
answer
> is no.
> > > Stat-Analysis
> > > > reads
> > > > >> the .stat
> > > > >> output files created by the MET
> tools. It
> > does not
> > > read the
> > > > >> gridded NetCDF
> > > > >> matched pairs files generated
by the
> > Grid-Stat tool.
> > > > >> But I do have two
recommendations to
> make.
> > > > >>
> > > > >> 1. Grid-Stat computes
statistics and
> > partial sums that
> > > > represent
> > > > >> a spatial
> > > > >> summary. You can configure
Grid-Stat
> to
> > compute its
> > > output
> > > > over
> > > > >> whatever
> > > > >> spatial masking regions you'd
like.
> > Stat-Analysis can
> > > > >> read/aggregate those
> > > > >> spatial summaries through time.
The
> SL1L2
> > line type
> > > contains
> > > > >> scalar L1 and
> > > > >> L2 norms. Those are written
because
> they
> > allow you to
> > > easily
> > > > >> aggregate
> > > > >> stats over multiple runs. Many
(but
> not
> > all) of the
> > > > continuous
> > > > >> statistics
> > > > >> (like ME, RMSE, MAE, ACC, and
so on)
> can be
> > derived
> > > from the
> > > > >> SL1L2 line
> > > > >> type. So I'd recommend trying
this
> type of
> > > stat_analysis
> > > > job:
> > > > >>
> > > > >> -job aggregate_stat -line_type
SL1L2
> > -out_line_type
> > > CNT -by
> > > > >> FCST_VAR,FCST_LEV,VX_MASK
-out_stat
> > sl1l2_to_cnt.stat
> > > > >>
> > > > >> For each unique combination of
> variable,
> > level, and
> > > masking
> > > > >> region, it'll
> > > > >> aggregate all the SL1L2 lines
it
> finds in
> > the input
> > > and
> > > > report the
> > > > >> corresponding continuous
statistics.
> > > > >>
> > > > >> 2. Another option is running
the
> NetCDF
> > matched pairs
> > > > produced by
> > > > >> Grid-Stat
> > > > >> through the Series-Analysis
tool.
> That'll
> > compute
> > > stats
> > > > through
> > > > >> time
> > > > >> separately for each individual
grid
> point.
> > The result
> > > with
> > > > be a
> > > > >> map of
> > > > >> statistics rather than a
spatial
> summary.
> > > > >>
> > > > >> For the first question, I see
that
> you're
> > using python
> > > > embedding
> > > > >> and are
> > > > >> trying to compute vector
partial sums
> > output. In
> > > general,
> > > > when
> > > > >> you request
> > > > >> an output line type, Grid-Stat
will
> create
> > the
> > > corresponding
> > > > >> output file.
> > > > >> But if it's empty, that means
that it
> > didn't have any
> > > input
> > > > data
> > > > >> suitable
> > > > >> for creating that output type.
So
> you've
> > requested
> > > vector
> > > > stats,
> > > > >> but
> > > > >> Grid-Stat isn't computing any.
> > > > >>
> > > > >> Thanks for sending the sample
output.
> > Looking in:
> > > > >>
> > > > >>
> > > >
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> > > > >>
> > > > >> I do see several warning that
you
> should
> > take a look
> > > at:
> > > > >> WARNING: python_dataplane() ->
an
> error
> > occurred
> > > importing
> > > > module
> > > > >>
> > > > >>
> > > >
> > >
> >
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> > > > >>
> > > > >> But I do also see that Grid-
Stat is
> > processing the U
> > > and V
> > > > data
> > > > >> and
> > > > >> reporting output as scalar
variables
> in the
> > CNT.txt
> > > file:
> > > > >>
> > > > >>
> > > >
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> > > > >>
> > > > >> Now I haven't tested this on
your
> data to
> > confirm
> > > that it'll
> > > > >> actually work,
> > > > >> but we want to set these flags
in the
> MET
> > config file:
> > > > >> is_u_wind = true
> > > > >> is_v_wind = true
> > > > >>
> > > > >>
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VnwsgB2i2RQwVAIUEfnO3xZLhClxh%2Bqb0q7b78HQ0YU%3D&reserved=0
> > > > >>
> > > > >> Checking in
> > > > >>
> > RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> > > > >> see that var 4 is for U and var
5 is
> for V.
> > > > >> Add those via the METplus
config file
> by
> > adding:
> > > > >> FCST_VAR4_OPTIONS = is_u_wind =
TRUE;
> > > > >> OBS_VAR4_OPTIONS = is_u_wind =
TRUE;
> > > > >> FCST_VAR5_OPTIONS = is_v_wind =
TRUE;
> > > > >> OBS_VAR5_OPTIONS = is_v_wind =
TRUE;
> > > > >>
> > > > >> Please give that a shot and let
me
> know how
> > it goes.
> > > The
> > > > goal is
> > > > >> NOT having
> > > > >> any U and V show up in the CNT
output
> > file... and
> > > instead
> > > > have
> > > > >> entries show
> > > > >> up in the VCNT output.
> > > > >>
> > > > >> One last detail... if you do
want to
> > aggregate results
> > > > through
> > > > >> time, I'd
> > > > >> recommend writing the SL1L2 and
VL1L2
> line
> > types to
> > > enable
> > > > that
> > > > >> aggregation
> > > > >> to be done later.
> > > > >>
> > > > >> Thanks,
> > > > >> John
> > > > >>
> > > > >> On Thu, Jun 3, 2021 at 1:49 PM
Minna
> Win
> > via RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
> > > > >>
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > > >> >
> > > > >> >
> > > > >> > Hi Scott,
> > > > >> >
> > > > >> > It looks like you've set your
MET
> config
> > file to
> > > correctly
> > > > >> identify which
> > > > >> > linetypes you'd like to see.
I'm
> not sure
> > why you
> > > are
> > > > >> essentially getting
> > > > >> > empty output files. I'll
need to
> find
> > someone
> > > who can
> > > > answer
> > > > >> your python
> > > > >> > embedding and grid-stat
questions.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Minna
> > > > >> > ---------------
> > > > >> > Minna Win
> > > > >> > Pronouns: she/her
> > > > >> > National Center for
Atmospheric
> Research
> > > > >> > Developmental Testbed Center
> > > > >> > Phone: 303-497-8423
> > > > >> > Fax: 303-497-8401
> > > > >> > ---------------
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Jun 3, 2021 at 12:04
PM
> > Rabenhorst, Scott D.
> > > > >> (GSFC-610.1)[SCIENCE
> > > > >> > SYSTEMS AND APPLICATIONS INC]
via
> RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > Thu Jun 03 12:04:14 2021:
Request
> > 100090 was
> > > acted upon.
> > > > >> > > Transaction: Ticket created
by
> > > > scott.d.rabenhorst at nasa.gov
> > > > >> > > Queue: met_help
> > > > >> > > Subject: 2 questions
> regarding how
> > to
> > > generate
> > > > specific
> > > > >> stats
> > > > >> > > Owner: Nobody
> > > > >> > > Requestors:
> > scott.d.rabenhorst at nasa.gov
> > > > >> > > Status: new
> > > > >> > > Ticket <URL:
> > > > >>
> > > >
> > >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
> > > > >> >
> > > > >> > >
> > > > >> > >
> > > > >> > > Hi met support,
> > > > >> > >
> > > > >> > > I have 2 questions
regarding how
> to
> > output
> > > specific
> > > > >> statistics for my
> > > > >> > > experiment (see files in my
> attached
> > tarball). I
> > > have
> > > > looked
> > > > >> through the
> > > > >> > > online documentation, but I
have
> had a
> > hard time
> > > > finding a
> > > > >> direct
> > > > >> > > answer/solution – perhaps
I’ve
> simply
> > missed
> > > something
> > > > >> obvious.
> > > > >> > >
> > > > >> > > QUESTION 1: how to output
> wind/vector
> > (VL1L2,
> > > VCNT)
> > > > >> statistics. I have
> > > > >> > > added:
> > > > >> > >
> > > > >> > > GRID_STAT_OUTPUT_FLAG_CNT
=
> BOTH
> > > > >> > > GRID_STAT_OUTPUT_FLAG_VCNT
=
> BOTH
> > > > >> > > GRID_STAT_OUTPUT_FLAG_VL1L2
=
> BOTH
> > > > >> > >
> > > > >> > > … to my
> > > > ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-
01.conf
> > > > >> file. I
> > > > >> > have
> > > > >> > > also added:
> > > > >> > >
> > > > >> > > is_u_wind : True
> > > > >> > > is_grid_relative: False
> > > > >> > >
> > > > >> > > … attributes to my
PYTHON_NUMPY
> object
> > through
> > > python
> > > > >> embedding. However,
> > > > >> > > when I look at the output
> *_vl1l2.txt
> > files, the
> > > only
> > > > >> information is the
> > > > >> > > header lines. I have
included 2
> sample
> > outputs
> > > from my
> > > > >> embedding scripts
> > > > >> > > when called alone
> > > (./scripts/read_*2xr_v2.example.log)
> > > > which
> > > > >> show all the
> > > > >> > > data is as expected. It
seems like
> > something else
> > > needs
> > > > to be
> > > > >> activated
> > > > >> > > somewhere in the conf
files, but I
> > can’t seem to
> > > find
> > > > the
> > > > >> correct switch.
> > > > >> > > If you have any ideas for
other
> things
> > to try,
> > > please
> > > > let me
> > > > >> know.
> > > > >> > >
> > > > >> > > QUESTION 2: is there a way
to use
> > stat_analysis to
> > > > >> recompute/aggregate
> > > > >> > > over my *_pairs.nc files
> containing
> > this
> > > information
> > > > >> (example ncdump in
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> > > > >> > > and output CNT stats for
the
> total of
> > all the
> > > matched
> > > > pair
> > > > >> data? I am
> > > > >> > aware
> > > > >> > > of the “-job summary
-line_type
> CNT
> > -column RMSE
> > > -by
> > > > >> FCST_VAR,FCST_LEV
> > > > >> > > -dump_row” but this looks
like it
> only
> > provides
> > > basic
> > > > >> statistics of all
> > > > >> > the
> > > > >> > > RMSE values in each
*_cnt.txt
> files,
> > correct?
> > > What I
> > > > really
> > > > >> want is to
> > > > >> > > generate actual CNT data
from all
> > matched pairs
> > > over all
> > > > >> times. I’ve
> > > > >> > tried
> > > > >> > > various combinations of
aggregate
> and
> > > aggregate_stat but
> > > > >> can’t seem to
> > > > >> > find
> > > > >> > > a combination that will
work. Is
> this
> > possible?
> > > If so,
> > > > what
> > > > >> should my
> > > > >> > > stat_analysis job command
look
> like?
> > > > >> > >
> > > > >> > > Thanks so much for your
time in
> > advance. I have
> > > asked a
> > > > lot
> > > > >> of questions
> > > > >> > > lately, but I think I’m
almost
> there.
> > Any help
> > > you can
> > > > >> provide on filling
> > > > >> > > in my knowledge gaps would
be
> very much
> > > appreciated.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > Scott
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working
day.
> Please
> > do not feel
> > > > obliged to
> > > > > reply to this email outside of your
normal
> working
> > hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working
day.
> Please do
> > not feel
> > > obliged
> > > > to
> > > > reply to this email outside of your normal
> working
> > hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day.
Please do
> not
> > feel obliged
> > > to
> > > reply to this email outside of your normal
working
> hours.
> > >
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do
not
> feel
> > obliged to
> > reply to this email outside of your normal working
hours.
> >
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Tue Jun 22 06:50:06 2021
Hi Scott,
This morning I thought to try to use set_attr_level to override the
level
values to see if that would allow the U/V components to be combined
properly, but that also did not seem to have any affect. While
searching
for another example of computing wind vectors from U/V I found another
use
case that reads U and V fields in a Python embedding script and
computes
U/V to be used in calculating IVT. I'm not sure that this will give
you the
same result that you are expecting, but you could potentially combine
U and
V inside your Python embedding script instead of relying on the MET
tools
to perform that calculation. I am still going to write up an issue to
investigate the potential deficiency in the logic, but this may help
you
move forward with your analysis for the time being.
Thanks,
George
On Mon, Jun 21, 2021 at 6:50 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Scott,
>
> Thanks for sending the input files. I did some digging and I believe
this
> may require an enhancement to the logic for handling Python
embedding. The
> value for level is typically not needed for Python embedding because
all of
> the information is provided to the name value. However, it looks
like the
> logic to compute U/V vector partial sums uses the level value to
match up
> the U and V fields. The developer(s) who are more familiar with this
logic
> in the C++ code are currently on vacation, but I will create a
GitHub issue
> to investigate this and create a fix.
>
> Thanks,
> George
>
> On Mon, Jun 21, 2021 at 10:17 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>>
>> Hi George,
>>
>> Thanks for getting back in touch with me so soon. Sorry for the
delay.
>> I'm back at work today.
>>
>> This morning I simplified my grid_stat config file so it is only
reading
>> a few select fields on pressure levels. Hopefully, this will avoid
the
>> complications of the surface data with time offsets and all the
other
>> non-conventional comparisons with masking that you have noticed.
This
>> latest run demonstrates the problem - still no vector stats. I
uploaded 3
>> files to the ucar ftp site:
>>
>> 1. geosgcm_pres.20191223_0000z.nc4.gz (forecast data time sliced to
index
>> 26 the observation comparison time)
>> 2. era5_pl-allvar.20191223_00z.nc4.gz (observation data)
>> 3. expfiles.5.tar.gz (logs/scripts from my latest run)
>>
>> Just as a reminder, my python read scripts search through a
directory
>> hierarchy to locate the proper forecast and observation files for
>> comparison. This will not work for you as is. You will have to
change and
>> hard-code the file paths to make this work on your side.
>>
>> Please let me know if you have any issues or need any further
>> information. I am eager and hopeful we can finally solve this
issue.
>> Looking forward to hearing from you.
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/18/21, 11:43 AM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>>
>> Hi Scott,
>>
>> Thanks for sending the new log file. We do have an FTP site
that you
>> can
>> use to send us data. Information on how is on this webpage
under the
>> "How
>> To Send Us Data" heading:
>>
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fget-
help&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074114075%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yQDC8isdwK7buTn0M8Hk%2F0Bb0QoI7Y6lw81ppYhBKuw%3D&reserved=0
>> Those files are quite large, but I would really only need the U
and V
>> fields from each data set to debug why that logic is not
working
>> properly.
>>
>> I did also notice that there are some warnings in your log file
>> saying that
>> there is a time mismatch between the forecast and observation
data
>> for some
>> of the other fields. It will still use the data, but the
results may
>> not
>> reflect what you are expecting. The valid time that comes from
the
>> forecast
>> script doesn't match the valid time passed into the script, so
there
>> may be
>> something going on there. This isn't related to the U/V issue
but I
>> thought
>> I would point it out.
>>
>>
>> {
>>
>>
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>>
>>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
>> TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
>> gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
>>
>> DEBUG 3: Running user's python script
>>
>>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
>> DEBUG 4:
>> DEBUG 4: Data plane information:
>> DEBUG 4: plane min: 0
>> DEBUG 4: plane max: 0.01865353266
>> DEBUG 4: valid time: *20191223_003000*
>> DEBUG 4: lead time: 273000
>> DEBUG 4: init time: 20191221_210000
>> DEBUG 4: accum time: 000000
>> DEBUG 4:
>> DEBUG 3: Reading forecast data for TTPRCP/Surface.
>> DEBUG 3: Running user's python script
>>
>>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
>> DEBUG 4:
>> DEBUG 4: Data plane information:
>> DEBUG 4: plane min: 0
>> DEBUG 4: plane max: 0.02649974823
>> DEBUG 4: valid time: *20191223_000000*
>> DEBUG 4: lead time: 000000
>> DEBUG 4: init time: 20191223_000000
>> DEBUG 4: accum time: 010000
>> DEBUG 4:
>> DEBUG 3: Reading observation data for tp/Surface.
>> WARNING:
>> WARNING: process_scores() -> Forecast and observation valid
times do
>> not
>> match 20191223_003000 != 20191223_000000 for TTPRCP/Surface
versus
>> tp/Surface.
>>
>> On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D.
>> (GSFC-610.1)[SCIENCE
>> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>>
>> >
>> > <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> >
>> >
>> > Hi George,
>> >
>> > I apologize, my last email sent prematurely for some weird
reason.
>> As I
>> > was saying below, our forecast and observation files are very
>> large. I
>> > could ncks a subset, but that really wouldn't be the same
file. I
>> suppose
>> > it would contain the same fields, but even a subset would be
too
>> large for
>> > an email attachment.
>> >
>> > Regarding the output below, you are correct. I did not modify
the
>> 10m U/V
>> > winds (U10M, V10M, etc). I only changed the 3D pressure level
U/V
>> winds to
>> > have the same name and attributes. I figured if those started
>> working I
>> > would propagate the changes to the surface python scripts
too. So I
>> would
>> > recommend to only look at the output pertaining to pressure
level
>> winds
>> > from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py
scripts.
>> Those have
>> > the changes.
>> >
>> > Please let me know how to proceed with posting the
>> forecast/observation
>> > files if you think that will help. Do you have an ftp site?
>> >
>> > Thanks,
>> > Scott
>> >
>> >
>> >
>> >
>> > On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
>> SYSTEMS
>> > AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov> wrote:
>> >
>> > Hi George,
>> >
>> > I have attached the log file - that is easy. However, I'm
not
>> sure how
>> > to go about sending you the data. The experiment (forecast)
3D file
>> is 734G
>> > and the surface (single) level files are 619M each.
>> >
>> > On 6/17/21, 5:01 PM, "George McCabe via RT"
<met_help at ucar.edu>
>> wrote:
>> >
>> > Hi Scott,
>> >
>> > The VL1L2 flag should accept BOTH. I'm not familiar
enough
>> with
>> > the U/V
>> > calculations to know if it needs a threshold but John
didn't
>> > mention that
>> > it was a requirement in his first email. Could you
send me
>> a set
>> > of files
>> > that are used as input for this use case? Also, the
latest
>> log
>> > file would
>> > be helpful. I noticed that the name of the fields did
>> reflect the
>> > script
>> > name, so I'm curious to see if it was overridden
properly
>> in your
>> > last run.
>> >
>> > | | | | | | | Dictionary
Entry[0] ...
>> > | | | | | | | | Name =
*name*
>> > | | | | | | | | Type =
StringType
>> > | | | | | | | | *String Value
=
>> >
>> >
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>> >
>> >
>>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
>> > U10M 20191221_2100 20191223_0000"*
>> > | | | | | | | Dictionary
Entry[1] ...
>> > | | | | | | | | Name =
is_u_wind
>> > | | | | | | | | Type =
BooleanType
>> > | | | | | | | | Boolean Value
= true
>> > | | | | | | | Dictionary
Entry[2] ...
>> > | | | | | | | | Name =
>> is_grid_relative
>> > | | | | | | | | Type =
BooleanType
>> > | | | | | | | | Boolean Value
=
>> false
>> >
>> > Thanks,
>> > George
>> >
>> > On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst, Scott D.
>> > (GSFC-610.1)[SCIENCE
>> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
>> wrote:
>> >
>> > >
>> > > <URL:
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> > >
>> > >
>> > > Hi George,
>> > >
>> > > I your suggestion and double checked for typos, but
>> > unfortunately, still
>> > > nothing different - just headers in the output
files.
>> Could
>> > there be a bug
>> > > in something indirectly related to this, such as
the BOTH
>> option
>> > does not
>> > > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe U/V
winds
>> need a
>> > threshold.
>> > > Just a WAG.
>> > >
>> > > ~Scott
>> > >
>> > >
>> > >
>> > > On 6/17/21, 1:28 PM, "George McCabe via RT" <
>> met_help at ucar.edu>
>> > wrote:
>> > >
>> > > Hi Scott,
>> > >
>> > > You could try adding
>> > >
>> > > set_attr_name = "U";
>> > >
>> > > to the [FCST/OBS]_VAR4_OPTIONS and "V" for the
VAR5
>> options
>> > to see if
>> > > that
>> > > helps? It may be using the script as the name,
so
>> setting
>> > this would
>> > > override the input name that is used.
>> > >
>> > > FCST_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
>> =
>> > "U";
>> > > OBS_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
>> =
>> > "U";
>> > > FCST_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
>> =
>> > "V";
>> > > OBS_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
>> =
>> > "V";
>> > >
>> > > Let me know if that doesn't work and I can
continue
>> digging.
>> > >
>> > > Thanks,
>> > > George
>> > >
>> > > On Thu, Jun 17, 2021 at 10:10 AM Rabenhorst,
Scott D.
>> > > (GSFC-610.1)[SCIENCE
>> > > SYSTEMS AND APPLICATIONS INC] via RT <
>> met_help at ucar.edu>
>> > wrote:
>> > >
>> > > >
>> > > > <URL:
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> > > >
>> > > >
>> > > > Hi George,
>> > > >
>> > > > Thanks so much for taking a look at the code
once
>> again.
>> > You had
>> > > another
>> > > > great idea, and something rather easy to
change via
>> python
>> > embedding
>> > > > compared to using ncatted/ncrename on so many
>> files. I
>> > modified my
>> > > scripts
>> > > > to ensure all my attrs={...} values matched
for the
>> U and
>> > V fields,
>> > > but
>> > > > still not luck! I can't believe it. Since
metplus
>> doesn't
>> > appear to
>> > > display
>> > > > the output from my python embedding scripts,
I ran
>> both
>> > scripts
>> > > stand-alone
>> > > > by copying/pasting metplus' name field onto
the
>> command
>> > line. The
>> > > output is
>> > > > in the scripts/read_*2xr_v2.log files. All
>> attributes
>> > match exactly
>> > > (I even
>> > > > hardcoded the init time just to be sure). I
ran
>> with and
>> > without the
>> > > > updating the is_u/v_wind and is_grid_relative
>> attributes
>> > in the
>> > > python
>> > > > attrs object (which shouldn't be necessary),
but
>> that made
>> > no
>> > > difference
>> > > > either. The vl1l2 and vcnt files are still
just
>> header
>> > information
>> > > only
>> > > > (see tarball). I increased the verbosity of
>> grid_stat to
>> > 10, but my
>> > > eye
>> > > > didn't catch anything offhand. Maybe yo!
>> > > > u have a better idea what to look for. This
whole
>> thing
>> > is super
>> > > weird.
>> > > > Do you have any other ideas I can try?
>> > > >
>> > > > Thanks,
>> > > > Scott
>> > > >
>> > > >
>> > > >
>> > > > On 6/16/21, 4:13 PM, "George McCabe via RT" <
>> > met_help at ucar.edu>
>> > > wrote:
>> > > >
>> > > > Hi Scott,
>> > > >
>> > > > I think I see what is happening with the
U/V
>> fields. I
>> > checked
>> > > the code
>> > > > where it is checking the U/V fields and
it
>> expects the
>> > name and
>> > > level
>> > > > of
>> > > > the fcst and obs to match to make sure it
it
>> > processing the
>> > > correct U/V
>> > > > data. I noticed in the log output from
your
>> python
>> > embedding
>> > > call that
>> > > > era5pl2xr has the name "u" while
g5fcst2xr has
>> "U". I
>> > would
>> > > suggest
>> > > > modifying the python embedding script to
ensure
>> that
>> > both of the
>> > > > varname
>> > > > values are the same case.
>> > > >
>> > > > If that doesn't work, I would suggest
bumping
>> up the
>> > log
>> > > verbosity for
>> > > > GridStat (LOG_GRID_STAT_VERBOSITY) to see
if
>> there are
>> > any clues
>> > > in the
>> > > > higher number log output. If you are
still
>> having
>> > issues, you
>> > > could
>> > > > send me
>> > > > the input files for each and I can play
around
>> with
>> > the settings
>> > > to see
>> > > > what will work.
>> > > >
>> > > > Thanks,
>> > > > George
>> > > >
>> > > > On Wed, Jun 16, 2021 at 1:07 PM George
McCabe <
>> > mccabe at ucar.edu>
>> > > wrote:
>> > > >
>> > > > > Hi Scott,
>> > > > >
>> > > > > Regarding SeriesAnalysis -- there is an
>> argument to
>> > the tool
>> > > called
>> > > > > "-both" that can be used to set both
the fcst
>> and
>> > obs lists to
>> > > use
>> > > > the same
>> > > > > files. This use case runs
SeriesAnalysis on
>> GridStat
>> > output:
>> > > > >
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CaEvxnPTd%2FXCcR5PZGjT4reGDAdbN0110%2BSAqw%2BXXQI%3D&reserved=0
>> > > > >
>> > > > > This example uses
>> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
>> > > the
>> > > > input
>> > > > > file path and specifies the fcst and
obs
>> input fields
>> > > separately:
>> > > > >
>> > > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
>> FCST_precip_FULL
>> > > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
>> > > > >
>> > > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
>> OBS_precip_FULL
>> > > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
>> > > > >
>> > > > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
>> > > > >
>> > > >
>> > >
>> >
>> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
>> > > > > pairs.nc
>> > > > >
>> > > > > Regarding not being able to produce
wind
>> output,
>> > I'll take a
>> > > look at
>> > > > your
>> > > > > files and see if I can narrow down the
issue.
>> I will
>> > report
>> > > back
>> > > > with more
>> > > > > information.
>> > > > >
>> > > > > Thanks,
>> > > > > George
>> > > > >
>> > > > > On Mon, Jun 7, 2021 at 9:09 AM
Rabenhorst,
>> Scott D.
>> > > > (GSFC-610.1)[SCIENCE
>> > > > > SYSTEMS AND APPLICATIONS INC] via RT <
>> > met_help at ucar.edu>
>> > > wrote:
>> > > > >
>> > > > >>
>> > > > >> <URL:
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> > > > >
>> > > > >>
>> > > > >> Hi George,
>> > > > >>
>> > > > >> Just as a follow up thought to problem
below
>> > regarding winds
>> > > stats,
>> > > > when
>> > > > >> I look through the CNT and SL1L2
statistics
>> output,
>> > I see U
>> > > and V
>> > > > listed
>> > > > >> with all respective stats. So this
issue is
>> not a
>> > problem
>> > > reading U
>> > > > and V,
>> > > > >> it is a problem recognizing they are
vector
>> > components. Are
>> > > your
>> > > > sure there
>> > > > >> is not another flag or option to set
either
>> in the
>> > > PTHON_NUMPY/attrs
>> > > > >> objects or another option in the
metplus
>> conf file?
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Scott
>> > > > >>
>> > > > >>
>> > > > >> On 6/4/21, 12:28 PM, "Rabenhorst,
Scott D.
>> > > (GSFC-610.1)[SCIENCE
>> > > > SYSTEMS
>> > > > >> AND APPLICATIONS INC]" <
>> scott.d.rabenhorst at nasa.gov>
>> > wrote:
>> > > > >>
>> > > > >> Hi George,
>> > > > >>
>> > > > >> Thanks for answering my questions
below.
>> So
>> > sorry for
>> > > taking so
>> > > > much
>> > > > >> of your time lately. From your
description,
>> it
>> > sounds like
>> > > the best
>> > > > option
>> > > > >> for my needs is to use Stat-Analysis
to
>> aggregate
>> > SL1L2/VL1L2
>> > > data.
>> > > > My goal
>> > > > >> is to generate single-value statistics
for
>> each
>> > variable/level
>> > > > through all
>> > > > >> valid times and spatial areas. These
will be
>> > plugged into a
>> > > Green's
>> > > > >> Function algorithm for our model.
>> > > > >>
>> > > > >> However, the Series-Analysis is
also an
>> > intriguing
>> > > approach at
>> > > > least
>> > > > >> for looking at spatial errors. I guess
>> Grid-Stat
>> > could
>> > > operate on
>> > > > that data
>> > > > >> at the end to generate total stats? I
have
>> not
>> > tried using
>> > > > Series-Analysis
>> > > > >> yet, but what has confused me is the
>> documentation
>> > requires
>> > > > separate "-fcst
>> > > > >> file_1...file_n" and "-obs
file_1...file_n".
>> In
>> > most examples
>> > > I
>> > > > have seen,
>> > > > >> these are independent files. However,
you
>> make it
>> > sound as
>> > > though
>> > > > it can
>> > > > >> run on my *_pairs.nc files generated
by
>> Grid-Stat
>> > I presume
>> > > as
>> > > > long as
>> > > > >> RAW data is in there? In this case, I
assume
>> the
>> > list of files
>> > > > would be
>> > > > >> identical to both forecast and
observations?
>> Are
>> > there any
>> > > examples
>> > > > of
>> > > > >> using matched-pairs for the Series-
Analysis
>> online?
>> > > > >>
>> > > > >> Unfortunately, the larger problem
right
>> now is
>> > that I
>> > > still
>> > > > can't get
>> > > > >> Grid-Stat to process any my winds and
output
>> VCNT
>> > or VL1L2
>> > > data. I
>> > > > tried
>> > > > >> your suggestion below and added the
options
>> to my
>> > > > >>
>> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
>> > file
>> > > (please see
>> > > > >> updated attached tarball). I tried
running
>> with and
>> > without
>> > > each of
>> > > > these
>> > > > >> options "is_u(v)_wind=TRUE" and
>> > "is_grid_relative=FALSE" and
>> > > those
>> > > > ASCII
>> > > > >> files never contains any VL1L2/VCNT
data,
>> nor do
>> > the *.stat
>> > > files
>> > > > contain
>> > > > >> any of those line_types. I also
toggled
>> outputting
>> > those
>> > > attribute
>> > > > in my
>> > > > >> PYTHON_NUMPY attrs object, but that
didn't
>> work
>> > either. I
>> > > couldn't
>> > > > find any
>> > > > >> combinations that would work. In an
effort to
>> > narrow the
>> > > problem, I
>> > > > >> copy/pasted the first references to
the fcst
>> and
>> > obs fields
>> > > in my
>> > > > logfile
>> > > > >> and ran my python read scripts stand-
alone
>> and they
>> > produced
>> > > the
>> > > > expected
>> > > > >> output with correct values:
>> > > > >>
scripts/read_era5pl2xr_v2.example.log
>> > > > >>
scripts/read_g5fcst2xr_v2.example.log
>> > > > >>
>> > > > >> Furthermore, I decided to test
>> PlotDataPlane
>> > since I
>> > > recalled
>> > > > you
>> > > > >> saying that was used behind the
scenes. It
>> ran just
>> > fine and
>> > > > produced these
>> > > > >> beautiful U 500hPa images without
error:
>> > > > >>
>> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
>> > > > >>
>> > > > >> I have combed the log files for
any
>> errors or
>> > warning, but
>> > > > cannot
>> > > > >> find any at all, or anything else that
would
>> > indicate a
>> > > processing
>> > > > error.
>> > > > >> It is almost like there is some flag
that
>> has to be
>> > set in
>> > > order to
>> > > > trigger
>> > > > >> wind processing. Do you have any other
ideas
>> I can
>> > try?
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Scott
>> > > > >>
>> > > > >>
>> > > > >> On 6/3/21, 6:22 PM, "John Halley
Gotway
>> via RT"
>> > <
>> > > > met_help at ucar.edu>
>> > > > >> wrote:
>> > > > >>
>> > > > >> Hi Scott,
>> > > > >>
>> > > > >> Minna asked me to take a look
at your
>> > questions.
>> > > > >>
>> > > > >> For the second question, the
answer
>> is no.
>> > > Stat-Analysis
>> > > > reads
>> > > > >> the .stat
>> > > > >> output files created by the
MET
>> tools. It
>> > does not
>> > > read the
>> > > > >> gridded NetCDF
>> > > > >> matched pairs files generated
by the
>> > Grid-Stat tool.
>> > > > >> But I do have two
recommendations to
>> make.
>> > > > >>
>> > > > >> 1. Grid-Stat computes
statistics and
>> > partial sums that
>> > > > represent
>> > > > >> a spatial
>> > > > >> summary. You can configure
Grid-Stat
>> to
>> > compute its
>> > > output
>> > > > over
>> > > > >> whatever
>> > > > >> spatial masking regions you'd
like.
>> > Stat-Analysis can
>> > > > >> read/aggregate those
>> > > > >> spatial summaries through
time. The
>> SL1L2
>> > line type
>> > > contains
>> > > > >> scalar L1 and
>> > > > >> L2 norms. Those are written
because
>> they
>> > allow you to
>> > > easily
>> > > > >> aggregate
>> > > > >> stats over multiple runs. Many
(but
>> not
>> > all) of the
>> > > > continuous
>> > > > >> statistics
>> > > > >> (like ME, RMSE, MAE, ACC, and
so on)
>> can be
>> > derived
>> > > from the
>> > > > >> SL1L2 line
>> > > > >> type. So I'd recommend trying
this
>> type of
>> > > stat_analysis
>> > > > job:
>> > > > >>
>> > > > >> -job aggregate_stat -line_type
SL1L2
>> > -out_line_type
>> > > CNT -by
>> > > > >> FCST_VAR,FCST_LEV,VX_MASK
-out_stat
>> > sl1l2_to_cnt.stat
>> > > > >>
>> > > > >> For each unique combination of
>> variable,
>> > level, and
>> > > masking
>> > > > >> region, it'll
>> > > > >> aggregate all the SL1L2 lines
it
>> finds in
>> > the input
>> > > and
>> > > > report the
>> > > > >> corresponding continuous
statistics.
>> > > > >>
>> > > > >> 2. Another option is running
the
>> NetCDF
>> > matched pairs
>> > > > produced by
>> > > > >> Grid-Stat
>> > > > >> through the Series-Analysis
tool.
>> That'll
>> > compute
>> > > stats
>> > > > through
>> > > > >> time
>> > > > >> separately for each individual
grid
>> point.
>> > The result
>> > > with
>> > > > be a
>> > > > >> map of
>> > > > >> statistics rather than a
spatial
>> summary.
>> > > > >>
>> > > > >> For the first question, I see
that
>> you're
>> > using python
>> > > > embedding
>> > > > >> and are
>> > > > >> trying to compute vector
partial sums
>> > output. In
>> > > general,
>> > > > when
>> > > > >> you request
>> > > > >> an output line type, Grid-Stat
will
>> create
>> > the
>> > > corresponding
>> > > > >> output file.
>> > > > >> But if it's empty, that means
that it
>> > didn't have any
>> > > input
>> > > > data
>> > > > >> suitable
>> > > > >> for creating that output type.
So
>> you've
>> > requested
>> > > vector
>> > > > stats,
>> > > > >> but
>> > > > >> Grid-Stat isn't computing any.
>> > > > >>
>> > > > >> Thanks for sending the sample
output.
>> > Looking in:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
>> > > > >>
>> > > > >> I do see several warning that
you
>> should
>> > take a look
>> > > at:
>> > > > >> WARNING: python_dataplane() ->
an
>> error
>> > occurred
>> > > importing
>> > > > module
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
>> > > > >>
>> > > > >> But I do also see that Grid-
Stat is
>> > processing the U
>> > > and V
>> > > > data
>> > > > >> and
>> > > > >> reporting output as scalar
variables
>> in the
>> > CNT.txt
>> > > file:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
>> > > > >>
>> > > > >> Now I haven't tested this on
your
>> data to
>> > confirm
>> > > that it'll
>> > > > >> actually work,
>> > > > >> but we want to set these flags
in
>> the MET
>> > config file:
>> > > > >> is_u_wind = true
>> > > > >> is_v_wind = true
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VnwsgB2i2RQwVAIUEfnO3xZLhClxh%2Bqb0q7b78HQ0YU%3D&reserved=0
>> > > > >>
>> > > > >> Checking in
>> > > > >>
>> > RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
>> > > > >> see that var 4 is for U and
var 5 is
>> for V.
>> > > > >> Add those via the METplus
config
>> file by
>> > adding:
>> > > > >> FCST_VAR4_OPTIONS = is_u_wind
= TRUE;
>> > > > >> OBS_VAR4_OPTIONS = is_u_wind =
TRUE;
>> > > > >> FCST_VAR5_OPTIONS = is_v_wind
= TRUE;
>> > > > >> OBS_VAR5_OPTIONS = is_v_wind =
TRUE;
>> > > > >>
>> > > > >> Please give that a shot and
let me
>> know how
>> > it goes.
>> > > The
>> > > > goal is
>> > > > >> NOT having
>> > > > >> any U and V show up in the CNT
output
>> > file... and
>> > > instead
>> > > > have
>> > > > >> entries show
>> > > > >> up in the VCNT output.
>> > > > >>
>> > > > >> One last detail... if you do
want to
>> > aggregate results
>> > > > through
>> > > > >> time, I'd
>> > > > >> recommend writing the SL1L2
and
>> VL1L2 line
>> > types to
>> > > enable
>> > > > that
>> > > > >> aggregation
>> > > > >> to be done later.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> John
>> > > > >>
>> > > > >> On Thu, Jun 3, 2021 at 1:49 PM
Minna
>> Win
>> > via RT <
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >> >
>> > > > >> > <URL:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> > > > >> >
>> > > > >> >
>> > > > >> > Hi Scott,
>> > > > >> >
>> > > > >> > It looks like you've set
your MET
>> config
>> > file to
>> > > correctly
>> > > > >> identify which
>> > > > >> > linetypes you'd like to see.
I'm
>> not sure
>> > why you
>> > > are
>> > > > >> essentially getting
>> > > > >> > empty output files. I'll
need
>> to find
>> > someone
>> > > who can
>> > > > answer
>> > > > >> your python
>> > > > >> > embedding and grid-stat
questions.
>> > > > >> >
>> > > > >> > Regards,
>> > > > >> > Minna
>> > > > >> > ---------------
>> > > > >> > Minna Win
>> > > > >> > Pronouns: she/her
>> > > > >> > National Center for
Atmospheric
>> Research
>> > > > >> > Developmental Testbed Center
>> > > > >> > Phone: 303-497-8423
>> > > > >> > Fax: 303-497-8401
>> > > > >> > ---------------
>> > > > >> >
>> > > > >> >
>> > > > >> >
>> > > > >> > On Thu, Jun 3, 2021 at 12:04
PM
>> > Rabenhorst, Scott D.
>> > > > >> (GSFC-610.1)[SCIENCE
>> > > > >> > SYSTEMS AND APPLICATIONS
INC] via
>> RT <
>> > > met_help at ucar.edu>
>> > > > wrote:
>> > > > >> >
>> > > > >> > >
>> > > > >> > > Thu Jun 03 12:04:14 2021:
Request
>> > 100090 was
>> > > acted upon.
>> > > > >> > > Transaction: Ticket
created by
>> > > > scott.d.rabenhorst at nasa.gov
>> > > > >> > > Queue: met_help
>> > > > >> > > Subject: 2 questions
>> regarding how
>> > to
>> > > generate
>> > > > specific
>> > > > >> stats
>> > > > >> > > Owner: Nobody
>> > > > >> > > Requestors:
>> > scott.d.rabenhorst at nasa.gov
>> > > > >> > > Status: new
>> > > > >> > > Ticket <URL:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C70969ee8e7d34e4d28f508d9326fcf07%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637596278074124031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=en1bAf5uI87UbWWun1c5dIXfT42Dp7CtuTcRZGE5QOk%3D&reserved=0
>> > > > >> >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Hi met support,
>> > > > >> > >
>> > > > >> > > I have 2 questions
regarding how
>> to
>> > output
>> > > specific
>> > > > >> statistics for my
>> > > > >> > > experiment (see files in
my
>> attached
>> > tarball). I
>> > > have
>> > > > looked
>> > > > >> through the
>> > > > >> > > online documentation, but
I have
>> had a
>> > hard time
>> > > > finding a
>> > > > >> direct
>> > > > >> > > answer/solution – perhaps
I’ve
>> simply
>> > missed
>> > > something
>> > > > >> obvious.
>> > > > >> > >
>> > > > >> > > QUESTION 1: how to output
>> wind/vector
>> > (VL1L2,
>> > > VCNT)
>> > > > >> statistics. I have
>> > > > >> > > added:
>> > > > >> > >
>> > > > >> > > GRID_STAT_OUTPUT_FLAG_CNT
=
>> BOTH
>> > > > >> > > GRID_STAT_OUTPUT_FLAG_VCNT
=
>> BOTH
>> > > > >> > >
GRID_STAT_OUTPUT_FLAG_VL1L2 =
>> BOTH
>> > > > >> > >
>> > > > >> > > … to my
>> > > >
>> ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
>> > > > >> file. I
>> > > > >> > have
>> > > > >> > > also added:
>> > > > >> > >
>> > > > >> > > is_u_wind : True
>> > > > >> > > is_grid_relative: False
>> > > > >> > >
>> > > > >> > > … attributes to my
PYTHON_NUMPY
>> object
>> > through
>> > > python
>> > > > >> embedding. However,
>> > > > >> > > when I look at the output
>> *_vl1l2.txt
>> > files, the
>> > > only
>> > > > >> information is the
>> > > > >> > > header lines. I have
included 2
>> sample
>> > outputs
>> > > from my
>> > > > >> embedding scripts
>> > > > >> > > when called alone
>> > > (./scripts/read_*2xr_v2.example.log)
>> > > > which
>> > > > >> show all the
>> > > > >> > > data is as expected. It
seems
>> like
>> > something else
>> > > needs
>> > > > to be
>> > > > >> activated
>> > > > >> > > somewhere in the conf
files, but
>> I
>> > can’t seem to
>> > > find
>> > > > the
>> > > > >> correct switch.
>> > > > >> > > If you have any ideas for
other
>> things
>> > to try,
>> > > please
>> > > > let me
>> > > > >> know.
>> > > > >> > >
>> > > > >> > > QUESTION 2: is there a way
to use
>> > stat_analysis to
>> > > > >> recompute/aggregate
>> > > > >> > > over my *_pairs.nc files
>> containing
>> > this
>> > > information
>> > > > >> (example ncdump in
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
>> > > > >> > > and output CNT stats for
the
>> total of
>> > all the
>> > > matched
>> > > > pair
>> > > > >> data? I am
>> > > > >> > aware
>> > > > >> > > of the “-job summary
-line_type
>> CNT
>> > -column RMSE
>> > > -by
>> > > > >> FCST_VAR,FCST_LEV
>> > > > >> > > -dump_row” but this looks
like
>> it only
>> > provides
>> > > basic
>> > > > >> statistics of all
>> > > > >> > the
>> > > > >> > > RMSE values in each
*_cnt.txt
>> files,
>> > correct?
>> > > What I
>> > > > really
>> > > > >> want is to
>> > > > >> > > generate actual CNT data
from all
>> > matched pairs
>> > > over all
>> > > > >> times. I’ve
>> > > > >> > tried
>> > > > >> > > various combinations of
>> aggregate and
>> > > aggregate_stat but
>> > > > >> can’t seem to
>> > > > >> > find
>> > > > >> > > a combination that will
work. Is
>> this
>> > possible?
>> > > If so,
>> > > > what
>> > > > >> should my
>> > > > >> > > stat_analysis job command
look
>> like?
>> > > > >> > >
>> > > > >> > > Thanks so much for your
time in
>> > advance. I have
>> > > asked a
>> > > > lot
>> > > > >> of questions
>> > > > >> > > lately, but I think I’m
almost
>> there.
>> > Any help
>> > > you can
>> > > > >> provide on filling
>> > > > >> > > in my knowledge gaps would
be
>> very much
>> > > appreciated.
>> > > > >> > >
>> > > > >> > > Thanks,
>> > > > >> > > Scott
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >
>> > > > > --
>> > > > > George McCabe - Software Engineer III
>> > > > > National Center for Atmospheric
Research
>> > > > > Research Applications Laboratory
>> > > > > 303-497-2768
>> > > > > ---
>> > > > > My working day may not be your working
day.
>> Please
>> > do not feel
>> > > > obliged to
>> > > > > reply to this email outside of your
normal
>> working
>> > hours.
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > George McCabe - Software Engineer III
>> > > > National Center for Atmospheric Research
>> > > > Research Applications Laboratory
>> > > > 303-497-2768
>> > > > ---
>> > > > My working day may not be your working
day.
>> Please do
>> > not feel
>> > > obliged
>> > > > to
>> > > > reply to this email outside of your
normal
>> working
>> > hours.
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> > > --
>> > > George McCabe - Software Engineer III
>> > > National Center for Atmospheric Research
>> > > Research Applications Laboratory
>> > > 303-497-2768
>> > > ---
>> > > My working day may not be your working day.
Please do
>> not
>> > feel obliged
>> > > to
>> > > reply to this email outside of your normal
working
>> hours.
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>> > --
>> > George McCabe - Software Engineer III
>> > National Center for Atmospheric Research
>> > Research Applications Laboratory
>> > 303-497-2768
>> > ---
>> > My working day may not be your working day. Please do
not
>> feel
>> > obliged to
>> > reply to this email outside of your normal working
hours.
>> >
>> >
>> >
>> >
>> >
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
>> obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>>
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100090] 2 questions regarding how to generate specific stats
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Tue Jun 22 08:14:26 2021
Hi George,
Thanks so much for creating an issue for this and for continuing to
ponder a work around. In the interim, I am certainly open to combining
u/v winds in python somehow, but I'm unsure exactly how the MET
package does this in order to drive the VL1L2, VCNT output. Are u and
v paired in an combined array somehow that met uses, or is it
sufficient to just generated additional wind speed and direction
variables? Again, are there key attributes specifically designated to
recognize wind speed and direction variables when using python
embedding (similar to "is_u_wind") or does met key off a standard
name?
Thanks,
Scott
On 6/22/21, 8:50 AM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
This morning I thought to try to use set_attr_level to override
the level
values to see if that would allow the U/V components to be
combined
properly, but that also did not seem to have any affect. While
searching
for another example of computing wind vectors from U/V I found
another use
case that reads U and V fields in a Python embedding script and
computes
U/V to be used in calculating IVT. I'm not sure that this will
give you the
same result that you are expecting, but you could potentially
combine U and
V inside your Python embedding script instead of relying on the
MET tools
to perform that calculation. I am still going to write up an issue
to
investigate the potential deficiency in the logic, but this may
help you
move forward with your analysis for the time being.
Thanks,
George
On Mon, Jun 21, 2021 at 6:50 PM George McCabe <mccabe at ucar.edu>
wrote:
> Hi Scott,
>
> Thanks for sending the input files. I did some digging and I
believe this
> may require an enhancement to the logic for handling Python
embedding. The
> value for level is typically not needed for Python embedding
because all of
> the information is provided to the name value. However, it looks
like the
> logic to compute U/V vector partial sums uses the level value to
match up
> the U and V fields. The developer(s) who are more familiar with
this logic
> in the C++ code are currently on vacation, but I will create a
GitHub issue
> to investigate this and create a fix.
>
> Thanks,
> George
>
> On Mon, Jun 21, 2021 at 10:17 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
>>
>> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
>
>>
>> Hi George,
>>
>> Thanks for getting back in touch with me so soon. Sorry for the
delay.
>> I'm back at work today.
>>
>> This morning I simplified my grid_stat config file so it is
only reading
>> a few select fields on pressure levels. Hopefully, this will
avoid the
>> complications of the surface data with time offsets and all the
other
>> non-conventional comparisons with masking that you have
noticed. This
>> latest run demonstrates the problem - still no vector stats. I
uploaded 3
>> files to the ucar ftp site:
>>
>> 1. geosgcm_pres.20191223_0000z.nc4.gz (forecast data time
sliced to index
>> 26 the observation comparison time)
>> 2. era5_pl-allvar.20191223_00z.nc4.gz (observation data)
>> 3. expfiles.5.tar.gz (logs/scripts from my latest run)
>>
>> Just as a reminder, my python read scripts search through a
directory
>> hierarchy to locate the proper forecast and observation files
for
>> comparison. This will not work for you as is. You will have to
change and
>> hard-code the file paths to make this work on your side.
>>
>> Please let me know if you have any issues or need any further
>> information. I am eager and hopeful we can finally solve this
issue.
>> Looking forward to hearing from you.
>>
>> Thanks,
>> Scott
>>
>>
>> On 6/18/21, 11:43 AM, "George McCabe via RT"
<met_help at ucar.edu> wrote:
>>
>> Hi Scott,
>>
>> Thanks for sending the new log file. We do have an FTP site
that you
>> can
>> use to send us data. Information on how is on this webpage
under the
>> "How
>> To Send Us Data" heading:
>>
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fget-
help&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8sUOP2ha66gn6HCbXgHov9p%2Fw%2BlKakoVAHQvTlPoFCI%3D&reserved=0
>> Those files are quite large, but I would really only need
the U and V
>> fields from each data set to debug why that logic is not
working
>> properly.
>>
>> I did also notice that there are some warnings in your log
file
>> saying that
>> there is a time mismatch between the forecast and
observation data
>> for some
>> of the other fields. It will still use the data, but the
results may
>> not
>> reflect what you are expecting. The valid time that comes
from the
>> forecast
>> script doesn't match the valid time passed into the script,
so there
>> may be
>> something going on there. This isn't related to the U/V
issue but I
>> thought
>> I would point it out.
>>
>>
>> {
>>
>>
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>>
>>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
>> TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
>> gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
>>
>> DEBUG 3: Running user's python script
>>
>>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
>> DEBUG 4:
>> DEBUG 4: Data plane information:
>> DEBUG 4: plane min: 0
>> DEBUG 4: plane max: 0.01865353266
>> DEBUG 4: valid time: *20191223_003000*
>> DEBUG 4: lead time: 273000
>> DEBUG 4: init time: 20191221_210000
>> DEBUG 4: accum time: 000000
>> DEBUG 4:
>> DEBUG 3: Reading forecast data for TTPRCP/Surface.
>> DEBUG 3: Running user's python script
>>
>>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
>> DEBUG 4:
>> DEBUG 4: Data plane information:
>> DEBUG 4: plane min: 0
>> DEBUG 4: plane max: 0.02649974823
>> DEBUG 4: valid time: *20191223_000000*
>> DEBUG 4: lead time: 000000
>> DEBUG 4: init time: 20191223_000000
>> DEBUG 4: accum time: 010000
>> DEBUG 4:
>> DEBUG 3: Reading observation data for tp/Surface.
>> WARNING:
>> WARNING: process_scores() -> Forecast and observation valid
times do
>> not
>> match 20191223_003000 != 20191223_000000 for TTPRCP/Surface
versus
>> tp/Surface.
>>
>> On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D.
>> (GSFC-610.1)[SCIENCE
>> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>>
>> >
>> > <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
>> >
>> >
>> > Hi George,
>> >
>> > I apologize, my last email sent prematurely for some
weird reason.
>> As I
>> > was saying below, our forecast and observation files are
very
>> large. I
>> > could ncks a subset, but that really wouldn't be the same
file. I
>> suppose
>> > it would contain the same fields, but even a subset would
be too
>> large for
>> > an email attachment.
>> >
>> > Regarding the output below, you are correct. I did not
modify the
>> 10m U/V
>> > winds (U10M, V10M, etc). I only changed the 3D pressure
level U/V
>> winds to
>> > have the same name and attributes. I figured if those
started
>> working I
>> > would propagate the changes to the surface python scripts
too. So I
>> would
>> > recommend to only look at the output pertaining to
pressure level
>> winds
>> > from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py
scripts.
>> Those have
>> > the changes.
>> >
>> > Please let me know how to proceed with posting the
>> forecast/observation
>> > files if you think that will help. Do you have an ftp
site?
>> >
>> > Thanks,
>> > Scott
>> >
>> >
>> >
>> >
>> > On 6/17/21, 10:32 PM, "Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
>> SYSTEMS
>> > AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov>
wrote:
>> >
>> > Hi George,
>> >
>> > I have attached the log file - that is easy. However,
I'm not
>> sure how
>> > to go about sending you the data. The experiment
(forecast) 3D file
>> is 734G
>> > and the surface (single) level files are 619M each.
>> >
>> > On 6/17/21, 5:01 PM, "George McCabe via RT"
<met_help at ucar.edu>
>> wrote:
>> >
>> > Hi Scott,
>> >
>> > The VL1L2 flag should accept BOTH. I'm not
familiar enough
>> with
>> > the U/V
>> > calculations to know if it needs a threshold but
John didn't
>> > mention that
>> > it was a requirement in his first email. Could
you send me
>> a set
>> > of files
>> > that are used as input for this use case? Also,
the latest
>> log
>> > file would
>> > be helpful. I noticed that the name of the fields
did
>> reflect the
>> > script
>> > name, so I'm curious to see if it was overridden
properly
>> in your
>> > last run.
>> >
>> > | | | | | | | Dictionary
Entry[0] ...
>> > | | | | | | | | Name =
*name*
>> > | | | | | | | | Type =
StringType
>> > | | | | | | | | *String
Value =
>> >
>> >
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
>> >
>> >
>>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
>> > U10M 20191221_2100 20191223_0000"*
>> > | | | | | | | Dictionary
Entry[1] ...
>> > | | | | | | | | Name =
is_u_wind
>> > | | | | | | | | Type =
BooleanType
>> > | | | | | | | | Boolean
Value = true
>> > | | | | | | | Dictionary
Entry[2] ...
>> > | | | | | | | | Name =
>> is_grid_relative
>> > | | | | | | | | Type =
BooleanType
>> > | | | | | | | | Boolean
Value =
>> false
>> >
>> > Thanks,
>> > George
>> >
>> > On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst,
Scott D.
>> > (GSFC-610.1)[SCIENCE
>> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
>> wrote:
>> >
>> > >
>> > > <URL:
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
>> > >
>> > >
>> > > Hi George,
>> > >
>> > > I your suggestion and double checked for typos,
but
>> > unfortunately, still
>> > > nothing different - just headers in the output
files.
>> Could
>> > there be a bug
>> > > in something indirectly related to this, such
as the BOTH
>> option
>> > does not
>> > > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe
U/V winds
>> need a
>> > threshold.
>> > > Just a WAG.
>> > >
>> > > ~Scott
>> > >
>> > >
>> > >
>> > > On 6/17/21, 1:28 PM, "George McCabe via RT" <
>> met_help at ucar.edu>
>> > wrote:
>> > >
>> > > Hi Scott,
>> > >
>> > > You could try adding
>> > >
>> > > set_attr_name = "U";
>> > >
>> > > to the [FCST/OBS]_VAR4_OPTIONS and "V" for
the VAR5
>> options
>> > to see if
>> > > that
>> > > helps? It may be using the script as the
name, so
>> setting
>> > this would
>> > > override the input name that is used.
>> > >
>> > > FCST_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
>> =
>> > "U";
>> > > OBS_VAR4_OPTIONS = is_u_wind = TRUE;
set_attr_name
>> =
>> > "U";
>> > > FCST_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
>> =
>> > "V";
>> > > OBS_VAR5_OPTIONS = is_v_wind = TRUE;
set_attr_name
>> =
>> > "V";
>> > >
>> > > Let me know if that doesn't work and I can
continue
>> digging.
>> > >
>> > > Thanks,
>> > > George
>> > >
>> > > On Thu, Jun 17, 2021 at 10:10 AM
Rabenhorst, Scott D.
>> > > (GSFC-610.1)[SCIENCE
>> > > SYSTEMS AND APPLICATIONS INC] via RT <
>> met_help at ucar.edu>
>> > wrote:
>> > >
>> > > >
>> > > > <URL:
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
>> > > >
>> > > >
>> > > > Hi George,
>> > > >
>> > > > Thanks so much for taking a look at the
code once
>> again.
>> > You had
>> > > another
>> > > > great idea, and something rather easy to
change via
>> python
>> > embedding
>> > > > compared to using ncatted/ncrename on so
many
>> files. I
>> > modified my
>> > > scripts
>> > > > to ensure all my attrs={...} values
matched for the
>> U and
>> > V fields,
>> > > but
>> > > > still not luck! I can't believe it. Since
metplus
>> doesn't
>> > appear to
>> > > display
>> > > > the output from my python embedding
scripts, I ran
>> both
>> > scripts
>> > > stand-alone
>> > > > by copying/pasting metplus' name field
onto the
>> command
>> > line. The
>> > > output is
>> > > > in the scripts/read_*2xr_v2.log files.
All
>> attributes
>> > match exactly
>> > > (I even
>> > > > hardcoded the init time just to be sure).
I ran
>> with and
>> > without the
>> > > > updating the is_u/v_wind and
is_grid_relative
>> attributes
>> > in the
>> > > python
>> > > > attrs object (which shouldn't be
necessary), but
>> that made
>> > no
>> > > difference
>> > > > either. The vl1l2 and vcnt files are
still just
>> header
>> > information
>> > > only
>> > > > (see tarball). I increased the verbosity
of
>> grid_stat to
>> > 10, but my
>> > > eye
>> > > > didn't catch anything offhand. Maybe yo!
>> > > > u have a better idea what to look for.
This whole
>> thing
>> > is super
>> > > weird.
>> > > > Do you have any other ideas I can try?
>> > > >
>> > > > Thanks,
>> > > > Scott
>> > > >
>> > > >
>> > > >
>> > > > On 6/16/21, 4:13 PM, "George McCabe via
RT" <
>> > met_help at ucar.edu>
>> > > wrote:
>> > > >
>> > > > Hi Scott,
>> > > >
>> > > > I think I see what is happening with
the U/V
>> fields. I
>> > checked
>> > > the code
>> > > > where it is checking the U/V fields
and it
>> expects the
>> > name and
>> > > level
>> > > > of
>> > > > the fcst and obs to match to make
sure it it
>> > processing the
>> > > correct U/V
>> > > > data. I noticed in the log output
from your
>> python
>> > embedding
>> > > call that
>> > > > era5pl2xr has the name "u" while
g5fcst2xr has
>> "U". I
>> > would
>> > > suggest
>> > > > modifying the python embedding script
to ensure
>> that
>> > both of the
>> > > > varname
>> > > > values are the same case.
>> > > >
>> > > > If that doesn't work, I would suggest
bumping
>> up the
>> > log
>> > > verbosity for
>> > > > GridStat (LOG_GRID_STAT_VERBOSITY) to
see if
>> there are
>> > any clues
>> > > in the
>> > > > higher number log output. If you are
still
>> having
>> > issues, you
>> > > could
>> > > > send me
>> > > > the input files for each and I can
play around
>> with
>> > the settings
>> > > to see
>> > > > what will work.
>> > > >
>> > > > Thanks,
>> > > > George
>> > > >
>> > > > On Wed, Jun 16, 2021 at 1:07 PM
George McCabe <
>> > mccabe at ucar.edu>
>> > > wrote:
>> > > >
>> > > > > Hi Scott,
>> > > > >
>> > > > > Regarding SeriesAnalysis -- there
is an
>> argument to
>> > the tool
>> > > called
>> > > > > "-both" that can be used to set
both the fcst
>> and
>> > obs lists to
>> > > use
>> > > > the same
>> > > > > files. This use case runs
SeriesAnalysis on
>> GridStat
>> > output:
>> > > > >
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mS8sL5XTA6idlmqx%2FHdJOFSq0RXaRLRFW71c7hJYHcs%3D&reserved=0
>> > > > >
>> > > > > This example uses
>> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
>> > > the
>> > > > input
>> > > > > file path and specifies the fcst
and obs
>> input fields
>> > > separately:
>> > > > >
>> > > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
>> FCST_precip_FULL
>> > > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
>> > > > >
>> > > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
>> OBS_precip_FULL
>> > > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS =
"(*,*)"
>> > > > >
>> > > > > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE
=
>> > > > >
>> > > >
>> > >
>> >
>> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
>> > > > > pairs.nc
>> > > > >
>> > > > > Regarding not being able to produce
wind
>> output,
>> > I'll take a
>> > > look at
>> > > > your
>> > > > > files and see if I can narrow down
the issue.
>> I will
>> > report
>> > > back
>> > > > with more
>> > > > > information.
>> > > > >
>> > > > > Thanks,
>> > > > > George
>> > > > >
>> > > > > On Mon, Jun 7, 2021 at 9:09 AM
Rabenhorst,
>> Scott D.
>> > > > (GSFC-610.1)[SCIENCE
>> > > > > SYSTEMS AND APPLICATIONS INC] via
RT <
>> > met_help at ucar.edu>
>> > > wrote:
>> > > > >
>> > > > >>
>> > > > >> <URL:
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
>> > > > >
>> > > > >>
>> > > > >> Hi George,
>> > > > >>
>> > > > >> Just as a follow up thought to
problem below
>> > regarding winds
>> > > stats,
>> > > > when
>> > > > >> I look through the CNT and SL1L2
statistics
>> output,
>> > I see U
>> > > and V
>> > > > listed
>> > > > >> with all respective stats. So this
issue is
>> not a
>> > problem
>> > > reading U
>> > > > and V,
>> > > > >> it is a problem recognizing they
are vector
>> > components. Are
>> > > your
>> > > > sure there
>> > > > >> is not another flag or option to
set either
>> in the
>> > > PTHON_NUMPY/attrs
>> > > > >> objects or another option in the
metplus
>> conf file?
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Scott
>> > > > >>
>> > > > >>
>> > > > >> On 6/4/21, 12:28 PM, "Rabenhorst,
Scott D.
>> > > (GSFC-610.1)[SCIENCE
>> > > > SYSTEMS
>> > > > >> AND APPLICATIONS INC]" <
>> scott.d.rabenhorst at nasa.gov>
>> > wrote:
>> > > > >>
>> > > > >> Hi George,
>> > > > >>
>> > > > >> Thanks for answering my
questions below.
>> So
>> > sorry for
>> > > taking so
>> > > > much
>> > > > >> of your time lately. From your
description,
>> it
>> > sounds like
>> > > the best
>> > > > option
>> > > > >> for my needs is to use Stat-
Analysis to
>> aggregate
>> > SL1L2/VL1L2
>> > > data.
>> > > > My goal
>> > > > >> is to generate single-value
statistics for
>> each
>> > variable/level
>> > > > through all
>> > > > >> valid times and spatial areas.
These will be
>> > plugged into a
>> > > Green's
>> > > > >> Function algorithm for our model.
>> > > > >>
>> > > > >> However, the Series-Analysis
is also an
>> > intriguing
>> > > approach at
>> > > > least
>> > > > >> for looking at spatial errors. I
guess
>> Grid-Stat
>> > could
>> > > operate on
>> > > > that data
>> > > > >> at the end to generate total
stats? I have
>> not
>> > tried using
>> > > > Series-Analysis
>> > > > >> yet, but what has confused me is
the
>> documentation
>> > requires
>> > > > separate "-fcst
>> > > > >> file_1...file_n" and "-obs
file_1...file_n".
>> In
>> > most examples
>> > > I
>> > > > have seen,
>> > > > >> these are independent files.
However, you
>> make it
>> > sound as
>> > > though
>> > > > it can
>> > > > >> run on my *_pairs.nc files
generated by
>> Grid-Stat
>> > I presume
>> > > as
>> > > > long as
>> > > > >> RAW data is in there? In this
case, I assume
>> the
>> > list of files
>> > > > would be
>> > > > >> identical to both forecast and
observations?
>> Are
>> > there any
>> > > examples
>> > > > of
>> > > > >> using matched-pairs for the
Series-Analysis
>> online?
>> > > > >>
>> > > > >> Unfortunately, the larger
problem right
>> now is
>> > that I
>> > > still
>> > > > can't get
>> > > > >> Grid-Stat to process any my winds
and output
>> VCNT
>> > or VL1L2
>> > > data. I
>> > > > tried
>> > > > >> your suggestion below and added
the options
>> to my
>> > > > >>
>> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
>> > file
>> > > (please see
>> > > > >> updated attached tarball). I tried
running
>> with and
>> > without
>> > > each of
>> > > > these
>> > > > >> options "is_u(v)_wind=TRUE" and
>> > "is_grid_relative=FALSE" and
>> > > those
>> > > > ASCII
>> > > > >> files never contains any
VL1L2/VCNT data,
>> nor do
>> > the *.stat
>> > > files
>> > > > contain
>> > > > >> any of those line_types. I also
toggled
>> outputting
>> > those
>> > > attribute
>> > > > in my
>> > > > >> PYTHON_NUMPY attrs object, but
that didn't
>> work
>> > either. I
>> > > couldn't
>> > > > find any
>> > > > >> combinations that would work. In
an effort to
>> > narrow the
>> > > problem, I
>> > > > >> copy/pasted the first references
to the fcst
>> and
>> > obs fields
>> > > in my
>> > > > logfile
>> > > > >> and ran my python read scripts
stand-alone
>> and they
>> > produced
>> > > the
>> > > > expected
>> > > > >> output with correct values:
>> > > > >>
scripts/read_era5pl2xr_v2.example.log
>> > > > >>
scripts/read_g5fcst2xr_v2.example.log
>> > > > >>
>> > > > >> Furthermore, I decided to test
>> PlotDataPlane
>> > since I
>> > > recalled
>> > > > you
>> > > > >> saying that was used behind the
scenes. It
>> ran just
>> > fine and
>> > > > produced these
>> > > > >> beautiful U 500hPa images without
error:
>> > > > >>
>> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
>> > > > >>
>> > > > >> I have combed the log files
for any
>> errors or
>> > warning, but
>> > > > cannot
>> > > > >> find any at all, or anything else
that would
>> > indicate a
>> > > processing
>> > > > error.
>> > > > >> It is almost like there is some
flag that
>> has to be
>> > set in
>> > > order to
>> > > > trigger
>> > > > >> wind processing. Do you have any
other ideas
>> I can
>> > try?
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Scott
>> > > > >>
>> > > > >>
>> > > > >> On 6/3/21, 6:22 PM, "John
Halley Gotway
>> via RT"
>> > <
>> > > > met_help at ucar.edu>
>> > > > >> wrote:
>> > > > >>
>> > > > >> Hi Scott,
>> > > > >>
>> > > > >> Minna asked me to take a
look at your
>> > questions.
>> > > > >>
>> > > > >> For the second question,
the answer
>> is no.
>> > > Stat-Analysis
>> > > > reads
>> > > > >> the .stat
>> > > > >> output files created by
the MET
>> tools. It
>> > does not
>> > > read the
>> > > > >> gridded NetCDF
>> > > > >> matched pairs files
generated by the
>> > Grid-Stat tool.
>> > > > >> But I do have two
recommendations to
>> make.
>> > > > >>
>> > > > >> 1. Grid-Stat computes
statistics and
>> > partial sums that
>> > > > represent
>> > > > >> a spatial
>> > > > >> summary. You can configure
Grid-Stat
>> to
>> > compute its
>> > > output
>> > > > over
>> > > > >> whatever
>> > > > >> spatial masking regions
you'd like.
>> > Stat-Analysis can
>> > > > >> read/aggregate those
>> > > > >> spatial summaries through
time. The
>> SL1L2
>> > line type
>> > > contains
>> > > > >> scalar L1 and
>> > > > >> L2 norms. Those are
written because
>> they
>> > allow you to
>> > > easily
>> > > > >> aggregate
>> > > > >> stats over multiple runs.
Many (but
>> not
>> > all) of the
>> > > > continuous
>> > > > >> statistics
>> > > > >> (like ME, RMSE, MAE, ACC,
and so on)
>> can be
>> > derived
>> > > from the
>> > > > >> SL1L2 line
>> > > > >> type. So I'd recommend
trying this
>> type of
>> > > stat_analysis
>> > > > job:
>> > > > >>
>> > > > >> -job aggregate_stat
-line_type SL1L2
>> > -out_line_type
>> > > CNT -by
>> > > > >> FCST_VAR,FCST_LEV,VX_MASK
-out_stat
>> > sl1l2_to_cnt.stat
>> > > > >>
>> > > > >> For each unique
combination of
>> variable,
>> > level, and
>> > > masking
>> > > > >> region, it'll
>> > > > >> aggregate all the SL1L2
lines it
>> finds in
>> > the input
>> > > and
>> > > > report the
>> > > > >> corresponding continuous
statistics.
>> > > > >>
>> > > > >> 2. Another option is
running the
>> NetCDF
>> > matched pairs
>> > > > produced by
>> > > > >> Grid-Stat
>> > > > >> through the Series-
Analysis tool.
>> That'll
>> > compute
>> > > stats
>> > > > through
>> > > > >> time
>> > > > >> separately for each
individual grid
>> point.
>> > The result
>> > > with
>> > > > be a
>> > > > >> map of
>> > > > >> statistics rather than a
spatial
>> summary.
>> > > > >>
>> > > > >> For the first question, I
see that
>> you're
>> > using python
>> > > > embedding
>> > > > >> and are
>> > > > >> trying to compute vector
partial sums
>> > output. In
>> > > general,
>> > > > when
>> > > > >> you request
>> > > > >> an output line type, Grid-
Stat will
>> create
>> > the
>> > > corresponding
>> > > > >> output file.
>> > > > >> But if it's empty, that
means that it
>> > didn't have any
>> > > input
>> > > > data
>> > > > >> suitable
>> > > > >> for creating that output
type. So
>> you've
>> > requested
>> > > vector
>> > > > stats,
>> > > > >> but
>> > > > >> Grid-Stat isn't computing
any.
>> > > > >>
>> > > > >> Thanks for sending the
sample output.
>> > Looking in:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
>> > > > >>
>> > > > >> I do see several warning
that you
>> should
>> > take a look
>> > > at:
>> > > > >> WARNING:
python_dataplane() -> an
>> error
>> > occurred
>> > > importing
>> > > > module
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
>> > > > >>
>> > > > >> But I do also see that
Grid-Stat is
>> > processing the U
>> > > and V
>> > > > data
>> > > > >> and
>> > > > >> reporting output as scalar
variables
>> in the
>> > CNT.txt
>> > > file:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
>> > > > >>
>> > > > >> Now I haven't tested this
on your
>> data to
>> > confirm
>> > > that it'll
>> > > > >> actually work,
>> > > > >> but we want to set these
flags in
>> the MET
>> > config file:
>> > > > >> is_u_wind = true
>> > > > >> is_v_wind = true
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TSbsmVExDOLnIasq22jGMa1QQtf%2B4zyE4VMy5PYlMik%3D&reserved=0
>> > > > >>
>> > > > >> Checking in
>> > > > >>
>> >
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
>> > > > >> see that var 4 is for U
and var 5 is
>> for V.
>> > > > >> Add those via the METplus
config
>> file by
>> > adding:
>> > > > >> FCST_VAR4_OPTIONS =
is_u_wind = TRUE;
>> > > > >> OBS_VAR4_OPTIONS =
is_u_wind = TRUE;
>> > > > >> FCST_VAR5_OPTIONS =
is_v_wind = TRUE;
>> > > > >> OBS_VAR5_OPTIONS =
is_v_wind = TRUE;
>> > > > >>
>> > > > >> Please give that a shot
and let me
>> know how
>> > it goes.
>> > > The
>> > > > goal is
>> > > > >> NOT having
>> > > > >> any U and V show up in the
CNT output
>> > file... and
>> > > instead
>> > > > have
>> > > > >> entries show
>> > > > >> up in the VCNT output.
>> > > > >>
>> > > > >> One last detail... if you
do want to
>> > aggregate results
>> > > > through
>> > > > >> time, I'd
>> > > > >> recommend writing the
SL1L2 and
>> VL1L2 line
>> > types to
>> > > enable
>> > > > that
>> > > > >> aggregation
>> > > > >> to be done later.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> John
>> > > > >>
>> > > > >> On Thu, Jun 3, 2021 at
1:49 PM Minna
>> Win
>> > via RT <
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >> >
>> > > > >> > <URL:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
>> > > > >> >
>> > > > >> >
>> > > > >> > Hi Scott,
>> > > > >> >
>> > > > >> > It looks like you've set
your MET
>> config
>> > file to
>> > > correctly
>> > > > >> identify which
>> > > > >> > linetypes you'd like to
see. I'm
>> not sure
>> > why you
>> > > are
>> > > > >> essentially getting
>> > > > >> > empty output files.
I'll need
>> to find
>> > someone
>> > > who can
>> > > > answer
>> > > > >> your python
>> > > > >> > embedding and grid-stat
questions.
>> > > > >> >
>> > > > >> > Regards,
>> > > > >> > Minna
>> > > > >> > ---------------
>> > > > >> > Minna Win
>> > > > >> > Pronouns: she/her
>> > > > >> > National Center for
Atmospheric
>> Research
>> > > > >> > Developmental Testbed
Center
>> > > > >> > Phone: 303-497-8423
>> > > > >> > Fax: 303-497-8401
>> > > > >> > ---------------
>> > > > >> >
>> > > > >> >
>> > > > >> >
>> > > > >> > On Thu, Jun 3, 2021 at
12:04 PM
>> > Rabenhorst, Scott D.
>> > > > >> (GSFC-610.1)[SCIENCE
>> > > > >> > SYSTEMS AND APPLICATIONS
INC] via
>> RT <
>> > > met_help at ucar.edu>
>> > > > wrote:
>> > > > >> >
>> > > > >> > >
>> > > > >> > > Thu Jun 03 12:04:14
2021: Request
>> > 100090 was
>> > > acted upon.
>> > > > >> > > Transaction: Ticket
created by
>> > > > scott.d.rabenhorst at nasa.gov
>> > > > >> > > Queue: met_help
>> > > > >> > > Subject: 2
questions
>> regarding how
>> > to
>> > > generate
>> > > > specific
>> > > > >> stats
>> > > > >> > > Owner: Nobody
>> > > > >> > > Requestors:
>> > scott.d.rabenhorst at nasa.gov
>> > > > >> > > Status: new
>> > > > >> > > Ticket <URL:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
>> > > > >> >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Hi met support,
>> > > > >> > >
>> > > > >> > > I have 2 questions
regarding how
>> to
>> > output
>> > > specific
>> > > > >> statistics for my
>> > > > >> > > experiment (see files
in my
>> attached
>> > tarball). I
>> > > have
>> > > > looked
>> > > > >> through the
>> > > > >> > > online documentation,
but I have
>> had a
>> > hard time
>> > > > finding a
>> > > > >> direct
>> > > > >> > > answer/solution –
perhaps I’ve
>> simply
>> > missed
>> > > something
>> > > > >> obvious.
>> > > > >> > >
>> > > > >> > > QUESTION 1: how to
output
>> wind/vector
>> > (VL1L2,
>> > > VCNT)
>> > > > >> statistics. I have
>> > > > >> > > added:
>> > > > >> > >
>> > > > >> > >
GRID_STAT_OUTPUT_FLAG_CNT =
>> BOTH
>> > > > >> > >
GRID_STAT_OUTPUT_FLAG_VCNT =
>> BOTH
>> > > > >> > >
GRID_STAT_OUTPUT_FLAG_VL1L2 =
>> BOTH
>> > > > >> > >
>> > > > >> > > … to my
>> > > >
>> ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
>> > > > >> file. I
>> > > > >> > have
>> > > > >> > > also added:
>> > > > >> > >
>> > > > >> > > is_u_wind : True
>> > > > >> > > is_grid_relative:
False
>> > > > >> > >
>> > > > >> > > … attributes to my
PYTHON_NUMPY
>> object
>> > through
>> > > python
>> > > > >> embedding. However,
>> > > > >> > > when I look at the
output
>> *_vl1l2.txt
>> > files, the
>> > > only
>> > > > >> information is the
>> > > > >> > > header lines. I have
included 2
>> sample
>> > outputs
>> > > from my
>> > > > >> embedding scripts
>> > > > >> > > when called alone
>> > > (./scripts/read_*2xr_v2.example.log)
>> > > > which
>> > > > >> show all the
>> > > > >> > > data is as expected.
It seems
>> like
>> > something else
>> > > needs
>> > > > to be
>> > > > >> activated
>> > > > >> > > somewhere in the conf
files, but
>> I
>> > can’t seem to
>> > > find
>> > > > the
>> > > > >> correct switch.
>> > > > >> > > If you have any ideas
for other
>> things
>> > to try,
>> > > please
>> > > > let me
>> > > > >> know.
>> > > > >> > >
>> > > > >> > > QUESTION 2: is there a
way to use
>> > stat_analysis to
>> > > > >> recompute/aggregate
>> > > > >> > > over my *_pairs.nc
files
>> containing
>> > this
>> > > information
>> > > > >> (example ncdump in
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
>> > > > >> > > and output CNT stats
for the
>> total of
>> > all the
>> > > matched
>> > > > pair
>> > > > >> data? I am
>> > > > >> > aware
>> > > > >> > > of the “-job summary
-line_type
>> CNT
>> > -column RMSE
>> > > -by
>> > > > >> FCST_VAR,FCST_LEV
>> > > > >> > > -dump_row” but this
looks like
>> it only
>> > provides
>> > > basic
>> > > > >> statistics of all
>> > > > >> > the
>> > > > >> > > RMSE values in each
*_cnt.txt
>> files,
>> > correct?
>> > > What I
>> > > > really
>> > > > >> want is to
>> > > > >> > > generate actual CNT
data from all
>> > matched pairs
>> > > over all
>> > > > >> times. I’ve
>> > > > >> > tried
>> > > > >> > > various combinations
of
>> aggregate and
>> > > aggregate_stat but
>> > > > >> can’t seem to
>> > > > >> > find
>> > > > >> > > a combination that
will work. Is
>> this
>> > possible?
>> > > If so,
>> > > > what
>> > > > >> should my
>> > > > >> > > stat_analysis job
command look
>> like?
>> > > > >> > >
>> > > > >> > > Thanks so much for
your time in
>> > advance. I have
>> > > asked a
>> > > > lot
>> > > > >> of questions
>> > > > >> > > lately, but I think
I’m almost
>> there.
>> > Any help
>> > > you can
>> > > > >> provide on filling
>> > > > >> > > in my knowledge gaps
would be
>> very much
>> > > appreciated.
>> > > > >> > >
>> > > > >> > > Thanks,
>> > > > >> > > Scott
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >
>> > > > > --
>> > > > > George McCabe - Software Engineer
III
>> > > > > National Center for Atmospheric
Research
>> > > > > Research Applications Laboratory
>> > > > > 303-497-2768
>> > > > > ---
>> > > > > My working day may not be your
working day.
>> Please
>> > do not feel
>> > > > obliged to
>> > > > > reply to this email outside of your
normal
>> working
>> > hours.
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > George McCabe - Software Engineer III
>> > > > National Center for Atmospheric
Research
>> > > > Research Applications Laboratory
>> > > > 303-497-2768
>> > > > ---
>> > > > My working day may not be your
working day.
>> Please do
>> > not feel
>> > > obliged
>> > > > to
>> > > > reply to this email outside of your
normal
>> working
>> > hours.
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> > > --
>> > > George McCabe - Software Engineer III
>> > > National Center for Atmospheric Research
>> > > Research Applications Laboratory
>> > > 303-497-2768
>> > > ---
>> > > My working day may not be your working day.
Please do
>> not
>> > feel obliged
>> > > to
>> > > reply to this email outside of your normal
working
>> hours.
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>> > --
>> > George McCabe - Software Engineer III
>> > National Center for Atmospheric Research
>> > Research Applications Laboratory
>> > 303-497-2768
>> > ---
>> > My working day may not be your working day.
Please do not
>> feel
>> > obliged to
>> > reply to this email outside of your normal
working hours.
>> >
>> >
>> >
>> >
>> >
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not
feel
>> obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>>
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: 2 questions regarding how to generate specific stats
From: George McCabe
Time: Tue Jun 22 08:32:17 2021
Hi Scott,
Here is the reference page from the MET User's Guide about Vectors and
Vector Statistics:
https://met.readthedocs.io/en/latest/Users_Guide/appendixG.html
Also, here is the Python embedding script that computes wind:
https://github.com/dtcenter/METplus/blob/main_v4.0/parm/use_cases/model_applications/medium_range/TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead_PyEmbed_IVT/gfs_ivt_fcst.py#L70
uv = np.sqrt(u**2+v**2) # Calculate wind
I thought I had included that link in the last email, so I apologize
for
excluding that detail. I hope that helps!
FYI, here is the GitHub issue that I created:
https://github.com/dtcenter/MET/issues/1839
- George
On Tue, Jun 22, 2021 at 8:14 AM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100090 >
>
> Hi George,
>
> Thanks so much for creating an issue for this and for continuing to
ponder
> a work around. In the interim, I am certainly open to combining u/v
winds
> in python somehow, but I'm unsure exactly how the MET package does
this in
> order to drive the VL1L2, VCNT output. Are u and v paired in an
combined
> array somehow that met uses, or is it sufficient to just generated
> additional wind speed and direction variables? Again, are there key
> attributes specifically designated to recognize wind speed and
direction
> variables when using python embedding (similar to "is_u_wind") or
does met
> key off a standard name?
>
> Thanks,
> Scott
>
>
> On 6/22/21, 8:50 AM, "George McCabe via RT" <met_help at ucar.edu>
wrote:
>
> Hi Scott,
>
> This morning I thought to try to use set_attr_level to override
the
> level
> values to see if that would allow the U/V components to be
combined
> properly, but that also did not seem to have any affect. While
> searching
> for another example of computing wind vectors from U/V I found
another
> use
> case that reads U and V fields in a Python embedding script and
> computes
> U/V to be used in calculating IVT. I'm not sure that this will
give
> you the
> same result that you are expecting, but you could potentially
combine
> U and
> V inside your Python embedding script instead of relying on the
MET
> tools
> to perform that calculation. I am still going to write up an
issue to
> investigate the potential deficiency in the logic, but this may
help
> you
> move forward with your analysis for the time being.
>
> Thanks,
> George
>
> On Mon, Jun 21, 2021 at 6:50 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Scott,
> >
> > Thanks for sending the input files. I did some digging and I
believe
> this
> > may require an enhancement to the logic for handling Python
> embedding. The
> > value for level is typically not needed for Python embedding
because
> all of
> > the information is provided to the name value. However, it
looks
> like the
> > logic to compute U/V vector partial sums uses the level value
to
> match up
> > the U and V fields. The developer(s) who are more familiar
with this
> logic
> > in the C++ code are currently on vacation, but I will create a
> GitHub issue
> > to investigate this and create a fix.
> >
> > Thanks,
> > George
> >
> > On Mon, Jun 21, 2021 at 10:17 AM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >
> >>
> >> <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
> >
> >>
> >> Hi George,
> >>
> >> Thanks for getting back in touch with me so soon. Sorry for
the
> delay.
> >> I'm back at work today.
> >>
> >> This morning I simplified my grid_stat config file so it is
only
> reading
> >> a few select fields on pressure levels. Hopefully, this will
avoid
> the
> >> complications of the surface data with time offsets and all
the
> other
> >> non-conventional comparisons with masking that you have
noticed.
> This
> >> latest run demonstrates the problem - still no vector stats.
I
> uploaded 3
> >> files to the ucar ftp site:
> >>
> >> 1. geosgcm_pres.20191223_0000z.nc4.gz (forecast data time
sliced to
> index
> >> 26 the observation comparison time)
> >> 2. era5_pl-allvar.20191223_00z.nc4.gz (observation data)
> >> 3. expfiles.5.tar.gz (logs/scripts from my latest run)
> >>
> >> Just as a reminder, my python read scripts search through a
> directory
> >> hierarchy to locate the proper forecast and observation files
for
> >> comparison. This will not work for you as is. You will have
to
> change and
> >> hard-code the file paths to make this work on your side.
> >>
> >> Please let me know if you have any issues or need any further
> >> information. I am eager and hopeful we can finally solve this
issue.
> >> Looking forward to hearing from you.
> >>
> >> Thanks,
> >> Scott
> >>
> >>
> >> On 6/18/21, 11:43 AM, "George McCabe via RT"
<met_help at ucar.edu>
> wrote:
> >>
> >> Hi Scott,
> >>
> >> Thanks for sending the new log file. We do have an FTP
site
> that you
> >> can
> >> use to send us data. Information on how is on this
webpage
> under the
> >> "How
> >> To Send Us Data" heading:
> >>
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fget-
help&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8sUOP2ha66gn6HCbXgHov9p%2Fw%2BlKakoVAHQvTlPoFCI%3D&reserved=0
> >> Those files are quite large, but I would really only need
the U
> and V
> >> fields from each data set to debug why that logic is not
working
> >> properly.
> >>
> >> I did also notice that there are some warnings in your
log file
> >> saying that
> >> there is a time mismatch between the forecast and
observation
> data
> >> for some
> >> of the other fields. It will still use the data, but the
> results may
> >> not
> >> reflect what you are expecting. The valid time that comes
from
> the
> >> forecast
> >> script doesn't match the valid time passed into the
script, so
> there
> >> may be
> >> something going on there. This isn't related to the U/V
issue
> but I
> >> thought
> >> I would point it out.
> >>
> >>
> >> {
> >>
> >>
>
name=\"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
> >>
> >>
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> >> TTPRCP 20191221_2100 *20191223_0000*\"; cat_thresh=[
> >> gt0.0,gt0.00071,gt0.00141,gt0.00212 ]; }
> >>
> >> DEBUG 3: Running user's python script
> >>
> >>
>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2).
> >> DEBUG 4:
> >> DEBUG 4: Data plane information:
> >> DEBUG 4: plane min: 0
> >> DEBUG 4: plane max: 0.01865353266
> >> DEBUG 4: valid time: *20191223_003000*
> >> DEBUG 4: lead time: 273000
> >> DEBUG 4: init time: 20191221_210000
> >> DEBUG 4: accum time: 000000
> >> DEBUG 4:
> >> DEBUG 3: Reading forecast data for TTPRCP/Surface.
> >> DEBUG 3: Running user's python script
> >>
> >>
>
(/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5sl2xr_v2).
> >> DEBUG 4:
> >> DEBUG 4: Data plane information:
> >> DEBUG 4: plane min: 0
> >> DEBUG 4: plane max: 0.02649974823
> >> DEBUG 4: valid time: *20191223_000000*
> >> DEBUG 4: lead time: 000000
> >> DEBUG 4: init time: 20191223_000000
> >> DEBUG 4: accum time: 010000
> >> DEBUG 4:
> >> DEBUG 3: Reading observation data for tp/Surface.
> >> WARNING:
> >> WARNING: process_scores() -> Forecast and observation
valid
> times do
> >> not
> >> match 20191223_003000 != 20191223_000000 for
TTPRCP/Surface
> versus
> >> tp/Surface.
> >>
> >> On Thu, Jun 17, 2021 at 8:51 PM Rabenhorst, Scott D.
> >> (GSFC-610.1)[SCIENCE
> >> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >>
> >> >
> >> > <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
> >> >
> >> >
> >> > Hi George,
> >> >
> >> > I apologize, my last email sent prematurely for some
weird
> reason.
> >> As I
> >> > was saying below, our forecast and observation files
are very
> >> large. I
> >> > could ncks a subset, but that really wouldn't be the
same
> file. I
> >> suppose
> >> > it would contain the same fields, but even a subset
would be
> too
> >> large for
> >> > an email attachment.
> >> >
> >> > Regarding the output below, you are correct. I did not
modify
> the
> >> 10m U/V
> >> > winds (U10M, V10M, etc). I only changed the 3D pressure
level
> U/V
> >> winds to
> >> > have the same name and attributes. I figured if those
started
> >> working I
> >> > would propagate the changes to the surface python
scripts
> too. So I
> >> would
> >> > recommend to only look at the output pertaining to
pressure
> level
> >> winds
> >> > from the read_era5pl2xr_v2.py and read_g5fcst2xr_v2.py
> scripts.
> >> Those have
> >> > the changes.
> >> >
> >> > Please let me know how to proceed with posting the
> >> forecast/observation
> >> > files if you think that will help. Do you have an ftp
site?
> >> >
> >> > Thanks,
> >> > Scott
> >> >
> >> >
> >> >
> >> >
> >> > On 6/17/21, 10:32 PM, "Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> >> SYSTEMS
> >> > AND APPLICATIONS INC]" <scott.d.rabenhorst at nasa.gov>
wrote:
> >> >
> >> > Hi George,
> >> >
> >> > I have attached the log file - that is easy.
However, I'm
> not
> >> sure how
> >> > to go about sending you the data. The experiment
(forecast)
> 3D file
> >> is 734G
> >> > and the surface (single) level files are 619M each.
> >> >
> >> > On 6/17/21, 5:01 PM, "George McCabe via RT" <
> met_help at ucar.edu>
> >> wrote:
> >> >
> >> > Hi Scott,
> >> >
> >> > The VL1L2 flag should accept BOTH. I'm not
familiar
> enough
> >> with
> >> > the U/V
> >> > calculations to know if it needs a threshold
but John
> didn't
> >> > mention that
> >> > it was a requirement in his first email. Could
you
> send me
> >> a set
> >> > of files
> >> > that are used as input for this use case? Also,
the
> latest
> >> log
> >> > file would
> >> > be helpful. I noticed that the name of the
fields did
> >> reflect the
> >> > script
> >> > name, so I'm curious to see if it was
overridden
> properly
> >> in your
> >> > last run.
> >> >
> >> > | | | | | | | Dictionary
> Entry[0] ...
> >> > | | | | | | | | Name =
*name*
> >> > | | | | | | | | Type =
> StringType
> >> > | | | | | | | | *String
Value
> =
> >> >
> >> >
> >>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5surf2xr_v2.py
> >> >
> >> >
> >>
>
/discover/nobackup/projects/gmao/osse2/Ops12KM_R/GMAO_PROPOSAL_BENCHMARK_SETUPS/B_Bench-
develop-feature-e20191221_21z-
RFCSTNLv10p19nhGF2020greens001_25KM_GFDL_RTElite_L181_0137_030x0180_40x01/scratch
> >> > U10M 20191221_2100 20191223_0000"*
> >> > | | | | | | | Dictionary
> Entry[1] ...
> >> > | | | | | | | | Name =
> is_u_wind
> >> > | | | | | | | | Type =
> BooleanType
> >> > | | | | | | | | Boolean
Value
> = true
> >> > | | | | | | | Dictionary
> Entry[2] ...
> >> > | | | | | | | | Name =
> >> is_grid_relative
> >> > | | | | | | | | Type =
> BooleanType
> >> > | | | | | | | | Boolean
Value
> =
> >> false
> >> >
> >> > Thanks,
> >> > George
> >> >
> >> > On Thu, Jun 17, 2021 at 12:29 PM Rabenhorst,
Scott D.
> >> > (GSFC-610.1)[SCIENCE
> >> > SYSTEMS AND APPLICATIONS INC] via RT <
> met_help at ucar.edu>
> >> wrote:
> >> >
> >> > >
> >> > > <URL:
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314375039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kEqWpnPWsAU7OHJWD6jeROEIIOVOjyAvfjyZxcT3Sn8%3D&reserved=0
> >> > >
> >> > >
> >> > > Hi George,
> >> > >
> >> > > I your suggestion and double checked for
typos, but
> >> > unfortunately, still
> >> > > nothing different - just headers in the
output
> files.
> >> Could
> >> > there be a bug
> >> > > in something indirectly related to this, such
as
> the BOTH
> >> option
> >> > does not
> >> > > work for GRID_STAT_OUTPUT_FLAG_VL1L2 or maybe
U/V
> winds
> >> need a
> >> > threshold.
> >> > > Just a WAG.
> >> > >
> >> > > ~Scott
> >> > >
> >> > >
> >> > >
> >> > > On 6/17/21, 1:28 PM, "George McCabe via RT" <
> >> met_help at ucar.edu>
> >> > wrote:
> >> > >
> >> > > Hi Scott,
> >> > >
> >> > > You could try adding
> >> > >
> >> > > set_attr_name = "U";
> >> > >
> >> > > to the [FCST/OBS]_VAR4_OPTIONS and "V"
for the
> VAR5
> >> options
> >> > to see if
> >> > > that
> >> > > helps? It may be using the script as the
name,
> so
> >> setting
> >> > this would
> >> > > override the input name that is used.
> >> > >
> >> > > FCST_VAR4_OPTIONS = is_u_wind = TRUE;
> set_attr_name
> >> =
> >> > "U";
> >> > > OBS_VAR4_OPTIONS = is_u_wind = TRUE;
> set_attr_name
> >> =
> >> > "U";
> >> > > FCST_VAR5_OPTIONS = is_v_wind = TRUE;
> set_attr_name
> >> =
> >> > "V";
> >> > > OBS_VAR5_OPTIONS = is_v_wind = TRUE;
> set_attr_name
> >> =
> >> > "V";
> >> > >
> >> > > Let me know if that doesn't work and I
can
> continue
> >> digging.
> >> > >
> >> > > Thanks,
> >> > > George
> >> > >
> >> > > On Thu, Jun 17, 2021 at 10:10 AM
Rabenhorst,
> Scott D.
> >> > > (GSFC-610.1)[SCIENCE
> >> > > SYSTEMS AND APPLICATIONS INC] via RT <
> >> met_help at ucar.edu>
> >> > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
> >> > > >
> >> > > >
> >> > > > Hi George,
> >> > > >
> >> > > > Thanks so much for taking a look at the
code
> once
> >> again.
> >> > You had
> >> > > another
> >> > > > great idea, and something rather easy
to
> change via
> >> python
> >> > embedding
> >> > > > compared to using ncatted/ncrename on
so many
> >> files. I
> >> > modified my
> >> > > scripts
> >> > > > to ensure all my attrs={...} values
matched
> for the
> >> U and
> >> > V fields,
> >> > > but
> >> > > > still not luck! I can't believe it.
Since
> metplus
> >> doesn't
> >> > appear to
> >> > > display
> >> > > > the output from my python embedding
scripts,
> I ran
> >> both
> >> > scripts
> >> > > stand-alone
> >> > > > by copying/pasting metplus' name field
onto
> the
> >> command
> >> > line. The
> >> > > output is
> >> > > > in the scripts/read_*2xr_v2.log files.
All
> >> attributes
> >> > match exactly
> >> > > (I even
> >> > > > hardcoded the init time just to be
sure). I
> ran
> >> with and
> >> > without the
> >> > > > updating the is_u/v_wind and
is_grid_relative
> >> attributes
> >> > in the
> >> > > python
> >> > > > attrs object (which shouldn't be
necessary),
> but
> >> that made
> >> > no
> >> > > difference
> >> > > > either. The vl1l2 and vcnt files are
still
> just
> >> header
> >> > information
> >> > > only
> >> > > > (see tarball). I increased the
verbosity of
> >> grid_stat to
> >> > 10, but my
> >> > > eye
> >> > > > didn't catch anything offhand. Maybe
yo!
> >> > > > u have a better idea what to look for.
This
> whole
> >> thing
> >> > is super
> >> > > weird.
> >> > > > Do you have any other ideas I can try?
> >> > > >
> >> > > > Thanks,
> >> > > > Scott
> >> > > >
> >> > > >
> >> > > >
> >> > > > On 6/16/21, 4:13 PM, "George McCabe via
RT" <
> >> > met_help at ucar.edu>
> >> > > wrote:
> >> > > >
> >> > > > Hi Scott,
> >> > > >
> >> > > > I think I see what is happening
with the
> U/V
> >> fields. I
> >> > checked
> >> > > the code
> >> > > > where it is checking the U/V fields
and it
> >> expects the
> >> > name and
> >> > > level
> >> > > > of
> >> > > > the fcst and obs to match to make
sure it
> it
> >> > processing the
> >> > > correct U/V
> >> > > > data. I noticed in the log output
from
> your
> >> python
> >> > embedding
> >> > > call that
> >> > > > era5pl2xr has the name "u" while
> g5fcst2xr has
> >> "U". I
> >> > would
> >> > > suggest
> >> > > > modifying the python embedding
script to
> ensure
> >> that
> >> > both of the
> >> > > > varname
> >> > > > values are the same case.
> >> > > >
> >> > > > If that doesn't work, I would
suggest
> bumping
> >> up the
> >> > log
> >> > > verbosity for
> >> > > > GridStat (LOG_GRID_STAT_VERBOSITY)
to see
> if
> >> there are
> >> > any clues
> >> > > in the
> >> > > > higher number log output. If you
are still
> >> having
> >> > issues, you
> >> > > could
> >> > > > send me
> >> > > > the input files for each and I can
play
> around
> >> with
> >> > the settings
> >> > > to see
> >> > > > what will work.
> >> > > >
> >> > > > Thanks,
> >> > > > George
> >> > > >
> >> > > > On Wed, Jun 16, 2021 at 1:07 PM
George
> McCabe <
> >> > mccabe at ucar.edu>
> >> > > wrote:
> >> > > >
> >> > > > > Hi Scott,
> >> > > > >
> >> > > > > Regarding SeriesAnalysis -- there
is an
> >> argument to
> >> > the tool
> >> > > called
> >> > > > > "-both" that can be used to set
both
> the fcst
> >> and
> >> > obs lists to
> >> > > use
> >> > > > the same
> >> > > > > files. This use case runs
> SeriesAnalysis on
> >> GridStat
> >> > output:
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetplus.readthedocs.io%2Fen%2Fv4.0.0%2Fgenerated%2Fmodel_applications%2Fs2s%2FGridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.html%3Fhighlight%3DSeriesAnalysisUseCase%2520%2520GridStatToolUseCase&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mS8sL5XTA6idlmqx%2FHdJOFSq0RXaRLRFW71c7hJYHcs%3D&reserved=0
> >> > > > >
> >> > > > > This example uses
> >> > BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE to set
> >> > > the
> >> > > > input
> >> > > > > file path and specifies the fcst
and obs
> >> input fields
> >> > > separately:
> >> > > > >
> >> > > > > FCST_SERIES_ANALYSIS_VAR1_NAME =
> >> FCST_precip_FULL
> >> > > > > FCST_SERIES_ANALYSIS_VAR1_LEVELS
=
> "(*,*)"
> >> > > > >
> >> > > > > OBS_SERIES_ANALYSIS_VAR1_NAME =
> >> OBS_precip_FULL
> >> > > > > OBS_SERIES_ANALYSIS_VAR1_LEVELS =
> "(*,*)"
> >> > > > >
> >> > > > >
BOTH_SERIES_ANALYSIS_INPUT_TEMPLATE =
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> grid_stat_{MODEL}-
hindcast_precip_vs_{OBTYPE}_IC{init?fmt=%Y%b}_V{valid?fmt=%Y%2m}01_*
> >> > > > > pairs.nc
> >> > > > >
> >> > > > > Regarding not being able to
produce wind
> >> output,
> >> > I'll take a
> >> > > look at
> >> > > > your
> >> > > > > files and see if I can narrow
down the
> issue.
> >> I will
> >> > report
> >> > > back
> >> > > > with more
> >> > > > > information.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > George
> >> > > > >
> >> > > > > On Mon, Jun 7, 2021 at 9:09 AM
> Rabenhorst,
> >> Scott D.
> >> > > > (GSFC-610.1)[SCIENCE
> >> > > > > SYSTEMS AND APPLICATIONS INC] via
RT <
> >> > met_help at ucar.edu>
> >> > > wrote:
> >> > > > >
> >> > > > >>
> >> > > > >> <URL:
> >> > > >
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
> >> > > > >
> >> > > > >>
> >> > > > >> Hi George,
> >> > > > >>
> >> > > > >> Just as a follow up thought to
problem
> below
> >> > regarding winds
> >> > > stats,
> >> > > > when
> >> > > > >> I look through the CNT and SL1L2
> statistics
> >> output,
> >> > I see U
> >> > > and V
> >> > > > listed
> >> > > > >> with all respective stats. So
this
> issue is
> >> not a
> >> > problem
> >> > > reading U
> >> > > > and V,
> >> > > > >> it is a problem recognizing they
are
> vector
> >> > components. Are
> >> > > your
> >> > > > sure there
> >> > > > >> is not another flag or option to
set
> either
> >> in the
> >> > > PTHON_NUMPY/attrs
> >> > > > >> objects or another option in the
> metplus
> >> conf file?
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> Scott
> >> > > > >>
> >> > > > >>
> >> > > > >> On 6/4/21, 12:28 PM,
"Rabenhorst,
> Scott D.
> >> > > (GSFC-610.1)[SCIENCE
> >> > > > SYSTEMS
> >> > > > >> AND APPLICATIONS INC]" <
> >> scott.d.rabenhorst at nasa.gov>
> >> > wrote:
> >> > > > >>
> >> > > > >> Hi George,
> >> > > > >>
> >> > > > >> Thanks for answering my
questions
> below.
> >> So
> >> > sorry for
> >> > > taking so
> >> > > > much
> >> > > > >> of your time lately. From your
> description,
> >> it
> >> > sounds like
> >> > > the best
> >> > > > option
> >> > > > >> for my needs is to use Stat-
Analysis to
> >> aggregate
> >> > SL1L2/VL1L2
> >> > > data.
> >> > > > My goal
> >> > > > >> is to generate single-value
statistics
> for
> >> each
> >> > variable/level
> >> > > > through all
> >> > > > >> valid times and spatial areas.
These
> will be
> >> > plugged into a
> >> > > Green's
> >> > > > >> Function algorithm for our
model.
> >> > > > >>
> >> > > > >> However, the Series-Analysis
is
> also an
> >> > intriguing
> >> > > approach at
> >> > > > least
> >> > > > >> for looking at spatial errors. I
guess
> >> Grid-Stat
> >> > could
> >> > > operate on
> >> > > > that data
> >> > > > >> at the end to generate total
stats? I
> have
> >> not
> >> > tried using
> >> > > > Series-Analysis
> >> > > > >> yet, but what has confused me is
the
> >> documentation
> >> > requires
> >> > > > separate "-fcst
> >> > > > >> file_1...file_n" and "-obs
> file_1...file_n".
> >> In
> >> > most examples
> >> > > I
> >> > > > have seen,
> >> > > > >> these are independent files.
However,
> you
> >> make it
> >> > sound as
> >> > > though
> >> > > > it can
> >> > > > >> run on my *_pairs.nc files
generated
> by
> >> Grid-Stat
> >> > I presume
> >> > > as
> >> > > > long as
> >> > > > >> RAW data is in there? In this
case, I
> assume
> >> the
> >> > list of files
> >> > > > would be
> >> > > > >> identical to both forecast and
> observations?
> >> Are
> >> > there any
> >> > > examples
> >> > > > of
> >> > > > >> using matched-pairs for the
> Series-Analysis
> >> online?
> >> > > > >>
> >> > > > >> Unfortunately, the larger
problem
> right
> >> now is
> >> > that I
> >> > > still
> >> > > > can't get
> >> > > > >> Grid-Stat to process any my
winds and
> output
> >> VCNT
> >> > or VL1L2
> >> > > data. I
> >> > > > tried
> >> > > > >> your suggestion below and added
the
> options
> >> to my
> >> > > > >>
> >> RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> >> > file
> >> > > (please see
> >> > > > >> updated attached tarball). I
tried
> running
> >> with and
> >> > without
> >> > > each of
> >> > > > these
> >> > > > >> options "is_u(v)_wind=TRUE" and
> >> > "is_grid_relative=FALSE" and
> >> > > those
> >> > > > ASCII
> >> > > > >> files never contains any
VL1L2/VCNT
> data,
> >> nor do
> >> > the *.stat
> >> > > files
> >> > > > contain
> >> > > > >> any of those line_types. I also
toggled
> >> outputting
> >> > those
> >> > > attribute
> >> > > > in my
> >> > > > >> PYTHON_NUMPY attrs object, but
that
> didn't
> >> work
> >> > either. I
> >> > > couldn't
> >> > > > find any
> >> > > > >> combinations that would work. In
an
> effort to
> >> > narrow the
> >> > > problem, I
> >> > > > >> copy/pasted the first references
to
> the fcst
> >> and
> >> > obs fields
> >> > > in my
> >> > > > logfile
> >> > > > >> and ran my python read scripts
> stand-alone
> >> and they
> >> > produced
> >> > > the
> >> > > > expected
> >> > > > >> output with correct values:
> >> > > > >>
> scripts/read_era5pl2xr_v2.example.log
> >> > > > >>
> scripts/read_g5fcst2xr_v2.example.log
> >> > > > >>
> >> > > > >> Furthermore, I decided to
test
> >> PlotDataPlane
> >> > since I
> >> > > recalled
> >> > > > you
> >> > > > >> saying that was used behind the
> scenes. It
> >> ran just
> >> > fine and
> >> > > > produced these
> >> > > > >> beautiful U 500hPa images
without
> error:
> >> > > > >>
> >> PlotDataPlane_py-embed_test/plot_data_plane/*.ps
> >> > > > >>
> >> > > > >> I have combed the log files
for any
> >> errors or
> >> > warning, but
> >> > > > cannot
> >> > > > >> find any at all, or anything
else that
> would
> >> > indicate a
> >> > > processing
> >> > > > error.
> >> > > > >> It is almost like there is some
flag
> that
> >> has to be
> >> > set in
> >> > > order to
> >> > > > trigger
> >> > > > >> wind processing. Do you have any
other
> ideas
> >> I can
> >> > try?
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> Scott
> >> > > > >>
> >> > > > >>
> >> > > > >> On 6/3/21, 6:22 PM, "John
Halley
> Gotway
> >> via RT"
> >> > <
> >> > > > met_help at ucar.edu>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> Hi Scott,
> >> > > > >>
> >> > > > >> Minna asked me to take a
look
> at your
> >> > questions.
> >> > > > >>
> >> > > > >> For the second question,
the
> answer
> >> is no.
> >> > > Stat-Analysis
> >> > > > reads
> >> > > > >> the .stat
> >> > > > >> output files created by
the MET
> >> tools. It
> >> > does not
> >> > > read the
> >> > > > >> gridded NetCDF
> >> > > > >> matched pairs files
generated
> by the
> >> > Grid-Stat tool.
> >> > > > >> But I do have two
> recommendations to
> >> make.
> >> > > > >>
> >> > > > >> 1. Grid-Stat computes
> statistics and
> >> > partial sums that
> >> > > > represent
> >> > > > >> a spatial
> >> > > > >> summary. You can
configure
> Grid-Stat
> >> to
> >> > compute its
> >> > > output
> >> > > > over
> >> > > > >> whatever
> >> > > > >> spatial masking regions
you'd
> like.
> >> > Stat-Analysis can
> >> > > > >> read/aggregate those
> >> > > > >> spatial summaries
through
> time. The
> >> SL1L2
> >> > line type
> >> > > contains
> >> > > > >> scalar L1 and
> >> > > > >> L2 norms. Those are
written
> because
> >> they
> >> > allow you to
> >> > > easily
> >> > > > >> aggregate
> >> > > > >> stats over multiple
runs. Many
> (but
> >> not
> >> > all) of the
> >> > > > continuous
> >> > > > >> statistics
> >> > > > >> (like ME, RMSE, MAE,
ACC, and
> so on)
> >> can be
> >> > derived
> >> > > from the
> >> > > > >> SL1L2 line
> >> > > > >> type. So I'd recommend
trying
> this
> >> type of
> >> > > stat_analysis
> >> > > > job:
> >> > > > >>
> >> > > > >> -job aggregate_stat
-line_type
> SL1L2
> >> > -out_line_type
> >> > > CNT -by
> >> > > > >>
FCST_VAR,FCST_LEV,VX_MASK
> -out_stat
> >> > sl1l2_to_cnt.stat
> >> > > > >>
> >> > > > >> For each unique
combination of
> >> variable,
> >> > level, and
> >> > > masking
> >> > > > >> region, it'll
> >> > > > >> aggregate all the SL1L2
lines
> it
> >> finds in
> >> > the input
> >> > > and
> >> > > > report the
> >> > > > >> corresponding continuous
> statistics.
> >> > > > >>
> >> > > > >> 2. Another option is
running
> the
> >> NetCDF
> >> > matched pairs
> >> > > > produced by
> >> > > > >> Grid-Stat
> >> > > > >> through the Series-
Analysis
> tool.
> >> That'll
> >> > compute
> >> > > stats
> >> > > > through
> >> > > > >> time
> >> > > > >> separately for each
individual
> grid
> >> point.
> >> > The result
> >> > > with
> >> > > > be a
> >> > > > >> map of
> >> > > > >> statistics rather than a
> spatial
> >> summary.
> >> > > > >>
> >> > > > >> For the first question,
I see
> that
> >> you're
> >> > using python
> >> > > > embedding
> >> > > > >> and are
> >> > > > >> trying to compute vector
> partial sums
> >> > output. In
> >> > > general,
> >> > > > when
> >> > > > >> you request
> >> > > > >> an output line type,
Grid-Stat
> will
> >> create
> >> > the
> >> > > corresponding
> >> > > > >> output file.
> >> > > > >> But if it's empty, that
means
> that it
> >> > didn't have any
> >> > > input
> >> > > > data
> >> > > > >> suitable
> >> > > > >> for creating that output
type.
> So
> >> you've
> >> > requested
> >> > > vector
> >> > > > stats,
> >> > > > >> but
> >> > > > >> Grid-Stat isn't
computing any.
> >> > > > >>
> >> > > > >> Thanks for sending the
sample
> output.
> >> > Looking in:
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/logs/metplus.log.20210603112534
> >> > > > >>
> >> > > > >> I do see several warning
that
> you
> >> should
> >> > take a look
> >> > > at:
> >> > > > >> WARNING:
python_dataplane() ->
> an
> >> error
> >> > occurred
> >> > > importing
> >> > > > module
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
"/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr_v2"
> >> > > > >>
> >> > > > >> But I do also see that
> Grid-Stat is
> >> > processing the U
> >> > > and V
> >> > > > data
> >> > > > >> and
> >> > > > >> reporting output as
scalar
> variables
> >> in the
> >> > CNT.txt
> >> > > file:
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_010000V_cnt.txt
> >> > > > >>
> >> > > > >> Now I haven't tested
this on
> your
> >> data to
> >> > confirm
> >> > > that it'll
> >> > > > >> actually work,
> >> > > > >> but we want to set these
flags
> in
> >> the MET
> >> > config file:
> >> > > > >> is_u_wind = true
> >> > > > >> is_v_wind = true
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fconfig_options.html%3Fhighlight%3Dis_u_wind%23settings-
common-to-multiple-
tools&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TSbsmVExDOLnIasq22jGMa1QQtf%2B4zyE4VMy5PYlMik%3D&reserved=0
> >> > > > >>
> >> > > > >> Checking in
> >> > > > >>
> >> >
RFCSTNLv10p19nhGF2020_20191221/greens001/metplus_final.conf I
> >> > > > >> see that var 4 is for U
and
> var 5 is
> >> for V.
> >> > > > >> Add those via the
METplus
> config
> >> file by
> >> > adding:
> >> > > > >> FCST_VAR4_OPTIONS =
is_u_wind
> = TRUE;
> >> > > > >> OBS_VAR4_OPTIONS =
is_u_wind =
> TRUE;
> >> > > > >> FCST_VAR5_OPTIONS =
is_v_wind
> = TRUE;
> >> > > > >> OBS_VAR5_OPTIONS =
is_v_wind =
> TRUE;
> >> > > > >>
> >> > > > >> Please give that a shot
and
> let me
> >> know how
> >> > it goes.
> >> > > The
> >> > > > goal is
> >> > > > >> NOT having
> >> > > > >> any U and V show up in
the CNT
> output
> >> > file... and
> >> > > instead
> >> > > > have
> >> > > > >> entries show
> >> > > > >> up in the VCNT output.
> >> > > > >>
> >> > > > >> One last detail... if
you do
> want to
> >> > aggregate results
> >> > > > through
> >> > > > >> time, I'd
> >> > > > >> recommend writing the
SL1L2 and
> >> VL1L2 line
> >> > types to
> >> > > enable
> >> > > > that
> >> > > > >> aggregation
> >> > > > >> to be done later.
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> John
> >> > > > >>
> >> > > > >> On Thu, Jun 3, 2021 at
1:49 PM
> Minna
> >> Win
> >> > via RT <
> >> > > > >> met_help at ucar.edu> wrote:
> >> > > > >>
> >> > > > >> >
> >> > > > >> > <URL:
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > Hi Scott,
> >> > > > >> >
> >> > > > >> > It looks like you've
set
> your MET
> >> config
> >> > file to
> >> > > correctly
> >> > > > >> identify which
> >> > > > >> > linetypes you'd like
to see.
> I'm
> >> not sure
> >> > why you
> >> > > are
> >> > > > >> essentially getting
> >> > > > >> > empty output files.
I'll
> need
> >> to find
> >> > someone
> >> > > who can
> >> > > > answer
> >> > > > >> your python
> >> > > > >> > embedding and grid-
stat
> questions.
> >> > > > >> >
> >> > > > >> > Regards,
> >> > > > >> > Minna
> >> > > > >> > ---------------
> >> > > > >> > Minna Win
> >> > > > >> > Pronouns: she/her
> >> > > > >> > National Center for
> Atmospheric
> >> Research
> >> > > > >> > Developmental Testbed
Center
> >> > > > >> > Phone: 303-497-8423
> >> > > > >> > Fax: 303-497-8401
> >> > > > >> > ---------------
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > On Thu, Jun 3, 2021 at
12:04
> PM
> >> > Rabenhorst, Scott D.
> >> > > > >> (GSFC-610.1)[SCIENCE
> >> > > > >> > SYSTEMS AND
APPLICATIONS
> INC] via
> >> RT <
> >> > > met_help at ucar.edu>
> >> > > > wrote:
> >> > > > >> >
> >> > > > >> > >
> >> > > > >> > > Thu Jun 03 12:04:14
2021:
> Request
> >> > 100090 was
> >> > > acted upon.
> >> > > > >> > > Transaction: Ticket
> created by
> >> > > > scott.d.rabenhorst at nasa.gov
> >> > > > >> > > Queue:
met_help
> >> > > > >> > > Subject: 2
questions
> >> regarding how
> >> > to
> >> > > generate
> >> > > > specific
> >> > > > >> stats
> >> > > > >> > > Owner: Nobody
> >> > > > >> > > Requestors:
> >> > scott.d.rabenhorst at nasa.gov
> >> > > > >> > > Status: new
> >> > > > >> > > Ticket <URL:
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100090&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C998f30582c57460f1e1408d9357c508e%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637599630314384988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=21OnX6peAeW8D9VUhG4AkxCi%2BM3EN1YJ6%2FMvqR8T%2BO4%3D&reserved=0
> >> > > > >> >
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > Hi met support,
> >> > > > >> > >
> >> > > > >> > > I have 2 questions
> regarding how
> >> to
> >> > output
> >> > > specific
> >> > > > >> statistics for my
> >> > > > >> > > experiment (see
files in my
> >> attached
> >> > tarball). I
> >> > > have
> >> > > > looked
> >> > > > >> through the
> >> > > > >> > > online
documentation, but
> I have
> >> had a
> >> > hard time
> >> > > > finding a
> >> > > > >> direct
> >> > > > >> > > answer/solution –
perhaps
> I’ve
> >> simply
> >> > missed
> >> > > something
> >> > > > >> obvious.
> >> > > > >> > >
> >> > > > >> > > QUESTION 1: how to
output
> >> wind/vector
> >> > (VL1L2,
> >> > > VCNT)
> >> > > > >> statistics. I have
> >> > > > >> > > added:
> >> > > > >> > >
> >> > > > >> > >
GRID_STAT_OUTPUT_FLAG_CNT
> =
> >> BOTH
> >> > > > >> > >
> GRID_STAT_OUTPUT_FLAG_VCNT =
> >> BOTH
> >> > > > >> > >
> GRID_STAT_OUTPUT_FLAG_VL1L2 =
> >> BOTH
> >> > > > >> > >
> >> > > > >> > > … to my
> >> > > >
> >> ./RFCSTNLv10p19nhGF2020_20191221/GridStat_cfg-01.conf
> >> > > > >> file. I
> >> > > > >> > have
> >> > > > >> > > also added:
> >> > > > >> > >
> >> > > > >> > > is_u_wind : True
> >> > > > >> > > is_grid_relative:
False
> >> > > > >> > >
> >> > > > >> > > … attributes to my
> PYTHON_NUMPY
> >> object
> >> > through
> >> > > python
> >> > > > >> embedding. However,
> >> > > > >> > > when I look at the
output
> >> *_vl1l2.txt
> >> > files, the
> >> > > only
> >> > > > >> information is the
> >> > > > >> > > header lines. I have
> included 2
> >> sample
> >> > outputs
> >> > > from my
> >> > > > >> embedding scripts
> >> > > > >> > > when called alone
> >> > > (./scripts/read_*2xr_v2.example.log)
> >> > > > which
> >> > > > >> show all the
> >> > > > >> > > data is as expected.
It
> seems
> >> like
> >> > something else
> >> > > needs
> >> > > > to be
> >> > > > >> activated
> >> > > > >> > > somewhere in the
conf
> files, but
> >> I
> >> > can’t seem to
> >> > > find
> >> > > > the
> >> > > > >> correct switch.
> >> > > > >> > > If you have any
ideas for
> other
> >> things
> >> > to try,
> >> > > please
> >> > > > let me
> >> > > > >> know.
> >> > > > >> > >
> >> > > > >> > > QUESTION 2: is there
a way
> to use
> >> > stat_analysis to
> >> > > > >> recompute/aggregate
> >> > > > >> > > over my *_pairs.nc
files
> >> containing
> >> > this
> >> > > information
> >> > > > >> (example ncdump in
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
RFCSTNLv10p19nhGF2020_20191221/greens001/grid_stat/grid_stat_greens001_vs_ERA5-
PL_000000L_20191223_000000V_pairs.ncdump)
> >> > > > >> > > and output CNT stats
for
> the
> >> total of
> >> > all the
> >> > > matched
> >> > > > pair
> >> > > > >> data? I am
> >> > > > >> > aware
> >> > > > >> > > of the “-job summary
> -line_type
> >> CNT
> >> > -column RMSE
> >> > > -by
> >> > > > >> FCST_VAR,FCST_LEV
> >> > > > >> > > -dump_row” but this
looks
> like
> >> it only
> >> > provides
> >> > > basic
> >> > > > >> statistics of all
> >> > > > >> > the
> >> > > > >> > > RMSE values in each
> *_cnt.txt
> >> files,
> >> > correct?
> >> > > What I
> >> > > > really
> >> > > > >> want is to
> >> > > > >> > > generate actual CNT
data
> from all
> >> > matched pairs
> >> > > over all
> >> > > > >> times. I’ve
> >> > > > >> > tried
> >> > > > >> > > various combinations
of
> >> aggregate and
> >> > > aggregate_stat but
> >> > > > >> can’t seem to
> >> > > > >> > find
> >> > > > >> > > a combination that
will
> work. Is
> >> this
> >> > possible?
> >> > > If so,
> >> > > > what
> >> > > > >> should my
> >> > > > >> > > stat_analysis job
command
> look
> >> like?
> >> > > > >> > >
> >> > > > >> > > Thanks so much for
your
> time in
> >> > advance. I have
> >> > > asked a
> >> > > > lot
> >> > > > >> of questions
> >> > > > >> > > lately, but I think
I’m
> almost
> >> there.
> >> > Any help
> >> > > you can
> >> > > > >> provide on filling
> >> > > > >> > > in my knowledge gaps
would
> be
> >> very much
> >> > > appreciated.
> >> > > > >> > >
> >> > > > >> > > Thanks,
> >> > > > >> > > Scott
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >> >
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >
> >> > > > > --
> >> > > > > George McCabe - Software Engineer
III
> >> > > > > National Center for Atmospheric
Research
> >> > > > > Research Applications Laboratory
> >> > > > > 303-497-2768
> >> > > > > ---
> >> > > > > My working day may not be your
working
> day.
> >> Please
> >> > do not feel
> >> > > > obliged to
> >> > > > > reply to this email outside of
your
> normal
> >> working
> >> > hours.
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > George McCabe - Software Engineer
III
> >> > > > National Center for Atmospheric
Research
> >> > > > Research Applications Laboratory
> >> > > > 303-497-2768
> >> > > > ---
> >> > > > My working day may not be your
working
> day.
> >> Please do
> >> > not feel
> >> > > obliged
> >> > > > to
> >> > > > reply to this email outside of your
normal
> >> working
> >> > hours.
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> > > --
> >> > > George McCabe - Software Engineer III
> >> > > National Center for Atmospheric Research
> >> > > Research Applications Laboratory
> >> > > 303-497-2768
> >> > > ---
> >> > > My working day may not be your working
day.
> Please do
> >> not
> >> > feel obliged
> >> > > to
> >> > > reply to this email outside of your
normal
> working
> >> hours.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > George McCabe - Software Engineer III
> >> > National Center for Atmospheric Research
> >> > Research Applications Laboratory
> >> > 303-497-2768
> >> > ---
> >> > My working day may not be your working day.
Please do
> not
> >> feel
> >> > obliged to
> >> > reply to this email outside of your normal
working
> hours.
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not
feel
> >> obliged to
> >> reply to this email outside of your normal working hours.
> >>
> >>
> >>
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to
> reply to this email outside of your normal working hours.
>
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list