[Met_help] [rt.rap.ucar.edu #96463] History for calculate multiple variables in config file
John Halley Gotway via RT
met_help at ucar.edu
Thu Oct 1 14:34:33 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am wondering how to get new variables by using multiple known variables
in the config file? e.g:
I have "UGRD' and "VGRD" defined in the ensemble_stat as below, but I want
to get WIND instead of the U/V component by using sqrt(UGRD^2+VGRD^2), how
to do that in the config file?
name = "UGRD";
level = [ "P850", "P250" ];
cat_thresh = [ <=5, >5&&<=10, >10&&<=15, >15&&<=20, >20&&<=25,
>25&&<=30, >30&&<=35, >35&&<=40, >40&&<=45, >45&&<=50, > 50 ];
},
{
name = "VGRD";
level = [ "P850", "P250" ];
cat_thresh = [ <=5, >5&&<=10, >10&&<=15, >15&&<=20, >20&&<=25,
>25&&<=30, >30&&<=35, >35&&<=40, >40&&<=45, >45&&<=50, > 50 ];
The MET document only talks about how to do the conversion for a single
variable, but not multiple ones.
Thank you.
Binyu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: calculate multiple variables in config file
From: John Halley Gotway
Time: Thu Aug 27 13:28:45 2020
Binyu,
In general, MET does not provide a way of reading multiple input
variables
and deriving a new field in the configuration file. However, we did
add
support for python-embedding with this use in mind. Generally
speaking, you
can write a python script to read data in whatever way you'd like...
compute whatever sort of values you'd like... and then pass the result
to
the MET tools via memory. So that's the recommended way of deriving
new
datasets for MET to process.
However, python-embedding is not required for WIND speed. You've
stumbled
across a very particular derivation type that is already supported
directly
in MET. When using input GRIB1 or GRIB2 files, if you request the
"WIND"
variable, MET will first try to read that from the input data file. If
found, it'll use it. If not found, it'll read the corresponding UGRD
and
VGRD records and use them to derive WIND speed on the fly.
I'd recommend running the plot_data_plane tool to try that out:
plot_data_plane in.grb wind_speed.ps 'name="WIND"; level="P850";' -v 4
And then look closely at the log messages that are printed to screen.
It
should tell you that it's deriving WIND speed for you on the fly.
Thanks,
John
On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: calculate multiple variables in config file
> Owner: johnhg
> Requestors: binyu.wang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: calculate multiple variables in config file
From: binyu.wang at noaa.gov
Time: Thu Aug 27 13:51:41 2020
Cool, that works. It seems it works for WDIR as well!
Thank you very much.
On Thu, Aug 27, 2020 at 3:28 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> In general, MET does not provide a way of reading multiple input
variables
> and deriving a new field in the configuration file. However, we did
add
> support for python-embedding with this use in mind. Generally
speaking, you
> can write a python script to read data in whatever way you'd like...
> compute whatever sort of values you'd like... and then pass the
result to
> the MET tools via memory. So that's the recommended way of deriving
new
> datasets for MET to process.
>
> However, python-embedding is not required for WIND speed. You've
stumbled
> across a very particular derivation type that is already supported
directly
> in MET. When using input GRIB1 or GRIB2 files, if you request the
"WIND"
> variable, MET will first try to read that from the input data file.
If
> found, it'll use it. If not found, it'll read the corresponding UGRD
and
> VGRD records and use them to derive WIND speed on the fly.
>
> I'd recommend running the plot_data_plane tool to try that out:
>
> plot_data_plane in.grb wind_speed.ps 'name="WIND"; level="P850";' -v
4
>
> And then look closely at the log messages that are printed to
screen. It
> should tell you that it's deriving WIND speed for you on the fly.
>
> Thanks,
> John
>
> On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: calculate multiple variables in config file
> > Owner: johnhg
> > Requestors: binyu.wang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
> >
> >
> > This transaction appears to have no content
> >
>
>
------------------------------------------------
Subject: calculate multiple variables in config file
From: binyu.wang at noaa.gov
Time: Fri Aug 28 11:02:03 2020
Hello John,
I want to double check about using MET for the wind verification:
You told me I could use UGRD/VGRD to get wind direction information by
setting " vl1l2 = BOTH; and vcnt = BOTH;" before, and I did get it
done
successfully but with "v9.1_beta2" unfortunately. Yesterday you said
MET
can get WIND by UGRD/VGRD, so I am thinking maybe I can get wind
direction
information directly using the config below (by doing that, I don't
need
worry about the rule that UGRD must be immediately
followed by a VGRD observation at the same level.). My Obs. has WIND
variable and my model has UGRD/VGRD.
fcst = {
message_type = [ "ADPUPA" ];
field = [
{
name = "WIND";
level = [ "P250" ];
cat_thresh = [ <=1.0, >1&&<=3.0,>3&&<=10.0 ];
},
I didn't get any error message in the log file, but the output is
empty.
Does that mean I can not get wind direction stat. information by
reading in
"WIND" and MUST use "UGRD/VGRD" instead?
FYI: my log file is:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.test.log
Thank you.
Binyu
On Thu, Aug 27, 2020 at 3:28 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> In general, MET does not provide a way of reading multiple input
variables
> and deriving a new field in the configuration file. However, we did
add
> support for python-embedding with this use in mind. Generally
speaking, you
> can write a python script to read data in whatever way you'd like...
> compute whatever sort of values you'd like... and then pass the
result to
> the MET tools via memory. So that's the recommended way of deriving
new
> datasets for MET to process.
>
> However, python-embedding is not required for WIND speed. You've
stumbled
> across a very particular derivation type that is already supported
directly
> in MET. When using input GRIB1 or GRIB2 files, if you request the
"WIND"
> variable, MET will first try to read that from the input data file.
If
> found, it'll use it. If not found, it'll read the corresponding UGRD
and
> VGRD records and use them to derive WIND speed on the fly.
>
> I'd recommend running the plot_data_plane tool to try that out:
>
> plot_data_plane in.grb wind_speed.ps 'name="WIND"; level="P850";' -v
4
>
> And then look closely at the log messages that are printed to
screen. It
> should tell you that it's deriving WIND speed for you on the fly.
>
> Thanks,
> John
>
> On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: calculate multiple variables in config file
> > Owner: johnhg
> > Requestors: binyu.wang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
> >
> >
> > This transaction appears to have no content
> >
>
>
------------------------------------------------
Subject: calculate multiple variables in config file
From: John Halley Gotway
Time: Fri Aug 28 14:40:32 2020
Binyu,
Yes, MET does know how to derive wind direction (WDIR) on the fly.
However,
the MET statistics tools DO NOT SUPPORT direct support of verification
of
the wind direction field.
The traditional statistics, such as RMSE, are not appropriate for wind
direction... in which values of 359 and 1 degrees are very close.
Instead,
wind vector statistics are provided by the VL1L2 partial sums and VCNT
statistics.
If you request WDIR when running Grid-Stat, you should see the
following
error message:
ERROR :
ERROR : GridStatVxOpt::process_config() -> wind direction may not be
verified using grid_stat.
ERROR :
Thanks,
John
On Fri, Aug 28, 2020 at 11:02 AM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
>
> Hello John,
>
> I want to double check about using MET for the wind verification:
>
> You told me I could use UGRD/VGRD to get wind direction information
by
> setting " vl1l2 = BOTH; and vcnt = BOTH;" before, and I did get
it done
> successfully but with "v9.1_beta2" unfortunately. Yesterday you
said MET
> can get WIND by UGRD/VGRD, so I am thinking maybe I can get wind
direction
> information directly using the config below (by doing that, I don't
need
> worry about the rule that UGRD must be immediately
> followed by a VGRD observation at the same level.). My Obs. has WIND
> variable and my model has UGRD/VGRD.
>
> fcst = {
> message_type = [ "ADPUPA" ];
> field = [
> {
> name = "WIND";
> level = [ "P250" ];
> cat_thresh = [ <=1.0, >1&&<=3.0,>3&&<=10.0 ];
> },
>
> I didn't get any error message in the log file, but the output is
empty.
> Does that mean I can not get wind direction stat. information by
reading in
> "WIND" and MUST use "UGRD/VGRD" instead?
>
> FYI: my log file is:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.test.log
>
> Thank you.
> Binyu
> On Thu, Aug 27, 2020 at 3:28 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > In general, MET does not provide a way of reading multiple input
> variables
> > and deriving a new field in the configuration file. However, we
did add
> > support for python-embedding with this use in mind. Generally
speaking,
> you
> > can write a python script to read data in whatever way you'd
like...
> > compute whatever sort of values you'd like... and then pass the
result to
> > the MET tools via memory. So that's the recommended way of
deriving new
> > datasets for MET to process.
> >
> > However, python-embedding is not required for WIND speed. You've
stumbled
> > across a very particular derivation type that is already supported
> directly
> > in MET. When using input GRIB1 or GRIB2 files, if you request the
"WIND"
> > variable, MET will first try to read that from the input data
file. If
> > found, it'll use it. If not found, it'll read the corresponding
UGRD and
> > VGRD records and use them to derive WIND speed on the fly.
> >
> > I'd recommend running the plot_data_plane tool to try that out:
> >
> > plot_data_plane in.grb wind_speed.ps 'name="WIND"; level="P850";'
-v 4
> >
> > And then look closely at the log messages that are printed to
screen. It
> > should tell you that it's deriving WIND speed for you on the fly.
> >
> > Thanks,
> > John
> >
> > On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > > Queue: met_help
> > > Subject: calculate multiple variables in config file
> > > Owner: johnhg
> > > Requestors: binyu.wang at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463
> >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
>
>
------------------------------------------------
Subject: calculate multiple variables in config file
From: binyu.wang at noaa.gov
Time: Sat Aug 29 19:29:02 2020
Yes RMSE is not appropriate. But I could get " FDIR" and "ODIR" from
VCNT "
with UGRD and VGRD, why I can not get the same information using WIND?
Since MET has the ability to derive UGRD/VGRD from wind speed and vise
versa?
Binyu
On Fri, Aug 28, 2020 at 4:40 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> Yes, MET does know how to derive wind direction (WDIR) on the fly.
However,
> the MET statistics tools DO NOT SUPPORT direct support of
verification of
> the wind direction field.
>
> The traditional statistics, such as RMSE, are not appropriate for
wind
> direction... in which values of 359 and 1 degrees are very close.
Instead,
> wind vector statistics are provided by the VL1L2 partial sums and
VCNT
> statistics.
>
> If you request WDIR when running Grid-Stat, you should see the
following
> error message:
>
> ERROR :
>
> ERROR : GridStatVxOpt::process_config() -> wind direction may not
be
> verified using grid_stat.
>
> ERROR :
>
>
> Thanks,
> John
>
> On Fri, Aug 28, 2020 at 11:02 AM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
> >
> > Hello John,
> >
> > I want to double check about using MET for the wind verification:
> >
> > You told me I could use UGRD/VGRD to get wind direction
information by
> > setting " vl1l2 = BOTH; and vcnt = BOTH;" before, and I did get
it
> done
> > successfully but with "v9.1_beta2" unfortunately. Yesterday you
said MET
> > can get WIND by UGRD/VGRD, so I am thinking maybe I can get wind
> direction
> > information directly using the config below (by doing that, I
don't need
> > worry about the rule that UGRD must be immediately
> > followed by a VGRD observation at the same level.). My Obs. has
WIND
> > variable and my model has UGRD/VGRD.
> >
> > fcst = {
> > message_type = [ "ADPUPA" ];
> > field = [
> > {
> > name = "WIND";
> > level = [ "P250" ];
> > cat_thresh = [ <=1.0, >1&&<=3.0,>3&&<=10.0 ];
> > },
> >
> > I didn't get any error message in the log file, but the output is
empty.
> > Does that mean I can not get wind direction stat. information by
reading
> in
> > "WIND" and MUST use "UGRD/VGRD" instead?
> >
> > FYI: my log file is:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.test.log
> >
> > Thank you.
> > Binyu
> > On Thu, Aug 27, 2020 at 3:28 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > >
> > > In general, MET does not provide a way of reading multiple input
> > variables
> > > and deriving a new field in the configuration file. However, we
did add
> > > support for python-embedding with this use in mind. Generally
speaking,
> > you
> > > can write a python script to read data in whatever way you'd
like...
> > > compute whatever sort of values you'd like... and then pass the
result
> to
> > > the MET tools via memory. So that's the recommended way of
deriving new
> > > datasets for MET to process.
> > >
> > > However, python-embedding is not required for WIND speed. You've
> stumbled
> > > across a very particular derivation type that is already
supported
> > directly
> > > in MET. When using input GRIB1 or GRIB2 files, if you request
the
> "WIND"
> > > variable, MET will first try to read that from the input data
file. If
> > > found, it'll use it. If not found, it'll read the corresponding
UGRD
> and
> > > VGRD records and use them to derive WIND speed on the fly.
> > >
> > > I'd recommend running the plot_data_plane tool to try that out:
> > >
> > > plot_data_plane in.grb wind_speed.ps 'name="WIND";
level="P850";' -v 4
> > >
> > > And then look closely at the log messages that are printed to
screen.
> It
> > > should tell you that it's deriving WIND speed for you on the
fly.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> > > > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > > > Queue: met_help
> > > > Subject: calculate multiple variables in config file
> > > > Owner: johnhg
> > > > Requestors: binyu.wang at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463
> > >
> > > >
> > > >
> > > > This transaction appears to have no content
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: calculate multiple variables in config file
From: John Halley Gotway
Time: Mon Aug 31 13:35:04 2020
Binyu,
MET does have the ability to read UGRD and VGRD and derive WDIR and
WIND.
But it does not include logic to do the reverse... reading WDIR and
WIND
and deriving UGRD and VGRD. We just have not had a funded project that
required this logic.
Yes, you can process UGRD, VGRD, and WIND (speed) as scalars in the
MET
tools and derive continuous statistics, like RMSE.
However, you cannot verify WDIR directly. Instead, we advise verifying
UGRD
and VGRD and requesting the VL1L2 and VCNT output line types, which
contain
vector partials sums and statistics.
I'm not sure I'm answering your question. If not, please try re-
stating it
and I'll try again.
Thanks,
John
On Sat, Aug 29, 2020 at 7:29 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
>
> Yes RMSE is not appropriate. But I could get " FDIR" and "ODIR" from
VCNT "
> with UGRD and VGRD, why I can not get the same information using
WIND?
> Since MET has the ability to derive UGRD/VGRD from wind speed and
vise
> versa?
>
> Binyu
>
> On Fri, Aug 28, 2020 at 4:40 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > Yes, MET does know how to derive wind direction (WDIR) on the fly.
> However,
> > the MET statistics tools DO NOT SUPPORT direct support of
verification of
> > the wind direction field.
> >
> > The traditional statistics, such as RMSE, are not appropriate for
wind
> > direction... in which values of 359 and 1 degrees are very close.
> Instead,
> > wind vector statistics are provided by the VL1L2 partial sums and
VCNT
> > statistics.
> >
> > If you request WDIR when running Grid-Stat, you should see the
following
> > error message:
> >
> > ERROR :
> >
> > ERROR : GridStatVxOpt::process_config() -> wind direction may not
be
> > verified using grid_stat.
> >
> > ERROR :
> >
> >
> > Thanks,
> > John
> >
> > On Fri, Aug 28, 2020 at 11:02 AM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463 >
> > >
> > > Hello John,
> > >
> > > I want to double check about using MET for the wind
verification:
> > >
> > > You told me I could use UGRD/VGRD to get wind direction
information by
> > > setting " vl1l2 = BOTH; and vcnt = BOTH;" before, and I did
get it
> > done
> > > successfully but with "v9.1_beta2" unfortunately. Yesterday you
said
> MET
> > > can get WIND by UGRD/VGRD, so I am thinking maybe I can get wind
> > direction
> > > information directly using the config below (by doing that, I
don't
> need
> > > worry about the rule that UGRD must be immediately
> > > followed by a VGRD observation at the same level.). My Obs. has
WIND
> > > variable and my model has UGRD/VGRD.
> > >
> > > fcst = {
> > > message_type = [ "ADPUPA" ];
> > > field = [
> > > {
> > > name = "WIND";
> > > level = [ "P250" ];
> > > cat_thresh = [ <=1.0, >1&&<=3.0,>3&&<=10.0 ];
> > > },
> > >
> > > I didn't get any error message in the log file, but the output
is
> empty.
> > > Does that mean I can not get wind direction stat. information by
> reading
> > in
> > > "WIND" and MUST use "UGRD/VGRD" instead?
> > >
> > > FYI: my log file is:
> > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met_wdir/ecf/02.test.log
> > >
> > > Thank you.
> > > Binyu
> > > On Thu, Aug 27, 2020 at 3:28 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Binyu,
> > > >
> > > > In general, MET does not provide a way of reading multiple
input
> > > variables
> > > > and deriving a new field in the configuration file. However,
we did
> add
> > > > support for python-embedding with this use in mind. Generally
> speaking,
> > > you
> > > > can write a python script to read data in whatever way you'd
like...
> > > > compute whatever sort of values you'd like... and then pass
the
> result
> > to
> > > > the MET tools via memory. So that's the recommended way of
deriving
> new
> > > > datasets for MET to process.
> > > >
> > > > However, python-embedding is not required for WIND speed.
You've
> > stumbled
> > > > across a very particular derivation type that is already
supported
> > > directly
> > > > in MET. When using input GRIB1 or GRIB2 files, if you request
the
> > "WIND"
> > > > variable, MET will first try to read that from the input data
file.
> If
> > > > found, it'll use it. If not found, it'll read the
corresponding UGRD
> > and
> > > > VGRD records and use them to derive WIND speed on the fly.
> > > >
> > > > I'd recommend running the plot_data_plane tool to try that
out:
> > > >
> > > > plot_data_plane in.grb wind_speed.ps 'name="WIND";
level="P850";'
> -v 4
> > > >
> > > > And then look closely at the log messages that are printed to
screen.
> > It
> > > > should tell you that it's deriving WIND speed for you on the
fly.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Aug 26, 2020 at 3:28 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Wed Aug 26 15:27:36 2020: Request 96463 was acted upon.
> > > > > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > > > > Queue: met_help
> > > > > Subject: calculate multiple variables in config file
> > > > > Owner: johnhg
> > > > > Requestors: binyu.wang at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96463
> > > >
> > > > >
> > > > >
> > > > > This transaction appears to have no content
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list