[Met_help] [rt.rap.ucar.edu #99031] History for MXUPHL PcpCombine use case with METplus-4.0_beta4
George McCabe via RT
met_help at ucar.edu
Thu Mar 18 09:46:22 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I've been in touch with George about updates to the PcpCombine wrapper to
pass in multiple fields and levels to the pcp_combine command. The
enhancement is captured in GitHub issue #718
<https://github.com/dtcenter/METplus/issues/718> and was merged into the
develop branch before the METplus-4.0 beta4 release.
I didn't get my testing done with George's feature branch, so I've just
been able to test with the beta4 release now. I'm still running into one
issue with my use case, which I've detailed below.
Here are some of the relevant configuration settings I have defined for the
use case (24-h max accumulation of MXUPHL and RELV from 1-hrly files):
FCST_PCP_COMBINE_INPUT_ACCUMS = 1
FCST_PCP_COMBINE_INPUT_NAMES = MXUPHL
FCST_PCP_COMBINE_INPUT_LEVELS = Z2000-5000
FCST_PCP_COMBINE_EXTRA_NAMES = MXUPHL, RELV, RELV
FCST_PCP_COMBINE_EXTRA_LEVELS = Z0-3000, Z0-2000, Z0-1000
FCST_PCP_COMBINE_OUTPUT_ACCUM = 24
FCST_PCP_COMBINE_DERIVE_LOOKBACK = 24
This is the resultant MET command:
/gpfs/dell2/emc/verification/noscrub/emc.metplus/met/10.0.0-beta4/exec/pcp_combine
-v 5 -derive MAX (list of 24 files) -field 'name="MXUPHL";
level="Z2000-5000";' -field 'name="MXUPHL"; level="Z0-3000";' -field
'name="RELV"; level="Z0-2000";' -field 'name="RELV"; level="Z0-1000";'
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
An error results from this setup. Because the level is used to specify a
height for MXUPHL as opposed to an accumulation length, there's a
disconnect where the level attribute comes out as "A1", and the MXUPHL
accumulations overwrite one another. From the log file:
DEBUG 1: Creating output file:
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
DEBUG 2: Writing output variable "MXUPHL_01_MAX" for the "MAX" of "MXUPHL/
Z5000-2000".
(pcp_combine works on second -field entry)
DEBUG 2: Writing output variable "MXUPHL_01_MAX" for the "MAX" of
"MXUPHL/Z3000-0".
terminate called after throwing an instance of
'netCDF::exceptions::NcNameInUse'
what(): NetCDF: String match to name in use
file: ncGroup.cpp line:496
I was already aware of the issue with -field 'name="MXUPHL";
level="Z2000-5000";' writing out to "MXUPHL_01_MAX", so I have been using
the FCST_PCP_COMBINE_OUTPUT_NAME variable to distinguish different
variables and levels from one another. When I add FCST_PCP_COMBINE_OUTPUT_NAME
= MXUPHL25_A24 to my configuration file, this is the command I get:
/gpfs/dell2/emc/verification/noscrub/emc.metplus/met/10.0.0-beta4/exec/pcp_combine
-v 5 -derive MAX (list of 24 files) -field 'name="MXUPHL";
level="Z2000-5000";' -field 'name="RELV"; level="Z0-2000";' -name
"MXUPHL25_A24"
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
And MET throws a quick error:
DEBUG 2: Since the "-field" command line option was used, parsing the
command line arguments as a list of files.
ERROR :
ERROR : process_command_line() -> expected 2 entries for the "-name"
command line option but got 1!
I was able to trick METplus into writing two entries in the command by
using the setting:
FCST_PCP_COMBINE_OUTPUT_NAME = MXUPHL25_A24","RELV02_A24
It works, but it would probably make sense if there was a configuration
like FCST_PCP_COMBINE_EXTRA_OUTPUT_NAMES to allow for modification of
multiple output names in instances when multiple input variables and levels
are specified.
Thanks,
Logan
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
From: George McCabe
Time: Mon Mar 15 12:37:30 2021
Hi Logan,
I think you are exactly right that we should add another configuration
to
specify the extra output names. I didn't think about making sure the
number
of input names matched the number of output names. I'll re-open the
GitHub
issue and take a look. Thanks for testing this and discovered the
missing
piece needed.
- George
On Fri, Mar 12, 2021 at 11:24 AM Logan Dawson - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Fri Mar 12 11:24:09 2021: Request 99031 was acted upon.
> Transaction: Ticket created by logan.dawson at noaa.gov
> Queue: met_help
> Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
> Owner: Nobody
> Requestors: logan.dawson at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99031 >
>
>
> Hello,
>
> I've been in touch with George about updates to the PcpCombine
wrapper to
> pass in multiple fields and levels to the pcp_combine command. The
> enhancement is captured in GitHub issue #718
> <https://github.com/dtcenter/METplus/issues/718> and was merged into
the
> develop branch before the METplus-4.0 beta4 release.
>
> I didn't get my testing done with George's feature branch, so I've
just
> been able to test with the beta4 release now. I'm still running into
one
> issue with my use case, which I've detailed below.
>
> Here are some of the relevant configuration settings I have defined
for the
> use case (24-h max accumulation of MXUPHL and RELV from 1-hrly
files):
>
> FCST_PCP_COMBINE_INPUT_ACCUMS = 1
>
> FCST_PCP_COMBINE_INPUT_NAMES = MXUPHL
>
> FCST_PCP_COMBINE_INPUT_LEVELS = Z2000-5000
>
> FCST_PCP_COMBINE_EXTRA_NAMES = MXUPHL, RELV, RELV
>
> FCST_PCP_COMBINE_EXTRA_LEVELS = Z0-3000, Z0-2000, Z0-1000
>
> FCST_PCP_COMBINE_OUTPUT_ACCUM = 24
>
> FCST_PCP_COMBINE_DERIVE_LOOKBACK = 24
>
> This is the resultant MET command:
>
>
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/met/10.0.0-
beta4/exec/pcp_combine
> -v 5 -derive MAX (list of 24 files) -field 'name="MXUPHL";
> level="Z2000-5000";' -field 'name="MXUPHL"; level="Z0-3000";' -field
> 'name="RELV"; level="Z0-2000";' -field 'name="RELV"; level="Z0-
1000";'
>
>
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
> HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
>
> An error results from this setup. Because the level is used to
specify a
> height for MXUPHL as opposed to an accumulation length, there's a
> disconnect where the level attribute comes out as "A1", and the
MXUPHL
> accumulations overwrite one another. From the log file:
>
> DEBUG 1: Creating output file:
>
>
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
> HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
>
> DEBUG 2: Writing output variable "MXUPHL_01_MAX" for the "MAX" of
"MXUPHL/
> Z5000-2000".
>
>
> (pcp_combine works on second -field entry)
>
>
> DEBUG 2: Writing output variable "MXUPHL_01_MAX" for the "MAX" of
> "MXUPHL/Z3000-0".
>
> terminate called after throwing an instance of
> 'netCDF::exceptions::NcNameInUse'
>
> what(): NetCDF: String match to name in use
>
> file: ncGroup.cpp line:496
>
>
> I was already aware of the issue with -field 'name="MXUPHL";
> level="Z2000-5000";' writing out to "MXUPHL_01_MAX", so I have been
using
> the FCST_PCP_COMBINE_OUTPUT_NAME variable to distinguish different
> variables and levels from one another. When I add
> FCST_PCP_COMBINE_OUTPUT_NAME
> = MXUPHL25_A24 to my configuration file, this is the command I get:
>
>
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/met/10.0.0-
beta4/exec/pcp_combine
> -v 5 -derive MAX (list of 24 files) -field 'name="MXUPHL";
> level="Z2000-5000";' -field 'name="RELV"; level="Z0-2000";' -name
> "MXUPHL25_A24"
>
>
/gpfs/dell2/ptmp/Logan.Dawson/CAM_verif/METplus/metplus.out/surrogate_severe/pcp_combine/2021031212/
> HRRR.20210311.t12z.2021031212-2021031212.wrfprsf24.nc
>
> And MET throws a quick error:
>
> DEBUG 2: Since the "-field" command line option was used, parsing
the
> command line arguments as a list of files.
>
> ERROR :
>
> ERROR : process_command_line() -> expected 2 entries for the "-
name"
> command line option but got 1!
>
> I was able to trick METplus into writing two entries in the command
by
> using the setting:
>
> FCST_PCP_COMBINE_OUTPUT_NAME = MXUPHL25_A24","RELV02_A24
>
> It works, but it would probably make sense if there was a
configuration
> like FCST_PCP_COMBINE_EXTRA_OUTPUT_NAMES to allow for modification
of
> multiple output names in instances when multiple input variables and
levels
> are specified.
>
> Thanks,
> Logan
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
From: George McCabe
Time: Mon Mar 15 15:07:53 2021
Hi Logan,
I made these changes to a branch called
feature_718_pcp_combine_output_names. Could you test out these changes
with
your use case to make sure they work as expected? The new variable you
would use is called FCST_PCP_COMBINE_EXTRA_OUTPUT_NAMES . Note that if
you
don't have FCST_PCP_COMBINE_OUTPUT_NAME set to anything, you will have
to
set that as well. If not, it will not read the extra output names. Let
me
know if you have any questions and how it goes.
Thanks,
George
On Mon, Mar 15, 2021 at 12:38 PM The RT System itself via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 15 12:37:30 2021: Request 99031 was acted upon.
> Transaction: Given to mccabe (George McCabe) by RT_System
> Queue: met_help
> Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
> Owner: mccabe
> Requestors: logan.dawson at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99031 >
>
>
> This transaction appears to have no content
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
From: George McCabe
Time: Mon Mar 15 15:27:25 2021
Hi Logan,
I assigned you as the reviewer for this pull request regarding these
changes:
https://github.com/dtcenter/METplus/pull/843
You can test that the changes work for your use case and approve the
pull
request if they do. Let me know if you have any questions.
Thanks,
George
On Mon, Mar 15, 2021 at 3:07 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Logan,
> I made these changes to a branch called
> feature_718_pcp_combine_output_names. Could you test out these
changes with
> your use case to make sure they work as expected? The new variable
you
> would use is called FCST_PCP_COMBINE_EXTRA_OUTPUT_NAMES . Note that
if you
> don't have FCST_PCP_COMBINE_OUTPUT_NAME set to anything, you will
have to
> set that as well. If not, it will not read the extra output names.
Let me
> know if you have any questions and how it goes.
>
> Thanks,
> George
>
> On Mon, Mar 15, 2021 at 12:38 PM The RT System itself via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Mon Mar 15 12:37:30 2021: Request 99031 was acted upon.
>> Transaction: Given to mccabe (George McCabe) by RT_System
>> Queue: met_help
>> Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
>> Owner: mccabe
>> Requestors: logan.dawson at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99031 >
>>
>>
>> This transaction appears to have no content
>>
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
From: George McCabe
Time: Thu Mar 18 09:46:12 2021
Thank you for reviewing and approving the pull request, Logan. The
changes are now in the develop branch and will be available in METplus
4.0.0. Closing this ticket.
- George
On Mon Mar 15 15:27:25 2021, mccabe wrote:
> Hi Logan,
>
> I assigned you as the reviewer for this pull request regarding these
> changes:
> https://github.com/dtcenter/METplus/pull/843
>
> You can test that the changes work for your use case and approve the
> pull
> request if they do. Let me know if you have any questions.
>
> Thanks,
> George
>
> On Mon, Mar 15, 2021 at 3:07 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Logan,
> > I made these changes to a branch called
> > feature_718_pcp_combine_output_names. Could you test out these
> > changes with
> > your use case to make sure they work as expected? The new variable
> > you
> > would use is called FCST_PCP_COMBINE_EXTRA_OUTPUT_NAMES . Note
that
> > if you
> > don't have FCST_PCP_COMBINE_OUTPUT_NAME set to anything, you will
> > have to
> > set that as well. If not, it will not read the extra output names.
> > Let me
> > know if you have any questions and how it goes.
> >
> > Thanks,
> > George
> >
> > On Mon, Mar 15, 2021 at 12:38 PM The RT System itself via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Mon Mar 15 12:37:30 2021: Request 99031 was acted upon.
> >> Transaction: Given to mccabe (George McCabe) by RT_System
> >> Queue: met_help
> >> Subject: MXUPHL PcpCombine use case with METplus-4.0_beta4
> >> Owner: mccabe
> >> Requestors: logan.dawson at noaa.gov
> >> Status: new
> >> Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99031 >
> >>
> >>
> >> This transaction appears to have no content
> >>
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to
> > reply to this email outside of your normal working hours.
> >
------------------------------------------------
More information about the Met_help
mailing list