[Met_help] [rt.rap.ucar.edu #99740] History for Libpng warning with MET-9.1.3 in production on WCOSS Dell
Julie Prestopnik via RT
met_help at ucar.edu
Thu May 6 09:10:02 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
The MET-9.1.3 installed into operations on WCOSS Dell (
/gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng warning when
trying to read MRMS files. Here's an example of the output from
regrid_data_plane showing that MET reads all missing data from a MRMS file:
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field 1 of GRIB2
file
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-000000.grib2'
*libpng warning: Application built with libpng-1.2.59 but running with
1.5.13*
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: -999
*DEBUG 4: plane max: -999*
DEBUG 4: scan mode: 0
DEBUG 4: valid time: 20210426_000034
DEBUG 4: lead time: 000000
DEBUG 4: init time: 20210426_000034
DEBUG 4: bitmap flag: 255
DEBUG 4:
*DEBUG 2: Range of input data (name="MergedReflectivityQCComposite";
level="Z500";) is -999 to -999.*
*DEBUG 2: Range of regridded data (name="MergedReflectivityQCComposite";
level="Z500";) is -999 to -999.*
DEBUG 1: Writing output file:
/gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_20210426-000000_g227.nc
This same issue previously occurred when MET-9.0.1 was installed on the
Dells in May/June 2020, and Julie found a resolution for the issue before
the MET-9.0.2 installation (see MET-help ticket #95364). The issue was
resolved with MET installs in the noscrub/emc.metplus space up through
MET-9.1. However, once this libpng warning issue was identified in the
nwprod/met.v9.1.3 installation, I found that the same issue shows up with
the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2 installations on the
Dell.
It seems as if the resolution from the previous MET-help ticket #95364
(adding LIBPNG to LDFLAGS in the compilation script) maybe didn't prevent
this problem from resurfacing in MET-9.1.1 and later installations? It
seems that we should take a look to see if we can find a fix before
MET-10.0 is released and installed on WCOSS.
Thanks,
Logan
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Thu Apr 29 12:22:47 2021
Hi Logan.
Thank you for bringing this problem to our attention, for reminding me
of
this past issue, and for doing the research and adding "see MET-help
ticket
#95364" in the email. That was very helpful information, and I
appreciate
your effort!
Can you please send me the command line you are using and the location
of
your data so I can test with the version that Mallory put together for
nco
before the official release of met-9.1.3?
The script we provided should already put the value for LIBPNG in
LDFLAGS,
so I suspect the value may not be set correctly, but I will look into
that.
I'll follow up once I'm able to run your test case with the other
version
and compare to see what's going on.
We definitely want to get this fixed before the installation of met-
10.0.0,
so thanks for staying on top of this and alerting us now.
Julie
On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> Transaction: Ticket created by logan.dawson at noaa.gov
> Queue: met_help
> Subject: Libpng warning with MET-9.1.3 in production on WCOSS
Dell
> Owner: Nobody
> Requestors: logan.dawson at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
>
> Hello,
>
> The MET-9.1.3 installed into operations on WCOSS Dell (
> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng
warning when
> trying to read MRMS files. Here's an example of the output from
> regrid_data_plane showing that MET reads all missing data from a
MRMS file:
>
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field 1 of
GRIB2
> file
>
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
>
> *libpng warning: Application built with libpng-1.2.59 but running
with
> 1.5.13*
>
> DEBUG 4:
>
> DEBUG 4: Data plane information:
>
> DEBUG 4: plane min: -999
>
> *DEBUG 4: plane max: -999*
>
> DEBUG 4: scan mode: 0
>
> DEBUG 4: valid time: 20210426_000034
>
> DEBUG 4: lead time: 000000
>
> DEBUG 4: init time: 20210426_000034
>
> DEBUG 4: bitmap flag: 255
>
> DEBUG 4:
>
> *DEBUG 2: Range of input data (name="MergedReflectivityQCComposite";
> level="Z500";) is -999 to -999.*
>
> *DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
> level="Z500";) is -999 to -999.*
>
> DEBUG 1: Writing output file:
> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> 20210426-000000_g227.nc
>
> This same issue previously occurred when MET-9.0.1 was installed on
the
> Dells in May/June 2020, and Julie found a resolution for the issue
before
> the MET-9.0.2 installation (see MET-help ticket #95364). The issue
was
> resolved with MET installs in the noscrub/emc.metplus space up
through
> MET-9.1. However, once this libpng warning issue was identified in
the
> nwprod/met.v9.1.3 installation, I found that the same issue shows up
with
> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2 installations on
the
> Dell.
>
> It seems as if the resolution from the previous MET-help ticket
#95364
> (adding LIBPNG to LDFLAGS in the compilation script) maybe didn't
prevent
> this problem from resurfacing in MET-9.1.1 and later installations?
It
> seems that we should take a look to see if we can find a fix before
> MET-10.0 is released and installed on WCOSS.
>
> Thanks,
> Logan
>
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Thu Apr 29 13:04:45 2021
Unfortunately, I am wrong on all accounts.
The script did not contain the addition of "-L${LIB_LIBPNG}" in rpath,
and
the value of LIB_LIBPNG is set correctly.
The script currently contains:
> export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} -L${MET_HDF5}/lib"
but should contain:
> export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-L${LIB_LIBPNG}*
> -L${MET_HDF5}/lib"
I have added Mallory to this email to see if there is anything we can
do to
get this fixed for the already installed version of met-9.1.3. If
not, we
will be releasing met-10.0.0 soon and hopefully the install process
will go
more quickly since we recently went through it for 9.1.3.
Mallory, I have made some other additions and updates to the
compile_MET_all.sh script, which I will include for the release of
10.0.0.
Logan and Mallory, I would also like to test this addition to the
compile_MET_all.sh script to make sure it solves this problem (again).
I
plan to update my latest install of met-10.0.0-beta5 to test, but I
can
also test to get a good install of 9.1.3 if you would like, Logan.
Please
let me know what you would prefer.
Please let me know if you have any questions. My apologies for this
trouble!
Julie
On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <jpresto at ucar.edu>
wrote:
> Hi Logan.
>
> Thank you for bringing this problem to our attention, for reminding
me of
> this past issue, and for doing the research and adding "see MET-help
ticket
> #95364" in the email. That was very helpful information, and I
appreciate
> your effort!
>
> Can you please send me the command line you are using and the
location of
> your data so I can test with the version that Mallory put together
for nco
> before the official release of met-9.1.3?
>
> The script we provided should already put the value for LIBPNG in
LDFLAGS,
> so I suspect the value may not be set correctly, but I will look
into
> that.
>
> I'll follow up once I'm able to run your test case with the other
version
> and compare to see what's going on.
>
> We definitely want to get this fixed before the installation of
> met-10.0.0, so thanks for staying on top of this and alerting us
now.
>
> Julie
>
> On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
>> Transaction: Ticket created by logan.dawson at noaa.gov
>> Queue: met_help
>> Subject: Libpng warning with MET-9.1.3 in production on WCOSS
Dell
>> Owner: Nobody
>> Requestors: logan.dawson at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>>
>>
>> Hello,
>>
>> The MET-9.1.3 installed into operations on WCOSS Dell (
>> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng
warning when
>> trying to read MRMS files. Here's an example of the output from
>> regrid_data_plane showing that MET reads all missing data from a
MRMS
>> file:
>>
>> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
>> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field 1 of
GRIB2
>> file
>>
>>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
>>
>> *libpng warning: Application built with libpng-1.2.59 but running
with
>> 1.5.13*
>>
>> DEBUG 4:
>>
>> DEBUG 4: Data plane information:
>>
>> DEBUG 4: plane min: -999
>>
>> *DEBUG 4: plane max: -999*
>>
>> DEBUG 4: scan mode: 0
>>
>> DEBUG 4: valid time: 20210426_000034
>>
>> DEBUG 4: lead time: 000000
>>
>> DEBUG 4: init time: 20210426_000034
>>
>> DEBUG 4: bitmap flag: 255
>>
>> DEBUG 4:
>>
>> *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
>> level="Z500";) is -999 to -999.*
>>
>> *DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
>> level="Z500";) is -999 to -999.*
>>
>> DEBUG 1: Writing output file:
>> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
>> 20210426-000000_g227.nc
>>
>> This same issue previously occurred when MET-9.0.1 was installed on
the
>> Dells in May/June 2020, and Julie found a resolution for the issue
before
>> the MET-9.0.2 installation (see MET-help ticket #95364). The issue
was
>> resolved with MET installs in the noscrub/emc.metplus space up
through
>> MET-9.1. However, once this libpng warning issue was identified in
the
>> nwprod/met.v9.1.3 installation, I found that the same issue shows
up with
>> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2 installations
on the
>> Dell.
>>
>> It seems as if the resolution from the previous MET-help ticket
#95364
>> (adding LIBPNG to LDFLAGS in the compilation script) maybe didn't
prevent
>> this problem from resurfacing in MET-9.1.1 and later installations?
It
>> seems that we should take a look to see if we can find a fix before
>> MET-10.0 is released and installed on WCOSS.
>>
>> Thanks,
>> Logan
>>
>> --
>> Logan C. Dawson, Ph.D. (he/him/his)
>> Physical Scientist, Model Evaluation Group (VPPPG Branch)
>> NOAA/NWS/NCEP/EMC
>> (301) 683-3944
>>
>>
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Logan Dawson - NOAA Affiliate
Time: Thu Apr 29 13:31:16 2021
Hi Julie,
Here's the command that produced the output I previously sent:
/gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane -v 5
-method
BUDGET -width 2 -field 'name="MergedReflectivityQCComposite";
level="Z500";' -name MergedReflectivityQCComposite
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
/gpfs/dell2/ptmp/Logan.Dawson/
MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
I moved the input file to a location that won't be scrubbed while
we're
testing.
If it's not too much trouble, it might be good to test this with an
update
to the met-10.0.0-beta5 installation and a good install of met-9.1.3.
That
way, if NCO is able to update the version installed in nwprod, we'll
have
confirmation that the compilation update will fix the problem with
that
version. I assume it will, but it wouldn't hurt to confirm before
going
back to NCO.
Thanks very much for checking into this!
Logan
On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Unfortunately, I am wrong on all accounts.
>
> The script did not contain the addition of "-L${LIB_LIBPNG}" in
rpath, and
> the value of LIB_LIBPNG is set correctly.
>
> The script currently contains:
>
> > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} -L${MET_HDF5}/lib"
>
>
> but should contain:
>
> > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-L${LIB_LIBPNG}*
> > -L${MET_HDF5}/lib"
>
>
> I have added Mallory to this email to see if there is anything we
can do to
> get this fixed for the already installed version of met-9.1.3. If
not, we
> will be releasing met-10.0.0 soon and hopefully the install process
will go
> more quickly since we recently went through it for 9.1.3.
>
> Mallory, I have made some other additions and updates to the
> compile_MET_all.sh script, which I will include for the release of
10.0.0.
>
> Logan and Mallory, I would also like to test this addition to the
> compile_MET_all.sh script to make sure it solves this problem
(again). I
> plan to update my latest install of met-10.0.0-beta5 to test, but I
can
> also test to get a good install of 9.1.3 if you would like, Logan.
Please
> let me know what you would prefer.
>
> Please let me know if you have any questions. My apologies for this
> trouble!
>
> Julie
>
>
> On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <jpresto at ucar.edu>
> wrote:
>
> > Hi Logan.
> >
> > Thank you for bringing this problem to our attention, for
reminding me of
> > this past issue, and for doing the research and adding "see MET-
help
> ticket
> > #95364" in the email. That was very helpful information, and I
> appreciate
> > your effort!
> >
> > Can you please send me the command line you are using and the
location of
> > your data so I can test with the version that Mallory put together
for
> nco
> > before the official release of met-9.1.3?
> >
> > The script we provided should already put the value for LIBPNG in
> LDFLAGS,
> > so I suspect the value may not be set correctly, but I will look
into
> > that.
> >
> > I'll follow up once I'm able to run your test case with the other
version
> > and compare to see what's going on.
> >
> > We definitely want to get this fixed before the installation of
> > met-10.0.0, so thanks for staying on top of this and alerting us
now.
> >
> > Julie
> >
> > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> >> Transaction: Ticket created by logan.dawson at noaa.gov
> >> Queue: met_help
> >> Subject: Libpng warning with MET-9.1.3 in production on
WCOSS Dell
> >> Owner: Nobody
> >> Requestors: logan.dawson at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> >>
> >>
> >> Hello,
> >>
> >> The MET-9.1.3 installed into operations on WCOSS Dell (
> >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng
warning
> when
> >> trying to read MRMS files. Here's an example of the output from
> >> regrid_data_plane showing that MET reads all missing data from a
MRMS
> >> file:
> >>
> >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field 1 of
GRIB2
> >> file
> >>
> >>
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> >>
> >> *libpng warning: Application built with libpng-1.2.59 but running
with
> >> 1.5.13*
> >>
> >> DEBUG 4:
> >>
> >> DEBUG 4: Data plane information:
> >>
> >> DEBUG 4: plane min: -999
> >>
> >> *DEBUG 4: plane max: -999*
> >>
> >> DEBUG 4: scan mode: 0
> >>
> >> DEBUG 4: valid time: 20210426_000034
> >>
> >> DEBUG 4: lead time: 000000
> >>
> >> DEBUG 4: init time: 20210426_000034
> >>
> >> DEBUG 4: bitmap flag: 255
> >>
> >> DEBUG 4:
> >>
> >> *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> >> level="Z500";) is -999 to -999.*
> >>
> >> *DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
> >> level="Z500";) is -999 to -999.*
> >>
> >> DEBUG 1: Writing output file:
> >> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> >> 20210426-000000_g227.nc
> >>
> >> This same issue previously occurred when MET-9.0.1 was installed
on the
> >> Dells in May/June 2020, and Julie found a resolution for the
issue
> before
> >> the MET-9.0.2 installation (see MET-help ticket #95364). The
issue was
> >> resolved with MET installs in the noscrub/emc.metplus space up
through
> >> MET-9.1. However, once this libpng warning issue was identified
in the
> >> nwprod/met.v9.1.3 installation, I found that the same issue shows
up
> with
> >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2 installations
on the
> >> Dell.
> >>
> >> It seems as if the resolution from the previous MET-help ticket
#95364
> >> (adding LIBPNG to LDFLAGS in the compilation script) maybe didn't
> prevent
> >> this problem from resurfacing in MET-9.1.1 and later
installations? It
> >> seems that we should take a look to see if we can find a fix
before
> >> MET-10.0 is released and installed on WCOSS.
> >>
> >> Thanks,
> >> Logan
> >>
> >> --
> >> Logan C. Dawson, Ph.D. (he/him/his)
> >> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> >> NOAA/NWS/NCEP/EMC
> >> (301) 683-3944
> >>
> >>
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Mon May 03 11:05:48 2021
Hi Logan.
Thank you for sending the command you're using and for moving the
input
file to a location that wouldn't be scrubbed. I wish I better
understood
this problem. It is a bit confusing still since version 1.5.13 of the
png
library doesn't seem to be available on WCOSS.
I agree that testing the change with both is the best thing to do. I
have
done that and have tested with both and see that the warning is now
gone.
I did this testing using the following:
module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> module load met/10.0.0-beta5
and
module use
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> module load met/9.1.3_test
Logan, do you notice any problems with the output? I'm trying to
figure
out if the warning has any ill effects.
Thank you again for bringing this to our attention. It is much
appreciated.
Julie
On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
> Hi Julie,
>
> Here's the command that produced the output I previously sent:
>
> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane -v 5
-method
> BUDGET -width 2 -field 'name="MergedReflectivityQCComposite";
> level="Z500";' -name MergedReflectivityQCComposite
>
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
>
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> /gpfs/dell2/ptmp/Logan.Dawson/
> MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
>
> I moved the input file to a location that won't be scrubbed while
we're
> testing.
>
> If it's not too much trouble, it might be good to test this with an
update
> to the met-10.0.0-beta5 installation and a good install of met-
9.1.3. That
> way, if NCO is able to update the version installed in nwprod, we'll
have
> confirmation that the compilation update will fix the problem with
that
> version. I assume it will, but it wouldn't hurt to confirm before
going
> back to NCO.
>
> Thanks very much for checking into this!
> Logan
>
> On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> > Unfortunately, I am wrong on all accounts.
> >
> > The script did not contain the addition of "-L${LIB_LIBPNG}" in
rpath,
> and
> > the value of LIB_LIBPNG is set correctly.
> >
> > The script currently contains:
> >
> > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} -L${MET_HDF5}/lib"
> >
> >
> > but should contain:
> >
> > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-L${LIB_LIBPNG}*
> > > -L${MET_HDF5}/lib"
> >
> >
> > I have added Mallory to this email to see if there is anything we
can do
> to
> > get this fixed for the already installed version of met-9.1.3. If
not,
> we
> > will be releasing met-10.0.0 soon and hopefully the install
process will
> go
> > more quickly since we recently went through it for 9.1.3.
> >
> > Mallory, I have made some other additions and updates to the
> > compile_MET_all.sh script, which I will include for the release of
> 10.0.0.
> >
> > Logan and Mallory, I would also like to test this addition to the
> > compile_MET_all.sh script to make sure it solves this problem
(again). I
> > plan to update my latest install of met-10.0.0-beta5 to test, but
I can
> > also test to get a good install of 9.1.3 if you would like, Logan.
> Please
> > let me know what you would prefer.
> >
> > Please let me know if you have any questions. My apologies for
this
> > trouble!
> >
> > Julie
> >
> >
> > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik
<jpresto at ucar.edu>
> > wrote:
> >
> > > Hi Logan.
> > >
> > > Thank you for bringing this problem to our attention, for
reminding me
> of
> > > this past issue, and for doing the research and adding "see MET-
help
> > ticket
> > > #95364" in the email. That was very helpful information, and I
> > appreciate
> > > your effort!
> > >
> > > Can you please send me the command line you are using and the
location
> of
> > > your data so I can test with the version that Mallory put
together for
> > nco
> > > before the official release of met-9.1.3?
> > >
> > > The script we provided should already put the value for LIBPNG
in
> > LDFLAGS,
> > > so I suspect the value may not be set correctly, but I will look
into
> > > that.
> > >
> > > I'll follow up once I'm able to run your test case with the
other
> version
> > > and compare to see what's going on.
> > >
> > > We definitely want to get this fixed before the installation of
> > > met-10.0.0, so thanks for staying on top of this and alerting us
now.
> > >
> > > Julie
> > >
> > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > >> Queue: met_help
> > >> Subject: Libpng warning with MET-9.1.3 in production on
WCOSS
> Dell
> > >> Owner: Nobody
> > >> Requestors: logan.dawson at noaa.gov
> > >> Status: new
> > >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> >
> > >>
> > >>
> > >> Hello,
> > >>
> > >> The MET-9.1.3 installed into operations on WCOSS Dell (
> > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng
warning
> > when
> > >> trying to read MRMS files. Here's an example of the output from
> > >> regrid_data_plane showing that MET reads all missing data from
a MRMS
> > >> file:
> > >>
> > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match
for
> > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field 1
of
> GRIB2
> > >> file
> > >>
> > >>
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > >>
> > >> *libpng warning: Application built with libpng-1.2.59 but
running with
> > >> 1.5.13*
> > >>
> > >> DEBUG 4:
> > >>
> > >> DEBUG 4: Data plane information:
> > >>
> > >> DEBUG 4: plane min: -999
> > >>
> > >> *DEBUG 4: plane max: -999*
> > >>
> > >> DEBUG 4: scan mode: 0
> > >>
> > >> DEBUG 4: valid time: 20210426_000034
> > >>
> > >> DEBUG 4: lead time: 000000
> > >>
> > >> DEBUG 4: init time: 20210426_000034
> > >>
> > >> DEBUG 4: bitmap flag: 255
> > >>
> > >> DEBUG 4:
> > >>
> > >> *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> > >> level="Z500";) is -999 to -999.*
> > >>
> > >> *DEBUG 2: Range of regridded data
> (name="MergedReflectivityQCComposite";
> > >> level="Z500";) is -999 to -999.*
> > >>
> > >> DEBUG 1: Writing output file:
> > >> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > >> 20210426-000000_g227.nc
> > >>
> > >> This same issue previously occurred when MET-9.0.1 was
installed on
> the
> > >> Dells in May/June 2020, and Julie found a resolution for the
issue
> > before
> > >> the MET-9.0.2 installation (see MET-help ticket #95364). The
issue was
> > >> resolved with MET installs in the noscrub/emc.metplus space up
through
> > >> MET-9.1. However, once this libpng warning issue was identified
in the
> > >> nwprod/met.v9.1.3 installation, I found that the same issue
shows up
> > with
> > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
installations on
> the
> > >> Dell.
> > >>
> > >> It seems as if the resolution from the previous MET-help ticket
#95364
> > >> (adding LIBPNG to LDFLAGS in the compilation script) maybe
didn't
> > prevent
> > >> this problem from resurfacing in MET-9.1.1 and later
installations? It
> > >> seems that we should take a look to see if we can find a fix
before
> > >> MET-10.0 is released and installed on WCOSS.
> > >>
> > >> Thanks,
> > >> Logan
> > >>
> > >> --
> > >> Logan C. Dawson, Ph.D. (he/him/his)
> > >> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > >> NOAA/NWS/NCEP/EMC
> > >> (301) 683-3944
> > >>
> > >>
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> >
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Logan Dawson - NOAA Affiliate
Time: Mon May 03 11:44:12 2021
Hi Julie,
I just tested the command again with met/10.0.0-beta5 and
met/9.1.3_test
and see that the warning is now gone in both instances. I also see
that the
data plane information in the output is the same in both instances and
matches the output produced from the previously successful runs with
met/9.1:
DEBUG 4: Data plane information:
DEBUG 4: plane min: -999
*DEBUG 4: plane max: 59*
DEBUG 4: scan mode: 0
DEBUG 4: valid time: 20210426_000034
DEBUG 4: lead time: 000000
DEBUG 4: init time: 20210426_000034
DEBUG 4: bitmap flag: 255
DEBUG 4:
*DEBUG 2: Range of input data (name="MergedReflectivityQCComposite";
level="Z500";) is -999 to 59.*
*DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
level="Z500";) is -999 to 51.8493.*
Also, as one final sanity check, I read the output files in with
python and
can confirm the output data are the same between the two test runs and
the
met/9.1 run. Thanks for installing both test capabilities so we can
ensure
that the compilation fix works for 9.1.3 and 10.0.0!
Also, we're planning to add this specific use case to our WCOSS test
suite
so that we can catch any future issues ahead of time. I'm unsure of
why the
warning is produced when MRMS data are read on the Dells, but clearly
it's
something we should keep an eye on with future installations and new
machines.
Thanks,
Logan
On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi Logan.
>
> Thank you for sending the command you're using and for moving the
input
> file to a location that wouldn't be scrubbed. I wish I better
understood
> this problem. It is a bit confusing still since version 1.5.13 of
the png
> library doesn't seem to be available on WCOSS.
>
> I agree that testing the change with both is the best thing to do.
I have
> done that and have tested with both and see that the warning is now
gone.
> I did this testing using the following:
>
> module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > module load met/10.0.0-beta5
>
>
> and
>
> module use
> >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > module load met/9.1.3_test
>
>
> Logan, do you notice any problems with the output? I'm trying to
figure
> out if the warning has any ill effects.
>
> Thank you again for bringing this to our attention. It is much
> appreciated.
>
> Julie
>
> On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> >
> > Hi Julie,
> >
> > Here's the command that produced the output I previously sent:
> >
> > /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane -v 5
-method
> > BUDGET -width 2 -field 'name="MergedReflectivityQCComposite";
> > level="Z500";' -name MergedReflectivityQCComposite
> >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > /gpfs/dell2/ptmp/Logan.Dawson/
> > MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
> >
> > I moved the input file to a location that won't be scrubbed while
we're
> > testing.
> >
> > If it's not too much trouble, it might be good to test this with
an
> update
> > to the met-10.0.0-beta5 installation and a good install of met-
9.1.3.
> That
> > way, if NCO is able to update the version installed in nwprod,
we'll have
> > confirmation that the compilation update will fix the problem with
that
> > version. I assume it will, but it wouldn't hurt to confirm before
going
> > back to NCO.
> >
> > Thanks very much for checking into this!
> > Logan
> >
> > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > > Unfortunately, I am wrong on all accounts.
> > >
> > > The script did not contain the addition of "-L${LIB_LIBPNG}" in
rpath,
> > and
> > > the value of LIB_LIBPNG is set correctly.
> > >
> > > The script currently contains:
> > >
> > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} -L${MET_HDF5}/lib"
> > >
> > >
> > > but should contain:
> > >
> > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-L${LIB_LIBPNG}*
> > > > -L${MET_HDF5}/lib"
> > >
> > >
> > > I have added Mallory to this email to see if there is anything
we can
> do
> > to
> > > get this fixed for the already installed version of met-9.1.3.
If not,
> > we
> > > will be releasing met-10.0.0 soon and hopefully the install
process
> will
> > go
> > > more quickly since we recently went through it for 9.1.3.
> > >
> > > Mallory, I have made some other additions and updates to the
> > > compile_MET_all.sh script, which I will include for the release
of
> > 10.0.0.
> > >
> > > Logan and Mallory, I would also like to test this addition to
the
> > > compile_MET_all.sh script to make sure it solves this problem
> (again). I
> > > plan to update my latest install of met-10.0.0-beta5 to test,
but I can
> > > also test to get a good install of 9.1.3 if you would like,
Logan.
> > Please
> > > let me know what you would prefer.
> > >
> > > Please let me know if you have any questions. My apologies for
this
> > > trouble!
> > >
> > > Julie
> > >
> > >
> > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik
<jpresto at ucar.edu>
> > > wrote:
> > >
> > > > Hi Logan.
> > > >
> > > > Thank you for bringing this problem to our attention, for
reminding
> me
> > of
> > > > this past issue, and for doing the research and adding "see
MET-help
> > > ticket
> > > > #95364" in the email. That was very helpful information, and
I
> > > appreciate
> > > > your effort!
> > > >
> > > > Can you please send me the command line you are using and the
> location
> > of
> > > > your data so I can test with the version that Mallory put
together
> for
> > > nco
> > > > before the official release of met-9.1.3?
> > > >
> > > > The script we provided should already put the value for LIBPNG
in
> > > LDFLAGS,
> > > > so I suspect the value may not be set correctly, but I will
look into
> > > > that.
> > > >
> > > > I'll follow up once I'm able to run your test case with the
other
> > version
> > > > and compare to see what's going on.
> > > >
> > > > We definitely want to get this fixed before the installation
of
> > > > met-10.0.0, so thanks for staying on top of this and alerting
us now.
> > > >
> > > > Julie
> > > >
> > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> > > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > > >> Queue: met_help
> > > >> Subject: Libpng warning with MET-9.1.3 in production on
WCOSS
> > Dell
> > > >> Owner: Nobody
> > > >> Requestors: logan.dawson at noaa.gov
> > > >> Status: new
> > > >> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > >
> > > >>
> > > >>
> > > >> Hello,
> > > >>
> > > >> The MET-9.1.3 installed into operations on WCOSS Dell (
> > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a libpng
> warning
> > > when
> > > >> trying to read MRMS files. Here's an example of the output
from
> > > >> regrid_data_plane showing that MET reads all missing data
from a
> MRMS
> > > >> file:
> > > >>
> > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match
for
> > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1 field
1 of
> > GRIB2
> > > >> file
> > > >>
> > > >>
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > >>
> > > >> *libpng warning: Application built with libpng-1.2.59 but
running
> with
> > > >> 1.5.13*
> > > >>
> > > >> DEBUG 4:
> > > >>
> > > >> DEBUG 4: Data plane information:
> > > >>
> > > >> DEBUG 4: plane min: -999
> > > >>
> > > >> *DEBUG 4: plane max: -999*
> > > >>
> > > >> DEBUG 4: scan mode: 0
> > > >>
> > > >> DEBUG 4: valid time: 20210426_000034
> > > >>
> > > >> DEBUG 4: lead time: 000000
> > > >>
> > > >> DEBUG 4: init time: 20210426_000034
> > > >>
> > > >> DEBUG 4: bitmap flag: 255
> > > >>
> > > >> DEBUG 4:
> > > >>
> > > >> *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> > > >> level="Z500";) is -999 to -999.*
> > > >>
> > > >> *DEBUG 2: Range of regridded data
> > (name="MergedReflectivityQCComposite";
> > > >> level="Z500";) is -999 to -999.*
> > > >>
> > > >> DEBUG 1: Writing output file:
> > > >> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > >> 20210426-000000_g227.nc
> > > >>
> > > >> This same issue previously occurred when MET-9.0.1 was
installed on
> > the
> > > >> Dells in May/June 2020, and Julie found a resolution for the
issue
> > > before
> > > >> the MET-9.0.2 installation (see MET-help ticket #95364). The
issue
> was
> > > >> resolved with MET installs in the noscrub/emc.metplus space
up
> through
> > > >> MET-9.1. However, once this libpng warning issue was
identified in
> the
> > > >> nwprod/met.v9.1.3 installation, I found that the same issue
shows up
> > > with
> > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
installations on
> > the
> > > >> Dell.
> > > >>
> > > >> It seems as if the resolution from the previous MET-help
ticket
> #95364
> > > >> (adding LIBPNG to LDFLAGS in the compilation script) maybe
didn't
> > > prevent
> > > >> this problem from resurfacing in MET-9.1.1 and later
installations?
> It
> > > >> seems that we should take a look to see if we can find a fix
before
> > > >> MET-10.0 is released and installed on WCOSS.
> > > >>
> > > >> Thanks,
> > > >> Logan
> > > >>
> > > >> --
> > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > >> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > >> NOAA/NWS/NCEP/EMC
> > > >> (301) 683-3944
> > > >>
> > > >>
> > > >
> > > > --
> > > > Julie Prestopnik (she/her)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > Logan C. Dawson, Ph.D. (he/him/his)
> > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > NOAA/NWS/NCEP/EMC
> > (301) 683-3944
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Mon May 03 12:07:18 2021
Hi Logan. Thank you for all of this information. I had also planned
to add
this specific use case to the WCOSS test suite, but it sounds like you
and
Mallory are taking care of that. Please let me know if either of you
would
like me to do anything.
It is great that the data plane information in the output is the same
in
both instances and matches the output produced from the previously
successful runs with met/9.1. Can you please tell me if the output
file
has any differences when using the 9.1.3_test module and the 9.1.3
operational version?
Thanks!
Julie
On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
> Hi Julie,
>
> I just tested the command again with met/10.0.0-beta5 and
met/9.1.3_test
> and see that the warning is now gone in both instances. I also see
that the
> data plane information in the output is the same in both instances
and
> matches the output produced from the previously successful runs with
> met/9.1:
>
> DEBUG 4: Data plane information:
>
> DEBUG 4: plane min: -999
>
> *DEBUG 4: plane max: 59*
>
> DEBUG 4: scan mode: 0
>
> DEBUG 4: valid time: 20210426_000034
>
> DEBUG 4: lead time: 000000
>
> DEBUG 4: init time: 20210426_000034
>
> DEBUG 4: bitmap flag: 255
>
> DEBUG 4:
>
> *DEBUG 2: Range of input data (name="MergedReflectivityQCComposite";
> level="Z500";) is -999 to 59.*
>
> *DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
> level="Z500";) is -999 to 51.8493.*
>
> Also, as one final sanity check, I read the output files in with
python and
> can confirm the output data are the same between the two test runs
and the
> met/9.1 run. Thanks for installing both test capabilities so we can
ensure
> that the compilation fix works for 9.1.3 and 10.0.0!
>
> Also, we're planning to add this specific use case to our WCOSS test
suite
> so that we can catch any future issues ahead of time. I'm unsure of
why the
> warning is produced when MRMS data are read on the Dells, but
clearly it's
> something we should keep an eye on with future installations and new
> machines.
>
> Thanks,
> Logan
>
> On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> > Hi Logan.
> >
> > Thank you for sending the command you're using and for moving the
input
> > file to a location that wouldn't be scrubbed. I wish I better
understood
> > this problem. It is a bit confusing still since version 1.5.13 of
the
> png
> > library doesn't seem to be available on WCOSS.
> >
> > I agree that testing the change with both is the best thing to do.
I
> have
> > done that and have tested with both and see that the warning is
now gone.
> > I did this testing using the following:
> >
> > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > module load met/10.0.0-beta5
> >
> >
> > and
> >
> > module use
> > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > module load met/9.1.3_test
> >
> >
> > Logan, do you notice any problems with the output? I'm trying to
figure
> > out if the warning has any ill effects.
> >
> > Thank you again for bringing this to our attention. It is much
> > appreciated.
> >
> > Julie
> >
> > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > >
> > > Hi Julie,
> > >
> > > Here's the command that produced the output I previously sent:
> > >
> > > /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane -v
5
> -method
> > > BUDGET -width 2 -field 'name="MergedReflectivityQCComposite";
> > > level="Z500";' -name MergedReflectivityQCComposite
> > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
> > >
> > > I moved the input file to a location that won't be scrubbed
while we're
> > > testing.
> > >
> > > If it's not too much trouble, it might be good to test this with
an
> > update
> > > to the met-10.0.0-beta5 installation and a good install of met-
9.1.3.
> > That
> > > way, if NCO is able to update the version installed in nwprod,
we'll
> have
> > > confirmation that the compilation update will fix the problem
with that
> > > version. I assume it will, but it wouldn't hurt to confirm
before going
> > > back to NCO.
> > >
> > > Thanks very much for checking into this!
> > > Logan
> > >
> > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > > Unfortunately, I am wrong on all accounts.
> > > >
> > > > The script did not contain the addition of "-L${LIB_LIBPNG}"
in
> rpath,
> > > and
> > > > the value of LIB_LIBPNG is set correctly.
> > > >
> > > > The script currently contains:
> > > >
> > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
-L${MET_HDF5}/lib"
> > > >
> > > >
> > > > but should contain:
> > > >
> > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-L${LIB_LIBPNG}*
> > > > > -L${MET_HDF5}/lib"
> > > >
> > > >
> > > > I have added Mallory to this email to see if there is anything
we can
> > do
> > > to
> > > > get this fixed for the already installed version of met-9.1.3.
If
> not,
> > > we
> > > > will be releasing met-10.0.0 soon and hopefully the install
process
> > will
> > > go
> > > > more quickly since we recently went through it for 9.1.3.
> > > >
> > > > Mallory, I have made some other additions and updates to the
> > > > compile_MET_all.sh script, which I will include for the
release of
> > > 10.0.0.
> > > >
> > > > Logan and Mallory, I would also like to test this addition to
the
> > > > compile_MET_all.sh script to make sure it solves this problem
> > (again). I
> > > > plan to update my latest install of met-10.0.0-beta5 to test,
but I
> can
> > > > also test to get a good install of 9.1.3 if you would like,
Logan.
> > > Please
> > > > let me know what you would prefer.
> > > >
> > > > Please let me know if you have any questions. My apologies for
this
> > > > trouble!
> > > >
> > > > Julie
> > > >
> > > >
> > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik
<jpresto at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Logan.
> > > > >
> > > > > Thank you for bringing this problem to our attention, for
reminding
> > me
> > > of
> > > > > this past issue, and for doing the research and adding "see
> MET-help
> > > > ticket
> > > > > #95364" in the email. That was very helpful information,
and I
> > > > appreciate
> > > > > your effort!
> > > > >
> > > > > Can you please send me the command line you are using and
the
> > location
> > > of
> > > > > your data so I can test with the version that Mallory put
together
> > for
> > > > nco
> > > > > before the official release of met-9.1.3?
> > > > >
> > > > > The script we provided should already put the value for
LIBPNG in
> > > > LDFLAGS,
> > > > > so I suspect the value may not be set correctly, but I will
look
> into
> > > > > that.
> > > > >
> > > > > I'll follow up once I'm able to run your test case with the
other
> > > version
> > > > > and compare to see what's going on.
> > > > >
> > > > > We definitely want to get this fixed before the installation
of
> > > > > met-10.0.0, so thanks for staying on top of this and
alerting us
> now.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA
Affiliate via
> > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> > > > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > > > >> Queue: met_help
> > > > >> Subject: Libpng warning with MET-9.1.3 in production
on WCOSS
> > > Dell
> > > > >> Owner: Nobody
> > > > >> Requestors: logan.dawson at noaa.gov
> > > > >> Status: new
> > > > >> Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > >
> > > > >>
> > > > >>
> > > > >> Hello,
> > > > >>
> > > > >> The MET-9.1.3 installed into operations on WCOSS Dell (
> > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a
libpng
> > warning
> > > > when
> > > > >> trying to read MRMS files. Here's an example of the output
from
> > > > >> regrid_data_plane showing that MET reads all missing data
from a
> > MRMS
> > > > >> file:
> > > > >>
> > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match for
> > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1
field 1 of
> > > GRIB2
> > > > >> file
> > > > >>
> > > > >>
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > >>
> > > > >> *libpng warning: Application built with libpng-1.2.59 but
running
> > with
> > > > >> 1.5.13*
> > > > >>
> > > > >> DEBUG 4:
> > > > >>
> > > > >> DEBUG 4: Data plane information:
> > > > >>
> > > > >> DEBUG 4: plane min: -999
> > > > >>
> > > > >> *DEBUG 4: plane max: -999*
> > > > >>
> > > > >> DEBUG 4: scan mode: 0
> > > > >>
> > > > >> DEBUG 4: valid time: 20210426_000034
> > > > >>
> > > > >> DEBUG 4: lead time: 000000
> > > > >>
> > > > >> DEBUG 4: init time: 20210426_000034
> > > > >>
> > > > >> DEBUG 4: bitmap flag: 255
> > > > >>
> > > > >> DEBUG 4:
> > > > >>
> > > > >> *DEBUG 2: Range of input data
> (name="MergedReflectivityQCComposite";
> > > > >> level="Z500";) is -999 to -999.*
> > > > >>
> > > > >> *DEBUG 2: Range of regridded data
> > > (name="MergedReflectivityQCComposite";
> > > > >> level="Z500";) is -999 to -999.*
> > > > >>
> > > > >> DEBUG 1: Writing output file:
> > > > >>
/gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > >> 20210426-000000_g227.nc
> > > > >>
> > > > >> This same issue previously occurred when MET-9.0.1 was
installed
> on
> > > the
> > > > >> Dells in May/June 2020, and Julie found a resolution for
the issue
> > > > before
> > > > >> the MET-9.0.2 installation (see MET-help ticket #95364).
The issue
> > was
> > > > >> resolved with MET installs in the noscrub/emc.metplus space
up
> > through
> > > > >> MET-9.1. However, once this libpng warning issue was
identified in
> > the
> > > > >> nwprod/met.v9.1.3 installation, I found that the same issue
shows
> up
> > > > with
> > > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
installations
> on
> > > the
> > > > >> Dell.
> > > > >>
> > > > >> It seems as if the resolution from the previous MET-help
ticket
> > #95364
> > > > >> (adding LIBPNG to LDFLAGS in the compilation script) maybe
didn't
> > > > prevent
> > > > >> this problem from resurfacing in MET-9.1.1 and later
> installations?
> > It
> > > > >> seems that we should take a look to see if we can find a
fix
> before
> > > > >> MET-10.0 is released and installed on WCOSS.
> > > > >>
> > > > >> Thanks,
> > > > >> Logan
> > > > >>
> > > > >> --
> > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > >> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > >> NOAA/NWS/NCEP/EMC
> > > > >> (301) 683-3944
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > Julie Prestopnik (she/her)
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik (she/her)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > Logan C. Dawson, Ph.D. (he/him/his)
> > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > NOAA/NWS/NCEP/EMC
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Logan Dawson - NOAA Affiliate
Time: Mon May 03 13:01:05 2021
Hi Julie,
All three output files are all the same. I did a ncdump -h on the
output
files, and I plotted the data as well. The data values and plotted
output
are all the same as well, so I'm comfortable saying the two tests
match the
expected output from met/9.1.
Thanks,
Logan
On Mon, May 3, 2021 at 1:07 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi Logan. Thank you for all of this information. I had also planned
to add
> this specific use case to the WCOSS test suite, but it sounds like
you and
> Mallory are taking care of that. Please let me know if either of
you would
> like me to do anything.
>
> It is great that the data plane information in the output is the
same in
> both instances and matches the output produced from the previously
> successful runs with met/9.1. Can you please tell me if the output
file
> has any differences when using the 9.1.3_test module and the 9.1.3
> operational version?
>
> Thanks!
>
> Julie
>
>
>
> On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> >
> > Hi Julie,
> >
> > I just tested the command again with met/10.0.0-beta5 and
met/9.1.3_test
> > and see that the warning is now gone in both instances. I also see
that
> the
> > data plane information in the output is the same in both instances
and
> > matches the output produced from the previously successful runs
with
> > met/9.1:
> >
> > DEBUG 4: Data plane information:
> >
> > DEBUG 4: plane min: -999
> >
> > *DEBUG 4: plane max: 59*
> >
> > DEBUG 4: scan mode: 0
> >
> > DEBUG 4: valid time: 20210426_000034
> >
> > DEBUG 4: lead time: 000000
> >
> > DEBUG 4: init time: 20210426_000034
> >
> > DEBUG 4: bitmap flag: 255
> >
> > DEBUG 4:
> >
> > *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> > level="Z500";) is -999 to 59.*
> >
> > *DEBUG 2: Range of regridded data
(name="MergedReflectivityQCComposite";
> > level="Z500";) is -999 to 51.8493.*
> >
> > Also, as one final sanity check, I read the output files in with
python
> and
> > can confirm the output data are the same between the two test runs
and
> the
> > met/9.1 run. Thanks for installing both test capabilities so we
can
> ensure
> > that the compilation fix works for 9.1.3 and 10.0.0!
> >
> > Also, we're planning to add this specific use case to our WCOSS
test
> suite
> > so that we can catch any future issues ahead of time. I'm unsure
of why
> the
> > warning is produced when MRMS data are read on the Dells, but
clearly
> it's
> > something we should keep an eye on with future installations and
new
> > machines.
> >
> > Thanks,
> > Logan
> >
> > On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > > Hi Logan.
> > >
> > > Thank you for sending the command you're using and for moving
the input
> > > file to a location that wouldn't be scrubbed. I wish I better
> understood
> > > this problem. It is a bit confusing still since version 1.5.13
of the
> > png
> > > library doesn't seem to be available on WCOSS.
> > >
> > > I agree that testing the change with both is the best thing to
do. I
> > have
> > > done that and have tested with both and see that the warning is
now
> gone.
> > > I did this testing using the following:
> > >
> > > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > module load met/10.0.0-beta5
> > >
> > >
> > > and
> > >
> > > module use
> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > > module load met/9.1.3_test
> > >
> > >
> > > Logan, do you notice any problems with the output? I'm trying
to
> figure
> > > out if the warning has any ill effects.
> > >
> > > Thank you again for bringing this to our attention. It is much
> > > appreciated.
> > >
> > > Julie
> > >
> > > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
>
> > > >
> > > > Hi Julie,
> > > >
> > > > Here's the command that produced the output I previously sent:
> > > >
> > > > /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane
-v 5
> > -method
> > > > BUDGET -width 2 -field 'name="MergedReflectivityQCComposite";
> > > > level="Z500";' -name MergedReflectivityQCComposite
> > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > > MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
> > > >
> > > > I moved the input file to a location that won't be scrubbed
while
> we're
> > > > testing.
> > > >
> > > > If it's not too much trouble, it might be good to test this
with an
> > > update
> > > > to the met-10.0.0-beta5 installation and a good install of
met-9.1.3.
> > > That
> > > > way, if NCO is able to update the version installed in nwprod,
we'll
> > have
> > > > confirmation that the compilation update will fix the problem
with
> that
> > > > version. I assume it will, but it wouldn't hurt to confirm
before
> going
> > > > back to NCO.
> > > >
> > > > Thanks very much for checking into this!
> > > > Logan
> > > >
> > > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > > Unfortunately, I am wrong on all accounts.
> > > > >
> > > > > The script did not contain the addition of "-L${LIB_LIBPNG}"
in
> > rpath,
> > > > and
> > > > > the value of LIB_LIBPNG is set correctly.
> > > > >
> > > > > The script currently contains:
> > > > >
> > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
-L${MET_HDF5}/lib"
> > > > >
> > > > >
> > > > > but should contain:
> > > > >
> > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-
L${LIB_LIBPNG}*
> > > > > > -L${MET_HDF5}/lib"
> > > > >
> > > > >
> > > > > I have added Mallory to this email to see if there is
anything we
> can
> > > do
> > > > to
> > > > > get this fixed for the already installed version of met-
9.1.3. If
> > not,
> > > > we
> > > > > will be releasing met-10.0.0 soon and hopefully the install
process
> > > will
> > > > go
> > > > > more quickly since we recently went through it for 9.1.3.
> > > > >
> > > > > Mallory, I have made some other additions and updates to the
> > > > > compile_MET_all.sh script, which I will include for the
release of
> > > > 10.0.0.
> > > > >
> > > > > Logan and Mallory, I would also like to test this addition
to the
> > > > > compile_MET_all.sh script to make sure it solves this
problem
> > > (again). I
> > > > > plan to update my latest install of met-10.0.0-beta5 to
test, but I
> > can
> > > > > also test to get a good install of 9.1.3 if you would like,
Logan.
> > > > Please
> > > > > let me know what you would prefer.
> > > > >
> > > > > Please let me know if you have any questions. My apologies
for this
> > > > > trouble!
> > > > >
> > > > > Julie
> > > > >
> > > > >
> > > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <
> jpresto at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Logan.
> > > > > >
> > > > > > Thank you for bringing this problem to our attention, for
> reminding
> > > me
> > > > of
> > > > > > this past issue, and for doing the research and adding
"see
> > MET-help
> > > > > ticket
> > > > > > #95364" in the email. That was very helpful information,
and I
> > > > > appreciate
> > > > > > your effort!
> > > > > >
> > > > > > Can you please send me the command line you are using and
the
> > > location
> > > > of
> > > > > > your data so I can test with the version that Mallory put
> together
> > > for
> > > > > nco
> > > > > > before the official release of met-9.1.3?
> > > > > >
> > > > > > The script we provided should already put the value for
LIBPNG in
> > > > > LDFLAGS,
> > > > > > so I suspect the value may not be set correctly, but I
will look
> > into
> > > > > > that.
> > > > > >
> > > > > > I'll follow up once I'm able to run your test case with
the other
> > > > version
> > > > > > and compare to see what's going on.
> > > > > >
> > > > > > We definitely want to get this fixed before the
installation of
> > > > > > met-10.0.0, so thanks for staying on top of this and
alerting us
> > now.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA
Affiliate
> via
> > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > >>
> > > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> > > > > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > >> Queue: met_help
> > > > > >> Subject: Libpng warning with MET-9.1.3 in production
on
> WCOSS
> > > > Dell
> > > > > >> Owner: Nobody
> > > > > >> Requestors: logan.dawson at noaa.gov
> > > > > >> Status: new
> > > > > >> Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > > >
> > > > > >>
> > > > > >>
> > > > > >> Hello,
> > > > > >>
> > > > > >> The MET-9.1.3 installed into operations on WCOSS Dell (
> > > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a
libpng
> > > warning
> > > > > when
> > > > > >> trying to read MRMS files. Here's an example of the
output from
> > > > > >> regrid_data_plane showing that MET reads all missing data
from a
> > > MRMS
> > > > > >> file:
> > > > > >>
> > > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match for
> > > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1
field 1
> of
> > > > GRIB2
> > > > > >> file
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > > >>
> > > > > >> *libpng warning: Application built with libpng-1.2.59 but
> running
> > > with
> > > > > >> 1.5.13*
> > > > > >>
> > > > > >> DEBUG 4:
> > > > > >>
> > > > > >> DEBUG 4: Data plane information:
> > > > > >>
> > > > > >> DEBUG 4: plane min: -999
> > > > > >>
> > > > > >> *DEBUG 4: plane max: -999*
> > > > > >>
> > > > > >> DEBUG 4: scan mode: 0
> > > > > >>
> > > > > >> DEBUG 4: valid time: 20210426_000034
> > > > > >>
> > > > > >> DEBUG 4: lead time: 000000
> > > > > >>
> > > > > >> DEBUG 4: init time: 20210426_000034
> > > > > >>
> > > > > >> DEBUG 4: bitmap flag: 255
> > > > > >>
> > > > > >> DEBUG 4:
> > > > > >>
> > > > > >> *DEBUG 2: Range of input data
> > (name="MergedReflectivityQCComposite";
> > > > > >> level="Z500";) is -999 to -999.*
> > > > > >>
> > > > > >> *DEBUG 2: Range of regridded data
> > > > (name="MergedReflectivityQCComposite";
> > > > > >> level="Z500";) is -999 to -999.*
> > > > > >>
> > > > > >> DEBUG 1: Writing output file:
> > > > > >>
/gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > > >> 20210426-000000_g227.nc
> > > > > >>
> > > > > >> This same issue previously occurred when MET-9.0.1 was
installed
> > on
> > > > the
> > > > > >> Dells in May/June 2020, and Julie found a resolution for
the
> issue
> > > > > before
> > > > > >> the MET-9.0.2 installation (see MET-help ticket #95364).
The
> issue
> > > was
> > > > > >> resolved with MET installs in the noscrub/emc.metplus
space up
> > > through
> > > > > >> MET-9.1. However, once this libpng warning issue was
identified
> in
> > > the
> > > > > >> nwprod/met.v9.1.3 installation, I found that the same
issue
> shows
> > up
> > > > > with
> > > > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
> installations
> > on
> > > > the
> > > > > >> Dell.
> > > > > >>
> > > > > >> It seems as if the resolution from the previous MET-help
ticket
> > > #95364
> > > > > >> (adding LIBPNG to LDFLAGS in the compilation script)
maybe
> didn't
> > > > > prevent
> > > > > >> this problem from resurfacing in MET-9.1.1 and later
> > installations?
> > > It
> > > > > >> seems that we should take a look to see if we can find a
fix
> > before
> > > > > >> MET-10.0 is released and installed on WCOSS.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Logan
> > > > > >>
> > > > > >> --
> > > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > > >> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > > >> NOAA/NWS/NCEP/EMC
> > > > > >> (301) 683-3944
> > > > > >>
> > > > > >>
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik (she/her)
> > > > > > Software Engineer
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik (she/her)
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > > --
> > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > NOAA/NWS/NCEP/EMC
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > Logan C. Dawson, Ph.D. (he/him/his)
> > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > NOAA/NWS/NCEP/EMC
> > (301) 683-3944
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Mon May 03 18:49:33 2021
So is it correct to say that if 9.1.3 is not updated in the official
installation, it is ok because the output is the same, only a warning
is
thrown?
Julie
On Mon, May 3, 2021 at 1:01 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
> Hi Julie,
>
> All three output files are all the same. I did a ncdump -h on the
output
> files, and I plotted the data as well. The data values and plotted
output
> are all the same as well, so I'm comfortable saying the two tests
match the
> expected output from met/9.1.
>
> Thanks,
> Logan
>
> On Mon, May 3, 2021 at 1:07 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan. Thank you for all of this information. I had also
planned to
> add
> > this specific use case to the WCOSS test suite, but it sounds like
you
> and
> > Mallory are taking care of that. Please let me know if either of
you
> would
> > like me to do anything.
> >
> > It is great that the data plane information in the output is the
same in
> > both instances and matches the output produced from the previously
> > successful runs with met/9.1. Can you please tell me if the
output file
> > has any differences when using the 9.1.3_test module and the 9.1.3
> > operational version?
> >
> > Thanks!
> >
> > Julie
> >
> >
> >
> > On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > >
> > > Hi Julie,
> > >
> > > I just tested the command again with met/10.0.0-beta5 and
> met/9.1.3_test
> > > and see that the warning is now gone in both instances. I also
see that
> > the
> > > data plane information in the output is the same in both
instances and
> > > matches the output produced from the previously successful runs
with
> > > met/9.1:
> > >
> > > DEBUG 4: Data plane information:
> > >
> > > DEBUG 4: plane min: -999
> > >
> > > *DEBUG 4: plane max: 59*
> > >
> > > DEBUG 4: scan mode: 0
> > >
> > > DEBUG 4: valid time: 20210426_000034
> > >
> > > DEBUG 4: lead time: 000000
> > >
> > > DEBUG 4: init time: 20210426_000034
> > >
> > > DEBUG 4: bitmap flag: 255
> > >
> > > DEBUG 4:
> > >
> > > *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> > > level="Z500";) is -999 to 59.*
> > >
> > > *DEBUG 2: Range of regridded data
> (name="MergedReflectivityQCComposite";
> > > level="Z500";) is -999 to 51.8493.*
> > >
> > > Also, as one final sanity check, I read the output files in with
python
> > and
> > > can confirm the output data are the same between the two test
runs and
> > the
> > > met/9.1 run. Thanks for installing both test capabilities so we
can
> > ensure
> > > that the compilation fix works for 9.1.3 and 10.0.0!
> > >
> > > Also, we're planning to add this specific use case to our WCOSS
test
> > suite
> > > so that we can catch any future issues ahead of time. I'm unsure
of why
> > the
> > > warning is produced when MRMS data are read on the Dells, but
clearly
> > it's
> > > something we should keep an eye on with future installations and
new
> > > machines.
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > > Hi Logan.
> > > >
> > > > Thank you for sending the command you're using and for moving
the
> input
> > > > file to a location that wouldn't be scrubbed. I wish I better
> > understood
> > > > this problem. It is a bit confusing still since version
1.5.13 of
> the
> > > png
> > > > library doesn't seem to be available on WCOSS.
> > > >
> > > > I agree that testing the change with both is the best thing to
do. I
> > > have
> > > > done that and have tested with both and see that the warning
is now
> > gone.
> > > > I did this testing using the following:
> > > >
> > > > module use
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > module load met/10.0.0-beta5
> > > >
> > > >
> > > > and
> > > >
> > > > module use
> > > > >
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > > > module load met/9.1.3_test
> > > >
> > > >
> > > > Logan, do you notice any problems with the output? I'm trying
to
> > figure
> > > > out if the warning has any ill effects.
> > > >
> > > > Thank you again for bringing this to our attention. It is
much
> > > > appreciated.
> > > >
> > > > Julie
> > > >
> > > > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA Affiliate
via RT
> <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > > > >
> > > > > Hi Julie,
> > > > >
> > > > > Here's the command that produced the output I previously
sent:
> > > > >
> > > > > /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane
-v 5
> > > -method
> > > > > BUDGET -width 2 -field
'name="MergedReflectivityQCComposite";
> > > > > level="Z500";' -name MergedReflectivityQCComposite
> > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > > > >
> > > > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > > > MergedReflectivityQCComposite_00.50_20210426-000000_g227.nc
> > > > >
> > > > > I moved the input file to a location that won't be scrubbed
while
> > we're
> > > > > testing.
> > > > >
> > > > > If it's not too much trouble, it might be good to test this
with an
> > > > update
> > > > > to the met-10.0.0-beta5 installation and a good install of
> met-9.1.3.
> > > > That
> > > > > way, if NCO is able to update the version installed in
nwprod,
> we'll
> > > have
> > > > > confirmation that the compilation update will fix the
problem with
> > that
> > > > > version. I assume it will, but it wouldn't hurt to confirm
before
> > going
> > > > > back to NCO.
> > > > >
> > > > > Thanks very much for checking into this!
> > > > > Logan
> > > > >
> > > > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Unfortunately, I am wrong on all accounts.
> > > > > >
> > > > > > The script did not contain the addition of "-
L${LIB_LIBPNG}" in
> > > rpath,
> > > > > and
> > > > > > the value of LIB_LIBPNG is set correctly.
> > > > > >
> > > > > > The script currently contains:
> > > > > >
> > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
-L${MET_HDF5}/lib"
> > > > > >
> > > > > >
> > > > > > but should contain:
> > > > > >
> > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-
L${LIB_LIBPNG}*
> > > > > > > -L${MET_HDF5}/lib"
> > > > > >
> > > > > >
> > > > > > I have added Mallory to this email to see if there is
anything we
> > can
> > > > do
> > > > > to
> > > > > > get this fixed for the already installed version of met-
9.1.3.
> If
> > > not,
> > > > > we
> > > > > > will be releasing met-10.0.0 soon and hopefully the
install
> process
> > > > will
> > > > > go
> > > > > > more quickly since we recently went through it for 9.1.3.
> > > > > >
> > > > > > Mallory, I have made some other additions and updates to
the
> > > > > > compile_MET_all.sh script, which I will include for the
release
> of
> > > > > 10.0.0.
> > > > > >
> > > > > > Logan and Mallory, I would also like to test this addition
to the
> > > > > > compile_MET_all.sh script to make sure it solves this
problem
> > > > (again). I
> > > > > > plan to update my latest install of met-10.0.0-beta5 to
test,
> but I
> > > can
> > > > > > also test to get a good install of 9.1.3 if you would
like,
> Logan.
> > > > > Please
> > > > > > let me know what you would prefer.
> > > > > >
> > > > > > Please let me know if you have any questions. My apologies
for
> this
> > > > > > trouble!
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <
> > jpresto at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Logan.
> > > > > > >
> > > > > > > Thank you for bringing this problem to our attention,
for
> > reminding
> > > > me
> > > > > of
> > > > > > > this past issue, and for doing the research and adding
"see
> > > MET-help
> > > > > > ticket
> > > > > > > #95364" in the email. That was very helpful
information, and I
> > > > > > appreciate
> > > > > > > your effort!
> > > > > > >
> > > > > > > Can you please send me the command line you are using
and the
> > > > location
> > > > > of
> > > > > > > your data so I can test with the version that Mallory
put
> > together
> > > > for
> > > > > > nco
> > > > > > > before the official release of met-9.1.3?
> > > > > > >
> > > > > > > The script we provided should already put the value for
LIBPNG
> in
> > > > > > LDFLAGS,
> > > > > > > so I suspect the value may not be set correctly, but I
will
> look
> > > into
> > > > > > > that.
> > > > > > >
> > > > > > > I'll follow up once I'm able to run your test case with
the
> other
> > > > > version
> > > > > > > and compare to see what's going on.
> > > > > > >
> > > > > > > We definitely want to get this fixed before the
installation of
> > > > > > > met-10.0.0, so thanks for staying on top of this and
alerting
> us
> > > now.
> > > > > > >
> > > > > > > Julie
> > > > > > >
> > > > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > >>
> > > > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted upon.
> > > > > > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > > >> Queue: met_help
> > > > > > >> Subject: Libpng warning with MET-9.1.3 in
production on
> > WCOSS
> > > > > Dell
> > > > > > >> Owner: Nobody
> > > > > > >> Requestors: logan.dawson at noaa.gov
> > > > > > >> Status: new
> > > > > > >> Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > > > >
> > > > > > >>
> > > > > > >>
> > > > > > >> Hello,
> > > > > > >>
> > > > > > >> The MET-9.1.3 installed into operations on WCOSS Dell (
> > > > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning a
libpng
> > > > warning
> > > > > > when
> > > > > > >> trying to read MRMS files. Here's an example of the
output
> from
> > > > > > >> regrid_data_plane showing that MET reads all missing
data
> from a
> > > > MRMS
> > > > > > >> file:
> > > > > > >>
> > > > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match
> for
> > > > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record 1
field 1
> > of
> > > > > GRIB2
> > > > > > >> file
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > > > >>
> > > > > > >> *libpng warning: Application built with libpng-1.2.59
but
> > running
> > > > with
> > > > > > >> 1.5.13*
> > > > > > >>
> > > > > > >> DEBUG 4:
> > > > > > >>
> > > > > > >> DEBUG 4: Data plane information:
> > > > > > >>
> > > > > > >> DEBUG 4: plane min: -999
> > > > > > >>
> > > > > > >> *DEBUG 4: plane max: -999*
> > > > > > >>
> > > > > > >> DEBUG 4: scan mode: 0
> > > > > > >>
> > > > > > >> DEBUG 4: valid time: 20210426_000034
> > > > > > >>
> > > > > > >> DEBUG 4: lead time: 000000
> > > > > > >>
> > > > > > >> DEBUG 4: init time: 20210426_000034
> > > > > > >>
> > > > > > >> DEBUG 4: bitmap flag: 255
> > > > > > >>
> > > > > > >> DEBUG 4:
> > > > > > >>
> > > > > > >> *DEBUG 2: Range of input data
> > > (name="MergedReflectivityQCComposite";
> > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > >>
> > > > > > >> *DEBUG 2: Range of regridded data
> > > > > (name="MergedReflectivityQCComposite";
> > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > >>
> > > > > > >> DEBUG 1: Writing output file:
> > > > > > >>
/gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > > > >> 20210426-000000_g227.nc
> > > > > > >>
> > > > > > >> This same issue previously occurred when MET-9.0.1 was
> installed
> > > on
> > > > > the
> > > > > > >> Dells in May/June 2020, and Julie found a resolution
for the
> > issue
> > > > > > before
> > > > > > >> the MET-9.0.2 installation (see MET-help ticket
#95364). The
> > issue
> > > > was
> > > > > > >> resolved with MET installs in the noscrub/emc.metplus
space up
> > > > through
> > > > > > >> MET-9.1. However, once this libpng warning issue was
> identified
> > in
> > > > the
> > > > > > >> nwprod/met.v9.1.3 installation, I found that the same
issue
> > shows
> > > up
> > > > > > with
> > > > > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
> > installations
> > > on
> > > > > the
> > > > > > >> Dell.
> > > > > > >>
> > > > > > >> It seems as if the resolution from the previous MET-
help
> ticket
> > > > #95364
> > > > > > >> (adding LIBPNG to LDFLAGS in the compilation script)
maybe
> > didn't
> > > > > > prevent
> > > > > > >> this problem from resurfacing in MET-9.1.1 and later
> > > installations?
> > > > It
> > > > > > >> seems that we should take a look to see if we can find
a fix
> > > before
> > > > > > >> MET-10.0 is released and installed on WCOSS.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Logan
> > > > > > >>
> > > > > > >> --
> > > > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > >> Physical Scientist, Model Evaluation Group (VPPPG
Branch)
> > > > > > >> NOAA/NWS/NCEP/EMC
> > > > > > >> (301) 683-3944
> > > > > > >>
> > > > > > >>
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik (she/her)
> > > > > > > Software Engineer
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik (she/her)
> > > > > > Software Engineer
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > > NOAA/NWS/NCEP/EMC
> > > > > (301) 683-3944
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik (she/her)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > Logan C. Dawson, Ph.D. (he/him/his)
> > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > NOAA/NWS/NCEP/EMC
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: John Halley Gotway
Time: Tue May 04 09:56:38 2021
Logan,
I saw your conversation with Julie about this and noticed the log
messages
you sent:
DEBUG 4: Data plane information:
DEBUG 4: plane min: -999
DEBUG 4: plane max: 59
I'm wondering about the minimum value of "-999". Is that a real data
value
or is that supposed to be missing data?
If that latter, you could consider using data censoring to set it bad
data
in MET... which uses -9999.
Here's an example of how you could do this in a call to
plot_data_plane:
plot_data_plane data.grib plot.ps 'name="TMP"; level="Z2";
censor_thresh =
[ ==-999 ]; censor_val = [ -9999 ];'
Obviously you're not plotting 2-m temperature, but that's just an
example.
This will replace all instances of -999 with a value of -9999 which
MET
treats as missing data. Typically missing data is handled by a
_FillValue
variable attribute in NetCDF files or the right metadata in GRIB
files. But
if that metadata isn't correct, using censor_thresh and censor_val is
a
good way to handle it.
Thanks,
John
On Mon, May 3, 2021 at 6:50 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
> So is it correct to say that if 9.1.3 is not updated in the official
> installation, it is ok because the output is the same, only a
warning is
> thrown?
>
> Julie
>
> On Mon, May 3, 2021 at 1:01 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> >
> > Hi Julie,
> >
> > All three output files are all the same. I did a ncdump -h on the
output
> > files, and I plotted the data as well. The data values and plotted
output
> > are all the same as well, so I'm comfortable saying the two tests
match
> the
> > expected output from met/9.1.
> >
> > Thanks,
> > Logan
> >
> > On Mon, May 3, 2021 at 1:07 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Hi Logan. Thank you for all of this information. I had also
planned to
> > add
> > > this specific use case to the WCOSS test suite, but it sounds
like you
> > and
> > > Mallory are taking care of that. Please let me know if either
of you
> > would
> > > like me to do anything.
> > >
> > > It is great that the data plane information in the output is the
same
> in
> > > both instances and matches the output produced from the
previously
> > > successful runs with met/9.1. Can you please tell me if the
output
> file
> > > has any differences when using the 9.1.3_test module and the
9.1.3
> > > operational version?
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > >
> > >
> > > On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
>
> > > >
> > > > Hi Julie,
> > > >
> > > > I just tested the command again with met/10.0.0-beta5 and
> > met/9.1.3_test
> > > > and see that the warning is now gone in both instances. I also
see
> that
> > > the
> > > > data plane information in the output is the same in both
instances
> and
> > > > matches the output produced from the previously successful
runs with
> > > > met/9.1:
> > > >
> > > > DEBUG 4: Data plane information:
> > > >
> > > > DEBUG 4: plane min: -999
> > > >
> > > > *DEBUG 4: plane max: 59*
> > > >
> > > > DEBUG 4: scan mode: 0
> > > >
> > > > DEBUG 4: valid time: 20210426_000034
> > > >
> > > > DEBUG 4: lead time: 000000
> > > >
> > > > DEBUG 4: init time: 20210426_000034
> > > >
> > > > DEBUG 4: bitmap flag: 255
> > > >
> > > > DEBUG 4:
> > > >
> > > > *DEBUG 2: Range of input data
(name="MergedReflectivityQCComposite";
> > > > level="Z500";) is -999 to 59.*
> > > >
> > > > *DEBUG 2: Range of regridded data
> > (name="MergedReflectivityQCComposite";
> > > > level="Z500";) is -999 to 51.8493.*
> > > >
> > > > Also, as one final sanity check, I read the output files in
with
> python
> > > and
> > > > can confirm the output data are the same between the two test
runs
> and
> > > the
> > > > met/9.1 run. Thanks for installing both test capabilities so
we can
> > > ensure
> > > > that the compilation fix works for 9.1.3 and 10.0.0!
> > > >
> > > > Also, we're planning to add this specific use case to our
WCOSS test
> > > suite
> > > > so that we can catch any future issues ahead of time. I'm
unsure of
> why
> > > the
> > > > warning is produced when MRMS data are read on the Dells, but
clearly
> > > it's
> > > > something we should keep an eye on with future installations
and new
> > > > machines.
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Logan.
> > > > >
> > > > > Thank you for sending the command you're using and for
moving the
> > input
> > > > > file to a location that wouldn't be scrubbed. I wish I
better
> > > understood
> > > > > this problem. It is a bit confusing still since version
1.5.13 of
> > the
> > > > png
> > > > > library doesn't seem to be available on WCOSS.
> > > > >
> > > > > I agree that testing the change with both is the best thing
to
> do. I
> > > > have
> > > > > done that and have tested with both and see that the warning
is now
> > > gone.
> > > > > I did this testing using the following:
> > > > >
> > > > > module use
> > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > > module load met/10.0.0-beta5
> > > > >
> > > > >
> > > > > and
> > > > >
> > > > > module use
> > > > > >
> >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > > > > module load met/9.1.3_test
> > > > >
> > > > >
> > > > > Logan, do you notice any problems with the output? I'm
trying to
> > > figure
> > > > > out if the warning has any ill effects.
> > > > >
> > > > > Thank you again for bringing this to our attention. It is
much
> > > > > appreciated.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA
Affiliate via
> RT
> > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > > > > >
> > > > > > Hi Julie,
> > > > > >
> > > > > > Here's the command that produced the output I previously
sent:
> > > > > >
> > > > > >
/gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane -v 5
> > > > -method
> > > > > > BUDGET -width 2 -field
'name="MergedReflectivityQCComposite";
> > > > > > level="Z500";' -name MergedReflectivityQCComposite
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > > > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > > > > MergedReflectivityQCComposite_00.50_20210426-
000000_g227.nc
> > > > > >
> > > > > > I moved the input file to a location that won't be
scrubbed while
> > > we're
> > > > > > testing.
> > > > > >
> > > > > > If it's not too much trouble, it might be good to test
this with
> an
> > > > > update
> > > > > > to the met-10.0.0-beta5 installation and a good install of
> > met-9.1.3.
> > > > > That
> > > > > > way, if NCO is able to update the version installed in
nwprod,
> > we'll
> > > > have
> > > > > > confirmation that the compilation update will fix the
problem
> with
> > > that
> > > > > > version. I assume it will, but it wouldn't hurt to confirm
before
> > > going
> > > > > > back to NCO.
> > > > > >
> > > > > > Thanks very much for checking into this!
> > > > > > Logan
> > > > > >
> > > > > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Unfortunately, I am wrong on all accounts.
> > > > > > >
> > > > > > > The script did not contain the addition of "-
L${LIB_LIBPNG}" in
> > > > rpath,
> > > > > > and
> > > > > > > the value of LIB_LIBPNG is set correctly.
> > > > > > >
> > > > > > > The script currently contains:
> > > > > > >
> > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
-L${MET_HDF5}/lib"
> > > > > > >
> > > > > > >
> > > > > > > but should contain:
> > > > > > >
> > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER} *-
L${LIB_LIBPNG}*
> > > > > > > > -L${MET_HDF5}/lib"
> > > > > > >
> > > > > > >
> > > > > > > I have added Mallory to this email to see if there is
anything
> we
> > > can
> > > > > do
> > > > > > to
> > > > > > > get this fixed for the already installed version of met-
9.1.3.
> > If
> > > > not,
> > > > > > we
> > > > > > > will be releasing met-10.0.0 soon and hopefully the
install
> > process
> > > > > will
> > > > > > go
> > > > > > > more quickly since we recently went through it for
9.1.3.
> > > > > > >
> > > > > > > Mallory, I have made some other additions and updates to
the
> > > > > > > compile_MET_all.sh script, which I will include for the
release
> > of
> > > > > > 10.0.0.
> > > > > > >
> > > > > > > Logan and Mallory, I would also like to test this
addition to
> the
> > > > > > > compile_MET_all.sh script to make sure it solves this
problem
> > > > > (again). I
> > > > > > > plan to update my latest install of met-10.0.0-beta5 to
test,
> > but I
> > > > can
> > > > > > > also test to get a good install of 9.1.3 if you would
like,
> > Logan.
> > > > > > Please
> > > > > > > let me know what you would prefer.
> > > > > > >
> > > > > > > Please let me know if you have any questions. My
apologies for
> > this
> > > > > > > trouble!
> > > > > > >
> > > > > > > Julie
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <
> > > jpresto at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Logan.
> > > > > > > >
> > > > > > > > Thank you for bringing this problem to our attention,
for
> > > reminding
> > > > > me
> > > > > > of
> > > > > > > > this past issue, and for doing the research and adding
"see
> > > > MET-help
> > > > > > > ticket
> > > > > > > > #95364" in the email. That was very helpful
information,
> and I
> > > > > > > appreciate
> > > > > > > > your effort!
> > > > > > > >
> > > > > > > > Can you please send me the command line you are using
and the
> > > > > location
> > > > > > of
> > > > > > > > your data so I can test with the version that Mallory
put
> > > together
> > > > > for
> > > > > > > nco
> > > > > > > > before the official release of met-9.1.3?
> > > > > > > >
> > > > > > > > The script we provided should already put the value
for
> LIBPNG
> > in
> > > > > > > LDFLAGS,
> > > > > > > > so I suspect the value may not be set correctly, but I
will
> > look
> > > > into
> > > > > > > > that.
> > > > > > > >
> > > > > > > > I'll follow up once I'm able to run your test case
with the
> > other
> > > > > > version
> > > > > > > > and compare to see what's going on.
> > > > > > > >
> > > > > > > > We definitely want to get this fixed before the
installation
> of
> > > > > > > > met-10.0.0, so thanks for staying on top of this and
alerting
> > us
> > > > now.
> > > > > > > >
> > > > > > > > Julie
> > > > > > > >
> > > > > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > >>
> > > > > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted
upon.
> > > > > > > >> Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > > > >> Queue: met_help
> > > > > > > >> Subject: Libpng warning with MET-9.1.3 in
production on
> > > WCOSS
> > > > > > Dell
> > > > > > > >> Owner: Nobody
> > > > > > > >> Requestors: logan.dawson at noaa.gov
> > > > > > > >> Status: new
> > > > > > > >> Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > > > > >
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> Hello,
> > > > > > > >>
> > > > > > > >> The MET-9.1.3 installed into operations on WCOSS Dell
(
> > > > > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is returning
a
> libpng
> > > > > warning
> > > > > > > when
> > > > > > > >> trying to read MRMS files. Here's an example of the
output
> > from
> > > > > > > >> regrid_data_plane showing that MET reads all missing
data
> > from a
> > > > > MRMS
> > > > > > > >> file:
> > > > > > > >>
> > > > > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact match
> > for
> > > > > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2 record
1
> field 1
> > > of
> > > > > > GRIB2
> > > > > > > >> file
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > > > > >>
> > > > > > > >> *libpng warning: Application built with libpng-1.2.59
but
> > > running
> > > > > with
> > > > > > > >> 1.5.13*
> > > > > > > >>
> > > > > > > >> DEBUG 4:
> > > > > > > >>
> > > > > > > >> DEBUG 4: Data plane information:
> > > > > > > >>
> > > > > > > >> DEBUG 4: plane min: -999
> > > > > > > >>
> > > > > > > >> *DEBUG 4: plane max: -999*
> > > > > > > >>
> > > > > > > >> DEBUG 4: scan mode: 0
> > > > > > > >>
> > > > > > > >> DEBUG 4: valid time: 20210426_000034
> > > > > > > >>
> > > > > > > >> DEBUG 4: lead time: 000000
> > > > > > > >>
> > > > > > > >> DEBUG 4: init time: 20210426_000034
> > > > > > > >>
> > > > > > > >> DEBUG 4: bitmap flag: 255
> > > > > > > >>
> > > > > > > >> DEBUG 4:
> > > > > > > >>
> > > > > > > >> *DEBUG 2: Range of input data
> > > > (name="MergedReflectivityQCComposite";
> > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > >>
> > > > > > > >> *DEBUG 2: Range of regridded data
> > > > > > (name="MergedReflectivityQCComposite";
> > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > >>
> > > > > > > >> DEBUG 1: Writing output file:
> > > > > > > >>
/gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > > > > >> 20210426-000000_g227.nc
> > > > > > > >>
> > > > > > > >> This same issue previously occurred when MET-9.0.1
was
> > installed
> > > > on
> > > > > > the
> > > > > > > >> Dells in May/June 2020, and Julie found a resolution
for the
> > > issue
> > > > > > > before
> > > > > > > >> the MET-9.0.2 installation (see MET-help ticket
#95364). The
> > > issue
> > > > > was
> > > > > > > >> resolved with MET installs in the noscrub/emc.metplus
space
> up
> > > > > through
> > > > > > > >> MET-9.1. However, once this libpng warning issue was
> > identified
> > > in
> > > > > the
> > > > > > > >> nwprod/met.v9.1.3 installation, I found that the same
issue
> > > shows
> > > > up
> > > > > > > with
> > > > > > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
> > > installations
> > > > on
> > > > > > the
> > > > > > > >> Dell.
> > > > > > > >>
> > > > > > > >> It seems as if the resolution from the previous MET-
help
> > ticket
> > > > > #95364
> > > > > > > >> (adding LIBPNG to LDFLAGS in the compilation script)
maybe
> > > didn't
> > > > > > > prevent
> > > > > > > >> this problem from resurfacing in MET-9.1.1 and later
> > > > installations?
> > > > > It
> > > > > > > >> seems that we should take a look to see if we can
find a fix
> > > > before
> > > > > > > >> MET-10.0 is released and installed on WCOSS.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Logan
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > > >> Physical Scientist, Model Evaluation Group (VPPPG
Branch)
> > > > > > > >> NOAA/NWS/NCEP/EMC
> > > > > > > >> (301) 683-3944
> > > > > > > >>
> > > > > > > >>
> > > > > > > >
> > > > > > > > --
> > > > > > > > Julie Prestopnik (she/her)
> > > > > > > > Software Engineer
> > > > > > > > National Center for Atmospheric Research
> > > > > > > > Research Applications Laboratory
> > > > > > > > Email: jpresto at ucar.edu
> > > > > > > >
> > > > > > > > My working day may not be your working day. Please do
not
> feel
> > > > > obliged
> > > > > > > to
> > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik (she/her)
> > > > > > > Software Engineer
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > > > NOAA/NWS/NCEP/EMC
> > > > > > (301) 683-3944
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik (she/her)
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > > --
> > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > NOAA/NWS/NCEP/EMC
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > Logan C. Dawson, Ph.D. (he/him/his)
> > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > NOAA/NWS/NCEP/EMC
> > (301) 683-3944
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Logan Dawson - NOAA Affiliate
Time: Tue May 04 10:46:11 2021
Julie,
I'm sorry for the confusion. I completely misread this question
yesterday
afternoon:
> Can you please tell me if the output file has any differences when
using
> the 9.1.3_test module and the 9.1.3 operational version?
Yes, there are differences between the output files from those two
versions. The output file from the 9.1.3_test module is correct and
matches
the output from the emc.metplus/met/9.1 installation. The output from
the
9.1.3 operational version still throws the libpng warning and has an
incorrect data plane min and max = -999. The libpng warning in the
operational version is still meaningful.
My apologies for misreading your question yesterday - doing a few too
many
things at once. I was referring to the output from the 9.1.3_test,
10.0.0_beta5, and the 9.1 installations when I said all three outputs
were
the same.
John,
Thanks for mentioning the data censoring. I actually do have a
censor_thresh set in the METplus conf file that runs RegridDataPlane
on the
MRMS files. Example:
OBS_VAR1_NAME = MergedReflectivityQCComposite
OBS_VAR1_LEVELS = Z500
OBS_VAR1_OPTIONS = censor_thresh = eq-999; censor_val = -9999;
But it seems like METplus isn't applying those two options as
environment
variables when the MET command is built. Regardless, it's not a
problem for
my current setup because I have the same two options set in my
GridStat
conf file, so those missing values are actually censored during the
process
of running grid_stat.
And as an FYI, I run RegridDataPlane on the MRMS observation data as
somewhat of a pre-processing step because I verify ~10 different model
runs
at the same time, so it's advantageous to interpolate the obs to the
common
grid one time with a single, separate step.
Thanks,
Logan
On Tue, May 4, 2021 at 10:56 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Logan,
>
> I saw your conversation with Julie about this and noticed the log
messages
> you sent:
>
> DEBUG 4: Data plane information:
> DEBUG 4: plane min: -999
> DEBUG 4: plane max: 59
>
> I'm wondering about the minimum value of "-999". Is that a real data
value
> or is that supposed to be missing data?
>
> If that latter, you could consider using data censoring to set it
bad data
> in MET... which uses -9999.
> Here's an example of how you could do this in a call to
plot_data_plane:
>
> plot_data_plane data.grib plot.ps 'name="TMP"; level="Z2";
censor_thresh =
> [ ==-999 ]; censor_val = [ -9999 ];'
>
> Obviously you're not plotting 2-m temperature, but that's just an
example.
> This will replace all instances of -999 with a value of -9999 which
MET
> treats as missing data. Typically missing data is handled by a
_FillValue
> variable attribute in NetCDF files or the right metadata in GRIB
files. But
> if that metadata isn't correct, using censor_thresh and censor_val
is a
> good way to handle it.
>
> Thanks,
> John
>
> On Mon, May 3, 2021 at 6:50 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> >
> > So is it correct to say that if 9.1.3 is not updated in the
official
> > installation, it is ok because the output is the same, only a
warning is
> > thrown?
> >
> > Julie
> >
> > On Mon, May 3, 2021 at 1:01 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > >
> > > Hi Julie,
> > >
> > > All three output files are all the same. I did a ncdump -h on
the
> output
> > > files, and I plotted the data as well. The data values and
plotted
> output
> > > are all the same as well, so I'm comfortable saying the two
tests match
> > the
> > > expected output from met/9.1.
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Mon, May 3, 2021 at 1:07 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Logan. Thank you for all of this information. I had also
planned
> to
> > > add
> > > > this specific use case to the WCOSS test suite, but it sounds
like
> you
> > > and
> > > > Mallory are taking care of that. Please let me know if either
of you
> > > would
> > > > like me to do anything.
> > > >
> > > > It is great that the data plane information in the output is
the same
> > in
> > > > both instances and matches the output produced from the
previously
> > > > successful runs with met/9.1. Can you please tell me if the
output
> > file
> > > > has any differences when using the 9.1.3_test module and the
9.1.3
> > > > operational version?
> > > >
> > > > Thanks!
> > > >
> > > > Julie
> > > >
> > > >
> > > >
> > > > On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA Affiliate
via RT
> <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > > > >
> > > > > Hi Julie,
> > > > >
> > > > > I just tested the command again with met/10.0.0-beta5 and
> > > met/9.1.3_test
> > > > > and see that the warning is now gone in both instances. I
also see
> > that
> > > > the
> > > > > data plane information in the output is the same in both
instances
> > and
> > > > > matches the output produced from the previously successful
runs
> with
> > > > > met/9.1:
> > > > >
> > > > > DEBUG 4: Data plane information:
> > > > >
> > > > > DEBUG 4: plane min: -999
> > > > >
> > > > > *DEBUG 4: plane max: 59*
> > > > >
> > > > > DEBUG 4: scan mode: 0
> > > > >
> > > > > DEBUG 4: valid time: 20210426_000034
> > > > >
> > > > > DEBUG 4: lead time: 000000
> > > > >
> > > > > DEBUG 4: init time: 20210426_000034
> > > > >
> > > > > DEBUG 4: bitmap flag: 255
> > > > >
> > > > > DEBUG 4:
> > > > >
> > > > > *DEBUG 2: Range of input data
> (name="MergedReflectivityQCComposite";
> > > > > level="Z500";) is -999 to 59.*
> > > > >
> > > > > *DEBUG 2: Range of regridded data
> > > (name="MergedReflectivityQCComposite";
> > > > > level="Z500";) is -999 to 51.8493.*
> > > > >
> > > > > Also, as one final sanity check, I read the output files in
with
> > python
> > > > and
> > > > > can confirm the output data are the same between the two
test runs
> > and
> > > > the
> > > > > met/9.1 run. Thanks for installing both test capabilities so
we can
> > > > ensure
> > > > > that the compilation fix works for 9.1.3 and 10.0.0!
> > > > >
> > > > > Also, we're planning to add this specific use case to our
WCOSS
> test
> > > > suite
> > > > > so that we can catch any future issues ahead of time. I'm
unsure of
> > why
> > > > the
> > > > > warning is produced when MRMS data are read on the Dells,
but
> clearly
> > > > it's
> > > > > something we should keep an eye on with future installations
and
> new
> > > > > machines.
> > > > >
> > > > > Thanks,
> > > > > Logan
> > > > >
> > > > > On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Logan.
> > > > > >
> > > > > > Thank you for sending the command you're using and for
moving the
> > > input
> > > > > > file to a location that wouldn't be scrubbed. I wish I
better
> > > > understood
> > > > > > this problem. It is a bit confusing still since version
1.5.13
> of
> > > the
> > > > > png
> > > > > > library doesn't seem to be available on WCOSS.
> > > > > >
> > > > > > I agree that testing the change with both is the best
thing to
> > do. I
> > > > > have
> > > > > > done that and have tested with both and see that the
warning is
> now
> > > > gone.
> > > > > > I did this testing using the following:
> > > > > >
> > > > > > module use
> > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > > > module load met/10.0.0-beta5
> > > > > >
> > > > > >
> > > > > > and
> > > > > >
> > > > > > module use
> > > > > > >
> > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > > > > > module load met/9.1.3_test
> > > > > >
> > > > > >
> > > > > > Logan, do you notice any problems with the output? I'm
trying to
> > > > figure
> > > > > > out if the warning has any ill effects.
> > > > > >
> > > > > > Thank you again for bringing this to our attention. It is
much
> > > > > > appreciated.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA
Affiliate via
> > RT
> > > <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> >
> > > > > > >
> > > > > > > Hi Julie,
> > > > > > >
> > > > > > > Here's the command that produced the output I previously
sent:
> > > > > > >
> > > > > > >
/gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane
> -v 5
> > > > > -method
> > > > > > > BUDGET -width 2 -field
'name="MergedReflectivityQCComposite";
> > > > > > > level="Z500";' -name MergedReflectivityQCComposite
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > > > > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > > > > > MergedReflectivityQCComposite_00.50_20210426-
000000_g227.nc
> > > > > > >
> > > > > > > I moved the input file to a location that won't be
scrubbed
> while
> > > > we're
> > > > > > > testing.
> > > > > > >
> > > > > > > If it's not too much trouble, it might be good to test
this
> with
> > an
> > > > > > update
> > > > > > > to the met-10.0.0-beta5 installation and a good install
of
> > > met-9.1.3.
> > > > > > That
> > > > > > > way, if NCO is able to update the version installed in
nwprod,
> > > we'll
> > > > > have
> > > > > > > confirmation that the compilation update will fix the
problem
> > with
> > > > that
> > > > > > > version. I assume it will, but it wouldn't hurt to
confirm
> before
> > > > going
> > > > > > > back to NCO.
> > > > > > >
> > > > > > > Thanks very much for checking into this!
> > > > > > > Logan
> > > > > > >
> > > > > > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via RT
<
> > > > > > met_help at ucar.edu
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Unfortunately, I am wrong on all accounts.
> > > > > > > >
> > > > > > > > The script did not contain the addition of "-
L${LIB_LIBPNG}"
> in
> > > > > rpath,
> > > > > > > and
> > > > > > > > the value of LIB_LIBPNG is set correctly.
> > > > > > > >
> > > > > > > > The script currently contains:
> > > > > > > >
> > > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
> -L${MET_HDF5}/lib"
> > > > > > > >
> > > > > > > >
> > > > > > > > but should contain:
> > > > > > > >
> > > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
> *-L${LIB_LIBPNG}*
> > > > > > > > > -L${MET_HDF5}/lib"
> > > > > > > >
> > > > > > > >
> > > > > > > > I have added Mallory to this email to see if there is
> anything
> > we
> > > > can
> > > > > > do
> > > > > > > to
> > > > > > > > get this fixed for the already installed version of
> met-9.1.3.
> > > If
> > > > > not,
> > > > > > > we
> > > > > > > > will be releasing met-10.0.0 soon and hopefully the
install
> > > process
> > > > > > will
> > > > > > > go
> > > > > > > > more quickly since we recently went through it for
9.1.3.
> > > > > > > >
> > > > > > > > Mallory, I have made some other additions and updates
to the
> > > > > > > > compile_MET_all.sh script, which I will include for
the
> release
> > > of
> > > > > > > 10.0.0.
> > > > > > > >
> > > > > > > > Logan and Mallory, I would also like to test this
addition to
> > the
> > > > > > > > compile_MET_all.sh script to make sure it solves this
problem
> > > > > > (again). I
> > > > > > > > plan to update my latest install of met-10.0.0-beta5
to test,
> > > but I
> > > > > can
> > > > > > > > also test to get a good install of 9.1.3 if you would
like,
> > > Logan.
> > > > > > > Please
> > > > > > > > let me know what you would prefer.
> > > > > > > >
> > > > > > > > Please let me know if you have any questions. My
apologies
> for
> > > this
> > > > > > > > trouble!
> > > > > > > >
> > > > > > > > Julie
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <
> > > > jpresto at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Logan.
> > > > > > > > >
> > > > > > > > > Thank you for bringing this problem to our
attention, for
> > > > reminding
> > > > > > me
> > > > > > > of
> > > > > > > > > this past issue, and for doing the research and
adding "see
> > > > > MET-help
> > > > > > > > ticket
> > > > > > > > > #95364" in the email. That was very helpful
information,
> > and I
> > > > > > > > appreciate
> > > > > > > > > your effort!
> > > > > > > > >
> > > > > > > > > Can you please send me the command line you are
using and
> the
> > > > > > location
> > > > > > > of
> > > > > > > > > your data so I can test with the version that
Mallory put
> > > > together
> > > > > > for
> > > > > > > > nco
> > > > > > > > > before the official release of met-9.1.3?
> > > > > > > > >
> > > > > > > > > The script we provided should already put the value
for
> > LIBPNG
> > > in
> > > > > > > > LDFLAGS,
> > > > > > > > > so I suspect the value may not be set correctly, but
I will
> > > look
> > > > > into
> > > > > > > > > that.
> > > > > > > > >
> > > > > > > > > I'll follow up once I'm able to run your test case
with the
> > > other
> > > > > > > version
> > > > > > > > > and compare to see what's going on.
> > > > > > > > >
> > > > > > > > > We definitely want to get this fixed before the
> installation
> > of
> > > > > > > > > met-10.0.0, so thanks for staying on top of this and
> alerting
> > > us
> > > > > now.
> > > > > > > > >
> > > > > > > > > Julie
> > > > > > > > >
> > > > > > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson - NOAA
> > Affiliate
> > > > via
> > > > > > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > >>
> > > > > > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted
upon.
> > > > > > > > >> Transaction: Ticket created by
logan.dawson at noaa.gov
> > > > > > > > >> Queue: met_help
> > > > > > > > >> Subject: Libpng warning with MET-9.1.3 in
production
> on
> > > > WCOSS
> > > > > > > Dell
> > > > > > > > >> Owner: Nobody
> > > > > > > > >> Requestors: logan.dawson at noaa.gov
> > > > > > > > >> Status: new
> > > > > > > > >> Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > > > > > >
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Hello,
> > > > > > > > >>
> > > > > > > > >> The MET-9.1.3 installed into operations on WCOSS
Dell (
> > > > > > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is
returning a
> > libpng
> > > > > > warning
> > > > > > > > when
> > > > > > > > >> trying to read MRMS files. Here's an example of the
output
> > > from
> > > > > > > > >> regrid_data_plane showing that MET reads all
missing data
> > > from a
> > > > > > MRMS
> > > > > > > > >> file:
> > > > > > > > >>
> > > > > > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact
> match
> > > for
> > > > > > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2
record 1
> > field 1
> > > > of
> > > > > > > GRIB2
> > > > > > > > >> file
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > > > > > >>
> > > > > > > > >> *libpng warning: Application built with libpng-
1.2.59 but
> > > > running
> > > > > > with
> > > > > > > > >> 1.5.13*
> > > > > > > > >>
> > > > > > > > >> DEBUG 4:
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: Data plane information:
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: plane min: -999
> > > > > > > > >>
> > > > > > > > >> *DEBUG 4: plane max: -999*
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: scan mode: 0
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: valid time: 20210426_000034
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: lead time: 000000
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: init time: 20210426_000034
> > > > > > > > >>
> > > > > > > > >> DEBUG 4: bitmap flag: 255
> > > > > > > > >>
> > > > > > > > >> DEBUG 4:
> > > > > > > > >>
> > > > > > > > >> *DEBUG 2: Range of input data
> > > > > (name="MergedReflectivityQCComposite";
> > > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > > >>
> > > > > > > > >> *DEBUG 2: Range of regridded data
> > > > > > > (name="MergedReflectivityQCComposite";
> > > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > > >>
> > > > > > > > >> DEBUG 1: Writing output file:
> > > > > > > > >>
> /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > > > > > >> 20210426-000000_g227.nc
> > > > > > > > >>
> > > > > > > > >> This same issue previously occurred when MET-9.0.1
was
> > > installed
> > > > > on
> > > > > > > the
> > > > > > > > >> Dells in May/June 2020, and Julie found a
resolution for
> the
> > > > issue
> > > > > > > > before
> > > > > > > > >> the MET-9.0.2 installation (see MET-help ticket
#95364).
> The
> > > > issue
> > > > > > was
> > > > > > > > >> resolved with MET installs in the
noscrub/emc.metplus
> space
> > up
> > > > > > through
> > > > > > > > >> MET-9.1. However, once this libpng warning issue
was
> > > identified
> > > > in
> > > > > > the
> > > > > > > > >> nwprod/met.v9.1.3 installation, I found that the
same
> issue
> > > > shows
> > > > > up
> > > > > > > > with
> > > > > > > > >> the emc.metplus/met/9.1.1 and emc.metplus/met/9.1.2
> > > > installations
> > > > > on
> > > > > > > the
> > > > > > > > >> Dell.
> > > > > > > > >>
> > > > > > > > >> It seems as if the resolution from the previous
MET-help
> > > ticket
> > > > > > #95364
> > > > > > > > >> (adding LIBPNG to LDFLAGS in the compilation
script) maybe
> > > > didn't
> > > > > > > > prevent
> > > > > > > > >> this problem from resurfacing in MET-9.1.1 and
later
> > > > > installations?
> > > > > > It
> > > > > > > > >> seems that we should take a look to see if we can
find a
> fix
> > > > > before
> > > > > > > > >> MET-10.0 is released and installed on WCOSS.
> > > > > > > > >>
> > > > > > > > >> Thanks,
> > > > > > > > >> Logan
> > > > > > > > >>
> > > > > > > > >> --
> > > > > > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > > > >> Physical Scientist, Model Evaluation Group (VPPPG
Branch)
> > > > > > > > >> NOAA/NWS/NCEP/EMC
> > > > > > > > >> (301) 683-3944
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Julie Prestopnik (she/her)
> > > > > > > > > Software Engineer
> > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > Research Applications Laboratory
> > > > > > > > > Email: jpresto at ucar.edu
> > > > > > > > >
> > > > > > > > > My working day may not be your working day. Please
do not
> > feel
> > > > > > obliged
> > > > > > > > to
> > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Julie Prestopnik (she/her)
> > > > > > > > Software Engineer
> > > > > > > > National Center for Atmospheric Research
> > > > > > > > Research Applications Laboratory
> > > > > > > > Email: jpresto at ucar.edu
> > > > > > > >
> > > > > > > > My working day may not be your working day. Please do
not
> feel
> > > > > obliged
> > > > > > > to
> > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > > Physical Scientist, Model Evaluation Group (VPPPG
Branch)
> > > > > > > NOAA/NWS/NCEP/EMC
> > > > > > > (301) 683-3944
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik (she/her)
> > > > > > Software Engineer
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > > NOAA/NWS/NCEP/EMC
> > > > > (301) 683-3944
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik (she/her)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > Logan C. Dawson, Ph.D. (he/him/his)
> > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > NOAA/NWS/NCEP/EMC
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
Logan C. Dawson, Ph.D. (he/him/his)
Physical Scientist, Model Evaluation Group (VPPPG Branch)
NOAA/NWS/NCEP/EMC
(301) 683-3944
------------------------------------------------
Subject: Libpng warning with MET-9.1.3 in production on WCOSS Dell
From: Julie Prestopnik
Time: Tue May 04 14:42:41 2021
Hi Logan. Thanks for letting me know you misread the question. I
thought
you might have, but wasn't sure, so I thought I should restate my
question
to get to the heart of what I was wondering, just in case. I totally
understand doing a few too many things at once! No worries at all and
thank you for the follow up. I am so discouraged to hear about the
differences and apologize for this change not making it into the
script.
Is there any word on reinstalling 9.1.3 operationally? Thanks for any
information you can provide.
Julie
On Tue, May 4, 2021 at 10:46 AM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
>
> Julie,
>
> I'm sorry for the confusion. I completely misread this question
yesterday
> afternoon:
>
> > Can you please tell me if the output file has any differences when
using
> > the 9.1.3_test module and the 9.1.3 operational version?
>
>
> Yes, there are differences between the output files from those two
> versions. The output file from the 9.1.3_test module is correct and
matches
> the output from the emc.metplus/met/9.1 installation. The output
from the
> 9.1.3 operational version still throws the libpng warning and has an
> incorrect data plane min and max = -999. The libpng warning in the
> operational version is still meaningful.
>
> My apologies for misreading your question yesterday - doing a few
too many
> things at once. I was referring to the output from the 9.1.3_test,
> 10.0.0_beta5, and the 9.1 installations when I said all three
outputs were
> the same.
>
>
> John,
> Thanks for mentioning the data censoring. I actually do have a
> censor_thresh set in the METplus conf file that runs RegridDataPlane
on the
> MRMS files. Example:
>
> OBS_VAR1_NAME = MergedReflectivityQCComposite
>
> OBS_VAR1_LEVELS = Z500
>
> OBS_VAR1_OPTIONS = censor_thresh = eq-999; censor_val = -9999;
>
> But it seems like METplus isn't applying those two options as
environment
> variables when the MET command is built. Regardless, it's not a
problem for
> my current setup because I have the same two options set in my
GridStat
> conf file, so those missing values are actually censored during the
process
> of running grid_stat.
>
> And as an FYI, I run RegridDataPlane on the MRMS observation data as
> somewhat of a pre-processing step because I verify ~10 different
model runs
> at the same time, so it's advantageous to interpolate the obs to the
common
> grid one time with a single, separate step.
>
> Thanks,
> Logan
>
> On Tue, May 4, 2021 at 10:56 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Logan,
> >
> > I saw your conversation with Julie about this and noticed the log
> messages
> > you sent:
> >
> > DEBUG 4: Data plane information:
> > DEBUG 4: plane min: -999
> > DEBUG 4: plane max: 59
> >
> > I'm wondering about the minimum value of "-999". Is that a real
data
> value
> > or is that supposed to be missing data?
> >
> > If that latter, you could consider using data censoring to set it
bad
> data
> > in MET... which uses -9999.
> > Here's an example of how you could do this in a call to
plot_data_plane:
> >
> > plot_data_plane data.grib plot.ps 'name="TMP"; level="Z2";
> censor_thresh =
> > [ ==-999 ]; censor_val = [ -9999 ];'
> >
> > Obviously you're not plotting 2-m temperature, but that's just an
> example.
> > This will replace all instances of -999 with a value of -9999
which MET
> > treats as missing data. Typically missing data is handled by a
_FillValue
> > variable attribute in NetCDF files or the right metadata in GRIB
files.
> But
> > if that metadata isn't correct, using censor_thresh and censor_val
is a
> > good way to handle it.
> >
> > Thanks,
> > John
> >
> > On Mon, May 3, 2021 at 6:50 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > >
> > > So is it correct to say that if 9.1.3 is not updated in the
official
> > > installation, it is ok because the output is the same, only a
warning
> is
> > > thrown?
> > >
> > > Julie
> > >
> > > On Mon, May 3, 2021 at 1:01 PM Logan Dawson - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
>
> > > >
> > > > Hi Julie,
> > > >
> > > > All three output files are all the same. I did a ncdump -h on
the
> > output
> > > > files, and I plotted the data as well. The data values and
plotted
> > output
> > > > are all the same as well, so I'm comfortable saying the two
tests
> match
> > > the
> > > > expected output from met/9.1.
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > On Mon, May 3, 2021 at 1:07 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Logan. Thank you for all of this information. I had also
> planned
> > to
> > > > add
> > > > > this specific use case to the WCOSS test suite, but it
sounds like
> > you
> > > > and
> > > > > Mallory are taking care of that. Please let me know if
either of
> you
> > > > would
> > > > > like me to do anything.
> > > > >
> > > > > It is great that the data plane information in the output is
the
> same
> > > in
> > > > > both instances and matches the output produced from the
previously
> > > > > successful runs with met/9.1. Can you please tell me if the
output
> > > file
> > > > > has any differences when using the 9.1.3_test module and the
9.1.3
> > > > > operational version?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Julie
> > > > >
> > > > >
> > > > >
> > > > > On Mon, May 3, 2021 at 11:44 AM Logan Dawson - NOAA
Affiliate via
> RT
> > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740 >
> > > > > >
> > > > > > Hi Julie,
> > > > > >
> > > > > > I just tested the command again with met/10.0.0-beta5 and
> > > > met/9.1.3_test
> > > > > > and see that the warning is now gone in both instances. I
also
> see
> > > that
> > > > > the
> > > > > > data plane information in the output is the same in both
> instances
> > > and
> > > > > > matches the output produced from the previously successful
runs
> > with
> > > > > > met/9.1:
> > > > > >
> > > > > > DEBUG 4: Data plane information:
> > > > > >
> > > > > > DEBUG 4: plane min: -999
> > > > > >
> > > > > > *DEBUG 4: plane max: 59*
> > > > > >
> > > > > > DEBUG 4: scan mode: 0
> > > > > >
> > > > > > DEBUG 4: valid time: 20210426_000034
> > > > > >
> > > > > > DEBUG 4: lead time: 000000
> > > > > >
> > > > > > DEBUG 4: init time: 20210426_000034
> > > > > >
> > > > > > DEBUG 4: bitmap flag: 255
> > > > > >
> > > > > > DEBUG 4:
> > > > > >
> > > > > > *DEBUG 2: Range of input data
> > (name="MergedReflectivityQCComposite";
> > > > > > level="Z500";) is -999 to 59.*
> > > > > >
> > > > > > *DEBUG 2: Range of regridded data
> > > > (name="MergedReflectivityQCComposite";
> > > > > > level="Z500";) is -999 to 51.8493.*
> > > > > >
> > > > > > Also, as one final sanity check, I read the output files
in with
> > > python
> > > > > and
> > > > > > can confirm the output data are the same between the two
test
> runs
> > > and
> > > > > the
> > > > > > met/9.1 run. Thanks for installing both test capabilities
so we
> can
> > > > > ensure
> > > > > > that the compilation fix works for 9.1.3 and 10.0.0!
> > > > > >
> > > > > > Also, we're planning to add this specific use case to our
WCOSS
> > test
> > > > > suite
> > > > > > so that we can catch any future issues ahead of time. I'm
unsure
> of
> > > why
> > > > > the
> > > > > > warning is produced when MRMS data are read on the Dells,
but
> > clearly
> > > > > it's
> > > > > > something we should keep an eye on with future
installations and
> > new
> > > > > > machines.
> > > > > >
> > > > > > Thanks,
> > > > > > Logan
> > > > > >
> > > > > > On Mon, May 3, 2021 at 12:05 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Logan.
> > > > > > >
> > > > > > > Thank you for sending the command you're using and for
moving
> the
> > > > input
> > > > > > > file to a location that wouldn't be scrubbed. I wish I
better
> > > > > understood
> > > > > > > this problem. It is a bit confusing still since version
1.5.13
> > of
> > > > the
> > > > > > png
> > > > > > > library doesn't seem to be available on WCOSS.
> > > > > > >
> > > > > > > I agree that testing the change with both is the best
thing to
> > > do. I
> > > > > > have
> > > > > > > done that and have tested with both and see that the
warning is
> > now
> > > > > gone.
> > > > > > > I did this testing using the following:
> > > > > > >
> > > > > > > module use
> > > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > > > > module load met/10.0.0-beta5
> > > > > > >
> > > > > > >
> > > > > > > and
> > > > > > >
> > > > > > > module use
> > > > > > > >
> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/for_nco/modulefiles
> > > > > > > > module load met/9.1.3_test
> > > > > > >
> > > > > > >
> > > > > > > Logan, do you notice any problems with the output? I'm
trying
> to
> > > > > figure
> > > > > > > out if the warning has any ill effects.
> > > > > > >
> > > > > > > Thank you again for bringing this to our attention. It
is much
> > > > > > > appreciated.
> > > > > > >
> > > > > > > Julie
> > > > > > >
> > > > > > > On Thu, Apr 29, 2021 at 1:31 PM Logan Dawson - NOAA
Affiliate
> via
> > > RT
> > > > <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > >
> > > > > > > >
> > > > > > > > Hi Julie,
> > > > > > > >
> > > > > > > > Here's the command that produced the output I
previously
> sent:
> > > > > > > >
> > > > > > > >
/gpfs/dell1/nco/ops/nwprod/met.v9.1.3/exec/regrid_data_plane
> > -v 5
> > > > > > -method
> > > > > > > > BUDGET -width 2 -field
'name="MergedReflectivityQCComposite";
> > > > > > > > level="Z500";' -name MergedReflectivityQCComposite
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
"/gpfs/dell2/emc/verification/noscrub/Logan.Dawson/CAM_verif/masks/grid_files/G227.nc"
> > > > > > > > /gpfs/dell2/ptmp/Logan.Dawson/
> > > > > > > > MergedReflectivityQCComposite_00.50_20210426-
000000_g227.nc
> > > > > > > >
> > > > > > > > I moved the input file to a location that won't be
scrubbed
> > while
> > > > > we're
> > > > > > > > testing.
> > > > > > > >
> > > > > > > > If it's not too much trouble, it might be good to test
this
> > with
> > > an
> > > > > > > update
> > > > > > > > to the met-10.0.0-beta5 installation and a good
install of
> > > > met-9.1.3.
> > > > > > > That
> > > > > > > > way, if NCO is able to update the version installed in
> nwprod,
> > > > we'll
> > > > > > have
> > > > > > > > confirmation that the compilation update will fix the
problem
> > > with
> > > > > that
> > > > > > > > version. I assume it will, but it wouldn't hurt to
confirm
> > before
> > > > > going
> > > > > > > > back to NCO.
> > > > > > > >
> > > > > > > > Thanks very much for checking into this!
> > > > > > > > Logan
> > > > > > > >
> > > > > > > > On Thu, Apr 29, 2021 at 2:04 PM Julie Prestopnik via
RT <
> > > > > > > met_help at ucar.edu
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Unfortunately, I am wrong on all accounts.
> > > > > > > > >
> > > > > > > > > The script did not contain the addition of
> "-L${LIB_LIBPNG}"
> > in
> > > > > > rpath,
> > > > > > > > and
> > > > > > > > > the value of LIB_LIBPNG is set correctly.
> > > > > > > > >
> > > > > > > > > The script currently contains:
> > > > > > > > >
> > > > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
> > -L${MET_HDF5}/lib"
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > but should contain:
> > > > > > > > >
> > > > > > > > > > export LDFLAGS="${LDFLAGS} -L${LIB_JASPER}
> > *-L${LIB_LIBPNG}*
> > > > > > > > > > -L${MET_HDF5}/lib"
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I have added Mallory to this email to see if there
is
> > anything
> > > we
> > > > > can
> > > > > > > do
> > > > > > > > to
> > > > > > > > > get this fixed for the already installed version of
> > met-9.1.3.
> > > > If
> > > > > > not,
> > > > > > > > we
> > > > > > > > > will be releasing met-10.0.0 soon and hopefully the
install
> > > > process
> > > > > > > will
> > > > > > > > go
> > > > > > > > > more quickly since we recently went through it for
9.1.3.
> > > > > > > > >
> > > > > > > > > Mallory, I have made some other additions and
updates to
> the
> > > > > > > > > compile_MET_all.sh script, which I will include for
the
> > release
> > > > of
> > > > > > > > 10.0.0.
> > > > > > > > >
> > > > > > > > > Logan and Mallory, I would also like to test this
addition
> to
> > > the
> > > > > > > > > compile_MET_all.sh script to make sure it solves
this
> problem
> > > > > > > (again). I
> > > > > > > > > plan to update my latest install of met-10.0.0-beta5
to
> test,
> > > > but I
> > > > > > can
> > > > > > > > > also test to get a good install of 9.1.3 if you
would like,
> > > > Logan.
> > > > > > > > Please
> > > > > > > > > let me know what you would prefer.
> > > > > > > > >
> > > > > > > > > Please let me know if you have any questions. My
apologies
> > for
> > > > this
> > > > > > > > > trouble!
> > > > > > > > >
> > > > > > > > > Julie
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Apr 29, 2021 at 12:22 PM Julie Prestopnik <
> > > > > jpresto at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Logan.
> > > > > > > > > >
> > > > > > > > > > Thank you for bringing this problem to our
attention, for
> > > > > reminding
> > > > > > > me
> > > > > > > > of
> > > > > > > > > > this past issue, and for doing the research and
adding
> "see
> > > > > > MET-help
> > > > > > > > > ticket
> > > > > > > > > > #95364" in the email. That was very helpful
information,
> > > and I
> > > > > > > > > appreciate
> > > > > > > > > > your effort!
> > > > > > > > > >
> > > > > > > > > > Can you please send me the command line you are
using and
> > the
> > > > > > > location
> > > > > > > > of
> > > > > > > > > > your data so I can test with the version that
Mallory put
> > > > > together
> > > > > > > for
> > > > > > > > > nco
> > > > > > > > > > before the official release of met-9.1.3?
> > > > > > > > > >
> > > > > > > > > > The script we provided should already put the
value for
> > > LIBPNG
> > > > in
> > > > > > > > > LDFLAGS,
> > > > > > > > > > so I suspect the value may not be set correctly,
but I
> will
> > > > look
> > > > > > into
> > > > > > > > > > that.
> > > > > > > > > >
> > > > > > > > > > I'll follow up once I'm able to run your test case
with
> the
> > > > other
> > > > > > > > version
> > > > > > > > > > and compare to see what's going on.
> > > > > > > > > >
> > > > > > > > > > We definitely want to get this fixed before the
> > installation
> > > of
> > > > > > > > > > met-10.0.0, so thanks for staying on top of this
and
> > alerting
> > > > us
> > > > > > now.
> > > > > > > > > >
> > > > > > > > > > Julie
> > > > > > > > > >
> > > > > > > > > > On Thu, Apr 29, 2021 at 11:39 AM Logan Dawson -
NOAA
> > > Affiliate
> > > > > via
> > > > > > > RT <
> > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > >
> > > > > > > > > >>
> > > > > > > > > >> Thu Apr 29 11:39:14 2021: Request 99740 was acted
upon.
> > > > > > > > > >> Transaction: Ticket created by
logan.dawson at noaa.gov
> > > > > > > > > >> Queue: met_help
> > > > > > > > > >> Subject: Libpng warning with MET-9.1.3 in
> production
> > on
> > > > > WCOSS
> > > > > > > > Dell
> > > > > > > > > >> Owner: Nobody
> > > > > > > > > >> Requestors: logan.dawson at noaa.gov
> > > > > > > > > >> Status: new
> > > > > > > > > >> Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99740
> > > > > > > > >
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> Hello,
> > > > > > > > > >>
> > > > > > > > > >> The MET-9.1.3 installed into operations on WCOSS
Dell (
> > > > > > > > > >> /gpfs/dell1/nco/ops/nwprod/met.v9.1.3/) is
returning a
> > > libpng
> > > > > > > warning
> > > > > > > > > when
> > > > > > > > > >> trying to read MRMS files. Here's an example of
the
> output
> > > > from
> > > > > > > > > >> regrid_data_plane showing that MET reads all
missing
> data
> > > > from a
> > > > > > > MRMS
> > > > > > > > > >> file:
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact
> > match
> > > > for
> > > > > > > > > >> 'MergedReflectivityQCComposite/Z500' in GRIB2
record 1
> > > field 1
> > > > > of
> > > > > > > > GRIB2
> > > > > > > > > >> file
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
'/gpfs/dell2/ptmp/Logan.Dawson/com/MRMS/20210426/MergedReflectivityQCComposite_00.50_20210426-
000000.grib2'
> > > > > > > > > >>
> > > > > > > > > >> *libpng warning: Application built with libpng-
1.2.59
> but
> > > > > running
> > > > > > > with
> > > > > > > > > >> 1.5.13*
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4:
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: Data plane information:
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: plane min: -999
> > > > > > > > > >>
> > > > > > > > > >> *DEBUG 4: plane max: -999*
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: scan mode: 0
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: valid time: 20210426_000034
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: lead time: 000000
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: init time: 20210426_000034
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4: bitmap flag: 255
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 4:
> > > > > > > > > >>
> > > > > > > > > >> *DEBUG 2: Range of input data
> > > > > > (name="MergedReflectivityQCComposite";
> > > > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > > > >>
> > > > > > > > > >> *DEBUG 2: Range of regridded data
> > > > > > > > (name="MergedReflectivityQCComposite";
> > > > > > > > > >> level="Z500";) is -999 to -999.*
> > > > > > > > > >>
> > > > > > > > > >> DEBUG 1: Writing output file:
> > > > > > > > > >>
> > /gpfs/dell2/ptmp/Logan.Dawson/MergedReflectivityQCComposite_
> > > > > > > > > >> 20210426-000000_g227.nc
> > > > > > > > > >>
> > > > > > > > > >> This same issue previously occurred when MET-
9.0.1 was
> > > > installed
> > > > > > on
> > > > > > > > the
> > > > > > > > > >> Dells in May/June 2020, and Julie found a
resolution for
> > the
> > > > > issue
> > > > > > > > > before
> > > > > > > > > >> the MET-9.0.2 installation (see MET-help ticket
#95364).
> > The
> > > > > issue
> > > > > > > was
> > > > > > > > > >> resolved with MET installs in the
noscrub/emc.metplus
> > space
> > > up
> > > > > > > through
> > > > > > > > > >> MET-9.1. However, once this libpng warning issue
was
> > > > identified
> > > > > in
> > > > > > > the
> > > > > > > > > >> nwprod/met.v9.1.3 installation, I found that the
same
> > issue
> > > > > shows
> > > > > > up
> > > > > > > > > with
> > > > > > > > > >> the emc.metplus/met/9.1.1 and
emc.metplus/met/9.1.2
> > > > > installations
> > > > > > on
> > > > > > > > the
> > > > > > > > > >> Dell.
> > > > > > > > > >>
> > > > > > > > > >> It seems as if the resolution from the previous
MET-help
> > > > ticket
> > > > > > > #95364
> > > > > > > > > >> (adding LIBPNG to LDFLAGS in the compilation
script)
> maybe
> > > > > didn't
> > > > > > > > > prevent
> > > > > > > > > >> this problem from resurfacing in MET-9.1.1 and
later
> > > > > > installations?
> > > > > > > It
> > > > > > > > > >> seems that we should take a look to see if we can
find a
> > fix
> > > > > > before
> > > > > > > > > >> MET-10.0 is released and installed on WCOSS.
> > > > > > > > > >>
> > > > > > > > > >> Thanks,
> > > > > > > > > >> Logan
> > > > > > > > > >>
> > > > > > > > > >> --
> > > > > > > > > >> Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > > > > >> Physical Scientist, Model Evaluation Group (VPPPG
> Branch)
> > > > > > > > > >> NOAA/NWS/NCEP/EMC
> > > > > > > > > >> (301) 683-3944
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Julie Prestopnik (she/her)
> > > > > > > > > > Software Engineer
> > > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > > Research Applications Laboratory
> > > > > > > > > > Email: jpresto at ucar.edu
> > > > > > > > > >
> > > > > > > > > > My working day may not be your working day.
Please do
> not
> > > feel
> > > > > > > obliged
> > > > > > > > > to
> > > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Julie Prestopnik (she/her)
> > > > > > > > > Software Engineer
> > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > Research Applications Laboratory
> > > > > > > > > Email: jpresto at ucar.edu
> > > > > > > > >
> > > > > > > > > My working day may not be your working day. Please
do not
> > feel
> > > > > > obliged
> > > > > > > > to
> > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > > > Physical Scientist, Model Evaluation Group (VPPPG
Branch)
> > > > > > > > NOAA/NWS/NCEP/EMC
> > > > > > > > (301) 683-3944
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik (she/her)
> > > > > > > Software Engineer
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > > > NOAA/NWS/NCEP/EMC
> > > > > > (301) 683-3944
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik (she/her)
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > > --
> > > > Logan C. Dawson, Ph.D. (he/him/his)
> > > > Physical Scientist, Model Evaluation Group (VPPPG Branch)
> > > > NOAA/NWS/NCEP/EMC
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> Logan C. Dawson, Ph.D. (he/him/his)
> Physical Scientist, Model Evaluation Group (VPPPG Branch)
> NOAA/NWS/NCEP/EMC
> (301) 683-3944
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list