[Met_help] [rt.rap.ucar.edu #98594] History for HDF5 library version mismatched errors
John Halley Gotway via RT
met_help at ucar.edu
Mon Feb 8 17:34:55 MST 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm getting HDF5 library version mismatched errors when I try to run MET in
a conda environment with h5py (h5py is running against HDF5 1.8.18 when it
was built against 1.10.6, this may cause problems). Does MET have to be
recompiled with a newer version of h5 or is there a way to tell MET to use
the h5 in the conda environment? How is this handled in the container
build?
Thanks,
Liz
Elizabeth Satterfield
Editor, Monthly Weather Review
Meteorologist, Code 7531
Marine Meteorology Division
U.S. Naval Research Laboratory
T 831.656.4056 F 831.656.4026
<http://www.nrl.navy.mil/> www.nrl.navy.mil
__________________________________________________________________
FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
Any misuse or unauthorized disclosure may result in both civil and criminal
penalties.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: HDF5 library version mismatched errors
From: John Halley Gotway
Time: Mon Feb 08 14:16:58 2021
Liz,
Ah yes, version-itis of the HDF5 library. The NetCDF4 library is built
on
top of HDF5, and MET is compiled against it at runtime.
But that version may differ from the version used to compile the h5py
python package. And those environment settings can lead to the types
of
warnings or errors you describe.
I don't have a clear and definite answer for you here, just a
collection of
thoughts.
Julie and George, if you have any advice here, please chime in!
First, if you don't have MET_PYTHON_EXE set and pointing to the python
binary you want to execute, please try that. Any luck?
You can read more about that here:
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met-
python-exe
Next, I don't think there's a way to tell MET to "use" different HDF5
library versions, where the compiled version differs from the runtime
version. That's the source of the warning... the compilation version
differs from the runtime version.
I would be interested to see HOW your version of MET was compiled. A
couple
years ago, we got advice from our sys admins on compiling MET, trying
to
make the executables NOT require specific LD_LIBRARY_PATH settings.
That's
what the "-rpath" settings for LDFLAGS do starting on this line:
https://github.com/dtcenter/MET/blob/develop/scripts/installation/compile_MET_all.sh#L741
So I wonder if MET had been compiled using "-rpath" in this way, if
we'd
still have this HDF5 problem at runtime?
Here's the Dockerfile which compiles MET:
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfile
And the METplus Docker image is built on top of the MET one. George,
any
idea if we're using the METplus Docker image to run python embedding
for
MET where the python script uses the h5py module? And if so, if there
are
any library version issues?
Thanks,
John
On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
> Transaction: Ticket created by Elizabeth.Satterfield at nrlmry.navy.mil
> Queue: met_help
> Subject: HDF5 library version mismatched errors
> Owner: Nobody
> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594 >
>
>
> Hi,
>
>
>
> I'm getting HDF5 library version mismatched errors when I try to run
MET in
> a conda environment with h5py (h5py is running against HDF5 1.8.18
when it
> was built against 1.10.6, this may cause problems). Does MET have
to be
> recompiled with a newer version of h5 or is there a way to tell MET
to use
> the h5 in the conda environment? How is this handled in the
container
> build?
>
>
>
> Thanks,
>
> Liz
>
>
>
>
>
>
>
>
>
>
> Elizabeth Satterfield
> Editor, Monthly Weather Review
>
> Meteorologist, Code 7531
> Marine Meteorology Division
> U.S. Naval Research Laboratory
> T 831.656.4056 F 831.656.4026
> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>
> __________________________________________________________________
> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
> Any misuse or unauthorized disclosure may result in both civil and
criminal
> penalties.
>
>
>
>
>
>
>
>
>
------------------------------------------------
Subject: HDF5 library version mismatched errors
From: George McCabe
Time: Mon Feb 08 15:33:43 2021
Hi All,
In the automated tests, we do have a use case that uses the h5py
python
library. We run in the METplus Docker image, which has all of the MET
tools
built. Then we install the h5py python library using pip3 and use
MET_PYTHON_EXE to use the python binary that has the h5py package
installed.
On Mon, Feb 8, 2021 at 2:16 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Liz,
>
> Ah yes, version-itis of the HDF5 library. The NetCDF4 library is
built on
> top of HDF5, and MET is compiled against it at runtime.
> But that version may differ from the version used to compile the
h5py
> python package. And those environment settings can lead to the types
of
> warnings or errors you describe.
>
> I don't have a clear and definite answer for you here, just a
collection
> of thoughts.
>
> Julie and George, if you have any advice here, please chime in!
>
> First, if you don't have MET_PYTHON_EXE set and pointing to the
python
> binary you want to execute, please try that. Any luck?
> You can read more about that here:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met-
python-exe
>
> Next, I don't think there's a way to tell MET to "use" different
HDF5
> library versions, where the compiled version differs from the
runtime
> version. That's the source of the warning... the compilation version
> differs from the runtime version.
>
> I would be interested to see HOW your version of MET was compiled. A
> couple years ago, we got advice from our sys admins on compiling
MET,
> trying to make the executables NOT require specific LD_LIBRARY_PATH
> settings. That's what the "-rpath" settings for LDFLAGS do starting
on this
> line:
>
>
https://github.com/dtcenter/MET/blob/develop/scripts/installation/compile_MET_all.sh#L741
>
> So I wonder if MET had been compiled using "-rpath" in this way, if
we'd
> still have this HDF5 problem at runtime?
>
> Here's the Dockerfile which compiles MET:
>
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfile
>
> And the METplus Docker image is built on top of the MET one. George,
any
> idea if we're using the METplus Docker image to run python embedding
for
> MET where the python script uses the h5py module? And if so, if
there are
> any library version issues?
>
> Thanks,
> John
>
> On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
>> Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
>> Queue: met_help
>> Subject: HDF5 library version mismatched errors
>> Owner: Nobody
>> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594 >
>>
>>
>> Hi,
>>
>>
>>
>> I'm getting HDF5 library version mismatched errors when I try to
run MET
>> in
>> a conda environment with h5py (h5py is running against HDF5 1.8.18
when
>> it
>> was built against 1.10.6, this may cause problems). Does MET have
to be
>> recompiled with a newer version of h5 or is there a way to tell MET
to use
>> the h5 in the conda environment? How is this handled in the
container
>> build?
>>
>>
>>
>> Thanks,
>>
>> Liz
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Elizabeth Satterfield
>> Editor, Monthly Weather Review
>>
>> Meteorologist, Code 7531
>> Marine Meteorology Division
>> U.S. Naval Research Laboratory
>> T 831.656.4056 F 831.656.4026
>> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>>
>> __________________________________________________________________
>> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
>> Any misuse or unauthorized disclosure may result in both civil and
>> criminal
>> penalties.
>>
>>
>>
>>
>>
>>
>>
>>
>>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: HDF5 library version mismatched errors
From: Julie Prestopnik
Time: Mon Feb 08 15:36:30 2021
Unfortunately, I don't have anything to add here. I was going to
suggest
setting MET_PYTHON_EXE as well. Hopefully, that'll do the trick.
If not, compiling with rpath could be considered as well. I'd also be
interested in knowing if MET had been compiled using "-rpath" in this
way,
if we'd still have this HDF5 problem at runtime.
Julie
I
On Mon, Feb 8, 2021 at 3:33 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi All,
>
> In the automated tests, we do have a use case that uses the h5py
python
> library. We run in the METplus Docker image, which has all of the
MET tools
> built. Then we install the h5py python library using pip3 and use
> MET_PYTHON_EXE to use the python binary that has the h5py package
installed.
>
> On Mon, Feb 8, 2021 at 2:16 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
>> Liz,
>>
>> Ah yes, version-itis of the HDF5 library. The NetCDF4 library is
built on
>> top of HDF5, and MET is compiled against it at runtime.
>> But that version may differ from the version used to compile the
h5py
>> python package. And those environment settings can lead to the
types of
>> warnings or errors you describe.
>>
>> I don't have a clear and definite answer for you here, just a
collection
>> of thoughts.
>>
>> Julie and George, if you have any advice here, please chime in!
>>
>> First, if you don't have MET_PYTHON_EXE set and pointing to the
python
>> binary you want to execute, please try that. Any luck?
>> You can read more about that here:
>>
>>
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met-
python-exe
>>
>> Next, I don't think there's a way to tell MET to "use" different
HDF5
>> library versions, where the compiled version differs from the
runtime
>> version. That's the source of the warning... the compilation
version
>> differs from the runtime version.
>>
>> I would be interested to see HOW your version of MET was compiled.
A
>> couple years ago, we got advice from our sys admins on compiling
MET,
>> trying to make the executables NOT require specific LD_LIBRARY_PATH
>> settings. That's what the "-rpath" settings for LDFLAGS do starting
on this
>> line:
>>
>>
https://github.com/dtcenter/MET/blob/develop/scripts/installation/compile_MET_all.sh#L741
>>
>> So I wonder if MET had been compiled using "-rpath" in this way, if
we'd
>> still have this HDF5 problem at runtime?
>>
>> Here's the Dockerfile which compiles MET:
>>
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfile
>>
>> And the METplus Docker image is built on top of the MET one.
George, any
>> idea if we're using the METplus Docker image to run python
embedding for
>> MET where the python script uses the h5py module? And if so, if
there are
>> any library version issues?
>>
>> Thanks,
>> John
>>
>> On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT <
>> met_help at ucar.edu> wrote:
>>
>>>
>>> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
>>> Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
>>> Queue: met_help
>>> Subject: HDF5 library version mismatched errors
>>> Owner: Nobody
>>> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594 >
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I'm getting HDF5 library version mismatched errors when I try to
run MET
>>> in
>>> a conda environment with h5py (h5py is running against HDF5
1.8.18 when
>>> it
>>> was built against 1.10.6, this may cause problems). Does MET have
to be
>>> recompiled with a newer version of h5 or is there a way to tell
MET to
>>> use
>>> the h5 in the conda environment? How is this handled in the
container
>>> build?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Liz
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Elizabeth Satterfield
>>> Editor, Monthly Weather Review
>>>
>>> Meteorologist, Code 7531
>>> Marine Meteorology Division
>>> U.S. Naval Research Laboratory
>>> T 831.656.4056 F 831.656.4026
>>> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>>>
>>> __________________________________________________________________
>>> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE
>>> Any misuse or unauthorized disclosure may result in both civil and
>>> criminal
>>> penalties.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: HDF5 library version mismatched errors
From: Satterfield, Dr. Elizabeth
Time: Mon Feb 08 15:37:10 2021
Thanks, all
Setting MET_PYTHON_EXE seemed to fix the issue both for container and
non-container
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: Monday, February 8, 2021 2:34 PM
To: Satterfield, Dr. Elizabeth <Elizabeth.Satterfield at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98594] HDF5 library version mismatched
errors
Hi All,
In the automated tests, we do have a use case that uses the h5py
python
library. We run in the METplus Docker image, which has all of the MET
tools
built. Then we install the h5py python library using pip3 and use
MET_PYTHON_EXE to use the python binary that has the h5py package
installed.
On Mon, Feb 8, 2021 at 2:16 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Liz,
>
> Ah yes, version-itis of the HDF5 library. The NetCDF4 library is
built
> on top of HDF5, and MET is compiled against it at runtime.
> But that version may differ from the version used to compile the
h5py
> python package. And those environment settings can lead to the types
> of warnings or errors you describe.
>
> I don't have a clear and definite answer for you here, just a
> collection of thoughts.
>
> Julie and George, if you have any advice here, please chime in!
>
> First, if you don't have MET_PYTHON_EXE set and pointing to the
python
> binary you want to execute, please try that. Any luck?
> You can read more about that here:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met-p
> ython-exe
>
> Next, I don't think there's a way to tell MET to "use" different
HDF5
> library versions, where the compiled version differs from the
runtime
> version. That's the source of the warning... the compilation version
> differs from the runtime version.
>
> I would be interested to see HOW your version of MET was compiled. A
> couple years ago, we got advice from our sys admins on compiling
MET,
> trying to make the executables NOT require specific LD_LIBRARY_PATH
> settings. That's what the "-rpath" settings for LDFLAGS do starting
on
> this
> line:
>
>
https://github.com/dtcenter/MET/blob/develop/scripts/installation/comp
> ile_MET_all.sh#L741
>
> So I wonder if MET had been compiled using "-rpath" in this way, if
> we'd still have this HDF5 problem at runtime?
>
> Here's the Dockerfile which compiles MET:
>
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfile
>
> And the METplus Docker image is built on top of the MET one. George,
> any idea if we're using the METplus Docker image to run python
> embedding for MET where the python script uses the h5py module? And
if
> so, if there are any library version issues?
>
> Thanks,
> John
>
> On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
>> Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
>> Queue: met_help
>> Subject: HDF5 library version mismatched errors
>> Owner: Nobody
>> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594
>> >
>>
>>
>> Hi,
>>
>>
>>
>> I'm getting HDF5 library version mismatched errors when I try to
run
>> MET in a conda environment with h5py (h5py is running against HDF5
>> 1.8.18 when it was built against 1.10.6, this may cause problems).
>> Does MET have to be recompiled with a newer version of h5 or is
there
>> a way to tell MET to use the h5 in the conda environment? How is
>> this handled in the container build?
>>
>>
>>
>> Thanks,
>>
>> Liz
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Elizabeth Satterfield
>> Editor, Monthly Weather Review
>>
>> Meteorologist, Code 7531
>> Marine Meteorology Division
>> U.S. Naval Research Laboratory
>> T 831.656.4056 F 831.656.4026
>> <http://www.nrl.navy.mil/> www.nrl.navy.mil
>>
>> __________________________________________________________________
>> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE Any misuse or
unauthorized
>> disclosure may result in both civil and criminal penalties.
>>
>>
>>
>>
>>
>>
>>
>>
>>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: HDF5 library version mismatched errors
From: John Halley Gotway
Time: Mon Feb 08 16:55:41 2021
Excellent! Thanks for confirming. I’ll go ahead and resolve this
ticket.
John
On Mon, Feb 8, 2021 at 3:37 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594 >
>
> Thanks, all
> Setting MET_PYTHON_EXE seemed to fix the issue both for container
and
> non-container
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: Monday, February 8, 2021 2:34 PM
> To: Satterfield, Dr. Elizabeth
<Elizabeth.Satterfield at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98594] HDF5 library version
mismatched
> errors
>
> Hi All,
>
> In the automated tests, we do have a use case that uses the h5py
python
> library. We run in the METplus Docker image, which has all of the
MET
> tools
> built. Then we install the h5py python library using pip3 and use
> MET_PYTHON_EXE to use the python binary that has the h5py package
> installed.
>
> On Mon, Feb 8, 2021 at 2:16 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > Liz,
> >
> > Ah yes, version-itis of the HDF5 library. The NetCDF4 library is
built
> > on top of HDF5, and MET is compiled against it at runtime.
> > But that version may differ from the version used to compile the
h5py
> > python package. And those environment settings can lead to the
types
> > of warnings or errors you describe.
> >
> > I don't have a clear and definite answer for you here, just a
> > collection of thoughts.
> >
> > Julie and George, if you have any advice here, please chime in!
> >
> > First, if you don't have MET_PYTHON_EXE set and pointing to the
python
> > binary you want to execute, please try that. Any luck?
> > You can read more about that here:
> >
> >
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met-p
> > ython-exe
> >
> > Next, I don't think there's a way to tell MET to "use" different
HDF5
> > library versions, where the compiled version differs from the
runtime
> > version. That's the source of the warning... the compilation
version
> > differs from the runtime version.
> >
> > I would be interested to see HOW your version of MET was compiled.
A
> > couple years ago, we got advice from our sys admins on compiling
MET,
> > trying to make the executables NOT require specific
LD_LIBRARY_PATH
> > settings. That's what the "-rpath" settings for LDFLAGS do
starting on
> > this
> > line:
> >
> >
https://github.com/dtcenter/MET/blob/develop/scripts/installation/comp
> > ile_MET_all.sh#L741
> >
> > So I wonder if MET had been compiled using "-rpath" in this way,
if
> > we'd still have this HDF5 problem at runtime?
> >
> > Here's the Dockerfile which compiles MET:
> >
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfile
> >
> > And the METplus Docker image is built on top of the MET one.
George,
> > any idea if we're using the METplus Docker image to run python
> > embedding for MET where the python script uses the h5py module?
And if
> > so, if there are any library version issues?
> >
> > Thanks,
> > John
> >
> > On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT
<
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
> >> Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
> >> Queue: met_help
> >> Subject: HDF5 library version mismatched errors
> >> Owner: Nobody
> >> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594
> >> >
> >>
> >>
> >> Hi,
> >>
> >>
> >>
> >> I'm getting HDF5 library version mismatched errors when I try to
run
> >> MET in a conda environment with h5py (h5py is running against
HDF5
> >> 1.8.18 when it was built against 1.10.6, this may cause
problems).
> >> Does MET have to be recompiled with a newer version of h5 or is
there
> >> a way to tell MET to use the h5 in the conda environment? How is
> >> this handled in the container build?
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Liz
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Elizabeth Satterfield
> >> Editor, Monthly Weather Review
> >>
> >> Meteorologist, Code 7531
> >> Marine Meteorology Division
> >> U.S. Naval Research Laboratory
> >> T 831.656.4056 F 831.656.4026
> >> <http://www.nrl.navy.mil/> www.nrl.navy.mil
> >>
> >>
__________________________________________________________________
> >> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE Any misuse or
unauthorized
> >> disclosure may result in both civil and criminal penalties.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
------------------------------------------------
Subject: HDF5 library version mismatched errors
From: Satterfield, Dr. Elizabeth
Time: Mon Feb 08 16:58:35 2021
Thanks for your help!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 8, 2021 3:56 PM
To: Satterfield, Dr. Elizabeth <Elizabeth.Satterfield at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98594] HDF5 library version mismatched
errors
Excellent! Thanks for confirming. I’ll go ahead and resolve this
ticket.
John
On Mon, Feb 8, 2021 at 3:37 PM Satterfield, Dr. Elizabeth via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594 >
>
> Thanks, all
> Setting MET_PYTHON_EXE seemed to fix the issue both for container
and
> non-container
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: Monday, February 8, 2021 2:34 PM
> To: Satterfield, Dr. Elizabeth
<Elizabeth.Satterfield at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98594] HDF5 library version
mismatched
> errors
>
> Hi All,
>
> In the automated tests, we do have a use case that uses the h5py
> python library. We run in the METplus Docker image, which has all of
> the MET tools built. Then we install the h5py python library using
> pip3 and use MET_PYTHON_EXE to use the python binary that has the
h5py
> package installed.
>
> On Mon, Feb 8, 2021 at 2:16 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > Liz,
> >
> > Ah yes, version-itis of the HDF5 library. The NetCDF4 library is
> > built on top of HDF5, and MET is compiled against it at runtime.
> > But that version may differ from the version used to compile the
> > h5py python package. And those environment settings can lead to
the
> > types of warnings or errors you describe.
> >
> > I don't have a clear and definite answer for you here, just a
> > collection of thoughts.
> >
> > Julie and George, if you have any advice here, please chime in!
> >
> > First, if you don't have MET_PYTHON_EXE set and pointing to the
> > python binary you want to execute, please try that. Any luck?
> > You can read more about that here:
> >
> >
https://dtcenter.github.io/MET/latest/Users_Guide/appendixF.html#met
> > -p
> > ython-exe
> >
> > Next, I don't think there's a way to tell MET to "use" different
> > HDF5 library versions, where the compiled version differs from the
> > runtime version. That's the source of the warning... the
compilation
> > version differs from the runtime version.
> >
> > I would be interested to see HOW your version of MET was compiled.
A
> > couple years ago, we got advice from our sys admins on compiling
> > MET, trying to make the executables NOT require specific
> > LD_LIBRARY_PATH settings. That's what the "-rpath" settings for
> > LDFLAGS do starting on this
> > line:
> >
> >
https://github.com/dtcenter/MET/blob/develop/scripts/installation/co
> > mp
> > ile_MET_all.sh#L741
> >
> > So I wonder if MET had been compiled using "-rpath" in this way,
if
> > we'd still have this HDF5 problem at runtime?
> >
> > Here's the Dockerfile which compiles MET:
> >
https://github.com/dtcenter/MET/blob/develop/scripts/docker/Dockerfi
> > le
> >
> > And the METplus Docker image is built on top of the MET one.
George,
> > any idea if we're using the METplus Docker image to run python
> > embedding for MET where the python script uses the h5py module?
And
> > if so, if there are any library version issues?
> >
> > Thanks,
> > John
> >
> > On Mon, Feb 8, 2021 at 12:19 PM Satterfield, Dr. Elizabeth via RT
<
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Mon Feb 08 12:18:45 2021: Request 98594 was acted upon.
> >> Transaction: Ticket created by
Elizabeth.Satterfield at nrlmry.navy.mil
> >> Queue: met_help
> >> Subject: HDF5 library version mismatched errors
> >> Owner: Nobody
> >> Requestors: Elizabeth.Satterfield at nrlmry.navy.mil
> >> Status: new
> >> Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98594
> >> >
> >>
> >>
> >> Hi,
> >>
> >>
> >>
> >> I'm getting HDF5 library version mismatched errors when I try to
> >> run MET in a conda environment with h5py (h5py is running
against
> >> HDF5
> >> 1.8.18 when it was built against 1.10.6, this may cause
problems).
> >> Does MET have to be recompiled with a newer version of h5 or is
> >> there a way to tell MET to use the h5 in the conda environment?
> >> How is this handled in the container build?
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Liz
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Elizabeth Satterfield
> >> Editor, Monthly Weather Review
> >>
> >> Meteorologist, Code 7531
> >> Marine Meteorology Division
> >> U.S. Naval Research Laboratory
> >> T 831.656.4056 F 831.656.4026
> >> <http://www.nrl.navy.mil/> www.nrl.navy.mil
> >>
> >>
__________________________________________________________________
> >> FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE Any misuse or
> >> unauthorized disclosure may result in both civil and criminal
penalties.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
------------------------------------------------
More information about the Met_help
mailing list