[Met_help] [rt.rap.ucar.edu #96467] History for Python embedding help
John Halley Gotway via RT
met_help at ucar.edu
Thu Aug 27 14:01:55 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hey John,
I have a question about python embedding. In my python script, I have the following:
60 tokens = os.path.basename(input_file).replace('-', '_').split('_');
61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
62 data = np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
63 met_data = np.reshape(data,(dimy,dimx)).copy()
Where input file is some ieee64 binary /DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld. Running this script produces this:
In [1]: met_data
Out[1]:
array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
299.79425],
[298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
299.79425],
[298.68155, 298.68155, 298.9081 , ..., 299.3087 , 299.1046 ,
299.1046 ],
...,
[305.22366, 305.22366, 305.1536 , ..., 296.87302, 297.37988,
297.37988],
[304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
295.63745],
[304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
295.63745]], dtype=float32)
Met_data is an array full of sensical values. However, when I run regrid_data_plane on the same file:
[tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator 127 127 2.028 248.843 49.555 304.033' out.nc -field 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py /DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
DEBUG 1: Reading data file: PYTHON_NUMPY
DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127 Lat_LL_radians: 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians: 0.9768 Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127 Lat_LL_radians: 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians: 0.9768 Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
DEBUG 2: Interpolation options: method = NEAREST, width = 1, shape = SQUARE, vld_thresh = 0.5
DEBUG 2: Range of input data (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py /DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";) is -1.66213e+308 to 8.59046e+307.
DEBUG 2: Range of regridded data (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py /DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";) is -1.66213e+308 to 8.59046e+307.
DEBUG 1: Writing output file: out.nc
>From the highlighted portion, we can see that the met_data matrix no longer contains sensical values but instead has been replaced with very large Inf values. What is going on? What is python embedding expecting from the python script?
Justin
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Python embedding help
From: George McCabe
Time: Wed Aug 26 16:25:31 2020
Hi Justin,
I see you are having issues with python embedding. I'm going to assign
this
ticket to John HG since he has more experience with python embedding
than I
do.
One thing I noticed is your numpy array has a data type of float32.
The
examples I have seen use float64, so this could be causing an issue. I
would try that to see if it helps.
If that doesn't help, then John will likely have another suggestion.
Thanks,
George
On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> Queue: met_help
> Subject: Python embedding help
> Owner: Nobody
> Requestors: justin.tsu at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
>
> Hey John,
>
> I have a question about python embedding. In my python script, I
have the
> following:
>
> 60 tokens = os.path.basename(input_file).replace('-',
'_').split('_');
> 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> 62 data = np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> 63 met_data = np.reshape(data,(dimy,dimx)).copy()
>
> Where input file is some ieee64 binary
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> Running this script produces this:
>
> In [1]: met_data
> Out[1]:
> array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
> 299.79425],
> [298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
> 299.79425],
> [298.68155, 298.68155, 298.9081 , ..., 299.3087 , 299.1046 ,
> 299.1046 ],
> ...,
> [305.22366, 305.22366, 305.1536 , ..., 296.87302, 297.37988,
> 297.37988],
> [304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
> 295.63745],
> [304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
> 295.63745]], dtype=float32)
>
> Met_data is an array full of sensical values. However, when I run
> regrid_data_plane on the same file:
> [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator 127
127
> 2.028 248.843 49.555 304.033' out.nc -field
> 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> DEBUG 1: Reading data file: PYTHON_NUMPY
> DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians:
0.9768
> Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians:
0.9768
> Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> DEBUG 2: Interpolation options: method = NEAREST, width = 1, shape =
> SQUARE, vld_thresh = 0.5
> DEBUG 2: Range of input data
> (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> is -1.66213e+308 to 8.59046e+307.
> DEBUG 2: Range of regridded data
> (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> is -1.66213e+308 to 8.59046e+307.
> DEBUG 1: Writing output file: out.nc
>
> From the highlighted portion, we can see that the met_data matrix no
> longer contains sensical values but instead has been replaced with
very
> large Inf values. What is going on? What is python embedding
expecting
> from the python script?
>
> Justin
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96467] Python embedding help
From: Tsu, Mr. Justin
Time: Thu Aug 27 11:26:54 2020
Thanks George,
I figured it out right after I sent you that. All is working now.
Justin
-----Original Message-----
From: George McCabe via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, August 26, 2020 3:26 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
Hi Justin,
I see you are having issues with python embedding. I'm going to assign
this
ticket to John HG since he has more experience with python embedding
than I
do.
One thing I noticed is your numpy array has a data type of float32.
The
examples I have seen use float64, so this could be causing an issue. I
would try that to see if it helps.
If that doesn't help, then John will likely have another suggestion.
Thanks,
George
On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> Queue: met_help
> Subject: Python embedding help
> Owner: Nobody
> Requestors: justin.tsu at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
>
> Hey John,
>
> I have a question about python embedding. In my python script, I
have the
> following:
>
> 60 tokens = os.path.basename(input_file).replace('-',
'_').split('_');
> 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> 62 data = np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> 63 met_data = np.reshape(data,(dimy,dimx)).copy()
>
> Where input file is some ieee64 binary
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> Running this script produces this:
>
> In [1]: met_data
> Out[1]:
> array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
> 299.79425],
> [298.2424 , 298.2424 , 297.98398, ..., 299.6279 , 299.79425,
> 299.79425],
> [298.68155, 298.68155, 298.9081 , ..., 299.3087 , 299.1046 ,
> 299.1046 ],
> ...,
> [305.22366, 305.22366, 305.1536 , ..., 296.87302, 297.37988,
> 297.37988],
> [304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
> 295.63745],
> [304.95667, 304.95667, 304.9788 , ..., 296.63263, 295.63745,
> 295.63745]], dtype=float32)
>
> Met_data is an array full of sensical values. However, when I run
> regrid_data_plane on the same file:
> [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator 127
127
> 2.028 248.843 49.555 304.033' out.nc -field
> 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> DEBUG 1: Reading data file: PYTHON_NUMPY
> DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians:
0.9768
> Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649 Lon_UR_radians:
0.9768
> Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> DEBUG 2: Interpolation options: method = NEAREST, width = 1, shape =
> SQUARE, vld_thresh = 0.5
> DEBUG 2: Range of input data
> (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> is -1.66213e+308 to 8.59046e+307.
> DEBUG 2: Range of regridded data
> (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> is -1.66213e+308 to 8.59046e+307.
> DEBUG 1: Writing output file: out.nc
>
> From the highlighted portion, we can see that the met_data matrix no
> longer contains sensical values but instead has been replaced with
very
> large Inf values. What is going on? What is python embedding
expecting
> from the python script?
>
> Justin
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Python embedding help
From: George McCabe
Time: Thu Aug 27 11:29:04 2020
Great to hear! I'm resolving this issue now.
- George
On Thu, Aug 27, 2020 at 11:26 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
> Thanks George,
>
> I figured it out right after I sent you that. All is working now.
>
> Justin
>
> -----Original Message-----
> From: George McCabe via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, August 26, 2020 3:26 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
>
> Hi Justin,
>
> I see you are having issues with python embedding. I'm going to
assign this
> ticket to John HG since he has more experience with python embedding
than I
> do.
>
> One thing I noticed is your numpy array has a data type of float32.
The
> examples I have seen use float64, so this could be causing an issue.
I
> would try that to see if it helps.
>
> If that doesn't help, then John will likely have another suggestion.
>
> Thanks,
> George
>
> On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Python embedding help
> > Owner: Nobody
> > Requestors: justin.tsu at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
> >
> >
> > Hey John,
> >
> > I have a question about python embedding. In my python script, I
have
> the
> > following:
> >
> > 60 tokens = os.path.basename(input_file).replace('-',
> '_').split('_');
> > 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> > 62 data = np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> > 63 met_data = np.reshape(data,(dimy,dimx)).copy()
> >
> > Where input file is some ieee64 binary
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> > Running this script produces this:
> >
> > In [1]: met_data
> > Out[1]:
> > array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > 299.79425],
> > [298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > 299.79425],
> > [298.68155, 298.68155, 298.9081 , ..., 299.3087 , 299.1046
,
> > 299.1046 ],
> > ...,
> > [305.22366, 305.22366, 305.1536 , ..., 296.87302,
297.37988,
> > 297.37988],
> > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > 295.63745],
> > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > 295.63745]], dtype=float32)
> >
> > Met_data is an array full of sensical values. However, when I run
> > regrid_data_plane on the same file:
> > [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator 127
127
> > 2.028 248.843 49.555 304.033' out.nc -field
> > 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> > DEBUG 1: Reading data file: PYTHON_NUMPY
> > DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> 0.9768
> > Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> > DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
> Lat_LL_radians:
> > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> 0.9768
> > Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> > DEBUG 2: Interpolation options: method = NEAREST, width = 1, shape
=
> > SQUARE, vld_thresh = 0.5
> > DEBUG 2: Range of input data
> > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > is -1.66213e+308 to 8.59046e+307.
> > DEBUG 2: Range of regridded data
> > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > is -1.66213e+308 to 8.59046e+307.
> > DEBUG 1: Writing output file: out.nc
> >
> > From the highlighted portion, we can see that the met_data matrix
no
> > longer contains sensical values but instead has been replaced with
very
> > large Inf values. What is going on? What is python embedding
expecting
> > from the python script?
> >
> > Justin
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96467] Python embedding help
From: Tsu, Mr. Justin
Time: Thu Aug 27 11:32:44 2020
Oh wait I do have one more unrelated question about point_stat that I
came up with just now!
When I run point_stat to output CNT files, I notice that there is a
single statistic line for each tau and variable. Is there any
information in these CNT lines that tell me how many matched pairs
were used to calculate these stats?
Justin
-----Original Message-----
From: George McCabe via RT [mailto:met_help at ucar.edu]
Sent: Thursday, August 27, 2020 10:29 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
Great to hear! I'm resolving this issue now.
- George
On Thu, Aug 27, 2020 at 11:26 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
> Thanks George,
>
> I figured it out right after I sent you that. All is working now.
>
> Justin
>
> -----Original Message-----
> From: George McCabe via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, August 26, 2020 3:26 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
>
> Hi Justin,
>
> I see you are having issues with python embedding. I'm going to
assign this
> ticket to John HG since he has more experience with python embedding
than I
> do.
>
> One thing I noticed is your numpy array has a data type of float32.
The
> examples I have seen use float64, so this could be causing an issue.
I
> would try that to see if it helps.
>
> If that doesn't help, then John will likely have another suggestion.
>
> Thanks,
> George
>
> On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Python embedding help
> > Owner: Nobody
> > Requestors: justin.tsu at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
> >
> >
> > Hey John,
> >
> > I have a question about python embedding. In my python script, I
have
> the
> > following:
> >
> > 60 tokens = os.path.basename(input_file).replace('-',
> '_').split('_');
> > 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> > 62 data = np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> > 63 met_data = np.reshape(data,(dimy,dimx)).copy()
> >
> > Where input file is some ieee64 binary
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> > Running this script produces this:
> >
> > In [1]: met_data
> > Out[1]:
> > array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > 299.79425],
> > [298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > 299.79425],
> > [298.68155, 298.68155, 298.9081 , ..., 299.3087 , 299.1046
,
> > 299.1046 ],
> > ...,
> > [305.22366, 305.22366, 305.1536 , ..., 296.87302,
297.37988,
> > 297.37988],
> > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > 295.63745],
> > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > 295.63745]], dtype=float32)
> >
> > Met_data is an array full of sensical values. However, when I run
> > regrid_data_plane on the same file:
> > [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator 127
127
> > 2.028 248.843 49.555 304.033' out.nc -field
> > 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> > DEBUG 1: Reading data file: PYTHON_NUMPY
> > DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians:
> > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> 0.9768
> > Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> > DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
> Lat_LL_radians:
> > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> 0.9768
> > Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> > DEBUG 2: Interpolation options: method = NEAREST, width = 1, shape
=
> > SQUARE, vld_thresh = 0.5
> > DEBUG 2: Range of input data
> > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > is -1.66213e+308 to 8.59046e+307.
> > DEBUG 2: Range of regridded data
> > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > is -1.66213e+308 to 8.59046e+307.
> > DEBUG 1: Writing output file: out.nc
> >
> > From the highlighted portion, we can see that the met_data matrix
no
> > longer contains sensical values but instead has been replaced with
very
> > large Inf values. What is going on? What is python embedding
expecting
> > from the python script?
> >
> > Justin
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Python embedding help
From: John Halley Gotway
Time: Thu Aug 27 12:22:15 2020
Justin,
The contents of the CNT line type are described in this table:
https://dtcenter.github.io/MET/Users_Guide/point-
stat.html?highlight=cnt%20line%20type#id10
Note that column number 25, for TOTAL, lists the total number of
matched
pairs.
The TOTAL column is included in almost all of the line types that MET
writes and consistently indicates the number of matched pairs.
Thanks,
John
On Thu, Aug 27, 2020 at 11:32 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
> Oh wait I do have one more unrelated question about point_stat that
I came
> up with just now!
>
> When I run point_stat to output CNT files, I notice that there is a
single
> statistic line for each tau and variable. Is there any information
in
> these CNT lines that tell me how many matched pairs were used to
calculate
> these stats?
>
> Justin
>
> -----Original Message-----
> From: George McCabe via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, August 27, 2020 10:29 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
>
> Great to hear! I'm resolving this issue now.
>
> - George
>
> On Thu, Aug 27, 2020 at 11:26 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
> >
> > Thanks George,
> >
> > I figured it out right after I sent you that. All is working now.
> >
> > Justin
> >
> > -----Original Message-----
> > From: George McCabe via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, August 26, 2020 3:26 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
> >
> > Hi Justin,
> >
> > I see you are having issues with python embedding. I'm going to
assign
> this
> > ticket to John HG since he has more experience with python
embedding
> than I
> > do.
> >
> > One thing I noticed is your numpy array has a data type of
float32. The
> > examples I have seen use float64, so this could be causing an
issue. I
> > would try that to see if it helps.
> >
> > If that doesn't help, then John will likely have another
suggestion.
> >
> > Thanks,
> > George
> >
> > On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Python embedding help
> > > Owner: Nobody
> > > Requestors: justin.tsu at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467
> >
> > >
> > >
> > > Hey John,
> > >
> > > I have a question about python embedding. In my python script,
I have
> > the
> > > following:
> > >
> > > 60 tokens = os.path.basename(input_file).replace('-',
> > '_').split('_');
> > > 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> > > 62 data =
np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> > > 63 met_data = np.reshape(data,(dimy,dimx)).copy()
> > >
> > > Where input file is some ieee64 binary
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> > > Running this script produces this:
> > >
> > > In [1]: met_data
> > > Out[1]:
> > > array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > > 299.79425],
> > > [298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > > 299.79425],
> > > [298.68155, 298.68155, 298.9081 , ..., 299.3087 ,
299.1046 ,
> > > 299.1046 ],
> > > ...,
> > > [305.22366, 305.22366, 305.1536 , ..., 296.87302,
297.37988,
> > > 297.37988],
> > > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > > 295.63745],
> > > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > > 295.63745]], dtype=float32)
> > >
> > > Met_data is an array full of sensical values. However, when I
run
> > > regrid_data_plane on the same file:
> > > [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator
127 127
> > > 2.028 248.843 49.555 304.033' out.nc -field
> > > 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> > > DEBUG 1: Reading data file: PYTHON_NUMPY
> > > DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
> Lat_LL_radians:
> > > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> > 0.9768
> > > Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> > > DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
> > Lat_LL_radians:
> > > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> > 0.9768
> > > Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> > > DEBUG 2: Interpolation options: method = NEAREST, width = 1,
shape =
> > > SQUARE, vld_thresh = 0.5
> > > DEBUG 2: Range of input data
> > > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > > is -1.66213e+308 to 8.59046e+307.
> > > DEBUG 2: Range of regridded data
> > > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > > is -1.66213e+308 to 8.59046e+307.
> > > DEBUG 1: Writing output file: out.nc
> > >
> > > From the highlighted portion, we can see that the met_data
matrix no
> > > longer contains sensical values but instead has been replaced
with very
> > > large Inf values. What is going on? What is python embedding
expecting
> > > from the python script?
> > >
> > > Justin
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96467] Python embedding help
From: Tsu, Mr. Justin
Time: Thu Aug 27 12:28:29 2020
Thanks John
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, August 27, 2020 11:22 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
Justin,
The contents of the CNT line type are described in this table:
https://dtcenter.github.io/MET/Users_Guide/point-
stat.html?highlight=cnt%20line%20type#id10
Note that column number 25, for TOTAL, lists the total number of
matched
pairs.
The TOTAL column is included in almost all of the line types that MET
writes and consistently indicates the number of matched pairs.
Thanks,
John
On Thu, Aug 27, 2020 at 11:32 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
>
> Oh wait I do have one more unrelated question about point_stat that
I came
> up with just now!
>
> When I run point_stat to output CNT files, I notice that there is a
single
> statistic line for each tau and variable. Is there any information
in
> these CNT lines that tell me how many matched pairs were used to
calculate
> these stats?
>
> Justin
>
> -----Original Message-----
> From: George McCabe via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, August 27, 2020 10:29 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
>
> Great to hear! I'm resolving this issue now.
>
> - George
>
> On Thu, Aug 27, 2020 at 11:26 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467 >
> >
> > Thanks George,
> >
> > I figured it out right after I sent you that. All is working now.
> >
> > Justin
> >
> > -----Original Message-----
> > From: George McCabe via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, August 26, 2020 3:26 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #96467] Python embedding help
> >
> > Hi Justin,
> >
> > I see you are having issues with python embedding. I'm going to
assign
> this
> > ticket to John HG since he has more experience with python
embedding
> than I
> > do.
> >
> > One thing I noticed is your numpy array has a data type of
float32. The
> > examples I have seen use float64, so this could be causing an
issue. I
> > would try that to see if it helps.
> >
> > If that doesn't help, then John will likely have another
suggestion.
> >
> > Thanks,
> > George
> >
> > On Wed, Aug 26, 2020 at 3:41 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed Aug 26 15:41:46 2020: Request 96467 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Python embedding help
> > > Owner: Nobody
> > > Requestors: justin.tsu at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96467
> >
> > >
> > >
> > > Hey John,
> > >
> > > I have a question about python embedding. In my python script,
I have
> > the
> > > following:
> > >
> > > 60 tokens = os.path.basename(input_file).replace('-',
> > '_').split('_');
> > > 61 dimx,dimy = [int(x[-3:]) for x in tokens[4].split('x')]
> > > 62 data =
np.fromfile(input_file,dtype='>f4',count=dimx*dimy)
> > > 63 met_data = np.reshape(data,(dimy,dimx)).copy()
> > >
> > > Where input file is some ieee64 binary
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld.
> > > Running this script produces this:
> > >
> > > In [1]: met_data
> > > Out[1]:
> > > array([[298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > > 299.79425],
> > > [298.2424 , 298.2424 , 297.98398, ..., 299.6279 ,
299.79425,
> > > 299.79425],
> > > [298.68155, 298.68155, 298.9081 , ..., 299.3087 ,
299.1046 ,
> > > 299.1046 ],
> > > ...,
> > > [305.22366, 305.22366, 305.1536 , ..., 296.87302,
297.37988,
> > > 297.37988],
> > > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > > 295.63745],
> > > [304.95667, 304.95667, 304.9788 , ..., 296.63263,
295.63745,
> > > 295.63745]], dtype=float32)
> > >
> > > Met_data is an array full of sensical values. However, when I
run
> > > regrid_data_plane on the same file:
> > > [tsu at lorenz 04_CAPE]$ regrid_data_plane PYTHON_NUMPY 'mercator
127 127
> > > 2.028 248.843 49.555 304.033' out.nc -field
> > > 'name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";'
> > > DEBUG 1: Reading data file: PYTHON_NUMPY
> > > DEBUG 2: Input grid: Projection: Mercator Nx: 127 Ny: 127
> Lat_LL_radians:
> > > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> > 0.9768
> > > Mx: 130.8055 My: 130.8055 Bx: 253.7709 By: -4.6313
> > > DEBUG 2: Output grid: Projection: Mercator Nx: 127 Ny: 127
> > Lat_LL_radians:
> > > 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians:
> > 0.9768
> > > Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
> > > DEBUG 2: Interpolation options: method = NEAREST, width = 1,
shape =
> > > SQUARE, vld_thresh = 0.5
> > > DEBUG 2: Range of input data
> > > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > > is -1.66213e+308 to 8.59046e+307.
> > > DEBUG 2: Range of regridded data
> > > (name="/users/tsu/MET/work/04_CAPE/read_COAMPS_general.py
> > >
> >
>
/DATA/tsu/repos/flds/airtmp_zht_000002_000000_1a0127x0127_2019080100_00000000_fcstfld";)
> > > is -1.66213e+308 to 8.59046e+307.
> > > DEBUG 1: Writing output file: out.nc
> > >
> > > From the highlighted portion, we can see that the met_data
matrix no
> > > longer contains sensical values but instead has been replaced
with very
> > > large Inf values. What is going on? What is python embedding
expecting
> > > from the python script?
> > >
> > > Justin
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list