[Met_help] [rt.rap.ucar.edu #89634] History for MET Tool 8.0 support of rotated pole data
John Halley Gotway via RT
met_help at ucar.edu
Wed Jul 10 17:01:06 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Met Tool Help:
How are you?
If I understood correctly, Met Tool 8.0 supports rotated pole data via a Python interface. I looked at the official Met Tool 8.0 documentation -- it only states it support rotated pole, but I wasn't able to find any specific information and examples in how that is actually done.
Does it follow conventions similar to the read_ascii_numpy.py in which data is read into met_data variable, plus the appropriate information in attrs dictionary? In this case, certain specific and correct information needed for attrs['grid']?
Thank you
Steven Chan
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Thu Apr 11 03:47:12 2019
Dear Met Help:
Please ignore last email. I was not thinking and remembering clearly
we have communicated over this last year. Please forgive the mistake.
Steven
________________________________
From: met_help at ucar.edu via RT <met_help at ucar.edu>
Sent: 08 April 2019 17:35:22
To: Chan, Steven
Subject: [rt.rap.ucar.edu #89634] AutoReply: MET Tool 8.0 support of
rotated pole data
Greetings,
This message has been automatically generated in response to the
creation of a trouble ticket regarding:
"MET Tool 8.0 support of rotated pole data",
a summary of which appears below.
There is no need to reply to this message right now. Your ticket has
been assigned an ID of [rt.rap.ucar.edu #89634].
Please include the string:
[rt.rap.ucar.edu #89634]
in the subject line of all future correspondence about this issue. To
do so, you may reply to this message.
For more information, please see:
MET Online Tutorial:
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
MET Users Guide:
https://www.dtcenter.org/met/users/docs/overview.php
MET FAQs:
https://www.dtcenter.org/met/users/support/faqs/index.php
MET-Help Email Archive:
http://mailman.ucar.edu/pipermail/met_help
Thank you,
met_help at ucar.edu
-------------------------------------------------------------------------
Dear Met Tool Help:
How are you?
If I understood correctly, Met Tool 8.0 supports rotated pole data via
a Python interface. I looked at the official Met Tool 8.0
documentation -- it only states it support rotated pole, but I wasn't
able to find any specific information and examples in how that is
actually done.
Does it follow conventions similar to the read_ascii_numpy.py in which
data is read into met_data variable, plus the appropriate information
in attrs dictionary? In this case, certain specific and correct
information needed for attrs['grid']?
Thank you
Steven Chan
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: John Halley Gotway
Time: Fri Apr 12 12:31:20 2019
Steven,
It sounds like you're asking for an example of encoding a rotated
lat/lon
projection in MET's Python interface. You'd like to know what
elements
should be included in "attrs['grid']" to define a rotated lat/lon
grid.
I'm going to assign this ticket to Randy, who knows the most about the
Python interface and projection information. He should be in touch
with
you next week about this.
Thanks,
John
On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> Transaction: Ticket created by steven.chan at metoffice.gov.uk
> Queue: met_help
> Subject: MET Tool 8.0 support of rotated pole data
> Owner: Nobody
> Requestors: steven.chan at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
>
> Dear Met Tool Help:
>
>
> How are you?
>
> If I understood correctly, Met Tool 8.0 supports rotated pole data
via a
> Python interface. I looked at the official Met Tool 8.0
documentation -- it
> only states it support rotated pole, but I wasn't able to find any
specific
> information and examples in how that is actually done.
>
>
> Does it follow conventions similar to the read_ascii_numpy.py in
which
> data is read into met_data variable, plus the appropriate
information in
> attrs dictionary? In this case, certain specific and correct
information
> needed for attrs['grid']?
>
>
> Thank you
>
> Steven Chan
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Randy Bullock
Time: Sat Apr 13 17:28:37 2019
Hi Steven -
Specifying a rotated lat/lon grid from python should be possible. The
needed entries in the "atts" dictionary are
//
// name (string)
//
// rot_lat_ll, rot_lon_ll (double)
//
// delta_rot_lat, delta_rot_lon (double)
//
// Nlat, Nlon (int)
//
// true_lat_south_pole, true_lon_south_pole (double)
//
// aux_rotation (double)
I copied these lines from the MET C++ source file
"grid_from_python_dict.cc". This code's job is to instantiate a grid
(along with the associated map projection) from the entries in a
python
dictionary. The entries make a distinction between quantities that
refer
to rotated lat/lon (prefix "rot_" in the list above) and entries that
refer
to true (ie unrotated) lat/lon (prefix "true" in the list).
As for the meaning of each entry, we have patterned it after the GRIB2
specification for rotated lat/lon grids (see chart here
<https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml>,
especially the notes below the chart).
To summarize ---
Good news: Doing this should be possible.
Bad news: It may take some experimentation with the rotation
parameters.
Most people (including those at key institutions who create the grid
specs!) don't have a good intuition for 3D space and rotations, so you
might have to work at it a bit.
Hope this helps.
Randy
On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Steven,
>
> It sounds like you're asking for an example of encoding a rotated
lat/lon
> projection in MET's Python interface. You'd like to know what
elements
> should be included in "attrs['grid']" to define a rotated lat/lon
grid.
>
> I'm going to assign this ticket to Randy, who knows the most about
the
> Python interface and projection information. He should be in touch
with
> you next week about this.
>
> Thanks,
> John
>
>
>
> On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > Queue: met_help
> > Subject: MET Tool 8.0 support of rotated pole data
> > Owner: Nobody
> > Requestors: steven.chan at metoffice.gov.uk
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> >
> > Dear Met Tool Help:
> >
> >
> > How are you?
> >
> > If I understood correctly, Met Tool 8.0 supports rotated pole data
via a
> > Python interface. I looked at the official Met Tool 8.0
documentation --
> it
> > only states it support rotated pole, but I wasn't able to find any
> specific
> > information and examples in how that is actually done.
> >
> >
> > Does it follow conventions similar to the read_ascii_numpy.py in
which
> > data is read into met_data variable, plus the appropriate
information in
> > attrs dictionary? In this case, certain specific and correct
information
> > needed for attrs['grid']?
> >
> >
> > Thank you
> >
> > Steven Chan
> >
> >
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Mon Apr 15 10:19:45 2019
Dear Randy:
Thank you very much for the reply. I was not aware of the Grib2
standards, and that site's info clarify much of the issues. I have
also resolved some of the unusual aspects of Python library in the Met
Office network.
Now some newer issues popped up -- I do not know if this is specific
to MTD tool or to the MET 8.0, but I got the following unexpected
results and errors:
DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
Input File: './infilelist'
Data Name : 'FCST'
['ym200209d10z23.nc']
Data Shape: (1536, 1536)
Data Type: dtype('float64')
dlat = 0.019999999553
dlon = 0.019999999553
Nlat = 1536
Nlon = 1536
true_lat_south_pole = -43.0
true_lon_south_pole = 10.0
rot_lat_ll = -14.7999992371
rot_lon_ll = 341.199981689
Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
'valid': '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat': 0.019999999552965164, 'delta_rot_lon':
0.019999999552965164, 'Nlon': 1536, 'true_lat_south_pole': -43.0,
'name': 'UKMO', 'rot_lat_ll': -14.799999237060547, 'Nlat': 1536,
'aux_rotation': 0.0, 'rot_lon_ll': 341.1999816894531, 'type': 'Rotated
LatLon'}, 'name': 'FCST', 'lead': '00', 'level': 'Surface', 'units':
'mm/hr', 'accum': '01'}
DEBUG 2: regridding, if needed ...
ERROR :
ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:
ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800 rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation: 0.000 !=
ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800 rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation: 0.000
ERROR : Specify regridding logic in the config file "regrid" section.
ERROR :
There are two problems with the output -- the obvious one being the
error and a second not-so-obvious one (but it could be related to
error).
1) The error: It is caused by trying to match projections between FCST
and OBS. MTD is running single mode, so it is the same file with the
same projections and uses the same dict "attrs".
2) Unexpected value for rot_lon_ll: "attrs" has set the value to near
341.2. but somehow the value becomes negative (-341.2, as indicated in
the error message).
There also seem to be floating point comparison and rounding problems,
which may be related to the above two problems.
Best Regards
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 14 April 2019 00:28:37
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Hi Steven -
Specifying a rotated lat/lon grid from python should be possible. The
needed entries in the "atts" dictionary are
//
// name (string)
//
// rot_lat_ll, rot_lon_ll (double)
//
// delta_rot_lat, delta_rot_lon (double)
//
// Nlat, Nlon (int)
//
// true_lat_south_pole, true_lon_south_pole (double)
//
// aux_rotation (double)
I copied these lines from the MET C++ source file
"grid_from_python_dict.cc". This code's job is to instantiate a grid
(along with the associated map projection) from the entries in a
python
dictionary. The entries make a distinction between quantities that
refer
to rotated lat/lon (prefix "rot_" in the list above) and entries that
refer
to true (ie unrotated) lat/lon (prefix "true" in the list).
As for the meaning of each entry, we have patterned it after the GRIB2
specification for rotated lat/lon grids (see chart here
<https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml>,
especially the notes below the chart).
To summarize ---
Good news: Doing this should be possible.
Bad news: It may take some experimentation with the rotation
parameters.
Most people (including those at key institutions who create the grid
specs!) don't have a good intuition for 3D space and rotations, so you
might have to work at it a bit.
Hope this helps.
Randy
On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Steven,
>
> It sounds like you're asking for an example of encoding a rotated
lat/lon
> projection in MET's Python interface. You'd like to know what
elements
> should be included in "attrs['grid']" to define a rotated lat/lon
grid.
>
> I'm going to assign this ticket to Randy, who knows the most about
the
> Python interface and projection information. He should be in touch
with
> you next week about this.
>
> Thanks,
> John
>
>
>
> On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > Queue: met_help
> > Subject: MET Tool 8.0 support of rotated pole data
> > Owner: Nobody
> > Requestors: steven.chan at metoffice.gov.uk
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> >
> > Dear Met Tool Help:
> >
> >
> > How are you?
> >
> > If I understood correctly, Met Tool 8.0 supports rotated pole data
via a
> > Python interface. I looked at the official Met Tool 8.0
documentation --
> it
> > only states it support rotated pole, but I wasn't able to find any
> specific
> > information and examples in how that is actually done.
> >
> >
> > Does it follow conventions similar to the read_ascii_numpy.py in
which
> > data is read into met_data variable, plus the appropriate
information in
> > attrs dictionary? In this case, certain specific and correct
information
> > needed for attrs['grid']?
> >
> >
> > Thank you
> >
> > Steven Chan
> >
> >
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Randy Bullock
Time: Tue Apr 16 11:40:57 2019
Hello again, Steven -
We're aware of the problems that MET sometimes has with testing
whether two
grids are the same or not. Your speculation about floating-point
rounding
issues no doubt explains at least part of the trouble.
Could you provide us with the data files you're using? Also the
config
file. If the data files are small enough, you could compress them and
attach them to the email. Otherwise I can give you an ftp site where
you
can put them.
Thanks in advance.
Randy
On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the reply. I was not aware of the Grib2
standards,
> and that site's info clarify much of the issues. I have also
resolved some
> of the unusual aspects of Python library in the Met Office network.
>
>
> Now some newer issues popped up -- I do not know if this is specific
to
> MTD tool or to the MET 8.0, but I got the following unexpected
results and
> errors:
>
>
> DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> Input File: './infilelist'
> Data Name : 'FCST'
> ['ym200209d10z23.nc']
> Data Shape: (1536, 1536)
> Data Type: dtype('float64')
> dlat = 0.019999999553
> dlon = 0.019999999553
> Nlat = 1536
> Nlon = 1536
> true_lat_south_pole = -43.0
> true_lon_south_pole = 10.0
> rot_lat_ll = -14.7999992371
> rot_lon_ll = 341.199981689
> Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
'valid':
> '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat':
> 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164, 'Nlon':
1536,
> 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead': '00',
> 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> DEBUG 2: regridding, if needed ...
> ERROR :
> ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> match:
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000 !=
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000
> ERROR : Specify regridding logic in the config file "regrid"
section.
> ERROR :
>
> There are two problems with the output -- the obvious one being the
error
> and a second not-so-obvious one (but it could be related to error).
>
>
> 1) The error: It is caused by trying to match projections between
FCST and
> OBS. MTD is running single mode, so it is the same file with the
same
> projections and uses the same dict "attrs".
>
>
> 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> 341.2. but somehow the value becomes negative (-341.2, as indicated
in the
> error message).
>
>
> There also seem to be floating point comparison and rounding
problems,
> which may be related to the above two problems.
>
>
> Best Regards
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 14 April 2019 00:28:37
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hi Steven -
>
> Specifying a rotated lat/lon grid from python should be possible.
The
> needed entries in the "atts" dictionary are
>
>
> //
> // name (string)
> //
> // rot_lat_ll, rot_lon_ll (double)
> //
> // delta_rot_lat, delta_rot_lon (double)
> //
> // Nlat, Nlon (int)
> //
> // true_lat_south_pole, true_lon_south_pole (double)
> //
> // aux_rotation (double)
>
> I copied these lines from the MET C++ source file
> "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> (along with the associated map projection) from the entries in a
python
> dictionary. The entries make a distinction between quantities that
refer
> to rotated lat/lon (prefix "rot_" in the list above) and entries
that refer
> to true (ie unrotated) lat/lon (prefix "true" in the list).
>
> As for the meaning of each entry, we have patterned it after the
GRIB2
> specification for rotated lat/lon grids (see chart here
> <
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> >,
> especially the notes below the chart).
>
> To summarize ---
>
> Good news: Doing this should be possible.
>
> Bad news: It may take some experimentation with the rotation
parameters.
> Most people (including those at key institutions who create the grid
> specs!) don't have a good intuition for 3D space and rotations, so
you
> might have to work at it a bit.
>
>
> Hope this helps.
>
> Randy
>
>
>
> On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Steven,
> >
> > It sounds like you're asking for an example of encoding a rotated
lat/lon
> > projection in MET's Python interface. You'd like to know what
elements
> > should be included in "attrs['grid']" to define a rotated lat/lon
grid.
> >
> > I'm going to assign this ticket to Randy, who knows the most about
the
> > Python interface and projection information. He should be in
touch with
> > you next week about this.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > Queue: met_help
> > > Subject: MET Tool 8.0 support of rotated pole data
> > > Owner: Nobody
> > > Requestors: steven.chan at metoffice.gov.uk
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> >
> > >
> > >
> > > Dear Met Tool Help:
> > >
> > >
> > > How are you?
> > >
> > > If I understood correctly, Met Tool 8.0 supports rotated pole
data via
> a
> > > Python interface. I looked at the official Met Tool 8.0
documentation
> --
> > it
> > > only states it support rotated pole, but I wasn't able to find
any
> > specific
> > > information and examples in how that is actually done.
> > >
> > >
> > > Does it follow conventions similar to the read_ascii_numpy.py in
which
> > > data is read into met_data variable, plus the appropriate
information
> in
> > > attrs dictionary? In this case, certain specific and correct
> information
> > > needed for attrs['grid']?
> > >
> > >
> > > Thank you
> > >
> > > Steven Chan
> > >
> > >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Wed Apr 17 03:53:45 2019
Dear Randy:
Thank you for your email. The model data used in tests won't fit in an
email, hence can I have information about the FTP please?
Thank you
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 16 April 2019 18:40:58
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Hello again, Steven -
We're aware of the problems that MET sometimes has with testing
whether two
grids are the same or not. Your speculation about floating-point
rounding
issues no doubt explains at least part of the trouble.
Could you provide us with the data files you're using? Also the
config
file. If the data files are small enough, you could compress them and
attach them to the email. Otherwise I can give you an ftp site where
you
can put them.
Thanks in advance.
Randy
On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the reply. I was not aware of the Grib2
standards,
> and that site's info clarify much of the issues. I have also
resolved some
> of the unusual aspects of Python library in the Met Office network.
>
>
> Now some newer issues popped up -- I do not know if this is specific
to
> MTD tool or to the MET 8.0, but I got the following unexpected
results and
> errors:
>
>
> DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> Input File: './infilelist'
> Data Name : 'FCST'
> ['ym200209d10z23.nc']
> Data Shape: (1536, 1536)
> Data Type: dtype('float64')
> dlat = 0.019999999553
> dlon = 0.019999999553
> Nlat = 1536
> Nlon = 1536
> true_lat_south_pole = -43.0
> true_lon_south_pole = 10.0
> rot_lat_ll = -14.7999992371
> rot_lon_ll = 341.199981689
> Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
'valid':
> '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat':
> 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164, 'Nlon':
1536,
> 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead': '00',
> 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> DEBUG 2: regridding, if needed ...
> ERROR :
> ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> match:
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000 !=
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000
> ERROR : Specify regridding logic in the config file "regrid"
section.
> ERROR :
>
> There are two problems with the output -- the obvious one being the
error
> and a second not-so-obvious one (but it could be related to error).
>
>
> 1) The error: It is caused by trying to match projections between
FCST and
> OBS. MTD is running single mode, so it is the same file with the
same
> projections and uses the same dict "attrs".
>
>
> 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> 341.2. but somehow the value becomes negative (-341.2, as indicated
in the
> error message).
>
>
> There also seem to be floating point comparison and rounding
problems,
> which may be related to the above two problems.
>
>
> Best Regards
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 14 April 2019 00:28:37
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hi Steven -
>
> Specifying a rotated lat/lon grid from python should be possible.
The
> needed entries in the "atts" dictionary are
>
>
> //
> // name (string)
> //
> // rot_lat_ll, rot_lon_ll (double)
> //
> // delta_rot_lat, delta_rot_lon (double)
> //
> // Nlat, Nlon (int)
> //
> // true_lat_south_pole, true_lon_south_pole (double)
> //
> // aux_rotation (double)
>
> I copied these lines from the MET C++ source file
> "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> (along with the associated map projection) from the entries in a
python
> dictionary. The entries make a distinction between quantities that
refer
> to rotated lat/lon (prefix "rot_" in the list above) and entries
that refer
> to true (ie unrotated) lat/lon (prefix "true" in the list).
>
> As for the meaning of each entry, we have patterned it after the
GRIB2
> specification for rotated lat/lon grids (see chart here
> <
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> >,
> especially the notes below the chart).
>
> To summarize ---
>
> Good news: Doing this should be possible.
>
> Bad news: It may take some experimentation with the rotation
parameters.
> Most people (including those at key institutions who create the grid
> specs!) don't have a good intuition for 3D space and rotations, so
you
> might have to work at it a bit.
>
>
> Hope this helps.
>
> Randy
>
>
>
> On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Steven,
> >
> > It sounds like you're asking for an example of encoding a rotated
lat/lon
> > projection in MET's Python interface. You'd like to know what
elements
> > should be included in "attrs['grid']" to define a rotated lat/lon
grid.
> >
> > I'm going to assign this ticket to Randy, who knows the most about
the
> > Python interface and projection information. He should be in
touch with
> > you next week about this.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > Queue: met_help
> > > Subject: MET Tool 8.0 support of rotated pole data
> > > Owner: Nobody
> > > Requestors: steven.chan at metoffice.gov.uk
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> >
> > >
> > >
> > > Dear Met Tool Help:
> > >
> > >
> > > How are you?
> > >
> > > If I understood correctly, Met Tool 8.0 supports rotated pole
data via
> a
> > > Python interface. I looked at the official Met Tool 8.0
documentation
> --
> > it
> > > only states it support rotated pole, but I wasn't able to find
any
> > specific
> > > information and examples in how that is actually done.
> > >
> > >
> > > Does it follow conventions similar to the read_ascii_numpy.py in
which
> > > data is read into met_data variable, plus the appropriate
information
> in
> > > attrs dictionary? In this case, certain specific and correct
> information
> > > needed for attrs['grid']?
> > >
> > >
> > > Thank you
> > >
> > > Steven Chan
> > >
> > >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Wed Apr 17 07:28:38 2019
Dear Randy:
I think I have found out how to upload to your FTP. The file is in
/incoming/irap/met_help/schan_UKMO and is massif_2002_sample.tar.gz.
I have included 2 model days of data, python script used, config file,
and sample output that shows the STDOUT and STDERR messages. The
Python script uses a Met Office Python package, but all it does is to
load binary data and is used to read the rotated pole metadata
automatically; the file run_MTD.out (the STDOUT/STDERR messages) shows
the actual rotated pole values used in Python script dict.
Best Regards,
Steven Chan
________________________________
From: Chan, Steven
Sent: 17 April 2019 10:53:38
To: met_help at ucar.edu
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Dear Randy:
Thank you for your email. The model data used in tests won't fit in an
email, hence can I have information about the FTP please?
Thank you
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 16 April 2019 18:40:58
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Hello again, Steven -
We're aware of the problems that MET sometimes has with testing
whether two
grids are the same or not. Your speculation about floating-point
rounding
issues no doubt explains at least part of the trouble.
Could you provide us with the data files you're using? Also the
config
file. If the data files are small enough, you could compress them and
attach them to the email. Otherwise I can give you an ftp site where
you
can put them.
Thanks in advance.
Randy
On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the reply. I was not aware of the Grib2
standards,
> and that site's info clarify much of the issues. I have also
resolved some
> of the unusual aspects of Python library in the Met Office network.
>
>
> Now some newer issues popped up -- I do not know if this is specific
to
> MTD tool or to the MET 8.0, but I got the following unexpected
results and
> errors:
>
>
> DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> Input File: './infilelist'
> Data Name : 'FCST'
> ['ym200209d10z23.nc']
> Data Shape: (1536, 1536)
> Data Type: dtype('float64')
> dlat = 0.019999999553
> dlon = 0.019999999553
> Nlat = 1536
> Nlon = 1536
> true_lat_south_pole = -43.0
> true_lon_south_pole = 10.0
> rot_lat_ll = -14.7999992371
> rot_lon_ll = 341.199981689
> Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
'valid':
> '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat':
> 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164, 'Nlon':
1536,
> 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead': '00',
> 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> DEBUG 2: regridding, if needed ...
> ERROR :
> ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> match:
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000 !=
> ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
-14.800
> rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> 0.000
> ERROR : Specify regridding logic in the config file "regrid"
section.
> ERROR :
>
> There are two problems with the output -- the obvious one being the
error
> and a second not-so-obvious one (but it could be related to error).
>
>
> 1) The error: It is caused by trying to match projections between
FCST and
> OBS. MTD is running single mode, so it is the same file with the
same
> projections and uses the same dict "attrs".
>
>
> 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> 341.2. but somehow the value becomes negative (-341.2, as indicated
in the
> error message).
>
>
> There also seem to be floating point comparison and rounding
problems,
> which may be related to the above two problems.
>
>
> Best Regards
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 14 April 2019 00:28:37
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hi Steven -
>
> Specifying a rotated lat/lon grid from python should be possible.
The
> needed entries in the "atts" dictionary are
>
>
> //
> // name (string)
> //
> // rot_lat_ll, rot_lon_ll (double)
> //
> // delta_rot_lat, delta_rot_lon (double)
> //
> // Nlat, Nlon (int)
> //
> // true_lat_south_pole, true_lon_south_pole (double)
> //
> // aux_rotation (double)
>
> I copied these lines from the MET C++ source file
> "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> (along with the associated map projection) from the entries in a
python
> dictionary. The entries make a distinction between quantities that
refer
> to rotated lat/lon (prefix "rot_" in the list above) and entries
that refer
> to true (ie unrotated) lat/lon (prefix "true" in the list).
>
> As for the meaning of each entry, we have patterned it after the
GRIB2
> specification for rotated lat/lon grids (see chart here
> <
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> >,
> especially the notes below the chart).
>
> To summarize ---
>
> Good news: Doing this should be possible.
>
> Bad news: It may take some experimentation with the rotation
parameters.
> Most people (including those at key institutions who create the grid
> specs!) don't have a good intuition for 3D space and rotations, so
you
> might have to work at it a bit.
>
>
> Hope this helps.
>
> Randy
>
>
>
> On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Steven,
> >
> > It sounds like you're asking for an example of encoding a rotated
lat/lon
> > projection in MET's Python interface. You'd like to know what
elements
> > should be included in "attrs['grid']" to define a rotated lat/lon
grid.
> >
> > I'm going to assign this ticket to Randy, who knows the most about
the
> > Python interface and projection information. He should be in
touch with
> > you next week about this.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > Queue: met_help
> > > Subject: MET Tool 8.0 support of rotated pole data
> > > Owner: Nobody
> > > Requestors: steven.chan at metoffice.gov.uk
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> >
> > >
> > >
> > > Dear Met Tool Help:
> > >
> > >
> > > How are you?
> > >
> > > If I understood correctly, Met Tool 8.0 supports rotated pole
data via
> a
> > > Python interface. I looked at the official Met Tool 8.0
documentation
> --
> > it
> > > only states it support rotated pole, but I wasn't able to find
any
> > specific
> > > information and examples in how that is actually done.
> > >
> > >
> > > Does it follow conventions similar to the read_ascii_numpy.py in
which
> > > data is read into met_data variable, plus the appropriate
information
> in
> > > attrs dictionary? In this case, certain specific and correct
> information
> > > needed for attrs['grid']?
> > >
> > >
> > > Thank you
> > >
> > > Steven Chan
> > >
> > >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Randy Bullock
Time: Wed Apr 17 12:24:07 2019
Steven -
I'll have a look at the data. Can't promise anything, though ...
Randy
On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> I think I have found out how to upload to your FTP. The file is in
> /incoming/irap/met_help/schan_UKMO and is massif_2002_sample.tar.gz.
>
>
> I have included 2 model days of data, python script used, config
file, and
> sample output that shows the STDOUT and STDERR messages. The Python
script
> uses a Met Office Python package, but all it does is to load binary
data
> and is used to read the rotated pole metadata automatically; the
file
> run_MTD.out (the STDOUT/STDERR messages) shows the actual rotated
pole
> values used in Python script dict.
>
>
> Best Regards,
>
> Steven Chan
>
> ________________________________
> From: Chan, Steven
> Sent: 17 April 2019 10:53:38
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
>
> Dear Randy:
>
>
> Thank you for your email. The model data used in tests won't fit in
an
> email, hence can I have information about the FTP please?
>
>
> Thank you
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 16 April 2019 18:40:58
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hello again, Steven -
>
> We're aware of the problems that MET sometimes has with testing
whether two
> grids are the same or not. Your speculation about floating-point
rounding
> issues no doubt explains at least part of the trouble.
>
> Could you provide us with the data files you're using? Also the
config
> file. If the data files are small enough, you could compress them
and
> attach them to the email. Otherwise I can give you an ftp site
where you
> can put them.
>
> Thanks in advance.
>
> Randy
>
> On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > Thank you very much for the reply. I was not aware of the Grib2
> standards,
> > and that site's info clarify much of the issues. I have also
resolved
> some
> > of the unusual aspects of Python library in the Met Office
network.
> >
> >
> > Now some newer issues popped up -- I do not know if this is
specific to
> > MTD tool or to the MET 8.0, but I got the following unexpected
results
> and
> > errors:
> >
> >
> > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > Input File: './infilelist'
> > Data Name : 'FCST'
> > ['ym200209d10z23.nc']
> > Data Shape: (1536, 1536)
> > Data Type: dtype('float64')
> > dlat = 0.019999999553
> > dlon = 0.019999999553
> > Nlat = 1536
> > Nlon = 1536
> > true_lat_south_pole = -43.0
> > true_lon_south_pole = 10.0
> > rot_lat_ll = -14.7999992371
> > rot_lon_ll = 341.199981689
> > Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
> 'valid':
> > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat':
> > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
'Nlon':
> 1536,
> > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead':
> '00',
> > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > DEBUG 2: regridding, if needed ...
> > ERROR :
> > ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> > match:
> > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
> -14.800
> > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > 0.000 !=
> > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
> -14.800
> > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > 0.000
> > ERROR : Specify regridding logic in the config file "regrid"
section.
> > ERROR :
> >
> > There are two problems with the output -- the obvious one being
the error
> > and a second not-so-obvious one (but it could be related to
error).
> >
> >
> > 1) The error: It is caused by trying to match projections between
FCST
> and
> > OBS. MTD is running single mode, so it is the same file with the
same
> > projections and uses the same dict "attrs".
> >
> >
> > 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> > 341.2. but somehow the value becomes negative (-341.2, as
indicated in
> the
> > error message).
> >
> >
> > There also seem to be floating point comparison and rounding
problems,
> > which may be related to the above two problems.
> >
> >
> > Best Regards
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 14 April 2019 00:28:37
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Hi Steven -
> >
> > Specifying a rotated lat/lon grid from python should be possible.
The
> > needed entries in the "atts" dictionary are
> >
> >
> > //
> > // name (string)
> > //
> > // rot_lat_ll, rot_lon_ll (double)
> > //
> > // delta_rot_lat, delta_rot_lon (double)
> > //
> > // Nlat, Nlon (int)
> > //
> > // true_lat_south_pole, true_lon_south_pole (double)
> > //
> > // aux_rotation (double)
> >
> > I copied these lines from the MET C++ source file
> > "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> > (along with the associated map projection) from the entries in a
python
> > dictionary. The entries make a distinction between quantities
that
> refer
> > to rotated lat/lon (prefix "rot_" in the list above) and entries
that
> refer
> > to true (ie unrotated) lat/lon (prefix "true" in the list).
> >
> > As for the meaning of each entry, we have patterned it after the
GRIB2
> > specification for rotated lat/lon grids (see chart here
> > <
> >
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> > >,
> > especially the notes below the chart).
> >
> > To summarize ---
> >
> > Good news: Doing this should be possible.
> >
> > Bad news: It may take some experimentation with the rotation
parameters.
> > Most people (including those at key institutions who create the
grid
> > specs!) don't have a good intuition for 3D space and rotations, so
you
> > might have to work at it a bit.
> >
> >
> > Hope this helps.
> >
> > Randy
> >
> >
> >
> > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Steven,
> > >
> > > It sounds like you're asking for an example of encoding a
rotated
> lat/lon
> > > projection in MET's Python interface. You'd like to know what
elements
> > > should be included in "attrs['grid']" to define a rotated
lat/lon grid.
> > >
> > > I'm going to assign this ticket to Randy, who knows the most
about the
> > > Python interface and projection information. He should be in
touch
> with
> > > you next week about this.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > > Queue: met_help
> > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > Owner: Nobody
> > > > Requestors: steven.chan at metoffice.gov.uk
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > >
> > > >
> > > >
> > > > Dear Met Tool Help:
> > > >
> > > >
> > > > How are you?
> > > >
> > > > If I understood correctly, Met Tool 8.0 supports rotated pole
data
> via
> > a
> > > > Python interface. I looked at the official Met Tool 8.0
documentation
> > --
> > > it
> > > > only states it support rotated pole, but I wasn't able to find
any
> > > specific
> > > > information and examples in how that is actually done.
> > > >
> > > >
> > > > Does it follow conventions similar to the read_ascii_numpy.py
in
> which
> > > > data is read into met_data variable, plus the appropriate
information
> > in
> > > > attrs dictionary? In this case, certain specific and correct
> > information
> > > > needed for attrs['grid']?
> > > >
> > > >
> > > > Thank you
> > > >
> > > > Steven Chan
> > > >
> > > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Thu Apr 18 06:25:27 2019
Dear Randy:
Thank you very much for the help again. If there is something I can
help testing, I am more than willing to.
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 17 April 2019 19:24:07
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Steven -
I'll have a look at the data. Can't promise anything, though ...
Randy
On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> I think I have found out how to upload to your FTP. The file is in
> /incoming/irap/met_help/schan_UKMO and is massif_2002_sample.tar.gz.
>
>
> I have included 2 model days of data, python script used, config
file, and
> sample output that shows the STDOUT and STDERR messages. The Python
script
> uses a Met Office Python package, but all it does is to load binary
data
> and is used to read the rotated pole metadata automatically; the
file
> run_MTD.out (the STDOUT/STDERR messages) shows the actual rotated
pole
> values used in Python script dict.
>
>
> Best Regards,
>
> Steven Chan
>
> ________________________________
> From: Chan, Steven
> Sent: 17 April 2019 10:53:38
> To: met_help at ucar.edu
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
>
> Dear Randy:
>
>
> Thank you for your email. The model data used in tests won't fit in
an
> email, hence can I have information about the FTP please?
>
>
> Thank you
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 16 April 2019 18:40:58
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hello again, Steven -
>
> We're aware of the problems that MET sometimes has with testing
whether two
> grids are the same or not. Your speculation about floating-point
rounding
> issues no doubt explains at least part of the trouble.
>
> Could you provide us with the data files you're using? Also the
config
> file. If the data files are small enough, you could compress them
and
> attach them to the email. Otherwise I can give you an ftp site
where you
> can put them.
>
> Thanks in advance.
>
> Randy
>
> On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > Thank you very much for the reply. I was not aware of the Grib2
> standards,
> > and that site's info clarify much of the issues. I have also
resolved
> some
> > of the unusual aspects of Python library in the Met Office
network.
> >
> >
> > Now some newer issues popped up -- I do not know if this is
specific to
> > MTD tool or to the MET 8.0, but I got the following unexpected
results
> and
> > errors:
> >
> >
> > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > Input File: './infilelist'
> > Data Name : 'FCST'
> > ['ym200209d10z23.nc']
> > Data Shape: (1536, 1536)
> > Data Type: dtype('float64')
> > dlat = 0.019999999553
> > dlon = 0.019999999553
> > Nlat = 1536
> > Nlon = 1536
> > true_lat_south_pole = -43.0
> > true_lon_south_pole = 10.0
> > rot_lat_ll = -14.7999992371
> > rot_lon_ll = 341.199981689
> > Attributes: {'long_name': 'FCST_word', 'init': '20020910_230000',
> 'valid':
> > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
'delta_rot_lat':
> > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
'Nlon':
> 1536,
> > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead':
> '00',
> > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > DEBUG 2: regridding, if needed ...
> > ERROR :
> > ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> > match:
> > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
> -14.800
> > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > 0.000 !=
> > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536 rot_lat_ll:
> -14.800
> > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > 0.000
> > ERROR : Specify regridding logic in the config file "regrid"
section.
> > ERROR :
> >
> > There are two problems with the output -- the obvious one being
the error
> > and a second not-so-obvious one (but it could be related to
error).
> >
> >
> > 1) The error: It is caused by trying to match projections between
FCST
> and
> > OBS. MTD is running single mode, so it is the same file with the
same
> > projections and uses the same dict "attrs".
> >
> >
> > 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> > 341.2. but somehow the value becomes negative (-341.2, as
indicated in
> the
> > error message).
> >
> >
> > There also seem to be floating point comparison and rounding
problems,
> > which may be related to the above two problems.
> >
> >
> > Best Regards
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 14 April 2019 00:28:37
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Hi Steven -
> >
> > Specifying a rotated lat/lon grid from python should be possible.
The
> > needed entries in the "atts" dictionary are
> >
> >
> > //
> > // name (string)
> > //
> > // rot_lat_ll, rot_lon_ll (double)
> > //
> > // delta_rot_lat, delta_rot_lon (double)
> > //
> > // Nlat, Nlon (int)
> > //
> > // true_lat_south_pole, true_lon_south_pole (double)
> > //
> > // aux_rotation (double)
> >
> > I copied these lines from the MET C++ source file
> > "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> > (along with the associated map projection) from the entries in a
python
> > dictionary. The entries make a distinction between quantities
that
> refer
> > to rotated lat/lon (prefix "rot_" in the list above) and entries
that
> refer
> > to true (ie unrotated) lat/lon (prefix "true" in the list).
> >
> > As for the meaning of each entry, we have patterned it after the
GRIB2
> > specification for rotated lat/lon grids (see chart here
> > <
> >
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> > >,
> > especially the notes below the chart).
> >
> > To summarize ---
> >
> > Good news: Doing this should be possible.
> >
> > Bad news: It may take some experimentation with the rotation
parameters.
> > Most people (including those at key institutions who create the
grid
> > specs!) don't have a good intuition for 3D space and rotations, so
you
> > might have to work at it a bit.
> >
> >
> > Hope this helps.
> >
> > Randy
> >
> >
> >
> > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Steven,
> > >
> > > It sounds like you're asking for an example of encoding a
rotated
> lat/lon
> > > projection in MET's Python interface. You'd like to know what
elements
> > > should be included in "attrs['grid']" to define a rotated
lat/lon grid.
> > >
> > > I'm going to assign this ticket to Randy, who knows the most
about the
> > > Python interface and projection information. He should be in
touch
> with
> > > you next week about this.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > > Queue: met_help
> > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > Owner: Nobody
> > > > Requestors: steven.chan at metoffice.gov.uk
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > >
> > > >
> > > >
> > > > Dear Met Tool Help:
> > > >
> > > >
> > > > How are you?
> > > >
> > > > If I understood correctly, Met Tool 8.0 supports rotated pole
data
> via
> > a
> > > > Python interface. I looked at the official Met Tool 8.0
documentation
> > --
> > > it
> > > > only states it support rotated pole, but I wasn't able to find
any
> > > specific
> > > > information and examples in how that is actually done.
> > > >
> > > >
> > > > Does it follow conventions similar to the read_ascii_numpy.py
in
> which
> > > > data is read into met_data variable, plus the appropriate
information
> > in
> > > > attrs dictionary? In this case, certain specific and correct
> > information
> > > > needed for attrs['grid']?
> > > >
> > > >
> > > > Thank you
> > > >
> > > > Steven Chan
> > > >
> > > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Randy Bullock
Time: Thu Apr 18 12:54:19 2019
Hi Steve -
I got your tarball from the FTP site and copied it to my local
machine.
It'll likely be tuesday or so before I have a chance to look at it
though.
Sorry.
Randy
On Thu, Apr 18, 2019 at 6:26 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the help again. If there is something I can
help
> testing, I am more than willing to.
>
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 17 April 2019 19:24:07
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Steven -
>
> I'll have a look at the data. Can't promise anything, though ...
>
> Randy
>
> On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > I think I have found out how to upload to your FTP. The file is in
> > /incoming/irap/met_help/schan_UKMO and is
massif_2002_sample.tar.gz.
> >
> >
> > I have included 2 model days of data, python script used, config
file,
> and
> > sample output that shows the STDOUT and STDERR messages. The
Python
> script
> > uses a Met Office Python package, but all it does is to load
binary data
> > and is used to read the rotated pole metadata automatically; the
file
> > run_MTD.out (the STDOUT/STDERR messages) shows the actual rotated
pole
> > values used in Python script dict.
> >
> >
> > Best Regards,
> >
> > Steven Chan
> >
> > ________________________________
> > From: Chan, Steven
> > Sent: 17 April 2019 10:53:38
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> >
> > Dear Randy:
> >
> >
> > Thank you for your email. The model data used in tests won't fit
in an
> > email, hence can I have information about the FTP please?
> >
> >
> > Thank you
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 16 April 2019 18:40:58
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Hello again, Steven -
> >
> > We're aware of the problems that MET sometimes has with testing
whether
> two
> > grids are the same or not. Your speculation about floating-point
> rounding
> > issues no doubt explains at least part of the trouble.
> >
> > Could you provide us with the data files you're using? Also the
config
> > file. If the data files are small enough, you could compress them
and
> > attach them to the email. Otherwise I can give you an ftp site
where you
> > can put them.
> >
> > Thanks in advance.
> >
> > Randy
> >
> > On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Dear Randy:
> > >
> > >
> > > Thank you very much for the reply. I was not aware of the Grib2
> > standards,
> > > and that site's info clarify much of the issues. I have also
resolved
> > some
> > > of the unusual aspects of Python library in the Met Office
network.
> > >
> > >
> > > Now some newer issues popped up -- I do not know if this is
specific to
> > > MTD tool or to the MET 8.0, but I got the following unexpected
results
> > and
> > > errors:
> > >
> > >
> > > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > > Input File: './infilelist'
> > > Data Name : 'FCST'
> > > ['ym200209d10z23.nc']
> > > Data Shape: (1536, 1536)
> > > Data Type: dtype('float64')
> > > dlat = 0.019999999553
> > > dlon = 0.019999999553
> > > Nlat = 1536
> > > Nlon = 1536
> > > true_lat_south_pole = -43.0
> > > true_lon_south_pole = 10.0
> > > rot_lat_ll = -14.7999992371
> > > rot_lon_ll = 341.199981689
> > > Attributes: {'long_name': 'FCST_word', 'init':
'20020910_230000',
> > 'valid':
> > > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
> 'delta_rot_lat':
> > > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
'Nlon':
> > 1536,
> > > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> > > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead':
> > '00',
> > > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > > DEBUG 2: regridding, if needed ...
> > > ERROR :
> > > ERROR : parse_vx_grid() -> The forecast and observation grids
do not
> > > match:
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000 !=
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000
> > > ERROR : Specify regridding logic in the config file "regrid"
section.
> > > ERROR :
> > >
> > > There are two problems with the output -- the obvious one being
the
> error
> > > and a second not-so-obvious one (but it could be related to
error).
> > >
> > >
> > > 1) The error: It is caused by trying to match projections
between FCST
> > and
> > > OBS. MTD is running single mode, so it is the same file with the
same
> > > projections and uses the same dict "attrs".
> > >
> > >
> > > 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> > > 341.2. but somehow the value becomes negative (-341.2, as
indicated in
> > the
> > > error message).
> > >
> > >
> > > There also seem to be floating point comparison and rounding
problems,
> > > which may be related to the above two problems.
> > >
> > >
> > > Best Regards
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > Sent: 14 April 2019 00:28:37
> > > To: Chan, Steven
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > > pole data
> > >
> > > Hi Steven -
> > >
> > > Specifying a rotated lat/lon grid from python should be
possible. The
> > > needed entries in the "atts" dictionary are
> > >
> > >
> > > //
> > > // name (string)
> > > //
> > > // rot_lat_ll, rot_lon_ll (double)
> > > //
> > > // delta_rot_lat, delta_rot_lon (double)
> > > //
> > > // Nlat, Nlon (int)
> > > //
> > > // true_lat_south_pole, true_lon_south_pole (double)
> > > //
> > > // aux_rotation (double)
> > >
> > > I copied these lines from the MET C++ source file
> > > "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> > > (along with the associated map projection) from the entries in a
python
> > > dictionary. The entries make a distinction between quantities
that
> > refer
> > > to rotated lat/lon (prefix "rot_" in the list above) and entries
that
> > refer
> > > to true (ie unrotated) lat/lon (prefix "true" in the list).
> > >
> > > As for the meaning of each entry, we have patterned it after the
GRIB2
> > > specification for rotated lat/lon grids (see chart here
> > > <
> > >
> >
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> > > >,
> > > especially the notes below the chart).
> > >
> > > To summarize ---
> > >
> > > Good news: Doing this should be possible.
> > >
> > > Bad news: It may take some experimentation with the rotation
> parameters.
> > > Most people (including those at key institutions who create the
grid
> > > specs!) don't have a good intuition for 3D space and rotations,
so you
> > > might have to work at it a bit.
> > >
> > >
> > > Hope this helps.
> > >
> > > Randy
> > >
> > >
> > >
> > > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
>
> > > >
> > > > Steven,
> > > >
> > > > It sounds like you're asking for an example of encoding a
rotated
> > lat/lon
> > > > projection in MET's Python interface. You'd like to know what
> elements
> > > > should be included in "attrs['grid']" to define a rotated
lat/lon
> grid.
> > > >
> > > > I'm going to assign this ticket to Randy, who knows the most
about
> the
> > > > Python interface and projection information. He should be in
touch
> > with
> > > > you next week about this.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > > > Queue: met_help
> > > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > > Owner: Nobody
> > > > > Requestors: steven.chan at metoffice.gov.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > > >
> > > > >
> > > > >
> > > > > Dear Met Tool Help:
> > > > >
> > > > >
> > > > > How are you?
> > > > >
> > > > > If I understood correctly, Met Tool 8.0 supports rotated
pole data
> > via
> > > a
> > > > > Python interface. I looked at the official Met Tool 8.0
> documentation
> > > --
> > > > it
> > > > > only states it support rotated pole, but I wasn't able to
find any
> > > > specific
> > > > > information and examples in how that is actually done.
> > > > >
> > > > >
> > > > > Does it follow conventions similar to the
read_ascii_numpy.py in
> > which
> > > > > data is read into met_data variable, plus the appropriate
> information
> > > in
> > > > > attrs dictionary? In this case, certain specific and
correct
> > > information
> > > > > needed for attrs['grid']?
> > > > >
> > > > >
> > > > > Thank you
> > > > >
> > > > > Steven Chan
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Tue Apr 23 03:05:30 2019
Dear Randy:
It is no worry at all. Thank you very much for the help.
Steven
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 18 April 2019 19:54:19
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Hi Steve -
I got your tarball from the FTP site and copied it to my local
machine.
It'll likely be tuesday or so before I have a chance to look at it
though.
Sorry.
Randy
On Thu, Apr 18, 2019 at 6:26 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the help again. If there is something I can
help
> testing, I am more than willing to.
>
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 17 April 2019 19:24:07
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Steven -
>
> I'll have a look at the data. Can't promise anything, though ...
>
> Randy
>
> On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > I think I have found out how to upload to your FTP. The file is in
> > /incoming/irap/met_help/schan_UKMO and is
massif_2002_sample.tar.gz.
> >
> >
> > I have included 2 model days of data, python script used, config
file,
> and
> > sample output that shows the STDOUT and STDERR messages. The
Python
> script
> > uses a Met Office Python package, but all it does is to load
binary data
> > and is used to read the rotated pole metadata automatically; the
file
> > run_MTD.out (the STDOUT/STDERR messages) shows the actual rotated
pole
> > values used in Python script dict.
> >
> >
> > Best Regards,
> >
> > Steven Chan
> >
> > ________________________________
> > From: Chan, Steven
> > Sent: 17 April 2019 10:53:38
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> >
> > Dear Randy:
> >
> >
> > Thank you for your email. The model data used in tests won't fit
in an
> > email, hence can I have information about the FTP please?
> >
> >
> > Thank you
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 16 April 2019 18:40:58
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Hello again, Steven -
> >
> > We're aware of the problems that MET sometimes has with testing
whether
> two
> > grids are the same or not. Your speculation about floating-point
> rounding
> > issues no doubt explains at least part of the trouble.
> >
> > Could you provide us with the data files you're using? Also the
config
> > file. If the data files are small enough, you could compress them
and
> > attach them to the email. Otherwise I can give you an ftp site
where you
> > can put them.
> >
> > Thanks in advance.
> >
> > Randy
> >
> > On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Dear Randy:
> > >
> > >
> > > Thank you very much for the reply. I was not aware of the Grib2
> > standards,
> > > and that site's info clarify much of the issues. I have also
resolved
> > some
> > > of the unusual aspects of Python library in the Met Office
network.
> > >
> > >
> > > Now some newer issues popped up -- I do not know if this is
specific to
> > > MTD tool or to the MET 8.0, but I got the following unexpected
results
> > and
> > > errors:
> > >
> > >
> > > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > > Input File: './infilelist'
> > > Data Name : 'FCST'
> > > ['ym200209d10z23.nc']
> > > Data Shape: (1536, 1536)
> > > Data Type: dtype('float64')
> > > dlat = 0.019999999553
> > > dlon = 0.019999999553
> > > Nlat = 1536
> > > Nlon = 1536
> > > true_lat_south_pole = -43.0
> > > true_lon_south_pole = 10.0
> > > rot_lat_ll = -14.7999992371
> > > rot_lon_ll = 341.199981689
> > > Attributes: {'long_name': 'FCST_word', 'init':
'20020910_230000',
> > 'valid':
> > > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
> 'delta_rot_lat':
> > > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
'Nlon':
> > 1536,
> > > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> > > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead':
> > '00',
> > > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > > DEBUG 2: regridding, if needed ...
> > > ERROR :
> > > ERROR : parse_vx_grid() -> The forecast and observation grids
do not
> > > match:
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000 !=
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000
> > > ERROR : Specify regridding logic in the config file "regrid"
section.
> > > ERROR :
> > >
> > > There are two problems with the output -- the obvious one being
the
> error
> > > and a second not-so-obvious one (but it could be related to
error).
> > >
> > >
> > > 1) The error: It is caused by trying to match projections
between FCST
> > and
> > > OBS. MTD is running single mode, so it is the same file with the
same
> > > projections and uses the same dict "attrs".
> > >
> > >
> > > 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> > > 341.2. but somehow the value becomes negative (-341.2, as
indicated in
> > the
> > > error message).
> > >
> > >
> > > There also seem to be floating point comparison and rounding
problems,
> > > which may be related to the above two problems.
> > >
> > >
> > > Best Regards
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > Sent: 14 April 2019 00:28:37
> > > To: Chan, Steven
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > > pole data
> > >
> > > Hi Steven -
> > >
> > > Specifying a rotated lat/lon grid from python should be
possible. The
> > > needed entries in the "atts" dictionary are
> > >
> > >
> > > //
> > > // name (string)
> > > //
> > > // rot_lat_ll, rot_lon_ll (double)
> > > //
> > > // delta_rot_lat, delta_rot_lon (double)
> > > //
> > > // Nlat, Nlon (int)
> > > //
> > > // true_lat_south_pole, true_lon_south_pole (double)
> > > //
> > > // aux_rotation (double)
> > >
> > > I copied these lines from the MET C++ source file
> > > "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> > > (along with the associated map projection) from the entries in a
python
> > > dictionary. The entries make a distinction between quantities
that
> > refer
> > > to rotated lat/lon (prefix "rot_" in the list above) and entries
that
> > refer
> > > to true (ie unrotated) lat/lon (prefix "true" in the list).
> > >
> > > As for the meaning of each entry, we have patterned it after the
GRIB2
> > > specification for rotated lat/lon grids (see chart here
> > > <
> > >
> >
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> > > >,
> > > especially the notes below the chart).
> > >
> > > To summarize ---
> > >
> > > Good news: Doing this should be possible.
> > >
> > > Bad news: It may take some experimentation with the rotation
> parameters.
> > > Most people (including those at key institutions who create the
grid
> > > specs!) don't have a good intuition for 3D space and rotations,
so you
> > > might have to work at it a bit.
> > >
> > >
> > > Hope this helps.
> > >
> > > Randy
> > >
> > >
> > >
> > > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
>
> > > >
> > > > Steven,
> > > >
> > > > It sounds like you're asking for an example of encoding a
rotated
> > lat/lon
> > > > projection in MET's Python interface. You'd like to know what
> elements
> > > > should be included in "attrs['grid']" to define a rotated
lat/lon
> grid.
> > > >
> > > > I'm going to assign this ticket to Randy, who knows the most
about
> the
> > > > Python interface and projection information. He should be in
touch
> > with
> > > > you next week about this.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > > > Queue: met_help
> > > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > > Owner: Nobody
> > > > > Requestors: steven.chan at metoffice.gov.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > > >
> > > > >
> > > > >
> > > > > Dear Met Tool Help:
> > > > >
> > > > >
> > > > > How are you?
> > > > >
> > > > > If I understood correctly, Met Tool 8.0 supports rotated
pole data
> > via
> > > a
> > > > > Python interface. I looked at the official Met Tool 8.0
> documentation
> > > --
> > > > it
> > > > > only states it support rotated pole, but I wasn't able to
find any
> > > > specific
> > > > > information and examples in how that is actually done.
> > > > >
> > > > >
> > > > > Does it follow conventions similar to the
read_ascii_numpy.py in
> > which
> > > > > data is read into met_data variable, plus the appropriate
> information
> > > in
> > > > > attrs dictionary? In this case, certain specific and
correct
> > > information
> > > > > needed for attrs['grid']?
> > > > >
> > > > >
> > > > > Thank you
> > > > >
> > > > > Steven Chan
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Tue May 14 03:05:54 2019
Dear Randy:
How are you?
I notice Met tool 8.1 have been released with certain Python interface
updates (no mention about the rotated poles). So does this imply if I
upgrade to 8.1, the problem that we had discussed about rotated poles
loaded and configured through Python interface for mtd is still likely
to encounter problems?
Thank you and Best Regards
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 18 April 2019 19:54:19
To: Chan, Steven
Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated
pole data
Hi Steve -
I got your tarball from the FTP site and copied it to my local
machine.
It'll likely be tuesday or so before I have a chance to look at it
though.
Sorry.
Randy
On Thu, Apr 18, 2019 at 6:26 AM Chan, Steven via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
>
> Dear Randy:
>
>
> Thank you very much for the help again. If there is something I can
help
> testing, I am more than willing to.
>
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 17 April 2019 19:24:07
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Steven -
>
> I'll have a look at the data. Can't promise anything, though ...
>
> Randy
>
> On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > I think I have found out how to upload to your FTP. The file is in
> > /incoming/irap/met_help/schan_UKMO and is
massif_2002_sample.tar.gz.
> >
> >
> > I have included 2 model days of data, python script used, config
file,
> and
> > sample output that shows the STDOUT and STDERR messages. The
Python
> script
> > uses a Met Office Python package, but all it does is to load
binary data
> > and is used to read the rotated pole metadata automatically; the
file
> > run_MTD.out (the STDOUT/STDERR messages) shows the actual rotated
pole
> > values used in Python script dict.
> >
> >
> > Best Regards,
> >
> > Steven Chan
> >
> > ________________________________
> > From: Chan, Steven
> > Sent: 17 April 2019 10:53:38
> > To: met_help at ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> >
> > Dear Randy:
> >
> >
> > Thank you for your email. The model data used in tests won't fit
in an
> > email, hence can I have information about the FTP please?
> >
> >
> > Thank you
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 16 April 2019 18:40:58
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Hello again, Steven -
> >
> > We're aware of the problems that MET sometimes has with testing
whether
> two
> > grids are the same or not. Your speculation about floating-point
> rounding
> > issues no doubt explains at least part of the trouble.
> >
> > Could you provide us with the data files you're using? Also the
config
> > file. If the data files are small enough, you could compress them
and
> > attach them to the email. Otherwise I can give you an ftp site
where you
> > can put them.
> >
> > Thanks in advance.
> >
> > Randy
> >
> > On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Dear Randy:
> > >
> > >
> > > Thank you very much for the reply. I was not aware of the Grib2
> > standards,
> > > and that site's info clarify much of the issues. I have also
resolved
> > some
> > > of the unusual aspects of Python library in the Met Office
network.
> > >
> > >
> > > Now some newer issues popped up -- I do not know if this is
specific to
> > > MTD tool or to the MET 8.0, but I got the following unexpected
results
> > and
> > > errors:
> > >
> > >
> > > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > > Input File: './infilelist'
> > > Data Name : 'FCST'
> > > ['ym200209d10z23.nc']
> > > Data Shape: (1536, 1536)
> > > Data Type: dtype('float64')
> > > dlat = 0.019999999553
> > > dlon = 0.019999999553
> > > Nlat = 1536
> > > Nlon = 1536
> > > true_lat_south_pole = -43.0
> > > true_lon_south_pole = 10.0
> > > rot_lat_ll = -14.7999992371
> > > rot_lon_ll = 341.199981689
> > > Attributes: {'long_name': 'FCST_word', 'init':
'20020910_230000',
> > 'valid':
> > > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
> 'delta_rot_lat':
> > > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
'Nlon':
> > 1536,
> > > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
'rot_lon_ll':
> > > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
'lead':
> > '00',
> > > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > > DEBUG 2: regridding, if needed ...
> > > ERROR :
> > > ERROR : parse_vx_grid() -> The forecast and observation grids
do not
> > > match:
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000 !=
> > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
rot_lat_ll:
> > -14.800
> > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
aux_rotation:
> > > 0.000
> > > ERROR : Specify regridding logic in the config file "regrid"
section.
> > > ERROR :
> > >
> > > There are two problems with the output -- the obvious one being
the
> error
> > > and a second not-so-obvious one (but it could be related to
error).
> > >
> > >
> > > 1) The error: It is caused by trying to match projections
between FCST
> > and
> > > OBS. MTD is running single mode, so it is the same file with the
same
> > > projections and uses the same dict "attrs".
> > >
> > >
> > > 2) Unexpected value for rot_lon_ll: "attrs" has set the value to
near
> > > 341.2. but somehow the value becomes negative (-341.2, as
indicated in
> > the
> > > error message).
> > >
> > >
> > > There also seem to be floating point comparison and rounding
problems,
> > > which may be related to the above two problems.
> > >
> > >
> > > Best Regards
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > Sent: 14 April 2019 00:28:37
> > > To: Chan, Steven
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > > pole data
> > >
> > > Hi Steven -
> > >
> > > Specifying a rotated lat/lon grid from python should be
possible. The
> > > needed entries in the "atts" dictionary are
> > >
> > >
> > > //
> > > // name (string)
> > > //
> > > // rot_lat_ll, rot_lon_ll (double)
> > > //
> > > // delta_rot_lat, delta_rot_lon (double)
> > > //
> > > // Nlat, Nlon (int)
> > > //
> > > // true_lat_south_pole, true_lon_south_pole (double)
> > > //
> > > // aux_rotation (double)
> > >
> > > I copied these lines from the MET C++ source file
> > > "grid_from_python_dict.cc". This code's job is to instantiate a
grid
> > > (along with the associated map projection) from the entries in a
python
> > > dictionary. The entries make a distinction between quantities
that
> > refer
> > > to rotated lat/lon (prefix "rot_" in the list above) and entries
that
> > refer
> > > to true (ie unrotated) lat/lon (prefix "true" in the list).
> > >
> > > As for the meaning of each entry, we have patterned it after the
GRIB2
> > > specification for rotated lat/lon grids (see chart here
> > > <
> > >
> >
> https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
1.shtml
> > > >,
> > > especially the notes below the chart).
> > >
> > > To summarize ---
> > >
> > > Good news: Doing this should be possible.
> > >
> > > Bad news: It may take some experimentation with the rotation
> parameters.
> > > Most people (including those at key institutions who create the
grid
> > > specs!) don't have a good intuition for 3D space and rotations,
so you
> > > might have to work at it a bit.
> > >
> > >
> > > Hope this helps.
> > >
> > > Randy
> > >
> > >
> > >
> > > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
>
> > > >
> > > > Steven,
> > > >
> > > > It sounds like you're asking for an example of encoding a
rotated
> > lat/lon
> > > > projection in MET's Python interface. You'd like to know what
> elements
> > > > should be included in "attrs['grid']" to define a rotated
lat/lon
> grid.
> > > >
> > > > I'm going to assign this ticket to Randy, who knows the most
about
> the
> > > > Python interface and projection information. He should be in
touch
> > with
> > > > you next week about this.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > > Transaction: Ticket created by steven.chan at metoffice.gov.uk
> > > > > Queue: met_help
> > > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > > Owner: Nobody
> > > > > Requestors: steven.chan at metoffice.gov.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > > >
> > > > >
> > > > >
> > > > > Dear Met Tool Help:
> > > > >
> > > > >
> > > > > How are you?
> > > > >
> > > > > If I understood correctly, Met Tool 8.0 supports rotated
pole data
> > via
> > > a
> > > > > Python interface. I looked at the official Met Tool 8.0
> documentation
> > > --
> > > > it
> > > > > only states it support rotated pole, but I wasn't able to
find any
> > > > specific
> > > > > information and examples in how that is actually done.
> > > > >
> > > > >
> > > > > Does it follow conventions similar to the
read_ascii_numpy.py in
> > which
> > > > > data is read into met_data variable, plus the appropriate
> information
> > > in
> > > > > attrs dictionary? In this case, certain specific and
correct
> > > information
> > > > > needed for attrs['grid']?
> > > > >
> > > > >
> > > > > Thank you
> > > > >
> > > > > Steven Chan
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Randy Bullock
Time: Thu May 16 11:43:28 2019
Hi again Steven -
I don't think the rotated pole lat/lon grid was part of anyh python
upgrades we did, so I think the problem likely persists.
I'm still looking at your data (in case you were wondering)
Randy
On Tue May 14 03:05:54 2019, steven.chan at metoffice.gov.uk wrote:
> Dear Randy:
>
>
> How are you?
>
> I notice Met tool 8.1 have been released with certain Python
interface
> updates (no mention about the rotated poles). So does this imply if
I
> upgrade to 8.1, the problem that we had discussed about rotated
poles
> loaded and configured through Python interface for mtd is still
likely
> to encounter problems?
>
> Thank you and Best Regards
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 18 April 2019 19:54:19
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hi Steve -
>
> I got your tarball from the FTP site and copied it to my local
> machine.
>
> It'll likely be tuesday or so before I have a chance to look at it
> though.
> Sorry.
>
> Randy
>
> On Thu, Apr 18, 2019 at 6:26 AM Chan, Steven via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > Thank you very much for the help again. If there is something I
can
> > help
> > testing, I am more than willing to.
> >
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 17 April 2019 19:24:07
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Steven -
> >
> > I'll have a look at the data. Can't promise anything, though ...
> >
> > Randy
> >
> > On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Dear Randy:
> > >
> > >
> > > I think I have found out how to upload to your FTP. The file is
in
> > > /incoming/irap/met_help/schan_UKMO and is
> > > massif_2002_sample.tar.gz.
> > >
> > >
> > > I have included 2 model days of data, python script used, config
> > > file,
> > and
> > > sample output that shows the STDOUT and STDERR messages. The
Python
> > script
> > > uses a Met Office Python package, but all it does is to load
binary
> > > data
> > > and is used to read the rotated pole metadata automatically; the
> > > file
> > > run_MTD.out (the STDOUT/STDERR messages) shows the actual
rotated
> > > pole
> > > values used in Python script dict.
> > >
> > >
> > > Best Regards,
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Chan, Steven
> > > Sent: 17 April 2019 10:53:38
> > > To: met_help at ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > rotated
> > > pole data
> > >
> > >
> > > Dear Randy:
> > >
> > >
> > > Thank you for your email. The model data used in tests won't fit
in
> > > an
> > > email, hence can I have information about the FTP please?
> > >
> > >
> > > Thank you
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > Sent: 16 April 2019 18:40:58
> > > To: Chan, Steven
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > rotated
> > > pole data
> > >
> > > Hello again, Steven -
> > >
> > > We're aware of the problems that MET sometimes has with testing
> > > whether
> > two
> > > grids are the same or not. Your speculation about floating-
point
> > rounding
> > > issues no doubt explains at least part of the trouble.
> > >
> > > Could you provide us with the data files you're using? Also the
> > > config
> > > file. If the data files are small enough, you could compress
them
> > > and
> > > attach them to the email. Otherwise I can give you an ftp site
> > > where you
> > > can put them.
> > >
> > > Thanks in advance.
> > >
> > > Randy
> > >
> > > On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
>
> > > >
> > > > Dear Randy:
> > > >
> > > >
> > > > Thank you very much for the reply. I was not aware of the
Grib2
> > > standards,
> > > > and that site's info clarify much of the issues. I have also
> > > > resolved
> > > some
> > > > of the unusual aspects of Python library in the Met Office
> > > > network.
> > > >
> > > >
> > > > Now some newer issues popped up -- I do not know if this is
> > > > specific to
> > > > MTD tool or to the MET 8.0, but I got the following unexpected
> > > > results
> > > and
> > > > errors:
> > > >
> > > >
> > > > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > > > Input File: './infilelist'
> > > > Data Name : 'FCST'
> > > > ['ym200209d10z23.nc']
> > > > Data Shape: (1536, 1536)
> > > > Data Type: dtype('float64')
> > > > dlat = 0.019999999553
> > > > dlon = 0.019999999553
> > > > Nlat = 1536
> > > > Nlon = 1536
> > > > true_lat_south_pole = -43.0
> > > > true_lon_south_pole = 10.0
> > > > rot_lat_ll = -14.7999992371
> > > > rot_lon_ll = 341.199981689
> > > > Attributes: {'long_name': 'FCST_word', 'init':
'20020910_230000',
> > > 'valid':
> > > > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
> > 'delta_rot_lat':
> > > > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
> > > > 'Nlon':
> > > 1536,
> > > > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > > > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
> > > > 'rot_lon_ll':
> > > > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
> > > > 'lead':
> > > '00',
> > > > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > > > DEBUG 2: regridding, if needed ...
> > > > ERROR :
> > > > ERROR : parse_vx_grid() -> The forecast and observation grids
do
> > > > not
> > > > match:
> > > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
> > > > rot_lat_ll:
> > > -14.800
> > > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
> > > > aux_rotation:
> > > > 0.000 !=
> > > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
> > > > rot_lat_ll:
> > > -14.800
> > > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
> > > > aux_rotation:
> > > > 0.000
> > > > ERROR : Specify regridding logic in the config file "regrid"
> > > > section.
> > > > ERROR :
> > > >
> > > > There are two problems with the output -- the obvious one
being
> > > > the
> > error
> > > > and a second not-so-obvious one (but it could be related to
> > > > error).
> > > >
> > > >
> > > > 1) The error: It is caused by trying to match projections
between
> > > > FCST
> > > and
> > > > OBS. MTD is running single mode, so it is the same file with
the
> > > > same
> > > > projections and uses the same dict "attrs".
> > > >
> > > >
> > > > 2) Unexpected value for rot_lon_ll: "attrs" has set the value
to
> > > > near
> > > > 341.2. but somehow the value becomes negative (-341.2, as
> > > > indicated in
> > > the
> > > > error message).
> > > >
> > > >
> > > > There also seem to be floating point comparison and rounding
> > > > problems,
> > > > which may be related to the above two problems.
> > > >
> > > >
> > > > Best Regards
> > > >
> > > > Steven Chan
> > > >
> > > > ________________________________
> > > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > > Sent: 14 April 2019 00:28:37
> > > > To: Chan, Steven
> > > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > > rotated
> > > > pole data
> > > >
> > > > Hi Steven -
> > > >
> > > > Specifying a rotated lat/lon grid from python should be
possible.
> > > > The
> > > > needed entries in the "atts" dictionary are
> > > >
> > > >
> > > > //
> > > > // name (string)
> > > > //
> > > > // rot_lat_ll, rot_lon_ll (double)
> > > > //
> > > > // delta_rot_lat, delta_rot_lon (double)
> > > > //
> > > > // Nlat, Nlon (int)
> > > > //
> > > > // true_lat_south_pole, true_lon_south_pole (double)
> > > > //
> > > > // aux_rotation (double)
> > > >
> > > > I copied these lines from the MET C++ source file
> > > > "grid_from_python_dict.cc". This code's job is to instantiate
a
> > > > grid
> > > > (along with the associated map projection) from the entries in
a
> > > > python
> > > > dictionary. The entries make a distinction between
quantities
> > > > that
> > > refer
> > > > to rotated lat/lon (prefix "rot_" in the list above) and
entries
> > > > that
> > > refer
> > > > to true (ie unrotated) lat/lon (prefix "true" in the list).
> > > >
> > > > As for the meaning of each entry, we have patterned it after
the
> > > > GRIB2
> > > > specification for rotated lat/lon grids (see chart here
> > > > <
> > > >
> > >
> >
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
> > 1.shtml
> > > > > ,
> > > > especially the notes below the chart).
> > > >
> > > > To summarize ---
> > > >
> > > > Good news: Doing this should be possible.
> > > >
> > > > Bad news: It may take some experimentation with the rotation
> > parameters.
> > > > Most people (including those at key institutions who create
the
> > > > grid
> > > > specs!) don't have a good intuition for 3D space and
rotations,
> > > > so you
> > > > might have to work at it a bit.
> > > >
> > > >
> > > > Hope this helps.
> > > >
> > > > Randy
> > > >
> > > >
> > > >
> > > > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > > > >
> > > > > Steven,
> > > > >
> > > > > It sounds like you're asking for an example of encoding a
> > > > > rotated
> > > lat/lon
> > > > > projection in MET's Python interface. You'd like to know
what
> > elements
> > > > > should be included in "attrs['grid']" to define a rotated
> > > > > lat/lon
> > grid.
> > > > >
> > > > > I'm going to assign this ticket to Randy, who knows the most
> > > > > about
> > the
> > > > > Python interface and projection information. He should be
in
> > > > > touch
> > > with
> > > > > you next week about this.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT <
> > met_help at ucar.edu
> > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > > > Transaction: Ticket created by
steven.chan at metoffice.gov.uk
> > > > > > Queue: met_help
> > > > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > > > Owner: Nobody
> > > > > > Requestors: steven.chan at metoffice.gov.uk
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > > > >
> > > > > >
> > > > > >
> > > > > > Dear Met Tool Help:
> > > > > >
> > > > > >
> > > > > > How are you?
> > > > > >
> > > > > > If I understood correctly, Met Tool 8.0 supports rotated
pole
> > > > > > data
> > > via
> > > > a
> > > > > > Python interface. I looked at the official Met Tool 8.0
> > documentation
> > > > --
> > > > > it
> > > > > > only states it support rotated pole, but I wasn't able to
> > > > > > find any
> > > > > specific
> > > > > > information and examples in how that is actually done.
> > > > > >
> > > > > >
> > > > > > Does it follow conventions similar to the
read_ascii_numpy.py
> > > > > > in
> > > which
> > > > > > data is read into met_data variable, plus the appropriate
> > information
> > > > in
> > > > > > attrs dictionary? In this case, certain specific and
correct
> > > > information
> > > > > > needed for attrs['grid']?
> > > > > >
> > > > > >
> > > > > > Thank you
> > > > > >
> > > > > > Steven Chan
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
------------------------------------------------
Subject: MET Tool 8.0 support of rotated pole data
From: Chan, Steven
Time: Fri May 17 03:08:20 2019
Dear Randy:
It is not a problem at all. Thank you very much for the help again.
Steven Chan
________________________________
From: Randy Bullock via RT <met_help at ucar.edu>
Sent: 16 May 2019 18:43:28
To: Chan, Steven
Subject: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of rotated pole
data
Hi again Steven -
I don't think the rotated pole lat/lon grid was part of anyh python
upgrades we did, so I think the problem likely persists.
I'm still looking at your data (in case you were wondering)
Randy
On Tue May 14 03:05:54 2019, steven.chan at metoffice.gov.uk wrote:
> Dear Randy:
>
>
> How are you?
>
> I notice Met tool 8.1 have been released with certain Python
interface
> updates (no mention about the rotated poles). So does this imply if
I
> upgrade to 8.1, the problem that we had discussed about rotated
poles
> loaded and configured through Python interface for mtd is still
likely
> to encounter problems?
>
> Thank you and Best Regards
>
> Steven Chan
>
> ________________________________
> From: Randy Bullock via RT <met_help at ucar.edu>
> Sent: 18 April 2019 19:54:19
> To: Chan, Steven
> Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> pole data
>
> Hi Steve -
>
> I got your tarball from the FTP site and copied it to my local
> machine.
>
> It'll likely be tuesday or so before I have a chance to look at it
> though.
> Sorry.
>
> Randy
>
> On Thu, Apr 18, 2019 at 6:26 AM Chan, Steven via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> >
> > Dear Randy:
> >
> >
> > Thank you very much for the help again. If there is something I
can
> > help
> > testing, I am more than willing to.
> >
> >
> > Steven Chan
> >
> > ________________________________
> > From: Randy Bullock via RT <met_help at ucar.edu>
> > Sent: 17 April 2019 19:24:07
> > To: Chan, Steven
> > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
rotated
> > pole data
> >
> > Steven -
> >
> > I'll have a look at the data. Can't promise anything, though ...
> >
> > Randy
> >
> > On Wed, Apr 17, 2019 at 7:28 AM Chan, Steven via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > >
> > > Dear Randy:
> > >
> > >
> > > I think I have found out how to upload to your FTP. The file is
in
> > > /incoming/irap/met_help/schan_UKMO and is
> > > massif_2002_sample.tar.gz.
> > >
> > >
> > > I have included 2 model days of data, python script used, config
> > > file,
> > and
> > > sample output that shows the STDOUT and STDERR messages. The
Python
> > script
> > > uses a Met Office Python package, but all it does is to load
binary
> > > data
> > > and is used to read the rotated pole metadata automatically; the
> > > file
> > > run_MTD.out (the STDOUT/STDERR messages) shows the actual
rotated
> > > pole
> > > values used in Python script dict.
> > >
> > >
> > > Best Regards,
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Chan, Steven
> > > Sent: 17 April 2019 10:53:38
> > > To: met_help at ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > rotated
> > > pole data
> > >
> > >
> > > Dear Randy:
> > >
> > >
> > > Thank you for your email. The model data used in tests won't fit
in
> > > an
> > > email, hence can I have information about the FTP please?
> > >
> > >
> > > Thank you
> > >
> > > Steven Chan
> > >
> > > ________________________________
> > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > Sent: 16 April 2019 18:40:58
> > > To: Chan, Steven
> > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > rotated
> > > pole data
> > >
> > > Hello again, Steven -
> > >
> > > We're aware of the problems that MET sometimes has with testing
> > > whether
> > two
> > > grids are the same or not. Your speculation about floating-
point
> > rounding
> > > issues no doubt explains at least part of the trouble.
> > >
> > > Could you provide us with the data files you're using? Also the
> > > config
> > > file. If the data files are small enough, you could compress
them
> > > and
> > > attach them to the email. Otherwise I can give you an ftp site
> > > where you
> > > can put them.
> > >
> > > Thanks in advance.
> > >
> > > Randy
> > >
> > > On Mon, Apr 15, 2019 at 10:19 AM Chan, Steven via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
>
> > > >
> > > > Dear Randy:
> > > >
> > > >
> > > > Thank you very much for the reply. I was not aware of the
Grib2
> > > standards,
> > > > and that site's info clarify much of the issues. I have also
> > > > resolved
> > > some
> > > > of the unusual aspects of Python library in the Met Office
> > > > network.
> > > >
> > > >
> > > > Now some newer issues popped up -- I do not know if this is
> > > > specific to
> > > > MTD tool or to the MET 8.0, but I got the following unexpected
> > > > results
> > > and
> > > > errors:
> > > >
> > > >
> > > > DEBUG 2: mtd_read_data() -> processing file "PYTHON_NUMPY"
> > > > Input File: './infilelist'
> > > > Data Name : 'FCST'
> > > > ['ym200209d10z23.nc']
> > > > Data Shape: (1536, 1536)
> > > > Data Type: dtype('float64')
> > > > dlat = 0.019999999553
> > > > dlon = 0.019999999553
> > > > Nlat = 1536
> > > > Nlon = 1536
> > > > true_lat_south_pole = -43.0
> > > > true_lon_south_pole = 10.0
> > > > rot_lat_ll = -14.7999992371
> > > > rot_lon_ll = 341.199981689
> > > > Attributes: {'long_name': 'FCST_word', 'init':
'20020910_230000',
> > > 'valid':
> > > > '20020910_230000', 'grid': {'true_lon_south_pole': 10.0,
> > 'delta_rot_lat':
> > > > 0.019999999552965164, 'delta_rot_lon': 0.019999999552965164,
> > > > 'Nlon':
> > > 1536,
> > > > 'true_lat_south_pole': -43.0, 'name': 'UKMO', 'rot_lat_ll':
> > > > -14.799999237060547, 'Nlat': 1536, 'aux_rotation': 0.0,
> > > > 'rot_lon_ll':
> > > > 341.1999816894531, 'type': 'Rotated LatLon'}, 'name': 'FCST',
> > > > 'lead':
> > > '00',
> > > > 'level': 'Surface', 'units': 'mm/hr', 'accum': '01'}
> > > > DEBUG 2: regridding, if needed ...
> > > > ERROR :
> > > > ERROR : parse_vx_grid() -> The forecast and observation grids
do
> > > > not
> > > > match:
> > > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
> > > > rot_lat_ll:
> > > -14.800
> > > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
> > > > aux_rotation:
> > > > 0.000 !=
> > > > ERROR : Projection: Rotated Lat/Lon Nx: 1536 Ny: 1536
> > > > rot_lat_ll:
> > > -14.800
> > > > rot_lon_ll: -341.200 delta_rot_lat: 0.020 delta_rot_lon: 0.020
> > > > true_lat_south_pole: -43.000 true_lon_south_pole: -10.000
> > > > aux_rotation:
> > > > 0.000
> > > > ERROR : Specify regridding logic in the config file "regrid"
> > > > section.
> > > > ERROR :
> > > >
> > > > There are two problems with the output -- the obvious one
being
> > > > the
> > error
> > > > and a second not-so-obvious one (but it could be related to
> > > > error).
> > > >
> > > >
> > > > 1) The error: It is caused by trying to match projections
between
> > > > FCST
> > > and
> > > > OBS. MTD is running single mode, so it is the same file with
the
> > > > same
> > > > projections and uses the same dict "attrs".
> > > >
> > > >
> > > > 2) Unexpected value for rot_lon_ll: "attrs" has set the value
to
> > > > near
> > > > 341.2. but somehow the value becomes negative (-341.2, as
> > > > indicated in
> > > the
> > > > error message).
> > > >
> > > >
> > > > There also seem to be floating point comparison and rounding
> > > > problems,
> > > > which may be related to the above two problems.
> > > >
> > > >
> > > > Best Regards
> > > >
> > > > Steven Chan
> > > >
> > > > ________________________________
> > > > From: Randy Bullock via RT <met_help at ucar.edu>
> > > > Sent: 14 April 2019 00:28:37
> > > > To: Chan, Steven
> > > > Subject: Re: [rt.rap.ucar.edu #89634] MET Tool 8.0 support of
> > > > rotated
> > > > pole data
> > > >
> > > > Hi Steven -
> > > >
> > > > Specifying a rotated lat/lon grid from python should be
possible.
> > > > The
> > > > needed entries in the "atts" dictionary are
> > > >
> > > >
> > > > //
> > > > // name (string)
> > > > //
> > > > // rot_lat_ll, rot_lon_ll (double)
> > > > //
> > > > // delta_rot_lat, delta_rot_lon (double)
> > > > //
> > > > // Nlat, Nlon (int)
> > > > //
> > > > // true_lat_south_pole, true_lon_south_pole (double)
> > > > //
> > > > // aux_rotation (double)
> > > >
> > > > I copied these lines from the MET C++ source file
> > > > "grid_from_python_dict.cc". This code's job is to instantiate
a
> > > > grid
> > > > (along with the associated map projection) from the entries in
a
> > > > python
> > > > dictionary. The entries make a distinction between
quantities
> > > > that
> > > refer
> > > > to rotated lat/lon (prefix "rot_" in the list above) and
entries
> > > > that
> > > refer
> > > > to true (ie unrotated) lat/lon (prefix "true" in the list).
> > > >
> > > > As for the meaning of each entry, we have patterned it after
the
> > > > GRIB2
> > > > specification for rotated lat/lon grids (see chart here
> > > > <
> > > >
> > >
> >
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-
> > 1.shtml
> > > > > ,
> > > > especially the notes below the chart).
> > > >
> > > > To summarize ---
> > > >
> > > > Good news: Doing this should be possible.
> > > >
> > > > Bad news: It may take some experimentation with the rotation
> > parameters.
> > > > Most people (including those at key institutions who create
the
> > > > grid
> > > > specs!) don't have a good intuition for 3D space and
rotations,
> > > > so you
> > > > might have to work at it a bit.
> > > >
> > > >
> > > > Hope this helps.
> > > >
> > > > Randy
> > > >
> > > >
> > > >
> > > > On Fri, Apr 12, 2019 at 12:31 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634 >
> > > > >
> > > > > Steven,
> > > > >
> > > > > It sounds like you're asking for an example of encoding a
> > > > > rotated
> > > lat/lon
> > > > > projection in MET's Python interface. You'd like to know
what
> > elements
> > > > > should be included in "attrs['grid']" to define a rotated
> > > > > lat/lon
> > grid.
> > > > >
> > > > > I'm going to assign this ticket to Randy, who knows the most
> > > > > about
> > the
> > > > > Python interface and projection information. He should be
in
> > > > > touch
> > > with
> > > > > you next week about this.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Apr 8, 2019 at 10:35 AM Chan, Steven via RT <
> > met_help at ucar.edu
> > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Mon Apr 08 10:35:22 2019: Request 89634 was acted upon.
> > > > > > Transaction: Ticket created by
steven.chan at metoffice.gov.uk
> > > > > > Queue: met_help
> > > > > > Subject: MET Tool 8.0 support of rotated pole data
> > > > > > Owner: Nobody
> > > > > > Requestors: steven.chan at metoffice.gov.uk
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89634
> > > > >
> > > > > >
> > > > > >
> > > > > > Dear Met Tool Help:
> > > > > >
> > > > > >
> > > > > > How are you?
> > > > > >
> > > > > > If I understood correctly, Met Tool 8.0 supports rotated
pole
> > > > > > data
> > > via
> > > > a
> > > > > > Python interface. I looked at the official Met Tool 8.0
> > documentation
> > > > --
> > > > > it
> > > > > > only states it support rotated pole, but I wasn't able to
> > > > > > find any
> > > > > specific
> > > > > > information and examples in how that is actually done.
> > > > > >
> > > > > >
> > > > > > Does it follow conventions similar to the
read_ascii_numpy.py
> > > > > > in
> > > which
> > > > > > data is read into met_data variable, plus the appropriate
> > information
> > > > in
> > > > > > attrs dictionary? In this case, certain specific and
correct
> > > > information
> > > > > > needed for attrs['grid']?
> > > > > >
> > > > > >
> > > > > > Thank you
> > > > > >
> > > > > > Steven Chan
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
------------------------------------------------
More information about the Met_help
mailing list