[Met_help] [rt.rap.ucar.edu #92748] History for using MET with IMS snow data
John Halley Gotway via RT
met_help at ucar.edu
Wed Oct 30 17:07:14 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good Morning,
I would love to use MET/MET Plus to verify the NOAA/NESDIS
Interactive Multisensor Snow and Ice Mapping System (IMS) data with ERA5
data. The IMS data is on a Polar Stereographic 4km grid the data format is
ASCII. The latitude and longitude definitions are in a separate file. I
can read in the data and plot with NCL. The ERA5 data that I am using is
the ERA5 land data. It is in grib format. I can choose the resolution
when down loading the data. Is this possible?
I will be working on Hera.
I hope this makes sense,
Many Thanks,
Sherrie
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Tue Oct 22 13:57:51 2019
Hello Sherrie,
Yes this is possible, but you'll need to use the python embedding
feature
of MET. Specifically, we'll need a python script which reads data
from
that ASCII file and defines the metadata for it (e.g. timing and grid
info). This is a relatively new feature in MET, and we're revising
the
logic for it for met-9.0, but reading data from ASCII should be very
straight-forward. I'd expect this to work well.
Examples of using python embedding to read gridded data into MET can
be
found here:
https://dtcenter.org/community-code/model-evaluation-tools-met/sample-
analysis-scripts
The last example on that page reads data from ASCII, so that might be
a
good place to start:
plot_data_plane PYTHON_NUMPY crain.ps 'name="read_AWS_CMORPH.py
AWS_CMORPH_crain-2017043006.txt";
I highly recommend starting with the plot_data_plane tool since you
can
compare the output plot to your NCL output to make sure you're passing
the
data to MET in the correct orientation with the correct grid info. If
you're comfortable with python, feel free to adapt that script to read
your
data. If you need help with this, please send us a sample data file,
the
lat/lon data file, and an NCL plot of that data.
Thanks,
John
On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> Queue: met_help
> Subject: using MET with IMS snow data
> Owner: Nobody
> Requestors: sherrie.fredrick at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
>
> Good Morning,
>
> I would love to use MET/MET Plus to verify the NOAA/NESDIS
> Interactive Multisensor Snow and Ice Mapping System (IMS) data with
ERA5
> data. The IMS data is on a Polar Stereographic 4km grid the data
format is
> ASCII. The latitude and longitude definitions are in a separate
file. I
> can read in the data and plot with NCL. The ERA5 data that I am
using is
> the ERA5 land data. It is in grib format. I can choose the
resolution
> when down loading the data. Is this possible?
>
> I will be working on Hera.
>
> I hope this makes sense,
> Many Thanks,
> Sherrie
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Tue Oct 22 15:19:31 2019
Hi John,
Thank you so much! I will give it a try.
cheers,
Sherrie
On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hello Sherrie,
>
> Yes this is possible, but you'll need to use the python embedding
feature
> of MET. Specifically, we'll need a python script which reads data
from
> that ASCII file and defines the metadata for it (e.g. timing and
grid
> info). This is a relatively new feature in MET, and we're revising
the
> logic for it for met-9.0, but reading data from ASCII should be very
> straight-forward. I'd expect this to work well.
>
> Examples of using python embedding to read gridded data into MET can
be
> found here:
>
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
>
> The last example on that page reads data from ASCII, so that might
be a
> good place to start:
> plot_data_plane PYTHON_NUMPY crain.ps 'name="read_AWS_CMORPH.py
> AWS_CMORPH_crain-2017043006.txt";
>
> I highly recommend starting with the plot_data_plane tool since you
can
> compare the output plot to your NCL output to make sure you're
passing the
> data to MET in the correct orientation with the correct grid info.
If
> you're comfortable with python, feel free to adapt that script to
read your
> data. If you need help with this, please send us a sample data
file, the
> lat/lon data file, and an NCL plot of that data.
>
> Thanks,
> John
>
> On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > Queue: met_help
> > Subject: using MET with IMS snow data
> > Owner: Nobody
> > Requestors: sherrie.fredrick at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> >
> > Good Morning,
> >
> > I would love to use MET/MET Plus to verify the NOAA/NESDIS
> > Interactive Multisensor Snow and Ice Mapping System (IMS) data
with ERA5
> > data. The IMS data is on a Polar Stereographic 4km grid the data
format
> is
> > ASCII. The latitude and longitude definitions are in a separate
file. I
> > can read in the data and plot with NCL. The ERA5 data that I am
using is
> > the ERA5 land data. It is in grib format. I can choose the
resolution
> > when down loading the data. Is this possible?
> >
> > I will be working on Hera.
> >
> > I hope this makes sense,
> > Many Thanks,
> > Sherrie
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Tue Oct 22 17:15:37 2019
Sherrie,
OK, great. FYI, unfortunately, the name of the required entries vary
depending on the projection type and we haven't done a good job
documenting
all of the variations yet. However, those entries are the same as the
global attributes you'll find in the NetCDF output from MET tools.
Here's an example of those attributes for a Polar Stereographic
projection:
:Projection = "Polar Stereographic" ;
:hemisphere = "N" ;
:scale_lat = "60.000000 degrees_north" ;
:lat_pin = "31.357000" ;
:lon_pin = "-129.770000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-120.500000" ;
:d_km = "10.395000 km" ;
:r_km = "6371.200000 km" ;
:nx = "169" ;
:ny = "154" ;
Hopefully that'll help.
Thanks,
John
On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Hi John,
> Thank you so much! I will give it a try.
>
> cheers,
> Sherrie
>
> On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hello Sherrie,
> >
> > Yes this is possible, but you'll need to use the python embedding
feature
> > of MET. Specifically, we'll need a python script which reads data
from
> > that ASCII file and defines the metadata for it (e.g. timing and
grid
> > info). This is a relatively new feature in MET, and we're
revising the
> > logic for it for met-9.0, but reading data from ASCII should be
very
> > straight-forward. I'd expect this to work well.
> >
> > Examples of using python embedding to read gridded data into MET
can be
> > found here:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> >
> > The last example on that page reads data from ASCII, so that might
be a
> > good place to start:
> > plot_data_plane PYTHON_NUMPY crain.ps 'name="read_AWS_CMORPH.py
> > AWS_CMORPH_crain-2017043006.txt";
> >
> > I highly recommend starting with the plot_data_plane tool since
you can
> > compare the output plot to your NCL output to make sure you're
passing
> the
> > data to MET in the correct orientation with the correct grid info.
If
> > you're comfortable with python, feel free to adapt that script to
read
> your
> > data. If you need help with this, please send us a sample data
file, the
> > lat/lon data file, and an NCL plot of that data.
> >
> > Thanks,
> > John
> >
> > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > Queue: met_help
> > > Subject: using MET with IMS snow data
> > > Owner: Nobody
> > > Requestors: sherrie.fredrick at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >
> > >
> > >
> > > Good Morning,
> > >
> > > I would love to use MET/MET Plus to verify the
NOAA/NESDIS
> > > Interactive Multisensor Snow and Ice Mapping System (IMS) data
with
> ERA5
> > > data. The IMS data is on a Polar Stereographic 4km grid the
data
> format
> > is
> > > ASCII. The latitude and longitude definitions are in a separate
> file. I
> > > can read in the data and plot with NCL. The ERA5 data that I am
using
> is
> > > the ERA5 land data. It is in grib format. I can choose the
> resolution
> > > when down loading the data. Is this possible?
> > >
> > > I will be working on Hera.
> > >
> > > I hope this makes sense,
> > > Many Thanks,
> > > Sherrie
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Wed Oct 23 11:05:49 2019
Good Morning John,
I have a couple more questions at your convenience,
I took the IMS ascii data and the lat/lon data and wrote a
Netcdf file that has both the IMS data and the lat/lon data in
it.dimensions:
ny = 6144 ;
nx = 6144 ;
variables:
float lat(ny, nx) ;
lat:_FillValue = 9.96921e+36f ;
float lon(ny, nx) ;
lon:_FillValue = 9.96921e+36f ;
float snowC(ny, nx) ;
snowC:_FillValue = 9.96921e+36f ;
}
I am not sure I understand the all of the metadata above.
I am not sure of what the following should be or how I should get it
:scale_lat = "? degrees_north" ;
:lat_pin = ?;
:lon_pin = ?;
:x_pin = ? ;
:y_pin = "0.000000" ;
:lon_orient = ? ;
The following I think is correct for the IMS polar stereographic
grid
:d_km = "4km" ;
:r_km = "6378.137km" ;
Since it is a 4km grid I believe I need to use a radius of 6378.137
according to the documentation.
The IMS grid is the WGS-84 ellipse.
Also my forecast is going to at 1kmX1km and the IMS grid is a 4km.
Do I
need to interpolate the IMS grid to a 1km grid?
Sorry for all the questions.
Many thanks for your help.
Sherrie
On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Sherrie,
>
> OK, great. FYI, unfortunately, the name of the required entries
vary
> depending on the projection type and we haven't done a good job
documenting
> all of the variations yet. However, those entries are the same as
the
> global attributes you'll find in the NetCDF output from MET tools.
>
> Here's an example of those attributes for a Polar Stereographic
projection:
>
> :Projection = "Polar Stereographic" ;
> :hemisphere = "N" ;
> :scale_lat = "60.000000 degrees_north" ;
> :lat_pin = "31.357000" ;
> :lon_pin = "-129.770000" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-120.500000" ;
> :d_km = "10.395000 km" ;
> :r_km = "6371.200000 km" ;
> :nx = "169" ;
> :ny = "154" ;
>
> Hopefully that'll help.
>
> Thanks,
> John
>
> On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Hi John,
> > Thank you so much! I will give it a try.
> >
> > cheers,
> > Sherrie
> >
> > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hello Sherrie,
> > >
> > > Yes this is possible, but you'll need to use the python
embedding
> feature
> > > of MET. Specifically, we'll need a python script which reads
data from
> > > that ASCII file and defines the metadata for it (e.g. timing and
grid
> > > info). This is a relatively new feature in MET, and we're
revising the
> > > logic for it for met-9.0, but reading data from ASCII should be
very
> > > straight-forward. I'd expect this to work well.
> > >
> > > Examples of using python embedding to read gridded data into MET
can be
> > > found here:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > >
> > > The last example on that page reads data from ASCII, so that
might be a
> > > good place to start:
> > > plot_data_plane PYTHON_NUMPY crain.ps
'name="read_AWS_CMORPH.py
> > > AWS_CMORPH_crain-2017043006.txt";
> > >
> > > I highly recommend starting with the plot_data_plane tool since
you can
> > > compare the output plot to your NCL output to make sure you're
passing
> > the
> > > data to MET in the correct orientation with the correct grid
info. If
> > > you're comfortable with python, feel free to adapt that script
to read
> > your
> > > data. If you need help with this, please send us a sample data
file,
> the
> > > lat/lon data file, and an NCL plot of that data.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > Queue: met_help
> > > > Subject: using MET with IMS snow data
> > > > Owner: Nobody
> > > > Requestors: sherrie.fredrick at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > >
> > > >
> > > >
> > > > Good Morning,
> > > >
> > > > I would love to use MET/MET Plus to verify the
NOAA/NESDIS
> > > > Interactive Multisensor Snow and Ice Mapping System (IMS) data
with
> > ERA5
> > > > data. The IMS data is on a Polar Stereographic 4km grid the
data
> > format
> > > is
> > > > ASCII. The latitude and longitude definitions are in a
separate
> > file. I
> > > > can read in the data and plot with NCL. The ERA5 data that I
am
> using
> > is
> > > > the ERA5 land data. It is in grib format. I can choose the
> > resolution
> > > > when down loading the data. Is this possible?
> > > >
> > > > I will be working on Hera.
> > > >
> > > > I hope this makes sense,
> > > > Many Thanks,
> > > > Sherrie
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Wed Oct 23 13:27:50 2019
Sherrie,
So we need to figure out the grid specification for the IMS polar
stereographic grid on which your data resides.
I did some hunting around online and suspect that you're processing
this
data:
https://nsidc.org/data/G02156
And the 4km resolution version of this data is on a 6444x6444 polar
stereographic grid.
On the "User's Guide" tab of that page, table 4 lists some relevant
information:
Latitude of true scale = 60 North
Longitude below the pole = 80 West
Based on that I'd say you want to try using the following settings:
:Projection = "Polar Stereographic" ;
:hemisphere = "N" ;
:scale_lat = "60.0 degrees_north" ;
:lat_pin = "???" ;
:lon_pin = "???" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-80.0" ;
:d_km = "4.0 km" ;
:r_km = "6371.2 km" ;
:nx = "6144" ;
:ny = "6144" ;
x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin values you
specify are for the "lower-left" corner of the grid. Look at the
lat/lon
values you have and try just specifying that lat/lon of the first
point.
Presumably, that'd be one of the 4 corners. And then run
plot_data_plane
to see what the image looks like. If it differs from your NCL plot,
we can
go from there.
Thanks,
John
On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Good Morning John,
>
> I have a couple more questions at your convenience,
> I took the IMS ascii data and the lat/lon data and wrote a
> Netcdf file that has both the IMS data and the lat/lon data in
> it.dimensions:
> ny = 6144 ;
> nx = 6144 ;
> variables:
> float lat(ny, nx) ;
> lat:_FillValue = 9.96921e+36f ;
> float lon(ny, nx) ;
> lon:_FillValue = 9.96921e+36f ;
> float snowC(ny, nx) ;
> snowC:_FillValue = 9.96921e+36f ;
> }
>
> I am not sure I understand the all of the metadata above.
> I am not sure of what the following should be or how I should get it
>
> :scale_lat = "? degrees_north" ;
> :lat_pin = ?;
> :lon_pin = ?;
> :x_pin = ? ;
> :y_pin = "0.000000" ;
> :lon_orient = ? ;
> The following I think is correct for the IMS polar stereographic
grid
> :d_km = "4km" ;
> :r_km = "6378.137km" ;
> Since it is a 4km grid I believe I need to use a radius of
6378.137
> according to the documentation.
> The IMS grid is the WGS-84 ellipse.
>
> Also my forecast is going to at 1kmX1km and the IMS grid is a
4km. Do I
> need to interpolate the IMS grid to a 1km grid?
>
> Sorry for all the questions.
> Many thanks for your help.
> Sherrie
>
> On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > OK, great. FYI, unfortunately, the name of the required entries
vary
> > depending on the projection type and we haven't done a good job
> documenting
> > all of the variations yet. However, those entries are the same as
the
> > global attributes you'll find in the NetCDF output from MET tools.
> >
> > Here's an example of those attributes for a Polar Stereographic
> projection:
> >
> > :Projection = "Polar Stereographic" ;
> > :hemisphere = "N" ;
> > :scale_lat = "60.000000 degrees_north" ;
> > :lat_pin = "31.357000" ;
> > :lon_pin = "-129.770000" ;
> > :x_pin = "0.000000" ;
> > :y_pin = "0.000000" ;
> > :lon_orient = "-120.500000" ;
> > :d_km = "10.395000 km" ;
> > :r_km = "6371.200000 km" ;
> > :nx = "169" ;
> > :ny = "154" ;
> >
> > Hopefully that'll help.
> >
> > Thanks,
> > John
> >
> > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >
> > > Hi John,
> > > Thank you so much! I will give it a try.
> > >
> > > cheers,
> > > Sherrie
> > >
> > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hello Sherrie,
> > > >
> > > > Yes this is possible, but you'll need to use the python
embedding
> > feature
> > > > of MET. Specifically, we'll need a python script which reads
data
> from
> > > > that ASCII file and defines the metadata for it (e.g. timing
and grid
> > > > info). This is a relatively new feature in MET, and we're
revising
> the
> > > > logic for it for met-9.0, but reading data from ASCII should
be very
> > > > straight-forward. I'd expect this to work well.
> > > >
> > > > Examples of using python embedding to read gridded data into
MET can
> be
> > > > found here:
> > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > >
> > > > The last example on that page reads data from ASCII, so that
might
> be a
> > > > good place to start:
> > > > plot_data_plane PYTHON_NUMPY crain.ps
'name="read_AWS_CMORPH.py
> > > > AWS_CMORPH_crain-2017043006.txt";
> > > >
> > > > I highly recommend starting with the plot_data_plane tool
since you
> can
> > > > compare the output plot to your NCL output to make sure you're
> passing
> > > the
> > > > data to MET in the correct orientation with the correct grid
info.
> If
> > > > you're comfortable with python, feel free to adapt that script
to
> read
> > > your
> > > > data. If you need help with this, please send us a sample
data file,
> > the
> > > > lat/lon data file, and an NCL plot of that data.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: using MET with IMS snow data
> > > > > Owner: Nobody
> > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > >
> > > > >
> > > > >
> > > > > Good Morning,
> > > > >
> > > > > I would love to use MET/MET Plus to verify the
NOAA/NESDIS
> > > > > Interactive Multisensor Snow and Ice Mapping System (IMS)
data with
> > > ERA5
> > > > > data. The IMS data is on a Polar Stereographic 4km grid the
data
> > > format
> > > > is
> > > > > ASCII. The latitude and longitude definitions are in a
separate
> > > file. I
> > > > > can read in the data and plot with NCL. The ERA5 data that
I am
> > using
> > > is
> > > > > the ERA5 land data. It is in grib format. I can choose
the
> > > resolution
> > > > > when down loading the data. Is this possible?
> > > > >
> > > > > I will be working on Hera.
> > > > >
> > > > > I hope this makes sense,
> > > > > Many Thanks,
> > > > > Sherrie
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Wed Oct 23 14:37:00 2019
Hi John,
Does plot_data_plane work on Netcdf files. I have the following
in my
netcdf file.
The error I get is
plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data file
Do I need to tell plot_data_plane that I have a netcdf file.
Thanks,
Sherrie
dimensions:
ny = 6144 ;
nx = 6144 ;
variables:
float lat(ny, nx) ;
lat:long_name = "latitude" ;
lat:lon_pin = 145.f ;
lat:lat_pin = -21.49193f ;
lat:ny = "6144" ;lat:nx = "6144" ;
lat:r_km = "6371.2 km" ;
lat:d_km = "4.0km" ;
lat:lon_orient = "-80.0" ;
lat:y_pin = "0.000000" ;
lat:x_pin = "0.000000" ;
lat:scale_lat = "60.0 degrees_north" ;
lat:hemisphere = "N" ;
lat:projection = "Polar Stereographic" ;
lat:_FillValue = 9.96921e+36f ;
float lon(ny, nx) ;
lon:long_name = "longitude" ;
lon:lon_pin = 145.f ;
lon:lat_pin = -21.49193f ;
lon:ny = "6144" ;
lon:nx = "6144" ;
lon:r_km = "6371.2 km" ;
lon:d_km = "4.0km" ;
lon:lon_orient = "-80.0" ;
lon:y_pin = "0.000000" ;
lon:x_pin = "0.000000" ;
lon:scale_lat = "60.0 degrees_north" ;
lon:hemisphere = "N" ;
lon:projection = "Polar Stereographic" ;
lon:_FillValue = 9.96921e+36f ;
float snowC(ny, nx) ;
snowC:units = "%" ;
snowC:long_name = "Snow Cover" ;
snowC:lon_pin = 145.f ;
snowC:lat_pin = -21.49193f ;
snowC:ny = "6144" ;
snowC:nx = "6144" ;
snowC:r_km = "6371.2 km" ;
snowC:d_km = "4.0km" ;
snowC:lon_orient = "-80.0" ;
snowC:y_pin = "0.000000" ;
snowC:x_pin = "0.000000" ;
snowC:scale_lat = "60.0 degrees_north" ;
snowC:hemisphere = "N" ;
snowC:projection = "Polar Stereographic" ;
snowC:_FillValue = 9.96921e+36f ;
On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Sherrie,
>
> So we need to figure out the grid specification for the IMS polar
> stereographic grid on which your data resides.
>
> I did some hunting around online and suspect that you're processing
this
> data:
> https://nsidc.org/data/G02156
>
> And the 4km resolution version of this data is on a 6444x6444 polar
> stereographic grid.
>
> On the "User's Guide" tab of that page, table 4 lists some relevant
> information:
> Latitude of true scale = 60 North
> Longitude below the pole = 80 West
>
> Based on that I'd say you want to try using the following settings:
>
> :Projection = "Polar Stereographic" ;
> :hemisphere = "N" ;
> :scale_lat = "60.0 degrees_north" ;
> :lat_pin = "???" ;
> :lon_pin = "???" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-80.0" ;
> :d_km = "4.0 km" ;
> :r_km = "6371.2 km" ;
> :nx = "6144" ;
> :ny = "6144" ;
>
> x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin values
you
> specify are for the "lower-left" corner of the grid. Look at the
lat/lon
> values you have and try just specifying that lat/lon of the first
point.
> Presumably, that'd be one of the 4 corners. And then run
plot_data_plane
> to see what the image looks like. If it differs from your NCL plot,
we can
> go from there.
>
> Thanks,
> John
>
> On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Good Morning John,
> >
> > I have a couple more questions at your convenience,
> > I took the IMS ascii data and the lat/lon data and wrote a
> > Netcdf file that has both the IMS data and the lat/lon data in
> > it.dimensions:
> > ny = 6144 ;
> > nx = 6144 ;
> > variables:
> > float lat(ny, nx) ;
> > lat:_FillValue = 9.96921e+36f ;
> > float lon(ny, nx) ;
> > lon:_FillValue = 9.96921e+36f ;
> > float snowC(ny, nx) ;
> > snowC:_FillValue = 9.96921e+36f ;
> > }
> >
> > I am not sure I understand the all of the metadata above.
> > I am not sure of what the following should be or how I should get
it
> >
> > :scale_lat = "? degrees_north" ;
> > :lat_pin = ?;
> > :lon_pin = ?;
> > :x_pin = ? ;
> > :y_pin = "0.000000" ;
> > :lon_orient = ? ;
> > The following I think is correct for the IMS polar stereographic
grid
> > :d_km = "4km" ;
> > :r_km = "6378.137km" ;
> > Since it is a 4km grid I believe I need to use a radius of
6378.137
> > according to the documentation.
> > The IMS grid is the WGS-84 ellipse.
> >
> > Also my forecast is going to at 1kmX1km and the IMS grid is a
4km.
> Do I
> > need to interpolate the IMS grid to a 1km grid?
> >
> > Sorry for all the questions.
> > Many thanks for your help.
> > Sherrie
> >
> > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > OK, great. FYI, unfortunately, the name of the required entries
vary
> > > depending on the projection type and we haven't done a good job
> > documenting
> > > all of the variations yet. However, those entries are the same
as the
> > > global attributes you'll find in the NetCDF output from MET
tools.
> > >
> > > Here's an example of those attributes for a Polar Stereographic
> > projection:
> > >
> > > :Projection = "Polar Stereographic" ;
> > > :hemisphere = "N" ;
> > > :scale_lat = "60.000000 degrees_north" ;
> > > :lat_pin = "31.357000" ;
> > > :lon_pin = "-129.770000" ;
> > > :x_pin = "0.000000" ;
> > > :y_pin = "0.000000" ;
> > > :lon_orient = "-120.500000" ;
> > > :d_km = "10.395000 km" ;
> > > :r_km = "6371.200000 km" ;
> > > :nx = "169" ;
> > > :ny = "154" ;
> > >
> > > Hopefully that'll help.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
Affiliate via
> RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
> > > >
> > > > Hi John,
> > > > Thank you so much! I will give it a try.
> > > >
> > > > cheers,
> > > > Sherrie
> > > >
> > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hello Sherrie,
> > > > >
> > > > > Yes this is possible, but you'll need to use the python
embedding
> > > feature
> > > > > of MET. Specifically, we'll need a python script which
reads data
> > from
> > > > > that ASCII file and defines the metadata for it (e.g. timing
and
> grid
> > > > > info). This is a relatively new feature in MET, and we're
revising
> > the
> > > > > logic for it for met-9.0, but reading data from ASCII should
be
> very
> > > > > straight-forward. I'd expect this to work well.
> > > > >
> > > > > Examples of using python embedding to read gridded data into
MET
> can
> > be
> > > > > found here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > >
> > > > > The last example on that page reads data from ASCII, so that
might
> > be a
> > > > > good place to start:
> > > > > plot_data_plane PYTHON_NUMPY crain.ps
'name="read_AWS_CMORPH.py
> > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > >
> > > > > I highly recommend starting with the plot_data_plane tool
since you
> > can
> > > > > compare the output plot to your NCL output to make sure
you're
> > passing
> > > > the
> > > > > data to MET in the correct orientation with the correct grid
info.
> > If
> > > > > you're comfortable with python, feel free to adapt that
script to
> > read
> > > > your
> > > > > data. If you need help with this, please send us a sample
data
> file,
> > > the
> > > > > lat/lon data file, and an NCL plot of that data.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA
Affiliate
> > via
> > > > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > > > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: using MET with IMS snow data
> > > > > > Owner: Nobody
> > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > >
> > > > > >
> > > > > >
> > > > > > Good Morning,
> > > > > >
> > > > > > I would love to use MET/MET Plus to verify the
NOAA/NESDIS
> > > > > > Interactive Multisensor Snow and Ice Mapping System (IMS)
data
> with
> > > > ERA5
> > > > > > data. The IMS data is on a Polar Stereographic 4km grid
the data
> > > > format
> > > > > is
> > > > > > ASCII. The latitude and longitude definitions are in a
separate
> > > > file. I
> > > > > > can read in the data and plot with NCL. The ERA5 data
that I am
> > > using
> > > > is
> > > > > > the ERA5 land data. It is in grib format. I can choose
the
> > > > resolution
> > > > > > when down loading the data. Is this possible?
> > > > > >
> > > > > > I will be working on Hera.
> > > > > >
> > > > > > I hope this makes sense,
> > > > > > Many Thanks,
> > > > > > Sherrie
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Wed Oct 23 16:04:04 2019
Sherrie,
My intention was to have you use the Python embedding option to read
this
data, but it looks like you're reformatting it into NetCDF instead.
And
that's fine too.
But there's some issues based on the NetCDF header you sent.
(1) Those projection attributes should be defined as global
attributes, not
separately for each variable.
(2) You are formatting a NetCDF file to follow the conventions used
for the
NetCDF output generated by the MET tools. If you add the global
attribute
"MET_version = "V8.0";" then MET will correctly process that file as
being
the NetCDF output of another MET tool.
(3) You'll need to add timing information as attributes for each data
variable.
By way of example, I've attached a header dump of some NetCDF output
from
MET.
Ultimately, whether you reformat the data into NetCDF for MET to read
or
pass it via python embedding, you're still defining the same info...
the
grid spec and timing information. The downside of reformatting to
NetCDF
is that pre-processing step and duplicating the data in ascii and
NetCDF
formats. But either way can get the job done.
If you're stuck, feel free to send me a file with your work in
progress,
and I can edit it to make MET happy.
Thanks,
John
dimensions:
lat = 154 ;
lon = 169 ;
variables:
float APCP_24_A24_ENS_MEAN(lat, lon) ;
APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at A24
Ensemble
Mean" ;
APCP_24_A24_ENS_MEAN:level = "A24" ;
APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
APCP_24_A24_ENS_MEAN:init_time = "20091231_120000" ;
APCP_24_A24_ENS_MEAN:init_time_ut = "1262260800" ;
APCP_24_A24_ENS_MEAN:valid_time = "20100101_120000" ;
APCP_24_A24_ENS_MEAN:valid_time_ut = "1262347200" ;
APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
// global attributes:
:FileOrigins = "File
../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc generated
20191023_165124 UTC on host number5 by t
he MET ensemble_stat tool" ;
:MET_version = "V9.0" ;
:MET_tool = "ensemble_stat" ;
:model = "WRF" ;
:obtype = "MC_PCP" ;
:Projection = "Polar Stereographic" ;
:hemisphere = "N" ;
:scale_lat = "60.000000 degrees_north" ;
:lat_pin = "31.357000" ;
:lon_pin = "-129.770000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-120.500000" ;
:d_km = "10.395000 km" ;
:r_km = "6371.200000 km" ;
:nx = "169" ;
:ny = "154" ;
On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Hi John,
>
> Does plot_data_plane work on Netcdf files. I have the
following in my
> netcdf file.
> The error I get is
> plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data file
> Do I need to tell plot_data_plane that I have a netcdf file.
>
> Thanks,
> Sherrie
> dimensions:
> ny = 6144 ;
> nx = 6144 ;
> variables:
> float lat(ny, nx) ;
> lat:long_name = "latitude" ;
> lat:lon_pin = 145.f ;
> lat:lat_pin = -21.49193f ;
> lat:ny = "6144" ;lat:nx = "6144" ;
> lat:r_km = "6371.2 km" ;
> lat:d_km = "4.0km" ;
> lat:lon_orient = "-80.0" ;
> lat:y_pin = "0.000000" ;
> lat:x_pin = "0.000000" ;
> lat:scale_lat = "60.0 degrees_north" ;
> lat:hemisphere = "N" ;
> lat:projection = "Polar Stereographic" ;
> lat:_FillValue = 9.96921e+36f ;
> float lon(ny, nx) ;
> lon:long_name = "longitude" ;
> lon:lon_pin = 145.f ;
> lon:lat_pin = -21.49193f ;
> lon:ny = "6144" ;
> lon:nx = "6144" ;
> lon:r_km = "6371.2 km" ;
> lon:d_km = "4.0km" ;
> lon:lon_orient = "-80.0" ;
> lon:y_pin = "0.000000" ;
> lon:x_pin = "0.000000" ;
> lon:scale_lat = "60.0 degrees_north" ;
> lon:hemisphere = "N" ;
> lon:projection = "Polar Stereographic" ;
> lon:_FillValue = 9.96921e+36f ;
> float snowC(ny, nx) ;
> snowC:units = "%" ;
> snowC:long_name = "Snow Cover" ;
> snowC:lon_pin = 145.f ;
> snowC:lat_pin = -21.49193f ;
> snowC:ny = "6144" ;
> snowC:nx = "6144" ;
> snowC:r_km = "6371.2 km" ;
> snowC:d_km = "4.0km" ;
> snowC:lon_orient = "-80.0" ;
> snowC:y_pin = "0.000000" ;
> snowC:x_pin = "0.000000" ;
> snowC:scale_lat = "60.0 degrees_north" ;
> snowC:hemisphere = "N" ;
> snowC:projection = "Polar Stereographic" ;
> snowC:_FillValue = 9.96921e+36f ;
>
> On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > So we need to figure out the grid specification for the IMS polar
> > stereographic grid on which your data resides.
> >
> > I did some hunting around online and suspect that you're
processing this
> > data:
> > https://nsidc.org/data/G02156
> >
> > And the 4km resolution version of this data is on a 6444x6444
polar
> > stereographic grid.
> >
> > On the "User's Guide" tab of that page, table 4 lists some
relevant
> > information:
> > Latitude of true scale = 60 North
> > Longitude below the pole = 80 West
> >
> > Based on that I'd say you want to try using the following
settings:
> >
> > :Projection = "Polar Stereographic" ;
> > :hemisphere = "N" ;
> > :scale_lat = "60.0 degrees_north" ;
> > :lat_pin = "???" ;
> > :lon_pin = "???" ;
> > :x_pin = "0.000000" ;
> > :y_pin = "0.000000" ;
> > :lon_orient = "-80.0" ;
> > :d_km = "4.0 km" ;
> > :r_km = "6371.2 km" ;
> > :nx = "6144" ;
> > :ny = "6144" ;
> >
> > x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin values
you
> > specify are for the "lower-left" corner of the grid. Look at the
lat/lon
> > values you have and try just specifying that lat/lon of the first
point.
> > Presumably, that'd be one of the 4 corners. And then run
plot_data_plane
> > to see what the image looks like. If it differs from your NCL
plot, we
> can
> > go from there.
> >
> > Thanks,
> > John
> >
> > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >
> > > Good Morning John,
> > >
> > > I have a couple more questions at your convenience,
> > > I took the IMS ascii data and the lat/lon data and wrote a
> > > Netcdf file that has both the IMS data and the lat/lon data in
> > > it.dimensions:
> > > ny = 6144 ;
> > > nx = 6144 ;
> > > variables:
> > > float lat(ny, nx) ;
> > > lat:_FillValue = 9.96921e+36f ;
> > > float lon(ny, nx) ;
> > > lon:_FillValue = 9.96921e+36f ;
> > > float snowC(ny, nx) ;
> > > snowC:_FillValue = 9.96921e+36f ;
> > > }
> > >
> > > I am not sure I understand the all of the metadata above.
> > > I am not sure of what the following should be or how I should
get it
> > >
> > > :scale_lat = "? degrees_north" ;
> > > :lat_pin = ?;
> > > :lon_pin = ?;
> > > :x_pin = ? ;
> > > :y_pin = "0.000000" ;
> > > :lon_orient = ? ;
> > > The following I think is correct for the IMS polar
stereographic grid
> > > :d_km = "4km" ;
> > > :r_km = "6378.137km" ;
> > > Since it is a 4km grid I believe I need to use a radius of
6378.137
> > > according to the documentation.
> > > The IMS grid is the WGS-84 ellipse.
> > >
> > > Also my forecast is going to at 1kmX1km and the IMS grid is a
4km.
> > Do I
> > > need to interpolate the IMS grid to a 1km grid?
> > >
> > > Sorry for all the questions.
> > > Many thanks for your help.
> > > Sherrie
> > >
> > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > OK, great. FYI, unfortunately, the name of the required
entries vary
> > > > depending on the projection type and we haven't done a good
job
> > > documenting
> > > > all of the variations yet. However, those entries are the
same as
> the
> > > > global attributes you'll find in the NetCDF output from MET
tools.
> > > >
> > > > Here's an example of those attributes for a Polar
Stereographic
> > > projection:
> > > >
> > > > :Projection = "Polar Stereographic" ;
> > > > :hemisphere = "N" ;
> > > > :scale_lat = "60.000000 degrees_north" ;
> > > > :lat_pin = "31.357000" ;
> > > > :lon_pin = "-129.770000" ;
> > > > :x_pin = "0.000000" ;
> > > > :y_pin = "0.000000" ;
> > > > :lon_orient = "-120.500000" ;
> > > > :d_km = "10.395000 km" ;
> > > > :r_km = "6371.200000 km" ;
> > > > :nx = "169" ;
> > > > :ny = "154" ;
> > > >
> > > > Hopefully that'll help.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
Affiliate via
> > RT
> > > <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > >
> > > > > Hi John,
> > > > > Thank you so much! I will give it a try.
> > > > >
> > > > > cheers,
> > > > > Sherrie
> > > > >
> > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hello Sherrie,
> > > > > >
> > > > > > Yes this is possible, but you'll need to use the python
embedding
> > > > feature
> > > > > > of MET. Specifically, we'll need a python script which
reads
> data
> > > from
> > > > > > that ASCII file and defines the metadata for it (e.g.
timing and
> > grid
> > > > > > info). This is a relatively new feature in MET, and we're
> revising
> > > the
> > > > > > logic for it for met-9.0, but reading data from ASCII
should be
> > very
> > > > > > straight-forward. I'd expect this to work well.
> > > > > >
> > > > > > Examples of using python embedding to read gridded data
into MET
> > can
> > > be
> > > > > > found here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > > >
> > > > > > The last example on that page reads data from ASCII, so
that
> might
> > > be a
> > > > > > good place to start:
> > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> 'name="read_AWS_CMORPH.py
> > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > > >
> > > > > > I highly recommend starting with the plot_data_plane tool
since
> you
> > > can
> > > > > > compare the output plot to your NCL output to make sure
you're
> > > passing
> > > > > the
> > > > > > data to MET in the correct orientation with the correct
grid
> info.
> > > If
> > > > > > you're comfortable with python, feel free to adapt that
script to
> > > read
> > > > > your
> > > > > > data. If you need help with this, please send us a sample
data
> > file,
> > > > the
> > > > > > lat/lon data file, and an NCL plot of that data.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted upon.
> > > > > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: using MET with IMS snow data
> > > > > > > Owner: Nobody
> > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Good Morning,
> > > > > > >
> > > > > > > I would love to use MET/MET Plus to verify the
> NOAA/NESDIS
> > > > > > > Interactive Multisensor Snow and Ice Mapping System
(IMS) data
> > with
> > > > > ERA5
> > > > > > > data. The IMS data is on a Polar Stereographic 4km grid
the
> data
> > > > > format
> > > > > > is
> > > > > > > ASCII. The latitude and longitude definitions are in a
> separate
> > > > > file. I
> > > > > > > can read in the data and plot with NCL. The ERA5 data
that I
> am
> > > > using
> > > > > is
> > > > > > > the ERA5 land data. It is in grib format. I can
choose the
> > > > > resolution
> > > > > > > when down loading the data. Is this possible?
> > > > > > >
> > > > > > > I will be working on Hera.
> > > > > > >
> > > > > > > I hope this makes sense,
> > > > > > > Many Thanks,
> > > > > > > Sherrie
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Fri Oct 25 09:52:51 2019
Hi John,
I was out yesterday, but am working on it today.
Thanks for the information. I havn't qute got it working yet.
But it
no longer tells me that I have the wrong
format.
I mainly converted the file to netcdf because I feel most
comfortable
with netcdf files, but I very much appreciate your
comments.
Many Thanks,
Sherrie
On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Sherrie,
>
> My intention was to have you use the Python embedding option to read
this
> data, but it looks like you're reformatting it into NetCDF instead.
And
> that's fine too.
>
> But there's some issues based on the NetCDF header you sent.
>
> (1) Those projection attributes should be defined as global
attributes, not
> separately for each variable.
> (2) You are formatting a NetCDF file to follow the conventions used
for the
> NetCDF output generated by the MET tools. If you add the global
attribute
> "MET_version = "V8.0";" then MET will correctly process that file as
being
> the NetCDF output of another MET tool.
> (3) You'll need to add timing information as attributes for each
data
> variable.
>
> By way of example, I've attached a header dump of some NetCDF output
from
> MET.
>
> Ultimately, whether you reformat the data into NetCDF for MET to
read or
> pass it via python embedding, you're still defining the same info...
the
> grid spec and timing information. The downside of reformatting to
NetCDF
> is that pre-processing step and duplicating the data in ascii and
NetCDF
> formats. But either way can get the job done.
>
> If you're stuck, feel free to send me a file with your work in
progress,
> and I can edit it to make MET happy.
>
> Thanks,
> John
>
>
> dimensions:
> lat = 154 ;
> lon = 169 ;
> variables:
> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at A24
Ensemble
> Mean" ;
> APCP_24_A24_ENS_MEAN:level = "A24" ;
> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
> APCP_24_A24_ENS_MEAN:init_time = "20091231_120000" ;
> APCP_24_A24_ENS_MEAN:init_time_ut = "1262260800" ;
> APCP_24_A24_ENS_MEAN:valid_time = "20100101_120000" ;
> APCP_24_A24_ENS_MEAN:valid_time_ut = "1262347200" ;
> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
> // global attributes:
> :FileOrigins = "File
> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc generated
> 20191023_165124 UTC on host number5 by t
> he MET ensemble_stat tool" ;
> :MET_version = "V9.0" ;
> :MET_tool = "ensemble_stat" ;
> :model = "WRF" ;
> :obtype = "MC_PCP" ;
> :Projection = "Polar Stereographic" ;
> :hemisphere = "N" ;
> :scale_lat = "60.000000 degrees_north" ;
> :lat_pin = "31.357000" ;
> :lon_pin = "-129.770000" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-120.500000" ;
> :d_km = "10.395000 km" ;
> :r_km = "6371.200000 km" ;
> :nx = "169" ;
> :ny = "154" ;
>
>
> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Hi John,
> >
> > Does plot_data_plane work on Netcdf files. I have the
following in
> my
> > netcdf file.
> > The error I get is
> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data file
> > Do I need to tell plot_data_plane that I have a netcdf file.
> >
> > Thanks,
> > Sherrie
> > dimensions:
> > ny = 6144 ;
> > nx = 6144 ;
> > variables:
> > float lat(ny, nx) ;
> > lat:long_name = "latitude" ;
> > lat:lon_pin = 145.f ;
> > lat:lat_pin = -21.49193f ;
> > lat:ny = "6144" ;lat:nx = "6144" ;
> > lat:r_km = "6371.2 km" ;
> > lat:d_km = "4.0km" ;
> > lat:lon_orient = "-80.0" ;
> > lat:y_pin = "0.000000" ;
> > lat:x_pin = "0.000000" ;
> > lat:scale_lat = "60.0 degrees_north" ;
> > lat:hemisphere = "N" ;
> > lat:projection = "Polar Stereographic" ;
> > lat:_FillValue = 9.96921e+36f ;
> > float lon(ny, nx) ;
> > lon:long_name = "longitude" ;
> > lon:lon_pin = 145.f ;
> > lon:lat_pin = -21.49193f ;
> > lon:ny = "6144" ;
> > lon:nx = "6144" ;
> > lon:r_km = "6371.2 km" ;
> > lon:d_km = "4.0km" ;
> > lon:lon_orient = "-80.0" ;
> > lon:y_pin = "0.000000" ;
> > lon:x_pin = "0.000000" ;
> > lon:scale_lat = "60.0 degrees_north" ;
> > lon:hemisphere = "N" ;
> > lon:projection = "Polar Stereographic" ;
> > lon:_FillValue = 9.96921e+36f ;
> > float snowC(ny, nx) ;
> > snowC:units = "%" ;
> > snowC:long_name = "Snow Cover" ;
> > snowC:lon_pin = 145.f ;
> > snowC:lat_pin = -21.49193f ;
> > snowC:ny = "6144" ;
> > snowC:nx = "6144" ;
> > snowC:r_km = "6371.2 km" ;
> > snowC:d_km = "4.0km" ;
> > snowC:lon_orient = "-80.0" ;
> > snowC:y_pin = "0.000000" ;
> > snowC:x_pin = "0.000000" ;
> > snowC:scale_lat = "60.0 degrees_north" ;
> > snowC:hemisphere = "N" ;
> > snowC:projection = "Polar Stereographic" ;
> > snowC:_FillValue = 9.96921e+36f ;
> >
> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > So we need to figure out the grid specification for the IMS
polar
> > > stereographic grid on which your data resides.
> > >
> > > I did some hunting around online and suspect that you're
processing
> this
> > > data:
> > > https://nsidc.org/data/G02156
> > >
> > > And the 4km resolution version of this data is on a 6444x6444
polar
> > > stereographic grid.
> > >
> > > On the "User's Guide" tab of that page, table 4 lists some
relevant
> > > information:
> > > Latitude of true scale = 60 North
> > > Longitude below the pole = 80 West
> > >
> > > Based on that I'd say you want to try using the following
settings:
> > >
> > > :Projection = "Polar Stereographic" ;
> > > :hemisphere = "N" ;
> > > :scale_lat = "60.0 degrees_north" ;
> > > :lat_pin = "???" ;
> > > :lon_pin = "???" ;
> > > :x_pin = "0.000000" ;
> > > :y_pin = "0.000000" ;
> > > :lon_orient = "-80.0" ;
> > > :d_km = "4.0 km" ;
> > > :r_km = "6371.2 km" ;
> > > :nx = "6144" ;
> > > :ny = "6144" ;
> > >
> > > x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin
values you
> > > specify are for the "lower-left" corner of the grid. Look at
the
> lat/lon
> > > values you have and try just specifying that lat/lon of the
first
> point.
> > > Presumably, that'd be one of the 4 corners. And then run
> plot_data_plane
> > > to see what the image looks like. If it differs from your NCL
plot, we
> > can
> > > go from there.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
> > > >
> > > > Good Morning John,
> > > >
> > > > I have a couple more questions at your convenience,
> > > > I took the IMS ascii data and the lat/lon data and wrote a
> > > > Netcdf file that has both the IMS data and the lat/lon data in
> > > > it.dimensions:
> > > > ny = 6144 ;
> > > > nx = 6144 ;
> > > > variables:
> > > > float lat(ny, nx) ;
> > > > lat:_FillValue = 9.96921e+36f ;
> > > > float lon(ny, nx) ;
> > > > lon:_FillValue = 9.96921e+36f ;
> > > > float snowC(ny, nx) ;
> > > > snowC:_FillValue = 9.96921e+36f ;
> > > > }
> > > >
> > > > I am not sure I understand the all of the metadata above.
> > > > I am not sure of what the following should be or how I should
get it
> > > >
> > > > :scale_lat = "? degrees_north" ;
> > > > :lat_pin = ?;
> > > > :lon_pin = ?;
> > > > :x_pin = ? ;
> > > > :y_pin = "0.000000" ;
> > > > :lon_orient = ? ;
> > > > The following I think is correct for the IMS polar
stereographic
> grid
> > > > :d_km = "4km" ;
> > > > :r_km = "6378.137km" ;
> > > > Since it is a 4km grid I believe I need to use a radius of
6378.137
> > > > according to the documentation.
> > > > The IMS grid is the WGS-84 ellipse.
> > > >
> > > > Also my forecast is going to at 1kmX1km and the IMS grid is
a 4km.
> > > Do I
> > > > need to interpolate the IMS grid to a 1km grid?
> > > >
> > > > Sorry for all the questions.
> > > > Many thanks for your help.
> > > > Sherrie
> > > >
> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Sherrie,
> > > > >
> > > > > OK, great. FYI, unfortunately, the name of the required
entries
> vary
> > > > > depending on the projection type and we haven't done a good
job
> > > > documenting
> > > > > all of the variations yet. However, those entries are the
same as
> > the
> > > > > global attributes you'll find in the NetCDF output from MET
tools.
> > > > >
> > > > > Here's an example of those attributes for a Polar
Stereographic
> > > > projection:
> > > > >
> > > > > :Projection = "Polar Stereographic" ;
> > > > > :hemisphere = "N" ;
> > > > > :scale_lat = "60.000000 degrees_north" ;
> > > > > :lat_pin = "31.357000" ;
> > > > > :lon_pin = "-129.770000" ;
> > > > > :x_pin = "0.000000" ;
> > > > > :y_pin = "0.000000" ;
> > > > > :lon_orient = "-120.500000" ;
> > > > > :d_km = "10.395000 km" ;
> > > > > :r_km = "6371.200000 km" ;
> > > > > :nx = "169" ;
> > > > > :ny = "154" ;
> > > > >
> > > > > Hopefully that'll help.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT
> > > > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > > >
> > > > > > Hi John,
> > > > > > Thank you so much! I will give it a try.
> > > > > >
> > > > > > cheers,
> > > > > > Sherrie
> > > > > >
> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Sherrie,
> > > > > > >
> > > > > > > Yes this is possible, but you'll need to use the python
> embedding
> > > > > feature
> > > > > > > of MET. Specifically, we'll need a python script which
reads
> > data
> > > > from
> > > > > > > that ASCII file and defines the metadata for it (e.g.
timing
> and
> > > grid
> > > > > > > info). This is a relatively new feature in MET, and
we're
> > revising
> > > > the
> > > > > > > logic for it for met-9.0, but reading data from ASCII
should be
> > > very
> > > > > > > straight-forward. I'd expect this to work well.
> > > > > > >
> > > > > > > Examples of using python embedding to read gridded data
into
> MET
> > > can
> > > > be
> > > > > > > found here:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > > > >
> > > > > > > The last example on that page reads data from ASCII, so
that
> > might
> > > > be a
> > > > > > > good place to start:
> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > 'name="read_AWS_CMORPH.py
> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > > > >
> > > > > > > I highly recommend starting with the plot_data_plane
tool since
> > you
> > > > can
> > > > > > > compare the output plot to your NCL output to make sure
you're
> > > > passing
> > > > > > the
> > > > > > > data to MET in the correct orientation with the correct
grid
> > info.
> > > > If
> > > > > > > you're comfortable with python, feel free to adapt that
script
> to
> > > > read
> > > > > > your
> > > > > > > data. If you need help with this, please send us a
sample data
> > > file,
> > > > > the
> > > > > > > lat/lon data file, and an NCL plot of that data.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick - NOAA
> > Affiliate
> > > > via
> > > > > > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted
upon.
> > > > > > > > Transaction: Ticket created by
sherrie.fredrick at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: using MET with IMS snow data
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Good Morning,
> > > > > > > >
> > > > > > > > I would love to use MET/MET Plus to verify the
> > NOAA/NESDIS
> > > > > > > > Interactive Multisensor Snow and Ice Mapping System
(IMS)
> data
> > > with
> > > > > > ERA5
> > > > > > > > data. The IMS data is on a Polar Stereographic 4km
grid the
> > data
> > > > > > format
> > > > > > > is
> > > > > > > > ASCII. The latitude and longitude definitions are in
a
> > separate
> > > > > > file. I
> > > > > > > > can read in the data and plot with NCL. The ERA5 data
that I
> > am
> > > > > using
> > > > > > is
> > > > > > > > the ERA5 land data. It is in grib format. I can
choose the
> > > > > > resolution
> > > > > > > > when down loading the data. Is this possible?
> > > > > > > >
> > > > > > > > I will be working on Hera.
> > > > > > > >
> > > > > > > > I hope this makes sense,
> > > > > > > > Many Thanks,
> > > > > > > > Sherrie
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Mon Oct 28 10:30:05 2019
Good Morning John,
I have my ERA5 data and my IMS data. MET is still unable to read
my
IMS data. The message I get is
DEBUG 1: Observation File:
/scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/imc_oct-01-
2017.nc
DEBUG 2:
DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &, DataPlane &)
const
-> data was flipped to north.
ERROR :
ERROR : NcCfFile::getData(NcVar *, const LongArray &, DataPlane &)
const
-> star found in bad slot
ERROR :
I have no idea about this error. I am sorry to need so much help.
A header dump of my IMS Netcdf file is
dimensions:
time = UNLIMITED ; // (1 currently)
ny = 6144 ;
nx = 6144 ;
variables:
double time(time) ;
time:calendar = "standard" ;
time:units = "hours since 1900-01-01 00:00:00" ;
time:long_name = "time" ;
float lat(ny, nx) ;
lat:units = "degrees_north" ;
lat:long_name = "latitude" ;
lat:forecast_time = 0 ;
lat:forecast_time_units = "hours" ;
lat:initial_time = "10/01/2017 (00:00)" ;
lat:_FillValue = 9.96921e+36f ;
float lon(ny, nx) ;
lon:units = "degrees_east" ;
lon:long_name = "longitude" ;
lon:forecast_time = 0 ;
lon:forecast_time_units = "hours" ;
lon:initial_time = "10/01/2017 (00:00)" ;
lon:_FillValue = 9.96921e+36f ;
float snowC(ny, nx) ;
snowC:level = "Z0" ;
snowC:units = "%" ;
snowC:long_name = "Snow Cover" ;
snowC:forecast_time = 0 ;
snowC:forecast_time_units = "hours" ;
snowC:initial_time = "10/01/2017 (00:00)" ;
snowC:_FillValue = 9.96921e+36f ;
// global attributes:
:lon_pin = 145.f ;
:lat_pin = -21.49193f ;
:ny = "6144" ;
:nx = "6144" ;
:r_km = "6371.2 km" ;
:d_km = "4.0km" ;
:lon_orient = "-80.0" ;
:y_pin = "0.000000" ;
:x_pin = "0.000000" ;
:scale_lat = "60.0 degrees_north" ;
:hemisphere = "N" ;
:projection = "Polar Stereographic" ;
:MET_version = "V8.1" ;
I am working on Hera in the directory
/scratch2/BMC/refcst2/Sherrie.Fredrick/MET
My Netcdf IMS files are in
/scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
Thanks,
Sherrie
>
> On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Sherrie,
>>
>> My intention was to have you use the Python embedding option to
read this
>> data, but it looks like you're reformatting it into NetCDF instead.
And
>> that's fine too.
>>
>> But there's some issues based on the NetCDF header you sent.
>>
>> (1) Those projection attributes should be defined as global
attributes,
>> not
>> separately for each variable.
>> (2) You are formatting a NetCDF file to follow the conventions used
for
>> the
>> NetCDF output generated by the MET tools. If you add the global
attribute
>> "MET_version = "V8.0";" then MET will correctly process that file
as being
>> the NetCDF output of another MET tool.
>> (3) You'll need to add timing information as attributes for each
data
>> variable.
>>
>> By way of example, I've attached a header dump of some NetCDF
output from
>> MET.
>>
>> Ultimately, whether you reformat the data into NetCDF for MET to
read or
>> pass it via python embedding, you're still defining the same
info... the
>> grid spec and timing information. The downside of reformatting to
NetCDF
>> is that pre-processing step and duplicating the data in ascii and
NetCDF
>> formats. But either way can get the job done.
>>
>> If you're stuck, feel free to send me a file with your work in
progress,
>> and I can edit it to make MET happy.
>>
>> Thanks,
>> John
>>
>>
>> dimensions:
>> lat = 154 ;
>> lon = 169 ;
>> variables:
>> float APCP_24_A24_ENS_MEAN(lat, lon) ;
>> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
>> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at A24
Ensemble
>> Mean" ;
>> APCP_24_A24_ENS_MEAN:level = "A24" ;
>> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
>> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
>> APCP_24_A24_ENS_MEAN:init_time = "20091231_120000" ;
>> APCP_24_A24_ENS_MEAN:init_time_ut = "1262260800" ;
>> APCP_24_A24_ENS_MEAN:valid_time = "20100101_120000"
;
>> APCP_24_A24_ENS_MEAN:valid_time_ut = "1262347200" ;
>> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
>> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
>> // global attributes:
>> :FileOrigins = "File
>> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
generated
>> 20191023_165124 UTC on host number5 by t
>> he MET ensemble_stat tool" ;
>> :MET_version = "V9.0" ;
>> :MET_tool = "ensemble_stat" ;
>> :model = "WRF" ;
>> :obtype = "MC_PCP" ;
>> :Projection = "Polar Stereographic" ;
>> :hemisphere = "N" ;
>> :scale_lat = "60.000000 degrees_north" ;
>> :lat_pin = "31.357000" ;
>> :lon_pin = "-129.770000" ;
>> :x_pin = "0.000000" ;
>> :y_pin = "0.000000" ;
>> :lon_orient = "-120.500000" ;
>> :d_km = "10.395000 km" ;
>> :r_km = "6371.200000 km" ;
>> :nx = "169" ;
>> :ny = "154" ;
>>
>>
>> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>> >
>> > Hi John,
>> >
>> > Does plot_data_plane work on Netcdf files. I have the
following
>> in my
>> > netcdf file.
>> > The error I get is
>> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data
file
>> > Do I need to tell plot_data_plane that I have a netcdf file.
>> >
>> > Thanks,
>> > Sherrie
>> > dimensions:
>> > ny = 6144 ;
>> > nx = 6144 ;
>> > variables:
>> > float lat(ny, nx) ;
>> > lat:long_name = "latitude" ;
>> > lat:lon_pin = 145.f ;
>> > lat:lat_pin = -21.49193f ;
>> > lat:ny = "6144" ;lat:nx = "6144" ;
>> > lat:r_km = "6371.2 km" ;
>> > lat:d_km = "4.0km" ;
>> > lat:lon_orient = "-80.0" ;
>> > lat:y_pin = "0.000000" ;
>> > lat:x_pin = "0.000000" ;
>> > lat:scale_lat = "60.0 degrees_north" ;
>> > lat:hemisphere = "N" ;
>> > lat:projection = "Polar Stereographic" ;
>> > lat:_FillValue = 9.96921e+36f ;
>> > float lon(ny, nx) ;
>> > lon:long_name = "longitude" ;
>> > lon:lon_pin = 145.f ;
>> > lon:lat_pin = -21.49193f ;
>> > lon:ny = "6144" ;
>> > lon:nx = "6144" ;
>> > lon:r_km = "6371.2 km" ;
>> > lon:d_km = "4.0km" ;
>> > lon:lon_orient = "-80.0" ;
>> > lon:y_pin = "0.000000" ;
>> > lon:x_pin = "0.000000" ;
>> > lon:scale_lat = "60.0 degrees_north" ;
>> > lon:hemisphere = "N" ;
>> > lon:projection = "Polar Stereographic" ;
>> > lon:_FillValue = 9.96921e+36f ;
>> > float snowC(ny, nx) ;
>> > snowC:units = "%" ;
>> > snowC:long_name = "Snow Cover" ;
>> > snowC:lon_pin = 145.f ;
>> > snowC:lat_pin = -21.49193f ;
>> > snowC:ny = "6144" ;
>> > snowC:nx = "6144" ;
>> > snowC:r_km = "6371.2 km" ;
>> > snowC:d_km = "4.0km" ;
>> > snowC:lon_orient = "-80.0" ;
>> > snowC:y_pin = "0.000000" ;
>> > snowC:x_pin = "0.000000" ;
>> > snowC:scale_lat = "60.0 degrees_north" ;
>> > snowC:hemisphere = "N" ;
>> > snowC:projection = "Polar Stereographic" ;
>> > snowC:_FillValue = 9.96921e+36f ;
>> >
>> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Sherrie,
>> > >
>> > > So we need to figure out the grid specification for the IMS
polar
>> > > stereographic grid on which your data resides.
>> > >
>> > > I did some hunting around online and suspect that you're
processing
>> this
>> > > data:
>> > > https://nsidc.org/data/G02156
>> > >
>> > > And the 4km resolution version of this data is on a 6444x6444
polar
>> > > stereographic grid.
>> > >
>> > > On the "User's Guide" tab of that page, table 4 lists some
relevant
>> > > information:
>> > > Latitude of true scale = 60 North
>> > > Longitude below the pole = 80 West
>> > >
>> > > Based on that I'd say you want to try using the following
settings:
>> > >
>> > > :Projection = "Polar Stereographic" ;
>> > > :hemisphere = "N" ;
>> > > :scale_lat = "60.0 degrees_north" ;
>> > > :lat_pin = "???" ;
>> > > :lon_pin = "???" ;
>> > > :x_pin = "0.000000" ;
>> > > :y_pin = "0.000000" ;
>> > > :lon_orient = "-80.0" ;
>> > > :d_km = "4.0 km" ;
>> > > :r_km = "6371.2 km" ;
>> > > :nx = "6144" ;
>> > > :ny = "6144" ;
>> > >
>> > > x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin
values you
>> > > specify are for the "lower-left" corner of the grid. Look at
the
>> lat/lon
>> > > values you have and try just specifying that lat/lon of the
first
>> point.
>> > > Presumably, that'd be one of the 4 corners. And then run
>> plot_data_plane
>> > > to see what the image looks like. If it differs from your NCL
plot,
>> we
>> > can
>> > > go from there.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA
Affiliate via
>> > RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
>> > > >
>> > > > Good Morning John,
>> > > >
>> > > > I have a couple more questions at your convenience,
>> > > > I took the IMS ascii data and the lat/lon data and wrote
a
>> > > > Netcdf file that has both the IMS data and the lat/lon data
in
>> > > > it.dimensions:
>> > > > ny = 6144 ;
>> > > > nx = 6144 ;
>> > > > variables:
>> > > > float lat(ny, nx) ;
>> > > > lat:_FillValue = 9.96921e+36f ;
>> > > > float lon(ny, nx) ;
>> > > > lon:_FillValue = 9.96921e+36f ;
>> > > > float snowC(ny, nx) ;
>> > > > snowC:_FillValue = 9.96921e+36f ;
>> > > > }
>> > > >
>> > > > I am not sure I understand the all of the metadata above.
>> > > > I am not sure of what the following should be or how I should
get it
>> > > >
>> > > > :scale_lat = "? degrees_north" ;
>> > > > :lat_pin = ?;
>> > > > :lon_pin = ?;
>> > > > :x_pin = ? ;
>> > > > :y_pin = "0.000000" ;
>> > > > :lon_orient = ? ;
>> > > > The following I think is correct for the IMS polar
stereographic
>> grid
>> > > > :d_km = "4km" ;
>> > > > :r_km = "6378.137km" ;
>> > > > Since it is a 4km grid I believe I need to use a radius of
>> 6378.137
>> > > > according to the documentation.
>> > > > The IMS grid is the WGS-84 ellipse.
>> > > >
>> > > > Also my forecast is going to at 1kmX1km and the IMS grid
is a
>> 4km.
>> > > Do I
>> > > > need to interpolate the IMS grid to a 1km grid?
>> > > >
>> > > > Sorry for all the questions.
>> > > > Many thanks for your help.
>> > > > Sherrie
>> > > >
>> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
>> > > > met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Sherrie,
>> > > > >
>> > > > > OK, great. FYI, unfortunately, the name of the required
entries
>> vary
>> > > > > depending on the projection type and we haven't done a good
job
>> > > > documenting
>> > > > > all of the variations yet. However, those entries are the
same as
>> > the
>> > > > > global attributes you'll find in the NetCDF output from MET
tools.
>> > > > >
>> > > > > Here's an example of those attributes for a Polar
Stereographic
>> > > > projection:
>> > > > >
>> > > > > :Projection = "Polar Stereographic" ;
>> > > > > :hemisphere = "N" ;
>> > > > > :scale_lat = "60.000000 degrees_north" ;
>> > > > > :lat_pin = "31.357000" ;
>> > > > > :lon_pin = "-129.770000" ;
>> > > > > :x_pin = "0.000000" ;
>> > > > > :y_pin = "0.000000" ;
>> > > > > :lon_orient = "-120.500000" ;
>> > > > > :d_km = "10.395000 km" ;
>> > > > > :r_km = "6371.200000 km" ;
>> > > > > :nx = "169" ;
>> > > > > :ny = "154" ;
>> > > > >
>> > > > > Hopefully that'll help.
>> > > > >
>> > > > > Thanks,
>> > > > > John
>> > > > >
>> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
Affiliate
>> via
>> > > RT
>> > > > <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>> > > > > >
>> > > > > > Hi John,
>> > > > > > Thank you so much! I will give it a try.
>> > > > > >
>> > > > > > cheers,
>> > > > > > Sherrie
>> > > > > >
>> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via RT
<
>> > > > > > met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hello Sherrie,
>> > > > > > >
>> > > > > > > Yes this is possible, but you'll need to use the python
>> embedding
>> > > > > feature
>> > > > > > > of MET. Specifically, we'll need a python script which
reads
>> > data
>> > > > from
>> > > > > > > that ASCII file and defines the metadata for it (e.g.
timing
>> and
>> > > grid
>> > > > > > > info). This is a relatively new feature in MET, and
we're
>> > revising
>> > > > the
>> > > > > > > logic for it for met-9.0, but reading data from ASCII
should
>> be
>> > > very
>> > > > > > > straight-forward. I'd expect this to work well.
>> > > > > > >
>> > > > > > > Examples of using python embedding to read gridded data
into
>> MET
>> > > can
>> > > > be
>> > > > > > > found here:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
>> > > > > > >
>> > > > > > > The last example on that page reads data from ASCII, so
that
>> > might
>> > > > be a
>> > > > > > > good place to start:
>> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
>> > 'name="read_AWS_CMORPH.py
>> > > > > > > AWS_CMORPH_crain-2017043006.txt";
>> > > > > > >
>> > > > > > > I highly recommend starting with the plot_data_plane
tool
>> since
>> > you
>> > > > can
>> > > > > > > compare the output plot to your NCL output to make sure
you're
>> > > > passing
>> > > > > > the
>> > > > > > > data to MET in the correct orientation with the correct
grid
>> > info.
>> > > > If
>> > > > > > > you're comfortable with python, feel free to adapt that
>> script to
>> > > > read
>> > > > > > your
>> > > > > > > data. If you need help with this, please send us a
sample
>> data
>> > > file,
>> > > > > the
>> > > > > > > lat/lon data file, and an NCL plot of that data.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > John
>> > > > > > >
>> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick -
NOAA
>> > Affiliate
>> > > > via
>> > > > > > RT <
>> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted
upon.
>> > > > > > > > Transaction: Ticket created by
sherrie.fredrick at noaa.gov
>> > > > > > > > Queue: met_help
>> > > > > > > > Subject: using MET with IMS snow data
>> > > > > > > > Owner: Nobody
>> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
>> > > > > > > > Status: new
>> > > > > > > > Ticket <URL:
>> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Good Morning,
>> > > > > > > >
>> > > > > > > > I would love to use MET/MET Plus to verify the
>> > NOAA/NESDIS
>> > > > > > > > Interactive Multisensor Snow and Ice Mapping System
(IMS)
>> data
>> > > with
>> > > > > > ERA5
>> > > > > > > > data. The IMS data is on a Polar Stereographic 4km
grid the
>> > data
>> > > > > > format
>> > > > > > > is
>> > > > > > > > ASCII. The latitude and longitude definitions are in
a
>> > separate
>> > > > > > file. I
>> > > > > > > > can read in the data and plot with NCL. The ERA5
data that
>> I
>> > am
>> > > > > using
>> > > > > > is
>> > > > > > > > the ERA5 land data. It is in grib format. I can
choose
>> the
>> > > > > > resolution
>> > > > > > > > when down loading the data. Is this possible?
>> > > > > > > >
>> > > > > > > > I will be working on Hera.
>> > > > > > > >
>> > > > > > > > I hope this makes sense,
>> > > > > > > > Many Thanks,
>> > > > > > > > Sherrie
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Mon Oct 28 11:43:27 2019
Hi Sherrie,
I just tried logging on to Hera but am having problems... so I wrote
for
help from Hera's helpdesk.
In the meantime, is there any chance you could post sample files to
our
anonymous ftp site? I'd need the latest iteration of the ERA5 and IMS
files and your Grid-Stat config file. Here are instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Thanks,
John
On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Good Morning John,
>
> I have my ERA5 data and my IMS data. MET is still unable to
read my
> IMS data. The message I get is
> DEBUG 1: Observation File:
> /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/imc_oct-01-
2017.nc
> DEBUG 2:
> DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &, DataPlane &)
const
> -> data was flipped to north.
> ERROR :
> ERROR : NcCfFile::getData(NcVar *, const LongArray &, DataPlane &)
const
> -> star found in bad slot
> ERROR :
> I have no idea about this error. I am sorry to need so much help.
>
> A header dump of my IMS Netcdf file is
> dimensions:
> time = UNLIMITED ; // (1 currently)
> ny = 6144 ;
> nx = 6144 ;
> variables:
> double time(time) ;
> time:calendar = "standard" ;
> time:units = "hours since 1900-01-01 00:00:00" ;
> time:long_name = "time" ;
>
> float lat(ny, nx) ;
> lat:units = "degrees_north" ;
> lat:long_name = "latitude" ;
> lat:forecast_time = 0 ;
> lat:forecast_time_units = "hours" ;
> lat:initial_time = "10/01/2017 (00:00)" ;
> lat:_FillValue = 9.96921e+36f ;
>
> float lon(ny, nx) ;
> lon:units = "degrees_east" ;
> lon:long_name = "longitude" ;
> lon:forecast_time = 0 ;
> lon:forecast_time_units = "hours" ;
> lon:initial_time = "10/01/2017 (00:00)" ;
> lon:_FillValue = 9.96921e+36f ;
>
> float snowC(ny, nx) ;
> snowC:level = "Z0" ;
> snowC:units = "%" ;
> snowC:long_name = "Snow Cover" ;
> snowC:forecast_time = 0 ;
> snowC:forecast_time_units = "hours" ;
> snowC:initial_time = "10/01/2017 (00:00)" ;
> snowC:_FillValue = 9.96921e+36f ;
>
> // global attributes:
> :lon_pin = 145.f ;
> :lat_pin = -21.49193f ;
> :ny = "6144" ;
> :nx = "6144" ;
> :r_km = "6371.2 km" ;
> :d_km = "4.0km" ;
> :lon_orient = "-80.0" ;
> :y_pin = "0.000000" ;
> :x_pin = "0.000000" ;
> :scale_lat = "60.0 degrees_north" ;
> :hemisphere = "N" ;
> :projection = "Polar Stereographic" ;
> :MET_version = "V8.1" ;
>
> I am working on Hera in the directory
> /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
>
> My Netcdf IMS files are in
> /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> Thanks,
> Sherrie
>
>
> >
> > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Sherrie,
> >>
> >> My intention was to have you use the Python embedding option to
read
> this
> >> data, but it looks like you're reformatting it into NetCDF
instead. And
> >> that's fine too.
> >>
> >> But there's some issues based on the NetCDF header you sent.
> >>
> >> (1) Those projection attributes should be defined as global
attributes,
> >> not
> >> separately for each variable.
> >> (2) You are formatting a NetCDF file to follow the conventions
used for
> >> the
> >> NetCDF output generated by the MET tools. If you add the global
> attribute
> >> "MET_version = "V8.0";" then MET will correctly process that file
as
> being
> >> the NetCDF output of another MET tool.
> >> (3) You'll need to add timing information as attributes for each
data
> >> variable.
> >>
> >> By way of example, I've attached a header dump of some NetCDF
output
> from
> >> MET.
> >>
> >> Ultimately, whether you reformat the data into NetCDF for MET to
read or
> >> pass it via python embedding, you're still defining the same
info... the
> >> grid spec and timing information. The downside of reformatting
to
> NetCDF
> >> is that pre-processing step and duplicating the data in ascii and
NetCDF
> >> formats. But either way can get the job done.
> >>
> >> If you're stuck, feel free to send me a file with your work in
progress,
> >> and I can edit it to make MET happy.
> >>
> >> Thanks,
> >> John
> >>
> >>
> >> dimensions:
> >> lat = 154 ;
> >> lon = 169 ;
> >> variables:
> >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> >> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at A24
Ensemble
> >> Mean" ;
> >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> >> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
> >> APCP_24_A24_ENS_MEAN:init_time = "20091231_120000"
;
> >> APCP_24_A24_ENS_MEAN:init_time_ut = "1262260800" ;
> >> APCP_24_A24_ENS_MEAN:valid_time =
"20100101_120000" ;
> >> APCP_24_A24_ENS_MEAN:valid_time_ut = "1262347200"
;
> >> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
> >> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
> >> // global attributes:
> >> :FileOrigins = "File
> >> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
generated
> >> 20191023_165124 UTC on host number5 by t
> >> he MET ensemble_stat tool" ;
> >> :MET_version = "V9.0" ;
> >> :MET_tool = "ensemble_stat" ;
> >> :model = "WRF" ;
> >> :obtype = "MC_PCP" ;
> >> :Projection = "Polar Stereographic" ;
> >> :hemisphere = "N" ;
> >> :scale_lat = "60.000000 degrees_north" ;
> >> :lat_pin = "31.357000" ;
> >> :lon_pin = "-129.770000" ;
> >> :x_pin = "0.000000" ;
> >> :y_pin = "0.000000" ;
> >> :lon_orient = "-120.500000" ;
> >> :d_km = "10.395000 km" ;
> >> :r_km = "6371.200000 km" ;
> >> :nx = "169" ;
> >> :ny = "154" ;
> >>
> >>
> >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >> >
> >> > Hi John,
> >> >
> >> > Does plot_data_plane work on Netcdf files. I have the
following
> >> in my
> >> > netcdf file.
> >> > The error I get is
> >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data
file
> >> > Do I need to tell plot_data_plane that I have a netcdf file.
> >> >
> >> > Thanks,
> >> > Sherrie
> >> > dimensions:
> >> > ny = 6144 ;
> >> > nx = 6144 ;
> >> > variables:
> >> > float lat(ny, nx) ;
> >> > lat:long_name = "latitude" ;
> >> > lat:lon_pin = 145.f ;
> >> > lat:lat_pin = -21.49193f ;
> >> > lat:ny = "6144" ;lat:nx = "6144" ;
> >> > lat:r_km = "6371.2 km" ;
> >> > lat:d_km = "4.0km" ;
> >> > lat:lon_orient = "-80.0" ;
> >> > lat:y_pin = "0.000000" ;
> >> > lat:x_pin = "0.000000" ;
> >> > lat:scale_lat = "60.0 degrees_north" ;
> >> > lat:hemisphere = "N" ;
> >> > lat:projection = "Polar Stereographic" ;
> >> > lat:_FillValue = 9.96921e+36f ;
> >> > float lon(ny, nx) ;
> >> > lon:long_name = "longitude" ;
> >> > lon:lon_pin = 145.f ;
> >> > lon:lat_pin = -21.49193f ;
> >> > lon:ny = "6144" ;
> >> > lon:nx = "6144" ;
> >> > lon:r_km = "6371.2 km" ;
> >> > lon:d_km = "4.0km" ;
> >> > lon:lon_orient = "-80.0" ;
> >> > lon:y_pin = "0.000000" ;
> >> > lon:x_pin = "0.000000" ;
> >> > lon:scale_lat = "60.0 degrees_north" ;
> >> > lon:hemisphere = "N" ;
> >> > lon:projection = "Polar Stereographic" ;
> >> > lon:_FillValue = 9.96921e+36f ;
> >> > float snowC(ny, nx) ;
> >> > snowC:units = "%" ;
> >> > snowC:long_name = "Snow Cover" ;
> >> > snowC:lon_pin = 145.f ;
> >> > snowC:lat_pin = -21.49193f ;
> >> > snowC:ny = "6144" ;
> >> > snowC:nx = "6144" ;
> >> > snowC:r_km = "6371.2 km" ;
> >> > snowC:d_km = "4.0km" ;
> >> > snowC:lon_orient = "-80.0" ;
> >> > snowC:y_pin = "0.000000" ;
> >> > snowC:x_pin = "0.000000" ;
> >> > snowC:scale_lat = "60.0 degrees_north" ;
> >> > snowC:hemisphere = "N" ;
> >> > snowC:projection = "Polar Stereographic" ;
> >> > snowC:_FillValue = 9.96921e+36f ;
> >> >
> >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Sherrie,
> >> > >
> >> > > So we need to figure out the grid specification for the IMS
polar
> >> > > stereographic grid on which your data resides.
> >> > >
> >> > > I did some hunting around online and suspect that you're
processing
> >> this
> >> > > data:
> >> > > https://nsidc.org/data/G02156
> >> > >
> >> > > And the 4km resolution version of this data is on a 6444x6444
polar
> >> > > stereographic grid.
> >> > >
> >> > > On the "User's Guide" tab of that page, table 4 lists some
relevant
> >> > > information:
> >> > > Latitude of true scale = 60 North
> >> > > Longitude below the pole = 80 West
> >> > >
> >> > > Based on that I'd say you want to try using the following
settings:
> >> > >
> >> > > :Projection = "Polar Stereographic" ;
> >> > > :hemisphere = "N" ;
> >> > > :scale_lat = "60.0 degrees_north" ;
> >> > > :lat_pin = "???" ;
> >> > > :lon_pin = "???" ;
> >> > > :x_pin = "0.000000" ;
> >> > > :y_pin = "0.000000" ;
> >> > > :lon_orient = "-80.0" ;
> >> > > :d_km = "4.0 km" ;
> >> > > :r_km = "6371.2 km" ;
> >> > > :nx = "6144" ;
> >> > > :ny = "6144" ;
> >> > >
> >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin
values
> you
> >> > > specify are for the "lower-left" corner of the grid. Look at
the
> >> lat/lon
> >> > > values you have and try just specifying that lat/lon of the
first
> >> point.
> >> > > Presumably, that'd be one of the 4 corners. And then run
> >> plot_data_plane
> >> > > to see what the image looks like. If it differs from your
NCL plot,
> >> we
> >> > can
> >> > > go from there.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA
Affiliate
> via
> >> > RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >> > > >
> >> > > > Good Morning John,
> >> > > >
> >> > > > I have a couple more questions at your convenience,
> >> > > > I took the IMS ascii data and the lat/lon data and
wrote a
> >> > > > Netcdf file that has both the IMS data and the lat/lon data
in
> >> > > > it.dimensions:
> >> > > > ny = 6144 ;
> >> > > > nx = 6144 ;
> >> > > > variables:
> >> > > > float lat(ny, nx) ;
> >> > > > lat:_FillValue = 9.96921e+36f ;
> >> > > > float lon(ny, nx) ;
> >> > > > lon:_FillValue = 9.96921e+36f ;
> >> > > > float snowC(ny, nx) ;
> >> > > > snowC:_FillValue = 9.96921e+36f ;
> >> > > > }
> >> > > >
> >> > > > I am not sure I understand the all of the metadata above.
> >> > > > I am not sure of what the following should be or how I
should get
> it
> >> > > >
> >> > > > :scale_lat = "? degrees_north" ;
> >> > > > :lat_pin = ?;
> >> > > > :lon_pin = ?;
> >> > > > :x_pin = ? ;
> >> > > > :y_pin = "0.000000" ;
> >> > > > :lon_orient = ? ;
> >> > > > The following I think is correct for the IMS polar
stereographic
> >> grid
> >> > > > :d_km = "4km" ;
> >> > > > :r_km = "6378.137km" ;
> >> > > > Since it is a 4km grid I believe I need to use a radius
of
> >> 6378.137
> >> > > > according to the documentation.
> >> > > > The IMS grid is the WGS-84 ellipse.
> >> > > >
> >> > > > Also my forecast is going to at 1kmX1km and the IMS grid
is a
> >> 4km.
> >> > > Do I
> >> > > > need to interpolate the IMS grid to a 1km grid?
> >> > > >
> >> > > > Sorry for all the questions.
> >> > > > Many thanks for your help.
> >> > > > Sherrie
> >> > > >
> >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT <
> >> > > > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Sherrie,
> >> > > > >
> >> > > > > OK, great. FYI, unfortunately, the name of the required
entries
> >> vary
> >> > > > > depending on the projection type and we haven't done a
good job
> >> > > > documenting
> >> > > > > all of the variations yet. However, those entries are
the same
> as
> >> > the
> >> > > > > global attributes you'll find in the NetCDF output from
MET
> tools.
> >> > > > >
> >> > > > > Here's an example of those attributes for a Polar
Stereographic
> >> > > > projection:
> >> > > > >
> >> > > > > :Projection = "Polar Stereographic" ;
> >> > > > > :hemisphere = "N" ;
> >> > > > > :scale_lat = "60.000000 degrees_north" ;
> >> > > > > :lat_pin = "31.357000" ;
> >> > > > > :lon_pin = "-129.770000" ;
> >> > > > > :x_pin = "0.000000" ;
> >> > > > > :y_pin = "0.000000" ;
> >> > > > > :lon_orient = "-120.500000" ;
> >> > > > > :d_km = "10.395000 km" ;
> >> > > > > :r_km = "6371.200000 km" ;
> >> > > > > :nx = "169" ;
> >> > > > > :ny = "154" ;
> >> > > > >
> >> > > > > Hopefully that'll help.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > John
> >> > > > >
> >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
> Affiliate
> >> via
> >> > > RT
> >> > > > <
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >
> >> > > > > >
> >> > > > > > Hi John,
> >> > > > > > Thank you so much! I will give it a try.
> >> > > > > >
> >> > > > > > cheers,
> >> > > > > > Sherrie
> >> > > > > >
> >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway via
RT <
> >> > > > > > met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hello Sherrie,
> >> > > > > > >
> >> > > > > > > Yes this is possible, but you'll need to use the
python
> >> embedding
> >> > > > > feature
> >> > > > > > > of MET. Specifically, we'll need a python script
which
> reads
> >> > data
> >> > > > from
> >> > > > > > > that ASCII file and defines the metadata for it (e.g.
timing
> >> and
> >> > > grid
> >> > > > > > > info). This is a relatively new feature in MET, and
we're
> >> > revising
> >> > > > the
> >> > > > > > > logic for it for met-9.0, but reading data from ASCII
should
> >> be
> >> > > very
> >> > > > > > > straight-forward. I'd expect this to work well.
> >> > > > > > >
> >> > > > > > > Examples of using python embedding to read gridded
data into
> >> MET
> >> > > can
> >> > > > be
> >> > > > > > > found here:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> >> > > > > > >
> >> > > > > > > The last example on that page reads data from ASCII,
so that
> >> > might
> >> > > > be a
> >> > > > > > > good place to start:
> >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> >> > 'name="read_AWS_CMORPH.py
> >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> >> > > > > > >
> >> > > > > > > I highly recommend starting with the plot_data_plane
tool
> >> since
> >> > you
> >> > > > can
> >> > > > > > > compare the output plot to your NCL output to make
sure
> you're
> >> > > > passing
> >> > > > > > the
> >> > > > > > > data to MET in the correct orientation with the
correct grid
> >> > info.
> >> > > > If
> >> > > > > > > you're comfortable with python, feel free to adapt
that
> >> script to
> >> > > > read
> >> > > > > > your
> >> > > > > > > data. If you need help with this, please send us a
sample
> >> data
> >> > > file,
> >> > > > > the
> >> > > > > > > lat/lon data file, and an NCL plot of that data.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > John
> >> > > > > > >
> >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick -
NOAA
> >> > Affiliate
> >> > > > via
> >> > > > > > RT <
> >> > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted
upon.
> >> > > > > > > > Transaction: Ticket created by
sherrie.fredrick at noaa.gov
> >> > > > > > > > Queue: met_help
> >> > > > > > > > Subject: using MET with IMS snow data
> >> > > > > > > > Owner: Nobody
> >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> >> > > > > > > > Status: new
> >> > > > > > > > Ticket <URL:
> >> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Good Morning,
> >> > > > > > > >
> >> > > > > > > > I would love to use MET/MET Plus to verify
the
> >> > NOAA/NESDIS
> >> > > > > > > > Interactive Multisensor Snow and Ice Mapping System
(IMS)
> >> data
> >> > > with
> >> > > > > > ERA5
> >> > > > > > > > data. The IMS data is on a Polar Stereographic 4km
grid
> the
> >> > data
> >> > > > > > format
> >> > > > > > > is
> >> > > > > > > > ASCII. The latitude and longitude definitions are
in a
> >> > separate
> >> > > > > > file. I
> >> > > > > > > > can read in the data and plot with NCL. The ERA5
data
> that
> >> I
> >> > am
> >> > > > > using
> >> > > > > > is
> >> > > > > > > > the ERA5 land data. It is in grib format. I can
choose
> >> the
> >> > > > > > resolution
> >> > > > > > > > when down loading the data. Is this possible?
> >> > > > > > > >
> >> > > > > > > > I will be working on Hera.
> >> > > > > > > >
> >> > > > > > > > I hope this makes sense,
> >> > > > > > > > Many Thanks,
> >> > > > > > > > Sherrie
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Mon Oct 28 12:33:50 2019
Hi John,
I have placed the following files on .../met_help/sherrie_data.
grid_stat_config
ims2017274_4km_v1.3.asc - This is the original ascii file
downloaded from the IMS web site.
Northern
Hemisphere, Polar stereographic, 4kmX4km grid resolution.
imslat_4km.bin - The binary latitude file
from
the IMS web site
imslon_4km.bin - The binary longitude file
from
the IMS web site
ims_oct-01-2017.nc - The Netcdf file that I
produced.
oct201701.nc - The ERA5 netcdf file from
the
ERA5-Land data. Downloaded from Copernicus.
I downloaded
the
ERA5 data at a 4kmX4km grid resolution.
I included the original files just in case. I think there will
be a
lot of interest in using MET to verify model data. My marching orders
are
to use only MET for verification. I will be running the FV3workflow
to get
FV3 model data. Right now we are not ready to do that so
we want to use ERA5 data with the IMS data.
Your help is much appreciated,
Sherrie
On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Sherrie,
>
> I just tried logging on to Hera but am having problems... so I wrote
for
> help from Hera's helpdesk.
>
> In the meantime, is there any chance you could post sample files to
our
> anonymous ftp site? I'd need the latest iteration of the ERA5 and
IMS
> files and your Grid-Stat config file. Here are instructions:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>
> Thanks,
> John
>
> On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Good Morning John,
> >
> > I have my ERA5 data and my IMS data. MET is still unable to
read my
> > IMS data. The message I get is
> > DEBUG 1: Observation File:
> > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> imc_oct-01-2017.nc
> > DEBUG 2:
> > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &, DataPlane
&) const
> > -> data was flipped to north.
> > ERROR :
> > ERROR : NcCfFile::getData(NcVar *, const LongArray &, DataPlane
&) const
> > -> star found in bad slot
> > ERROR :
> > I have no idea about this error. I am sorry to need so much help.
> >
> > A header dump of my IMS Netcdf file is
> > dimensions:
> > time = UNLIMITED ; // (1 currently)
> > ny = 6144 ;
> > nx = 6144 ;
> > variables:
> > double time(time) ;
> > time:calendar = "standard" ;
> > time:units = "hours since 1900-01-01 00:00:00" ;
> > time:long_name = "time" ;
> >
> > float lat(ny, nx) ;
> > lat:units = "degrees_north" ;
> > lat:long_name = "latitude" ;
> > lat:forecast_time = 0 ;
> > lat:forecast_time_units = "hours" ;
> > lat:initial_time = "10/01/2017 (00:00)" ;
> > lat:_FillValue = 9.96921e+36f ;
> >
> > float lon(ny, nx) ;
> > lon:units = "degrees_east" ;
> > lon:long_name = "longitude" ;
> > lon:forecast_time = 0 ;
> > lon:forecast_time_units = "hours" ;
> > lon:initial_time = "10/01/2017 (00:00)" ;
> > lon:_FillValue = 9.96921e+36f ;
> >
> > float snowC(ny, nx) ;
> > snowC:level = "Z0" ;
> > snowC:units = "%" ;
> > snowC:long_name = "Snow Cover" ;
> > snowC:forecast_time = 0 ;
> > snowC:forecast_time_units = "hours" ;
> > snowC:initial_time = "10/01/2017 (00:00)" ;
> > snowC:_FillValue = 9.96921e+36f ;
> >
> > // global attributes:
> > :lon_pin = 145.f ;
> > :lat_pin = -21.49193f ;
> > :ny = "6144" ;
> > :nx = "6144" ;
> > :r_km = "6371.2 km" ;
> > :d_km = "4.0km" ;
> > :lon_orient = "-80.0" ;
> > :y_pin = "0.000000" ;
> > :x_pin = "0.000000" ;
> > :scale_lat = "60.0 degrees_north" ;
> > :hemisphere = "N" ;
> > :projection = "Polar Stereographic" ;
> > :MET_version = "V8.1" ;
> >
> > I am working on Hera in the directory
> > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> >
> > My Netcdf IMS files are in
> > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > Thanks,
> > Sherrie
> >
> >
> > >
> > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Sherrie,
> > >>
> > >> My intention was to have you use the Python embedding option to
read
> > this
> > >> data, but it looks like you're reformatting it into NetCDF
instead.
> And
> > >> that's fine too.
> > >>
> > >> But there's some issues based on the NetCDF header you sent.
> > >>
> > >> (1) Those projection attributes should be defined as global
> attributes,
> > >> not
> > >> separately for each variable.
> > >> (2) You are formatting a NetCDF file to follow the conventions
used
> for
> > >> the
> > >> NetCDF output generated by the MET tools. If you add the
global
> > attribute
> > >> "MET_version = "V8.0";" then MET will correctly process that
file as
> > being
> > >> the NetCDF output of another MET tool.
> > >> (3) You'll need to add timing information as attributes for
each data
> > >> variable.
> > >>
> > >> By way of example, I've attached a header dump of some NetCDF
output
> > from
> > >> MET.
> > >>
> > >> Ultimately, whether you reformat the data into NetCDF for MET
to read
> or
> > >> pass it via python embedding, you're still defining the same
info...
> the
> > >> grid spec and timing information. The downside of reformatting
to
> > NetCDF
> > >> is that pre-processing step and duplicating the data in ascii
and
> NetCDF
> > >> formats. But either way can get the job done.
> > >>
> > >> If you're stuck, feel free to send me a file with your work in
> progress,
> > >> and I can edit it to make MET happy.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >>
> > >> dimensions:
> > >> lat = 154 ;
> > >> lon = 169 ;
> > >> variables:
> > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> > >> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at A24
> Ensemble
> > >> Mean" ;
> > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> > >> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
> > >> APCP_24_A24_ENS_MEAN:init_time =
"20091231_120000" ;
> > >> APCP_24_A24_ENS_MEAN:init_time_ut = "1262260800"
;
> > >> APCP_24_A24_ENS_MEAN:valid_time =
"20100101_120000" ;
> > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
"1262347200" ;
> > >> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
> > >> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
> > >> // global attributes:
> > >> :FileOrigins = "File
> > >> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
generated
> > >> 20191023_165124 UTC on host number5 by t
> > >> he MET ensemble_stat tool" ;
> > >> :MET_version = "V9.0" ;
> > >> :MET_tool = "ensemble_stat" ;
> > >> :model = "WRF" ;
> > >> :obtype = "MC_PCP" ;
> > >> :Projection = "Polar Stereographic" ;
> > >> :hemisphere = "N" ;
> > >> :scale_lat = "60.000000 degrees_north" ;
> > >> :lat_pin = "31.357000" ;
> > >> :lon_pin = "-129.770000" ;
> > >> :x_pin = "0.000000" ;
> > >> :y_pin = "0.000000" ;
> > >> :lon_orient = "-120.500000" ;
> > >> :d_km = "10.395000 km" ;
> > >> :r_km = "6371.200000 km" ;
> > >> :nx = "169" ;
> > >> :ny = "154" ;
> > >>
> > >>
> > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
> > >> >
> > >> > Hi John,
> > >> >
> > >> > Does plot_data_plane work on Netcdf files. I have the
> following
> > >> in my
> > >> > netcdf file.
> > >> > The error I get is
> > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid data
file
> > >> > Do I need to tell plot_data_plane that I have a netcdf file.
> > >> >
> > >> > Thanks,
> > >> > Sherrie
> > >> > dimensions:
> > >> > ny = 6144 ;
> > >> > nx = 6144 ;
> > >> > variables:
> > >> > float lat(ny, nx) ;
> > >> > lat:long_name = "latitude" ;
> > >> > lat:lon_pin = 145.f ;
> > >> > lat:lat_pin = -21.49193f ;
> > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > >> > lat:r_km = "6371.2 km" ;
> > >> > lat:d_km = "4.0km" ;
> > >> > lat:lon_orient = "-80.0" ;
> > >> > lat:y_pin = "0.000000" ;
> > >> > lat:x_pin = "0.000000" ;
> > >> > lat:scale_lat = "60.0 degrees_north" ;
> > >> > lat:hemisphere = "N" ;
> > >> > lat:projection = "Polar Stereographic" ;
> > >> > lat:_FillValue = 9.96921e+36f ;
> > >> > float lon(ny, nx) ;
> > >> > lon:long_name = "longitude" ;
> > >> > lon:lon_pin = 145.f ;
> > >> > lon:lat_pin = -21.49193f ;
> > >> > lon:ny = "6144" ;
> > >> > lon:nx = "6144" ;
> > >> > lon:r_km = "6371.2 km" ;
> > >> > lon:d_km = "4.0km" ;
> > >> > lon:lon_orient = "-80.0" ;
> > >> > lon:y_pin = "0.000000" ;
> > >> > lon:x_pin = "0.000000" ;
> > >> > lon:scale_lat = "60.0 degrees_north" ;
> > >> > lon:hemisphere = "N" ;
> > >> > lon:projection = "Polar Stereographic" ;
> > >> > lon:_FillValue = 9.96921e+36f ;
> > >> > float snowC(ny, nx) ;
> > >> > snowC:units = "%" ;
> > >> > snowC:long_name = "Snow Cover" ;
> > >> > snowC:lon_pin = 145.f ;
> > >> > snowC:lat_pin = -21.49193f ;
> > >> > snowC:ny = "6144" ;
> > >> > snowC:nx = "6144" ;
> > >> > snowC:r_km = "6371.2 km" ;
> > >> > snowC:d_km = "4.0km" ;
> > >> > snowC:lon_orient = "-80.0" ;
> > >> > snowC:y_pin = "0.000000" ;
> > >> > snowC:x_pin = "0.000000" ;
> > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > >> > snowC:hemisphere = "N" ;
> > >> > snowC:projection = "Polar Stereographic" ;
> > >> > snowC:_FillValue = 9.96921e+36f ;
> > >> >
> > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
> > >> > met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Sherrie,
> > >> > >
> > >> > > So we need to figure out the grid specification for the IMS
polar
> > >> > > stereographic grid on which your data resides.
> > >> > >
> > >> > > I did some hunting around online and suspect that you're
> processing
> > >> this
> > >> > > data:
> > >> > > https://nsidc.org/data/G02156
> > >> > >
> > >> > > And the 4km resolution version of this data is on a
6444x6444
> polar
> > >> > > stereographic grid.
> > >> > >
> > >> > > On the "User's Guide" tab of that page, table 4 lists some
> relevant
> > >> > > information:
> > >> > > Latitude of true scale = 60 North
> > >> > > Longitude below the pole = 80 West
> > >> > >
> > >> > > Based on that I'd say you want to try using the following
> settings:
> > >> > >
> > >> > > :Projection = "Polar Stereographic" ;
> > >> > > :hemisphere = "N" ;
> > >> > > :scale_lat = "60.0 degrees_north" ;
> > >> > > :lat_pin = "???" ;
> > >> > > :lon_pin = "???" ;
> > >> > > :x_pin = "0.000000" ;
> > >> > > :y_pin = "0.000000" ;
> > >> > > :lon_orient = "-80.0" ;
> > >> > > :d_km = "4.0 km" ;
> > >> > > :r_km = "6371.2 km" ;
> > >> > > :nx = "6144" ;
> > >> > > :ny = "6144" ;
> > >> > >
> > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and lon_pin
values
> > you
> > >> > > specify are for the "lower-left" corner of the grid. Look
at the
> > >> lat/lon
> > >> > > values you have and try just specifying that lat/lon of the
first
> > >> point.
> > >> > > Presumably, that'd be one of the 4 corners. And then run
> > >> plot_data_plane
> > >> > > to see what the image looks like. If it differs from your
NCL
> plot,
> > >> we
> > >> > can
> > >> > > go from there.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA
Affiliate
> > via
> > >> > RT <
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >> > > >
> > >> > > > Good Morning John,
> > >> > > >
> > >> > > > I have a couple more questions at your convenience,
> > >> > > > I took the IMS ascii data and the lat/lon data and
wrote a
> > >> > > > Netcdf file that has both the IMS data and the lat/lon
data in
> > >> > > > it.dimensions:
> > >> > > > ny = 6144 ;
> > >> > > > nx = 6144 ;
> > >> > > > variables:
> > >> > > > float lat(ny, nx) ;
> > >> > > > lat:_FillValue = 9.96921e+36f ;
> > >> > > > float lon(ny, nx) ;
> > >> > > > lon:_FillValue = 9.96921e+36f ;
> > >> > > > float snowC(ny, nx) ;
> > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > >> > > > }
> > >> > > >
> > >> > > > I am not sure I understand the all of the metadata above.
> > >> > > > I am not sure of what the following should be or how I
should
> get
> > it
> > >> > > >
> > >> > > > :scale_lat = "? degrees_north" ;
> > >> > > > :lat_pin = ?;
> > >> > > > :lon_pin = ?;
> > >> > > > :x_pin = ? ;
> > >> > > > :y_pin = "0.000000" ;
> > >> > > > :lon_orient = ? ;
> > >> > > > The following I think is correct for the IMS polar
> stereographic
> > >> grid
> > >> > > > :d_km = "4km" ;
> > >> > > > :r_km = "6378.137km" ;
> > >> > > > Since it is a 4km grid I believe I need to use a radius
of
> > >> 6378.137
> > >> > > > according to the documentation.
> > >> > > > The IMS grid is the WGS-84 ellipse.
> > >> > > >
> > >> > > > Also my forecast is going to at 1kmX1km and the IMS
grid is a
> > >> 4km.
> > >> > > Do I
> > >> > > > need to interpolate the IMS grid to a 1km grid?
> > >> > > >
> > >> > > > Sorry for all the questions.
> > >> > > > Many thanks for your help.
> > >> > > > Sherrie
> > >> > > >
> > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via RT
<
> > >> > > > met_help at ucar.edu>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Sherrie,
> > >> > > > >
> > >> > > > > OK, great. FYI, unfortunately, the name of the
required
> entries
> > >> vary
> > >> > > > > depending on the projection type and we haven't done a
good
> job
> > >> > > > documenting
> > >> > > > > all of the variations yet. However, those entries are
the
> same
> > as
> > >> > the
> > >> > > > > global attributes you'll find in the NetCDF output from
MET
> > tools.
> > >> > > > >
> > >> > > > > Here's an example of those attributes for a Polar
> Stereographic
> > >> > > > projection:
> > >> > > > >
> > >> > > > > :Projection = "Polar Stereographic" ;
> > >> > > > > :hemisphere = "N" ;
> > >> > > > > :scale_lat = "60.000000 degrees_north" ;
> > >> > > > > :lat_pin = "31.357000" ;
> > >> > > > > :lon_pin = "-129.770000" ;
> > >> > > > > :x_pin = "0.000000" ;
> > >> > > > > :y_pin = "0.000000" ;
> > >> > > > > :lon_orient = "-120.500000" ;
> > >> > > > > :d_km = "10.395000 km" ;
> > >> > > > > :r_km = "6371.200000 km" ;
> > >> > > > > :nx = "169" ;
> > >> > > > > :ny = "154" ;
> > >> > > > >
> > >> > > > > Hopefully that'll help.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > John
> > >> > > > >
> > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick - NOAA
> > Affiliate
> > >> via
> > >> > > RT
> > >> > > > <
> > >> > > > > met_help at ucar.edu> wrote:
> > >> > > > >
> > >> > > > > >
> > >> > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > >
> > >> > > > > >
> > >> > > > > > Hi John,
> > >> > > > > > Thank you so much! I will give it a try.
> > >> > > > > >
> > >> > > > > > cheers,
> > >> > > > > > Sherrie
> > >> > > > > >
> > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway
via RT <
> > >> > > > > > met_help at ucar.edu>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hello Sherrie,
> > >> > > > > > >
> > >> > > > > > > Yes this is possible, but you'll need to use the
python
> > >> embedding
> > >> > > > > feature
> > >> > > > > > > of MET. Specifically, we'll need a python script
which
> > reads
> > >> > data
> > >> > > > from
> > >> > > > > > > that ASCII file and defines the metadata for it
(e.g.
> timing
> > >> and
> > >> > > grid
> > >> > > > > > > info). This is a relatively new feature in MET,
and we're
> > >> > revising
> > >> > > > the
> > >> > > > > > > logic for it for met-9.0, but reading data from
ASCII
> should
> > >> be
> > >> > > very
> > >> > > > > > > straight-forward. I'd expect this to work well.
> > >> > > > > > >
> > >> > > > > > > Examples of using python embedding to read gridded
data
> into
> > >> MET
> > >> > > can
> > >> > > > be
> > >> > > > > > > found here:
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > >> > > > > > >
> > >> > > > > > > The last example on that page reads data from
ASCII, so
> that
> > >> > might
> > >> > > > be a
> > >> > > > > > > good place to start:
> > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > >> > 'name="read_AWS_CMORPH.py
> > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > >> > > > > > >
> > >> > > > > > > I highly recommend starting with the
plot_data_plane tool
> > >> since
> > >> > you
> > >> > > > can
> > >> > > > > > > compare the output plot to your NCL output to make
sure
> > you're
> > >> > > > passing
> > >> > > > > > the
> > >> > > > > > > data to MET in the correct orientation with the
correct
> grid
> > >> > info.
> > >> > > > If
> > >> > > > > > > you're comfortable with python, feel free to adapt
that
> > >> script to
> > >> > > > read
> > >> > > > > > your
> > >> > > > > > > data. If you need help with this, please send us a
sample
> > >> data
> > >> > > file,
> > >> > > > > the
> > >> > > > > > > lat/lon data file, and an NCL plot of that data.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > John
> > >> > > > > > >
> > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick -
NOAA
> > >> > Affiliate
> > >> > > > via
> > >> > > > > > RT <
> > >> > > > > > > met_help at ucar.edu> wrote:
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was acted
upon.
> > >> > > > > > > > Transaction: Ticket created by
> sherrie.fredrick at noaa.gov
> > >> > > > > > > > Queue: met_help
> > >> > > > > > > > Subject: using MET with IMS snow data
> > >> > > > > > > > Owner: Nobody
> > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > >> > > > > > > > Status: new
> > >> > > > > > > > Ticket <URL:
> > >> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Good Morning,
> > >> > > > > > > >
> > >> > > > > > > > I would love to use MET/MET Plus to verify
the
> > >> > NOAA/NESDIS
> > >> > > > > > > > Interactive Multisensor Snow and Ice Mapping
System
> (IMS)
> > >> data
> > >> > > with
> > >> > > > > > ERA5
> > >> > > > > > > > data. The IMS data is on a Polar Stereographic
4km grid
> > the
> > >> > data
> > >> > > > > > format
> > >> > > > > > > is
> > >> > > > > > > > ASCII. The latitude and longitude definitions
are in a
> > >> > separate
> > >> > > > > > file. I
> > >> > > > > > > > can read in the data and plot with NCL. The ERA5
data
> > that
> > >> I
> > >> > am
> > >> > > > > using
> > >> > > > > > is
> > >> > > > > > > > the ERA5 land data. It is in grib format. I
can
> choose
> > >> the
> > >> > > > > > resolution
> > >> > > > > > > > when down loading the data. Is this possible?
> > >> > > > > > > >
> > >> > > > > > > > I will be working on Hera.
> > >> > > > > > > >
> > >> > > > > > > > I hope this makes sense,
> > >> > > > > > > > Many Thanks,
> > >> > > > > > > > Sherrie
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Mon Oct 28 13:50:21 2019
Sherrie,
Great, thanks. I ran plot_data_plane on the ims NetCDF and got the
same
error you did. Here are the issues I found and how to fix them.
(1) Rename ny/nx dimension to lat/lon:
Looking at that NetCDF I see that you're formatting it as if it were a
NetCDF file created by another MET tool. The library in MET that
reads
this data assumes the dimensions are named lat and lon.
* ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
<http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
(2) The "projection" global attribute should be capitalized as
"Projection":
* ncatted -a projection,global,d,c,'' -a
Projection,global,a,c,"Polar
Stereographic" ims_oct-01-2017_MOD.nc*
(3) I changed to reference point to be the north pole: (x, y) = (3072,
3072) is the north pole at (lat, lon) = (90, 0):
* ncatted -a lon_pin,global,o,c,"0" -a lat_pin,global,o,c,"90" -a
x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072" ims_oct-01-
2017_MOD.nc*
After all of these changes, I ran plot_data_plane to produce the
attached
plot.
*plot_data_plane ims_oct-01-2017_MOD.nc ims.ps <http://ims.ps>
'name="snowC"; level="(*,*)";'*
And the data lines up very well with the map data!
However, issues with the timing info remain. Your snowC data variable
doesn't define the timing info in the way expected by the NetCDF
output
from MET. It should contain variable attributes, like this:
APCP_12:init_time = "20050807_000000" ;
APCP_12:init_time_ut = "1123372800" ;
APCP_12:valid_time = "20050807_120000" ;
APCP_12:valid_time_ut = "1123416000" ;
APCP_12:accum_time = "120000" ;
APCP_12:accum_time_sec = 43200 ;
Technically, I believe the code is parsing the timing info from the
"init_time_ut" and "valid_time_ut" attributes, where UT is unixtime,
the
number of seconds since Jan 1, 1970.
Here's how you can use the unix date command to convert between
October 1,
2017 and unixtime:
*date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
And then going back the other way:
*date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
*2019100100*
Thanks,
John
On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Hi John,
>
> I have placed the following files on .../met_help/sherrie_data.
> grid_stat_config
> ims2017274_4km_v1.3.asc - This is the original ascii file
> downloaded from the IMS web site.
> Northern
> Hemisphere, Polar stereographic, 4kmX4km grid resolution.
> imslat_4km.bin - The binary latitude
file from
> the IMS web site
> imslon_4km.bin - The binary longitude
file from
> the IMS web site
> ims_oct-01-2017.nc - The Netcdf file that I
produced.
> oct201701.nc - The ERA5 netcdf file
from the
> ERA5-Land data. Downloaded from Copernicus.
> I downloaded
the
> ERA5 data at a 4kmX4km grid resolution.
> I included the original files just in case. I think there will
be a
> lot of interest in using MET to verify model data. My marching
orders are
> to use only MET for verification. I will be running the FV3workflow
to get
> FV3 model data. Right now we are not ready to do that so
> we want to use ERA5 data with the IMS data.
> Your help is much appreciated,
> Sherrie
>
> On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Sherrie,
> >
> > I just tried logging on to Hera but am having problems... so I
wrote for
> > help from Hera's helpdesk.
> >
> > In the meantime, is there any chance you could post sample files
to our
> > anonymous ftp site? I'd need the latest iteration of the ERA5 and
IMS
> > files and your Grid-Stat config file. Here are instructions:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> >
> > Thanks,
> > John
> >
> > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >
> > > Good Morning John,
> > >
> > > I have my ERA5 data and my IMS data. MET is still unable
to read
> my
> > > IMS data. The message I get is
> > > DEBUG 1: Observation File:
> > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> > imc_oct-01-2017.nc
> > > DEBUG 2:
> > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &, DataPlane
&)
> const
> > > -> data was flipped to north.
> > > ERROR :
> > > ERROR : NcCfFile::getData(NcVar *, const LongArray &, DataPlane
&)
> const
> > > -> star found in bad slot
> > > ERROR :
> > > I have no idea about this error. I am sorry to need so much
help.
> > >
> > > A header dump of my IMS Netcdf file is
> > > dimensions:
> > > time = UNLIMITED ; // (1 currently)
> > > ny = 6144 ;
> > > nx = 6144 ;
> > > variables:
> > > double time(time) ;
> > > time:calendar = "standard" ;
> > > time:units = "hours since 1900-01-01 00:00:00" ;
> > > time:long_name = "time" ;
> > >
> > > float lat(ny, nx) ;
> > > lat:units = "degrees_north" ;
> > > lat:long_name = "latitude" ;
> > > lat:forecast_time = 0 ;
> > > lat:forecast_time_units = "hours" ;
> > > lat:initial_time = "10/01/2017 (00:00)" ;
> > > lat:_FillValue = 9.96921e+36f ;
> > >
> > > float lon(ny, nx) ;
> > > lon:units = "degrees_east" ;
> > > lon:long_name = "longitude" ;
> > > lon:forecast_time = 0 ;
> > > lon:forecast_time_units = "hours" ;
> > > lon:initial_time = "10/01/2017 (00:00)" ;
> > > lon:_FillValue = 9.96921e+36f ;
> > >
> > > float snowC(ny, nx) ;
> > > snowC:level = "Z0" ;
> > > snowC:units = "%" ;
> > > snowC:long_name = "Snow Cover" ;
> > > snowC:forecast_time = 0 ;
> > > snowC:forecast_time_units = "hours" ;
> > > snowC:initial_time = "10/01/2017 (00:00)" ;
> > > snowC:_FillValue = 9.96921e+36f ;
> > >
> > > // global attributes:
> > > :lon_pin = 145.f ;
> > > :lat_pin = -21.49193f ;
> > > :ny = "6144" ;
> > > :nx = "6144" ;
> > > :r_km = "6371.2 km" ;
> > > :d_km = "4.0km" ;
> > > :lon_orient = "-80.0" ;
> > > :y_pin = "0.000000" ;
> > > :x_pin = "0.000000" ;
> > > :scale_lat = "60.0 degrees_north" ;
> > > :hemisphere = "N" ;
> > > :projection = "Polar Stereographic" ;
> > > :MET_version = "V8.1" ;
> > >
> > > I am working on Hera in the directory
> > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> > >
> > > My Netcdf IMS files are in
> > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > > Thanks,
> > > Sherrie
> > >
> > >
> > > >
> > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Sherrie,
> > > >>
> > > >> My intention was to have you use the Python embedding option
to read
> > > this
> > > >> data, but it looks like you're reformatting it into NetCDF
instead.
> > And
> > > >> that's fine too.
> > > >>
> > > >> But there's some issues based on the NetCDF header you sent.
> > > >>
> > > >> (1) Those projection attributes should be defined as global
> > attributes,
> > > >> not
> > > >> separately for each variable.
> > > >> (2) You are formatting a NetCDF file to follow the
conventions used
> > for
> > > >> the
> > > >> NetCDF output generated by the MET tools. If you add the
global
> > > attribute
> > > >> "MET_version = "V8.0";" then MET will correctly process that
file as
> > > being
> > > >> the NetCDF output of another MET tool.
> > > >> (3) You'll need to add timing information as attributes for
each
> data
> > > >> variable.
> > > >>
> > > >> By way of example, I've attached a header dump of some NetCDF
output
> > > from
> > > >> MET.
> > > >>
> > > >> Ultimately, whether you reformat the data into NetCDF for MET
to
> read
> > or
> > > >> pass it via python embedding, you're still defining the same
info...
> > the
> > > >> grid spec and timing information. The downside of
reformatting to
> > > NetCDF
> > > >> is that pre-processing step and duplicating the data in ascii
and
> > NetCDF
> > > >> formats. But either way can get the job done.
> > > >>
> > > >> If you're stuck, feel free to send me a file with your work
in
> > progress,
> > > >> and I can edit it to make MET happy.
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >>
> > > >> dimensions:
> > > >> lat = 154 ;
> > > >> lon = 169 ;
> > > >> variables:
> > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> > > >> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at
A24
> > Ensemble
> > > >> Mean" ;
> > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> > > >> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
> > > >> APCP_24_A24_ENS_MEAN:init_time =
"20091231_120000" ;
> > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
"1262260800" ;
> > > >> APCP_24_A24_ENS_MEAN:valid_time =
"20100101_120000" ;
> > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
"1262347200" ;
> > > >> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
> > > >> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400 ;
> > > >> // global attributes:
> > > >> :FileOrigins = "File
> > > >> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
generated
> > > >> 20191023_165124 UTC on host number5 by t
> > > >> he MET ensemble_stat tool" ;
> > > >> :MET_version = "V9.0" ;
> > > >> :MET_tool = "ensemble_stat" ;
> > > >> :model = "WRF" ;
> > > >> :obtype = "MC_PCP" ;
> > > >> :Projection = "Polar Stereographic" ;
> > > >> :hemisphere = "N" ;
> > > >> :scale_lat = "60.000000 degrees_north" ;
> > > >> :lat_pin = "31.357000" ;
> > > >> :lon_pin = "-129.770000" ;
> > > >> :x_pin = "0.000000" ;
> > > >> :y_pin = "0.000000" ;
> > > >> :lon_orient = "-120.500000" ;
> > > >> :d_km = "10.395000 km" ;
> > > >> :r_km = "6371.200000 km" ;
> > > >> :nx = "169" ;
> > > >> :ny = "154" ;
> > > >>
> > > >>
> > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > >> >
> > > >> > Hi John,
> > > >> >
> > > >> > Does plot_data_plane work on Netcdf files. I have the
> > following
> > > >> in my
> > > >> > netcdf file.
> > > >> > The error I get is
> > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid
data
> file
> > > >> > Do I need to tell plot_data_plane that I have a netcdf
file.
> > > >> >
> > > >> > Thanks,
> > > >> > Sherrie
> > > >> > dimensions:
> > > >> > ny = 6144 ;
> > > >> > nx = 6144 ;
> > > >> > variables:
> > > >> > float lat(ny, nx) ;
> > > >> > lat:long_name = "latitude" ;
> > > >> > lat:lon_pin = 145.f ;
> > > >> > lat:lat_pin = -21.49193f ;
> > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > > >> > lat:r_km = "6371.2 km" ;
> > > >> > lat:d_km = "4.0km" ;
> > > >> > lat:lon_orient = "-80.0" ;
> > > >> > lat:y_pin = "0.000000" ;
> > > >> > lat:x_pin = "0.000000" ;
> > > >> > lat:scale_lat = "60.0 degrees_north" ;
> > > >> > lat:hemisphere = "N" ;
> > > >> > lat:projection = "Polar Stereographic" ;
> > > >> > lat:_FillValue = 9.96921e+36f ;
> > > >> > float lon(ny, nx) ;
> > > >> > lon:long_name = "longitude" ;
> > > >> > lon:lon_pin = 145.f ;
> > > >> > lon:lat_pin = -21.49193f ;
> > > >> > lon:ny = "6144" ;
> > > >> > lon:nx = "6144" ;
> > > >> > lon:r_km = "6371.2 km" ;
> > > >> > lon:d_km = "4.0km" ;
> > > >> > lon:lon_orient = "-80.0" ;
> > > >> > lon:y_pin = "0.000000" ;
> > > >> > lon:x_pin = "0.000000" ;
> > > >> > lon:scale_lat = "60.0 degrees_north" ;
> > > >> > lon:hemisphere = "N" ;
> > > >> > lon:projection = "Polar Stereographic" ;
> > > >> > lon:_FillValue = 9.96921e+36f ;
> > > >> > float snowC(ny, nx) ;
> > > >> > snowC:units = "%" ;
> > > >> > snowC:long_name = "Snow Cover" ;
> > > >> > snowC:lon_pin = 145.f ;
> > > >> > snowC:lat_pin = -21.49193f ;
> > > >> > snowC:ny = "6144" ;
> > > >> > snowC:nx = "6144" ;
> > > >> > snowC:r_km = "6371.2 km" ;
> > > >> > snowC:d_km = "4.0km" ;
> > > >> > snowC:lon_orient = "-80.0" ;
> > > >> > snowC:y_pin = "0.000000" ;
> > > >> > snowC:x_pin = "0.000000" ;
> > > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > > >> > snowC:hemisphere = "N" ;
> > > >> > snowC:projection = "Polar Stereographic" ;
> > > >> > snowC:_FillValue = 9.96921e+36f ;
> > > >> >
> > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT <
> > > >> > met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> > > Sherrie,
> > > >> > >
> > > >> > > So we need to figure out the grid specification for the
IMS
> polar
> > > >> > > stereographic grid on which your data resides.
> > > >> > >
> > > >> > > I did some hunting around online and suspect that you're
> > processing
> > > >> this
> > > >> > > data:
> > > >> > > https://nsidc.org/data/G02156
> > > >> > >
> > > >> > > And the 4km resolution version of this data is on a
6444x6444
> > polar
> > > >> > > stereographic grid.
> > > >> > >
> > > >> > > On the "User's Guide" tab of that page, table 4 lists
some
> > relevant
> > > >> > > information:
> > > >> > > Latitude of true scale = 60 North
> > > >> > > Longitude below the pole = 80 West
> > > >> > >
> > > >> > > Based on that I'd say you want to try using the following
> > settings:
> > > >> > >
> > > >> > > :Projection = "Polar Stereographic" ;
> > > >> > > :hemisphere = "N" ;
> > > >> > > :scale_lat = "60.0 degrees_north" ;
> > > >> > > :lat_pin = "???" ;
> > > >> > > :lon_pin = "???" ;
> > > >> > > :x_pin = "0.000000" ;
> > > >> > > :y_pin = "0.000000" ;
> > > >> > > :lon_orient = "-80.0" ;
> > > >> > > :d_km = "4.0 km" ;
> > > >> > > :r_km = "6371.2 km" ;
> > > >> > > :nx = "6144" ;
> > > >> > > :ny = "6144" ;
> > > >> > >
> > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and
lon_pin
> values
> > > you
> > > >> > > specify are for the "lower-left" corner of the grid.
Look at
> the
> > > >> lat/lon
> > > >> > > values you have and try just specifying that lat/lon of
the
> first
> > > >> point.
> > > >> > > Presumably, that'd be one of the 4 corners. And then run
> > > >> plot_data_plane
> > > >> > > to see what the image looks like. If it differs from
your NCL
> > plot,
> > > >> we
> > > >> > can
> > > >> > > go from there.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > John
> > > >> > >
> > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick - NOAA
> Affiliate
> > > via
> > > >> > RT <
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >
> > > >> > > >
> > > >> > > > Good Morning John,
> > > >> > > >
> > > >> > > > I have a couple more questions at your convenience,
> > > >> > > > I took the IMS ascii data and the lat/lon data and
wrote a
> > > >> > > > Netcdf file that has both the IMS data and the lat/lon
data in
> > > >> > > > it.dimensions:
> > > >> > > > ny = 6144 ;
> > > >> > > > nx = 6144 ;
> > > >> > > > variables:
> > > >> > > > float lat(ny, nx) ;
> > > >> > > > lat:_FillValue = 9.96921e+36f ;
> > > >> > > > float lon(ny, nx) ;
> > > >> > > > lon:_FillValue = 9.96921e+36f ;
> > > >> > > > float snowC(ny, nx) ;
> > > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > > >> > > > }
> > > >> > > >
> > > >> > > > I am not sure I understand the all of the metadata
above.
> > > >> > > > I am not sure of what the following should be or how I
should
> > get
> > > it
> > > >> > > >
> > > >> > > > :scale_lat = "? degrees_north" ;
> > > >> > > > :lat_pin = ?;
> > > >> > > > :lon_pin = ?;
> > > >> > > > :x_pin = ? ;
> > > >> > > > :y_pin = "0.000000" ;
> > > >> > > > :lon_orient = ? ;
> > > >> > > > The following I think is correct for the IMS polar
> > stereographic
> > > >> grid
> > > >> > > > :d_km = "4km" ;
> > > >> > > > :r_km = "6378.137km" ;
> > > >> > > > Since it is a 4km grid I believe I need to use a
radius of
> > > >> 6378.137
> > > >> > > > according to the documentation.
> > > >> > > > The IMS grid is the WGS-84 ellipse.
> > > >> > > >
> > > >> > > > Also my forecast is going to at 1kmX1km and the IMS
grid
> is a
> > > >> 4km.
> > > >> > > Do I
> > > >> > > > need to interpolate the IMS grid to a 1km grid?
> > > >> > > >
> > > >> > > > Sorry for all the questions.
> > > >> > > > Many thanks for your help.
> > > >> > > > Sherrie
> > > >> > > >
> > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway via
RT <
> > > >> > > > met_help at ucar.edu>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Sherrie,
> > > >> > > > >
> > > >> > > > > OK, great. FYI, unfortunately, the name of the
required
> > entries
> > > >> vary
> > > >> > > > > depending on the projection type and we haven't done
a good
> > job
> > > >> > > > documenting
> > > >> > > > > all of the variations yet. However, those entries
are the
> > same
> > > as
> > > >> > the
> > > >> > > > > global attributes you'll find in the NetCDF output
from MET
> > > tools.
> > > >> > > > >
> > > >> > > > > Here's an example of those attributes for a Polar
> > Stereographic
> > > >> > > > projection:
> > > >> > > > >
> > > >> > > > > :Projection = "Polar Stereographic" ;
> > > >> > > > > :hemisphere = "N" ;
> > > >> > > > > :scale_lat = "60.000000 degrees_north"
;
> > > >> > > > > :lat_pin = "31.357000" ;
> > > >> > > > > :lon_pin = "-129.770000" ;
> > > >> > > > > :x_pin = "0.000000" ;
> > > >> > > > > :y_pin = "0.000000" ;
> > > >> > > > > :lon_orient = "-120.500000" ;
> > > >> > > > > :d_km = "10.395000 km" ;
> > > >> > > > > :r_km = "6371.200000 km" ;
> > > >> > > > > :nx = "169" ;
> > > >> > > > > :ny = "154" ;
> > > >> > > > >
> > > >> > > > > Hopefully that'll help.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > John
> > > >> > > > >
> > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick -
NOAA
> > > Affiliate
> > > >> via
> > > >> > > RT
> > > >> > > > <
> > > >> > > > > met_help at ucar.edu> wrote:
> > > >> > > > >
> > > >> > > > > >
> > > >> > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > >
> > > >> > > > > >
> > > >> > > > > > Hi John,
> > > >> > > > > > Thank you so much! I will give it a try.
> > > >> > > > > >
> > > >> > > > > > cheers,
> > > >> > > > > > Sherrie
> > > >> > > > > >
> > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley Gotway
via RT
> <
> > > >> > > > > > met_help at ucar.edu>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hello Sherrie,
> > > >> > > > > > >
> > > >> > > > > > > Yes this is possible, but you'll need to use the
python
> > > >> embedding
> > > >> > > > > feature
> > > >> > > > > > > of MET. Specifically, we'll need a python script
which
> > > reads
> > > >> > data
> > > >> > > > from
> > > >> > > > > > > that ASCII file and defines the metadata for it
(e.g.
> > timing
> > > >> and
> > > >> > > grid
> > > >> > > > > > > info). This is a relatively new feature in MET,
and
> we're
> > > >> > revising
> > > >> > > > the
> > > >> > > > > > > logic for it for met-9.0, but reading data from
ASCII
> > should
> > > >> be
> > > >> > > very
> > > >> > > > > > > straight-forward. I'd expect this to work well.
> > > >> > > > > > >
> > > >> > > > > > > Examples of using python embedding to read
gridded data
> > into
> > > >> MET
> > > >> > > can
> > > >> > > > be
> > > >> > > > > > > found here:
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > >> > > > > > >
> > > >> > > > > > > The last example on that page reads data from
ASCII, so
> > that
> > > >> > might
> > > >> > > > be a
> > > >> > > > > > > good place to start:
> > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > > >> > 'name="read_AWS_CMORPH.py
> > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > >> > > > > > >
> > > >> > > > > > > I highly recommend starting with the
plot_data_plane
> tool
> > > >> since
> > > >> > you
> > > >> > > > can
> > > >> > > > > > > compare the output plot to your NCL output to
make sure
> > > you're
> > > >> > > > passing
> > > >> > > > > > the
> > > >> > > > > > > data to MET in the correct orientation with the
correct
> > grid
> > > >> > info.
> > > >> > > > If
> > > >> > > > > > > you're comfortable with python, feel free to
adapt that
> > > >> script to
> > > >> > > > read
> > > >> > > > > > your
> > > >> > > > > > > data. If you need help with this, please send us
a
> sample
> > > >> data
> > > >> > > file,
> > > >> > > > > the
> > > >> > > > > > > lat/lon data file, and an NCL plot of that data.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > John
> > > >> > > > > > >
> > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie Fredrick
- NOAA
> > > >> > Affiliate
> > > >> > > > via
> > > >> > > > > > RT <
> > > >> > > > > > > met_help at ucar.edu> wrote:
> > > >> > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was
acted
> upon.
> > > >> > > > > > > > Transaction: Ticket created by
> > sherrie.fredrick at noaa.gov
> > > >> > > > > > > > Queue: met_help
> > > >> > > > > > > > Subject: using MET with IMS snow data
> > > >> > > > > > > > Owner: Nobody
> > > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > >> > > > > > > > Status: new
> > > >> > > > > > > > Ticket <URL:
> > > >> > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > >> > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Good Morning,
> > > >> > > > > > > >
> > > >> > > > > > > > I would love to use MET/MET Plus to
verify the
> > > >> > NOAA/NESDIS
> > > >> > > > > > > > Interactive Multisensor Snow and Ice Mapping
System
> > (IMS)
> > > >> data
> > > >> > > with
> > > >> > > > > > ERA5
> > > >> > > > > > > > data. The IMS data is on a Polar Stereographic
4km
> grid
> > > the
> > > >> > data
> > > >> > > > > > format
> > > >> > > > > > > is
> > > >> > > > > > > > ASCII. The latitude and longitude definitions
are in
> a
> > > >> > separate
> > > >> > > > > > file. I
> > > >> > > > > > > > can read in the data and plot with NCL. The
ERA5 data
> > > that
> > > >> I
> > > >> > am
> > > >> > > > > using
> > > >> > > > > > is
> > > >> > > > > > > > the ERA5 land data. It is in grib format. I
can
> > choose
> > > >> the
> > > >> > > > > > resolution
> > > >> > > > > > > > when down loading the data. Is this possible?
> > > >> > > > > > > >
> > > >> > > > > > > > I will be working on Hera.
> > > >> > > > > > > >
> > > >> > > > > > > > I hope this makes sense,
> > > >> > > > > > > > Many Thanks,
> > > >> > > > > > > > Sherrie
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Mon Oct 28 13:59:14 2019
Hi John,
Yea! Thanks very much for all your help. I will make the
changes
you suggested.
Sherrie
On Mon, Oct 28, 2019 at 1:50 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Sherrie,
>
> Great, thanks. I ran plot_data_plane on the ims NetCDF and got the
same
> error you did. Here are the issues I found and how to fix them.
>
> (1) Rename ny/nx dimension to lat/lon:
> Looking at that NetCDF I see that you're formatting it as if it were
a
> NetCDF file created by another MET tool. The library in MET that
reads
> this data assumes the dimensions are named lat and lon.
> * ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
> <http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
>
> (2) The "projection" global attribute should be capitalized as
> "Projection":
> * ncatted -a projection,global,d,c,'' -a
Projection,global,a,c,"Polar
> Stereographic" ims_oct-01-2017_MOD.nc*
>
> (3) I changed to reference point to be the north pole: (x, y) =
(3072,
> 3072) is the north pole at (lat, lon) = (90, 0):
> * ncatted -a lon_pin,global,o,c,"0" -a lat_pin,global,o,c,"90" -a
> x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072" ims_oct-01-
2017_MOD.nc*
>
> After all of these changes, I ran plot_data_plane to produce the
attached
> plot.
> *plot_data_plane ims_oct-01-2017_MOD.nc ims.ps <http://ims.ps>
> 'name="snowC"; level="(*,*)";'*
>
> And the data lines up very well with the map data!
>
> However, issues with the timing info remain. Your snowC data
variable
> doesn't define the timing info in the way expected by the NetCDF
output
> from MET. It should contain variable attributes, like this:
> APCP_12:init_time = "20050807_000000" ;
>
> APCP_12:init_time_ut = "1123372800" ;
>
> APCP_12:valid_time = "20050807_120000" ;
>
> APCP_12:valid_time_ut = "1123416000" ;
>
> APCP_12:accum_time = "120000" ;
>
> APCP_12:accum_time_sec = 43200 ;
> Technically, I believe the code is parsing the timing info from the
> "init_time_ut" and "valid_time_ut" attributes, where UT is unixtime,
the
> number of seconds since Jan 1, 1970.
>
> Here's how you can use the unix date command to convert between
October 1,
> 2017 and unixtime:
>
> *date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
>
> And then going back the other way:
> *date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
> *2019100100*
>
> Thanks,
> John
>
>
> On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Hi John,
> >
> > I have placed the following files on
.../met_help/sherrie_data.
> > grid_stat_config
> > ims2017274_4km_v1.3.asc - This is the original ascii file
> > downloaded from the IMS web site.
> > Northern
> > Hemisphere, Polar stereographic, 4kmX4km grid resolution.
> > imslat_4km.bin - The binary latitude
file
> from
> > the IMS web site
> > imslon_4km.bin - The binary longitude
file
> from
> > the IMS web site
> > ims_oct-01-2017.nc - The Netcdf file that I
> produced.
> > oct201701.nc - The ERA5 netcdf file
from
> the
> > ERA5-Land data. Downloaded from Copernicus.
> > I
downloaded the
> > ERA5 data at a 4kmX4km grid resolution.
> > I included the original files just in case. I think there
will be a
> > lot of interest in using MET to verify model data. My marching
orders
> are
> > to use only MET for verification. I will be running the
FV3workflow to
> get
> > FV3 model data. Right now we are not ready to do that so
> > we want to use ERA5 data with the IMS data.
> > Your help is much appreciated,
> > Sherrie
> >
> > On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hi Sherrie,
> > >
> > > I just tried logging on to Hera but am having problems... so I
wrote
> for
> > > help from Hera's helpdesk.
> > >
> > > In the meantime, is there any chance you could post sample files
to our
> > > anonymous ftp site? I'd need the latest iteration of the ERA5
and IMS
> > > files and your Grid-Stat config file. Here are instructions:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > >
> > > Thanks,
> > > John
> > >
> > > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
> > > >
> > > > Good Morning John,
> > > >
> > > > I have my ERA5 data and my IMS data. MET is still unable
to
> read
> > my
> > > > IMS data. The message I get is
> > > > DEBUG 1: Observation File:
> > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> > > imc_oct-01-2017.nc
> > > > DEBUG 2:
> > > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &,
DataPlane &)
> > const
> > > > -> data was flipped to north.
> > > > ERROR :
> > > > ERROR : NcCfFile::getData(NcVar *, const LongArray &,
DataPlane &)
> > const
> > > > -> star found in bad slot
> > > > ERROR :
> > > > I have no idea about this error. I am sorry to need so much
help.
> > > >
> > > > A header dump of my IMS Netcdf file is
> > > > dimensions:
> > > > time = UNLIMITED ; // (1 currently)
> > > > ny = 6144 ;
> > > > nx = 6144 ;
> > > > variables:
> > > > double time(time) ;
> > > > time:calendar = "standard" ;
> > > > time:units = "hours since 1900-01-01 00:00:00" ;
> > > > time:long_name = "time" ;
> > > >
> > > > float lat(ny, nx) ;
> > > > lat:units = "degrees_north" ;
> > > > lat:long_name = "latitude" ;
> > > > lat:forecast_time = 0 ;
> > > > lat:forecast_time_units = "hours" ;
> > > > lat:initial_time = "10/01/2017 (00:00)" ;
> > > > lat:_FillValue = 9.96921e+36f ;
> > > >
> > > > float lon(ny, nx) ;
> > > > lon:units = "degrees_east" ;
> > > > lon:long_name = "longitude" ;
> > > > lon:forecast_time = 0 ;
> > > > lon:forecast_time_units = "hours" ;
> > > > lon:initial_time = "10/01/2017 (00:00)" ;
> > > > lon:_FillValue = 9.96921e+36f ;
> > > >
> > > > float snowC(ny, nx) ;
> > > > snowC:level = "Z0" ;
> > > > snowC:units = "%" ;
> > > > snowC:long_name = "Snow Cover" ;
> > > > snowC:forecast_time = 0 ;
> > > > snowC:forecast_time_units = "hours" ;
> > > > snowC:initial_time = "10/01/2017 (00:00)" ;
> > > > snowC:_FillValue = 9.96921e+36f ;
> > > >
> > > > // global attributes:
> > > > :lon_pin = 145.f ;
> > > > :lat_pin = -21.49193f ;
> > > > :ny = "6144" ;
> > > > :nx = "6144" ;
> > > > :r_km = "6371.2 km" ;
> > > > :d_km = "4.0km" ;
> > > > :lon_orient = "-80.0" ;
> > > > :y_pin = "0.000000" ;
> > > > :x_pin = "0.000000" ;
> > > > :scale_lat = "60.0 degrees_north" ;
> > > > :hemisphere = "N" ;
> > > > :projection = "Polar Stereographic" ;
> > > > :MET_version = "V8.1" ;
> > > >
> > > > I am working on Hera in the directory
> > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> > > >
> > > > My Netcdf IMS files are in
> > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > > > Thanks,
> > > > Sherrie
> > > >
> > > >
> > > > >
> > > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Sherrie,
> > > > >>
> > > > >> My intention was to have you use the Python embedding
option to
> read
> > > > this
> > > > >> data, but it looks like you're reformatting it into NetCDF
> instead.
> > > And
> > > > >> that's fine too.
> > > > >>
> > > > >> But there's some issues based on the NetCDF header you
sent.
> > > > >>
> > > > >> (1) Those projection attributes should be defined as global
> > > attributes,
> > > > >> not
> > > > >> separately for each variable.
> > > > >> (2) You are formatting a NetCDF file to follow the
conventions
> used
> > > for
> > > > >> the
> > > > >> NetCDF output generated by the MET tools. If you add the
global
> > > > attribute
> > > > >> "MET_version = "V8.0";" then MET will correctly process
that file
> as
> > > > being
> > > > >> the NetCDF output of another MET tool.
> > > > >> (3) You'll need to add timing information as attributes for
each
> > data
> > > > >> variable.
> > > > >>
> > > > >> By way of example, I've attached a header dump of some
NetCDF
> output
> > > > from
> > > > >> MET.
> > > > >>
> > > > >> Ultimately, whether you reformat the data into NetCDF for
MET to
> > read
> > > or
> > > > >> pass it via python embedding, you're still defining the
same
> info...
> > > the
> > > > >> grid spec and timing information. The downside of
reformatting to
> > > > NetCDF
> > > > >> is that pre-processing step and duplicating the data in
ascii and
> > > NetCDF
> > > > >> formats. But either way can get the job done.
> > > > >>
> > > > >> If you're stuck, feel free to send me a file with your work
in
> > > progress,
> > > > >> and I can edit it to make MET happy.
> > > > >>
> > > > >> Thanks,
> > > > >> John
> > > > >>
> > > > >>
> > > > >> dimensions:
> > > > >> lat = 154 ;
> > > > >> lon = 169 ;
> > > > >> variables:
> > > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> > > > >> APCP_24_A24_ENS_MEAN:long_name = "APCP_24 at
A24
> > > Ensemble
> > > > >> Mean" ;
> > > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> > > > >> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f ;
> > > > >> APCP_24_A24_ENS_MEAN:init_time =
"20091231_120000"
> ;
> > > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
"1262260800" ;
> > > > >> APCP_24_A24_ENS_MEAN:valid_time =
> "20100101_120000" ;
> > > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
"1262347200" ;
> > > > >> APCP_24_A24_ENS_MEAN:accum_time = "240000" ;
> > > > >> APCP_24_A24_ENS_MEAN:accum_time_sec = 86400
;
> > > > >> // global attributes:
> > > > >> :FileOrigins = "File
> > > > >> ../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
> generated
> > > > >> 20191023_165124 UTC on host number5 by t
> > > > >> he MET ensemble_stat tool" ;
> > > > >> :MET_version = "V9.0" ;
> > > > >> :MET_tool = "ensemble_stat" ;
> > > > >> :model = "WRF" ;
> > > > >> :obtype = "MC_PCP" ;
> > > > >> :Projection = "Polar Stereographic" ;
> > > > >> :hemisphere = "N" ;
> > > > >> :scale_lat = "60.000000 degrees_north" ;
> > > > >> :lat_pin = "31.357000" ;
> > > > >> :lon_pin = "-129.770000" ;
> > > > >> :x_pin = "0.000000" ;
> > > > >> :y_pin = "0.000000" ;
> > > > >> :lon_orient = "-120.500000" ;
> > > > >> :d_km = "10.395000 km" ;
> > > > >> :r_km = "6371.200000 km" ;
> > > > >> :nx = "169" ;
> > > > >> :ny = "154" ;
> > > > >>
> > > > >>
> > > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA
Affiliate
> > via
> > > > RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > >> >
> > > > >> > Hi John,
> > > > >> >
> > > > >> > Does plot_data_plane work on Netcdf files. I have
the
> > > following
> > > > >> in my
> > > > >> > netcdf file.
> > > > >> > The error I get is
> > > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a valid
data
> > file
> > > > >> > Do I need to tell plot_data_plane that I have a netcdf
file.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Sherrie
> > > > >> > dimensions:
> > > > >> > ny = 6144 ;
> > > > >> > nx = 6144 ;
> > > > >> > variables:
> > > > >> > float lat(ny, nx) ;
> > > > >> > lat:long_name = "latitude" ;
> > > > >> > lat:lon_pin = 145.f ;
> > > > >> > lat:lat_pin = -21.49193f ;
> > > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > > > >> > lat:r_km = "6371.2 km" ;
> > > > >> > lat:d_km = "4.0km" ;
> > > > >> > lat:lon_orient = "-80.0" ;
> > > > >> > lat:y_pin = "0.000000" ;
> > > > >> > lat:x_pin = "0.000000" ;
> > > > >> > lat:scale_lat = "60.0 degrees_north" ;
> > > > >> > lat:hemisphere = "N" ;
> > > > >> > lat:projection = "Polar Stereographic" ;
> > > > >> > lat:_FillValue = 9.96921e+36f ;
> > > > >> > float lon(ny, nx) ;
> > > > >> > lon:long_name = "longitude" ;
> > > > >> > lon:lon_pin = 145.f ;
> > > > >> > lon:lat_pin = -21.49193f ;
> > > > >> > lon:ny = "6144" ;
> > > > >> > lon:nx = "6144" ;
> > > > >> > lon:r_km = "6371.2 km" ;
> > > > >> > lon:d_km = "4.0km" ;
> > > > >> > lon:lon_orient = "-80.0" ;
> > > > >> > lon:y_pin = "0.000000" ;
> > > > >> > lon:x_pin = "0.000000" ;
> > > > >> > lon:scale_lat = "60.0 degrees_north" ;
> > > > >> > lon:hemisphere = "N" ;
> > > > >> > lon:projection = "Polar Stereographic" ;
> > > > >> > lon:_FillValue = 9.96921e+36f ;
> > > > >> > float snowC(ny, nx) ;
> > > > >> > snowC:units = "%" ;
> > > > >> > snowC:long_name = "Snow Cover" ;
> > > > >> > snowC:lon_pin = 145.f ;
> > > > >> > snowC:lat_pin = -21.49193f ;
> > > > >> > snowC:ny = "6144" ;
> > > > >> > snowC:nx = "6144" ;
> > > > >> > snowC:r_km = "6371.2 km" ;
> > > > >> > snowC:d_km = "4.0km" ;
> > > > >> > snowC:lon_orient = "-80.0" ;
> > > > >> > snowC:y_pin = "0.000000" ;
> > > > >> > snowC:x_pin = "0.000000" ;
> > > > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > > > >> > snowC:hemisphere = "N" ;
> > > > >> > snowC:projection = "Polar Stereographic" ;
> > > > >> > snowC:_FillValue = 9.96921e+36f ;
> > > > >> >
> > > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via RT
<
> > > > >> > met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Sherrie,
> > > > >> > >
> > > > >> > > So we need to figure out the grid specification for the
IMS
> > polar
> > > > >> > > stereographic grid on which your data resides.
> > > > >> > >
> > > > >> > > I did some hunting around online and suspect that
you're
> > > processing
> > > > >> this
> > > > >> > > data:
> > > > >> > > https://nsidc.org/data/G02156
> > > > >> > >
> > > > >> > > And the 4km resolution version of this data is on a
6444x6444
> > > polar
> > > > >> > > stereographic grid.
> > > > >> > >
> > > > >> > > On the "User's Guide" tab of that page, table 4 lists
some
> > > relevant
> > > > >> > > information:
> > > > >> > > Latitude of true scale = 60 North
> > > > >> > > Longitude below the pole = 80 West
> > > > >> > >
> > > > >> > > Based on that I'd say you want to try using the
following
> > > settings:
> > > > >> > >
> > > > >> > > :Projection = "Polar Stereographic" ;
> > > > >> > > :hemisphere = "N" ;
> > > > >> > > :scale_lat = "60.0 degrees_north" ;
> > > > >> > > :lat_pin = "???" ;
> > > > >> > > :lon_pin = "???" ;
> > > > >> > > :x_pin = "0.000000" ;
> > > > >> > > :y_pin = "0.000000" ;
> > > > >> > > :lon_orient = "-80.0" ;
> > > > >> > > :d_km = "4.0 km" ;
> > > > >> > > :r_km = "6371.2 km" ;
> > > > >> > > :nx = "6144" ;
> > > > >> > > :ny = "6144" ;
> > > > >> > >
> > > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and
lon_pin
> > values
> > > > you
> > > > >> > > specify are for the "lower-left" corner of the grid.
Look at
> > the
> > > > >> lat/lon
> > > > >> > > values you have and try just specifying that lat/lon of
the
> > first
> > > > >> point.
> > > > >> > > Presumably, that'd be one of the 4 corners. And then
run
> > > > >> plot_data_plane
> > > > >> > > to see what the image looks like. If it differs from
your NCL
> > > plot,
> > > > >> we
> > > > >> > can
> > > > >> > > go from there.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > John
> > > > >> > >
> > > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick -
NOAA
> > Affiliate
> > > > via
> > > > >> > RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > >
> > > > >> > > >
> > > > >> > > > Good Morning John,
> > > > >> > > >
> > > > >> > > > I have a couple more questions at your
convenience,
> > > > >> > > > I took the IMS ascii data and the lat/lon data
and
> wrote a
> > > > >> > > > Netcdf file that has both the IMS data and the
lat/lon data
> in
> > > > >> > > > it.dimensions:
> > > > >> > > > ny = 6144 ;
> > > > >> > > > nx = 6144 ;
> > > > >> > > > variables:
> > > > >> > > > float lat(ny, nx) ;
> > > > >> > > > lat:_FillValue = 9.96921e+36f ;
> > > > >> > > > float lon(ny, nx) ;
> > > > >> > > > lon:_FillValue = 9.96921e+36f ;
> > > > >> > > > float snowC(ny, nx) ;
> > > > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > > > >> > > > }
> > > > >> > > >
> > > > >> > > > I am not sure I understand the all of the metadata
above.
> > > > >> > > > I am not sure of what the following should be or how
I
> should
> > > get
> > > > it
> > > > >> > > >
> > > > >> > > > :scale_lat = "? degrees_north" ;
> > > > >> > > > :lat_pin = ?;
> > > > >> > > > :lon_pin = ?;
> > > > >> > > > :x_pin = ? ;
> > > > >> > > > :y_pin = "0.000000" ;
> > > > >> > > > :lon_orient = ? ;
> > > > >> > > > The following I think is correct for the IMS polar
> > > stereographic
> > > > >> grid
> > > > >> > > > :d_km = "4km" ;
> > > > >> > > > :r_km = "6378.137km" ;
> > > > >> > > > Since it is a 4km grid I believe I need to use a
radius of
> > > > >> 6378.137
> > > > >> > > > according to the documentation.
> > > > >> > > > The IMS grid is the WGS-84 ellipse.
> > > > >> > > >
> > > > >> > > > Also my forecast is going to at 1kmX1km and the
IMS grid
> > is a
> > > > >> 4km.
> > > > >> > > Do I
> > > > >> > > > need to interpolate the IMS grid to a 1km grid?
> > > > >> > > >
> > > > >> > > > Sorry for all the questions.
> > > > >> > > > Many thanks for your help.
> > > > >> > > > Sherrie
> > > > >> > > >
> > > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway
via RT <
> > > > >> > > > met_help at ucar.edu>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Sherrie,
> > > > >> > > > >
> > > > >> > > > > OK, great. FYI, unfortunately, the name of the
required
> > > entries
> > > > >> vary
> > > > >> > > > > depending on the projection type and we haven't
done a
> good
> > > job
> > > > >> > > > documenting
> > > > >> > > > > all of the variations yet. However, those entries
are the
> > > same
> > > > as
> > > > >> > the
> > > > >> > > > > global attributes you'll find in the NetCDF output
from
> MET
> > > > tools.
> > > > >> > > > >
> > > > >> > > > > Here's an example of those attributes for a Polar
> > > Stereographic
> > > > >> > > > projection:
> > > > >> > > > >
> > > > >> > > > > :Projection = "Polar Stereographic"
;
> > > > >> > > > > :hemisphere = "N" ;
> > > > >> > > > > :scale_lat = "60.000000
degrees_north" ;
> > > > >> > > > > :lat_pin = "31.357000" ;
> > > > >> > > > > :lon_pin = "-129.770000" ;
> > > > >> > > > > :x_pin = "0.000000" ;
> > > > >> > > > > :y_pin = "0.000000" ;
> > > > >> > > > > :lon_orient = "-120.500000" ;
> > > > >> > > > > :d_km = "10.395000 km" ;
> > > > >> > > > > :r_km = "6371.200000 km" ;
> > > > >> > > > > :nx = "169" ;
> > > > >> > > > > :ny = "154" ;
> > > > >> > > > >
> > > > >> > > > > Hopefully that'll help.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > John
> > > > >> > > > >
> > > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick -
NOAA
> > > > Affiliate
> > > > >> via
> > > > >> > > RT
> > > > >> > > > <
> > > > >> > > > > met_help at ucar.edu> wrote:
> > > > >> > > > >
> > > > >> > > > > >
> > > > >> > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > >
> > > > >> > > > > >
> > > > >> > > > > > Hi John,
> > > > >> > > > > > Thank you so much! I will give it a try.
> > > > >> > > > > >
> > > > >> > > > > > cheers,
> > > > >> > > > > > Sherrie
> > > > >> > > > > >
> > > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley
Gotway via
> RT
> > <
> > > > >> > > > > > met_help at ucar.edu>
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hello Sherrie,
> > > > >> > > > > > >
> > > > >> > > > > > > Yes this is possible, but you'll need to use
the
> python
> > > > >> embedding
> > > > >> > > > > feature
> > > > >> > > > > > > of MET. Specifically, we'll need a python
script
> which
> > > > reads
> > > > >> > data
> > > > >> > > > from
> > > > >> > > > > > > that ASCII file and defines the metadata for it
(e.g.
> > > timing
> > > > >> and
> > > > >> > > grid
> > > > >> > > > > > > info). This is a relatively new feature in
MET, and
> > we're
> > > > >> > revising
> > > > >> > > > the
> > > > >> > > > > > > logic for it for met-9.0, but reading data from
ASCII
> > > should
> > > > >> be
> > > > >> > > very
> > > > >> > > > > > > straight-forward. I'd expect this to work
well.
> > > > >> > > > > > >
> > > > >> > > > > > > Examples of using python embedding to read
gridded
> data
> > > into
> > > > >> MET
> > > > >> > > can
> > > > >> > > > be
> > > > >> > > > > > > found here:
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > >> > > > > > >
> > > > >> > > > > > > The last example on that page reads data from
ASCII,
> so
> > > that
> > > > >> > might
> > > > >> > > > be a
> > > > >> > > > > > > good place to start:
> > > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > > > >> > 'name="read_AWS_CMORPH.py
> > > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > >> > > > > > >
> > > > >> > > > > > > I highly recommend starting with the
plot_data_plane
> > tool
> > > > >> since
> > > > >> > you
> > > > >> > > > can
> > > > >> > > > > > > compare the output plot to your NCL output to
make
> sure
> > > > you're
> > > > >> > > > passing
> > > > >> > > > > > the
> > > > >> > > > > > > data to MET in the correct orientation with the
> correct
> > > grid
> > > > >> > info.
> > > > >> > > > If
> > > > >> > > > > > > you're comfortable with python, feel free to
adapt
> that
> > > > >> script to
> > > > >> > > > read
> > > > >> > > > > > your
> > > > >> > > > > > > data. If you need help with this, please send
us a
> > sample
> > > > >> data
> > > > >> > > file,
> > > > >> > > > > the
> > > > >> > > > > > > lat/lon data file, and an NCL plot of that
data.
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks,
> > > > >> > > > > > > John
> > > > >> > > > > > >
> > > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie
Fredrick -
> NOAA
> > > > >> > Affiliate
> > > > >> > > > via
> > > > >> > > > > > RT <
> > > > >> > > > > > > met_help at ucar.edu> wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was
acted
> > upon.
> > > > >> > > > > > > > Transaction: Ticket created by
> > > sherrie.fredrick at noaa.gov
> > > > >> > > > > > > > Queue: met_help
> > > > >> > > > > > > > Subject: using MET with IMS snow data
> > > > >> > > > > > > > Owner: Nobody
> > > > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > >> > > > > > > > Status: new
> > > > >> > > > > > > > Ticket <URL:
> > > > >> > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > >> > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Good Morning,
> > > > >> > > > > > > >
> > > > >> > > > > > > > I would love to use MET/MET Plus to
verify the
> > > > >> > NOAA/NESDIS
> > > > >> > > > > > > > Interactive Multisensor Snow and Ice Mapping
System
> > > (IMS)
> > > > >> data
> > > > >> > > with
> > > > >> > > > > > ERA5
> > > > >> > > > > > > > data. The IMS data is on a Polar
Stereographic 4km
> > grid
> > > > the
> > > > >> > data
> > > > >> > > > > > format
> > > > >> > > > > > > is
> > > > >> > > > > > > > ASCII. The latitude and longitude
definitions are
> in
> > a
> > > > >> > separate
> > > > >> > > > > > file. I
> > > > >> > > > > > > > can read in the data and plot with NCL. The
ERA5
> data
> > > > that
> > > > >> I
> > > > >> > am
> > > > >> > > > > using
> > > > >> > > > > > is
> > > > >> > > > > > > > the ERA5 land data. It is in grib format.
I can
> > > choose
> > > > >> the
> > > > >> > > > > > resolution
> > > > >> > > > > > > > when down loading the data. Is this possible?
> > > > >> > > > > > > >
> > > > >> > > > > > > > I will be working on Hera.
> > > > >> > > > > > > >
> > > > >> > > > > > > > I hope this makes sense,
> > > > >> > > > > > > > Many Thanks,
> > > > >> > > > > > > > Sherrie
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Mon Oct 28 17:15:45 2019
Sherrie,
Just to show you an alternative, I worked up an example of using
python
embedding to read this 4km data.
Here's an example of calling plot_data_plane using python embedding to
read
the ASCII data directly:
plot_data_plane PYTHON_NUMPY ims.ps 'name="read_NSIDC_IMS_4km.py
ims2017010_4km_v1.3.asc";'
And that produces the same image we got before: ims.png
I've attached the python script I used. The advantage here is not
needing
to write intermediate NetCDF files and being able to "fix" issues that
arise directly in the python script.
Thanks,
John
On Mon, Oct 28, 2019 at 1:59 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Hi John,
>
> Yea! Thanks very much for all your help. I will make the
changes
> you suggested.
> Sherrie
>
> On Mon, Oct 28, 2019 at 1:50 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > Great, thanks. I ran plot_data_plane on the ims NetCDF and got
the same
> > error you did. Here are the issues I found and how to fix them.
> >
> > (1) Rename ny/nx dimension to lat/lon:
> > Looking at that NetCDF I see that you're formatting it as if it
were a
> > NetCDF file created by another MET tool. The library in MET that
reads
> > this data assumes the dimensions are named lat and lon.
> > * ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
> > <http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
> >
> > (2) The "projection" global attribute should be capitalized as
> > "Projection":
> > * ncatted -a projection,global,d,c,'' -a
Projection,global,a,c,"Polar
> > Stereographic" ims_oct-01-2017_MOD.nc*
> >
> > (3) I changed to reference point to be the north pole: (x, y) =
(3072,
> > 3072) is the north pole at (lat, lon) = (90, 0):
> > * ncatted -a lon_pin,global,o,c,"0" -a lat_pin,global,o,c,"90"
-a
> > x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072"
> ims_oct-01-2017_MOD.nc*
> >
> > After all of these changes, I ran plot_data_plane to produce the
attached
> > plot.
> > *plot_data_plane ims_oct-01-2017_MOD.nc ims.ps <http://ims.ps>
> > 'name="snowC"; level="(*,*)";'*
> >
> > And the data lines up very well with the map data!
> >
> > However, issues with the timing info remain. Your snowC data
variable
> > doesn't define the timing info in the way expected by the NetCDF
output
> > from MET. It should contain variable attributes, like this:
> > APCP_12:init_time = "20050807_000000" ;
> >
> > APCP_12:init_time_ut = "1123372800" ;
> >
> > APCP_12:valid_time = "20050807_120000" ;
> >
> > APCP_12:valid_time_ut = "1123416000" ;
> >
> > APCP_12:accum_time = "120000" ;
> >
> > APCP_12:accum_time_sec = 43200 ;
> > Technically, I believe the code is parsing the timing info from
the
> > "init_time_ut" and "valid_time_ut" attributes, where UT is
unixtime, the
> > number of seconds since Jan 1, 1970.
> >
> > Here's how you can use the unix date command to convert between
October
> 1,
> > 2017 and unixtime:
> >
> > *date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
> >
> > And then going back the other way:
> > *date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
> > *2019100100*
> >
> > Thanks,
> > John
> >
> >
> > On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >
> > > Hi John,
> > >
> > > I have placed the following files on
.../met_help/sherrie_data.
> > > grid_stat_config
> > > ims2017274_4km_v1.3.asc - This is the original ascii
file
> > > downloaded from the IMS web site.
> > > Northern
> > > Hemisphere, Polar stereographic, 4kmX4km grid resolution.
> > > imslat_4km.bin - The binary latitude
file
> > from
> > > the IMS web site
> > > imslon_4km.bin - The binary longitude
file
> > from
> > > the IMS web site
> > > ims_oct-01-2017.nc - The Netcdf file that I
> > produced.
> > > oct201701.nc - The ERA5 netcdf
file from
> > the
> > > ERA5-Land data. Downloaded from Copernicus.
> > > I
downloaded
> the
> > > ERA5 data at a 4kmX4km grid resolution.
> > > I included the original files just in case. I think there
will
> be a
> > > lot of interest in using MET to verify model data. My marching
orders
> > are
> > > to use only MET for verification. I will be running the
FV3workflow to
> > get
> > > FV3 model data. Right now we are not ready to do that so
> > > we want to use ERA5 data with the IMS data.
> > > Your help is much appreciated,
> > > Sherrie
> > >
> > > On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Hi Sherrie,
> > > >
> > > > I just tried logging on to Hera but am having problems... so I
wrote
> > for
> > > > help from Hera's helpdesk.
> > > >
> > > > In the meantime, is there any chance you could post sample
files to
> our
> > > > anonymous ftp site? I'd need the latest iteration of the ERA5
and
> IMS
> > > > files and your Grid-Stat config file. Here are instructions:
> > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > >
> > > > > Good Morning John,
> > > > >
> > > > > I have my ERA5 data and my IMS data. MET is still
unable to
> > read
> > > my
> > > > > IMS data. The message I get is
> > > > > DEBUG 1: Observation File:
> > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> > > > imc_oct-01-2017.nc
> > > > > DEBUG 2:
> > > > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &,
DataPlane &)
> > > const
> > > > > -> data was flipped to north.
> > > > > ERROR :
> > > > > ERROR : NcCfFile::getData(NcVar *, const LongArray &,
DataPlane &)
> > > const
> > > > > -> star found in bad slot
> > > > > ERROR :
> > > > > I have no idea about this error. I am sorry to need so much
help.
> > > > >
> > > > > A header dump of my IMS Netcdf file is
> > > > > dimensions:
> > > > > time = UNLIMITED ; // (1 currently)
> > > > > ny = 6144 ;
> > > > > nx = 6144 ;
> > > > > variables:
> > > > > double time(time) ;
> > > > > time:calendar = "standard" ;
> > > > > time:units = "hours since 1900-01-01 00:00:00" ;
> > > > > time:long_name = "time" ;
> > > > >
> > > > > float lat(ny, nx) ;
> > > > > lat:units = "degrees_north" ;
> > > > > lat:long_name = "latitude" ;
> > > > > lat:forecast_time = 0 ;
> > > > > lat:forecast_time_units = "hours" ;
> > > > > lat:initial_time = "10/01/2017 (00:00)" ;
> > > > > lat:_FillValue = 9.96921e+36f ;
> > > > >
> > > > > float lon(ny, nx) ;
> > > > > lon:units = "degrees_east" ;
> > > > > lon:long_name = "longitude" ;
> > > > > lon:forecast_time = 0 ;
> > > > > lon:forecast_time_units = "hours" ;
> > > > > lon:initial_time = "10/01/2017 (00:00)" ;
> > > > > lon:_FillValue = 9.96921e+36f ;
> > > > >
> > > > > float snowC(ny, nx) ;
> > > > > snowC:level = "Z0" ;
> > > > > snowC:units = "%" ;
> > > > > snowC:long_name = "Snow Cover" ;
> > > > > snowC:forecast_time = 0 ;
> > > > > snowC:forecast_time_units = "hours" ;
> > > > > snowC:initial_time = "10/01/2017 (00:00)" ;
> > > > > snowC:_FillValue = 9.96921e+36f ;
> > > > >
> > > > > // global attributes:
> > > > > :lon_pin = 145.f ;
> > > > > :lat_pin = -21.49193f ;
> > > > > :ny = "6144" ;
> > > > > :nx = "6144" ;
> > > > > :r_km = "6371.2 km" ;
> > > > > :d_km = "4.0km" ;
> > > > > :lon_orient = "-80.0" ;
> > > > > :y_pin = "0.000000" ;
> > > > > :x_pin = "0.000000" ;
> > > > > :scale_lat = "60.0 degrees_north" ;
> > > > > :hemisphere = "N" ;
> > > > > :projection = "Polar Stereographic" ;
> > > > > :MET_version = "V8.1" ;
> > > > >
> > > > > I am working on Hera in the directory
> > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> > > > >
> > > > > My Netcdf IMS files are in
> > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > > > > Thanks,
> > > > > Sherrie
> > > > >
> > > > >
> > > > > >
> > > > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > >> Sherrie,
> > > > > >>
> > > > > >> My intention was to have you use the Python embedding
option to
> > read
> > > > > this
> > > > > >> data, but it looks like you're reformatting it into
NetCDF
> > instead.
> > > > And
> > > > > >> that's fine too.
> > > > > >>
> > > > > >> But there's some issues based on the NetCDF header you
sent.
> > > > > >>
> > > > > >> (1) Those projection attributes should be defined as
global
> > > > attributes,
> > > > > >> not
> > > > > >> separately for each variable.
> > > > > >> (2) You are formatting a NetCDF file to follow the
conventions
> > used
> > > > for
> > > > > >> the
> > > > > >> NetCDF output generated by the MET tools. If you add the
global
> > > > > attribute
> > > > > >> "MET_version = "V8.0";" then MET will correctly process
that
> file
> > as
> > > > > being
> > > > > >> the NetCDF output of another MET tool.
> > > > > >> (3) You'll need to add timing information as attributes
for each
> > > data
> > > > > >> variable.
> > > > > >>
> > > > > >> By way of example, I've attached a header dump of some
NetCDF
> > output
> > > > > from
> > > > > >> MET.
> > > > > >>
> > > > > >> Ultimately, whether you reformat the data into NetCDF for
MET to
> > > read
> > > > or
> > > > > >> pass it via python embedding, you're still defining the
same
> > info...
> > > > the
> > > > > >> grid spec and timing information. The downside of
reformatting
> to
> > > > > NetCDF
> > > > > >> is that pre-processing step and duplicating the data in
ascii
> and
> > > > NetCDF
> > > > > >> formats. But either way can get the job done.
> > > > > >>
> > > > > >> If you're stuck, feel free to send me a file with your
work in
> > > > progress,
> > > > > >> and I can edit it to make MET happy.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> John
> > > > > >>
> > > > > >>
> > > > > >> dimensions:
> > > > > >> lat = 154 ;
> > > > > >> lon = 169 ;
> > > > > >> variables:
> > > > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > > > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> > > > > >> APCP_24_A24_ENS_MEAN:long_name = "APCP_24
at A24
> > > > Ensemble
> > > > > >> Mean" ;
> > > > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > > > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> > > > > >> APCP_24_A24_ENS_MEAN:_FillValue = -9999.f
;
> > > > > >> APCP_24_A24_ENS_MEAN:init_time =
> "20091231_120000"
> > ;
> > > > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
"1262260800"
> ;
> > > > > >> APCP_24_A24_ENS_MEAN:valid_time =
> > "20100101_120000" ;
> > > > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
> "1262347200" ;
> > > > > >> APCP_24_A24_ENS_MEAN:accum_time = "240000"
;
> > > > > >> APCP_24_A24_ENS_MEAN:accum_time_sec =
86400 ;
> > > > > >> // global attributes:
> > > > > >> :FileOrigins = "File
> > > > > >>
../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
> > generated
> > > > > >> 20191023_165124 UTC on host number5 by t
> > > > > >> he MET ensemble_stat tool" ;
> > > > > >> :MET_version = "V9.0" ;
> > > > > >> :MET_tool = "ensemble_stat" ;
> > > > > >> :model = "WRF" ;
> > > > > >> :obtype = "MC_PCP" ;
> > > > > >> :Projection = "Polar Stereographic" ;
> > > > > >> :hemisphere = "N" ;
> > > > > >> :scale_lat = "60.000000 degrees_north" ;
> > > > > >> :lat_pin = "31.357000" ;
> > > > > >> :lon_pin = "-129.770000" ;
> > > > > >> :x_pin = "0.000000" ;
> > > > > >> :y_pin = "0.000000" ;
> > > > > >> :lon_orient = "-120.500000" ;
> > > > > >> :d_km = "10.395000 km" ;
> > > > > >> :r_km = "6371.200000 km" ;
> > > > > >> :nx = "169" ;
> > > > > >> :ny = "154" ;
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > >> met_help at ucar.edu> wrote:
> > > > > >>
> > > > > >> >
> > > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >
> > > > > >> >
> > > > > >> > Hi John,
> > > > > >> >
> > > > > >> > Does plot_data_plane work on Netcdf files. I have
the
> > > > following
> > > > > >> in my
> > > > > >> > netcdf file.
> > > > > >> > The error I get is
> > > > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a
valid data
> > > file
> > > > > >> > Do I need to tell plot_data_plane that I have a netcdf
file.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Sherrie
> > > > > >> > dimensions:
> > > > > >> > ny = 6144 ;
> > > > > >> > nx = 6144 ;
> > > > > >> > variables:
> > > > > >> > float lat(ny, nx) ;
> > > > > >> > lat:long_name = "latitude" ;
> > > > > >> > lat:lon_pin = 145.f ;
> > > > > >> > lat:lat_pin = -21.49193f ;
> > > > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > > > > >> > lat:r_km = "6371.2 km" ;
> > > > > >> > lat:d_km = "4.0km" ;
> > > > > >> > lat:lon_orient = "-80.0" ;
> > > > > >> > lat:y_pin = "0.000000" ;
> > > > > >> > lat:x_pin = "0.000000" ;
> > > > > >> > lat:scale_lat = "60.0 degrees_north" ;
> > > > > >> > lat:hemisphere = "N" ;
> > > > > >> > lat:projection = "Polar Stereographic" ;
> > > > > >> > lat:_FillValue = 9.96921e+36f ;
> > > > > >> > float lon(ny, nx) ;
> > > > > >> > lon:long_name = "longitude" ;
> > > > > >> > lon:lon_pin = 145.f ;
> > > > > >> > lon:lat_pin = -21.49193f ;
> > > > > >> > lon:ny = "6144" ;
> > > > > >> > lon:nx = "6144" ;
> > > > > >> > lon:r_km = "6371.2 km" ;
> > > > > >> > lon:d_km = "4.0km" ;
> > > > > >> > lon:lon_orient = "-80.0" ;
> > > > > >> > lon:y_pin = "0.000000" ;
> > > > > >> > lon:x_pin = "0.000000" ;
> > > > > >> > lon:scale_lat = "60.0 degrees_north" ;
> > > > > >> > lon:hemisphere = "N" ;
> > > > > >> > lon:projection = "Polar Stereographic" ;
> > > > > >> > lon:_FillValue = 9.96921e+36f ;
> > > > > >> > float snowC(ny, nx) ;
> > > > > >> > snowC:units = "%" ;
> > > > > >> > snowC:long_name = "Snow Cover" ;
> > > > > >> > snowC:lon_pin = 145.f ;
> > > > > >> > snowC:lat_pin = -21.49193f ;
> > > > > >> > snowC:ny = "6144" ;
> > > > > >> > snowC:nx = "6144" ;
> > > > > >> > snowC:r_km = "6371.2 km" ;
> > > > > >> > snowC:d_km = "4.0km" ;
> > > > > >> > snowC:lon_orient = "-80.0" ;
> > > > > >> > snowC:y_pin = "0.000000" ;
> > > > > >> > snowC:x_pin = "0.000000" ;
> > > > > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > > > > >> > snowC:hemisphere = "N" ;
> > > > > >> > snowC:projection = "Polar Stereographic" ;
> > > > > >> > snowC:_FillValue = 9.96921e+36f ;
> > > > > >> >
> > > > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway via
RT <
> > > > > >> > met_help at ucar.edu>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > Sherrie,
> > > > > >> > >
> > > > > >> > > So we need to figure out the grid specification for
the IMS
> > > polar
> > > > > >> > > stereographic grid on which your data resides.
> > > > > >> > >
> > > > > >> > > I did some hunting around online and suspect that
you're
> > > > processing
> > > > > >> this
> > > > > >> > > data:
> > > > > >> > > https://nsidc.org/data/G02156
> > > > > >> > >
> > > > > >> > > And the 4km resolution version of this data is on a
> 6444x6444
> > > > polar
> > > > > >> > > stereographic grid.
> > > > > >> > >
> > > > > >> > > On the "User's Guide" tab of that page, table 4 lists
some
> > > > relevant
> > > > > >> > > information:
> > > > > >> > > Latitude of true scale = 60 North
> > > > > >> > > Longitude below the pole = 80 West
> > > > > >> > >
> > > > > >> > > Based on that I'd say you want to try using the
following
> > > > settings:
> > > > > >> > >
> > > > > >> > > :Projection = "Polar Stereographic" ;
> > > > > >> > > :hemisphere = "N" ;
> > > > > >> > > :scale_lat = "60.0 degrees_north" ;
> > > > > >> > > :lat_pin = "???" ;
> > > > > >> > > :lon_pin = "???" ;
> > > > > >> > > :x_pin = "0.000000" ;
> > > > > >> > > :y_pin = "0.000000" ;
> > > > > >> > > :lon_orient = "-80.0" ;
> > > > > >> > > :d_km = "4.0 km" ;
> > > > > >> > > :r_km = "6371.2 km" ;
> > > > > >> > > :nx = "6144" ;
> > > > > >> > > :ny = "6144" ;
> > > > > >> > >
> > > > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and
lon_pin
> > > values
> > > > > you
> > > > > >> > > specify are for the "lower-left" corner of the grid.
Look
> at
> > > the
> > > > > >> lat/lon
> > > > > >> > > values you have and try just specifying that lat/lon
of the
> > > first
> > > > > >> point.
> > > > > >> > > Presumably, that'd be one of the 4 corners. And then
run
> > > > > >> plot_data_plane
> > > > > >> > > to see what the image looks like. If it differs from
your
> NCL
> > > > plot,
> > > > > >> we
> > > > > >> > can
> > > > > >> > > go from there.
> > > > > >> > >
> > > > > >> > > Thanks,
> > > > > >> > > John
> > > > > >> > >
> > > > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick -
NOAA
> > > Affiliate
> > > > > via
> > > > > >> > RT <
> > > > > >> > > met_help at ucar.edu> wrote:
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > >
> > > > > >> > > >
> > > > > >> > > > Good Morning John,
> > > > > >> > > >
> > > > > >> > > > I have a couple more questions at your
convenience,
> > > > > >> > > > I took the IMS ascii data and the lat/lon data
and
> > wrote a
> > > > > >> > > > Netcdf file that has both the IMS data and the
lat/lon
> data
> > in
> > > > > >> > > > it.dimensions:
> > > > > >> > > > ny = 6144 ;
> > > > > >> > > > nx = 6144 ;
> > > > > >> > > > variables:
> > > > > >> > > > float lat(ny, nx) ;
> > > > > >> > > > lat:_FillValue = 9.96921e+36f ;
> > > > > >> > > > float lon(ny, nx) ;
> > > > > >> > > > lon:_FillValue = 9.96921e+36f ;
> > > > > >> > > > float snowC(ny, nx) ;
> > > > > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > > > > >> > > > }
> > > > > >> > > >
> > > > > >> > > > I am not sure I understand the all of the metadata
above.
> > > > > >> > > > I am not sure of what the following should be or
how I
> > should
> > > > get
> > > > > it
> > > > > >> > > >
> > > > > >> > > > :scale_lat = "? degrees_north" ;
> > > > > >> > > > :lat_pin = ?;
> > > > > >> > > > :lon_pin = ?;
> > > > > >> > > > :x_pin = ? ;
> > > > > >> > > > :y_pin = "0.000000" ;
> > > > > >> > > > :lon_orient = ? ;
> > > > > >> > > > The following I think is correct for the IMS
polar
> > > > stereographic
> > > > > >> grid
> > > > > >> > > > :d_km = "4km" ;
> > > > > >> > > > :r_km = "6378.137km" ;
> > > > > >> > > > Since it is a 4km grid I believe I need to use a
radius
> of
> > > > > >> 6378.137
> > > > > >> > > > according to the documentation.
> > > > > >> > > > The IMS grid is the WGS-84 ellipse.
> > > > > >> > > >
> > > > > >> > > > Also my forecast is going to at 1kmX1km and the
IMS
> grid
> > > is a
> > > > > >> 4km.
> > > > > >> > > Do I
> > > > > >> > > > need to interpolate the IMS grid to a 1km grid?
> > > > > >> > > >
> > > > > >> > > > Sorry for all the questions.
> > > > > >> > > > Many thanks for your help.
> > > > > >> > > > Sherrie
> > > > > >> > > >
> > > > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley Gotway
via RT
> <
> > > > > >> > > > met_help at ucar.edu>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Sherrie,
> > > > > >> > > > >
> > > > > >> > > > > OK, great. FYI, unfortunately, the name of the
required
> > > > entries
> > > > > >> vary
> > > > > >> > > > > depending on the projection type and we haven't
done a
> > good
> > > > job
> > > > > >> > > > documenting
> > > > > >> > > > > all of the variations yet. However, those
entries are
> the
> > > > same
> > > > > as
> > > > > >> > the
> > > > > >> > > > > global attributes you'll find in the NetCDF
output from
> > MET
> > > > > tools.
> > > > > >> > > > >
> > > > > >> > > > > Here's an example of those attributes for a Polar
> > > > Stereographic
> > > > > >> > > > projection:
> > > > > >> > > > >
> > > > > >> > > > > :Projection = "Polar
Stereographic" ;
> > > > > >> > > > > :hemisphere = "N" ;
> > > > > >> > > > > :scale_lat = "60.000000
degrees_north" ;
> > > > > >> > > > > :lat_pin = "31.357000" ;
> > > > > >> > > > > :lon_pin = "-129.770000" ;
> > > > > >> > > > > :x_pin = "0.000000" ;
> > > > > >> > > > > :y_pin = "0.000000" ;
> > > > > >> > > > > :lon_orient = "-120.500000" ;
> > > > > >> > > > > :d_km = "10.395000 km" ;
> > > > > >> > > > > :r_km = "6371.200000 km" ;
> > > > > >> > > > > :nx = "169" ;
> > > > > >> > > > > :ny = "154" ;
> > > > > >> > > > >
> > > > > >> > > > > Hopefully that'll help.
> > > > > >> > > > >
> > > > > >> > > > > Thanks,
> > > > > >> > > > > John
> > > > > >> > > > >
> > > > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie Fredrick
- NOAA
> > > > > Affiliate
> > > > > >> via
> > > > > >> > > RT
> > > > > >> > > > <
> > > > > >> > > > > met_help at ucar.edu> wrote:
> > > > > >> > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > Hi John,
> > > > > >> > > > > > Thank you so much! I will give it a try.
> > > > > >> > > > > >
> > > > > >> > > > > > cheers,
> > > > > >> > > > > > Sherrie
> > > > > >> > > > > >
> > > > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley
Gotway via
> > RT
> > > <
> > > > > >> > > > > > met_help at ucar.edu>
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > > Hello Sherrie,
> > > > > >> > > > > > >
> > > > > >> > > > > > > Yes this is possible, but you'll need to use
the
> > python
> > > > > >> embedding
> > > > > >> > > > > feature
> > > > > >> > > > > > > of MET. Specifically, we'll need a python
script
> > which
> > > > > reads
> > > > > >> > data
> > > > > >> > > > from
> > > > > >> > > > > > > that ASCII file and defines the metadata for
it
> (e.g.
> > > > timing
> > > > > >> and
> > > > > >> > > grid
> > > > > >> > > > > > > info). This is a relatively new feature in
MET, and
> > > we're
> > > > > >> > revising
> > > > > >> > > > the
> > > > > >> > > > > > > logic for it for met-9.0, but reading data
from
> ASCII
> > > > should
> > > > > >> be
> > > > > >> > > very
> > > > > >> > > > > > > straight-forward. I'd expect this to work
well.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Examples of using python embedding to read
gridded
> > data
> > > > into
> > > > > >> MET
> > > > > >> > > can
> > > > > >> > > > be
> > > > > >> > > > > > > found here:
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > > >> > > > > > >
> > > > > >> > > > > > > The last example on that page reads data from
ASCII,
> > so
> > > > that
> > > > > >> > might
> > > > > >> > > > be a
> > > > > >> > > > > > > good place to start:
> > > > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > > > > >> > 'name="read_AWS_CMORPH.py
> > > > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > > >> > > > > > >
> > > > > >> > > > > > > I highly recommend starting with the
plot_data_plane
> > > tool
> > > > > >> since
> > > > > >> > you
> > > > > >> > > > can
> > > > > >> > > > > > > compare the output plot to your NCL output to
make
> > sure
> > > > > you're
> > > > > >> > > > passing
> > > > > >> > > > > > the
> > > > > >> > > > > > > data to MET in the correct orientation with
the
> > correct
> > > > grid
> > > > > >> > info.
> > > > > >> > > > If
> > > > > >> > > > > > > you're comfortable with python, feel free to
adapt
> > that
> > > > > >> script to
> > > > > >> > > > read
> > > > > >> > > > > > your
> > > > > >> > > > > > > data. If you need help with this, please
send us a
> > > sample
> > > > > >> data
> > > > > >> > > file,
> > > > > >> > > > > the
> > > > > >> > > > > > > lat/lon data file, and an NCL plot of that
data.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Thanks,
> > > > > >> > > > > > > John
> > > > > >> > > > > > >
> > > > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie
Fredrick -
> > NOAA
> > > > > >> > Affiliate
> > > > > >> > > > via
> > > > > >> > > > > > RT <
> > > > > >> > > > > > > met_help at ucar.edu> wrote:
> > > > > >> > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748 was
acted
> > > upon.
> > > > > >> > > > > > > > Transaction: Ticket created by
> > > > sherrie.fredrick at noaa.gov
> > > > > >> > > > > > > > Queue: met_help
> > > > > >> > > > > > > > Subject: using MET with IMS snow data
> > > > > >> > > > > > > > Owner: Nobody
> > > > > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > >> > > > > > > > Status: new
> > > > > >> > > > > > > > Ticket <URL:
> > > > > >> > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > >> > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Good Morning,
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > I would love to use MET/MET Plus to
verify
> the
> > > > > >> > NOAA/NESDIS
> > > > > >> > > > > > > > Interactive Multisensor Snow and Ice
Mapping
> System
> > > > (IMS)
> > > > > >> data
> > > > > >> > > with
> > > > > >> > > > > > ERA5
> > > > > >> > > > > > > > data. The IMS data is on a Polar
Stereographic
> 4km
> > > grid
> > > > > the
> > > > > >> > data
> > > > > >> > > > > > format
> > > > > >> > > > > > > is
> > > > > >> > > > > > > > ASCII. The latitude and longitude
definitions are
> > in
> > > a
> > > > > >> > separate
> > > > > >> > > > > > file. I
> > > > > >> > > > > > > > can read in the data and plot with NCL.
The ERA5
> > data
> > > > > that
> > > > > >> I
> > > > > >> > am
> > > > > >> > > > > using
> > > > > >> > > > > > is
> > > > > >> > > > > > > > the ERA5 land data. It is in grib format.
I can
> > > > choose
> > > > > >> the
> > > > > >> > > > > > resolution
> > > > > >> > > > > > > > when down loading the data. Is this
possible?
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > I will be working on Hera.
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > I hope this makes sense,
> > > > > >> > > > > > > > Many Thanks,
> > > > > >> > > > > > > > Sherrie
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: Sherrie Fredrick - NOAA Affiliate
Time: Mon Oct 28 19:53:41 2019
Hi John,
That will save a bunch of time. I also have other data that are
ascii
from a different source. I am sure you
script will help with that also.
Many Thanks,
Sherrie
On Mon, Oct 28, 2019 at 5:15 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Sherrie,
>
> Just to show you an alternative, I worked up an example of using
python
> embedding to read this 4km data.
>
> Here's an example of calling plot_data_plane using python embedding
to read
> the ASCII data directly:
>
> plot_data_plane PYTHON_NUMPY ims.ps 'name="read_NSIDC_IMS_4km.py
> ims2017010_4km_v1.3.asc";'
>
> And that produces the same image we got before: ims.png
>
> I've attached the python script I used. The advantage here is not
needing
> to write intermediate NetCDF files and being able to "fix" issues
that
> arise directly in the python script.
>
> Thanks,
> John
>
> On Mon, Oct 28, 2019 at 1:59 PM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> >
> > Hi John,
> >
> > Yea! Thanks very much for all your help. I will make the
changes
> > you suggested.
> > Sherrie
> >
> > On Mon, Oct 28, 2019 at 1:50 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > Great, thanks. I ran plot_data_plane on the ims NetCDF and got
the
> same
> > > error you did. Here are the issues I found and how to fix them.
> > >
> > > (1) Rename ny/nx dimension to lat/lon:
> > > Looking at that NetCDF I see that you're formatting it as if it
were a
> > > NetCDF file created by another MET tool. The library in MET
that reads
> > > this data assumes the dimensions are named lat and lon.
> > > * ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
> > > <http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
> > >
> > > (2) The "projection" global attribute should be capitalized as
> > > "Projection":
> > > * ncatted -a projection,global,d,c,'' -a
> Projection,global,a,c,"Polar
> > > Stereographic" ims_oct-01-2017_MOD.nc*
> > >
> > > (3) I changed to reference point to be the north pole: (x, y) =
(3072,
> > > 3072) is the north pole at (lat, lon) = (90, 0):
> > > * ncatted -a lon_pin,global,o,c,"0" -a
lat_pin,global,o,c,"90" -a
> > > x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072"
> > ims_oct-01-2017_MOD.nc*
> > >
> > > After all of these changes, I ran plot_data_plane to produce the
> attached
> > > plot.
> > > *plot_data_plane ims_oct-01-2017_MOD.nc ims.ps
<http://ims.ps>
> > > 'name="snowC"; level="(*,*)";'*
> > >
> > > And the data lines up very well with the map data!
> > >
> > > However, issues with the timing info remain. Your snowC data
variable
> > > doesn't define the timing info in the way expected by the NetCDF
output
> > > from MET. It should contain variable attributes, like this:
> > > APCP_12:init_time = "20050807_000000" ;
> > >
> > > APCP_12:init_time_ut = "1123372800" ;
> > >
> > > APCP_12:valid_time = "20050807_120000" ;
> > >
> > > APCP_12:valid_time_ut = "1123416000" ;
> > >
> > > APCP_12:accum_time = "120000" ;
> > >
> > > APCP_12:accum_time_sec = 43200 ;
> > > Technically, I believe the code is parsing the timing info from
the
> > > "init_time_ut" and "valid_time_ut" attributes, where UT is
unixtime,
> the
> > > number of seconds since Jan 1, 1970.
> > >
> > > Here's how you can use the unix date command to convert between
October
> > 1,
> > > 2017 and unixtime:
> > >
> > > *date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
> > >
> > > And then going back the other way:
> > > *date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
> > > *2019100100*
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>
> > > >
> > > > Hi John,
> > > >
> > > > I have placed the following files on
.../met_help/sherrie_data.
> > > > grid_stat_config
> > > > ims2017274_4km_v1.3.asc - This is the original ascii
file
> > > > downloaded from the IMS web site.
> > > >
Northern
> > > > Hemisphere, Polar stereographic, 4kmX4km grid resolution.
> > > > imslat_4km.bin - The binary
latitude file
> > > from
> > > > the IMS web site
> > > > imslon_4km.bin - The binary
longitude file
> > > from
> > > > the IMS web site
> > > > ims_oct-01-2017.nc - The Netcdf file that
I
> > > produced.
> > > > oct201701.nc - The ERA5 netcdf
file
> from
> > > the
> > > > ERA5-Land data. Downloaded from Copernicus.
> > > > I
downloaded
> > the
> > > > ERA5 data at a 4kmX4km grid resolution.
> > > > I included the original files just in case. I think
there will
> > be a
> > > > lot of interest in using MET to verify model data. My
marching
> orders
> > > are
> > > > to use only MET for verification. I will be running the
FV3workflow
> to
> > > get
> > > > FV3 model data. Right now we are not ready to do that so
> > > > we want to use ERA5 data with the IMS data.
> > > > Your help is much appreciated,
> > > > Sherrie
> > > >
> > > > On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Hi Sherrie,
> > > > >
> > > > > I just tried logging on to Hera but am having problems... so
I
> wrote
> > > for
> > > > > help from Hera's helpdesk.
> > > > >
> > > > > In the meantime, is there any chance you could post sample
files to
> > our
> > > > > anonymous ftp site? I'd need the latest iteration of the
ERA5 and
> > IMS
> > > > > files and your Grid-Stat config file. Here are
instructions:
> > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA
Affiliate
> > via
> > > > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > > >
> > > > > > Good Morning John,
> > > > > >
> > > > > > I have my ERA5 data and my IMS data. MET is still
unable to
> > > read
> > > > my
> > > > > > IMS data. The message I get is
> > > > > > DEBUG 1: Observation File:
> > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> > > > > imc_oct-01-2017.nc
> > > > > > DEBUG 2:
> > > > > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &,
DataPlane
> &)
> > > > const
> > > > > > -> data was flipped to north.
> > > > > > ERROR :
> > > > > > ERROR : NcCfFile::getData(NcVar *, const LongArray &,
DataPlane
> &)
> > > > const
> > > > > > -> star found in bad slot
> > > > > > ERROR :
> > > > > > I have no idea about this error. I am sorry to need so
much
> help.
> > > > > >
> > > > > > A header dump of my IMS Netcdf file is
> > > > > > dimensions:
> > > > > > time = UNLIMITED ; // (1 currently)
> > > > > > ny = 6144 ;
> > > > > > nx = 6144 ;
> > > > > > variables:
> > > > > > double time(time) ;
> > > > > > time:calendar = "standard" ;
> > > > > > time:units = "hours since 1900-01-01 00:00:00" ;
> > > > > > time:long_name = "time" ;
> > > > > >
> > > > > > float lat(ny, nx) ;
> > > > > > lat:units = "degrees_north" ;
> > > > > > lat:long_name = "latitude" ;
> > > > > > lat:forecast_time = 0 ;
> > > > > > lat:forecast_time_units = "hours" ;
> > > > > > lat:initial_time = "10/01/2017 (00:00)" ;
> > > > > > lat:_FillValue = 9.96921e+36f ;
> > > > > >
> > > > > > float lon(ny, nx) ;
> > > > > > lon:units = "degrees_east" ;
> > > > > > lon:long_name = "longitude" ;
> > > > > > lon:forecast_time = 0 ;
> > > > > > lon:forecast_time_units = "hours" ;
> > > > > > lon:initial_time = "10/01/2017 (00:00)" ;
> > > > > > lon:_FillValue = 9.96921e+36f ;
> > > > > >
> > > > > > float snowC(ny, nx) ;
> > > > > > snowC:level = "Z0" ;
> > > > > > snowC:units = "%" ;
> > > > > > snowC:long_name = "Snow Cover" ;
> > > > > > snowC:forecast_time = 0 ;
> > > > > > snowC:forecast_time_units = "hours" ;
> > > > > > snowC:initial_time = "10/01/2017 (00:00)" ;
> > > > > > snowC:_FillValue = 9.96921e+36f ;
> > > > > >
> > > > > > // global attributes:
> > > > > > :lon_pin = 145.f ;
> > > > > > :lat_pin = -21.49193f ;
> > > > > > :ny = "6144" ;
> > > > > > :nx = "6144" ;
> > > > > > :r_km = "6371.2 km" ;
> > > > > > :d_km = "4.0km" ;
> > > > > > :lon_orient = "-80.0" ;
> > > > > > :y_pin = "0.000000" ;
> > > > > > :x_pin = "0.000000" ;
> > > > > > :scale_lat = "60.0 degrees_north" ;
> > > > > > :hemisphere = "N" ;
> > > > > > :projection = "Polar Stereographic" ;
> > > > > > :MET_version = "V8.1" ;
> > > > > >
> > > > > > I am working on Hera in the directory
> > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> > > > > >
> > > > > > My Netcdf IMS files are in
> > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > > > > > Thanks,
> > > > > > Sherrie
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > >> Sherrie,
> > > > > > >>
> > > > > > >> My intention was to have you use the Python embedding
option
> to
> > > read
> > > > > > this
> > > > > > >> data, but it looks like you're reformatting it into
NetCDF
> > > instead.
> > > > > And
> > > > > > >> that's fine too.
> > > > > > >>
> > > > > > >> But there's some issues based on the NetCDF header you
sent.
> > > > > > >>
> > > > > > >> (1) Those projection attributes should be defined as
global
> > > > > attributes,
> > > > > > >> not
> > > > > > >> separately for each variable.
> > > > > > >> (2) You are formatting a NetCDF file to follow the
conventions
> > > used
> > > > > for
> > > > > > >> the
> > > > > > >> NetCDF output generated by the MET tools. If you add
the
> global
> > > > > > attribute
> > > > > > >> "MET_version = "V8.0";" then MET will correctly process
that
> > file
> > > as
> > > > > > being
> > > > > > >> the NetCDF output of another MET tool.
> > > > > > >> (3) You'll need to add timing information as attributes
for
> each
> > > > data
> > > > > > >> variable.
> > > > > > >>
> > > > > > >> By way of example, I've attached a header dump of some
NetCDF
> > > output
> > > > > > from
> > > > > > >> MET.
> > > > > > >>
> > > > > > >> Ultimately, whether you reformat the data into NetCDF
for MET
> to
> > > > read
> > > > > or
> > > > > > >> pass it via python embedding, you're still defining the
same
> > > info...
> > > > > the
> > > > > > >> grid spec and timing information. The downside of
> reformatting
> > to
> > > > > > NetCDF
> > > > > > >> is that pre-processing step and duplicating the data in
ascii
> > and
> > > > > NetCDF
> > > > > > >> formats. But either way can get the job done.
> > > > > > >>
> > > > > > >> If you're stuck, feel free to send me a file with your
work in
> > > > > progress,
> > > > > > >> and I can edit it to make MET happy.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> John
> > > > > > >>
> > > > > > >>
> > > > > > >> dimensions:
> > > > > > >> lat = 154 ;
> > > > > > >> lon = 169 ;
> > > > > > >> variables:
> > > > > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > > > > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:long_name =
"APCP_24 at
> A24
> > > > > Ensemble
> > > > > > >> Mean" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:_FillValue =
-9999.f ;
> > > > > > >> APCP_24_A24_ENS_MEAN:init_time =
> > "20091231_120000"
> > > ;
> > > > > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
> "1262260800"
> > ;
> > > > > > >> APCP_24_A24_ENS_MEAN:valid_time =
> > > "20100101_120000" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
> > "1262347200" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:accum_time =
"240000" ;
> > > > > > >> APCP_24_A24_ENS_MEAN:accum_time_sec =
86400 ;
> > > > > > >> // global attributes:
> > > > > > >> :FileOrigins = "File
> > > > > > >>
../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
> > > generated
> > > > > > >> 20191023_165124 UTC on host number5 by t
> > > > > > >> he MET ensemble_stat tool" ;
> > > > > > >> :MET_version = "V9.0" ;
> > > > > > >> :MET_tool = "ensemble_stat" ;
> > > > > > >> :model = "WRF" ;
> > > > > > >> :obtype = "MC_PCP" ;
> > > > > > >> :Projection = "Polar Stereographic" ;
> > > > > > >> :hemisphere = "N" ;
> > > > > > >> :scale_lat = "60.000000 degrees_north" ;
> > > > > > >> :lat_pin = "31.357000" ;
> > > > > > >> :lon_pin = "-129.770000" ;
> > > > > > >> :x_pin = "0.000000" ;
> > > > > > >> :y_pin = "0.000000" ;
> > > > > > >> :lon_orient = "-120.500000" ;
> > > > > > >> :d_km = "10.395000 km" ;
> > > > > > >> :r_km = "6371.200000 km" ;
> > > > > > >> :nx = "169" ;
> > > > > > >> :ny = "154" ;
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick - NOAA
> > Affiliate
> > > > via
> > > > > > RT <
> > > > > > >> met_help at ucar.edu> wrote:
> > > > > > >>
> > > > > > >> >
> > > > > > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > >
> > > > > > >> >
> > > > > > >> > Hi John,
> > > > > > >> >
> > > > > > >> > Does plot_data_plane work on Netcdf files. I
have the
> > > > > following
> > > > > > >> in my
> > > > > > >> > netcdf file.
> > > > > > >> > The error I get is
> > > > > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a
valid
> data
> > > > file
> > > > > > >> > Do I need to tell plot_data_plane that I have a
netcdf file.
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Sherrie
> > > > > > >> > dimensions:
> > > > > > >> > ny = 6144 ;
> > > > > > >> > nx = 6144 ;
> > > > > > >> > variables:
> > > > > > >> > float lat(ny, nx) ;
> > > > > > >> > lat:long_name = "latitude" ;
> > > > > > >> > lat:lon_pin = 145.f ;
> > > > > > >> > lat:lat_pin = -21.49193f ;
> > > > > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > > > > > >> > lat:r_km = "6371.2 km" ;
> > > > > > >> > lat:d_km = "4.0km" ;
> > > > > > >> > lat:lon_orient = "-80.0" ;
> > > > > > >> > lat:y_pin = "0.000000" ;
> > > > > > >> > lat:x_pin = "0.000000" ;
> > > > > > >> > lat:scale_lat = "60.0 degrees_north" ;
> > > > > > >> > lat:hemisphere = "N" ;
> > > > > > >> > lat:projection = "Polar Stereographic" ;
> > > > > > >> > lat:_FillValue = 9.96921e+36f ;
> > > > > > >> > float lon(ny, nx) ;
> > > > > > >> > lon:long_name = "longitude" ;
> > > > > > >> > lon:lon_pin = 145.f ;
> > > > > > >> > lon:lat_pin = -21.49193f ;
> > > > > > >> > lon:ny = "6144" ;
> > > > > > >> > lon:nx = "6144" ;
> > > > > > >> > lon:r_km = "6371.2 km" ;
> > > > > > >> > lon:d_km = "4.0km" ;
> > > > > > >> > lon:lon_orient = "-80.0" ;
> > > > > > >> > lon:y_pin = "0.000000" ;
> > > > > > >> > lon:x_pin = "0.000000" ;
> > > > > > >> > lon:scale_lat = "60.0 degrees_north" ;
> > > > > > >> > lon:hemisphere = "N" ;
> > > > > > >> > lon:projection = "Polar Stereographic" ;
> > > > > > >> > lon:_FillValue = 9.96921e+36f ;
> > > > > > >> > float snowC(ny, nx) ;
> > > > > > >> > snowC:units = "%" ;
> > > > > > >> > snowC:long_name = "Snow Cover" ;
> > > > > > >> > snowC:lon_pin = 145.f ;
> > > > > > >> > snowC:lat_pin = -21.49193f ;
> > > > > > >> > snowC:ny = "6144" ;
> > > > > > >> > snowC:nx = "6144" ;
> > > > > > >> > snowC:r_km = "6371.2 km" ;
> > > > > > >> > snowC:d_km = "4.0km" ;
> > > > > > >> > snowC:lon_orient = "-80.0" ;
> > > > > > >> > snowC:y_pin = "0.000000" ;
> > > > > > >> > snowC:x_pin = "0.000000" ;
> > > > > > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > > > > > >> > snowC:hemisphere = "N" ;
> > > > > > >> > snowC:projection = "Polar Stereographic" ;
> > > > > > >> > snowC:_FillValue = 9.96921e+36f ;
> > > > > > >> >
> > > > > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway
via RT <
> > > > > > >> > met_help at ucar.edu>
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > > Sherrie,
> > > > > > >> > >
> > > > > > >> > > So we need to figure out the grid specification for
the
> IMS
> > > > polar
> > > > > > >> > > stereographic grid on which your data resides.
> > > > > > >> > >
> > > > > > >> > > I did some hunting around online and suspect that
you're
> > > > > processing
> > > > > > >> this
> > > > > > >> > > data:
> > > > > > >> > > https://nsidc.org/data/G02156
> > > > > > >> > >
> > > > > > >> > > And the 4km resolution version of this data is on a
> > 6444x6444
> > > > > polar
> > > > > > >> > > stereographic grid.
> > > > > > >> > >
> > > > > > >> > > On the "User's Guide" tab of that page, table 4
lists some
> > > > > relevant
> > > > > > >> > > information:
> > > > > > >> > > Latitude of true scale = 60 North
> > > > > > >> > > Longitude below the pole = 80 West
> > > > > > >> > >
> > > > > > >> > > Based on that I'd say you want to try using the
following
> > > > > settings:
> > > > > > >> > >
> > > > > > >> > > :Projection = "Polar Stereographic"
;
> > > > > > >> > > :hemisphere = "N" ;
> > > > > > >> > > :scale_lat = "60.0 degrees_north" ;
> > > > > > >> > > :lat_pin = "???" ;
> > > > > > >> > > :lon_pin = "???" ;
> > > > > > >> > > :x_pin = "0.000000" ;
> > > > > > >> > > :y_pin = "0.000000" ;
> > > > > > >> > > :lon_orient = "-80.0" ;
> > > > > > >> > > :d_km = "4.0 km" ;
> > > > > > >> > > :r_km = "6371.2 km" ;
> > > > > > >> > > :nx = "6144" ;
> > > > > > >> > > :ny = "6144" ;
> > > > > > >> > >
> > > > > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin and
lon_pin
> > > > values
> > > > > > you
> > > > > > >> > > specify are for the "lower-left" corner of the
grid. Look
> > at
> > > > the
> > > > > > >> lat/lon
> > > > > > >> > > values you have and try just specifying that
lat/lon of
> the
> > > > first
> > > > > > >> point.
> > > > > > >> > > Presumably, that'd be one of the 4 corners. And
then run
> > > > > > >> plot_data_plane
> > > > > > >> > > to see what the image looks like. If it differs
from your
> > NCL
> > > > > plot,
> > > > > > >> we
> > > > > > >> > can
> > > > > > >> > > go from there.
> > > > > > >> > >
> > > > > > >> > > Thanks,
> > > > > > >> > > John
> > > > > > >> > >
> > > > > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick -
NOAA
> > > > Affiliate
> > > > > > via
> > > > > > >> > RT <
> > > > > > >> > > met_help at ucar.edu> wrote:
> > > > > > >> > >
> > > > > > >> > > >
> > > > > > >> > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > >
> > > > > > >> > > >
> > > > > > >> > > > Good Morning John,
> > > > > > >> > > >
> > > > > > >> > > > I have a couple more questions at your
convenience,
> > > > > > >> > > > I took the IMS ascii data and the lat/lon
data and
> > > wrote a
> > > > > > >> > > > Netcdf file that has both the IMS data and the
lat/lon
> > data
> > > in
> > > > > > >> > > > it.dimensions:
> > > > > > >> > > > ny = 6144 ;
> > > > > > >> > > > nx = 6144 ;
> > > > > > >> > > > variables:
> > > > > > >> > > > float lat(ny, nx) ;
> > > > > > >> > > > lat:_FillValue = 9.96921e+36f ;
> > > > > > >> > > > float lon(ny, nx) ;
> > > > > > >> > > > lon:_FillValue = 9.96921e+36f ;
> > > > > > >> > > > float snowC(ny, nx) ;
> > > > > > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > > > > > >> > > > }
> > > > > > >> > > >
> > > > > > >> > > > I am not sure I understand the all of the
metadata
> above.
> > > > > > >> > > > I am not sure of what the following should be or
how I
> > > should
> > > > > get
> > > > > > it
> > > > > > >> > > >
> > > > > > >> > > > :scale_lat = "? degrees_north" ;
> > > > > > >> > > > :lat_pin = ?;
> > > > > > >> > > > :lon_pin = ?;
> > > > > > >> > > > :x_pin = ? ;
> > > > > > >> > > > :y_pin = "0.000000" ;
> > > > > > >> > > > :lon_orient = ? ;
> > > > > > >> > > > The following I think is correct for the IMS
polar
> > > > > stereographic
> > > > > > >> grid
> > > > > > >> > > > :d_km = "4km" ;
> > > > > > >> > > > :r_km = "6378.137km" ;
> > > > > > >> > > > Since it is a 4km grid I believe I need to use
a
> radius
> > of
> > > > > > >> 6378.137
> > > > > > >> > > > according to the documentation.
> > > > > > >> > > > The IMS grid is the WGS-84 ellipse.
> > > > > > >> > > >
> > > > > > >> > > > Also my forecast is going to at 1kmX1km and
the IMS
> > grid
> > > > is a
> > > > > > >> 4km.
> > > > > > >> > > Do I
> > > > > > >> > > > need to interpolate the IMS grid to a 1km grid?
> > > > > > >> > > >
> > > > > > >> > > > Sorry for all the questions.
> > > > > > >> > > > Many thanks for your help.
> > > > > > >> > > > Sherrie
> > > > > > >> > > >
> > > > > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley
Gotway via
> RT
> > <
> > > > > > >> > > > met_help at ucar.edu>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Sherrie,
> > > > > > >> > > > >
> > > > > > >> > > > > OK, great. FYI, unfortunately, the name of the
> required
> > > > > entries
> > > > > > >> vary
> > > > > > >> > > > > depending on the projection type and we haven't
done a
> > > good
> > > > > job
> > > > > > >> > > > documenting
> > > > > > >> > > > > all of the variations yet. However, those
entries are
> > the
> > > > > same
> > > > > > as
> > > > > > >> > the
> > > > > > >> > > > > global attributes you'll find in the NetCDF
output
> from
> > > MET
> > > > > > tools.
> > > > > > >> > > > >
> > > > > > >> > > > > Here's an example of those attributes for a
Polar
> > > > > Stereographic
> > > > > > >> > > > projection:
> > > > > > >> > > > >
> > > > > > >> > > > > :Projection = "Polar
Stereographic" ;
> > > > > > >> > > > > :hemisphere = "N" ;
> > > > > > >> > > > > :scale_lat = "60.000000
degrees_north"
> ;
> > > > > > >> > > > > :lat_pin = "31.357000" ;
> > > > > > >> > > > > :lon_pin = "-129.770000" ;
> > > > > > >> > > > > :x_pin = "0.000000" ;
> > > > > > >> > > > > :y_pin = "0.000000" ;
> > > > > > >> > > > > :lon_orient = "-120.500000" ;
> > > > > > >> > > > > :d_km = "10.395000 km" ;
> > > > > > >> > > > > :r_km = "6371.200000 km" ;
> > > > > > >> > > > > :nx = "169" ;
> > > > > > >> > > > > :ny = "154" ;
> > > > > > >> > > > >
> > > > > > >> > > > > Hopefully that'll help.
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks,
> > > > > > >> > > > > John
> > > > > > >> > > > >
> > > > > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie
Fredrick -
> NOAA
> > > > > > Affiliate
> > > > > > >> via
> > > > > > >> > > RT
> > > > > > >> > > > <
> > > > > > >> > > > > met_help at ucar.edu> wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > Hi John,
> > > > > > >> > > > > > Thank you so much! I will give it a try.
> > > > > > >> > > > > >
> > > > > > >> > > > > > cheers,
> > > > > > >> > > > > > Sherrie
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley
Gotway
> via
> > > RT
> > > > <
> > > > > > >> > > > > > met_help at ucar.edu>
> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > > > Hello Sherrie,
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Yes this is possible, but you'll need to
use the
> > > python
> > > > > > >> embedding
> > > > > > >> > > > > feature
> > > > > > >> > > > > > > of MET. Specifically, we'll need a python
script
> > > which
> > > > > > reads
> > > > > > >> > data
> > > > > > >> > > > from
> > > > > > >> > > > > > > that ASCII file and defines the metadata
for it
> > (e.g.
> > > > > timing
> > > > > > >> and
> > > > > > >> > > grid
> > > > > > >> > > > > > > info). This is a relatively new feature in
MET,
> and
> > > > we're
> > > > > > >> > revising
> > > > > > >> > > > the
> > > > > > >> > > > > > > logic for it for met-9.0, but reading data
from
> > ASCII
> > > > > should
> > > > > > >> be
> > > > > > >> > > very
> > > > > > >> > > > > > > straight-forward. I'd expect this to work
well.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Examples of using python embedding to read
gridded
> > > data
> > > > > into
> > > > > > >> MET
> > > > > > >> > > can
> > > > > > >> > > > be
> > > > > > >> > > > > > > found here:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > The last example on that page reads data
from
> ASCII,
> > > so
> > > > > that
> > > > > > >> > might
> > > > > > >> > > > be a
> > > > > > >> > > > > > > good place to start:
> > > > > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > > > > > >> > 'name="read_AWS_CMORPH.py
> > > > > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > I highly recommend starting with the
> plot_data_plane
> > > > tool
> > > > > > >> since
> > > > > > >> > you
> > > > > > >> > > > can
> > > > > > >> > > > > > > compare the output plot to your NCL output
to make
> > > sure
> > > > > > you're
> > > > > > >> > > > passing
> > > > > > >> > > > > > the
> > > > > > >> > > > > > > data to MET in the correct orientation with
the
> > > correct
> > > > > grid
> > > > > > >> > info.
> > > > > > >> > > > If
> > > > > > >> > > > > > > you're comfortable with python, feel free
to adapt
> > > that
> > > > > > >> script to
> > > > > > >> > > > read
> > > > > > >> > > > > > your
> > > > > > >> > > > > > > data. If you need help with this, please
send us
> a
> > > > sample
> > > > > > >> data
> > > > > > >> > > file,
> > > > > > >> > > > > the
> > > > > > >> > > > > > > lat/lon data file, and an NCL plot of that
data.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Thanks,
> > > > > > >> > > > > > > John
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie
Fredrick
> -
> > > NOAA
> > > > > > >> > Affiliate
> > > > > > >> > > > via
> > > > > > >> > > > > > RT <
> > > > > > >> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748
was
> acted
> > > > upon.
> > > > > > >> > > > > > > > Transaction: Ticket created by
> > > > > sherrie.fredrick at noaa.gov
> > > > > > >> > > > > > > > Queue: met_help
> > > > > > >> > > > > > > > Subject: using MET with IMS snow
data
> > > > > > >> > > > > > > > Owner: Nobody
> > > > > > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > > >> > > > > > > > Status: new
> > > > > > >> > > > > > > > Ticket <URL:
> > > > > > >> > > > >
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > Good Morning,
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > I would love to use MET/MET Plus to
verify
> > the
> > > > > > >> > NOAA/NESDIS
> > > > > > >> > > > > > > > Interactive Multisensor Snow and Ice
Mapping
> > System
> > > > > (IMS)
> > > > > > >> data
> > > > > > >> > > with
> > > > > > >> > > > > > ERA5
> > > > > > >> > > > > > > > data. The IMS data is on a Polar
Stereographic
> > 4km
> > > > grid
> > > > > > the
> > > > > > >> > data
> > > > > > >> > > > > > format
> > > > > > >> > > > > > > is
> > > > > > >> > > > > > > > ASCII. The latitude and longitude
definitions
> are
> > > in
> > > > a
> > > > > > >> > separate
> > > > > > >> > > > > > file. I
> > > > > > >> > > > > > > > can read in the data and plot with NCL.
The
> ERA5
> > > data
> > > > > > that
> > > > > > >> I
> > > > > > >> > am
> > > > > > >> > > > > using
> > > > > > >> > > > > > is
> > > > > > >> > > > > > > > the ERA5 land data. It is in grib
format. I
> can
> > > > > choose
> > > > > > >> the
> > > > > > >> > > > > > resolution
> > > > > > >> > > > > > > > when down loading the data. Is this
possible?
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > I will be working on Hera.
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > I hope this makes sense,
> > > > > > >> > > > > > > > Many Thanks,
> > > > > > >> > > > > > > > Sherrie
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Tue Oct 29 09:52:31 2019
Sherrie,
Great, glad that's helpful. I'll go ahead and post this to the MET
website
as another example. Ideally, the script would handle all 3
resolutions
(1km, 4km, and 24km) but something's better than nothing.
Thanks,
John
On Mon, Oct 28, 2019 at 7:53 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>
> Hi John,
> That will save a bunch of time. I also have other data that
are ascii
> from a different source. I am sure you
> script will help with that also.
>
> Many Thanks,
> Sherrie
>
> On Mon, Oct 28, 2019 at 5:15 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > Just to show you an alternative, I worked up an example of using
python
> > embedding to read this 4km data.
> >
> > Here's an example of calling plot_data_plane using python
embedding to
> read
> > the ASCII data directly:
> >
> > plot_data_plane PYTHON_NUMPY ims.ps 'name="read_NSIDC_IMS_4km.py
> > ims2017010_4km_v1.3.asc";'
> >
> > And that produces the same image we got before: ims.png
> >
> > I've attached the python script I used. The advantage here is not
> needing
> > to write intermediate NetCDF files and being able to "fix" issues
that
> > arise directly in the python script.
> >
> > Thanks,
> > John
> >
> > On Mon, Oct 28, 2019 at 1:59 PM Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > >
> > > Hi John,
> > >
> > > Yea! Thanks very much for all your help. I will make the
> changes
> > > you suggested.
> > > Sherrie
> > >
> > > On Mon, Oct 28, 2019 at 1:50 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > Great, thanks. I ran plot_data_plane on the ims NetCDF and
got the
> > same
> > > > error you did. Here are the issues I found and how to fix
them.
> > > >
> > > > (1) Rename ny/nx dimension to lat/lon:
> > > > Looking at that NetCDF I see that you're formatting it as if
it were
> a
> > > > NetCDF file created by another MET tool. The library in MET
that
> reads
> > > > this data assumes the dimensions are named lat and lon.
> > > > * ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
> > > > <http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
> > > >
> > > > (2) The "projection" global attribute should be capitalized as
> > > > "Projection":
> > > > * ncatted -a projection,global,d,c,'' -a
> > Projection,global,a,c,"Polar
> > > > Stereographic" ims_oct-01-2017_MOD.nc*
> > > >
> > > > (3) I changed to reference point to be the north pole: (x, y)
=
> (3072,
> > > > 3072) is the north pole at (lat, lon) = (90, 0):
> > > > * ncatted -a lon_pin,global,o,c,"0" -a
lat_pin,global,o,c,"90" -a
> > > > x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072"
> > > ims_oct-01-2017_MOD.nc*
> > > >
> > > > After all of these changes, I ran plot_data_plane to produce
the
> > attached
> > > > plot.
> > > > *plot_data_plane ims_oct-01-2017_MOD.nc ims.ps
<http://ims.ps>
> > > > 'name="snowC"; level="(*,*)";'*
> > > >
> > > > And the data lines up very well with the map data!
> > > >
> > > > However, issues with the timing info remain. Your snowC data
> variable
> > > > doesn't define the timing info in the way expected by the
NetCDF
> output
> > > > from MET. It should contain variable attributes, like this:
> > > > APCP_12:init_time = "20050807_000000" ;
> > > >
> > > > APCP_12:init_time_ut = "1123372800" ;
> > > >
> > > > APCP_12:valid_time = "20050807_120000" ;
> > > >
> > > > APCP_12:valid_time_ut = "1123416000" ;
> > > >
> > > > APCP_12:accum_time = "120000" ;
> > > >
> > > > APCP_12:accum_time_sec = 43200 ;
> > > > Technically, I believe the code is parsing the timing info
from the
> > > > "init_time_ut" and "valid_time_ut" attributes, where UT is
unixtime,
> > the
> > > > number of seconds since Jan 1, 1970.
> > > >
> > > > Here's how you can use the unix date command to convert
between
> October
> > > 1,
> > > > 2017 and unixtime:
> > > >
> > > > *date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
> > > >
> > > > And then going back the other way:
> > > > *date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
> > > > *2019100100*
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > > I have placed the following files on
> .../met_help/sherrie_data.
> > > > > grid_stat_config
> > > > > ims2017274_4km_v1.3.asc - This is the original ascii
file
> > > > > downloaded from the IMS web site.
> > > > >
Northern
> > > > > Hemisphere, Polar stereographic, 4kmX4km grid resolution.
> > > > > imslat_4km.bin - The binary
latitude
> file
> > > > from
> > > > > the IMS web site
> > > > > imslon_4km.bin - The binary
longitude
> file
> > > > from
> > > > > the IMS web site
> > > > > ims_oct-01-2017.nc - The Netcdf file
that I
> > > > produced.
> > > > > oct201701.nc - The ERA5 netcdf
file
> > from
> > > > the
> > > > > ERA5-Land data. Downloaded from Copernicus.
> > > > > I
> downloaded
> > > the
> > > > > ERA5 data at a 4kmX4km grid resolution.
> > > > > I included the original files just in case. I think
there
> will
> > > be a
> > > > > lot of interest in using MET to verify model data. My
marching
> > orders
> > > > are
> > > > > to use only MET for verification. I will be running the
> FV3workflow
> > to
> > > > get
> > > > > FV3 model data. Right now we are not ready to do that so
> > > > > we want to use ERA5 data with the IMS data.
> > > > > Your help is much appreciated,
> > > > > Sherrie
> > > > >
> > > > > On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Hi Sherrie,
> > > > > >
> > > > > > I just tried logging on to Hera but am having problems...
so I
> > wrote
> > > > for
> > > > > > help from Hera's helpdesk.
> > > > > >
> > > > > > In the meantime, is there any chance you could post sample
files
> to
> > > our
> > > > > > anonymous ftp site? I'd need the latest iteration of the
ERA5
> and
> > > IMS
> > > > > > files and your Grid-Stat config file. Here are
instructions:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> >
> > > > > > >
> > > > > > > Good Morning John,
> > > > > > >
> > > > > > > I have my ERA5 data and my IMS data. MET is still
unable
> to
> > > > read
> > > > > my
> > > > > > > IMS data. The message I get is
> > > > > > > DEBUG 1: Observation File:
> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
> > > > > > imc_oct-01-2017.nc
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &,
> DataPlane
> > &)
> > > > > const
> > > > > > > -> data was flipped to north.
> > > > > > > ERROR :
> > > > > > > ERROR : NcCfFile::getData(NcVar *, const LongArray &,
> DataPlane
> > &)
> > > > > const
> > > > > > > -> star found in bad slot
> > > > > > > ERROR :
> > > > > > > I have no idea about this error. I am sorry to need so
much
> > help.
> > > > > > >
> > > > > > > A header dump of my IMS Netcdf file is
> > > > > > > dimensions:
> > > > > > > time = UNLIMITED ; // (1 currently)
> > > > > > > ny = 6144 ;
> > > > > > > nx = 6144 ;
> > > > > > > variables:
> > > > > > > double time(time) ;
> > > > > > > time:calendar = "standard" ;
> > > > > > > time:units = "hours since 1900-01-01 00:00:00" ;
> > > > > > > time:long_name = "time" ;
> > > > > > >
> > > > > > > float lat(ny, nx) ;
> > > > > > > lat:units = "degrees_north" ;
> > > > > > > lat:long_name = "latitude" ;
> > > > > > > lat:forecast_time = 0 ;
> > > > > > > lat:forecast_time_units = "hours" ;
> > > > > > > lat:initial_time = "10/01/2017 (00:00)" ;
> > > > > > > lat:_FillValue = 9.96921e+36f ;
> > > > > > >
> > > > > > > float lon(ny, nx) ;
> > > > > > > lon:units = "degrees_east" ;
> > > > > > > lon:long_name = "longitude" ;
> > > > > > > lon:forecast_time = 0 ;
> > > > > > > lon:forecast_time_units = "hours" ;
> > > > > > > lon:initial_time = "10/01/2017 (00:00)" ;
> > > > > > > lon:_FillValue = 9.96921e+36f ;
> > > > > > >
> > > > > > > float snowC(ny, nx) ;
> > > > > > > snowC:level = "Z0" ;
> > > > > > > snowC:units = "%" ;
> > > > > > > snowC:long_name = "Snow Cover" ;
> > > > > > > snowC:forecast_time = 0 ;
> > > > > > > snowC:forecast_time_units = "hours" ;
> > > > > > > snowC:initial_time = "10/01/2017 (00:00)" ;
> > > > > > > snowC:_FillValue = 9.96921e+36f ;
> > > > > > >
> > > > > > > // global attributes:
> > > > > > > :lon_pin = 145.f ;
> > > > > > > :lat_pin = -21.49193f ;
> > > > > > > :ny = "6144" ;
> > > > > > > :nx = "6144" ;
> > > > > > > :r_km = "6371.2 km" ;
> > > > > > > :d_km = "4.0km" ;
> > > > > > > :lon_orient = "-80.0" ;
> > > > > > > :y_pin = "0.000000" ;
> > > > > > > :x_pin = "0.000000" ;
> > > > > > > :scale_lat = "60.0 degrees_north" ;
> > > > > > > :hemisphere = "N" ;
> > > > > > > :projection = "Polar Stereographic" ;
> > > > > > > :MET_version = "V8.1" ;
> > > > > > >
> > > > > > > I am working on Hera in the directory
> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
> > > > > > >
> > > > > > > My Netcdf IMS files are in
> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
> > > > > > > Thanks,
> > > > > > > Sherrie
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway via
RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > >> Sherrie,
> > > > > > > >>
> > > > > > > >> My intention was to have you use the Python embedding
option
> > to
> > > > read
> > > > > > > this
> > > > > > > >> data, but it looks like you're reformatting it into
NetCDF
> > > > instead.
> > > > > > And
> > > > > > > >> that's fine too.
> > > > > > > >>
> > > > > > > >> But there's some issues based on the NetCDF header
you sent.
> > > > > > > >>
> > > > > > > >> (1) Those projection attributes should be defined as
global
> > > > > > attributes,
> > > > > > > >> not
> > > > > > > >> separately for each variable.
> > > > > > > >> (2) You are formatting a NetCDF file to follow the
> conventions
> > > > used
> > > > > > for
> > > > > > > >> the
> > > > > > > >> NetCDF output generated by the MET tools. If you add
the
> > global
> > > > > > > attribute
> > > > > > > >> "MET_version = "V8.0";" then MET will correctly
process that
> > > file
> > > > as
> > > > > > > being
> > > > > > > >> the NetCDF output of another MET tool.
> > > > > > > >> (3) You'll need to add timing information as
attributes for
> > each
> > > > > data
> > > > > > > >> variable.
> > > > > > > >>
> > > > > > > >> By way of example, I've attached a header dump of
some
> NetCDF
> > > > output
> > > > > > > from
> > > > > > > >> MET.
> > > > > > > >>
> > > > > > > >> Ultimately, whether you reformat the data into NetCDF
for
> MET
> > to
> > > > > read
> > > > > > or
> > > > > > > >> pass it via python embedding, you're still defining
the same
> > > > info...
> > > > > > the
> > > > > > > >> grid spec and timing information. The downside of
> > reformatting
> > > to
> > > > > > > NetCDF
> > > > > > > >> is that pre-processing step and duplicating the data
in
> ascii
> > > and
> > > > > > NetCDF
> > > > > > > >> formats. But either way can get the job done.
> > > > > > > >>
> > > > > > > >> If you're stuck, feel free to send me a file with
your work
> in
> > > > > > progress,
> > > > > > > >> and I can edit it to make MET happy.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> John
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> dimensions:
> > > > > > > >> lat = 154 ;
> > > > > > > >> lon = 169 ;
> > > > > > > >> variables:
> > > > > > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24"
;
> > > > > > > >> APCP_24_A24_ENS_MEAN:long_name =
"APCP_24 at
> > A24
> > > > > > Ensemble
> > > > > > > >> Mean" ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2"
;
> > > > > > > >> APCP_24_A24_ENS_MEAN:_FillValue =
-9999.f ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:init_time =
> > > "20091231_120000"
> > > > ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
> > "1262260800"
> > > ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:valid_time =
> > > > "20100101_120000" ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
> > > "1262347200" ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:accum_time =
"240000" ;
> > > > > > > >> APCP_24_A24_ENS_MEAN:accum_time_sec =
86400 ;
> > > > > > > >> // global attributes:
> > > > > > > >> :FileOrigins = "File
> > > > > > > >>
../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
> > > > generated
> > > > > > > >> 20191023_165124 UTC on host number5 by t
> > > > > > > >> he MET ensemble_stat tool" ;
> > > > > > > >> :MET_version = "V9.0" ;
> > > > > > > >> :MET_tool = "ensemble_stat" ;
> > > > > > > >> :model = "WRF" ;
> > > > > > > >> :obtype = "MC_PCP" ;
> > > > > > > >> :Projection = "Polar Stereographic" ;
> > > > > > > >> :hemisphere = "N" ;
> > > > > > > >> :scale_lat = "60.000000 degrees_north"
;
> > > > > > > >> :lat_pin = "31.357000" ;
> > > > > > > >> :lon_pin = "-129.770000" ;
> > > > > > > >> :x_pin = "0.000000" ;
> > > > > > > >> :y_pin = "0.000000" ;
> > > > > > > >> :lon_orient = "-120.500000" ;
> > > > > > > >> :d_km = "10.395000 km" ;
> > > > > > > >> :r_km = "6371.200000 km" ;
> > > > > > > >> :nx = "169" ;
> > > > > > > >> :ny = "154" ;
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick -
NOAA
> > > Affiliate
> > > > > via
> > > > > > > RT <
> > > > > > > >> met_help at ucar.edu> wrote:
> > > > > > > >>
> > > > > > > >> >
> > > > > > > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > >
> > > > > > > >> >
> > > > > > > >> > Hi John,
> > > > > > > >> >
> > > > > > > >> > Does plot_data_plane work on Netcdf files. I
have
> the
> > > > > > following
> > > > > > > >> in my
> > > > > > > >> > netcdf file.
> > > > > > > >> > The error I get is
> > > > > > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a
valid
> > data
> > > > > file
> > > > > > > >> > Do I need to tell plot_data_plane that I have a
netcdf
> file.
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Sherrie
> > > > > > > >> > dimensions:
> > > > > > > >> > ny = 6144 ;
> > > > > > > >> > nx = 6144 ;
> > > > > > > >> > variables:
> > > > > > > >> > float lat(ny, nx) ;
> > > > > > > >> > lat:long_name = "latitude" ;
> > > > > > > >> > lat:lon_pin = 145.f ;
> > > > > > > >> > lat:lat_pin = -21.49193f ;
> > > > > > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
> > > > > > > >> > lat:r_km = "6371.2 km" ;
> > > > > > > >> > lat:d_km = "4.0km" ;
> > > > > > > >> > lat:lon_orient = "-80.0" ;
> > > > > > > >> > lat:y_pin = "0.000000" ;
> > > > > > > >> > lat:x_pin = "0.000000" ;
> > > > > > > >> > lat:scale_lat = "60.0 degrees_north" ;
> > > > > > > >> > lat:hemisphere = "N" ;
> > > > > > > >> > lat:projection = "Polar Stereographic" ;
> > > > > > > >> > lat:_FillValue = 9.96921e+36f ;
> > > > > > > >> > float lon(ny, nx) ;
> > > > > > > >> > lon:long_name = "longitude" ;
> > > > > > > >> > lon:lon_pin = 145.f ;
> > > > > > > >> > lon:lat_pin = -21.49193f ;
> > > > > > > >> > lon:ny = "6144" ;
> > > > > > > >> > lon:nx = "6144" ;
> > > > > > > >> > lon:r_km = "6371.2 km" ;
> > > > > > > >> > lon:d_km = "4.0km" ;
> > > > > > > >> > lon:lon_orient = "-80.0" ;
> > > > > > > >> > lon:y_pin = "0.000000" ;
> > > > > > > >> > lon:x_pin = "0.000000" ;
> > > > > > > >> > lon:scale_lat = "60.0 degrees_north" ;
> > > > > > > >> > lon:hemisphere = "N" ;
> > > > > > > >> > lon:projection = "Polar Stereographic" ;
> > > > > > > >> > lon:_FillValue = 9.96921e+36f ;
> > > > > > > >> > float snowC(ny, nx) ;
> > > > > > > >> > snowC:units = "%" ;
> > > > > > > >> > snowC:long_name = "Snow Cover" ;
> > > > > > > >> > snowC:lon_pin = 145.f ;
> > > > > > > >> > snowC:lat_pin = -21.49193f ;
> > > > > > > >> > snowC:ny = "6144" ;
> > > > > > > >> > snowC:nx = "6144" ;
> > > > > > > >> > snowC:r_km = "6371.2 km" ;
> > > > > > > >> > snowC:d_km = "4.0km" ;
> > > > > > > >> > snowC:lon_orient = "-80.0" ;
> > > > > > > >> > snowC:y_pin = "0.000000" ;
> > > > > > > >> > snowC:x_pin = "0.000000" ;
> > > > > > > >> > snowC:scale_lat = "60.0 degrees_north" ;
> > > > > > > >> > snowC:hemisphere = "N" ;
> > > > > > > >> > snowC:projection = "Polar Stereographic" ;
> > > > > > > >> > snowC:_FillValue = 9.96921e+36f ;
> > > > > > > >> >
> > > > > > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway
via RT
> <
> > > > > > > >> > met_help at ucar.edu>
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> > > Sherrie,
> > > > > > > >> > >
> > > > > > > >> > > So we need to figure out the grid specification
for the
> > IMS
> > > > > polar
> > > > > > > >> > > stereographic grid on which your data resides.
> > > > > > > >> > >
> > > > > > > >> > > I did some hunting around online and suspect that
you're
> > > > > > processing
> > > > > > > >> this
> > > > > > > >> > > data:
> > > > > > > >> > > https://nsidc.org/data/G02156
> > > > > > > >> > >
> > > > > > > >> > > And the 4km resolution version of this data is on
a
> > > 6444x6444
> > > > > > polar
> > > > > > > >> > > stereographic grid.
> > > > > > > >> > >
> > > > > > > >> > > On the "User's Guide" tab of that page, table 4
lists
> some
> > > > > > relevant
> > > > > > > >> > > information:
> > > > > > > >> > > Latitude of true scale = 60 North
> > > > > > > >> > > Longitude below the pole = 80 West
> > > > > > > >> > >
> > > > > > > >> > > Based on that I'd say you want to try using the
> following
> > > > > > settings:
> > > > > > > >> > >
> > > > > > > >> > > :Projection = "Polar
Stereographic" ;
> > > > > > > >> > > :hemisphere = "N" ;
> > > > > > > >> > > :scale_lat = "60.0 degrees_north"
;
> > > > > > > >> > > :lat_pin = "???" ;
> > > > > > > >> > > :lon_pin = "???" ;
> > > > > > > >> > > :x_pin = "0.000000" ;
> > > > > > > >> > > :y_pin = "0.000000" ;
> > > > > > > >> > > :lon_orient = "-80.0" ;
> > > > > > > >> > > :d_km = "4.0 km" ;
> > > > > > > >> > > :r_km = "6371.2 km" ;
> > > > > > > >> > > :nx = "6144" ;
> > > > > > > >> > > :ny = "6144" ;
> > > > > > > >> > >
> > > > > > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin
and
> lon_pin
> > > > > values
> > > > > > > you
> > > > > > > >> > > specify are for the "lower-left" corner of the
grid.
> Look
> > > at
> > > > > the
> > > > > > > >> lat/lon
> > > > > > > >> > > values you have and try just specifying that
lat/lon of
> > the
> > > > > first
> > > > > > > >> point.
> > > > > > > >> > > Presumably, that'd be one of the 4 corners. And
then
> run
> > > > > > > >> plot_data_plane
> > > > > > > >> > > to see what the image looks like. If it differs
from
> your
> > > NCL
> > > > > > plot,
> > > > > > > >> we
> > > > > > > >> > can
> > > > > > > >> > > go from there.
> > > > > > > >> > >
> > > > > > > >> > > Thanks,
> > > > > > > >> > > John
> > > > > > > >> > >
> > > > > > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie Fredrick
- NOAA
> > > > > Affiliate
> > > > > > > via
> > > > > > > >> > RT <
> > > > > > > >> > > met_help at ucar.edu> wrote:
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > >
> > > > > > > >> > > >
> > > > > > > >> > > > Good Morning John,
> > > > > > > >> > > >
> > > > > > > >> > > > I have a couple more questions at your
> convenience,
> > > > > > > >> > > > I took the IMS ascii data and the lat/lon
data and
> > > > wrote a
> > > > > > > >> > > > Netcdf file that has both the IMS data and the
lat/lon
> > > data
> > > > in
> > > > > > > >> > > > it.dimensions:
> > > > > > > >> > > > ny = 6144 ;
> > > > > > > >> > > > nx = 6144 ;
> > > > > > > >> > > > variables:
> > > > > > > >> > > > float lat(ny, nx) ;
> > > > > > > >> > > > lat:_FillValue = 9.96921e+36f ;
> > > > > > > >> > > > float lon(ny, nx) ;
> > > > > > > >> > > > lon:_FillValue = 9.96921e+36f ;
> > > > > > > >> > > > float snowC(ny, nx) ;
> > > > > > > >> > > > snowC:_FillValue = 9.96921e+36f ;
> > > > > > > >> > > > }
> > > > > > > >> > > >
> > > > > > > >> > > > I am not sure I understand the all of the
metadata
> > above.
> > > > > > > >> > > > I am not sure of what the following should be
or how I
> > > > should
> > > > > > get
> > > > > > > it
> > > > > > > >> > > >
> > > > > > > >> > > > :scale_lat = "? degrees_north" ;
> > > > > > > >> > > > :lat_pin = ?;
> > > > > > > >> > > > :lon_pin = ?;
> > > > > > > >> > > > :x_pin = ? ;
> > > > > > > >> > > > :y_pin = "0.000000" ;
> > > > > > > >> > > > :lon_orient = ? ;
> > > > > > > >> > > > The following I think is correct for the IMS
polar
> > > > > > stereographic
> > > > > > > >> grid
> > > > > > > >> > > > :d_km = "4km" ;
> > > > > > > >> > > > :r_km = "6378.137km" ;
> > > > > > > >> > > > Since it is a 4km grid I believe I need to
use a
> > radius
> > > of
> > > > > > > >> 6378.137
> > > > > > > >> > > > according to the documentation.
> > > > > > > >> > > > The IMS grid is the WGS-84 ellipse.
> > > > > > > >> > > >
> > > > > > > >> > > > Also my forecast is going to at 1kmX1km and
the IMS
> > > grid
> > > > > is a
> > > > > > > >> 4km.
> > > > > > > >> > > Do I
> > > > > > > >> > > > need to interpolate the IMS grid to a 1km grid?
> > > > > > > >> > > >
> > > > > > > >> > > > Sorry for all the questions.
> > > > > > > >> > > > Many thanks for your help.
> > > > > > > >> > > > Sherrie
> > > > > > > >> > > >
> > > > > > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley
Gotway via
> > RT
> > > <
> > > > > > > >> > > > met_help at ucar.edu>
> > > > > > > >> > > > wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > Sherrie,
> > > > > > > >> > > > >
> > > > > > > >> > > > > OK, great. FYI, unfortunately, the name of
the
> > required
> > > > > > entries
> > > > > > > >> vary
> > > > > > > >> > > > > depending on the projection type and we
haven't
> done a
> > > > good
> > > > > > job
> > > > > > > >> > > > documenting
> > > > > > > >> > > > > all of the variations yet. However, those
entries
> are
> > > the
> > > > > > same
> > > > > > > as
> > > > > > > >> > the
> > > > > > > >> > > > > global attributes you'll find in the NetCDF
output
> > from
> > > > MET
> > > > > > > tools.
> > > > > > > >> > > > >
> > > > > > > >> > > > > Here's an example of those attributes for a
Polar
> > > > > > Stereographic
> > > > > > > >> > > > projection:
> > > > > > > >> > > > >
> > > > > > > >> > > > > :Projection = "Polar
Stereographic" ;
> > > > > > > >> > > > > :hemisphere = "N" ;
> > > > > > > >> > > > > :scale_lat = "60.000000
> degrees_north"
> > ;
> > > > > > > >> > > > > :lat_pin = "31.357000" ;
> > > > > > > >> > > > > :lon_pin = "-129.770000" ;
> > > > > > > >> > > > > :x_pin = "0.000000" ;
> > > > > > > >> > > > > :y_pin = "0.000000" ;
> > > > > > > >> > > > > :lon_orient = "-120.500000" ;
> > > > > > > >> > > > > :d_km = "10.395000 km" ;
> > > > > > > >> > > > > :r_km = "6371.200000 km" ;
> > > > > > > >> > > > > :nx = "169" ;
> > > > > > > >> > > > > :ny = "154" ;
> > > > > > > >> > > > >
> > > > > > > >> > > > > Hopefully that'll help.
> > > > > > > >> > > > >
> > > > > > > >> > > > > Thanks,
> > > > > > > >> > > > > John
> > > > > > > >> > > > >
> > > > > > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie
Fredrick -
> > NOAA
> > > > > > > Affiliate
> > > > > > > >> via
> > > > > > > >> > > RT
> > > > > > > >> > > > <
> > > > > > > >> > > > > met_help at ucar.edu> wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Hi John,
> > > > > > > >> > > > > > Thank you so much! I will give it a try.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > cheers,
> > > > > > > >> > > > > > Sherrie
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John Halley
Gotway
> > via
> > > > RT
> > > > > <
> > > > > > > >> > > > > > met_help at ucar.edu>
> > > > > > > >> > > > > > wrote:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > > Hello Sherrie,
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Yes this is possible, but you'll need to
use the
> > > > python
> > > > > > > >> embedding
> > > > > > > >> > > > > feature
> > > > > > > >> > > > > > > of MET. Specifically, we'll need a
python
> script
> > > > which
> > > > > > > reads
> > > > > > > >> > data
> > > > > > > >> > > > from
> > > > > > > >> > > > > > > that ASCII file and defines the metadata
for it
> > > (e.g.
> > > > > > timing
> > > > > > > >> and
> > > > > > > >> > > grid
> > > > > > > >> > > > > > > info). This is a relatively new feature
in MET,
> > and
> > > > > we're
> > > > > > > >> > revising
> > > > > > > >> > > > the
> > > > > > > >> > > > > > > logic for it for met-9.0, but reading
data from
> > > ASCII
> > > > > > should
> > > > > > > >> be
> > > > > > > >> > > very
> > > > > > > >> > > > > > > straight-forward. I'd expect this to
work well.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Examples of using python embedding to
read
> gridded
> > > > data
> > > > > > into
> > > > > > > >> MET
> > > > > > > >> > > can
> > > > > > > >> > > > be
> > > > > > > >> > > > > > > found here:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > The last example on that page reads data
from
> > ASCII,
> > > > so
> > > > > > that
> > > > > > > >> > might
> > > > > > > >> > > > be a
> > > > > > > >> > > > > > > good place to start:
> > > > > > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
> > > > > > > >> > 'name="read_AWS_CMORPH.py
> > > > > > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I highly recommend starting with the
> > plot_data_plane
> > > > > tool
> > > > > > > >> since
> > > > > > > >> > you
> > > > > > > >> > > > can
> > > > > > > >> > > > > > > compare the output plot to your NCL
output to
> make
> > > > sure
> > > > > > > you're
> > > > > > > >> > > > passing
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > > data to MET in the correct orientation
with the
> > > > correct
> > > > > > grid
> > > > > > > >> > info.
> > > > > > > >> > > > If
> > > > > > > >> > > > > > > you're comfortable with python, feel free
to
> adapt
> > > > that
> > > > > > > >> script to
> > > > > > > >> > > > read
> > > > > > > >> > > > > > your
> > > > > > > >> > > > > > > data. If you need help with this, please
send
> us
> > a
> > > > > sample
> > > > > > > >> data
> > > > > > > >> > > file,
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > > lat/lon data file, and an NCL plot of
that data.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Thanks,
> > > > > > > >> > > > > > > John
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie
> Fredrick
> > -
> > > > NOAA
> > > > > > > >> > Affiliate
> > > > > > > >> > > > via
> > > > > > > >> > > > > > RT <
> > > > > > > >> > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request 92748
was
> > acted
> > > > > upon.
> > > > > > > >> > > > > > > > Transaction: Ticket created by
> > > > > > sherrie.fredrick at noaa.gov
> > > > > > > >> > > > > > > > Queue: met_help
> > > > > > > >> > > > > > > > Subject: using MET with IMS snow
data
> > > > > > > >> > > > > > > > Owner: Nobody
> > > > > > > >> > > > > > > > Requestors: sherrie.fredrick at noaa.gov
> > > > > > > >> > > > > > > > Status: new
> > > > > > > >> > > > > > > > Ticket <URL:
> > > > > > > >> > > > >
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > Good Morning,
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > I would love to use MET/MET Plus
to
> verify
> > > the
> > > > > > > >> > NOAA/NESDIS
> > > > > > > >> > > > > > > > Interactive Multisensor Snow and Ice
Mapping
> > > System
> > > > > > (IMS)
> > > > > > > >> data
> > > > > > > >> > > with
> > > > > > > >> > > > > > ERA5
> > > > > > > >> > > > > > > > data. The IMS data is on a Polar
> Stereographic
> > > 4km
> > > > > grid
> > > > > > > the
> > > > > > > >> > data
> > > > > > > >> > > > > > format
> > > > > > > >> > > > > > > is
> > > > > > > >> > > > > > > > ASCII. The latitude and longitude
definitions
> > are
> > > > in
> > > > > a
> > > > > > > >> > separate
> > > > > > > >> > > > > > file. I
> > > > > > > >> > > > > > > > can read in the data and plot with NCL.
The
> > ERA5
> > > > data
> > > > > > > that
> > > > > > > >> I
> > > > > > > >> > am
> > > > > > > >> > > > > using
> > > > > > > >> > > > > > is
> > > > > > > >> > > > > > > > the ERA5 land data. It is in grib
format. I
> > can
> > > > > > choose
> > > > > > > >> the
> > > > > > > >> > > > > > resolution
> > > > > > > >> > > > > > > > when down loading the data. Is this
possible?
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > I will be working on Hera.
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > I hope this makes sense,
> > > > > > > >> > > > > > > > Many Thanks,
> > > > > > > >> > > > > > > > Sherrie
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: using MET with IMS snow data
From: John Halley Gotway
Time: Wed Oct 30 17:06:59 2019
Sherrie,
FYI - I added this python embedding example of NSIDC IMS data to the
MET
website:
https://dtcenter.org/community-code/model-evaluation-tools-met/sample-
analysis-scripts
And I'll resolve this issue now.
Thanks,
John
On Tue, Oct 29, 2019 at 9:52 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Sherrie,
>
> Great, glad that's helpful. I'll go ahead and post this to the MET
> website as another example. Ideally, the script would handle all 3
> resolutions (1km, 4km, and 24km) but something's better than
nothing.
>
> Thanks,
> John
>
> On Mon, Oct 28, 2019 at 7:53 PM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>>
>> Hi John,
>> That will save a bunch of time. I also have other data that
are
>> ascii
>> from a different source. I am sure you
>> script will help with that also.
>>
>> Many Thanks,
>> Sherrie
>>
>> On Mon, Oct 28, 2019 at 5:15 PM John Halley Gotway via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> > Sherrie,
>> >
>> > Just to show you an alternative, I worked up an example of using
python
>> > embedding to read this 4km data.
>> >
>> > Here's an example of calling plot_data_plane using python
embedding to
>> read
>> > the ASCII data directly:
>> >
>> > plot_data_plane PYTHON_NUMPY ims.ps 'name="read_NSIDC_IMS_4km.py
>> > ims2017010_4km_v1.3.asc";'
>> >
>> > And that produces the same image we got before: ims.png
>> >
>> > I've attached the python script I used. The advantage here is
not
>> needing
>> > to write intermediate NetCDF files and being able to "fix" issues
that
>> > arise directly in the python script.
>> >
>> > Thanks,
>> > John
>> >
>> > On Mon, Oct 28, 2019 at 1:59 PM Sherrie Fredrick - NOAA Affiliate
via
>> RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>> > >
>> > > Hi John,
>> > >
>> > > Yea! Thanks very much for all your help. I will make
the
>> changes
>> > > you suggested.
>> > > Sherrie
>> > >
>> > > On Mon, Oct 28, 2019 at 1:50 PM John Halley Gotway via RT <
>> > > met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > > Sherrie,
>> > > >
>> > > > Great, thanks. I ran plot_data_plane on the ims NetCDF and
got the
>> > same
>> > > > error you did. Here are the issues I found and how to fix
them.
>> > > >
>> > > > (1) Rename ny/nx dimension to lat/lon:
>> > > > Looking at that NetCDF I see that you're formatting it as if
it
>> were a
>> > > > NetCDF file created by another MET tool. The library in MET
that
>> reads
>> > > > this data assumes the dimensions are named lat and lon.
>> > > > * ncrename -d ny,lat -d nx,lon ims_oct-01-2017.nc
>> > > > <http://ims_oct-01-2017.nc> -o ims_oct-01-2017_MOD.nc*
>> > > >
>> > > > (2) The "projection" global attribute should be capitalized
as
>> > > > "Projection":
>> > > > * ncatted -a projection,global,d,c,'' -a
>> > Projection,global,a,c,"Polar
>> > > > Stereographic" ims_oct-01-2017_MOD.nc*
>> > > >
>> > > > (3) I changed to reference point to be the north pole: (x, y)
=
>> (3072,
>> > > > 3072) is the north pole at (lat, lon) = (90, 0):
>> > > > * ncatted -a lon_pin,global,o,c,"0" -a
lat_pin,global,o,c,"90" -a
>> > > > x_pin,global,o,c,"3072" -a y_pin,global,o,c,"3072"
>> > > ims_oct-01-2017_MOD.nc*
>> > > >
>> > > > After all of these changes, I ran plot_data_plane to produce
the
>> > attached
>> > > > plot.
>> > > > *plot_data_plane ims_oct-01-2017_MOD.nc ims.ps
<http://ims.ps>
>> > > > 'name="snowC"; level="(*,*)";'*
>> > > >
>> > > > And the data lines up very well with the map data!
>> > > >
>> > > > However, issues with the timing info remain. Your snowC data
>> variable
>> > > > doesn't define the timing info in the way expected by the
NetCDF
>> output
>> > > > from MET. It should contain variable attributes, like this:
>> > > > APCP_12:init_time = "20050807_000000" ;
>> > > >
>> > > > APCP_12:init_time_ut = "1123372800" ;
>> > > >
>> > > > APCP_12:valid_time = "20050807_120000" ;
>> > > >
>> > > > APCP_12:valid_time_ut = "1123416000" ;
>> > > >
>> > > > APCP_12:accum_time = "120000" ;
>> > > >
>> > > > APCP_12:accum_time_sec = 43200 ;
>> > > > Technically, I believe the code is parsing the timing info
from the
>> > > > "init_time_ut" and "valid_time_ut" attributes, where UT is
unixtime,
>> > the
>> > > > number of seconds since Jan 1, 1970.
>> > > >
>> > > > Here's how you can use the unix date command to convert
between
>> October
>> > > 1,
>> > > > 2017 and unixtime:
>> > > >
>> > > > *date -ud ''2019-10-01' UTC '00:00:00'' +%s 1569888000*
>> > > >
>> > > > And then going back the other way:
>> > > > *date -ud '1970-01-01 UTC '1569888000' seconds' +%Y%m%d%H*
>> > > > *2019100100*
>> > > >
>> > > > Thanks,
>> > > > John
>> > > >
>> > > >
>> > > > On Mon, Oct 28, 2019 at 12:33 PM Sherrie Fredrick - NOAA
Affiliate
>> via
>> > > RT <
>> > > > met_help at ucar.edu> wrote:
>> > > >
>> > > > >
>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748 >
>> > > > >
>> > > > > Hi John,
>> > > > >
>> > > > > I have placed the following files on
>> .../met_help/sherrie_data.
>> > > > > grid_stat_config
>> > > > > ims2017274_4km_v1.3.asc - This is the original
ascii file
>> > > > > downloaded from the IMS web site.
>> > > > >
Northern
>> > > > > Hemisphere, Polar stereographic, 4kmX4km grid resolution.
>> > > > > imslat_4km.bin - The binary
latitude
>> file
>> > > > from
>> > > > > the IMS web site
>> > > > > imslon_4km.bin - The binary
longitude
>> file
>> > > > from
>> > > > > the IMS web site
>> > > > > ims_oct-01-2017.nc - The Netcdf file
that I
>> > > > produced.
>> > > > > oct201701.nc - The ERA5
netcdf file
>> > from
>> > > > the
>> > > > > ERA5-Land data. Downloaded from Copernicus.
>> > > > > I
>> downloaded
>> > > the
>> > > > > ERA5 data at a 4kmX4km grid resolution.
>> > > > > I included the original files just in case. I think
there
>> will
>> > > be a
>> > > > > lot of interest in using MET to verify model data. My
marching
>> > orders
>> > > > are
>> > > > > to use only MET for verification. I will be running the
>> FV3workflow
>> > to
>> > > > get
>> > > > > FV3 model data. Right now we are not ready to do that so
>> > > > > we want to use ERA5 data with the IMS data.
>> > > > > Your help is much appreciated,
>> > > > > Sherrie
>> > > > >
>> > > > > On Mon, Oct 28, 2019 at 11:43 AM John Halley Gotway via RT
<
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > > > Hi Sherrie,
>> > > > > >
>> > > > > > I just tried logging on to Hera but am having problems...
so I
>> > wrote
>> > > > for
>> > > > > > help from Hera's helpdesk.
>> > > > > >
>> > > > > > In the meantime, is there any chance you could post
sample
>> files to
>> > > our
>> > > > > > anonymous ftp site? I'd need the latest iteration of the
ERA5
>> and
>> > > IMS
>> > > > > > files and your Grid-Stat config file. Here are
instructions:
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>> > > > > >
>> > > > > > Thanks,
>> > > > > > John
>> > > > > >
>> > > > > > On Mon, Oct 28, 2019 at 10:30 AM Sherrie Fredrick - NOAA
>> Affiliate
>> > > via
>> > > > > RT <
>> > > > > > met_help at ucar.edu> wrote:
>> > > > > >
>> > > > > > >
>> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> >
>> > > > > > >
>> > > > > > > Good Morning John,
>> > > > > > >
>> > > > > > > I have my ERA5 data and my IMS data. MET is still
>> unable to
>> > > > read
>> > > > > my
>> > > > > > > IMS data. The message I get is
>> > > > > > > DEBUG 1: Observation File:
>> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims/
>> > > > > > imc_oct-01-2017.nc
>> > > > > > > DEBUG 2:
>> > > > > > > DEBUG 2: NcCfFile::getData(NcVar *, const LongArray &,
>> DataPlane
>> > &)
>> > > > > const
>> > > > > > > -> data was flipped to north.
>> > > > > > > ERROR :
>> > > > > > > ERROR : NcCfFile::getData(NcVar *, const LongArray &,
>> DataPlane
>> > &)
>> > > > > const
>> > > > > > > -> star found in bad slot
>> > > > > > > ERROR :
>> > > > > > > I have no idea about this error. I am sorry to need so
much
>> > help.
>> > > > > > >
>> > > > > > > A header dump of my IMS Netcdf file is
>> > > > > > > dimensions:
>> > > > > > > time = UNLIMITED ; // (1 currently)
>> > > > > > > ny = 6144 ;
>> > > > > > > nx = 6144 ;
>> > > > > > > variables:
>> > > > > > > double time(time) ;
>> > > > > > > time:calendar = "standard" ;
>> > > > > > > time:units = "hours since 1900-01-01 00:00:00" ;
>> > > > > > > time:long_name = "time" ;
>> > > > > > >
>> > > > > > > float lat(ny, nx) ;
>> > > > > > > lat:units = "degrees_north" ;
>> > > > > > > lat:long_name = "latitude" ;
>> > > > > > > lat:forecast_time = 0 ;
>> > > > > > > lat:forecast_time_units = "hours" ;
>> > > > > > > lat:initial_time = "10/01/2017 (00:00)" ;
>> > > > > > > lat:_FillValue = 9.96921e+36f ;
>> > > > > > >
>> > > > > > > float lon(ny, nx) ;
>> > > > > > > lon:units = "degrees_east" ;
>> > > > > > > lon:long_name = "longitude" ;
>> > > > > > > lon:forecast_time = 0 ;
>> > > > > > > lon:forecast_time_units = "hours" ;
>> > > > > > > lon:initial_time = "10/01/2017 (00:00)" ;
>> > > > > > > lon:_FillValue = 9.96921e+36f ;
>> > > > > > >
>> > > > > > > float snowC(ny, nx) ;
>> > > > > > > snowC:level = "Z0" ;
>> > > > > > > snowC:units = "%" ;
>> > > > > > > snowC:long_name = "Snow Cover" ;
>> > > > > > > snowC:forecast_time = 0 ;
>> > > > > > > snowC:forecast_time_units = "hours" ;
>> > > > > > > snowC:initial_time = "10/01/2017 (00:00)" ;
>> > > > > > > snowC:_FillValue = 9.96921e+36f ;
>> > > > > > >
>> > > > > > > // global attributes:
>> > > > > > > :lon_pin = 145.f ;
>> > > > > > > :lat_pin = -21.49193f ;
>> > > > > > > :ny = "6144" ;
>> > > > > > > :nx = "6144" ;
>> > > > > > > :r_km = "6371.2 km" ;
>> > > > > > > :d_km = "4.0km" ;
>> > > > > > > :lon_orient = "-80.0" ;
>> > > > > > > :y_pin = "0.000000" ;
>> > > > > > > :x_pin = "0.000000" ;
>> > > > > > > :scale_lat = "60.0 degrees_north" ;
>> > > > > > > :hemisphere = "N" ;
>> > > > > > > :projection = "Polar Stereographic" ;
>> > > > > > > :MET_version = "V8.1" ;
>> > > > > > >
>> > > > > > > I am working on Hera in the directory
>> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/MET
>> > > > > > >
>> > > > > > > My Netcdf IMS files are in
>> > > > > > > /scratch2/BMC/refcst2/Sherrie.Fredrick/FV3workflow/ims
>> > > > > > > Thanks,
>> > > > > > > Sherrie
>> > > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > On Wed, Oct 23, 2019 at 4:04 PM John Halley Gotway
via RT <
>> > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > >
>> > > > > > > >> Sherrie,
>> > > > > > > >>
>> > > > > > > >> My intention was to have you use the Python
embedding
>> option
>> > to
>> > > > read
>> > > > > > > this
>> > > > > > > >> data, but it looks like you're reformatting it into
NetCDF
>> > > > instead.
>> > > > > > And
>> > > > > > > >> that's fine too.
>> > > > > > > >>
>> > > > > > > >> But there's some issues based on the NetCDF header
you
>> sent.
>> > > > > > > >>
>> > > > > > > >> (1) Those projection attributes should be defined as
global
>> > > > > > attributes,
>> > > > > > > >> not
>> > > > > > > >> separately for each variable.
>> > > > > > > >> (2) You are formatting a NetCDF file to follow the
>> conventions
>> > > > used
>> > > > > > for
>> > > > > > > >> the
>> > > > > > > >> NetCDF output generated by the MET tools. If you
add the
>> > global
>> > > > > > > attribute
>> > > > > > > >> "MET_version = "V8.0";" then MET will correctly
process
>> that
>> > > file
>> > > > as
>> > > > > > > being
>> > > > > > > >> the NetCDF output of another MET tool.
>> > > > > > > >> (3) You'll need to add timing information as
attributes for
>> > each
>> > > > > data
>> > > > > > > >> variable.
>> > > > > > > >>
>> > > > > > > >> By way of example, I've attached a header dump of
some
>> NetCDF
>> > > > output
>> > > > > > > from
>> > > > > > > >> MET.
>> > > > > > > >>
>> > > > > > > >> Ultimately, whether you reformat the data into
NetCDF for
>> MET
>> > to
>> > > > > read
>> > > > > > or
>> > > > > > > >> pass it via python embedding, you're still defining
the
>> same
>> > > > info...
>> > > > > > the
>> > > > > > > >> grid spec and timing information. The downside of
>> > reformatting
>> > > to
>> > > > > > > NetCDF
>> > > > > > > >> is that pre-processing step and duplicating the data
in
>> ascii
>> > > and
>> > > > > > NetCDF
>> > > > > > > >> formats. But either way can get the job done.
>> > > > > > > >>
>> > > > > > > >> If you're stuck, feel free to send me a file with
your
>> work in
>> > > > > > progress,
>> > > > > > > >> and I can edit it to make MET happy.
>> > > > > > > >>
>> > > > > > > >> Thanks,
>> > > > > > > >> John
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> dimensions:
>> > > > > > > >> lat = 154 ;
>> > > > > > > >> lon = 169 ;
>> > > > > > > >> variables:
>> > > > > > > >> float APCP_24_A24_ENS_MEAN(lat, lon) ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:name = "APCP_24"
;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:long_name =
"APCP_24 at
>> > A24
>> > > > > > Ensemble
>> > > > > > > >> Mean" ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:level = "A24" ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:units = "kg/m^2"
;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:_FillValue =
-9999.f ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:init_time =
>> > > "20091231_120000"
>> > > > ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:init_time_ut =
>> > "1262260800"
>> > > ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:valid_time =
>> > > > "20100101_120000" ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:valid_time_ut =
>> > > "1262347200" ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:accum_time =
"240000" ;
>> > > > > > > >> APCP_24_A24_ENS_MEAN:accum_time_sec =
86400
>> ;
>> > > > > > > >> // global attributes:
>> > > > > > > >> :FileOrigins = "File
>> > > > > > > >>
../out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
>> > > > generated
>> > > > > > > >> 20191023_165124 UTC on host number5 by t
>> > > > > > > >> he MET ensemble_stat tool" ;
>> > > > > > > >> :MET_version = "V9.0" ;
>> > > > > > > >> :MET_tool = "ensemble_stat" ;
>> > > > > > > >> :model = "WRF" ;
>> > > > > > > >> :obtype = "MC_PCP" ;
>> > > > > > > >> :Projection = "Polar Stereographic" ;
>> > > > > > > >> :hemisphere = "N" ;
>> > > > > > > >> :scale_lat = "60.000000
degrees_north" ;
>> > > > > > > >> :lat_pin = "31.357000" ;
>> > > > > > > >> :lon_pin = "-129.770000" ;
>> > > > > > > >> :x_pin = "0.000000" ;
>> > > > > > > >> :y_pin = "0.000000" ;
>> > > > > > > >> :lon_orient = "-120.500000" ;
>> > > > > > > >> :d_km = "10.395000 km" ;
>> > > > > > > >> :r_km = "6371.200000 km" ;
>> > > > > > > >> :nx = "169" ;
>> > > > > > > >> :ny = "154" ;
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Wed, Oct 23, 2019 at 2:37 PM Sherrie Fredrick -
NOAA
>> > > Affiliate
>> > > > > via
>> > > > > > > RT <
>> > > > > > > >> met_help at ucar.edu> wrote:
>> > > > > > > >>
>> > > > > > > >> >
>> > > > > > > >> > <URL:
>> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> > > >
>> > > > > > > >> >
>> > > > > > > >> > Hi John,
>> > > > > > > >> >
>> > > > > > > >> > Does plot_data_plane work on Netcdf files. I
have
>> the
>> > > > > > following
>> > > > > > > >> in my
>> > > > > > > >> > netcdf file.
>> > > > > > > >> > The error I get is
>> > > > > > > >> > plot_data_plane -> file "imc_oct-01-2017.nc" not a
valid
>> > data
>> > > > > file
>> > > > > > > >> > Do I need to tell plot_data_plane that I have a
netcdf
>> file.
>> > > > > > > >> >
>> > > > > > > >> > Thanks,
>> > > > > > > >> > Sherrie
>> > > > > > > >> > dimensions:
>> > > > > > > >> > ny = 6144 ;
>> > > > > > > >> > nx = 6144 ;
>> > > > > > > >> > variables:
>> > > > > > > >> > float lat(ny, nx) ;
>> > > > > > > >> > lat:long_name = "latitude" ;
>> > > > > > > >> > lat:lon_pin = 145.f ;
>> > > > > > > >> > lat:lat_pin = -21.49193f ;
>> > > > > > > >> > lat:ny = "6144" ;lat:nx = "6144" ;
>> > > > > > > >> > lat:r_km = "6371.2 km" ;
>> > > > > > > >> > lat:d_km = "4.0km" ;
>> > > > > > > >> > lat:lon_orient = "-80.0" ;
>> > > > > > > >> > lat:y_pin = "0.000000" ;
>> > > > > > > >> > lat:x_pin = "0.000000" ;
>> > > > > > > >> > lat:scale_lat = "60.0 degrees_north" ;
>> > > > > > > >> > lat:hemisphere = "N" ;
>> > > > > > > >> > lat:projection = "Polar Stereographic" ;
>> > > > > > > >> > lat:_FillValue = 9.96921e+36f ;
>> > > > > > > >> > float lon(ny, nx) ;
>> > > > > > > >> > lon:long_name = "longitude" ;
>> > > > > > > >> > lon:lon_pin = 145.f ;
>> > > > > > > >> > lon:lat_pin = -21.49193f ;
>> > > > > > > >> > lon:ny = "6144" ;
>> > > > > > > >> > lon:nx = "6144" ;
>> > > > > > > >> > lon:r_km = "6371.2 km" ;
>> > > > > > > >> > lon:d_km = "4.0km" ;
>> > > > > > > >> > lon:lon_orient = "-80.0" ;
>> > > > > > > >> > lon:y_pin = "0.000000" ;
>> > > > > > > >> > lon:x_pin = "0.000000" ;
>> > > > > > > >> > lon:scale_lat = "60.0 degrees_north" ;
>> > > > > > > >> > lon:hemisphere = "N" ;
>> > > > > > > >> > lon:projection = "Polar Stereographic" ;
>> > > > > > > >> > lon:_FillValue = 9.96921e+36f ;
>> > > > > > > >> > float snowC(ny, nx) ;
>> > > > > > > >> > snowC:units = "%" ;
>> > > > > > > >> > snowC:long_name = "Snow Cover" ;
>> > > > > > > >> > snowC:lon_pin = 145.f ;
>> > > > > > > >> > snowC:lat_pin = -21.49193f ;
>> > > > > > > >> > snowC:ny = "6144" ;
>> > > > > > > >> > snowC:nx = "6144" ;
>> > > > > > > >> > snowC:r_km = "6371.2 km" ;
>> > > > > > > >> > snowC:d_km = "4.0km" ;
>> > > > > > > >> > snowC:lon_orient = "-80.0" ;
>> > > > > > > >> > snowC:y_pin = "0.000000" ;
>> > > > > > > >> > snowC:x_pin = "0.000000" ;
>> > > > > > > >> > snowC:scale_lat = "60.0 degrees_north" ;
>> > > > > > > >> > snowC:hemisphere = "N" ;
>> > > > > > > >> > snowC:projection = "Polar Stereographic" ;
>> > > > > > > >> > snowC:_FillValue = 9.96921e+36f ;
>> > > > > > > >> >
>> > > > > > > >> > On Wed, Oct 23, 2019 at 1:27 PM John Halley Gotway
via
>> RT <
>> > > > > > > >> > met_help at ucar.edu>
>> > > > > > > >> > wrote:
>> > > > > > > >> >
>> > > > > > > >> > > Sherrie,
>> > > > > > > >> > >
>> > > > > > > >> > > So we need to figure out the grid specification
for the
>> > IMS
>> > > > > polar
>> > > > > > > >> > > stereographic grid on which your data resides.
>> > > > > > > >> > >
>> > > > > > > >> > > I did some hunting around online and suspect
that
>> you're
>> > > > > > processing
>> > > > > > > >> this
>> > > > > > > >> > > data:
>> > > > > > > >> > > https://nsidc.org/data/G02156
>> > > > > > > >> > >
>> > > > > > > >> > > And the 4km resolution version of this data is
on a
>> > > 6444x6444
>> > > > > > polar
>> > > > > > > >> > > stereographic grid.
>> > > > > > > >> > >
>> > > > > > > >> > > On the "User's Guide" tab of that page, table 4
lists
>> some
>> > > > > > relevant
>> > > > > > > >> > > information:
>> > > > > > > >> > > Latitude of true scale = 60 North
>> > > > > > > >> > > Longitude below the pole = 80 West
>> > > > > > > >> > >
>> > > > > > > >> > > Based on that I'd say you want to try using the
>> following
>> > > > > > settings:
>> > > > > > > >> > >
>> > > > > > > >> > > :Projection = "Polar
Stereographic" ;
>> > > > > > > >> > > :hemisphere = "N" ;
>> > > > > > > >> > > :scale_lat = "60.0 degrees_north"
;
>> > > > > > > >> > > :lat_pin = "???" ;
>> > > > > > > >> > > :lon_pin = "???" ;
>> > > > > > > >> > > :x_pin = "0.000000" ;
>> > > > > > > >> > > :y_pin = "0.000000" ;
>> > > > > > > >> > > :lon_orient = "-80.0" ;
>> > > > > > > >> > > :d_km = "4.0 km" ;
>> > > > > > > >> > > :r_km = "6371.2 km" ;
>> > > > > > > >> > > :nx = "6144" ;
>> > > > > > > >> > > :ny = "6144" ;
>> > > > > > > >> > >
>> > > > > > > >> > > x_pin = y_pin = 0.0 indicates that the lat_pin
and
>> lon_pin
>> > > > > values
>> > > > > > > you
>> > > > > > > >> > > specify are for the "lower-left" corner of the
grid.
>> Look
>> > > at
>> > > > > the
>> > > > > > > >> lat/lon
>> > > > > > > >> > > values you have and try just specifying that
lat/lon of
>> > the
>> > > > > first
>> > > > > > > >> point.
>> > > > > > > >> > > Presumably, that'd be one of the 4 corners. And
then
>> run
>> > > > > > > >> plot_data_plane
>> > > > > > > >> > > to see what the image looks like. If it differs
from
>> your
>> > > NCL
>> > > > > > plot,
>> > > > > > > >> we
>> > > > > > > >> > can
>> > > > > > > >> > > go from there.
>> > > > > > > >> > >
>> > > > > > > >> > > Thanks,
>> > > > > > > >> > > John
>> > > > > > > >> > >
>> > > > > > > >> > > On Wed, Oct 23, 2019 at 11:06 AM Sherrie
Fredrick -
>> NOAA
>> > > > > Affiliate
>> > > > > > > via
>> > > > > > > >> > RT <
>> > > > > > > >> > > met_help at ucar.edu> wrote:
>> > > > > > > >> > >
>> > > > > > > >> > > >
>> > > > > > > >> > > > <URL:
>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> > > > > >
>> > > > > > > >> > > >
>> > > > > > > >> > > > Good Morning John,
>> > > > > > > >> > > >
>> > > > > > > >> > > > I have a couple more questions at your
>> convenience,
>> > > > > > > >> > > > I took the IMS ascii data and the lat/lon
data
>> and
>> > > > wrote a
>> > > > > > > >> > > > Netcdf file that has both the IMS data and the
>> lat/lon
>> > > data
>> > > > in
>> > > > > > > >> > > > it.dimensions:
>> > > > > > > >> > > > ny = 6144 ;
>> > > > > > > >> > > > nx = 6144 ;
>> > > > > > > >> > > > variables:
>> > > > > > > >> > > > float lat(ny, nx) ;
>> > > > > > > >> > > > lat:_FillValue = 9.96921e+36f ;
>> > > > > > > >> > > > float lon(ny, nx) ;
>> > > > > > > >> > > > lon:_FillValue = 9.96921e+36f ;
>> > > > > > > >> > > > float snowC(ny, nx) ;
>> > > > > > > >> > > > snowC:_FillValue = 9.96921e+36f ;
>> > > > > > > >> > > > }
>> > > > > > > >> > > >
>> > > > > > > >> > > > I am not sure I understand the all of the
metadata
>> > above.
>> > > > > > > >> > > > I am not sure of what the following should be
or how
>> I
>> > > > should
>> > > > > > get
>> > > > > > > it
>> > > > > > > >> > > >
>> > > > > > > >> > > > :scale_lat = "? degrees_north"
;
>> > > > > > > >> > > > :lat_pin = ?;
>> > > > > > > >> > > > :lon_pin = ?;
>> > > > > > > >> > > > :x_pin = ? ;
>> > > > > > > >> > > > :y_pin = "0.000000" ;
>> > > > > > > >> > > > :lon_orient = ? ;
>> > > > > > > >> > > > The following I think is correct for the IMS
polar
>> > > > > > stereographic
>> > > > > > > >> grid
>> > > > > > > >> > > > :d_km = "4km" ;
>> > > > > > > >> > > > :r_km = "6378.137km" ;
>> > > > > > > >> > > > Since it is a 4km grid I believe I need to
use a
>> > radius
>> > > of
>> > > > > > > >> 6378.137
>> > > > > > > >> > > > according to the documentation.
>> > > > > > > >> > > > The IMS grid is the WGS-84 ellipse.
>> > > > > > > >> > > >
>> > > > > > > >> > > > Also my forecast is going to at 1kmX1km and
the
>> IMS
>> > > grid
>> > > > > is a
>> > > > > > > >> 4km.
>> > > > > > > >> > > Do I
>> > > > > > > >> > > > need to interpolate the IMS grid to a 1km
grid?
>> > > > > > > >> > > >
>> > > > > > > >> > > > Sorry for all the questions.
>> > > > > > > >> > > > Many thanks for your help.
>> > > > > > > >> > > > Sherrie
>> > > > > > > >> > > >
>> > > > > > > >> > > > On Tue, Oct 22, 2019 at 5:15 PM John Halley
Gotway
>> via
>> > RT
>> > > <
>> > > > > > > >> > > > met_help at ucar.edu>
>> > > > > > > >> > > > wrote:
>> > > > > > > >> > > >
>> > > > > > > >> > > > > Sherrie,
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > OK, great. FYI, unfortunately, the name of
the
>> > required
>> > > > > > entries
>> > > > > > > >> vary
>> > > > > > > >> > > > > depending on the projection type and we
haven't
>> done a
>> > > > good
>> > > > > > job
>> > > > > > > >> > > > documenting
>> > > > > > > >> > > > > all of the variations yet. However, those
entries
>> are
>> > > the
>> > > > > > same
>> > > > > > > as
>> > > > > > > >> > the
>> > > > > > > >> > > > > global attributes you'll find in the NetCDF
output
>> > from
>> > > > MET
>> > > > > > > tools.
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > Here's an example of those attributes for a
Polar
>> > > > > > Stereographic
>> > > > > > > >> > > > projection:
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > :Projection = "Polar
Stereographic"
>> ;
>> > > > > > > >> > > > > :hemisphere = "N" ;
>> > > > > > > >> > > > > :scale_lat = "60.000000
>> degrees_north"
>> > ;
>> > > > > > > >> > > > > :lat_pin = "31.357000" ;
>> > > > > > > >> > > > > :lon_pin = "-129.770000" ;
>> > > > > > > >> > > > > :x_pin = "0.000000" ;
>> > > > > > > >> > > > > :y_pin = "0.000000" ;
>> > > > > > > >> > > > > :lon_orient = "-120.500000" ;
>> > > > > > > >> > > > > :d_km = "10.395000 km" ;
>> > > > > > > >> > > > > :r_km = "6371.200000 km" ;
>> > > > > > > >> > > > > :nx = "169" ;
>> > > > > > > >> > > > > :ny = "154" ;
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > Hopefully that'll help.
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > Thanks,
>> > > > > > > >> > > > > John
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > On Tue, Oct 22, 2019 at 3:19 PM Sherrie
Fredrick -
>> > NOAA
>> > > > > > > Affiliate
>> > > > > > > >> via
>> > > > > > > >> > > RT
>> > > > > > > >> > > > <
>> > > > > > > >> > > > > met_help at ucar.edu> wrote:
>> > > > > > > >> > > > >
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > > <URL:
>> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> > > > > > > >
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > > Hi John,
>> > > > > > > >> > > > > > Thank you so much! I will give it a try.
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > > cheers,
>> > > > > > > >> > > > > > Sherrie
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > > On Tue, Oct 22, 2019 at 1:57 PM John
Halley
>> Gotway
>> > via
>> > > > RT
>> > > > > <
>> > > > > > > >> > > > > > met_help at ucar.edu>
>> > > > > > > >> > > > > > wrote:
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > > > Hello Sherrie,
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > Yes this is possible, but you'll need to
use
>> the
>> > > > python
>> > > > > > > >> embedding
>> > > > > > > >> > > > > feature
>> > > > > > > >> > > > > > > of MET. Specifically, we'll need a
python
>> script
>> > > > which
>> > > > > > > reads
>> > > > > > > >> > data
>> > > > > > > >> > > > from
>> > > > > > > >> > > > > > > that ASCII file and defines the metadata
for it
>> > > (e.g.
>> > > > > > timing
>> > > > > > > >> and
>> > > > > > > >> > > grid
>> > > > > > > >> > > > > > > info). This is a relatively new feature
in
>> MET,
>> > and
>> > > > > we're
>> > > > > > > >> > revising
>> > > > > > > >> > > > the
>> > > > > > > >> > > > > > > logic for it for met-9.0, but reading
data from
>> > > ASCII
>> > > > > > should
>> > > > > > > >> be
>> > > > > > > >> > > very
>> > > > > > > >> > > > > > > straight-forward. I'd expect this to
work
>> well.
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > Examples of using python embedding to
read
>> gridded
>> > > > data
>> > > > > > into
>> > > > > > > >> MET
>> > > > > > > >> > > can
>> > > > > > > >> > > > be
>> > > > > > > >> > > > > > > found here:
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > >
>> > > > > > > >> > > >
>> > > > > > > >> > >
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://dtcenter.org/community-code/model-evaluation-tools-
met/sample-analysis-scripts
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > The last example on that page reads data
from
>> > ASCII,
>> > > > so
>> > > > > > that
>> > > > > > > >> > might
>> > > > > > > >> > > > be a
>> > > > > > > >> > > > > > > good place to start:
>> > > > > > > >> > > > > > > plot_data_plane PYTHON_NUMPY crain.ps
>> > > > > > > >> > 'name="read_AWS_CMORPH.py
>> > > > > > > >> > > > > > > AWS_CMORPH_crain-2017043006.txt";
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > I highly recommend starting with the
>> > plot_data_plane
>> > > > > tool
>> > > > > > > >> since
>> > > > > > > >> > you
>> > > > > > > >> > > > can
>> > > > > > > >> > > > > > > compare the output plot to your NCL
output to
>> make
>> > > > sure
>> > > > > > > you're
>> > > > > > > >> > > > passing
>> > > > > > > >> > > > > > the
>> > > > > > > >> > > > > > > data to MET in the correct orientation
with the
>> > > > correct
>> > > > > > grid
>> > > > > > > >> > info.
>> > > > > > > >> > > > If
>> > > > > > > >> > > > > > > you're comfortable with python, feel
free to
>> adapt
>> > > > that
>> > > > > > > >> script to
>> > > > > > > >> > > > read
>> > > > > > > >> > > > > > your
>> > > > > > > >> > > > > > > data. If you need help with this,
please send
>> us
>> > a
>> > > > > sample
>> > > > > > > >> data
>> > > > > > > >> > > file,
>> > > > > > > >> > > > > the
>> > > > > > > >> > > > > > > lat/lon data file, and an NCL plot of
that
>> data.
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > Thanks,
>> > > > > > > >> > > > > > > John
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > On Tue, Oct 22, 2019 at 11:23 AM Sherrie
>> Fredrick
>> > -
>> > > > NOAA
>> > > > > > > >> > Affiliate
>> > > > > > > >> > > > via
>> > > > > > > >> > > > > > RT <
>> > > > > > > >> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > > Tue Oct 22 11:23:31 2019: Request
92748 was
>> > acted
>> > > > > upon.
>> > > > > > > >> > > > > > > > Transaction: Ticket created by
>> > > > > > sherrie.fredrick at noaa.gov
>> > > > > > > >> > > > > > > > Queue: met_help
>> > > > > > > >> > > > > > > > Subject: using MET with IMS snow
data
>> > > > > > > >> > > > > > > > Owner: Nobody
>> > > > > > > >> > > > > > > > Requestors:
sherrie.fredrick at noaa.gov
>> > > > > > > >> > > > > > > > Status: new
>> > > > > > > >> > > > > > > > Ticket <URL:
>> > > > > > > >> > > > >
>> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92748
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > > Good Morning,
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > > I would love to use MET/MET Plus
to
>> verify
>> > > the
>> > > > > > > >> > NOAA/NESDIS
>> > > > > > > >> > > > > > > > Interactive Multisensor Snow and Ice
Mapping
>> > > System
>> > > > > > (IMS)
>> > > > > > > >> data
>> > > > > > > >> > > with
>> > > > > > > >> > > > > > ERA5
>> > > > > > > >> > > > > > > > data. The IMS data is on a Polar
>> Stereographic
>> > > 4km
>> > > > > grid
>> > > > > > > the
>> > > > > > > >> > data
>> > > > > > > >> > > > > > format
>> > > > > > > >> > > > > > > is
>> > > > > > > >> > > > > > > > ASCII. The latitude and longitude
>> definitions
>> > are
>> > > > in
>> > > > > a
>> > > > > > > >> > separate
>> > > > > > > >> > > > > > file. I
>> > > > > > > >> > > > > > > > can read in the data and plot with
NCL. The
>> > ERA5
>> > > > data
>> > > > > > > that
>> > > > > > > >> I
>> > > > > > > >> > am
>> > > > > > > >> > > > > using
>> > > > > > > >> > > > > > is
>> > > > > > > >> > > > > > > > the ERA5 land data. It is in grib
format.
>> I
>> > can
>> > > > > > choose
>> > > > > > > >> the
>> > > > > > > >> > > > > > resolution
>> > > > > > > >> > > > > > > > when down loading the data. Is this
possible?
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > > I will be working on Hera.
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > > I hope this makes sense,
>> > > > > > > >> > > > > > > > Many Thanks,
>> > > > > > > >> > > > > > > > Sherrie
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > > >
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > > >
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > > >
>> > > > > > > >> > > > >
>> > > > > > > >> > > > >
>> > > > > > > >> > > >
>> > > > > > > >> > > >
>> > > > > > > >> > >
>> > > > > > > >> > >
>> > > > > > > >> >
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
------------------------------------------------
More information about the Met_help
mailing list