[Met_help] [rt.rap.ucar.edu #74975] History for Trying MODE out and having issues
John Halley Gotway via RT
met_help at ucar.edu
Wed Feb 17 13:31:07 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
met_help,
I'm trying to run MODE on my data and am having issues. Could to help me figure out the error? I did the tutorial for MODE with success but am having problems with my own data (NetCDF data). My files are located here
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0
The file structure is the same as in the tutorial, but instead of calling it tutorial I call it gfs_test. So everything is under gfs_test. I have not opened up my directory fully yet because I think you should be able to cp the config file over to your directory and look at it. However, if you can't even do that let me know and I'll open my folders up.
Here is the command I use to run MODE and the error I get when I run the code.
Theia08:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>$METPATH/bin/mode data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc data/gfs_test/NR2006.pl.1by1.7278.2006022818.nc gfs_test/config/MODEConfig_gfstest -outdir gfs_test/out/mode/ -v 2
DEBUG 1: Default Config File: /scratch4/BMC/dtc/MET/met-5.0/share/met/config/MODEConfig_default
DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
NetCDF: Attribute not found
Theia08:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
So the error I'm getting is NetCDF: Attribute not found. Am I doing something wrong with how I'm specifying the variable name or level? I saw some notes in the manual about how NetCDF is specified differently in the config files sometimes but it was confusing.
Thanks!
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Wed Feb 03 15:25:32 2016
Hi Tanya,
I took a look at the gridded NetCDF data you're trying to pass to
MODE. Unfortunately, MET doesn't support all types of gridded NetCDF
files. Instead, it only supports the MET NetCDF format (e.g. the
output of the pcp_combine tool), the output of the WRF pinterp (or
wrfinterp) utilities, and CF-compliant NetCDF files.
Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I see that it
follows none of those conventions. However, it is closest to CF-
compliant NetCDF.
When using a new data file format, instead of running it through MODE,
I'd suggest using plot_data_plane to make sure MET is reading it
correctly. I ran met-5.1/plot_data_plane as follows:
/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
The resulting image is attached. There are several issues here:
- I had to tell plot_data_plane to interpret this as CF-compliant
NetCDF using NETCDF_NCCF.
- The timing info in that file isn't specified in the CF-compliant
way... so MET can't discern the initialization, valid, and lead times.
- And then there's the obvious problem (in the image) of the world
being upside down and backwards!
So while we did get *something*, there are obviously a lot of issues.
Basically, MET doesn't support the gridded NetCDF file format you are
using. Is this data possibly available in GRIB? Or do you have
control over the NetCDF file format in use?
Thanks,
John
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Wed Feb 03 15:53:59 2016
John,
We generated the NetCDF files ourselves so we could probably make it
work
for MET. Do you know how to get it into CF-compliant NetCDF format?
Yes, we have grib files but the reason we created NetCDF files was
because
original the NR... file had accumulated precip over the whole forecast
where the other file has 6h accumulation data, so we need to convert
the
NR... precip information into 6h accumulation (they had different
units
too, so we need to change that too). Since we had to change the units
too I
think use generating our own NetCDF file is best but we just need to
get it
in the right format.
What is your opinion on the best way to approach this?
If we get it into the right format is how I specify the 'name' and
'level'
of the field correct? It was hard to figure out the correct notation
from
the manual.
Thanks,
Tanya
On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Hi Tanya,
>
> I took a look at the gridded NetCDF data you're trying to pass to
MODE.
> Unfortunately, MET doesn't support all types of gridded NetCDF
files.
> Instead, it only supports the MET NetCDF format (e.g. the output of
the
> pcp_combine tool), the output of the WRF pinterp (or wrfinterp)
utilities,
> and CF-compliant NetCDF files.
>
> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I see that
it
> follows none of those conventions. However, it is closest to CF-
compliant
> NetCDF.
>
> When using a new data file format, instead of running it through
MODE, I'd
> suggest using plot_data_plane to make sure MET is reading it
correctly. I
> ran met-5.1/plot_data_plane as follows:
>
> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
>
> The resulting image is attached. There are several issues here:
> - I had to tell plot_data_plane to interpret this as CF-compliant
NetCDF
> using NETCDF_NCCF.
> - The timing info in that file isn't specified in the CF-compliant
way...
> so MET can't discern the initialization, valid, and lead times.
> - And then there's the obvious problem (in the image) of the world
being
> upside down and backwards!
>
> So while we did get *something*, there are obviously a lot of
issues.
> Basically, MET doesn't support the gridded NetCDF file format you
are
> using. Is this data possibly available in GRIB? Or do you have
control
> over the NetCDF file format in use?
>
> Thanks,
> John
>
>
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Thu Feb 04 11:21:47 2016
John,
To clarify my previous request of information. I meant to ask was the
following: Do you know of any good documentation on how to make a
NetCDF
file CF-compliant? Also, do you have an example .nc file that is
NetCDF
CF-compliant that we could compare to?
Thanks,
Tanya
On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate <
tanya.peevey at noaa.gov> wrote:
> John,
>
> We generated the NetCDF files ourselves so we could probably make it
work
> for MET. Do you know how to get it into CF-compliant NetCDF format?
> Yes, we have grib files but the reason we created NetCDF files was
because
> original the NR... file had accumulated precip over the whole
forecast
> where the other file has 6h accumulation data, so we need to convert
the
> NR... precip information into 6h accumulation (they had different
units
> too, so we need to change that too). Since we had to change the
units too I
> think use generating our own NetCDF file is best but we just need to
get it
> in the right format.
> What is your opinion on the best way to approach this?
>
> If we get it into the right format is how I specify the 'name' and
'level'
> of the field correct? It was hard to figure out the correct notation
from
> the manual.
>
> Thanks,
> Tanya
>
>
>
> On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Hi Tanya,
>>
>> I took a look at the gridded NetCDF data you're trying to pass to
MODE.
>> Unfortunately, MET doesn't support all types of gridded NetCDF
files.
>> Instead, it only supports the MET NetCDF format (e.g. the output of
the
>> pcp_combine tool), the output of the WRF pinterp (or wrfinterp)
utilities,
>> and CF-compliant NetCDF files.
>>
>> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I see that
it
>> follows none of those conventions. However, it is closest to CF-
compliant
>> NetCDF.
>>
>> When using a new data file format, instead of running it through
MODE,
>> I'd suggest using plot_data_plane to make sure MET is reading it
>> correctly. I ran met-5.1/plot_data_plane as follows:
>>
>> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
>>
>> The resulting image is attached. There are several issues here:
>> - I had to tell plot_data_plane to interpret this as CF-compliant
NetCDF
>> using NETCDF_NCCF.
>> - The timing info in that file isn't specified in the CF-compliant
way...
>> so MET can't discern the initialization, valid, and lead times.
>> - And then there's the obvious problem (in the image) of the world
being
>> upside down and backwards!
>>
>> So while we did get *something*, there are obviously a lot of
issues.
>> Basically, MET doesn't support the gridded NetCDF file format you
are
>> using. Is this data possibly available in GRIB? Or do you have
control
>> over the NetCDF file format in use?
>>
>> Thanks,
>> John
>>
>>
>>
>>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Thu Feb 04 11:32:58 2016
Tanya,
Here's a website about it:
http://cfconventions.org/latest.html
And I posted a sample CF-compliant NetCDF file here:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
The relevant pieces in there are...
- The global "Conventions" attribute set to "CF-...".
- The time dimension/variable define the forecast valid time.
- The forecast_reference_time variable defines the model
initialization
time.
- The grid_mapping, x, and y variables define the grid definition
information.
- The grid_mapping variable attributes map those variables to the
relevant
grid definition.
Hope this helps.
Thanks,
John
On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> To clarify my previous request of information. I meant to ask was
the
> following: Do you know of any good documentation on how to make a
NetCDF
> file CF-compliant? Also, do you have an example .nc file that is
NetCDF
> CF-compliant that we could compare to?
>
> Thanks,
> Tanya
>
>
>
> On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate <
> tanya.peevey at noaa.gov> wrote:
>
> > John,
> >
> > We generated the NetCDF files ourselves so we could probably make
it work
> > for MET. Do you know how to get it into CF-compliant NetCDF
format?
> > Yes, we have grib files but the reason we created NetCDF files was
> because
> > original the NR... file had accumulated precip over the whole
forecast
> > where the other file has 6h accumulation data, so we need to
convert the
> > NR... precip information into 6h accumulation (they had different
units
> > too, so we need to change that too). Since we had to change the
units
> too I
> > think use generating our own NetCDF file is best but we just need
to get
> it
> > in the right format.
> > What is your opinion on the best way to approach this?
> >
> > If we get it into the right format is how I specify the 'name' and
> 'level'
> > of the field correct? It was hard to figure out the correct
notation from
> > the manual.
> >
> > Thanks,
> > Tanya
> >
> >
> >
> > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Hi Tanya,
> >>
> >> I took a look at the gridded NetCDF data you're trying to pass to
MODE.
> >> Unfortunately, MET doesn't support all types of gridded NetCDF
files.
> >> Instead, it only supports the MET NetCDF format (e.g. the output
of the
> >> pcp_combine tool), the output of the WRF pinterp (or wrfinterp)
> utilities,
> >> and CF-compliant NetCDF files.
> >>
> >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I see
that it
> >> follows none of those conventions. However, it is closest to
> CF-compliant
> >> NetCDF.
> >>
> >> When using a new data file format, instead of running it through
MODE,
> >> I'd suggest using plot_data_plane to make sure MET is reading it
> >> correctly. I ran met-5.1/plot_data_plane as follows:
> >>
> >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
> >>
> >> The resulting image is attached. There are several issues here:
> >> - I had to tell plot_data_plane to interpret this as CF-compliant
NetCDF
> >> using NETCDF_NCCF.
> >> - The timing info in that file isn't specified in the CF-
compliant
> way...
> >> so MET can't discern the initialization, valid, and lead times.
> >> - And then there's the obvious problem (in the image) of the
world being
> >> upside down and backwards!
> >>
> >> So while we did get *something*, there are obviously a lot of
issues.
> >> Basically, MET doesn't support the gridded NetCDF file format you
are
> >> using. Is this data possibly available in GRIB? Or do you have
control
> >> over the NetCDF file format in use?
> >>
> >> Thanks,
> >> John
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Thu Feb 04 15:07:03 2016
John,
Thanks for the info. We are going to work on changing our file format
and
then I'll get back to you.
Did you see my other question about how I specified 'name' and 'level'
for
the precip. field in my config file for MODE? Was how I did it ok? I
ask
because the manual was confusing on this point. If my syntax was
incorrect
could you please email me the correction.
Thanks,
Tanya
On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Tanya,
>
> Here's a website about it:
> http://cfconventions.org/latest.html
>
> And I posted a sample CF-compliant NetCDF file here:
>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
>
> The relevant pieces in there are...
> - The global "Conventions" attribute set to "CF-...".
> - The time dimension/variable define the forecast valid time.
> - The forecast_reference_time variable defines the model
initialization
> time.
> - The grid_mapping, x, and y variables define the grid definition
> information.
> - The grid_mapping variable attributes map those variables to the
relevant
> grid definition.
>
> Hope this helps.
>
> Thanks,
> John
>
>
> On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >
> > John,
> >
> > To clarify my previous request of information. I meant to ask was
the
> > following: Do you know of any good documentation on how to make a
NetCDF
> > file CF-compliant? Also, do you have an example .nc file that is
NetCDF
> > CF-compliant that we could compare to?
> >
> > Thanks,
> > Tanya
> >
> >
> >
> > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate <
> > tanya.peevey at noaa.gov> wrote:
> >
> > > John,
> > >
> > > We generated the NetCDF files ourselves so we could probably
make it
> work
> > > for MET. Do you know how to get it into CF-compliant NetCDF
format?
> > > Yes, we have grib files but the reason we created NetCDF files
was
> > because
> > > original the NR... file had accumulated precip over the whole
forecast
> > > where the other file has 6h accumulation data, so we need to
convert
> the
> > > NR... precip information into 6h accumulation (they had
different units
> > > too, so we need to change that too). Since we had to change the
units
> > too I
> > > think use generating our own NetCDF file is best but we just
need to
> get
> > it
> > > in the right format.
> > > What is your opinion on the best way to approach this?
> > >
> > > If we get it into the right format is how I specify the 'name'
and
> > 'level'
> > > of the field correct? It was hard to figure out the correct
notation
> from
> > > the manual.
> > >
> > > Thanks,
> > > Tanya
> > >
> > >
> > >
> > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Hi Tanya,
> > >>
> > >> I took a look at the gridded NetCDF data you're trying to pass
to
> MODE.
> > >> Unfortunately, MET doesn't support all types of gridded NetCDF
files.
> > >> Instead, it only supports the MET NetCDF format (e.g. the
output of
> the
> > >> pcp_combine tool), the output of the WRF pinterp (or wrfinterp)
> > utilities,
> > >> and CF-compliant NetCDF files.
> > >>
> > >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I see
that
> it
> > >> follows none of those conventions. However, it is closest to
> > CF-compliant
> > >> NetCDF.
> > >>
> > >> When using a new data file format, instead of running it
through MODE,
> > >> I'd suggest using plot_data_plane to make sure MET is reading
it
> > >> correctly. I ran met-5.1/plot_data_plane as follows:
> > >>
> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
> > >>
> > >> The resulting image is attached. There are several issues
here:
> > >> - I had to tell plot_data_plane to interpret this as CF-
compliant
> NetCDF
> > >> using NETCDF_NCCF.
> > >> - The timing info in that file isn't specified in the CF-
compliant
> > way...
> > >> so MET can't discern the initialization, valid, and lead times.
> > >> - And then there's the obvious problem (in the image) of the
world
> being
> > >> upside down and backwards!
> > >>
> > >> So while we did get *something*, there are obviously a lot of
issues.
> > >> Basically, MET doesn't support the gridded NetCDF file format
you are
> > >> using. Is this data possibly available in GRIB? Or do you
have
> control
> > >> over the NetCDF file format in use?
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> >
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Mon Feb 08 09:40:56 2016
Tanya,
Sorry for the delay. I was out of the office on Friday. Using
"LSP_6h"
for the name and "(*,*)" for the level should do the trick.
When trying to get the MET tools to read a new dataset, I always start
by
using the plot_data_plane tool:
/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
I use that to make sure the name and level settings work as expected,
and
it also creates an output plot so I can see that MET is (or is not)
reading/plotting the data as expected.
That configuration string you pass to the the plot_data_plane tool is
exactly the same settings as are supported in the configuration files.
In
fact, we read that string with the same code that parses the config
files.
I find using plot_data_plane very helpful.
Thanks,
John
On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> Thanks for the info. We are going to work on changing our file
format and
> then I'll get back to you.
> Did you see my other question about how I specified 'name' and
'level' for
> the precip. field in my config file for MODE? Was how I did it ok? I
ask
> because the manual was confusing on this point. If my syntax was
incorrect
> could you please email me the correction.
>
> Thanks,
> Tanya
>
>
>
> On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Tanya,
> >
> > Here's a website about it:
> > http://cfconventions.org/latest.html
> >
> > And I posted a sample CF-compliant NetCDF file here:
> >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> >
> > The relevant pieces in there are...
> > - The global "Conventions" attribute set to "CF-...".
> > - The time dimension/variable define the forecast valid time.
> > - The forecast_reference_time variable defines the model
initialization
> > time.
> > - The grid_mapping, x, and y variables define the grid definition
> > information.
> > - The grid_mapping variable attributes map those variables to the
> relevant
> > grid definition.
> >
> > Hope this helps.
> >
> > Thanks,
> > John
> >
> >
> > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >
> > > John,
> > >
> > > To clarify my previous request of information. I meant to ask
was the
> > > following: Do you know of any good documentation on how to make
a
> NetCDF
> > > file CF-compliant? Also, do you have an example .nc file that is
NetCDF
> > > CF-compliant that we could compare to?
> > >
> > > Thanks,
> > > Tanya
> > >
> > >
> > >
> > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate <
> > > tanya.peevey at noaa.gov> wrote:
> > >
> > > > John,
> > > >
> > > > We generated the NetCDF files ourselves so we could probably
make it
> > work
> > > > for MET. Do you know how to get it into CF-compliant NetCDF
format?
> > > > Yes, we have grib files but the reason we created NetCDF files
was
> > > because
> > > > original the NR... file had accumulated precip over the whole
> forecast
> > > > where the other file has 6h accumulation data, so we need to
convert
> > the
> > > > NR... precip information into 6h accumulation (they had
different
> units
> > > > too, so we need to change that too). Since we had to change
the units
> > > too I
> > > > think use generating our own NetCDF file is best but we just
need to
> > get
> > > it
> > > > in the right format.
> > > > What is your opinion on the best way to approach this?
> > > >
> > > > If we get it into the right format is how I specify the 'name'
and
> > > 'level'
> > > > of the field correct? It was hard to figure out the correct
notation
> > from
> > > > the manual.
> > > >
> > > > Thanks,
> > > > Tanya
> > > >
> > > >
> > > >
> > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Hi Tanya,
> > > >>
> > > >> I took a look at the gridded NetCDF data you're trying to
pass to
> > MODE.
> > > >> Unfortunately, MET doesn't support all types of gridded
NetCDF
> files.
> > > >> Instead, it only supports the MET NetCDF format (e.g. the
output of
> > the
> > > >> pcp_combine tool), the output of the WRF pinterp (or
wrfinterp)
> > > utilities,
> > > >> and CF-compliant NetCDF files.
> > > >>
> > > >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I
see that
> > it
> > > >> follows none of those conventions. However, it is closest to
> > > CF-compliant
> > > >> NetCDF.
> > > >>
> > > >> When using a new data file format, instead of running it
through
> MODE,
> > > >> I'd suggest using plot_data_plane to make sure MET is reading
it
> > > >> correctly. I ran met-5.1/plot_data_plane as follows:
> > > >>
> > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
> > > >>
> > > >> The resulting image is attached. There are several issues
here:
> > > >> - I had to tell plot_data_plane to interpret this as CF-
compliant
> > NetCDF
> > > >> using NETCDF_NCCF.
> > > >> - The timing info in that file isn't specified in the CF-
compliant
> > > way...
> > > >> so MET can't discern the initialization, valid, and lead
times.
> > > >> - And then there's the obvious problem (in the image) of the
world
> > being
> > > >> upside down and backwards!
> > > >>
> > > >> So while we did get *something*, there are obviously a lot of
> issues.
> > > >> Basically, MET doesn't support the gridded NetCDF file format
you
> are
> > > >> using. Is this data possibly available in GRIB? Or do you
have
> > control
> > > >> over the NetCDF file format in use?
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > Tanya R. Peevey, PhD
> > > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > > NOAA ESRL Global Systems Division
> > > > 325 Broadway, Boulder, CO 80305
> > > > (303) 497-5847
> > > >
> > >
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> > >
> >
> >
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Mon Feb 08 16:33:30 2016
John,
Thank you for the information. We generated a new file and I tried the
plot_data_plane tool on it and got the following error. Do you have an
idea
of what could be wrong? Sorry to bug you so much and thanks in advance
for
the help.
Here's the location of the file:
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
Here's the error:
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
data/gfs_test/
lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
DEBUG 1: Opening data file: data/gfs_test/
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
ERROR :
ERROR : StringArray::operator[](int) const -> range check error!
ERROR :
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
Thank you ,
Tanya
On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Tanya,
>
> Sorry for the delay. I was out of the office on Friday. Using
"LSP_6h"
> for the name and "(*,*)" for the level should do the trick.
>
> When trying to get the MET tools to read a new dataset, I always
start by
> using the plot_data_plane tool:
>
> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
>
> I use that to make sure the name and level settings work as
expected, and
> it also creates an output plot so I can see that MET is (or is not)
> reading/plotting the data as expected.
>
> That configuration string you pass to the the plot_data_plane tool
is
> exactly the same settings as are supported in the configuration
files. In
> fact, we read that string with the same code that parses the config
files.
>
> I find using plot_data_plane very helpful.
>
> Thanks,
> John
>
>
> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >
> > John,
> >
> > Thanks for the info. We are going to work on changing our file
format and
> > then I'll get back to you.
> > Did you see my other question about how I specified 'name' and
'level'
> for
> > the precip. field in my config file for MODE? Was how I did it ok?
I ask
> > because the manual was confusing on this point. If my syntax was
> incorrect
> > could you please email me the correction.
> >
> > Thanks,
> > Tanya
> >
> >
> >
> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Tanya,
> > >
> > > Here's a website about it:
> > > http://cfconventions.org/latest.html
> > >
> > > And I posted a sample CF-compliant NetCDF file here:
> > >
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > >
> > > The relevant pieces in there are...
> > > - The global "Conventions" attribute set to "CF-...".
> > > - The time dimension/variable define the forecast valid time.
> > > - The forecast_reference_time variable defines the model
initialization
> > > time.
> > > - The grid_mapping, x, and y variables define the grid
definition
> > > information.
> > > - The grid_mapping variable attributes map those variables to
the
> > relevant
> > > grid definition.
> > >
> > > Hope this helps.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> > > >
> > > > John,
> > > >
> > > > To clarify my previous request of information. I meant to ask
was the
> > > > following: Do you know of any good documentation on how to
make a
> > NetCDF
> > > > file CF-compliant? Also, do you have an example .nc file that
is
> NetCDF
> > > > CF-compliant that we could compare to?
> > > >
> > > > Thanks,
> > > > Tanya
> > > >
> > > >
> > > >
> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate
<
> > > > tanya.peevey at noaa.gov> wrote:
> > > >
> > > > > John,
> > > > >
> > > > > We generated the NetCDF files ourselves so we could probably
make
> it
> > > work
> > > > > for MET. Do you know how to get it into CF-compliant NetCDF
format?
> > > > > Yes, we have grib files but the reason we created NetCDF
files was
> > > > because
> > > > > original the NR... file had accumulated precip over the
whole
> > forecast
> > > > > where the other file has 6h accumulation data, so we need to
> convert
> > > the
> > > > > NR... precip information into 6h accumulation (they had
different
> > units
> > > > > too, so we need to change that too). Since we had to change
the
> units
> > > > too I
> > > > > think use generating our own NetCDF file is best but we just
need
> to
> > > get
> > > > it
> > > > > in the right format.
> > > > > What is your opinion on the best way to approach this?
> > > > >
> > > > > If we get it into the right format is how I specify the
'name' and
> > > > 'level'
> > > > > of the field correct? It was hard to figure out the correct
> notation
> > > from
> > > > > the manual.
> > > > >
> > > > > Thanks,
> > > > > Tanya
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Hi Tanya,
> > > > >>
> > > > >> I took a look at the gridded NetCDF data you're trying to
pass to
> > > MODE.
> > > > >> Unfortunately, MET doesn't support all types of gridded
NetCDF
> > files.
> > > > >> Instead, it only supports the MET NetCDF format (e.g. the
output
> of
> > > the
> > > > >> pcp_combine tool), the output of the WRF pinterp (or
wrfinterp)
> > > > utilities,
> > > > >> and CF-compliant NetCDF files.
> > > > >>
> > > > >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I
see
> that
> > > it
> > > > >> follows none of those conventions. However, it is closest
to
> > > > CF-compliant
> > > > >> NetCDF.
> > > > >>
> > > > >> When using a new data file format, instead of running it
through
> > MODE,
> > > > >> I'd suggest using plot_data_plane to make sure MET is
reading it
> > > > >> correctly. I ran met-5.1/plot_data_plane as follows:
> > > > >>
> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
> > > > >>
> > > > >> The resulting image is attached. There are several issues
here:
> > > > >> - I had to tell plot_data_plane to interpret this as CF-
compliant
> > > NetCDF
> > > > >> using NETCDF_NCCF.
> > > > >> - The timing info in that file isn't specified in the CF-
compliant
> > > > way...
> > > > >> so MET can't discern the initialization, valid, and lead
times.
> > > > >> - And then there's the obvious problem (in the image) of
the world
> > > being
> > > > >> upside down and backwards!
> > > > >>
> > > > >> So while we did get *something*, there are obviously a lot
of
> > issues.
> > > > >> Basically, MET doesn't support the gridded NetCDF file
format you
> > are
> > > > >> using. Is this data possibly available in GRIB? Or do you
have
> > > control
> > > > >> over the NetCDF file format in use?
> > > > >>
> > > > >> Thanks,
> > > > >> John
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Tanya R. Peevey, PhD
> > > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> > > > > NOAA ESRL Global Systems Division
> > > > > 325 Broadway, Boulder, CO 80305
> > > > > (303) 497-5847
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Tanya R. Peevey, PhD
> > > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > > NOAA ESRL Global Systems Division
> > > > 325 Broadway, Boulder, CO 80305
> > > > (303) 497-5847
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Mon Feb 08 16:46:37 2016
John,
My colleague, Jason English (cc'd on email), that is generating the
NETCF_NCCF files has another question about the CF format.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Do we need the grid_mapping x and y variables if we are using the
lambert
cylindrical equal area grid?
If so, how do we convert 1-dimensional lat/lon coordinates to the 2d
coordinates with y,x?
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thank you,
Tanya Peevey
On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate <
tanya.peevey at noaa.gov> wrote:
> John,
>
> Thank you for the information. We generated a new file and I tried
the
> plot_data_plane tool on it and got the following error. Do you have
an idea
> of what could be wrong? Sorry to bug you so much and thanks in
advance for
> the help.
>
> Here's the location of the file:
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>
> Here's the error:
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> file_type=NETCDF_NCCF;'
> DEBUG 1: Opening data file: data/gfs_test/
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> ERROR :
> ERROR : StringArray::operator[](int) const -> range check error!
> ERROR :
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
>
> Thank you ,
> Tanya
>
>
>
> On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Tanya,
>>
>> Sorry for the delay. I was out of the office on Friday. Using
"LSP_6h"
>> for the name and "(*,*)" for the level should do the trick.
>>
>> When trying to get the MET tools to read a new dataset, I always
start by
>> using the plot_data_plane tool:
>>
>> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> lsp_6h.ps 'name="LSP_6h"; level="(*,*)"; file_type=NETCDF_NCCF;'
>>
>> I use that to make sure the name and level settings work as
expected, and
>> it also creates an output plot so I can see that MET is (or is not)
>> reading/plotting the data as expected.
>>
>> That configuration string you pass to the the plot_data_plane tool
is
>> exactly the same settings as are supported in the configuration
files. In
>> fact, we read that string with the same code that parses the config
files.
>>
>> I find using plot_data_plane very helpful.
>>
>> Thanks,
>> John
>>
>>
>> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate via
RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> >
>> > John,
>> >
>> > Thanks for the info. We are going to work on changing our file
format
>> and
>> > then I'll get back to you.
>> > Did you see my other question about how I specified 'name' and
'level'
>> for
>> > the precip. field in my config file for MODE? Was how I did it
ok? I ask
>> > because the manual was confusing on this point. If my syntax was
>> incorrect
>> > could you please email me the correction.
>> >
>> > Thanks,
>> > Tanya
>> >
>> >
>> >
>> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > > Tanya,
>> > >
>> > > Here's a website about it:
>> > > http://cfconventions.org/latest.html
>> > >
>> > > And I posted a sample CF-compliant NetCDF file here:
>> > >
>> > >
>> >
>>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
>> > >
>> > > The relevant pieces in there are...
>> > > - The global "Conventions" attribute set to "CF-...".
>> > > - The time dimension/variable define the forecast valid time.
>> > > - The forecast_reference_time variable defines the model
>> initialization
>> > > time.
>> > > - The grid_mapping, x, and y variables define the grid
definition
>> > > information.
>> > > - The grid_mapping variable attributes map those variables to
the
>> > relevant
>> > > grid definition.
>> > >
>> > > Hope this helps.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > >
>> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA Affiliate
via RT
>> <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
>> > > >
>> > > > John,
>> > > >
>> > > > To clarify my previous request of information. I meant to ask
was
>> the
>> > > > following: Do you know of any good documentation on how to
make a
>> > NetCDF
>> > > > file CF-compliant? Also, do you have an example .nc file that
is
>> NetCDF
>> > > > CF-compliant that we could compare to?
>> > > >
>> > > > Thanks,
>> > > > Tanya
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA Affiliate
<
>> > > > tanya.peevey at noaa.gov> wrote:
>> > > >
>> > > > > John,
>> > > > >
>> > > > > We generated the NetCDF files ourselves so we could
probably make
>> it
>> > > work
>> > > > > for MET. Do you know how to get it into CF-compliant NetCDF
>> format?
>> > > > > Yes, we have grib files but the reason we created NetCDF
files was
>> > > > because
>> > > > > original the NR... file had accumulated precip over the
whole
>> > forecast
>> > > > > where the other file has 6h accumulation data, so we need
to
>> convert
>> > > the
>> > > > > NR... precip information into 6h accumulation (they had
different
>> > units
>> > > > > too, so we need to change that too). Since we had to change
the
>> units
>> > > > too I
>> > > > > think use generating our own NetCDF file is best but we
just need
>> to
>> > > get
>> > > > it
>> > > > > in the right format.
>> > > > > What is your opinion on the best way to approach this?
>> > > > >
>> > > > > If we get it into the right format is how I specify the
'name' and
>> > > > 'level'
>> > > > > of the field correct? It was hard to figure out the correct
>> notation
>> > > from
>> > > > > the manual.
>> > > > >
>> > > > > Thanks,
>> > > > > Tanya
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > >> Hi Tanya,
>> > > > >>
>> > > > >> I took a look at the gridded NetCDF data you're trying to
pass to
>> > > MODE.
>> > > > >> Unfortunately, MET doesn't support all types of gridded
NetCDF
>> > files.
>> > > > >> Instead, it only supports the MET NetCDF format (e.g. the
output
>> of
>> > > the
>> > > > >> pcp_combine tool), the output of the WRF pinterp (or
wrfinterp)
>> > > > utilities,
>> > > > >> and CF-compliant NetCDF files.
>> > > > >>
>> > > > >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I
see
>> that
>> > > it
>> > > > >> follows none of those conventions. However, it is closest
to
>> > > > CF-compliant
>> > > > >> NetCDF.
>> > > > >>
>> > > > >> When using a new data file format, instead of running it
through
>> > MODE,
>> > > > >> I'd suggest using plot_data_plane to make sure MET is
reading it
>> > > > >> correctly. I ran met-5.1/plot_data_plane as follows:
>> > > > >>
>> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
>> > > > >>
>> > > > >> The resulting image is attached. There are several issues
here:
>> > > > >> - I had to tell plot_data_plane to interpret this as CF-
compliant
>> > > NetCDF
>> > > > >> using NETCDF_NCCF.
>> > > > >> - The timing info in that file isn't specified in the
>> CF-compliant
>> > > > way...
>> > > > >> so MET can't discern the initialization, valid, and lead
times.
>> > > > >> - And then there's the obvious problem (in the image) of
the
>> world
>> > > being
>> > > > >> upside down and backwards!
>> > > > >>
>> > > > >> So while we did get *something*, there are obviously a lot
of
>> > issues.
>> > > > >> Basically, MET doesn't support the gridded NetCDF file
format you
>> > are
>> > > > >> using. Is this data possibly available in GRIB? Or do
you have
>> > > control
>> > > > >> over the NetCDF file format in use?
>> > > > >>
>> > > > >> Thanks,
>> > > > >> John
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Tanya R. Peevey, PhD
>> > > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> Group
>> > > > > NOAA ESRL Global Systems Division
>> > > > > 325 Broadway, Boulder, CO 80305
>> > > > > (303) 497-5847
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Tanya R. Peevey, PhD
>> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
>> > > > NOAA ESRL Global Systems Division
>> > > > 325 Broadway, Boulder, CO 80305
>> > > > (303) 497-5847
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>> > --
>> > Tanya R. Peevey, PhD
>> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> > NOAA ESRL Global Systems Division
>> > 325 Broadway, Boulder, CO 80305
>> > (303) 497-5847
>> >
>> >
>>
>>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Tue Feb 09 10:20:42 2016
Tanya,
I took a close look at your current data file:
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
Running it through the debugger, I see that the problem comes from
parsing
this time string:
"hours since 2005-05-01 (12:00)"
The parsing code is expecting that to be in HH:MM:SS format, not
HH:MM. I
changed it to this:
"hours since 2005-05-01 12:00:00"
And then I was able to run the plot_data_plane tool on it. However,
the
resulting image is still upside-down and backwards. See attached
(test.png).
To answer the question about grid x/y mapping, the answer is no. For
lat/lon grids with no projection info defined, MET uses the lat/lon
values
and intuits the projection info. For non-lat/lon grids, the
projection
info would need to be explicitly defined since there's no simple way
of
intuiting a grid definition from latitude and longitude values.
So about the data being upside-down and backwards... there are flags
in
GRIB files which indicate the order in which the data is written.
When
reading data from GRIB files, MET uses those flags to determine where
to
place things and orient the data right-side-up. The CF-compliant
NetCDF
code logic is much simpler... it just reads the data in the order it
is
encoded in the file and doesn't check which way is up!
I'm not really sure what to do here. Our support for CF-compliant
NetCDF
is still relatively new, and I'm not sure what the expectations should
be.
Should we add logic to the code to detect this situation and handle it
better? Or is the code working as expected? I'm not really sure.
I do see that ncview handles it better!
With the current version of MET, you have 2 choices:
(1) You could write lat/lon and data values in reverse order (i.e.
increasing latitude values) to "right the ship".
(2) Or you could leave things as-is and make use of the automated
regridding capability within the MET tools.
For example:
# run regrid_data_plane utility to regrid to NCEP grid 4
regrid_data_plane \
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
-field 'name="T2"; level="(*,*)";'
# run plot_data_plane on the output
plot_data_plane \
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
test_regrid.ps 'name="T2"; level="(*,*)";'
The result (test_regrid.png) is attached.
John
On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> My colleague, Jason English (cc'd on email), that is generating the
> NETCF_NCCF files has another question about the CF format.
>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Do we need the grid_mapping x and y variables if we are using the
lambert
> cylindrical equal area grid?
> If so, how do we convert 1-dimensional lat/lon coordinates to the 2d
> coordinates with y,x?
>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Thank you,
> Tanya Peevey
>
>
>
> On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate <
> tanya.peevey at noaa.gov> wrote:
>
> > John,
> >
> > Thank you for the information. We generated a new file and I tried
the
> > plot_data_plane tool on it and got the following error. Do you
have an
> idea
> > of what could be wrong? Sorry to bug you so much and thanks in
advance
> for
> > the help.
> >
> > Here's the location of the file:
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >
> > Here's the error:
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > file_type=NETCDF_NCCF;'
> > DEBUG 1: Opening data file: data/gfs_test/
> > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > ERROR :
> > ERROR : StringArray::operator[](int) const -> range check error!
> > ERROR :
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >
> > Thank you ,
> > Tanya
> >
> >
> >
> > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Tanya,
> >>
> >> Sorry for the delay. I was out of the office on Friday. Using
"LSP_6h"
> >> for the name and "(*,*)" for the level should do the trick.
> >>
> >> When trying to get the MET tools to read a new dataset, I always
start
> by
> >> using the plot_data_plane tool:
> >>
> >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
> >>
> >> I use that to make sure the name and level settings work as
expected,
> and
> >> it also creates an output plot so I can see that MET is (or is
not)
> >> reading/plotting the data as expected.
> >>
> >> That configuration string you pass to the the plot_data_plane
tool is
> >> exactly the same settings as are supported in the configuration
files.
> In
> >> fact, we read that string with the same code that parses the
config
> files.
> >>
> >> I find using plot_data_plane very helpful.
> >>
> >> Thanks,
> >> John
> >>
> >>
> >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate via
RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >> >
> >> > John,
> >> >
> >> > Thanks for the info. We are going to work on changing our file
format
> >> and
> >> > then I'll get back to you.
> >> > Did you see my other question about how I specified 'name' and
'level'
> >> for
> >> > the precip. field in my config file for MODE? Was how I did it
ok? I
> ask
> >> > because the manual was confusing on this point. If my syntax
was
> >> incorrect
> >> > could you please email me the correction.
> >> >
> >> > Thanks,
> >> > Tanya
> >> >
> >> >
> >> >
> >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > > Tanya,
> >> > >
> >> > > Here's a website about it:
> >> > > http://cfconventions.org/latest.html
> >> > >
> >> > > And I posted a sample CF-compliant NetCDF file here:
> >> > >
> >> > >
> >> >
> >>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> >> > >
> >> > > The relevant pieces in there are...
> >> > > - The global "Conventions" attribute set to "CF-...".
> >> > > - The time dimension/variable define the forecast valid time.
> >> > > - The forecast_reference_time variable defines the model
> >> initialization
> >> > > time.
> >> > > - The grid_mapping, x, and y variables define the grid
definition
> >> > > information.
> >> > > - The grid_mapping variable attributes map those variables to
the
> >> > relevant
> >> > > grid definition.
> >> > >
> >> > > Hope this helps.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > >
> >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
Affiliate via
> RT
> >> <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >> > > >
> >> > > > John,
> >> > > >
> >> > > > To clarify my previous request of information. I meant to
ask was
> >> the
> >> > > > following: Do you know of any good documentation on how to
make a
> >> > NetCDF
> >> > > > file CF-compliant? Also, do you have an example .nc file
that is
> >> NetCDF
> >> > > > CF-compliant that we could compare to?
> >> > > >
> >> > > > Thanks,
> >> > > > Tanya
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA
Affiliate <
> >> > > > tanya.peevey at noaa.gov> wrote:
> >> > > >
> >> > > > > John,
> >> > > > >
> >> > > > > We generated the NetCDF files ourselves so we could
probably
> make
> >> it
> >> > > work
> >> > > > > for MET. Do you know how to get it into CF-compliant
NetCDF
> >> format?
> >> > > > > Yes, we have grib files but the reason we created NetCDF
files
> was
> >> > > > because
> >> > > > > original the NR... file had accumulated precip over the
whole
> >> > forecast
> >> > > > > where the other file has 6h accumulation data, so we need
to
> >> convert
> >> > > the
> >> > > > > NR... precip information into 6h accumulation (they had
> different
> >> > units
> >> > > > > too, so we need to change that too). Since we had to
change the
> >> units
> >> > > > too I
> >> > > > > think use generating our own NetCDF file is best but we
just
> need
> >> to
> >> > > get
> >> > > > it
> >> > > > > in the right format.
> >> > > > > What is your opinion on the best way to approach this?
> >> > > > >
> >> > > > > If we get it into the right format is how I specify the
'name'
> and
> >> > > > 'level'
> >> > > > > of the field correct? It was hard to figure out the
correct
> >> notation
> >> > > from
> >> > > > > the manual.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Tanya
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via RT
<
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > >> Hi Tanya,
> >> > > > >>
> >> > > > >> I took a look at the gridded NetCDF data you're trying
to pass
> to
> >> > > MODE.
> >> > > > >> Unfortunately, MET doesn't support all types of gridded
NetCDF
> >> > files.
> >> > > > >> Instead, it only supports the MET NetCDF format (e.g.
the
> output
> >> of
> >> > > the
> >> > > > >> pcp_combine tool), the output of the WRF pinterp (or
wrfinterp)
> >> > > > utilities,
> >> > > > >> and CF-compliant NetCDF files.
> >> > > > >>
> >> > > > >> Looking at "prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc"
I see
> >> that
> >> > > it
> >> > > > >> follows none of those conventions. However, it is
closest to
> >> > > > CF-compliant
> >> > > > >> NetCDF.
> >> > > > >>
> >> > > > >> When using a new data file format, instead of running it
> through
> >> > MODE,
> >> > > > >> I'd suggest using plot_data_plane to make sure MET is
reading
> it
> >> > > > >> correctly. I ran met-5.1/plot_data_plane as follows:
> >> > > > >>
> >> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> file_type=NETCDF_NCCF;'
> >> > > > >>
> >> > > > >> The resulting image is attached. There are several
issues
> here:
> >> > > > >> - I had to tell plot_data_plane to interpret this as
> CF-compliant
> >> > > NetCDF
> >> > > > >> using NETCDF_NCCF.
> >> > > > >> - The timing info in that file isn't specified in the
> >> CF-compliant
> >> > > > way...
> >> > > > >> so MET can't discern the initialization, valid, and lead
times.
> >> > > > >> - And then there's the obvious problem (in the image) of
the
> >> world
> >> > > being
> >> > > > >> upside down and backwards!
> >> > > > >>
> >> > > > >> So while we did get *something*, there are obviously a
lot of
> >> > issues.
> >> > > > >> Basically, MET doesn't support the gridded NetCDF file
format
> you
> >> > are
> >> > > > >> using. Is this data possibly available in GRIB? Or do
you
> have
> >> > > control
> >> > > > >> over the NetCDF file format in use?
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> John
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Tanya R. Peevey, PhD
> >> > > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> >> Group
> >> > > > > NOAA ESRL Global Systems Division
> >> > > > > 325 Broadway, Boulder, CO 80305
> >> > > > > (303) 497-5847
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Tanya R. Peevey, PhD
> >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> >> > > > NOAA ESRL Global Systems Division
> >> > > > 325 Broadway, Boulder, CO 80305
> >> > > > (303) 497-5847
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > Tanya R. Peevey, PhD
> >> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> >> > NOAA ESRL Global Systems Division
> >> > 325 Broadway, Boulder, CO 80305
> >> > (303) 497-5847
> >> >
> >> >
> >>
> >>
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Thu Feb 11 14:13:44 2016
John,
So Jason and I got our data working with MODE. However, I have one
question. Right now MODE automatically plots the full grid available,
in
this case a global plot. We are only focused on North America. I was
thinking of using the regrid tool to change the grid from global to
North
American using NCEP grid 209. Can the regrid tool take any of the NCEP
grids found on the following website (
http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and apply it
to a
netcdf file (I ask becuase 209 had 'See GRIB Specifications' by it)?
Also,
right now we reverse the latitude indices so that everything is right
side
up but this makes it so another product we use can't view the .nc
files. So
my question to you is would we be able to do reverse the latitudes
using
the MET regrid tool with NCEP grid 209 or would we need to use the
regrid
tool twice, once with grid 004 and once with grid 209, or would
another
NCEP grid work?
Oh, and, could you direct me to some documentation on the regrid tool.
I
can see the usage from the command line but that doesn't really
address my
above question. Also, I didn't see anything in the MET manual.
Thanks,
Tanya
On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Tanya,
>
> I took a close look at your current data file:
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>
> Running it through the debugger, I see that the problem comes from
parsing
> this time string:
> "hours since 2005-05-01 (12:00)"
>
> The parsing code is expecting that to be in HH:MM:SS format, not
HH:MM. I
> changed it to this:
> "hours since 2005-05-01 12:00:00"
>
> And then I was able to run the plot_data_plane tool on it. However,
the
> resulting image is still upside-down and backwards. See attached
> (test.png).
>
> To answer the question about grid x/y mapping, the answer is no.
For
> lat/lon grids with no projection info defined, MET uses the lat/lon
values
> and intuits the projection info. For non-lat/lon grids, the
projection
> info would need to be explicitly defined since there's no simple way
of
> intuiting a grid definition from latitude and longitude values.
>
> So about the data being upside-down and backwards... there are flags
in
> GRIB files which indicate the order in which the data is written.
When
> reading data from GRIB files, MET uses those flags to determine
where to
> place things and orient the data right-side-up. The CF-compliant
NetCDF
> code logic is much simpler... it just reads the data in the order it
is
> encoded in the file and doesn't check which way is up!
>
> I'm not really sure what to do here. Our support for CF-compliant
NetCDF
> is still relatively new, and I'm not sure what the expectations
should be.
> Should we add logic to the code to detect this situation and handle
it
> better? Or is the code working as expected? I'm not really sure.
>
> I do see that ncview handles it better!
>
> With the current version of MET, you have 2 choices:
>
> (1) You could write lat/lon and data values in reverse order (i.e.
> increasing latitude values) to "right the ship".
>
> (2) Or you could leave things as-is and make use of the automated
> regridding capability within the MET tools.
>
> For example:
>
> # run regrid_data_plane utility to regrid to NCEP grid 4
> regrid_data_plane \
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> -field 'name="T2"; level="(*,*)";'
>
> # run plot_data_plane on the output
> plot_data_plane \
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> test_regrid.ps 'name="T2"; level="(*,*)";'
>
> The result (test_regrid.png) is attached.
>
> John
>
>
>
>
> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >
> > John,
> >
> > My colleague, Jason English (cc'd on email), that is generating
the
> > NETCF_NCCF files has another question about the CF format.
> >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > Do we need the grid_mapping x and y variables if we are using the
lambert
> > cylindrical equal area grid?
> > If so, how do we convert 1-dimensional lat/lon coordinates to the
2d
> > coordinates with y,x?
> >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Thank you,
> > Tanya Peevey
> >
> >
> >
> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate <
> > tanya.peevey at noaa.gov> wrote:
> >
> > > John,
> > >
> > > Thank you for the information. We generated a new file and I
tried the
> > > plot_data_plane tool on it and got the following error. Do you
have an
> > idea
> > > of what could be wrong? Sorry to bug you so much and thanks in
advance
> > for
> > > the help.
> > >
> > > Here's the location of the file:
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >
> > > Here's the error:
> > > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > >
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > > data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > file_type=NETCDF_NCCF;'
> > > DEBUG 1: Opening data file: data/gfs_test/
> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > ERROR :
> > > ERROR : StringArray::operator[](int) const -> range check
error!
> > > ERROR :
> > > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > >
> > > Thank you ,
> > > Tanya
> > >
> > >
> > >
> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Tanya,
> > >>
> > >> Sorry for the delay. I was out of the office on Friday. Using
> "LSP_6h"
> > >> for the name and "(*,*)" for the level should do the trick.
> > >>
> > >> When trying to get the MET tools to read a new dataset, I
always start
> > by
> > >> using the plot_data_plane tool:
> > >>
> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
> > >>
> > >> I use that to make sure the name and level settings work as
expected,
> > and
> > >> it also creates an output plot so I can see that MET is (or is
not)
> > >> reading/plotting the data as expected.
> > >>
> > >> That configuration string you pass to the the plot_data_plane
tool is
> > >> exactly the same settings as are supported in the configuration
files.
> > In
> > >> fact, we read that string with the same code that parses the
config
> > files.
> > >>
> > >> I find using plot_data_plane very helpful.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >>
> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate
via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> > >> >
> > >> > John,
> > >> >
> > >> > Thanks for the info. We are going to work on changing our
file
> format
> > >> and
> > >> > then I'll get back to you.
> > >> > Did you see my other question about how I specified 'name'
and
> 'level'
> > >> for
> > >> > the precip. field in my config file for MODE? Was how I did
it ok? I
> > ask
> > >> > because the manual was confusing on this point. If my syntax
was
> > >> incorrect
> > >> > could you please email me the correction.
> > >> >
> > >> > Thanks,
> > >> > Tanya
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> > > Tanya,
> > >> > >
> > >> > > Here's a website about it:
> > >> > > http://cfconventions.org/latest.html
> > >> > >
> > >> > > And I posted a sample CF-compliant NetCDF file here:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > >> > >
> > >> > > The relevant pieces in there are...
> > >> > > - The global "Conventions" attribute set to "CF-...".
> > >> > > - The time dimension/variable define the forecast valid
time.
> > >> > > - The forecast_reference_time variable defines the model
> > >> initialization
> > >> > > time.
> > >> > > - The grid_mapping, x, and y variables define the grid
definition
> > >> > > information.
> > >> > > - The grid_mapping variable attributes map those variables
to the
> > >> > relevant
> > >> > > grid definition.
> > >> > >
> > >> > > Hope this helps.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > >
> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
Affiliate via
> > RT
> > >> <
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >> > > >
> > >> > > > John,
> > >> > > >
> > >> > > > To clarify my previous request of information. I meant to
ask
> was
> > >> the
> > >> > > > following: Do you know of any good documentation on how
to make
> a
> > >> > NetCDF
> > >> > > > file CF-compliant? Also, do you have an example .nc file
that is
> > >> NetCDF
> > >> > > > CF-compliant that we could compare to?
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Tanya
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA
Affiliate <
> > >> > > > tanya.peevey at noaa.gov> wrote:
> > >> > > >
> > >> > > > > John,
> > >> > > > >
> > >> > > > > We generated the NetCDF files ourselves so we could
probably
> > make
> > >> it
> > >> > > work
> > >> > > > > for MET. Do you know how to get it into CF-compliant
NetCDF
> > >> format?
> > >> > > > > Yes, we have grib files but the reason we created
NetCDF files
> > was
> > >> > > > because
> > >> > > > > original the NR... file had accumulated precip over the
whole
> > >> > forecast
> > >> > > > > where the other file has 6h accumulation data, so we
need to
> > >> convert
> > >> > > the
> > >> > > > > NR... precip information into 6h accumulation (they had
> > different
> > >> > units
> > >> > > > > too, so we need to change that too). Since we had to
change
> the
> > >> units
> > >> > > > too I
> > >> > > > > think use generating our own NetCDF file is best but we
just
> > need
> > >> to
> > >> > > get
> > >> > > > it
> > >> > > > > in the right format.
> > >> > > > > What is your opinion on the best way to approach this?
> > >> > > > >
> > >> > > > > If we get it into the right format is how I specify the
'name'
> > and
> > >> > > > 'level'
> > >> > > > > of the field correct? It was hard to figure out the
correct
> > >> notation
> > >> > > from
> > >> > > > > the manual.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Tanya
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via
RT <
> > >> > > > > met_help at ucar.edu> wrote:
> > >> > > > >
> > >> > > > >> Hi Tanya,
> > >> > > > >>
> > >> > > > >> I took a look at the gridded NetCDF data you're trying
to
> pass
> > to
> > >> > > MODE.
> > >> > > > >> Unfortunately, MET doesn't support all types of
gridded
> NetCDF
> > >> > files.
> > >> > > > >> Instead, it only supports the MET NetCDF format (e.g.
the
> > output
> > >> of
> > >> > > the
> > >> > > > >> pcp_combine tool), the output of the WRF pinterp (or
> wrfinterp)
> > >> > > > utilities,
> > >> > > > >> and CF-compliant NetCDF files.
> > >> > > > >>
> > >> > > > >> Looking at
"prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I
> see
> > >> that
> > >> > > it
> > >> > > > >> follows none of those conventions. However, it is
closest to
> > >> > > > CF-compliant
> > >> > > > >> NetCDF.
> > >> > > > >>
> > >> > > > >> When using a new data file format, instead of running
it
> > through
> > >> > MODE,
> > >> > > > >> I'd suggest using plot_data_plane to make sure MET is
reading
> > it
> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as follows:
> > >> > > > >>
> > >> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > file_type=NETCDF_NCCF;'
> > >> > > > >>
> > >> > > > >> The resulting image is attached. There are several
issues
> > here:
> > >> > > > >> - I had to tell plot_data_plane to interpret this as
> > CF-compliant
> > >> > > NetCDF
> > >> > > > >> using NETCDF_NCCF.
> > >> > > > >> - The timing info in that file isn't specified in the
> > >> CF-compliant
> > >> > > > way...
> > >> > > > >> so MET can't discern the initialization, valid, and
lead
> times.
> > >> > > > >> - And then there's the obvious problem (in the image)
of the
> > >> world
> > >> > > being
> > >> > > > >> upside down and backwards!
> > >> > > > >>
> > >> > > > >> So while we did get *something*, there are obviously a
lot of
> > >> > issues.
> > >> > > > >> Basically, MET doesn't support the gridded NetCDF file
format
> > you
> > >> > are
> > >> > > > >> using. Is this data possibly available in GRIB? Or
do you
> > have
> > >> > > control
> > >> > > > >> over the NetCDF file format in use?
> > >> > > > >>
> > >> > > > >> Thanks,
> > >> > > > >> John
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Tanya R. Peevey, PhD
> > >> > > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > >> Group
> > >> > > > > NOAA ESRL Global Systems Division
> > >> > > > > 325 Broadway, Boulder, CO 80305
> > >> > > > > (303) 497-5847
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Tanya R. Peevey, PhD
> > >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > Group
> > >> > > > NOAA ESRL Global Systems Division
> > >> > > > 325 Broadway, Boulder, CO 80305
> > >> > > > (303) 497-5847
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > Tanya R. Peevey, PhD
> > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> > >> > NOAA ESRL Global Systems Division
> > >> > 325 Broadway, Boulder, CO 80305
> > >> > (303) 497-5847
> > >> >
> > >> >
> > >>
> > >>
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> >
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Thu Feb 11 16:10:03 2016
John,
To clarify my previous email, I'm trying to use the regrid tool to
reverse
the latitudes in our .nc files and focus on North America rather than
the
whole global. I just tried the regrid tool with the grid set to G209
and it
worked (both corrected the latitude issue and focused on North
America) but
with one small problem, there are some weird lines where Russia is
suppose
to be. Do you know what the problem is? Is there another grid you
could
recommend if the output from the regrid tool on the G209 grid can't be
fixed? Or could I define my own grid using another MET tool?
If all of the above fails we can modify our NetCDF files but I hope to
be
able to use the regrid tool.
All plots from my tests with the regrid tool can be found here:
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
Data used to make the plots can be found here:
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
Still would like to see some documentation on the regrid tool.
Lastly, after using the regrid tool I had to use the newest version of
MET
(met-5.1). Is this version good to go? Should I change my MET path to
this
version? Is there documentation on this version yet?
Thanks!
Tanya
On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA Affiliate <
tanya.peevey at noaa.gov> wrote:
> John,
>
> So Jason and I got our data working with MODE. However, I have one
> question. Right now MODE automatically plots the full grid
available, in
> this case a global plot. We are only focused on North America. I was
> thinking of using the regrid tool to change the grid from global to
North
> American using NCEP grid 209. Can the regrid tool take any of the
NCEP
> grids found on the following website (
> http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and apply
it to
> a netcdf file (I ask becuase 209 had 'See GRIB Specifications' by
it)?
> Also, right now we reverse the latitude indices so that everything
is right
> side up but this makes it so another product we use can't view the
.nc
> files. So my question to you is would we be able to do reverse the
> latitudes using the MET regrid tool with NCEP grid 209 or would we
need to
> use the regrid tool twice, once with grid 004 and once with grid
209, or
> would another NCEP grid work?
>
> Oh, and, could you direct me to some documentation on the regrid
tool. I
> can see the usage from the command line but that doesn't really
address my
> above question. Also, I didn't see anything in the MET manual.
>
> Thanks,
> Tanya
>
>
>
> On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Tanya,
>>
>> I took a close look at your current data file:
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>>
>> Running it through the debugger, I see that the problem comes from
parsing
>> this time string:
>> "hours since 2005-05-01 (12:00)"
>>
>> The parsing code is expecting that to be in HH:MM:SS format, not
HH:MM. I
>> changed it to this:
>> "hours since 2005-05-01 12:00:00"
>>
>> And then I was able to run the plot_data_plane tool on it.
However, the
>> resulting image is still upside-down and backwards. See attached
>> (test.png).
>>
>> To answer the question about grid x/y mapping, the answer is no.
For
>> lat/lon grids with no projection info defined, MET uses the lat/lon
values
>> and intuits the projection info. For non-lat/lon grids, the
projection
>> info would need to be explicitly defined since there's no simple
way of
>> intuiting a grid definition from latitude and longitude values.
>>
>> So about the data being upside-down and backwards... there are
flags in
>> GRIB files which indicate the order in which the data is written.
When
>> reading data from GRIB files, MET uses those flags to determine
where to
>> place things and orient the data right-side-up. The CF-compliant
NetCDF
>> code logic is much simpler... it just reads the data in the order
it is
>> encoded in the file and doesn't check which way is up!
>>
>> I'm not really sure what to do here. Our support for CF-compliant
NetCDF
>> is still relatively new, and I'm not sure what the expectations
should be.
>> Should we add logic to the code to detect this situation and handle
it
>> better? Or is the code working as expected? I'm not really sure.
>>
>> I do see that ncview handles it better!
>>
>> With the current version of MET, you have 2 choices:
>>
>> (1) You could write lat/lon and data values in reverse order (i.e.
>> increasing latitude values) to "right the ship".
>>
>> (2) Or you could leave things as-is and make use of the automated
>> regridding capability within the MET tools.
>>
>> For example:
>>
>> # run regrid_data_plane utility to regrid to NCEP grid 4
>> regrid_data_plane \
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> -field 'name="T2"; level="(*,*)";'
>>
>> # run plot_data_plane on the output
>> plot_data_plane \
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> test_regrid.ps 'name="T2"; level="(*,*)";'
>>
>> The result (test_regrid.png) is attached.
>>
>> John
>>
>>
>>
>>
>> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate via
RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> >
>> > John,
>> >
>> > My colleague, Jason English (cc'd on email), that is generating
the
>> > NETCF_NCCF files has another question about the CF format.
>> >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > Do we need the grid_mapping x and y variables if we are using the
>> lambert
>> > cylindrical equal area grid?
>> > If so, how do we convert 1-dimensional lat/lon coordinates to the
2d
>> > coordinates with y,x?
>> >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> >
>> > Thank you,
>> > Tanya Peevey
>> >
>> >
>> >
>> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate <
>> > tanya.peevey at noaa.gov> wrote:
>> >
>> > > John,
>> > >
>> > > Thank you for the information. We generated a new file and I
tried the
>> > > plot_data_plane tool on it and got the following error. Do you
have an
>> > idea
>> > > of what could be wrong? Sorry to bug you so much and thanks in
advance
>> > for
>> > > the help.
>> > >
>> > > Here's the location of the file:
>> > >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
>> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > >
>> > > Here's the error:
>> > > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > >
>> >
>> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
>> > > data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > > file_type=NETCDF_NCCF;'
>> > > DEBUG 1: Opening data file: data/gfs_test/
>> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > ERROR :
>> > > ERROR : StringArray::operator[](int) const -> range check
error!
>> > > ERROR :
>> > > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > >
>> > > Thank you ,
>> > > Tanya
>> > >
>> > >
>> > >
>> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > >> Tanya,
>> > >>
>> > >> Sorry for the delay. I was out of the office on Friday.
Using
>> "LSP_6h"
>> > >> for the name and "(*,*)" for the level should do the trick.
>> > >>
>> > >> When trying to get the MET tools to read a new dataset, I
always
>> start
>> > by
>> > >> using the plot_data_plane tool:
>> > >>
>> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
file_type=NETCDF_NCCF;'
>> > >>
>> > >> I use that to make sure the name and level settings work as
expected,
>> > and
>> > >> it also creates an output plot so I can see that MET is (or is
not)
>> > >> reading/plotting the data as expected.
>> > >>
>> > >> That configuration string you pass to the the plot_data_plane
tool is
>> > >> exactly the same settings as are supported in the
configuration
>> files.
>> > In
>> > >> fact, we read that string with the same code that parses the
config
>> > files.
>> > >>
>> > >> I find using plot_data_plane very helpful.
>> > >>
>> > >> Thanks,
>> > >> John
>> > >>
>> > >>
>> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA Affiliate
via RT
>> <
>> > >> met_help at ucar.edu> wrote:
>> > >>
>> > >> >
>> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > >> >
>> > >> > John,
>> > >> >
>> > >> > Thanks for the info. We are going to work on changing our
file
>> format
>> > >> and
>> > >> > then I'll get back to you.
>> > >> > Did you see my other question about how I specified 'name'
and
>> 'level'
>> > >> for
>> > >> > the precip. field in my config file for MODE? Was how I did
it ok?
>> I
>> > ask
>> > >> > because the manual was confusing on this point. If my syntax
was
>> > >> incorrect
>> > >> > could you please email me the correction.
>> > >> >
>> > >> > Thanks,
>> > >> > Tanya
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT <
>> > >> > met_help at ucar.edu> wrote:
>> > >> >
>> > >> > > Tanya,
>> > >> > >
>> > >> > > Here's a website about it:
>> > >> > > http://cfconventions.org/latest.html
>> > >> > >
>> > >> > > And I posted a sample CF-compliant NetCDF file here:
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> >
>>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
>> > >> > >
>> > >> > > The relevant pieces in there are...
>> > >> > > - The global "Conventions" attribute set to "CF-...".
>> > >> > > - The time dimension/variable define the forecast valid
time.
>> > >> > > - The forecast_reference_time variable defines the model
>> > >> initialization
>> > >> > > time.
>> > >> > > - The grid_mapping, x, and y variables define the grid
definition
>> > >> > > information.
>> > >> > > - The grid_mapping variable attributes map those variables
to the
>> > >> > relevant
>> > >> > > grid definition.
>> > >> > >
>> > >> > > Hope this helps.
>> > >> > >
>> > >> > > Thanks,
>> > >> > > John
>> > >> > >
>> > >> > >
>> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
Affiliate
>> via
>> > RT
>> > >> <
>> > >> > > met_help at ucar.edu> wrote:
>> > >> > >
>> > >> > > >
>> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> >
>> > >> > > >
>> > >> > > > John,
>> > >> > > >
>> > >> > > > To clarify my previous request of information. I meant
to ask
>> was
>> > >> the
>> > >> > > > following: Do you know of any good documentation on how
to
>> make a
>> > >> > NetCDF
>> > >> > > > file CF-compliant? Also, do you have an example .nc file
that
>> is
>> > >> NetCDF
>> > >> > > > CF-compliant that we could compare to?
>> > >> > > >
>> > >> > > > Thanks,
>> > >> > > > Tanya
>> > >> > > >
>> > >> > > >
>> > >> > > >
>> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA
Affiliate <
>> > >> > > > tanya.peevey at noaa.gov> wrote:
>> > >> > > >
>> > >> > > > > John,
>> > >> > > > >
>> > >> > > > > We generated the NetCDF files ourselves so we could
probably
>> > make
>> > >> it
>> > >> > > work
>> > >> > > > > for MET. Do you know how to get it into CF-compliant
NetCDF
>> > >> format?
>> > >> > > > > Yes, we have grib files but the reason we created
NetCDF
>> files
>> > was
>> > >> > > > because
>> > >> > > > > original the NR... file had accumulated precip over
the whole
>> > >> > forecast
>> > >> > > > > where the other file has 6h accumulation data, so we
need to
>> > >> convert
>> > >> > > the
>> > >> > > > > NR... precip information into 6h accumulation (they
had
>> > different
>> > >> > units
>> > >> > > > > too, so we need to change that too). Since we had to
change
>> the
>> > >> units
>> > >> > > > too I
>> > >> > > > > think use generating our own NetCDF file is best but
we just
>> > need
>> > >> to
>> > >> > > get
>> > >> > > > it
>> > >> > > > > in the right format.
>> > >> > > > > What is your opinion on the best way to approach this?
>> > >> > > > >
>> > >> > > > > If we get it into the right format is how I specify
the
>> 'name'
>> > and
>> > >> > > > 'level'
>> > >> > > > > of the field correct? It was hard to figure out the
correct
>> > >> notation
>> > >> > > from
>> > >> > > > > the manual.
>> > >> > > > >
>> > >> > > > > Thanks,
>> > >> > > > > Tanya
>> > >> > > > >
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway via
RT <
>> > >> > > > > met_help at ucar.edu> wrote:
>> > >> > > > >
>> > >> > > > >> Hi Tanya,
>> > >> > > > >>
>> > >> > > > >> I took a look at the gridded NetCDF data you're
trying to
>> pass
>> > to
>> > >> > > MODE.
>> > >> > > > >> Unfortunately, MET doesn't support all types of
gridded
>> NetCDF
>> > >> > files.
>> > >> > > > >> Instead, it only supports the MET NetCDF format (e.g.
the
>> > output
>> > >> of
>> > >> > > the
>> > >> > > > >> pcp_combine tool), the output of the WRF pinterp (or
>> wrfinterp)
>> > >> > > > utilities,
>> > >> > > > >> and CF-compliant NetCDF files.
>> > >> > > > >>
>> > >> > > > >> Looking at
"prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc" I
>> see
>> > >> that
>> > >> > > it
>> > >> > > > >> follows none of those conventions. However, it is
closest
>> to
>> > >> > > > CF-compliant
>> > >> > > > >> NetCDF.
>> > >> > > > >>
>> > >> > > > >> When using a new data file format, instead of running
it
>> > through
>> > >> > MODE,
>> > >> > > > >> I'd suggest using plot_data_plane to make sure MET is
>> reading
>> > it
>> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as follows:
>> > >> > > > >>
>> > >> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > file_type=NETCDF_NCCF;'
>> > >> > > > >>
>> > >> > > > >> The resulting image is attached. There are several
issues
>> > here:
>> > >> > > > >> - I had to tell plot_data_plane to interpret this as
>> > CF-compliant
>> > >> > > NetCDF
>> > >> > > > >> using NETCDF_NCCF.
>> > >> > > > >> - The timing info in that file isn't specified in the
>> > >> CF-compliant
>> > >> > > > way...
>> > >> > > > >> so MET can't discern the initialization, valid, and
lead
>> times.
>> > >> > > > >> - And then there's the obvious problem (in the image)
of the
>> > >> world
>> > >> > > being
>> > >> > > > >> upside down and backwards!
>> > >> > > > >>
>> > >> > > > >> So while we did get *something*, there are obviously
a lot
>> of
>> > >> > issues.
>> > >> > > > >> Basically, MET doesn't support the gridded NetCDF
file
>> format
>> > you
>> > >> > are
>> > >> > > > >> using. Is this data possibly available in GRIB? Or
do you
>> > have
>> > >> > > control
>> > >> > > > >> over the NetCDF file format in use?
>> > >> > > > >>
>> > >> > > > >> Thanks,
>> > >> > > > >> John
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > --
>> > >> > > > > Tanya R. Peevey, PhD
>> > >> > > > > Research Scientist I, Global Observing Systems
Analysis
>> (GOSA)
>> > >> Group
>> > >> > > > > NOAA ESRL Global Systems Division
>> > >> > > > > 325 Broadway, Boulder, CO 80305
>> > >> > > > > (303) 497-5847
>> > >> > > > >
>> > >> > > >
>> > >> > > >
>> > >> > > >
>> > >> > > > --
>> > >> > > > Tanya R. Peevey, PhD
>> > >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> > Group
>> > >> > > > NOAA ESRL Global Systems Division
>> > >> > > > 325 Broadway, Boulder, CO 80305
>> > >> > > > (303) 497-5847
>> > >> > > >
>> > >> > > >
>> > >> > >
>> > >> > >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > Tanya R. Peevey, PhD
>> > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> Group
>> > >> > NOAA ESRL Global Systems Division
>> > >> > 325 Broadway, Boulder, CO 80305
>> > >> > (303) 497-5847
>> > >> >
>> > >> >
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > Tanya R. Peevey, PhD
>> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> > > NOAA ESRL Global Systems Division
>> > > 325 Broadway, Boulder, CO 80305
>> > > (303) 497-5847
>> > >
>> >
>> >
>> >
>> > --
>> > Tanya R. Peevey, PhD
>> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> > NOAA ESRL Global Systems Division
>> > 325 Broadway, Boulder, CO 80305
>> > (303) 497-5847
>> >
>> >
>>
>>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Thu Feb 11 17:12:39 2016
Tanya,
I can look more closely tomorrow, but some initial thoughts are:
- yes, please use met-5.1 and use the default mode 5.1 config file
- you could use the regrid tool, but instead fill out the regrid
section of
the mode config file. Try setting to_grid = "G209";
That'll automatically regrid the global data in the fly.
That's all new in 5.1.
John
On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> To clarify my previous email, I'm trying to use the regrid tool to
reverse
> the latitudes in our .nc files and focus on North America rather
than the
> whole global. I just tried the regrid tool with the grid set to G209
and it
> worked (both corrected the latitude issue and focused on North
America) but
> with one small problem, there are some weird lines where Russia is
suppose
> to be. Do you know what the problem is? Is there another grid you
could
> recommend if the output from the regrid tool on the G209 grid can't
be
> fixed? Or could I define my own grid using another MET tool?
>
> If all of the above fails we can modify our NetCDF files but I hope
to be
> able to use the regrid tool.
>
> All plots from my tests with the regrid tool can be found here:
>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> Data used to make the plots can be found here:
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
>
> Still would like to see some documentation on the regrid tool.
>
> Lastly, after using the regrid tool I had to use the newest version
of MET
> (met-5.1). Is this version good to go? Should I change my MET path
to this
> version? Is there documentation on this version yet?
>
> Thanks!
> Tanya
>
>
>
> On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA Affiliate <
> tanya.peevey at noaa.gov <javascript:;>> wrote:
>
> > John,
> >
> > So Jason and I got our data working with MODE. However, I have one
> > question. Right now MODE automatically plots the full grid
available, in
> > this case a global plot. We are only focused on North America. I
was
> > thinking of using the regrid tool to change the grid from global
to North
> > American using NCEP grid 209. Can the regrid tool take any of the
NCEP
> > grids found on the following website (
> > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and apply
it to
> > a netcdf file (I ask becuase 209 had 'See GRIB Specifications' by
it)?
> > Also, right now we reverse the latitude indices so that everything
is
> right
> > side up but this makes it so another product we use can't view the
.nc
> > files. So my question to you is would we be able to do reverse the
> > latitudes using the MET regrid tool with NCEP grid 209 or would we
need
> to
> > use the regrid tool twice, once with grid 004 and once with grid
209, or
> > would another NCEP grid work?
> >
> > Oh, and, could you direct me to some documentation on the regrid
tool. I
> > can see the usage from the command line but that doesn't really
address
> my
> > above question. Also, I didn't see anything in the MET manual.
> >
> > Thanks,
> > Tanya
> >
> >
> >
> > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
> > met_help at ucar.edu <javascript:;>> wrote:
> >
> >> Tanya,
> >>
> >> I took a close look at your current data file:
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >>
> >> Running it through the debugger, I see that the problem comes
from
> parsing
> >> this time string:
> >> "hours since 2005-05-01 (12:00)"
> >>
> >> The parsing code is expecting that to be in HH:MM:SS format, not
> HH:MM. I
> >> changed it to this:
> >> "hours since 2005-05-01 12:00:00"
> >>
> >> And then I was able to run the plot_data_plane tool on it.
However, the
> >> resulting image is still upside-down and backwards. See attached
> >> (test.png).
> >>
> >> To answer the question about grid x/y mapping, the answer is no.
For
> >> lat/lon grids with no projection info defined, MET uses the
lat/lon
> values
> >> and intuits the projection info. For non-lat/lon grids, the
projection
> >> info would need to be explicitly defined since there's no simple
way of
> >> intuiting a grid definition from latitude and longitude values.
> >>
> >> So about the data being upside-down and backwards... there are
flags in
> >> GRIB files which indicate the order in which the data is written.
When
> >> reading data from GRIB files, MET uses those flags to determine
where to
> >> place things and orient the data right-side-up. The CF-
compliant
> NetCDF
> >> code logic is much simpler... it just reads the data in the order
it is
> >> encoded in the file and doesn't check which way is up!
> >>
> >> I'm not really sure what to do here. Our support for CF-
compliant
> NetCDF
> >> is still relatively new, and I'm not sure what the expectations
should
> be.
> >> Should we add logic to the code to detect this situation and
handle it
> >> better? Or is the code working as expected? I'm not really
sure.
> >>
> >> I do see that ncview handles it better!
> >>
> >> With the current version of MET, you have 2 choices:
> >>
> >> (1) You could write lat/lon and data values in reverse order
(i.e.
> >> increasing latitude values) to "right the ship".
> >>
> >> (2) Or you could leave things as-is and make use of the automated
> >> regridding capability within the MET tools.
> >>
> >> For example:
> >>
> >> # run regrid_data_plane utility to regrid to NCEP grid 4
> >> regrid_data_plane \
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> -field 'name="T2"; level="(*,*)";'
> >>
> >> # run plot_data_plane on the output
> >> plot_data_plane \
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> test_regrid.ps 'name="T2"; level="(*,*)";'
> >>
> >> The result (test_regrid.png) is attached.
> >>
> >> John
> >>
> >>
> >>
> >>
> >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate via
RT <
> >> met_help at ucar.edu <javascript:;>> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >> >
> >> > John,
> >> >
> >> > My colleague, Jason English (cc'd on email), that is generating
the
> >> > NETCF_NCCF files has another question about the CF format.
> >> >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > Do we need the grid_mapping x and y variables if we are using
the
> >> lambert
> >> > cylindrical equal area grid?
> >> > If so, how do we convert 1-dimensional lat/lon coordinates to
the 2d
> >> > coordinates with y,x?
> >> >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> >
> >> > Thank you,
> >> > Tanya Peevey
> >> >
> >> >
> >> >
> >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate <
> >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> >
> >> > > John,
> >> > >
> >> > > Thank you for the information. We generated a new file and I
tried
> the
> >> > > plot_data_plane tool on it and got the following error. Do
you have
> an
> >> > idea
> >> > > of what could be wrong? Sorry to bug you so much and thanks
in
> advance
> >> > for
> >> > > the help.
> >> > >
> >> > > Here's the location of the file:
> >> > >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > >
> >> > > Here's the error:
> >> > >
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> >> > >
> >> >
> >>
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> >> > > data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > > file_type=NETCDF_NCCF;'
> >> > > DEBUG 1: Opening data file: data/gfs_test/
> >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > ERROR :
> >> > > ERROR : StringArray::operator[](int) const -> range check
error!
> >> > > ERROR :
> >> > >
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> >> > >
> >> > > Thank you ,
> >> > > Tanya
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
> >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > >
> >> > >> Tanya,
> >> > >>
> >> > >> Sorry for the delay. I was out of the office on Friday.
Using
> >> "LSP_6h"
> >> > >> for the name and "(*,*)" for the level should do the trick.
> >> > >>
> >> > >> When trying to get the MET tools to read a new dataset, I
always
> >> start
> >> > by
> >> > >> using the plot_data_plane tool:
> >> > >>
> >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> file_type=NETCDF_NCCF;'
> >> > >>
> >> > >> I use that to make sure the name and level settings work as
> expected,
> >> > and
> >> > >> it also creates an output plot so I can see that MET is (or
is not)
> >> > >> reading/plotting the data as expected.
> >> > >>
> >> > >> That configuration string you pass to the the
plot_data_plane tool
> is
> >> > >> exactly the same settings as are supported in the
configuration
> >> files.
> >> > In
> >> > >> fact, we read that string with the same code that parses the
config
> >> > files.
> >> > >>
> >> > >> I find using plot_data_plane very helpful.
> >> > >>
> >> > >> Thanks,
> >> > >> John
> >> > >>
> >> > >>
> >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
Affiliate via
> RT
> >> <
> >> > >> met_help at ucar.edu <javascript:;>> wrote:
> >> > >>
> >> > >> >
> >> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >> > >> >
> >> > >> > John,
> >> > >> >
> >> > >> > Thanks for the info. We are going to work on changing our
file
> >> format
> >> > >> and
> >> > >> > then I'll get back to you.
> >> > >> > Did you see my other question about how I specified 'name'
and
> >> 'level'
> >> > >> for
> >> > >> > the precip. field in my config file for MODE? Was how I
did it
> ok?
> >> I
> >> > ask
> >> > >> > because the manual was confusing on this point. If my
syntax was
> >> > >> incorrect
> >> > >> > could you please email me the correction.
> >> > >> >
> >> > >> > Thanks,
> >> > >> > Tanya
> >> > >> >
> >> > >> >
> >> > >> >
> >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via RT
<
> >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> >> > >> >
> >> > >> > > Tanya,
> >> > >> > >
> >> > >> > > Here's a website about it:
> >> > >> > > http://cfconventions.org/latest.html
> >> > >> > >
> >> > >> > > And I posted a sample CF-compliant NetCDF file here:
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> >> > >> > >
> >> > >> > > The relevant pieces in there are...
> >> > >> > > - The global "Conventions" attribute set to "CF-...".
> >> > >> > > - The time dimension/variable define the forecast valid
time.
> >> > >> > > - The forecast_reference_time variable defines the model
> >> > >> initialization
> >> > >> > > time.
> >> > >> > > - The grid_mapping, x, and y variables define the grid
> definition
> >> > >> > > information.
> >> > >> > > - The grid_mapping variable attributes map those
variables to
> the
> >> > >> > relevant
> >> > >> > > grid definition.
> >> > >> > >
> >> > >> > > Hope this helps.
> >> > >> > >
> >> > >> > > Thanks,
> >> > >> > > John
> >> > >> > >
> >> > >> > >
> >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
Affiliate
> >> via
> >> > RT
> >> > >> <
> >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > >> > >
> >> > >> > > >
> >> > >> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> >
> >> > >> > > >
> >> > >> > > > John,
> >> > >> > > >
> >> > >> > > > To clarify my previous request of information. I meant
to ask
> >> was
> >> > >> the
> >> > >> > > > following: Do you know of any good documentation on
how to
> >> make a
> >> > >> > NetCDF
> >> > >> > > > file CF-compliant? Also, do you have an example .nc
file that
> >> is
> >> > >> NetCDF
> >> > >> > > > CF-compliant that we could compare to?
> >> > >> > > >
> >> > >> > > > Thanks,
> >> > >> > > > Tanya
> >> > >> > > >
> >> > >> > > >
> >> > >> > > >
> >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA
> Affiliate <
> >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > >> > > >
> >> > >> > > > > John,
> >> > >> > > > >
> >> > >> > > > > We generated the NetCDF files ourselves so we could
> probably
> >> > make
> >> > >> it
> >> > >> > > work
> >> > >> > > > > for MET. Do you know how to get it into CF-compliant
NetCDF
> >> > >> format?
> >> > >> > > > > Yes, we have grib files but the reason we created
NetCDF
> >> files
> >> > was
> >> > >> > > > because
> >> > >> > > > > original the NR... file had accumulated precip over
the
> whole
> >> > >> > forecast
> >> > >> > > > > where the other file has 6h accumulation data, so we
need
> to
> >> > >> convert
> >> > >> > > the
> >> > >> > > > > NR... precip information into 6h accumulation (they
had
> >> > different
> >> > >> > units
> >> > >> > > > > too, so we need to change that too). Since we had to
change
> >> the
> >> > >> units
> >> > >> > > > too I
> >> > >> > > > > think use generating our own NetCDF file is best but
we
> just
> >> > need
> >> > >> to
> >> > >> > > get
> >> > >> > > > it
> >> > >> > > > > in the right format.
> >> > >> > > > > What is your opinion on the best way to approach
this?
> >> > >> > > > >
> >> > >> > > > > If we get it into the right format is how I specify
the
> >> 'name'
> >> > and
> >> > >> > > > 'level'
> >> > >> > > > > of the field correct? It was hard to figure out the
correct
> >> > >> notation
> >> > >> > > from
> >> > >> > > > > the manual.
> >> > >> > > > >
> >> > >> > > > > Thanks,
> >> > >> > > > > Tanya
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway
via RT <
> >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> >> > >> > > > >
> >> > >> > > > >> Hi Tanya,
> >> > >> > > > >>
> >> > >> > > > >> I took a look at the gridded NetCDF data you're
trying to
> >> pass
> >> > to
> >> > >> > > MODE.
> >> > >> > > > >> Unfortunately, MET doesn't support all types of
gridded
> >> NetCDF
> >> > >> > files.
> >> > >> > > > >> Instead, it only supports the MET NetCDF format
(e.g. the
> >> > output
> >> > >> of
> >> > >> > > the
> >> > >> > > > >> pcp_combine tool), the output of the WRF pinterp
(or
> >> wrfinterp)
> >> > >> > > > utilities,
> >> > >> > > > >> and CF-compliant NetCDF files.
> >> > >> > > > >>
> >> > >> > > > >> Looking at
"prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc"
> I
> >> see
> >> > >> that
> >> > >> > > it
> >> > >> > > > >> follows none of those conventions. However, it is
closest
> >> to
> >> > >> > > > CF-compliant
> >> > >> > > > >> NetCDF.
> >> > >> > > > >>
> >> > >> > > > >> When using a new data file format, instead of
running it
> >> > through
> >> > >> > MODE,
> >> > >> > > > >> I'd suggest using plot_data_plane to make sure MET
is
> >> reading
> >> > it
> >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as
follows:
> >> > >> > > > >>
> >> > >> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > file_type=NETCDF_NCCF;'
> >> > >> > > > >>
> >> > >> > > > >> The resulting image is attached. There are several
issues
> >> > here:
> >> > >> > > > >> - I had to tell plot_data_plane to interpret this
as
> >> > CF-compliant
> >> > >> > > NetCDF
> >> > >> > > > >> using NETCDF_NCCF.
> >> > >> > > > >> - The timing info in that file isn't specified in
the
> >> > >> CF-compliant
> >> > >> > > > way...
> >> > >> > > > >> so MET can't discern the initialization, valid, and
lead
> >> times.
> >> > >> > > > >> - And then there's the obvious problem (in the
image) of
> the
> >> > >> world
> >> > >> > > being
> >> > >> > > > >> upside down and backwards!
> >> > >> > > > >>
> >> > >> > > > >> So while we did get *something*, there are
obviously a lot
> >> of
> >> > >> > issues.
> >> > >> > > > >> Basically, MET doesn't support the gridded NetCDF
file
> >> format
> >> > you
> >> > >> > are
> >> > >> > > > >> using. Is this data possibly available in GRIB?
Or do
> you
> >> > have
> >> > >> > > control
> >> > >> > > > >> over the NetCDF file format in use?
> >> > >> > > > >>
> >> > >> > > > >> Thanks,
> >> > >> > > > >> John
> >> > >> > > > >>
> >> > >> > > > >>
> >> > >> > > > >>
> >> > >> > > > >>
> >> > >> > > > >
> >> > >> > > > >
> >> > >> > > > > --
> >> > >> > > > > Tanya R. Peevey, PhD
> >> > >> > > > > Research Scientist I, Global Observing Systems
Analysis
> >> (GOSA)
> >> > >> Group
> >> > >> > > > > NOAA ESRL Global Systems Division
> >> > >> > > > > 325 Broadway, Boulder, CO 80305
> >> > >> > > > > (303) 497-5847
> >> > >> > > > >
> >> > >> > > >
> >> > >> > > >
> >> > >> > > >
> >> > >> > > > --
> >> > >> > > > Tanya R. Peevey, PhD
> >> > >> > > > Research Scientist I, Global Observing Systems
Analysis
> (GOSA)
> >> > Group
> >> > >> > > > NOAA ESRL Global Systems Division
> >> > >> > > > 325 Broadway, Boulder, CO 80305
> >> > >> > > > (303) 497-5847
> >> > >> > > >
> >> > >> > > >
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >> >
> >> > >> > --
> >> > >> > Tanya R. Peevey, PhD
> >> > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> >> Group
> >> > >> > NOAA ESRL Global Systems Division
> >> > >> > 325 Broadway, Boulder, CO 80305
> >> > >> > (303) 497-5847
> >> > >> >
> >> > >> >
> >> > >>
> >> > >>
> >> > >
> >> > >
> >> > > --
> >> > > Tanya R. Peevey, PhD
> >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> >> > > NOAA ESRL Global Systems Division
> >> > > 325 Broadway, Boulder, CO 80305
> >> > > (303) 497-5847
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Tanya R. Peevey, PhD
> >> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> >> > NOAA ESRL Global Systems Division
> >> > 325 Broadway, Boulder, CO 80305
> >> > (303) 497-5847
> >> >
> >> >
> >>
> >>
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
>
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Fri Feb 12 09:15:10 2016
John,
Ok, tried G209 in the new MODE and I got results but the lines on the
map
that are suppose to define Russia are all still wrong. See following
directory,
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
So the above worked with a NetCDF file we generated that has CF-
compliant
time and grid_mapping information. We found out that the grid_mapping
variable in the CF-compliant file was what was not allowing us to use
Panoply (a package for quickly viewing data in an NetCDF file). So we
removed the grid_mapping information and tried again, to see if MODE
would
work and it did not. Here are where the two files are location
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
with *.cf.nc being the file without grid_mapping and *.old_cf.nc being
the
file with grid_mapping.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
data/gfs_test/
NR2006.pl.1by1.7188.2006022500.cf.nc
gfs_test/config/MODEConfig_gfstest
-outdir gfs_test/out/mode/ -v 2
DEBUG 1: Default Config File:
/scratch4/BMC/dtc/MET/met-5.1/share/met/config/MODEConfig_default
DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
ERROR :
ERROR : Trouble reading forecast file "data/gfs_test/
prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
ERROR :
Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Basically, want to try and fix the regridding to the G209 grid so
Russia
looks normal and want to confirm that we do need to use the
*.old_cf.nc
files instead of the other within MET.
Thanks,
Tanya
P.S. Having the regird option in MODE is great. Now I don't have to
create
another set of files before processing the data. Saves me some disk
space
:-).
On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Tanya,
>
> I can look more closely tomorrow, but some initial thoughts are:
>
> - yes, please use met-5.1 and use the default mode 5.1 config file
> - you could use the regrid tool, but instead fill out the regrid
section of
> the mode config file. Try setting to_grid = "G209";
>
> That'll automatically regrid the global data in the fly.
>
> That's all new in 5.1.
>
> John
>
> On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >
> > John,
> >
> > To clarify my previous email, I'm trying to use the regrid tool to
> reverse
> > the latitudes in our .nc files and focus on North America rather
than the
> > whole global. I just tried the regrid tool with the grid set to
G209 and
> it
> > worked (both corrected the latitude issue and focused on North
America)
> but
> > with one small problem, there are some weird lines where Russia is
> suppose
> > to be. Do you know what the problem is? Is there another grid you
could
> > recommend if the output from the regrid tool on the G209 grid
can't be
> > fixed? Or could I define my own grid using another MET tool?
> >
> > If all of the above fails we can modify our NetCDF files but I
hope to be
> > able to use the regrid tool.
> >
> > All plots from my tests with the regrid tool can be found here:
> >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> > Data used to make the plots can be found here:
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> >
> > Still would like to see some documentation on the regrid tool.
> >
> > Lastly, after using the regrid tool I had to use the newest
version of
> MET
> > (met-5.1). Is this version good to go? Should I change my MET path
to
> this
> > version? Is there documentation on this version yet?
> >
> > Thanks!
> > Tanya
> >
> >
> >
> > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA Affiliate <
> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >
> > > John,
> > >
> > > So Jason and I got our data working with MODE. However, I have
one
> > > question. Right now MODE automatically plots the full grid
available,
> in
> > > this case a global plot. We are only focused on North America. I
was
> > > thinking of using the regrid tool to change the grid from global
to
> North
> > > American using NCEP grid 209. Can the regrid tool take any of
the NCEP
> > > grids found on the following website (
> > > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and
apply it
> to
> > > a netcdf file (I ask becuase 209 had 'See GRIB Specifications'
by it)?
> > > Also, right now we reverse the latitude indices so that
everything is
> > right
> > > side up but this makes it so another product we use can't view
the .nc
> > > files. So my question to you is would we be able to do reverse
the
> > > latitudes using the MET regrid tool with NCEP grid 209 or would
we need
> > to
> > > use the regrid tool twice, once with grid 004 and once with grid
209,
> or
> > > would another NCEP grid work?
> > >
> > > Oh, and, could you direct me to some documentation on the regrid
tool.
> I
> > > can see the usage from the command line but that doesn't really
address
> > my
> > > above question. Also, I didn't see anything in the MET manual.
> > >
> > > Thanks,
> > > Tanya
> > >
> > >
> > >
> > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
> > > met_help at ucar.edu <javascript:;>> wrote:
> > >
> > >> Tanya,
> > >>
> > >> I took a close look at your current data file:
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >>
> > >> Running it through the debugger, I see that the problem comes
from
> > parsing
> > >> this time string:
> > >> "hours since 2005-05-01 (12:00)"
> > >>
> > >> The parsing code is expecting that to be in HH:MM:SS format,
not
> > HH:MM. I
> > >> changed it to this:
> > >> "hours since 2005-05-01 12:00:00"
> > >>
> > >> And then I was able to run the plot_data_plane tool on it.
However,
> the
> > >> resulting image is still upside-down and backwards. See
attached
> > >> (test.png).
> > >>
> > >> To answer the question about grid x/y mapping, the answer is
no. For
> > >> lat/lon grids with no projection info defined, MET uses the
lat/lon
> > values
> > >> and intuits the projection info. For non-lat/lon grids, the
> projection
> > >> info would need to be explicitly defined since there's no
simple way
> of
> > >> intuiting a grid definition from latitude and longitude values.
> > >>
> > >> So about the data being upside-down and backwards... there are
flags
> in
> > >> GRIB files which indicate the order in which the data is
written.
> When
> > >> reading data from GRIB files, MET uses those flags to determine
where
> to
> > >> place things and orient the data right-side-up. The CF-
compliant
> > NetCDF
> > >> code logic is much simpler... it just reads the data in the
order it
> is
> > >> encoded in the file and doesn't check which way is up!
> > >>
> > >> I'm not really sure what to do here. Our support for CF-
compliant
> > NetCDF
> > >> is still relatively new, and I'm not sure what the expectations
should
> > be.
> > >> Should we add logic to the code to detect this situation and
handle it
> > >> better? Or is the code working as expected? I'm not really
sure.
> > >>
> > >> I do see that ncview handles it better!
> > >>
> > >> With the current version of MET, you have 2 choices:
> > >>
> > >> (1) You could write lat/lon and data values in reverse order
(i.e.
> > >> increasing latitude values) to "right the ship".
> > >>
> > >> (2) Or you could leave things as-is and make use of the
automated
> > >> regridding capability within the MET tools.
> > >>
> > >> For example:
> > >>
> > >> # run regrid_data_plane utility to regrid to NCEP grid 4
> > >> regrid_data_plane \
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > >> -field 'name="T2"; level="(*,*)";'
> > >>
> > >> # run plot_data_plane on the output
> > >> plot_data_plane \
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> > >>
> > >> The result (test_regrid.png) is attached.
> > >>
> > >> John
> > >>
> > >>
> > >>
> > >>
> > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate
via RT <
> > >> met_help at ucar.edu <javascript:;>> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> > >> >
> > >> > John,
> > >> >
> > >> > My colleague, Jason English (cc'd on email), that is
generating the
> > >> > NETCF_NCCF files has another question about the CF format.
> > >> >
> > >> >
> > >>
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >> > Do we need the grid_mapping x and y variables if we are using
the
> > >> lambert
> > >> > cylindrical equal area grid?
> > >> > If so, how do we convert 1-dimensional lat/lon coordinates to
the 2d
> > >> > coordinates with y,x?
> > >> >
> > >> >
> > >>
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >> >
> > >> > Thank you,
> > >> > Tanya Peevey
> > >> >
> > >> >
> > >> >
> > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA Affiliate
<
> > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >> >
> > >> > > John,
> > >> > >
> > >> > > Thank you for the information. We generated a new file and
I tried
> > the
> > >> > > plot_data_plane tool on it and got the following error. Do
you
> have
> > an
> > >> > idea
> > >> > > of what could be wrong? Sorry to bug you so much and thanks
in
> > advance
> > >> > for
> > >> > > the help.
> > >> > >
> > >> > > Here's the location of the file:
> > >> > >
> > >>
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > >
> > >> > > Here's the error:
> > >> > >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > >> > >
> > >> >
> > >>
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > >> > >
data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > >> > > file_type=NETCDF_NCCF;'
> > >> > > DEBUG 1: Opening data file: data/gfs_test/
> > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > ERROR :
> > >> > > ERROR : StringArray::operator[](int) const -> range check
error!
> > >> > > ERROR :
> > >> > >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > >> > >
> > >> > > Thank you ,
> > >> > > Tanya
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT <
> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > >
> > >> > >> Tanya,
> > >> > >>
> > >> > >> Sorry for the delay. I was out of the office on Friday.
Using
> > >> "LSP_6h"
> > >> > >> for the name and "(*,*)" for the level should do the
trick.
> > >> > >>
> > >> > >> When trying to get the MET tools to read a new dataset, I
always
> > >> start
> > >> > by
> > >> > >> using the plot_data_plane tool:
> > >> > >>
> > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > file_type=NETCDF_NCCF;'
> > >> > >>
> > >> > >> I use that to make sure the name and level settings work
as
> > expected,
> > >> > and
> > >> > >> it also creates an output plot so I can see that MET is
(or is
> not)
> > >> > >> reading/plotting the data as expected.
> > >> > >>
> > >> > >> That configuration string you pass to the the
plot_data_plane
> tool
> > is
> > >> > >> exactly the same settings as are supported in the
configuration
> > >> files.
> > >> > In
> > >> > >> fact, we read that string with the same code that parses
the
> config
> > >> > files.
> > >> > >>
> > >> > >> I find using plot_data_plane very helpful.
> > >> > >>
> > >> > >> Thanks,
> > >> > >> John
> > >> > >>
> > >> > >>
> > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
Affiliate via
> > RT
> > >> <
> > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> > >> > >>
> > >> > >> >
> > >> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >
> > >> > >> >
> > >> > >> > John,
> > >> > >> >
> > >> > >> > Thanks for the info. We are going to work on changing
our file
> > >> format
> > >> > >> and
> > >> > >> > then I'll get back to you.
> > >> > >> > Did you see my other question about how I specified
'name' and
> > >> 'level'
> > >> > >> for
> > >> > >> > the precip. field in my config file for MODE? Was how I
did it
> > ok?
> > >> I
> > >> > ask
> > >> > >> > because the manual was confusing on this point. If my
syntax
> was
> > >> > >> incorrect
> > >> > >> > could you please email me the correction.
> > >> > >> >
> > >> > >> > Thanks,
> > >> > >> > Tanya
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway via
RT <
> > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> > >> > >> >
> > >> > >> > > Tanya,
> > >> > >> > >
> > >> > >> > > Here's a website about it:
> > >> > >> > > http://cfconventions.org/latest.html
> > >> > >> > >
> > >> > >> > > And I posted a sample CF-compliant NetCDF file here:
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > >> > >> > >
> > >> > >> > > The relevant pieces in there are...
> > >> > >> > > - The global "Conventions" attribute set to "CF-...".
> > >> > >> > > - The time dimension/variable define the forecast
valid time.
> > >> > >> > > - The forecast_reference_time variable defines the
model
> > >> > >> initialization
> > >> > >> > > time.
> > >> > >> > > - The grid_mapping, x, and y variables define the grid
> > definition
> > >> > >> > > information.
> > >> > >> > > - The grid_mapping variable attributes map those
variables to
> > the
> > >> > >> > relevant
> > >> > >> > > grid definition.
> > >> > >> > >
> > >> > >> > > Hope this helps.
> > >> > >> > >
> > >> > >> > > Thanks,
> > >> > >> > > John
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
> Affiliate
> > >> via
> > >> > RT
> > >> > >> <
> > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > >> > >
> > >> > >> > > >
> > >> > >> > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > >> >
> > >> > >> > > >
> > >> > >> > > > John,
> > >> > >> > > >
> > >> > >> > > > To clarify my previous request of information. I
meant to
> ask
> > >> was
> > >> > >> the
> > >> > >> > > > following: Do you know of any good documentation on
how to
> > >> make a
> > >> > >> > NetCDF
> > >> > >> > > > file CF-compliant? Also, do you have an example .nc
file
> that
> > >> is
> > >> > >> NetCDF
> > >> > >> > > > CF-compliant that we could compare to?
> > >> > >> > > >
> > >> > >> > > > Thanks,
> > >> > >> > > > Tanya
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey - NOAA
> > Affiliate <
> > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >> > >> > > >
> > >> > >> > > > > John,
> > >> > >> > > > >
> > >> > >> > > > > We generated the NetCDF files ourselves so we
could
> > probably
> > >> > make
> > >> > >> it
> > >> > >> > > work
> > >> > >> > > > > for MET. Do you know how to get it into CF-
compliant
> NetCDF
> > >> > >> format?
> > >> > >> > > > > Yes, we have grib files but the reason we created
NetCDF
> > >> files
> > >> > was
> > >> > >> > > > because
> > >> > >> > > > > original the NR... file had accumulated precip
over the
> > whole
> > >> > >> > forecast
> > >> > >> > > > > where the other file has 6h accumulation data, so
we need
> > to
> > >> > >> convert
> > >> > >> > > the
> > >> > >> > > > > NR... precip information into 6h accumulation
(they had
> > >> > different
> > >> > >> > units
> > >> > >> > > > > too, so we need to change that too). Since we had
to
> change
> > >> the
> > >> > >> units
> > >> > >> > > > too I
> > >> > >> > > > > think use generating our own NetCDF file is best
but we
> > just
> > >> > need
> > >> > >> to
> > >> > >> > > get
> > >> > >> > > > it
> > >> > >> > > > > in the right format.
> > >> > >> > > > > What is your opinion on the best way to approach
this?
> > >> > >> > > > >
> > >> > >> > > > > If we get it into the right format is how I
specify the
> > >> 'name'
> > >> > and
> > >> > >> > > > 'level'
> > >> > >> > > > > of the field correct? It was hard to figure out
the
> correct
> > >> > >> notation
> > >> > >> > > from
> > >> > >> > > > > the manual.
> > >> > >> > > > >
> > >> > >> > > > > Thanks,
> > >> > >> > > > > Tanya
> > >> > >> > > > >
> > >> > >> > > > >
> > >> > >> > > > >
> > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley Gotway
via
> RT <
> > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > >> > > > >
> > >> > >> > > > >> Hi Tanya,
> > >> > >> > > > >>
> > >> > >> > > > >> I took a look at the gridded NetCDF data you're
trying
> to
> > >> pass
> > >> > to
> > >> > >> > > MODE.
> > >> > >> > > > >> Unfortunately, MET doesn't support all types of
gridded
> > >> NetCDF
> > >> > >> > files.
> > >> > >> > > > >> Instead, it only supports the MET NetCDF format
(e.g.
> the
> > >> > output
> > >> > >> of
> > >> > >> > > the
> > >> > >> > > > >> pcp_combine tool), the output of the WRF pinterp
(or
> > >> wrfinterp)
> > >> > >> > > > utilities,
> > >> > >> > > > >> and CF-compliant NetCDF files.
> > >> > >> > > > >>
> > >> > >> > > > >> Looking at
"prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> "
> > I
> > >> see
> > >> > >> that
> > >> > >> > > it
> > >> > >> > > > >> follows none of those conventions. However, it
is
> closest
> > >> to
> > >> > >> > > > CF-compliant
> > >> > >> > > > >> NetCDF.
> > >> > >> > > > >>
> > >> > >> > > > >> When using a new data file format, instead of
running it
> > >> > through
> > >> > >> > MODE,
> > >> > >> > > > >> I'd suggest using plot_data_plane to make sure
MET is
> > >> reading
> > >> > it
> > >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as
follows:
> > >> > >> > > > >>
> > >> > >> > > > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
\
> > >> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > >> > file_type=NETCDF_NCCF;'
> > >> > >> > > > >>
> > >> > >> > > > >> The resulting image is attached. There are
several
> issues
> > >> > here:
> > >> > >> > > > >> - I had to tell plot_data_plane to interpret this
as
> > >> > CF-compliant
> > >> > >> > > NetCDF
> > >> > >> > > > >> using NETCDF_NCCF.
> > >> > >> > > > >> - The timing info in that file isn't specified in
the
> > >> > >> CF-compliant
> > >> > >> > > > way...
> > >> > >> > > > >> so MET can't discern the initialization, valid,
and lead
> > >> times.
> > >> > >> > > > >> - And then there's the obvious problem (in the
image) of
> > the
> > >> > >> world
> > >> > >> > > being
> > >> > >> > > > >> upside down and backwards!
> > >> > >> > > > >>
> > >> > >> > > > >> So while we did get *something*, there are
obviously a
> lot
> > >> of
> > >> > >> > issues.
> > >> > >> > > > >> Basically, MET doesn't support the gridded NetCDF
file
> > >> format
> > >> > you
> > >> > >> > are
> > >> > >> > > > >> using. Is this data possibly available in GRIB?
Or do
> > you
> > >> > have
> > >> > >> > > control
> > >> > >> > > > >> over the NetCDF file format in use?
> > >> > >> > > > >>
> > >> > >> > > > >> Thanks,
> > >> > >> > > > >> John
> > >> > >> > > > >>
> > >> > >> > > > >>
> > >> > >> > > > >>
> > >> > >> > > > >>
> > >> > >> > > > >
> > >> > >> > > > >
> > >> > >> > > > > --
> > >> > >> > > > > Tanya R. Peevey, PhD
> > >> > >> > > > > Research Scientist I, Global Observing Systems
Analysis
> > >> (GOSA)
> > >> > >> Group
> > >> > >> > > > > NOAA ESRL Global Systems Division
> > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> > >> > >> > > > > (303) 497-5847
> > >> > >> > > > >
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > > > --
> > >> > >> > > > Tanya R. Peevey, PhD
> > >> > >> > > > Research Scientist I, Global Observing Systems
Analysis
> > (GOSA)
> > >> > Group
> > >> > >> > > > NOAA ESRL Global Systems Division
> > >> > >> > > > 325 Broadway, Boulder, CO 80305
> > >> > >> > > > (303) 497-5847
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >> >
> > >> > >> > --
> > >> > >> > Tanya R. Peevey, PhD
> > >> > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > >> Group
> > >> > >> > NOAA ESRL Global Systems Division
> > >> > >> > 325 Broadway, Boulder, CO 80305
> > >> > >> > (303) 497-5847
> > >> > >> >
> > >> > >> >
> > >> > >>
> > >> > >>
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Tanya R. Peevey, PhD
> > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> > >> > > NOAA ESRL Global Systems Division
> > >> > > 325 Broadway, Boulder, CO 80305
> > >> > > (303) 497-5847
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Tanya R. Peevey, PhD
> > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> > >> > NOAA ESRL Global Systems Division
> > >> > 325 Broadway, Boulder, CO 80305
> > >> > (303) 497-5847
> > >> >
> > >> >
> > >>
> > >>
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> >
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Fri Feb 12 11:50:00 2016
Tanya,
I took a close look at the issue with plotting the map for grid 209.
The
issue is caused by the map data straddling the international date
line. I
tried rescaling the longitudes from -180,180 to 0,360 but wasn't able
to
resolve this issue. So this is not easily fixed. I'll need to have
Randy
Bullock look for a solution.
In the meantime, you have a couple options...
- If you're not tied to grid 209, you could easily switch to using a
different grid.
- Or you could set up the config file to plot a set of map data files
which
does not include Russia.
Take a look in met-5.1/share/met/config/ConfigMap. The "map_data"
section
controls the map data files to be plotted. I've attached a map data
file
to this message from which I've removed all the Russia data:
country_data_minus_russia
Plotting on this map data file results in the attached plot of grid
209.
FYI, you can just copy and paste the "map_data" config file section to
the
bottom of your MODE config file. And then set the path to the
attached map
data file. That'll overwrite the default values set in ConfigMapData.
John
On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> Ok, tried G209 in the new MODE and I got results but the lines on
the map
> that are suppose to define Russia are all still wrong. See following
> directory,
>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
>
> So the above worked with a NetCDF file we generated that has CF-
compliant
> time and grid_mapping information. We found out that the
grid_mapping
> variable in the CF-compliant file was what was not allowing us to
use
> Panoply (a package for quickly viewing data in an NetCDF file). So
we
> removed the grid_mapping information and tried again, to see if MODE
would
> work and it did not. Here are where the two files are location
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> with *.cf.nc being the file without grid_mapping and *.old_cf.nc
being the
> file with grid_mapping.
>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> data/gfs_test/
> NR2006.pl.1by1.7188.2006022500.cf.nc
gfs_test/config/MODEConfig_gfstest
> -outdir gfs_test/out/mode/ -v 2
> DEBUG 1: Default Config File:
> /scratch4/BMC/dtc/MET/met-5.1/share/met/config/MODEConfig_default
> DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
> DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
> ERROR :
> ERROR : Trouble reading forecast file "data/gfs_test/
> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> ERROR :
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Basically, want to try and fix the regridding to the G209 grid so
Russia
> looks normal and want to confirm that we do need to use the
*.old_cf.nc
> files instead of the other within MET.
>
> Thanks,
> Tanya
>
> P.S. Having the regird option in MODE is great. Now I don't have to
create
> another set of files before processing the data. Saves me some disk
space
> :-).
>
>
> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Tanya,
> >
> > I can look more closely tomorrow, but some initial thoughts are:
> >
> > - yes, please use met-5.1 and use the default mode 5.1 config file
> > - you could use the regrid tool, but instead fill out the regrid
section
> of
> > the mode config file. Try setting to_grid = "G209";
> >
> > That'll automatically regrid the global data in the fly.
> >
> > That's all new in 5.1.
> >
> > John
> >
> > On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >
> > > John,
> > >
> > > To clarify my previous email, I'm trying to use the regrid tool
to
> > reverse
> > > the latitudes in our .nc files and focus on North America rather
than
> the
> > > whole global. I just tried the regrid tool with the grid set to
G209
> and
> > it
> > > worked (both corrected the latitude issue and focused on North
America)
> > but
> > > with one small problem, there are some weird lines where Russia
is
> > suppose
> > > to be. Do you know what the problem is? Is there another grid
you could
> > > recommend if the output from the regrid tool on the G209 grid
can't be
> > > fixed? Or could I define my own grid using another MET tool?
> > >
> > > If all of the above fails we can modify our NetCDF files but I
hope to
> be
> > > able to use the regrid tool.
> > >
> > > All plots from my tests with the regrid tool can be found here:
> > >
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> > > Data used to make the plots can be found here:
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> > >
> > > Still would like to see some documentation on the regrid tool.
> > >
> > > Lastly, after using the regrid tool I had to use the newest
version of
> > MET
> > > (met-5.1). Is this version good to go? Should I change my MET
path to
> > this
> > > version? Is there documentation on this version yet?
> > >
> > > Thanks!
> > > Tanya
> > >
> > >
> > >
> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA Affiliate <
> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >
> > > > John,
> > > >
> > > > So Jason and I got our data working with MODE. However, I have
one
> > > > question. Right now MODE automatically plots the full grid
available,
> > in
> > > > this case a global plot. We are only focused on North America.
I was
> > > > thinking of using the regrid tool to change the grid from
global to
> > North
> > > > American using NCEP grid 209. Can the regrid tool take any of
the
> NCEP
> > > > grids found on the following website (
> > > > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and
apply
> it
> > to
> > > > a netcdf file (I ask becuase 209 had 'See GRIB Specifications'
by
> it)?
> > > > Also, right now we reverse the latitude indices so that
everything is
> > > right
> > > > side up but this makes it so another product we use can't view
the
> .nc
> > > > files. So my question to you is would we be able to do reverse
the
> > > > latitudes using the MET regrid tool with NCEP grid 209 or
would we
> need
> > > to
> > > > use the regrid tool twice, once with grid 004 and once with
grid 209,
> > or
> > > > would another NCEP grid work?
> > > >
> > > > Oh, and, could you direct me to some documentation on the
regrid
> tool.
> > I
> > > > can see the usage from the command line but that doesn't
really
> address
> > > my
> > > > above question. Also, I didn't see anything in the MET manual.
> > > >
> > > > Thanks,
> > > > Tanya
> > > >
> > > >
> > > >
> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
> > > > met_help at ucar.edu <javascript:;>> wrote:
> > > >
> > > >> Tanya,
> > > >>
> > > >> I took a close look at your current data file:
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >>
> > > >> Running it through the debugger, I see that the problem comes
from
> > > parsing
> > > >> this time string:
> > > >> "hours since 2005-05-01 (12:00)"
> > > >>
> > > >> The parsing code is expecting that to be in HH:MM:SS format,
not
> > > HH:MM. I
> > > >> changed it to this:
> > > >> "hours since 2005-05-01 12:00:00"
> > > >>
> > > >> And then I was able to run the plot_data_plane tool on it.
However,
> > the
> > > >> resulting image is still upside-down and backwards. See
attached
> > > >> (test.png).
> > > >>
> > > >> To answer the question about grid x/y mapping, the answer is
no.
> For
> > > >> lat/lon grids with no projection info defined, MET uses the
lat/lon
> > > values
> > > >> and intuits the projection info. For non-lat/lon grids, the
> > projection
> > > >> info would need to be explicitly defined since there's no
simple way
> > of
> > > >> intuiting a grid definition from latitude and longitude
values.
> > > >>
> > > >> So about the data being upside-down and backwards... there
are flags
> > in
> > > >> GRIB files which indicate the order in which the data is
written.
> > When
> > > >> reading data from GRIB files, MET uses those flags to
determine
> where
> > to
> > > >> place things and orient the data right-side-up. The CF-
compliant
> > > NetCDF
> > > >> code logic is much simpler... it just reads the data in the
order it
> > is
> > > >> encoded in the file and doesn't check which way is up!
> > > >>
> > > >> I'm not really sure what to do here. Our support for CF-
compliant
> > > NetCDF
> > > >> is still relatively new, and I'm not sure what the
expectations
> should
> > > be.
> > > >> Should we add logic to the code to detect this situation and
handle
> it
> > > >> better? Or is the code working as expected? I'm not really
sure.
> > > >>
> > > >> I do see that ncview handles it better!
> > > >>
> > > >> With the current version of MET, you have 2 choices:
> > > >>
> > > >> (1) You could write lat/lon and data values in reverse order
(i.e.
> > > >> increasing latitude values) to "right the ship".
> > > >>
> > > >> (2) Or you could leave things as-is and make use of the
automated
> > > >> regridding capability within the MET tools.
> > > >>
> > > >> For example:
> > > >>
> > > >> # run regrid_data_plane utility to regrid to NCEP grid 4
> > > >> regrid_data_plane \
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > > >> -field 'name="T2"; level="(*,*)";'
> > > >>
> > > >> # run plot_data_plane on the output
> > > >> plot_data_plane \
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> > > >>
> > > >> The result (test_regrid.png) is attached.
> > > >>
> > > >> John
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA Affiliate
via
> RT <
> > > >> met_help at ucar.edu <javascript:;>> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > > >> >
> > > >> > John,
> > > >> >
> > > >> > My colleague, Jason English (cc'd on email), that is
generating
> the
> > > >> > NETCF_NCCF files has another question about the CF format.
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >> > Do we need the grid_mapping x and y variables if we are
using the
> > > >> lambert
> > > >> > cylindrical equal area grid?
> > > >> > If so, how do we convert 1-dimensional lat/lon coordinates
to the
> 2d
> > > >> > coordinates with y,x?
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >> >
> > > >> > Thank you,
> > > >> > Tanya Peevey
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA
Affiliate <
> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > >> >
> > > >> > > John,
> > > >> > >
> > > >> > > Thank you for the information. We generated a new file
and I
> tried
> > > the
> > > >> > > plot_data_plane tool on it and got the following error.
Do you
> > have
> > > an
> > > >> > idea
> > > >> > > of what could be wrong? Sorry to bug you so much and
thanks in
> > > advance
> > > >> > for
> > > >> > > the help.
> > > >> > >
> > > >> > > Here's the location of the file:
> > > >> > >
> > > >>
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > >
> > > >> > > Here's the error:
> > > >> > >
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > > >> > >
data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > >> > > file_type=NETCDF_NCCF;'
> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > ERROR :
> > > >> > > ERROR : StringArray::operator[](int) const -> range
check
> error!
> > > >> > > ERROR :
> > > >> > >
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > > >> > >
> > > >> > > Thank you ,
> > > >> > > Tanya
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via RT
<
> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > >
> > > >> > >> Tanya,
> > > >> > >>
> > > >> > >> Sorry for the delay. I was out of the office on Friday.
Using
> > > >> "LSP_6h"
> > > >> > >> for the name and "(*,*)" for the level should do the
trick.
> > > >> > >>
> > > >> > >> When trying to get the MET tools to read a new dataset,
I
> always
> > > >> start
> > > >> > by
> > > >> > >> using the plot_data_plane tool:
> > > >> > >>
> > > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > file_type=NETCDF_NCCF;'
> > > >> > >>
> > > >> > >> I use that to make sure the name and level settings work
as
> > > expected,
> > > >> > and
> > > >> > >> it also creates an output plot so I can see that MET is
(or is
> > not)
> > > >> > >> reading/plotting the data as expected.
> > > >> > >>
> > > >> > >> That configuration string you pass to the the
plot_data_plane
> > tool
> > > is
> > > >> > >> exactly the same settings as are supported in the
configuration
> > > >> files.
> > > >> > In
> > > >> > >> fact, we read that string with the same code that parses
the
> > config
> > > >> > files.
> > > >> > >>
> > > >> > >> I find using plot_data_plane very helpful.
> > > >> > >>
> > > >> > >> Thanks,
> > > >> > >> John
> > > >> > >>
> > > >> > >>
> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
Affiliate
> via
> > > RT
> > > >> <
> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> > > >> > >>
> > > >> > >> >
> > > >> > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > >
> > > >> > >> >
> > > >> > >> > John,
> > > >> > >> >
> > > >> > >> > Thanks for the info. We are going to work on changing
our
> file
> > > >> format
> > > >> > >> and
> > > >> > >> > then I'll get back to you.
> > > >> > >> > Did you see my other question about how I specified
'name'
> andhow to round to nearest 0.5
> > > >> 'level'
> > > >> > >> for
> > > >> > >> > the precip. field in my config file for MODE? Was how
I did
> it
> > > ok?
> > > >> I
> > > >> > ask
> > > >> > >> > because the manual was confusing on this point. If my
syntax
> > was
> > > >> > >> incorrect
> > > >> > >> > could you please email me the correction.
> > > >> > >> >
> > > >> > >> > Thanks,
> > > >> > >> > Tanya
> > > >> > >> >
> > > >> > >> >
> > > >> > >> >
> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway
via RT <
> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > >> >
> > > >> > >> > > Tanya,
> > > >> > >> > >
> > > >> > >> > > Here's a website about it:
> > > >> > >> > > http://cfconventions.org/latest.html
> > > >> > >> > >
> > > >> > >> > > And I posted a sample CF-compliant NetCDF file here:
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > > >> > >> > >
> > > >> > >> > > The relevant pieces in there are...
> > > >> > >> > > - The global "Conventions" attribute set to "CF-
...".
> > > >> > >> > > - The time dimension/variable define the forecast
valid
> time.
> > > >> > >> > > - The forecast_reference_time variable defines the
model
> > > >> > >> initialization
> > > >> > >> > > time.
> > > >> > >> > > - The grid_mapping, x, and y variables define the
grid
> > > definition
> > > >> > >> > > information.
> > > >> > >> > > - The grid_mapping variable attributes map those
variables
> to
> > > the
> > > >> > >> > relevant
> > > >> > >> > > grid definition.
> > > >> > >> > >
> > > >> > >> > > Hope this helps.
> > > >> > >> > >
> > > >> > >> > > Thanks,
> > > >> > >> > > John
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey - NOAA
> > Affiliate
> > > >> via
> > > >> > RT
> > > >> > >> <
> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > >> > >
> > > >> > >> > > >
> > > >> > >> > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > >> >
> > > >> > >> > > >
> > > >> > >> > > > John,
> > > >> > >> > > >
> > > >> > >> > > > To clarify my previous request of information. I
meant to
> > ask
> > > >> was
> > > >> > >> the
> > > >> > >> > > > following: Do you know of any good documentation
on how
> to
> > > >> make a
> > > >> > >> > NetCDF
> > > >> > >> > > > file CF-compliant? Also, do you have an example
.nc file
> > that
> > > >> is
> > > >> > >> NetCDF
> > > >> > >> > > > CF-compliant that we could compare to?
> > > >> > >> > > >
> > > >> > >> > > > Thanks,
> > > >> > >> > > > Tanya
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey -
NOAA
> > > Affiliate <
> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > >> > >> > > >
> > > >> > >> > > > > John,
> > > >> > >> > > > >
> > > >> > >> > > > > We generated the NetCDF files ourselves so we
could
> > > probably
> > > >> > make
> > > >> > >> it
> > > >> > >> > > work
> > > >> > >> > > > > for MET. Do you know how to get it into CF-
compliant
> > NetCDF
> > > >> > >> format?
> > > >> > >> > > > > Yes, we have grib files but the reason we
created
> NetCDF
> > > >> files
> > > >> > was
> > > >> > >> > > > because
> > > >> > >> > > > > original the NR... file had accumulated precip
over the
> > > whole
> > > >> > >> > forecast
> > > >> > >> > > > > where the other file has 6h accumulation data,
so we
> need
> > > to
> > > >> > >> convert
> > > >> > >> > > the
> > > >> > >> > > > > NR... precip information into 6h accumulation
(they had
> > > >> > different
> > > >> > >> > units
> > > >> > >> > > > > too, so we need to change that too). Since we
had to
> > change
> > > >> the
> > > >> > >> units
> > > >> > >> > > > too I
> > > >> > >> > > > > think use generating our own NetCDF file is best
but we
> > > just
> > > >> > need
> > > >> > >> to
> > > >> > >> > > get
> > > >> > >> > > > it
> > > >> > >> > > > > in the right format.
> > > >> > >> > > > > What is your opinion on the best way to approach
this?
> > > >> > >> > > > >
> > > >> > >> > > > > If we get it into the right format is how I
specify the
> > > >> 'name'
> > > >> > and
> > > >> > >> > > > 'level'
> > > >> > >> > > > > of the field correct? It was hard to figure out
the
> > correct
> > > >> > >> notation
> > > >> > >> > > from
> > > >> > >> > > > > the manual.
> > > >> > >> > > > >
> > > >> > >> > > > > Thanks,
> > > >> > >> > > > > Tanya
> > > >> > >> > > > >
> > > >> > >> > > > >
> > > >> > >> > > > >
> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley
Gotway via
> > RT <
> > > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > >> > > > >
> > > >> > >> > > > >> Hi Tanya,
> > > >> > >> > > > >>
> > > >> > >> > > > >> I took a look at the gridded NetCDF data you're
trying
> > to
> > > >> pass
> > > >> > to
> > > >> > >> > > MODE.
> > > >> > >> > > > >> Unfortunately, MET doesn't support all types of
> gridded
> > > >> NetCDF
> > > >> > >> > files.
> > > >> > >> > > > >> Instead, it only supports the MET NetCDF format
(e.g.
> > the
> > > >> > output
> > > >> > >> of
> > > >> > >> > > the
> > > >> > >> > > > >> pcp_combine tool), the output of the WRF
pinterp (or
> > > >> wrfinterp)
> > > >> > >> > > > utilities,
> > > >> > >> > > > >> and CF-compliant NetCDF files.
> > > >> > >> > > > >>
> > > >> > >> > > > >> Looking at "
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> > "
> > > I
> > > >> see
> > > >> > >> that
> > > >> > >> > > it
> > > >> > >> > > > >> follows none of those conventions. However, it
is
> > closest
> > > >> to
> > > >> > >> > > > CF-compliant
> > > >> > >> > > > >> NetCDF.
> > > >> > >> > > > >>
> > > >> > >> > > > >> When using a new data file format, instead of
running
> it
> > > >> > through
> > > >> > >> > MODE,
> > > >> > >> > > > >> I'd suggest using plot_data_plane to make sure
MET is
> > > >> reading
> > > >> > it
> > > >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as
follows:
> > > >> > >> > > > >>
> > > >> > >> > > > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
> > > >> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > >> > file_type=NETCDF_NCCF;'
> > > >> > >> > > > >>
> > > >> > >> > > > >> The resulting image is attached. There are
several
> > issues
> > > >> > here:
> > > >> > >> > > > >> - I had to tell plot_data_plane to interpret
this as
> > > >> > CF-compliant
> > > >> > >> > > NetCDF
> > > >> > >> > > > >> using NETCDF_NCCF.
> > > >> > >> > > > >> - The timing info in that file isn't specified
in the
> > > >> > >> CF-compliant
> > > >> > >> > > > way...
> > > >> > >> > > > >> so MET can't discern the initialization, valid,
and
> lead
> > > >> times.
> > > >> > >> > > > >> - And then there's the obvious problem (in the
image)
> of
> > > the
> > > >> > >> world
> > > >> > >> > > being
> > > >> > >> > > > >> upside down and backwards!
> > > >> > >> > > > >>
> > > >> > >> > > > >> So while we did get *something*, there are
obviously a
> > lot
> > > >> of
> > > >> > >> > issues.
> > > >> > >> > > > >> Basically, MET doesn't support the gridded
NetCDF file
> > > >> format
> > > >> > you
> > > >> > >> > are
> > > >> > >> > > > >> using. Is this data possibly available in
GRIB? Or
> do
> > > you
> > > >> > have
> > > >> > >> > > control
> > > >> > >> > > > >> over the NetCDF file format in use?
> > > >> > >> > > > >>
> > > >> > >> > > > >> Thanks,
> > > >> > >> > > > >> John
> > > >> > >> > > > >>
> > > >> > >> > > > >>
> > > >> > >> > > > >>
> > > >> > >> > > > >>
> > > >> > >> > > > >
> > > >> > >> > > > >
> > > >> > >> > > > > --
> > > >> > >> > > > > Tanya R. Peevey, PhD
> > > >> > >> > > > > Research Scientist I, Global Observing Systems
Analysis
> > > >> (GOSA)
> > > >> > >> Group
> > > >> > >> > > > > NOAA ESRL Global Systems Division
> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> > > >> > >> > > > > (303) 497-5847
> > > >> > >> > > > >
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > > > --
> > > >> > >> > > > Tanya R. Peevey, PhD
> > > >> > >> > > > Research Scientist I, Global Observing Systems
Analysis
> > > (GOSA)
> > > >> > Group
> > > >> > >> > > > NOAA ESRL Global Systems Division
> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> > > >> > >> > > > (303) 497-5847
> > > >> > >> > > >
> > > >> > >> > > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >> >
> > > >> > >> > --
> > > >> > >> > Tanya R. Peevey, PhD
> > > >> > >> > Research Scientist I, Global Observing Systems
Analysis
> (GOSA)
> > > >> Group
> > > >> > >> > NOAA ESRL Global Systems Division
> > > >> > >> > 325 Broadway, Boulder, CO 80305
> > > >> > >> > (303) 497-5847
> > > >> > >> >
> > > >> > >> >
> > > >> > >>
> > > >> > >>
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Tanya R. Peevey, PhD
> > > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > Group
> > > >> > > NOAA ESRL Global Systems Division
> > > >> > > 325 Broadway, Boulder, CO 80305
> > > >> > > (303) 497-5847
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Tanya R. Peevey, PhD
> > > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> > > >> > NOAA ESRL Global Systems Division
> > > >> > 325 Broadway, Boulder, CO 80305
> > > >> > (303) 497-5847
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > Tanya R. Peevey, PhD
> > > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > > NOAA ESRL Global Systems Division
> > > > 325 Broadway, Boulder, CO 80305
> > > > (303) 497-5847
> > > >
> > >
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> > >
> >
> >
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Fri Feb 12 12:00:16 2016
Tanya,
On to more issues. I think that not having the grid_mapping should
work
fine, but only because this is lat/lon data and MET can use the
lat/lon
variables to infer the grid.
For example, the following commands works fine:
/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
NR2006.pl.1by1.7278.2006022818.cf.nc \
~/without_grid_map.ps \
'name="LSP_6h"; level="(*,*)";'
Next, looking in your MODE output file:
/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
I see that the lead time is listed as 3169835842. The units are
supposed
to be HHMMSS so this string looks pretty odd! So there's still an
issue
with the times.
John
On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Tanya,
>
> I took a close look at the issue with plotting the map for grid 209.
The
> issue is caused by the map data straddling the international date
line. I
> tried rescaling the longitudes from -180,180 to 0,360 but wasn't
able to
> resolve this issue. So this is not easily fixed. I'll need to have
Randy
> Bullock look for a solution.
>
> In the meantime, you have a couple options...
> - If you're not tied to grid 209, you could easily switch to using a
> different grid.
> - Or you could set up the config file to plot a set of map data
files
> which does not include Russia.
>
> Take a look in met-5.1/share/met/config/ConfigMap. The "map_data"
section
> controls the map data files to be plotted. I've attached a map data
file
> to this message from which I've removed all the Russia data:
> country_data_minus_russia
>
> Plotting on this map data file results in the attached plot of grid
209.
>
> FYI, you can just copy and paste the "map_data" config file section
to the
> bottom of your MODE config file. And then set the path to the
attached map
> data file. That'll overwrite the default values set in
ConfigMapData.
>
> John
>
>
> On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>>
>> John,
>>
>> Ok, tried G209 in the new MODE and I got results but the lines on
the map
>> that are suppose to define Russia are all still wrong. See
following
>> directory,
>>
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
>>
>> So the above worked with a NetCDF file we generated that has CF-
compliant
>> time and grid_mapping information. We found out that the
grid_mapping
>> variable in the CF-compliant file was what was not allowing us to
use
>> Panoply (a package for quickly viewing data in an NetCDF file). So
we
>> removed the grid_mapping information and tried again, to see if
MODE would
>> work and it did not. Here are where the two files are location
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
>> with *.cf.nc being the file without grid_mapping and *.old_cf.nc
being
>> the
>> file with grid_mapping.
>>
>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
>> data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
>> data/gfs_test/
>> NR2006.pl.1by1.7188.2006022500.cf.nc
gfs_test/config/MODEConfig_gfstest
>> -outdir gfs_test/out/mode/ -v 2
>> DEBUG 1: Default Config File:
>> /scratch4/BMC/dtc/MET/met-5.1/share/met/config/MODEConfig_default
>> DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
>> DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
>> ERROR :
>> ERROR : Trouble reading forecast file "data/gfs_test/
>> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
>> ERROR :
>> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
>>
>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> Basically, want to try and fix the regridding to the G209 grid so
Russia
>> looks normal and want to confirm that we do need to use the
*.old_cf.nc
>> files instead of the other within MET.
>>
>> Thanks,
>> Tanya
>>
>> P.S. Having the regird option in MODE is great. Now I don't have to
create
>> another set of files before processing the data. Saves me some disk
space
>> :-).
>>
>>
>> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Tanya,
>> >
>> > I can look more closely tomorrow, but some initial thoughts are:
>> >
>> > - yes, please use met-5.1 and use the default mode 5.1 config
file
>> > - you could use the regrid tool, but instead fill out the regrid
>> section of
>> > the mode config file. Try setting to_grid = "G209";
>> >
>> > That'll automatically regrid the global data in the fly.
>> >
>> > That's all new in 5.1.
>> >
>> > John
>> >
>> > On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate via
RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > >
>> > > John,
>> > >
>> > > To clarify my previous email, I'm trying to use the regrid tool
to
>> > reverse
>> > > the latitudes in our .nc files and focus on North America
rather than
>> the
>> > > whole global. I just tried the regrid tool with the grid set to
G209
>> and
>> > it
>> > > worked (both corrected the latitude issue and focused on North
>> America)
>> > but
>> > > with one small problem, there are some weird lines where Russia
is
>> > suppose
>> > > to be. Do you know what the problem is? Is there another grid
you
>> could
>> > > recommend if the output from the regrid tool on the G209 grid
can't be
>> > > fixed? Or could I define my own grid using another MET tool?
>> > >
>> > > If all of the above fails we can modify our NetCDF files but I
hope
>> to be
>> > > able to use the regrid tool.
>> > >
>> > > All plots from my tests with the regrid tool can be found here:
>> > >
>> > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
>> > > Data used to make the plots can be found here:
>> > >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
>> > >
>> > > Still would like to see some documentation on the regrid tool.
>> > >
>> > > Lastly, after using the regrid tool I had to use the newest
version of
>> > MET
>> > > (met-5.1). Is this version good to go? Should I change my MET
path to
>> > this
>> > > version? Is there documentation on this version yet?
>> > >
>> > > Thanks!
>> > > Tanya
>> > >
>> > >
>> > >
>> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA Affiliate
<
>> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
>> > >
>> > > > John,
>> > > >
>> > > > So Jason and I got our data working with MODE. However, I
have one
>> > > > question. Right now MODE automatically plots the full grid
>> available,
>> > in
>> > > > this case a global plot. We are only focused on North
America. I was
>> > > > thinking of using the regrid tool to change the grid from
global to
>> > North
>> > > > American using NCEP grid 209. Can the regrid tool take any of
the
>> NCEP
>> > > > grids found on the following website (
>> > > > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and
apply
>> it
>> > to
>> > > > a netcdf file (I ask becuase 209 had 'See GRIB
Specifications' by
>> it)?
>> > > > Also, right now we reverse the latitude indices so that
everything
>> is
>> > > right
>> > > > side up but this makes it so another product we use can't
view the
>> .nc
>> > > > files. So my question to you is would we be able to do
reverse the
>> > > > latitudes using the MET regrid tool with NCEP grid 209 or
would we
>> need
>> > > to
>> > > > use the regrid tool twice, once with grid 004 and once with
grid
>> 209,
>> > or
>> > > > would another NCEP grid work?
>> > > >
>> > > > Oh, and, could you direct me to some documentation on the
regrid
>> tool.
>> > I
>> > > > can see the usage from the command line but that doesn't
really
>> address
>> > > my
>> > > > above question. Also, I didn't see anything in the MET
manual.
>> > > >
>> > > > Thanks,
>> > > > Tanya
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT <
>> > > > met_help at ucar.edu <javascript:;>> wrote:
>> > > >
>> > > >> Tanya,
>> > > >>
>> > > >> I took a close look at your current data file:
>> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > >>
>> > > >> Running it through the debugger, I see that the problem
comes from
>> > > parsing
>> > > >> this time string:
>> > > >> "hours since 2005-05-01 (12:00)"
>> > > >>
>> > > >> The parsing code is expecting that to be in HH:MM:SS format,
not
>> > > HH:MM. I
>> > > >> changed it to this:
>> > > >> "hours since 2005-05-01 12:00:00"
>> > > >>
>> > > >> And then I was able to run the plot_data_plane tool on it.
>> However,
>> > the
>> > > >> resulting image is still upside-down and backwards. See
attached
>> > > >> (test.png).
>> > > >>
>> > > >> To answer the question about grid x/y mapping, the answer is
no.
>> For
>> > > >> lat/lon grids with no projection info defined, MET uses the
lat/lon
>> > > values
>> > > >> and intuits the projection info. For non-lat/lon grids, the
>> > projection
>> > > >> info would need to be explicitly defined since there's no
simple
>> way
>> > of
>> > > >> intuiting a grid definition from latitude and longitude
values.
>> > > >>
>> > > >> So about the data being upside-down and backwards... there
are
>> flags
>> > in
>> > > >> GRIB files which indicate the order in which the data is
written.
>> > When
>> > > >> reading data from GRIB files, MET uses those flags to
determine
>> where
>> > to
>> > > >> place things and orient the data right-side-up. The CF-
compliant
>> > > NetCDF
>> > > >> code logic is much simpler... it just reads the data in the
order
>> it
>> > is
>> > > >> encoded in the file and doesn't check which way is up!
>> > > >>
>> > > >> I'm not really sure what to do here. Our support for CF-
compliant
>> > > NetCDF
>> > > >> is still relatively new, and I'm not sure what the
expectations
>> should
>> > > be.
>> > > >> Should we add logic to the code to detect this situation and
>> handle it
>> > > >> better? Or is the code working as expected? I'm not really
sure.
>> > > >>
>> > > >> I do see that ncview handles it better!
>> > > >>
>> > > >> With the current version of MET, you have 2 choices:
>> > > >>
>> > > >> (1) You could write lat/lon and data values in reverse order
(i.e.
>> > > >> increasing latitude values) to "right the ship".
>> > > >>
>> > > >> (2) Or you could leave things as-is and make use of the
automated
>> > > >> regridding capability within the MET tools.
>> > > >>
>> > > >> For example:
>> > > >>
>> > > >> # run regrid_data_plane utility to regrid to NCEP grid 4
>> > > >> regrid_data_plane \
>> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
>> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> > > >> -field 'name="T2"; level="(*,*)";'
>> > > >>
>> > > >> # run plot_data_plane on the output
>> > > >> plot_data_plane \
>> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
>> > > >>
>> > > >> The result (test_regrid.png) is attached.
>> > > >>
>> > > >> John
>> > > >>
>> > > >>
>> > > >>
>> > > >>
>> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
Affiliate via
>> RT <
>> > > >> met_help at ucar.edu <javascript:;>> wrote:
>> > > >>
>> > > >> >
>> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > > >> >
>> > > >> > John,
>> > > >> >
>> > > >> > My colleague, Jason English (cc'd on email), that is
generating
>> the
>> > > >> > NETCF_NCCF files has another question about the CF format.
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > >> > Do we need the grid_mapping x and y variables if we are
using the
>> > > >> lambert
>> > > >> > cylindrical equal area grid?
>> > > >> > If so, how do we convert 1-dimensional lat/lon coordinates
to
>> the 2d
>> > > >> > coordinates with y,x?
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > >> >
>> > > >> > Thank you,
>> > > >> > Tanya Peevey
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA
Affiliate <
>> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
>> > > >> >
>> > > >> > > John,
>> > > >> > >
>> > > >> > > Thank you for the information. We generated a new file
and I
>> tried
>> > > the
>> > > >> > > plot_data_plane tool on it and got the following error.
Do you
>> > have
>> > > an
>> > > >> > idea
>> > > >> > > of what could be wrong? Sorry to bug you so much and
thanks in
>> > > advance
>> > > >> > for
>> > > >> > > the help.
>> > > >> > >
>> > > >> > > Here's the location of the file:
>> > > >> > >
>> > > >>
>> > >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
>> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > >> > >
>> > > >> > > Here's the error:
>> > > >> > >
>> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
>> > > >> > >
data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > > >> > > file_type=NETCDF_NCCF;'
>> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
>> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > >> > > ERROR :
>> > > >> > > ERROR : StringArray::operator[](int) const -> range
check
>> error!
>> > > >> > > ERROR :
>> > > >> > >
>> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > > >> > >
>> > > >> > > Thank you ,
>> > > >> > > Tanya
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via
RT <
>> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
>> > > >> > >
>> > > >> > >> Tanya,
>> > > >> > >>
>> > > >> > >> Sorry for the delay. I was out of the office on
Friday.
>> Using
>> > > >> "LSP_6h"
>> > > >> > >> for the name and "(*,*)" for the level should do the
trick.
>> > > >> > >>
>> > > >> > >> When trying to get the MET tools to read a new dataset,
I
>> always
>> > > >> start
>> > > >> > by
>> > > >> > >> using the plot_data_plane tool:
>> > > >> > >>
>> > > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > > file_type=NETCDF_NCCF;'
>> > > >> > >>
>> > > >> > >> I use that to make sure the name and level settings
work as
>> > > expected,
>> > > >> > and
>> > > >> > >> it also creates an output plot so I can see that MET is
(or is
>> > not)
>> > > >> > >> reading/plotting the data as expected.
>> > > >> > >>
>> > > >> > >> That configuration string you pass to the the
plot_data_plane
>> > tool
>> > > is
>> > > >> > >> exactly the same settings as are supported in the
>> configuration
>> > > >> files.
>> > > >> > In
>> > > >> > >> fact, we read that string with the same code that
parses the
>> > config
>> > > >> > files.
>> > > >> > >>
>> > > >> > >> I find using plot_data_plane very helpful.
>> > > >> > >>
>> > > >> > >> Thanks,
>> > > >> > >> John
>> > > >> > >>
>> > > >> > >>
>> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
Affiliate
>> via
>> > > RT
>> > > >> <
>> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
>> > > >> > >>
>> > > >> > >> >
>> > > >> > >> > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> > >
>> > > >> > >> >
>> > > >> > >> > John,
>> > > >> > >> >
>> > > >> > >> > Thanks for the info. We are going to work on changing
our
>> file
>> > > >> format
>> > > >> > >> and
>> > > >> > >> > then I'll get back to you.
>> > > >> > >> > Did you see my other question about how I specified
'name'
>> andhow to round to nearest 0.5
>>
>> > > >> 'level'
>> > > >> > >> for
>> > > >> > >> > the precip. field in my config file for MODE? Was how
I did
>> it
>> > > ok?
>> > > >> I
>> > > >> > ask
>> > > >> > >> > because the manual was confusing on this point. If my
syntax
>> > was
>> > > >> > >> incorrect
>> > > >> > >> > could you please email me the correction.
>> > > >> > >> >
>> > > >> > >> > Thanks,
>> > > >> > >> > Tanya
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway
via RT <
>> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
>> > > >> > >> >
>> > > >> > >> > > Tanya,
>> > > >> > >> > >
>> > > >> > >> > > Here's a website about it:
>> > > >> > >> > > http://cfconventions.org/latest.html
>> > > >> > >> > >
>> > > >> > >> > > And I posted a sample CF-compliant NetCDF file
here:
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >>
>> > > >> >
>> > > >>
>> > >
>> >
>>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
>> > > >> > >> > >
>> > > >> > >> > > The relevant pieces in there are...
>> > > >> > >> > > - The global "Conventions" attribute set to "CF-
...".
>> > > >> > >> > > - The time dimension/variable define the forecast
valid
>> time.
>> > > >> > >> > > - The forecast_reference_time variable defines the
model
>> > > >> > >> initialization
>> > > >> > >> > > time.
>> > > >> > >> > > - The grid_mapping, x, and y variables define the
grid
>> > > definition
>> > > >> > >> > > information.
>> > > >> > >> > > - The grid_mapping variable attributes map those
>> variables to
>> > > the
>> > > >> > >> > relevant
>> > > >> > >> > > grid definition.
>> > > >> > >> > >
>> > > >> > >> > > Hope this helps.
>> > > >> > >> > >
>> > > >> > >> > > Thanks,
>> > > >> > >> > > John
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey -
NOAA
>> > Affiliate
>> > > >> via
>> > > >> > RT
>> > > >> > >> <
>> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
>> > > >> > >> > >
>> > > >> > >> > > >
>> > > >> > >> > > > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> > > >> >
>> > > >> > >> > > >
>> > > >> > >> > > > John,
>> > > >> > >> > > >
>> > > >> > >> > > > To clarify my previous request of information. I
meant
>> to
>> > ask
>> > > >> was
>> > > >> > >> the
>> > > >> > >> > > > following: Do you know of any good documentation
on how
>> to
>> > > >> make a
>> > > >> > >> > NetCDF
>> > > >> > >> > > > file CF-compliant? Also, do you have an example
.nc file
>> > that
>> > > >> is
>> > > >> > >> NetCDF
>> > > >> > >> > > > CF-compliant that we could compare to?
>> > > >> > >> > > >
>> > > >> > >> > > > Thanks,
>> > > >> > >> > > > Tanya
>> > > >> > >> > > >
>> > > >> > >> > > >
>> > > >> > >> > > >
>> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey -
NOAA
>> > > Affiliate <
>> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
>> > > >> > >> > > >
>> > > >> > >> > > > > John,
>> > > >> > >> > > > >
>> > > >> > >> > > > > We generated the NetCDF files ourselves so we
could
>> > > probably
>> > > >> > make
>> > > >> > >> it
>> > > >> > >> > > work
>> > > >> > >> > > > > for MET. Do you know how to get it into CF-
compliant
>> > NetCDF
>> > > >> > >> format?
>> > > >> > >> > > > > Yes, we have grib files but the reason we
created
>> NetCDF
>> > > >> files
>> > > >> > was
>> > > >> > >> > > > because
>> > > >> > >> > > > > original the NR... file had accumulated precip
over
>> the
>> > > whole
>> > > >> > >> > forecast
>> > > >> > >> > > > > where the other file has 6h accumulation data,
so we
>> need
>> > > to
>> > > >> > >> convert
>> > > >> > >> > > the
>> > > >> > >> > > > > NR... precip information into 6h accumulation
(they
>> had
>> > > >> > different
>> > > >> > >> > units
>> > > >> > >> > > > > too, so we need to change that too). Since we
had to
>> > change
>> > > >> the
>> > > >> > >> units
>> > > >> > >> > > > too I
>> > > >> > >> > > > > think use generating our own NetCDF file is
best but
>> we
>> > > just
>> > > >> > need
>> > > >> > >> to
>> > > >> > >> > > get
>> > > >> > >> > > > it
>> > > >> > >> > > > > in the right format.
>> > > >> > >> > > > > What is your opinion on the best way to
approach this?
>> > > >> > >> > > > >
>> > > >> > >> > > > > If we get it into the right format is how I
specify
>> the
>> > > >> 'name'
>> > > >> > and
>> > > >> > >> > > > 'level'
>> > > >> > >> > > > > of the field correct? It was hard to figure out
the
>> > correct
>> > > >> > >> notation
>> > > >> > >> > > from
>> > > >> > >> > > > > the manual.
>> > > >> > >> > > > >
>> > > >> > >> > > > > Thanks,
>> > > >> > >> > > > > Tanya
>> > > >> > >> > > > >
>> > > >> > >> > > > >
>> > > >> > >> > > > >
>> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley
Gotway via
>> > RT <
>> > > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
>> > > >> > >> > > > >
>> > > >> > >> > > > >> Hi Tanya,
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> I took a look at the gridded NetCDF data
you're
>> trying
>> > to
>> > > >> pass
>> > > >> > to
>> > > >> > >> > > MODE.
>> > > >> > >> > > > >> Unfortunately, MET doesn't support all types
of
>> gridded
>> > > >> NetCDF
>> > > >> > >> > files.
>> > > >> > >> > > > >> Instead, it only supports the MET NetCDF
format (e.g.
>> > the
>> > > >> > output
>> > > >> > >> of
>> > > >> > >> > > the
>> > > >> > >> > > > >> pcp_combine tool), the output of the WRF
pinterp (or
>> > > >> wrfinterp)
>> > > >> > >> > > > utilities,
>> > > >> > >> > > > >> and CF-compliant NetCDF files.
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> Looking at "
>> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
>> > "
>> > > I
>> > > >> see
>> > > >> > >> that
>> > > >> > >> > > it
>> > > >> > >> > > > >> follows none of those conventions. However,
it is
>> > closest
>> > > >> to
>> > > >> > >> > > > CF-compliant
>> > > >> > >> > > > >> NetCDF.
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> When using a new data file format, instead of
>> running it
>> > > >> > through
>> > > >> > >> > MODE,
>> > > >> > >> > > > >> I'd suggest using plot_data_plane to make sure
MET is
>> > > >> reading
>> > > >> > it
>> > > >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as
follows:
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
>> > > >> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
>> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > > >> > file_type=NETCDF_NCCF;'
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> The resulting image is attached. There are
several
>> > issues
>> > > >> > here:
>> > > >> > >> > > > >> - I had to tell plot_data_plane to interpret
this as
>> > > >> > CF-compliant
>> > > >> > >> > > NetCDF
>> > > >> > >> > > > >> using NETCDF_NCCF.
>> > > >> > >> > > > >> - The timing info in that file isn't specified
in the
>> > > >> > >> CF-compliant
>> > > >> > >> > > > way...
>> > > >> > >> > > > >> so MET can't discern the initialization,
valid, and
>> lead
>> > > >> times.
>> > > >> > >> > > > >> - And then there's the obvious problem (in the
>> image) of
>> > > the
>> > > >> > >> world
>> > > >> > >> > > being
>> > > >> > >> > > > >> upside down and backwards!
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> So while we did get *something*, there are
obviously
>> a
>> > lot
>> > > >> of
>> > > >> > >> > issues.
>> > > >> > >> > > > >> Basically, MET doesn't support the gridded
NetCDF
>> file
>> > > >> format
>> > > >> > you
>> > > >> > >> > are
>> > > >> > >> > > > >> using. Is this data possibly available in
GRIB? Or
>> do
>> > > you
>> > > >> > have
>> > > >> > >> > > control
>> > > >> > >> > > > >> over the NetCDF file format in use?
>> > > >> > >> > > > >>
>> > > >> > >> > > > >> Thanks,
>> > > >> > >> > > > >> John
>> > > >> > >> > > > >>
>> > > >> > >> > > > >>
>> > > >> > >> > > > >>
>> > > >> > >> > > > >>
>> > > >> > >> > > > >
>> > > >> > >> > > > >
>> > > >> > >> > > > > --
>> > > >> > >> > > > > Tanya R. Peevey, PhD
>> > > >> > >> > > > > Research Scientist I, Global Observing Systems
>> Analysis
>> > > >> (GOSA)
>> > > >> > >> Group
>> > > >> > >> > > > > NOAA ESRL Global Systems Division
>> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
>> > > >> > >> > > > > (303) 497-5847
>> > > >> > >> > > > >
>> > > >> > >> > > >
>> > > >> > >> > > >
>> > > >> > >> > > >
>> > > >> > >> > > > --
>> > > >> > >> > > > Tanya R. Peevey, PhD
>> > > >> > >> > > > Research Scientist I, Global Observing Systems
Analysis
>> > > (GOSA)
>> > > >> > Group
>> > > >> > >> > > > NOAA ESRL Global Systems Division
>> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
>> > > >> > >> > > > (303) 497-5847
>> > > >> > >> > > >
>> > > >> > >> > > >
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >> > --
>> > > >> > >> > Tanya R. Peevey, PhD
>> > > >> > >> > Research Scientist I, Global Observing Systems
Analysis
>> (GOSA)
>> > > >> Group
>> > > >> > >> > NOAA ESRL Global Systems Division
>> > > >> > >> > 325 Broadway, Boulder, CO 80305
>> > > >> > >> > (303) 497-5847
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >>
>> > > >> > >>
>> > > >> > >
>> > > >> > >
>> > > >> > > --
>> > > >> > > Tanya R. Peevey, PhD
>> > > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> > Group
>> > > >> > > NOAA ESRL Global Systems Division
>> > > >> > > 325 Broadway, Boulder, CO 80305
>> > > >> > > (303) 497-5847
>> > > >> > >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > --
>> > > >> > Tanya R. Peevey, PhD
>> > > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> Group
>> > > >> > NOAA ESRL Global Systems Division
>> > > >> > 325 Broadway, Boulder, CO 80305
>> > > >> > (303) 497-5847
>> > > >> >
>> > > >> >
>> > > >>
>> > > >>
>> > > >
>> > > >
>> > > > --
>> > > > Tanya R. Peevey, PhD
>> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
>> > > > NOAA ESRL Global Systems Division
>> > > > 325 Broadway, Boulder, CO 80305
>> > > > (303) 497-5847
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Tanya R. Peevey, PhD
>> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> > > NOAA ESRL Global Systems Division
>> > > 325 Broadway, Boulder, CO 80305
>> > > (303) 497-5847
>> > >
>> > >
>> >
>> >
>>
>>
>> --
>> Tanya R. Peevey, PhD
>> Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> NOAA ESRL Global Systems Division
>> 325 Broadway, Boulder, CO 80305
>> (303) 497-5847
>>
>>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Tue Feb 16 10:59:09 2016
John,
Thanks for the information. We are going to look at the time
information
again today. Also, I found the issue with the last file I tried to use
in
MODE, the issue wasn't the grid_mapping but something else on our end.
I tried different grids on the *old_cf* files (the ones that worked
previously) and all of them had Russia looking funny. The grids I
tried
were G221 and G241, both North America grids. So basically the only
options
I have are CONUS, Northern Hemisphere or the Globe, none of which are
appealing. I also don't like the idea of excluding Russia but I'll
copy
over your code so I have it. In the meantime we will probably create a
subset of data over North America and then run that through MODE. As
soon
as you get the North American centric grids working please let me
know.
Lastly, if you know of another grid that would work please let me know
but
I believe I've tried all of the North American centric grids that are
of
~equivalent resolution of the original dataset (1x1 degree) and are
available in MODE.
Thanks,
Tanya
On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Tanya,
>
> On to more issues. I think that not having the grid_mapping should
work
> fine, but only because this is lat/lon data and MET can use the
lat/lon
> variables to infer the grid.
>
> For example, the following commands works fine:
>
> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
> NR2006.pl.1by1.7278.2006022818.cf.nc \
> ~/without_grid_map.ps \
> 'name="LSP_6h"; level="(*,*)";'
>
> Next, looking in your MODE output file:
>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
>
> I see that the lead time is listed as 3169835842. The units are
supposed
> to be HHMMSS so this string looks pretty odd! So there's still an
issue
> with the times.
>
> John
>
>
>
> On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Tanya,
> >
> > I took a close look at the issue with plotting the map for grid
209. The
> > issue is caused by the map data straddling the international date
line.
> I
> > tried rescaling the longitudes from -180,180 to 0,360 but wasn't
able to
> > resolve this issue. So this is not easily fixed. I'll need to
have
> Randy
> > Bullock look for a solution.
> >
> > In the meantime, you have a couple options...
> > - If you're not tied to grid 209, you could easily switch to using
a
> > different grid.
> > - Or you could set up the config file to plot a set of map data
files
> > which does not include Russia.
> >
> > Take a look in met-5.1/share/met/config/ConfigMap. The "map_data"
> section
> > controls the map data files to be plotted. I've attached a map
data file
> > to this message from which I've removed all the Russia data:
> > country_data_minus_russia
> >
> > Plotting on this map data file results in the attached plot of
grid 209.
> >
> > FYI, you can just copy and paste the "map_data" config file
section to
> the
> > bottom of your MODE config file. And then set the path to the
attached
> map
> > data file. That'll overwrite the default values set in
ConfigMapData.
> >
> > John
> >
> >
> > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >>
> >> John,
> >>
> >> Ok, tried G209 in the new MODE and I got results but the lines on
the
> map
> >> that are suppose to define Russia are all still wrong. See
following
> >> directory,
> >>
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
> >>
> >> So the above worked with a NetCDF file we generated that has
> CF-compliant
> >> time and grid_mapping information. We found out that the
grid_mapping
> >> variable in the CF-compliant file was what was not allowing us to
use
> >> Panoply (a package for quickly viewing data in an NetCDF file).
So we
> >> removed the grid_mapping information and tried again, to see if
MODE
> would
> >> work and it did not. Here are where the two files are location
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> >> with *.cf.nc being the file without grid_mapping and *.old_cf.nc
being
> >> the
> >> file with grid_mapping.
> >>
> >>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>
> >>
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> >> data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> >> data/gfs_test/
> >> NR2006.pl.1by1.7188.2006022500.cf.nc
gfs_test/config/MODEConfig_gfstest
> >> -outdir gfs_test/out/mode/ -v 2
> >> DEBUG 1: Default Config File:
> >> /scratch4/BMC/dtc/MET/met-5.1/share/met/config/MODEConfig_default
> >> DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
> >> DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
> >> ERROR :
> >> ERROR : Trouble reading forecast file "data/gfs_test/
> >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> >> ERROR :
> >> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>
> >>
> >>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>
> >> Basically, want to try and fix the regridding to the G209 grid so
Russia
> >> looks normal and want to confirm that we do need to use the
*.old_cf.nc
> >> files instead of the other within MET.
> >>
> >> Thanks,
> >> Tanya
> >>
> >> P.S. Having the regird option in MODE is great. Now I don't have
to
> create
> >> another set of files before processing the data. Saves me some
disk
> space
> >> :-).
> >>
> >>
> >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Tanya,
> >> >
> >> > I can look more closely tomorrow, but some initial thoughts
are:
> >> >
> >> > - yes, please use met-5.1 and use the default mode 5.1 config
file
> >> > - you could use the regrid tool, but instead fill out the
regrid
> >> section of
> >> > the mode config file. Try setting to_grid = "G209";
> >> >
> >> > That'll automatically regrid the global data in the fly.
> >> >
> >> > That's all new in 5.1.
> >> >
> >> > John
> >> >
> >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate
via RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> >> > >
> >> > > John,
> >> > >
> >> > > To clarify my previous email, I'm trying to use the regrid
tool to
> >> > reverse
> >> > > the latitudes in our .nc files and focus on North America
rather
> than
> >> the
> >> > > whole global. I just tried the regrid tool with the grid set
to G209
> >> and
> >> > it
> >> > > worked (both corrected the latitude issue and focused on
North
> >> America)
> >> > but
> >> > > with one small problem, there are some weird lines where
Russia is
> >> > suppose
> >> > > to be. Do you know what the problem is? Is there another grid
you
> >> could
> >> > > recommend if the output from the regrid tool on the G209 grid
can't
> be
> >> > > fixed? Or could I define my own grid using another MET tool?
> >> > >
> >> > > If all of the above fails we can modify our NetCDF files but
I hope
> >> to be
> >> > > able to use the regrid tool.
> >> > >
> >> > > All plots from my tests with the regrid tool can be found
here:
> >> > >
> >> > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> >> > > Data used to make the plots can be found here:
> >> > >
> >> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> >> > >
> >> > > Still would like to see some documentation on the regrid
tool.
> >> > >
> >> > > Lastly, after using the regrid tool I had to use the newest
version
> of
> >> > MET
> >> > > (met-5.1). Is this version good to go? Should I change my MET
path
> to
> >> > this
> >> > > version? Is there documentation on this version yet?
> >> > >
> >> > > Thanks!
> >> > > Tanya
> >> > >
> >> > >
> >> > >
> >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
Affiliate <
> >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > >
> >> > > > John,
> >> > > >
> >> > > > So Jason and I got our data working with MODE. However, I
have one
> >> > > > question. Right now MODE automatically plots the full grid
> >> available,
> >> > in
> >> > > > this case a global plot. We are only focused on North
America. I
> was
> >> > > > thinking of using the regrid tool to change the grid from
global
> to
> >> > North
> >> > > > American using NCEP grid 209. Can the regrid tool take any
of the
> >> NCEP
> >> > > > grids found on the following website (
> >> > > > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html)
and
> apply
> >> it
> >> > to
> >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
Specifications' by
> >> it)?
> >> > > > Also, right now we reverse the latitude indices so that
everything
> >> is
> >> > > right
> >> > > > side up but this makes it so another product we use can't
view the
> >> .nc
> >> > > > files. So my question to you is would we be able to do
reverse the
> >> > > > latitudes using the MET regrid tool with NCEP grid 209 or
would we
> >> need
> >> > > to
> >> > > > use the regrid tool twice, once with grid 004 and once with
grid
> >> 209,
> >> > or
> >> > > > would another NCEP grid work?
> >> > > >
> >> > > > Oh, and, could you direct me to some documentation on the
regrid
> >> tool.
> >> > I
> >> > > > can see the usage from the command line but that doesn't
really
> >> address
> >> > > my
> >> > > > above question. Also, I didn't see anything in the MET
manual.
> >> > > >
> >> > > > Thanks,
> >> > > > Tanya
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via RT
<
> >> > > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > >
> >> > > >> Tanya,
> >> > > >>
> >> > > >> I took a close look at your current data file:
> >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > >>
> >> > > >> Running it through the debugger, I see that the problem
comes
> from
> >> > > parsing
> >> > > >> this time string:
> >> > > >> "hours since 2005-05-01 (12:00)"
> >> > > >>
> >> > > >> The parsing code is expecting that to be in HH:MM:SS
format, not
> >> > > HH:MM. I
> >> > > >> changed it to this:
> >> > > >> "hours since 2005-05-01 12:00:00"
> >> > > >>
> >> > > >> And then I was able to run the plot_data_plane tool on it.
> >> However,
> >> > the
> >> > > >> resulting image is still upside-down and backwards. See
attached
> >> > > >> (test.png).
> >> > > >>
> >> > > >> To answer the question about grid x/y mapping, the answer
is no.
> >> For
> >> > > >> lat/lon grids with no projection info defined, MET uses
the
> lat/lon
> >> > > values
> >> > > >> and intuits the projection info. For non-lat/lon grids,
the
> >> > projection
> >> > > >> info would need to be explicitly defined since there's no
simple
> >> way
> >> > of
> >> > > >> intuiting a grid definition from latitude and longitude
values.
> >> > > >>
> >> > > >> So about the data being upside-down and backwards... there
are
> >> flags
> >> > in
> >> > > >> GRIB files which indicate the order in which the data is
written.
> >> > When
> >> > > >> reading data from GRIB files, MET uses those flags to
determine
> >> where
> >> > to
> >> > > >> place things and orient the data right-side-up. The
> CF-compliant
> >> > > NetCDF
> >> > > >> code logic is much simpler... it just reads the data in
the order
> >> it
> >> > is
> >> > > >> encoded in the file and doesn't check which way is up!
> >> > > >>
> >> > > >> I'm not really sure what to do here. Our support for
> CF-compliant
> >> > > NetCDF
> >> > > >> is still relatively new, and I'm not sure what the
expectations
> >> should
> >> > > be.
> >> > > >> Should we add logic to the code to detect this situation
and
> >> handle it
> >> > > >> better? Or is the code working as expected? I'm not
really
> sure.
> >> > > >>
> >> > > >> I do see that ncview handles it better!
> >> > > >>
> >> > > >> With the current version of MET, you have 2 choices:
> >> > > >>
> >> > > >> (1) You could write lat/lon and data values in reverse
order
> (i.e.
> >> > > >> increasing latitude values) to "right the ship".
> >> > > >>
> >> > > >> (2) Or you could leave things as-is and make use of the
automated
> >> > > >> regridding capability within the MET tools.
> >> > > >>
> >> > > >> For example:
> >> > > >>
> >> > > >> # run regrid_data_plane utility to regrid to NCEP grid 4
> >> > > >> regrid_data_plane \
> >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004 \
> >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> > > >> -field 'name="T2"; level="(*,*)";'
> >> > > >>
> >> > > >> # run plot_data_plane on the output
> >> > > >> plot_data_plane \
> >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> >> > > >>
> >> > > >> The result (test_regrid.png) is attached.
> >> > > >>
> >> > > >> John
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
Affiliate via
> >> RT <
> >> > > >> met_help at ucar.edu <javascript:;>> wrote:
> >> > > >>
> >> > > >> >
> >> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >
> >> > > >> >
> >> > > >> > John,
> >> > > >> >
> >> > > >> > My colleague, Jason English (cc'd on email), that is
generating
> >> the
> >> > > >> > NETCF_NCCF files has another question about the CF
format.
> >> > > >> >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > >> > Do we need the grid_mapping x and y variables if we are
using
> the
> >> > > >> lambert
> >> > > >> > cylindrical equal area grid?
> >> > > >> > If so, how do we convert 1-dimensional lat/lon
coordinates to
> >> the 2d
> >> > > >> > coordinates with y,x?
> >> > > >> >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > >> >
> >> > > >> > Thank you,
> >> > > >> > Tanya Peevey
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA
Affiliate <
> >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > > >> >
> >> > > >> > > John,
> >> > > >> > >
> >> > > >> > > Thank you for the information. We generated a new file
and I
> >> tried
> >> > > the
> >> > > >> > > plot_data_plane tool on it and got the following
error. Do
> you
> >> > have
> >> > > an
> >> > > >> > idea
> >> > > >> > > of what could be wrong? Sorry to bug you so much and
thanks
> in
> >> > > advance
> >> > > >> > for
> >> > > >> > > the help.
> >> > > >> > >
> >> > > >> > > Here's the location of the file:
> >> > > >> > >
> >> > > >>
> >> > >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > >> > >
> >> > > >> > > Here's the error:
> >> > > >> > >
> >> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> >> > > >> > >
data/gfs_test/prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > > >> > > file_type=NETCDF_NCCF;'
> >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > >> > > ERROR :
> >> > > >> > > ERROR : StringArray::operator[](int) const -> range
check
> >> error!
> >> > > >> > > ERROR :
> >> > > >> > >
> >> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >> > > >> > >
> >> > > >> > > Thank you ,
> >> > > >> > > Tanya
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway via
RT <
> >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > >> > >
> >> > > >> > >> Tanya,
> >> > > >> > >>
> >> > > >> > >> Sorry for the delay. I was out of the office on
Friday.
> >> Using
> >> > > >> "LSP_6h"
> >> > > >> > >> for the name and "(*,*)" for the level should do the
trick.
> >> > > >> > >>
> >> > > >> > >> When trying to get the MET tools to read a new
dataset, I
> >> always
> >> > > >> start
> >> > > >> > by
> >> > > >> > >> using the plot_data_plane tool:
> >> > > >> > >>
> >> > > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > > file_type=NETCDF_NCCF;'
> >> > > >> > >>
> >> > > >> > >> I use that to make sure the name and level settings
work as
> >> > > expected,
> >> > > >> > and
> >> > > >> > >> it also creates an output plot so I can see that MET
is (or
> is
> >> > not)
> >> > > >> > >> reading/plotting the data as expected.
> >> > > >> > >>
> >> > > >> > >> That configuration string you pass to the the
> plot_data_plane
> >> > tool
> >> > > is
> >> > > >> > >> exactly the same settings as are supported in the
> >> configuration
> >> > > >> files.
> >> > > >> > In
> >> > > >> > >> fact, we read that string with the same code that
parses the
> >> > config
> >> > > >> > files.
> >> > > >> > >>
> >> > > >> > >> I find using plot_data_plane very helpful.
> >> > > >> > >>
> >> > > >> > >> Thanks,
> >> > > >> > >> John
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
> Affiliate
> >> via
> >> > > RT
> >> > > >> <
> >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> >> > > >> > >>
> >> > > >> > >> >
> >> > > >> > >> > <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> > >
> >> > > >> > >> >
> >> > > >> > >> > John,
> >> > > >> > >> >
> >> > > >> > >> > Thanks for the info. We are going to work on
changing our
> >> file
> >> > > >> format
> >> > > >> > >> and
> >> > > >> > >> > then I'll get back to you.
> >> > > >> > >> > Did you see my other question about how I specified
'name'
> >> andhow to round to nearest 0.5
> >>
> >> > > >> 'level'
> >> > > >> > >> for
> >> > > >> > >> > the precip. field in my config file for MODE? Was
how I
> did
> >> it
> >> > > ok?
> >> > > >> I
> >> > > >> > ask
> >> > > >> > >> > because the manual was confusing on this point. If
my
> syntax
> >> > was
> >> > > >> > >> incorrect
> >> > > >> > >> > could you please email me the correction.
> >> > > >> > >> >
> >> > > >> > >> > Thanks,
> >> > > >> > >> > Tanya
> >> > > >> > >> >
> >> > > >> > >> >
> >> > > >> > >> >
> >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley Gotway
via
> RT <
> >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> >> > > >> > >> >
> >> > > >> > >> > > Tanya,
> >> > > >> > >> > >
> >> > > >> > >> > > Here's a website about it:
> >> > > >> > >> > > http://cfconventions.org/latest.html
> >> > > >> > >> > >
> >> > > >> > >> > > And I posted a sample CF-compliant NetCDF file
here:
> >> > > >> > >> > >
> >> > > >> > >> > >
> >> > > >> > >> >
> >> > > >> > >>
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> >> > > >> > >> > >
> >> > > >> > >> > > The relevant pieces in there are...
> >> > > >> > >> > > - The global "Conventions" attribute set to "CF-
...".
> >> > > >> > >> > > - The time dimension/variable define the forecast
valid
> >> time.
> >> > > >> > >> > > - The forecast_reference_time variable defines
the model
> >> > > >> > >> initialization
> >> > > >> > >> > > time.
> >> > > >> > >> > > - The grid_mapping, x, and y variables define the
grid
> >> > > definition
> >> > > >> > >> > > information.
> >> > > >> > >> > > - The grid_mapping variable attributes map those
> >> variables to
> >> > > the
> >> > > >> > >> > relevant
> >> > > >> > >> > > grid definition.
> >> > > >> > >> > >
> >> > > >> > >> > > Hope this helps.
> >> > > >> > >> > >
> >> > > >> > >> > > Thanks,
> >> > > >> > >> > > John
> >> > > >> > >> > >
> >> > > >> > >> > >
> >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey -
NOAA
> >> > Affiliate
> >> > > >> via
> >> > > >> > RT
> >> > > >> > >> <
> >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > >> > >> > >
> >> > > >> > >> > > >
> >> > > >> > >> > > > <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> > > >> >
> >> > > >> > >> > > >
> >> > > >> > >> > > > John,
> >> > > >> > >> > > >
> >> > > >> > >> > > > To clarify my previous request of information.
I meant
> >> to
> >> > ask
> >> > > >> was
> >> > > >> > >> the
> >> > > >> > >> > > > following: Do you know of any good
documentation on
> how
> >> to
> >> > > >> make a
> >> > > >> > >> > NetCDF
> >> > > >> > >> > > > file CF-compliant? Also, do you have an example
.nc
> file
> >> > that
> >> > > >> is
> >> > > >> > >> NetCDF
> >> > > >> > >> > > > CF-compliant that we could compare to?
> >> > > >> > >> > > >
> >> > > >> > >> > > > Thanks,
> >> > > >> > >> > > > Tanya
> >> > > >> > >> > > >
> >> > > >> > >> > > >
> >> > > >> > >> > > >
> >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey -
NOAA
> >> > > Affiliate <
> >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > > >> > >> > > >
> >> > > >> > >> > > > > John,
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > We generated the NetCDF files ourselves so we
could
> >> > > probably
> >> > > >> > make
> >> > > >> > >> it
> >> > > >> > >> > > work
> >> > > >> > >> > > > > for MET. Do you know how to get it into CF-
compliant
> >> > NetCDF
> >> > > >> > >> format?
> >> > > >> > >> > > > > Yes, we have grib files but the reason we
created
> >> NetCDF
> >> > > >> files
> >> > > >> > was
> >> > > >> > >> > > > because
> >> > > >> > >> > > > > original the NR... file had accumulated
precip over
> >> the
> >> > > whole
> >> > > >> > >> > forecast
> >> > > >> > >> > > > > where the other file has 6h accumulation
data, so we
> >> need
> >> > > to
> >> > > >> > >> convert
> >> > > >> > >> > > the
> >> > > >> > >> > > > > NR... precip information into 6h accumulation
(they
> >> had
> >> > > >> > different
> >> > > >> > >> > units
> >> > > >> > >> > > > > too, so we need to change that too). Since we
had to
> >> > change
> >> > > >> the
> >> > > >> > >> units
> >> > > >> > >> > > > too I
> >> > > >> > >> > > > > think use generating our own NetCDF file is
best but
> >> we
> >> > > just
> >> > > >> > need
> >> > > >> > >> to
> >> > > >> > >> > > get
> >> > > >> > >> > > > it
> >> > > >> > >> > > > > in the right format.
> >> > > >> > >> > > > > What is your opinion on the best way to
approach
> this?
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > If we get it into the right format is how I
specify
> >> the
> >> > > >> 'name'
> >> > > >> > and
> >> > > >> > >> > > > 'level'
> >> > > >> > >> > > > > of the field correct? It was hard to figure
out the
> >> > correct
> >> > > >> > >> notation
> >> > > >> > >> > > from
> >> > > >> > >> > > > > the manual.
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > Thanks,
> >> > > >> > >> > > > > Tanya
> >> > > >> > >> > > > >
> >> > > >> > >> > > > >
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley
Gotway
> via
> >> > RT <
> >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > >> > >> > > > >
> >> > > >> > >> > > > >> Hi Tanya,
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> I took a look at the gridded NetCDF data
you're
> >> trying
> >> > to
> >> > > >> pass
> >> > > >> > to
> >> > > >> > >> > > MODE.
> >> > > >> > >> > > > >> Unfortunately, MET doesn't support all types
of
> >> gridded
> >> > > >> NetCDF
> >> > > >> > >> > files.
> >> > > >> > >> > > > >> Instead, it only supports the MET NetCDF
format
> (e.g.
> >> > the
> >> > > >> > output
> >> > > >> > >> of
> >> > > >> > >> > > the
> >> > > >> > >> > > > >> pcp_combine tool), the output of the WRF
pinterp
> (or
> >> > > >> wrfinterp)
> >> > > >> > >> > > > utilities,
> >> > > >> > >> > > > >> and CF-compliant NetCDF files.
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> Looking at "
> >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> >> > "
> >> > > I
> >> > > >> see
> >> > > >> > >> that
> >> > > >> > >> > > it
> >> > > >> > >> > > > >> follows none of those conventions. However,
it is
> >> > closest
> >> > > >> to
> >> > > >> > >> > > > CF-compliant
> >> > > >> > >> > > > >> NetCDF.
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> When using a new data file format, instead
of
> >> running it
> >> > > >> > through
> >> > > >> > >> > MODE,
> >> > > >> > >> > > > >> I'd suggest using plot_data_plane to make
sure MET
> is
> >> > > >> reading
> >> > > >> > it
> >> > > >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane as
> follows:
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
> >> > > >> > >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
\
> >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > > >> > file_type=NETCDF_NCCF;'
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> The resulting image is attached. There are
several
> >> > issues
> >> > > >> > here:
> >> > > >> > >> > > > >> - I had to tell plot_data_plane to interpret
this
> as
> >> > > >> > CF-compliant
> >> > > >> > >> > > NetCDF
> >> > > >> > >> > > > >> using NETCDF_NCCF.
> >> > > >> > >> > > > >> - The timing info in that file isn't
specified in
> the
> >> > > >> > >> CF-compliant
> >> > > >> > >> > > > way...
> >> > > >> > >> > > > >> so MET can't discern the initialization,
valid, and
> >> lead
> >> > > >> times.
> >> > > >> > >> > > > >> - And then there's the obvious problem (in
the
> >> image) of
> >> > > the
> >> > > >> > >> world
> >> > > >> > >> > > being
> >> > > >> > >> > > > >> upside down and backwards!
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> So while we did get *something*, there are
> obviously
> >> a
> >> > lot
> >> > > >> of
> >> > > >> > >> > issues.
> >> > > >> > >> > > > >> Basically, MET doesn't support the gridded
NetCDF
> >> file
> >> > > >> format
> >> > > >> > you
> >> > > >> > >> > are
> >> > > >> > >> > > > >> using. Is this data possibly available in
GRIB?
> Or
> >> do
> >> > > you
> >> > > >> > have
> >> > > >> > >> > > control
> >> > > >> > >> > > > >> over the NetCDF file format in use?
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >> Thanks,
> >> > > >> > >> > > > >> John
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >>
> >> > > >> > >> > > > >
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > --
> >> > > >> > >> > > > > Tanya R. Peevey, PhD
> >> > > >> > >> > > > > Research Scientist I, Global Observing
Systems
> >> Analysis
> >> > > >> (GOSA)
> >> > > >> > >> Group
> >> > > >> > >> > > > > NOAA ESRL Global Systems Division
> >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> >> > > >> > >> > > > > (303) 497-5847
> >> > > >> > >> > > > >
> >> > > >> > >> > > >
> >> > > >> > >> > > >
> >> > > >> > >> > > >
> >> > > >> > >> > > > --
> >> > > >> > >> > > > Tanya R. Peevey, PhD
> >> > > >> > >> > > > Research Scientist I, Global Observing Systems
> Analysis
> >> > > (GOSA)
> >> > > >> > Group
> >> > > >> > >> > > > NOAA ESRL Global Systems Division
> >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> >> > > >> > >> > > > (303) 497-5847
> >> > > >> > >> > > >
> >> > > >> > >> > > >
> >> > > >> > >> > >
> >> > > >> > >> > >
> >> > > >> > >> >
> >> > > >> > >> >
> >> > > >> > >> > --
> >> > > >> > >> > Tanya R. Peevey, PhD
> >> > > >> > >> > Research Scientist I, Global Observing Systems
Analysis
> >> (GOSA)
> >> > > >> Group
> >> > > >> > >> > NOAA ESRL Global Systems Division
> >> > > >> > >> > 325 Broadway, Boulder, CO 80305
> >> > > >> > >> > (303) 497-5847
> >> > > >> > >> >
> >> > > >> > >> >
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > --
> >> > > >> > > Tanya R. Peevey, PhD
> >> > > >> > > Research Scientist I, Global Observing Systems
Analysis
> (GOSA)
> >> > Group
> >> > > >> > > NOAA ESRL Global Systems Division
> >> > > >> > > 325 Broadway, Boulder, CO 80305
> >> > > >> > > (303) 497-5847
> >> > > >> > >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > --
> >> > > >> > Tanya R. Peevey, PhD
> >> > > >> > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> >> Group
> >> > > >> > NOAA ESRL Global Systems Division
> >> > > >> > 325 Broadway, Boulder, CO 80305
> >> > > >> > (303) 497-5847
> >> > > >> >
> >> > > >> >
> >> > > >>
> >> > > >>
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Tanya R. Peevey, PhD
> >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> >> > > > NOAA ESRL Global Systems Division
> >> > > > 325 Broadway, Boulder, CO 80305
> >> > > > (303) 497-5847
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Tanya R. Peevey, PhD
> >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> >> > > NOAA ESRL Global Systems Division
> >> > > 325 Broadway, Boulder, CO 80305
> >> > > (303) 497-5847
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >> --
> >> Tanya R. Peevey, PhD
> >> Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> >> NOAA ESRL Global Systems Division
> >> 325 Broadway, Boulder, CO 80305
> >> (303) 497-5847
> >>
> >>
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Wed Feb 17 10:00:36 2016
Tanya,
Grid 212 is commonly used:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid212.gif
However that is limited to CONUS.
To be clear, there really isn't a problem in the output of MODE on
grid
221, other than the plotting of Russia's coastlines. The object
definition
and distances should all be fine. Are you planning to use the
PostScript
output of MODE systematically? Do you need the graphics? MODE can be
configured to write a NetCDF object file which can be plotted using
whatever graphics package you'd like.
For example, for HWT or HMT a few years back we plotted the MODE
NetCDF
output using NCL to overlay the forecast and observation objects.
Thanks,
John
On Tue, Feb 16, 2016 at 10:59 AM, Tanya Peevey - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> Thanks for the information. We are going to look at the time
information
> again today. Also, I found the issue with the last file I tried to
use in
> MODE, the issue wasn't the grid_mapping but something else on our
end.
>
> I tried different grids on the *old_cf* files (the ones that worked
> previously) and all of them had Russia looking funny. The grids I
tried
> were G221 and G241, both North America grids. So basically the only
options
> I have are CONUS, Northern Hemisphere or the Globe, none of which
are
> appealing. I also don't like the idea of excluding Russia but I'll
copy
> over your code so I have it. In the meantime we will probably create
a
> subset of data over North America and then run that through MODE. As
soon
> as you get the North American centric grids working please let me
know.
>
> Lastly, if you know of another grid that would work please let me
know but
> I believe I've tried all of the North American centric grids that
are of
> ~equivalent resolution of the original dataset (1x1 degree) and are
> available in MODE.
>
> Thanks,
> Tanya
>
>
>
> On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Tanya,
> >
> > On to more issues. I think that not having the grid_mapping
should work
> > fine, but only because this is lat/lon data and MET can use the
lat/lon
> > variables to infer the grid.
> >
> > For example, the following commands works fine:
> >
> > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
> > NR2006.pl.1by1.7278.2006022818.cf.nc \
> > ~/without_grid_map.ps \
> > 'name="LSP_6h"; level="(*,*)";'
> >
> > Next, looking in your MODE output file:
> >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
> >
> > I see that the lead time is listed as 3169835842. The units are
> supposed
> > to be HHMMSS so this string looks pretty odd! So there's still an
issue
> > with the times.
> >
> > John
> >
> >
> >
> > On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > Tanya,
> > >
> > > I took a close look at the issue with plotting the map for grid
209.
> The
> > > issue is caused by the map data straddling the international
date line.
> > I
> > > tried rescaling the longitudes from -180,180 to 0,360 but wasn't
able
> to
> > > resolve this issue. So this is not easily fixed. I'll need to
have
> > Randy
> > > Bullock look for a solution.
> > >
> > > In the meantime, you have a couple options...
> > > - If you're not tied to grid 209, you could easily switch to
using a
> > > different grid.
> > > - Or you could set up the config file to plot a set of map data
files
> > > which does not include Russia.
> > >
> > > Take a look in met-5.1/share/met/config/ConfigMap. The
"map_data"
> > section
> > > controls the map data files to be plotted. I've attached a map
data
> file
> > > to this message from which I've removed all the Russia data:
> > > country_data_minus_russia
> > >
> > > Plotting on this map data file results in the attached plot of
grid
> 209.
> > >
> > > FYI, you can just copy and paste the "map_data" config file
section to
> > the
> > > bottom of your MODE config file. And then set the path to the
attached
> > map
> > > data file. That'll overwrite the default values set in
ConfigMapData.
> > >
> > > John
> > >
> > >
> > > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >>
> > >> John,
> > >>
> > >> Ok, tried G209 in the new MODE and I got results but the lines
on the
> > map
> > >> that are suppose to define Russia are all still wrong. See
following
> > >> directory,
> > >>
> > >>
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
> > >>
> > >> So the above worked with a NetCDF file we generated that has
> > CF-compliant
> > >> time and grid_mapping information. We found out that the
grid_mapping
> > >> variable in the CF-compliant file was what was not allowing us
to use
> > >> Panoply (a package for quickly viewing data in an NetCDF file).
So we
> > >> removed the grid_mapping information and tried again, to see if
MODE
> > would
> > >> work and it did not. Here are where the two files are location
> > >>
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> > >> with *.cf.nc being the file without grid_mapping and
*.old_cf.nc
> being
> > >> the
> > >> file with grid_mapping.
> > >>
> > >>
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >>
> > >>
> >
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> > >> data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> > >> data/gfs_test/
> > >> NR2006.pl.1by1.7188.2006022500.cf.nc
> gfs_test/config/MODEConfig_gfstest
> > >> -outdir gfs_test/out/mode/ -v 2
> > >> DEBUG 1: Default Config File:
> > >> /scratch4/BMC/dtc/MET/met-
5.1/share/met/config/MODEConfig_default
> > >> DEBUG 1: Match Config File: gfs_test/config/MODEConfig_gfstest
> > >> DEBUG 1: Merge Config File: gfs_test/config/MODEConfig_gfstest
> > >> ERROR :
> > >> ERROR : Trouble reading forecast file "data/gfs_test/
> > >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> > >> ERROR :
> > >> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>
> > >>
> > >>
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >>
> > >> Basically, want to try and fix the regridding to the G209 grid
so
> Russia
> > >> looks normal and want to confirm that we do need to use the *.
> old_cf.nc
> > >> files instead of the other within MET.
> > >>
> > >> Thanks,
> > >> Tanya
> > >>
> > >> P.S. Having the regird option in MODE is great. Now I don't
have to
> > create
> > >> another set of files before processing the data. Saves me some
disk
> > space
> > >> :-).
> > >>
> > >>
> > >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> > Tanya,
> > >> >
> > >> > I can look more closely tomorrow, but some initial thoughts
are:
> > >> >
> > >> > - yes, please use met-5.1 and use the default mode 5.1 config
file
> > >> > - you could use the regrid tool, but instead fill out the
regrid
> > >> section of
> > >> > the mode config file. Try setting to_grid = "G209";
> > >> >
> > >> > That'll automatically regrid the global data in the fly.
> > >> >
> > >> > That's all new in 5.1.
> > >> >
> > >> > John
> > >> >
> > >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA Affiliate
via
> RT <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> > >
> > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >> > >
> > >> > > John,
> > >> > >
> > >> > > To clarify my previous email, I'm trying to use the regrid
tool to
> > >> > reverse
> > >> > > the latitudes in our .nc files and focus on North America
rather
> > than
> > >> the
> > >> > > whole global. I just tried the regrid tool with the grid
set to
> G209
> > >> and
> > >> > it
> > >> > > worked (both corrected the latitude issue and focused on
North
> > >> America)
> > >> > but
> > >> > > with one small problem, there are some weird lines where
Russia is
> > >> > suppose
> > >> > > to be. Do you know what the problem is? Is there another
grid you
> > >> could
> > >> > > recommend if the output from the regrid tool on the G209
grid
> can't
> > be
> > >> > > fixed? Or could I define my own grid using another MET
tool?
> > >> > >
> > >> > > If all of the above fails we can modify our NetCDF files
but I
> hope
> > >> to be
> > >> > > able to use the regrid tool.
> > >> > >
> > >> > > All plots from my tests with the regrid tool can be found
here:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> > >> > > Data used to make the plots can be found here:
> > >> > >
> > >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> > >> > >
> > >> > > Still would like to see some documentation on the regrid
tool.
> > >> > >
> > >> > > Lastly, after using the regrid tool I had to use the newest
> version
> > of
> > >> > MET
> > >> > > (met-5.1). Is this version good to go? Should I change my
MET path
> > to
> > >> > this
> > >> > > version? Is there documentation on this version yet?
> > >> > >
> > >> > > Thanks!
> > >> > > Tanya
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
Affiliate <
> > >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >> > >
> > >> > > > John,
> > >> > > >
> > >> > > > So Jason and I got our data working with MODE. However, I
have
> one
> > >> > > > question. Right now MODE automatically plots the full
grid
> > >> available,
> > >> > in
> > >> > > > this case a global plot. We are only focused on North
America. I
> > was
> > >> > > > thinking of using the regrid tool to change the grid from
global
> > to
> > >> > North
> > >> > > > American using NCEP grid 209. Can the regrid tool take
any of
> the
> > >> NCEP
> > >> > > > grids found on the following website (
> > >> > > > http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html)
and
> > apply
> > >> it
> > >> > to
> > >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
Specifications'
> by
> > >> it)?
> > >> > > > Also, right now we reverse the latitude indices so that
> everything
> > >> is
> > >> > > right
> > >> > > > side up but this makes it so another product we use can't
view
> the
> > >> .nc
> > >> > > > files. So my question to you is would we be able to do
reverse
> the
> > >> > > > latitudes using the MET regrid tool with NCEP grid 209 or
would
> we
> > >> need
> > >> > > to
> > >> > > > use the regrid tool twice, once with grid 004 and once
with grid
> > >> 209,
> > >> > or
> > >> > > > would another NCEP grid work?
> > >> > > >
> > >> > > > Oh, and, could you direct me to some documentation on the
regrid
> > >> tool.
> > >> > I
> > >> > > > can see the usage from the command line but that doesn't
really
> > >> address
> > >> > > my
> > >> > > > above question. Also, I didn't see anything in the MET
manual.
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Tanya
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via
RT <
> > >> > > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >
> > >> > > >> Tanya,
> > >> > > >>
> > >> > > >> I took a close look at your current data file:
> > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > >>
> > >> > > >> Running it through the debugger, I see that the problem
comes
> > from
> > >> > > parsing
> > >> > > >> this time string:
> > >> > > >> "hours since 2005-05-01 (12:00)"
> > >> > > >>
> > >> > > >> The parsing code is expecting that to be in HH:MM:SS
format,
> not
> > >> > > HH:MM. I
> > >> > > >> changed it to this:
> > >> > > >> "hours since 2005-05-01 12:00:00"
> > >> > > >>
> > >> > > >> And then I was able to run the plot_data_plane tool on
it.
> > >> However,
> > >> > the
> > >> > > >> resulting image is still upside-down and backwards. See
> attached
> > >> > > >> (test.png).
> > >> > > >>
> > >> > > >> To answer the question about grid x/y mapping, the
answer is
> no.
> > >> For
> > >> > > >> lat/lon grids with no projection info defined, MET uses
the
> > lat/lon
> > >> > > values
> > >> > > >> and intuits the projection info. For non-lat/lon grids,
the
> > >> > projection
> > >> > > >> info would need to be explicitly defined since there's
no
> simple
> > >> way
> > >> > of
> > >> > > >> intuiting a grid definition from latitude and longitude
values.
> > >> > > >>
> > >> > > >> So about the data being upside-down and backwards...
there are
> > >> flags
> > >> > in
> > >> > > >> GRIB files which indicate the order in which the data is
> written.
> > >> > When
> > >> > > >> reading data from GRIB files, MET uses those flags to
determine
> > >> where
> > >> > to
> > >> > > >> place things and orient the data right-side-up. The
> > CF-compliant
> > >> > > NetCDF
> > >> > > >> code logic is much simpler... it just reads the data in
the
> order
> > >> it
> > >> > is
> > >> > > >> encoded in the file and doesn't check which way is up!
> > >> > > >>
> > >> > > >> I'm not really sure what to do here. Our support for
> > CF-compliant
> > >> > > NetCDF
> > >> > > >> is still relatively new, and I'm not sure what the
expectations
> > >> should
> > >> > > be.
> > >> > > >> Should we add logic to the code to detect this situation
and
> > >> handle it
> > >> > > >> better? Or is the code working as expected? I'm not
really
> > sure.
> > >> > > >>
> > >> > > >> I do see that ncview handles it better!
> > >> > > >>
> > >> > > >> With the current version of MET, you have 2 choices:
> > >> > > >>
> > >> > > >> (1) You could write lat/lon and data values in reverse
order
> > (i.e.
> > >> > > >> increasing latitude values) to "right the ship".
> > >> > > >>
> > >> > > >> (2) Or you could leave things as-is and make use of the
> automated
> > >> > > >> regridding capability within the MET tools.
> > >> > > >>
> > >> > > >> For example:
> > >> > > >>
> > >> > > >> # run regrid_data_plane utility to regrid to NCEP grid 4
> > >> > > >> regrid_data_plane \
> > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc G004
\
> > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > >> > > >> -field 'name="T2"; level="(*,*)";'
> > >> > > >>
> > >> > > >> # run plot_data_plane on the output
> > >> > > >> plot_data_plane \
> > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> > >> > > >>
> > >> > > >> The result (test_regrid.png) is attached.
> > >> > > >>
> > >> > > >> John
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
Affiliate
> via
> > >> RT <
> > >> > > >> met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >>
> > >> > > >> >
> > >> > > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > >
> > >> > > >> >
> > >> > > >> > John,
> > >> > > >> >
> > >> > > >> > My colleague, Jason English (cc'd on email), that is
> generating
> > >> the
> > >> > > >> > NETCF_NCCF files has another question about the CF
format.
> > >> > > >> >
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >> > > >> > Do we need the grid_mapping x and y variables if we
are using
> > the
> > >> > > >> lambert
> > >> > > >> > cylindrical equal area grid?
> > >> > > >> > If so, how do we convert 1-dimensional lat/lon
coordinates to
> > >> the 2d
> > >> > > >> > coordinates with y,x?
> > >> > > >> >
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >> > > >> >
> > >> > > >> > Thank you,
> > >> > > >> > Tanya Peevey
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA
> Affiliate <
> > >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >> > > >> >
> > >> > > >> > > John,
> > >> > > >> > >
> > >> > > >> > > Thank you for the information. We generated a new
file and
> I
> > >> tried
> > >> > > the
> > >> > > >> > > plot_data_plane tool on it and got the following
error. Do
> > you
> > >> > have
> > >> > > an
> > >> > > >> > idea
> > >> > > >> > > of what could be wrong? Sorry to bug you so much and
thanks
> > in
> > >> > > advance
> > >> > > >> > for
> > >> > > >> > > the help.
> > >> > > >> > >
> > >> > > >> > > Here's the location of the file:
> > >> > > >> > >
> > >> > > >>
> > >> > >
> > >>
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > >> > >
> > >> > > >> > > Here's the error:
> > >> > > >> > >
> > >> >
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > >> > > >> > >
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > >> > > >> > > data/gfs_test/
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
> > >> > > >> > > file_type=NETCDF_NCCF;'
> > >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > >> > > >> > > ERROR :
> > >> > > >> > > ERROR : StringArray::operator[](int) const -> range
check
> > >> error!
> > >> > > >> > > ERROR :
> > >> > > >> > >
> > >> >
Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > >> > > >> > >
> > >> > > >> > > Thank you ,
> > >> > > >> > > Tanya
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway
via RT <
> > >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >> > >
> > >> > > >> > >> Tanya,
> > >> > > >> > >>
> > >> > > >> > >> Sorry for the delay. I was out of the office on
Friday.
> > >> Using
> > >> > > >> "LSP_6h"
> > >> > > >> > >> for the name and "(*,*)" for the level should do
the
> trick.
> > >> > > >> > >>
> > >> > > >> > >> When trying to get the MET tools to read a new
dataset, I
> > >> always
> > >> > > >> start
> > >> > > >> > by
> > >> > > >> > >> using the plot_data_plane tool:
> > >> > > >> > >>
> > >> > > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > >> > > file_type=NETCDF_NCCF;'
> > >> > > >> > >>
> > >> > > >> > >> I use that to make sure the name and level settings
work
> as
> > >> > > expected,
> > >> > > >> > and
> > >> > > >> > >> it also creates an output plot so I can see that
MET is
> (or
> > is
> > >> > not)
> > >> > > >> > >> reading/plotting the data as expected.
> > >> > > >> > >>
> > >> > > >> > >> That configuration string you pass to the the
> > plot_data_plane
> > >> > tool
> > >> > > is
> > >> > > >> > >> exactly the same settings as are supported in the
> > >> configuration
> > >> > > >> files.
> > >> > > >> > In
> > >> > > >> > >> fact, we read that string with the same code that
parses
> the
> > >> > config
> > >> > > >> > files.
> > >> > > >> > >>
> > >> > > >> > >> I find using plot_data_plane very helpful.
> > >> > > >> > >>
> > >> > > >> > >> Thanks,
> > >> > > >> > >> John
> > >> > > >> > >>
> > >> > > >> > >>
> > >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey - NOAA
> > Affiliate
> > >> via
> > >> > > RT
> > >> > > >> <
> > >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >> > >>
> > >> > > >> > >> >
> > >> > > >> > >> > <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > >> > >
> > >> > > >> > >> >
> > >> > > >> > >> > John,
> > >> > > >> > >> >
> > >> > > >> > >> > Thanks for the info. We are going to work on
changing
> our
> > >> file
> > >> > > >> format
> > >> > > >> > >> and
> > >> > > >> > >> > then I'll get back to you.
> > >> > > >> > >> > Did you see my other question about how I
specified
> 'name'
> > >> andhow to round to nearest 0.5
> > >>
> > >> > > >> 'level'
> > >> > > >> > >> for
> > >> > > >> > >> > the precip. field in my config file for MODE? Was
how I
> > did
> > >> it
> > >> > > ok?
> > >> > > >> I
> > >> > > >> > ask
> > >> > > >> > >> > because the manual was confusing on this point.
If my
> > syntax
> > >> > was
> > >> > > >> > >> incorrect
> > >> > > >> > >> > could you please email me the correction.
> > >> > > >> > >> >
> > >> > > >> > >> > Thanks,
> > >> > > >> > >> > Tanya
> > >> > > >> > >> >
> > >> > > >> > >> >
> > >> > > >> > >> >
> > >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley
Gotway via
> > RT <
> > >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >> > >> >
> > >> > > >> > >> > > Tanya,
> > >> > > >> > >> > >
> > >> > > >> > >> > > Here's a website about it:
> > >> > > >> > >> > > http://cfconventions.org/latest.html
> > >> > > >> > >> > >
> > >> > > >> > >> > > And I posted a sample CF-compliant NetCDF file
here:
> > >> > > >> > >> > >
> > >> > > >> > >> > >
> > >> > > >> > >> >
> > >> > > >> > >>
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > >> > > >> > >> > >
> > >> > > >> > >> > > The relevant pieces in there are...
> > >> > > >> > >> > > - The global "Conventions" attribute set to
"CF-...".
> > >> > > >> > >> > > - The time dimension/variable define the
forecast
> valid
> > >> time.
> > >> > > >> > >> > > - The forecast_reference_time variable defines
the
> model
> > >> > > >> > >> initialization
> > >> > > >> > >> > > time.
> > >> > > >> > >> > > - The grid_mapping, x, and y variables define
the grid
> > >> > > definition
> > >> > > >> > >> > > information.
> > >> > > >> > >> > > - The grid_mapping variable attributes map
those
> > >> variables to
> > >> > > the
> > >> > > >> > >> > relevant
> > >> > > >> > >> > > grid definition.
> > >> > > >> > >> > >
> > >> > > >> > >> > > Hope this helps.
> > >> > > >> > >> > >
> > >> > > >> > >> > > Thanks,
> > >> > > >> > >> > > John
> > >> > > >> > >> > >
> > >> > > >> > >> > >
> > >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey -
NOAA
> > >> > Affiliate
> > >> > > >> via
> > >> > > >> > RT
> > >> > > >> > >> <
> > >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >> > >> > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > <URL:
> > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > >> > > >> >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > John,
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > To clarify my previous request of
information. I
> meant
> > >> to
> > >> > ask
> > >> > > >> was
> > >> > > >> > >> the
> > >> > > >> > >> > > > following: Do you know of any good
documentation on
> > how
> > >> to
> > >> > > >> make a
> > >> > > >> > >> > NetCDF
> > >> > > >> > >> > > > file CF-compliant? Also, do you have an
example .nc
> > file
> > >> > that
> > >> > > >> is
> > >> > > >> > >> NetCDF
> > >> > > >> > >> > > > CF-compliant that we could compare to?
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > Thanks,
> > >> > > >> > >> > > > Tanya
> > >> > > >> > >> > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya Peevey
- NOAA
> > >> > > Affiliate <
> > >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > > John,
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > We generated the NetCDF files ourselves so
we
> could
> > >> > > probably
> > >> > > >> > make
> > >> > > >> > >> it
> > >> > > >> > >> > > work
> > >> > > >> > >> > > > > for MET. Do you know how to get it into
> CF-compliant
> > >> > NetCDF
> > >> > > >> > >> format?
> > >> > > >> > >> > > > > Yes, we have grib files but the reason we
created
> > >> NetCDF
> > >> > > >> files
> > >> > > >> > was
> > >> > > >> > >> > > > because
> > >> > > >> > >> > > > > original the NR... file had accumulated
precip
> over
> > >> the
> > >> > > whole
> > >> > > >> > >> > forecast
> > >> > > >> > >> > > > > where the other file has 6h accumulation
data, so
> we
> > >> need
> > >> > > to
> > >> > > >> > >> convert
> > >> > > >> > >> > > the
> > >> > > >> > >> > > > > NR... precip information into 6h
accumulation
> (they
> > >> had
> > >> > > >> > different
> > >> > > >> > >> > units
> > >> > > >> > >> > > > > too, so we need to change that too). Since
we had
> to
> > >> > change
> > >> > > >> the
> > >> > > >> > >> units
> > >> > > >> > >> > > > too I
> > >> > > >> > >> > > > > think use generating our own NetCDF file is
best
> but
> > >> we
> > >> > > just
> > >> > > >> > need
> > >> > > >> > >> to
> > >> > > >> > >> > > get
> > >> > > >> > >> > > > it
> > >> > > >> > >> > > > > in the right format.
> > >> > > >> > >> > > > > What is your opinion on the best way to
approach
> > this?
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > If we get it into the right format is how I
> specify
> > >> the
> > >> > > >> 'name'
> > >> > > >> > and
> > >> > > >> > >> > > > 'level'
> > >> > > >> > >> > > > > of the field correct? It was hard to figure
out
> the
> > >> > correct
> > >> > > >> > >> notation
> > >> > > >> > >> > > from
> > >> > > >> > >> > > > > the manual.
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > Thanks,
> > >> > > >> > >> > > > > Tanya
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John Halley
Gotway
> > via
> > >> > RT <
> > >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > >> Hi Tanya,
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> I took a look at the gridded NetCDF data
you're
> > >> trying
> > >> > to
> > >> > > >> pass
> > >> > > >> > to
> > >> > > >> > >> > > MODE.
> > >> > > >> > >> > > > >> Unfortunately, MET doesn't support all
types of
> > >> gridded
> > >> > > >> NetCDF
> > >> > > >> > >> > files.
> > >> > > >> > >> > > > >> Instead, it only supports the MET NetCDF
format
> > (e.g.
> > >> > the
> > >> > > >> > output
> > >> > > >> > >> of
> > >> > > >> > >> > > the
> > >> > > >> > >> > > > >> pcp_combine tool), the output of the WRF
pinterp
> > (or
> > >> > > >> wrfinterp)
> > >> > > >> > >> > > > utilities,
> > >> > > >> > >> > > > >> and CF-compliant NetCDF files.
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> Looking at "
> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> > >> > "
> > >> > > I
> > >> > > >> see
> > >> > > >> > >> that
> > >> > > >> > >> > > it
> > >> > > >> > >> > > > >> follows none of those conventions.
However, it
> is
> > >> > closest
> > >> > > >> to
> > >> > > >> > >> > > > CF-compliant
> > >> > > >> > >> > > > >> NetCDF.
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> When using a new data file format, instead
of
> > >> running it
> > >> > > >> > through
> > >> > > >> > >> > MODE,
> > >> > > >> > >> > > > >> I'd suggest using plot_data_plane to make
sure
> MET
> > is
> > >> > > >> reading
> > >> > > >> > it
> > >> > > >> > >> > > > >> correctly. I ran met-5.1/plot_data_plane
as
> > follows:
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >>
> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >> > > >> > >> > > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > >> > > >> > file_type=NETCDF_NCCF;'
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> The resulting image is attached. There
are
> several
> > >> > issues
> > >> > > >> > here:
> > >> > > >> > >> > > > >> - I had to tell plot_data_plane to
interpret this
> > as
> > >> > > >> > CF-compliant
> > >> > > >> > >> > > NetCDF
> > >> > > >> > >> > > > >> using NETCDF_NCCF.
> > >> > > >> > >> > > > >> - The timing info in that file isn't
specified in
> > the
> > >> > > >> > >> CF-compliant
> > >> > > >> > >> > > > way...
> > >> > > >> > >> > > > >> so MET can't discern the initialization,
valid,
> and
> > >> lead
> > >> > > >> times.
> > >> > > >> > >> > > > >> - And then there's the obvious problem (in
the
> > >> image) of
> > >> > > the
> > >> > > >> > >> world
> > >> > > >> > >> > > being
> > >> > > >> > >> > > > >> upside down and backwards!
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> So while we did get *something*, there are
> > obviously
> > >> a
> > >> > lot
> > >> > > >> of
> > >> > > >> > >> > issues.
> > >> > > >> > >> > > > >> Basically, MET doesn't support the gridded
NetCDF
> > >> file
> > >> > > >> format
> > >> > > >> > you
> > >> > > >> > >> > are
> > >> > > >> > >> > > > >> using. Is this data possibly available in
GRIB?
> > Or
> > >> do
> > >> > > you
> > >> > > >> > have
> > >> > > >> > >> > > control
> > >> > > >> > >> > > > >> over the NetCDF file format in use?
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >> Thanks,
> > >> > > >> > >> > > > >> John
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >>
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > --
> > >> > > >> > >> > > > > Tanya R. Peevey, PhD
> > >> > > >> > >> > > > > Research Scientist I, Global Observing
Systems
> > >> Analysis
> > >> > > >> (GOSA)
> > >> > > >> > >> Group
> > >> > > >> > >> > > > > NOAA ESRL Global Systems Division
> > >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> > >> > > >> > >> > > > > (303) 497-5847
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > --
> > >> > > >> > >> > > > Tanya R. Peevey, PhD
> > >> > > >> > >> > > > Research Scientist I, Global Observing
Systems
> > Analysis
> > >> > > (GOSA)
> > >> > > >> > Group
> > >> > > >> > >> > > > NOAA ESRL Global Systems Division
> > >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> > >> > > >> > >> > > > (303) 497-5847
> > >> > > >> > >> > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > >
> > >> > > >> > >> > >
> > >> > > >> > >> >
> > >> > > >> > >> >
> > >> > > >> > >> > --
> > >> > > >> > >> > Tanya R. Peevey, PhD
> > >> > > >> > >> > Research Scientist I, Global Observing Systems
Analysis
> > >> (GOSA)
> > >> > > >> Group
> > >> > > >> > >> > NOAA ESRL Global Systems Division
> > >> > > >> > >> > 325 Broadway, Boulder, CO 80305
> > >> > > >> > >> > (303) 497-5847
> > >> > > >> > >> >
> > >> > > >> > >> >
> > >> > > >> > >>
> > >> > > >> > >>
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > > --
> > >> > > >> > > Tanya R. Peevey, PhD
> > >> > > >> > > Research Scientist I, Global Observing Systems
Analysis
> > (GOSA)
> > >> > Group
> > >> > > >> > > NOAA ESRL Global Systems Division
> > >> > > >> > > 325 Broadway, Boulder, CO 80305
> > >> > > >> > > (303) 497-5847
> > >> > > >> > >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > --
> > >> > > >> > Tanya R. Peevey, PhD
> > >> > > >> > Research Scientist I, Global Observing Systems
Analysis
> (GOSA)
> > >> Group
> > >> > > >> > NOAA ESRL Global Systems Division
> > >> > > >> > 325 Broadway, Boulder, CO 80305
> > >> > > >> > (303) 497-5847
> > >> > > >> >
> > >> > > >> >
> > >> > > >>
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Tanya R. Peevey, PhD
> > >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > Group
> > >> > > > NOAA ESRL Global Systems Division
> > >> > > > 325 Broadway, Boulder, CO 80305
> > >> > > > (303) 497-5847
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Tanya R. Peevey, PhD
> > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> > >> > > NOAA ESRL Global Systems Division
> > >> > > 325 Broadway, Boulder, CO 80305
> > >> > > (303) 497-5847
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> > >> --
> > >> Tanya R. Peevey, PhD
> > >> Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > >> NOAA ESRL Global Systems Division
> > >> 325 Broadway, Boulder, CO 80305
> > >> (303) 497-5847
> > >>
> > >>
> > >
> >
> >
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Wed Feb 17 12:07:25 2016
John,
Thanks for the clarification of the MODE output on grid 221. We will
use
the ps file/graphics produced by MODE regularly, but don't know if we
will
use them when we publish. For the moment they are fine for in-house
stuff,
but still let me know when those grids are fixed in terms of how they
look
in the MODE graphics. We will either create a subset of data and use
that
in MODE or do what you suggest by having MODE write an NetCDF object
file
if the grids aren't fixed and we are going to publish (this is still a
few
months away). How do you create that object netcdf file? What do I
need to
change in MODE? Also, I thought MODE already spit out a couple files
one of
which is a file with object information? Is that file ASCII?
So letting me know how to modify MODE would be great since it would be
good
to be able to overlay the objects onto other fields. Any scripts you
may
have for reading the NetCDF output would be helpful too.
Thanks,
Tanya
On Wed, Feb 17, 2016 at 10:00 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Tanya,
>
> Grid 212 is commonly used:
> http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid212.gif
>
> However that is limited to CONUS.
>
> To be clear, there really isn't a problem in the output of MODE on
grid
> 221, other than the plotting of Russia's coastlines. The object
definition
> and distances should all be fine. Are you planning to use the
PostScript
> output of MODE systematically? Do you need the graphics? MODE can
be
> configured to write a NetCDF object file which can be plotted using
> whatever graphics package you'd like.
>
> For example, for HWT or HMT a few years back we plotted the MODE
NetCDF
> output using NCL to overlay the forecast and observation objects.
>
> Thanks,
> John
>
>
>
>
> On Tue, Feb 16, 2016 at 10:59 AM, Tanya Peevey - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >
> > John,
> >
> > Thanks for the information. We are going to look at the time
information
> > again today. Also, I found the issue with the last file I tried to
use in
> > MODE, the issue wasn't the grid_mapping but something else on our
end.
> >
> > I tried different grids on the *old_cf* files (the ones that
worked
> > previously) and all of them had Russia looking funny. The grids I
tried
> > were G221 and G241, both North America grids. So basically the
only
> options
> > I have are CONUS, Northern Hemisphere or the Globe, none of which
are
> > appealing. I also don't like the idea of excluding Russia but I'll
copy
> > over your code so I have it. In the meantime we will probably
create a
> > subset of data over North America and then run that through MODE.
As soon
> > as you get the North American centric grids working please let me
know.
> >
> > Lastly, if you know of another grid that would work please let me
know
> but
> > I believe I've tried all of the North American centric grids that
are of
> > ~equivalent resolution of the original dataset (1x1 degree) and
are
> > available in MODE.
> >
> > Thanks,
> > Tanya
> >
> >
> >
> > On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Tanya,
> > >
> > > On to more issues. I think that not having the grid_mapping
should
> work
> > > fine, but only because this is lat/lon data and MET can use the
lat/lon
> > > variables to infer the grid.
> > >
> > > For example, the following commands works fine:
> > >
> > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
> > > NR2006.pl.1by1.7278.2006022818.cf.nc \
> > > ~/without_grid_map.ps \
> > > 'name="LSP_6h"; level="(*,*)";'
> > >
> > > Next, looking in your MODE output file:
> > >
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
> > >
> > > I see that the lead time is listed as 3169835842. The units are
> > supposed
> > > to be HHMMSS so this string looks pretty odd! So there's still
an
> issue
> > > with the times.
> > >
> > > John
> > >
> > >
> > >
> > > On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > >
> > > > Tanya,
> > > >
> > > > I took a close look at the issue with plotting the map for
grid 209.
> > The
> > > > issue is caused by the map data straddling the international
date
> line.
> > > I
> > > > tried rescaling the longitudes from -180,180 to 0,360 but
wasn't able
> > to
> > > > resolve this issue. So this is not easily fixed. I'll need
to have
> > > Randy
> > > > Bullock look for a solution.
> > > >
> > > > In the meantime, you have a couple options...
> > > > - If you're not tied to grid 209, you could easily switch to
using a
> > > > different grid.
> > > > - Or you could set up the config file to plot a set of map
data files
> > > > which does not include Russia.
> > > >
> > > > Take a look in met-5.1/share/met/config/ConfigMap. The
"map_data"
> > > section
> > > > controls the map data files to be plotted. I've attached a
map data
> > file
> > > > to this message from which I've removed all the Russia data:
> > > > country_data_minus_russia
> > > >
> > > > Plotting on this map data file results in the attached plot of
grid
> > 209.
> > > >
> > > > FYI, you can just copy and paste the "map_data" config file
section
> to
> > > the
> > > > bottom of your MODE config file. And then set the path to the
> attached
> > > map
> > > > data file. That'll overwrite the default values set in
> ConfigMapData.
> > > >
> > > > John
> > > >
> > > >
> > > > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> > > >>
> > > >> John,
> > > >>
> > > >> Ok, tried G209 in the new MODE and I got results but the
lines on
> the
> > > map
> > > >> that are suppose to define Russia are all still wrong. See
following
> > > >> directory,
> > > >>
> > > >>
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
> > > >>
> > > >> So the above worked with a NetCDF file we generated that has
> > > CF-compliant
> > > >> time and grid_mapping information. We found out that the
> grid_mapping
> > > >> variable in the CF-compliant file was what was not allowing
us to
> use
> > > >> Panoply (a package for quickly viewing data in an NetCDF
file). So
> we
> > > >> removed the grid_mapping information and tried again, to see
if MODE
> > > would
> > > >> work and it did not. Here are where the two files are
location
> > > >>
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> > > >> with *.cf.nc being the file without grid_mapping and
*.old_cf.nc
> > being
> > > >> the
> > > >> file with grid_mapping.
> > > >>
> > > >>
> > >
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >>
> > > >>
> > >
> >
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> > > >> data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> > > >> data/gfs_test/
> > > >> NR2006.pl.1by1.7188.2006022500.cf.nc
> > gfs_test/config/MODEConfig_gfstest
> > > >> -outdir gfs_test/out/mode/ -v 2
> > > >> DEBUG 1: Default Config File:
> > > >> /scratch4/BMC/dtc/MET/met-
5.1/share/met/config/MODEConfig_default
> > > >> DEBUG 1: Match Config File:
gfs_test/config/MODEConfig_gfstest
> > > >> DEBUG 1: Merge Config File:
gfs_test/config/MODEConfig_gfstest
> > > >> ERROR :
> > > >> ERROR : Trouble reading forecast file "data/gfs_test/
> > > >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> > > >> ERROR :
> > > >>
Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
> > > >>
> > > >>
> > >
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >>
> > > >> Basically, want to try and fix the regridding to the G209
grid so
> > Russia
> > > >> looks normal and want to confirm that we do need to use the
*.
> > old_cf.nc
> > > >> files instead of the other within MET.
> > > >>
> > > >> Thanks,
> > > >> Tanya
> > > >>
> > > >> P.S. Having the regird option in MODE is great. Now I don't
have to
> > > create
> > > >> another set of files before processing the data. Saves me
some disk
> > > space
> > > >> :-).
> > > >>
> > > >>
> > > >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> > Tanya,
> > > >> >
> > > >> > I can look more closely tomorrow, but some initial thoughts
are:
> > > >> >
> > > >> > - yes, please use met-5.1 and use the default mode 5.1
config file
> > > >> > - you could use the regrid tool, but instead fill out the
regrid
> > > >> section of
> > > >> > the mode config file. Try setting to_grid = "G209";
> > > >> >
> > > >> > That'll automatically regrid the global data in the fly.
> > > >> >
> > > >> > That's all new in 5.1.
> > > >> >
> > > >> > John
> > > >> >
> > > >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA
Affiliate via
> > RT <
> > > >> > met_help at ucar.edu> wrote:
> > > >> >
> > > >> > >
> > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > > >> > >
> > > >> > > John,
> > > >> > >
> > > >> > > To clarify my previous email, I'm trying to use the
regrid tool
> to
> > > >> > reverse
> > > >> > > the latitudes in our .nc files and focus on North America
rather
> > > than
> > > >> the
> > > >> > > whole global. I just tried the regrid tool with the grid
set to
> > G209
> > > >> and
> > > >> > it
> > > >> > > worked (both corrected the latitude issue and focused on
North
> > > >> America)
> > > >> > but
> > > >> > > with one small problem, there are some weird lines where
Russia
> is
> > > >> > suppose
> > > >> > > to be. Do you know what the problem is? Is there another
grid
> you
> > > >> could
> > > >> > > recommend if the output from the regrid tool on the G209
grid
> > can't
> > > be
> > > >> > > fixed? Or could I define my own grid using another MET
tool?
> > > >> > >
> > > >> > > If all of the above fails we can modify our NetCDF files
but I
> > hope
> > > >> to be
> > > >> > > able to use the regrid tool.
> > > >> > >
> > > >> > > All plots from my tests with the regrid tool can be found
here:
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> > > >> > > Data used to make the plots can be found here:
> > > >> > >
> > > >>
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> > > >> > >
> > > >> > > Still would like to see some documentation on the regrid
tool.
> > > >> > >
> > > >> > > Lastly, after using the regrid tool I had to use the
newest
> > version
> > > of
> > > >> > MET
> > > >> > > (met-5.1). Is this version good to go? Should I change my
MET
> path
> > > to
> > > >> > this
> > > >> > > version? Is there documentation on this version yet?
> > > >> > >
> > > >> > > Thanks!
> > > >> > > Tanya
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
Affiliate <
> > > >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > >> > >
> > > >> > > > John,
> > > >> > > >
> > > >> > > > So Jason and I got our data working with MODE. However,
I have
> > one
> > > >> > > > question. Right now MODE automatically plots the full
grid
> > > >> available,
> > > >> > in
> > > >> > > > this case a global plot. We are only focused on North
> America. I
> > > was
> > > >> > > > thinking of using the regrid tool to change the grid
from
> global
> > > to
> > > >> > North
> > > >> > > > American using NCEP grid 209. Can the regrid tool take
any of
> > the
> > > >> NCEP
> > > >> > > > grids found on the following website (
> > > >> > > >
http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) and
> > > apply
> > > >> it
> > > >> > to
> > > >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
Specifications'
> > by
> > > >> it)?
> > > >> > > > Also, right now we reverse the latitude indices so that
> > everything
> > > >> is
> > > >> > > right
> > > >> > > > side up but this makes it so another product we use
can't view
> > the
> > > >> .nc
> > > >> > > > files. So my question to you is would we be able to do
reverse
> > the
> > > >> > > > latitudes using the MET regrid tool with NCEP grid 209
or
> would
> > we
> > > >> need
> > > >> > > to
> > > >> > > > use the regrid tool twice, once with grid 004 and once
with
> grid
> > > >> 209,
> > > >> > or
> > > >> > > > would another NCEP grid work?
> > > >> > > >
> > > >> > > > Oh, and, could you direct me to some documentation on
the
> regrid
> > > >> tool.
> > > >> > I
> > > >> > > > can see the usage from the command line but that
doesn't
> really
> > > >> address
> > > >> > > my
> > > >> > > > above question. Also, I didn't see anything in the MET
manual.
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Tanya
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway via
RT <
> > > >> > > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >
> > > >> > > >> Tanya,
> > > >> > > >>
> > > >> > > >> I took a close look at your current data file:
> > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > >>
> > > >> > > >> Running it through the debugger, I see that the
problem comes
> > > from
> > > >> > > parsing
> > > >> > > >> this time string:
> > > >> > > >> "hours since 2005-05-01 (12:00)"
> > > >> > > >>
> > > >> > > >> The parsing code is expecting that to be in HH:MM:SS
format,
> > not
> > > >> > > HH:MM. I
> > > >> > > >> changed it to this:
> > > >> > > >> "hours since 2005-05-01 12:00:00"
> > > >> > > >>
> > > >> > > >> And then I was able to run the plot_data_plane tool on
it.
> > > >> However,
> > > >> > the
> > > >> > > >> resulting image is still upside-down and backwards.
See
> > attached
> > > >> > > >> (test.png).
> > > >> > > >>
> > > >> > > >> To answer the question about grid x/y mapping, the
answer is
> > no.
> > > >> For
> > > >> > > >> lat/lon grids with no projection info defined, MET
uses the
> > > lat/lon
> > > >> > > values
> > > >> > > >> and intuits the projection info. For non-lat/lon
grids, the
> > > >> > projection
> > > >> > > >> info would need to be explicitly defined since there's
no
> > simple
> > > >> way
> > > >> > of
> > > >> > > >> intuiting a grid definition from latitude and
longitude
> values.
> > > >> > > >>
> > > >> > > >> So about the data being upside-down and backwards...
there
> are
> > > >> flags
> > > >> > in
> > > >> > > >> GRIB files which indicate the order in which the data
is
> > written.
> > > >> > When
> > > >> > > >> reading data from GRIB files, MET uses those flags to
> determine
> > > >> where
> > > >> > to
> > > >> > > >> place things and orient the data right-side-up. The
> > > CF-compliant
> > > >> > > NetCDF
> > > >> > > >> code logic is much simpler... it just reads the data
in the
> > order
> > > >> it
> > > >> > is
> > > >> > > >> encoded in the file and doesn't check which way is up!
> > > >> > > >>
> > > >> > > >> I'm not really sure what to do here. Our support for
> > > CF-compliant
> > > >> > > NetCDF
> > > >> > > >> is still relatively new, and I'm not sure what the
> expectations
> > > >> should
> > > >> > > be.
> > > >> > > >> Should we add logic to the code to detect this
situation and
> > > >> handle it
> > > >> > > >> better? Or is the code working as expected? I'm not
really
> > > sure.
> > > >> > > >>
> > > >> > > >> I do see that ncview handles it better!
> > > >> > > >>
> > > >> > > >> With the current version of MET, you have 2 choices:
> > > >> > > >>
> > > >> > > >> (1) You could write lat/lon and data values in reverse
order
> > > (i.e.
> > > >> > > >> increasing latitude values) to "right the ship".
> > > >> > > >>
> > > >> > > >> (2) Or you could leave things as-is and make use of
the
> > automated
> > > >> > > >> regridding capability within the MET tools.
> > > >> > > >>
> > > >> > > >> For example:
> > > >> > > >>
> > > >> > > >> # run regrid_data_plane utility to regrid to NCEP grid
4
> > > >> > > >> regrid_data_plane \
> > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc
G004 \
> > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc
\
> > > >> > > >> -field 'name="T2"; level="(*,*)";'
> > > >> > > >>
> > > >> > > >> # run plot_data_plane on the output
> > > >> > > >> plot_data_plane \
> > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc
\
> > > >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> > > >> > > >>
> > > >> > > >> The result (test_regrid.png) is attached.
> > > >> > > >>
> > > >> > > >> John
> > > >> > > >>
> > > >> > > >>
> > > >> > > >>
> > > >> > > >>
> > > >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
Affiliate
> > via
> > > >> RT <
> > > >> > > >> met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >>
> > > >> > > >> >
> > > >> > > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > >
> > > >> > > >> >
> > > >> > > >> > John,
> > > >> > > >> >
> > > >> > > >> > My colleague, Jason English (cc'd on email), that is
> > generating
> > > >> the
> > > >> > > >> > NETCF_NCCF files has another question about the CF
format.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >> > > >> > Do we need the grid_mapping x and y variables if we
are
> using
> > > the
> > > >> > > >> lambert
> > > >> > > >> > cylindrical equal area grid?
> > > >> > > >> > If so, how do we convert 1-dimensional lat/lon
coordinates
> to
> > > >> the 2d
> > > >> > > >> > coordinates with y,x?
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > >> > > >> >
> > > >> > > >> > Thank you,
> > > >> > > >> > Tanya Peevey
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey - NOAA
> > Affiliate <
> > > >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > >> > > >> >
> > > >> > > >> > > John,
> > > >> > > >> > >
> > > >> > > >> > > Thank you for the information. We generated a new
file
> and
> > I
> > > >> tried
> > > >> > > the
> > > >> > > >> > > plot_data_plane tool on it and got the following
error.
> Do
> > > you
> > > >> > have
> > > >> > > an
> > > >> > > >> > idea
> > > >> > > >> > > of what could be wrong? Sorry to bug you so much
and
> thanks
> > > in
> > > >> > > advance
> > > >> > > >> > for
> > > >> > > >> > > the help.
> > > >> > > >> > >
> > > >> > > >> > > Here's the location of the file:
> > > >> > > >> > >
> > > >> > > >>
> > > >> > >
> > > >>
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > >> > >
> > > >> > > >> > > Here's the error:
> > > >> > > >> > >
> > > >> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > > >> > > >> > > data/gfs_test/
> > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
> > > >> > > >> > > file_type=NETCDF_NCCF;'
> > > >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > >> > > >> > > ERROR :
> > > >> > > >> > > ERROR : StringArray::operator[](int) const ->
range
> check
> > > >> error!
> > > >> > > >> > > ERROR :
> > > >> > > >> > >
> > > >> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.0>
> > > >> > > >> > >
> > > >> > > >> > > Thank you ,
> > > >> > > >> > > Tanya
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley Gotway
via
> RT <
> > > >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >> > >
> > > >> > > >> > >> Tanya,
> > > >> > > >> > >>
> > > >> > > >> > >> Sorry for the delay. I was out of the office on
Friday.
> > > >> Using
> > > >> > > >> "LSP_6h"
> > > >> > > >> > >> for the name and "(*,*)" for the level should do
the
> > trick.
> > > >> > > >> > >>
> > > >> > > >> > >> When trying to get the MET tools to read a new
dataset,
> I
> > > >> always
> > > >> > > >> start
> > > >> > > >> > by
> > > >> > > >> > >> using the plot_data_plane tool:
> > > >> > > >> > >>
> > > >> > > >> > >> /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
\
> > > >> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > >> > > file_type=NETCDF_NCCF;'
> > > >> > > >> > >>
> > > >> > > >> > >> I use that to make sure the name and level
settings work
> > as
> > > >> > > expected,
> > > >> > > >> > and
> > > >> > > >> > >> it also creates an output plot so I can see that
MET is
> > (or
> > > is
> > > >> > not)
> > > >> > > >> > >> reading/plotting the data as expected.
> > > >> > > >> > >>
> > > >> > > >> > >> That configuration string you pass to the the
> > > plot_data_plane
> > > >> > tool
> > > >> > > is
> > > >> > > >> > >> exactly the same settings as are supported in the
> > > >> configuration
> > > >> > > >> files.
> > > >> > > >> > In
> > > >> > > >> > >> fact, we read that string with the same code that
parses
> > the
> > > >> > config
> > > >> > > >> > files.
> > > >> > > >> > >>
> > > >> > > >> > >> I find using plot_data_plane very helpful.
> > > >> > > >> > >>
> > > >> > > >> > >> Thanks,
> > > >> > > >> > >> John
> > > >> > > >> > >>
> > > >> > > >> > >>
> > > >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey -
NOAA
> > > Affiliate
> > > >> via
> > > >> > > RT
> > > >> > > >> <
> > > >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >> > >>
> > > >> > > >> > >> >
> > > >> > > >> > >> > <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >> > John,
> > > >> > > >> > >> >
> > > >> > > >> > >> > Thanks for the info. We are going to work on
changing
> > our
> > > >> file
> > > >> > > >> format
> > > >> > > >> > >> and
> > > >> > > >> > >> > then I'll get back to you.
> > > >> > > >> > >> > Did you see my other question about how I
specified
> > 'name'
> > > >> andhow to round to nearest 0.5
> > > >>
> > > >> > > >> 'level'
> > > >> > > >> > >> for
> > > >> > > >> > >> > the precip. field in my config file for MODE?
Was how
> I
> > > did
> > > >> it
> > > >> > > ok?
> > > >> > > >> I
> > > >> > > >> > ask
> > > >> > > >> > >> > because the manual was confusing on this point.
If my
> > > syntax
> > > >> > was
> > > >> > > >> > >> incorrect
> > > >> > > >> > >> > could you please email me the correction.
> > > >> > > >> > >> >
> > > >> > > >> > >> > Thanks,
> > > >> > > >> > >> > Tanya
> > > >> > > >> > >> >
> > > >> > > >> > >> >
> > > >> > > >> > >> >
> > > >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley
Gotway
> via
> > > RT <
> > > >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >> > >> >
> > > >> > > >> > >> > > Tanya,
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > Here's a website about it:
> > > >> > > >> > >> > > http://cfconventions.org/latest.html
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > And I posted a sample CF-compliant NetCDF
file here:
> > > >> > > >> > >> > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > The relevant pieces in there are...
> > > >> > > >> > >> > > - The global "Conventions" attribute set to
> "CF-...".
> > > >> > > >> > >> > > - The time dimension/variable define the
forecast
> > valid
> > > >> time.
> > > >> > > >> > >> > > - The forecast_reference_time variable
defines the
> > model
> > > >> > > >> > >> initialization
> > > >> > > >> > >> > > time.
> > > >> > > >> > >> > > - The grid_mapping, x, and y variables define
the
> grid
> > > >> > > definition
> > > >> > > >> > >> > > information.
> > > >> > > >> > >> > > - The grid_mapping variable attributes map
those
> > > >> variables to
> > > >> > > the
> > > >> > > >> > >> > relevant
> > > >> > > >> > >> > > grid definition.
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > Hope this helps.
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > Thanks,
> > > >> > > >> > >> > > John
> > > >> > > >> > >> > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya Peevey
- NOAA
> > > >> > Affiliate
> > > >> > > >> via
> > > >> > > >> > RT
> > > >> > > >> > >> <
> > > >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > <URL:
> > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > >> > > >> >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > John,
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > To clarify my previous request of
information. I
> > meant
> > > >> to
> > > >> > ask
> > > >> > > >> was
> > > >> > > >> > >> the
> > > >> > > >> > >> > > > following: Do you know of any good
documentation
> on
> > > how
> > > >> to
> > > >> > > >> make a
> > > >> > > >> > >> > NetCDF
> > > >> > > >> > >> > > > file CF-compliant? Also, do you have an
example
> .nc
> > > file
> > > >> > that
> > > >> > > >> is
> > > >> > > >> > >> NetCDF
> > > >> > > >> > >> > > > CF-compliant that we could compare to?
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > Thanks,
> > > >> > > >> > >> > > > Tanya
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya
Peevey -
> NOAA
> > > >> > > Affiliate <
> > > >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>>
wrote:
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > > John,
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > We generated the NetCDF files ourselves
so we
> > could
> > > >> > > probably
> > > >> > > >> > make
> > > >> > > >> > >> it
> > > >> > > >> > >> > > work
> > > >> > > >> > >> > > > > for MET. Do you know how to get it into
> > CF-compliant
> > > >> > NetCDF
> > > >> > > >> > >> format?
> > > >> > > >> > >> > > > > Yes, we have grib files but the reason we
> created
> > > >> NetCDF
> > > >> > > >> files
> > > >> > > >> > was
> > > >> > > >> > >> > > > because
> > > >> > > >> > >> > > > > original the NR... file had accumulated
precip
> > over
> > > >> the
> > > >> > > whole
> > > >> > > >> > >> > forecast
> > > >> > > >> > >> > > > > where the other file has 6h accumulation
data,
> so
> > we
> > > >> need
> > > >> > > to
> > > >> > > >> > >> convert
> > > >> > > >> > >> > > the
> > > >> > > >> > >> > > > > NR... precip information into 6h
accumulation
> > (they
> > > >> had
> > > >> > > >> > different
> > > >> > > >> > >> > units
> > > >> > > >> > >> > > > > too, so we need to change that too).
Since we
> had
> > to
> > > >> > change
> > > >> > > >> the
> > > >> > > >> > >> units
> > > >> > > >> > >> > > > too I
> > > >> > > >> > >> > > > > think use generating our own NetCDF file
is best
> > but
> > > >> we
> > > >> > > just
> > > >> > > >> > need
> > > >> > > >> > >> to
> > > >> > > >> > >> > > get
> > > >> > > >> > >> > > > it
> > > >> > > >> > >> > > > > in the right format.
> > > >> > > >> > >> > > > > What is your opinion on the best way to
approach
> > > this?
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > If we get it into the right format is how
I
> > specify
> > > >> the
> > > >> > > >> 'name'
> > > >> > > >> > and
> > > >> > > >> > >> > > > 'level'
> > > >> > > >> > >> > > > > of the field correct? It was hard to
figure out
> > the
> > > >> > correct
> > > >> > > >> > >> notation
> > > >> > > >> > >> > > from
> > > >> > > >> > >> > > > > the manual.
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > Thanks,
> > > >> > > >> > >> > > > > Tanya
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John
Halley
> Gotway
> > > via
> > > >> > RT <
> > > >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>> wrote:
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > >> Hi Tanya,
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> I took a look at the gridded NetCDF data
you're
> > > >> trying
> > > >> > to
> > > >> > > >> pass
> > > >> > > >> > to
> > > >> > > >> > >> > > MODE.
> > > >> > > >> > >> > > > >> Unfortunately, MET doesn't support all
types of
> > > >> gridded
> > > >> > > >> NetCDF
> > > >> > > >> > >> > files.
> > > >> > > >> > >> > > > >> Instead, it only supports the MET NetCDF
format
> > > (e.g.
> > > >> > the
> > > >> > > >> > output
> > > >> > > >> > >> of
> > > >> > > >> > >> > > the
> > > >> > > >> > >> > > > >> pcp_combine tool), the output of the WRF
> pinterp
> > > (or
> > > >> > > >> wrfinterp)
> > > >> > > >> > >> > > > utilities,
> > > >> > > >> > >> > > > >> and CF-compliant NetCDF files.
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> Looking at "
> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> > > >> > "
> > > >> > > I
> > > >> > > >> see
> > > >> > > >> > >> that
> > > >> > > >> > >> > > it
> > > >> > > >> > >> > > > >> follows none of those conventions.
However, it
> > is
> > > >> > closest
> > > >> > > >> to
> > > >> > > >> > >> > > > CF-compliant
> > > >> > > >> > >> > > > >> NetCDF.
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> When using a new data file format,
instead of
> > > >> running it
> > > >> > > >> > through
> > > >> > > >> > >> > MODE,
> > > >> > > >> > >> > > > >> I'd suggest using plot_data_plane to
make sure
> > MET
> > > is
> > > >> > > >> reading
> > > >> > > >> > it
> > > >> > > >> > >> > > > >> correctly. I ran met-
5.1/plot_data_plane as
> > > follows:
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > >> > > >> > >> > > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> > > >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > >> > > >> > file_type=NETCDF_NCCF;'
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> The resulting image is attached. There
are
> > several
> > > >> > issues
> > > >> > > >> > here:
> > > >> > > >> > >> > > > >> - I had to tell plot_data_plane to
interpret
> this
> > > as
> > > >> > > >> > CF-compliant
> > > >> > > >> > >> > > NetCDF
> > > >> > > >> > >> > > > >> using NETCDF_NCCF.
> > > >> > > >> > >> > > > >> - The timing info in that file isn't
specified
> in
> > > the
> > > >> > > >> > >> CF-compliant
> > > >> > > >> > >> > > > way...
> > > >> > > >> > >> > > > >> so MET can't discern the initialization,
valid,
> > and
> > > >> lead
> > > >> > > >> times.
> > > >> > > >> > >> > > > >> - And then there's the obvious problem
(in the
> > > >> image) of
> > > >> > > the
> > > >> > > >> > >> world
> > > >> > > >> > >> > > being
> > > >> > > >> > >> > > > >> upside down and backwards!
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> So while we did get *something*, there
are
> > > obviously
> > > >> a
> > > >> > lot
> > > >> > > >> of
> > > >> > > >> > >> > issues.
> > > >> > > >> > >> > > > >> Basically, MET doesn't support the
gridded
> NetCDF
> > > >> file
> > > >> > > >> format
> > > >> > > >> > you
> > > >> > > >> > >> > are
> > > >> > > >> > >> > > > >> using. Is this data possibly available
in
> GRIB?
> > > Or
> > > >> do
> > > >> > > you
> > > >> > > >> > have
> > > >> > > >> > >> > > control
> > > >> > > >> > >> > > > >> over the NetCDF file format in use?
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> Thanks,
> > > >> > > >> > >> > > > >> John
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > --
> > > >> > > >> > >> > > > > Tanya R. Peevey, PhD
> > > >> > > >> > >> > > > > Research Scientist I, Global Observing
Systems
> > > >> Analysis
> > > >> > > >> (GOSA)
> > > >> > > >> > >> Group
> > > >> > > >> > >> > > > > NOAA ESRL Global Systems Division
> > > >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> > > >> > > >> > >> > > > > (303) 497-5847
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > --
> > > >> > > >> > >> > > > Tanya R. Peevey, PhD
> > > >> > > >> > >> > > > Research Scientist I, Global Observing
Systems
> > > Analysis
> > > >> > > (GOSA)
> > > >> > > >> > Group
> > > >> > > >> > >> > > > NOAA ESRL Global Systems Division
> > > >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> > > >> > > >> > >> > > > (303) 497-5847
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >> >
> > > >> > > >> > >> > --
> > > >> > > >> > >> > Tanya R. Peevey, PhD
> > > >> > > >> > >> > Research Scientist I, Global Observing Systems
> Analysis
> > > >> (GOSA)
> > > >> > > >> Group
> > > >> > > >> > >> > NOAA ESRL Global Systems Division
> > > >> > > >> > >> > 325 Broadway, Boulder, CO 80305
> > > >> > > >> > >> > (303) 497-5847
> > > >> > > >> > >> >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> > >>
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > --
> > > >> > > >> > > Tanya R. Peevey, PhD
> > > >> > > >> > > Research Scientist I, Global Observing Systems
Analysis
> > > (GOSA)
> > > >> > Group
> > > >> > > >> > > NOAA ESRL Global Systems Division
> > > >> > > >> > > 325 Broadway, Boulder, CO 80305
> > > >> > > >> > > (303) 497-5847
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > --
> > > >> > > >> > Tanya R. Peevey, PhD
> > > >> > > >> > Research Scientist I, Global Observing Systems
Analysis
> > (GOSA)
> > > >> Group
> > > >> > > >> > NOAA ESRL Global Systems Division
> > > >> > > >> > 325 Broadway, Boulder, CO 80305
> > > >> > > >> > (303) 497-5847
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >>
> > > >> > > >
> > > >> > > >
> > > >> > > > --
> > > >> > > > Tanya R. Peevey, PhD
> > > >> > > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > > Group
> > > >> > > > NOAA ESRL Global Systems Division
> > > >> > > > 325 Broadway, Boulder, CO 80305
> > > >> > > > (303) 497-5847
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Tanya R. Peevey, PhD
> > > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > Group
> > > >> > > NOAA ESRL Global Systems Division
> > > >> > > 325 Broadway, Boulder, CO 80305
> > > >> > > (303) 497-5847
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >> --
> > > >> Tanya R. Peevey, PhD
> > > >> Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> > > >> NOAA ESRL Global Systems Division
> > > >> 325 Broadway, Boulder, CO 80305
> > > >> (303) 497-5847
> > > >>
> > > >>
> > > >
> > >
> > >
> >
> >
> > --
> > Tanya R. Peevey, PhD
> > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > NOAA ESRL Global Systems Division
> > 325 Broadway, Boulder, CO 80305
> > (303) 497-5847
> >
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Wed Feb 17 12:41:10 2016
Tanya,
The output that MODE creates is controlled by the following entries in
the
config file:
ps_plot_flag = TRUE;
nc_pairs_flag = {
latlon = TRUE;
raw = TRUE;
object_raw = TRUE;
object_id = TRUE;
cluster_id = TRUE;
polylines = TRUE;
}
ct_stats_flag = TRUE;
You will always get a "_obj.txt" output file and by default all the
optional output files are also written. Use ps_plot_flag to disable
the
PostScript output. Use nc_pairs_flag to disable the NetCDF output.
And
use ct_stat_flag to disable the contingency table output file.
Since creating the NetCDF file is the default behavior, you probably
already have it.
I've attached an NCL script which plots some MODE NetCDF files along
with a
sample output image.
On Wed, Feb 17, 2016 at 12:07 PM, Tanya Peevey - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>
> John,
>
> Thanks for the clarification of the MODE output on grid 221. We will
use
> the ps file/graphics produced by MODE regularly, but don't know if
we will
> use them when we publish. For the moment they are fine for in-house
stuff,
> but still let me know when those grids are fixed in terms of how
they look
> in the MODE graphics. We will either create a subset of data and use
that
> in MODE or do what you suggest by having MODE write an NetCDF object
file
> if the grids aren't fixed and we are going to publish (this is still
a few
> months away). How do you create that object netcdf file? What do I
need to
> change in MODE? Also, I thought MODE already spit out a couple files
one of
> which is a file with object information? Is that file ASCII?
>
> So letting me know how to modify MODE would be great since it would
be good
> to be able to overlay the objects onto other fields. Any scripts you
may
> have for reading the NetCDF output would be helpful too.
>
> Thanks,
> Tanya
>
>
>
> On Wed, Feb 17, 2016 at 10:00 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Tanya,
> >
> > Grid 212 is commonly used:
> > http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid212.gif
> >
> > However that is limited to CONUS.
> >
> > To be clear, there really isn't a problem in the output of MODE on
grid
> > 221, other than the plotting of Russia's coastlines. The object
> definition
> > and distances should all be fine. Are you planning to use the
PostScript
> > output of MODE systematically? Do you need the graphics? MODE
can be
> > configured to write a NetCDF object file which can be plotted
using
> > whatever graphics package you'd like.
> >
> > For example, for HWT or HMT a few years back we plotted the MODE
NetCDF
> > output using NCL to overlay the forecast and observation objects.
> >
> > Thanks,
> > John
> >
> >
> >
> >
> > On Tue, Feb 16, 2016 at 10:59 AM, Tanya Peevey - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > >
> > > John,
> > >
> > > Thanks for the information. We are going to look at the time
> information
> > > again today. Also, I found the issue with the last file I tried
to use
> in
> > > MODE, the issue wasn't the grid_mapping but something else on
our end.
> > >
> > > I tried different grids on the *old_cf* files (the ones that
worked
> > > previously) and all of them had Russia looking funny. The grids
I tried
> > > were G221 and G241, both North America grids. So basically the
only
> > options
> > > I have are CONUS, Northern Hemisphere or the Globe, none of
which are
> > > appealing. I also don't like the idea of excluding Russia but
I'll copy
> > > over your code so I have it. In the meantime we will probably
create a
> > > subset of data over North America and then run that through
MODE. As
> soon
> > > as you get the North American centric grids working please let
me know.
> > >
> > > Lastly, if you know of another grid that would work please let
me know
> > but
> > > I believe I've tried all of the North American centric grids
that are
> of
> > > ~equivalent resolution of the original dataset (1x1 degree) and
are
> > > available in MODE.
> > >
> > > Thanks,
> > > Tanya
> > >
> > >
> > >
> > > On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Tanya,
> > > >
> > > > On to more issues. I think that not having the grid_mapping
should
> > work
> > > > fine, but only because this is lat/lon data and MET can use
the
> lat/lon
> > > > variables to infer the grid.
> > > >
> > > > For example, the following commands works fine:
> > > >
> > > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > >
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
> > > > NR2006.pl.1by1.7278.2006022818.cf.nc \
> > > > ~/without_grid_map.ps \
> > > > 'name="LSP_6h"; level="(*,*)";'
> > > >
> > > > Next, looking in your MODE output file:
> > > >
> > > >
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
> > > >
> > > > I see that the lead time is listed as 3169835842. The units
are
> > > supposed
> > > > to be HHMMSS so this string looks pretty odd! So there's
still an
> > issue
> > > > with the times.
> > > >
> > > > John
> > > >
> > > >
> > > >
> > > > On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway <
> johnhg at ucar.edu>
> > > > wrote:
> > > >
> > > > > Tanya,
> > > > >
> > > > > I took a close look at the issue with plotting the map for
grid
> 209.
> > > The
> > > > > issue is caused by the map data straddling the international
date
> > line.
> > > > I
> > > > > tried rescaling the longitudes from -180,180 to 0,360 but
wasn't
> able
> > > to
> > > > > resolve this issue. So this is not easily fixed. I'll need
to
> have
> > > > Randy
> > > > > Bullock look for a solution.
> > > > >
> > > > > In the meantime, you have a couple options...
> > > > > - If you're not tied to grid 209, you could easily switch to
using
> a
> > > > > different grid.
> > > > > - Or you could set up the config file to plot a set of map
data
> files
> > > > > which does not include Russia.
> > > > >
> > > > > Take a look in met-5.1/share/met/config/ConfigMap. The
"map_data"
> > > > section
> > > > > controls the map data files to be plotted. I've attached a
map
> data
> > > file
> > > > > to this message from which I've removed all the Russia data:
> > > > > country_data_minus_russia
> > > > >
> > > > > Plotting on this map data file results in the attached plot
of grid
> > > 209.
> > > > >
> > > > > FYI, you can just copy and paste the "map_data" config file
section
> > to
> > > > the
> > > > > bottom of your MODE config file. And then set the path to
the
> > attached
> > > > map
> > > > > data file. That'll overwrite the default values set in
> > ConfigMapData.
> > > > >
> > > > > John
> > > > >
> > > > >
> > > > > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA
Affiliate via
> > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> > > > >>
> > > > >> John,
> > > > >>
> > > > >> Ok, tried G209 in the new MODE and I got results but the
lines on
> > the
> > > > map
> > > > >> that are suppose to define Russia are all still wrong. See
> following
> > > > >> directory,
> > > > >>
> > > > >>
> > > >
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
> > > > >>
> > > > >> So the above worked with a NetCDF file we generated that
has
> > > > CF-compliant
> > > > >> time and grid_mapping information. We found out that the
> > grid_mapping
> > > > >> variable in the CF-compliant file was what was not allowing
us to
> > use
> > > > >> Panoply (a package for quickly viewing data in an NetCDF
file). So
> > we
> > > > >> removed the grid_mapping information and tried again, to
see if
> MODE
> > > > would
> > > > >> work and it did not. Here are where the two files are
location
> > > > >>
> > > >
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> > > > >> with *.cf.nc being the file without grid_mapping and
*.old_cf.nc
> > > being
> > > > >> the
> > > > >> file with grid_mapping.
> > > > >>
> > > > >>
> > > >
> > >
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > > >>
> > > > >>
> > > >
> > >
> >
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> > > > >>
data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> > > > >> data/gfs_test/
> > > > >> NR2006.pl.1by1.7188.2006022500.cf.nc
> > > gfs_test/config/MODEConfig_gfstest
> > > > >> -outdir gfs_test/out/mode/ -v 2
> > > > >> DEBUG 1: Default Config File:
> > > > >> /scratch4/BMC/dtc/MET/met-
5.1/share/met/config/MODEConfig_default
> > > > >> DEBUG 1: Match Config File:
gfs_test/config/MODEConfig_gfstest
> > > > >> DEBUG 1: Merge Config File:
gfs_test/config/MODEConfig_gfstest
> > > > >> ERROR :
> > > > >> ERROR : Trouble reading forecast file "data/gfs_test/
> > > > >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> > > > >> ERROR :
> > > > >>
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
> > > > >>
> > > > >>
> > > >
> > >
> >
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > > >>
> > > > >> Basically, want to try and fix the regridding to the G209
grid so
> > > Russia
> > > > >> looks normal and want to confirm that we do need to use the
*.
> > > old_cf.nc
> > > > >> files instead of the other within MET.
> > > > >>
> > > > >> Thanks,
> > > > >> Tanya
> > > > >>
> > > > >> P.S. Having the regird option in MODE is great. Now I don't
have
> to
> > > > create
> > > > >> another set of files before processing the data. Saves me
some
> disk
> > > > space
> > > > >> :-).
> > > > >>
> > > > >>
> > > > >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> > Tanya,
> > > > >> >
> > > > >> > I can look more closely tomorrow, but some initial
thoughts are:
> > > > >> >
> > > > >> > - yes, please use met-5.1 and use the default mode 5.1
config
> file
> > > > >> > - you could use the regrid tool, but instead fill out the
regrid
> > > > >> section of
> > > > >> > the mode config file. Try setting to_grid = "G209";
> > > > >> >
> > > > >> > That'll automatically regrid the global data in the fly.
> > > > >> >
> > > > >> > That's all new in 5.1.
> > > > >> >
> > > > >> > John
> > > > >> >
> > > > >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA
Affiliate
> via
> > > RT <
> > > > >> > met_help at ucar.edu> wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >
> > > > >> > >
> > > > >> > > John,
> > > > >> > >
> > > > >> > > To clarify my previous email, I'm trying to use the
regrid
> tool
> > to
> > > > >> > reverse
> > > > >> > > the latitudes in our .nc files and focus on North
America
> rather
> > > > than
> > > > >> the
> > > > >> > > whole global. I just tried the regrid tool with the
grid set
> to
> > > G209
> > > > >> and
> > > > >> > it
> > > > >> > > worked (both corrected the latitude issue and focused
on North
> > > > >> America)
> > > > >> > but
> > > > >> > > with one small problem, there are some weird lines
where
> Russia
> > is
> > > > >> > suppose
> > > > >> > > to be. Do you know what the problem is? Is there
another grid
> > you
> > > > >> could
> > > > >> > > recommend if the output from the regrid tool on the
G209 grid
> > > can't
> > > > be
> > > > >> > > fixed? Or could I define my own grid using another MET
tool?
> > > > >> > >
> > > > >> > > If all of the above fails we can modify our NetCDF
files but I
> > > hope
> > > > >> to be
> > > > >> > > able to use the regrid tool.
> > > > >> > >
> > > > >> > > All plots from my tests with the regrid tool can be
found
> here:
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> > > > >> > > Data used to make the plots can be found here:
> > > > >> > >
> > > > >>
> > >
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> > > > >> > >
> > > > >> > > Still would like to see some documentation on the
regrid tool.
> > > > >> > >
> > > > >> > > Lastly, after using the regrid tool I had to use the
newest
> > > version
> > > > of
> > > > >> > MET
> > > > >> > > (met-5.1). Is this version good to go? Should I change
my MET
> > path
> > > > to
> > > > >> > this
> > > > >> > > version? Is there documentation on this version yet?
> > > > >> > >
> > > > >> > > Thanks!
> > > > >> > > Tanya
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
> Affiliate <
> > > > >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > > >> > >
> > > > >> > > > John,
> > > > >> > > >
> > > > >> > > > So Jason and I got our data working with MODE.
However, I
> have
> > > one
> > > > >> > > > question. Right now MODE automatically plots the full
grid
> > > > >> available,
> > > > >> > in
> > > > >> > > > this case a global plot. We are only focused on North
> > America. I
> > > > was
> > > > >> > > > thinking of using the regrid tool to change the grid
from
> > global
> > > > to
> > > > >> > North
> > > > >> > > > American using NCEP grid 209. Can the regrid tool
take any
> of
> > > the
> > > > >> NCEP
> > > > >> > > > grids found on the following website (
> > > > >> > > >
http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html)
> and
> > > > apply
> > > > >> it
> > > > >> > to
> > > > >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
> Specifications'
> > > by
> > > > >> it)?
> > > > >> > > > Also, right now we reverse the latitude indices so
that
> > > everything
> > > > >> is
> > > > >> > > right
> > > > >> > > > side up but this makes it so another product we use
can't
> view
> > > the
> > > > >> .nc
> > > > >> > > > files. So my question to you is would we be able to
do
> reverse
> > > the
> > > > >> > > > latitudes using the MET regrid tool with NCEP grid
209 or
> > would
> > > we
> > > > >> need
> > > > >> > > to
> > > > >> > > > use the regrid tool twice, once with grid 004 and
once with
> > grid
> > > > >> 209,
> > > > >> > or
> > > > >> > > > would another NCEP grid work?
> > > > >> > > >
> > > > >> > > > Oh, and, could you direct me to some documentation on
the
> > regrid
> > > > >> tool.
> > > > >> > I
> > > > >> > > > can see the usage from the command line but that
doesn't
> > really
> > > > >> address
> > > > >> > > my
> > > > >> > > > above question. Also, I didn't see anything in the
MET
> manual.
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > Tanya
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway
via RT <
> > > > >> > > > met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >
> > > > >> > > >> Tanya,
> > > > >> > > >>
> > > > >> > > >> I took a close look at your current data file:
> > > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > > >> > > >>
> > > > >> > > >> Running it through the debugger, I see that the
problem
> comes
> > > > from
> > > > >> > > parsing
> > > > >> > > >> this time string:
> > > > >> > > >> "hours since 2005-05-01 (12:00)"
> > > > >> > > >>
> > > > >> > > >> The parsing code is expecting that to be in HH:MM:SS
> format,
> > > not
> > > > >> > > HH:MM. I
> > > > >> > > >> changed it to this:
> > > > >> > > >> "hours since 2005-05-01 12:00:00"
> > > > >> > > >>
> > > > >> > > >> And then I was able to run the plot_data_plane tool
on it.
> > > > >> However,
> > > > >> > the
> > > > >> > > >> resulting image is still upside-down and backwards.
See
> > > attached
> > > > >> > > >> (test.png).
> > > > >> > > >>
> > > > >> > > >> To answer the question about grid x/y mapping, the
answer
> is
> > > no.
> > > > >> For
> > > > >> > > >> lat/lon grids with no projection info defined, MET
uses the
> > > > lat/lon
> > > > >> > > values
> > > > >> > > >> and intuits the projection info. For non-lat/lon
grids,
> the
> > > > >> > projection
> > > > >> > > >> info would need to be explicitly defined since
there's no
> > > simple
> > > > >> way
> > > > >> > of
> > > > >> > > >> intuiting a grid definition from latitude and
longitude
> > values.
> > > > >> > > >>
> > > > >> > > >> So about the data being upside-down and backwards...
there
> > are
> > > > >> flags
> > > > >> > in
> > > > >> > > >> GRIB files which indicate the order in which the
data is
> > > written.
> > > > >> > When
> > > > >> > > >> reading data from GRIB files, MET uses those flags
to
> > determine
> > > > >> where
> > > > >> > to
> > > > >> > > >> place things and orient the data right-side-up.
The
> > > > CF-compliant
> > > > >> > > NetCDF
> > > > >> > > >> code logic is much simpler... it just reads the data
in the
> > > order
> > > > >> it
> > > > >> > is
> > > > >> > > >> encoded in the file and doesn't check which way is
up!
> > > > >> > > >>
> > > > >> > > >> I'm not really sure what to do here. Our support
for
> > > > CF-compliant
> > > > >> > > NetCDF
> > > > >> > > >> is still relatively new, and I'm not sure what the
> > expectations
> > > > >> should
> > > > >> > > be.
> > > > >> > > >> Should we add logic to the code to detect this
situation
> and
> > > > >> handle it
> > > > >> > > >> better? Or is the code working as expected? I'm
not
> really
> > > > sure.
> > > > >> > > >>
> > > > >> > > >> I do see that ncview handles it better!
> > > > >> > > >>
> > > > >> > > >> With the current version of MET, you have 2 choices:
> > > > >> > > >>
> > > > >> > > >> (1) You could write lat/lon and data values in
reverse
> order
> > > > (i.e.
> > > > >> > > >> increasing latitude values) to "right the ship".
> > > > >> > > >>
> > > > >> > > >> (2) Or you could leave things as-is and make use of
the
> > > automated
> > > > >> > > >> regridding capability within the MET tools.
> > > > >> > > >>
> > > > >> > > >> For example:
> > > > >> > > >>
> > > > >> > > >> # run regrid_data_plane utility to regrid to NCEP
grid 4
> > > > >> > > >> regrid_data_plane \
> > > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc
G004 \
> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > > > >> > > >> -field 'name="T2"; level="(*,*)";'
> > > > >> > > >>
> > > > >> > > >> # run plot_data_plane on the output
> > > > >> > > >> plot_data_plane \
> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> > > > >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> > > > >> > > >>
> > > > >> > > >> The result (test_regrid.png) is attached.
> > > > >> > > >>
> > > > >> > > >> John
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
> Affiliate
> > > via
> > > > >> RT <
> > > > >> > > >> met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >>
> > > > >> > > >> >
> > > > >> > > >> > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > > >
> > > > >> > > >> >
> > > > >> > > >> > John,
> > > > >> > > >> >
> > > > >> > > >> > My colleague, Jason English (cc'd on email), that
is
> > > generating
> > > > >> the
> > > > >> > > >> > NETCF_NCCF files has another question about the CF
> format.
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > > >> > > >> > Do we need the grid_mapping x and y variables if
we are
> > using
> > > > the
> > > > >> > > >> lambert
> > > > >> > > >> > cylindrical equal area grid?
> > > > >> > > >> > If so, how do we convert 1-dimensional lat/lon
> coordinates
> > to
> > > > >> the 2d
> > > > >> > > >> > coordinates with y,x?
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > > > >> > > >> >
> > > > >> > > >> > Thank you,
> > > > >> > > >> > Tanya Peevey
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey -
NOAA
> > > Affiliate <
> > > > >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> > > > >> > > >> >
> > > > >> > > >> > > John,
> > > > >> > > >> > >
> > > > >> > > >> > > Thank you for the information. We generated a
new file
> > and
> > > I
> > > > >> tried
> > > > >> > > the
> > > > >> > > >> > > plot_data_plane tool on it and got the following
error.
> > Do
> > > > you
> > > > >> > have
> > > > >> > > an
> > > > >> > > >> > idea
> > > > >> > > >> > > of what could be wrong? Sorry to bug you so much
and
> > thanks
> > > > in
> > > > >> > > advance
> > > > >> > > >> > for
> > > > >> > > >> > > the help.
> > > > >> > > >> > >
> > > > >> > > >> > > Here's the location of the file:
> > > > >> > > >> > >
> > > > >> > > >>
> > > > >> > >
> > > > >>
> > > >
> > /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> > > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > > >> > > >> > >
> > > > >> > > >> > > Here's the error:
> > > > >> > > >> > >
> > > > >> >
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > > > >> > > >> > >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> > > > >> > > >> > > data/gfs_test/
> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > > >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
> > > > >> > > >> > > file_type=NETCDF_NCCF;'
> > > > >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> > > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> > > > >> > > >> > > ERROR :
> > > > >> > > >> > > ERROR : StringArray::operator[](int) const ->
range
> > check
> > > > >> error!
> > > > >> > > >> > > ERROR :
> > > > >> > > >> > >
> > > > >> >
> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> > > > >> > > >> > >
> > > > >> > > >> > > Thank you ,
> > > > >> > > >> > > Tanya
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley
Gotway via
> > RT <
> > > > >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >> > >
> > > > >> > > >> > >> Tanya,
> > > > >> > > >> > >>
> > > > >> > > >> > >> Sorry for the delay. I was out of the office
on
> Friday.
> > > > >> Using
> > > > >> > > >> "LSP_6h"
> > > > >> > > >> > >> for the name and "(*,*)" for the level should
do the
> > > trick.
> > > > >> > > >> > >>
> > > > >> > > >> > >> When trying to get the MET tools to read a new
> dataset,
> > I
> > > > >> always
> > > > >> > > >> start
> > > > >> > > >> > by
> > > > >> > > >> > >> using the plot_data_plane tool:
> > > > >> > > >> > >>
> > > > >> > > >> > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
> > > > >> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
\
> > > > >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> > > > >> > > file_type=NETCDF_NCCF;'
> > > > >> > > >> > >>
> > > > >> > > >> > >> I use that to make sure the name and level
settings
> work
> > > as
> > > > >> > > expected,
> > > > >> > > >> > and
> > > > >> > > >> > >> it also creates an output plot so I can see
that MET
> is
> > > (or
> > > > is
> > > > >> > not)
> > > > >> > > >> > >> reading/plotting the data as expected.
> > > > >> > > >> > >>
> > > > >> > > >> > >> That configuration string you pass to the the
> > > > plot_data_plane
> > > > >> > tool
> > > > >> > > is
> > > > >> > > >> > >> exactly the same settings as are supported in
the
> > > > >> configuration
> > > > >> > > >> files.
> > > > >> > > >> > In
> > > > >> > > >> > >> fact, we read that string with the same code
that
> parses
> > > the
> > > > >> > config
> > > > >> > > >> > files.
> > > > >> > > >> > >>
> > > > >> > > >> > >> I find using plot_data_plane very helpful.
> > > > >> > > >> > >>
> > > > >> > > >> > >> Thanks,
> > > > >> > > >> > >> John
> > > > >> > > >> > >>
> > > > >> > > >> > >>
> > > > >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey -
NOAA
> > > > Affiliate
> > > > >> via
> > > > >> > > RT
> > > > >> > > >> <
> > > > >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >> > >>
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > > >> > >
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > John,
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > Thanks for the info. We are going to work on
> changing
> > > our
> > > > >> file
> > > > >> > > >> format
> > > > >> > > >> > >> and
> > > > >> > > >> > >> > then I'll get back to you.
> > > > >> > > >> > >> > Did you see my other question about how I
specified
> > > 'name'
> > > > >> andhow to round to nearest 0.5
> > > > >>
> > > > >> > > >> 'level'
> > > > >> > > >> > >> for
> > > > >> > > >> > >> > the precip. field in my config file for MODE?
Was
> how
> > I
> > > > did
> > > > >> it
> > > > >> > > ok?
> > > > >> > > >> I
> > > > >> > > >> > ask
> > > > >> > > >> > >> > because the manual was confusing on this
point. If
> my
> > > > syntax
> > > > >> > was
> > > > >> > > >> > >> incorrect
> > > > >> > > >> > >> > could you please email me the correction.
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > Thanks,
> > > > >> > > >> > >> > Tanya
> > > > >> > > >> > >> >
> > > > >> > > >> > >> >
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley
Gotway
> > via
> > > > RT <
> > > > >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > > Tanya,
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > Here's a website about it:
> > > > >> > > >> > >> > > http://cfconventions.org/latest.html
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > And I posted a sample CF-compliant NetCDF
file
> here:
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> >
> > > > >> > > >> > >>
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > The relevant pieces in there are...
> > > > >> > > >> > >> > > - The global "Conventions" attribute set to
> > "CF-...".
> > > > >> > > >> > >> > > - The time dimension/variable define the
forecast
> > > valid
> > > > >> time.
> > > > >> > > >> > >> > > - The forecast_reference_time variable
defines the
> > > model
> > > > >> > > >> > >> initialization
> > > > >> > > >> > >> > > time.
> > > > >> > > >> > >> > > - The grid_mapping, x, and y variables
define the
> > grid
> > > > >> > > definition
> > > > >> > > >> > >> > > information.
> > > > >> > > >> > >> > > - The grid_mapping variable attributes map
those
> > > > >> variables to
> > > > >> > > the
> > > > >> > > >> > >> > relevant
> > > > >> > > >> > >> > > grid definition.
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > Hope this helps.
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > Thanks,
> > > > >> > > >> > >> > > John
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya
Peevey -
> NOAA
> > > > >> > Affiliate
> > > > >> > > >> via
> > > > >> > > >> > RT
> > > > >> > > >> > >> <
> > > > >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > <URL:
> > > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> > > > >> > > >> >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > John,
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > To clarify my previous request of
information. I
> > > meant
> > > > >> to
> > > > >> > ask
> > > > >> > > >> was
> > > > >> > > >> > >> the
> > > > >> > > >> > >> > > > following: Do you know of any good
documentation
> > on
> > > > how
> > > > >> to
> > > > >> > > >> make a
> > > > >> > > >> > >> > NetCDF
> > > > >> > > >> > >> > > > file CF-compliant? Also, do you have an
example
> > .nc
> > > > file
> > > > >> > that
> > > > >> > > >> is
> > > > >> > > >> > >> NetCDF
> > > > >> > > >> > >> > > > CF-compliant that we could compare to?
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > Thanks,
> > > > >> > > >> > >> > > > Tanya
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya
Peevey -
> > NOAA
> > > > >> > > Affiliate <
> > > > >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>>
wrote:
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > > John,
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > > We generated the NetCDF files ourselves
so we
> > > could
> > > > >> > > probably
> > > > >> > > >> > make
> > > > >> > > >> > >> it
> > > > >> > > >> > >> > > work
> > > > >> > > >> > >> > > > > for MET. Do you know how to get it into
> > > CF-compliant
> > > > >> > NetCDF
> > > > >> > > >> > >> format?
> > > > >> > > >> > >> > > > > Yes, we have grib files but the reason
we
> > created
> > > > >> NetCDF
> > > > >> > > >> files
> > > > >> > > >> > was
> > > > >> > > >> > >> > > > because
> > > > >> > > >> > >> > > > > original the NR... file had accumulated
precip
> > > over
> > > > >> the
> > > > >> > > whole
> > > > >> > > >> > >> > forecast
> > > > >> > > >> > >> > > > > where the other file has 6h
accumulation data,
> > so
> > > we
> > > > >> need
> > > > >> > > to
> > > > >> > > >> > >> convert
> > > > >> > > >> > >> > > the
> > > > >> > > >> > >> > > > > NR... precip information into 6h
accumulation
> > > (they
> > > > >> had
> > > > >> > > >> > different
> > > > >> > > >> > >> > units
> > > > >> > > >> > >> > > > > too, so we need to change that too).
Since we
> > had
> > > to
> > > > >> > change
> > > > >> > > >> the
> > > > >> > > >> > >> units
> > > > >> > > >> > >> > > > too I
> > > > >> > > >> > >> > > > > think use generating our own NetCDF
file is
> best
> > > but
> > > > >> we
> > > > >> > > just
> > > > >> > > >> > need
> > > > >> > > >> > >> to
> > > > >> > > >> > >> > > get
> > > > >> > > >> > >> > > > it
> > > > >> > > >> > >> > > > > in the right format.
> > > > >> > > >> > >> > > > > What is your opinion on the best way to
> approach
> > > > this?
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > > If we get it into the right format is
how I
> > > specify
> > > > >> the
> > > > >> > > >> 'name'
> > > > >> > > >> > and
> > > > >> > > >> > >> > > > 'level'
> > > > >> > > >> > >> > > > > of the field correct? It was hard to
figure
> out
> > > the
> > > > >> > correct
> > > > >> > > >> > >> notation
> > > > >> > > >> > >> > > from
> > > > >> > > >> > >> > > > > the manual.
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > > Thanks,
> > > > >> > > >> > >> > > > > Tanya
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John
Halley
> > Gotway
> > > > via
> > > > >> > RT <
> > > > >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>>
wrote:
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > >> Hi Tanya,
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> I took a look at the gridded NetCDF
data
> you're
> > > > >> trying
> > > > >> > to
> > > > >> > > >> pass
> > > > >> > > >> > to
> > > > >> > > >> > >> > > MODE.
> > > > >> > > >> > >> > > > >> Unfortunately, MET doesn't support all
types
> of
> > > > >> gridded
> > > > >> > > >> NetCDF
> > > > >> > > >> > >> > files.
> > > > >> > > >> > >> > > > >> Instead, it only supports the MET
NetCDF
> format
> > > > (e.g.
> > > > >> > the
> > > > >> > > >> > output
> > > > >> > > >> > >> of
> > > > >> > > >> > >> > > the
> > > > >> > > >> > >> > > > >> pcp_combine tool), the output of the
WRF
> > pinterp
> > > > (or
> > > > >> > > >> wrfinterp)
> > > > >> > > >> > >> > > > utilities,
> > > > >> > > >> > >> > > > >> and CF-compliant NetCDF files.
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> Looking at "
> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> > > > >> > "
> > > > >> > > I
> > > > >> > > >> see
> > > > >> > > >> > >> that
> > > > >> > > >> > >> > > it
> > > > >> > > >> > >> > > > >> follows none of those conventions.
However,
> it
> > > is
> > > > >> > closest
> > > > >> > > >> to
> > > > >> > > >> > >> > > > CF-compliant
> > > > >> > > >> > >> > > > >> NetCDF.
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> When using a new data file format,
instead of
> > > > >> running it
> > > > >> > > >> > through
> > > > >> > > >> > >> > MODE,
> > > > >> > > >> > >> > > > >> I'd suggest using plot_data_plane to
make
> sure
> > > MET
> > > > is
> > > > >> > > >> reading
> > > > >> > > >> > it
> > > > >> > > >> > >> > > > >> correctly. I ran met-
5.1/plot_data_plane as
> > > > follows:
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >>
> > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> > > > >> > > >> > >> > > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> \
> > > > >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
> > > > >> > > >> > file_type=NETCDF_NCCF;'
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> The resulting image is attached.
There are
> > > several
> > > > >> > issues
> > > > >> > > >> > here:
> > > > >> > > >> > >> > > > >> - I had to tell plot_data_plane to
interpret
> > this
> > > > as
> > > > >> > > >> > CF-compliant
> > > > >> > > >> > >> > > NetCDF
> > > > >> > > >> > >> > > > >> using NETCDF_NCCF.
> > > > >> > > >> > >> > > > >> - The timing info in that file isn't
> specified
> > in
> > > > the
> > > > >> > > >> > >> CF-compliant
> > > > >> > > >> > >> > > > way...
> > > > >> > > >> > >> > > > >> so MET can't discern the
initialization,
> valid,
> > > and
> > > > >> lead
> > > > >> > > >> times.
> > > > >> > > >> > >> > > > >> - And then there's the obvious problem
(in
> the
> > > > >> image) of
> > > > >> > > the
> > > > >> > > >> > >> world
> > > > >> > > >> > >> > > being
> > > > >> > > >> > >> > > > >> upside down and backwards!
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> So while we did get *something*, there
are
> > > > obviously
> > > > >> a
> > > > >> > lot
> > > > >> > > >> of
> > > > >> > > >> > >> > issues.
> > > > >> > > >> > >> > > > >> Basically, MET doesn't support the
gridded
> > NetCDF
> > > > >> file
> > > > >> > > >> format
> > > > >> > > >> > you
> > > > >> > > >> > >> > are
> > > > >> > > >> > >> > > > >> using. Is this data possibly
available in
> > GRIB?
> > > > Or
> > > > >> do
> > > > >> > > you
> > > > >> > > >> > have
> > > > >> > > >> > >> > > control
> > > > >> > > >> > >> > > > >> over the NetCDF file format in use?
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >> Thanks,
> > > > >> > > >> > >> > > > >> John
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >>
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > > > --
> > > > >> > > >> > >> > > > > Tanya R. Peevey, PhD
> > > > >> > > >> > >> > > > > Research Scientist I, Global Observing
Systems
> > > > >> Analysis
> > > > >> > > >> (GOSA)
> > > > >> > > >> > >> Group
> > > > >> > > >> > >> > > > > NOAA ESRL Global Systems Division
> > > > >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> > > > >> > > >> > >> > > > > (303) 497-5847
> > > > >> > > >> > >> > > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > > --
> > > > >> > > >> > >> > > > Tanya R. Peevey, PhD
> > > > >> > > >> > >> > > > Research Scientist I, Global Observing
Systems
> > > > Analysis
> > > > >> > > (GOSA)
> > > > >> > > >> > Group
> > > > >> > > >> > >> > > > NOAA ESRL Global Systems Division
> > > > >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> > > > >> > > >> > >> > > > (303) 497-5847
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > > >
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> > >
> > > > >> > > >> > >> >
> > > > >> > > >> > >> >
> > > > >> > > >> > >> > --
> > > > >> > > >> > >> > Tanya R. Peevey, PhD
> > > > >> > > >> > >> > Research Scientist I, Global Observing
Systems
> > Analysis
> > > > >> (GOSA)
> > > > >> > > >> Group
> > > > >> > > >> > >> > NOAA ESRL Global Systems Division
> > > > >> > > >> > >> > 325 Broadway, Boulder, CO 80305
> > > > >> > > >> > >> > (303) 497-5847
> > > > >> > > >> > >> >
> > > > >> > > >> > >> >
> > > > >> > > >> > >>
> > > > >> > > >> > >>
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > --
> > > > >> > > >> > > Tanya R. Peevey, PhD
> > > > >> > > >> > > Research Scientist I, Global Observing Systems
Analysis
> > > > (GOSA)
> > > > >> > Group
> > > > >> > > >> > > NOAA ESRL Global Systems Division
> > > > >> > > >> > > 325 Broadway, Boulder, CO 80305
> > > > >> > > >> > > (303) 497-5847
> > > > >> > > >> > >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > --
> > > > >> > > >> > Tanya R. Peevey, PhD
> > > > >> > > >> > Research Scientist I, Global Observing Systems
Analysis
> > > (GOSA)
> > > > >> Group
> > > > >> > > >> > NOAA ESRL Global Systems Division
> > > > >> > > >> > 325 Broadway, Boulder, CO 80305
> > > > >> > > >> > (303) 497-5847
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Tanya R. Peevey, PhD
> > > > >> > > > Research Scientist I, Global Observing Systems
Analysis
> (GOSA)
> > > > Group
> > > > >> > > > NOAA ESRL Global Systems Division
> > > > >> > > > 325 Broadway, Boulder, CO 80305
> > > > >> > > > (303) 497-5847
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > --
> > > > >> > > Tanya R. Peevey, PhD
> > > > >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA)
> > > Group
> > > > >> > > NOAA ESRL Global Systems Division
> > > > >> > > 325 Broadway, Boulder, CO 80305
> > > > >> > > (303) 497-5847
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Tanya R. Peevey, PhD
> > > > >> Research Scientist I, Global Observing Systems Analysis
(GOSA)
> Group
> > > > >> NOAA ESRL Global Systems Division
> > > > >> 325 Broadway, Boulder, CO 80305
> > > > >> (303) 497-5847
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Tanya R. Peevey, PhD
> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> > > NOAA ESRL Global Systems Division
> > > 325 Broadway, Boulder, CO 80305
> > > (303) 497-5847
> > >
> > >
> >
> >
>
>
> --
> Tanya R. Peevey, PhD
> Research Scientist I, Global Observing Systems Analysis (GOSA) Group
> NOAA ESRL Global Systems Division
> 325 Broadway, Boulder, CO 80305
> (303) 497-5847
>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: John Halley Gotway
Time: Wed Feb 17 12:45:08 2016
Sorry, sent that too quickly. Here's the image.
This NCL script reads MODE object information from several different
ensemble members and the ensemble mean. It plots thin outlines for
ensemble member objects and a thick outline for the ensemble mean.
The
observations are plotting in solid colors: red for matched and blue
for
unmatched (matched and unmatched compared to the ensemble mean).
So this is an example of using the MODE NetCDF output to display
information about an ensemble.
Hope that helps.
Thanks,
John
On Wed, Feb 17, 2016 at 12:40 PM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Tanya,
>
> The output that MODE creates is controlled by the following entries
in the
> config file:
>
> ps_plot_flag = TRUE;
> nc_pairs_flag = {
> latlon = TRUE;
> raw = TRUE;
> object_raw = TRUE;
> object_id = TRUE;
> cluster_id = TRUE;
> polylines = TRUE;
> }
> ct_stats_flag = TRUE;
>
> You will always get a "_obj.txt" output file and by default all the
> optional output files are also written. Use ps_plot_flag to disable
the
> PostScript output. Use nc_pairs_flag to disable the NetCDF output.
And
> use ct_stat_flag to disable the contingency table output file.
>
> Since creating the NetCDF file is the default behavior, you probably
> already have it.
>
> I've attached an NCL script which plots some MODE NetCDF files along
with
> a sample output image.
>
>
>
>
>
> On Wed, Feb 17, 2016 at 12:07 PM, Tanya Peevey - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>>
>> John,
>>
>> Thanks for the clarification of the MODE output on grid 221. We
will use
>> the ps file/graphics produced by MODE regularly, but don't know if
we will
>> use them when we publish. For the moment they are fine for in-house
stuff,
>> but still let me know when those grids are fixed in terms of how
they look
>> in the MODE graphics. We will either create a subset of data and
use that
>> in MODE or do what you suggest by having MODE write an NetCDF
object file
>> if the grids aren't fixed and we are going to publish (this is
still a few
>> months away). How do you create that object netcdf file? What do I
need to
>> change in MODE? Also, I thought MODE already spit out a couple
files one
>> of
>> which is a file with object information? Is that file ASCII?
>>
>> So letting me know how to modify MODE would be great since it would
be
>> good
>> to be able to overlay the objects onto other fields. Any scripts
you may
>> have for reading the NetCDF output would be helpful too.
>>
>> Thanks,
>> Tanya
>>
>>
>>
>> On Wed, Feb 17, 2016 at 10:00 AM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Tanya,
>> >
>> > Grid 212 is commonly used:
>> > http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid212.gif
>> >
>> > However that is limited to CONUS.
>> >
>> > To be clear, there really isn't a problem in the output of MODE
on grid
>> > 221, other than the plotting of Russia's coastlines. The object
>> definition
>> > and distances should all be fine. Are you planning to use the
>> PostScript
>> > output of MODE systematically? Do you need the graphics? MODE
can be
>> > configured to write a NetCDF object file which can be plotted
using
>> > whatever graphics package you'd like.
>> >
>> > For example, for HWT or HMT a few years back we plotted the MODE
NetCDF
>> > output using NCL to overlay the forecast and observation objects.
>> >
>> > Thanks,
>> > John
>> >
>> >
>> >
>> >
>> > On Tue, Feb 16, 2016 at 10:59 AM, Tanya Peevey - NOAA Affiliate
via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > >
>> > > John,
>> > >
>> > > Thanks for the information. We are going to look at the time
>> information
>> > > again today. Also, I found the issue with the last file I tried
to
>> use in
>> > > MODE, the issue wasn't the grid_mapping but something else on
our end.
>> > >
>> > > I tried different grids on the *old_cf* files (the ones that
worked
>> > > previously) and all of them had Russia looking funny. The grids
I
>> tried
>> > > were G221 and G241, both North America grids. So basically the
only
>> > options
>> > > I have are CONUS, Northern Hemisphere or the Globe, none of
which are
>> > > appealing. I also don't like the idea of excluding Russia but
I'll
>> copy
>> > > over your code so I have it. In the meantime we will probably
create a
>> > > subset of data over North America and then run that through
MODE. As
>> soon
>> > > as you get the North American centric grids working please let
me
>> know.
>> > >
>> > > Lastly, if you know of another grid that would work please let
me know
>> > but
>> > > I believe I've tried all of the North American centric grids
that are
>> of
>> > > ~equivalent resolution of the original dataset (1x1 degree) and
are
>> > > available in MODE.
>> > >
>> > > Thanks,
>> > > Tanya
>> > >
>> > >
>> > >
>> > > On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > > Tanya,
>> > > >
>> > > > On to more issues. I think that not having the grid_mapping
should
>> > work
>> > > > fine, but only because this is lat/lon data and MET can use
the
>> lat/lon
>> > > > variables to infer the grid.
>> > > >
>> > > > For example, the following commands works fine:
>> > > >
>> > > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
>> > > > NR2006.pl.1by1.7278.2006022818.cf.nc \
>> > > > ~/without_grid_map.ps \
>> > > > 'name="LSP_6h"; level="(*,*)";'
>> > > >
>> > > > Next, looking in your MODE output file:
>> > > >
>> > > >
>> > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
>> > > >
>> > > > I see that the lead time is listed as 3169835842. The units
are
>> > > supposed
>> > > > to be HHMMSS so this string looks pretty odd! So there's
still an
>> > issue
>> > > > with the times.
>> > > >
>> > > > John
>> > > >
>> > > >
>> > > >
>> > > > On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway <
>> johnhg at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Tanya,
>> > > > >
>> > > > > I took a close look at the issue with plotting the map for
grid
>> 209.
>> > > The
>> > > > > issue is caused by the map data straddling the
international date
>> > line.
>> > > > I
>> > > > > tried rescaling the longitudes from -180,180 to 0,360 but
wasn't
>> able
>> > > to
>> > > > > resolve this issue. So this is not easily fixed. I'll
need to
>> have
>> > > > Randy
>> > > > > Bullock look for a solution.
>> > > > >
>> > > > > In the meantime, you have a couple options...
>> > > > > - If you're not tied to grid 209, you could easily switch
to
>> using a
>> > > > > different grid.
>> > > > > - Or you could set up the config file to plot a set of map
data
>> files
>> > > > > which does not include Russia.
>> > > > >
>> > > > > Take a look in met-5.1/share/met/config/ConfigMap. The
"map_data"
>> > > > section
>> > > > > controls the map data files to be plotted. I've attached a
map
>> data
>> > > file
>> > > > > to this message from which I've removed all the Russia
data:
>> > > > > country_data_minus_russia
>> > > > >
>> > > > > Plotting on this map data file results in the attached plot
of
>> grid
>> > > 209.
>> > > > >
>> > > > > FYI, you can just copy and paste the "map_data" config file
>> section
>> > to
>> > > > the
>> > > > > bottom of your MODE config file. And then set the path to
the
>> > attached
>> > > > map
>> > > > > data file. That'll overwrite the default values set in
>> > ConfigMapData.
>> > > > >
>> > > > > John
>> > > > >
>> > > > >
>> > > > > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA
Affiliate via
>> > RT <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > >>
>> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > > > >>
>> > > > >> John,
>> > > > >>
>> > > > >> Ok, tried G209 in the new MODE and I got results but the
lines on
>> > the
>> > > > map
>> > > > >> that are suppose to define Russia are all still wrong. See
>> following
>> > > > >> directory,
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
>> > > > >>
>> > > > >> So the above worked with a NetCDF file we generated that
has
>> > > > CF-compliant
>> > > > >> time and grid_mapping information. We found out that the
>> > grid_mapping
>> > > > >> variable in the CF-compliant file was what was not
allowing us to
>> > use
>> > > > >> Panoply (a package for quickly viewing data in an NetCDF
file).
>> So
>> > we
>> > > > >> removed the grid_mapping information and tried again, to
see if
>> MODE
>> > > > would
>> > > > >> work and it did not. Here are where the two files are
location
>> > > > >>
>> > > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
>> > > > >> with *.cf.nc being the file without grid_mapping and
*.old_cf.nc
>> > > being
>> > > > >> the
>> > > > >> file with grid_mapping.
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
>> > > > >>
data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
>> > > > >> data/gfs_test/
>> > > > >> NR2006.pl.1by1.7188.2006022500.cf.nc
>> > > gfs_test/config/MODEConfig_gfstest
>> > > > >> -outdir gfs_test/out/mode/ -v 2
>> > > > >> DEBUG 1: Default Config File:
>> > > > >> /scratch4/BMC/dtc/MET/met-
5.1/share/met/config/MODEConfig_default
>> > > > >> DEBUG 1: Match Config File:
gfs_test/config/MODEConfig_gfstest
>> > > > >> DEBUG 1: Merge Config File:
gfs_test/config/MODEConfig_gfstest
>> > > > >> ERROR :
>> > > > >> ERROR : Trouble reading forecast file "data/gfs_test/
>> > > > >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
>> > > > >> ERROR :
>> > > > >>
>> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-5.1>
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > > >>
>> > > > >> Basically, want to try and fix the regridding to the G209
grid so
>> > > Russia
>> > > > >> looks normal and want to confirm that we do need to use
the *.
>> > > old_cf.nc
>> > > > >> files instead of the other within MET.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Tanya
>> > > > >>
>> > > > >> P.S. Having the regird option in MODE is great. Now I
don't have
>> to
>> > > > create
>> > > > >> another set of files before processing the data. Saves me
some
>> disk
>> > > > space
>> > > > >> :-).
>> > > > >>
>> > > > >>
>> > > > >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via RT
<
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >> > Tanya,
>> > > > >> >
>> > > > >> > I can look more closely tomorrow, but some initial
thoughts
>> are:
>> > > > >> >
>> > > > >> > - yes, please use met-5.1 and use the default mode 5.1
config
>> file
>> > > > >> > - you could use the regrid tool, but instead fill out
the
>> regrid
>> > > > >> section of
>> > > > >> > the mode config file. Try setting to_grid = "G209";
>> > > > >> >
>> > > > >> > That'll automatically regrid the global data in the fly.
>> > > > >> >
>> > > > >> > That's all new in 5.1.
>> > > > >> >
>> > > > >> > John
>> > > > >> >
>> > > > >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA
Affiliate
>> via
>> > > RT <
>> > > > >> > met_help at ucar.edu> wrote:
>> > > > >> >
>> > > > >> > >
>> > > > >> > > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
>> > > > >> > >
>> > > > >> > > John,
>> > > > >> > >
>> > > > >> > > To clarify my previous email, I'm trying to use the
regrid
>> tool
>> > to
>> > > > >> > reverse
>> > > > >> > > the latitudes in our .nc files and focus on North
America
>> rather
>> > > > than
>> > > > >> the
>> > > > >> > > whole global. I just tried the regrid tool with the
grid set
>> to
>> > > G209
>> > > > >> and
>> > > > >> > it
>> > > > >> > > worked (both corrected the latitude issue and focused
on
>> North
>> > > > >> America)
>> > > > >> > but
>> > > > >> > > with one small problem, there are some weird lines
where
>> Russia
>> > is
>> > > > >> > suppose
>> > > > >> > > to be. Do you know what the problem is? Is there
another grid
>> > you
>> > > > >> could
>> > > > >> > > recommend if the output from the regrid tool on the
G209 grid
>> > > can't
>> > > > be
>> > > > >> > > fixed? Or could I define my own grid using another MET
tool?
>> > > > >> > >
>> > > > >> > > If all of the above fails we can modify our NetCDF
files but
>> I
>> > > hope
>> > > > >> to be
>> > > > >> > > able to use the regrid tool.
>> > > > >> > >
>> > > > >> > > All plots from my tests with the regrid tool can be
found
>> here:
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
>> > > > >> > > Data used to make the plots can be found here:
>> > > > >> > >
>> > > > >>
>> > >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
>> > > > >> > >
>> > > > >> > > Still would like to see some documentation on the
regrid
>> tool.
>> > > > >> > >
>> > > > >> > > Lastly, after using the regrid tool I had to use the
newest
>> > > version
>> > > > of
>> > > > >> > MET
>> > > > >> > > (met-5.1). Is this version good to go? Should I change
my MET
>> > path
>> > > > to
>> > > > >> > this
>> > > > >> > > version? Is there documentation on this version yet?
>> > > > >> > >
>> > > > >> > > Thanks!
>> > > > >> > > Tanya
>> > > > >> > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
>> Affiliate <
>> > > > >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
>> > > > >> > >
>> > > > >> > > > John,
>> > > > >> > > >
>> > > > >> > > > So Jason and I got our data working with MODE.
However, I
>> have
>> > > one
>> > > > >> > > > question. Right now MODE automatically plots the
full grid
>> > > > >> available,
>> > > > >> > in
>> > > > >> > > > this case a global plot. We are only focused on
North
>> > America. I
>> > > > was
>> > > > >> > > > thinking of using the regrid tool to change the grid
from
>> > global
>> > > > to
>> > > > >> > North
>> > > > >> > > > American using NCEP grid 209. Can the regrid tool
take any
>> of
>> > > the
>> > > > >> NCEP
>> > > > >> > > > grids found on the following website (
>> > > > >> > > >
http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html)
>> and
>> > > > apply
>> > > > >> it
>> > > > >> > to
>> > > > >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
>> Specifications'
>> > > by
>> > > > >> it)?
>> > > > >> > > > Also, right now we reverse the latitude indices so
that
>> > > everything
>> > > > >> is
>> > > > >> > > right
>> > > > >> > > > side up but this makes it so another product we use
can't
>> view
>> > > the
>> > > > >> .nc
>> > > > >> > > > files. So my question to you is would we be able to
do
>> reverse
>> > > the
>> > > > >> > > > latitudes using the MET regrid tool with NCEP grid
209 or
>> > would
>> > > we
>> > > > >> need
>> > > > >> > > to
>> > > > >> > > > use the regrid tool twice, once with grid 004 and
once with
>> > grid
>> > > > >> 209,
>> > > > >> > or
>> > > > >> > > > would another NCEP grid work?
>> > > > >> > > >
>> > > > >> > > > Oh, and, could you direct me to some documentation
on the
>> > regrid
>> > > > >> tool.
>> > > > >> > I
>> > > > >> > > > can see the usage from the command line but that
doesn't
>> > really
>> > > > >> address
>> > > > >> > > my
>> > > > >> > > > above question. Also, I didn't see anything in the
MET
>> manual.
>> > > > >> > > >
>> > > > >> > > > Thanks,
>> > > > >> > > > Tanya
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley Gotway
via RT
>> <
>> > > > >> > > > met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >
>> > > > >> > > >> Tanya,
>> > > > >> > > >>
>> > > > >> > > >> I took a close look at your current data file:
>> > > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > > >> > > >>
>> > > > >> > > >> Running it through the debugger, I see that the
problem
>> comes
>> > > > from
>> > > > >> > > parsing
>> > > > >> > > >> this time string:
>> > > > >> > > >> "hours since 2005-05-01 (12:00)"
>> > > > >> > > >>
>> > > > >> > > >> The parsing code is expecting that to be in
HH:MM:SS
>> format,
>> > > not
>> > > > >> > > HH:MM. I
>> > > > >> > > >> changed it to this:
>> > > > >> > > >> "hours since 2005-05-01 12:00:00"
>> > > > >> > > >>
>> > > > >> > > >> And then I was able to run the plot_data_plane tool
on it.
>> > > > >> However,
>> > > > >> > the
>> > > > >> > > >> resulting image is still upside-down and backwards.
See
>> > > attached
>> > > > >> > > >> (test.png).
>> > > > >> > > >>
>> > > > >> > > >> To answer the question about grid x/y mapping, the
answer
>> is
>> > > no.
>> > > > >> For
>> > > > >> > > >> lat/lon grids with no projection info defined, MET
uses
>> the
>> > > > lat/lon
>> > > > >> > > values
>> > > > >> > > >> and intuits the projection info. For non-lat/lon
grids,
>> the
>> > > > >> > projection
>> > > > >> > > >> info would need to be explicitly defined since
there's no
>> > > simple
>> > > > >> way
>> > > > >> > of
>> > > > >> > > >> intuiting a grid definition from latitude and
longitude
>> > values.
>> > > > >> > > >>
>> > > > >> > > >> So about the data being upside-down and
backwards... there
>> > are
>> > > > >> flags
>> > > > >> > in
>> > > > >> > > >> GRIB files which indicate the order in which the
data is
>> > > written.
>> > > > >> > When
>> > > > >> > > >> reading data from GRIB files, MET uses those flags
to
>> > determine
>> > > > >> where
>> > > > >> > to
>> > > > >> > > >> place things and orient the data right-side-up.
The
>> > > > CF-compliant
>> > > > >> > > NetCDF
>> > > > >> > > >> code logic is much simpler... it just reads the
data in
>> the
>> > > order
>> > > > >> it
>> > > > >> > is
>> > > > >> > > >> encoded in the file and doesn't check which way is
up!
>> > > > >> > > >>
>> > > > >> > > >> I'm not really sure what to do here. Our support
for
>> > > > CF-compliant
>> > > > >> > > NetCDF
>> > > > >> > > >> is still relatively new, and I'm not sure what the
>> > expectations
>> > > > >> should
>> > > > >> > > be.
>> > > > >> > > >> Should we add logic to the code to detect this
situation
>> and
>> > > > >> handle it
>> > > > >> > > >> better? Or is the code working as expected? I'm
not
>> really
>> > > > sure.
>> > > > >> > > >>
>> > > > >> > > >> I do see that ncview handles it better!
>> > > > >> > > >>
>> > > > >> > > >> With the current version of MET, you have 2
choices:
>> > > > >> > > >>
>> > > > >> > > >> (1) You could write lat/lon and data values in
reverse
>> order
>> > > > (i.e.
>> > > > >> > > >> increasing latitude values) to "right the ship".
>> > > > >> > > >>
>> > > > >> > > >> (2) Or you could leave things as-is and make use of
the
>> > > automated
>> > > > >> > > >> regridding capability within the MET tools.
>> > > > >> > > >>
>> > > > >> > > >> For example:
>> > > > >> > > >>
>> > > > >> > > >> # run regrid_data_plane utility to regrid to NCEP
grid 4
>> > > > >> > > >> regrid_data_plane \
>> > > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc
G004 \
>> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> > > > >> > > >> -field 'name="T2"; level="(*,*)";'
>> > > > >> > > >>
>> > > > >> > > >> # run plot_data_plane on the output
>> > > > >> > > >> plot_data_plane \
>> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
>> > > > >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
>> > > > >> > > >>
>> > > > >> > > >> The result (test_regrid.png) is attached.
>> > > > >> > > >>
>> > > > >> > > >> John
>> > > > >> > > >>
>> > > > >> > > >>
>> > > > >> > > >>
>> > > > >> > > >>
>> > > > >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey - NOAA
>> Affiliate
>> > > via
>> > > > >> RT <
>> > > > >> > > >> met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >>
>> > > > >> > > >> >
>> > > > >> > > >> > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> > > > >
>> > > > >> > > >> >
>> > > > >> > > >> > John,
>> > > > >> > > >> >
>> > > > >> > > >> > My colleague, Jason English (cc'd on email), that
is
>> > > generating
>> > > > >> the
>> > > > >> > > >> > NETCF_NCCF files has another question about the
CF
>> format.
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >>
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > > >> > > >> > Do we need the grid_mapping x and y variables if
we are
>> > using
>> > > > the
>> > > > >> > > >> lambert
>> > > > >> > > >> > cylindrical equal area grid?
>> > > > >> > > >> > If so, how do we convert 1-dimensional lat/lon
>> coordinates
>> > to
>> > > > >> the 2d
>> > > > >> > > >> > coordinates with y,x?
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >>
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> > > > >> > > >> >
>> > > > >> > > >> > Thank you,
>> > > > >> > > >> > Tanya Peevey
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey -
NOAA
>> > > Affiliate <
>> > > > >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
>> > > > >> > > >> >
>> > > > >> > > >> > > John,
>> > > > >> > > >> > >
>> > > > >> > > >> > > Thank you for the information. We generated a
new file
>> > and
>> > > I
>> > > > >> tried
>> > > > >> > > the
>> > > > >> > > >> > > plot_data_plane tool on it and got the
following
>> error.
>> > Do
>> > > > you
>> > > > >> > have
>> > > > >> > > an
>> > > > >> > > >> > idea
>> > > > >> > > >> > > of what could be wrong? Sorry to bug you so
much and
>> > thanks
>> > > > in
>> > > > >> > > advance
>> > > > >> > > >> > for
>> > > > >> > > >> > > the help.
>> > > > >> > > >> > >
>> > > > >> > > >> > > Here's the location of the file:
>> > > > >> > > >> > >
>> > > > >> > > >>
>> > > > >> > >
>> > > > >>
>> > > >
>> >
>> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
>> > > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > > >> > > >> > >
>> > > > >> > > >> > > Here's the error:
>> > > > >> > > >> > >
>> > > > >> >
>> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > > > >> > > >> > >
>> > > > >> > > >> >
>> > > > >> > > >>
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
>> > > > >> > > >> > > data/gfs_test/
>> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > > >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h";
>> level="(*,*)";
>> > > > >> > > >> > > file_type=NETCDF_NCCF;'
>> > > > >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
>> > > > >> > > >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
>> > > > >> > > >> > > ERROR :
>> > > > >> > > >> > > ERROR : StringArray::operator[](int) const ->
range
>> > check
>> > > > >> error!
>> > > > >> > > >> > > ERROR :
>> > > > >> > > >> > >
>> > > > >> >
>> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
>> > > > >> > > >> > >
>> > > > >> > > >> > > Thank you ,
>> > > > >> > > >> > > Tanya
>> > > > >> > > >> > >
>> > > > >> > > >> > >
>> > > > >> > > >> > >
>> > > > >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley
Gotway via
>> > RT <
>> > > > >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >> > >
>> > > > >> > > >> > >> Tanya,
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> Sorry for the delay. I was out of the office
on
>> Friday.
>> > > > >> Using
>> > > > >> > > >> "LSP_6h"
>> > > > >> > > >> > >> for the name and "(*,*)" for the level should
do the
>> > > trick.
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> When trying to get the MET tools to read a new
>> dataset,
>> > I
>> > > > >> always
>> > > > >> > > >> start
>> > > > >> > > >> > by
>> > > > >> > > >> > >> using the plot_data_plane tool:
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
>> > > > >> > > >> > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
\
>> > > > >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
>> > > > >> > > file_type=NETCDF_NCCF;'
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> I use that to make sure the name and level
settings
>> work
>> > > as
>> > > > >> > > expected,
>> > > > >> > > >> > and
>> > > > >> > > >> > >> it also creates an output plot so I can see
that MET
>> is
>> > > (or
>> > > > is
>> > > > >> > not)
>> > > > >> > > >> > >> reading/plotting the data as expected.
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> That configuration string you pass to the the
>> > > > plot_data_plane
>> > > > >> > tool
>> > > > >> > > is
>> > > > >> > > >> > >> exactly the same settings as are supported in
the
>> > > > >> configuration
>> > > > >> > > >> files.
>> > > > >> > > >> > In
>> > > > >> > > >> > >> fact, we read that string with the same code
that
>> parses
>> > > the
>> > > > >> > config
>> > > > >> > > >> > files.
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> I find using plot_data_plane very helpful.
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> Thanks,
>> > > > >> > > >> > >> John
>> > > > >> > > >> > >>
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey -
NOAA
>> > > > Affiliate
>> > > > >> via
>> > > > >> > > RT
>> > > > >> > > >> <
>> > > > >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >> > >>
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > <URL:
>> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> > > > >> > >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > John,
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > Thanks for the info. We are going to work on
>> changing
>> > > our
>> > > > >> file
>> > > > >> > > >> format
>> > > > >> > > >> > >> and
>> > > > >> > > >> > >> > then I'll get back to you.
>> > > > >> > > >> > >> > Did you see my other question about how I
specified
>> > > 'name'
>> > > > >> andhow to round to nearest 0.5
>> > > > >>
>> > > > >> > > >> 'level'
>> > > > >> > > >> > >> for
>> > > > >> > > >> > >> > the precip. field in my config file for
MODE? Was
>> how
>> > I
>> > > > did
>> > > > >> it
>> > > > >> > > ok?
>> > > > >> > > >> I
>> > > > >> > > >> > ask
>> > > > >> > > >> > >> > because the manual was confusing on this
point. If
>> my
>> > > > syntax
>> > > > >> > was
>> > > > >> > > >> > >> incorrect
>> > > > >> > > >> > >> > could you please email me the correction.
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > Thanks,
>> > > > >> > > >> > >> > Tanya
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John Halley
Gotway
>> > via
>> > > > RT <
>> > > > >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > > Tanya,
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > Here's a website about it:
>> > > > >> > > >> > >> > > http://cfconventions.org/latest.html
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > And I posted a sample CF-compliant NetCDF
file
>> here:
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >>
>> > > > >> > > >> >
>> > > > >> > > >>
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > The relevant pieces in there are...
>> > > > >> > > >> > >> > > - The global "Conventions" attribute set
to
>> > "CF-...".
>> > > > >> > > >> > >> > > - The time dimension/variable define the
forecast
>> > > valid
>> > > > >> time.
>> > > > >> > > >> > >> > > - The forecast_reference_time variable
defines
>> the
>> > > model
>> > > > >> > > >> > >> initialization
>> > > > >> > > >> > >> > > time.
>> > > > >> > > >> > >> > > - The grid_mapping, x, and y variables
define the
>> > grid
>> > > > >> > > definition
>> > > > >> > > >> > >> > > information.
>> > > > >> > > >> > >> > > - The grid_mapping variable attributes map
those
>> > > > >> variables to
>> > > > >> > > the
>> > > > >> > > >> > >> > relevant
>> > > > >> > > >> > >> > > grid definition.
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > Hope this helps.
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > Thanks,
>> > > > >> > > >> > >> > > John
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya
Peevey -
>> NOAA
>> > > > >> > Affiliate
>> > > > >> > > >> via
>> > > > >> > > >> > RT
>> > > > >> > > >> > >> <
>> > > > >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > <URL:
>> > > > >> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>> > > > >> > > >> >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > John,
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > To clarify my previous request of
information.
>> I
>> > > meant
>> > > > >> to
>> > > > >> > ask
>> > > > >> > > >> was
>> > > > >> > > >> > >> the
>> > > > >> > > >> > >> > > > following: Do you know of any good
>> documentation
>> > on
>> > > > how
>> > > > >> to
>> > > > >> > > >> make a
>> > > > >> > > >> > >> > NetCDF
>> > > > >> > > >> > >> > > > file CF-compliant? Also, do you have an
example
>> > .nc
>> > > > file
>> > > > >> > that
>> > > > >> > > >> is
>> > > > >> > > >> > >> NetCDF
>> > > > >> > > >> > >> > > > CF-compliant that we could compare to?
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > Thanks,
>> > > > >> > > >> > >> > > > Tanya
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya
Peevey -
>> > NOAA
>> > > > >> > > Affiliate <
>> > > > >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>>
wrote:
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > > John,
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > > We generated the NetCDF files
ourselves so we
>> > > could
>> > > > >> > > probably
>> > > > >> > > >> > make
>> > > > >> > > >> > >> it
>> > > > >> > > >> > >> > > work
>> > > > >> > > >> > >> > > > > for MET. Do you know how to get it
into
>> > > CF-compliant
>> > > > >> > NetCDF
>> > > > >> > > >> > >> format?
>> > > > >> > > >> > >> > > > > Yes, we have grib files but the reason
we
>> > created
>> > > > >> NetCDF
>> > > > >> > > >> files
>> > > > >> > > >> > was
>> > > > >> > > >> > >> > > > because
>> > > > >> > > >> > >> > > > > original the NR... file had
accumulated
>> precip
>> > > over
>> > > > >> the
>> > > > >> > > whole
>> > > > >> > > >> > >> > forecast
>> > > > >> > > >> > >> > > > > where the other file has 6h
accumulation
>> data,
>> > so
>> > > we
>> > > > >> need
>> > > > >> > > to
>> > > > >> > > >> > >> convert
>> > > > >> > > >> > >> > > the
>> > > > >> > > >> > >> > > > > NR... precip information into 6h
accumulation
>> > > (they
>> > > > >> had
>> > > > >> > > >> > different
>> > > > >> > > >> > >> > units
>> > > > >> > > >> > >> > > > > too, so we need to change that too).
Since we
>> > had
>> > > to
>> > > > >> > change
>> > > > >> > > >> the
>> > > > >> > > >> > >> units
>> > > > >> > > >> > >> > > > too I
>> > > > >> > > >> > >> > > > > think use generating our own NetCDF
file is
>> best
>> > > but
>> > > > >> we
>> > > > >> > > just
>> > > > >> > > >> > need
>> > > > >> > > >> > >> to
>> > > > >> > > >> > >> > > get
>> > > > >> > > >> > >> > > > it
>> > > > >> > > >> > >> > > > > in the right format.
>> > > > >> > > >> > >> > > > > What is your opinion on the best way
to
>> approach
>> > > > this?
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > > If we get it into the right format is
how I
>> > > specify
>> > > > >> the
>> > > > >> > > >> 'name'
>> > > > >> > > >> > and
>> > > > >> > > >> > >> > > > 'level'
>> > > > >> > > >> > >> > > > > of the field correct? It was hard to
figure
>> out
>> > > the
>> > > > >> > correct
>> > > > >> > > >> > >> notation
>> > > > >> > > >> > >> > > from
>> > > > >> > > >> > >> > > > > the manual.
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > > Thanks,
>> > > > >> > > >> > >> > > > > Tanya
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John
Halley
>> > Gotway
>> > > > via
>> > > > >> > RT <
>> > > > >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>>
wrote:
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > >> Hi Tanya,
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> I took a look at the gridded NetCDF
data
>> you're
>> > > > >> trying
>> > > > >> > to
>> > > > >> > > >> pass
>> > > > >> > > >> > to
>> > > > >> > > >> > >> > > MODE.
>> > > > >> > > >> > >> > > > >> Unfortunately, MET doesn't support
all
>> types of
>> > > > >> gridded
>> > > > >> > > >> NetCDF
>> > > > >> > > >> > >> > files.
>> > > > >> > > >> > >> > > > >> Instead, it only supports the MET
NetCDF
>> format
>> > > > (e.g.
>> > > > >> > the
>> > > > >> > > >> > output
>> > > > >> > > >> > >> of
>> > > > >> > > >> > >> > > the
>> > > > >> > > >> > >> > > > >> pcp_combine tool), the output of the
WRF
>> > pinterp
>> > > > (or
>> > > > >> > > >> wrfinterp)
>> > > > >> > > >> > >> > > > utilities,
>> > > > >> > > >> > >> > > > >> and CF-compliant NetCDF files.
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> Looking at "
>> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
>> > > > >> > "
>> > > > >> > > I
>> > > > >> > > >> see
>> > > > >> > > >> > >> that
>> > > > >> > > >> > >> > > it
>> > > > >> > > >> > >> > > > >> follows none of those conventions.
>> However, it
>> > > is
>> > > > >> > closest
>> > > > >> > > >> to
>> > > > >> > > >> > >> > > > CF-compliant
>> > > > >> > > >> > >> > > > >> NetCDF.
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> When using a new data file format,
instead
>> of
>> > > > >> running it
>> > > > >> > > >> > through
>> > > > >> > > >> > >> > MODE,
>> > > > >> > > >> > >> > > > >> I'd suggest using plot_data_plane to
make
>> sure
>> > > MET
>> > > > is
>> > > > >> > > >> reading
>> > > > >> > > >> > it
>> > > > >> > > >> > >> > > > >> correctly. I ran met-
5.1/plot_data_plane as
>> > > > follows:
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >>
>> > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
>> > > > >> > > >> > >> > > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
>> \
>> > > > >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
>> > > > >> > > >> > file_type=NETCDF_NCCF;'
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> The resulting image is attached.
There are
>> > > several
>> > > > >> > issues
>> > > > >> > > >> > here:
>> > > > >> > > >> > >> > > > >> - I had to tell plot_data_plane to
interpret
>> > this
>> > > > as
>> > > > >> > > >> > CF-compliant
>> > > > >> > > >> > >> > > NetCDF
>> > > > >> > > >> > >> > > > >> using NETCDF_NCCF.
>> > > > >> > > >> > >> > > > >> - The timing info in that file isn't
>> specified
>> > in
>> > > > the
>> > > > >> > > >> > >> CF-compliant
>> > > > >> > > >> > >> > > > way...
>> > > > >> > > >> > >> > > > >> so MET can't discern the
initialization,
>> valid,
>> > > and
>> > > > >> lead
>> > > > >> > > >> times.
>> > > > >> > > >> > >> > > > >> - And then there's the obvious
problem (in
>> the
>> > > > >> image) of
>> > > > >> > > the
>> > > > >> > > >> > >> world
>> > > > >> > > >> > >> > > being
>> > > > >> > > >> > >> > > > >> upside down and backwards!
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> So while we did get *something*,
there are
>> > > > obviously
>> > > > >> a
>> > > > >> > lot
>> > > > >> > > >> of
>> > > > >> > > >> > >> > issues.
>> > > > >> > > >> > >> > > > >> Basically, MET doesn't support the
gridded
>> > NetCDF
>> > > > >> file
>> > > > >> > > >> format
>> > > > >> > > >> > you
>> > > > >> > > >> > >> > are
>> > > > >> > > >> > >> > > > >> using. Is this data possibly
available in
>> > GRIB?
>> > > > Or
>> > > > >> do
>> > > > >> > > you
>> > > > >> > > >> > have
>> > > > >> > > >> > >> > > control
>> > > > >> > > >> > >> > > > >> over the NetCDF file format in use?
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >> Thanks,
>> > > > >> > > >> > >> > > > >> John
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >>
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > > > --
>> > > > >> > > >> > >> > > > > Tanya R. Peevey, PhD
>> > > > >> > > >> > >> > > > > Research Scientist I, Global Observing
>> Systems
>> > > > >> Analysis
>> > > > >> > > >> (GOSA)
>> > > > >> > > >> > >> Group
>> > > > >> > > >> > >> > > > > NOAA ESRL Global Systems Division
>> > > > >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
>> > > > >> > > >> > >> > > > > (303) 497-5847
>> > > > >> > > >> > >> > > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > > --
>> > > > >> > > >> > >> > > > Tanya R. Peevey, PhD
>> > > > >> > > >> > >> > > > Research Scientist I, Global Observing
Systems
>> > > > Analysis
>> > > > >> > > (GOSA)
>> > > > >> > > >> > Group
>> > > > >> > > >> > >> > > > NOAA ESRL Global Systems Division
>> > > > >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
>> > > > >> > > >> > >> > > > (303) 497-5847
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > > >
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> > >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> > --
>> > > > >> > > >> > >> > Tanya R. Peevey, PhD
>> > > > >> > > >> > >> > Research Scientist I, Global Observing
Systems
>> > Analysis
>> > > > >> (GOSA)
>> > > > >> > > >> Group
>> > > > >> > > >> > >> > NOAA ESRL Global Systems Division
>> > > > >> > > >> > >> > 325 Broadway, Boulder, CO 80305
>> > > > >> > > >> > >> > (303) 497-5847
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >> >
>> > > > >> > > >> > >>
>> > > > >> > > >> > >>
>> > > > >> > > >> > >
>> > > > >> > > >> > >
>> > > > >> > > >> > > --
>> > > > >> > > >> > > Tanya R. Peevey, PhD
>> > > > >> > > >> > > Research Scientist I, Global Observing Systems
>> Analysis
>> > > > (GOSA)
>> > > > >> > Group
>> > > > >> > > >> > > NOAA ESRL Global Systems Division
>> > > > >> > > >> > > 325 Broadway, Boulder, CO 80305
>> > > > >> > > >> > > (303) 497-5847
>> > > > >> > > >> > >
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >> > --
>> > > > >> > > >> > Tanya R. Peevey, PhD
>> > > > >> > > >> > Research Scientist I, Global Observing Systems
Analysis
>> > > (GOSA)
>> > > > >> Group
>> > > > >> > > >> > NOAA ESRL Global Systems Division
>> > > > >> > > >> > 325 Broadway, Boulder, CO 80305
>> > > > >> > > >> > (303) 497-5847
>> > > > >> > > >> >
>> > > > >> > > >> >
>> > > > >> > > >>
>> > > > >> > > >>
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > --
>> > > > >> > > > Tanya R. Peevey, PhD
>> > > > >> > > > Research Scientist I, Global Observing Systems
Analysis
>> (GOSA)
>> > > > Group
>> > > > >> > > > NOAA ESRL Global Systems Division
>> > > > >> > > > 325 Broadway, Boulder, CO 80305
>> > > > >> > > > (303) 497-5847
>> > > > >> > > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > --
>> > > > >> > > Tanya R. Peevey, PhD
>> > > > >> > > Research Scientist I, Global Observing Systems
Analysis
>> (GOSA)
>> > > Group
>> > > > >> > > NOAA ESRL Global Systems Division
>> > > > >> > > 325 Broadway, Boulder, CO 80305
>> > > > >> > > (303) 497-5847
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >> --
>> > > > >> Tanya R. Peevey, PhD
>> > > > >> Research Scientist I, Global Observing Systems Analysis
(GOSA)
>> Group
>> > > > >> NOAA ESRL Global Systems Division
>> > > > >> 325 Broadway, Boulder, CO 80305
>> > > > >> (303) 497-5847
>> > > > >>
>> > > > >>
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Tanya R. Peevey, PhD
>> > > Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> > > NOAA ESRL Global Systems Division
>> > > 325 Broadway, Boulder, CO 80305
>> > > (303) 497-5847
>> > >
>> > >
>> >
>> >
>>
>>
>> --
>> Tanya R. Peevey, PhD
>> Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
>> NOAA ESRL Global Systems Division
>> 325 Broadway, Boulder, CO 80305
>> (303) 497-5847
>>
>>
>
------------------------------------------------
Subject: Trying MODE out and having issues
From: Tanya Peevey - NOAA Affiliate
Time: Wed Feb 17 13:25:05 2016
John,
No worries. I noticed that and figured you'd catch it.
Feel free to close the ticket. I'm good for now and will email
met_help
with any further or additional questions that pop up.
Thanks!
Tanya
On Wed, Feb 17, 2016 at 12:45 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Sorry, sent that too quickly. Here's the image.
>
> This NCL script reads MODE object information from several different
> ensemble members and the ensemble mean. It plots thin outlines for
> ensemble member objects and a thick outline for the ensemble mean.
The
> observations are plotting in solid colors: red for matched and blue
for
> unmatched (matched and unmatched compared to the ensemble mean).
>
> So this is an example of using the MODE NetCDF output to display
> information about an ensemble.
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On Wed, Feb 17, 2016 at 12:40 PM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Tanya,
> >
> > The output that MODE creates is controlled by the following
entries in
> the
> > config file:
> >
> > ps_plot_flag = TRUE;
> > nc_pairs_flag = {
> > latlon = TRUE;
> > raw = TRUE;
> > object_raw = TRUE;
> > object_id = TRUE;
> > cluster_id = TRUE;
> > polylines = TRUE;
> > }
> > ct_stats_flag = TRUE;
> >
> > You will always get a "_obj.txt" output file and by default all
the
> > optional output files are also written. Use ps_plot_flag to
disable the
> > PostScript output. Use nc_pairs_flag to disable the NetCDF
output. And
> > use ct_stat_flag to disable the contingency table output file.
> >
> > Since creating the NetCDF file is the default behavior, you
probably
> > already have it.
> >
> > I've attached an NCL script which plots some MODE NetCDF files
along with
> > a sample output image.
> >
> >
> >
> >
> >
> > On Wed, Feb 17, 2016 at 12:07 PM, Tanya Peevey - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >>
> >> John,
> >>
> >> Thanks for the clarification of the MODE output on grid 221. We
will use
> >> the ps file/graphics produced by MODE regularly, but don't know
if we
> will
> >> use them when we publish. For the moment they are fine for in-
house
> stuff,
> >> but still let me know when those grids are fixed in terms of how
they
> look
> >> in the MODE graphics. We will either create a subset of data and
use
> that
> >> in MODE or do what you suggest by having MODE write an NetCDF
object
> file
> >> if the grids aren't fixed and we are going to publish (this is
still a
> few
> >> months away). How do you create that object netcdf file? What do
I need
> to
> >> change in MODE? Also, I thought MODE already spit out a couple
files one
> >> of
> >> which is a file with object information? Is that file ASCII?
> >>
> >> So letting me know how to modify MODE would be great since it
would be
> >> good
> >> to be able to overlay the objects onto other fields. Any scripts
you may
> >> have for reading the NetCDF output would be helpful too.
> >>
> >> Thanks,
> >> Tanya
> >>
> >>
> >>
> >> On Wed, Feb 17, 2016 at 10:00 AM, John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Tanya,
> >> >
> >> > Grid 212 is commonly used:
> >> >
http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid212.gif
> >> >
> >> > However that is limited to CONUS.
> >> >
> >> > To be clear, there really isn't a problem in the output of MODE
on
> grid
> >> > 221, other than the plotting of Russia's coastlines. The
object
> >> definition
> >> > and distances should all be fine. Are you planning to use the
> >> PostScript
> >> > output of MODE systematically? Do you need the graphics? MODE
can be
> >> > configured to write a NetCDF object file which can be plotted
using
> >> > whatever graphics package you'd like.
> >> >
> >> > For example, for HWT or HMT a few years back we plotted the
MODE
> NetCDF
> >> > output using NCL to overlay the forecast and observation
objects.
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, Feb 16, 2016 at 10:59 AM, Tanya Peevey - NOAA Affiliate
via
> RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
>
> >> > >
> >> > > John,
> >> > >
> >> > > Thanks for the information. We are going to look at the time
> >> information
> >> > > again today. Also, I found the issue with the last file I
tried to
> >> use in
> >> > > MODE, the issue wasn't the grid_mapping but something else on
our
> end.
> >> > >
> >> > > I tried different grids on the *old_cf* files (the ones that
worked
> >> > > previously) and all of them had Russia looking funny. The
grids I
> >> tried
> >> > > were G221 and G241, both North America grids. So basically
the only
> >> > options
> >> > > I have are CONUS, Northern Hemisphere or the Globe, none of
which
> are
> >> > > appealing. I also don't like the idea of excluding Russia but
I'll
> >> copy
> >> > > over your code so I have it. In the meantime we will probably
> create a
> >> > > subset of data over North America and then run that through
MODE. As
> >> soon
> >> > > as you get the North American centric grids working please
let me
> >> know.
> >> > >
> >> > > Lastly, if you know of another grid that would work please
let me
> know
> >> > but
> >> > > I believe I've tried all of the North American centric grids
that
> are
> >> of
> >> > > ~equivalent resolution of the original dataset (1x1 degree)
and are
> >> > > available in MODE.
> >> > >
> >> > > Thanks,
> >> > > Tanya
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Feb 12, 2016 at 12:00 PM, John Halley Gotway via RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > > Tanya,
> >> > > >
> >> > > > On to more issues. I think that not having the
grid_mapping
> should
> >> > work
> >> > > > fine, but only because this is lat/lon data and MET can use
the
> >> lat/lon
> >> > > > variables to infer the grid.
> >> > > >
> >> > > > For example, the following commands works fine:
> >> > > >
> >> > > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test/
> >> > > > NR2006.pl.1by1.7278.2006022818.cf.nc \
> >> > > > ~/without_grid_map.ps \
> >> > > > 'name="LSP_6h"; level="(*,*)";'
> >> > > >
> >> > > > Next, looking in your MODE output file:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode/mode_3169855830L_20060228_180000V_000000A_obj.txt
> >> > > >
> >> > > > I see that the lead time is listed as 3169835842. The
units are
> >> > > supposed
> >> > > > to be HHMMSS so this string looks pretty odd! So there's
still an
> >> > issue
> >> > > > with the times.
> >> > > >
> >> > > > John
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Fri, Feb 12, 2016 at 11:49 AM, John Halley Gotway <
> >> johnhg at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Tanya,
> >> > > > >
> >> > > > > I took a close look at the issue with plotting the map
for grid
> >> 209.
> >> > > The
> >> > > > > issue is caused by the map data straddling the
international
> date
> >> > line.
> >> > > > I
> >> > > > > tried rescaling the longitudes from -180,180 to 0,360 but
wasn't
> >> able
> >> > > to
> >> > > > > resolve this issue. So this is not easily fixed. I'll
need to
> >> have
> >> > > > Randy
> >> > > > > Bullock look for a solution.
> >> > > > >
> >> > > > > In the meantime, you have a couple options...
> >> > > > > - If you're not tied to grid 209, you could easily switch
to
> >> using a
> >> > > > > different grid.
> >> > > > > - Or you could set up the config file to plot a set of
map data
> >> files
> >> > > > > which does not include Russia.
> >> > > > >
> >> > > > > Take a look in met-5.1/share/met/config/ConfigMap. The
> "map_data"
> >> > > > section
> >> > > > > controls the map data files to be plotted. I've attached
a map
> >> data
> >> > > file
> >> > > > > to this message from which I've removed all the Russia
data:
> >> > > > > country_data_minus_russia
> >> > > > >
> >> > > > > Plotting on this map data file results in the attached
plot of
> >> grid
> >> > > 209.
> >> > > > >
> >> > > > > FYI, you can just copy and paste the "map_data" config
file
> >> section
> >> > to
> >> > > > the
> >> > > > > bottom of your MODE config file. And then set the path
to the
> >> > attached
> >> > > > map
> >> > > > > data file. That'll overwrite the default values set in
> >> > ConfigMapData.
> >> > > > >
> >> > > > > John
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Feb 12, 2016 at 9:15 AM, Tanya Peevey - NOAA
Affiliate
> via
> >> > RT <
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > >>
> >> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >
> >> > > > >>
> >> > > > >> John,
> >> > > > >>
> >> > > > >> Ok, tried G209 in the new MODE and I got results but the
lines
> on
> >> > the
> >> > > > map
> >> > > > >> that are suppose to define Russia are all still wrong.
See
> >> following
> >> > > > >> directory,
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/gfs_test/out/mode.
> >> > > > >>
> >> > > > >> So the above worked with a NetCDF file we generated that
has
> >> > > > CF-compliant
> >> > > > >> time and grid_mapping information. We found out that the
> >> > grid_mapping
> >> > > > >> variable in the CF-compliant file was what was not
allowing us
> to
> >> > use
> >> > > > >> Panoply (a package for quickly viewing data in an NetCDF
file).
> >> So
> >> > we
> >> > > > >> removed the grid_mapping information and tried again, to
see if
> >> MODE
> >> > > > would
> >> > > > >> work and it did not. Here are where the two files are
location
> >> > > > >>
> >> > > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1/data/gfs_test,
> >> > > > >> with *.cf.nc being the file without grid_mapping and *.
> old_cf.nc
> >> > > being
> >> > > > >> the
> >> > > > >> file with grid_mapping.
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>/scratch4/BMC/dtc/MET/met-5.1/bin/mode
> >> > > > >>
data/gfs_test/prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc
> >> > > > >> data/gfs_test/
> >> > > > >> NR2006.pl.1by1.7188.2006022500.cf.nc
> >> > > gfs_test/config/MODEConfig_gfstest
> >> > > > >> -outdir gfs_test/out/mode/ -v 2
> >> > > > >> DEBUG 1: Default Config File:
> >> > > > >>
> /scratch4/BMC/dtc/MET/met-5.1/share/met/config/MODEConfig_default
> >> > > > >> DEBUG 1: Match Config File:
gfs_test/config/MODEConfig_gfstest
> >> > > > >> DEBUG 1: Merge Config File:
gfs_test/config/MODEConfig_gfstest
> >> > > > >> ERROR :
> >> > > > >> ERROR : Trouble reading forecast file "data/gfs_test/
> >> > > > >> prwin_ctr.7188.pgbf00.gfs.2006022500.ver.cf.nc"
> >> > > > >> ERROR :
> >> > > > >>
> >> Theia05:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.1>
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > > >>
> >> > > > >> Basically, want to try and fix the regridding to the
G209 grid
> so
> >> > > Russia
> >> > > > >> looks normal and want to confirm that we do need to use
the *.
> >> > > old_cf.nc
> >> > > > >> files instead of the other within MET.
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> Tanya
> >> > > > >>
> >> > > > >> P.S. Having the regird option in MODE is great. Now I
don't
> have
> >> to
> >> > > > create
> >> > > > >> another set of files before processing the data. Saves
me some
> >> disk
> >> > > > space
> >> > > > >> :-).
> >> > > > >>
> >> > > > >>
> >> > > > >> On Thu, Feb 11, 2016 at 5:12 PM, John Halley Gotway via
RT <
> >> > > > >> met_help at ucar.edu> wrote:
> >> > > > >>
> >> > > > >> > Tanya,
> >> > > > >> >
> >> > > > >> > I can look more closely tomorrow, but some initial
thoughts
> >> are:
> >> > > > >> >
> >> > > > >> > - yes, please use met-5.1 and use the default mode 5.1
config
> >> file
> >> > > > >> > - you could use the regrid tool, but instead fill out
the
> >> regrid
> >> > > > >> section of
> >> > > > >> > the mode config file. Try setting to_grid = "G209";
> >> > > > >> >
> >> > > > >> > That'll automatically regrid the global data in the
fly.
> >> > > > >> >
> >> > > > >> > That's all new in 5.1.
> >> > > > >> >
> >> > > > >> > John
> >> > > > >> >
> >> > > > >> > On Thursday, February 11, 2016, Tanya Peevey - NOAA
Affiliate
> >> via
> >> > > RT <
> >> > > > >> > met_help at ucar.edu> wrote:
> >> > > > >> >
> >> > > > >> > >
> >> > > > >> > > <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975 >
> >> > > > >> > >
> >> > > > >> > > John,
> >> > > > >> > >
> >> > > > >> > > To clarify my previous email, I'm trying to use the
regrid
> >> tool
> >> > to
> >> > > > >> > reverse
> >> > > > >> > > the latitudes in our .nc files and focus on North
America
> >> rather
> >> > > > than
> >> > > > >> the
> >> > > > >> > > whole global. I just tried the regrid tool with the
grid
> set
> >> to
> >> > > G209
> >> > > > >> and
> >> > > > >> > it
> >> > > > >> > > worked (both corrected the latitude issue and
focused on
> >> North
> >> > > > >> America)
> >> > > > >> > but
> >> > > > >> > > with one small problem, there are some weird lines
where
> >> Russia
> >> > is
> >> > > > >> > suppose
> >> > > > >> > > to be. Do you know what the problem is? Is there
another
> grid
> >> > you
> >> > > > >> could
> >> > > > >> > > recommend if the output from the regrid tool on the
G209
> grid
> >> > > can't
> >> > > > be
> >> > > > >> > > fixed? Or could I define my own grid using another
MET
> tool?
> >> > > > >> > >
> >> > > > >> > > If all of the above fails we can modify our NetCDF
files
> but
> >> I
> >> > > hope
> >> > > > >> to be
> >> > > > >> > > able to use the regrid tool.
> >> > > > >> > >
> >> > > > >> > > All plots from my tests with the regrid tool can be
found
> >> here:
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/gfs_test/out/plot_data_plane
> >> > > > >> > > Data used to make the plots can be found here:
> >> > > > >> > >
> >> > > > >>
> >> > >
> >> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test
> >> > > > >> > >
> >> > > > >> > > Still would like to see some documentation on the
regrid
> >> tool.
> >> > > > >> > >
> >> > > > >> > > Lastly, after using the regrid tool I had to use the
newest
> >> > > version
> >> > > > of
> >> > > > >> > MET
> >> > > > >> > > (met-5.1). Is this version good to go? Should I
change my
> MET
> >> > path
> >> > > > to
> >> > > > >> > this
> >> > > > >> > > version? Is there documentation on this version yet?
> >> > > > >> > >
> >> > > > >> > > Thanks!
> >> > > > >> > > Tanya
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > On Thu, Feb 11, 2016 at 2:13 PM, Tanya Peevey - NOAA
> >> Affiliate <
> >> > > > >> > > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > > > >> > >
> >> > > > >> > > > John,
> >> > > > >> > > >
> >> > > > >> > > > So Jason and I got our data working with MODE.
However, I
> >> have
> >> > > one
> >> > > > >> > > > question. Right now MODE automatically plots the
full
> grid
> >> > > > >> available,
> >> > > > >> > in
> >> > > > >> > > > this case a global plot. We are only focused on
North
> >> > America. I
> >> > > > was
> >> > > > >> > > > thinking of using the regrid tool to change the
grid from
> >> > global
> >> > > > to
> >> > > > >> > North
> >> > > > >> > > > American using NCEP grid 209. Can the regrid tool
take
> any
> >> of
> >> > > the
> >> > > > >> NCEP
> >> > > > >> > > > grids found on the following website (
> >> > > > >> > > >
http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html)
> >> and
> >> > > > apply
> >> > > > >> it
> >> > > > >> > to
> >> > > > >> > > > a netcdf file (I ask becuase 209 had 'See GRIB
> >> Specifications'
> >> > > by
> >> > > > >> it)?
> >> > > > >> > > > Also, right now we reverse the latitude indices so
that
> >> > > everything
> >> > > > >> is
> >> > > > >> > > right
> >> > > > >> > > > side up but this makes it so another product we
use can't
> >> view
> >> > > the
> >> > > > >> .nc
> >> > > > >> > > > files. So my question to you is would we be able
to do
> >> reverse
> >> > > the
> >> > > > >> > > > latitudes using the MET regrid tool with NCEP grid
209 or
> >> > would
> >> > > we
> >> > > > >> need
> >> > > > >> > > to
> >> > > > >> > > > use the regrid tool twice, once with grid 004 and
once
> with
> >> > grid
> >> > > > >> 209,
> >> > > > >> > or
> >> > > > >> > > > would another NCEP grid work?
> >> > > > >> > > >
> >> > > > >> > > > Oh, and, could you direct me to some documentation
on the
> >> > regrid
> >> > > > >> tool.
> >> > > > >> > I
> >> > > > >> > > > can see the usage from the command line but that
doesn't
> >> > really
> >> > > > >> address
> >> > > > >> > > my
> >> > > > >> > > > above question. Also, I didn't see anything in the
MET
> >> manual.
> >> > > > >> > > >
> >> > > > >> > > > Thanks,
> >> > > > >> > > > Tanya
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > On Tue, Feb 9, 2016 at 10:20 AM, John Halley
Gotway via
> RT
> >> <
> >> > > > >> > > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >
> >> > > > >> > > >> Tanya,
> >> > > > >> > > >>
> >> > > > >> > > >> I took a close look at your current data file:
> >> > > > >> > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > > >> > > >>
> >> > > > >> > > >> Running it through the debugger, I see that the
problem
> >> comes
> >> > > > from
> >> > > > >> > > parsing
> >> > > > >> > > >> this time string:
> >> > > > >> > > >> "hours since 2005-05-01 (12:00)"
> >> > > > >> > > >>
> >> > > > >> > > >> The parsing code is expecting that to be in
HH:MM:SS
> >> format,
> >> > > not
> >> > > > >> > > HH:MM. I
> >> > > > >> > > >> changed it to this:
> >> > > > >> > > >> "hours since 2005-05-01 12:00:00"
> >> > > > >> > > >>
> >> > > > >> > > >> And then I was able to run the plot_data_plane
tool on
> it.
> >> > > > >> However,
> >> > > > >> > the
> >> > > > >> > > >> resulting image is still upside-down and
backwards. See
> >> > > attached
> >> > > > >> > > >> (test.png).
> >> > > > >> > > >>
> >> > > > >> > > >> To answer the question about grid x/y mapping,
the
> answer
> >> is
> >> > > no.
> >> > > > >> For
> >> > > > >> > > >> lat/lon grids with no projection info defined,
MET uses
> >> the
> >> > > > lat/lon
> >> > > > >> > > values
> >> > > > >> > > >> and intuits the projection info. For non-lat/lon
grids,
> >> the
> >> > > > >> > projection
> >> > > > >> > > >> info would need to be explicitly defined since
there's
> no
> >> > > simple
> >> > > > >> way
> >> > > > >> > of
> >> > > > >> > > >> intuiting a grid definition from latitude and
longitude
> >> > values.
> >> > > > >> > > >>
> >> > > > >> > > >> So about the data being upside-down and
backwards...
> there
> >> > are
> >> > > > >> flags
> >> > > > >> > in
> >> > > > >> > > >> GRIB files which indicate the order in which the
data is
> >> > > written.
> >> > > > >> > When
> >> > > > >> > > >> reading data from GRIB files, MET uses those
flags to
> >> > determine
> >> > > > >> where
> >> > > > >> > to
> >> > > > >> > > >> place things and orient the data right-side-up.
The
> >> > > > CF-compliant
> >> > > > >> > > NetCDF
> >> > > > >> > > >> code logic is much simpler... it just reads the
data in
> >> the
> >> > > order
> >> > > > >> it
> >> > > > >> > is
> >> > > > >> > > >> encoded in the file and doesn't check which way
is up!
> >> > > > >> > > >>
> >> > > > >> > > >> I'm not really sure what to do here. Our support
for
> >> > > > CF-compliant
> >> > > > >> > > NetCDF
> >> > > > >> > > >> is still relatively new, and I'm not sure what
the
> >> > expectations
> >> > > > >> should
> >> > > > >> > > be.
> >> > > > >> > > >> Should we add logic to the code to detect this
situation
> >> and
> >> > > > >> handle it
> >> > > > >> > > >> better? Or is the code working as expected? I'm
not
> >> really
> >> > > > sure.
> >> > > > >> > > >>
> >> > > > >> > > >> I do see that ncview handles it better!
> >> > > > >> > > >>
> >> > > > >> > > >> With the current version of MET, you have 2
choices:
> >> > > > >> > > >>
> >> > > > >> > > >> (1) You could write lat/lon and data values in
reverse
> >> order
> >> > > > (i.e.
> >> > > > >> > > >> increasing latitude values) to "right the ship".
> >> > > > >> > > >>
> >> > > > >> > > >> (2) Or you could leave things as-is and make use
of the
> >> > > automated
> >> > > > >> > > >> regridding capability within the MET tools.
> >> > > > >> > > >>
> >> > > > >> > > >> For example:
> >> > > > >> > > >>
> >> > > > >> > > >> # run regrid_data_plane utility to regrid to NCEP
grid 4
> >> > > > >> > > >> regrid_data_plane \
> >> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_JHG.nc
> G004 \
> >> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> > > > >> > > >> -field 'name="T2"; level="(*,*)";'
> >> > > > >> > > >>
> >> > > > >> > > >> # run plot_data_plane on the output
> >> > > > >> > > >> plot_data_plane \
> >> > > > >> > > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf_regrid.nc \
> >> > > > >> > > >> test_regrid.ps 'name="T2"; level="(*,*)";'
> >> > > > >> > > >>
> >> > > > >> > > >> The result (test_regrid.png) is attached.
> >> > > > >> > > >>
> >> > > > >> > > >> John
> >> > > > >> > > >>
> >> > > > >> > > >>
> >> > > > >> > > >>
> >> > > > >> > > >>
> >> > > > >> > > >> On Mon, Feb 8, 2016 at 4:46 PM, Tanya Peevey -
NOAA
> >> Affiliate
> >> > > via
> >> > > > >> RT <
> >> > > > >> > > >> met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >>
> >> > > > >> > > >> >
> >> > > > >> > > >> > <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> > > > >
> >> > > > >> > > >> >
> >> > > > >> > > >> > John,
> >> > > > >> > > >> >
> >> > > > >> > > >> > My colleague, Jason English (cc'd on email),
that is
> >> > > generating
> >> > > > >> the
> >> > > > >> > > >> > NETCF_NCCF files has another question about the
CF
> >> format.
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >>
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > > >> > > >> > Do we need the grid_mapping x and y variables
if we
> are
> >> > using
> >> > > > the
> >> > > > >> > > >> lambert
> >> > > > >> > > >> > cylindrical equal area grid?
> >> > > > >> > > >> > If so, how do we convert 1-dimensional lat/lon
> >> coordinates
> >> > to
> >> > > > >> the 2d
> >> > > > >> > > >> > coordinates with y,x?
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >>
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> > > > >> > > >> >
> >> > > > >> > > >> > Thank you,
> >> > > > >> > > >> > Tanya Peevey
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >> > On Mon, Feb 8, 2016 at 4:33 PM, Tanya Peevey -
NOAA
> >> > > Affiliate <
> >> > > > >> > > >> > tanya.peevey at noaa.gov <javascript:;>> wrote:
> >> > > > >> > > >> >
> >> > > > >> > > >> > > John,
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > Thank you for the information. We generated a
new
> file
> >> > and
> >> > > I
> >> > > > >> tried
> >> > > > >> > > the
> >> > > > >> > > >> > > plot_data_plane tool on it and got the
following
> >> error.
> >> > Do
> >> > > > you
> >> > > > >> > have
> >> > > > >> > > an
> >> > > > >> > > >> > idea
> >> > > > >> > > >> > > of what could be wrong? Sorry to bug you so
much and
> >> > thanks
> >> > > > in
> >> > > > >> > > advance
> >> > > > >> > > >> > for
> >> > > > >> > > >> > > the help.
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > Here's the location of the file:
> >> > > > >> > > >> > >
> >> > > > >> > > >>
> >> > > > >> > >
> >> > > > >>
> >> > > >
> >> >
> >>
> /scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0/data/gfs_test/
> >> > > > >> > > >> > >
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > Here's the error:
> >> > > > >> > > >> > >
> >> > > > >> >
> >> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >> > > > >> > > >> > >
> >> > > > >> > > >> >
> >> > > > >> > > >>
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>/scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane
> >> > > > >> > > >> > > data/gfs_test/
> >> > > prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > > >> > > >> > > data/gfs_test/lsp_6h.ps 'name="LSP_6h";
> >> level="(*,*)";
> >> > > > >> > > >> > > file_type=NETCDF_NCCF;'
> >> > > > >> > > >> > > DEBUG 1: Opening data file: data/gfs_test/
> >> > > > >> > > >> > >
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.cf.nc
> >> > > > >> > > >> > > ERROR :
> >> > > > >> > > >> > > ERROR : StringArray::operator[](int) const
-> range
> >> > check
> >> > > > >> error!
> >> > > > >> > > >> > > ERROR :
> >> > > > >> > > >> > >
> >> > > > >> >
> >> > Theia04:/scratch4/BMC/shout/Tanya.Peevey/noscrub/tools/MET/met-
5.0>
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > Thank you ,
> >> > > > >> > > >> > > Tanya
> >> > > > >> > > >> > >
> >> > > > >> > > >> > >
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > On Mon, Feb 8, 2016 at 9:40 AM, John Halley
Gotway
> via
> >> > RT <
> >> > > > >> > > >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >> > >
> >> > > > >> > > >> > >> Tanya,
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> Sorry for the delay. I was out of the
office on
> >> Friday.
> >> > > > >> Using
> >> > > > >> > > >> "LSP_6h"
> >> > > > >> > > >> > >> for the name and "(*,*)" for the level
should do
> the
> >> > > trick.
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> When trying to get the MET tools to read a
new
> >> dataset,
> >> > I
> >> > > > >> always
> >> > > > >> > > >> start
> >> > > > >> > > >> > by
> >> > > > >> > > >> > >> using the plot_data_plane tool:
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> /scratch4/BMC/dtc/MET/met-
5.1/bin/plot_data_plane \
> >> > > > >> > > >> > >>
prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc \
> >> > > > >> > > >> > >> lsp_6h.ps 'name="LSP_6h"; level="(*,*)";
> >> > > > >> > > file_type=NETCDF_NCCF;'
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> I use that to make sure the name and level
settings
> >> work
> >> > > as
> >> > > > >> > > expected,
> >> > > > >> > > >> > and
> >> > > > >> > > >> > >> it also creates an output plot so I can see
that
> MET
> >> is
> >> > > (or
> >> > > > is
> >> > > > >> > not)
> >> > > > >> > > >> > >> reading/plotting the data as expected.
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> That configuration string you pass to the
the
> >> > > > plot_data_plane
> >> > > > >> > tool
> >> > > > >> > > is
> >> > > > >> > > >> > >> exactly the same settings as are supported
in the
> >> > > > >> configuration
> >> > > > >> > > >> files.
> >> > > > >> > > >> > In
> >> > > > >> > > >> > >> fact, we read that string with the same code
that
> >> parses
> >> > > the
> >> > > > >> > config
> >> > > > >> > > >> > files.
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> I find using plot_data_plane very helpful.
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> Thanks,
> >> > > > >> > > >> > >> John
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> On Thu, Feb 4, 2016 at 3:07 PM, Tanya Peevey
- NOAA
> >> > > > Affiliate
> >> > > > >> via
> >> > > > >> > > RT
> >> > > > >> > > >> <
> >> > > > >> > > >> > >> met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > <URL:
> >> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> > > > >> > >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > John,
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > Thanks for the info. We are going to work
on
> >> changing
> >> > > our
> >> > > > >> file
> >> > > > >> > > >> format
> >> > > > >> > > >> > >> and
> >> > > > >> > > >> > >> > then I'll get back to you.
> >> > > > >> > > >> > >> > Did you see my other question about how I
> specified
> >> > > 'name'
> >> > > > >> andhow to round to nearest 0.5
> >> > > > >>
> >> > > > >> > > >> 'level'
> >> > > > >> > > >> > >> for
> >> > > > >> > > >> > >> > the precip. field in my config file for
MODE? Was
> >> how
> >> > I
> >> > > > did
> >> > > > >> it
> >> > > > >> > > ok?
> >> > > > >> > > >> I
> >> > > > >> > > >> > ask
> >> > > > >> > > >> > >> > because the manual was confusing on this
point.
> If
> >> my
> >> > > > syntax
> >> > > > >> > was
> >> > > > >> > > >> > >> incorrect
> >> > > > >> > > >> > >> > could you please email me the correction.
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > Thanks,
> >> > > > >> > > >> > >> > Tanya
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > On Thu, Feb 4, 2016 at 11:32 AM, John
Halley
> Gotway
> >> > via
> >> > > > RT <
> >> > > > >> > > >> > >> > met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > > Tanya,
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > Here's a website about it:
> >> > > > >> > > >> > >> > > http://cfconventions.org/latest.html
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > And I posted a sample CF-compliant
NetCDF file
> >> here:
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >>
> >> > > > >> > > >> >
> >> > > > >> > > >>
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/for_tanya/gtg_obs_forecast.20130827.i12.f03.nc
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > The relevant pieces in there are...
> >> > > > >> > > >> > >> > > - The global "Conventions" attribute set
to
> >> > "CF-...".
> >> > > > >> > > >> > >> > > - The time dimension/variable define the
> forecast
> >> > > valid
> >> > > > >> time.
> >> > > > >> > > >> > >> > > - The forecast_reference_time variable
defines
> >> the
> >> > > model
> >> > > > >> > > >> > >> initialization
> >> > > > >> > > >> > >> > > time.
> >> > > > >> > > >> > >> > > - The grid_mapping, x, and y variables
define
> the
> >> > grid
> >> > > > >> > > definition
> >> > > > >> > > >> > >> > > information.
> >> > > > >> > > >> > >> > > - The grid_mapping variable attributes
map
> those
> >> > > > >> variables to
> >> > > > >> > > the
> >> > > > >> > > >> > >> > relevant
> >> > > > >> > > >> > >> > > grid definition.
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > Hope this helps.
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > Thanks,
> >> > > > >> > > >> > >> > > John
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > On Thu, Feb 4, 2016 at 11:21 AM, Tanya
Peevey -
> >> NOAA
> >> > > > >> > Affiliate
> >> > > > >> > > >> via
> >> > > > >> > > >> > RT
> >> > > > >> > > >> > >> <
> >> > > > >> > > >> > >> > > met_help at ucar.edu <javascript:;>> wrote:
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > <URL:
> >> > > > >> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74975
> >> > > > >> > > >> >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > John,
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > To clarify my previous request of
> information.
> >> I
> >> > > meant
> >> > > > >> to
> >> > > > >> > ask
> >> > > > >> > > >> was
> >> > > > >> > > >> > >> the
> >> > > > >> > > >> > >> > > > following: Do you know of any good
> >> documentation
> >> > on
> >> > > > how
> >> > > > >> to
> >> > > > >> > > >> make a
> >> > > > >> > > >> > >> > NetCDF
> >> > > > >> > > >> > >> > > > file CF-compliant? Also, do you have
an
> example
> >> > .nc
> >> > > > file
> >> > > > >> > that
> >> > > > >> > > >> is
> >> > > > >> > > >> > >> NetCDF
> >> > > > >> > > >> > >> > > > CF-compliant that we could compare to?
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > Thanks,
> >> > > > >> > > >> > >> > > > Tanya
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > On Wed, Feb 3, 2016 at 3:53 PM, Tanya
Peevey
> -
> >> > NOAA
> >> > > > >> > > Affiliate <
> >> > > > >> > > >> > >> > > > tanya.peevey at noaa.gov <javascript:;>>
wrote:
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > > John,
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > > We generated the NetCDF files
ourselves so
> we
> >> > > could
> >> > > > >> > > probably
> >> > > > >> > > >> > make
> >> > > > >> > > >> > >> it
> >> > > > >> > > >> > >> > > work
> >> > > > >> > > >> > >> > > > > for MET. Do you know how to get it
into
> >> > > CF-compliant
> >> > > > >> > NetCDF
> >> > > > >> > > >> > >> format?
> >> > > > >> > > >> > >> > > > > Yes, we have grib files but the
reason we
> >> > created
> >> > > > >> NetCDF
> >> > > > >> > > >> files
> >> > > > >> > > >> > was
> >> > > > >> > > >> > >> > > > because
> >> > > > >> > > >> > >> > > > > original the NR... file had
accumulated
> >> precip
> >> > > over
> >> > > > >> the
> >> > > > >> > > whole
> >> > > > >> > > >> > >> > forecast
> >> > > > >> > > >> > >> > > > > where the other file has 6h
accumulation
> >> data,
> >> > so
> >> > > we
> >> > > > >> need
> >> > > > >> > > to
> >> > > > >> > > >> > >> convert
> >> > > > >> > > >> > >> > > the
> >> > > > >> > > >> > >> > > > > NR... precip information into 6h
> accumulation
> >> > > (they
> >> > > > >> had
> >> > > > >> > > >> > different
> >> > > > >> > > >> > >> > units
> >> > > > >> > > >> > >> > > > > too, so we need to change that too).
Since
> we
> >> > had
> >> > > to
> >> > > > >> > change
> >> > > > >> > > >> the
> >> > > > >> > > >> > >> units
> >> > > > >> > > >> > >> > > > too I
> >> > > > >> > > >> > >> > > > > think use generating our own NetCDF
file is
> >> best
> >> > > but
> >> > > > >> we
> >> > > > >> > > just
> >> > > > >> > > >> > need
> >> > > > >> > > >> > >> to
> >> > > > >> > > >> > >> > > get
> >> > > > >> > > >> > >> > > > it
> >> > > > >> > > >> > >> > > > > in the right format.
> >> > > > >> > > >> > >> > > > > What is your opinion on the best way
to
> >> approach
> >> > > > this?
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > > If we get it into the right format
is how I
> >> > > specify
> >> > > > >> the
> >> > > > >> > > >> 'name'
> >> > > > >> > > >> > and
> >> > > > >> > > >> > >> > > > 'level'
> >> > > > >> > > >> > >> > > > > of the field correct? It was hard to
figure
> >> out
> >> > > the
> >> > > > >> > correct
> >> > > > >> > > >> > >> notation
> >> > > > >> > > >> > >> > > from
> >> > > > >> > > >> > >> > > > > the manual.
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > > Thanks,
> >> > > > >> > > >> > >> > > > > Tanya
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > > On Wed, Feb 3, 2016 at 3:25 PM, John
Halley
> >> > Gotway
> >> > > > via
> >> > > > >> > RT <
> >> > > > >> > > >> > >> > > > > met_help at ucar.edu <javascript:;>>
wrote:
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > >> Hi Tanya,
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> I took a look at the gridded NetCDF
data
> >> you're
> >> > > > >> trying
> >> > > > >> > to
> >> > > > >> > > >> pass
> >> > > > >> > > >> > to
> >> > > > >> > > >> > >> > > MODE.
> >> > > > >> > > >> > >> > > > >> Unfortunately, MET doesn't support
all
> >> types of
> >> > > > >> gridded
> >> > > > >> > > >> NetCDF
> >> > > > >> > > >> > >> > files.
> >> > > > >> > > >> > >> > > > >> Instead, it only supports the MET
NetCDF
> >> format
> >> > > > (e.g.
> >> > > > >> > the
> >> > > > >> > > >> > output
> >> > > > >> > > >> > >> of
> >> > > > >> > > >> > >> > > the
> >> > > > >> > > >> > >> > > > >> pcp_combine tool), the output of
the WRF
> >> > pinterp
> >> > > > (or
> >> > > > >> > > >> wrfinterp)
> >> > > > >> > > >> > >> > > > utilities,
> >> > > > >> > > >> > >> > > > >> and CF-compliant NetCDF files.
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> Looking at "
> >> > > > >> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> >> > > > >> > "
> >> > > > >> > > I
> >> > > > >> > > >> see
> >> > > > >> > > >> > >> that
> >> > > > >> > > >> > >> > > it
> >> > > > >> > > >> > >> > > > >> follows none of those conventions.
> >> However, it
> >> > > is
> >> > > > >> > closest
> >> > > > >> > > >> to
> >> > > > >> > > >> > >> > > > CF-compliant
> >> > > > >> > > >> > >> > > > >> NetCDF.
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> When using a new data file format,
instead
> >> of
> >> > > > >> running it
> >> > > > >> > > >> > through
> >> > > > >> > > >> > >> > MODE,
> >> > > > >> > > >> > >> > > > >> I'd suggest using plot_data_plane
to make
> >> sure
> >> > > MET
> >> > > > is
> >> > > > >> > > >> reading
> >> > > > >> > > >> > it
> >> > > > >> > > >> > >> > > > >> correctly. I ran met-
5.1/plot_data_plane
> as
> >> > > > follows:
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >>
> >> > > /scratch4/BMC/dtc/MET/met-5.1/bin/plot_data_plane \
> >> > > > >> > > >> > >> > > > >>
> prwin_ctr.7278.pgbf90.gfs.2006022500.ver.nc
> >> \
> >> > > > >> > > >> > >> > > > >> lsp_6h.ps 'name="LSP_6h";
level="(*,*)";
> >> > > > >> > > >> > file_type=NETCDF_NCCF;'
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> The resulting image is attached.
There
> are
> >> > > several
> >> > > > >> > issues
> >> > > > >> > > >> > here:
> >> > > > >> > > >> > >> > > > >> - I had to tell plot_data_plane to
> interpret
> >> > this
> >> > > > as
> >> > > > >> > > >> > CF-compliant
> >> > > > >> > > >> > >> > > NetCDF
> >> > > > >> > > >> > >> > > > >> using NETCDF_NCCF.
> >> > > > >> > > >> > >> > > > >> - The timing info in that file
isn't
> >> specified
> >> > in
> >> > > > the
> >> > > > >> > > >> > >> CF-compliant
> >> > > > >> > > >> > >> > > > way...
> >> > > > >> > > >> > >> > > > >> so MET can't discern the
initialization,
> >> valid,
> >> > > and
> >> > > > >> lead
> >> > > > >> > > >> times.
> >> > > > >> > > >> > >> > > > >> - And then there's the obvious
problem (in
> >> the
> >> > > > >> image) of
> >> > > > >> > > the
> >> > > > >> > > >> > >> world
> >> > > > >> > > >> > >> > > being
> >> > > > >> > > >> > >> > > > >> upside down and backwards!
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> So while we did get *something*,
there are
> >> > > > obviously
> >> > > > >> a
> >> > > > >> > lot
> >> > > > >> > > >> of
> >> > > > >> > > >> > >> > issues.
> >> > > > >> > > >> > >> > > > >> Basically, MET doesn't support the
gridded
> >> > NetCDF
> >> > > > >> file
> >> > > > >> > > >> format
> >> > > > >> > > >> > you
> >> > > > >> > > >> > >> > are
> >> > > > >> > > >> > >> > > > >> using. Is this data possibly
available in
> >> > GRIB?
> >> > > > Or
> >> > > > >> do
> >> > > > >> > > you
> >> > > > >> > > >> > have
> >> > > > >> > > >> > >> > > control
> >> > > > >> > > >> > >> > > > >> over the NetCDF file format in use?
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >> Thanks,
> >> > > > >> > > >> > >> > > > >> John
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >>
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > > > --
> >> > > > >> > > >> > >> > > > > Tanya R. Peevey, PhD
> >> > > > >> > > >> > >> > > > > Research Scientist I, Global
Observing
> >> Systems
> >> > > > >> Analysis
> >> > > > >> > > >> (GOSA)
> >> > > > >> > > >> > >> Group
> >> > > > >> > > >> > >> > > > > NOAA ESRL Global Systems Division
> >> > > > >> > > >> > >> > > > > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > >> > >> > > > > (303) 497-5847
> >> > > > >> > > >> > >> > > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > > --
> >> > > > >> > > >> > >> > > > Tanya R. Peevey, PhD
> >> > > > >> > > >> > >> > > > Research Scientist I, Global Observing
> Systems
> >> > > > Analysis
> >> > > > >> > > (GOSA)
> >> > > > >> > > >> > Group
> >> > > > >> > > >> > >> > > > NOAA ESRL Global Systems Division
> >> > > > >> > > >> > >> > > > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > >> > >> > > > (303) 497-5847
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > > >
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> > >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> > --
> >> > > > >> > > >> > >> > Tanya R. Peevey, PhD
> >> > > > >> > > >> > >> > Research Scientist I, Global Observing
Systems
> >> > Analysis
> >> > > > >> (GOSA)
> >> > > > >> > > >> Group
> >> > > > >> > > >> > >> > NOAA ESRL Global Systems Division
> >> > > > >> > > >> > >> > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > >> > >> > (303) 497-5847
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >> >
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >>
> >> > > > >> > > >> > >
> >> > > > >> > > >> > >
> >> > > > >> > > >> > > --
> >> > > > >> > > >> > > Tanya R. Peevey, PhD
> >> > > > >> > > >> > > Research Scientist I, Global Observing
Systems
> >> Analysis
> >> > > > (GOSA)
> >> > > > >> > Group
> >> > > > >> > > >> > > NOAA ESRL Global Systems Division
> >> > > > >> > > >> > > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > >> > > (303) 497-5847
> >> > > > >> > > >> > >
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >> > --
> >> > > > >> > > >> > Tanya R. Peevey, PhD
> >> > > > >> > > >> > Research Scientist I, Global Observing Systems
> Analysis
> >> > > (GOSA)
> >> > > > >> Group
> >> > > > >> > > >> > NOAA ESRL Global Systems Division
> >> > > > >> > > >> > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > >> > (303) 497-5847
> >> > > > >> > > >> >
> >> > > > >> > > >> >
> >> > > > >> > > >>
> >> > > > >> > > >>
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > --
> >> > > > >> > > > Tanya R. Peevey, PhD
> >> > > > >> > > > Research Scientist I, Global Observing Systems
Analysis
> >> (GOSA)
> >> > > > Group
> >> > > > >> > > > NOAA ESRL Global Systems Division
> >> > > > >> > > > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > > (303) 497-5847
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > --
> >> > > > >> > > Tanya R. Peevey, PhD
> >> > > > >> > > Research Scientist I, Global Observing Systems
Analysis
> >> (GOSA)
> >> > > Group
> >> > > > >> > > NOAA ESRL Global Systems Division
> >> > > > >> > > 325 Broadway, Boulder, CO 80305
> >> > > > >> > > (303) 497-5847
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >> >
> >> > > > >>
> >> > > > >>
> >> > > > >> --
> >> > > > >> Tanya R. Peevey, PhD
> >> > > > >> Research Scientist I, Global Observing Systems Analysis
(GOSA)
> >> Group
> >> > > > >> NOAA ESRL Global Systems Division
> >> > > > >> 325 Broadway, Boulder, CO 80305
> >> > > > >> (303) 497-5847
> >> > > > >>
> >> > > > >>
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > Tanya R. Peevey, PhD
> >> > > Research Scientist I, Global Observing Systems Analysis
(GOSA) Group
> >> > > NOAA ESRL Global Systems Division
> >> > > 325 Broadway, Boulder, CO 80305
> >> > > (303) 497-5847
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >> --
> >> Tanya R. Peevey, PhD
> >> Research Scientist I, Global Observing Systems Analysis (GOSA)
Group
> >> NOAA ESRL Global Systems Division
> >> 325 Broadway, Boulder, CO 80305
> >> (303) 497-5847
> >>
> >>
> >
>
>
--
Tanya R. Peevey, PhD
Research Scientist I, Global Observing Systems Analysis (GOSA) Group
NOAA ESRL Global Systems Division
325 Broadway, Boulder, CO 80305
(303) 497-5847
------------------------------------------------
More information about the Met_help
mailing list