[Met_help] [rt.rap.ucar.edu #39089] History for MODE usage with netCDF
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Wed Feb 23 14:53:21 MST 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello, I work for the MIT Lincoln Laboratory's Weather Sensing group and I had some interest in using your object verification library.
I was wondering if the MET module's MODE utility is usable with netCDF input files other than those coming out of the pcp combine tool? The documentation seems to indicate that it is NOT and I just wanted some clarification on the issue. If this is the case, is there any plans in the near future for supporting other netCDF input data?
Thanks
Patrick Lamey
Group 43 Weather Sensing
MIT Lincoln Laboratory
Phone: (781)-981-4790
Email: lamey at ll.mit.edu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: John Halley Gotway
Time: Fri Jul 09 12:52:30 2010
Patrick,
Sorry for the delay in getting back to you.
As of METv2.0, the current release, the only NetCDF files that you can
use as input for MODE are those that look like the output of the PCP-
Combine tool. For METv3.0, the upcoming release, we're
working to enhance our support for NetCDF data. In particular, we're
trying to support the output of the pinterp tool which may be used to
post-process WRF-ARW output. Ultimately, we'd like to
support NetCDF files that conform to the CF-convention.
But for the time being, your stuck with our very limited support for
NetCDF.
Hope that helps clarify.
Thanks,
John Halley Gotway
met_help at ucar.edu
RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
> Wed Jul 07 13:16:25 2010: Request 39089 was acted upon.
> Transaction: Ticket created by lamey at ll.mit.edu
> Queue: met_help
> Subject: MODE usage with netCDF
> Owner: Nobody
> Requestors: lamey at ll.mit.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
>
>
> Hello, I work for the MIT Lincoln Laboratory's Weather Sensing group
and I had some interest in using your object verification library.
> I was wondering if the MET module's MODE utility is usable with
netCDF input files other than those coming out of the pcp combine
tool? The documentation seems to indicate that it is NOT and I just
wanted some clarification on the issue. If this is the case, is there
any plans in the near future for supporting other netCDF input data?
> Thanks
>
> Patrick Lamey
> Group 43 Weather Sensing
> MIT Lincoln Laboratory
> Phone: (781)-981-4790
> Email: lamey at ll.mit.edu
>
>
>
>
>
------------------------------------------------------------------------
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: Lamey, Patrick - 0443 - MITLL
Time: Wed Jul 28 13:42:27 2010
Hello, I inquired a while back about using NetCDF files with the MODE
module. I know that the netCDF input file needs to be in the PCP-
Combine format. I was wondering if there is a routine to reformat
other netCDF files into a PCP-Combine format? If not is there any
documentation that you could provide on the format of the PCP-Combine
output netCDF files?
Thanks,
Patrick Lamey
Group 43 Weather Sensing
MIT LINCOLN LABORATORY
-----Original Message-----
From: RAL
HelpDesk {for John Halley Gotway} [mailto:met_help at ucar.edu]
Sent:
Friday, July 09, 2010 2:53 PM
To: Lamey, Patrick - 0443 - MITLL
Subject: Re: [rt.rap.ucar.edu #39089] MODE usage with netCDF
Patrick,
Sorry for the delay in getting back to you.
As of
METv2.0, the current release, the only NetCDF files that you can use
as input for MODE are those that look like the output of the PCP-
Combine tool. For METv3.0, the upcoming release, we're
working to
enhance our support for NetCDF data. In particular, we're trying to
support the output of the pinterp tool which may be used to post-
process WRF-ARW output. Ultimately, we'd like to
support NetCDF
files that conform to the CF-convention.
But for the time being,
your stuck with our very limited support for NetCDF.
Hope that
helps clarify.
Thanks,
John Halley Gotway
met_help at ucar.edu
RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
> Wed Jul 07
13:16:25 2010: Request 39089 was acted upon.
> Transaction: Ticket
created by lamey at ll.mit.edu
> Queue: met_help
> Subject:
MODE usage with netCDF
> Owner: Nobody
> Requestors:
lamey at ll.mit.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
>
>
> Hello, I work for the MIT Lincoln Laboratory's Weather Sensing group
and I had some interest in using your object verification library.
>
I was wondering if the MET module's MODE utility is usable with netCDF
input files other than those coming out of the pcp combine tool? The
documentation seems to indicate that it is NOT and I just wanted some
clarification on the issue. If this is the case, is there any plans in
the near future for supporting other netCDF input data?
> Thanks
>
> Patrick Lamey
> Group 43 Weather Sensing
> MIT Lincoln Laboratory
> Phone: (781)-981-4790
> Email: lamey at ll.mit.edu
>
>
>
>
>
------------------------------------------------------------------------
>
------------------------------------------------
Subject: MODE usage with netCDF
From: John Halley Gotway
Time: Wed Jul 28 16:37:50 2010
Patrick,
We're currently finishing up development on METv3.0. We have a couple
more items to address, including our support for reading NetCDF data
files.
We're updating METv3.0 to read the NetCDF output of the WRF-ARW-
pinterp post-processing tool. And we'd like to make those changes as
general as we can in hopes of supporting a wider variety of NetCDF
formats.
I have examples of NetCDF files from a couple of users right now, that
we'll be using as we make these updates. I'm wondering if you'd be
willing to send me some of your NetCDF files to see what would need to
change to use them in MET.
As far as modifying your NetCDF files for use in MET, it's a bit of an
awkward time right now. The way we encode the grid definition
information in the NetCDF header has changed for METv3.0. So if
possible, I'd suggest waiting for METv3.0 before working on this. I'd
hate to see you get it working with METv2.0 and then have to change it
for METv3.0.
We don't have tools available for converting NetCDF files for use in
MET. I do know other MET users have used ncgen with a CDL file.
Really the best example is the NetCDF files that are created by the
pcp_combine test script in the MET tarball. I do have an old write-up
about the NetCDF format, but it's out of date for METv3.0 - which we
plan to release by the end of August.
If you'd be willing to send me some sample NetCDF files, you could
post them to our anonymous ftp site at:
ftp ftp.rap.ucar.edu
username=anonymous
password="your email address"
cd incoming/irap/met_help/lamey_data
put "your files"
bye
Thanks,
John Halley Gotway
met_help at ucar.edu
On Wed Jul 28 13:42:27 2010, lamey at ll.mit.edu wrote:
> Hello, I inquired a while back about using NetCDF files with the
MODE
> module. I know that the netCDF input file needs to be in the PCP-
> Combine format. I was wondering if there is a routine to reformat
> other netCDF files into a PCP-Combine format? If not is there any
> documentation that you could provide on the format of the PCP-
> Combine output netCDF files?
> Thanks,
> Patrick Lamey
> Group 43
> Weather Sensing
> MIT LINCOLN LABORATORY
>
> -----Original
> Message-----
> From: RAL HelpDesk {for John Halley Gotway}
> [mailto:met_help at ucar.edu]
> Sent: Friday, July 09, 2010 2:53 PM
> To: Lamey, Patrick - 0443 - MITLL
> Subject: Re: [rt.rap.ucar.edu
> #39089] MODE usage with netCDF
>
> Patrick,
>
> Sorry for the delay
> in getting back to you.
>
> As of METv2.0, the current release, the
> only NetCDF files that you can use as input for MODE are those
that
> look like the output of the PCP-Combine tool. For METv3.0, the
> upcoming release, we're
> working to enhance our support for NetCDF
> data. In particular, we're trying to support the output of the
> pinterp tool which may be used to post-process WRF-ARW output.
> Ultimately, we'd like to
> support NetCDF files that conform to the
> CF-convention.
>
> But for the time being, your stuck with our very
> limited support for NetCDF.
>
> Hope that helps clarify.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> RAL HelpDesk {for Lamey,
> Patrick - 0443 - MITLL} wrote:
> > Wed Jul 07 13:16:25 2010: Request
> 39089 was acted upon.
> > Transaction: Ticket created by
> lamey at ll.mit.edu
> > Queue: met_help
> > Subject: MODE
> usage with netCDF
> > Owner: Nobody
> > Requestors:
> lamey at ll.mit.edu
> > Status: new
> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
> >
> >
> >
> > Hello, I work for the MIT Lincoln Laboratory's Weather
> Sensing group and I had some interest in using your object
> verification library.
> > I was wondering if the MET module's MODE
> utility is usable with netCDF input files other than those coming
> out of the pcp combine tool? The documentation seems to indicate
> that it is NOT and I just wanted some clarification on the issue.
> If this is the case, is there any plans in the near future for
> supporting other netCDF input data?
> > Thanks
> >
> > Patrick Lamey
> > Group 43 Weather Sensing
> > MIT Lincoln Laboratory
> > Phone:
> (781)-981-4790
> > Email: lamey at ll.mit.edu
> >
> >
> >
> >
> >
>
------------------------------------------------------------------------
> >
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: Lamey, Patrick - 0443 - MITLL
Time: Thu Jul 29 12:01:19 2010
Hello,
I've placed examples of our NetCDF format files in the
directory "incoming/irap/met_help/lamey_data" on your ftp server. If
you need more examples please let me know.
Please note that the
format of these Netcdf files is very similar to a format that has been
agreed upon between MITLL and NCAR for transferring data between these
organizations as part of the FAA CoSPA project. The main difference is
that MITLL places multiple forecast time horizons in one file, whereas
NCAR breaks the time horizons out into separate files. The NCAR people
that we routinely work with are members of the NCAR RAL group,
specifically, Matthias Steiner, James Pinto, John Williams, David
Ahijevych, Huaqing Cai to name a few. So modifications to MODE to
accommodate this format may have broad appeal.
Patrick Lamey
Group 43 Weather Sensing
MIT Lincoln Laboratory
Phone: (781)-981-
4790
Email: lamey at ll.mit.edu
-----Original Message-----
From:
RAL HelpDesk {for John Halley Gotway} [mailto:met_help at ucar.edu]
Sent: Wednesday, July 28, 2010 6:38 PM
To: Lamey, Patrick - 0443 -
MITLL
Subject: [rt.rap.ucar.edu #39089] MODE usage with netCDF
Patrick,
We're currently finishing up development on METv3.0. We
have a couple more items to address, including our support for reading
NetCDF data files.
We're updating METv3.0 to read the NetCDF output
of the WRF-ARW-pinterp post-processing tool. And we'd like to make
those changes as general as we can in hopes of supporting a wider
variety of NetCDF formats.
I have examples of NetCDF files from a
couple of users right now, that we'll be using as we make these
updates. I'm wondering if you'd be willing to send me some of your
NetCDF files to see what would need to change to use them in MET.
As far as modifying your NetCDF files for use in MET, it's a bit of an
awkward time right now. The way we encode the grid definition
information in the NetCDF header has changed for METv3.0. So if
possible, I'd suggest waiting for METv3.0 before working on this. I'd
hate to see you get it working with METv2.0 and then have to change it
for METv3.0.
We don't have tools available for converting NetCDF
files for use in MET. I do know other MET users have used ncgen with
a CDL file. Really the best example is the NetCDF files that are
created by the pcp_combine test script in the MET tarball. I do have
an old write-up about the NetCDF format, but it's out of date for
METv3.0 - which we plan to release by the end of August.
If you'd
be willing to send me some sample NetCDF files, you could post them to
our anonymous ftp site at:
ftp ftp.rap.ucar.edu
username=anonymous
password="your email address"
cd incoming/irap/met_help/lamey_data
put "your files"
bye
Thanks,
John Halley Gotway
met_help at ucar.edu
On Wed Jul 28 13:42:27 2010, lamey at ll.mit.edu
wrote:
> Hello, I inquired a while back about using NetCDF files with
the MODE
> module. I know that the netCDF input file needs to be
in the PCP-
> Combine format. I was wondering if there is a
routine to reformat
> other netCDF files into a PCP-Combine
format? If not is there any
> documentation that you could provide
on the format of the PCP-
> Combine output netCDF files?
>
Thanks,
> Patrick Lamey
> Group 43
> Weather Sensing
> MIT
LINCOLN LABORATORY
>
> -----Original
> Message-----
> From:
RAL HelpDesk {for John Halley Gotway}
> [mailto:met_help at ucar.edu]
> Sent: Friday, July 09, 2010 2:53 PM
> To: Lamey, Patrick - 0443
- MITLL
> Subject: Re: [rt.rap.ucar.edu
> #39089] MODE usage with
netCDF
>
> Patrick,
>
> Sorry for the delay
> in getting
back to you.
>
> As of METv2.0, the current release, the
> only
NetCDF files that you can use as input for MODE are those that
>
look like the output of the PCP-Combine tool. For METv3.0, the
>
upcoming release, we're
> working to enhance our support for NetCDF
> data. In particular, we're trying to support the output of the
> pinterp tool which may be used to post-process WRF-ARW output.
>
Ultimately, we'd like to
> support NetCDF files that conform to the
> CF-convention.
>
> But for the time being, your stuck with our
very
> limited support for NetCDF.
>
> Hope that helps clarify.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> RAL
HelpDesk {for Lamey,
> Patrick - 0443 - MITLL} wrote:
> > Wed Jul
07 13:16:25 2010: Request
> 39089 was acted upon.
> >
Transaction: Ticket created by
> lamey at ll.mit.edu
> >
Queue: met_help
> > Subject: MODE
> usage with netCDF
> >
Owner: Nobody
> > Requestors:
> lamey at ll.mit.edu
> >
Status: new
> > Ticket <URL:
>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
> >
> >
> >
> > Hello, I work for the MIT Lincoln Laboratory's Weather
>
Sensing group and I had some interest in using your object
>
verification library.
> > I was wondering if the MET module's MODE
>
utility is usable with netCDF input files other than those coming
>
out of the pcp combine tool? The documentation seems to indicate
>
that it is NOT and I just wanted some clarification on the issue.
>
If this is the case, is there any plans in the near future for
>
supporting other netCDF input data?
> > Thanks
> >
> > Patrick
Lamey
> > Group 43 Weather Sensing
> > MIT Lincoln Laboratory
>
> Phone:
> (781)-981-4790
> > Email: lamey at ll.mit.edu
> >
> >
> >
> >
> >
>
------------------------------------------------------------------------
> >
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: John Halley Gotway
Time: Thu Jul 29 12:15:51 2010
Patrick,
Thanks for sending me those sample files. I see that they are in
NetCDF4 format following the CF-1.3 convention. Our goal is to work
toward supporting CF-compliant files. However, I can tell you
that we haven't made the switch over from NetCDF3 to NetCDF4 yet and
certainly won't be doing so prior to the upcoming release. So we will
not be able to read these files directly in METv3.0.
I have been in contact with James and Huaqing over the last couple
years about their use of MET, MODE in particular. Currently, they're
converting their MDV data into GRIB1 prior to running it
through MET. I do know they'd prefer to skip that conversion step and
just use NetCDF files directly. Unfortunately, we haven't had the
time or funding to add that capability.
I'm not sure how you'd like to proceed. If you have by chance have
tools available for writing your data in GRIB1 format, that would make
using MET the most straight-forward. Otherwise, you'd need
to reformat your NetCDF data into the NetCDF3 classic data model for
use in MET.
Sorry this is such a hassle, but handling data format often is!
Thanks,
John
RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
> Hello,
> I've placed examples of our NetCDF format files in the directory
"incoming/irap/met_help/lamey_data" on your ftp server. If you need
more examples please let me know.
>
> Please note that the format of these Netcdf files is very similar to
a format that has been agreed upon between MITLL and NCAR for
transferring data between these organizations as part of the FAA CoSPA
project. The main difference is that MITLL places multiple forecast
time horizons in one file, whereas NCAR breaks the time horizons out
into separate files. The NCAR people that we routinely work with are
members of the NCAR RAL group, specifically, Matthias Steiner, James
Pinto, John Williams, David Ahijevych, Huaqing Cai to name a few. So
modifications to MODE to accommodate this format may have broad
appeal.
>
> Patrick Lamey
> Group 43 Weather Sensing
> MIT Lincoln Laboratory
> Phone: (781)-981-4790
> Email: lamey at ll.mit.edu
>
>
> -----Original Message-----
> From: RAL HelpDesk {for John Halley Gotway}
[mailto:met_help at ucar.edu]
> Sent: Wednesday, July 28, 2010 6:38 PM
> To: Lamey, Patrick - 0443 - MITLL
> Subject: [rt.rap.ucar.edu #39089] MODE usage with netCDF
>
> Patrick,
>
> We're currently finishing up development on METv3.0. We have a
couple more items to address, including our support for reading NetCDF
data files.
>
> We're updating METv3.0 to read the NetCDF output of the WRF-ARW-
pinterp post-processing tool. And we'd like to make those changes as
general as we can in hopes of supporting a wider variety of NetCDF
formats.
>
> I have examples of NetCDF files from a couple of users right now,
that we'll be using as we make these updates. I'm wondering if you'd
be willing to send me some of your NetCDF files to see what would need
to change to use them in MET.
>
> As far as modifying your NetCDF files for use in MET, it's a bit of
an awkward time right now. The way we encode the grid definition
information in the NetCDF header has changed for METv3.0. So if
possible, I'd suggest waiting for METv3.0 before working on this. I'd
hate to see you get it working with METv2.0 and then have to change it
for METv3.0.
>
> We don't have tools available for converting NetCDF files for use in
MET. I do know other MET users have used ncgen with a CDL file.
Really the best example is the NetCDF files that are created by the
pcp_combine test script in the MET tarball. I do have an old write-up
about the NetCDF format, but it's out of date for METv3.0 - which we
plan to release by the end of August.
>
> If you'd be willing to send me some sample NetCDF files, you could
post them to our anonymous ftp site at:
>
> ftp ftp.rap.ucar.edu
> username=anonymous
> password="your email address"
> cd incoming/irap/met_help/lamey_data
> put "your files"
> bye
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> On Wed Jul 28 13:42:27 2010, lamey at ll.mit.edu wrote:
>> Hello, I inquired a while back about using NetCDF files with the
MODE
>> module. I know that the netCDF input file needs to be in the
PCP-
>> Combine format. I was wondering if there is a routine to
reformat
>> other netCDF files into a PCP-Combine format? If not is there
any
>> documentation that you could provide on the format of the PCP-
>> Combine output netCDF files?
>> Thanks,
>> Patrick Lamey
>> Group 43
>> Weather Sensing
>> MIT LINCOLN LABORATORY
>>
>> -----Original
>> Message-----
>> From: RAL HelpDesk {for John Halley Gotway}
>> [mailto:met_help at ucar.edu]
>> Sent: Friday, July 09, 2010 2:53 PM
>> To: Lamey, Patrick - 0443 - MITLL
>> Subject: Re: [rt.rap.ucar.edu
>> #39089] MODE usage with netCDF
>>
>> Patrick,
>>
>> Sorry for the delay
>> in getting back to you.
>>
>> As of METv2.0, the current release, the
>> only NetCDF files that you can use as input for MODE are those
that
>> look like the output of the PCP-Combine tool. For METv3.0, the
>> upcoming release, we're
>> working to enhance our support for NetCDF
>> data. In particular, we're trying to support the output of the
>> pinterp tool which may be used to post-process WRF-ARW output.
>> Ultimately, we'd like to
>> support NetCDF files that conform to the
>> CF-convention.
>>
>> But for the time being, your stuck with our very
>> limited support for NetCDF.
>>
>> Hope that helps clarify.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>> RAL HelpDesk {for Lamey,
>> Patrick - 0443 - MITLL} wrote:
>>> Wed Jul 07 13:16:25 2010: Request
>> 39089 was acted upon.
>>> Transaction: Ticket created by
>> lamey at ll.mit.edu
>>> Queue: met_help
>>> Subject: MODE
>> usage with netCDF
>>> Owner: Nobody
>>> Requestors:
>> lamey at ll.mit.edu
>>> Status: new
>>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>>
>> >
>>> Hello, I work for the MIT Lincoln Laboratory's Weather
>> Sensing group and I had some interest in using your object
>> verification library.
>>> I was wondering if the MET module's MODE
>> utility is usable with netCDF input files other than those
coming
>> out of the pcp combine tool? The documentation seems to indicate
>> that it is NOT and I just wanted some clarification on the
issue.
>> If this is the case, is there any plans in the near future for
>> supporting other netCDF input data?
>>> Thanks
>>>
>>> Patrick Lamey
>> > Group 43 Weather Sensing
>>> MIT Lincoln Laboratory
>>> Phone:
>> (781)-981-4790
>>> Email: lamey at ll.mit.edu
>>>
>>>
>>>
>>>
>>>
>>
------------------------------------------------------------------------
>> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: Lamey, Patrick - 0443 - MITLL
Time: Thu Jul 29 12:52:58 2010
Thanks for your help John. We have the capability of writing files
into NetCDF3 format. I have uploaded a few more examples to
"incoming/irap/met_help/lamey_data" on the ftp server. Could you
clarify what "NetCDF3 classic data model" refers to? Is that the data
format of the pcp_combine output?
Thanks,
Pat
-----Original
Message-----
From: RAL HelpDesk {for John Halley Gotway}
[mailto:met_help at ucar.edu]
Sent: Thursday, July 29, 2010 2:16 PM
To: Lamey, Patrick - 0443 - MITLL
Subject: Re: [rt.rap.ucar.edu
#39089] MODE usage with netCDF
Patrick,
Thanks for sending me
those sample files. I see that they are in NetCDF4 format following
the CF-1.3 convention. Our goal is to work toward supporting CF-
compliant files. However, I can tell you
that we haven't made the
switch over from NetCDF3 to NetCDF4 yet and certainly won't be doing
so prior to the upcoming release. So we will not be able to read
these files directly in METv3.0.
I have been in contact with James
and Huaqing over the last couple years about their use of MET, MODE in
particular. Currently, they're converting their MDV data into GRIB1
prior to running it
through MET. I do know they'd prefer to skip
that conversion step and just use NetCDF files directly.
Unfortunately, we haven't had the time or funding to add that
capability.
I'm not sure how you'd like to proceed. If you have by
chance have tools available for writing your data in GRIB1 format,
that would make using MET the most straight-forward. Otherwise, you'd
need
to reformat your NetCDF data into the NetCDF3 classic data model
for use in MET.
Sorry this is such a hassle, but handling data
format often is!
Thanks,
John
RAL HelpDesk {for Lamey, Patrick
- 0443 - MITLL} wrote:
> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
>
Hello,
> I've placed examples of our NetCDF format files in the
directory "incoming/irap/met_help/lamey_data" on your ftp server. If
you need more examples please let me know.
>
> Please note that
the format of these Netcdf files is very similar to a format that has
been agreed upon between MITLL and NCAR for transferring data between
these organizations as part of the FAA CoSPA project. The main
difference is that MITLL places multiple forecast time horizons in one
file, whereas NCAR breaks the time horizons out into separate files.
The NCAR people that we routinely work with are members of the NCAR
RAL group, specifically, Matthias Steiner, James Pinto, John Williams,
David Ahijevych, Huaqing Cai to name a few. So modifications to MODE
to accommodate this format may have broad appeal.
>
> Patrick
Lamey
> Group 43 Weather Sensing
> MIT Lincoln Laboratory
> Phone:
(781)-981-4790
> Email: lamey at ll.mit.edu
>
>
> -----Original
Message-----
> From: RAL HelpDesk {for John Halley Gotway}
[mailto:met_help at ucar.edu]
> Sent: Wednesday, July 28, 2010 6:38 PM
> To: Lamey, Patrick - 0443 - MITLL
> Subject: [rt.rap.ucar.edu
#39089] MODE usage with netCDF
>
> Patrick,
>
> We're currently
finishing up development on METv3.0. We have a couple more items to
address, including our support for reading NetCDF data files.
>
>
We're updating METv3.0 to read the NetCDF output of the WRF-ARW-
pinterp post-processing tool. And we'd like to make those changes as
general as we can in hopes of supporting a wider variety of NetCDF
formats.
>
> I have examples of NetCDF files from a couple of users
right now, that we'll be using as we make these updates. I'm
wondering if you'd be willing to send me some of your NetCDF files to
see what would need to change to use them in MET.
>
> As far as
modifying your NetCDF files for use in MET, it's a bit of an awkward
time right now. The way we encode the grid definition information in
the NetCDF header has changed for METv3.0. So if possible, I'd
suggest waiting for METv3.0 before working on this. I'd hate to see
you get it working with METv2.0 and then have to change it for
METv3.0.
>
> We don't have tools available for converting NetCDF
files for use in MET. I do know other MET users have used ncgen with
a CDL file. Really the best example is the NetCDF files that are
created by the pcp_combine test script in the MET tarball. I do have
an old write-up about the NetCDF format, but it's out of date for
METv3.0 - which we plan to release by the end of August.
>
> If
you'd be willing to send me some sample NetCDF files, you could post
them to our anonymous ftp site at:
>
> ftp ftp.rap.ucar.edu
>
username=anonymous
> password="your email address"
> cd
incoming/irap/met_help/lamey_data
> put "your files"
> bye
>
>
Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> On Wed Jul
28 13:42:27 2010, lamey at ll.mit.edu wrote:
>> Hello, I inquired a
while back about using NetCDF files with the MODE
>> module. I
know that the netCDF input file needs to be in the PCP-
>> Combine
format. I was wondering if there is a routine to reformat
>> other
netCDF files into a PCP-Combine format? If not is there any
>>
documentation that you could provide on the format of the PCP-
>>
Combine output netCDF files?
>> Thanks,
>> Patrick Lamey
>> Group
43
>> Weather Sensing
>> MIT LINCOLN LABORATORY
>>
>>
-----Original
>> Message-----
>> From: RAL HelpDesk {for John
Halley Gotway}
>> [mailto:met_help at ucar.edu]
>> Sent: Friday,
July 09, 2010 2:53 PM
>> To: Lamey, Patrick - 0443 - MITLL
>>
Subject: Re: [rt.rap.ucar.edu
>> #39089] MODE usage with netCDF
>>
>> Patrick,
>>
>> Sorry for the delay
>> in getting back to
you.
>>
>> As of METv2.0, the current release, the
>> only
NetCDF files that you can use as input for MODE are those that
>>
look like the output of the PCP-Combine tool. For METv3.0, the
>>
upcoming release, we're
>> working to enhance our support for NetCDF
>> data. In particular, we're trying to support the output of the
>> pinterp tool which may be used to post-process WRF-ARW output.
>> Ultimately, we'd like to
>> support NetCDF files that conform
to the
>> CF-convention.
>>
>> But for the time being, your
stuck with our very
>> limited support for NetCDF.
>>
>> Hope
that helps clarify.
>>
>> Thanks,
>> John Halley Gotway
>>
met_help at ucar.edu
>>
>> RAL HelpDesk {for Lamey,
>> Patrick -
0443 - MITLL} wrote:
>>> Wed Jul 07 13:16:25 2010: Request
>>
39089 was acted upon.
>>> Transaction: Ticket created by
>>
lamey at ll.mit.edu
>>> Queue: met_help
>>> Subject: MODE
>> usage with netCDF
>>> Owner: Nobody
>>> Requestors:
>> lamey at ll.mit.edu
>>> Status: new
>>> Ticket <URL:
>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>>
>>
>
>>> Hello, I work for the MIT Lincoln Laboratory's Weather
>>
Sensing group and I had some interest in using your object
>>
verification library.
>>> I was wondering if the MET module's MODE
>> utility is usable with netCDF input files other than those
coming
>> out of the pcp combine tool? The documentation seems to
indicate
>> that it is NOT and I just wanted some clarification on
the issue.
>> If this is the case, is there any plans in the near
future for
>> supporting other netCDF input data?
>>> Thanks
>>>
>>> Patrick Lamey
>> > Group 43 Weather Sensing
>>> MIT Lincoln
Laboratory
>>> Phone:
>> (781)-981-4790
>>> Email:
lamey at ll.mit.edu
>>>
>>>
>>>
>>>
>>>
>>
------------------------------------------------------------------------
>> >
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: John Halley Gotway
Time: Thu Jul 29 13:33:12 2010
Patrick,
Great. Thanks for sending me those NetCDF3 files. I'll refer to them
over the next couple of weeks when I get a chance to work on this.
During this week and next we'll be preparing for/teaching the
MET tutorial here. These files will serve as a good example of the
CF-compliant NetCDF files we'd ultimately like to support. I'll let
you know if any questions come up about them.
To answer your question, by "classic data model", I was drawing a
distinction between the older NetCDF3 file format (classic data model)
and the newer NetCDF4 file format that's built on top of HDF5.
Thanks,
John
RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
> Thanks for your help John. We have the capability of writing files
into NetCDF3 format. I have uploaded a few more examples to
"incoming/irap/met_help/lamey_data" on the ftp server. Could you
clarify what "NetCDF3 classic data model" refers to? Is that the data
format of the pcp_combine output?
> Thanks,
> Pat
>
> -----Original Message-----
> From: RAL HelpDesk {for John Halley Gotway}
[mailto:met_help at ucar.edu]
> Sent: Thursday, July 29, 2010 2:16 PM
> To: Lamey, Patrick - 0443 - MITLL
> Subject: Re: [rt.rap.ucar.edu #39089] MODE usage with netCDF
>
> Patrick,
>
> Thanks for sending me those sample files. I see that they are in
NetCDF4 format following the CF-1.3 convention. Our goal is to work
toward supporting CF-compliant files. However, I can tell you
> that we haven't made the switch over from NetCDF3 to NetCDF4 yet and
certainly won't be doing so prior to the upcoming release. So we will
not be able to read these files directly in METv3.0.
>
> I have been in contact with James and Huaqing over the last couple
years about their use of MET, MODE in particular. Currently, they're
converting their MDV data into GRIB1 prior to running it
> through MET. I do know they'd prefer to skip that conversion step
and just use NetCDF files directly. Unfortunately, we haven't had the
time or funding to add that capability.
>
> I'm not sure how you'd like to proceed. If you have by chance have
tools available for writing your data in GRIB1 format, that would make
using MET the most straight-forward. Otherwise, you'd need
> to reformat your NetCDF data into the NetCDF3 classic data model for
use in MET.
>
> Sorry this is such a hassle, but handling data format often is!
>
> Thanks,
> John
>
> RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>
>> Hello,
>> I've placed examples of our NetCDF format files in the directory
"incoming/irap/met_help/lamey_data" on your ftp server. If you need
more examples please let me know.
>>
>> Please note that the format of these Netcdf files is very similar
to a format that has been agreed upon between MITLL and NCAR for
transferring data between these organizations as part of the FAA CoSPA
project. The main difference is that MITLL places multiple forecast
time horizons in one file, whereas NCAR breaks the time horizons out
into separate files. The NCAR people that we routinely work with are
members of the NCAR RAL group, specifically, Matthias Steiner, James
Pinto, John Williams, David Ahijevych, Huaqing Cai to name a few. So
modifications to MODE to accommodate this format may have broad
appeal.
>>
>> Patrick Lamey
>> Group 43 Weather Sensing
>> MIT Lincoln Laboratory
>> Phone: (781)-981-4790
>> Email: lamey at ll.mit.edu
>>
>>
>> -----Original Message-----
>> From: RAL HelpDesk {for John Halley Gotway}
[mailto:met_help at ucar.edu]
>> Sent: Wednesday, July 28, 2010 6:38 PM
>> To: Lamey, Patrick - 0443 - MITLL
>> Subject: [rt.rap.ucar.edu #39089] MODE usage with netCDF
>>
>> Patrick,
>>
>> We're currently finishing up development on METv3.0. We have a
couple more items to address, including our support for reading NetCDF
data files.
>>
>> We're updating METv3.0 to read the NetCDF output of the WRF-ARW-
pinterp post-processing tool. And we'd like to make those changes as
general as we can in hopes of supporting a wider variety of NetCDF
formats.
>>
>> I have examples of NetCDF files from a couple of users right now,
that we'll be using as we make these updates. I'm wondering if you'd
be willing to send me some of your NetCDF files to see what would need
to change to use them in MET.
>>
>> As far as modifying your NetCDF files for use in MET, it's a bit of
an awkward time right now. The way we encode the grid definition
information in the NetCDF header has changed for METv3.0. So if
possible, I'd suggest waiting for METv3.0 before working on this. I'd
hate to see you get it working with METv2.0 and then have to change it
for METv3.0.
>>
>> We don't have tools available for converting NetCDF files for use
in MET. I do know other MET users have used ncgen with a CDL file.
Really the best example is the NetCDF files that are created by the
pcp_combine test script in the MET tarball. I do have an old write-up
about the NetCDF format, but it's out of date for METv3.0 - which we
plan to release by the end of August.
>>
>> If you'd be willing to send me some sample NetCDF files, you could
post them to our anonymous ftp site at:
>>
>> ftp ftp.rap.ucar.edu
>> username=anonymous
>> password="your email address"
>> cd incoming/irap/met_help/lamey_data
>> put "your files"
>> bye
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>> On Wed Jul 28 13:42:27 2010, lamey at ll.mit.edu wrote:
>>> Hello, I inquired a while back about using NetCDF files with the
MODE
>>> module. I know that the netCDF input file needs to be in the
PCP-
>>> Combine format. I was wondering if there is a routine to
reformat
>>> other netCDF files into a PCP-Combine format? If not is there
any
>>> documentation that you could provide on the format of the PCP-
>>> Combine output netCDF files?
>>> Thanks,
>>> Patrick Lamey
>>> Group 43
>>> Weather Sensing
>>> MIT LINCOLN LABORATORY
>>>
>>> -----Original
>>> Message-----
>>> From: RAL HelpDesk {for John Halley Gotway}
>>> [mailto:met_help at ucar.edu]
>>> Sent: Friday, July 09, 2010 2:53 PM
>>> To: Lamey, Patrick - 0443 - MITLL
>>> Subject: Re: [rt.rap.ucar.edu
>>> #39089] MODE usage with netCDF
>>>
>>> Patrick,
>>>
>>> Sorry for the delay
>>> in getting back to you.
>>>
>>> As of METv2.0, the current release, the
>>> only NetCDF files that you can use as input for MODE are those
that
>>> look like the output of the PCP-Combine tool. For METv3.0, the
>>> upcoming release, we're
>>> working to enhance our support for NetCDF
>>> data. In particular, we're trying to support the output of the
>>> pinterp tool which may be used to post-process WRF-ARW output.
>>> Ultimately, we'd like to
>>> support NetCDF files that conform to the
>>> CF-convention.
>>>
>>> But for the time being, your stuck with our very
>>> limited support for NetCDF.
>>>
>>> Hope that helps clarify.
>>>
>>> Thanks,
>>> John Halley Gotway
>>> met_help at ucar.edu
>>>
>>> RAL HelpDesk {for Lamey,
>>> Patrick - 0443 - MITLL} wrote:
>>>> Wed Jul 07 13:16:25 2010: Request
>>> 39089 was acted upon.
>>>> Transaction: Ticket created by
>>> lamey at ll.mit.edu
>>>> Queue: met_help
>>>> Subject: MODE
>>> usage with netCDF
>>>> Owner: Nobody
>>>> Requestors:
>>> lamey at ll.mit.edu
>>>> Status: new
>>>> Ticket <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>> >
>>>> Hello, I work for the MIT Lincoln Laboratory's Weather
>>> Sensing group and I had some interest in using your object
>>> verification library.
>>>> I was wondering if the MET module's MODE
>>> utility is usable with netCDF input files other than those
coming
>>> out of the pcp combine tool? The documentation seems to
indicate
>>> that it is NOT and I just wanted some clarification on the
issue.
>>> If this is the case, is there any plans in the near future for
>>> supporting other netCDF input data?
>>>> Thanks
>>>>
>>>> Patrick Lamey
>>> > Group 43 Weather Sensing
>>>> MIT Lincoln Laboratory
>>>> Phone:
>>> (781)-981-4790
>>>> Email: lamey at ll.mit.edu
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
------------------------------------------------------------------------
>>> >
>>
>>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #39089] MODE usage with netCDF
From: Lamey, Patrick - 0443 - MITLL
Time: Mon Aug 30 13:45:17 2010
Hello John,
I have a question about writing grib files. Hopefully
you'll have an answer! We are trying to convert the data in our grid
structures into grib1 format. Do you know of any grid size limitations
in the GRIB1 format? I haven't been able to find any documentation on
data size limitations. Typically we have grid sizes of 5120x3520. I've
been using the gribw library to write grib records and it works great
for smaller (order of 10 smaller) grid sizes. I think it's possible
that the limit is being hit somewhere in the gribw library. If there
is a size limit, do you know if there is any good way to break a grid
up into separate records? Ultimately we'd like to preserve all the
data without resampling. Any information you have would be of great
help.
Thanks,
Patrick Lamey
Group 43 Weather Sensing
MIT
Lincoln Laboratory
Phone: (781)-981-4790
Email: lamey at ll.mit.edu
-----Original Message-----
From: RAL HelpDesk {for John Halley
Gotway} [mailto:met_help at ucar.edu]
Sent: Thursday, July 29, 2010
3:33 PM
To: Lamey, Patrick - 0443 - MITLL
Subject: Re:
[rt.rap.ucar.edu #39089] MODE usage with netCDF
Patrick,
Great.
Thanks for sending me those NetCDF3 files. I'll refer to them over
the next couple of weeks when I get a chance to work on this. During
this week and next we'll be preparing for/teaching the
MET tutorial
here. These files will serve as a good example of the CF-compliant
NetCDF files we'd ultimately like to support. I'll let you know if
any questions come up about them.
To answer your question, by
"classic data model", I was drawing a distinction between the older
NetCDF3 file format (classic data model) and the newer NetCDF4 file
format that's built on top of HDF5.
Thanks,
John
RAL HelpDesk
{for Lamey, Patrick - 0443 - MITLL} wrote:
> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>
>
Thanks for your help John. We have the capability of writing files
into NetCDF3 format. I have uploaded a few more examples to
"incoming/irap/met_help/lamey_data" on the ftp server. Could you
clarify what "NetCDF3 classic data model" refers to? Is that the data
format of the pcp_combine output?
> Thanks,
> Pat
>
>
-----Original Message-----
> From: RAL HelpDesk {for John Halley
Gotway} [mailto:met_help at ucar.edu]
> Sent: Thursday, July 29, 2010
2:16 PM
> To: Lamey, Patrick - 0443 - MITLL
> Subject: Re:
[rt.rap.ucar.edu #39089] MODE usage with netCDF
>
> Patrick,
>
>
Thanks for sending me those sample files. I see that they are in
NetCDF4 format following the CF-1.3 convention. Our goal is to work
toward supporting CF-compliant files. However, I can tell you
> that
we haven't made the switch over from NetCDF3 to NetCDF4 yet and
certainly won't be doing so prior to the upcoming release. So we will
not be able to read these files directly in METv3.0.
>
> I have
been in contact with James and Huaqing over the last couple years
about their use of MET, MODE in particular. Currently, they're
converting their MDV data into GRIB1 prior to running it
> through
MET. I do know they'd prefer to skip that conversion step and just
use NetCDF files directly. Unfortunately, we haven't had the time or
funding to add that capability.
>
> I'm not sure how you'd like to
proceed. If you have by chance have tools available for writing your
data in GRIB1 format, that would make using MET the most straight-
forward. Otherwise, you'd need
> to reformat your NetCDF data into
the NetCDF3 classic data model for use in MET.
>
> Sorry this is
such a hassle, but handling data format often is!
>
> Thanks,
>
John
>
> RAL HelpDesk {for Lamey, Patrick - 0443 - MITLL} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>
>> Hello,
>> I've placed examples of our NetCDF format files in the
directory "incoming/irap/met_help/lamey_data" on your ftp server. If
you need more examples please let me know.
>>
>> Please note that
the format of these Netcdf files is very similar to a format that has
been agreed upon between MITLL and NCAR for transferring data between
these organizations as part of the FAA CoSPA project. The main
difference is that MITLL places multiple forecast time horizons in one
file, whereas NCAR breaks the time horizons out into separate files.
The NCAR people that we routinely work with are members of the NCAR
RAL group, specifically, Matthias Steiner, James Pinto, John Williams,
David Ahijevych, Huaqing Cai to name a few. So modifications to MODE
to accommodate this format may have broad appeal.
>>
>> Patrick
Lamey
>> Group 43 Weather Sensing
>> MIT Lincoln Laboratory
>>
Phone: (781)-981-4790
>> Email: lamey at ll.mit.edu
>>
>>
>>
-----Original Message-----
>> From: RAL HelpDesk {for John Halley
Gotway} [mailto:met_help at ucar.edu]
>> Sent: Wednesday, July 28, 2010
6:38 PM
>> To: Lamey, Patrick - 0443 - MITLL
>> Subject:
[rt.rap.ucar.edu #39089] MODE usage with netCDF
>>
>> Patrick,
>>
>> We're currently finishing up development on METv3.0. We have a
couple more items to address, including our support for reading NetCDF
data files.
>>
>> We're updating METv3.0 to read the NetCDF output
of the WRF-ARW-pinterp post-processing tool. And we'd like to make
those changes as general as we can in hopes of supporting a wider
variety of NetCDF formats.
>>
>> I have examples of NetCDF files
from a couple of users right now, that we'll be using as we make these
updates. I'm wondering if you'd be willing to send me some of your
NetCDF files to see what would need to change to use them in MET.
>>
>> As far as modifying your NetCDF files for use in MET, it's a bit of
an awkward time right now. The way we encode the grid definition
information in the NetCDF header has changed for METv3.0. So if
possible, I'd suggest waiting for METv3.0 before working on this. I'd
hate to see you get it working with METv2.0 and then have to change it
for METv3.0.
>>
>> We don't have tools available for converting
NetCDF files for use in MET. I do know other MET users have used
ncgen with a CDL file. Really the best example is the NetCDF files
that are created by the pcp_combine test script in the MET tarball. I
do have an old write-up about the NetCDF format, but it's out of date
for METv3.0 - which we plan to release by the end of August.
>>
>>
If you'd be willing to send me some sample NetCDF files, you could
post them to our anonymous ftp site at:
>>
>> ftp ftp.rap.ucar.edu
>> username=anonymous
>> password="your email address"
>> cd
incoming/irap/met_help/lamey_data
>> put "your files"
>> bye
>>
>>
Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>> On Wed
Jul 28 13:42:27 2010, lamey at ll.mit.edu wrote:
>>> Hello, I inquired a
while back about using NetCDF files with the MODE
>>> module. I
know that the netCDF input file needs to be in the PCP-
>>>
Combine format. I was wondering if there is a routine to reformat
>>>
other netCDF files into a PCP-Combine format? If not is there any
>>>
documentation that you could provide on the format of the PCP-
>>>
Combine output netCDF files?
>>> Thanks,
>>> Patrick Lamey
>>>
Group 43
>>> Weather Sensing
>>> MIT LINCOLN LABORATORY
>>>
>>>
-----Original
>>> Message-----
>>> From: RAL HelpDesk {for John
Halley Gotway}
>>> [mailto:met_help at ucar.edu]
>>> Sent: Friday,
July 09, 2010 2:53 PM
>>> To: Lamey, Patrick - 0443 - MITLL
>>>
Subject: Re: [rt.rap.ucar.edu
>>> #39089] MODE usage with netCDF
>>>
>>> Patrick,
>>>
>>> Sorry for the delay
>>> in getting
back to you.
>>>
>>> As of METv2.0, the current release, the
>>>
only NetCDF files that you can use as input for MODE are those that
>>> look like the output of the PCP-Combine tool. For METv3.0, the
>>> upcoming release, we're
>>> working to enhance our support for
NetCDF
>>> data. In particular, we're trying to support the
output of the
>>> pinterp tool which may be used to post-process
WRF-ARW output.
>>> Ultimately, we'd like to
>>> support NetCDF
files that conform to the
>>> CF-convention.
>>>
>>> But for the
time being, your stuck with our very
>>> limited support for
NetCDF.
>>>
>>> Hope that helps clarify.
>>>
>>> Thanks,
>>>
John Halley Gotway
>>> met_help at ucar.edu
>>>
>>> RAL HelpDesk {for
Lamey,
>>> Patrick - 0443 - MITLL} wrote:
>>>> Wed Jul 07
13:16:25 2010: Request
>>> 39089 was acted upon.
>>>>
Transaction: Ticket created by
>>> lamey at ll.mit.edu
>>>>
Queue: met_help
>>>> Subject: MODE
>>> usage with netCDF
>>>> Owner: Nobody
>>>> Requestors:
>>> lamey at ll.mit.edu
>>>> Status: new
>>>> Ticket <URL:
>>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39089 >
>>> >
>>>> Hello, I work for the MIT Lincoln Laboratory's Weather
>>>
Sensing group and I had some interest in using your object
>>>
verification library.
>>>> I was wondering if the MET module's MODE
>>> utility is usable with netCDF input files other than those
coming
>>> out of the pcp combine tool? The documentation seems to
indicate
>>> that it is NOT and I just wanted some clarification
on the issue.
>>> If this is the case, is there any plans in the
near future for
>>> supporting other netCDF input data?
>>>>
Thanks
>>>>
>>>> Patrick Lamey
>>> > Group 43 Weather Sensing
>>>> MIT Lincoln Laboratory
>>>> Phone:
>>> (781)-981-4790
>>>>
Email: lamey at ll.mit.edu
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
------------------------------------------------------------------------
>>> >
>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list