[Met_help] [rt.rap.ucar.edu #40389] History for MODE NetCDF Input

Wed Sep 8 09:09:28 MDT 2010

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I'm currently trying to take CF compliant NetCDF files from model output
files from NARCCAP into MODE to due some not so standard analysis with MODE.
 It says it cannot find the NetCDF attribute I'm trying to give it (precip
and only 1 time step in the file).  I know the NetCDF that MODE accepts from
the PCP Combine tool is not CF compliant, but I was wondering if there is
some way to take CF compliant NetCDF and getting MODE to read it?

Thank you,
Casey Zoellick
czoellic at iastate.edu

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MODE NetCDF Input
From: John Halley Gotway
Time: Fri Aug 27 10:30:37 2010

Casey,

In order to get MODE to read that data, you'd need to make it look
very much like the output of PCP-Combine.  In particular, you'd need
to name the dimensions the same as the output of PCP-Combine.  The
NetCDF variable should have only two dimensions and should be named
using the corresponding GRIB code abbreviation.  For precip, that'd be
"APCP_HH" where HH is the accumulation interval for the precip.  The
variable would need to have the same attributes.  And the global
attributes would need to include parameters for defining the grid on
which the data resides.

Basically, it sounds like you need to write a script to reformat your
data.  Fortunately, there are many scripting languages that can
read/write NetCDF data, including NCL and R, which I've used before.

Sorry for the hassle required to reformat your data.  Unfortunately,
data formatting is always an issue.  For the upcoming release of MET,
METv3.0, we are expanding our support for NetCDF to include the NetCDF
output of pinterp post-processing tool for WRF-ARW.  But we will not
have full support for the CF-convention yet.

If you get stuck in your reformatting step, feel free to send us a
NetCDF file and we'd be happy to offer some suggestions.

Thanks,
John Halley Gotway
met_help at ucar.edu

------------------------------------------------
Subject: MODE NetCDF Input
From: Casey Zoellick
Time: Mon Aug 30 13:24:52 2010

I decided to shoot you guys an example of the NetCDF files I'm using.
Technically it's not an accumulation yet, but in the script that I was
going
to use that would change the metadata I was going to have it convert
the
data from a flux to accumulations.  Either way the metadata is the
same.

Thanks again,
Casey Zoellick

------------------------------------------------
Subject: MODE NetCDF Input
From: Casey Zoellick
Time: Thu Sep 02 22:24:08 2010

John,

I tried using a script I wrote to match the metadata for the PCP-
Combine
output, but it still can't seem to find the NetCDF attribute.  I
attached
the original file (the pr one) and a file my script made (the test
one) if
it's possible for you to make any suggestions like you mentioned?

Thanks again,
Casey Zoellick

On Fri, Aug 27, 2010 at 11:30 AM, RAL HelpDesk {for John Halley
Gotway} <
met_help at ucar.edu> wrote:

> Casey,
>
> In order to get MODE to read that data, you'd need to make it look
very
> much like the output of PCP-Combine.  In particular, you'd need to
name the
> dimensions the same as the output of PCP-Combine.  The NetCDF
variable
> should have only two dimensions and should be named using the
corresponding
> GRIB code abbreviation.  For precip, that'd be "APCP_HH" where HH is
the
> accumulation interval for the precip.  The variable would need to
have the
> same attributes.  And the global attributes would need to include
parameters
> for defining the grid on which the data resides.
>
> Basically, it sounds like you need to write a script to reformat
your data.
>  Fortunately, there are many scripting languages that can read/write
NetCDF
> data, including NCL and R, which I've used before.
>
> Sorry for the hassle required to reformat your data.  Unfortunately,
data
> formatting is always an issue.  For the upcoming release of MET,
METv3.0, we
> are expanding our support for NetCDF to include the NetCDF output of
pinterp
> post-processing tool for WRF-ARW.  But we will not have full support
for the
> CF-convention yet.
>
> If you get stuck in your reformatting step, feel free to send us a
NetCDF
> file and we'd be happy to offer some suggestions.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>

--
CASEY L. ZOELLICK, C/Maj, AFROTC
Mission Support Group Deputy Commander
Detachment 250
630-890-2911
czoellic at iastate.edu

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40389] MODE NetCDF Input
From: John Halley Gotway
Time: Fri Sep 03 09:10:18 2010

Casey,

Don't you wish those NetCDF error messages were more helpful than
simply "NetCDF: Attribute not found"!

I see a few problems in the test file you sent:
(1) You've defined the global attribute named "yx", when it should be
named "ny" - that's the reason for the error message you're seeing.
(2) In the variable attributes, you've written out init_time_ut,
valid_time_ut, and accum_time_sec as strings when they should really
be numbers.  Try writing them out in the same way that you're
writing out the "grib_code" variable attribute, as a number.
(3) The APCP_03 variable in the test.nc just contains all zeros, but
you probably already know that.

Hopefully that'll help.

Thanks,
John

RAL HelpDesk {for Casey Zoellick} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40389 >
>
> John,
>
> I tried using a script I wrote to match the metadata for the PCP-
Combine
> output, but it still can't seem to find the NetCDF attribute.  I
attached
> the original file (the pr one) and a file my script made (the test
one) if
> it's possible for you to make any suggestions like you mentioned?
>
> Thanks again,
> Casey Zoellick
>
>
> On Fri, Aug 27, 2010 at 11:30 AM, RAL HelpDesk {for John Halley
Gotway} <
> met_help at ucar.edu> wrote:
>
>> Casey,
>>
>> In order to get MODE to read that data, you'd need to make it look
very
>> much like the output of PCP-Combine.  In particular, you'd need to
name the
>> dimensions the same as the output of PCP-Combine.  The NetCDF
variable
>> should have only two dimensions and should be named using the
corresponding
>> GRIB code abbreviation.  For precip, that'd be "APCP_HH" where HH
is the
>> accumulation interval for the precip.  The variable would need to
have the
>> same attributes.  And the global attributes would need to include
parameters
>> for defining the grid on which the data resides.
>>
>> Basically, it sounds like you need to write a script to reformat
your data.
>>  Fortunately, there are many scripting languages that can
read/write NetCDF
>> data, including NCL and R, which I've used before.
>>
>> Sorry for the hassle required to reformat your data.
Unfortunately, data
>> formatting is always an issue.  For the upcoming release of MET,
METv3.0, we
>> are expanding our support for NetCDF to include the NetCDF output
of pinterp
>> post-processing tool for WRF-ARW.  But we will not have full
support for the
>> CF-convention yet.
>>
>> If you get stuck in your reformatting step, feel free to send us a
NetCDF
>> file and we'd be happy to offer some suggestions.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>
>
>

------------------------------------------------
Subject: MODE NetCDF Input
From: Casey Zoellick
Time: Tue Sep 07 20:59:54 2010

John,

Those issues seemed to have cleared up my problem!  I'm aware of the
fact
that my data is currently all zeros, but I'm sure that's a problem
with my
script that I need to debug yet.  MODE accepts the file, which is what
I was
more concerned about.

Thanks for all your help!
Casey

On Fri, Sep 3, 2010 at 10:10 AM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:

> Casey,
>
> Don't you wish those NetCDF error messages were more helpful than
simply
> "NetCDF: Attribute not found"!
>
> I see a few problems in the test file you sent:
> (1) You've defined the global attribute named "yx", when it should
be named
> "ny" - that's the reason for the error message you're seeing.
> (2) In the variable attributes, you've written out init_time_ut,
> valid_time_ut, and accum_time_sec as strings when they should really
be
> numbers.  Try writing them out in the same way that you're
> writing out the "grib_code" variable attribute, as a number.
> (3) The APCP_03 variable in the test.nc just contains all zeros, but
you
> probably already know that.
>
> Hopefully that'll help.
>
> Thanks,
> John
>
> RAL HelpDesk {for Casey Zoellick} wrote:
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40389 >
> >
> > John,
> >
> > I tried using a script I wrote to match the metadata for the PCP-
Combine
> > output, but it still can't seem to find the NetCDF attribute.  I
attached
> > the original file (the pr one) and a file my script made (the test
one)
> if
> > it's possible for you to make any suggestions like you mentioned?
> >
> > Thanks again,
> > Casey Zoellick
> >
> >
> > On Fri, Aug 27, 2010 at 11:30 AM, RAL HelpDesk {for John Halley
Gotway} <
> > met_help at ucar.edu> wrote:
> >
> >> Casey,
> >>
> >> In order to get MODE to read that data, you'd need to make it
look very
> >> much like the output of PCP-Combine.  In particular, you'd need
to name
> the
> >> dimensions the same as the output of PCP-Combine.  The NetCDF
variable
> >> should have only two dimensions and should be named using the
> corresponding
> >> GRIB code abbreviation.  For precip, that'd be "APCP_HH" where HH
is the
> >> accumulation interval for the precip.  The variable would need to
have
> the
> >> same attributes.  And the global attributes would need to include
> parameters
> >> for defining the grid on which the data resides.
> >>
> >> Basically, it sounds like you need to write a script to reformat
your
> data.
> >>  Fortunately, there are many scripting languages that can
read/write
> NetCDF
> >> data, including NCL and R, which I've used before.
> >>
> >> Sorry for the hassle required to reformat your data.
Unfortunately,
> data
> >> formatting is always an issue.  For the upcoming release of MET,
> METv3.0, we
> >> are expanding our support for NetCDF to include the NetCDF output
of
> pinterp
> >> post-processing tool for WRF-ARW.  But we will not have full
support for
> the
> >> CF-convention yet.
> >>
> >> If you get stuck in your reformatting step, feel free to send us
a
> NetCDF
> >> file and we'd be happy to offer some suggestions.
> >>
> >> Thanks,
> >> John Halley Gotway
> >> met_help at ucar.edu
> >>
> >
> >
> >
>
>

--
CASEY L. ZOELLICK, C/Maj, AFROTC
Mission Support Group Deputy Commander
Detachment 250
630-890-2911
czoellic at iastate.edu

------------------------------------------------