[Met_help] [rt.rap.ucar.edu #87632] History for Format of netcdf input to MTD
John Halley Gotway via RT
met_help at ucar.edu
Mon Nov 12 15:49:57 MST 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I am having trouble reformatting my netcdf files into something I can use
with the MODE-Time-Domain tool. I have netcdf files with daily 1 hourly
total precipitation rate (24 timesteps per file). I was able to get the
tool to run with two of my daily files (i.e.):
mtd -single day1_precip.nc day2_precip.nc -outdir /home/aepayne -config
> MTDConfig
but it looks like the tool is only using the first timestep of each file. I
tried breaking the files down into hourly files, but I get this error:
NcCfFile::open() -> could not determine the valid time, using 0.
Is there an existing tool that will help reformat this data into a format
that can be used with mtd? I've been working through the online tutorial
but I'm afraid I'm just confusing myself.
Thank you for your help.
Ashley
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Format of netcdf input to MTD
From: John Halley Gotway
Time: Fri Nov 02 15:34:51 2018
Hi Ashley,
I see you have a question about preparing data for the MODE time
domain
tool. I understand that you have a NetCDF file containing multiple
output
times but MTD is only processing the first timestamp in each file.
Unfortunately, you're stuck with a messy reformatting task.
I have also run into this problem in the past when applying mtd to
climate
simulations. We do have a development ticket defined to enhance mtd
to
read multiple timestamps from the same file. Unfortunately though
that
work isn't done yet. Also new in met-8.0 we've added support for
calling
python scripts to read data and pass it as input to the MET tools.
However, that logic currently only handles a single field of data...
and
therefore isn't yet useful for mtd. We would like to extend that
functionality to pass in a time series of fields... which would be
useful
for your case.
But unfortunately, with met-8.0 we're stuck doing reformatting.
Here's a trick I've used in the past...
The Grid-Stat tool in MET reads in a gridded forecast field and a
gridded
observation field. It compute statistics for one or more variables
over
one or more spatial regions. But it also has the ability to write the
data
it reads to an output NetCDF file. When processing NetCDF files with
multiple time steps, I wrote a shell script to loop over each
timestep.
And for each time, I ran Grid-Stat, computed some stats, and also
wrote a
NetCDF matched pairs output file. After looping over all the times, I
ran
mtd on those NetCDF matched pairs output files.
And that approach made my life easier.
So now the question is, are the MET tools able to read your input data
files at all?
Feel free to post a sample file to our anonymous ftp site following
these
instructions:
https://dtcenter.org/met/users/support/met_help.php#ftp
When I hear back from you, I'll go grab it and see if I have any
suggestions. Also, what version of MET are you using?
Thanks,
John Halley Gotway
On Fri, Nov 2, 2018 at 12:25 PM Ashley Payne via RT
<met_help at ucar.edu>
wrote:
>
> Fri Nov 02 12:25:14 2018: Request 87632 was acted upon.
> Transaction: Ticket created by aepayne at umich.edu
> Queue: met_help
> Subject: Format of netcdf input to MTD
> Owner: Nobody
> Requestors: aepayne at umich.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
>
>
> Hi,
>
> I am having trouble reformatting my netcdf files into something I
can use
> with the MODE-Time-Domain tool. I have netcdf files with daily 1
hourly
> total precipitation rate (24 timesteps per file). I was able to get
the
> tool to run with two of my daily files (i.e.):
>
> mtd -single day1_precip.nc day2_precip.nc -outdir /home/aepayne
-config
> > MTDConfig
>
>
> but it looks like the tool is only using the first timestep of each
file. I
> tried breaking the files down into hourly files, but I get this
error:
>
> NcCfFile::open() -> could not determine the valid time, using 0.
>
>
> Is there an existing tool that will help reformat this data into a
format
> that can be used with mtd? I've been working through the online
tutorial
> but I'm afraid I'm just confusing myself.
>
> Thank you for your help.
>
> Ashley
>
>
------------------------------------------------
Subject: Format of netcdf input to MTD
From: Ashley Payne
Time: Fri Nov 02 21:34:10 2018
Hi John,
Thank you for getting back to me so quickly!
To give you more context, I am trying to use MTD on regridded output
from a
variable resolution CESM simulation. I use a simple NCL script with
ESMF_regrid to convert from the original unstructured grid to regular
lat/lon. After adding in some of the required metadata, MTD does
recognize
the netcdf file itself, but with the issue I mentioned in my initial
email
(that the tool only used the first timestep of each 24 hour file).
My thought then was to simply modify the NCL script to produce hourly
files
rather than the combined daily file with 24 hours, but I ran into some
trouble with the MTD tool not recognizing time (NcCfFile::open() ->
could
not determine the valid time, using 0.) in the hourly files.
I've uploaded three of the daily files to the ftp site under
/incoming/irap/met_help/aepayne_data. The file names are a bit odd,
but
basically they are three consecutive 24 hour periods starting 2008-05-
22.
Please let me know if you need anything else.
Ashley
--
Assistant Professor
Department of Climate and Space Sciences and Engineering
CSRB 2517B | 512.636.9390
https://aepayne.me
On Fri, Nov 2, 2018 at 5:34 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Ashley,
>
> I see you have a question about preparing data for the MODE time
domain
> tool. I understand that you have a NetCDF file containing multiple
output
> times but MTD is only processing the first timestamp in each file.
> Unfortunately, you're stuck with a messy reformatting task.
>
> I have also run into this problem in the past when applying mtd to
climate
> simulations. We do have a development ticket defined to enhance mtd
to
> read multiple timestamps from the same file. Unfortunately though
that
> work isn't done yet. Also new in met-8.0 we've added support for
calling
> python scripts to read data and pass it as input to the MET tools.
> However, that logic currently only handles a single field of data...
and
> therefore isn't yet useful for mtd. We would like to extend that
> functionality to pass in a time series of fields... which would be
useful
> for your case.
>
> But unfortunately, with met-8.0 we're stuck doing reformatting.
>
> Here's a trick I've used in the past...
>
> The Grid-Stat tool in MET reads in a gridded forecast field and a
gridded
> observation field. It compute statistics for one or more variables
over
> one or more spatial regions. But it also has the ability to write
the data
> it reads to an output NetCDF file. When processing NetCDF files
with
> multiple time steps, I wrote a shell script to loop over each
timestep.
> And for each time, I ran Grid-Stat, computed some stats, and also
wrote a
> NetCDF matched pairs output file. After looping over all the times,
I ran
> mtd on those NetCDF matched pairs output files.
>
> And that approach made my life easier.
>
> So now the question is, are the MET tools able to read your input
data
> files at all?
>
> Feel free to post a sample file to our anonymous ftp site following
these
> instructions:
> https://dtcenter.org/met/users/support/met_help.php#ftp
>
> When I hear back from you, I'll go grab it and see if I have any
> suggestions. Also, what version of MET are you using?
>
> Thanks,
> John Halley Gotway
>
>
> On Fri, Nov 2, 2018 at 12:25 PM Ashley Payne via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Nov 02 12:25:14 2018: Request 87632 was acted upon.
> > Transaction: Ticket created by aepayne at umich.edu
> > Queue: met_help
> > Subject: Format of netcdf input to MTD
> > Owner: Nobody
> > Requestors: aepayne at umich.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
> >
> >
> > Hi,
> >
> > I am having trouble reformatting my netcdf files into something I
can use
> > with the MODE-Time-Domain tool. I have netcdf files with daily 1
hourly
> > total precipitation rate (24 timesteps per file). I was able to
get the
> > tool to run with two of my daily files (i.e.):
> >
> > mtd -single day1_precip.nc day2_precip.nc -outdir /home/aepayne
-config
> > > MTDConfig
> >
> >
> > but it looks like the tool is only using the first timestep of
each
> file. I
> > tried breaking the files down into hourly files, but I get this
error:
> >
> > NcCfFile::open() -> could not determine the valid time, using 0.
> >
> >
> > Is there an existing tool that will help reformat this data into a
format
> > that can be used with mtd? I've been working through the online
tutorial
> > but I'm afraid I'm just confusing myself.
> >
> > Thank you for your help.
> >
> > Ashley
> >
> >
>
>
------------------------------------------------
Subject: Format of netcdf input to MTD
From: John Halley Gotway
Time: Mon Nov 05 09:42:43 2018
Ashley,
OK, if you're able to modify the NCL script to dump hourly files, then
that
would be the easiest... just keep the file format exactly the same,
but
have the time dimension be length1 instead of 24.
When I run your sample file through MET's plot_data_plane tool, I
don't see
any warnings about the valid time. For example, the following command
plots the 11th (since it's 0-based) time step from the input file:
/usr/local/met-8.0/bin/plot_data_plane \
tmp_ne30_0508.PRC.2008-05-22-00000.2008-05-22-03600.reg0.125deg.nc
\
plot.ps \
'name="PRECCr"; level="(10,*,*)";' -v 4
And png version of the resulting image is attached. If you continue
to run
into that warning message about the valid time, please send me a
sample
hourly file which causes it.
Thanks,
John
On Fri, Nov 2, 2018 at 9:34 PM Ashley Payne via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
>
> Hi John,
>
> Thank you for getting back to me so quickly!
>
> To give you more context, I am trying to use MTD on regridded output
from a
> variable resolution CESM simulation. I use a simple NCL script with
> ESMF_regrid to convert from the original unstructured grid to
regular
> lat/lon. After adding in some of the required metadata, MTD does
recognize
> the netcdf file itself, but with the issue I mentioned in my initial
email
> (that the tool only used the first timestep of each 24 hour file).
>
> My thought then was to simply modify the NCL script to produce
hourly files
> rather than the combined daily file with 24 hours, but I ran into
some
> trouble with the MTD tool not recognizing time (NcCfFile::open() ->
could
> not determine the valid time, using 0.) in the hourly files.
>
> I've uploaded three of the daily files to the ftp site under
> /incoming/irap/met_help/aepayne_data. The file names are a bit odd,
but
> basically they are three consecutive 24 hour periods starting 2008-
05-22.
>
> Please let me know if you need anything else.
>
> Ashley
> --
> Assistant Professor
> Department of Climate and Space Sciences and Engineering
> CSRB 2517B | 512.636.9390
> https://aepayne.me
>
>
> On Fri, Nov 2, 2018 at 5:34 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Ashley,
> >
> > I see you have a question about preparing data for the MODE time
domain
> > tool. I understand that you have a NetCDF file containing
multiple
> output
> > times but MTD is only processing the first timestamp in each file.
> > Unfortunately, you're stuck with a messy reformatting task.
> >
> > I have also run into this problem in the past when applying mtd to
> climate
> > simulations. We do have a development ticket defined to enhance
mtd to
> > read multiple timestamps from the same file. Unfortunately though
that
> > work isn't done yet. Also new in met-8.0 we've added support for
calling
> > python scripts to read data and pass it as input to the MET tools.
> > However, that logic currently only handles a single field of
data... and
> > therefore isn't yet useful for mtd. We would like to extend that
> > functionality to pass in a time series of fields... which would be
useful
> > for your case.
> >
> > But unfortunately, with met-8.0 we're stuck doing reformatting.
> >
> > Here's a trick I've used in the past...
> >
> > The Grid-Stat tool in MET reads in a gridded forecast field and a
gridded
> > observation field. It compute statistics for one or more
variables over
> > one or more spatial regions. But it also has the ability to write
the
> data
> > it reads to an output NetCDF file. When processing NetCDF files
with
> > multiple time steps, I wrote a shell script to loop over each
timestep.
> > And for each time, I ran Grid-Stat, computed some stats, and also
wrote a
> > NetCDF matched pairs output file. After looping over all the
times, I
> ran
> > mtd on those NetCDF matched pairs output files.
> >
> > And that approach made my life easier.
> >
> > So now the question is, are the MET tools able to read your input
data
> > files at all?
> >
> > Feel free to post a sample file to our anonymous ftp site
following these
> > instructions:
> > https://dtcenter.org/met/users/support/met_help.php#ftp
> >
> > When I hear back from you, I'll go grab it and see if I have any
> > suggestions. Also, what version of MET are you using?
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Fri, Nov 2, 2018 at 12:25 PM Ashley Payne via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri Nov 02 12:25:14 2018: Request 87632 was acted upon.
> > > Transaction: Ticket created by aepayne at umich.edu
> > > Queue: met_help
> > > Subject: Format of netcdf input to MTD
> > > Owner: Nobody
> > > Requestors: aepayne at umich.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632
> >
> > >
> > >
> > > Hi,
> > >
> > > I am having trouble reformatting my netcdf files into something
I can
> use
> > > with the MODE-Time-Domain tool. I have netcdf files with daily 1
hourly
> > > total precipitation rate (24 timesteps per file). I was able to
get the
> > > tool to run with two of my daily files (i.e.):
> > >
> > > mtd -single day1_precip.nc day2_precip.nc -outdir /home/aepayne
> -config
> > > > MTDConfig
> > >
> > >
> > > but it looks like the tool is only using the first timestep of
each
> > file. I
> > > tried breaking the files down into hourly files, but I get this
error:
> > >
> > > NcCfFile::open() -> could not determine the valid time, using 0.
> > >
> > >
> > > Is there an existing tool that will help reformat this data into
a
> format
> > > that can be used with mtd? I've been working through the online
> tutorial
> > > but I'm afraid I'm just confusing myself.
> > >
> > > Thank you for your help.
> > >
> > > Ashley
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Format of netcdf input to MTD
From: Ashley Payne
Time: Mon Nov 12 12:59:23 2018
Hi John,
Sorry for the delay in responding to your email. I was able to adapt
my NCL
script to produce hourly files with the needed attributes for MTD. It
looks
like everything is working now.
Thank you for your help!
Ashley
--
Assistant Professor
Department of Climate and Space Sciences and Engineering
CSRB 2517B | 512.636.9390
https://aepayne.me
On Mon, Nov 5, 2018 at 11:42 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ashley,
>
> OK, if you're able to modify the NCL script to dump hourly files,
then that
> would be the easiest... just keep the file format exactly the same,
but
> have the time dimension be length1 instead of 24.
>
> When I run your sample file through MET's plot_data_plane tool, I
don't see
> any warnings about the valid time. For example, the following
command
> plots the 11th (since it's 0-based) time step from the input file:
>
> /usr/local/met-8.0/bin/plot_data_plane \
> tmp_ne30_0508.PRC.2008-05-22-00000.2008-05-22-
03600.reg0.125deg.nc \
> plot.ps \
> 'name="PRECCr"; level="(10,*,*)";' -v 4
>
> And png version of the resulting image is attached. If you continue
to run
> into that warning message about the valid time, please send me a
sample
> hourly file which causes it.
>
> Thanks,
> John
>
> On Fri, Nov 2, 2018 at 9:34 PM Ashley Payne via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
> >
> > Hi John,
> >
> > Thank you for getting back to me so quickly!
> >
> > To give you more context, I am trying to use MTD on regridded
output
> from a
> > variable resolution CESM simulation. I use a simple NCL script
with
> > ESMF_regrid to convert from the original unstructured grid to
regular
> > lat/lon. After adding in some of the required metadata, MTD does
> recognize
> > the netcdf file itself, but with the issue I mentioned in my
initial
> email
> > (that the tool only used the first timestep of each 24 hour file).
> >
> > My thought then was to simply modify the NCL script to produce
hourly
> files
> > rather than the combined daily file with 24 hours, but I ran into
some
> > trouble with the MTD tool not recognizing time (NcCfFile::open()
-> could
> > not determine the valid time, using 0.) in the hourly files.
> >
> > I've uploaded three of the daily files to the ftp site under
> > /incoming/irap/met_help/aepayne_data. The file names are a bit
odd, but
> > basically they are three consecutive 24 hour periods starting
2008-05-22.
> >
> > Please let me know if you need anything else.
> >
> > Ashley
> > --
> > Assistant Professor
> > Department of Climate and Space Sciences and Engineering
> > CSRB 2517B | 512.636.9390
> > https://aepayne.me
> >
> >
> > On Fri, Nov 2, 2018 at 5:34 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Ashley,
> > >
> > > I see you have a question about preparing data for the MODE time
domain
> > > tool. I understand that you have a NetCDF file containing
multiple
> > output
> > > times but MTD is only processing the first timestamp in each
file.
> > > Unfortunately, you're stuck with a messy reformatting task.
> > >
> > > I have also run into this problem in the past when applying mtd
to
> > climate
> > > simulations. We do have a development ticket defined to enhance
mtd to
> > > read multiple timestamps from the same file. Unfortunately
though that
> > > work isn't done yet. Also new in met-8.0 we've added support
for
> calling
> > > python scripts to read data and pass it as input to the MET
tools.
> > > However, that logic currently only handles a single field of
data...
> and
> > > therefore isn't yet useful for mtd. We would like to extend
that
> > > functionality to pass in a time series of fields... which would
be
> useful
> > > for your case.
> > >
> > > But unfortunately, with met-8.0 we're stuck doing reformatting.
> > >
> > > Here's a trick I've used in the past...
> > >
> > > The Grid-Stat tool in MET reads in a gridded forecast field and
a
> gridded
> > > observation field. It compute statistics for one or more
variables
> over
> > > one or more spatial regions. But it also has the ability to
write the
> > data
> > > it reads to an output NetCDF file. When processing NetCDF files
with
> > > multiple time steps, I wrote a shell script to loop over each
timestep.
> > > And for each time, I ran Grid-Stat, computed some stats, and
also
> wrote a
> > > NetCDF matched pairs output file. After looping over all the
times, I
> > ran
> > > mtd on those NetCDF matched pairs output files.
> > >
> > > And that approach made my life easier.
> > >
> > > So now the question is, are the MET tools able to read your
input data
> > > files at all?
> > >
> > > Feel free to post a sample file to our anonymous ftp site
following
> these
> > > instructions:
> > > https://dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > When I hear back from you, I'll go grab it and see if I have any
> > > suggestions. Also, what version of MET are you using?
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > >
> > > On Fri, Nov 2, 2018 at 12:25 PM Ashley Payne via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > >
> > > > Fri Nov 02 12:25:14 2018: Request 87632 was acted upon.
> > > > Transaction: Ticket created by aepayne at umich.edu
> > > > Queue: met_help
> > > > Subject: Format of netcdf input to MTD
> > > > Owner: Nobody
> > > > Requestors: aepayne at umich.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I am having trouble reformatting my netcdf files into
something I can
> > use
> > > > with the MODE-Time-Domain tool. I have netcdf files with daily
1
> hourly
> > > > total precipitation rate (24 timesteps per file). I was able
to get
> the
> > > > tool to run with two of my daily files (i.e.):
> > > >
> > > > mtd -single day1_precip.nc day2_precip.nc -outdir
/home/aepayne
> > -config
> > > > > MTDConfig
> > > >
> > > >
> > > > but it looks like the tool is only using the first timestep of
each
> > > file. I
> > > > tried breaking the files down into hourly files, but I get
this
> error:
> > > >
> > > > NcCfFile::open() -> could not determine the valid time, using
0.
> > > >
> > > >
> > > > Is there an existing tool that will help reformat this data
into a
> > format
> > > > that can be used with mtd? I've been working through the
online
> > tutorial
> > > > but I'm afraid I'm just confusing myself.
> > > >
> > > > Thank you for your help.
> > > >
> > > > Ashley
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Format of netcdf input to MTD
From: John Halley Gotway
Time: Mon Nov 12 15:49:36 2018
Ashley,
Glad to hear it. Thanks for confirming. I'll resolve this ticket
now.
John
On Mon, Nov 12, 2018 at 12:59 PM Ashley Payne via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
>
> Hi John,
>
> Sorry for the delay in responding to your email. I was able to adapt
my NCL
> script to produce hourly files with the needed attributes for MTD.
It looks
> like everything is working now.
>
> Thank you for your help!
>
> Ashley
> --
> Assistant Professor
> Department of Climate and Space Sciences and Engineering
> CSRB 2517B | 512.636.9390
> https://aepayne.me
>
>
> On Mon, Nov 5, 2018 at 11:42 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ashley,
> >
> > OK, if you're able to modify the NCL script to dump hourly files,
then
> that
> > would be the easiest... just keep the file format exactly the
same, but
> > have the time dimension be length1 instead of 24.
> >
> > When I run your sample file through MET's plot_data_plane tool, I
don't
> see
> > any warnings about the valid time. For example, the following
command
> > plots the 11th (since it's 0-based) time step from the input file:
> >
> > /usr/local/met-8.0/bin/plot_data_plane \
> > tmp_ne30_0508.PRC.2008-05-22-00000.2008-05-22-
03600.reg0.125deg.nc \
> > plot.ps \
> > 'name="PRECCr"; level="(10,*,*)";' -v 4
> >
> > And png version of the resulting image is attached. If you
continue to
> run
> > into that warning message about the valid time, please send me a
sample
> > hourly file which causes it.
> >
> > Thanks,
> > John
> >
> > On Fri, Nov 2, 2018 at 9:34 PM Ashley Payne via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632 >
> > >
> > > Hi John,
> > >
> > > Thank you for getting back to me so quickly!
> > >
> > > To give you more context, I am trying to use MTD on regridded
output
> > from a
> > > variable resolution CESM simulation. I use a simple NCL script
with
> > > ESMF_regrid to convert from the original unstructured grid to
regular
> > > lat/lon. After adding in some of the required metadata, MTD does
> > recognize
> > > the netcdf file itself, but with the issue I mentioned in my
initial
> > email
> > > (that the tool only used the first timestep of each 24 hour
file).
> > >
> > > My thought then was to simply modify the NCL script to produce
hourly
> > files
> > > rather than the combined daily file with 24 hours, but I ran
into some
> > > trouble with the MTD tool not recognizing time (NcCfFile::open()
->
> could
> > > not determine the valid time, using 0.) in the hourly files.
> > >
> > > I've uploaded three of the daily files to the ftp site under
> > > /incoming/irap/met_help/aepayne_data. The file names are a bit
odd, but
> > > basically they are three consecutive 24 hour periods starting
> 2008-05-22.
> > >
> > > Please let me know if you need anything else.
> > >
> > > Ashley
> > > --
> > > Assistant Professor
> > > Department of Climate and Space Sciences and Engineering
> > > CSRB 2517B | 512.636.9390
> > > https://aepayne.me
> > >
> > >
> > > On Fri, Nov 2, 2018 at 5:34 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Ashley,
> > > >
> > > > I see you have a question about preparing data for the MODE
time
> domain
> > > > tool. I understand that you have a NetCDF file containing
multiple
> > > output
> > > > times but MTD is only processing the first timestamp in each
file.
> > > > Unfortunately, you're stuck with a messy reformatting task.
> > > >
> > > > I have also run into this problem in the past when applying
mtd to
> > > climate
> > > > simulations. We do have a development ticket defined to
enhance mtd
> to
> > > > read multiple timestamps from the same file. Unfortunately
though
> that
> > > > work isn't done yet. Also new in met-8.0 we've added support
for
> > calling
> > > > python scripts to read data and pass it as input to the MET
tools.
> > > > However, that logic currently only handles a single field of
data...
> > and
> > > > therefore isn't yet useful for mtd. We would like to extend
that
> > > > functionality to pass in a time series of fields... which
would be
> > useful
> > > > for your case.
> > > >
> > > > But unfortunately, with met-8.0 we're stuck doing
reformatting.
> > > >
> > > > Here's a trick I've used in the past...
> > > >
> > > > The Grid-Stat tool in MET reads in a gridded forecast field
and a
> > gridded
> > > > observation field. It compute statistics for one or more
variables
> > over
> > > > one or more spatial regions. But it also has the ability to
write
> the
> > > data
> > > > it reads to an output NetCDF file. When processing NetCDF
files with
> > > > multiple time steps, I wrote a shell script to loop over each
> timestep.
> > > > And for each time, I ran Grid-Stat, computed some stats, and
also
> > wrote a
> > > > NetCDF matched pairs output file. After looping over all the
times,
> I
> > > ran
> > > > mtd on those NetCDF matched pairs output files.
> > > >
> > > > And that approach made my life easier.
> > > >
> > > > So now the question is, are the MET tools able to read your
input
> data
> > > > files at all?
> > > >
> > > > Feel free to post a sample file to our anonymous ftp site
following
> > these
> > > > instructions:
> > > > https://dtcenter.org/met/users/support/met_help.php#ftp
> > > >
> > > > When I hear back from you, I'll go grab it and see if I have
any
> > > > suggestions. Also, what version of MET are you using?
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > >
> > > > On Fri, Nov 2, 2018 at 12:25 PM Ashley Payne via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > Fri Nov 02 12:25:14 2018: Request 87632 was acted upon.
> > > > > Transaction: Ticket created by aepayne at umich.edu
> > > > > Queue: met_help
> > > > > Subject: Format of netcdf input to MTD
> > > > > Owner: Nobody
> > > > > Requestors: aepayne at umich.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87632
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I am having trouble reformatting my netcdf files into
something I
> can
> > > use
> > > > > with the MODE-Time-Domain tool. I have netcdf files with
daily 1
> > hourly
> > > > > total precipitation rate (24 timesteps per file). I was able
to get
> > the
> > > > > tool to run with two of my daily files (i.e.):
> > > > >
> > > > > mtd -single day1_precip.nc day2_precip.nc -outdir
/home/aepayne
> > > -config
> > > > > > MTDConfig
> > > > >
> > > > >
> > > > > but it looks like the tool is only using the first timestep
of each
> > > > file. I
> > > > > tried breaking the files down into hourly files, but I get
this
> > error:
> > > > >
> > > > > NcCfFile::open() -> could not determine the valid time,
using 0.
> > > > >
> > > > >
> > > > > Is there an existing tool that will help reformat this data
into a
> > > format
> > > > > that can be used with mtd? I've been working through the
online
> > > tutorial
> > > > > but I'm afraid I'm just confusing myself.
> > > > >
> > > > > Thank you for your help.
> > > > >
> > > > > Ashley
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list