[Met_help] [rt.rap.ucar.edu #95850] History for GridStat Question
George McCabe via RT
met_help at ucar.edu
Tue Jul 28 08:19:55 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
My start date for GridStat is e.g. 2016060118 for FCSTs and OBSs. FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and OBSs at 3-hr intervals. For this time GridStat.conf is configured to take 3rd FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th OBS in the other input file: (6,1,*,*) That only works properly for FCSTs/OBSs starting at 18z with intervals of 24 hours when running over e.g. a month-long period. However, if I wanted to run GridStat to get statistics with data at intervals of 6 hours e.g. SDATE=2016060118 and EDATE=2016063118 that is incorrect since for 00Z (3,1,*,*) needs to become (0,1,*,*) etc. Is there a way to pass times of FCSTs/OBSs to GridStat.conf when running master_metplus.py ?
Thanks again,
Mariusz
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: GridStat Question
From: Julie Prestopnik
Time: Wed Jul 08 10:47:03 2020
Hi Mariusz.
I moved your question over to met_help so that others could benefit
from your questions.
I asked a co-worker for some help with this one and he said that if
the data is in NetCDF and in a format that MET can read, you can
specify the time to use in the level, i.e. (YYYYMMDD_HHMMSS,*,*).
There is a METplus use case that does this here:
parm/use_cases/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf
FCST_GRID_STAT_VAR1_NAME = pr
FCST_GRID_STAT_VAR1_LEVELS = "({valid?fmt=%Y%m01_000000},*,*)"
He said that likely won't work with this data, but there is likely a
clever way to use the valid time to create the appropriate level value
for each run time in a similar way. If you aren't able to figure it
out, you could provide a sample dataset and he could help figure out
how to configure it.
Thanks,
Julie
On Wed Jul 08 10:39:59 2020, jpresto wrote:
> My start date for GridStat is e.g. 2016060118 for FCSTs and OBSs.
> FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and OBSs at
3-
> hr intervals. For this time GridStat.conf is configured to take 3rd
> FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th OBS in
> the other input file: (6,1,*,*) That only works properly for
> FCSTs/OBSs starting at 18z with intervals of 24 hours when running
> over e.g. a month-long period. However, if I wanted to run GridStat
to
> get statistics with data at intervals of 6 hours e.g.
SDATE=2016060118
> and EDATE=2016063118 that is incorrect since for 00Z (3,1,*,*) needs
> to become (0,1,*,*) etc. Is there a way to pass times of FCSTs/OBSs
to
> GridStat.conf when running master_metplus.py ?
>
> Thanks again,
> Mariusz
------------------------------------------------
Subject: GridStat Question
From: Mariusz Pagowski
Time: Thu Jul 23 19:06:34 2020
Julie,
It was hanging over my head but only today I tried to input forecast
time
to the conf and did not figure the solution.
valid?fmt string does not produce for me a correct time and GridStat
fails.
If your colleague looked into our problem
we would very much appreciate it, thanks,
Mariusz
models for GridStat processing
/scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/cams/pll_new
/scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/merra-2/pll
/scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/fv3/pll
other stuff:
/home/Mariusz.Pagowski/MAPP_2018/scripts/met/met_grid_stat.sh
/scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/metstats/logs
/scratch1/BMC/gsd-fv3-dev/MAPP_2018/pagowski/tmpdir/workdir_gridstat
On Wed, Jul 8, 2020 at 10:47 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi Mariusz.
>
> I moved your question over to met_help so that others could benefit
from
> your questions.
>
> I asked a co-worker for some help with this one and he said that if
the
> data is in NetCDF and in a format that MET can read, you can specify
the
> time to use in the level, i.e. (YYYYMMDD_HHMMSS,*,*). There is a
METplus
> use case that does this here:
>
>
>
parm/use_cases/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf
>
> FCST_GRID_STAT_VAR1_NAME = pr
> FCST_GRID_STAT_VAR1_LEVELS = "({valid?fmt=%Y%m01_000000},*,*)"
>
> He said that likely won't work with this data, but there is likely a
> clever way to use the valid time to create the appropriate level
value for
> each run time in a similar way. If you aren't able to figure it out,
you
> could provide a sample dataset and he could help figure out how to
> configure it.
>
> Thanks,
> Julie
>
> On Wed Jul 08 10:39:59 2020, jpresto wrote:
> > My start date for GridStat is e.g. 2016060118 for FCSTs and OBSs.
> > FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and OBSs at
3-
> > hr intervals. For this time GridStat.conf is configured to take
3rd
> > FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th OBS
in
> > the other input file: (6,1,*,*) That only works properly for
> > FCSTs/OBSs starting at 18z with intervals of 24 hours when running
> > over e.g. a month-long period. However, if I wanted to run
GridStat to
> > get statistics with data at intervals of 6 hours e.g.
SDATE=2016060118
> > and EDATE=2016063118 that is incorrect since for 00Z (3,1,*,*)
needs
> > to become (0,1,*,*) etc. Is there a way to pass times of
FCSTs/OBSs to
> > GridStat.conf when running master_metplus.py ?
> >
> > Thanks again,
> > Mariusz
>
>
>
>
------------------------------------------------
Subject: GridStat Question
From: George McCabe
Time: Mon Jul 27 11:13:14 2020
Hi Mariusz,
To specify the time in the NetCDF structure, you need to use the
format
YYYYMMDD_HHMMSS, so your FTIME variable should look like this:
{valid?fmt=%Y%m%d_%H%M%S}
If the NetCDF time info is formatted in the way the MET tools expect,
the
code is smart enough to find the correct data from the time info.
If that doesn't work, you may need to do something clever to
manipulate the
hour value to the corresponding number of the NetCDF dimension. It
looks
like your data has a time dimension of size 4, so passing in the hour
(18)
is outside of the valid range. I would recommend trying to format the
valid
time with YYYYMMDD_HHMMSS. If that doesn't work, let me know and I try
to
help you figure out a way to get the correct number for the time you
need.
Thanks,
George
On Thu, Jul 23, 2020 at 7:06 PM Mariusz Pagowski via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95850 >
>
> Julie,
> It was hanging over my head but only today I tried to input forecast
time
> to the conf and did not figure the solution.
> valid?fmt string does not produce for me a correct time and GridStat
fails.
> If your colleague looked into our problem
> we would very much appreciate it, thanks,
> Mariusz
>
> models for GridStat processing
> /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/cams/pll_new
> /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/merra-2/pll
> /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/fv3/pll
>
> other stuff:
> /home/Mariusz.Pagowski/MAPP_2018/scripts/met/met_grid_stat.sh
> /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/metstats/logs
> /scratch1/BMC/gsd-fv3-dev/MAPP_2018/pagowski/tmpdir/workdir_gridstat
>
> On Wed, Jul 8, 2020 at 10:47 AM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> > Hi Mariusz.
> >
> > I moved your question over to met_help so that others could
benefit from
> > your questions.
> >
> > I asked a co-worker for some help with this one and he said that
if the
> > data is in NetCDF and in a format that MET can read, you can
specify the
> > time to use in the level, i.e. (YYYYMMDD_HHMMSS,*,*). There is a
METplus
> > use case that does this here:
> >
> >
> >
>
parm/use_cases/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf
> >
> > FCST_GRID_STAT_VAR1_NAME = pr
> > FCST_GRID_STAT_VAR1_LEVELS = "({valid?fmt=%Y%m01_000000},*,*)"
> >
> > He said that likely won't work with this data, but there is likely
a
> > clever way to use the valid time to create the appropriate level
value
> for
> > each run time in a similar way. If you aren't able to figure it
out, you
> > could provide a sample dataset and he could help figure out how to
> > configure it.
> >
> > Thanks,
> > Julie
> >
> > On Wed Jul 08 10:39:59 2020, jpresto wrote:
> > > My start date for GridStat is e.g. 2016060118 for FCSTs and
OBSs.
> > > FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and OBSs
at 3-
> > > hr intervals. For this time GridStat.conf is configured to take
3rd
> > > FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th OBS
in
> > > the other input file: (6,1,*,*) That only works properly for
> > > FCSTs/OBSs starting at 18z with intervals of 24 hours when
running
> > > over e.g. a month-long period. However, if I wanted to run
GridStat to
> > > get statistics with data at intervals of 6 hours e.g.
SDATE=2016060118
> > > and EDATE=2016063118 that is incorrect since for 00Z (3,1,*,*)
needs
> > > to become (0,1,*,*) etc. Is there a way to pass times of
FCSTs/OBSs to
> > > GridStat.conf when running master_metplus.py ?
> > >
> > > Thanks again,
> > > Mariusz
> >
> >
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: GridStat Question
From: Mariusz Pagowski
Time: Mon Jul 27 22:14:31 2020
Thanks, George, works as promised,
Mariusz
On Mon, Jul 27, 2020 at 11:13 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Mariusz,
>
> To specify the time in the NetCDF structure, you need to use the
format
> YYYYMMDD_HHMMSS, so your FTIME variable should look like this:
>
> {valid?fmt=%Y%m%d_%H%M%S}
>
> If the NetCDF time info is formatted in the way the MET tools
expect, the
> code is smart enough to find the correct data from the time info.
>
> If that doesn't work, you may need to do something clever to
manipulate the
> hour value to the corresponding number of the NetCDF dimension. It
looks
> like your data has a time dimension of size 4, so passing in the
hour (18)
> is outside of the valid range. I would recommend trying to format
the valid
> time with YYYYMMDD_HHMMSS. If that doesn't work, let me know and I
try to
> help you figure out a way to get the correct number for the time you
need.
>
> Thanks,
> George
>
> On Thu, Jul 23, 2020 at 7:06 PM Mariusz Pagowski via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95850 >
> >
> > Julie,
> > It was hanging over my head but only today I tried to input
forecast time
> > to the conf and did not figure the solution.
> > valid?fmt string does not produce for me a correct time and
GridStat
> fails.
> > If your colleague looked into our problem
> > we would very much appreciate it, thanks,
> > Mariusz
> >
> > models for GridStat processing
> > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/cams/pll_new
> > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/merra-2/pll
> > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/fv3/pll
> >
> > other stuff:
> > /home/Mariusz.Pagowski/MAPP_2018/scripts/met/met_grid_stat.sh
> > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/metstats/logs
> > /scratch1/BMC/gsd-fv3-
dev/MAPP_2018/pagowski/tmpdir/workdir_gridstat
> >
> > On Wed, Jul 8, 2020 at 10:47 AM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > > Hi Mariusz.
> > >
> > > I moved your question over to met_help so that others could
benefit
> from
> > > your questions.
> > >
> > > I asked a co-worker for some help with this one and he said that
if the
> > > data is in NetCDF and in a format that MET can read, you can
specify
> the
> > > time to use in the level, i.e. (YYYYMMDD_HHMMSS,*,*). There is a
> METplus
> > > use case that does this here:
> > >
> > >
> > >
> >
>
parm/use_cases/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf
> > >
> > > FCST_GRID_STAT_VAR1_NAME = pr
> > > FCST_GRID_STAT_VAR1_LEVELS = "({valid?fmt=%Y%m01_000000},*,*)"
> > >
> > > He said that likely won't work with this data, but there is
likely a
> > > clever way to use the valid time to create the appropriate level
value
> > for
> > > each run time in a similar way. If you aren't able to figure it
out,
> you
> > > could provide a sample dataset and he could help figure out how
to
> > > configure it.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Wed Jul 08 10:39:59 2020, jpresto wrote:
> > > > My start date for GridStat is e.g. 2016060118 for FCSTs and
OBSs.
> > > > FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and
OBSs at 3-
> > > > hr intervals. For this time GridStat.conf is configured to
take 3rd
> > > > FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th
OBS in
> > > > the other input file: (6,1,*,*) That only works properly for
> > > > FCSTs/OBSs starting at 18z with intervals of 24 hours when
running
> > > > over e.g. a month-long period. However, if I wanted to run
GridStat
> to
> > > > get statistics with data at intervals of 6 hours e.g.
> SDATE=2016060118
> > > > and EDATE=2016063118 that is incorrect since for 00Z (3,1,*,*)
needs
> > > > to become (0,1,*,*) etc. Is there a way to pass times of
FCSTs/OBSs
> to
> > > > GridStat.conf when running master_metplus.py ?
> > > >
> > > > Thanks again,
> > > > Mariusz
> > >
> > >
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: GridStat Question
From: George McCabe
Time: Tue Jul 28 08:19:32 2020
Good to hear! I am going to close this ticket, but feel free to open a
new
ticket via met_help at ucar.edu if you have any other issues.
Thanks,
George
On Mon, Jul 27, 2020 at 10:15 PM Mariusz Pagowski via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95850 >
>
> Thanks, George, works as promised,
> Mariusz
>
> On Mon, Jul 27, 2020 at 11:13 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Mariusz,
> >
> > To specify the time in the NetCDF structure, you need to use the
format
> > YYYYMMDD_HHMMSS, so your FTIME variable should look like this:
> >
> > {valid?fmt=%Y%m%d_%H%M%S}
> >
> > If the NetCDF time info is formatted in the way the MET tools
expect, the
> > code is smart enough to find the correct data from the time info.
> >
> > If that doesn't work, you may need to do something clever to
manipulate
> the
> > hour value to the corresponding number of the NetCDF dimension. It
looks
> > like your data has a time dimension of size 4, so passing in the
hour
> (18)
> > is outside of the valid range. I would recommend trying to format
the
> valid
> > time with YYYYMMDD_HHMMSS. If that doesn't work, let me know and I
try to
> > help you figure out a way to get the correct number for the time
you
> need.
> >
> > Thanks,
> > George
> >
> > On Thu, Jul 23, 2020 at 7:06 PM Mariusz Pagowski via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95850 >
> > >
> > > Julie,
> > > It was hanging over my head but only today I tried to input
forecast
> time
> > > to the conf and did not figure the solution.
> > > valid?fmt string does not produce for me a correct time and
GridStat
> > fails.
> > > If your colleague looked into our problem
> > > we would very much appreciate it, thanks,
> > > Mariusz
> > >
> > > models for GridStat processing
> > > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/cams/pll_new
> > > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/merra-2/pll
> > > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/fv3/pll
> > >
> > > other stuff:
> > > /home/Mariusz.Pagowski/MAPP_2018/scripts/met/met_grid_stat.sh
> > > /scratch1/BMC/wrf-chem/pagowski/MAPP_2018/MODEL/metstats/logs
> > > /scratch1/BMC/gsd-fv3-
dev/MAPP_2018/pagowski/tmpdir/workdir_gridstat
> > >
> > > On Wed, Jul 8, 2020 at 10:47 AM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > > Hi Mariusz.
> > > >
> > > > I moved your question over to met_help so that others could
benefit
> > from
> > > > your questions.
> > > >
> > > > I asked a co-worker for some help with this one and he said
that if
> the
> > > > data is in NetCDF and in a format that MET can read, you can
specify
> > the
> > > > time to use in the level, i.e. (YYYYMMDD_HHMMSS,*,*). There is
a
> > METplus
> > > > use case that does this here:
> > > >
> > > >
> > > >
> > >
> >
>
parm/use_cases/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf
> > > >
> > > > FCST_GRID_STAT_VAR1_NAME = pr
> > > > FCST_GRID_STAT_VAR1_LEVELS = "({valid?fmt=%Y%m01_000000},*,*)"
> > > >
> > > > He said that likely won't work with this data, but there is
likely a
> > > > clever way to use the valid time to create the appropriate
level
> value
> > > for
> > > > each run time in a similar way. If you aren't able to figure
it out,
> > you
> > > > could provide a sample dataset and he could help figure out
how to
> > > > configure it.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Wed Jul 08 10:39:59 2020, jpresto wrote:
> > > > > My start date for GridStat is e.g. 2016060118 for FCSTs and
OBSs.
> > > > > FCSTs are output at 6-hr intervals (00Z, 06Z,12Z, 18Z) and
OBSs at
> 3-
> > > > > hr intervals. For this time GridStat.conf is configured to
take 3rd
> > > > > FCST in the input file [in GridStat.conf: (3,1,*,*) and 6th
OBS in
> > > > > the other input file: (6,1,*,*) That only works properly for
> > > > > FCSTs/OBSs starting at 18z with intervals of 24 hours when
running
> > > > > over e.g. a month-long period. However, if I wanted to run
GridStat
> > to
> > > > > get statistics with data at intervals of 6 hours e.g.
> > SDATE=2016060118
> > > > > and EDATE=2016063118 that is incorrect since for 00Z
(3,1,*,*)
> needs
> > > > > to become (0,1,*,*) etc. Is there a way to pass times of
FCSTs/OBSs
> > to
> > > > > GridStat.conf when running master_metplus.py ?
> > > > >
> > > > > Thanks again,
> > > > > Mariusz
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list