[Met_help] [rt.rap.ucar.edu #91549] History for MTD help - time increments failure
John Halley Gotway via RT
met_help at ucar.edu
Mon Aug 19 09:20:11 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I'm trying to understand what triggers this error and how to get around it.
vld497:/net/home/h02/frmm/MET/scripts > ./run_mtd.chl
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-01_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-02_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-03_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-04_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-05_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-06_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-07_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-08_Forecast_00.-12.nc"
DEBUG 2: mtd_read_data() -> processing file "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-09_Forecast_00.-12.nc"
ERROR :
ERROR :
ERROR : mtd_read_data() -> file time increments are not constant!
ERROR :
It happens whether I run single or with fcst/obs arguments but if I run single with the forecast it runs fine so the problem is with the analyses. I've attached the set of files used to run the above ... as far as I can tell there's nothing wrong with them but MTD doesn't want to know.
Thanks
Marion
--
Dr Marion Mittermaier Manager: Model diagnostics and novel verification
Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392 885681
E-mail: marion.mittermaier at metoffice.gov.uk<mailto:marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<http://www.metoffice.gov.uk/>
http://www.metoffice.gov.uk/research/people/marion-mittermaier
I am co-chair of the WMO WWRP Joint Working Group on Forecast Verification Research (JWGFVR) http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MTD help - time increments failure
From: John Halley Gotway
Time: Fri Aug 16 17:33:50 2019
Marion,
Thanks for sending the sample data to demonstrate the issue. I ran
the
following command and was able to reproduce the error you describe:
/usr/local/met-8.0/bin/mtd -single Met*.nc -config MTDConfig.CHL
ERROR : mtd_read_data() -> file time increments are not constant!
Increasing the verbosity level to 4 and carefully inspecting the
timestamps
reveals the source of the problem (see below). The punch line is that
there's a discrepancy between the timestamp of the file name and the
actual
time inside the last file in the list:
*DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-09_Forecast_00.-12.nc
<http://12.nc>"DEBUG 4:
valid time: 20190512_120000*
I looked inside that file to confirm the discrepancy:
*ncdump -v time MetO-NWS-BIO-dm-CPWC_2019-05-09_Forecast_00.-12.nc
<http://12.nc>*
* double time(time) ; time:units = "seconds
since
2014-01-01 00:00:00" ;*
* time = 169128000 ;*
Running these numbers through the date command confirms that MET is
correctly parsing the timestamp as 20190512... even though we were
expecting it to be 2019-05-08:
*date '+%Y-%m-%d' --date="2014-01-01 + 169128000 seconds"2019-05-12*
The timestamp for the last file jumps from 20190507_120000 to
20190512_120000... and that's a 5 day increment... not a 1 day
increment
like all the previous ones.
Hope that helps.
Thanks,
John
/usr/local/met-8.0/bin/mtd -single Met*.nc -config MTDConfig.CHL -v 4
|
egrep "processing file|valid time:"
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-01_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190430_120000
DEBUG 4: valid time: 20190430_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-02_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190501_120000
DEBUG 4: valid time: 20190501_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-03_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190502_120000
DEBUG 4: valid time: 20190502_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-04_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190503_120000
DEBUG 4: valid time: 20190503_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-05_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190504_120000
DEBUG 4: valid time: 20190504_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-06_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190505_120000
DEBUG 4: valid time: 20190505_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-07_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190506_120000
DEBUG 4: valid time: 20190506_120000
DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-08_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190507_120000
DEBUG 4: valid time: 20190507_120000
*DEBUG 2: mtd_read_data() -> processing file
"MetO-NWS-BIO-dm-CPWC_2019-05-09_Forecast_00.-12.nc
<http://12.nc>"DEBUG 4:
valid time: 20190512_120000*
DEBUG 4: valid time: 20190512_120000
ERROR :
ERROR :
ERROR : mtd_read_data() -> file time increments are not constant!
On Fri, Aug 16, 2019 at 8:29 AM marion.mittermaier at metoffice.gov.uk
via RT <
met_help at ucar.edu> wrote:
>
> Fri Aug 16 08:28:51 2019: Request 91549 was acted upon.
> Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> Queue: met_help
> Subject: MTD help - time increments failure
> Owner: Nobody
> Requestors: marion.mittermaier at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91549 >
>
>
> Hello,
>
> I'm trying to understand what triggers this error and how to get
around it.
>
> vld497:/net/home/h02/frmm/MET/scripts > ./run_mtd.chl
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
01_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
02_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
03_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
04_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
05_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
06_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
07_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
08_Forecast_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
09_Forecast_00.-
> 12.nc"
> ERROR :
> ERROR :
> ERROR : mtd_read_data() -> file time increments are not constant!
> ERROR :
>
> It happens whether I run single or with fcst/obs arguments but if I
run
> single with the forecast it runs fine so the problem is with the
analyses.
> I've attached the set of files used to run the above ... as far as I
can
> tell there's nothing wrong with them but MTD doesn't want to know.
>
> Thanks
> Marion
> --
> Dr Marion Mittermaier Manager: Model diagnostics and novel
verification
>
> Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
885681
> E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> http://www.metoffice.gov.uk/>
>
> http://www.metoffice.gov.uk/research/people/marion-mittermaier
>
> I am co-chair of the WMO WWRP Joint Working Group on Forecast
Verification
> Research (JWGFVR)
>
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #91549] MTD help - time increments failure
From: marion.mittermaier at metoffice.gov.uk
Time: Mon Aug 19 03:08:41 2019
Thanks John.
I wasn't expecting that. As I'm not the one generating the files I'm
not that conversant with netcdf (yet).
I should have increased the verbosity to see that myself...! However,
I'll have to wait for the person who does produce these files to come
back from holiday before anything can be done.
Thanks!
Marion
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: 17 August 2019 00:34
To: Mittermaier, Marion <marion.mittermaier at metoffice.gov.uk>
Subject: Re: [rt.rap.ucar.edu #91549] MTD help - time increments
failure
Marion,
Thanks for sending the sample data to demonstrate the issue. I ran
the following command and was able to reproduce the error you
describe:
/usr/local/met-8.0/bin/mtd -single Met*.nc -config MTDConfig.CHL
ERROR : mtd_read_data() -> file time increments are not constant!
Increasing the verbosity level to 4 and carefully inspecting the
timestamps reveals the source of the problem (see below). The punch
line is that there's a discrepancy between the timestamp of the file
name and the actual time inside the last file in the list:
*DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-09_Forecast_00.-12.nc <http://12.nc>"DEBUG 4:
valid time: 20190512_120000*
I looked inside that file to confirm the discrepancy:
*ncdump -v time MetO-NWS-BIO-dm-CPWC_2019-05-09_Forecast_00.-12.nc
<http://12.nc>*
* double time(time) ; time:units = "seconds
since
2014-01-01 00:00:00" ;*
* time = 169128000 ;*
Running these numbers through the date command confirms that MET is
correctly parsing the timestamp as 20190512... even though we were
expecting it to be 2019-05-08:
*date '+%Y-%m-%d' --date="2014-01-01 + 169128000 seconds"2019-05-12*
The timestamp for the last file jumps from 20190507_120000 to
20190512_120000... and that's a 5 day increment... not a 1 day
increment like all the previous ones.
Hope that helps.
Thanks,
John
/usr/local/met-8.0/bin/mtd -single Met*.nc -config MTDConfig.CHL -v 4
| egrep "processing file|valid time:"
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-01_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190430_120000
DEBUG 4: valid time: 20190430_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-02_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190501_120000
DEBUG 4: valid time: 20190501_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-03_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190502_120000
DEBUG 4: valid time: 20190502_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-04_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190503_120000
DEBUG 4: valid time: 20190503_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-05_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190504_120000
DEBUG 4: valid time: 20190504_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-06_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190505_120000
DEBUG 4: valid time: 20190505_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-07_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190506_120000
DEBUG 4: valid time: 20190506_120000
DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-08_Forecast_00.-12.nc"
DEBUG 4: valid time: 20190507_120000
DEBUG 4: valid time: 20190507_120000
*DEBUG 2: mtd_read_data() -> processing file "MetO-NWS-BIO-dm-
CPWC_2019-05-09_Forecast_00.-12.nc <http://12.nc>"DEBUG 4:
valid time: 20190512_120000*
DEBUG 4: valid time: 20190512_120000
ERROR :
ERROR :
ERROR : mtd_read_data() -> file time increments are not constant!
On Fri, Aug 16, 2019 at 8:29 AM marion.mittermaier at metoffice.gov.uk
via RT < met_help at ucar.edu> wrote:
>
> Fri Aug 16 08:28:51 2019: Request 91549 was acted upon.
> Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> Queue: met_help
> Subject: MTD help - time increments failure
> Owner: Nobody
> Requestors: marion.mittermaier at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91549
> >
>
>
> Hello,
>
> I'm trying to understand what triggers this error and how to get
around it.
>
> vld497:/net/home/h02/frmm/MET/scripts > ./run_mtd.chl DEBUG 2:
> mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
01_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
02_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
03_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
04_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
05_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
06_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
07_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
08_Forecas
> t_00.-
> 12.nc"
> DEBUG 2: mtd_read_data() -> processing file
> "/project/hive/input/model/BIO/MetO-NWS-BIO-dm-CPWC_2019-05-
09_Forecas
> t_00.-
> 12.nc"
> ERROR :
> ERROR :
> ERROR : mtd_read_data() -> file time increments are not constant!
> ERROR :
>
> It happens whether I run single or with fcst/obs arguments but if I
> run single with the forecast it runs fine so the problem is with the
analyses.
> I've attached the set of files used to run the above ... as far as I
> can tell there's nothing wrong with them but MTD doesn't want to
know.
>
> Thanks
> Marion
> --
> Dr Marion Mittermaier Manager: Model diagnostics and novel
verification
>
> Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
> 885681
> E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> http://www.metoffice.gov.uk/>
>
> http://www.metoffice.gov.uk/research/people/marion-mittermaier
>
> I am co-chair of the WMO WWRP Joint Working Group on Forecast
> Verification Research (JWGFVR)
>
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
>
>
>
------------------------------------------------
More information about the Met_help
mailing list