[Met_help] [rt.rap.ucar.edu #99789] History for Debugging MODE
George McCabe via RT
met_help at ucar.edu
Tue May 18 11:33:02 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I'm trying to use the MODE tool on some reanalysis- and satellite data I wish to analyse for my PhD research, and don't really know how to debug in light of my (most recent) error message. (Initially I got errors associated with a lack of CF conventions in my home-made observation product, so to proceed I am working with two reanalysis time-steps which I know to be CF compliant.) I have attached the two netCDFs that I am trying to operate on, as well as my config file. The error message I get is:
[cid:image001.png at 01D740F2.E0A65760]
This is, I suppose, the problem with using software as a 'black box'. If it's immediately clear what the issue is here, that would be fab - otherwise I'm really only looking for pointers on how to find the location of the issue in the software so that I can get my head around the syntax and debug it. Incidentally, I have tried running MODE between (identical) files with a time dimension, as opposed to a single step, and get the same error message.
Any help much appreciated!
Tom
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Tue May 04 08:38:00 2021
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field
information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default
settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://github.com/dtcenter/MET/blob/6b951f395d35b81f28cea8aa150734abdc600866/met/data/config/MODEConfig_default#L55-
L71
The last line from the excerpt sets the obs (observation) dictionary
to the
same values as the fcst dictionary. The field dictionary inside fcst
tells
MET to look for a field called 'APCP' at level 'A03.' You will need to
set
the name value to "hcc" which should remove the warning alerting you
that
it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can
only read 2D data, so you will have to specify which 2D volume to
read. It
looks like the NetCDF files you provided were not attached to the
ticket,
so I can't see how the data is stored. You can use * for the lat and
lon
dimensions to get all data points and set a number for the 3rd
dimension to
specify which slab of data to pass in. If the hcc field has (time,
lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the
first dimension is indeed time and it follows CF conventions for
specifying
time, MET even supports supplying the desired time as the value and it
will
find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://met.readthedocs.io/en/latest/Users_Guide/data_io.html#met-
configuration-file-options
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You
could either print the headers using ncdump -h and send me that
information
or upload the data to our FTP server (see How To Send Us Data in
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data I
> wish to analyse for my PhD research, and don't really know how to
debug in
> light of my (most recent) error message. (Initially I got errors
associated
> with a lack of CF conventions in my home-made observation product,
so to
> proceed I am working with two reanalysis time-steps which I know to
be CF
> compliant.) I have attached the two netCDFs that I am trying to
operate on,
> as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'. If
> it's immediately clear what the issue is here, that would be fab -
> otherwise I'm really only looking for pointers on how to find the
location
> of the issue in the software so that I can get my head around the
syntax
> and debug it. Incidentally, I have tried running MODE between
(identical)
> files with a time dimension, as opposed to a single step, and get
the same
> error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 08:49:46 2021
Hi George,
Sorry, I meant to attach my files. I've done so now; will have a look
at your advice below but if these are illuminating feel free to share
your thoughts!
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 15:38
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RULINH9sa4kJB8ikglLjghlVFgIfCC90QHoUSFc%2FXeg%3D&reserved=0
The last line from the excerpt sets the obs (observation) dictionary
to the same values as the fcst dictionary. The field dictionary inside
fcst tells MET to look for a field called 'APCP' at level 'A03.' You
will need to set the name value to "hcc" which should remove the
warning alerting you that it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can only read 2D data, so you will have to specify which 2D volume to
read. It looks like the NetCDF files you provided were not attached to
the ticket, so I can't see how the data is stored. You can use * for
the lat and lon dimensions to get all data points and set a number for
the 3rd dimension to specify which slab of data to pass in. If the hcc
field has (time, lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the first dimension is indeed time and it follows CF conventions for
specifying time, MET even supports supplying the desired time as the
value and it will find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
configuration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ke6eAM9sgM6L3LoLzjhaEvmjbpBEADD6BotxO%2FgXHp4%3D&reserved=0
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You could either print the headers using ncdump -h and send me that
information or upload the data to our FTP server (see How To Send Us
Data in
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zywwR%2BR9wwrqH2V%2FwfTgAB0hMQ0o9hdCc15gXnG%2F61s%3D&reserved=0
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAeflSN
> Oeww%3D&reserved=0 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data
> I wish to analyse for my PhD research, and don't really know how to
> debug in light of my (most recent) error message. (Initially I got
> errors associated with a lack of CF conventions in my home-made
> observation product, so to proceed I am working with two reanalysis
> time-steps which I know to be CF
> compliant.) I have attached the two netCDFs that I am trying to
> operate on, as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'.
> If it's immediately clear what the issue is here, that would be fab
-
> otherwise I'm really only looking for pointers on how to find the
> location of the issue in the software so that I can get my head
around
> the syntax and debug it. Incidentally, I have tried running MODE
> between (identical) files with a time dimension, as opposed to a
> single step, and get the same error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 08:54:21 2021
Hi again George,
Much as I'm sure you were excited to get your teeth into MyConfig,
your initial response was sufficiently helpful that I am now able to
run things (I had removed the "level" part of the field dictionary,
rather than specifying the single level).
Thanks a bunch!
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 15:38
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RULINH9sa4kJB8ikglLjghlVFgIfCC90QHoUSFc%2FXeg%3D&reserved=0
The last line from the excerpt sets the obs (observation) dictionary
to the same values as the fcst dictionary. The field dictionary inside
fcst tells MET to look for a field called 'APCP' at level 'A03.' You
will need to set the name value to "hcc" which should remove the
warning alerting you that it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can only read 2D data, so you will have to specify which 2D volume to
read. It looks like the NetCDF files you provided were not attached to
the ticket, so I can't see how the data is stored. You can use * for
the lat and lon dimensions to get all data points and set a number for
the 3rd dimension to specify which slab of data to pass in. If the hcc
field has (time, lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the first dimension is indeed time and it follows CF conventions for
specifying time, MET even supports supplying the desired time as the
value and it will find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
configuration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ke6eAM9sgM6L3LoLzjhaEvmjbpBEADD6BotxO%2FgXHp4%3D&reserved=0
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You could either print the headers using ncdump -h and send me that
information or upload the data to our FTP server (see How To Send Us
Data in
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zywwR%2BR9wwrqH2V%2FwfTgAB0hMQ0o9hdCc15gXnG%2F61s%3D&reserved=0
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAeflSN
> Oeww%3D&reserved=0 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data
> I wish to analyse for my PhD research, and don't really know how to
> debug in light of my (most recent) error message. (Initially I got
> errors associated with a lack of CF conventions in my home-made
> observation product, so to proceed I am working with two reanalysis
> time-steps which I know to be CF
> compliant.) I have attached the two netCDFs that I am trying to
> operate on, as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'.
> If it's immediately clear what the issue is here, that would be fab
-
> otherwise I'm really only looking for pointers on how to find the
> location of the issue in the software so that I can get my head
around
> the syntax and debug it. Incidentally, I have tried running MODE
> between (identical) files with a time dimension, as opposed to a
> single step, and get the same error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Tue May 04 08:59:03 2021
Tom,
Glad to hear it! I will close this ticket now, but feel free to open a
new
one if you have any other issues or questions.
Thanks,
George
On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Hi again George,
>
> Much as I'm sure you were excited to get your teeth into MyConfig,
your
> initial response was sufficiently helpful that I am now able to run
things
> (I had removed the "level" part of the field dictionary, rather than
> specifying the single level).
>
> Thanks a bunch!
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 15:38
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> I see you are having issues with running MODE using NetCDF data.
>
> The warning and error messages you are receiving suggests that the
field
> information you are providing in your config file are incorrect.
>
> I can't see what you have set inside your MyConfig file, but the
default
> settings for MODE set the fcst ( forecast) dictionary to:
>
> fcst = {
> field = {
> name = "APCP";
> level = "A03";
> }
>
>
> censor_thresh = [];
> censor_val = [];
> conv_radius = 60.0/grid_res; // in grid squares
> conv_thresh = >=5.0;
> vld_thresh = 0.5;
> filter_attr_name = [];
> filter_attr_thresh = [];
> merge_thresh = >=1.25;
> merge_flag = THRESH;
> }
> obs = fcst;
>
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RULINH9sa4kJB8ikglLjghlVFgIfCC90QHoUSFc%2FXeg%3D&reserved=0
>
> The last line from the excerpt sets the obs (observation) dictionary
to
> the same values as the fcst dictionary. The field dictionary inside
fcst
> tells MET to look for a field called 'APCP' at level 'A03.' You will
need
> to set the name value to "hcc" which should remove the warning
alerting you
> that it is using the first field that it finds.
>
> The error message saying "needed 3 arguments for variable hcc, got
2"
> implies that the hcc field is a NetCDF file that has 3 dimensions.
MET can
> only read 2D data, so you will have to specify which 2D volume to
read. It
> looks like the NetCDF files you provided were not attached to the
ticket,
> so I can't see how the data is stored. You can use * for the lat and
lon
> dimensions to get all data points and set a number for the 3rd
dimension to
> specify which slab of data to pass in. If the hcc field has (time,
lat,
> lon) as the dimensions, you can set:
>
> level = "(0,*,*)";
>
> to extract the 2D volume for the first time dimension in the field.
If the
> first dimension is indeed time and it follows CF conventions for
specifying
> time, MET even supports supplying the desired time as the value and
it will
> find the appropriate 2D volume:
>
> level = "(20210504_120000,*,*)";
>
> More information on MET I/O configurations can be found in the
User's Guide
> here:
>
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
configuration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ke6eAM9sgM6L3LoLzjhaEvmjbpBEADD6BotxO%2FgXHp4%3D&reserved=0
>
> Please let me know if you have any issues getting this configured.
The
> NetCDF files may not have been included because of file size limits.
You
> could either print the headers using ncdump -h and send me that
information
> or upload the data to our FTP server (see How To Send Us Data in
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zywwR%2BR9wwrqH2V%2FwfTgAB0hMQ0o9hdCc15gXnG%2F61s%3D&reserved=0
> ).
>
> Thanks,
> George
>
> On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > Queue: met_help
> > Subject: Debugging MODE
> > Owner: Nobody
> > Requestors: T.Eldridge at pgr.reading.ac.uk
> > Status: new
> > Ticket <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAeflSN
> > Oeww%3D&reserved=0 >
> >
> >
> > I'm trying to use the MODE tool on some reanalysis- and satellite
data
> > I wish to analyse for my PhD research, and don't really know how
to
> > debug in light of my (most recent) error message. (Initially I got
> > errors associated with a lack of CF conventions in my home-made
> > observation product, so to proceed I am working with two
reanalysis
> > time-steps which I know to be CF
> > compliant.) I have attached the two netCDFs that I am trying to
> > operate on, as well as my config file. The error message I get is:
> >
> > [cid:image001.png at 01D740F2.E0A65760]
> >
> > This is, I suppose, the problem with using software as a 'black
box'.
> > If it's immediately clear what the issue is here, that would be
fab -
> > otherwise I'm really only looking for pointers on how to find the
> > location of the issue in the software so that I can get my head
around
> > the syntax and debug it. Incidentally, I have tried running MODE
> > between (identical) files with a time dimension, as opposed to a
> > single step, and get the same error message.
> >
> > Any help much appreciated!
> >
> > Tom
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 09:00:46 2021
One quick thing while I have you: I noticed that the output netCDF
features an attribute called Projection whose value is "LatLon" -
would you happen to know if I can specify this for my non-CF compliant
file? No biggie if not, would just save me some time...
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 15:59
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Tom,
Glad to hear it! I will close this ticket now, but feel free to open a
new one if you have any other issues or questions.
Thanks,
George
On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a20%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2BvwFSq
> %2Br%2BslCTY%3D&reserved=0 >
>
> Hi again George,
>
> Much as I'm sure you were excited to get your teeth into MyConfig,
> your initial response was sufficiently helpful that I am now able to
> run things (I had removed the "level" part of the field dictionary,
> rather than specifying the single level).
>
> Thanks a bunch!
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 15:38
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> I see you are having issues with running MODE using NetCDF data.
>
> The warning and error messages you are receiving suggests that the
> field information you are providing in your config file are
incorrect.
>
> I can't see what you have set inside your MyConfig file, but the
> default settings for MODE set the fcst ( forecast) dictionary to:
>
> fcst = {
> field = {
> name = "APCP";
> level = "A03";
> }
>
>
> censor_thresh = [];
> censor_val = [];
> conv_radius = 60.0/grid_res; // in grid squares
> conv_thresh = >=5.0;
> vld_thresh = 0.5;
> filter_attr_name = [];
> filter_attr_thresh = [];
> merge_thresh = >=1.25;
> merge_flag = THRESH;
> }
> obs = fcst;
>
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
>
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600
> 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7
>
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a2
>
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%7CUn
>
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW
>
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4Ib6tH
> G3g4N%2BeRCpvCU%3D&reserved=0
>
> The last line from the excerpt sets the obs (observation) dictionary
> to the same values as the fcst dictionary. The field dictionary
inside
> fcst tells MET to look for a field called 'APCP' at level 'A03.' You
> will need to set the name value to "hcc" which should remove the
> warning alerting you that it is using the first field that it finds.
>
> The error message saying "needed 3 arguments for variable hcc, got
2"
> implies that the hcc field is a NetCDF file that has 3 dimensions.
MET
> can only read 2D data, so you will have to specify which 2D volume
to
> read. It looks like the NetCDF files you provided were not attached
to
> the ticket, so I can't see how the data is stored. You can use * for
> the lat and lon dimensions to get all data points and set a number
for
> the 3rd dimension to specify which slab of data to pass in. If the
hcc
> field has (time, lat,
> lon) as the dimensions, you can set:
>
> level = "(0,*,*)";
>
> to extract the 2D volume for the first time dimension in the field.
If
> the first dimension is indeed time and it follows CF conventions for
> specifying time, MET even supports supplying the desired time as the
> value and it will find the appropriate 2D volume:
>
> level = "(20210504_120000,*,*)";
>
> More information on MET I/O configurations can be found in the
User's
> Guide
> here:
>
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
config
> uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk
>
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f
>
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
>
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Tt
> GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
>
> Please let me know if you have any issues getting this configured.
The
> NetCDF files may not have been included because of file size limits.
> You could either print the headers using ncdump -h and send me that
> information or upload the data to our FTP server (see How To Send Us
> Data in
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk
>
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95b
>
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371
>
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi
>
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCVW4JE
> Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> ).
>
> Thanks,
> George
>
> On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > Queue: met_help
> > Subject: Debugging MODE
> > Owner: Nobody
> > Requestors: T.Eldridge at pgr.reading.ac.uk
> > Status: new
> > Ticket <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAefl
> > SN
> > Oeww%3D&reserved=0 >
> >
> >
> > I'm trying to use the MODE tool on some reanalysis- and satellite
> > data I wish to analyse for my PhD research, and don't really know
> > how to debug in light of my (most recent) error message.
(Initially
> > I got errors associated with a lack of CF conventions in my
> > home-made observation product, so to proceed I am working with two
> > reanalysis time-steps which I know to be CF
> > compliant.) I have attached the two netCDFs that I am trying to
> > operate on, as well as my config file. The error message I get is:
> >
> > [cid:image001.png at 01D740F2.E0A65760]
> >
> > This is, I suppose, the problem with using software as a 'black
box'.
> > If it's immediately clear what the issue is here, that would be
fab
> > - otherwise I'm really only looking for pointers on how to find
the
> > location of the issue in the software so that I can get my head
> > around the syntax and debug it. Incidentally, I have tried running
> > MODE between (identical) files with a time dimension, as opposed
to
> > a single step, and get the same error message.
> >
> > Any help much appreciated!
> >
> > Tom
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Tue May 04 09:14:27 2021
Hi Tom,
The output from the MET tools are a specific NetCDF format defined and
read
by the MET tools. I am not exactly sure of the criteria used by the
tools
to determine if a NetCDF file was created by one of its own. It would
likely be easier to do one of two things:
1) Modify the non-CF compliant file to add the required information to
make
it CF compliant
OR
2) Utilize the Python Embedding functionality to read the NetCDF file
The latter involves writing a Python script to read the data, then
define a
2D data volume and attributes to define how the data is formatted
(time,
fields, grid, etc.). To use this functionality, you need to have the
MET
tools built with Python enabled and the required Python packages
available.
More information on Python Embedding is here:
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html
Sample scripts that utilize this logic can be found here:
https://dtcenter.org/community-code/model-evaluation-tools-met/sample-
analysis-scripts
And a specific example that reads NetCDF data from that page is here:
https://dtcenter.org/sites/default/files/community-code/met/python-
scripts/read_VIIRS_AOD_NetCDF.py.txt
Let me know if you have any other questions.
Thanks,
George
On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> One quick thing while I have you: I noticed that the output netCDF
> features an attribute called Projection whose value is "LatLon" -
would you
> happen to know if I can specify this for my non-CF compliant file?
No
> biggie if not, would just save me some time...
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 15:59
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Tom,
>
> Glad to hear it! I will close this ticket now, but feel free to open
a new
> one if you have any other issues or questions.
>
> Thanks,
> George
>
> On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a20%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2BvwFSq
> > %2Br%2BslCTY%3D&reserved=0 >
> >
> > Hi again George,
> >
> > Much as I'm sure you were excited to get your teeth into MyConfig,
> > your initial response was sufficiently helpful that I am now able
to
> > run things (I had removed the "level" part of the field
dictionary,
> > rather than specifying the single level).
> >
> > Thanks a bunch!
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 15:38
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > I see you are having issues with running MODE using NetCDF data.
> >
> > The warning and error messages you are receiving suggests that the
> > field information you are providing in your config file are
incorrect.
> >
> > I can't see what you have set inside your MyConfig file, but the
> > default settings for MODE set the fcst ( forecast) dictionary to:
> >
> > fcst = {
> > field = {
> > name = "APCP";
> > level = "A03";
> > }
> >
> >
> > censor_thresh = [];
> > censor_val = [];
> > conv_radius = 60.0/grid_res; // in grid squares
> > conv_thresh = >=5.0;
> > vld_thresh = 0.5;
> > filter_attr_name = [];
> > filter_attr_thresh = [];
> > merge_thresh = >=1.25;
> > merge_flag = THRESH;
> > }
> > obs = fcst;
> >
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600
> > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7
> >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a2
> >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%7CUn
> >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW
> >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4Ib6tH
> > G3g4N%2BeRCpvCU%3D&reserved=0
> >
> > The last line from the excerpt sets the obs (observation)
dictionary
> > to the same values as the fcst dictionary. The field dictionary
inside
> > fcst tells MET to look for a field called 'APCP' at level 'A03.'
You
> > will need to set the name value to "hcc" which should remove the
> > warning alerting you that it is using the first field that it
finds.
> >
> > The error message saying "needed 3 arguments for variable hcc, got
2"
> > implies that the hcc field is a NetCDF file that has 3 dimensions.
MET
> > can only read 2D data, so you will have to specify which 2D volume
to
> > read. It looks like the NetCDF files you provided were not
attached to
> > the ticket, so I can't see how the data is stored. You can use *
for
> > the lat and lon dimensions to get all data points and set a number
for
> > the 3rd dimension to specify which slab of data to pass in. If the
hcc
> > field has (time, lat,
> > lon) as the dimensions, you can set:
> >
> > level = "(0,*,*)";
> >
> > to extract the 2D volume for the first time dimension in the
field. If
> > the first dimension is indeed time and it follows CF conventions
for
> > specifying time, MET even supports supplying the desired time as
the
> > value and it will find the appropriate 2D volume:
> >
> > level = "(20210504_120000,*,*)";
> >
> > More information on MET I/O configurations can be found in the
User's
> > Guide
> > here:
> >
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
config
> > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk
> >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f
> >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Tt
> > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> >
> > Please let me know if you have any issues getting this configured.
The
> > NetCDF files may not have been included because of file size
limits.
> > You could either print the headers using ncdump -h and send me
that
> > information or upload the data to our FTP server (see How To Send
Us
> > Data in
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk
> >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95b
> >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371
> >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi
> >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCVW4JE
> > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > ).
> >
> > Thanks,
> > George
> >
> > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > Queue: met_help
> > > Subject: Debugging MODE
> > > Owner: Nobody
> > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > Status: new
> > > Ticket <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAefl
> > > SN
> > > Oeww%3D&reserved=0 >
> > >
> > >
> > > I'm trying to use the MODE tool on some reanalysis- and
satellite
> > > data I wish to analyse for my PhD research, and don't really
know
> > > how to debug in light of my (most recent) error message.
(Initially
> > > I got errors associated with a lack of CF conventions in my
> > > home-made observation product, so to proceed I am working with
two
> > > reanalysis time-steps which I know to be CF
> > > compliant.) I have attached the two netCDFs that I am trying to
> > > operate on, as well as my config file. The error message I get
is:
> > >
> > > [cid:image001.png at 01D740F2.E0A65760]
> > >
> > > This is, I suppose, the problem with using software as a 'black
box'.
> > > If it's immediately clear what the issue is here, that would be
fab
> > > - otherwise I'm really only looking for pointers on how to find
the
> > > location of the issue in the software so that I can get my head
> > > around the syntax and debug it. Incidentally, I have tried
running
> > > MODE between (identical) files with a time dimension, as opposed
to
> > > a single step, and get the same error message.
> > >
> > > Any help much appreciated!
> > >
> > > Tom
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 09:16:24 2021
Thanks George, this is all great stuff, I'm sure the links you've
provided will cover my needs for now - expect to hear back from me
later this week if I keep banging my head against the wall.
All the best,
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 16:14
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
The output from the MET tools are a specific NetCDF format defined and
read by the MET tools. I am not exactly sure of the criteria used by
the tools to determine if a NetCDF file was created by one of its own.
It would likely be easier to do one of two things:
1) Modify the non-CF compliant file to add the required information to
make it CF compliant
OR
2) Utilize the Python Embedding functionality to read the NetCDF file
The latter involves writing a Python script to read the data, then
define a 2D data volume and attributes to define how the data is
formatted (time, fields, grid, etc.). To use this functionality, you
need to have the MET tools built with Python enabled and the required
Python packages available.
More information on Python Embedding is here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722006874%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=P68NUQeagwiGD5ifgntxaZD0eY3yy%2B%2Fli%2B6HP4OG5GI%3D&reserved=0
Sample scripts that utilize this logic can be found here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fsample-analysis-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722006874%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUutYoG2PrLnG%2F9GdS5tExrUYit6nVkX9qwDOp9fGHo%3D&reserved=0
And a specific example that reads NetCDF data from that page is here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-
scripts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1legGTr7BSX5rOk2SA9hxWECEKOIwXrhVvmiASg%2BkbU%3D&reserved=0
Let me know if you have any other questions.
Thanks,
George
On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%2FnU
> fX%2Bkh0%3D&reserved=0 >
>
> One quick thing while I have you: I noticed that the output netCDF
> features an attribute called Projection whose value is "LatLon" -
> would you happen to know if I can specify this for my non-CF
compliant
> file? No biggie if not, would just save me some time...
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 15:59
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Tom,
>
> Glad to hear it! I will close this ticket now, but feel free to open
a
> new one if you have any other issues or questions.
>
> Thanks,
> George
>
> On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a20
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2BvwF
> > Sq
> > %2Br%2BslCTY%3D&reserved=0 >
> >
> > Hi again George,
> >
> > Much as I'm sure you were excited to get your teeth into MyConfig,
> > your initial response was sufficiently helpful that I am now able
to
> > run things (I had removed the "level" part of the field
dictionary,
> > rather than specifying the single level).
> >
> > Thanks a bunch!
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 15:38
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > I see you are having issues with running MODE using NetCDF data.
> >
> > The warning and error messages you are receiving suggests that the
> > field information you are providing in your config file are
incorrect.
> >
> > I can't see what you have set inside your MyConfig file, but the
> > default settings for MODE set the fcst ( forecast) dictionary to:
> >
> > fcst = {
> > field = {
> > name = "APCP";
> > level = "A03";
> > }
> >
> >
> > censor_thresh = [];
> > censor_val = [];
> > conv_radius = 60.0/grid_res; // in grid squares
> > conv_thresh = >=5.0;
> > vld_thresh = 0.5;
> > filter_attr_name = [];
> > filter_attr_thresh = [];
> > merge_thresh = >=1.25;
> > merge_flag = THRESH;
> > }
> > obs = fcst;
> >
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > th
> >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc6
> > 00
> > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04
> > %7
> >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2
> > a2
> >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%7C
> > Un
> >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> > aW
> >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4Ib6
> > tH
> > G3g4N%2BeRCpvCU%3D&reserved=0
> >
> > The last line from the excerpt sets the obs (observation)
dictionary
> > to the same values as the fcst dictionary. The field dictionary
> > inside fcst tells MET to look for a field called 'APCP' at level
> > 'A03.' You will need to set the name value to "hcc" which should
> > remove the warning alerting you that it is using the first field
that it finds.
> >
> > The error message saying "needed 3 arguments for variable hcc, got
2"
> > implies that the hcc field is a NetCDF file that has 3 dimensions.
> > MET can only read 2D data, so you will have to specify which 2D
> > volume to read. It looks like the NetCDF files you provided were
not
> > attached to the ticket, so I can't see how the data is stored. You
> > can use * for the lat and lon dimensions to get all data points
and
> > set a number for the 3rd dimension to specify which slab of data
to
> > pass in. If the hcc field has (time, lat,
> > lon) as the dimensions, you can set:
> >
> > level = "(0,*,*)";
> >
> > to extract the 2D volume for the first time dimension in the
field.
> > If the first dimension is indeed time and it follows CF
conventions
> > for specifying time, MET even supports supplying the desired time
as
> > the value and it will find the appropriate 2D volume:
> >
> > level = "(20210504_120000,*,*)";
> >
> > More information on MET I/O configurations can be found in the
> > User's Guide
> > here:
> >
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
conf
> > ig
> > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > uk
> >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e
> > 5f
> >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> > Aw
> >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=
> > Tt
> > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> >
> > Please let me know if you have any issues getting this configured.
> > The NetCDF files may not have been included because of file size
limits.
> > You could either print the headers using ncdump -h and send me
that
> > information or upload the data to our FTP server (see How To Send
Us
> > Data in
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > ce
> > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-help-
de
> > sk
> >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a9
> > 5b
> >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573
> > 71
> >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> > Ii
> >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCVW4
> > JE
> > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > ).
> >
> > Thanks,
> > George
> >
> > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > Queue: met_help
> > > Subject: Debugging MODE
> > > Owner: Nobody
> > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > Status: new
> > > Ticket <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39
> > > cf
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAe
> > > fl
> > > SN
> > > Oeww%3D&reserved=0 >
> > >
> > >
> > > I'm trying to use the MODE tool on some reanalysis- and
satellite
> > > data I wish to analyse for my PhD research, and don't really
know
> > > how to debug in light of my (most recent) error message.
> > > (Initially I got errors associated with a lack of CF conventions
> > > in my home-made observation product, so to proceed I am working
> > > with two reanalysis time-steps which I know to be CF
> > > compliant.) I have attached the two netCDFs that I am trying to
> > > operate on, as well as my config file. The error message I get
is:
> > >
> > > [cid:image001.png at 01D740F2.E0A65760]
> > >
> > > This is, I suppose, the problem with using software as a 'black
box'.
> > > If it's immediately clear what the issue is here, that would be
> > > fab
> > > - otherwise I'm really only looking for pointers on how to find
> > > the location of the issue in the software so that I can get my
> > > head around the syntax and debug it. Incidentally, I have tried
> > > running MODE between (identical) files with a time dimension, as
> > > opposed to a single step, and get the same error message.
> > >
> > > Any help much appreciated!
> > >
> > > Tom
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Tue May 04 09:33:10 2021
Hi Tom,
Sounds good. Please don't bang your head for too long!
- George
On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Thanks George, this is all great stuff, I'm sure the links you've
provided
> will cover my needs for now - expect to hear back from me later this
week
> if I keep banging my head against the wall.
>
> All the best,
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 16:14
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> The output from the MET tools are a specific NetCDF format defined
and
> read by the MET tools. I am not exactly sure of the criteria used by
the
> tools to determine if a NetCDF file was created by one of its own.
It would
> likely be easier to do one of two things:
>
> 1) Modify the non-CF compliant file to add the required information
to
> make it CF compliant
>
> OR
>
> 2) Utilize the Python Embedding functionality to read the NetCDF
file
>
> The latter involves writing a Python script to read the data, then
define
> a 2D data volume and attributes to define how the data is formatted
(time,
> fields, grid, etc.). To use this functionality, you need to have the
MET
> tools built with Python enabled and the required Python packages
available.
>
> More information on Python Embedding is here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722006874%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=P68NUQeagwiGD5ifgntxaZD0eY3yy%2B%2Fli%2B6HP4OG5GI%3D&reserved=0
>
> Sample scripts that utilize this logic can be found here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fsample-analysis-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722006874%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUutYoG2PrLnG%2F9GdS5tExrUYit6nVkX9qwDOp9fGHo%3D&reserved=0
>
> And a specific example that reads NetCDF data from that page is
here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-
scripts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1legGTr7BSX5rOk2SA9hxWECEKOIwXrhVvmiASg%2BkbU%3D&reserved=0
>
> Let me know if you have any other questions.
>
> Thanks,
> George
>
>
> On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%2FnU
> > fX%2Bkh0%3D&reserved=0 >
> >
> > One quick thing while I have you: I noticed that the output netCDF
> > features an attribute called Projection whose value is "LatLon" -
> > would you happen to know if I can specify this for my non-CF
compliant
> > file? No biggie if not, would just save me some time...
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 15:59
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Tom,
> >
> > Glad to hear it! I will close this ticket now, but feel free to
open a
> > new one if you have any other issues or questions.
> >
> > Thanks,
> > George
> >
> > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a20
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2BvwF
> > > Sq
> > > %2Br%2BslCTY%3D&reserved=0 >
> > >
> > > Hi again George,
> > >
> > > Much as I'm sure you were excited to get your teeth into
MyConfig,
> > > your initial response was sufficiently helpful that I am now
able to
> > > run things (I had removed the "level" part of the field
dictionary,
> > > rather than specifying the single level).
> > >
> > > Thanks a bunch!
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 15:38
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > I see you are having issues with running MODE using NetCDF data.
> > >
> > > The warning and error messages you are receiving suggests that
the
> > > field information you are providing in your config file are
incorrect.
> > >
> > > I can't see what you have set inside your MyConfig file, but the
> > > default settings for MODE set the fcst ( forecast) dictionary
to:
> > >
> > > fcst = {
> > > field = {
> > > name = "APCP";
> > > level = "A03";
> > > }
> > >
> > >
> > > censor_thresh = [];
> > > censor_val = [];
> > > conv_radius = 60.0/grid_res; // in grid squares
> > > conv_thresh = >=5.0;
> > > vld_thresh = 0.5;
> > > filter_attr_name = [];
> > > filter_attr_thresh = [];
> > > merge_thresh = >=1.25;
> > > merge_flag = THRESH;
> > > }
> > > obs = fcst;
> > >
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > > th
> > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc6
> > > 00
> > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04
> > > %7
> > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2
> > > a2
> > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%7C
> > > Un
> > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> > > aW
> > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4Ib6
> > > tH
> > > G3g4N%2BeRCpvCU%3D&reserved=0
> > >
> > > The last line from the excerpt sets the obs (observation)
dictionary
> > > to the same values as the fcst dictionary. The field dictionary
> > > inside fcst tells MET to look for a field called 'APCP' at level
> > > 'A03.' You will need to set the name value to "hcc" which should
> > > remove the warning alerting you that it is using the first field
that
> it finds.
> > >
> > > The error message saying "needed 3 arguments for variable hcc,
got 2"
> > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > MET can only read 2D data, so you will have to specify which 2D
> > > volume to read. It looks like the NetCDF files you provided were
not
> > > attached to the ticket, so I can't see how the data is stored.
You
> > > can use * for the lat and lon dimensions to get all data points
and
> > > set a number for the 3rd dimension to specify which slab of data
to
> > > pass in. If the hcc field has (time, lat,
> > > lon) as the dimensions, you can set:
> > >
> > > level = "(0,*,*)";
> > >
> > > to extract the 2D volume for the first time dimension in the
field.
> > > If the first dimension is indeed time and it follows CF
conventions
> > > for specifying time, MET even supports supplying the desired
time as
> > > the value and it will find the appropriate 2D volume:
> > >
> > > level = "(20210504_120000,*,*)";
> > >
> > > More information on MET I/O configurations can be found in the
> > > User's Guide
> > > here:
> > >
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
conf
> > > ig
> > > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > > uk
> > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e
> > > 5f
> > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> > > Aw
> > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=
> > > Tt
> > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > >
> > > Please let me know if you have any issues getting this
configured.
> > > The NetCDF files may not have been included because of file size
> limits.
> > > You could either print the headers using ncdump -h and send me
that
> > > information or upload the data to our FTP server (see How To
Send Us
> > > Data in
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > > ce
> > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-
help-de
> > > sk
> > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a9
> > > 5b
> > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573
> > > 71
> > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> > > Ii
> > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCVW4
> > > JE
> > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > ).
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > > Queue: met_help
> > > > Subject: Debugging MODE
> > > > Owner: Nobody
> > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > Status: new
> > > > Ticket <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39
> > > > cf
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAe
> > > > fl
> > > > SN
> > > > Oeww%3D&reserved=0 >
> > > >
> > > >
> > > > I'm trying to use the MODE tool on some reanalysis- and
satellite
> > > > data I wish to analyse for my PhD research, and don't really
know
> > > > how to debug in light of my (most recent) error message.
> > > > (Initially I got errors associated with a lack of CF
conventions
> > > > in my home-made observation product, so to proceed I am
working
> > > > with two reanalysis time-steps which I know to be CF
> > > > compliant.) I have attached the two netCDFs that I am trying
to
> > > > operate on, as well as my config file. The error message I get
is:
> > > >
> > > > [cid:image001.png at 01D740F2.E0A65760]
> > > >
> > > > This is, I suppose, the problem with using software as a
'black box'.
> > > > If it's immediately clear what the issue is here, that would
be
> > > > fab
> > > > - otherwise I'm really only looking for pointers on how to
find
> > > > the location of the issue in the software so that I can get my
> > > > head around the syntax and debug it. Incidentally, I have
tried
> > > > running MODE between (identical) files with a time dimension,
as
> > > > opposed to a single step, and get the same error message.
> > > >
> > > > Any help much appreciated!
> > > >
> > > > Tom
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: Thomas Eldridge
Time: Wed May 05 08:11:32 2021
Hi George,
Progress made; new issue encountered. I have been able to update my
THIR files to be CF compliant, and have successfully run MODE on some
fields; however, on one example (which I hope will be particularly
insightful RE what MODE does and how to interpret it, since the
feature is very visible in the data), I am getting the following error
message:
ERROR :
ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"! ...
t = nan
ERROR :
I've attached my files, which I've renamed based on the directory
structure I'm using, in case you'd like to take a look. These feature
a single time-step, and MyConfigNoTime has been adjusted accordingly.
Will keep plugging away to see if I can identify the cause myself.
Thanks,
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 16:33
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
Sounds good. Please don't bang your head for too long!
- George
On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZCx1J%
> 2BZ7Lluk%3D&reserved=0 >
>
> Thanks George, this is all great stuff, I'm sure the links you've
> provided will cover my needs for now - expect to hear back from me
> later this week if I keep banging my head against the wall.
>
> All the best,
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 16:14
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> The output from the MET tools are a specific NetCDF format defined
and
> read by the MET tools. I am not exactly sure of the criteria used by
> the tools to determine if a NetCDF file was created by one of its
own.
> It would likely be easier to do one of two things:
>
> 1) Modify the non-CF compliant file to add the required information
to
> make it CF compliant
>
> OR
>
> 2) Utilize the Python Embedding functionality to read the NetCDF
file
>
> The latter involves writing a Python script to read the data, then
> define a 2D data volume and attributes to define how the data is
> formatted (time, fields, grid, etc.). To use this functionality, you
> need to have the MET tools built with Python enabled and the
required Python packages available.
>
> More information on Python Embedding is here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
>
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data=0
>
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11
>
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7
>
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
>
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifjbomL
> xgWzU5tgpA%3D&reserved=0
>
> Sample scripts that utilize this logic can be found here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
analys
> is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297e
>
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7
>
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj
>
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9lk65
> fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
>
> And a specific example that reads NetCDF data from that page is
here:
>
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-code%2Fmet%2Fpython-
scr
>
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40pg
>
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09
>
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb3d8e
>
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C10
>
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&reserv
> ed=0
>
> Let me know if you have any other questions.
>
> Thanks,
> George
>
>
> On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%2F
> > nU
> > fX%2Bkh0%3D&reserved=0 >
> >
> > One quick thing while I have you: I noticed that the output netCDF
> > features an attribute called Projection whose value is "LatLon" -
> > would you happen to know if I can specify this for my non-CF
> > compliant file? No biggie if not, would just save me some time...
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 15:59
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Tom,
> >
> > Glad to hear it! I will close this ticket now, but feel free to
open
> > a new one if you have any other issues or questions.
> >
> > Thanks,
> > George
> >
> > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a
> > > 20
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2Bv
> > > wF
> > > Sq
> > > %2Br%2BslCTY%3D&reserved=0 >
> > >
> > > Hi again George,
> > >
> > > Much as I'm sure you were excited to get your teeth into
MyConfig,
> > > your initial response was sufficiently helpful that I am now
able
> > > to run things (I had removed the "level" part of the field
> > > dictionary, rather than specifying the single level).
> > >
> > > Thanks a bunch!
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 15:38
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > I see you are having issues with running MODE using NetCDF data.
> > >
> > > The warning and error messages you are receiving suggests that
the
> > > field information you are providing in your config file are
incorrect.
> > >
> > > I can't see what you have set inside your MyConfig file, but the
> > > default settings for MODE set the fcst ( forecast) dictionary
to:
> > >
> > > fcst = {
> > > field = {
> > > name = "APCP";
> > > level = "A03";
> > > }
> > >
> > >
> > > censor_thresh = [];
> > > censor_val = [];
> > > conv_radius = 60.0/grid_res; // in grid squares
> > > conv_thresh = >=5.0;
> > > vld_thresh = 0.5;
> > > filter_attr_name = [];
> > > filter_attr_thresh = [];
> > > merge_thresh = >=1.25;
> > > merge_flag = THRESH;
> > > }
> > > obs = fcst;
> > >
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > gi
> > > th
> > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abd
> > > c6
> > > 00
> > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=
> > > 04
> > > %7
> > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0
> > > d2
> > > a2
> > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%
> > > 7C
> > > Un
> > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik
> > > 1h
> > > aW
> > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4I
> > > b6
> > > tH
> > > G3g4N%2BeRCpvCU%3D&reserved=0
> > >
> > > The last line from the excerpt sets the obs (observation)
> > > dictionary to the same values as the fcst dictionary. The field
> > > dictionary inside fcst tells MET to look for a field called
'APCP'
> > > at level 'A03.' You will need to set the name value to "hcc"
which
> > > should remove the warning alerting you that it is using the
first
> > > field that
> it finds.
> > >
> > > The error message saying "needed 3 arguments for variable hcc,
got 2"
> > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > MET can only read 2D data, so you will have to specify which 2D
> > > volume to read. It looks like the NetCDF files you provided were
> > > not attached to the ticket, so I can't see how the data is
stored.
> > > You can use * for the lat and lon dimensions to get all data
> > > points and set a number for the 3rd dimension to specify which
> > > slab of data to pass in. If the hcc field has (time, lat,
> > > lon) as the dimensions, you can set:
> > >
> > > level = "(0,*,*)";
> > >
> > > to extract the 2D volume for the first time dimension in the
field.
> > > If the first dimension is indeed time and it follows CF
> > > conventions for specifying time, MET even supports supplying the
> > > desired time as the value and it will find the appropriate 2D
volume:
> > >
> > > level = "(20210504_120000,*,*)";
> > >
> > > More information on MET I/O configurations can be found in the
> > > User's Guide
> > > here:
> > >
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
co
> > > nf
> > > ig
> > > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > > uk
> > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff9
> > > 0e
> > > 5f
> > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> > > Lj
> > > Aw
> > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdat
> > > a=
> > > Tt
> > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > >
> > > Please let me know if you have any issues getting this
configured.
> > > The NetCDF files may not have been included because of file size
> limits.
> > > You could either print the headers using ncdump -h and send me
> > > that information or upload the data to our FTP server (see How
To
> > > Send Us Data in
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > dt
> > > ce
> > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-
help-
> > > de
> > > sk
> > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4
> > > a9
> > > 5b
> > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755
> > > 73
> > > 71
> > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu
> > > Mz
> > > Ii
> > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCV
> > > W4
> > > JE
> > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > ).
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > > Queue: met_help
> > > > Subject: Debugging MODE
> > > > Owner: Nobody
> > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > Status: new
> > > > Ticket <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > 4%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a
> > > > 39
> > > > cf
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7
> > > > CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEve
> > > > Ae
> > > > fl
> > > > SN
> > > > Oeww%3D&reserved=0 >
> > > >
> > > >
> > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > satellite data I wish to analyse for my PhD research, and
don't
> > > > really know how to debug in light of my (most recent) error
message.
> > > > (Initially I got errors associated with a lack of CF
conventions
> > > > in my home-made observation product, so to proceed I am
working
> > > > with two reanalysis time-steps which I know to be CF
> > > > compliant.) I have attached the two netCDFs that I am trying
to
> > > > operate on, as well as my config file. The error message I get
is:
> > > >
> > > > [cid:image001.png at 01D740F2.E0A65760]
> > > >
> > > > This is, I suppose, the problem with using software as a
'black box'.
> > > > If it's immediately clear what the issue is here, that would
be
> > > > fab
> > > > - otherwise I'm really only looking for pointers on how to
find
> > > > the location of the issue in the software so that I can get my
> > > > head around the syntax and debug it. Incidentally, I have
tried
> > > > running MODE between (identical) files with a time dimension,
as
> > > > opposed to a single step, and get the same error message.
> > > >
> > > > Any help much appreciated!
> > > >
> > > > Tom
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Wed May 05 08:17:27 2021
Hi Tom,
Thank you for providing the data and config file. Could you share the
command you used to get this error? Also, it looks like you mentioned
a
file called MyConfigNoTime but the file you shared is called MyConfig.
I am
just checking to make sure I have the correct file to use for testing.
Thanks,
George
On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Hi George,
>
> Progress made; new issue encountered. I have been able to update my
THIR
> files to be CF compliant, and have successfully run MODE on some
fields;
> however, on one example (which I hope will be particularly
insightful RE
> what MODE does and how to interpret it, since the feature is very
visible
> in the data), I am getting the following error message:
>
> ERROR :
> ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"!
... t =
> nan
> ERROR :
>
> I've attached my files, which I've renamed based on the directory
> structure I'm using, in case you'd like to take a look. These
feature a
> single time-step, and MyConfigNoTime has been adjusted accordingly.
Will
> keep plugging away to see if I can identify the cause myself.
>
> Thanks,
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 16:33
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> Sounds good. Please don't bang your head for too long!
>
> - George
>
> On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZCx1J%
> > 2BZ7Lluk%3D&reserved=0 >
> >
> > Thanks George, this is all great stuff, I'm sure the links you've
> > provided will cover my needs for now - expect to hear back from me
> > later this week if I keep banging my head against the wall.
> >
> > All the best,
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 16:14
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > The output from the MET tools are a specific NetCDF format defined
and
> > read by the MET tools. I am not exactly sure of the criteria used
by
> > the tools to determine if a NetCDF file was created by one of its
own.
> > It would likely be easier to do one of two things:
> >
> > 1) Modify the non-CF compliant file to add the required
information to
> > make it CF compliant
> >
> > OR
> >
> > 2) Utilize the Python Embedding functionality to read the NetCDF
file
> >
> > The latter involves writing a Python script to read the data, then
> > define a 2D data volume and attributes to define how the data is
> > formatted (time, fields, grid, etc.). To use this functionality,
you
> > need to have the MET tools built with Python enabled and the
required
> Python packages available.
> >
> > More information on Python Embedding is here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data=0
> >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11
> >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7
> >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifjbomL
> > xgWzU5tgpA%3D&reserved=0
> >
> > Sample scripts that utilize this logic can be found here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
analys
> > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297e
> >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7
> >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj
> >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9lk65
> > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> >
> > And a specific example that reads NetCDF data from that page is
here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtce
> > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-scr
> >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40pg
> >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09
> >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb3d8e
> >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C10
> >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&reserv
> > ed=0
> >
> > Let me know if you have any other questions.
> >
> > Thanks,
> > George
> >
> >
> > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50e7
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%2F
> > > nU
> > > fX%2Bkh0%3D&reserved=0 >
> > >
> > > One quick thing while I have you: I noticed that the output
netCDF
> > > features an attribute called Projection whose value is "LatLon"
-
> > > would you happen to know if I can specify this for my non-CF
> > > compliant file? No biggie if not, would just save me some
time...
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 15:59
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Tom,
> > >
> > > Glad to hear it! I will close this ticket now, but feel free to
open
> > > a new one if you have any other issues or questions.
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d2a
> > > > 20
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2Bv
> > > > wF
> > > > Sq
> > > > %2Br%2BslCTY%3D&reserved=0 >
> > > >
> > > > Hi again George,
> > > >
> > > > Much as I'm sure you were excited to get your teeth into
MyConfig,
> > > > your initial response was sufficiently helpful that I am now
able
> > > > to run things (I had removed the "level" part of the field
> > > > dictionary, rather than specifying the single level).
> > > >
> > > > Thanks a bunch!
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 15:38
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > I see you are having issues with running MODE using NetCDF
data.
> > > >
> > > > The warning and error messages you are receiving suggests that
the
> > > > field information you are providing in your config file are
> incorrect.
> > > >
> > > > I can't see what you have set inside your MyConfig file, but
the
> > > > default settings for MODE set the fcst ( forecast) dictionary
to:
> > > >
> > > > fcst = {
> > > > field = {
> > > > name = "APCP";
> > > > level = "A03";
> > > > }
> > > >
> > > >
> > > > censor_thresh = [];
> > > > censor_val = [];
> > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > conv_thresh = >=5.0;
> > > > vld_thresh = 0.5;
> > > > filter_attr_name = [];
> > > > filter_attr_thresh = [];
> > > > merge_thresh = >=1.25;
> > > > merge_flag = THRESH;
> > > > }
> > > > obs = fcst;
> > > >
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > gi
> > > > th
> > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abd
> > > > c6
> > > > 00
> > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=
> > > > 04
> > > > %7
> > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0
> > > > d2
> > > > a2
> > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231875%
> > > > 7C
> > > > Un
> > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik
> > > > 1h
> > > > aW
> > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F4I
> > > > b6
> > > > tH
> > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > >
> > > > The last line from the excerpt sets the obs (observation)
> > > > dictionary to the same values as the fcst dictionary. The
field
> > > > dictionary inside fcst tells MET to look for a field called
'APCP'
> > > > at level 'A03.' You will need to set the name value to "hcc"
which
> > > > should remove the warning alerting you that it is using the
first
> > > > field that
> > it finds.
> > > >
> > > > The error message saying "needed 3 arguments for variable hcc,
got 2"
> > > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > > MET can only read 2D data, so you will have to specify which
2D
> > > > volume to read. It looks like the NetCDF files you provided
were
> > > > not attached to the ticket, so I can't see how the data is
stored.
> > > > You can use * for the lat and lon dimensions to get all data
> > > > points and set a number for the 3rd dimension to specify which
> > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > lon) as the dimensions, you can set:
> > > >
> > > > level = "(0,*,*)";
> > > >
> > > > to extract the 2D volume for the first time dimension in the
field.
> > > > If the first dimension is indeed time and it follows CF
> > > > conventions for specifying time, MET even supports supplying
the
> > > > desired time as the value and it will find the appropriate 2D
volume:
> > > >
> > > > level = "(20210504_120000,*,*)";
> > > >
> > > > More information on MET I/O configurations can be found in the
> > > > User's Guide
> > > > here:
> > > >
> > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-co
> > > > nf
> > > > ig
> > > > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > > > uk
> > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff9
> > > > 0e
> > > > 5f
> > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> > > > Lj
> > > > Aw
> > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdat
> > > > a=
> > > > Tt
> > > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > >
> > > > Please let me know if you have any issues getting this
configured.
> > > > The NetCDF files may not have been included because of file
size
> > limits.
> > > > You could either print the headers using ncdump -h and send me
> > > > that information or upload the data to our FTP server (see How
To
> > > > Send Us Data in
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > dt
> > > > ce
> > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-
help-
> > > > de
> > > > sk
> > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4
> > > > a9
> > > > 5b
> > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755
> > > > 73
> > > > 71
> > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu
> > > > Mz
> > > > Ii
> > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWVCV
> > > > W4
> > > > JE
> > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > ).
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > > > Queue: met_help
> > > > > Subject: Debugging MODE
> > > > > Owner: Nobody
> > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a
> > > > > 39
> > > > > cf
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEve
> > > > > Ae
> > > > > fl
> > > > > SN
> > > > > Oeww%3D&reserved=0 >
> > > > >
> > > > >
> > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > satellite data I wish to analyse for my PhD research, and
don't
> > > > > really know how to debug in light of my (most recent) error
> message.
> > > > > (Initially I got errors associated with a lack of CF
conventions
> > > > > in my home-made observation product, so to proceed I am
working
> > > > > with two reanalysis time-steps which I know to be CF
> > > > > compliant.) I have attached the two netCDFs that I am trying
to
> > > > > operate on, as well as my config file. The error message I
get is:
> > > > >
> > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > >
> > > > > This is, I suppose, the problem with using software as a
'black
> box'.
> > > > > If it's immediately clear what the issue is here, that would
be
> > > > > fab
> > > > > - otherwise I'm really only looking for pointers on how to
find
> > > > > the location of the issue in the software so that I can get
my
> > > > > head around the syntax and debug it. Incidentally, I have
tried
> > > > > running MODE between (identical) files with a time
dimension, as
> > > > > opposed to a single step, and get the same error message.
> > > > >
> > > > > Any help much appreciated!
> > > > >
> > > > > Tom
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: Thomas Eldridge
Time: Wed May 05 08:21:32 2021
You're totally right, my mistake - in fact all three files I attached
are incorrect, I was on autopilot. I've attached the correct versions.
The exact syntax of my command is
mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
config/MyConfigNoTime
I've edited the filenames to reflect the path, since otherwise they
would be identical in the attachment.
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 05 May 2021 15:17
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
Thank you for providing the data and config file. Could you share the
command you used to get this error? Also, it looks like you mentioned
a file called MyConfigNoTime but the file you shared is called
MyConfig. I am just checking to make sure I have the correct file to
use for testing.
Thanks,
George
On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd08496%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6jloV
> O9cA7E%3D&reserved=0 >
>
> Hi George,
>
> Progress made; new issue encountered. I have been able to update my
> THIR files to be CF compliant, and have successfully run MODE on
some
> fields; however, on one example (which I hope will be particularly
> insightful RE what MODE does and how to interpret it, since the
> feature is very visible in the data), I am getting the following
error message:
>
> ERROR :
> ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"!
...
> t = nan ERROR :
>
> I've attached my files, which I've renamed based on the directory
> structure I'm using, in case you'd like to take a look. These
feature
> a single time-step, and MyConfigNoTime has been adjusted
accordingly.
> Will keep plugging away to see if I can identify the cause myself.
>
> Thanks,
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 16:33
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> Sounds good. Please don't bang your head for too long!
>
> - George
>
> On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZCx1
> > J%
> > 2BZ7Lluk%3D&reserved=0 >
> >
> > Thanks George, this is all great stuff, I'm sure the links you've
> > provided will cover my needs for now - expect to hear back from me
> > later this week if I keep banging my head against the wall.
> >
> > All the best,
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 16:14
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > The output from the MET tools are a specific NetCDF format defined
> > and read by the MET tools. I am not exactly sure of the criteria
> > used by the tools to determine if a NetCDF file was created by one
of its own.
> > It would likely be easier to do one of two things:
> >
> > 1) Modify the non-CF compliant file to add the required
information
> > to make it CF compliant
> >
> > OR
> >
> > 2) Utilize the Python Embedding functionality to read the NetCDF
> > file
> >
> > The latter involves writing a Python script to read the data, then
> > define a 2D data volume and attributes to define how the data is
> > formatted (time, fields, grid, etc.). To use this functionality,
you
> > need to have the MET tools built with Python enabled and the
> > required
> Python packages available.
> >
> > More information on Python Embedding is here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data
> > =0
> >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f
> > 11
> >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730
> > %7
> >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > k1
> >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifjbo
> > mL
> > xgWzU5tgpA%3D&reserved=0
> >
> > Sample scripts that utilize this logic can be found here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > ce
> > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
anal
> > ys
> > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da29
> > 7e
> >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0
> > %7
> >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > Ij
> >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9lk
> > 65
> > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> >
> > And a specific example that reads NetCDF data from that page is
here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > ce
> > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-s
> > cr
> >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40
> > pg
> >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c
> > 09
> >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb3d
> > 8e
> >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > 10
> >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&rese
> > rv
> > ed=0
> >
> > Let me know if you have any other questions.
> >
> > Thanks,
> > George
> >
> >
> > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50
> > > e7
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%
> > > 2F
> > > nU
> > > fX%2Bkh0%3D&reserved=0 >
> > >
> > > One quick thing while I have you: I noticed that the output
netCDF
> > > features an attribute called Projection whose value is "LatLon"
-
> > > would you happen to know if I can specify this for my non-CF
> > > compliant file? No biggie if not, would just save me some
time...
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 15:59
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Tom,
> > >
> > > Glad to hear it! I will close this ticket now, but feel free to
> > > open a new one if you have any other issues or questions.
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > 4%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d
> > > > 2a
> > > > 20
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7
> > > > CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2
> > > > Bv
> > > > wF
> > > > Sq
> > > > %2Br%2BslCTY%3D&reserved=0 >
> > > >
> > > > Hi again George,
> > > >
> > > > Much as I'm sure you were excited to get your teeth into
> > > > MyConfig, your initial response was sufficiently helpful that
I
> > > > am now able to run things (I had removed the "level" part of
the
> > > > field dictionary, rather than specifying the single level).
> > > >
> > > > Thanks a bunch!
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 15:38
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > I see you are having issues with running MODE using NetCDF
data.
> > > >
> > > > The warning and error messages you are receiving suggests that
> > > > the field information you are providing in your config file
are
> incorrect.
> > > >
> > > > I can't see what you have set inside your MyConfig file, but
the
> > > > default settings for MODE set the fcst ( forecast) dictionary
to:
> > > >
> > > > fcst = {
> > > > field = {
> > > > name = "APCP";
> > > > level = "A03";
> > > > }
> > > >
> > > >
> > > > censor_thresh = [];
> > > > censor_val = [];
> > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > conv_thresh = >=5.0;
> > > > vld_thresh = 0.5;
> > > > filter_attr_name = [];
> > > > filter_attr_thresh = [];
> > > > merge_thresh = >=1.25;
> > > > merge_flag = THRESH;
> > > > }
> > > > obs = fcst;
> > > >
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > gi
> > > > th
> > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734a
> > > > bd
> > > > c6
> > > > 00
> > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&dat
> > > > a=
> > > > 04
> > > > %7
> > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90
> > > > f0
> > > > d2
> > > > a2
> > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755737149223187
> > > > 5%
> > > > 7C
> > > > Un
> > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6
> > > > Ik
> > > > 1h
> > > > aW
> > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F
> > > > 4I
> > > > b6
> > > > tH
> > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > >
> > > > The last line from the excerpt sets the obs (observation)
> > > > dictionary to the same values as the fcst dictionary. The
field
> > > > dictionary inside fcst tells MET to look for a field called
'APCP'
> > > > at level 'A03.' You will need to set the name value to "hcc"
> > > > which should remove the warning alerting you that it is using
> > > > the first field that
> > it finds.
> > > >
> > > > The error message saying "needed 3 arguments for variable hcc,
got 2"
> > > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > > MET can only read 2D data, so you will have to specify which
2D
> > > > volume to read. It looks like the NetCDF files you provided
were
> > > > not attached to the ticket, so I can't see how the data is
stored.
> > > > You can use * for the lat and lon dimensions to get all data
> > > > points and set a number for the 3rd dimension to specify which
> > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > lon) as the dimensions, you can set:
> > > >
> > > > level = "(0,*,*)";
> > > >
> > > > to extract the 2D volume for the first time dimension in the
field.
> > > > If the first dimension is indeed time and it follows CF
> > > > conventions for specifying time, MET even supports supplying
the
> > > > desired time as the value and it will find the appropriate 2D
volume:
> > > >
> > > > level = "(20210504_120000,*,*)";
> > > >
> > > > More information on MET I/O configurations can be found in the
> > > > User's Guide
> > > > here:
> > > >
> > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
> > > > co
> > > > nf
> > > > ig
> > > > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > > > uk
> > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43f
> > > > f9
> > > > 0e
> > > > 5f
> > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> > > > 4w
> > > > Lj
> > > > Aw
> > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sd
> > > > at
> > > > a=
> > > > Tt
> > > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > >
> > > > Please let me know if you have any issues getting this
configured.
> > > > The NetCDF files may not have been included because of file
size
> > limits.
> > > > You could either print the headers using ncdump -h and send me
> > > > that information or upload the data to our FTP server (see How
> > > > To Send Us Data in
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > dt
> > > > ce
> > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-
hel
> > > > p-
> > > > de
> > > > sk
> > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117
> > > > f4
> > > > a9
> > > > 5b
> > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637
> > > > 55
> > > > 73
> > > > 71
> > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2
> > > > lu
> > > > Mz
> > > > Ii
> > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWV
> > > > CV
> > > > W4
> > > > JE
> > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > ).
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > > > Queue: met_help
> > > > > Subject: Debugging MODE
> > > > > Owner: Nobody
> > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > =0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f
> > > > > 0a
> > > > > 39
> > > > > cf
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230
> > > > > %7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > k1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxE
> > > > > ve
> > > > > Ae
> > > > > fl
> > > > > SN
> > > > > Oeww%3D&reserved=0 >
> > > > >
> > > > >
> > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > satellite data I wish to analyse for my PhD research, and
> > > > > don't really know how to debug in light of my (most recent)
> > > > > error
> message.
> > > > > (Initially I got errors associated with a lack of CF
> > > > > conventions in my home-made observation product, so to
proceed
> > > > > I am working with two reanalysis time-steps which I know to
be
> > > > > CF
> > > > > compliant.) I have attached the two netCDFs that I am trying
> > > > > to operate on, as well as my config file. The error message
I get is:
> > > > >
> > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > >
> > > > > This is, I suppose, the problem with using software as a
> > > > > 'black
> box'.
> > > > > If it's immediately clear what the issue is here, that would
> > > > > be fab
> > > > > - otherwise I'm really only looking for pointers on how to
> > > > > find the location of the issue in the software so that I can
> > > > > get my head around the syntax and debug it. Incidentally, I
> > > > > have tried running MODE between (identical) files with a
time
> > > > > dimension, as opposed to a single step, and get the same
error message.
> > > > >
> > > > > Any help much appreciated!
> > > > >
> > > > > Tom
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: Thomas Eldridge
Time: Wed May 12 04:36:58 2021
Hi George,
Any update on this? No mad rush, just hadn't heard back from you and
thought I'd check to see you got my message. FYI I ran MODE on all my
files to see how many cases this is an issue for, and it turns out to
be three (out of 530) - I can't identify anything that these files
have in common so no clue as to the cause of the problem there.
Irritatingly the bug has hit one of my best examples, so would be nice
to have a fix!
Thanks for all your help,
Tom
-----Original Message-----
From: Thomas Eldridge
Sent: 05 May 2021 15:21
To: met_help at ucar.edu
Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
You're totally right, my mistake - in fact all three files I attached
are incorrect, I was on autopilot. I've attached the correct versions.
The exact syntax of my command is
mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
config/MyConfigNoTime
I've edited the filenames to reflect the path, since otherwise they
would be identical in the attachment.
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 05 May 2021 15:17
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
Thank you for providing the data and config file. Could you share the
command you used to get this error? Also, it looks like you mentioned
a file called MyConfigNoTime but the file you shared is called
MyConfig. I am just checking to make sure I have the correct file to
use for testing.
Thanks,
George
On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd08496%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6jloV
> O9cA7E%3D&reserved=0 >
>
> Hi George,
>
> Progress made; new issue encountered. I have been able to update my
> THIR files to be CF compliant, and have successfully run MODE on
some
> fields; however, on one example (which I hope will be particularly
> insightful RE what MODE does and how to interpret it, since the
> feature is very visible in the data), I am getting the following
error message:
>
> ERROR :
> ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"!
...
> t = nan ERROR :
>
> I've attached my files, which I've renamed based on the directory
> structure I'm using, in case you'd like to take a look. These
feature
> a single time-step, and MyConfigNoTime has been adjusted
accordingly.
> Will keep plugging away to see if I can identify the cause myself.
>
> Thanks,
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 04 May 2021 16:33
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> Sounds good. Please don't bang your head for too long!
>
> - George
>
> On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZCx1
> > J%
> > 2BZ7Lluk%3D&reserved=0 >
> >
> > Thanks George, this is all great stuff, I'm sure the links you've
> > provided will cover my needs for now - expect to hear back from me
> > later this week if I keep banging my head against the wall.
> >
> > All the best,
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 16:14
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > The output from the MET tools are a specific NetCDF format defined
> > and read by the MET tools. I am not exactly sure of the criteria
> > used by the tools to determine if a NetCDF file was created by one
of its own.
> > It would likely be easier to do one of two things:
> >
> > 1) Modify the non-CF compliant file to add the required
information
> > to make it CF compliant
> >
> > OR
> >
> > 2) Utilize the Python Embedding functionality to read the NetCDF
> > file
> >
> > The latter involves writing a Python script to read the data, then
> > define a 2D data volume and attributes to define how the data is
> > formatted (time, fields, grid, etc.). To use this functionality,
you
> > need to have the MET tools built with Python enabled and the
> > required
> Python packages available.
> >
> > More information on Python Embedding is here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data
> > =0
> >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f
> > 11
> >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730
> > %7
> >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > k1
> >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifjbo
> > mL
> > xgWzU5tgpA%3D&reserved=0
> >
> > Sample scripts that utilize this logic can be found here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > ce
> > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
anal
> > ys
> > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da29
> > 7e
> >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0
> > %7
> >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > Ij
> >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9lk
> > 65
> > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> >
> > And a specific example that reads NetCDF data from that page is
here:
> >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > ce
> > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-s
> > cr
> >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40
> > pg
> >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c
> > 09
> >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb3d
> > 8e
> >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > 10
> >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&rese
> > rv
> > ed=0
> >
> > Let me know if you have any other questions.
> >
> > Thanks,
> > George
> >
> >
> > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50
> > > e7
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%
> > > 2F
> > > nU
> > > fX%2Bkh0%3D&reserved=0 >
> > >
> > > One quick thing while I have you: I noticed that the output
netCDF
> > > features an attribute called Projection whose value is "LatLon"
-
> > > would you happen to know if I can specify this for my non-CF
> > > compliant file? No biggie if not, would just save me some
time...
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 15:59
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Tom,
> > >
> > > Glad to hear it! I will close this ticket now, but feel free to
> > > open a new one if you have any other issues or questions.
> > >
> > > Thanks,
> > > George
> > >
> > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > 4%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d
> > > > 2a
> > > > 20
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7
> > > > CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2
> > > > Bv
> > > > wF
> > > > Sq
> > > > %2Br%2BslCTY%3D&reserved=0 >
> > > >
> > > > Hi again George,
> > > >
> > > > Much as I'm sure you were excited to get your teeth into
> > > > MyConfig, your initial response was sufficiently helpful that
I
> > > > am now able to run things (I had removed the "level" part of
the
> > > > field dictionary, rather than specifying the single level).
> > > >
> > > > Thanks a bunch!
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 15:38
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > I see you are having issues with running MODE using NetCDF
data.
> > > >
> > > > The warning and error messages you are receiving suggests that
> > > > the field information you are providing in your config file
are
> incorrect.
> > > >
> > > > I can't see what you have set inside your MyConfig file, but
the
> > > > default settings for MODE set the fcst ( forecast) dictionary
to:
> > > >
> > > > fcst = {
> > > > field = {
> > > > name = "APCP";
> > > > level = "A03";
> > > > }
> > > >
> > > >
> > > > censor_thresh = [];
> > > > censor_val = [];
> > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > conv_thresh = >=5.0;
> > > > vld_thresh = 0.5;
> > > > filter_attr_name = [];
> > > > filter_attr_thresh = [];
> > > > merge_thresh = >=1.25;
> > > > merge_flag = THRESH;
> > > > }
> > > > obs = fcst;
> > > >
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > gi
> > > > th
> > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734a
> > > > bd
> > > > c6
> > > > 00
> > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&dat
> > > > a=
> > > > 04
> > > > %7
> > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90
> > > > f0
> > > > d2
> > > > a2
> > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755737149223187
> > > > 5%
> > > > 7C
> > > > Un
> > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6
> > > > Ik
> > > > 1h
> > > > aW
> > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F
> > > > 4I
> > > > b6
> > > > tH
> > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > >
> > > > The last line from the excerpt sets the obs (observation)
> > > > dictionary to the same values as the fcst dictionary. The
field
> > > > dictionary inside fcst tells MET to look for a field called
'APCP'
> > > > at level 'A03.' You will need to set the name value to "hcc"
> > > > which should remove the warning alerting you that it is using
> > > > the first field that
> > it finds.
> > > >
> > > > The error message saying "needed 3 arguments for variable hcc,
got 2"
> > > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > > MET can only read 2D data, so you will have to specify which
2D
> > > > volume to read. It looks like the NetCDF files you provided
were
> > > > not attached to the ticket, so I can't see how the data is
stored.
> > > > You can use * for the lat and lon dimensions to get all data
> > > > points and set a number for the 3rd dimension to specify which
> > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > lon) as the dimensions, you can set:
> > > >
> > > > level = "(0,*,*)";
> > > >
> > > > to extract the 2D volume for the first time dimension in the
field.
> > > > If the first dimension is indeed time and it follows CF
> > > > conventions for specifying time, MET even supports supplying
the
> > > > desired time as the value and it will find the appropriate 2D
volume:
> > > >
> > > > level = "(20210504_120000,*,*)";
> > > >
> > > > More information on MET I/O configurations can be found in the
> > > > User's Guide
> > > > here:
> > > >
> > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
> > > > co
> > > > nf
> > > > ig
> > > > uration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.
> > > > uk
> > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43f
> > > > f9
> > > > 0e
> > > > 5f
> > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> > > > 4w
> > > > Lj
> > > > Aw
> > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sd
> > > > at
> > > > a=
> > > > Tt
> > > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > >
> > > > Please let me know if you have any issues getting this
configured.
> > > > The NetCDF files may not have been included because of file
size
> > limits.
> > > > You could either print the headers using ncdump -h and send me
> > > > that information or upload the data to our FTP server (see How
> > > > To Send Us Data in
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > dt
> > > > ce
> > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fmet-
hel
> > > > p-
> > > > de
> > > > sk
> > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117
> > > > f4
> > > > a9
> > > > 5b
> > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637
> > > > 55
> > > > 73
> > > > 71
> > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2
> > > > lu
> > > > Mz
> > > > Ii
> > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWV
> > > > CV
> > > > W4
> > > > JE
> > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > ).
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> > > > > Queue: met_help
> > > > > Subject: Debugging MODE
> > > > > Owner: Nobody
> > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > Status: new
> > > > > Ticket <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > =0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f
> > > > > 0a
> > > > > 39
> > > > > cf
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230
> > > > > %7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > k1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxE
> > > > > ve
> > > > > Ae
> > > > > fl
> > > > > SN
> > > > > Oeww%3D&reserved=0 >
> > > > >
> > > > >
> > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > satellite data I wish to analyse for my PhD research, and
> > > > > don't really know how to debug in light of my (most recent)
> > > > > error
> message.
> > > > > (Initially I got errors associated with a lack of CF
> > > > > conventions in my home-made observation product, so to
proceed
> > > > > I am working with two reanalysis time-steps which I know to
be
> > > > > CF
> > > > > compliant.) I have attached the two netCDFs that I am trying
> > > > > to operate on, as well as my config file. The error message
I get is:
> > > > >
> > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > >
> > > > > This is, I suppose, the problem with using software as a
> > > > > 'black
> box'.
> > > > > If it's immediately clear what the issue is here, that would
> > > > > be fab
> > > > > - otherwise I'm really only looking for pointers on how to
> > > > > find the location of the issue in the software so that I can
> > > > > get my head around the syntax and debug it. Incidentally, I
> > > > > have tried running MODE between (identical) files with a
time
> > > > > dimension, as opposed to a single step, and get the same
error message.
> > > > >
> > > > > Any help much appreciated!
> > > > >
> > > > > Tom
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Wed May 12 07:22:27 2021
Hi Tom,
I'm very sorry for the delay. I was pulled away from this to take care
of
some unexpected tasks for the METplus 4.0.0 release (which is now
available!) and this ticket was buried in my task list. Thank you for
following up to remind me. I will do some testing with your files and
get
back to you shortly.
- George
On Wed, May 12, 2021 at 4:37 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Hi George,
>
> Any update on this? No mad rush, just hadn't heard back from you and
> thought I'd check to see you got my message. FYI I ran MODE on all
my files
> to see how many cases this is an issue for, and it turns out to be
three
> (out of 530) - I can't identify anything that these files have in
common so
> no clue as to the cause of the problem there. Irritatingly the bug
has hit
> one of my best examples, so would be nice to have a fix!
>
> Thanks for all your help,
>
> Tom
>
> -----Original Message-----
> From: Thomas Eldridge
> Sent: 05 May 2021 15:21
> To: met_help at ucar.edu
> Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
>
> You're totally right, my mistake - in fact all three files I
attached are
> incorrect, I was on autopilot. I've attached the correct versions.
The
> exact syntax of my command is
>
> mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
> config/MyConfigNoTime
>
> I've edited the filenames to reflect the path, since otherwise they
would
> be identical in the attachment.
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 05 May 2021 15:17
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> Thank you for providing the data and config file. Could you share
the
> command you used to get this error? Also, it looks like you
mentioned a
> file called MyConfigNoTime but the file you shared is called
MyConfig. I am
> just checking to make sure I have the correct file to use for
testing.
>
> Thanks,
> George
>
> On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd08496%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6jloV
> > O9cA7E%3D&reserved=0 >
> >
> > Hi George,
> >
> > Progress made; new issue encountered. I have been able to update
my
> > THIR files to be CF compliant, and have successfully run MODE on
some
> > fields; however, on one example (which I hope will be particularly
> > insightful RE what MODE does and how to interpret it, since the
> > feature is very visible in the data), I am getting the following
error
> message:
> >
> > ERROR :
> > ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"!
...
> > t = nan ERROR :
> >
> > I've attached my files, which I've renamed based on the directory
> > structure I'm using, in case you'd like to take a look. These
feature
> > a single time-step, and MyConfigNoTime has been adjusted
accordingly.
> > Will keep plugging away to see if I can identify the cause myself.
> >
> > Thanks,
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 16:33
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > Sounds good. Please don't bang your head for too long!
> >
> > - George
> >
> > On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZCx1
> > > J%
> > > 2BZ7Lluk%3D&reserved=0 >
> > >
> > > Thanks George, this is all great stuff, I'm sure the links
you've
> > > provided will cover my needs for now - expect to hear back from
me
> > > later this week if I keep banging my head against the wall.
> > >
> > > All the best,
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 16:14
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > The output from the MET tools are a specific NetCDF format
defined
> > > and read by the MET tools. I am not exactly sure of the criteria
> > > used by the tools to determine if a NetCDF file was created by
one of
> its own.
> > > It would likely be easier to do one of two things:
> > >
> > > 1) Modify the non-CF compliant file to add the required
information
> > > to make it CF compliant
> > >
> > > OR
> > >
> > > 2) Utilize the Python Embedding functionality to read the NetCDF
> > > file
> > >
> > > The latter involves writing a Python script to read the data,
then
> > > define a 2D data volume and attributes to define how the data is
> > > formatted (time, fields, grid, etc.). To use this functionality,
you
> > > need to have the MET tools built with Python enabled and the
> > > required
> > Python packages available.
> > >
> > > More information on Python Embedding is here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&data
> > > =0
> > >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f
> > > 11
> > >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730
> > > %7
> > >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > k1
> > >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifjbo
> > > mL
> > > xgWzU5tgpA%3D&reserved=0
> > >
> > > Sample scripts that utilize this logic can be found here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > > ce
> > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
anal
> > > ys
> > > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da29
> > > 7e
> > >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0
> > > %7
> > >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > Ij
> > >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9lk
> > > 65
> > > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> > >
> > > And a specific example that reads NetCDF data from that page is
here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdt
> > > ce
> > > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython-s
> > > cr
> > >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%40
> > > pg
> > >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c
> > > 09
> > >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb3d
> > > 8e
> > >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > > 10
> > >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&rese
> > > rv
> > > ed=0
> > >
> > > Let me know if you have any other questions.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f50
> > > > e7
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY7%
> > > > 2F
> > > > nU
> > > > fX%2Bkh0%3D&reserved=0 >
> > > >
> > > > One quick thing while I have you: I noticed that the output
netCDF
> > > > features an attribute called Projection whose value is
"LatLon" -
> > > > would you happen to know if I can specify this for my non-CF
> > > > compliant file? No biggie if not, would just save me some
time...
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 15:59
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Tom,
> > > >
> > > > Glad to hear it! I will close this ticket now, but feel free
to
> > > > open a new one if you have any other issues or questions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f0d
> > > > > 2a
> > > > > 20
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918%7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK%2
> > > > > Bv
> > > > > wF
> > > > > Sq
> > > > > %2Br%2BslCTY%3D&reserved=0 >
> > > > >
> > > > > Hi again George,
> > > > >
> > > > > Much as I'm sure you were excited to get your teeth into
> > > > > MyConfig, your initial response was sufficiently helpful
that I
> > > > > am now able to run things (I had removed the "level" part of
the
> > > > > field dictionary, rather than specifying the single level).
> > > > >
> > > > > Thanks a bunch!
> > > > >
> > > > > Tom
> > > > >
> > > > > -----Original Message-----
> > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > Sent: 04 May 2021 15:38
> > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > Cc: met_help at mailman.ucar.edu
> > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > >
> > > > > Hi Tom,
> > > > >
> > > > > I see you are having issues with running MODE using NetCDF
data.
> > > > >
> > > > > The warning and error messages you are receiving suggests
that
> > > > > the field information you are providing in your config file
are
> > incorrect.
> > > > >
> > > > > I can't see what you have set inside your MyConfig file, but
the
> > > > > default settings for MODE set the fcst ( forecast)
dictionary to:
> > > > >
> > > > > fcst = {
> > > > > field = {
> > > > > name = "APCP";
> > > > > level = "A03";
> > > > > }
> > > > >
> > > > >
> > > > > censor_thresh = [];
> > > > > censor_val = [];
> > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > conv_thresh = >=5.0;
> > > > > vld_thresh = 0.5;
> > > > > filter_attr_name = [];
> > > > > filter_attr_thresh = [];
> > > > > merge_thresh = >=1.25;
> > > > > merge_flag = THRESH;
> > > > > }
> > > > > obs = fcst;
> > > > >
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > gi
> > > > > th
> > > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734a
> > > > > bd
> > > > > c6
> > > > > 00
> > > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&dat
> > > > > a=
> > > > > 04
> > > > > %7
> > > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90
> > > > > f0
> > > > > d2
> > > > > a2
> > > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755737149223187
> > > > > 5%
> > > > > 7C
> > > > > Un
> > > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6
> > > > > Ik
> > > > > 1h
> > > > > aW
> > > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%2F
> > > > > 4I
> > > > > b6
> > > > > tH
> > > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > > >
> > > > > The last line from the excerpt sets the obs (observation)
> > > > > dictionary to the same values as the fcst dictionary. The
field
> > > > > dictionary inside fcst tells MET to look for a field called
'APCP'
> > > > > at level 'A03.' You will need to set the name value to "hcc"
> > > > > which should remove the warning alerting you that it is
using
> > > > > the first field that
> > > it finds.
> > > > >
> > > > > The error message saying "needed 3 arguments for variable
hcc, got
> 2"
> > > > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > > > MET can only read 2D data, so you will have to specify which
2D
> > > > > volume to read. It looks like the NetCDF files you provided
were
> > > > > not attached to the ticket, so I can't see how the data is
stored.
> > > > > You can use * for the lat and lon dimensions to get all data
> > > > > points and set a number for the 3rd dimension to specify
which
> > > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > > lon) as the dimensions, you can set:
> > > > >
> > > > > level = "(0,*,*)";
> > > > >
> > > > > to extract the 2D volume for the first time dimension in the
field.
> > > > > If the first dimension is indeed time and it follows CF
> > > > > conventions for specifying time, MET even supports supplying
the
> > > > > desired time as the value and it will find the appropriate
2D
> volume:
> > > > >
> > > > > level = "(20210504_120000,*,*)";
> > > > >
> > > > > More information on MET I/O configurations can be found in
the
> > > > > User's Guide
> > > > > here:
> > > > >
> > > > >
> > > > >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
> > > > > co
> > > > > nf
> > > > > ig
> > > > > uration-file-options&data=04%7C01%7CT.Eldridge%
> 40pgr.reading.ac.
> > > > > uk
> > > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43f
> > > > > f9
> > > > > 0e
> > > > > 5f
> > > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> > > > > 4w
> > > > > Lj
> > > > > Aw
> > > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sd
> > > > > at
> > > > > a=
> > > > > Tt
> > > > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > > >
> > > > > Please let me know if you have any issues getting this
configured.
> > > > > The NetCDF files may not have been included because of file
size
> > > limits.
> > > > > You could either print the headers using ncdump -h and send
me
> > > > > that information or upload the data to our FTP server (see
How
> > > > > To Send Us Data in
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > dt
> > > > > ce
> > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fmet-hel
> > > > > p-
> > > > > de
> > > > > sk
> > > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117
> > > > > f4
> > > > > a9
> > > > > 5b
> > > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637
> > > > > 55
> > > > > 73
> > > > > 71
> > > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2
> > > > > lu
> > > > > Mz
> > > > > Ii
> > > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACsWV
> > > > > CV
> > > > > W4
> > > > > JE
> > > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > > ).
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > > Transaction: Ticket created by
T.Eldridge at pgr.reading.ac.uk
> > > > > > Queue: met_help
> > > > > > Subject: Debugging MODE
> > > > > > Owner: Nobody
> > > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > > F%
> > > > > > 2F
> > > > > > rt
> > > > > > .r
> > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > > =0
> > > > > > 4%
> > > > > > 7C
> > > > > > 01
> > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f
> > > > > > 0a
> > > > > > 39
> > > > > > cf
> > > > > > %7
> > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230
> > > > > > %7
> > > > > > CU
> > > > > > nk
> > > > > > no
> > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > > k1
> > > > > > ha
> > > > > > Ww
> > > > > > iL
> > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxE
> > > > > > ve
> > > > > > Ae
> > > > > > fl
> > > > > > SN
> > > > > > Oeww%3D&reserved=0 >
> > > > > >
> > > > > >
> > > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > > satellite data I wish to analyse for my PhD research, and
> > > > > > don't really know how to debug in light of my (most
recent)
> > > > > > error
> > message.
> > > > > > (Initially I got errors associated with a lack of CF
> > > > > > conventions in my home-made observation product, so to
proceed
> > > > > > I am working with two reanalysis time-steps which I know
to be
> > > > > > CF
> > > > > > compliant.) I have attached the two netCDFs that I am
trying
> > > > > > to operate on, as well as my config file. The error
message I
> get is:
> > > > > >
> > > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > > >
> > > > > > This is, I suppose, the problem with using software as a
> > > > > > 'black
> > box'.
> > > > > > If it's immediately clear what the issue is here, that
would
> > > > > > be fab
> > > > > > - otherwise I'm really only looking for pointers on how to
> > > > > > find the location of the issue in the software so that I
can
> > > > > > get my head around the syntax and debug it. Incidentally,
I
> > > > > > have tried running MODE between (identical) files with a
time
> > > > > > dimension, as opposed to a single step, and get the same
error
> message.
> > > > > >
> > > > > > Any help much appreciated!
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III National Center for
> > > > > Atmospheric Research Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to reply to this email outside of your normal
working
> hours.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: FW: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Wed May 12 07:47:25 2021
Thanks George, like I said don't hurry, just wanted to make sure you'd
seen it. Congratulations on the new release! Looking forward to
hearing from you when you have a chance.
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 12 May 2021 14:22
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
I'm very sorry for the delay. I was pulled away from this to take care
of some unexpected tasks for the METplus 4.0.0 release (which is now
available!) and this ticket was buried in my task list. Thank you for
following up to remind me. I will do some testing with your files and
get back to you shortly.
- George
On Wed, May 12, 2021 at 4:37 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C262ff8ff522d4aea517508d91548ff27%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564225534655556%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=nJhG9yKC%2F3RQFhfD0v4Eh%2BtqHMHOxFDrnXw
> W%2FjyjQFc%3D&reserved=0 >
>
> Hi George,
>
> Any update on this? No mad rush, just hadn't heard back from you and
> thought I'd check to see you got my message. FYI I ran MODE on all
my
> files to see how many cases this is an issue for, and it turns out
to
> be three (out of 530) - I can't identify anything that these files
> have in common so no clue as to the cause of the problem there.
> Irritatingly the bug has hit one of my best examples, so would be
nice to have a fix!
>
> Thanks for all your help,
>
> Tom
>
> -----Original Message-----
> From: Thomas Eldridge
> Sent: 05 May 2021 15:21
> To: met_help at ucar.edu
> Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
>
> You're totally right, my mistake - in fact all three files I
attached
> are incorrect, I was on autopilot. I've attached the correct
versions.
> The exact syntax of my command is
>
> mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
> config/MyConfigNoTime
>
> I've edited the filenames to reflect the path, since otherwise they
> would be identical in the attachment.
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 05 May 2021 15:17
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> Thank you for providing the data and config file. Could you share
the
> command you used to get this error? Also, it looks like you
mentioned
> a file called MyConfigNoTime but the file you shared is called
> MyConfig. I am just checking to make sure I have the correct file to
use for testing.
>
> Thanks,
> George
>
> On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd08496
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6jl
> > oV
> > O9cA7E%3D&reserved=0 >
> >
> > Hi George,
> >
> > Progress made; new issue encountered. I have been able to update
my
> > THIR files to be CF compliant, and have successfully run MODE on
> > some fields; however, on one example (which I hope will be
> > particularly insightful RE what MODE does and how to interpret it,
> > since the feature is very visible in the data), I am getting the
> > following error
> message:
> >
> > ERROR :
> > ERROR : PiecewiseLinear::operator()(double) -> error in "(nul)"!
...
> > t = nan ERROR :
> >
> > I've attached my files, which I've renamed based on the directory
> > structure I'm using, in case you'd like to take a look. These
> > feature a single time-step, and MyConfigNoTime has been adjusted
accordingly.
> > Will keep plugging away to see if I can identify the cause myself.
> >
> > Thanks,
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 04 May 2021 16:33
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > Sounds good. Please don't bang your head for too long!
> >
> > - George
> >
> > On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11ee
> > > cc
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZC
> > > x1
> > > J%
> > > 2BZ7Lluk%3D&reserved=0 >
> > >
> > > Thanks George, this is all great stuff, I'm sure the links
you've
> > > provided will cover my needs for now - expect to hear back from
me
> > > later this week if I keep banging my head against the wall.
> > >
> > > All the best,
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 16:14
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > The output from the MET tools are a specific NetCDF format
defined
> > > and read by the MET tools. I am not exactly sure of the criteria
> > > used by the tools to determine if a NetCDF file was created by
one
> > > of
> its own.
> > > It would likely be easier to do one of two things:
> > >
> > > 1) Modify the non-CF compliant file to add the required
> > > information to make it CF compliant
> > >
> > > OR
> > >
> > > 2) Utilize the Python Embedding functionality to read the NetCDF
> > > file
> > >
> > > The latter involves writing a Python script to read the data,
then
> > > define a 2D data volume and attributes to define how the data is
> > > formatted (time, fields, grid, etc.). To use this functionality,
> > > you need to have the MET tools built with Python enabled and the
> > > required
> > Python packages available.
> > >
> > > More information on Python Embedding is here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&da
> > > ta
> > > =0
> > >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d9
> > > 0f
> > > 11
> > >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573919735317
> > > 30
> > > %7
> > >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > 6I
> > > k1
> > >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifj
> > > bo
> > > mL
> > > xgWzU5tgpA%3D&reserved=0
> > >
> > > Sample scripts that utilize this logic can be found here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > dt
> > > ce
> > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-met%2Fsample-
an
> > > al
> > > ys
> > > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da
> > > 29
> > > 7e
> > >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7
> > > C0
> > > %7
> > >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> > > JQ
> > > Ij
> > >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9
> > > lk
> > > 65
> > > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> > >
> > > And a specific example that reads NetCDF data from that page is
here:
> > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > dt
> > > ce
> > > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython
> > > -s
> > > cr
> > >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%
> > > 40
> > > pg
> > >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc4
> > > 8c
> > > 09
> > >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb
> > > 3d
> > > 8e
> > >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%
> > > 7C
> > > 10
> > >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&re
> > > se
> > > rv
> > > ed=0
> > >
> > > Let me know if you have any other questions.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > 4%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f
> > > > 50
> > > > e7
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7
> > > > CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY
> > > > 7%
> > > > 2F
> > > > nU
> > > > fX%2Bkh0%3D&reserved=0 >
> > > >
> > > > One quick thing while I have you: I noticed that the output
> > > > netCDF features an attribute called Projection whose value is
> > > > "LatLon" - would you happen to know if I can specify this for
my
> > > > non-CF compliant file? No biggie if not, would just save me
some time...
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 15:59
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Tom,
> > > >
> > > > Glad to hear it! I will close this ticket now, but feel free
to
> > > > open a new one if you have any other issues or questions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > =0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f
> > > > > 0d
> > > > > 2a
> > > > > 20
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918
> > > > > %7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > k1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK
> > > > > %2
> > > > > Bv
> > > > > wF
> > > > > Sq
> > > > > %2Br%2BslCTY%3D&reserved=0 >
> > > > >
> > > > > Hi again George,
> > > > >
> > > > > Much as I'm sure you were excited to get your teeth into
> > > > > MyConfig, your initial response was sufficiently helpful
that
> > > > > I am now able to run things (I had removed the "level" part
of
> > > > > the field dictionary, rather than specifying the single
level).
> > > > >
> > > > > Thanks a bunch!
> > > > >
> > > > > Tom
> > > > >
> > > > > -----Original Message-----
> > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > Sent: 04 May 2021 15:38
> > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > Cc: met_help at mailman.ucar.edu
> > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > >
> > > > > Hi Tom,
> > > > >
> > > > > I see you are having issues with running MODE using NetCDF
data.
> > > > >
> > > > > The warning and error messages you are receiving suggests
that
> > > > > the field information you are providing in your config file
> > > > > are
> > incorrect.
> > > > >
> > > > > I can't see what you have set inside your MyConfig file, but
> > > > > the default settings for MODE set the fcst ( forecast)
dictionary to:
> > > > >
> > > > > fcst = {
> > > > > field = {
> > > > > name = "APCP";
> > > > > level = "A03";
> > > > > }
> > > > >
> > > > >
> > > > > censor_thresh = [];
> > > > > censor_val = [];
> > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > conv_thresh = >=5.0;
> > > > > vld_thresh = 0.5;
> > > > > filter_attr_name = [];
> > > > > filter_attr_thresh = [];
> > > > > merge_thresh = >=1.25;
> > > > > merge_flag = THRESH;
> > > > > }
> > > > > obs = fcst;
> > > > >
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > gi
> > > > > th
> > > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa15073
> > > > > 4a
> > > > > bd
> > > > > c6
> > > > > 00
> > > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&d
> > > > > at
> > > > > a=
> > > > > 04
> > > > > %7
> > > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d
> > > > > 90
> > > > > f0
> > > > > d2
> > > > > a2
> > > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231
> > > > > 87
> > > > > 5%
> > > > > 7C
> > > > > Un
> > > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > > > > I6
> > > > > Ik
> > > > > 1h
> > > > > aW
> > > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%
> > > > > 2F
> > > > > 4I
> > > > > b6
> > > > > tH
> > > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > > >
> > > > > The last line from the excerpt sets the obs (observation)
> > > > > dictionary to the same values as the fcst dictionary. The
> > > > > field dictionary inside fcst tells MET to look for a field
called 'APCP'
> > > > > at level 'A03.' You will need to set the name value to "hcc"
> > > > > which should remove the warning alerting you that it is
using
> > > > > the first field that
> > > it finds.
> > > > >
> > > > > The error message saying "needed 3 arguments for variable
hcc,
> > > > > got
> 2"
> > > > > implies that the hcc field is a NetCDF file that has 3
dimensions.
> > > > > MET can only read 2D data, so you will have to specify which
> > > > > 2D volume to read. It looks like the NetCDF files you
provided
> > > > > were not attached to the ticket, so I can't see how the data
is stored.
> > > > > You can use * for the lat and lon dimensions to get all data
> > > > > points and set a number for the 3rd dimension to specify
which
> > > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > > lon) as the dimensions, you can set:
> > > > >
> > > > > level = "(0,*,*)";
> > > > >
> > > > > to extract the 2D volume for the first time dimension in the
field.
> > > > > If the first dimension is indeed time and it follows CF
> > > > > conventions for specifying time, MET even supports supplying
> > > > > the desired time as the value and it will find the
appropriate
> > > > > 2D
> volume:
> > > > >
> > > > > level = "(20210504_120000,*,*)";
> > > > >
> > > > > More information on MET I/O configurations can be found in
the
> > > > > User's Guide
> > > > > here:
> > > > >
> > > > >
> > > > >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23me
> > > > > t-
> > > > > co
> > > > > nf
> > > > > ig
> > > > > uration-file-options&data=04%7C01%7CT.Eldridge%
> 40pgr.reading.ac.
> > > > > uk
> > > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e4
> > > > > 3f
> > > > > f9
> > > > > 0e
> > > > > 5f
> > > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi
> > > > > MC
> > > > > 4w
> > > > > Lj
> > > > > Aw
> > > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&
> > > > > sd
> > > > > at
> > > > > a=
> > > > > Tt
> > > > > GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > > >
> > > > > Please let me know if you have any issues getting this
configured.
> > > > > The NetCDF files may not have been included because of file
> > > > > size
> > > limits.
> > > > > You could either print the headers using ncdump -h and send
me
> > > > > that information or upload the data to our FTP server (see
How
> > > > > To Send Us Data in
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > dt
> > > > > ce
> > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fmet-h
> > > > > el
> > > > > p-
> > > > > de
> > > > > sk
> > > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa7271
> > > > > 17
> > > > > f4
> > > > > a9
> > > > > 5b
> > > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6
> > > > > 37
> > > > > 55
> > > > > 73
> > > > > 71
> > > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > > > > V2
> > > > > lu
> > > > > Mz
> > > > > Ii
> > > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACs
> > > > > WV
> > > > > CV
> > > > > W4
> > > > > JE
> > > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > > ).
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > > Transaction: Ticket created by
T.Eldridge at pgr.reading.ac.uk
> > > > > > Queue: met_help
> > > > > > Subject: Debugging MODE
> > > > > > Owner: Nobody
> > > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > %2
> > > > > > F%
> > > > > > 2F
> > > > > > rt
> > > > > > .r
> > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&da
> > > > > > ta
> > > > > > =0
> > > > > > 4%
> > > > > > 7C
> > > > > > 01
> > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d9
> > > > > > 0f
> > > > > > 0a
> > > > > > 39
> > > > > > cf
> > > > > > %7
> > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573588586752
> > > > > > 30
> > > > > > %7
> > > > > > CU
> > > > > > nk
> > > > > > no
> > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > > > > 6I
> > > > > > k1
> > > > > > ha
> > > > > > Ww
> > > > > > iL
> > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlW
> > > > > > xE
> > > > > > ve
> > > > > > Ae
> > > > > > fl
> > > > > > SN
> > > > > > Oeww%3D&reserved=0 >
> > > > > >
> > > > > >
> > > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > > satellite data I wish to analyse for my PhD research, and
> > > > > > don't really know how to debug in light of my (most
recent)
> > > > > > error
> > message.
> > > > > > (Initially I got errors associated with a lack of CF
> > > > > > conventions in my home-made observation product, so to
> > > > > > proceed I am working with two reanalysis time-steps which
I
> > > > > > know to be CF
> > > > > > compliant.) I have attached the two netCDFs that I am
trying
> > > > > > to operate on, as well as my config file. The error
message
> > > > > > I
> get is:
> > > > > >
> > > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > > >
> > > > > > This is, I suppose, the problem with using software as a
> > > > > > 'black
> > box'.
> > > > > > If it's immediately clear what the issue is here, that
would
> > > > > > be fab
> > > > > > - otherwise I'm really only looking for pointers on how to
> > > > > > find the location of the issue in the software so that I
can
> > > > > > get my head around the syntax and debug it. Incidentally,
I
> > > > > > have tried running MODE between (identical) files with a
> > > > > > time dimension, as opposed to a single step, and get the
> > > > > > same error
> message.
> > > > > >
> > > > > > Any help much appreciated!
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III National Center for
> > > > > Atmospheric Research Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to reply to this email outside of your normal
working
> hours.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: John Halley Gotway
Time: Wed May 12 09:39:20 2021
Hi Tom,
This is John Halley Gotway. I work with George on providing MET-Help
support. I ran the sample data you sent via MODE and was able to track
down
the problem to the convolution radius setting.
conv_radius = [ 60.0/grid_res, 60.0/grid_res, 60.0/grid_res ];
I realize why you used that... since it's the default in the config
file!
In your case, with grid_res = 25, that works out to a value of 2.4.
And
MODE actually parse that as an integer... so it's a value of 2. When I
run
with exactly 2 (conv_radius = [ 2, 2, 2 ];), I get the same error
message.
However, running with a radius of 0 (to apply no smoothing), 3 (to
apply a
very small amount of smoothing), or a larger number for more
smoothing, it
runs just fine.
There's something about a convolution radius of 2 which makes the
logic
breakdown.
That's good news for you since it's easy to to change the conv_radius
setting. But I'll try to dig into the details to determine where the
logic's breaking down and hopefully add some more checks instead of
aborting.
Thanks,
John
On Wed, May 12, 2021 at 7:47 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Thanks George, like I said don't hurry, just wanted to make sure
you'd
> seen it. Congratulations on the new release! Looking forward to
hearing
> from you when you have a chance.
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 12 May 2021 14:22
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> I'm very sorry for the delay. I was pulled away from this to take
care of
> some unexpected tasks for the METplus 4.0.0 release (which is now
> available!) and this ticket was buried in my task list. Thank you
for
> following up to remind me. I will do some testing with your files
and get
> back to you shortly.
>
> - George
>
> On Wed, May 12, 2021 at 4:37 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C262ff8ff522d4aea517508d91548ff27%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564225534655556%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=nJhG9yKC%2F3RQFhfD0v4Eh%2BtqHMHOxFDrnXw
> > W%2FjyjQFc%3D&reserved=0 >
> >
> > Hi George,
> >
> > Any update on this? No mad rush, just hadn't heard back from you
and
> > thought I'd check to see you got my message. FYI I ran MODE on all
my
> > files to see how many cases this is an issue for, and it turns out
to
> > be three (out of 530) - I can't identify anything that these files
> > have in common so no clue as to the cause of the problem there.
> > Irritatingly the bug has hit one of my best examples, so would be
nice
> to have a fix!
> >
> > Thanks for all your help,
> >
> > Tom
> >
> > -----Original Message-----
> > From: Thomas Eldridge
> > Sent: 05 May 2021 15:21
> > To: met_help at ucar.edu
> > Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > You're totally right, my mistake - in fact all three files I
attached
> > are incorrect, I was on autopilot. I've attached the correct
versions.
> > The exact syntax of my command is
> >
> > mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
> > config/MyConfigNoTime
> >
> > I've edited the filenames to reflect the path, since otherwise
they
> > would be identical in the attachment.
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 05 May 2021 15:17
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > Thank you for providing the data and config file. Could you share
the
> > command you used to get this error? Also, it looks like you
mentioned
> > a file called MyConfigNoTime but the file you shared is called
> > MyConfig. I am just checking to make sure I have the correct file
to use
> for testing.
> >
> > Thanks,
> > George
> >
> > On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd08496
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6jl
> > > oV
> > > O9cA7E%3D&reserved=0 >
> > >
> > > Hi George,
> > >
> > > Progress made; new issue encountered. I have been able to update
my
> > > THIR files to be CF compliant, and have successfully run MODE on
> > > some fields; however, on one example (which I hope will be
> > > particularly insightful RE what MODE does and how to interpret
it,
> > > since the feature is very visible in the data), I am getting the
> > > following error
> > message:
> > >
> > > ERROR :
> > > ERROR : PiecewiseLinear::operator()(double) -> error in
"(nul)"! ...
> > > t = nan ERROR :
> > >
> > > I've attached my files, which I've renamed based on the
directory
> > > structure I'm using, in case you'd like to take a look. These
> > > feature a single time-step, and MyConfigNoTime has been adjusted
> accordingly.
> > > Will keep plugging away to see if I can identify the cause
myself.
> > >
> > > Thanks,
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 16:33
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > Sounds good. Please don't bang your head for too long!
> > >
> > > - George
> > >
> > > On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11ee
> > > > cc
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIqZC
> > > > x1
> > > > J%
> > > > 2BZ7Lluk%3D&reserved=0 >
> > > >
> > > > Thanks George, this is all great stuff, I'm sure the links
you've
> > > > provided will cover my needs for now - expect to hear back
from me
> > > > later this week if I keep banging my head against the wall.
> > > >
> > > > All the best,
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 16:14
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > The output from the MET tools are a specific NetCDF format
defined
> > > > and read by the MET tools. I am not exactly sure of the
criteria
> > > > used by the tools to determine if a NetCDF file was created by
one
> > > > of
> > its own.
> > > > It would likely be easier to do one of two things:
> > > >
> > > > 1) Modify the non-CF compliant file to add the required
> > > > information to make it CF compliant
> > > >
> > > > OR
> > > >
> > > > 2) Utilize the Python Embedding functionality to read the
NetCDF
> > > > file
> > > >
> > > > The latter involves writing a Python script to read the data,
then
> > > > define a 2D data volume and attributes to define how the data
is
> > > > formatted (time, fields, grid, etc.). To use this
functionality,
> > > > you need to have the MET tools built with Python enabled and
the
> > > > required
> > > Python packages available.
> > > >
> > > > More information on Python Embedding is here:
> > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&da
> > > > ta
> > > > =0
> > > >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d9
> > > > 0f
> > > > 11
> > > >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573919735317
> > > > 30
> > > > %7
> > > >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > > 6I
> > > > k1
> > > >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2ifj
> > > > bo
> > > > mL
> > > > xgWzU5tgpA%3D&reserved=0
> > > >
> > > > Sample scripts that utilize this logic can be found here:
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > dt
> > > > ce
> > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fsample-an
> > > > al
> > > > ys
> > > > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da
> > > > 29
> > > > 7e
> > > >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7
> > > > C0
> > > > %7
> > > >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> > > > JQ
> > > > Ij
> > > >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2B9
> > > > lk
> > > > 65
> > > > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> > > >
> > > > And a specific example that reads NetCDF data from that page
is here:
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > dt
> > > > ce
> > > > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpython
> > > > -s
> > > > cr
> > > >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridge%
> > > > 40
> > > > pg
> > > >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecfc4
> > > > 8c
> > > > 09
> > > >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZsb
> > > > 3d
> > > > 8e
> > > >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%
> > > > 7C
> > > > 10
> > > >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&re
> > > > se
> > > > rv
> > > > ed=0
> > > >
> > > > Let me know if you have any other questions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f0f
> > > > > 50
> > > > > e7
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825%7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkThY
> > > > > 7%
> > > > > 2F
> > > > > nU
> > > > > fX%2Bkh0%3D&reserved=0 >
> > > > >
> > > > > One quick thing while I have you: I noticed that the output
> > > > > netCDF features an attribute called Projection whose value
is
> > > > > "LatLon" - would you happen to know if I can specify this
for my
> > > > > non-CF compliant file? No biggie if not, would just save me
some
> time...
> > > > >
> > > > > Tom
> > > > >
> > > > > -----Original Message-----
> > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > Sent: 04 May 2021 15:59
> > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > Cc: met_help at mailman.ucar.edu
> > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > >
> > > > > Tom,
> > > > >
> > > > > Glad to hear it! I will close this ticket now, but feel free
to
> > > > > open a new one if you have any other issues or questions.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > > F%
> > > > > > 2F
> > > > > > rt
> > > > > > .r
> > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > > =0
> > > > > > 4%
> > > > > > 7C
> > > > > > 01
> > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d90f
> > > > > > 0d
> > > > > > 2a
> > > > > > 20
> > > > > > %7
> > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492221918
> > > > > > %7
> > > > > > CU
> > > > > > nk
> > > > > > no
> > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > > k1
> > > > > > ha
> > > > > > Ww
> > > > > > iL
> > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBXQK
> > > > > > %2
> > > > > > Bv
> > > > > > wF
> > > > > > Sq
> > > > > > %2Br%2BslCTY%3D&reserved=0 >
> > > > > >
> > > > > > Hi again George,
> > > > > >
> > > > > > Much as I'm sure you were excited to get your teeth into
> > > > > > MyConfig, your initial response was sufficiently helpful
that
> > > > > > I am now able to run things (I had removed the "level"
part of
> > > > > > the field dictionary, rather than specifying the single
level).
> > > > > >
> > > > > > Thanks a bunch!
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > > Sent: 04 May 2021 15:38
> > > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > > Cc: met_help at mailman.ucar.edu
> > > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > > >
> > > > > > Hi Tom,
> > > > > >
> > > > > > I see you are having issues with running MODE using NetCDF
data.
> > > > > >
> > > > > > The warning and error messages you are receiving suggests
that
> > > > > > the field information you are providing in your config
file
> > > > > > are
> > > incorrect.
> > > > > >
> > > > > > I can't see what you have set inside your MyConfig file,
but
> > > > > > the default settings for MODE set the fcst ( forecast)
> dictionary to:
> > > > > >
> > > > > > fcst = {
> > > > > > field = {
> > > > > > name = "APCP";
> > > > > > level = "A03";
> > > > > > }
> > > > > >
> > > > > >
> > > > > > censor_thresh = [];
> > > > > > censor_val = [];
> > > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > > conv_thresh = >=5.0;
> > > > > > vld_thresh = 0.5;
> > > > > > filter_attr_name = [];
> > > > > > filter_attr_thresh = [];
> > > > > > merge_thresh = >=1.25;
> > > > > > merge_flag = THRESH;
> > > > > > }
> > > > > > obs = fcst;
> > > > > >
> > > > > >
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > > F%
> > > > > > 2F
> > > > > > gi
> > > > > > th
> > > > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa15073
> > > > > > 4a
> > > > > > bd
> > > > > > c6
> > > > > > 00
> > > > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&d
> > > > > > at
> > > > > > a=
> > > > > > 04
> > > > > > %7
> > > > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d
> > > > > > 90
> > > > > > f0
> > > > > > d2
> > > > > > a2
> > > > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557371492231
> > > > > > 87
> > > > > > 5%
> > > > > > 7C
> > > > > > Un
> > > > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > > > > > I6
> > > > > > Ik
> > > > > > 1h
> > > > > > aW
> > > > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPHn%
> > > > > > 2F
> > > > > > 4I
> > > > > > b6
> > > > > > tH
> > > > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > > > >
> > > > > > The last line from the excerpt sets the obs (observation)
> > > > > > dictionary to the same values as the fcst dictionary. The
> > > > > > field dictionary inside fcst tells MET to look for a field
> called 'APCP'
> > > > > > at level 'A03.' You will need to set the name value to
"hcc"
> > > > > > which should remove the warning alerting you that it is
using
> > > > > > the first field that
> > > > it finds.
> > > > > >
> > > > > > The error message saying "needed 3 arguments for variable
hcc,
> > > > > > got
> > 2"
> > > > > > implies that the hcc field is a NetCDF file that has 3
> dimensions.
> > > > > > MET can only read 2D data, so you will have to specify
which
> > > > > > 2D volume to read. It looks like the NetCDF files you
provided
> > > > > > were not attached to the ticket, so I can't see how the
data is
> stored.
> > > > > > You can use * for the lat and lon dimensions to get all
data
> > > > > > points and set a number for the 3rd dimension to specify
which
> > > > > > slab of data to pass in. If the hcc field has (time, lat,
> > > > > > lon) as the dimensions, you can set:
> > > > > >
> > > > > > level = "(0,*,*)";
> > > > > >
> > > > > > to extract the 2D volume for the first time dimension in
the
> field.
> > > > > > If the first dimension is indeed time and it follows CF
> > > > > > conventions for specifying time, MET even supports
supplying
> > > > > > the desired time as the value and it will find the
appropriate
> > > > > > 2D
> > volume:
> > > > > >
> > > > > > level = "(20210504_120000,*,*)";
> > > > > >
> > > > > > More information on MET I/O configurations can be found in
the
> > > > > > User's Guide
> > > > > > here:
> > > > > >
> > > > > >
> > > > > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23me
> > > > > > t-
> > > > > > co
> > > > > > nf
> > > > > > ig
> > > > > > uration-file-options&data=04%7C01%7CT.Eldridge%
> > 40pgr.reading.ac.
> > > > > > uk
> > > > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e4
> > > > > > 3f
> > > > > > f9
> > > > > > 0e
> > > > > > 5f
> > > > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi
> > > > > > MC
> > > > > > 4w
> > > > > > Lj
> > > > > > Aw
> > > > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&
> > > > > > sd
> > > > > > at
> > > > > > a=
> > > > > > Tt
> > > > > >
GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > > > >
> > > > > > Please let me know if you have any issues getting this
> configured.
> > > > > > The NetCDF files may not have been included because of
file
> > > > > > size
> > > > limits.
> > > > > > You could either print the headers using ncdump -h and
send me
> > > > > > that information or upload the data to our FTP server (see
How
> > > > > > To Send Us Data in
> > > > > >
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > > F%
> > > > > > 2F
> > > > > > dt
> > > > > > ce
> > > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fmet-h
> > > > > > el
> > > > > > p-
> > > > > > de
> > > > > > sk
> > > > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa7271
> > > > > > 17
> > > > > > f4
> > > > > > a9
> > > > > > 5b
> > > > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6
> > > > > > 37
> > > > > > 55
> > > > > > 73
> > > > > > 71
> > > > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > > > > > V2
> > > > > > lu
> > > > > > Mz
> > > > > > Ii
> > > > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoACs
> > > > > > WV
> > > > > > CV
> > > > > > W4
> > > > > > JE
> > > > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > > > ).
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > > > Transaction: Ticket created by
T.Eldridge at pgr.reading.ac.uk
> > > > > > > Queue: met_help
> > > > > > > Subject: Debugging MODE
> > > > > > > Owner: Nobody
> > > > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > > %2
> > > > > > > F%
> > > > > > > 2F
> > > > > > > rt
> > > > > > > .r
> > > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&da
> > > > > > > ta
> > > > > > > =0
> > > > > > > 4%
> > > > > > > 7C
> > > > > > > 01
> > > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d9
> > > > > > > 0f
> > > > > > > 0a
> > > > > > > 39
> > > > > > > cf
> > > > > > > %7
> > > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573588586752
> > > > > > > 30
> > > > > > > %7
> > > > > > > CU
> > > > > > > nk
> > > > > > > no
> > > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > > > > > 6I
> > > > > > > k1
> > > > > > > ha
> > > > > > > Ww
> > > > > > > iL
> > > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlW
> > > > > > > xE
> > > > > > > ve
> > > > > > > Ae
> > > > > > > fl
> > > > > > > SN
> > > > > > > Oeww%3D&reserved=0 >
> > > > > > >
> > > > > > >
> > > > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > > > satellite data I wish to analyse for my PhD research,
and
> > > > > > > don't really know how to debug in light of my (most
recent)
> > > > > > > error
> > > message.
> > > > > > > (Initially I got errors associated with a lack of CF
> > > > > > > conventions in my home-made observation product, so to
> > > > > > > proceed I am working with two reanalysis time-steps
which I
> > > > > > > know to be CF
> > > > > > > compliant.) I have attached the two netCDFs that I am
trying
> > > > > > > to operate on, as well as my config file. The error
message
> > > > > > > I
> > get is:
> > > > > > >
> > > > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > > > >
> > > > > > > This is, I suppose, the problem with using software as a
> > > > > > > 'black
> > > box'.
> > > > > > > If it's immediately clear what the issue is here, that
would
> > > > > > > be fab
> > > > > > > - otherwise I'm really only looking for pointers on how
to
> > > > > > > find the location of the issue in the software so that I
can
> > > > > > > get my head around the syntax and debug it.
Incidentally, I
> > > > > > > have tried running MODE between (identical) files with a
> > > > > > > time dimension, as opposed to a single step, and get the
> > > > > > > same error
> > message.
> > > > > > >
> > > > > > > Any help much appreciated!
> > > > > > >
> > > > > > > Tom
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III National Center for
> > > > > > Atmospheric Research Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > > > > obliged to reply to this email outside of your normal
working
> > hours.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III National Center for
> > > > > Atmospheric Research Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to reply to this email outside of your normal
working
> hours.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
Subject: RE: FW: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Wed May 12 09:43:47 2021
Hi John, thanks for the response. It's funny that the error should
only happen in a few cases then - I mentioned to George that this
breaks only three out of several hundred times, and all of them have
the same grid resolution. I'll have a go at changing grid_res in the
config file to some number which divides 60 and see what we get!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: 12 May 2021 16:39
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
This is John Halley Gotway. I work with George on providing MET-Help
support. I ran the sample data you sent via MODE and was able to track
down the problem to the convolution radius setting.
conv_radius = [ 60.0/grid_res, 60.0/grid_res, 60.0/grid_res ];
I realize why you used that... since it's the default in the config
file!
In your case, with grid_res = 25, that works out to a value of 2.4.
And MODE actually parse that as an integer... so it's a value of 2.
When I run with exactly 2 (conv_radius = [ 2, 2, 2 ];), I get the same
error message.
However, running with a radius of 0 (to apply no smoothing), 3 (to
apply a very small amount of smoothing), or a larger number for more
smoothing, it runs just fine.
There's something about a convolution radius of 2 which makes the
logic breakdown.
That's good news for you since it's easy to to change the conv_radius
setting. But I'll try to dig into the details to determine where the
logic's breaking down and hopefully add some more checks instead of
aborting.
Thanks,
John
On Wed, May 12, 2021 at 7:47 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C3de906ef0a2a401c5a3808d9155c1f40%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564307669198563%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=7eL0Gr01jTsbk3JAU0o7MpbVSMqi5q6D6JKAJTE
> TvUE%3D&reserved=0 >
>
> Thanks George, like I said don't hurry, just wanted to make sure
you'd
> seen it. Congratulations on the new release! Looking forward to
> hearing from you when you have a chance.
>
> Tom
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: 12 May 2021 14:22
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> I'm very sorry for the delay. I was pulled away from this to take
care
> of some unexpected tasks for the METplus 4.0.0 release (which is now
> available!) and this ticket was buried in my task list. Thank you
for
> following up to remind me. I will do some testing with your files
and
> get back to you shortly.
>
> - George
>
> On Wed, May 12, 2021 at 4:37 AM Thomas Eldridge via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > .r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > 01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C262ff8ff522d4aea517508d91548ff27
> > %7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564225534655556%7CUnk
> > no
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > iL
> >
CJXVCI6Mn0%3D%7C1000&sdata=nJhG9yKC%2F3RQFhfD0v4Eh%2BtqHMHOxFDrn
> > Xw
> > W%2FjyjQFc%3D&reserved=0 >
> >
> > Hi George,
> >
> > Any update on this? No mad rush, just hadn't heard back from you
and
> > thought I'd check to see you got my message. FYI I ran MODE on all
> > my files to see how many cases this is an issue for, and it turns
> > out to be three (out of 530) - I can't identify anything that
these
> > files have in common so no clue as to the cause of the problem
there.
> > Irritatingly the bug has hit one of my best examples, so would be
> > nice
> to have a fix!
> >
> > Thanks for all your help,
> >
> > Tom
> >
> > -----Original Message-----
> > From: Thomas Eldridge
> > Sent: 05 May 2021 15:21
> > To: met_help at ucar.edu
> > Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > You're totally right, my mistake - in fact all three files I
> > attached are incorrect, I was on autopilot. I've attached the
correct versions.
> > The exact syntax of my command is
> >
> > mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
> > config/MyConfigNoTime
> >
> > I've edited the filenames to reflect the path, since otherwise
they
> > would be identical in the attachment.
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 05 May 2021 15:17
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > Thank you for providing the data and config file. Could you share
> > the command you used to get this error? Also, it looks like you
> > mentioned a file called MyConfigNoTime but the file you shared is
> > called MyConfig. I am just checking to make sure I have the
correct
> > file to use
> for testing.
> >
> > Thanks,
> > George
> >
> > On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > rt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > 7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd084
> > > 96
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CU
> > > nk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > Ww
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6
> > > jl
> > > oV
> > > O9cA7E%3D&reserved=0 >
> > >
> > > Hi George,
> > >
> > > Progress made; new issue encountered. I have been able to update
> > > my THIR files to be CF compliant, and have successfully run MODE
> > > on some fields; however, on one example (which I hope will be
> > > particularly insightful RE what MODE does and how to interpret
it,
> > > since the feature is very visible in the data), I am getting the
> > > following error
> > message:
> > >
> > > ERROR :
> > > ERROR : PiecewiseLinear::operator()(double) -> error in
"(nul)"! ...
> > > t = nan ERROR :
> > >
> > > I've attached my files, which I've renamed based on the
directory
> > > structure I'm using, in case you'd like to take a look. These
> > > feature a single time-step, and MyConfigNoTime has been adjusted
> accordingly.
> > > Will keep plugging away to see if I can identify the cause
myself.
> > >
> > > Thanks,
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 04 May 2021 16:33
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > Sounds good. Please don't bang your head for too long!
> > >
> > > - George
> > >
> > > On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > 4%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11
> > > > ee
> > > > cc
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7
> > > > CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIq
> > > > ZC
> > > > x1
> > > > J%
> > > > 2BZ7Lluk%3D&reserved=0 >
> > > >
> > > > Thanks George, this is all great stuff, I'm sure the links
> > > > you've provided will cover my needs for now - expect to hear
> > > > back from me later this week if I keep banging my head against
the wall.
> > > >
> > > > All the best,
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 16:14
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > The output from the MET tools are a specific NetCDF format
> > > > defined and read by the MET tools. I am not exactly sure of
the
> > > > criteria used by the tools to determine if a NetCDF file was
> > > > created by one of
> > its own.
> > > > It would likely be easier to do one of two things:
> > > >
> > > > 1) Modify the non-CF compliant file to add the required
> > > > information to make it CF compliant
> > > >
> > > > OR
> > > >
> > > > 2) Utilize the Python Embedding functionality to read the
NetCDF
> > > > file
> > > >
> > > > The latter involves writing a Python script to read the data,
> > > > then define a 2D data volume and attributes to define how the
> > > > data is formatted (time, fields, grid, etc.). To use this
> > > > functionality, you need to have the MET tools built with
Python
> > > > enabled and the required
> > > Python packages available.
> > > >
> > > > More information on Python Embedding is here:
> > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&
> > > > da
> > > > ta
> > > > =0
> > > >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708
> > > > d9
> > > > 0f
> > > > 11
> > > >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755739197353
> > > > 17
> > > > 30
> > > > %7
> > > >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> > > > iI
> > > > 6I
> > > > k1
> > > >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2i
> > > > fj
> > > > bo
> > > > mL
> > > > xgWzU5tgpA%3D&reserved=0
> > > >
> > > > Sample scripts that utilize this logic can be found here:
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > dt
> > > > ce
> > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fsample-
> > > > an
> > > > al
> > > > ys
> > > > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7
> > > > da
> > > > 29
> > > > 7e
> > > >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0
> > > > %7
> > > > C0
> > > > %7
> > > >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAi
> > > > LC
> > > > JQ
> > > > Ij
> > > >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2
> > > > B9
> > > > lk
> > > > 65
> > > > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> > > >
> > > > And a specific example that reads NetCDF data from that page
is here:
> > > >
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > dt
> > > > ce
> > > > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpyth
> > > > on
> > > > -s
> > > > cr
> > > >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridg
> > > > e%
> > > > 40
> > > > pg
> > > >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecf
> > > > c4
> > > > 8c
> > > > 09
> > > >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZ
> > > > sb
> > > > 3d
> > > > 8e
> > > >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
> > > > D%
> > > > 7C
> > > > 10
> > > >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&
> > > > re
> > > > se
> > > > rv
> > > > ed=0
> > > >
> > > > Let me know if you have any other questions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > =0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f
> > > > > 0f
> > > > > 50
> > > > > e7
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825
> > > > > %7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > k1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkT
> > > > > hY
> > > > > 7%
> > > > > 2F
> > > > > nU
> > > > > fX%2Bkh0%3D&reserved=0 >
> > > > >
> > > > > One quick thing while I have you: I noticed that the output
> > > > > netCDF features an attribute called Projection whose value
is
> > > > > "LatLon" - would you happen to know if I can specify this
for
> > > > > my non-CF compliant file? No biggie if not, would just save
me
> > > > > some
> time...
> > > > >
> > > > > Tom
> > > > >
> > > > > -----Original Message-----
> > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > Sent: 04 May 2021 15:59
> > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > Cc: met_help at mailman.ucar.edu
> > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > >
> > > > > Tom,
> > > > >
> > > > > Glad to hear it! I will close this ticket now, but feel free
> > > > > to open a new one if you have any other issues or questions.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > %2
> > > > > > F%
> > > > > > 2F
> > > > > > rt
> > > > > > .r
> > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&da
> > > > > > ta
> > > > > > =0
> > > > > > 4%
> > > > > > 7C
> > > > > > 01
> > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d9
> > > > > > 0f
> > > > > > 0d
> > > > > > 2a
> > > > > > 20
> > > > > > %7
> > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573714922219
> > > > > > 18
> > > > > > %7
> > > > > > CU
> > > > > > nk
> > > > > > no
> > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > > > > 6I
> > > > > > k1
> > > > > > ha
> > > > > > Ww
> > > > > > iL
> > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBX
> > > > > > QK
> > > > > > %2
> > > > > > Bv
> > > > > > wF
> > > > > > Sq
> > > > > > %2Br%2BslCTY%3D&reserved=0 >
> > > > > >
> > > > > > Hi again George,
> > > > > >
> > > > > > Much as I'm sure you were excited to get your teeth into
> > > > > > MyConfig, your initial response was sufficiently helpful
> > > > > > that I am now able to run things (I had removed the
"level"
> > > > > > part of the field dictionary, rather than specifying the
single level).
> > > > > >
> > > > > > Thanks a bunch!
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > > Sent: 04 May 2021 15:38
> > > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > > Cc: met_help at mailman.ucar.edu
> > > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > > >
> > > > > > Hi Tom,
> > > > > >
> > > > > > I see you are having issues with running MODE using NetCDF
data.
> > > > > >
> > > > > > The warning and error messages you are receiving suggests
> > > > > > that the field information you are providing in your
config
> > > > > > file are
> > > incorrect.
> > > > > >
> > > > > > I can't see what you have set inside your MyConfig file,
but
> > > > > > the default settings for MODE set the fcst ( forecast)
> dictionary to:
> > > > > >
> > > > > > fcst = {
> > > > > > field = {
> > > > > > name = "APCP";
> > > > > > level = "A03";
> > > > > > }
> > > > > >
> > > > > >
> > > > > > censor_thresh = [];
> > > > > > censor_val = [];
> > > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > > conv_thresh = >=5.0;
> > > > > > vld_thresh = 0.5;
> > > > > > filter_attr_name = [];
> > > > > > filter_attr_thresh = [];
> > > > > > merge_thresh = >=1.25;
> > > > > > merge_flag = THRESH;
> > > > > > }
> > > > > > obs = fcst;
> > > > > >
> > > > > >
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > %2
> > > > > > F%
> > > > > > 2F
> > > > > > gi
> > > > > > th
> > > > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150
> > > > > > 73
> > > > > > 4a
> > > > > > bd
> > > > > > c6
> > > > > > 00
> > > > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&
> > > > > > ;d
> > > > > > at
> > > > > > a=
> > > > > > 04
> > > > > > %7
> > > > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc60
> > > > > > 8d
> > > > > > 90
> > > > > > f0
> > > > > > d2
> > > > > > a2
> > > > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573714922
> > > > > > 31
> > > > > > 87
> > > > > > 5%
> > > > > > 7C
> > > > > > Un
> > > > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
> > > > > > Ti
> > > > > > I6
> > > > > > Ik
> > > > > > 1h
> > > > > > aW
> > > > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPH
> > > > > > n%
> > > > > > 2F
> > > > > > 4I
> > > > > > b6
> > > > > > tH
> > > > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > > > >
> > > > > > The last line from the excerpt sets the obs (observation)
> > > > > > dictionary to the same values as the fcst dictionary. The
> > > > > > field dictionary inside fcst tells MET to look for a field
> called 'APCP'
> > > > > > at level 'A03.' You will need to set the name value to
"hcc"
> > > > > > which should remove the warning alerting you that it is
> > > > > > using the first field that
> > > > it finds.
> > > > > >
> > > > > > The error message saying "needed 3 arguments for variable
> > > > > > hcc, got
> > 2"
> > > > > > implies that the hcc field is a NetCDF file that has 3
> dimensions.
> > > > > > MET can only read 2D data, so you will have to specify
which
> > > > > > 2D volume to read. It looks like the NetCDF files you
> > > > > > provided were not attached to the ticket, so I can't see
how
> > > > > > the data is
> stored.
> > > > > > You can use * for the lat and lon dimensions to get all
data
> > > > > > points and set a number for the 3rd dimension to specify
> > > > > > which slab of data to pass in. If the hcc field has (time,
> > > > > > lat,
> > > > > > lon) as the dimensions, you can set:
> > > > > >
> > > > > > level = "(0,*,*)";
> > > > > >
> > > > > > to extract the 2D volume for the first time dimension in
the
> field.
> > > > > > If the first dimension is indeed time and it follows CF
> > > > > > conventions for specifying time, MET even supports
supplying
> > > > > > the desired time as the value and it will find the
> > > > > > appropriate 2D
> > volume:
> > > > > >
> > > > > > level = "(20210504_120000,*,*)";
> > > > > >
> > > > > > More information on MET I/O configurations can be found in
> > > > > > the User's Guide
> > > > > > here:
> > > > > >
> > > > > >
> > > > > >
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23
> > > > > > me
> > > > > > t-
> > > > > > co
> > > > > > nf
> > > > > > ig
> > > > > > uration-file-options&data=04%7C01%7CT.Eldridge%
> > 40pgr.reading.ac.
> > > > > > uk
> > > > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5
> > > > > > e4
> > > > > > 3f
> > > > > > f9
> > > > > > 0e
> > > > > > 5f
> > > > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> > > > > > oi
> > > > > > MC
> > > > > > 4w
> > > > > > Lj
> > > > > > Aw
> > > > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&am
> > > > > > p;
> > > > > > sd
> > > > > > at
> > > > > > a=
> > > > > > Tt
> > > > > >
GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > > > >
> > > > > > Please let me know if you have any issues getting this
> configured.
> > > > > > The NetCDF files may not have been included because of
file
> > > > > > size
> > > > limits.
> > > > > > You could either print the headers using ncdump -h and
send
> > > > > > me that information or upload the data to our FTP server
> > > > > > (see How To Send Us Data in
> > > > > >
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > %2
> > > > > > F%
> > > > > > 2F
> > > > > > dt
> > > > > > ce
> > > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fmet
> > > > > > -h
> > > > > > el
> > > > > > p-
> > > > > > de
> > > > > > sk
> > > > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa72
> > > > > > 71
> > > > > > 17
> > > > > > f4
> > > > > > a9
> > > > > > 5b
> > > > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7
> > > > > > C6
> > > > > > 37
> > > > > > 55
> > > > > > 73
> > > > > > 71
> > > > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj
> > > > > > oi
> > > > > > V2
> > > > > > lu
> > > > > > Mz
> > > > > > Ii
> > > > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoA
> > > > > > Cs
> > > > > > WV
> > > > > > CV
> > > > > > W4
> > > > > > JE
> > > > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > > > ).
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> > > > > > > Transaction: Ticket created by
T.Eldridge at pgr.reading.ac.uk
> > > > > > > Queue: met_help
> > > > > > > Subject: Debugging MODE
> > > > > > > Owner: Nobody
> > > > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%
> > > > > > > 3A
> > > > > > > %2
> > > > > > > F%
> > > > > > > 2F
> > > > > > > rt
> > > > > > > .r
> > > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&
> > > > > > > da
> > > > > > > ta
> > > > > > > =0
> > > > > > > 4%
> > > > > > > 7C
> > > > > > > 01
> > > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308
> > > > > > > d9
> > > > > > > 0f
> > > > > > > 0a
> > > > > > > 39
> > > > > > > cf
> > > > > > > %7
> > > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755735885867
> > > > > > > 52
> > > > > > > 30
> > > > > > > %7
> > > > > > > CU
> > > > > > > nk
> > > > > > > no
> > > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> > > > > > > iI
> > > > > > > 6I
> > > > > > > k1
> > > > > > > ha
> > > > > > > Ww
> > > > > > > iL
> > > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOT
> > > > > > > lW
> > > > > > > xE
> > > > > > > ve
> > > > > > > Ae
> > > > > > > fl
> > > > > > > SN
> > > > > > > Oeww%3D&reserved=0 >
> > > > > > >
> > > > > > >
> > > > > > > I'm trying to use the MODE tool on some reanalysis- and
> > > > > > > satellite data I wish to analyse for my PhD research,
and
> > > > > > > don't really know how to debug in light of my (most
> > > > > > > recent) error
> > > message.
> > > > > > > (Initially I got errors associated with a lack of CF
> > > > > > > conventions in my home-made observation product, so to
> > > > > > > proceed I am working with two reanalysis time-steps
which
> > > > > > > I know to be CF
> > > > > > > compliant.) I have attached the two netCDFs that I am
> > > > > > > trying to operate on, as well as my config file. The
error
> > > > > > > message I
> > get is:
> > > > > > >
> > > > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > > > >
> > > > > > > This is, I suppose, the problem with using software as a
> > > > > > > 'black
> > > box'.
> > > > > > > If it's immediately clear what the issue is here, that
> > > > > > > would be fab
> > > > > > > - otherwise I'm really only looking for pointers on how
to
> > > > > > > find the location of the issue in the software so that I
> > > > > > > can get my head around the syntax and debug it.
> > > > > > > Incidentally, I have tried running MODE between
> > > > > > > (identical) files with a time dimension, as opposed to a
> > > > > > > single step, and get the same error
> > message.
> > > > > > >
> > > > > > > Any help much appreciated!
> > > > > > >
> > > > > > > Tom
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III National Center for
> > > > > > Atmospheric Research Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
> > > > > > feel obliged to reply to this email outside of your normal
> > > > > > working
> > hours.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III National Center for
> > > > > Atmospheric Research Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to reply to this email outside of your normal
working
> hours.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III National Center for
> > Atmospheric Research Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
Subject: Debugging MODE
From: John Halley Gotway
Time: Wed May 12 09:49:56 2021
Tom,
You do not need to set grid_res to some factor of 60. You could just
modify
the config file to set the convolution radius to some specific value,
like
0 or 3 or something larger.
It is not necessary to define it relative to grid_res. That's just how
the
default value is set.
Hope that helps clarify.
John
On Wed, May 12, 2021 at 9:44 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
> Hi John, thanks for the response. It's funny that the error should
only
> happen in a few cases then - I mentioned to George that this breaks
only
> three out of several hundred times, and all of them have the same
grid
> resolution. I'll have a go at changing grid_res in the config file
to some
> number which divides 60 and see what we get!
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: 12 May 2021 16:39
> To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> Cc: met_help at mailman.ucar.edu
> Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
>
> Hi Tom,
>
> This is John Halley Gotway. I work with George on providing MET-Help
> support. I ran the sample data you sent via MODE and was able to
track down
> the problem to the convolution radius setting.
>
> conv_radius = [ 60.0/grid_res, 60.0/grid_res, 60.0/grid_res ];
>
> I realize why you used that... since it's the default in the config
file!
> In your case, with grid_res = 25, that works out to a value of 2.4.
And
> MODE actually parse that as an integer... so it's a value of 2. When
I run
> with exactly 2 (conv_radius = [ 2, 2, 2 ];), I get the same error
message.
> However, running with a radius of 0 (to apply no smoothing), 3 (to
apply a
> very small amount of smoothing), or a larger number for more
smoothing, it
> runs just fine.
>
> There's something about a convolution radius of 2 which makes the
logic
> breakdown.
>
> That's good news for you since it's easy to to change the
conv_radius
> setting. But I'll try to dig into the details to determine where the
> logic's breaking down and hopefully add some more checks instead of
> aborting.
>
> Thanks,
> John
>
>
>
>
>
>
>
>
> On Wed, May 12, 2021 at 7:47 AM Thomas Eldridge via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL:
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
> >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
> >
%7CT.Eldridge%40pgr.reading.ac.uk%7C3de906ef0a2a401c5a3808d9155c1f40%7
> >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564307669198563%7CUnkno
> >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >
CJXVCI6Mn0%3D%7C1000&sdata=7eL0Gr01jTsbk3JAU0o7MpbVSMqi5q6D6JKAJTE
> > TvUE%3D&reserved=0 >
> >
> > Thanks George, like I said don't hurry, just wanted to make sure
you'd
> > seen it. Congratulations on the new release! Looking forward to
> > hearing from you when you have a chance.
> >
> > Tom
> >
> > -----Original Message-----
> > From: George McCabe via RT <met_help at ucar.edu>
> > Sent: 12 May 2021 14:22
> > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > Cc: met_help at mailman.ucar.edu
> > Subject: Re: FW: [rt.rap.ucar.edu #99789] Debugging MODE
> >
> > Hi Tom,
> >
> > I'm very sorry for the delay. I was pulled away from this to take
care
> > of some unexpected tasks for the METplus 4.0.0 release (which is
now
> > available!) and this ticket was buried in my task list. Thank you
for
> > following up to remind me. I will do some testing with your files
and
> > get back to you shortly.
> >
> > - George
> >
> > On Wed, May 12, 2021 at 4:37 AM Thomas Eldridge via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL:
> > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt
> > > .r
> > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C
> > > 01
> > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C262ff8ff522d4aea517508d91548ff27
> > > %7
> > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637564225534655556%7CUnk
> > > no
> > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw
> > > iL
> > >
CJXVCI6Mn0%3D%7C1000&sdata=nJhG9yKC%2F3RQFhfD0v4Eh%2BtqHMHOxFDrn
> > > Xw
> > > W%2FjyjQFc%3D&reserved=0 >
> > >
> > > Hi George,
> > >
> > > Any update on this? No mad rush, just hadn't heard back from you
and
> > > thought I'd check to see you got my message. FYI I ran MODE on
all
> > > my files to see how many cases this is an issue for, and it
turns
> > > out to be three (out of 530) - I can't identify anything that
these
> > > files have in common so no clue as to the cause of the problem
there.
> > > Irritatingly the bug has hit one of my best examples, so would
be
> > > nice
> > to have a fix!
> > >
> > > Thanks for all your help,
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: Thomas Eldridge
> > > Sent: 05 May 2021 15:21
> > > To: met_help at ucar.edu
> > > Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > You're totally right, my mistake - in fact all three files I
> > > attached are incorrect, I was on autopilot. I've attached the
correct
> versions.
> > > The exact syntax of my command is
> > >
> > > mode 10/ERA5/1970-05-27_15h00.nc 10/THIR/1970-05-27_15h00.nc
> > > config/MyConfigNoTime
> > >
> > > I've edited the filenames to reflect the path, since otherwise
they
> > > would be identical in the attachment.
> > >
> > > Tom
> > >
> > > -----Original Message-----
> > > From: George McCabe via RT <met_help at ucar.edu>
> > > Sent: 05 May 2021 15:17
> > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > Cc: met_help at mailman.ucar.edu
> > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > >
> > > Hi Tom,
> > >
> > > Thank you for providing the data and config file. Could you
share
> > > the command you used to get this error? Also, it looks like you
> > > mentioned a file called MyConfigNoTime but the file you shared
is
> > > called MyConfig. I am just checking to make sure I have the
correct
> > > file to use
> > for testing.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, May 5, 2021 at 8:12 AM Thomas Eldridge via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL:
> > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > rt
> > > > .r
> > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%
> > > > 7C
> > > > 01
> > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cfadf35a89e9f4e87c3e608d90fd084
> > > > 96
> > > > %7
> > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637558210518517047%7CU
> > > > nk
> > > > no
> > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> > > > Ww
> > > > iL
> > > >
CJXVCI6Mn0%3D%7C1000&sdata=hRx2HW5dirOlr12NH3fNOJ8Oz6iW0Vm%2B6
> > > > jl
> > > > oV
> > > > O9cA7E%3D&reserved=0 >
> > > >
> > > > Hi George,
> > > >
> > > > Progress made; new issue encountered. I have been able to
update
> > > > my THIR files to be CF compliant, and have successfully run
MODE
> > > > on some fields; however, on one example (which I hope will be
> > > > particularly insightful RE what MODE does and how to interpret
it,
> > > > since the feature is very visible in the data), I am getting
the
> > > > following error
> > > message:
> > > >
> > > > ERROR :
> > > > ERROR : PiecewiseLinear::operator()(double) -> error in
"(nul)"! ...
> > > > t = nan ERROR :
> > > >
> > > > I've attached my files, which I've renamed based on the
directory
> > > > structure I'm using, in case you'd like to take a look. These
> > > > feature a single time-step, and MyConfigNoTime has been
adjusted
> > accordingly.
> > > > Will keep plugging away to see if I can identify the cause
myself.
> > > >
> > > > Thanks,
> > > >
> > > > Tom
> > > >
> > > > -----Original Message-----
> > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > Sent: 04 May 2021 16:33
> > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > Cc: met_help at mailman.ucar.edu
> > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > >
> > > > Hi Tom,
> > > >
> > > > Sounds good. Please don't bang your head for too long!
> > > >
> > > > - George
> > > >
> > > > On Tue, May 4, 2021 at 9:16 AM Thomas Eldridge via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > rt
> > > > > .r
> > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=0
> > > > > 4%
> > > > > 7C
> > > > > 01
> > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708d90f11
> > > > > ee
> > > > > cc
> > > > > %7
> > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557391973531730%7
> > > > > CU
> > > > > nk
> > > > > no
> > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> > > > > ha
> > > > > Ww
> > > > > iL
> > > > >
CJXVCI6Mn0%3D%7C1000&sdata=0cgiCX7GjZaNtDq9NjY%2FfgZ2vUxMyIq
> > > > > ZC
> > > > > x1
> > > > > J%
> > > > > 2BZ7Lluk%3D&reserved=0 >
> > > > >
> > > > > Thanks George, this is all great stuff, I'm sure the links
> > > > > you've provided will cover my needs for now - expect to hear
> > > > > back from me later this week if I keep banging my head
against the
> wall.
> > > > >
> > > > > All the best,
> > > > >
> > > > > Tom
> > > > >
> > > > > -----Original Message-----
> > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > Sent: 04 May 2021 16:14
> > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > Cc: met_help at mailman.ucar.edu
> > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > >
> > > > > Hi Tom,
> > > > >
> > > > > The output from the MET tools are a specific NetCDF format
> > > > > defined and read by the MET tools. I am not exactly sure of
the
> > > > > criteria used by the tools to determine if a NetCDF file was
> > > > > created by one of
> > > its own.
> > > > > It would likely be easier to do one of two things:
> > > > >
> > > > > 1) Modify the non-CF compliant file to add the required
> > > > > information to make it CF compliant
> > > > >
> > > > > OR
> > > > >
> > > > > 2) Utilize the Python Embedding functionality to read the
NetCDF
> > > > > file
> > > > >
> > > > > The latter involves writing a Python script to read the
data,
> > > > > then define a 2D data volume and attributes to define how
the
> > > > > data is formatted (time, fields, grid, etc.). To use this
> > > > > functionality, you need to have the MET tools built with
Python
> > > > > enabled and the required
> > > > Python packages available.
> > > > >
> > > > > More information on Python Embedding is here:
> > > > >
> > > > >
> >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html&
> > > > > da
> > > > > ta
> > > > > =0
> > > > >
4%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7da297ef788041df58e708
> > > > > d9
> > > > > 0f
> > > > > 11
> > > > >
eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755739197353
> > > > > 17
> > > > > 30
> > > > > %7
> > > > >
CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> > > > > iI
> > > > > 6I
> > > > > k1
> > > > >
haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StrP56CHjs2IOqX8lQwKQZEmk2i
> > > > > fj
> > > > > bo
> > > > > mL
> > > > > xgWzU5tgpA%3D&reserved=0
> > > > >
> > > > > Sample scripts that utilize this logic can be found here:
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > dt
> > > > > ce
> > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fsample-
> > > > > an
> > > > > al
> > > > > ys
> > > > > is-
scripts&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C7
> > > > > da
> > > > > 29
> > > > > 7e
> > > > >
f788041df58e708d90f11eecc%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0
> > > > > %7
> > > > > C0
> > > > > %7
> > > > >
C637557391973531730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAi
> > > > > LC
> > > > > JQ
> > > > > Ij
> > > > >
oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CWEp%2
> > > > > B9
> > > > > lk
> > > > > 65
> > > > > fvErP3EYZWIkNEs4EFgwlKzwoLqBV3sDo%3D&reserved=0
> > > > >
> > > > > And a specific example that reads NetCDF data from that page
is
> here:
> > > > >
> > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > dt
> > > > > ce
> > > > > nter.org%2Fsites%2Fdefault%2Ffiles%2Fcommunity-
code%2Fmet%2Fpyth
> > > > > on
> > > > > -s
> > > > > cr
> > > > >
ipts%2Fread_VIIRS_AOD_NetCDF.py.txt&data=04%7C01%7CT.Eldridg
> > > > > e%
> > > > > 40
> > > > > pg
> > > > >
r.reading.ac.uk%7C7da297ef788041df58e708d90f11eecc%7C4ffa3bc4ecf
> > > > > c4
> > > > > 8c
> > > > > 09
> > > > >
080f5e43ff90e5f%7C0%7C0%7C637557391973541683%7CUnknown%7CTWFpbGZ
> > > > > sb
> > > > > 3d
> > > > > 8e
> > > > >
yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
> > > > > D%
> > > > > 7C
> > > > > 10
> > > > >
00&sdata=hBPfCijjFnideBCUWBt7Du0zRwO7KHl87YG9RRrGulw%3D&
> > > > > re
> > > > > se
> > > > > rv
> > > > > ed=0
> > > > >
> > > > > Let me know if you have any other questions.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > > On Tue, May 4, 2021 at 9:00 AM Thomas Eldridge via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
> > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2
> > > > > > F%
> > > > > > 2F
> > > > > > rt
> > > > > > .r
> > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data
> > > > > > =0
> > > > > > 4%
> > > > > > 7C
> > > > > > 01
> > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C63ec92fcd8e54eedcc9508d90f
> > > > > > 0f
> > > > > > 50
> > > > > > e7
> > > > > > %7
> > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557380722016825
> > > > > > %7
> > > > > > CU
> > > > > > nk
> > > > > > no
> > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> > > > > > k1
> > > > > > ha
> > > > > > Ww
> > > > > > iL
> > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WXAxWAF821SQwOxcn0xmKYYUoW4KPkT
> > > > > > hY
> > > > > > 7%
> > > > > > 2F
> > > > > > nU
> > > > > > fX%2Bkh0%3D&reserved=0 >
> > > > > >
> > > > > > One quick thing while I have you: I noticed that the
output
> > > > > > netCDF features an attribute called Projection whose value
is
> > > > > > "LatLon" - would you happen to know if I can specify this
for
> > > > > > my non-CF compliant file? No biggie if not, would just
save me
> > > > > > some
> > time...
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > > Sent: 04 May 2021 15:59
> > > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > > Cc: met_help at mailman.ucar.edu
> > > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > > >
> > > > > > Tom,
> > > > > >
> > > > > > Glad to hear it! I will close this ticket now, but feel
free
> > > > > > to open a new one if you have any other issues or
questions.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > > On Tue, May 4, 2021 at 8:54 AM Thomas Eldridge via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
> > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > > %2
> > > > > > > F%
> > > > > > > 2F
> > > > > > > rt
> > > > > > > .r
> > > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&da
> > > > > > > ta
> > > > > > > =0
> > > > > > > 4%
> > > > > > > 7C
> > > > > > > 01
> > > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc608d9
> > > > > > > 0f
> > > > > > > 0d
> > > > > > > 2a
> > > > > > > 20
> > > > > > > %7
> > > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573714922219
> > > > > > > 18
> > > > > > > %7
> > > > > > > CU
> > > > > > > nk
> > > > > > > no
> > > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> > > > > > > 6I
> > > > > > > k1
> > > > > > > ha
> > > > > > > Ww
> > > > > > > iL
> > > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=WKZj0XzQI6KV36bS4MEOwj%2FEiBX
> > > > > > > QK
> > > > > > > %2
> > > > > > > Bv
> > > > > > > wF
> > > > > > > Sq
> > > > > > > %2Br%2BslCTY%3D&reserved=0 >
> > > > > > >
> > > > > > > Hi again George,
> > > > > > >
> > > > > > > Much as I'm sure you were excited to get your teeth into
> > > > > > > MyConfig, your initial response was sufficiently helpful
> > > > > > > that I am now able to run things (I had removed the
"level"
> > > > > > > part of the field dictionary, rather than specifying the
> single level).
> > > > > > >
> > > > > > > Thanks a bunch!
> > > > > > >
> > > > > > > Tom
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: George McCabe via RT <met_help at ucar.edu>
> > > > > > > Sent: 04 May 2021 15:38
> > > > > > > To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
> > > > > > > Cc: met_help at mailman.ucar.edu
> > > > > > > Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
> > > > > > >
> > > > > > > Hi Tom,
> > > > > > >
> > > > > > > I see you are having issues with running MODE using
NetCDF
> data.
> > > > > > >
> > > > > > > The warning and error messages you are receiving
suggests
> > > > > > > that the field information you are providing in your
config
> > > > > > > file are
> > > > incorrect.
> > > > > > >
> > > > > > > I can't see what you have set inside your MyConfig file,
but
> > > > > > > the default settings for MODE set the fcst ( forecast)
> > dictionary to:
> > > > > > >
> > > > > > > fcst = {
> > > > > > > field = {
> > > > > > > name = "APCP";
> > > > > > > level = "A03";
> > > > > > > }
> > > > > > >
> > > > > > >
> > > > > > > censor_thresh = [];
> > > > > > > censor_val = [];
> > > > > > > conv_radius = 60.0/grid_res; // in grid
squares
> > > > > > > conv_thresh = >=5.0;
> > > > > > > vld_thresh = 0.5;
> > > > > > > filter_attr_name = [];
> > > > > > > filter_attr_thresh = [];
> > > > > > > merge_thresh = >=1.25;
> > > > > > > merge_flag = THRESH;
> > > > > > > }
> > > > > > > obs = fcst;
> > > > > > >
> > > > > > >
> > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > > %2
> > > > > > > F%
> > > > > > > 2F
> > > > > > > gi
> > > > > > > th
> > > > > > >
ub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150
> > > > > > > 73
> > > > > > > 4a
> > > > > > > bd
> > > > > > > c6
> > > > > > > 00
> > > > > > > 866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&
> > > > > > > ;d
> > > > > > > at
> > > > > > > a=
> > > > > > > 04
> > > > > > > %7
> > > > > > >
C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa727117f4a95bfc60
> > > > > > > 8d
> > > > > > > 90
> > > > > > > f0
> > > > > > > d2
> > > > > > > a2
> > > > > > >
0%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C6375573714922
> > > > > > > 31
> > > > > > > 87
> > > > > > > 5%
> > > > > > > 7C
> > > > > > > Un
> > > > > > >
known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
> > > > > > > Ti
> > > > > > > I6
> > > > > > > Ik
> > > > > > > 1h
> > > > > > > aW
> > > > > > >
wiLCJXVCI6Mn0%3D%7C1000&sdata=1IymI8uVZ%2FkoIo4%2FKHsdPH
> > > > > > > n%
> > > > > > > 2F
> > > > > > > 4I
> > > > > > > b6
> > > > > > > tH
> > > > > > > G3g4N%2BeRCpvCU%3D&reserved=0
> > > > > > >
> > > > > > > The last line from the excerpt sets the obs
(observation)
> > > > > > > dictionary to the same values as the fcst dictionary.
The
> > > > > > > field dictionary inside fcst tells MET to look for a
field
> > called 'APCP'
> > > > > > > at level 'A03.' You will need to set the name value to
"hcc"
> > > > > > > which should remove the warning alerting you that it is
> > > > > > > using the first field that
> > > > > it finds.
> > > > > > >
> > > > > > > The error message saying "needed 3 arguments for
variable
> > > > > > > hcc, got
> > > 2"
> > > > > > > implies that the hcc field is a NetCDF file that has 3
> > dimensions.
> > > > > > > MET can only read 2D data, so you will have to specify
which
> > > > > > > 2D volume to read. It looks like the NetCDF files you
> > > > > > > provided were not attached to the ticket, so I can't see
how
> > > > > > > the data is
> > stored.
> > > > > > > You can use * for the lat and lon dimensions to get all
data
> > > > > > > points and set a number for the 3rd dimension to specify
> > > > > > > which slab of data to pass in. If the hcc field has
(time,
> > > > > > > lat,
> > > > > > > lon) as the dimensions, you can set:
> > > > > > >
> > > > > > > level = "(0,*,*)";
> > > > > > >
> > > > > > > to extract the 2D volume for the first time dimension in
the
> > field.
> > > > > > > If the first dimension is indeed time and it follows CF
> > > > > > > conventions for specifying time, MET even supports
supplying
> > > > > > > the desired time as the value and it will find the
> > > > > > > appropriate 2D
> > > volume:
> > > > > > >
> > > > > > > level = "(20210504_120000,*,*)";
> > > > > > >
> > > > > > > More information on MET I/O configurations can be found
in
> > > > > > > the User's Guide
> > > > > > > here:
> > > > > > >
> > > > > > >
> > > > > > >
> > > >
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.
> > > > > > >
readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23
> > > > > > > me
> > > > > > > t-
> > > > > > > co
> > > > > > > nf
> > > > > > > ig
> > > > > > > uration-file-options&data=04%7C01%7CT.Eldridge%
> > > 40pgr.reading.ac.
> > > > > > > uk
> > > > > > >
%7Cf89fa727117f4a95bfc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5
> > > > > > > e4
> > > > > > > 3f
> > > > > > > f9
> > > > > > > 0e
> > > > > > > 5f
> > > > > > >
%7C0%7C0%7C637557371492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> > > > > > > oi
> > > > > > > MC
> > > > > > > 4w
> > > > > > > Lj
> > > > > > > Aw
> > > > > > >
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&am
> > > > > > > p;
> > > > > > > sd
> > > > > > > at
> > > > > > > a=
> > > > > > > Tt
> > > > > > >
GXNire6NnhZrOYds76V5KO7Na903HI4WLnCiCHSHA%3D&reserved=0
> > > > > > >
> > > > > > > Please let me know if you have any issues getting this
> > configured.
> > > > > > > The NetCDF files may not have been included because of
file
> > > > > > > size
> > > > > limits.
> > > > > > > You could either print the headers using ncdump -h and
send
> > > > > > > me that information or upload the data to our FTP server
> > > > > > > (see How To Send Us Data in
> > > > > > >
> > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%3A
> > > > > > > %2
> > > > > > > F%
> > > > > > > 2F
> > > > > > > dt
> > > > > > > ce
> > > > > > > nter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2Fmet
> > > > > > > -h
> > > > > > > el
> > > > > > > p-
> > > > > > > de
> > > > > > > sk
> > > > > > >
&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7Cf89fa72
> > > > > > > 71
> > > > > > > 17
> > > > > > > f4
> > > > > > > a9
> > > > > > > 5b
> > > > > > >
fc608d90f0d2a20%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7
> > > > > > > C6
> > > > > > > 37
> > > > > > > 55
> > > > > > > 73
> > > > > > > 71
> > > > > > >
492231875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj
> > > > > > > oi
> > > > > > > V2
> > > > > > > lu
> > > > > > > Mz
> > > > > > > Ii
> > > > > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=W7bIq5lfIcoA
> > > > > > > Cs
> > > > > > > WV
> > > > > > > CV
> > > > > > > W4
> > > > > > > JE
> > > > > > > Q9C8aymbCn%2FbD%2B0VFreqw%3D&reserved=0
> > > > > > > ).
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > > On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
> > > > > > > <met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue May 04 07:42:17 2021: Request 99789 was acted
upon.
> > > > > > > > Transaction: Ticket created by
T.Eldridge at pgr.reading.ac.uk
> > > > > > > > Queue: met_help
> > > > > > > > Subject: Debugging MODE
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: T.Eldridge at pgr.reading.ac.uk
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > > > >
https://eur03.safelinks.protection.outlook.com/?url=https%
> > > > > > > > 3A
> > > > > > > > %2
> > > > > > > > F%
> > > > > > > > 2F
> > > > > > > > rt
> > > > > > > > .r
> > > > > > > >
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&
> > > > > > > > da
> > > > > > > > ta
> > > > > > > > =0
> > > > > > > > 4%
> > > > > > > > 7C
> > > > > > > > 01
> > > > > > > >
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308
> > > > > > > > d9
> > > > > > > > 0f
> > > > > > > > 0a
> > > > > > > > 39
> > > > > > > > cf
> > > > > > > > %7
> > > > > > > >
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C63755735885867
> > > > > > > > 52
> > > > > > > > 30
> > > > > > > > %7
> > > > > > > > CU
> > > > > > > > nk
> > > > > > > > no
> > > > > > > >
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> > > > > > > > iI
> > > > > > > > 6I
> > > > > > > > k1
> > > > > > > > ha
> > > > > > > > Ww
> > > > > > > > iL
> > > > > > > >
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOT
> > > > > > > > lW
> > > > > > > > xE
> > > > > > > > ve
> > > > > > > > Ae
> > > > > > > > fl
> > > > > > > > SN
> > > > > > > > Oeww%3D&reserved=0 >
> > > > > > > >
> > > > > > > >
> > > > > > > > I'm trying to use the MODE tool on some reanalysis-
and
> > > > > > > > satellite data I wish to analyse for my PhD research,
and
> > > > > > > > don't really know how to debug in light of my (most
> > > > > > > > recent) error
> > > > message.
> > > > > > > > (Initially I got errors associated with a lack of CF
> > > > > > > > conventions in my home-made observation product, so to
> > > > > > > > proceed I am working with two reanalysis time-steps
which
> > > > > > > > I know to be CF
> > > > > > > > compliant.) I have attached the two netCDFs that I am
> > > > > > > > trying to operate on, as well as my config file. The
error
> > > > > > > > message I
> > > get is:
> > > > > > > >
> > > > > > > > [cid:image001.png at 01D740F2.E0A65760]
> > > > > > > >
> > > > > > > > This is, I suppose, the problem with using software as
a
> > > > > > > > 'black
> > > > box'.
> > > > > > > > If it's immediately clear what the issue is here, that
> > > > > > > > would be fab
> > > > > > > > - otherwise I'm really only looking for pointers on
how to
> > > > > > > > find the location of the issue in the software so that
I
> > > > > > > > can get my head around the syntax and debug it.
> > > > > > > > Incidentally, I have tried running MODE between
> > > > > > > > (identical) files with a time dimension, as opposed to
a
> > > > > > > > single step, and get the same error
> > > message.
> > > > > > > >
> > > > > > > > Any help much appreciated!
> > > > > > > >
> > > > > > > > Tom
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > George McCabe - Software Engineer III National Center
for
> > > > > > > Atmospheric Research Research Applications Laboratory
> > > > > > > 303-497-2768
> > > > > > > ---
> > > > > > > My working day may not be your working day. Please do
not
> > > > > > > feel obliged to reply to this email outside of your
normal
> > > > > > > working
> > > hours.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III National Center for
> > > > > > Atmospheric Research Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > > > > obliged to reply to this email outside of your normal
working
> > hours.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III National Center for
> > > > > Atmospheric Research Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to reply to this email outside of your normal
working
> hours.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III National Center for
> > > > Atmospheric Research Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to reply to this email outside of your normal working
hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III National Center for
> > > Atmospheric Research Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to reply to this email outside of your normal working
hours.
> > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged
> > to reply to this email outside of your normal working hours.
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list