[Met_help] [rt.rap.ucar.edu #99789] History for Debugging MODE
George McCabe via RT
met_help at ucar.edu
Tue May 4 08:59:02 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I'm trying to use the MODE tool on some reanalysis- and satellite data I wish to analyse for my PhD research, and don't really know how to debug in light of my (most recent) error message. (Initially I got errors associated with a lack of CF conventions in my home-made observation product, so to proceed I am working with two reanalysis time-steps which I know to be CF compliant.) I have attached the two netCDFs that I am trying to operate on, as well as my config file. The error message I get is:
[cid:image001.png at 01D740F2.E0A65760]
This is, I suppose, the problem with using software as a 'black box'. If it's immediately clear what the issue is here, that would be fab - otherwise I'm really only looking for pointers on how to find the location of the issue in the software so that I can get my head around the syntax and debug it. Incidentally, I have tried running MODE between (identical) files with a time dimension, as opposed to a single step, and get the same error message.
Any help much appreciated!
Tom
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Debugging MODE
From: George McCabe
Time: Tue May 04 08:38:00 2021
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field
information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default
settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://github.com/dtcenter/MET/blob/6b951f395d35b81f28cea8aa150734abdc600866/met/data/config/MODEConfig_default#L55-
L71
The last line from the excerpt sets the obs (observation) dictionary
to the
same values as the fcst dictionary. The field dictionary inside fcst
tells
MET to look for a field called 'APCP' at level 'A03.' You will need to
set
the name value to "hcc" which should remove the warning alerting you
that
it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can
only read 2D data, so you will have to specify which 2D volume to
read. It
looks like the NetCDF files you provided were not attached to the
ticket,
so I can't see how the data is stored. You can use * for the lat and
lon
dimensions to get all data points and set a number for the 3rd
dimension to
specify which slab of data to pass in. If the hcc field has (time,
lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the
first dimension is indeed time and it follows CF conventions for
specifying
time, MET even supports supplying the desired time as the value and it
will
find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://met.readthedocs.io/en/latest/Users_Guide/data_io.html#met-
configuration-file-options
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You
could either print the headers using ncdump -h and send me that
information
or upload the data to our FTP server (see How To Send Us Data in
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99789 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data I
> wish to analyse for my PhD research, and don't really know how to
debug in
> light of my (most recent) error message. (Initially I got errors
associated
> with a lack of CF conventions in my home-made observation product,
so to
> proceed I am working with two reanalysis time-steps which I know to
be CF
> compliant.) I have attached the two netCDFs that I am trying to
operate on,
> as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'. If
> it's immediately clear what the issue is here, that would be fab -
> otherwise I'm really only looking for pointers on how to find the
location
> of the issue in the software so that I can get my head around the
syntax
> and debug it. Incidentally, I have tried running MODE between
(identical)
> files with a time dimension, as opposed to a single step, and get
the same
> error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 08:49:46 2021
Hi George,
Sorry, I meant to attach my files. I've done so now; will have a look
at your advice below but if these are illuminating feel free to share
your thoughts!
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 15:38
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RULINH9sa4kJB8ikglLjghlVFgIfCC90QHoUSFc%2FXeg%3D&reserved=0
The last line from the excerpt sets the obs (observation) dictionary
to the same values as the fcst dictionary. The field dictionary inside
fcst tells MET to look for a field called 'APCP' at level 'A03.' You
will need to set the name value to "hcc" which should remove the
warning alerting you that it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can only read 2D data, so you will have to specify which 2D volume to
read. It looks like the NetCDF files you provided were not attached to
the ticket, so I can't see how the data is stored. You can use * for
the lat and lon dimensions to get all data points and set a number for
the 3rd dimension to specify which slab of data to pass in. If the hcc
field has (time, lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the first dimension is indeed time and it follows CF conventions for
specifying time, MET even supports supplying the desired time as the
value and it will find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
configuration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ke6eAM9sgM6L3LoLzjhaEvmjbpBEADD6BotxO%2FgXHp4%3D&reserved=0
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You could either print the headers using ncdump -h and send me that
information or upload the data to our FTP server (see How To Send Us
Data in
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zywwR%2BR9wwrqH2V%2FwfTgAB0hMQ0o9hdCc15gXnG%2F61s%3D&reserved=0
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAeflSN
> Oeww%3D&reserved=0 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data
> I wish to analyse for my PhD research, and don't really know how to
> debug in light of my (most recent) error message. (Initially I got
> errors associated with a lack of CF conventions in my home-made
> observation product, so to proceed I am working with two reanalysis
> time-steps which I know to be CF
> compliant.) I have attached the two netCDFs that I am trying to
> operate on, as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'.
> If it's immediately clear what the issue is here, that would be fab
-
> otherwise I'm really only looking for pointers on how to find the
> location of the issue in the software so that I can get my head
around
> the syntax and debug it. Incidentally, I have tried running MODE
> between (identical) files with a time dimension, as opposed to a
> single step, and get the same error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #99789] Debugging MODE
From: Thomas Eldridge
Time: Tue May 04 08:54:21 2021
Hi again George,
Much as I'm sure you were excited to get your teeth into MyConfig,
your initial response was sufficiently helpful that I am now able to
run things (I had removed the "level" part of the field dictionary,
rather than specifying the single level).
Thanks a bunch!
Tom
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: 04 May 2021 15:38
To: Thomas Eldridge <T.Eldridge at pgr.reading.ac.uk>
Cc: met_help at mailman.ucar.edu
Subject: Re: [rt.rap.ucar.edu #99789] Debugging MODE
Hi Tom,
I see you are having issues with running MODE using NetCDF data.
The warning and error messages you are receiving suggests that the
field information you are providing in your config file are incorrect.
I can't see what you have set inside your MyConfig file, but the
default settings for MODE set the fcst ( forecast) dictionary to:
fcst = {
field = {
name = "APCP";
level = "A03";
}
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh = >=5.0;
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = THRESH;
}
obs = fcst;
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdtcenter%2FMET%2Fblob%2F6b951f395d35b81f28cea8aa150734abdc600866%2Fmet%2Fdata%2Fconfig%2FMODEConfig_default%23L55-
L71&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RULINH9sa4kJB8ikglLjghlVFgIfCC90QHoUSFc%2FXeg%3D&reserved=0
The last line from the excerpt sets the obs (observation) dictionary
to the same values as the fcst dictionary. The field dictionary inside
fcst tells MET to look for a field called 'APCP' at level 'A03.' You
will need to set the name value to "hcc" which should remove the
warning alerting you that it is using the first field that it finds.
The error message saying "needed 3 arguments for variable hcc, got 2"
implies that the hcc field is a NetCDF file that has 3 dimensions. MET
can only read 2D data, so you will have to specify which 2D volume to
read. It looks like the NetCDF files you provided were not attached to
the ticket, so I can't see how the data is stored. You can use * for
the lat and lon dimensions to get all data points and set a number for
the 3rd dimension to specify which slab of data to pass in. If the hcc
field has (time, lat,
lon) as the dimensions, you can set:
level = "(0,*,*)";
to extract the 2D volume for the first time dimension in the field. If
the first dimension is indeed time and it follows CF conventions for
specifying time, MET even supports supplying the desired time as the
value and it will find the appropriate 2D volume:
level = "(20210504_120000,*,*)";
More information on MET I/O configurations can be found in the User's
Guide
here:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2Fdata_io.html%23met-
configuration-file-
options&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ke6eAM9sgM6L3LoLzjhaEvmjbpBEADD6BotxO%2FgXHp4%3D&reserved=0
Please let me know if you have any issues getting this configured. The
NetCDF files may not have been included because of file size limits.
You could either print the headers using ncdump -h and send me that
information or upload the data to our FTP server (see How To Send Us
Data in
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdtcenter.org%2Fcommunity-
code%2Fmodel-evaluation-tools-met%2Fmet-help-
desk&data=04%7C01%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zywwR%2BR9wwrqH2V%2FwfTgAB0hMQ0o9hdCc15gXnG%2F61s%3D&reserved=0
).
Thanks,
George
On Tue, May 4, 2021 at 7:42 AM Thomas Eldridge via RT
<met_help at ucar.edu>
wrote:
>
> Tue May 04 07:42:17 2021: Request 99789 was acted upon.
> Transaction: Ticket created by T.Eldridge at pgr.reading.ac.uk
> Queue: met_help
> Subject: Debugging MODE
> Owner: Nobody
> Requestors: T.Eldridge at pgr.reading.ac.uk
> Status: new
> Ticket <URL:
>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.r
>
ap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D99789&data=04%7C01
>
%7CT.Eldridge%40pgr.reading.ac.uk%7C955b1eae71a446bb648308d90f0a39cf%7
>
C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637557358858675230%7CUnkno
>
wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>
CJXVCI6Mn0%3D%7C1000&sdata=ObuSp4KnWkTek8yN3R3kQ0mThOTlWxEveAeflSN
> Oeww%3D&reserved=0 >
>
>
> I'm trying to use the MODE tool on some reanalysis- and satellite
data
> I wish to analyse for my PhD research, and don't really know how to
> debug in light of my (most recent) error message. (Initially I got
> errors associated with a lack of CF conventions in my home-made
> observation product, so to proceed I am working with two reanalysis
> time-steps which I know to be CF
> compliant.) I have attached the two netCDFs that I am trying to
> operate on, as well as my config file. The error message I get is:
>
> [cid:image001.png at 01D740F2.E0A65760]
>
> This is, I suppose, the problem with using software as a 'black
box'.
> If it's immediately clear what the issue is here, that would be fab
-
> otherwise I'm really only looking for pointers on how to find the
> location of the issue in the software so that I can get my head
around
> the syntax and debug it. Incidentally, I have tried running MODE
> between (identical) files with a time dimension, as opposed to a
> single step, and get the same error message.
>
> Any help much appreciated!
>
> Tom
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list