[Met_help] [rt.rap.ucar.edu #80583] History for Problem reading CF-Compliant NetCDF

Tue May 23 12:42:03 MDT 2017

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Thanks for the updated docker version.
I've installed it and run a subset of the tutorial without a problem.

However, when I try to use my data - a netcdf file that worked for version 5.2 - version 6.0 can't read it.  (The netcdf file was originally WRF output, re-gridded in ncl to a regular lat/lon grid, with the conventions attribute set to CF-1/0) 
Could you let me know if you have any suggestions?  

Thanks!
-Laurel

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Problem reading CF-Compliant NetCDF
From: John Halley Gotway
Time: Tue May 23 11:15:43 2017

Hi Laurel,

I see you're having trouble getting version 6.0 of MET to read a CF-
compliant NetCDF file that version 5.2 was able to read fine.  This
behavior may or may not be related to fact that you're running MET via
docker.

It'd probably be easiest to just have you send me the problematic
file.  You can post it to our anonymous ftp site following these
instructions:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

Please write back once you've posted it, and I'll go grab it.

I'm in the office today, but will be out tomorrow through June 5th.
So hopefully we can move quickly and resolve this issue today.

Thanks,
John

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant NetCDF
From: Dehaan, Laurel
Time: Tue May 23 11:33:17 2017

Thank you John!
I just posted a file (wrf.ivt.2016041212.nc).
Of note - I am not confident this is a CF-compliant file.
I am working with integrated vapor transport (IVT), and could not find
that in the CF table - only the vector components of IVT.  However,
when it worked in 5.2 I decided it was "good enough".

I appreciate your help on this!

-Laurel

________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, May 23, 2017 11:15 AM
To: Dehaan, Laurel
Subject: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant NetCDF

Hi Laurel,

I see you're having trouble getting version 6.0 of MET to read a CF-
compliant NetCDF file that version 5.2 was able to read fine.  This
behavior may or may not be related to fact that you're running MET via
docker.

It'd probably be easiest to just have you send me the problematic
file.  You can post it to our anonymous ftp site following these
instructions:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

Please write back once you've posted it, and I'll go grab it.

I'm in the office today, but will be out tomorrow through June 5th.
So hopefully we can move quickly and resolve this issue today.

Thanks,
John

------------------------------------------------
Subject: Problem reading CF-Compliant NetCDF
From: John Halley Gotway
Time: Tue May 23 11:39:07 2017

Laurel,

OK, I'm able to replicate a problem here.  I used plot_data_plane to
plot
this data for met-5.2 and met-6.0:

# 5.2 works fine
met-5.2/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
level="(*,*)";'
DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
DEBUG 1: Creating postscript file: ivt_5.2.ps

# 6.0 errors out
met-6.0/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
level="(*,*)";'
DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
terminate called after throwing an instance of
'netCDF::exceptions::NcChar'
  what():  NcChar: Attempt to convert between text & numbers
file: ncVar.cpp  line:1521
Abort

Does this error match the behavior you're seeing?

Thanks,
John

On Tue, May 23, 2017 at 11:33 AM, Dehaan, Laurel via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80583 >
>
> Thank you John!
> I just posted a file (wrf.ivt.2016041212.nc).
> Of note - I am not confident this is a CF-compliant file.
> I am working with integrated vapor transport (IVT), and could not
find
> that in the CF table - only the vector components of IVT.  However,
when it
> worked in 5.2 I decided it was "good enough".
>
> I appreciate your help on this!
>
> -Laurel
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, May 23, 2017 11:15 AM
> To: Dehaan, Laurel
> Subject: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF
>
> Hi Laurel,
>
> I see you're having trouble getting version 6.0 of MET to read a
> CF-compliant NetCDF file that version 5.2 was able to read fine.
This
> behavior may or may not be related to fact that you're running MET
via
> docker.
>
> It'd probably be easiest to just have you send me the problematic
file.
> You can post it to our anonymous ftp site following these
instructions:
>    http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Please write back once you've posted it, and I'll go grab it.
>
> I'm in the office today, but will be out tomorrow through June 5th.
So
> hopefully we can move quickly and resolve this issue today.
>
> Thanks,
> John
>
>
>

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant NetCDF
From: Dehaan, Laurel
Time: Tue May 23 11:51:51 2017

Yes - that's exactly it
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, May 23, 2017 11:39 AM
To: Dehaan, Laurel
Subject: Re: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF

Laurel,

OK, I'm able to replicate a problem here.  I used plot_data_plane to
plot
this data for met-5.2 and met-6.0:

# 5.2 works fine
met-5.2/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
level="(*,*)";'
DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
DEBUG 1: Creating postscript file: ivt_5.2.ps

# 6.0 errors out
met-6.0/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
level="(*,*)";'
DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
terminate called after throwing an instance of
'netCDF::exceptions::NcChar'
  what():  NcChar: Attempt to convert between text & numbers
file: ncVar.cpp  line:1521
Abort

Does this error match the behavior you're seeing?

Thanks,
John

On Tue, May 23, 2017 at 11:33 AM, Dehaan, Laurel via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80583 >
>
> Thank you John!
> I just posted a file (wrf.ivt.2016041212.nc).
> Of note - I am not confident this is a CF-compliant file.
> I am working with integrated vapor transport (IVT), and could not
find
> that in the CF table - only the vector components of IVT.  However,
when it
> worked in 5.2 I decided it was "good enough".
>
> I appreciate your help on this!
>
> -Laurel
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, May 23, 2017 11:15 AM
> To: Dehaan, Laurel
> Subject: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF
>
> Hi Laurel,
>
> I see you're having trouble getting version 6.0 of MET to read a
> CF-compliant NetCDF file that version 5.2 was able to read fine.
This
> behavior may or may not be related to fact that you're running MET
via
> docker.
>
> It'd probably be easiest to just have you send me the problematic
file.
> You can post it to our anonymous ftp site following these
instructions:
>    http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Please write back once you've posted it, and I'll go grab it.
>
> I'm in the office today, but will be out tomorrow through June 5th.
So
> hopefully we can move quickly and resolve this issue today.
>
> Thanks,
> John
>
>
>

------------------------------------------------
Subject: Problem reading CF-Compliant NetCDF
From: John Halley Gotway
Time: Tue May 23 12:04:23 2017

Laurel,

I see the cause of the different behavior.  met-6.0 is trying to
parse/store the timing information, and that's where the error
happens.

Your time isn't stored in a CF-compliant way.  Ultimately, MET is
trying to
read "2016-04-12_12:00:00" as a double which leads to an error.  It'd
obviously be preferable for MET to write a useful error message rather
than
aborting.  I'll use your data file to add one for the next version.

But for now, I'd suggest making your file "more CF-compliant".  The
"time"
variable should a double or float and have a single dimension.  It
should
have an attribute named "units" which defines how to interpret the
time
values.

I did the following to get this working:

(1) Run "ncdump" to dump the file you sent me to ascii:
   ncdump wrf.ivt.2016041212.nc > wrf.ivt.2016041212.ncdump

(2) Edit wrf.ivt.2016041212.ncdump as follows:
   - Remove the "DateStrLen" dimension.
   - Change "time" variable from "char" to "float" and make it one
dimensional by removing the "DateStrLen" dimension.
   - Add "units" attribute to the "time" variable:
        float time(Time) ;
                time:units = "seconds since 2016-04-12_12:00:00" ;
   - Edit the value for time, changing...
  FROM:
    time = "2016-04-12_12:00:00" ;
  TO:
    time = 0 ;

(3) Run "ncgen" to generate a new file with these changes:
   ncgen -o wrf.ivt.2016041212.mod.nc wrf.ivt.2016041212.ncdump

(4) Run the new file through plot_data_plane:
   met-6.0/bin/plot_data_plane wrf.ivt.2016041212.mod.nc ivt_6.0.ps
'name="IVT"; level="(*,*)";'

And I attached the resulting output.

Hopefully that'll get you back up and running.  FYI, information for
the CF
convention time coordinate can be found here:

http://cfconventions.org/cf-conventions/v1.6.0/cf-
conventions.html#time-coordinate

Thanks,
John

On Tue, May 23, 2017 at 11:51 AM, Dehaan, Laurel via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80583 >
>
> Yes - that's exactly it
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, May 23, 2017 11:39 AM
> To: Dehaan, Laurel
> Subject: Re: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF
>
> Laurel,
>
> OK, I'm able to replicate a problem here.  I used plot_data_plane to
plot
> this data for met-5.2 and met-6.0:
>
> # 5.2 works fine
> met-5.2/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
> level="(*,*)";'
> DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
> DEBUG 1: Creating postscript file: ivt_5.2.ps
>
> # 6.0 errors out
> met-6.0/bin/plot_data_plane wrf.ivt.2016041212.nc ivt_5.2.ps
'name="IVT";
> level="(*,*)";'
> DEBUG 1: Opening data file: wrf.ivt.2016041212.nc
> terminate called after throwing an instance of
'netCDF::exceptions::NcChar'
>   what():  NcChar: Attempt to convert between text & numbers
> file: ncVar.cpp  line:1521
> Abort
>
> Does this error match the behavior you're seeing?
>
> Thanks,
> John
>
>
> On Tue, May 23, 2017 at 11:33 AM, Dehaan, Laurel via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80583 >
> >
> > Thank you John!
> > I just posted a file (wrf.ivt.2016041212.nc).
> > Of note - I am not confident this is a CF-compliant file.
> > I am working with integrated vapor transport (IVT), and could not
find
> > that in the CF table - only the vector components of IVT.
However, when
> it
> > worked in 5.2 I decided it was "good enough".
> >
> > I appreciate your help on this!
> >
> > -Laurel
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Tuesday, May 23, 2017 11:15 AM
> > To: Dehaan, Laurel
> > Subject: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF
> >
> > Hi Laurel,
> >
> > I see you're having trouble getting version 6.0 of MET to read a
> > CF-compliant NetCDF file that version 5.2 was able to read fine.
This
> > behavior may or may not be related to fact that you're running MET
via
> > docker.
> >
> > It'd probably be easiest to just have you send me the problematic
file.
> > You can post it to our anonymous ftp site following these
instructions:
> >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > Please write back once you've posted it, and I'll go grab it.
> >
> > I'm in the office today, but will be out tomorrow through June
5th.  So
> > hopefully we can move quickly and resolve this issue today.
> >
> > Thanks,
> > John
> >
> >
> >
>
>
>
>

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant NetCDF
From: Dehaan, Laurel
Time: Tue May 23 12:08:34 2017

Thanks John.
That's extremely helpful!
I was worried about IVT itself being CF-compliant.
I wasn't thinking about time.

-Laurel
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, May 23, 2017 12:04 PM
To: Dehaan, Laurel
Subject: Re: [rt.rap.ucar.edu #80583] Problem reading CF-Compliant
NetCDF

Laurel,