[Met_help] [rt.rap.ucar.edu #82266] History for Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
John Halley Gotway via RT
met_help at ucar.edu
Thu Nov 9 11:24:52 MST 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi Tara and MET support,
I am in the early stages of a water level verification study that will
attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
either gridded or point observational ground truth. I would like to
obtain traditional statistics such as Bias, MAE, Correlation, etc.
Rather than re-invent the wheel and develop lots of coding to output
continuous verification statistics, I was wanting to look at other
options like the MET since I will have a decent number of 12-hour,
24-hour, 36-hour, and 48-hour forecasts. I would prefer to operate on
my Windows PC if at all possible. I see that you have a "docker"
option. I have not used this software but does this offer the ability
to install the MET on a Windows platform using docker? If so, do you
think this would be a viable options for me? Is there a compiled
version of MET available that could be downloaded to quickly get running?
Besides the MET, are there any other software and/or scripts that I
could use in a Windows environment to produce statistics and plots for
this kind of study?
Thanks.
Mike Buchanan
Science and Operations Officer
NWS Corpus Christi TX
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: John Halley Gotway
Time: Thu Oct 05 15:16:48 2017
Hello Mike,
I see that you're interested in running a docker image of MET on your
windows machine. I think that it is certainly worth considering!
Docker
is still a relatively new technology for us, but we did use a
dockerized
version of MET for a tutorial last year. And we are currently working
on a
dockerized version of the METViewer database and display package for
displaying the output of MET.
There are a few questions up front that would be good to answer.
First, are you actually able to run docker containers on your windows
machine?
Please go through the following instructions for downloading and
"loading"
the MET-6.0 docker container on your machine:
https://www.dtcenter.org/met/users/downloads/docker_container.php
Once you've loaded that container, the last instruction is to run
"bin/bash" to log onto it. In this way, it serves as a little virtual
machine for running through the MET online tutorial instructions.
Feel
free to go through whatever tutorial exercises you find interesting.
Second, what are the file formats for your data?
ESTOFS, ETSS, and NGOFS
Can MET already read them or is more development work required?
Hope that helps get you started. Feel free to send us some sample
files,
and I'll try running them through MET to see if there are any
formatting
issues. Here's how you can post data to our ftp site:
https://www.dtcenter.org/met/users/support/met_help.php#ftp
As I mentioned, we're still pretty new to docker. Ultimately, I image
you'd like to run a series of commands to perform your verification
routinely. I certainly know what commands you'd run on a Linux
machine.
The trick will be figuring out what command line options are required
to
get the I/O for the docker container working correct.
Thanks,
John Halley Gotway
On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT
<met_help at ucar.edu>
wrote:
>
> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
> Transaction: Ticket created by mike.buchanan at noaa.gov
> Queue: met_help
> Subject: Water Level Forecast Verification from the ESTOFS,
ETSS, and
> NGOFS models
> Owner: Nobody
> Requestors: mike.buchanan at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
>
> Hi Tara and MET support,
>
> I am in the early stages of a water level verification study that
will
> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
> either gridded or point observational ground truth. I would like to
> obtain traditional statistics such as Bias, MAE, Correlation, etc.
> Rather than re-invent the wheel and develop lots of coding to output
> continuous verification statistics, I was wanting to look at other
> options like the MET since I will have a decent number of 12-hour,
> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to operate
on
> my Windows PC if at all possible. I see that you have a "docker"
> option. I have not used this software but does this offer the
ability
> to install the MET on a Windows platform using docker? If so, do you
> think this would be a viable options for me? Is there a compiled
> version of MET available that could be downloaded to quickly get
running?
>
> Besides the MET, are there any other software and/or scripts that I
> could use in a Windows environment to produce statistics and plots
for
> this kind of study?
>
> Thanks.
>
> Mike Buchanan
> Science and Operations Officer
> NWS Corpus Christi TX
>
>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: mike.buchanan
Time: Fri Oct 06 12:43:57 2017
Hi John,
My ITO is going to install Docker on my Windows 10 PC on Monday. Once
Docker is installed, will I need administrator privileges to install
MET
within Docker?
The files formats of these models are as follows:
* ESTOFS (Grib-2 and netCDF)
* ETSS (Grib-2 only)
* NGOFS (netCDF only)
I tried uploading a file to your ftp site for each of the 3 models but
there was no incoming/irap/met_help directory and I could not upload
into the incoming directory due to a permissions error.
I believe I would be most interested in using the Point-Stat Tool
since
I have a set of point observations from one location going back 2
years.
Mike
On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
> Hello Mike,
>
> I see that you're interested in running a docker image of MET on
your
> windows machine. I think that it is certainly worth considering!
Docker
> is still a relatively new technology for us, but we did use a
dockerized
> version of MET for a tutorial last year. And we are currently
working on a
> dockerized version of the METViewer database and display package for
> displaying the output of MET.
>
> There are a few questions up front that would be good to answer.
>
> First, are you actually able to run docker containers on your
windows
> machine?
>
> Please go through the following instructions for downloading and
"loading"
> the MET-6.0 docker container on your machine:
>
https://www.dtcenter.org/met/users/downloads/docker_container.php
>
> Once you've loaded that container, the last instruction is to run
> "bin/bash" to log onto it. In this way, it serves as a little
virtual
> machine for running through the MET online tutorial instructions.
Feel
> free to go through whatever tutorial exercises you find interesting.
>
> Second, what are the file formats for your data?
> ESTOFS, ETSS, and NGOFS
>
> Can MET already read them or is more development work required?
>
> Hope that helps get you started. Feel free to send us some sample
files,
> and I'll try running them through MET to see if there are any
formatting
> issues. Here's how you can post data to our ftp site:
> https://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> As I mentioned, we're still pretty new to docker. Ultimately, I
image
> you'd like to run a series of commands to perform your verification
> routinely. I certainly know what commands you'd run on a Linux
machine.
> The trick will be figuring out what command line options are
required to
> get the I/O for the docker container working correct.
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT
<met_help at ucar.edu>
> wrote:
>
>> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
>> Transaction: Ticket created by mike.buchanan at noaa.gov
>> Queue: met_help
>> Subject: Water Level Forecast Verification from the ESTOFS,
ETSS, and
>> NGOFS models
>> Owner: Nobody
>> Requestors: mike.buchanan at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>>
>>
>> Hi Tara and MET support,
>>
>> I am in the early stages of a water level verification study that
will
>> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
>> either gridded or point observational ground truth. I would like
to
>> obtain traditional statistics such as Bias, MAE, Correlation, etc.
>> Rather than re-invent the wheel and develop lots of coding to
output
>> continuous verification statistics, I was wanting to look at other
>> options like the MET since I will have a decent number of 12-hour,
>> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to operate
on
>> my Windows PC if at all possible. I see that you have a "docker"
>> option. I have not used this software but does this offer the
ability
>> to install the MET on a Windows platform using docker? If so, do
you
>> think this would be a viable options for me? Is there a compiled
>> version of MET available that could be downloaded to quickly get
running?
>>
>> Besides the MET, are there any other software and/or scripts that I
>> could use in a Windows environment to produce statistics and plots
for
>> this kind of study?
>>
>> Thanks.
>>
>> Mike Buchanan
>> Science and Operations Officer
>> NWS Corpus Christi TX
>>
>>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: mike.buchanan
Time: Fri Oct 13 14:07:36 2017
MET Support,
I have Docker installed on my Windows 10 machine and installed the MET
container. The problem is that my government machine uses a CAC card
for credentials and Docker is not able to use these credentials so I
am
unable to mount my external drive. Online answers on this mounting
issue did not seem to offer a nice solution to the problem I am
having.
Any ideas?
Mike
On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
> Hello Mike,
>
> I see that you're interested in running a docker image of MET on
your
> windows machine. I think that it is certainly worth considering!
Docker
> is still a relatively new technology for us, but we did use a
dockerized
> version of MET for a tutorial last year. And we are currently
working on a
> dockerized version of the METViewer database and display package for
> displaying the output of MET.
>
> There are a few questions up front that would be good to answer.
>
> First, are you actually able to run docker containers on your
windows
> machine?
>
> Please go through the following instructions for downloading and
"loading"
> the MET-6.0 docker container on your machine:
>
https://www.dtcenter.org/met/users/downloads/docker_container.php
>
> Once you've loaded that container, the last instruction is to run
> "bin/bash" to log onto it. In this way, it serves as a little
virtual
> machine for running through the MET online tutorial instructions.
Feel
> free to go through whatever tutorial exercises you find interesting.
>
> Second, what are the file formats for your data?
> ESTOFS, ETSS, and NGOFS
>
> Can MET already read them or is more development work required?
>
> Hope that helps get you started. Feel free to send us some sample
files,
> and I'll try running them through MET to see if there are any
formatting
> issues. Here's how you can post data to our ftp site:
> https://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> As I mentioned, we're still pretty new to docker. Ultimately, I
image
> you'd like to run a series of commands to perform your verification
> routinely. I certainly know what commands you'd run on a Linux
machine.
> The trick will be figuring out what command line options are
required to
> get the I/O for the docker container working correct.
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT
<met_help at ucar.edu>
> wrote:
>
>> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
>> Transaction: Ticket created by mike.buchanan at noaa.gov
>> Queue: met_help
>> Subject: Water Level Forecast Verification from the ESTOFS,
ETSS, and
>> NGOFS models
>> Owner: Nobody
>> Requestors: mike.buchanan at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>>
>>
>> Hi Tara and MET support,
>>
>> I am in the early stages of a water level verification study that
will
>> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
>> either gridded or point observational ground truth. I would like
to
>> obtain traditional statistics such as Bias, MAE, Correlation, etc.
>> Rather than re-invent the wheel and develop lots of coding to
output
>> continuous verification statistics, I was wanting to look at other
>> options like the MET since I will have a decent number of 12-hour,
>> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to operate
on
>> my Windows PC if at all possible. I see that you have a "docker"
>> option. I have not used this software but does this offer the
ability
>> to install the MET on a Windows platform using docker? If so, do
you
>> think this would be a viable options for me? Is there a compiled
>> version of MET available that could be downloaded to quickly get
running?
>>
>> Besides the MET, are there any other software and/or scripts that I
>> could use in a Windows environment to produce statistics and plots
for
>> this kind of study?
>>
>> Thanks.
>>
>> Mike Buchanan
>> Science and Operations Officer
>> NWS Corpus Christi TX
>>
>>
>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: John Halley Gotway
Time: Sun Oct 15 19:09:28 2017
Mike,
I'm sorry, no, I don't have any ideas. Docker is still a relatively
new
technology for the weather community, and these sorts of configuration
issues are inevitable. I'd suggest asking your local sys admins to
see if
they have any suggestions.
So are you not able to run *any* docker containers on your machine for
the
same reason?
We could go through and compile all the external libraries and the MET
tools directly... but of course that negates the benefit of using
containers in the first place!
Thanks,
John
On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
> MET Support,
>
> I have Docker installed on my Windows 10 machine and installed the
MET
> container. The problem is that my government machine uses a CAC
card
> for credentials and Docker is not able to use these credentials so I
am
> unable to mount my external drive. Online answers on this mounting
> issue did not seem to offer a nice solution to the problem I am
having.
> Any ideas?
>
> Mike
>
> On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
> > Hello Mike,
> >
> > I see that you're interested in running a docker image of MET on
your
> > windows machine. I think that it is certainly worth considering!
Docker
> > is still a relatively new technology for us, but we did use a
dockerized
> > version of MET for a tutorial last year. And we are currently
working
> on a
> > dockerized version of the METViewer database and display package
for
> > displaying the output of MET.
> >
> > There are a few questions up front that would be good to answer.
> >
> > First, are you actually able to run docker containers on your
windows
> > machine?
> >
> > Please go through the following instructions for downloading and
> "loading"
> > the MET-6.0 docker container on your machine:
> >
https://www.dtcenter.org/met/users/downloads/docker_container.php
> >
> > Once you've loaded that container, the last instruction is to run
> > "bin/bash" to log onto it. In this way, it serves as a little
virtual
> > machine for running through the MET online tutorial instructions.
Feel
> > free to go through whatever tutorial exercises you find
interesting.
> >
> > Second, what are the file formats for your data?
> > ESTOFS, ETSS, and NGOFS
> >
> > Can MET already read them or is more development work required?
> >
> > Hope that helps get you started. Feel free to send us some sample
files,
> > and I'll try running them through MET to see if there are any
formatting
> > issues. Here's how you can post data to our ftp site:
> > https://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > As I mentioned, we're still pretty new to docker. Ultimately, I
image
> > you'd like to run a series of commands to perform your
verification
> > routinely. I certainly know what commands you'd run on a Linux
machine.
> > The trick will be figuring out what command line options are
required to
> > get the I/O for the docker container working correct.
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
> >> Transaction: Ticket created by mike.buchanan at noaa.gov
> >> Queue: met_help
> >> Subject: Water Level Forecast Verification from the ESTOFS,
ETSS,
> and
> >> NGOFS models
> >> Owner: Nobody
> >> Requestors: mike.buchanan at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266
> >
> >>
> >>
> >> Hi Tara and MET support,
> >>
> >> I am in the early stages of a water level verification study that
will
> >> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output
to
> >> either gridded or point observational ground truth. I would like
to
> >> obtain traditional statistics such as Bias, MAE, Correlation,
etc.
> >> Rather than re-invent the wheel and develop lots of coding to
output
> >> continuous verification statistics, I was wanting to look at
other
> >> options like the MET since I will have a decent number of 12-
hour,
> >> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to
operate on
> >> my Windows PC if at all possible. I see that you have a "docker"
> >> option. I have not used this software but does this offer the
ability
> >> to install the MET on a Windows platform using docker? If so, do
you
> >> think this would be a viable options for me? Is there a compiled
> >> version of MET available that could be downloaded to quickly get
> running?
> >>
> >> Besides the MET, are there any other software and/or scripts that
I
> >> could use in a Windows environment to produce statistics and
plots for
> >> this kind of study?
> >>
> >> Thanks.
> >>
> >> Mike Buchanan
> >> Science and Operations Officer
> >> NWS Corpus Christi TX
> >>
> >>
> >
>
>
>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: mike.buchanan
Time: Sun Oct 15 21:26:08 2017
Hi John,
Instead of using my government PC (which I do not have admin rights
and
CAC Card authentication really limits what I can do), I think my
better
option may be to use my home laptop. I installed VMWare 12.5 player
with macOS Sierra as a guest OS. I was able to install Docker for Mac
in this VM. I have the MET Docker image installed. I think I am real
close to getting this to work. I just need to mount my local external
disk (where I have my observation and forecast data) into a Docker
container.
I already did some prep work to format the observation data into the
11-field format that ASCIINC requires.
Mike
On 10/15/2017 8:09 PM, John Halley Gotway via RT wrote:
> Mike,
>
> I'm sorry, no, I don't have any ideas. Docker is still a relatively
new
> technology for the weather community, and these sorts of
configuration
> issues are inevitable. I'd suggest asking your local sys admins to
see if
> they have any suggestions.
>
> So are you not able to run *any* docker containers on your machine
for the
> same reason?
>
> We could go through and compile all the external libraries and the
MET
> tools directly... but of course that negates the benefit of using
> containers in the first place!
>
> Thanks,
> John
>
> On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu>
> wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>>
>> MET Support,
>>
>> I have Docker installed on my Windows 10 machine and installed the
MET
>> container. The problem is that my government machine uses a CAC
card
>> for credentials and Docker is not able to use these credentials so
I am
>> unable to mount my external drive. Online answers on this mounting
>> issue did not seem to offer a nice solution to the problem I am
having.
>> Any ideas?
>>
>> Mike
>>
>> On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
>>> Hello Mike,
>>>
>>> I see that you're interested in running a docker image of MET on
your
>>> windows machine. I think that it is certainly worth considering!
Docker
>>> is still a relatively new technology for us, but we did use a
dockerized
>>> version of MET for a tutorial last year. And we are currently
working
>> on a
>>> dockerized version of the METViewer database and display package
for
>>> displaying the output of MET.
>>>
>>> There are a few questions up front that would be good to answer.
>>>
>>> First, are you actually able to run docker containers on your
windows
>>> machine?
>>>
>>> Please go through the following instructions for downloading and
>> "loading"
>>> the MET-6.0 docker container on your machine:
>>>
https://www.dtcenter.org/met/users/downloads/docker_container.php
>>>
>>> Once you've loaded that container, the last instruction is to run
>>> "bin/bash" to log onto it. In this way, it serves as a little
virtual
>>> machine for running through the MET online tutorial instructions.
Feel
>>> free to go through whatever tutorial exercises you find
interesting.
>>>
>>> Second, what are the file formats for your data?
>>> ESTOFS, ETSS, and NGOFS
>>>
>>> Can MET already read them or is more development work required?
>>>
>>> Hope that helps get you started. Feel free to send us some sample
files,
>>> and I'll try running them through MET to see if there are any
formatting
>>> issues. Here's how you can post data to our ftp site:
>>> https://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> As I mentioned, we're still pretty new to docker. Ultimately, I
image
>>> you'd like to run a series of commands to perform your
verification
>>> routinely. I certainly know what commands you'd run on a Linux
machine.
>>> The trick will be figuring out what command line options are
required to
>>> get the I/O for the docker container working correct.
>>>
>>> Thanks,
>>> John Halley Gotway
>>>
>>>
>>> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT
<met_help at ucar.edu>
>>> wrote:
>>>
>>>> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
>>>> Transaction: Ticket created by mike.buchanan at noaa.gov
>>>> Queue: met_help
>>>> Subject: Water Level Forecast Verification from the
ESTOFS, ETSS,
>> and
>>>> NGOFS models
>>>> Owner: Nobody
>>>> Requestors: mike.buchanan at noaa.gov
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266
>>>>
>>>> Hi Tara and MET support,
>>>>
>>>> I am in the early stages of a water level verification study that
will
>>>> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output
to
>>>> either gridded or point observational ground truth. I would like
to
>>>> obtain traditional statistics such as Bias, MAE, Correlation,
etc.
>>>> Rather than re-invent the wheel and develop lots of coding to
output
>>>> continuous verification statistics, I was wanting to look at
other
>>>> options like the MET since I will have a decent number of 12-
hour,
>>>> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to
operate on
>>>> my Windows PC if at all possible. I see that you have a "docker"
>>>> option. I have not used this software but does this offer the
ability
>>>> to install the MET on a Windows platform using docker? If so, do
you
>>>> think this would be a viable options for me? Is there a compiled
>>>> version of MET available that could be downloaded to quickly get
>> running?
>>>> Besides the MET, are there any other software and/or scripts that
I
>>>> could use in a Windows environment to produce statistics and
plots for
>>>> this kind of study?
>>>>
>>>> Thanks.
>>>>
>>>> Mike Buchanan
>>>> Science and Operations Officer
>>>> NWS Corpus Christi TX
>>>>
>>>>
>>
>>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: John Halley Gotway
Time: Wed Oct 18 09:55:03 2017
Mike,
Glad you've been able to make progress!
Placing your input forecast and observation data into a docker
container
certainly is an option, but I don't think that it's required. I
certainly
am not an expert on this... but another option is just mounting local
directories from your machine into the container.
Below, I've cut-and-pasted 2 lines from this file
(https://github.com/NCAR/
container-dtc-met/blob/master/met/README.docker.txt):
docker run -it -v ${MET_TUTORIAL_DIR}/tutorial:/met/met-
6.0/tutorial
met-6.0-tutorial /bin/bash
cd /met/met-6.0
That launches a bash shell inside the container. But look closely at
the
"-v" option which is used to mount a "volume". It maps the
${MET_TUTORIAL_DIR}/tutorial directory from your local machine into
the
/met/met-6.0/tutorial directory inside the container.
You can use one or more "-v" options to map the location of your input
forecast and observation data into a convenient location inside the
container. When running commands inside the container, you'd just
reference the mapped location.
Make sense?
Thanks,
John
On Sun, Oct 15, 2017 at 9:26 PM, mike.buchanan via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
> Hi John,
>
> Instead of using my government PC (which I do not have admin rights
and
> CAC Card authentication really limits what I can do), I think my
better
> option may be to use my home laptop. I installed VMWare 12.5 player
> with macOS Sierra as a guest OS. I was able to install Docker for
Mac
> in this VM. I have the MET Docker image installed. I think I am
real
> close to getting this to work. I just need to mount my local
external
> disk (where I have my observation and forecast data) into a Docker
> container.
>
> I already did some prep work to format the observation data into the
> 11-field format that ASCIINC requires.
>
> Mike
>
> On 10/15/2017 8:09 PM, John Halley Gotway via RT wrote:
> > Mike,
> >
> > I'm sorry, no, I don't have any ideas. Docker is still a
relatively new
> > technology for the weather community, and these sorts of
configuration
> > issues are inevitable. I'd suggest asking your local sys admins
to see
> if
> > they have any suggestions.
> >
> > So are you not able to run *any* docker containers on your machine
for
> the
> > same reason?
> >
> > We could go through and compile all the external libraries and the
MET
> > tools directly... but of course that negates the benefit of using
> > containers in the first place!
> >
> > Thanks,
> > John
> >
> > On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
> >>
> >> MET Support,
> >>
> >> I have Docker installed on my Windows 10 machine and installed
the MET
> >> container. The problem is that my government machine uses a CAC
card
> >> for credentials and Docker is not able to use these credentials
so I am
> >> unable to mount my external drive. Online answers on this
mounting
> >> issue did not seem to offer a nice solution to the problem I am
having.
> >> Any ideas?
> >>
> >> Mike
> >>
> >> On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
> >>> Hello Mike,
> >>>
> >>> I see that you're interested in running a docker image of MET on
your
> >>> windows machine. I think that it is certainly worth
considering!
> Docker
> >>> is still a relatively new technology for us, but we did use a
> dockerized
> >>> version of MET for a tutorial last year. And we are currently
working
> >> on a
> >>> dockerized version of the METViewer database and display package
for
> >>> displaying the output of MET.
> >>>
> >>> There are a few questions up front that would be good to answer.
> >>>
> >>> First, are you actually able to run docker containers on your
windows
> >>> machine?
> >>>
> >>> Please go through the following instructions for downloading and
> >> "loading"
> >>> the MET-6.0 docker container on your machine:
> >>>
https://www.dtcenter.org/met/users/downloads/docker_container.php
> >>>
> >>> Once you've loaded that container, the last instruction is to
run
> >>> "bin/bash" to log onto it. In this way, it serves as a little
virtual
> >>> machine for running through the MET online tutorial
instructions. Feel
> >>> free to go through whatever tutorial exercises you find
interesting.
> >>>
> >>> Second, what are the file formats for your data?
> >>> ESTOFS, ETSS, and NGOFS
> >>>
> >>> Can MET already read them or is more development work required?
> >>>
> >>> Hope that helps get you started. Feel free to send us some
sample
> files,
> >>> and I'll try running them through MET to see if there are any
> formatting
> >>> issues. Here's how you can post data to our ftp site:
> >>>
https://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>>
> >>> As I mentioned, we're still pretty new to docker. Ultimately, I
image
> >>> you'd like to run a series of commands to perform your
verification
> >>> routinely. I certainly know what commands you'd run on a Linux
> machine.
> >>> The trick will be figuring out what command line options are
required
> to
> >>> get the I/O for the docker container working correct.
> >>>
> >>> Thanks,
> >>> John Halley Gotway
> >>>
> >>>
> >>> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT <
> met_help at ucar.edu>
> >>> wrote:
> >>>
> >>>> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
> >>>> Transaction: Ticket created by mike.buchanan at noaa.gov
> >>>> Queue: met_help
> >>>> Subject: Water Level Forecast Verification from the
ESTOFS,
> ETSS,
> >> and
> >>>> NGOFS models
> >>>> Owner: Nobody
> >>>> Requestors: mike.buchanan at noaa.gov
> >>>> Status: new
> >>>> Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> ket/Display.html?id=82266
> >>>>
> >>>> Hi Tara and MET support,
> >>>>
> >>>> I am in the early stages of a water level verification study
that will
> >>>> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output
to
> >>>> either gridded or point observational ground truth. I would
like to
> >>>> obtain traditional statistics such as Bias, MAE, Correlation,
etc.
> >>>> Rather than re-invent the wheel and develop lots of coding to
output
> >>>> continuous verification statistics, I was wanting to look at
other
> >>>> options like the MET since I will have a decent number of 12-
hour,
> >>>> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to
operate on
> >>>> my Windows PC if at all possible. I see that you have a
"docker"
> >>>> option. I have not used this software but does this offer the
ability
> >>>> to install the MET on a Windows platform using docker? If so,
do you
> >>>> think this would be a viable options for me? Is there a
compiled
> >>>> version of MET available that could be downloaded to quickly
get
> >> running?
> >>>> Besides the MET, are there any other software and/or scripts
that I
> >>>> could use in a Windows environment to produce statistics and
plots for
> >>>> this kind of study?
> >>>>
> >>>> Thanks.
> >>>>
> >>>> Mike Buchanan
> >>>> Science and Operations Officer
> >>>> NWS Corpus Christi TX
> >>>>
> >>>>
> >>
> >>
>
>
>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: mike.buchanan
Time: Wed Oct 18 11:36:29 2017
Hi John,
I was finally able to successfully run the ascii2nc tool inside the
docker container last night! I did use the "-v" switch to mount my
NTFS-formatted windows external hard drive. I was able to output the
netCDF file to this external drive. I was able to write to a NTFS
drive
using Paragon's NTFS driver
<https://www.paragon-software.com/ufsdhome/ntfs-mac/>on my VMWare
Players' guest OS, macOS Sierra.
Now I need to edit the Point-Stat configuration file so I can obtain
continuous stats I need for my study. My initial thoughts are to
generate stats such as the Pearson correlation, MAE, RMSE, and Bias.
Perhaps, some CI stats as well.
My observations netCDF file contains 219 observations for one
location.
I have grib2 and netCDF water level forecasts for the ESTOFS, ETSS,
and
NGOFS models that go back at least 48 hours prior to the valid time of
my observations. So, while I would like stats for all forecasts, I
still would like to break this down into how well did a 12-hour,
24-hour, 36-hour, and/or 48-hour forecast performed. I am assuming I
would just simply change the "lead_time" variable to something like
"12", "24", "36", and/or "48" to obtain this kind of data for each
"lead
time?
Mike
On 10/18/2017 10:55 AM, John Halley Gotway via RT wrote:
> Mike,
>
> Glad you've been able to make progress!
>
> Placing your input forecast and observation data into a docker
container
> certainly is an option, but I don't think that it's required. I
certainly
> am not an expert on this... but another option is just mounting
local
> directories from your machine into the container.
>
> Below, I've cut-and-pasted 2 lines from this file
(https://github.com/NCAR/
> container-dtc-met/blob/master/met/README.docker.txt):
>
> docker run -it -v ${MET_TUTORIAL_DIR}/tutorial:/met/met-
6.0/tutorial
> met-6.0-tutorial /bin/bash
> cd /met/met-6.0
>
> That launches a bash shell inside the container. But look closely
at the
> "-v" option which is used to mount a "volume". It maps the
> ${MET_TUTORIAL_DIR}/tutorial directory from your local machine into
the
> /met/met-6.0/tutorial directory inside the container.
>
> You can use one or more "-v" options to map the location of your
input
> forecast and observation data into a convenient location inside the
> container. When running commands inside the container, you'd just
> reference the mapped location.
>
> Make sense?
>
> Thanks,
> John
>
>
> On Sun, Oct 15, 2017 at 9:26 PM, mike.buchanan via RT
<met_help at ucar.edu>
> wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>>
>> Hi John,
>>
>> Instead of using my government PC (which I do not have admin rights
and
>> CAC Card authentication really limits what I can do), I think my
better
>> option may be to use my home laptop. I installed VMWare 12.5
player
>> with macOS Sierra as a guest OS. I was able to install Docker for
Mac
>> in this VM. I have the MET Docker image installed. I think I am
real
>> close to getting this to work. I just need to mount my local
external
>> disk (where I have my observation and forecast data) into a Docker
>> container.
>>
>> I already did some prep work to format the observation data into
the
>> 11-field format that ASCIINC requires.
>>
>> Mike
>>
>> On 10/15/2017 8:09 PM, John Halley Gotway via RT wrote:
>>> Mike,
>>>
>>> I'm sorry, no, I don't have any ideas. Docker is still a
relatively new
>>> technology for the weather community, and these sorts of
configuration
>>> issues are inevitable. I'd suggest asking your local sys admins
to see
>> if
>>> they have any suggestions.
>>>
>>> So are you not able to run *any* docker containers on your machine
for
>> the
>>> same reason?
>>>
>>> We could go through and compile all the external libraries and the
MET
>>> tools directly... but of course that negates the benefit of using
>>> containers in the first place!
>>>
>>> Thanks,
>>> John
>>>
>>> On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu
>>>
>>> wrote:
>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>>>>
>>>> MET Support,
>>>>
>>>> I have Docker installed on my Windows 10 machine and installed
the MET
>>>> container. The problem is that my government machine uses a CAC
card
>>>> for credentials and Docker is not able to use these credentials
so I am
>>>> unable to mount my external drive. Online answers on this
mounting
>>>> issue did not seem to offer a nice solution to the problem I am
having.
>>>> Any ideas?
>>>>
>>>> Mike
>>>>
>>>> On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
>>>>> Hello Mike,
>>>>>
>>>>> I see that you're interested in running a docker image of MET on
your
>>>>> windows machine. I think that it is certainly worth
considering!
>> Docker
>>>>> is still a relatively new technology for us, but we did use a
>> dockerized
>>>>> version of MET for a tutorial last year. And we are currently
working
>>>> on a
>>>>> dockerized version of the METViewer database and display package
for
>>>>> displaying the output of MET.
>>>>>
>>>>> There are a few questions up front that would be good to answer.
>>>>>
>>>>> First, are you actually able to run docker containers on your
windows
>>>>> machine?
>>>>>
>>>>> Please go through the following instructions for downloading and
>>>> "loading"
>>>>> the MET-6.0 docker container on your machine:
>>>>>
https://www.dtcenter.org/met/users/downloads/docker_container.php
>>>>>
>>>>> Once you've loaded that container, the last instruction is to
run
>>>>> "bin/bash" to log onto it. In this way, it serves as a little
virtual
>>>>> machine for running through the MET online tutorial
instructions. Feel
>>>>> free to go through whatever tutorial exercises you find
interesting.
>>>>>
>>>>> Second, what are the file formats for your data?
>>>>> ESTOFS, ETSS, and NGOFS
>>>>>
>>>>> Can MET already read them or is more development work required?
>>>>>
>>>>> Hope that helps get you started. Feel free to send us some
sample
>> files,
>>>>> and I'll try running them through MET to see if there are any
>> formatting
>>>>> issues. Here's how you can post data to our ftp site:
>>>>>
https://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> As I mentioned, we're still pretty new to docker. Ultimately, I
image
>>>>> you'd like to run a series of commands to perform your
verification
>>>>> routinely. I certainly know what commands you'd run on a Linux
>> machine.
>>>>> The trick will be figuring out what command line options are
required
>> to
>>>>> get the I/O for the docker container working correct.
>>>>>
>>>>> Thanks,
>>>>> John Halley Gotway
>>>>>
>>>>>
>>>>> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT <
>> met_help at ucar.edu>
>>>>> wrote:
>>>>>
>>>>>> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
>>>>>> Transaction: Ticket created by mike.buchanan at noaa.gov
>>>>>> Queue: met_help
>>>>>> Subject: Water Level Forecast Verification from the
ESTOFS,
>> ETSS,
>>>> and
>>>>>> NGOFS models
>>>>>> Owner: Nobody
>>>>>> Requestors: mike.buchanan at noaa.gov
>>>>>> Status: new
>>>>>> Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>> ket/Display.html?id=82266
>>>>>> Hi Tara and MET support,
>>>>>>
>>>>>> I am in the early stages of a water level verification study
that will
>>>>>> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output
to
>>>>>> either gridded or point observational ground truth. I would
like to
>>>>>> obtain traditional statistics such as Bias, MAE, Correlation,
etc.
>>>>>> Rather than re-invent the wheel and develop lots of coding to
output
>>>>>> continuous verification statistics, I was wanting to look at
other
>>>>>> options like the MET since I will have a decent number of 12-
hour,
>>>>>> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to
operate on
>>>>>> my Windows PC if at all possible. I see that you have a
"docker"
>>>>>> option. I have not used this software but does this offer the
ability
>>>>>> to install the MET on a Windows platform using docker? If so,
do you
>>>>>> think this would be a viable options for me? Is there a
compiled
>>>>>> version of MET available that could be downloaded to quickly
get
>>>> running?
>>>>>> Besides the MET, are there any other software and/or scripts
that I
>>>>>> could use in a Windows environment to produce statistics and
plots for
>>>>>> this kind of study?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Mike Buchanan
>>>>>> Science and Operations Officer
>>>>>> NWS Corpus Christi TX
>>>>>>
>>>>>>
>>>>
>>
>>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: mike.buchanan
Time: Wed Oct 18 22:52:52 2017
Hi John,
Is the grib code used in ASCII2NC have to be GRIB1 or GRIB2? If the
former, I do not see a GRIB1 Code for "Extra Tropical Storm Surge
Combined
Surge and Tide", which is the field I am interested in using. In
running
the Point_Stat tool, I was getting these kinds of errors:
ERROR : VarInfoGrib2::set_dict() -> unrecognized GRIB2 field
abbreviation
'Extra_Tropical_Storm_Surge_Combined_Surge_and_Tide_surface'
or when I change the pointStats config file to use "ETCWL" , I get
ERROR : VarInfoGrib::add_grib_code() -> unrecognized GRIB1 field
abbreviation 'ETCWL' for table version 2
The forecast is in GRIB2 format.
Mike
*Michael BuchananScience & Operations Officer*National Weather Service
Corpus Christi, Texas
Office: (361) 371-3169
On Wed, Oct 18, 2017 at 12:36 PM, mike.buchanan
<mike.buchanan at noaa.gov>
wrote:
> Hi John,
>
> I was finally able to successfully run the ascii2nc tool inside the
docker
> container last night! I did use the "-v" switch to mount my NTFS-
formatted
> windows external hard drive. I was able to output the netCDF file
to this
> external drive. I was able to write to a NTFS drive using Paragon's
NTFS
> driver <https://www.paragon-software.com/ufsdhome/ntfs-mac/>on my
VMWare
> Players' guest OS, macOS Sierra.
>
> Now I need to edit the Point-Stat configuration file so I can obtain
> continuous stats I need for my study. My initial thoughts are to
generate
> stats such as the Pearson correlation, MAE, RMSE, and Bias.
Perhaps,
> some CI stats as well.
>
> My observations netCDF file contains 219 observations for one
location. I
> have grib2 and netCDF water level forecasts for the ESTOFS, ETSS,
and NGOFS
> models that go back at least 48 hours prior to the valid time of my
> observations. So, while I would like stats for all forecasts, I
still
> would like to break this down into how well did a 12-hour, 24-hour,
> 36-hour, and/or 48-hour forecast performed. I am assuming I would
just
> simply change the "lead_time" variable to something like "12", "24",
"36",
> and/or "48" to obtain this kind of data for each "lead time?
>
> Mike
>
> On 10/18/2017 10:55 AM, John Halley Gotway via RT wrote:
>
> Mike,
>
> Glad you've been able to make progress!
>
> Placing your input forecast and observation data into a docker
container
> certainly is an option, but I don't think that it's required. I
certainly
> am not an expert on this... but another option is just mounting
local
> directories from your machine into the container.
>
> Below, I've cut-and-pasted 2 lines from this file
(https://github.com/NCAR/
> container-dtc-met/blob/master/met/README.docker.txt):
>
> docker run -it -v ${MET_TUTORIAL_DIR}/tutorial:/met/met-
6.0/tutorial
> met-6.0-tutorial /bin/bash
> cd /met/met-6.0
>
> That launches a bash shell inside the container. But look closely
at the
> "-v" option which is used to mount a "volume". It maps the
> ${MET_TUTORIAL_DIR}/tutorial directory from your local machine into
the
> /met/met-6.0/tutorial directory inside the container.
>
> You can use one or more "-v" options to map the location of your
input
> forecast and observation data into a convenient location inside the
> container. When running commands inside the container, you'd just
> reference the mapped location.
>
> Make sense?
>
> Thanks,
> John
>
>
> On Sun, Oct 15, 2017 at 9:26 PM, mike.buchanan via RT
<met_help at ucar.edu> <met_help at ucar.edu>
> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
> Hi John,
>
> Instead of using my government PC (which I do not have admin rights
and
> CAC Card authentication really limits what I can do), I think my
better
> option may be to use my home laptop. I installed VMWare 12.5 player
> with macOS Sierra as a guest OS. I was able to install Docker for
Mac
> in this VM. I have the MET Docker image installed. I think I am
real
> close to getting this to work. I just need to mount my local
external
> disk (where I have my observation and forecast data) into a Docker
> container.
>
> I already did some prep work to format the observation data into the
> 11-field format that ASCIINC requires.
>
> Mike
>
> On 10/15/2017 8:09 PM, John Halley Gotway via RT wrote:
>
> Mike,
>
> I'm sorry, no, I don't have any ideas. Docker is still a relatively
new
> technology for the weather community, and these sorts of
configuration
> issues are inevitable. I'd suggest asking your local sys admins to
see
>
> if
>
> they have any suggestions.
>
> So are you not able to run *any* docker containers on your machine
for
>
> the
>
> same reason?
>
> We could go through and compile all the external libraries and the
MET
> tools directly... but of course that negates the benefit of using
> containers in the first place!
>
> Thanks,
> John
>
> On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu
>
> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
> MET Support,
>
> I have Docker installed on my Windows 10 machine and installed the
MET
> container. The problem is that my government machine uses a CAC
card
> for credentials and Docker is not able to use these credentials so I
am
> unable to mount my external drive. Online answers on this mounting
> issue did not seem to offer a nice solution to the problem I am
having.
> Any ideas?
>
> Mike
>
> On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
>
> Hello Mike,
>
> I see that you're interested in running a docker image of MET on
your
> windows machine. I think that it is certainly worth considering!
>
> Docker
>
> is still a relatively new technology for us, but we did use a
>
> dockerized
>
> version of MET for a tutorial last year. And we are currently
working
>
> on a
>
> dockerized version of the METViewer database and display package for
> displaying the output of MET.
>
> There are a few questions up front that would be good to answer.
>
> First, are you actually able to run docker containers on your
windows
> machine?
>
> Please go through the following instructions for downloading and
>
> "loading"
>
> the MET-6.0 docker container on your machine:
>
https://www.dtcenter.org/met/users/downloads/docker_container.php
>
> Once you've loaded that container, the last instruction is to run
> "bin/bash" to log onto it. In this way, it serves as a little
virtual
> machine for running through the MET online tutorial instructions.
Feel
> free to go through whatever tutorial exercises you find interesting.
>
> Second, what are the file formats for your data?
> ESTOFS, ETSS, and NGOFS
>
> Can MET already read them or is more development work required?
>
> Hope that helps get you started. Feel free to send us some sample
>
> files,
>
> and I'll try running them through MET to see if there are any
>
> formatting
>
> issues. Here's how you can post data to our ftp site:
> https://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> As I mentioned, we're still pretty new to docker. Ultimately, I
image
> you'd like to run a series of commands to perform your verification
> routinely. I certainly know what commands you'd run on a Linux
>
> machine.
>
> The trick will be figuring out what command line options are
required
>
> to
>
> get the I/O for the docker container working correct.
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT <
>
> met_help at ucar.edu>
>
> wrote:
>
>
> Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
> Transaction: Ticket created by mike.buchanan at noaa.gov
> Queue: met_help
> Subject: Water Level Forecast Verification from the ESTOFS,
>
> ETSS,
>
> and
>
> NGOFS models
> Owner: Nobody
> Requestors: mike.buchanan at noaa.gov
> Status: new
> Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>
> ket/Display.html?id=82266
>
> Hi Tara and MET support,
>
> I am in the early stages of a water level verification study that
will
> attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
> either gridded or point observational ground truth. I would like to
> obtain traditional statistics such as Bias, MAE, Correlation, etc.
> Rather than re-invent the wheel and develop lots of coding to output
> continuous verification statistics, I was wanting to look at other
> options like the MET since I will have a decent number of 12-hour,
> 24-hour, 36-hour, and 48-hour forecasts. I would prefer to operate
on
> my Windows PC if at all possible. I see that you have a "docker"
> option. I have not used this software but does this offer the
ability
> to install the MET on a Windows platform using docker? If so, do you
> think this would be a viable options for me? Is there a compiled
> version of MET available that could be downloaded to quickly get
>
> running?
>
> Besides the MET, are there any other software and/or scripts that I
> could use in a Windows environment to produce statistics and plots
for
> this kind of study?
>
> Thanks.
>
> Mike Buchanan
> Science and Operations Officer
> NWS Corpus Christi TX
>
>
>
>
>
>
------------------------------------------------
Subject: Water Level Forecast Verification from the ESTOFS, ETSS, and NGOFS models
From: John Halley Gotway
Time: Fri Oct 20 17:09:27 2017
Mike,
Sorry for the delay in getting back to you about this issue.
The answer is a little messy...
The current public release of MET is version 6.0. And here's a
description
of the 11-column ascii format expected by ascii2nc:
https://dtcenter.org/met/users/support/online_tutorial/
METv6.0/tutorial.php?name=ascii2nc&category=index
The 7th column must be a GRIB version 1 code which identifies the type
of
observation for that line. The GRIB1 table usage is defined here:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html
Perhaps you'd want to use 228 for "Storm Surge" from parameter table
version number 129.
Here's how you'd set up the "obs" section of your Point-Stat config
file:
obs = {
field = [
{
name = "SURGE";
level = [ "Z0" ];
cat_thresh = [ >80.0 ];
GRIB1_ptv = 129;
}
];
}
The important detail here is that I specified "GRIB1_ptv" as 129 to
tell
Point-Stat to look in parameter table version number 129 when trying
to
interpret the string "SURGE".
Now, we realize that this requirement of mapping observation types to
existing GRIB1 codes is a bit cumbersome. So we're generalizing
things a
bit in the next release, version 6.1. Instead of requiring that
column 7
be set to a GRIB code, ascii2nc will also support strings. So you
could
set column 7 as
"Extra_Tropical_Storm_Surge_Combined_Surge_and_Tide_surface"
or whatever other string you'd like. And then in the Point-Stat
config
file, you'd refer to that same name.
We hope that this change will make things more intuitive and flexible.
Hope that helps.
Thanks,
John
On Wed, Oct 18, 2017 at 10:52 PM, mike.buchanan via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
>
> Hi John,
>
> Is the grib code used in ASCII2NC have to be GRIB1 or GRIB2? If the
> former, I do not see a GRIB1 Code for "Extra Tropical Storm Surge
Combined
> Surge and Tide", which is the field I am interested in using. In
running
> the Point_Stat tool, I was getting these kinds of errors:
>
> ERROR : VarInfoGrib2::set_dict() -> unrecognized GRIB2 field
abbreviation
> 'Extra_Tropical_Storm_Surge_Combined_Surge_and_Tide_surface'
>
> or when I change the pointStats config file to use "ETCWL" , I get
>
> ERROR : VarInfoGrib::add_grib_code() -> unrecognized GRIB1 field
> abbreviation 'ETCWL' for table version 2
>
> The forecast is in GRIB2 format.
>
> Mike
>
>
>
> *Michael BuchananScience & Operations Officer*National Weather
Service
> Corpus Christi, Texas
>
> Office: (361) 371-3169
>
> On Wed, Oct 18, 2017 at 12:36 PM, mike.buchanan
<mike.buchanan at noaa.gov>
> wrote:
>
> > Hi John,
> >
> > I was finally able to successfully run the ascii2nc tool inside
the
> docker
> > container last night! I did use the "-v" switch to mount my
> NTFS-formatted
> > windows external hard drive. I was able to output the netCDF file
to
> this
> > external drive. I was able to write to a NTFS drive using
Paragon's NTFS
> > driver <https://www.paragon-software.com/ufsdhome/ntfs-mac/>on my
VMWare
> > Players' guest OS, macOS Sierra.
> >
> > Now I need to edit the Point-Stat configuration file so I can
obtain
> > continuous stats I need for my study. My initial thoughts are to
> generate
> > stats such as the Pearson correlation, MAE, RMSE, and Bias.
Perhaps,
> > some CI stats as well.
> >
> > My observations netCDF file contains 219 observations for one
location.
> I
> > have grib2 and netCDF water level forecasts for the ESTOFS, ETSS,
and
> NGOFS
> > models that go back at least 48 hours prior to the valid time of
my
> > observations. So, while I would like stats for all forecasts, I
still
> > would like to break this down into how well did a 12-hour, 24-
hour,
> > 36-hour, and/or 48-hour forecast performed. I am assuming I would
just
> > simply change the "lead_time" variable to something like "12",
"24",
> "36",
> > and/or "48" to obtain this kind of data for each "lead time?
> >
> > Mike
> >
> > On 10/18/2017 10:55 AM, John Halley Gotway via RT wrote:
> >
> > Mike,
> >
> > Glad you've been able to make progress!
> >
> > Placing your input forecast and observation data into a docker
container
> > certainly is an option, but I don't think that it's required. I
> certainly
> > am not an expert on this... but another option is just mounting
local
> > directories from your machine into the container.
> >
> > Below, I've cut-and-pasted 2 lines from this file (
> https://github.com/NCAR/
> > container-dtc-met/blob/master/met/README.docker.txt):
> >
> > docker run -it -v ${MET_TUTORIAL_DIR}/tutorial:/met/met-
6.0/tutorial
> > met-6.0-tutorial /bin/bash
> > cd /met/met-6.0
> >
> > That launches a bash shell inside the container. But look closely
at the
> > "-v" option which is used to mount a "volume". It maps the
> > ${MET_TUTORIAL_DIR}/tutorial directory from your local machine
into the
> > /met/met-6.0/tutorial directory inside the container.
> >
> > You can use one or more "-v" options to map the location of your
input
> > forecast and observation data into a convenient location inside
the
> > container. When running commands inside the container, you'd just
> > reference the mapped location.
> >
> > Make sense?
> >
> > Thanks,
> > John
> >
> >
> > On Sun, Oct 15, 2017 at 9:26 PM, mike.buchanan via RT
<met_help at ucar.edu>
> <met_help at ucar.edu>
> > wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
> >
> > Hi John,
> >
> > Instead of using my government PC (which I do not have admin
rights and
> > CAC Card authentication really limits what I can do), I think my
better
> > option may be to use my home laptop. I installed VMWare 12.5
player
> > with macOS Sierra as a guest OS. I was able to install Docker for
Mac
> > in this VM. I have the MET Docker image installed. I think I am
real
> > close to getting this to work. I just need to mount my local
external
> > disk (where I have my observation and forecast data) into a Docker
> > container.
> >
> > I already did some prep work to format the observation data into
the
> > 11-field format that ASCIINC requires.
> >
> > Mike
> >
> > On 10/15/2017 8:09 PM, John Halley Gotway via RT wrote:
> >
> > Mike,
> >
> > I'm sorry, no, I don't have any ideas. Docker is still a
relatively new
> > technology for the weather community, and these sorts of
configuration
> > issues are inevitable. I'd suggest asking your local sys admins
to see
> >
> > if
> >
> > they have any suggestions.
> >
> > So are you not able to run *any* docker containers on your machine
for
> >
> > the
> >
> > same reason?
> >
> > We could go through and compile all the external libraries and the
MET
> > tools directly... but of course that negates the benefit of using
> > containers in the first place!
> >
> > Thanks,
> > John
> >
> > On Fri, Oct 13, 2017 at 2:07 PM, mike.buchanan via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82266 >
> >
> > MET Support,
> >
> > I have Docker installed on my Windows 10 machine and installed the
MET
> > container. The problem is that my government machine uses a CAC
card
> > for credentials and Docker is not able to use these credentials so
I am
> > unable to mount my external drive. Online answers on this
mounting
> > issue did not seem to offer a nice solution to the problem I am
having.
> > Any ideas?
> >
> > Mike
> >
> > On 10/5/2017 4:16 PM, John Halley Gotway via RT wrote:
> >
> > Hello Mike,
> >
> > I see that you're interested in running a docker image of MET on
your
> > windows machine. I think that it is certainly worth considering!
> >
> > Docker
> >
> > is still a relatively new technology for us, but we did use a
> >
> > dockerized
> >
> > version of MET for a tutorial last year. And we are currently
working
> >
> > on a
> >
> > dockerized version of the METViewer database and display package
for
> > displaying the output of MET.
> >
> > There are a few questions up front that would be good to answer.
> >
> > First, are you actually able to run docker containers on your
windows
> > machine?
> >
> > Please go through the following instructions for downloading and
> >
> > "loading"
> >
> > the MET-6.0 docker container on your machine:
> >
https://www.dtcenter.org/met/users/downloads/docker_container.php
> >
> > Once you've loaded that container, the last instruction is to run
> > "bin/bash" to log onto it. In this way, it serves as a little
virtual
> > machine for running through the MET online tutorial instructions.
Feel
> > free to go through whatever tutorial exercises you find
interesting.
> >
> > Second, what are the file formats for your data?
> > ESTOFS, ETSS, and NGOFS
> >
> > Can MET already read them or is more development work required?
> >
> > Hope that helps get you started. Feel free to send us some sample
> >
> > files,
> >
> > and I'll try running them through MET to see if there are any
> >
> > formatting
> >
> > issues. Here's how you can post data to our ftp site:
> > https://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > As I mentioned, we're still pretty new to docker. Ultimately, I
image
> > you'd like to run a series of commands to perform your
verification
> > routinely. I certainly know what commands you'd run on a Linux
> >
> > machine.
> >
> > The trick will be figuring out what command line options are
required
> >
> > to
> >
> > get the I/O for the docker container working correct.
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Thu, Oct 5, 2017 at 2:31 PM, mike.buchanan via RT <
> >
> > met_help at ucar.edu>
> >
> > wrote:
> >
> >
> > Thu Oct 05 14:31:22 2017: Request 82266 was acted upon.
> > Transaction: Ticket created by mike.buchanan at noaa.gov
> > Queue: met_help
> > Subject: Water Level Forecast Verification from the ESTOFS,
> >
> > ETSS,
> >
> > and
> >
> > NGOFS models
> > Owner: Nobody
> > Requestors: mike.buchanan at noaa.gov
> > Status: new
> > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> >
> > ket/Display.html?id=82266
> >
> > Hi Tara and MET support,
> >
> > I am in the early stages of a water level verification study that
will
> > attempt to compare gridded ESTOFS, ETSS, and NGOFS model output to
> > either gridded or point observational ground truth. I would like
to
> > obtain traditional statistics such as Bias, MAE, Correlation, etc.
> > Rather than re-invent the wheel and develop lots of coding to
output
> > continuous verification statistics, I was wanting to look at other
> > options like the MET since I will have a decent number of 12-hour,
> > 24-hour, 36-hour, and 48-hour forecasts. I would prefer to
operate on
> > my Windows PC if at all possible. I see that you have a "docker"
> > option. I have not used this software but does this offer the
ability
> > to install the MET on a Windows platform using docker? If so, do
you
> > think this would be a viable options for me? Is there a compiled
> > version of MET available that could be downloaded to quickly get
> >
> > running?
> >
> > Besides the MET, are there any other software and/or scripts that
I
> > could use in a Windows environment to produce statistics and plots
for
> > this kind of study?
> >
> > Thanks.
> >
> > Mike Buchanan
> > Science and Operations Officer
> > NWS Corpus Christi TX
> >
> >
> >
> >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list