[Met_help] [rt.rap.ucar.edu #94392] History for Running MODE for months of Stage IV Data
John Halley Gotway via RT
met_help at ucar.edu
Wed Mar 4 14:11:04 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I'm currently running into an issue where I'm trying to run MODE, except I
want to loop through a few months of Stage IV 2-year exceedance data using
a python program so that I don't need to change the date and time every
time I run MODE. However, when I run my program, I'm not getting any output
MODE files. I've attached the type of files as well as my program in this
email. If there's any suggestions you have for me, please let me know.
Thanks,
Michael
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Minna Win
Time: Wed Mar 04 09:19:41 2020
Hello Michael,
I am assigning your issue to John Halley Gotway, who is best suited to
assist you. Please allow a few days for a response.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT
<met_help at ucar.edu>
wrote:
>
> Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> Queue: met_help
> Subject: Running MODE for months of Stage IV Data
> Owner: Nobody
> Requestors: mpletch1 at terpmail.umd.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
>
> Hello,
>
> I'm currently running into an issue where I'm trying to run MODE,
except I
> want to loop through a few months of Stage IV 2-year exceedance data
using
> a python program so that I don't need to change the date and time
every
> time I run MODE. However, when I run my program, I'm not getting any
output
> MODE files. I've attached the type of files as well as my program in
this
> email. If there's any suggestions you have for me, please let me
know.
>
> Thanks,
> Michael
>
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 09:50:16 2020
Michael,
I see that you're running MODE to compare a stageIV file to itself,
which
is fine. Looks like your MODE configuration file is named
"test.config".
Can you please send me that file as well?
Thanks,
John Halley Gotway
On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
> Hello Michael,
>
> I am assigning your issue to John Halley Gotway, who is best suited
to
> assist you. Please allow a few days for a response.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> > Queue: met_help
> > Subject: Running MODE for months of Stage IV Data
> > Owner: Nobody
> > Requestors: mpletch1 at terpmail.umd.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> >
> > Hello,
> >
> > I'm currently running into an issue where I'm trying to run MODE,
except
> I
> > want to loop through a few months of Stage IV 2-year exceedance
data
> using
> > a python program so that I don't need to change the date and time
every
> > time I run MODE. However, when I run my program, I'm not getting
any
> output
> > MODE files. I've attached the type of files as well as my program
in this
> > email. If there's any suggestions you have for me, please let me
know.
> >
> > Thanks,
> > Michael
> >
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
> >
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 09:56:03 2020
Hi John,
Thanks for the reply. I've attached the config file in this email.
-Michael
On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> I see that you're running MODE to compare a stageIV file to itself,
which
> is fine. Looks like your MODE configuration file is named
"test.config".
> Can you please send me that file as well?
>
> Thanks,
> John Halley Gotway
>
> On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> > Hello Michael,
> >
> > I am assigning your issue to John Halley Gotway, who is best
suited to
> > assist you. Please allow a few days for a response.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> > > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> > > Queue: met_help
> > > Subject: Running MODE for months of Stage IV Data
> > > Owner: Nobody
> > > Requestors: mpletch1 at terpmail.umd.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
> >
> > >
> > >
> > > Hello,
> > >
> > > I'm currently running into an issue where I'm trying to run
MODE,
> except
> > I
> > > want to loop through a few months of Stage IV 2-year exceedance
data
> > using
> > > a python program so that I don't need to change the date and
time every
> > > time I run MODE. However, when I run my program, I'm not getting
any
> > output
> > > MODE files. I've attached the type of files as well as my
program in
> this
> > > email. If there's any suggestions you have for me, please let me
know.
> > >
> > > Thanks,
> > > Michael
> > >
> > > Michael Pletcher
> > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > University of Maryland, College Park Class of 2020
> > > mpletch1 at terpmail.umd.edu|443-857-1127
> > >
> > >
> >
> >
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 10:02:55 2020
Michael,
Great, thanks for sending the config file. When I run MODE directly
on the
command line, I get the following error:
/usr/local/met-8.1.1/bin/mode ST4gARI_s2018030112_e2018030118_vhr18.nc
ST4gARI_s2018030112_e2018030118_vhr18
.nc test.config -outdir out
DEBUG 1: Default Config File:
/d1/johnhg/MET/MET_development/git/MET-
develop/met/share/met/config/MODEConfig_default
DEBUG 1: Match Config File: test.config
DEBUG 1: Merge Config File: test.config
ERROR :
ERROR : Trouble reading observation file
"ST4gARI_s2018030112_e2018030118_vhr18.nc"
ERROR :
Furthermore, when I try running ncdump on the input file to see what's
inside, I get another error:
ncdump -h ST4gARI_s2018030112_e2018030118_vhr18.nc
/usr/local/netcdf4/bin/ncdump:
ST4gARI_s2018030112_e2018030118_vhr18.nc:
NetCDF: HDF error
If ncdump can't read it, then MET surely won't be able to either. If
ncdump works for you, perhaps the file was corrupted when sent via
email?
If so, you could post it to our anonymous ftp site following these
instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Thanks,
John
On Wed, Mar 4, 2020 at 9:56 AM Michael Pletcher via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
> Hi John,
>
> Thanks for the reply. I've attached the config file in this email.
>
> -Michael
>
> On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Michael,
> >
> > I see that you're running MODE to compare a stageIV file to
itself, which
> > is fine. Looks like your MODE configuration file is named
"test.config".
> > Can you please send me that file as well?
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT
<met_help at ucar.edu>
> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> > >
> > > Hello Michael,
> > >
> > > I am assigning your issue to John Halley Gotway, who is best
suited to
> > > assist you. Please allow a few days for a response.
> > >
> > > Regards,
> > > Minna
> > > ---------------
> > > Minna Win
> > > National Center for Atmospheric Research
> > > Developmental Testbed Center
> > > Phone: 303-497-8423
> > > Fax: 303-497-8401
> > >
> > >
> > >
> > > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> > > > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> > > > Queue: met_help
> > > > Subject: Running MODE for months of Stage IV Data
> > > > Owner: Nobody
> > > > Requestors: mpletch1 at terpmail.umd.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
> > >
> > > >
> > > >
> > > > Hello,
> > > >
> > > > I'm currently running into an issue where I'm trying to run
MODE,
> > except
> > > I
> > > > want to loop through a few months of Stage IV 2-year
exceedance data
> > > using
> > > > a python program so that I don't need to change the date and
time
> every
> > > > time I run MODE. However, when I run my program, I'm not
getting any
> > > output
> > > > MODE files. I've attached the type of files as well as my
program in
> > this
> > > > email. If there's any suggestions you have for me, please let
me
> know.
> > > >
> > > > Thanks,
> > > > Michael
> > > >
> > > > Michael Pletcher
> > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > > University of Maryland, College Park Class of 2020
> > > > mpletch1 at terpmail.umd.edu|443-857-1127
> > > >
> > > >
> > >
> > >
> >
> >
>
> --
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 10:14:21 2020
Hi John,
That is odd, because I just ran MODE for that same input file and it
generated output files just fine. As you said, it may have been
corrupted
over email. I'll try posting it to the anonymous ftp site.
Thanks,
Michael
On Wed, Mar 4, 2020 at 12:02 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> Great, thanks for sending the config file. When I run MODE directly
on the
> command line, I get the following error:
>
> /usr/local/met-8.1.1/bin/mode
ST4gARI_s2018030112_e2018030118_vhr18.nc
> ST4gARI_s2018030112_e2018030118_vhr18
> .nc test.config -outdir out
> DEBUG 1: Default Config File:
>
> /d1/johnhg/MET/MET_development/git/MET-
develop/met/share/met/config/MODEConfig_default
>
> DEBUG 1: Match Config File: test.config
> DEBUG 1: Merge Config File: test.config
> ERROR :
> ERROR : Trouble reading observation file
> "ST4gARI_s2018030112_e2018030118_vhr18.nc"
> ERROR :
>
> Furthermore, when I try running ncdump on the input file to see
what's
> inside, I get another error:
>
> ncdump -h ST4gARI_s2018030112_e2018030118_vhr18.nc
> /usr/local/netcdf4/bin/ncdump:
ST4gARI_s2018030112_e2018030118_vhr18.nc:
> NetCDF: HDF error
>
> If ncdump can't read it, then MET surely won't be able to either.
If
> ncdump works for you, perhaps the file was corrupted when sent via
email?
> If so, you could post it to our anonymous ftp site following these
> instructions:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>
> Thanks,
> John
>
>
> On Wed, Mar 4, 2020 at 9:56 AM Michael Pletcher via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> > Hi John,
> >
> > Thanks for the reply. I've attached the config file in this email.
> >
> > -Michael
> >
> > On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Michael,
> > >
> > > I see that you're running MODE to compare a stageIV file to
itself,
> which
> > > is fine. Looks like your MODE configuration file is named
> "test.config".
> > > Can you please send me that file as well?
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
>
> > > >
> > > > Hello Michael,
> > > >
> > > > I am assigning your issue to John Halley Gotway, who is best
suited
> to
> > > > assist you. Please allow a few days for a response.
> > > >
> > > > Regards,
> > > > Minna
> > > > ---------------
> > > > Minna Win
> > > > National Center for Atmospheric Research
> > > > Developmental Testbed Center
> > > > Phone: 303-497-8423
> > > > Fax: 303-497-8401
> > > >
> > > >
> > > >
> > > > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > >
> > > > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> > > > > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> > > > > Queue: met_help
> > > > > Subject: Running MODE for months of Stage IV Data
> > > > > Owner: Nobody
> > > > > Requestors: mpletch1 at terpmail.umd.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
> > > >
> > > > >
> > > > >
> > > > > Hello,
> > > > >
> > > > > I'm currently running into an issue where I'm trying to run
MODE,
> > > except
> > > > I
> > > > > want to loop through a few months of Stage IV 2-year
exceedance
> data
> > > > using
> > > > > a python program so that I don't need to change the date and
time
> > every
> > > > > time I run MODE. However, when I run my program, I'm not
getting
> any
> > > > output
> > > > > MODE files. I've attached the type of files as well as my
program
> in
> > > this
> > > > > email. If there's any suggestions you have for me, please
let me
> > know.
> > > > >
> > > > > Thanks,
> > > > > Michael
> > > > >
> > > > > Michael Pletcher
> > > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > > > University of Maryland, College Park Class of 2020
> > > > > mpletch1 at terpmail.umd.edu|443-857-1127
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
> >
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 10:40:02 2020
John,
I attempted to upload my file, but I'm getting this error:
ftp> put
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
local:
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
remote:
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
227 Entering Passive Mode (128,117,14,132,196,68).
550
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc:
Operation not permitted
On Wed, Mar 4, 2020 at 12:14 PM Michael Pletcher
<mpletch1 at terpmail.umd.edu>
wrote:
> Hi John,
>
> That is odd, because I just ran MODE for that same input file and it
> generated output files just fine. As you said, it may have been
corrupted
> over email. I'll try posting it to the anonymous ftp site.
>
> Thanks,
> Michael
>
> On Wed, Mar 4, 2020 at 12:02 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Michael,
>>
>> Great, thanks for sending the config file. When I run MODE
directly on
>> the
>> command line, I get the following error:
>>
>> /usr/local/met-8.1.1/bin/mode
ST4gARI_s2018030112_e2018030118_vhr18.nc
>> ST4gARI_s2018030112_e2018030118_vhr18
>> .nc test.config -outdir out
>> DEBUG 1: Default Config File:
>>
>> /d1/johnhg/MET/MET_development/git/MET-
develop/met/share/met/config/MODEConfig_default
>>
>> DEBUG 1: Match Config File: test.config
>> DEBUG 1: Merge Config File: test.config
>> ERROR :
>> ERROR : Trouble reading observation file
>> "ST4gARI_s2018030112_e2018030118_vhr18.nc"
>> ERROR :
>>
>> Furthermore, when I try running ncdump on the input file to see
what's
>> inside, I get another error:
>>
>> ncdump -h ST4gARI_s2018030112_e2018030118_vhr18.nc
>> /usr/local/netcdf4/bin/ncdump:
ST4gARI_s2018030112_e2018030118_vhr18.nc:
>> NetCDF: HDF error
>>
>> If ncdump can't read it, then MET surely won't be able to either.
If
>> ncdump works for you, perhaps the file was corrupted when sent via
email?
>> If so, you could post it to our anonymous ftp site following these
>> instructions:
>>
>> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
>>
>> Thanks,
>> John
>>
>>
>> On Wed, Mar 4, 2020 at 9:56 AM Michael Pletcher via RT
<met_help at ucar.edu
>> >
>> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>> >
>> > Hi John,
>> >
>> > Thanks for the reply. I've attached the config file in this
email.
>> >
>> > -Michael
>> >
>> > On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Michael,
>> > >
>> > > I see that you're running MODE to compare a stageIV file to
itself,
>> which
>> > > is fine. Looks like your MODE configuration file is named
>> "test.config".
>> > > Can you please send me that file as well?
>> > >
>> > > Thanks,
>> > > John Halley Gotway
>> > >
>> > > On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT
<met_help at ucar.edu>
>> > wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
>
>> > > >
>> > > > Hello Michael,
>> > > >
>> > > > I am assigning your issue to John Halley Gotway, who is best
suited
>> to
>> > > > assist you. Please allow a few days for a response.
>> > > >
>> > > > Regards,
>> > > > Minna
>> > > > ---------------
>> > > > Minna Win
>> > > > National Center for Atmospheric Research
>> > > > Developmental Testbed Center
>> > > > Phone: 303-497-8423
>> > > > Fax: 303-497-8401
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
>> > > met_help at ucar.edu
>> > > > >
>> > > > wrote:
>> > > >
>> > > > >
>> > > > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
>> > > > > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
>> > > > > Queue: met_help
>> > > > > Subject: Running MODE for months of Stage IV Data
>> > > > > Owner: Nobody
>> > > > > Requestors: mpletch1 at terpmail.umd.edu
>> > > > > Status: new
>> > > > > Ticket <URL:
>> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
>> > > >
>> > > > >
>> > > > >
>> > > > > Hello,
>> > > > >
>> > > > > I'm currently running into an issue where I'm trying to run
MODE,
>> > > except
>> > > > I
>> > > > > want to loop through a few months of Stage IV 2-year
exceedance
>> data
>> > > > using
>> > > > > a python program so that I don't need to change the date
and time
>> > every
>> > > > > time I run MODE. However, when I run my program, I'm not
getting
>> any
>> > > > output
>> > > > > MODE files. I've attached the type of files as well as my
program
>> in
>> > > this
>> > > > > email. If there's any suggestions you have for me, please
let me
>> > know.
>> > > > >
>> > > > > Thanks,
>> > > > > Michael
>> > > > >
>> > > > > Michael Pletcher
>> > > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
>> > > > > University of Maryland, College Park Class of 2020
>> > > > > mpletch1 at terpmail.umd.edu|443-857-1127
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > Michael Pletcher
>> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
>> > University of Maryland, College Park Class of 2020
>> > mpletch1 at terpmail.umd.edu|443-857-1127
>> >
>> >
>>
>>
>
> --
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 11:04:00 2020
Michael,
Can you try these commands:
cd /homes/metogra/mpletch1/ResearchProject/st4
ftp ftp.rap.ucar.edu
username = anonymous
password = your email address
cd incoming/irap/met_help
mkdir pletcher_data
cd pletcher_data
put ST4gARI_s2018030112_e2018030118_vhr18.nc
bye
Any better luck?
Thanks,
John
On Wed, Mar 4, 2020 at 10:40 AM Michael Pletcher via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
> John,
>
> I attempted to upload my file, but I'm getting this error:
>
> ftp> put
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> local:
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> remote:
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> 227 Entering Passive Mode (128,117,14,132,196,68).
> 550
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc:
> Operation not permitted
>
>
> On Wed, Mar 4, 2020 at 12:14 PM Michael Pletcher <
> mpletch1 at terpmail.umd.edu>
> wrote:
>
> > Hi John,
> >
> > That is odd, because I just ran MODE for that same input file and
it
> > generated output files just fine. As you said, it may have been
corrupted
> > over email. I'll try posting it to the anonymous ftp site.
> >
> > Thanks,
> > Michael
> >
> > On Wed, Mar 4, 2020 at 12:02 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Michael,
> >>
> >> Great, thanks for sending the config file. When I run MODE
directly on
> >> the
> >> command line, I get the following error:
> >>
> >> /usr/local/met-8.1.1/bin/mode
ST4gARI_s2018030112_e2018030118_vhr18.nc
> >> ST4gARI_s2018030112_e2018030118_vhr18
> >> .nc test.config -outdir out
> >> DEBUG 1: Default Config File:
> >>
> >>
> /d1/johnhg/MET/MET_development/git/MET-
develop/met/share/met/config/MODEConfig_default
> >>
> >> DEBUG 1: Match Config File: test.config
> >> DEBUG 1: Merge Config File: test.config
> >> ERROR :
> >> ERROR : Trouble reading observation file
> >> "ST4gARI_s2018030112_e2018030118_vhr18.nc"
> >> ERROR :
> >>
> >> Furthermore, when I try running ncdump on the input file to see
what's
> >> inside, I get another error:
> >>
> >> ncdump -h ST4gARI_s2018030112_e2018030118_vhr18.nc
> >> /usr/local/netcdf4/bin/ncdump:
ST4gARI_s2018030112_e2018030118_vhr18.nc
> :
> >> NetCDF: HDF error
> >>
> >> If ncdump can't read it, then MET surely won't be able to either.
If
> >> ncdump works for you, perhaps the file was corrupted when sent
via
> email?
> >> If so, you could post it to our anonymous ftp site following
these
> >> instructions:
> >>
> >>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> >>
> >> Thanks,
> >> John
> >>
> >>
> >> On Wed, Mar 4, 2020 at 9:56 AM Michael Pletcher via RT <
> met_help at ucar.edu
> >> >
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >> >
> >> > Hi John,
> >> >
> >> > Thanks for the reply. I've attached the config file in this
email.
> >> >
> >> > -Michael
> >> >
> >> > On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Michael,
> >> > >
> >> > > I see that you're running MODE to compare a stageIV file to
itself,
> >> which
> >> > > is fine. Looks like your MODE configuration file is named
> >> "test.config".
> >> > > Can you please send me that file as well?
> >> > >
> >> > > Thanks,
> >> > > John Halley Gotway
> >> > >
> >> > > On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT
<met_help at ucar.edu>
> >> > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >> > > >
> >> > > > Hello Michael,
> >> > > >
> >> > > > I am assigning your issue to John Halley Gotway, who is
best
> suited
> >> to
> >> > > > assist you. Please allow a few days for a response.
> >> > > >
> >> > > > Regards,
> >> > > > Minna
> >> > > > ---------------
> >> > > > Minna Win
> >> > > > National Center for Atmospheric Research
> >> > > > Developmental Testbed Center
> >> > > > Phone: 303-497-8423
> >> > > > Fax: 303-497-8401
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
> >> > > met_help at ucar.edu
> >> > > > >
> >> > > > wrote:
> >> > > >
> >> > > > >
> >> > > > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> >> > > > > Transaction: Ticket created by mpletch1 at terpmail.umd.edu
> >> > > > > Queue: met_help
> >> > > > > Subject: Running MODE for months of Stage IV Data
> >> > > > > Owner: Nobody
> >> > > > > Requestors: mpletch1 at terpmail.umd.edu
> >> > > > > Status: new
> >> > > > > Ticket <URL:
> >> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > I'm currently running into an issue where I'm trying to
run
> MODE,
> >> > > except
> >> > > > I
> >> > > > > want to loop through a few months of Stage IV 2-year
exceedance
> >> data
> >> > > > using
> >> > > > > a python program so that I don't need to change the date
and
> time
> >> > every
> >> > > > > time I run MODE. However, when I run my program, I'm not
getting
> >> any
> >> > > > output
> >> > > > > MODE files. I've attached the type of files as well as my
> program
> >> in
> >> > > this
> >> > > > > email. If there's any suggestions you have for me, please
let me
> >> > know.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Michael
> >> > > > >
> >> > > > > Michael Pletcher
> >> > > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> >> > > > > University of Maryland, College Park Class of 2020
> >> > > > > mpletch1 at terpmail.umd.edu|443-857-1127
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > Michael Pletcher
> >> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> >> > University of Maryland, College Park Class of 2020
> >> > mpletch1 at terpmail.umd.edu|443-857-1127
> >> >
> >> >
> >>
> >>
> >
> > --
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
>
>
> --
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 11:07:05 2020
John,
That worked! Let me know if you need to do anything else.
On Wed, Mar 4, 2020 at 1:04 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> Can you try these commands:
>
> cd /homes/metogra/mpletch1/ResearchProject/st4
> ftp ftp.rap.ucar.edu
> username = anonymous
> password = your email address
> cd incoming/irap/met_help
> mkdir pletcher_data
> cd pletcher_data
> put ST4gARI_s2018030112_e2018030118_vhr18.nc
> bye
>
> Any better luck?
>
> Thanks,
> John
>
> On Wed, Mar 4, 2020 at 10:40 AM Michael Pletcher via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> > John,
> >
> > I attempted to upload my file, but I'm getting this error:
> >
> > ftp> put
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > local:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > remote:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > 227 Entering Passive Mode (128,117,14,132,196,68).
> > 550
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc:
> > Operation not permitted
> >
> >
> > On Wed, Mar 4, 2020 at 12:14 PM Michael Pletcher <
> > mpletch1 at terpmail.umd.edu>
> > wrote:
> >
> > > Hi John,
> > >
> > > That is odd, because I just ran MODE for that same input file
and it
> > > generated output files just fine. As you said, it may have been
> corrupted
> > > over email. I'll try posting it to the anonymous ftp site.
> > >
> > > Thanks,
> > > Michael
> > >
> > > On Wed, Mar 4, 2020 at 12:02 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Michael,
> > >>
> > >> Great, thanks for sending the config file. When I run MODE
directly
> on
> > >> the
> > >> command line, I get the following error:
> > >>
> > >> /usr/local/met-8.1.1/bin/mode ST4gARI_s2018030112_
> e2018030118_vhr18.nc
> > >> ST4gARI_s2018030112_e2018030118_vhr18
> > >> .nc test.config -outdir out
> > >> DEBUG 1: Default Config File:
> > >>
> > >>
> >
> /d1/johnhg/MET/MET_development/git/MET-
develop/met/share/met/config/MODEConfig_default
> > >>
> > >> DEBUG 1: Match Config File: test.config
> > >> DEBUG 1: Merge Config File: test.config
> > >> ERROR :
> > >> ERROR : Trouble reading observation file
> > >> "ST4gARI_s2018030112_e2018030118_vhr18.nc"
> > >> ERROR :
> > >>
> > >> Furthermore, when I try running ncdump on the input file to see
what's
> > >> inside, I get another error:
> > >>
> > >> ncdump -h ST4gARI_s2018030112_e2018030118_vhr18.nc
> > >> /usr/local/netcdf4/bin/ncdump: ST4gARI_s2018030112_
> e2018030118_vhr18.nc
> > :
> > >> NetCDF: HDF error
> > >>
> > >> If ncdump can't read it, then MET surely won't be able to
either. If
> > >> ncdump works for you, perhaps the file was corrupted when sent
via
> > email?
> > >> If so, you could post it to our anonymous ftp site following
these
> > >> instructions:
> > >>
> > >>
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >>
> > >> On Wed, Mar 4, 2020 at 9:56 AM Michael Pletcher via RT <
> > met_help at ucar.edu
> > >> >
> > >> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
>
> > >> >
> > >> > Hi John,
> > >> >
> > >> > Thanks for the reply. I've attached the config file in this
email.
> > >> >
> > >> > -Michael
> > >> >
> > >> > On Wed, Mar 4, 2020 at 11:50 AM John Halley Gotway via RT <
> > >> > met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Michael,
> > >> > >
> > >> > > I see that you're running MODE to compare a stageIV file to
> itself,
> > >> which
> > >> > > is fine. Looks like your MODE configuration file is named
> > >> "test.config".
> > >> > > Can you please send me that file as well?
> > >> > >
> > >> > > Thanks,
> > >> > > John Halley Gotway
> > >> > >
> > >> > > On Wed, Mar 4, 2020 at 9:20 AM Minna Win via RT <
> met_help at ucar.edu>
> > >> > wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> > >> > > >
> > >> > > > Hello Michael,
> > >> > > >
> > >> > > > I am assigning your issue to John Halley Gotway, who is
best
> > suited
> > >> to
> > >> > > > assist you. Please allow a few days for a response.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Minna
> > >> > > > ---------------
> > >> > > > Minna Win
> > >> > > > National Center for Atmospheric Research
> > >> > > > Developmental Testbed Center
> > >> > > > Phone: 303-497-8423
> > >> > > > Fax: 303-497-8401
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Mar 4, 2020 at 12:04 AM Michael Pletcher via RT <
> > >> > > met_help at ucar.edu
> > >> > > > >
> > >> > > > wrote:
> > >> > > >
> > >> > > > >
> > >> > > > > Tue Mar 03 17:04:20 2020: Request 94392 was acted upon.
> > >> > > > > Transaction: Ticket created by
mpletch1 at terpmail.umd.edu
> > >> > > > > Queue: met_help
> > >> > > > > Subject: Running MODE for months of Stage IV Data
> > >> > > > > Owner: Nobody
> > >> > > > > Requestors: mpletch1 at terpmail.umd.edu
> > >> > > > > Status: new
> > >> > > > > Ticket <URL:
> > >> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
> > >> > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > Hello,
> > >> > > > >
> > >> > > > > I'm currently running into an issue where I'm trying to
run
> > MODE,
> > >> > > except
> > >> > > > I
> > >> > > > > want to loop through a few months of Stage IV 2-year
> exceedance
> > >> data
> > >> > > > using
> > >> > > > > a python program so that I don't need to change the
date and
> > time
> > >> > every
> > >> > > > > time I run MODE. However, when I run my program, I'm
not
> getting
> > >> any
> > >> > > > output
> > >> > > > > MODE files. I've attached the type of files as well as
my
> > program
> > >> in
> > >> > > this
> > >> > > > > email. If there's any suggestions you have for me,
please let
> me
> > >> > know.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Michael
> > >> > > > >
> > >> > > > > Michael Pletcher
> > >> > > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS
Minor
> > >> > > > > University of Maryland, College Park Class of 2020
> > >> > > > > mpletch1 at terpmail.umd.edu|443-857-1127
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> > Michael Pletcher
> > >> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > >> > University of Maryland, College Park Class of 2020
> > >> > mpletch1 at terpmail.umd.edu|443-857-1127
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > > Michael Pletcher
> > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > University of Maryland, College Park Class of 2020
> > > mpletch1 at terpmail.umd.edu|443-857-1127
> > >
> >
> >
> > --
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
> >
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 11:25:05 2020
Michael,
OK, that's much better. MODE now runs and the log messages indicate
that
it finds 0 objects. That's why the output files are empty:
DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
It does however create a PostScript plot summarizing the run. I
usually
use "ghostview" to visualize PostScript files, but different systems
handle
them differently.
I've attached a PNG version of the second page of that file. If you
look
very closely, there's a tiny blip of non-zero data in northern CA.
If you reconfigure and rerun with no smoothing (conv_radius = 0;), the
lowest threshold possible (conv_thresh = >0;), and no object area
threshold
(area_thresh = >0;), MODE does, in fact, find a single object:
DEBUG 2: Identified: 1 forecast objects and 1 observation objects.
Checking the ascii output file (
mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of that
object is
4.
I'd recommend taking a step back to look at your input data to see if
it
really contains what you expect. The plot_data_plane tool in MET is
very
useful in this regard:
plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
level="(*,*)";'
It just reads a single 2D field of data and plots it. Of course, the
result is really the same as looking at the PostScript output from
MODE.
Thanks,
John
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 11:31:27 2020
John,
That's good news you were able to run MODE with the file. However,
I've
actually already configured the config file for my own methods, so I
most
likely will not be changing that anymore. My issue is that MODE is not
running at all from my time_loop.py program, not that zero objects are
identified. When MODE normally runs, it gives me an output like this:
DEBUG 1: Default Config File:
/homes/metogra/mpletch1/ResearchProject/met/share/met/config/MODEConfig_default
DEBUG 1: Match Config File:
/homes/metogra/mpletch1/ResearchProject/test.config
DEBUG 1: Merge Config File:
/homes/metogra/mpletch1/ResearchProject/test.config
DEBUG 1: Forecast File:
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
DEBUG 1: Observation File:
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
DEBUG 1: Forecast Field: ST4gARI at Surface
DEBUG 1: Observation Field: ST4gARI at Surface
DEBUG 2: Processing masking regions.
DEBUG 2: Identifying objects in the forecast and observation fields...
DEBUG 2: Computing contingency table statistics...
DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
DEBUG 2: Performing merging (THRESH) in the forecast field.
DEBUG 2: Performing merging (THRESH) in the observation field.
DEBUG 2: Remaining: 0 forecast objects and 0 observation objects.
DEBUG 2: Performing matching (MERGE_FCST) between the forecast and
observation fields.
DEBUG 1: Creating Fcst-Obs Object Statistics file:
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.txt
DEBUG 1: Creating Contingency Table Statistics file:
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_cts.txt
DEBUG 1: Creating Object NetCDF file:
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.nc
DEBUG 1: Loading forecast raw color table:
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
DEBUG 1: Loading observation raw color table:
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
DEBUG 1: Creating postscript file:
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A.ps
But since I have so much data, I wanted to create a program that would
automatically loop through all the data that I have, which is what I
was
trying to use time_loop.py for, which I included in the first email.
If
there's any changes you believe should be made to that program, please
let
me know.
Thanks,
Mchael
On Wed, Mar 4, 2020 at 1:25 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> OK, that's much better. MODE now runs and the log messages indicate
that
> it finds 0 objects. That's why the output files are empty:
> DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
>
> It does however create a PostScript plot summarizing the run. I
usually
> use "ghostview" to visualize PostScript files, but different systems
handle
> them differently.
> I've attached a PNG version of the second page of that file. If you
look
> very closely, there's a tiny blip of non-zero data in northern CA.
>
> If you reconfigure and rerun with no smoothing (conv_radius = 0;),
the
> lowest threshold possible (conv_thresh = >0;), and no object area
threshold
> (area_thresh = >0;), MODE does, in fact, find a single object:
> DEBUG 2: Identified: 1 forecast objects and 1 observation objects.
>
> Checking the ascii output file (
> mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of that
object is
> 4.
>
> I'd recommend taking a step back to look at your input data to see
if it
> really contains what you expect. The plot_data_plane tool in MET is
very
> useful in this regard:
> plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
> ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
level="(*,*)";'
>
> It just reads a single 2D field of data and plots it. Of course,
the
> result is really the same as looking at the PostScript output from
MODE.
>
> Thanks,
> John
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 12:07:44 2020
Michael,
Ah, you need help debugging your custom python script.
I'd like to point you to the METplus python wrappers. That's what our
team
has been working on the last couple of years to automate calls to the
MET
tools, very much like what you're doing with your time looping.
Here's
where that lives on GitHub:
https://github.com/NCAR/METplus
In time_loop.py, it looks like you're redirecting all the stdout and
stderr
to /dev/null, which makes debugging much harder! Remove that to see
the
output from the call and the problem becomes obvious:
file = subprocess.call(MET_BIN_PATH +
'~/ResearchProject/st4/ST4gARI_s2018'
+ curDT.strftime("%m%d%H") + '*' '~/ResearchProject/st4/ST4gARI_s2018'
+
curDT.strftime("%m%d%H") + '*' '~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects', shell=True)
Here's the command it's trying to execute:
"
homes/metogra/mpletch1/ResearchProject/met/bin/mode~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/test.config"
You're missing spaces between the command line args. So just add some
spaced when building the subprocess call:
file = subprocess.call(MET_BIN_PATH + '
~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") + '*'
'
~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") + '*'
'
~/ResearchProject/test.config -outdir ~/ResearchProject/mode_objects',
stdout=o
pen(os.devnull, 'wb'), stderr=open(os.devnull, 'wb'), shell=True)
Although, I would recommend printing output to screen (instead of
redirecting to /dev/null) until the script is working smoothly.
Thanks,
John
On Wed, Mar 4, 2020 at 11:31 AM Michael Pletcher via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
> John,
>
> That's good news you were able to run MODE with the file. However,
I've
> actually already configured the config file for my own methods, so I
most
> likely will not be changing that anymore. My issue is that MODE is
not
> running at all from my time_loop.py program, not that zero objects
are
> identified. When MODE normally runs, it gives me an output like
this:
>
> DEBUG 1: Default Config File:
>
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/config/MODEConfig_default
> DEBUG 1: Match Config File:
> /homes/metogra/mpletch1/ResearchProject/test.config
> DEBUG 1: Merge Config File:
> /homes/metogra/mpletch1/ResearchProject/test.config
> DEBUG 1: Forecast File:
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> DEBUG 1: Observation File:
>
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> DEBUG 1: Forecast Field: ST4gARI at Surface
> DEBUG 1: Observation Field: ST4gARI at Surface
> DEBUG 2: Processing masking regions.
> DEBUG 2: Identifying objects in the forecast and observation
fields...
> DEBUG 2: Computing contingency table statistics...
> DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
> DEBUG 2: Performing merging (THRESH) in the forecast field.
> DEBUG 2: Performing merging (THRESH) in the observation field.
> DEBUG 2: Remaining: 0 forecast objects and 0 observation objects.
> DEBUG 2: Performing matching (MERGE_FCST) between the forecast and
> observation fields.
> DEBUG 1: Creating Fcst-Obs Object Statistics file:
>
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.txt
> DEBUG 1: Creating Contingency Table Statistics file:
>
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_cts.txt
> DEBUG 1: Creating Object NetCDF file:
>
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.nc
> DEBUG 1: Loading forecast raw color table:
>
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> DEBUG 1: Loading observation raw color table:
>
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> DEBUG 1: Creating postscript file:
>
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A.ps
>
> But since I have so much data, I wanted to create a program that
would
> automatically loop through all the data that I have, which is what I
was
> trying to use time_loop.py for, which I included in the first email.
If
> there's any changes you believe should be made to that program,
please let
> me know.
>
> Thanks,
> Mchael
>
> On Wed, Mar 4, 2020 at 1:25 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Michael,
> >
> > OK, that's much better. MODE now runs and the log messages
indicate that
> > it finds 0 objects. That's why the output files are empty:
> > DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
> >
> > It does however create a PostScript plot summarizing the run. I
usually
> > use "ghostview" to visualize PostScript files, but different
systems
> handle
> > them differently.
> > I've attached a PNG version of the second page of that file. If
you look
> > very closely, there's a tiny blip of non-zero data in northern CA.
> >
> > If you reconfigure and rerun with no smoothing (conv_radius = 0;),
the
> > lowest threshold possible (conv_thresh = >0;), and no object area
> threshold
> > (area_thresh = >0;), MODE does, in fact, find a single object:
> > DEBUG 2: Identified: 1 forecast objects and 1 observation objects.
> >
> > Checking the ascii output file (
> > mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of that
object
> is
> > 4.
> >
> > I'd recommend taking a step back to look at your input data to see
if it
> > really contains what you expect. The plot_data_plane tool in MET
is very
> > useful in this regard:
> > plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
> > ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
> level="(*,*)";'
> >
> > It just reads a single 2D field of data and plots it. Of course,
the
> > result is really the same as looking at the PostScript output from
MODE.
> >
> > Thanks,
> > John
> >
> >
>
> --
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 12:42:43 2020
John,
I think I got the loop to work, but I keep getting the "no such file
or
directory" error message. I changed the call a bit to be more
accurate.
Below is my subprocess call:
file = subprocess.call(MET_BIN_PATH +
'~/ResearchProject/st4/ST4gARI_s2018'
+ curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
curDT2.strftime("%H") + '.nc' '~/ResearchProject/st4/ST4gARI_s2018' +
curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
curDT.strftime("%H") + '.nc' '~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects', shell=True)
I've also attached my time_loop.py file to see what I changed.
Thanks,
Michael
On Wed, Mar 4, 2020 at 2:07 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> Ah, you need help debugging your custom python script.
>
> I'd like to point you to the METplus python wrappers. That's what
our team
> has been working on the last couple of years to automate calls to
the MET
> tools, very much like what you're doing with your time looping.
Here's
> where that lives on GitHub:
> https://github.com/NCAR/METplus
>
> In time_loop.py, it looks like you're redirecting all the stdout and
stderr
> to /dev/null, which makes debugging much harder! Remove that to see
the
> output from the call and the problem becomes obvious:
>
> file = subprocess.call(MET_BIN_PATH +
'~/ResearchProject/st4/ST4gARI_s2018'
> + curDT.strftime("%m%d%H") + '*'
'~/ResearchProject/st4/ST4gARI_s2018' +
> curDT.strftime("%m%d%H") + '*' '~/ResearchProject/test.config
-outdir
> ~/ResearchProject/mode_objects', shell=True)
>
> Here's the command it's trying to execute:
> "
>
>
homes/metogra/mpletch1/ResearchProject/met/bin/mode~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/test.config"
>
> You're missing spaces between the command line args. So just add
some
> spaced when building the subprocess call:
>
> file = subprocess.call(MET_BIN_PATH + '
> ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") +
'*' '
> ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") +
'*' '
> ~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects',
> stdout=o
> pen(os.devnull, 'wb'), stderr=open(os.devnull, 'wb'), shell=True)
>
> Although, I would recommend printing output to screen (instead of
> redirecting to /dev/null) until the script is working smoothly.
>
> Thanks,
> John
>
> On Wed, Mar 4, 2020 at 11:31 AM Michael Pletcher via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> > John,
> >
> > That's good news you were able to run MODE with the file. However,
I've
> > actually already configured the config file for my own methods, so
I most
> > likely will not be changing that anymore. My issue is that MODE is
not
> > running at all from my time_loop.py program, not that zero objects
are
> > identified. When MODE normally runs, it gives me an output like
this:
> >
> > DEBUG 1: Default Config File:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/config/MODEConfig_default
> > DEBUG 1: Match Config File:
> > /homes/metogra/mpletch1/ResearchProject/test.config
> > DEBUG 1: Merge Config File:
> > /homes/metogra/mpletch1/ResearchProject/test.config
> > DEBUG 1: Forecast File:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > DEBUG 1: Observation File:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > DEBUG 1: Forecast Field: ST4gARI at Surface
> > DEBUG 1: Observation Field: ST4gARI at Surface
> > DEBUG 2: Processing masking regions.
> > DEBUG 2: Identifying objects in the forecast and observation
fields...
> > DEBUG 2: Computing contingency table statistics...
> > DEBUG 2: Identified: 0 forecast objects and 0 observation objects.
> > DEBUG 2: Performing merging (THRESH) in the forecast field.
> > DEBUG 2: Performing merging (THRESH) in the observation field.
> > DEBUG 2: Remaining: 0 forecast objects and 0 observation objects.
> > DEBUG 2: Performing matching (MERGE_FCST) between the forecast and
> > observation fields.
> > DEBUG 1: Creating Fcst-Obs Object Statistics file:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.txt
> > DEBUG 1: Creating Contingency Table Statistics file:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_cts.txt
> > DEBUG 1: Creating Object NetCDF file:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.nc
> > DEBUG 1: Loading forecast raw color table:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > DEBUG 1: Loading observation raw color table:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > DEBUG 1: Creating postscript file:
> >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A.ps
> >
> > But since I have so much data, I wanted to create a program that
would
> > automatically loop through all the data that I have, which is what
I was
> > trying to use time_loop.py for, which I included in the first
email. If
> > there's any changes you believe should be made to that program,
please
> let
> > me know.
> >
> > Thanks,
> > Mchael
> >
> > On Wed, Mar 4, 2020 at 1:25 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Michael,
> > >
> > > OK, that's much better. MODE now runs and the log messages
indicate
> that
> > > it finds 0 objects. That's why the output files are empty:
> > > DEBUG 2: Identified: 0 forecast objects and 0 observation
objects.
> > >
> > > It does however create a PostScript plot summarizing the run. I
> usually
> > > use "ghostview" to visualize PostScript files, but different
systems
> > handle
> > > them differently.
> > > I've attached a PNG version of the second page of that file. If
you
> look
> > > very closely, there's a tiny blip of non-zero data in northern
CA.
> > >
> > > If you reconfigure and rerun with no smoothing (conv_radius =
0;), the
> > > lowest threshold possible (conv_thresh = >0;), and no object
area
> > threshold
> > > (area_thresh = >0;), MODE does, in fact, find a single object:
> > > DEBUG 2: Identified: 1 forecast objects and 1 observation
objects.
> > >
> > > Checking the ascii output file (
> > > mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of that
object
> > is
> > > 4.
> > >
> > > I'd recommend taking a step back to look at your input data to
see if
> it
> > > really contains what you expect. The plot_data_plane tool in
MET is
> very
> > > useful in this regard:
> > > plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
> > level="(*,*)";'
> > >
> > > It just reads a single 2D field of data and plots it. Of
course, the
> > > result is really the same as looking at the PostScript output
from
> MODE.
> > >
> > > Thanks,
> > > John
> > >
> > >
> >
> > --
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
> >
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 12:42:43 2020
import datetime as dt
import time
import os
import subprocess
start_time = time.time()
curDT = dt.datetime(2018,3,1,12)
curDT2 = dt.datetime(2018,3,1,18)
endDT = dt.datetime(2018,10,31,6)
StMonth = curDT.strftime("%m")
StDay = curDT.strftime("%d")
StHour = curDT.strftime("%H")
MET_BIN_PATH = 'homes/metogra/mpletch1/ResearchProject/met/bin/mode'
while curDT < endDT:
file = subprocess.call(MET_BIN_PATH +
'~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") +
'_e2018' + curDT2.strftime("%m%d%H") + 'vhr' + curDT2.strftime("%H") +
'.nc' '~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H")
+ '_e2018' + curDT2.strftime("%m%d%H") + 'vhr' + curDT.strftime("%H")
+ '.nc' '~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects', shell=True)
curDT = curDT + dt.timedelta(hours=6)
curDT2 = curDT2 + dt.timedelta(hours=6)
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: John Halley Gotway
Time: Wed Mar 04 13:46:11 2020
Michael,
You need to add 3 spaces so that you construct a command that can be
executed...
file = subprocess.call(MET_BIN_PATH + '*ADD_A_SPACE_HERE*
~/ResearchProject/st4/ST4gARI_s2018'
+ curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
curDT2.strftime("%H") + '.nc'
'*ADD_A_SPACE_HERE*~/ResearchProject/st4/ST4gARI_s2018'
+
curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
curDT.strftime("%H") + '.nc'
'*ADD_A_SPACE_HERE*~/ResearchProject/test.config
-outdir
~/ResearchProject/mode_objects', shell=True)
On Wed, Mar 4, 2020 at 12:43 PM Michael Pletcher via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
>
> John,
>
> I think I got the loop to work, but I keep getting the "no such file
or
> directory" error message. I changed the call a bit to be more
accurate.
> Below is my subprocess call:
>
> file = subprocess.call(MET_BIN_PATH +
'~/ResearchProject/st4/ST4gARI_s2018'
> + curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
> curDT2.strftime("%H") + '.nc' '~/ResearchProject/st4/ST4gARI_s2018'
+
> curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
> curDT.strftime("%H") + '.nc' '~/ResearchProject/test.config -outdir
> ~/ResearchProject/mode_objects', shell=True)
>
> I've also attached my time_loop.py file to see what I changed.
>
> Thanks,
> Michael
>
> On Wed, Mar 4, 2020 at 2:07 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Michael,
> >
> > Ah, you need help debugging your custom python script.
> >
> > I'd like to point you to the METplus python wrappers. That's what
our
> team
> > has been working on the last couple of years to automate calls to
the MET
> > tools, very much like what you're doing with your time looping.
Here's
> > where that lives on GitHub:
> > https://github.com/NCAR/METplus
> >
> > In time_loop.py, it looks like you're redirecting all the stdout
and
> stderr
> > to /dev/null, which makes debugging much harder! Remove that to
see the
> > output from the call and the problem becomes obvious:
> >
> > file = subprocess.call(MET_BIN_PATH +
> '~/ResearchProject/st4/ST4gARI_s2018'
> > + curDT.strftime("%m%d%H") + '*'
'~/ResearchProject/st4/ST4gARI_s2018' +
> > curDT.strftime("%m%d%H") + '*' '~/ResearchProject/test.config
-outdir
> > ~/ResearchProject/mode_objects', shell=True)
> >
> > Here's the command it's trying to execute:
> > "
> >
> >
>
homes/metogra/mpletch1/ResearchProject/met/bin/mode~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/test.config"
> >
> > You're missing spaces between the command line args. So just add
some
> > spaced when building the subprocess call:
> >
> > file = subprocess.call(MET_BIN_PATH + '
> > ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") +
'*' '
> > ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H") +
'*' '
> > ~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects',
> > stdout=o
> > pen(os.devnull, 'wb'), stderr=open(os.devnull, 'wb'), shell=True)
> >
> > Although, I would recommend printing output to screen (instead of
> > redirecting to /dev/null) until the script is working smoothly.
> >
> > Thanks,
> > John
> >
> > On Wed, Mar 4, 2020 at 11:31 AM Michael Pletcher via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> > >
> > > John,
> > >
> > > That's good news you were able to run MODE with the file.
However, I've
> > > actually already configured the config file for my own methods,
so I
> most
> > > likely will not be changing that anymore. My issue is that MODE
is not
> > > running at all from my time_loop.py program, not that zero
objects are
> > > identified. When MODE normally runs, it gives me an output like
this:
> > >
> > > DEBUG 1: Default Config File:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/config/MODEConfig_default
> > > DEBUG 1: Match Config File:
> > > /homes/metogra/mpletch1/ResearchProject/test.config
> > > DEBUG 1: Merge Config File:
> > > /homes/metogra/mpletch1/ResearchProject/test.config
> > > DEBUG 1: Forecast File:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > DEBUG 1: Observation File:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > DEBUG 1: Forecast Field: ST4gARI at Surface
> > > DEBUG 1: Observation Field: ST4gARI at Surface
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 2: Identifying objects in the forecast and observation
fields...
> > > DEBUG 2: Computing contingency table statistics...
> > > DEBUG 2: Identified: 0 forecast objects and 0 observation
objects.
> > > DEBUG 2: Performing merging (THRESH) in the forecast field.
> > > DEBUG 2: Performing merging (THRESH) in the observation field.
> > > DEBUG 2: Remaining: 0 forecast objects and 0 observation
objects.
> > > DEBUG 2: Performing matching (MERGE_FCST) between the forecast
and
> > > observation fields.
> > > DEBUG 1: Creating Fcst-Obs Object Statistics file:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.txt
> > > DEBUG 1: Creating Contingency Table Statistics file:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_cts.txt
> > > DEBUG 1: Creating Object NetCDF file:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.nc
> > > DEBUG 1: Loading forecast raw color table:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > > DEBUG 1: Loading observation raw color table:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > > DEBUG 1: Creating postscript file:
> > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A.ps
> > >
> > > But since I have so much data, I wanted to create a program that
would
> > > automatically loop through all the data that I have, which is
what I
> was
> > > trying to use time_loop.py for, which I included in the first
email. If
> > > there's any changes you believe should be made to that program,
please
> > let
> > > me know.
> > >
> > > Thanks,
> > > Mchael
> > >
> > > On Wed, Mar 4, 2020 at 1:25 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Michael,
> > > >
> > > > OK, that's much better. MODE now runs and the log messages
indicate
> > that
> > > > it finds 0 objects. That's why the output files are empty:
> > > > DEBUG 2: Identified: 0 forecast objects and 0 observation
objects.
> > > >
> > > > It does however create a PostScript plot summarizing the run.
I
> > usually
> > > > use "ghostview" to visualize PostScript files, but different
systems
> > > handle
> > > > them differently.
> > > > I've attached a PNG version of the second page of that file.
If you
> > look
> > > > very closely, there's a tiny blip of non-zero data in northern
CA.
> > > >
> > > > If you reconfigure and rerun with no smoothing (conv_radius =
0;),
> the
> > > > lowest threshold possible (conv_thresh = >0;), and no object
area
> > > threshold
> > > > (area_thresh = >0;), MODE does, in fact, find a single object:
> > > > DEBUG 2: Identified: 1 forecast objects and 1 observation
objects.
> > > >
> > > > Checking the ascii output file (
> > > > mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of
that
> object
> > > is
> > > > 4.
> > > >
> > > > I'd recommend taking a step back to look at your input data to
see if
> > it
> > > > really contains what you expect. The plot_data_plane tool in
MET is
> > very
> > > > useful in this regard:
> > > > plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > > ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
> > > level="(*,*)";'
> > > >
> > > > It just reads a single 2D field of data and plots it. Of
course, the
> > > > result is really the same as looking at the PostScript output
from
> > MODE.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > >
> > > --
> > > Michael Pletcher
> > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > University of Maryland, College Park Class of 2020
> > > mpletch1 at terpmail.umd.edu|443-857-1127
> > >
> > >
> >
> >
>
> --
> Michael Pletcher
> B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> University of Maryland, College Park Class of 2020
> mpletch1 at terpmail.umd.edu|443-857-1127
>
>
------------------------------------------------
Subject: Running MODE for months of Stage IV Data
From: Michael Pletcher
Time: Wed Mar 04 13:55:50 2020
John,
I also realized that I did not add a slash before the first directory
in my
MET_BIN_PATH, and after doing what you recommended, the program is
running
now! Thank you so much for your help!
-Michael
On Wed, Mar 4, 2020 at 3:46 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Michael,
>
> You need to add 3 spaces so that you construct a command that can be
> executed...
>
> file = subprocess.call(MET_BIN_PATH + '*ADD_A_SPACE_HERE*
> ~/ResearchProject/st4/ST4gARI_s2018'
> + curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
> curDT2.strftime("%H") + '.nc'
> '*ADD_A_SPACE_HERE*~/ResearchProject/st4/ST4gARI_s2018'
> +
> curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
> curDT.strftime("%H") + '.nc'
> '*ADD_A_SPACE_HERE*~/ResearchProject/test.config
> -outdir
> ~/ResearchProject/mode_objects', shell=True)
>
> On Wed, Mar 4, 2020 at 12:43 PM Michael Pletcher via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392 >
> >
> > John,
> >
> > I think I got the loop to work, but I keep getting the "no such
file or
> > directory" error message. I changed the call a bit to be more
accurate.
> > Below is my subprocess call:
> >
> > file = subprocess.call(MET_BIN_PATH +
> '~/ResearchProject/st4/ST4gARI_s2018'
> > + curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H")
+
> 'vhr' +
> > curDT2.strftime("%H") + '.nc'
'~/ResearchProject/st4/ST4gARI_s2018' +
> > curDT.strftime("%m%d%H") + '_e2018' + curDT2.strftime("%m%d%H") +
'vhr' +
> > curDT.strftime("%H") + '.nc' '~/ResearchProject/test.config
-outdir
> > ~/ResearchProject/mode_objects', shell=True)
> >
> > I've also attached my time_loop.py file to see what I changed.
> >
> > Thanks,
> > Michael
> >
> > On Wed, Mar 4, 2020 at 2:07 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Michael,
> > >
> > > Ah, you need help debugging your custom python script.
> > >
> > > I'd like to point you to the METplus python wrappers. That's
what our
> > team
> > > has been working on the last couple of years to automate calls
to the
> MET
> > > tools, very much like what you're doing with your time looping.
Here's
> > > where that lives on GitHub:
> > > https://github.com/NCAR/METplus
> > >
> > > In time_loop.py, it looks like you're redirecting all the stdout
and
> > stderr
> > > to /dev/null, which makes debugging much harder! Remove that to
see
> the
> > > output from the call and the problem becomes obvious:
> > >
> > > file = subprocess.call(MET_BIN_PATH +
> > '~/ResearchProject/st4/ST4gARI_s2018'
> > > + curDT.strftime("%m%d%H") + '*'
'~/ResearchProject/st4/ST4gARI_s2018'
> +
> > > curDT.strftime("%m%d%H") + '*' '~/ResearchProject/test.config
-outdir
> > > ~/ResearchProject/mode_objects', shell=True)
> > >
> > > Here's the command it's trying to execute:
> > > "
> > >
> > >
> >
>
homes/metogra/mpletch1/ResearchProject/met/bin/mode~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/st4/ST4gARI_s2018061700*~/ResearchProject/test.config"
> > >
> > > You're missing spaces between the command line args. So just
add some
> > > spaced when building the subprocess call:
> > >
> > > file = subprocess.call(MET_BIN_PATH + '
> > > ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H")
+ '*' '
> > > ~/ResearchProject/st4/ST4gARI_s2018' + curDT.strftime("%m%d%H")
+ '*' '
> > > ~/ResearchProject/test.config -outdir
~/ResearchProject/mode_objects',
> > > stdout=o
> > > pen(os.devnull, 'wb'), stderr=open(os.devnull, 'wb'),
shell=True)
> > >
> > > Although, I would recommend printing output to screen (instead
of
> > > redirecting to /dev/null) until the script is working smoothly.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Mar 4, 2020 at 11:31 AM Michael Pletcher via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94392
>
> > > >
> > > > John,
> > > >
> > > > That's good news you were able to run MODE with the file.
However,
> I've
> > > > actually already configured the config file for my own
methods, so I
> > most
> > > > likely will not be changing that anymore. My issue is that
MODE is
> not
> > > > running at all from my time_loop.py program, not that zero
objects
> are
> > > > identified. When MODE normally runs, it gives me an output
like this:
> > > >
> > > > DEBUG 1: Default Config File:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/config/MODEConfig_default
> > > > DEBUG 1: Match Config File:
> > > > /homes/metogra/mpletch1/ResearchProject/test.config
> > > > DEBUG 1: Merge Config File:
> > > > /homes/metogra/mpletch1/ResearchProject/test.config
> > > > DEBUG 1: Forecast File:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > > DEBUG 1: Observation File:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/st4/ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > > DEBUG 1: Forecast Field: ST4gARI at Surface
> > > > DEBUG 1: Observation Field: ST4gARI at Surface
> > > > DEBUG 2: Processing masking regions.
> > > > DEBUG 2: Identifying objects in the forecast and observation
> fields...
> > > > DEBUG 2: Computing contingency table statistics...
> > > > DEBUG 2: Identified: 0 forecast objects and 0 observation
objects.
> > > > DEBUG 2: Performing merging (THRESH) in the forecast field.
> > > > DEBUG 2: Performing merging (THRESH) in the observation field.
> > > > DEBUG 2: Remaining: 0 forecast objects and 0 observation
objects.
> > > > DEBUG 2: Performing matching (MERGE_FCST) between the forecast
and
> > > > observation fields.
> > > > DEBUG 1: Creating Fcst-Obs Object Statistics file:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.txt
> > > > DEBUG 1: Creating Contingency Table Statistics file:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_cts.txt
> > > > DEBUG 1: Creating Object NetCDF file:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A_obj.nc
> > > > DEBUG 1: Loading forecast raw color table:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > > > DEBUG 1: Loading observation raw color table:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/met/share/met/colortables/met_default.ctable
> > > > DEBUG 1: Creating postscript file:
> > > >
> > > >
> > >
> >
>
/homes/metogra/mpletch1/ResearchProject/mode_objects/mode_060000L_20180301_180000V_060000A.ps
> > > >
> > > > But since I have so much data, I wanted to create a program
that
> would
> > > > automatically loop through all the data that I have, which is
what I
> > was
> > > > trying to use time_loop.py for, which I included in the first
email.
> If
> > > > there's any changes you believe should be made to that
program,
> please
> > > let
> > > > me know.
> > > >
> > > > Thanks,
> > > > Mchael
> > > >
> > > > On Wed, Mar 4, 2020 at 1:25 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Michael,
> > > > >
> > > > > OK, that's much better. MODE now runs and the log messages
> indicate
> > > that
> > > > > it finds 0 objects. That's why the output files are empty:
> > > > > DEBUG 2: Identified: 0 forecast objects and 0 observation
objects.
> > > > >
> > > > > It does however create a PostScript plot summarizing the
run. I
> > > usually
> > > > > use "ghostview" to visualize PostScript files, but different
> systems
> > > > handle
> > > > > them differently.
> > > > > I've attached a PNG version of the second page of that file.
If
> you
> > > look
> > > > > very closely, there's a tiny blip of non-zero data in
northern CA.
> > > > >
> > > > > If you reconfigure and rerun with no smoothing (conv_radius
= 0;),
> > the
> > > > > lowest threshold possible (conv_thresh = >0;), and no object
area
> > > > threshold
> > > > > (area_thresh = >0;), MODE does, in fact, find a single
object:
> > > > > DEBUG 2: Identified: 1 forecast objects and 1 observation
objects.
> > > > >
> > > > > Checking the ascii output file (
> > > > > mode_060000L_20180301_180000V_060000A_obj.txt), the AREA of
that
> > object
> > > > is
> > > > > 4.
> > > > >
> > > > > I'd recommend taking a step back to look at your input data
to see
> if
> > > it
> > > > > really contains what you expect. The plot_data_plane tool
in MET
> is
> > > very
> > > > > useful in this regard:
> > > > > plot_data_plane ST4gARI_s2018030112_e2018030118_vhr18.nc
> > > > > ST4gARI_s2018030112_e2018030118_vhr18.ps 'name="ST4gARI";
> > > > level="(*,*)";'
> > > > >
> > > > > It just reads a single 2D field of data and plots it. Of
course,
> the
> > > > > result is really the same as looking at the PostScript
output from
> > > MODE.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > >
> > > > --
> > > > Michael Pletcher
> > > > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > > > University of Maryland, College Park Class of 2020
> > > > mpletch1 at terpmail.umd.edu|443-857-1127
> > > >
> > > >
> > >
> > >
> >
> > --
> > Michael Pletcher
> > B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
> > University of Maryland, College Park Class of 2020
> > mpletch1 at terpmail.umd.edu|443-857-1127
> >
> >
>
>
--
Michael Pletcher
B.S. Candidate, Atmospheric & Oceanic Science / GIS Minor
University of Maryland, College Park Class of 2020
mpletch1 at terpmail.umd.edu|443-857-1127
------------------------------------------------
More information about the Met_help
mailing list