[Met_help] [rt.rap.ucar.edu #83389] History for grid_stat Error
John Halley Gotway via RT
met_help at ucar.edu
Wed Jul 10 16:39:18 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am trying to run MET grid_stat on Theia.
After running MET, I am getting results only for point_stat even though I
have the task grid_stat on the xml file.
/scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
The only error I had found in the log files is related to CMORPH data.
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900/met/log/
met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
ERROR : Trouble reading observation file
"/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
I am not sure if the way created the CMORPH data have something to do with
this error.
To get the CMORPH data I downloaded the data from
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.CMORPH/.3-hourly/.dataset_documentation.html
and the data is 0.25 degree 3-hourly Information.
After getting the data, I used NCO to create a mean of two 3 hr to get
netcdf files every 6 hr. Then I combined them to make a 24hr netcdf file
for MET.
Thanks,
Keren
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: grid_stat Error
From: Minna Win
Time: Fri Dec 22 11:35:31 2017
Hello Keren,
The MET development team is out of the office due to the holidays and
as a
result there will be a delay in METhelp response time. We will
address
your issue when everyone returns to the office on January 2, 2018.
Thank you for your patience.
Regards,
Minna
---------------
Minna Win
NCAR
Research Applications Lab
Phone: 303-497-8423
Fax: 302-497-8401
On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> Transaction: Ticket created by keren.rosado at noaa.gov
> Queue: met_help
> Subject: grid_stat Error
> Owner: Nobody
> Requestors: keren.rosado at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
>
>
> Hello,
>
> I am trying to run MET grid_stat on Theia.
> After running MET, I am getting results only for point_stat even
though I
> have the task grid_stat on the xml file.
> /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
>
> The only error I had found in the log files is related to CMORPH
data.
>
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900/met/log/
> met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
>
> ERROR : Trouble reading observation file
> "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
>
>
>
> I am not sure if the way created the CMORPH data have something to
do with
> this error.
>
>
> To get the CMORPH data I downloaded the data from
> https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> CMORPH/.3-hourly/.dataset_documentation.html
> and the data is 0.25 degree 3-hourly Information.
>
>
> After getting the data, I used NCO to create a mean of two 3 hr to
get
> netcdf files every 6 hr. Then I combined them to make a 24hr netcdf
file
> for MET.
>
>
>
> Thanks,
>
>
> Keren
>
>
------------------------------------------------
Subject: grid_stat Error
From: John Halley Gotway
Time: Thu Dec 28 13:51:19 2017
Hi Karen,
I logged on to theia and took a quick look. Yes, I agree that the
problem
is related to that NetCDF file that resulted in the error message:
/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/
CMORPH_20160930_120000_24h.nc
It sounds like you ultimately want 24-hour accumulations of CMORPH
data.
The immediate problem is that MET doesn't know how to interpret the
NetCDF
file format you're using.
There is a 24-hour CMORPH product available in this location:
ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
That would be easier to work with rather than 8 3-hourly files,
assuming
the end-points of the 24-hour time window match the end points you'd
like
to evaluate. I wrote some of my colleagues who have run scripts to
convert
GRADS data to GRIB but haven't heard back from them yet.
If we can get that logic to work well, hopefully we can get things
running
smoothly.
Thanks,
John
On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
>
> Hello Keren,
>
> The MET development team is out of the office due to the holidays
and as a
> result there will be a delay in METhelp response time. We will
address
> your issue when everyone returns to the office on January 2, 2018.
>
> Thank you for your patience.
>
> Regards,
> Minna
>
> ---------------
> Minna Win
> NCAR
> Research Applications Lab
> Phone: 303-497-8423
> Fax: 302-497-8401
>
>
> On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > Transaction: Ticket created by keren.rosado at noaa.gov
> > Queue: met_help
> > Subject: grid_stat Error
> > Owner: Nobody
> > Requestors: keren.rosado at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> >
> >
> > Hello,
> >
> > I am trying to run MET grid_stat on Theia.
> > After running MET, I am getting results only for point_stat even
though I
> > have the task grid_stat on the xml file.
> > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> >
> > The only error I had found in the log files is related to CMORPH
data.
> > /scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900
> /met/log/
> > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> >
> > ERROR : Trouble reading observation file
> > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> >
> >
> >
> > I am not sure if the way created the CMORPH data have something to
do
> with
> > this error.
> >
> >
> > To get the CMORPH data I downloaded the data from
> > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > CMORPH/.3-hourly/.dataset_documentation.html
> > and the data is 0.25 degree 3-hourly Information.
> >
> >
> > After getting the data, I used NCO to create a mean of two 3 hr to
get
> > netcdf files every 6 hr. Then I combined them to make a 24hr
netcdf file
> > for MET.
> >
> >
> >
> > Thanks,
> >
> >
> > Keren
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: Keren Rosado - NOAA Affiliate
Time: Tue Jan 02 09:11:55 2018
Hi John,
I tried that CMORPH product before, but it only has 2017 data and I
need
2016. That is why I am using the 3-hourly files.
Thanks,
Keren
----
Keren Rosado, Ph.D.
NCAS-M Postdoctoral Fellow
NOAA ESRL
Office: (303) 497-5907
On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Karen,
>
> I logged on to theia and took a quick look. Yes, I agree that the
problem
> is related to that NetCDF file that resulted in the error message:
>
>
> /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> cmorph/BUCKET_24h/20160930/
> CMORPH_20160930_120000_24h.nc
>
>
> It sounds like you ultimately want 24-hour accumulations of CMORPH
data.
> The immediate problem is that MET doesn't know how to interpret the
NetCDF
> file format you're using.
>
>
> There is a 24-hour CMORPH product available in this location:
>
> ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
>
>
> That would be easier to work with rather than 8 3-hourly files,
assuming
> the end-points of the 24-hour time window match the end points you'd
like
> to evaluate. I wrote some of my colleagues who have run scripts to
convert
> GRADS data to GRIB but haven't heard back from them yet.
>
>
> If we can get that logic to work well, hopefully we can get things
running
> smoothly.
>
>
> Thanks,
> John
>
>
>
>
>
>
> On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> >
> > Hello Keren,
> >
> > The MET development team is out of the office due to the holidays
and as
> a
> > result there will be a delay in METhelp response time. We will
address
> > your issue when everyone returns to the office on January 2, 2018.
> >
> > Thank you for your patience.
> >
> > Regards,
> > Minna
> >
> > ---------------
> > Minna Win
> > NCAR
> > Research Applications Lab
> > Phone: 303-497-8423
> > Fax: 302-497-8401
> >
> >
> > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > Queue: met_help
> > > Subject: grid_stat Error
> > > Owner: Nobody
> > > Requestors: keren.rosado at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389
> >
> > >
> > >
> > > Hello,
> > >
> > > I am trying to run MET grid_stat on Theia.
> > > After running MET, I am getting results only for point_stat even
> though I
> > > have the task grid_stat on the xml file.
> > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > >
> > > The only error I had found in the log files is related to
CMORPH data.
> > >
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900
> > /met/log/
> > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > >
> > > ERROR : Trouble reading observation file
> > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > >
> > >
> > >
> > > I am not sure if the way created the CMORPH data have something
to do
> > with
> > > this error.
> > >
> > >
> > > To get the CMORPH data I downloaded the data from
> > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > CMORPH/.3-hourly/.dataset_documentation.html
> > > and the data is 0.25 degree 3-hourly Information.
> > >
> > >
> > > After getting the data, I used NCO to create a mean of two 3 hr
to get
> > > netcdf files every 6 hr. Then I combined them to make a 24hr
netcdf
> file
> > > for MET.
> > >
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Keren
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: John Halley Gotway
Time: Tue Jan 02 09:59:28 2018
Hi Keren,
I do see that the CPC website claims to provide 3-hourly back to 2002:
http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_description.html
But when you actually click on the link, it only serves up data for
2017.
Where are you actually pulling this 3-hourly data? I see that you're
running on theia. Does the CMORPH data live somewhere on theia
already or
are you pulling it from HTTP or FTP?
Thanks,
John
On Tue, Jan 2, 2018 at 9:11 AM, Keren Rosado - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
>
> Hi John,
>
> I tried that CMORPH product before, but it only has 2017 data and I
need
> 2016. That is why I am using the 3-hourly files.
>
>
> Thanks,
>
> Keren
>
> ----
> Keren Rosado, Ph.D.
> NCAS-M Postdoctoral Fellow
> NOAA ESRL
> Office: (303) 497-5907
>
> On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Karen,
> >
> > I logged on to theia and took a quick look. Yes, I agree that the
> problem
> > is related to that NetCDF file that resulted in the error message:
> >
> >
> > /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > cmorph/BUCKET_24h/20160930/
> > CMORPH_20160930_120000_24h.nc
> >
> >
> > It sounds like you ultimately want 24-hour accumulations of CMORPH
data.
> > The immediate problem is that MET doesn't know how to interpret
the
> NetCDF
> > file format you're using.
> >
> >
> > There is a 24-hour CMORPH product available in this location:
> >
> > ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
> >
> >
> > That would be easier to work with rather than 8 3-hourly files,
assuming
> > the end-points of the 24-hour time window match the end points
you'd like
> > to evaluate. I wrote some of my colleagues who have run scripts
to
> convert
> > GRADS data to GRIB but haven't heard back from them yet.
> >
> >
> > If we can get that logic to work well, hopefully we can get things
> running
> > smoothly.
> >
> >
> > Thanks,
> > John
> >
> >
> >
> >
> >
> >
> > On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > >
> > > Hello Keren,
> > >
> > > The MET development team is out of the office due to the
holidays and
> as
> > a
> > > result there will be a delay in METhelp response time. We will
address
> > > your issue when everyone returns to the office on January 2,
2018.
> > >
> > > Thank you for your patience.
> > >
> > > Regards,
> > > Minna
> > >
> > > ---------------
> > > Minna Win
> > > NCAR
> > > Research Applications Lab
> > > Phone: 303-497-8423
> > > Fax: 302-497-8401
> > >
> > >
> > > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > > Queue: met_help
> > > > Subject: grid_stat Error
> > > > Owner: Nobody
> > > > Requestors: keren.rosado at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=83389
> > >
> > > >
> > > >
> > > > Hello,
> > > >
> > > > I am trying to run MET grid_stat on Theia.
> > > > After running MET, I am getting results only for point_stat
even
> > though I
> > > > have the task grid_stat on the xml file.
> > > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > > >
> > > > The only error I had found in the log files is related to
CMORPH
> data.
> > > >
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900
> > > /met/log/
> > > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > > >
> > > > ERROR : Trouble reading observation file
> > > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > > >
> > > >
> > > >
> > > > I am not sure if the way created the CMORPH data have
something to do
> > > with
> > > > this error.
> > > >
> > > >
> > > > To get the CMORPH data I downloaded the data from
> > > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > > CMORPH/.3-hourly/.dataset_documentation.html
> > > > and the data is 0.25 degree 3-hourly Information.
> > > >
> > > >
> > > > After getting the data, I used NCO to create a mean of two 3
hr to
> get
> > > > netcdf files every 6 hr. Then I combined them to make a 24hr
netcdf
> > file
> > > > for MET.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Keren
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: Keren Rosado - NOAA Affiliate
Time: Tue Jan 02 10:18:06 2018
Hi John,
I used this link to select the days I want.
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.CMORPH/.3-
hourly/.mean/.morphed/.cmorph/dataselection.html?limit.X.value=0.125E+to+0.125W&limit.Y.value=59.875N+to+59.875S&limit.T.value=0000-
0300+29+Sep+2016+to+0000-0300+04+Oct+2016
>From this link I downloaded the NetCDF file called data.nc witch has
the
requested days.
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.CMORPH/.3-
hourly/.mean/.morphed/.cmorph/T/%280000-
0300%2029%20Sep%202016%29%280000-
0300%2004%20Oct%202016%29RANGEEDGES/datafiles.html
After that, I created individual 3 hour NetCDF files, with later I
combined
to create a 24 hour NetCDF files.
Thanks,
Keren
----
Keren Rosado, Ph.D.
NCAS-M Postdoctoral Fellow
NOAA ESRL
Office: (303) 497-5907
On Tue, Jan 2, 2018 at 9:59 AM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Hi Keren,
>
> I do see that the CPC website claims to provide 3-hourly back to
2002:
>
http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_description.html
>
> But when you actually click on the link, it only serves up data for
2017.
> Where are you actually pulling this 3-hourly data? I see that
you're
> running on theia. Does the CMORPH data live somewhere on theia
already or
> are you pulling it from HTTP or FTP?
>
> Thanks,
> John
>
> On Tue, Jan 2, 2018 at 9:11 AM, Keren Rosado - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> >
> > Hi John,
> >
> > I tried that CMORPH product before, but it only has 2017 data and
I need
> > 2016. That is why I am using the 3-hourly files.
> >
> >
> > Thanks,
> >
> > Keren
> >
> > ----
> > Keren Rosado, Ph.D.
> > NCAS-M Postdoctoral Fellow
> > NOAA ESRL
> > Office: (303) 497-5907
> >
> > On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hi Karen,
> > >
> > > I logged on to theia and took a quick look. Yes, I agree that
the
> > problem
> > > is related to that NetCDF file that resulted in the error
message:
> > >
> > >
> > > /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > cmorph/BUCKET_24h/20160930/
> > > CMORPH_20160930_120000_24h.nc
> > >
> > >
> > > It sounds like you ultimately want 24-hour accumulations of
CMORPH
> data.
> > > The immediate problem is that MET doesn't know how to interpret
the
> > NetCDF
> > > file format you're using.
> > >
> > >
> > > There is a 24-hour CMORPH product available in this location:
> > >
> > >
ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
> > >
> > >
> > > That would be easier to work with rather than 8 3-hourly files,
> assuming
> > > the end-points of the 24-hour time window match the end points
you'd
> like
> > > to evaluate. I wrote some of my colleagues who have run scripts
to
> > convert
> > > GRADS data to GRIB but haven't heard back from them yet.
> > >
> > >
> > > If we can get that logic to work well, hopefully we can get
things
> > running
> > > smoothly.
> > >
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT
<met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389
>
> > > >
> > > > Hello Keren,
> > > >
> > > > The MET development team is out of the office due to the
holidays and
> > as
> > > a
> > > > result there will be a delay in METhelp response time. We
will
> address
> > > > your issue when everyone returns to the office on January 2,
2018.
> > > >
> > > > Thank you for your patience.
> > > >
> > > > Regards,
> > > > Minna
> > > >
> > > > ---------------
> > > > Minna Win
> > > > NCAR
> > > > Research Applications Lab
> > > > Phone: 303-497-8423
> > > > Fax: 302-497-8401
> > > >
> > > >
> > > > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > > > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: grid_stat Error
> > > > > Owner: Nobody
> > > > > Requestors: keren.rosado at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > Ticket/Display.html?id=83389
> > > >
> > > > >
> > > > >
> > > > > Hello,
> > > > >
> > > > > I am trying to run MET grid_stat on Theia.
> > > > > After running MET, I am getting results only for point_stat
even
> > > though I
> > > > > have the task grid_stat on the xml file.
> > > > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > > > >
> > > > > The only error I had found in the log files is related to
CMORPH
> > data.
> > > > >
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900
> > > > /met/log/
> > > > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > > > >
> > > > > ERROR : Trouble reading observation file
> > > > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > > > >
> > > > >
> > > > >
> > > > > I am not sure if the way created the CMORPH data have
something to
> do
> > > > with
> > > > > this error.
> > > > >
> > > > >
> > > > > To get the CMORPH data I downloaded the data from
> > > > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > > > CMORPH/.3-hourly/.dataset_documentation.html
> > > > > and the data is 0.25 degree 3-hourly Information.
> > > > >
> > > > >
> > > > > After getting the data, I used NCO to create a mean of two 3
hr to
> > get
> > > > > netcdf files every 6 hr. Then I combined them to make a 24hr
netcdf
> > > file
> > > > > for MET.
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > > Keren
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: John Halley Gotway
Time: Tue Jan 02 11:42:34 2018
Hi Keren,
This CMORPH data does live on the HPSS on theia in the following
files:
/NCEPPROD/hpssprod/runhistory/rhYYYY/YYYYMM/YYYYMMDD/dcom_
us007003_YYYYMMDD.tar
For example:
/NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
us007003_20160807.tar
I ran the following commands to extract CMORPH data files from these
files,
but apparently I don't have permissions to access that tar file:
module load hpss
time htar -xvf
/NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_us007003_20160807.tar
`htar -tf
/NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_us007003_20160807.tar
| grep wgrbbul | grep 'CMORPH' | awk '{print $7}'`
ERROR: [FATAL] no permission to open HPSS archive file:
/NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
us007003_20160807.tar
Perhaps you do have permission?
Alternatively, another source of this data is the NCAR Research Data
Archive (RDA):
https://rda.ucar.edu/datasets/ds502.0/
The RDA has the data in both binary format and a CF-compliant NetCDF
format
that MET should be able to read. To test this, I pulled some 3-hourly
NetCDF data from that site and then ran the following command to plot
one
of the time steps. I've attached the resulting plot:
/usr/local/met-6.1/bin/plot_data_plane cmorph.3hr-025deg.20160807.nc
cmorph.ps 'name="cmorph_precip"; level="(3,*,*)";' -v 4
Perhaps you could run you NCO commands on the CF-complaint data to
compute
the mean over the 8 3-hour time slices?
Hope this helps.
John
On Tue, Jan 2, 2018 at 10:18 AM, Keren Rosado - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
>
> Hi John,
>
>
>
> I used this link to select the days I want.
> https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> CMORPH/.3-hourly/.mean/.morphed/.cmorph/dataselection.
> html?limit.X.value=0.125E+to+0.125W&limit.Y.value=59.875N+
> to+59.875S&limit.T.value=0000-0300+29+Sep+2016+to+0000-
0300+04+Oct+2016
>
> From this link I downloaded the NetCDF file called data.nc witch has
the
> requested days.
> https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> CMORPH/.3-hourly/.mean/.morphed/.cmorph/T/%280000-
> 0300%2029%20Sep%202016%29%280000-0300%2004%20Oct%202016%
> 29RANGEEDGES/datafiles.html
>
> After that, I created individual 3 hour NetCDF files, with later I
combined
> to create a 24 hour NetCDF files.
>
>
> Thanks,
>
> Keren
>
>
> ----
> Keren Rosado, Ph.D.
> NCAS-M Postdoctoral Fellow
> NOAA ESRL
> Office: (303) 497-5907
>
> On Tue, Jan 2, 2018 at 9:59 AM, John Halley Gotway via RT <
> met_help at ucar.edu
> > wrote:
>
> > Hi Keren,
> >
> > I do see that the CPC website claims to provide 3-hourly back to
2002:
> > http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_
> description.html
> >
> > But when you actually click on the link, it only serves up data
for 2017.
> > Where are you actually pulling this 3-hourly data? I see that
you're
> > running on theia. Does the CMORPH data live somewhere on theia
already
> or
> > are you pulling it from HTTP or FTP?
> >
> > Thanks,
> > John
> >
> > On Tue, Jan 2, 2018 at 9:11 AM, Keren Rosado - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > >
> > > Hi John,
> > >
> > > I tried that CMORPH product before, but it only has 2017 data
and I
> need
> > > 2016. That is why I am using the 3-hourly files.
> > >
> > >
> > > Thanks,
> > >
> > > Keren
> > >
> > > ----
> > > Keren Rosado, Ph.D.
> > > NCAS-M Postdoctoral Fellow
> > > NOAA ESRL
> > > Office: (303) 497-5907
> > >
> > > On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Hi Karen,
> > > >
> > > > I logged on to theia and took a quick look. Yes, I agree that
the
> > > problem
> > > > is related to that NetCDF file that resulted in the error
message:
> > > >
> > > >
> > > > /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > cmorph/BUCKET_24h/20160930/
> > > > CMORPH_20160930_120000_24h.nc
> > > >
> > > >
> > > > It sounds like you ultimately want 24-hour accumulations of
CMORPH
> > data.
> > > > The immediate problem is that MET doesn't know how to
interpret the
> > > NetCDF
> > > > file format you're using.
> > > >
> > > >
> > > > There is a 24-hour CMORPH product available in this location:
> > > >
> > > >
ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
> > > >
> > > >
> > > > That would be easier to work with rather than 8 3-hourly
files,
> > assuming
> > > > the end-points of the 24-hour time window match the end points
you'd
> > like
> > > > to evaluate. I wrote some of my colleagues who have run
scripts to
> > > convert
> > > > GRADS data to GRIB but haven't heard back from them yet.
> > > >
> > > >
> > > > If we can get that logic to work well, hopefully we can get
things
> > > running
> > > > smoothly.
> > > >
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT <
> met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > > > >
> > > > > Hello Keren,
> > > > >
> > > > > The MET development team is out of the office due to the
holidays
> and
> > > as
> > > > a
> > > > > result there will be a delay in METhelp response time. We
will
> > address
> > > > > your issue when everyone returns to the office on January 2,
2018.
> > > > >
> > > > > Thank you for your patience.
> > > > >
> > > > > Regards,
> > > > > Minna
> > > > >
> > > > > ---------------
> > > > > Minna Win
> > > > > NCAR
> > > > > Research Applications Lab
> > > > > Phone: 303-497-8423
> > > > > Fax: 302-497-8401
> > > > >
> > > > >
> > > > > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA
Affiliate via
> > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > > > > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: grid_stat Error
> > > > > > Owner: Nobody
> > > > > > Requestors: keren.rosado at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > Ticket/Display.html?id=83389
> > > > >
> > > > > >
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I am trying to run MET grid_stat on Theia.
> > > > > > After running MET, I am getting results only for
point_stat even
> > > > though I
> > > > > > have the task grid_stat on the xml file.
> > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > > > > >
> > > > > > The only error I had found in the log files is related to
CMORPH
> > > data.
> > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/
> 2016092900
> > > > > /met/log/
> > > > > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > > > > >
> > > > > > ERROR : Trouble reading observation file
> > > > > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > > cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > > > > >
> > > > > >
> > > > > >
> > > > > > I am not sure if the way created the CMORPH data have
something
> to
> > do
> > > > > with
> > > > > > this error.
> > > > > >
> > > > > >
> > > > > > To get the CMORPH data I downloaded the data from
> > > > > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > > > > CMORPH/.3-hourly/.dataset_documentation.html
> > > > > > and the data is 0.25 degree 3-hourly Information.
> > > > > >
> > > > > >
> > > > > > After getting the data, I used NCO to create a mean of two
3 hr
> to
> > > get
> > > > > > netcdf files every 6 hr. Then I combined them to make a
24hr
> netcdf
> > > > file
> > > > > > for MET.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > >
> > > > > > Keren
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: Keren Rosado - NOAA Affiliate
Time: Fri Jan 19 14:48:59 2018
Hi John,
I have the same permission problem to access the CMORPH data on Theia.
In the last test, I used NCAR Research Data Archive (RDA) to run MET.
Now I have another error:
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900/met>
vi
met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
DEBUG 1: Observation File: /scratch3/BMC/fim/Keren
.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc
DEBUG 1: Regridding forecast APCP_24(*,*) to the verification grid.
WARNING:
WARNING: process_scores() -> APCP_24(*,*) not found in file:
/scratch3/BMC/fim/Keren
.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc
Thanks,
Keren
----
Keren Rosado, Ph.D.
NCAS-M Postdoctoral Fellow
NOAA ESRL
Office: (303) 497-5907
On Tue, Jan 2, 2018 at 11:42 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Keren,
>
> This CMORPH data does live on the HPSS on theia in the following
files:
> /NCEPPROD/hpssprod/runhistory/rhYYYY/YYYYMM/YYYYMMDD/dcom_
> us007003_YYYYMMDD.tar
>
> For example:
> /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> us007003_20160807.tar
>
> I ran the following commands to extract CMORPH data files from these
files,
> but apparently I don't have permissions to access that tar file:
>
> module load hpss
> time htar -xvf
> /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> us007003_20160807.tar
> `htar -tf /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> us007003_20160807.tar
> | grep wgrbbul | grep 'CMORPH' | awk '{print $7}'`
>
> ERROR: [FATAL] no permission to open HPSS archive file:
> /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> us007003_20160807.tar
>
> Perhaps you do have permission?
>
> Alternatively, another source of this data is the NCAR Research Data
> Archive (RDA):
> https://rda.ucar.edu/datasets/ds502.0/
>
> The RDA has the data in both binary format and a CF-compliant NetCDF
format
> that MET should be able to read. To test this, I pulled some 3-
hourly
> NetCDF data from that site and then ran the following command to
plot one
> of the time steps. I've attached the resulting plot:
>
> /usr/local/met-6.1/bin/plot_data_plane cmorph.3hr-025deg.20160807.nc
> cmorph.ps 'name="cmorph_precip"; level="(3,*,*)";' -v 4
>
> Perhaps you could run you NCO commands on the CF-complaint data to
compute
> the mean over the 8 3-hour time slices?
>
> Hope this helps.
>
> John
>
>
>
>
>
>
> On Tue, Jan 2, 2018 at 10:18 AM, Keren Rosado - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> >
> > Hi John,
> >
> >
> >
> > I used this link to select the days I want.
> > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > CMORPH/.3-hourly/.mean/.morphed/.cmorph/dataselection.
> > html?limit.X.value=0.125E+to+0.125W&limit.Y.value=59.875N+
> > to+59.875S&limit.T.value=0000-0300+29+Sep+2016+to+0000-
0300+04+Oct+2016
> >
> > From this link I downloaded the NetCDF file called data.nc witch
has the
> > requested days.
> > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > CMORPH/.3-hourly/.mean/.morphed/.cmorph/T/%280000-
> > 0300%2029%20Sep%202016%29%280000-0300%2004%20Oct%202016%
> > 29RANGEEDGES/datafiles.html
> >
> > After that, I created individual 3 hour NetCDF files, with later I
> combined
> > to create a 24 hour NetCDF files.
> >
> >
> > Thanks,
> >
> > Keren
> >
> >
> > ----
> > Keren Rosado, Ph.D.
> > NCAS-M Postdoctoral Fellow
> > NOAA ESRL
> > Office: (303) 497-5907
> >
> > On Tue, Jan 2, 2018 at 9:59 AM, John Halley Gotway via RT <
> > met_help at ucar.edu
> > > wrote:
> >
> > > Hi Keren,
> > >
> > > I do see that the CPC website claims to provide 3-hourly back to
2002:
> > > http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_
> > description.html
> > >
> > > But when you actually click on the link, it only serves up data
for
> 2017.
> > > Where are you actually pulling this 3-hourly data? I see that
you're
> > > running on theia. Does the CMORPH data live somewhere on theia
already
> > or
> > > are you pulling it from HTTP or FTP?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jan 2, 2018 at 9:11 AM, Keren Rosado - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389
>
> > > >
> > > > Hi John,
> > > >
> > > > I tried that CMORPH product before, but it only has 2017 data
and I
> > need
> > > > 2016. That is why I am using the 3-hourly files.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Keren
> > > >
> > > > ----
> > > > Keren Rosado, Ph.D.
> > > > NCAS-M Postdoctoral Fellow
> > > > NOAA ESRL
> > > > Office: (303) 497-5907
> > > >
> > > > On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Hi Karen,
> > > > >
> > > > > I logged on to theia and took a quick look. Yes, I agree
that the
> > > > problem
> > > > > is related to that NetCDF file that resulted in the error
message:
> > > > >
> > > > >
> > > > > /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > cmorph/BUCKET_24h/20160930/
> > > > > CMORPH_20160930_120000_24h.nc
> > > > >
> > > > >
> > > > > It sounds like you ultimately want 24-hour accumulations of
CMORPH
> > > data.
> > > > > The immediate problem is that MET doesn't know how to
interpret the
> > > > NetCDF
> > > > > file format you're using.
> > > > >
> > > > >
> > > > > There is a 24-hour CMORPH product available in this
location:
> > > > >
> > > > >
ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_025deg/
> > > > >
> > > > >
> > > > > That would be easier to work with rather than 8 3-hourly
files,
> > > assuming
> > > > > the end-points of the 24-hour time window match the end
points
> you'd
> > > like
> > > > > to evaluate. I wrote some of my colleagues who have run
scripts to
> > > > convert
> > > > > GRADS data to GRIB but haven't heard back from them yet.
> > > > >
> > > > >
> > > > > If we can get that logic to work well, hopefully we can get
things
> > > > running
> > > > > smoothly.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT <
> > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > > > > >
> > > > > > Hello Keren,
> > > > > >
> > > > > > The MET development team is out of the office due to the
holidays
> > and
> > > > as
> > > > > a
> > > > > > result there will be a delay in METhelp response time. We
will
> > > address
> > > > > > your issue when everyone returns to the office on January
2,
> 2018.
> > > > > >
> > > > > > Thank you for your patience.
> > > > > >
> > > > > > Regards,
> > > > > > Minna
> > > > > >
> > > > > > ---------------
> > > > > > Minna Win
> > > > > > NCAR
> > > > > > Research Applications Lab
> > > > > > Phone: 303-497-8423
> > > > > > Fax: 302-497-8401
> > > > > >
> > > > > >
> > > > > > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA
Affiliate
> via
> > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Fri Dec 22 10:38:41 2017: Request 83389 was acted upon.
> > > > > > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: grid_stat Error
> > > > > > > Owner: Nobody
> > > > > > > Requestors: keren.rosado at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > Ticket/Display.html?id=83389
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I am trying to run MET grid_stat on Theia.
> > > > > > > After running MET, I am getting results only for
point_stat
> even
> > > > > though I
> > > > > > > have the task grid_stat on the xml file.
> > > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > > > > > >
> > > > > > > The only error I had found in the log files is related
to
> CMORPH
> > > > data.
> > > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/
> > 2016092900
> > > > > > /met/log/
> > > > > > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > > > > > >
> > > > > > > ERROR : Trouble reading observation file
> > > > > > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > > >
cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I am not sure if the way created the CMORPH data have
something
> > to
> > > do
> > > > > > with
> > > > > > > this error.
> > > > > > >
> > > > > > >
> > > > > > > To get the CMORPH data I downloaded the data from
> > > > > > >
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > > > > > CMORPH/.3-hourly/.dataset_documentation.html
> > > > > > > and the data is 0.25 degree 3-hourly Information.
> > > > > > >
> > > > > > >
> > > > > > > After getting the data, I used NCO to create a mean of
two 3 hr
> > to
> > > > get
> > > > > > > netcdf files every 6 hr. Then I combined them to make a
24hr
> > netcdf
> > > > > file
> > > > > > > for MET.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > >
> > > > > > > Keren
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Error
From: John Halley Gotway
Time: Fri Jan 19 16:30:20 2018
Keren,
I see that you're having trouble accessing data from these NetCDF
files.
I'd suggest running 'ncdump -h' to list their contents:
ncdump -h
/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc
FYI, I had to load a couple of modules to us ncdump on theia:
module load intel/15.0.0
module load netcdf/4.4.0
The output of ncdump reveals that the data variable is named
"cmorph_precip":
float cmorph_precip(time, lat, lon) ;
cmorph_precip:long_name = "CMORPH precipitation
estimate" ;
cmorph_precip:standard_name = "lwe_precipitation_rate"
;
cmorph_precip:units = "mm hr^-1" ;
cmorph_precip:_FillValue = -9999.f ;
cmorph_precip:missing_value = -9999.f ;
For NetCDF files, you need to tell MET the name of the variable that
you'd
like to use. And notice that this is a precipitation rate over the
24-hour
period (millimeters per hour). I ran the following command to plot
this
data:
module use /contrib/modulefiles
module load met/6.1
plot_data_plane
/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc
\
~/cmorph_precip_rate.ps 'name="cmorph_precip"; level="(0,*,*)";'
And I added the "convert" option (new in met-6.1) to convert that 24-
hour
rate to a 24-hour accumulation:
plot_data_plane
/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc
\
~/cmorph_precip_accum.ps 'name="cmorph_precip"; level="(0,*,*)";
convert(x)=24*x;'
Copies of the resulting images are attached. Notice that the pattern
is
the same, but the values differ by a factor of 24.
Hope that helps.
Thanks,
John
On Fri, Jan 19, 2018 at 2:48 PM, Keren Rosado - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
>
> Hi John,
>
> I have the same permission problem to access the CMORPH data on
Theia.
>
> In the last test, I used NCAR Research Data Archive (RDA) to run
MET.
>
> Now I have another error:
>
>
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/2016092900/met>
vi
> met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
>
>
> DEBUG 1: Observation File: /scratch3/BMC/fim/Keren
> .Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/
> CMORPH_20160930_120000_24h.nc
>
> DEBUG 1: Regridding forecast APCP_24(*,*) to the verification grid.
>
> WARNING:
>
> WARNING: process_scores() -> APCP_24(*,*) not found in file:
> /scratch3/BMC/fim/Keren
> .Rosado/MET/data/vx_data/cmorph/BUCKET_24h/20160930/
> CMORPH_20160930_120000_24h.nc
>
>
>
> Thanks,
>
>
> Keren
>
>
>
>
> ----
> Keren Rosado, Ph.D.
> NCAS-M Postdoctoral Fellow
> NOAA ESRL
> Office: (303) 497-5907
>
> On Tue, Jan 2, 2018 at 11:42 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Keren,
> >
> > This CMORPH data does live on the HPSS on theia in the following
files:
> > /NCEPPROD/hpssprod/runhistory/rhYYYY/YYYYMM/YYYYMMDD/dcom_
> > us007003_YYYYMMDD.tar
> >
> > For example:
> > /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> > us007003_20160807.tar
> >
> > I ran the following commands to extract CMORPH data files from
these
> files,
> > but apparently I don't have permissions to access that tar file:
> >
> > module load hpss
> > time htar -xvf
> > /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> > us007003_20160807.tar
> > `htar -tf
/NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> > us007003_20160807.tar
> > | grep wgrbbul | grep 'CMORPH' | awk '{print $7}'`
> >
> > ERROR: [FATAL] no permission to open HPSS archive file:
> > /NCEPPROD/hpssprod/runhistory/rh2016/201608/20160807/dcom_
> > us007003_20160807.tar
> >
> > Perhaps you do have permission?
> >
> > Alternatively, another source of this data is the NCAR Research
Data
> > Archive (RDA):
> > https://rda.ucar.edu/datasets/ds502.0/
> >
> > The RDA has the data in both binary format and a CF-compliant
NetCDF
> format
> > that MET should be able to read. To test this, I pulled some 3-
hourly
> > NetCDF data from that site and then ran the following command to
plot one
> > of the time steps. I've attached the resulting plot:
> >
> > /usr/local/met-6.1/bin/plot_data_plane cmorph.3hr-
025deg.20160807.nc
> > cmorph.ps 'name="cmorph_precip"; level="(3,*,*)";' -v 4
> >
> > Perhaps you could run you NCO commands on the CF-complaint data to
> compute
> > the mean over the 8 3-hour time slices?
> >
> > Hope this helps.
> >
> > John
> >
> >
> >
> >
> >
> >
> > On Tue, Jan 2, 2018 at 10:18 AM, Keren Rosado - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > >
> > > Hi John,
> > >
> > >
> > >
> > > I used this link to select the days I want.
> > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > CMORPH/.3-hourly/.mean/.morphed/.cmorph/dataselection.
> > > html?limit.X.value=0.125E+to+0.125W&limit.Y.value=59.875N+
> > > to+59.875S&limit.T.value=0000-0300+29+Sep+2016+to+0000-0300+
> 04+Oct+2016
> > >
> > > From this link I downloaded the NetCDF file called data.nc witch
has
> the
> > > requested days.
> > > https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > CMORPH/.3-hourly/.mean/.morphed/.cmorph/T/%280000-
> > > 0300%2029%20Sep%202016%29%280000-0300%2004%20Oct%202016%
> > > 29RANGEEDGES/datafiles.html
> > >
> > > After that, I created individual 3 hour NetCDF files, with later
I
> > combined
> > > to create a 24 hour NetCDF files.
> > >
> > >
> > > Thanks,
> > >
> > > Keren
> > >
> > >
> > > ----
> > > Keren Rosado, Ph.D.
> > > NCAS-M Postdoctoral Fellow
> > > NOAA ESRL
> > > Office: (303) 497-5907
> > >
> > > On Tue, Jan 2, 2018 at 9:59 AM, John Halley Gotway via RT <
> > > met_help at ucar.edu
> > > > wrote:
> > >
> > > > Hi Keren,
> > > >
> > > > I do see that the CPC website claims to provide 3-hourly back
to
> 2002:
> > > > http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_
> > > description.html
> > > >
> > > > But when you actually click on the link, it only serves up
data for
> > 2017.
> > > > Where are you actually pulling this 3-hourly data? I see that
you're
> > > > running on theia. Does the CMORPH data live somewhere on
theia
> already
> > > or
> > > > are you pulling it from HTTP or FTP?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jan 2, 2018 at 9:11 AM, Keren Rosado - NOAA Affiliate
via RT
> <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > > I tried that CMORPH product before, but it only has 2017
data and I
> > > need
> > > > > 2016. That is why I am using the 3-hourly files.
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Keren
> > > > >
> > > > > ----
> > > > > Keren Rosado, Ph.D.
> > > > > NCAS-M Postdoctoral Fellow
> > > > > NOAA ESRL
> > > > > Office: (303) 497-5907
> > > > >
> > > > > On Thu, Dec 28, 2017 at 1:51 PM, John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Hi Karen,
> > > > > >
> > > > > > I logged on to theia and took a quick look. Yes, I agree
that
> the
> > > > > problem
> > > > > > is related to that NetCDF file that resulted in the error
> message:
> > > > > >
> > > > > >
> > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > > cmorph/BUCKET_24h/20160930/
> > > > > > CMORPH_20160930_120000_24h.nc
> > > > > >
> > > > > >
> > > > > > It sounds like you ultimately want 24-hour accumulations
of
> CMORPH
> > > > data.
> > > > > > The immediate problem is that MET doesn't know how to
interpret
> the
> > > > > NetCDF
> > > > > > file format you're using.
> > > > > >
> > > > > >
> > > > > > There is a 24-hour CMORPH product available in this
location:
> > > > > >
> > > > > > ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/daily_
> 025deg/
> > > > > >
> > > > > >
> > > > > > That would be easier to work with rather than 8 3-hourly
files,
> > > > assuming
> > > > > > the end-points of the 24-hour time window match the end
points
> > you'd
> > > > like
> > > > > > to evaluate. I wrote some of my colleagues who have run
scripts
> to
> > > > > convert
> > > > > > GRADS data to GRIB but haven't heard back from them yet.
> > > > > >
> > > > > >
> > > > > > If we can get that logic to work well, hopefully we can
get
> things
> > > > > running
> > > > > > smoothly.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Dec 22, 2017 at 11:35 AM, Minna Win via RT <
> > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83389
> >
> > > > > > >
> > > > > > > Hello Keren,
> > > > > > >
> > > > > > > The MET development team is out of the office due to the
> holidays
> > > and
> > > > > as
> > > > > > a
> > > > > > > result there will be a delay in METhelp response time.
We will
> > > > address
> > > > > > > your issue when everyone returns to the office on
January 2,
> > 2018.
> > > > > > >
> > > > > > > Thank you for your patience.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Minna
> > > > > > >
> > > > > > > ---------------
> > > > > > > Minna Win
> > > > > > > NCAR
> > > > > > > Research Applications Lab
> > > > > > > Phone: 303-497-8423
> > > > > > > Fax: 302-497-8401
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Dec 22, 2017 at 5:38 PM, Keren Rosado - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Fri Dec 22 10:38:41 2017: Request 83389 was acted
upon.
> > > > > > > > Transaction: Ticket created by keren.rosado at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: grid_stat Error
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: keren.rosado at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> > > > > Ticket/Display.html?id=83389
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > I am trying to run MET grid_stat on Theia.
> > > > > > > > After running MET, I am getting results only for
point_stat
> > even
> > > > > > though I
> > > > > > > > have the task grid_stat on the xml file.
> > > > > > > > /scratch3/BMC/fim/Keren.Rosado/MET/xml/gfMatthew.xml
> > > > > > > >
> > > > > > > > The only error I had found in the log files is
related to
> > CMORPH
> > > > > data.
> > > > > > > >
/scratch3/BMC/fim/Keren.Rosado/MET/log/gf_Matthew_0P25/
> > > 2016092900
> > > > > > > /met/log/
> > > > > > > > met_qpf_global_24h_gf_Matthew_0P25_2016092900_036.log
> > > > > > > >
> > > > > > > > ERROR : Trouble reading observation file
> > > > > > > > "/scratch3/BMC/fim/Keren.Rosado/MET/data/vx_data/
> > > > > > > >
cmorph/BUCKET_24h/20160930/CMORPH_20160930_120000_24h.nc"
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I am not sure if the way created the CMORPH data have
> something
> > > to
> > > > do
> > > > > > > with
> > > > > > > > this error.
> > > > > > > >
> > > > > > > >
> > > > > > > > To get the CMORPH data I downloaded the data from
> > > > > > > >
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.
> > > > > > > > CMORPH/.3-hourly/.dataset_documentation.html
> > > > > > > > and the data is 0.25 degree 3-hourly Information.
> > > > > > > >
> > > > > > > >
> > > > > > > > After getting the data, I used NCO to create a mean of
two 3
> hr
> > > to
> > > > > get
> > > > > > > > netcdf files every 6 hr. Then I combined them to make
a 24hr
> > > netcdf
> > > > > > file
> > > > > > > > for MET.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > >
> > > > > > > > Keren
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list