[Met_help] [rt.rap.ucar.edu #55823] History for Use of regular binary data in MET

Mon Apr 16 08:52:54 MDT 2012

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,

We have daily rainfall observation data for Africa in binary format
(0.1degree resolution). Is there a way of converting these data into MET
compatible formats (grib1 or nc)?

Best regards,

Endalk

-- 
-----------------------------------------------------------------------------------
Endalkachew Bekele
NOAA Climate Prediction Center / African Desk
5200 Auth Road, Rm 811, Camp Springs, MD 20746, USA
Tel: (301) 763 8000 Ext. 7014
e-mail: endalkachew.bekele at noaa.gov

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #55823] Use of regular binary data in MET
From: Paul Oldenburg
Time: Thu Apr 12 08:26:36 2012

Endalk,

The easiest way will be to convert this data, which sounds like it is
gridded, into the format of NetCDF written by the
MET tool pcp_combine.  You can see an example of this format if you
use the NetCDF utility ncdump on the output of the
pcp_combine test script in $MET_BASE/out/pcp_combine, where $MET_BASE
is the MET installation folder on your system.
First, of course, you must have run the test scripts.

In order for MET to read your data, you must determine the projection
information for the data that you are converting
and put it into the NetCDF file in a way that MET will understand.  It
sounds like your data might be in a lat/long
projection.  There is an example script on the MET website, written in
the R statistical language, that converts text
TRMM rainfall data into the MET format of NetCDF.  You can download
this script at the bottom of the following page:

http://www.dtcenter.org/met/users/downloads/analysis_scripts.php

I know this may sound confusing, but it can be done.  Please let me
know if you have any questions.

Thanks,

Paul

On 04/12/2012 07:16 AM, Endalkachew Bekele via RT wrote:
>
> Thu Apr 12 07:16:14 2012: Request 55823 was acted upon.
> Transaction: Ticket created by endalkachew.bekele at noaa.gov
>         Queue: met_help
>       Subject: Use of regular binary data in MET
>         Owner: Nobody
>    Requestors: endalkachew.bekele at noaa.gov
>        Status: new
>   Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
>
>
> Hello,
>
> We have daily rainfall observation data for Africa in binary format
> (0.1degree resolution). Is there a way of converting these data into
MET
> compatible formats (grib1 or nc)?
>
> Best regards,
>
> Endalk
>

------------------------------------------------
Subject: Use of regular binary data in MET
From: Endalkachew Bekele
Time: Thu Apr 12 08:55:24 2012

Thank you so much, Paul!

The data is grided in lat/long projection as in:

ftp://ftp.cpc.ncep.noaa.gov/fews/newalgo_est/

I will go through your sugested ways, and will let you know the
results.

Thank you again.

Endalk

On Thu, Apr 12, 2012 at 10:26 AM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:

> Endalk,
>
> The easiest way will be to convert this data, which sounds like it
is
> gridded, into the format of NetCDF written by the
> MET tool pcp_combine.  You can see an example of this format if you
use
> the NetCDF utility ncdump on the output of the
> pcp_combine test script in $MET_BASE/out/pcp_combine, where
$MET_BASE is
> the MET installation folder on your system.
> First, of course, you must have run the test scripts.
>
> In order for MET to read your data, you must determine the
projection
> information for the data that you are converting
> and put it into the NetCDF file in a way that MET will understand.
It
> sounds like your data might be in a lat/long
> projection.  There is an example script on the MET website, written
in the
> R statistical language, that converts text
> TRMM rainfall data into the MET format of NetCDF.  You can download
this
> script at the bottom of the following page:
>
> http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
>
> I know this may sound confusing, but it can be done.  Please let me
know
> if you have any questions.
>
> Thanks,
>
> Paul
>
>
> On 04/12/2012 07:16 AM, Endalkachew Bekele via RT wrote:
> >
> > Thu Apr 12 07:16:14 2012: Request 55823 was acted upon.
> > Transaction: Ticket created by endalkachew.bekele at noaa.gov
> >         Queue: met_help
> >       Subject: Use of regular binary data in MET
> >         Owner: Nobody
> >    Requestors: endalkachew.bekele at noaa.gov
> >        Status: new
> >   Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
>  >
> >
> > Hello,
> >
> > We have daily rainfall observation data for Africa in binary
format
> > (0.1degree resolution). Is there a way of converting these data
into MET
> > compatible formats (grib1 or nc)?
> >
> > Best regards,
> >
> > Endalk
> >
>
>
>

--
-----------------------------------------------------------------------------------
Endalkachew Bekele
NOAA Climate Prediction Center / African Desk
5200 Auth Road, Rm 811, Camp Springs, MD 20746, USA
Tel: (301) 763 8000 Ext. 7014
e-mail: endalkachew.bekele at noaa.gov

------------------------------------------------
Subject: Use of regular binary data in MET
From: Endalkachew Bekele
Time: Thu Apr 12 15:23:26 2012

Hi Paul,

I did check the shell scripts used to the run pcp_combine tool. The
pcp_combine tool was applied to "grib" files not to plain "binary"
files in
the test run. When I try to apply the pcp_combine tool to my binary
files,
it looks for a grib file and when it doesn't find one, it gives me the
following error:

 bin/pcp_combine \
? -sum 00000000_000000 24 20120411_000000 24 \
? temp\temp.nc \
? -pcpdir temp
ERROR  :
ERROR  : GribFile::skip_header() -> can't find "GRIB" magic cookie
ERROR  :
WARNING:
WARNING: check_file_time() -> can't open precipitation grib file:
temp/all_products.bin.20120411
WARNING:
ERROR  :
ERROR  : sum_grib_files() -> Cannot find a file with a valid time of
20120411_000000 and accumulation time of 240000 matching the regular
expression ".*"
ERROR  :

As an alternative, I tried to use the opengradss' "lats4d" tool to
convert
my binary file into "nc" file. The conversion went well but, the
pcp_combine tool looks for a "grib" file and it doesn't recognize the
new
"nc" file.

Kindly advise in this regard.

Endalk

On Thu, Apr 12, 2012 at 10:26 AM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:

> Endalk,
>
> The easiest way will be to convert this data, which sounds like it
is
> gridded, into the format of NetCDF written by the
> MET tool pcp_combine.  You can see an example of this format if you
use
> the NetCDF utility ncdump on the output of the
> pcp_combine test script in $MET_BASE/out/pcp_combine, where
$MET_BASE is
> the MET installation folder on your system.
> First, of course, you must have run the test scripts.
>
> In order for MET to read your data, you must determine the
projection
> information for the data that you are converting
> and put it into the NetCDF file in a way that MET will understand.
It
> sounds like your data might be in a lat/long
> projection.  There is an example script on the MET website, written
in the
> R statistical language, that converts text
> TRMM rainfall data into the MET format of NetCDF.  You can download
this
> script at the bottom of the following page:
>
> http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
>
> I know this may sound confusing, but it can be done.  Please let me
know
> if you have any questions.
>
> Thanks,
>
> Paul
>
>
> On 04/12/2012 07:16 AM, Endalkachew Bekele via RT wrote:
> >
> > Thu Apr 12 07:16:14 2012: Request 55823 was acted upon.
> > Transaction: Ticket created by endalkachew.bekele at noaa.gov
> >         Queue: met_help
> >       Subject: Use of regular binary data in MET
> >         Owner: Nobody
> >    Requestors: endalkachew.bekele at noaa.gov
> >        Status: new
> >   Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
> >
> >
> > Hello,
> >
> > We have daily rainfall observation data for Africa in binary
format
> > (0.1degree resolution). Is there a way of converting these data
into MET
> > compatible formats (grib1 or nc)?
> >
> > Best regards,
> >
> > Endalk
> >
>
>
>

--
-----------------------------------------------------------------------------------
Endalkachew Bekele
NOAA Climate Prediction Center / African Desk
5200 Auth Road, Rm 811, Camp Springs, MD 20746, USA
Tel: (301) 763 8000 Ext. 7014
e-mail: endalkachew.bekele at noaa.gov

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #55823] Use of regular binary data in MET
From: Paul Oldenburg
Time: Thu Apr 12 15:33:35 2012

Endalk,

I was not suggesting that you run pcp_combine on your binary files,
but only that you study the format of the NetCDF
output from the pcp_combine test script.  You will have to create a
tool to convert your binary files to NetCDF, but the
format of the NetCDF you generate should be the same as that created
by pcp_combine.  Does that make sense?

The trmm2nc.R script on the bottom of the MET website scripts page
that I mentioned below is an example of a tool that
converts arbitrary gridded text data (TRMM) into MET NetCDF.  If you
are proficient with R, you can modify this script
to read your binary data instead of the TRMM text data.  Then, the
output will be MET NetCDF that MET can read.

Paul

On 04/12/2012 03:23 PM, Endalkachew Bekele via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
>
> Hi Paul,
>
> I did check the shell scripts used to the run pcp_combine tool. The
> pcp_combine tool was applied to "grib" files not to plain "binary"
files in
> the test run. When I try to apply the pcp_combine tool to my binary
files,
> it looks for a grib file and when it doesn't find one, it gives me
the
> following error:
>
>   bin/pcp_combine \
> ? -sum 00000000_000000 24 20120411_000000 24 \
> ? temp\temp.nc \
> ? -pcpdir temp
> ERROR  :
> ERROR  : GribFile::skip_header() ->  can't find "GRIB" magic cookie
> ERROR  :
> WARNING:
> WARNING: check_file_time() ->  can't open precipitation grib file:
> temp/all_products.bin.20120411
> WARNING:
> ERROR  :
> ERROR  : sum_grib_files() ->  Cannot find a file with a valid time
of
> 20120411_000000 and accumulation time of 240000 matching the regular
> expression ".*"
> ERROR  :
>
> As an alternative, I tried to use the opengradss' "lats4d" tool to
convert
> my binary file into "nc" file. The conversion went well but, the
> pcp_combine tool looks for a "grib" file and it doesn't recognize
the new
> "nc" file.
>
> Kindly advise in this regard.
>
> Endalk
>
>
> On Thu, Apr 12, 2012 at 10:26 AM, Paul Oldenburg via RT
> <met_help at ucar.edu>wrote:
>
>> Endalk,
>>
>> The easiest way will be to convert this data, which sounds like it
is
>> gridded, into the format of NetCDF written by the
>> MET tool pcp_combine.  You can see an example of this format if you
use
>> the NetCDF utility ncdump on the output of the
>> pcp_combine test script in $MET_BASE/out/pcp_combine, where
$MET_BASE is
>> the MET installation folder on your system.
>> First, of course, you must have run the test scripts.
>>
>> In order for MET to read your data, you must determine the
projection
>> information for the data that you are converting
>> and put it into the NetCDF file in a way that MET will understand.
It
>> sounds like your data might be in a lat/long
>> projection.  There is an example script on the MET website, written
in the
>> R statistical language, that converts text
>> TRMM rainfall data into the MET format of NetCDF.  You can download
this
>> script at the bottom of the following page:
>>
>> http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
>>
>> I know this may sound confusing, but it can be done.  Please let me
know
>> if you have any questions.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 04/12/2012 07:16 AM, Endalkachew Bekele via RT wrote:
>>>
>>> Thu Apr 12 07:16:14 2012: Request 55823 was acted upon.
>>> Transaction: Ticket created by endalkachew.bekele at noaa.gov
>>>          Queue: met_help
>>>        Subject: Use of regular binary data in MET
>>>          Owner: Nobody
>>>     Requestors: endalkachew.bekele at noaa.gov
>>>         Status: new
>>>    Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
>>>
>>>
>>> Hello,
>>>
>>> We have daily rainfall observation data for Africa in binary
format
>>> (0.1degree resolution). Is there a way of converting these data
into MET
>>> compatible formats (grib1 or nc)?
>>>
>>> Best regards,
>>>
>>> Endalk
>>>
>>
>>
>>
>
>

------------------------------------------------
Subject: Use of regular binary data in MET
From: Endalkachew Bekele
Time: Sat Apr 14 20:14:58 2012

Paul,
I used the g2grb tool (
http://www.cpc.ncep.noaa.gov/products/wesley/g2grb.html) to convert my
grided data to to grib2 format, I then used to the STRC's gribmaster
tool
to convert it to grib1 format. I was able to run the grid_stat tool
successfully. Thanks again!
Endalk

On Thu, Apr 12, 2012 at 5:33 PM, Paul Oldenburg via RT
<met_help at ucar.edu>wrote:

> Endalk,
>
> I was not suggesting that you run pcp_combine on your binary files,
but
> only that you study the format of the NetCDF
> output from the pcp_combine test script.  You will have to create a
tool
> to convert your binary files to NetCDF, but the
> format of the NetCDF you generate should be the same as that created
by
> pcp_combine.  Does that make sense?
>
> The trmm2nc.R script on the bottom of the MET website scripts page
that I
> mentioned below is an example of a tool that
> converts arbitrary gridded text data (TRMM) into MET NetCDF.  If you
are
> proficient with R, you can modify this script
> to read your binary data instead of the TRMM text data.  Then, the
output
> will be MET NetCDF that MET can read.
>
> Paul
>
>
> On 04/12/2012 03:23 PM, Endalkachew Bekele via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823>
> >
> > Hi Paul,
> >
> > I did check the shell scripts used to the run pcp_combine tool.
The
> > pcp_combine tool was applied to "grib" files not to plain "binary"
files
> in
> > the test run. When I try to apply the pcp_combine tool to my
binary
> files,
> > it looks for a grib file and when it doesn't find one, it gives me
the
> > following error:
> >
> >   bin/pcp_combine \
> > ? -sum 00000000_000000 24 20120411_000000 24 \
> > ? temp\temp.nc \
> > ? -pcpdir temp
> > ERROR  :
> > ERROR  : GribFile::skip_header() ->  can't find "GRIB" magic
cookie
> > ERROR  :
> > WARNING:
> > WARNING: check_file_time() ->  can't open precipitation grib file:
> > temp/all_products.bin.20120411
> > WARNING:
> > ERROR  :
> > ERROR  : sum_grib_files() ->  Cannot find a file with a valid time
of
> > 20120411_000000 and accumulation time of 240000 matching the
regular
> > expression ".*"
> > ERROR  :
> >
> > As an alternative, I tried to use the opengradss' "lats4d" tool to
> convert
> > my binary file into "nc" file. The conversion went well but, the
> > pcp_combine tool looks for a "grib" file and it doesn't recognize
the new
> > "nc" file.
> >
> > Kindly advise in this regard.
> >
> > Endalk
> >
> >
> > On Thu, Apr 12, 2012 at 10:26 AM, Paul Oldenburg via RT
> > <met_help at ucar.edu>wrote:
> >
> >> Endalk,
> >>
> >> The easiest way will be to convert this data, which sounds like
it is
> >> gridded, into the format of NetCDF written by the
> >> MET tool pcp_combine.  You can see an example of this format if
you use
> >> the NetCDF utility ncdump on the output of the
> >> pcp_combine test script in $MET_BASE/out/pcp_combine, where
$MET_BASE is
> >> the MET installation folder on your system.
> >> First, of course, you must have run the test scripts.
> >>
> >> In order for MET to read your data, you must determine the
projection
> >> information for the data that you are converting
> >> and put it into the NetCDF file in a way that MET will
understand.  It
> >> sounds like your data might be in a lat/long
> >> projection.  There is an example script on the MET website,
written in
> the
> >> R statistical language, that converts text
> >> TRMM rainfall data into the MET format of NetCDF.  You can
download this
> >> script at the bottom of the following page:
> >>
> >> http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
> >>
> >> I know this may sound confusing, but it can be done.  Please let
me know
> >> if you have any questions.
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >>
> >> On 04/12/2012 07:16 AM, Endalkachew Bekele via RT wrote:
> >>>
> >>> Thu Apr 12 07:16:14 2012: Request 55823 was acted upon.
> >>> Transaction: Ticket created by endalkachew.bekele at noaa.gov
> >>>          Queue: met_help
> >>>        Subject: Use of regular binary data in MET
> >>>          Owner: Nobody
> >>>     Requestors: endalkachew.bekele at noaa.gov
> >>>         Status: new
> >>>    Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=55823
> >
> >>>
> >>>
> >>> Hello,
> >>>
> >>> We have daily rainfall observation data for Africa in binary
format
> >>> (0.1degree resolution). Is there a way of converting these data
into
> MET
> >>> compatible formats (grib1 or nc)?
> >>>
> >>> Best regards,
> >>>
> >>> Endalk
> >>>
> >>
> >>
> >>
> >
> >
>
>
>

--
-----------------------------------------------------------------------------------
Endalkachew Bekele
NOAA Climate Prediction Center / African Desk
5200 Auth Road, Rm 811, Camp Springs, MD 20746, USA
Tel: (301) 763 8000 Ext. 7014
e-mail: endalkachew.bekele at noaa.gov

------------------------------------------------