[Met_help] [rt.rap.ucar.edu #78596] History for observation precipitation format
Julie Prestopnik via RT
met_help at ucar.edu
Mon Dec 12 12:44:33 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Sir/Madam,
I am writing to ask about the observation precipitation used in pcp_combine.I have precipitation data in binary format associated with a ctl file and I have to transform it to a grib file.It works normally with 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is generated.
However,pcp_combine may not recongnize the file.grb.I check the info of file.grb with 'ncl_filedump file.grb' and find a warning message,warning:NclGRIB: Unrecognized parameter table (center 0, subcenter 0, table 255), defaulting to NCEP operational table: variable names and units may be incorrect,is produced.In addition,the variiable names are not the way I think.
Now my question is how to transfer the data.Actually,I have been struggling for several days and I really hope your help.
Any help is greatly appreciated thank you
Jiao B.F.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: observation precipitation format
From: Julie Prestopnik
Time: Mon Nov 07 10:10:22 2016
Hello. I understand that you have a ctl file that you would like to
convert to GRIB format so that MET can read it for pcp_combine. It
wasn't
clear if you tried using pcp_combine and had a problem or if you were
only
having problems converting the data to GRIB using "cdo". It sounds
like
you suspect there may have been a problem in converting your data from
"ctl" to GRIB.
If you experienced problems converting your "ctl" data format to GRIB
format using "cdo", please refer to the "How to Get Help or
Contribute"
section of the CDO website and contact them for support for "cdo":
https://code.zmaw.de/projects/cdo/wiki
If you experienced a problem with using pcp_combine, please copy and
paste
the error in a reply to this message along with the command you ran
and
please upload your data to our anonymous ftp site as follows so that I
can
take a look:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Thanks,
Julie
On Sun, Nov 6, 2016 at 1:47 AM, 焦宝峰 via RT <met_help at ucar.edu> wrote:
>
> Sun Nov 06 01:47:23 2016: Request 78596 was acted upon.
> Transaction: Ticket created by nuistjiao at 163.com
> Queue: met_help
> Subject: observation precipitation format
> Owner: Nobody
> Requestors: nuistjiao at 163.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>
>
> Dear Sir/Madam,
>
>
> I am writing to ask about the observation precipitation used in
> pcp_combine.I have precipitation data in binary format associated
with a
> ctl file and I have to transform it to a grib file.It works normally
with
> 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is
generated.
>
>
> However,pcp_combine may not recongnize the file.grb.I check the info
of
> file.grb with 'ncl_filedump file.grb' and find a warning
> message,warning:NclGRIB: Unrecognized parameter table (center 0,
subcenter
> 0, table 255), defaulting to NCEP operational table: variable names
and
> units may be incorrect,is produced.In addition,the variiable names
are not
> the way I think.
>
>
> Now my question is how to transfer the data.Actually,I have been
> struggling for several days and I really hope your help.
>
>
> Any help is greatly appreciated thank you
> Jiao B.F.
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: observation precipitation format
From: 焦宝峰
Time: Tue Nov 08 02:22:00 2016
Hi Julie,
Thank you for your reply. I have uploaded my data to your ftp server.
I’m sorry I didn't explain the problem clearly last time. I would like
to run grid-stat tool with WRF model forecasts and gridded
observations from CMA.I converted the original data to grib-format
using ‘cdo -f grb import_binary file.ctl file.grb’. Then, I tried to
run pcp_combine before running grid-stat tool to make accumulated
precipitation. However, pcp_combine might not recognize the file.grb
.My goal is to deal the binary data with pcp_combine. Intermediate
process is not important as well as intermediate file format.
The var crain in ctl means combined analysis precipitation in one hour
and it is used to calculate accumulated precipitation.
Thank you for all your assistance
Jiao B.F.
At 2016-11-08 01:10:23, "Julie Prestopnik via RT" <met_help at ucar.edu>
wrote:
>Hello. I understand that you have a ctl file that you would like to
>convert to GRIB format so that MET can read it for pcp_combine. It
wasn't
>clear if you tried using pcp_combine and had a problem or if you were
only
>having problems converting the data to GRIB using "cdo". It sounds
like
>you suspect there may have been a problem in converting your data
from
>"ctl" to GRIB.
>
>If you experienced problems converting your "ctl" data format to GRIB
>format using "cdo", please refer to the "How to Get Help or
Contribute"
>section of the CDO website and contact them for support for "cdo":
>
>https://code.zmaw.de/projects/cdo/wiki
>
>If you experienced a problem with using pcp_combine, please copy and
paste
>the error in a reply to this message along with the command you ran
and
>please upload your data to our anonymous ftp site as follows so that
I can
>take a look:
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
>
>Thanks,
>Julie
>
>
>On Sun, Nov 6, 2016 at 1:47 AM, 焦宝峰 via RT <met_help at ucar.edu> wrote:
>
>>
>> Sun Nov 06 01:47:23 2016: Request 78596 was acted upon.
>> Transaction: Ticket created by nuistjiao at 163.com
>> Queue: met_help
>> Subject: observation precipitation format
>> Owner: Nobody
>> Requestors: nuistjiao at 163.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>>
>>
>> Dear Sir/Madam,
>>
>>
>> I am writing to ask about the observation precipitation used in
>> pcp_combine.I have precipitation data in binary format associated
with a
>> ctl file and I have to transform it to a grib file.It works
normally with
>> 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is
generated.
>>
>>
>> However,pcp_combine may not recongnize the file.grb.I check the
info of
>> file.grb with 'ncl_filedump file.grb' and find a warning
>> message,warning:NclGRIB: Unrecognized parameter table (center 0,
subcenter
>> 0, table 255), defaulting to NCEP operational table: variable names
and
>> units may be incorrect,is produced.In addition,the variiable names
are not
>> the way I think.
>>
>>
>> Now my question is how to transfer the data.Actually,I have been
>> struggling for several days and I really hope your help.
>>
>>
>> Any help is greatly appreciated thank you
>> Jiao B.F.
>>
>>
>>
>>
>
>
>--
>Julie Prestopnik
>National Center for Atmospheric Research
>Research Applications Laboratory
>Phone: 303.497.8399
>Email: jpresto at ucar.edu
>
------------------------------------------------
Subject: observation precipitation format
From: Julie Prestopnik
Time: Wed Nov 09 13:57:38 2016
Thank you for clarifying what you would like to do with the data and
for
uploading your data to the ftp server.
I grabbed the data and converted a file to GRIB using the following
command:
*cdo -f grb import_binary file.ctl 2014080100.grb*
To see if MET could read the GRIB file, I tried running
plot_data_plane,
but was unsuccessful.
DEBUG 1: Opening data file: 2014080100.grb
WARNING:
WARNING: MetGrib1DataFile::data_plane() -> No exact match found for
VarInfo
"SALTY/L0" in GRIB file "2014080100.grb".
WARNING:
ERROR :
ERROR : plot_data_plane -> trouble getting field "name="SALTY";
level="L0"; lead_time="06";" from file "2014080100.grb"
ERROR :
The GRIB file needs a few adjustments before MET can read it.
In researching online how to modify the GRIB file using cdo, I came
across
this CDO reference card which you may find useful:
http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/sorc/rtofs_cdo-
1.4.0.1.fd/cdo-1.5.0/doc/cdo_refcard.pdf
Running *cdo sinfo 2014080100.grb*, I can see that the first variable
has
a Parameter ID of 1 and the second variable has a Parameter ID of 2.
File format : GRIB
-1 : Institut Source Ttype Levels Num Points Num Dtype :
Parameter ID
1 : unknown unknown instant 1 1 308000 1 P16 : 1
2 : unknown unknown instant 1 1 308000 1 P16 : 2
We can change these with the CDO operator *chparam*.
I ran:
*cdo chparam,1,88.2,2,150.2 2014080100.grb 2014080100.new.grb*
based on the values from ON388 - Table 2:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html
where 88 is the code number for table 2 for variable 1 and 150 is the
code
number for table 2 for variable 2. Hopefully, this is correct for
your
data.
Unfortunately, plot_data_plane still did not work so MET can still not
read
this data.
Using the wgrib command:
*wgrib 2014080100.new.grb*
I can see for the first and second records:
1:0:d=14080100:SALTY:kpds5=88:kpds6=99:kpds7=1:TR=0:P1=0:P2=0:TimeU=1::anl:NAve=0
2:163336:d=14080100:COVMZ:kpds5=150:kpds6=99:kpds7=1:TR=0:P1=0:P2=0:TimeU=1::anl:NAve=0
...
that kpds6=99 and kpds7=1
according to:
http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib/readme
kpds6 and kdps7 are the Type and Value of Level (Octets 10, 11 -12
respectively)
"KPDS5, KPDS6, KDPS7 (PDS Octets 9, 10, 11-12)"
>From here:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html
I see that for kdps6 (Octet 10) that values 0-99 are special values
and
that where kdps6 has a value of 0-99 the kpds7 (Octets 11 - 12) value
should be 0, and get referred to Table 3a for the kdps6 values:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html#TABLE3A
Assuming that your variables are "Ground or water surface" I changed
the
level type to 1 and the level to 0 using the following commands:
*cdo setltype,1 2014080100.new.grb 2014080100.new2.grbcdo setlevel,0
2014080100.new2.grb 2014080100.new3.grb*
After these changes we can see the values changed by running wgrib
again on
the new file:
*wgrib *
*2014080100.new3.grb*
1:0:d=14080100:SALTY:kpds5=88:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
2:163336:d=14080100:COVMZ:kpds5=150:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
...
I do not know that these values are correct for your data, so please
verify.
After making these changes, I am able to run plot_data_plane
successfully:
plot_data_plane 2014080100.new3.grb salty_06hr.ps 'name="SALTY";
level="L0"; lead_time="06";'
MET should know be able to read the output GRIB file.
Please try executing the following commands and let us know if you're
able
to successfully produce output from plot_data_plane indicating that
MET can
read your GRIB file successfully:
*cdo -f grb import_binary file.ctl 2014080100.grbcdo
chparam,1,88.2,2,150.2
2014080100.grb 2014080100.new.grbcdo setltype,1 2014080100.new.grb
2014080100.new2.grbcdo setlevel,0 2014080100.new2.grb
2014080100.new3.grbplot_data_plane 2014080100.new3.grb salty_06hr.ps
<http://salty_06hr.ps> 'name="SALTY"; level="L0"; lead_time="06";'*
I have attached the output file (converted to png format) for you to
view.
I hope this helps.
Please let us know how it goes.
Thanks,
Julie Prestopnik
On Tue, Nov 8, 2016 at 2:22 AM, 焦宝峰 via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>
> Hi Julie,
>
> Thank you for your reply. I have uploaded my data to your ftp
server.
>
> I’m sorry I didn't explain the problem clearly last time. I would
like to
> run grid-stat tool with WRF model forecasts and gridded observations
from
> CMA.I converted the original data to grib-format using ‘cdo -f grb
> import_binary file.ctl file.grb’. Then, I tried to run pcp_combine
before
> running grid-stat tool to make accumulated precipitation. However,
> pcp_combine might not recognize the file.grb .My goal is to deal the
binary
> data with pcp_combine. Intermediate process is not important as well
as
> intermediate file format.
>
> The var crain in ctl means combined analysis precipitation in one
hour and
> it is used to calculate accumulated precipitation.
>
> Thank you for all your assistance
>
> Jiao B.F.
>
> At 2016-11-08 01:10:23, "Julie Prestopnik via RT"
<met_help at ucar.edu>
> wrote:
> >Hello. I understand that you have a ctl file that you would like
to
> >convert to GRIB format so that MET can read it for pcp_combine. It
wasn't
> >clear if you tried using pcp_combine and had a problem or if you
were only
> >having problems converting the data to GRIB using "cdo". It sounds
like
> >you suspect there may have been a problem in converting your data
from
> >"ctl" to GRIB.
> >
> >If you experienced problems converting your "ctl" data format to
GRIB
> >format using "cdo", please refer to the "How to Get Help or
Contribute"
> >section of the CDO website and contact them for support for "cdo":
> >
> >https://code.zmaw.de/projects/cdo/wiki
> >
> >If you experienced a problem with using pcp_combine, please copy
and paste
> >the error in a reply to this message along with the command you ran
and
> >please upload your data to our anonymous ftp site as follows so
that I can
> >take a look:
> >
> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> >
> >Thanks,
> >Julie
> >
> >
> >On Sun, Nov 6, 2016 at 1:47 AM, 焦宝峰 via RT <met_help at ucar.edu>
wrote:
> >
> >>
> >> Sun Nov 06 01:47:23 2016: Request 78596 was acted upon.
> >> Transaction: Ticket created by nuistjiao at 163.com
> >> Queue: met_help
> >> Subject: observation precipitation format
> >> Owner: Nobody
> >> Requestors: nuistjiao at 163.com
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
> >>
> >>
> >> Dear Sir/Madam,
> >>
> >>
> >> I am writing to ask about the observation precipitation used in
> >> pcp_combine.I have precipitation data in binary format associated
with a
> >> ctl file and I have to transform it to a grib file.It works
normally
> with
> >> 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is
> generated.
> >>
> >>
> >> However,pcp_combine may not recongnize the file.grb.I check the
info of
> >> file.grb with 'ncl_filedump file.grb' and find a warning
> >> message,warning:NclGRIB: Unrecognized parameter table (center 0,
> subcenter
> >> 0, table 255), defaulting to NCEP operational table: variable
names and
> >> units may be incorrect,is produced.In addition,the variiable
names are
> not
> >> the way I think.
> >>
> >>
> >> Now my question is how to transfer the data.Actually,I have been
> >> struggling for several days and I really hope your help.
> >>
> >>
> >> Any help is greatly appreciated thank you
> >> Jiao B.F.
> >>
> >>
> >>
> >>
> >
> >
> >--
> >Julie Prestopnik
> >National Center for Atmospheric Research
> >Research Applications Laboratory
> >Phone: 303.497.8399
> >Email: jpresto at ucar.edu
> >
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: observation precipitation format
From: 焦宝峰
Time: Thu Nov 10 01:56:57 2016
Dear Julie
Thank you very much for your quik reply.Following your steps,I got the
same result.The first parameter ID was changed to 61 and the second
parameter ID was changed to 62.A new problem occurred when I ran:
‘pcp_combine -sum 00000000_000000 1 20140801_060000 3 sample.nc ’
ERROR :
ERROR : sum_data_files() -> Cannot find a file with a valid time of
20140801_060000 and accumulation time of 010000 matching the regular
expression ".*"
ERROR :
Then,I compared the information of “2014080100.new3.grb” and
“ST2ml2005080700.Grb_G212” located in $met_dir/sample_obs/ST2ml with
wgrib.
wgrib 2014080100.new3.grb
1:0:d=14080100:APCP:kpds5=61:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
2:163336:d=14080100:NCPCP:kpds5=62:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
...
wgrib ST2ml2005080700.Grb_G212
1:0:d=05080623:APCP:kpds5=61:kpds6=1:kpds7=0:TR=4:P1=0:P2=1:TimeU=1:sfc:0-
1hr acc:NAve=0
I guess the ‘TR’ and ‘p2’ in 2014080100.new3.grb should be changed.I
am poor at nco.Could you help me with this problem?
***********************************an other
problem***********************************************
Recently,I happened to see a Web page
,http://www.dtcenter.org/met/users/downloads/observation_data.php, in
which a script named trmmbin2nc.R was used to reformat binary data
into a NetCDF format that MET can read.I am not familiar with R,so I
can just understand the general meaning.Based on this,I wrote a NCL
script to trans my binary data to NC format.Then,I ran :
pcp_combine -add \
2014080102.grd.nc 'name="APCP_01";level="(*,*)";’ \
2014080103.grd.nc 'name="APCP_01";level="(*,*)";’ \
pcp_2h.nc
It worked successfully,but there is a problem.The _FillValue of APCP
in 2014080102.grd.nc is -999 while the _FillValue of APCP in pcp_2h.nc
is -9999.I have uploaded the test_jiao.ncl.
Thank you very much
Jiao B.F.
At 2016-11-10 04:57:38, "Julie Prestopnik via RT" <met_help at ucar.edu>
wrote:
>Thank you for clarifying what you would like to do with the data and
for
>uploading your data to the ftp server.
>
>I grabbed the data and converted a file to GRIB using the following
command:
>
>*cdo -f grb import_binary file.ctl 2014080100.grb*
>
>To see if MET could read the GRIB file, I tried running
plot_data_plane,
>but was unsuccessful.
>
>DEBUG 1: Opening data file: 2014080100.grb
>WARNING:
>WARNING: MetGrib1DataFile::data_plane() -> No exact match found for
VarInfo
>"SALTY/L0" in GRIB file "2014080100.grb".
>WARNING:
>ERROR :
>ERROR : plot_data_plane -> trouble getting field "name="SALTY";
>level="L0"; lead_time="06";" from file "2014080100.grb"
>ERROR :
>
>The GRIB file needs a few adjustments before MET can read it.
>
>In researching online how to modify the GRIB file using cdo, I came
across
>this CDO reference card which you may find useful:
>http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/sorc/rtofs_cdo-
1.4.0.1.fd/cdo-1.5.0/doc/cdo_refcard.pdf
>
>Running *cdo sinfo 2014080100.grb*, I can see that the first
variable has
>a Parameter ID of 1 and the second variable has a Parameter ID of 2.
>
>File format : GRIB
> -1 : Institut Source Ttype Levels Num Points Num Dtype :
>Parameter ID
> 1 : unknown unknown instant 1 1 308000 1 P16 :
1
> 2 : unknown unknown instant 1 1 308000 1 P16 :
2
>
>We can change these with the CDO operator *chparam*.
>
>I ran:
>
>*cdo chparam,1,88.2,2,150.2 2014080100.grb 2014080100.new.grb*
>
>based on the values from ON388 - Table 2:
>http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html
>
>where 88 is the code number for table 2 for variable 1 and 150 is the
code
>number for table 2 for variable 2. Hopefully, this is correct for
your
>data.
>
>Unfortunately, plot_data_plane still did not work so MET can still
not read
>this data.
>
>Using the wgrib command:
>
>*wgrib 2014080100.new.grb*
>
>I can see for the first and second records:
>
>1:0:d=14080100:SALTY:kpds5=88:kpds6=99:kpds7=1:TR=0:P1=0:P2=0:TimeU=1::anl:NAve=0
>2:163336:d=14080100:COVMZ:kpds5=150:kpds6=99:kpds7=1:TR=0:P1=0:P2=0:TimeU=1::anl:NAve=0
>...
>
>that kpds6=99 and kpds7=1
>
>according to:
>
>http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib/readme
>
>kpds6 and kdps7 are the Type and Value of Level (Octets 10, 11 -12
>respectively)
>
>"KPDS5, KPDS6, KDPS7 (PDS Octets 9, 10, 11-12)"
>
>From here:
>http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html
>
>I see that for kdps6 (Octet 10) that values 0-99 are special values
and
>that where kdps6 has a value of 0-99 the kpds7 (Octets 11 - 12) value
>should be 0, and get referred to Table 3a for the kdps6 values:
>http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html#TABLE3A
>
>Assuming that your variables are "Ground or water surface" I changed
the
>level type to 1 and the level to 0 using the following commands:
>
>
>
>*cdo setltype,1 2014080100.new.grb 2014080100.new2.grbcdo setlevel,0
>2014080100.new2.grb 2014080100.new3.grb*
>
>After these changes we can see the values changed by running wgrib
again on
>the new file:
>
>*wgrib *
>
>*2014080100.new3.grb*
>1:0:d=14080100:SALTY:kpds5=88:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
>2:163336:d=14080100:COVMZ:kpds5=150:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0
>...
>
>I do not know that these values are correct for your data, so please
verify.
>
>After making these changes, I am able to run plot_data_plane
successfully:
>
>plot_data_plane 2014080100.new3.grb salty_06hr.ps 'name="SALTY";
>level="L0"; lead_time="06";'
>
>MET should know be able to read the output GRIB file.
>
>Please try executing the following commands and let us know if you're
able
>to successfully produce output from plot_data_plane indicating that
MET can
>read your GRIB file successfully:
>
>
>
>
>
>*cdo -f grb import_binary file.ctl 2014080100.grbcdo
chparam,1,88.2,2,150.2
>2014080100.grb 2014080100.new.grbcdo setltype,1 2014080100.new.grb
>2014080100.new2.grbcdo setlevel,0 2014080100.new2.grb
>2014080100.new3.grbplot_data_plane 2014080100.new3.grb salty_06hr.ps
><http://salty_06hr.ps> 'name="SALTY"; level="L0"; lead_time="06";'*
>
>I have attached the output file (converted to png format) for you to
view.
>I hope this helps.
>
>Please let us know how it goes.
>
>Thanks,
>Julie Prestopnik
>
>
>On Tue, Nov 8, 2016 at 2:22 AM, 焦宝峰 via RT <met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>>
>> Hi Julie,
>>
>> Thank you for your reply. I have uploaded my data to your ftp
server.
>>
>> I’m sorry I didn't explain the problem clearly last time. I would
like to
>> run grid-stat tool with WRF model forecasts and gridded
observations from
>> CMA.I converted the original data to grib-format using ‘cdo -f grb
>> import_binary file.ctl file.grb’. Then, I tried to run pcp_combine
before
>> running grid-stat tool to make accumulated precipitation. However,
>> pcp_combine might not recognize the file.grb .My goal is to deal
the binary
>> data with pcp_combine. Intermediate process is not important as
well as
>> intermediate file format.
>>
>> The var crain in ctl means combined analysis precipitation in one
hour and
>> it is used to calculate accumulated precipitation.
>>
>> Thank you for all your assistance
>>
>> Jiao B.F.
>>
>> At 2016-11-08 01:10:23, "Julie Prestopnik via RT"
<met_help at ucar.edu>
>> wrote:
>> >Hello. I understand that you have a ctl file that you would like
to
>> >convert to GRIB format so that MET can read it for pcp_combine.
It wasn't
>> >clear if you tried using pcp_combine and had a problem or if you
were only
>> >having problems converting the data to GRIB using "cdo". It
sounds like
>> >you suspect there may have been a problem in converting your data
from
>> >"ctl" to GRIB.
>> >
>> >If you experienced problems converting your "ctl" data format to
GRIB
>> >format using "cdo", please refer to the "How to Get Help or
Contribute"
>> >section of the CDO website and contact them for support for "cdo":
>> >
>> >https://code.zmaw.de/projects/cdo/wiki
>> >
>> >If you experienced a problem with using pcp_combine, please copy
and paste
>> >the error in a reply to this message along with the command you
ran and
>> >please upload your data to our anonymous ftp site as follows so
that I can
>> >take a look:
>> >
>> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
>> >
>> >
>> >Thanks,
>> >Julie
>> >
>> >
>> >On Sun, Nov 6, 2016 at 1:47 AM, 焦宝峰 via RT <met_help at ucar.edu>
wrote:
>> >
>> >>
>> >> Sun Nov 06 01:47:23 2016: Request 78596 was acted upon.
>> >> Transaction: Ticket created by nuistjiao at 163.com
>> >> Queue: met_help
>> >> Subject: observation precipitation format
>> >> Owner: Nobody
>> >> Requestors: nuistjiao at 163.com
>> >> Status: new
>> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>> >>
>> >>
>> >> Dear Sir/Madam,
>> >>
>> >>
>> >> I am writing to ask about the observation precipitation used in
>> >> pcp_combine.I have precipitation data in binary format
associated with a
>> >> ctl file and I have to transform it to a grib file.It works
normally
>> with
>> >> 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is
>> generated.
>> >>
>> >>
>> >> However,pcp_combine may not recongnize the file.grb.I check the
info of
>> >> file.grb with 'ncl_filedump file.grb' and find a warning
>> >> message,warning:NclGRIB: Unrecognized parameter table (center 0,
>> subcenter
>> >> 0, table 255), defaulting to NCEP operational table: variable
names and
>> >> units may be incorrect,is produced.In addition,the variiable
names are
>> not
>> >> the way I think.
>> >>
>> >>
>> >> Now my question is how to transfer the data.Actually,I have been
>> >> struggling for several days and I really hope your help.
>> >>
>> >>
>> >> Any help is greatly appreciated thank you
>> >> Jiao B.F.
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >--
>> >Julie Prestopnik
>> >National Center for Atmospheric Research
>> >Research Applications Laboratory
>> >Phone: 303.497.8399
>> >Email: jpresto at ucar.edu
>> >
>>
>>
>
>
>--
>Julie Prestopnik
>National Center for Atmospheric Research
>Research Applications Laboratory
>Phone: 303.497.8399
>Email: jpresto at ucar.edu
>
------------------------------------------------
Subject: observation precipitation format
From: Julie Prestopnik
Time: Thu Nov 10 14:56:31 2016
It sounds like you have two routes to take now for getting the data in
a
format that MET can read: the GRIB format and the NetCDF format.
GRIB format
If you'd like to pursue the GRIB format, you'll need to change the
time
range indicator "TR" to 4 for Accumulation:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html
and change P1 and P2 to the begin and end of the accumulation period,
respectively:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/section1.html
You can use the CDO reference card:
http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/sorc/rtofs_cdo
-1.4.0.1.fd/cdo-1.5.0/doc/cdo_refcard.pdf
or contact CDO for support in getting those values changed.
https://code.zmaw.de/projects/cdo/wiki
NetCDF format
It sounds like the NetCDF formatting is going well. MET reads the
fill
value attribute in the NetCDF file and uses that value, so you don't
have
to worry about the switch in missing values. You can always run
plot_data_plane to see if you think the data looks correct.
I hope that helps.
Julie
On Thu, Nov 10, 2016 at 1:56 AM, 焦宝峰 via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
>
> Dear Julie
>
> Thank you very much for your quik reply.Following your steps,I got
the
> same result.The first parameter ID was changed to 61 and the second
> parameter ID was changed to 62.A new problem occurred when I ran:
>
> ‘pcp_combine -sum 00000000_000000 1 20140801_060000 3 sample.nc ’
>
> ERROR :
>
> ERROR : sum_data_files() -> Cannot find a file with a valid time of
> 20140801_060000 and accumulation time of 010000 matching the regular
> expression ".*"
>
> ERROR :
>
> Then,I compared the information of “2014080100.new3.grb” and
> “ST2ml2005080700.Grb_G212” located in $met_dir/sample_obs/ST2ml
with wgrib.
>
> wgrib 2014080100.new3.grb
>
> 1:0:d=14080100:APCP:kpds5=61:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:
> TimeU=1:sfc:anl:NAve=0
>
> 2:163336:d=14080100:NCPCP:kpds5=62:kpds6=1:kpds7=0:TR=0:P1=0
> :P2=0:TimeU=1:sfc:anl:NAve=0
>
> ...
>
> wgrib ST2ml2005080700.Grb_G212
>
>
1:0:d=05080623:APCP:kpds5=61:kpds6=1:kpds7=0:TR=4:P1=0:P2=1:TimeU=1:sfc:0-
1hr
> acc:NAve=0
>
> I guess the ‘TR’ and ‘p2’ in 2014080100.new3.grb should be changed.I
am
> poor at nco.Could you help me with this problem?
> ***********************************an other
problem***********************
> ************************
>
> Recently,I happened to see a Web page ,http://www.dtcenter.org/met/u
> sers/downloads/observation_data.php, in which a script named
> trmmbin2nc.R was used to reformat binary data into a NetCDF format
that MET
> can read.I am not familiar with R,so I can just understand the
general
> meaning.Based on this,I wrote a NCL script to trans my binary data
to NC
> format.Then,I ran :
>
> pcp_combine -add \
> 2014080102.grd.nc 'name="APCP_01";level="(*,*)";’ \
> 2014080103.grd.nc 'name="APCP_01";level="(*,*)";’ \
> pcp_2h.nc
>
>
> It worked successfully,but there is a problem.The _FillValue of APCP
in
> 2014080102.grd.nc is -999 while the _FillValue of APCP in pcp_2h.nc
is
> -9999.I have uploaded the test_jiao.ncl.
>
>
> Thank you very much
> Jiao B.F.
>
> At 2016-11-10 04:57:38, "Julie Prestopnik via RT"
<met_help at ucar.edu>
> wrote:
> >Thank you for clarifying what you would like to do with the data
and for
> >uploading your data to the ftp server.
> >
> >I grabbed the data and converted a file to GRIB using the following
> command:
> >
> >*cdo -f grb import_binary file.ctl 2014080100.grb*
> >
> >To see if MET could read the GRIB file, I tried running
plot_data_plane,
> >but was unsuccessful.
> >
> >DEBUG 1: Opening data file: 2014080100.grb
> >WARNING:
> >WARNING: MetGrib1DataFile::data_plane() -> No exact match found for
> VarInfo
> >"SALTY/L0" in GRIB file "2014080100.grb".
> >WARNING:
> >ERROR :
> >ERROR : plot_data_plane -> trouble getting field "name="SALTY";
> >level="L0"; lead_time="06";" from file "2014080100.grb"
> >ERROR :
> >
> >The GRIB file needs a few adjustments before MET can read it.
> >
> >In researching online how to modify the GRIB file using cdo, I came
across
> >this CDO reference card which you may find useful:
> >http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/sorc/rtofs_cd
> o-1.4.0.1.fd/cdo-1.5.0/doc/cdo_refcard.pdf
> >
> >Running *cdo sinfo 2014080100.grb*, I can see that the first
variable
> has
> >a Parameter ID of 1 and the second variable has a Parameter ID of
2.
> >
> >File format : GRIB
> > -1 : Institut Source Ttype Levels Num Points Num Dtype
:
> >Parameter ID
> > 1 : unknown unknown instant 1 1 308000 1 P16
: 1
> > 2 : unknown unknown instant 1 1 308000 1 P16
: 2
> >
> >We can change these with the CDO operator *chparam*.
> >
> >I ran:
> >
> >*cdo chparam,1,88.2,2,150.2 2014080100.grb 2014080100.new.grb*
> >
> >based on the values from ON388 - Table 2:
> >http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html
> >
> >where 88 is the code number for table 2 for variable 1 and 150 is
the code
> >number for table 2 for variable 2. Hopefully, this is correct for
your
> >data.
> >
> >Unfortunately, plot_data_plane still did not work so MET can still
not
> read
> >this data.
> >
> >Using the wgrib command:
> >
> >*wgrib 2014080100.new.grb*
> >
> >I can see for the first and second records:
> >
> >1:0:d=14080100:SALTY:kpds5=88:kpds6=99:kpds7=1:TR=0:P1=0:P2
> =0:TimeU=1::anl:NAve=0
> >2:163336:d=14080100:COVMZ:kpds5=150:kpds6=99:kpds7=1:TR=0:P
> 1=0:P2=0:TimeU=1::anl:NAve=0
> >...
> >
> >that kpds6=99 and kpds7=1
> >
> >according to:
> >
> >http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib/readme
> >
> >kpds6 and kdps7 are the Type and Value of Level (Octets 10, 11 -12
> >respectively)
> >
> >"KPDS5, KPDS6, KDPS7 (PDS Octets 9, 10, 11-12)"
> >
> >From here:
> >http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html
> >
> >I see that for kdps6 (Octet 10) that values 0-99 are special values
and
> >that where kdps6 has a value of 0-99 the kpds7 (Octets 11 - 12)
value
> >should be 0, and get referred to Table 3a for the kdps6 values:
> >http://www.nco.ncep.noaa.gov/pmb/docs/on388/table3.html#TABLE3A
> >
> >Assuming that your variables are "Ground or water surface" I
changed the
> >level type to 1 and the level to 0 using the following commands:
> >
> >
> >
> >*cdo setltype,1 2014080100.new.grb 2014080100.new2.grbcdo
setlevel,0
> >2014080100.new2.grb 2014080100.new3.grb*
> >
> >After these changes we can see the values changed by running wgrib
again
> on
> >the new file:
> >
> >*wgrib *
> >
> >*2014080100.new3.grb*
> >1:0:d=14080100:SALTY:kpds5=88:kpds6=1:kpds7=0:TR=0:P1=0:P2=
> 0:TimeU=1:sfc:anl:NAve=0
> >2:163336:d=14080100:COVMZ:kpds5=150:kpds6=1:kpds7=0:TR=0:P1
> =0:P2=0:TimeU=1:sfc:anl:NAve=0
> >...
> >
> >I do not know that these values are correct for your data, so
please
> verify.
> >
> >After making these changes, I am able to run plot_data_plane
successfully:
> >
> >plot_data_plane 2014080100.new3.grb salty_06hr.ps 'name="SALTY";
> >level="L0"; lead_time="06";'
> >
> >MET should know be able to read the output GRIB file.
> >
> >Please try executing the following commands and let us know if
you're able
> >to successfully produce output from plot_data_plane indicating that
MET
> can
> >read your GRIB file successfully:
> >
> >
> >
> >
> >
> >*cdo -f grb import_binary file.ctl 2014080100.grbcdo
> chparam,1,88.2,2,150.2
> >2014080100.grb 2014080100.new.grbcdo setltype,1 2014080100.new.grb
> >2014080100.new2.grbcdo setlevel,0 2014080100.new2.grb
> >2014080100.new3.grbplot_data_plane 2014080100.new3.grb
salty_06hr.ps
> ><http://salty_06hr.ps> 'name="SALTY"; level="L0"; lead_time="06";'*
> >
> >I have attached the output file (converted to png format) for you
to view.
> >I hope this helps.
> >
> >Please let us know how it goes.
> >
> >Thanks,
> >Julie Prestopnik
> >
> >
> >On Tue, Nov 8, 2016 at 2:22 AM, 焦宝峰 via RT <met_help at ucar.edu>
wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78596 >
> >>
> >> Hi Julie,
> >>
> >> Thank you for your reply. I have uploaded my data to your ftp
server.
> >>
> >> I’m sorry I didn't explain the problem clearly last time. I would
like
> to
> >> run grid-stat tool with WRF model forecasts and gridded
observations
> from
> >> CMA.I converted the original data to grib-format using ‘cdo -f
grb
> >> import_binary file.ctl file.grb’. Then, I tried to run
pcp_combine
> before
> >> running grid-stat tool to make accumulated precipitation.
However,
> >> pcp_combine might not recognize the file.grb .My goal is to deal
the
> binary
> >> data with pcp_combine. Intermediate process is not important as
well as
> >> intermediate file format.
> >>
> >> The var crain in ctl means combined analysis precipitation in one
hour
> and
> >> it is used to calculate accumulated precipitation.
> >>
> >> Thank you for all your assistance
> >>
> >> Jiao B.F.
> >>
> >> At 2016-11-08 01:10:23, "Julie Prestopnik via RT"
<met_help at ucar.edu>
> >> wrote:
> >> >Hello. I understand that you have a ctl file that you would
like to
> >> >convert to GRIB format so that MET can read it for pcp_combine.
It
> wasn't
> >> >clear if you tried using pcp_combine and had a problem or if you
were
> only
> >> >having problems converting the data to GRIB using "cdo". It
sounds
> like
> >> >you suspect there may have been a problem in converting your
data from
> >> >"ctl" to GRIB.
> >> >
> >> >If you experienced problems converting your "ctl" data format to
GRIB
> >> >format using "cdo", please refer to the "How to Get Help or
Contribute"
> >> >section of the CDO website and contact them for support for
"cdo":
> >> >
> >> >https://code.zmaw.de/projects/cdo/wiki
> >> >
> >> >If you experienced a problem with using pcp_combine, please copy
and
> paste
> >> >the error in a reply to this message along with the command you
ran and
> >> >please upload your data to our anonymous ftp site as follows so
that I
> can
> >> >take a look:
> >> >
> >> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >> >
> >> >
> >> >Thanks,
> >> >Julie
> >> >
> >> >
> >> >On Sun, Nov 6, 2016 at 1:47 AM, 焦宝峰 via RT <met_help at ucar.edu>
wrote:
> >> >
> >> >>
> >> >> Sun Nov 06 01:47:23 2016: Request 78596 was acted upon.
> >> >> Transaction: Ticket created by nuistjiao at 163.com
> >> >> Queue: met_help
> >> >> Subject: observation precipitation format
> >> >> Owner: Nobody
> >> >> Requestors: nuistjiao at 163.com
> >> >> Status: new
> >> >> Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> ket/Display.html?id=78596 >
> >> >>
> >> >>
> >> >> Dear Sir/Madam,
> >> >>
> >> >>
> >> >> I am writing to ask about the observation precipitation used
in
> >> >> pcp_combine.I have precipitation data in binary format
associated
> with a
> >> >> ctl file and I have to transform it to a grib file.It works
normally
> >> with
> >> >> 'cdo -f grb import_binary file.ctl file.grb' and a file.grb is
> >> generated.
> >> >>
> >> >>
> >> >> However,pcp_combine may not recongnize the file.grb.I check
the info
> of
> >> >> file.grb with 'ncl_filedump file.grb' and find a warning
> >> >> message,warning:NclGRIB: Unrecognized parameter table (center
0,
> >> subcenter
> >> >> 0, table 255), defaulting to NCEP operational table: variable
names
> and
> >> >> units may be incorrect,is produced.In addition,the variiable
names
> are
> >> not
> >> >> the way I think.
> >> >>
> >> >>
> >> >> Now my question is how to transfer the data.Actually,I have
been
> >> >> struggling for several days and I really hope your help.
> >> >>
> >> >>
> >> >> Any help is greatly appreciated thank you
> >> >> Jiao B.F.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> >--
> >> >Julie Prestopnik
> >> >National Center for Atmospheric Research
> >> >Research Applications Laboratory
> >> >Phone: 303.497.8399
> >> >Email: jpresto at ucar.edu
> >> >
> >>
> >>
> >
> >
> >--
> >Julie Prestopnik
> >National Center for Atmospheric Research
> >Research Applications Laboratory
> >Phone: 303.497.8399
> >Email: jpresto at ucar.edu
> >
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
More information about the Met_help
mailing list