[Met_help] [rt.rap.ucar.edu #58172] History for METv4.0 problems
John Halley Gotway via RT
met_help at ucar.edu
Tue Sep 25 08:07:10 MDT 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear John,
I am now in new troubles with MET and would like to ask your help after having changed for the new v4.0 package. Three small directories are attached to better understand my problems.
A. RUS_DEMO_METv3.1 directory.
The grib1 increment rounding was the previous trouble with METv3.1 though everything else went normally, including ‘gen_poly_mask’, ‘plot_point_obs’ and ‘point_stat’ steps.
Some results are given in RUS_DEMO_METv3.1 with the corresponding run-script.
Station positioning (very tiny red points though) near Sochi region in MET_01_TA_2012-08-01.pdf and numbers in ‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory showed that we are on the right track.
B. RUS_DEMO_METv4.0 directory.
The grib1 lat-lon increments were unacceptable for 1x1-km model, and we decided to reformat all files to grib2 with the ECMWF grib_convert procedure. Some additional testing showed that this procedure gave identical results on both data formats of our COSMO model.
But all METv4.0 steps were discouraging as given in RUS_DEMO_METv4.0.
1. No lat-lon INNER boarders were processed correctly by the ‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly) collected points inside the region as shown in ‘MET_01_TA_2012-08-01.plot_point.log’.
2. That something goes wrong is seen in the coastline drawn by ‘plot_point_obs’. It seems as if lats and lons have been mixed up somewhere (MET_01_TA_2012-08-01.pdf).
3. Together with the grid specification trouble I cannot understand the forecast lead time definition in ‘PointStatConfig’, which was no problem in v3.1 with hourly forecast fields gathered in separate variable-files.
C. USA_DEMO_METv3.1-4.0 directory.
Since several attempts to realize the error source failed, I decided to test the presentation WRF data in the METv4.0 data storage.
I took one model data file, reformatted it to grib2 and ran two parallel calculations with METv4.0/point_stat tool. Table statistics in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’ refer to grib2. THEY ARE DIFFERENT, so I realized that I missed something very important from the very beginning.
It is worth noting that all formatted RUS and US data are diagnosed with wgrib and wgrib2 procedures and included into corresponding directories. Data files in directories attached are given as links.
I would be very much obliged if you could explain my faults and misunderstanding in METv4.0 application for the Sochi region.
Thank you in advance.
Best regards,
Anatoly
........................................
Dr. Anatoly Muravyev
THE HYDROMETEOROLOGICAL RESEARCH CENTRE
OF THE RUSSIAN FEDERATION
11-13, Bolshoi Predtechensky Per.
123242, Moscow, Russia
Phone: + 7-499-7952127
Fax: + 7-499-2551582
E-mail: muravev at mecom.ru
........................................
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: METv4.0 problems
From: John Halley Gotway
Time: Mon Sep 10 08:43:33 2012
Anatoly,
Very good questions indeed. After reading your email, I stepped
through the same process you went through, using the MET test data to
see if the results differed for GRIB1 vs GRIB2. However, the big
difference is that I used the "cnvgrib" utility to do the conversion
from GRIB1 to GRIB2 (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/).
But that conversion errored out after processing the first 36
GRIB records. It died on the 37th record, which contains UGRD data.
So I ran the Point-Stat test script on both the GRIB1 and GRIB2
versions of the file, but only after removing the UGRD/VGRD
verification from the Point-Stat config file. And the output of
Point-Stat
was identical for GRIB1 and GRIB2.
But some obvious questions remain:
(1) Why did cnvgrib error out on record number 37?
I'm not sure and we are not funded to provide support for that
utility.
(2) Why do you see the difference when converting from GRIB1 to 2
using grib_convert?
I downloaded and built the ECMWF GRIB tools. And I ran the same
conversion you did using the grib_convert tool. And I was able to
reproduce the problem you're seeing in the output of Point-Stat.
So I took a step back and just plotted the same record from the
GRIB1 and GRIB2 files, TMP at 750mb. Listed below if the command I
used for plot_data_plane. I've attached two images showing those
plots.
$MET_BASE/bin/plot_data_plane \
nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
'name="TMP"; level="P750";'
Please take a look at those files and you'll notice a big discrepancy
in how the grid is defined. For some reason, the grib_convert utility
is doing something odd to the grid specification. Listed
below is the output of wgrib (for GRIB1) and wgrib2 (for the output of
grib_convert). As you can see, there is something very wrong with the
"LoV -2052.483648" setting in the wgrib2 output.
So there's your answer. The problem is coming from the GRIB 1 -> 2
conversion tool that you're using.
Hope that helps.
John
GRIB1 output:
rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100 kpds7=750
levels=(2,238) grid=218 750 mb 36hr fcst:
TMP=Temp. [K]
timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS grid 3 num_in_ave
0 missing 0
center 7 subcenter 0 process 84 Table 2 scan: WE:SN winds(grid)
Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP 0.000000
North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan 64 mode 8
min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0 BinScale
0
GRIB_CONVERT output:
21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature [K]:
ndata=262792:undef=0:mean=273.101:min=248:max=289
grid_template=30:winds(grid):
Lambert Conformal: (614 x 428) input WE:SN output WE:SN res 8
Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
LatD 60.000000 Latin1 25.000000 Latin2 25.000000
LatSP 0.000000 LonSP 0.000000
North Pole (614 x 428) Dx 12191.000000 m Dy 12191.000000 m
mode 8
On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>
> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
> Transaction: Ticket created by muravev at mecom.ru
> Queue: met_help
> Subject: METv4.0 problems
> Owner: Nobody
> Requestors: muravev at mecom.ru
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>
>
> Dear John,
>
> I am now in new troubles with MET and would like to ask your help
after having changed for the new v4.0 package. Three small directories
are attached to better understand my problems.
>
> A. RUS_DEMO_METv3.1 directory.
>
> The grib1 increment rounding was the previous trouble with METv3.1
though everything else went normally, including ‘gen_poly_mask’,
‘plot_point_obs’ and ‘point_stat’ steps.
>
> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>
> Station positioning (very tiny red points though) near Sochi region
in MET_01_TA_2012-08-01.pdf and numbers in
‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
showed that we are on the right track.
>
> B. RUS_DEMO_METv4.0 directory.
>
> The grib1 lat-lon increments were unacceptable for 1x1-km model, and
we decided to reformat all files to grib2 with the ECMWF grib_convert
procedure. Some additional testing showed that this procedure gave
identical results on both data formats of our COSMO model.
>
> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>
>
> 1. No lat-lon INNER boarders were processed correctly by the
‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
collected points inside the region as shown in ‘MET_01_TA_2012-08-
01.plot_point.log’.
>
> 2. That something goes wrong is seen in the coastline drawn by
‘plot_point_obs’. It seems as if lats and lons have been mixed up
somewhere (MET_01_TA_2012-08-01.pdf).
>
> 3. Together with the grid specification trouble I cannot understand
the forecast lead time definition in ‘PointStatConfig’, which was no
problem in v3.1 with hourly forecast fields gathered in separate
variable-files.
>
> C. USA_DEMO_METv3.1-4.0 directory.
>
> Since several attempts to realize the error source failed, I decided
to test the presentation WRF data in the METv4.0 data storage.
>
> I took one model data file, reformatted it to grib2 and ran two
parallel calculations with METv4.0/point_stat tool. Table statistics
in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’
refer to grib2. THEY ARE DIFFERENT, so I realized that I missed
something very important from the very beginning.
>
> It is worth noting that all formatted RUS and US data are diagnosed
with wgrib and wgrib2 procedures and included into corresponding
directories. Data files in directories attached are given as links.
>
> I would be very much obliged if you could explain my faults and
misunderstanding in METv4.0 application for the Sochi region.
>
> Thank you in advance.
>
> Best regards,
>
> Anatoly
>
> ........................................
> Dr. Anatoly Muravyev
> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
> OF THE RUSSIAN FEDERATION
> 11-13, Bolshoi Predtechensky Per.
> 123242, Moscow, Russia
> Phone: + 7-499-7952127
> Fax: + 7-499-2551582
> E-mail: muravev at mecom.ru
> ........................................
>
------------------------------------------------
Subject: Re[2]: [rt.rap.ucar.edu #58172] METv4.0 problems
From: muravev at mecom.ru
Time: Tue Sep 11 02:30:51 2012
Dear John,
Thank you very much for your detailed letter. I see several problems
remain.
Now I'll take a closer look at your points in the letter and I hope we
will find proper decisions with the help of your targeted answers.
Best regards,
Anatoly
JHGvR> Anatoly,
JHGvR> Very good questions indeed. After reading your email,
JHGvR> I stepped through the same process you went through, using the
JHGvR> MET test data to see if the results differed for GRIB1 vs
JHGvR> GRIB2. However, the big
JHGvR> difference is that I used the "cnvgrib" utility to do
JHGvR> the conversion from GRIB1 to GRIB2
JHGvR> (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/). But that
JHGvR> conversion errored out after processing the first 36
JHGvR> GRIB records. It died on the 37th record, which contains UGRD
data.
JHGvR> So I ran the Point-Stat test script on both the GRIB1
JHGvR> and GRIB2 versions of the file, but only after removing the
JHGvR> UGRD/VGRD verification from the Point-Stat config file. And
JHGvR> the output of Point-Stat
JHGvR> was identical for GRIB1 and GRIB2.
JHGvR> But some obvious questions remain:
JHGvR> (1) Why did cnvgrib error out on record number 37?
JHGvR> I'm not sure and we are not funded to provide support for
that utility.
JHGvR> (2) Why do you see the difference when converting from
JHGvR> GRIB1 to 2 using grib_convert?
JHGvR> I downloaded and built the ECMWF GRIB tools. And I
JHGvR> ran the same conversion you did using the grib_convert tool.
JHGvR> And I was able to reproduce the problem you're seeing in the
JHGvR> output of Point-Stat.
JHGvR> So I took a step back and just plotted the same
JHGvR> record from the GRIB1 and GRIB2 files, TMP at 750mb. Listed
JHGvR> below if the command I used for plot_data_plane. I've attached
JHGvR> two images showing those
JHGvR> plots.
JHGvR> $MET_BASE/bin/plot_data_plane \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
JHGvR> 'name="TMP"; level="P750";'
JHGvR> Please take a look at those files and you'll notice a
JHGvR> big discrepancy in how the grid is defined. For some reason,
JHGvR> the grib_convert utility is doing something odd to the grid
JHGvR> specification. Listed
JHGvR> below is the output of wgrib (for GRIB1) and wgrib2
JHGvR> (for the output of grib_convert). As you can see, there is
JHGvR> something very wrong with the "LoV -2052.483648" setting in the
JHGvR> wgrib2 output.
JHGvR> So there's your answer. The problem is coming from the
JHGvR> GRIB 1 -> 2 conversion tool that you're using.
JHGvR> Hope that helps.
JHGvR> John
JHGvR> GRIB1 output:
JHGvR> rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100
JHGvR> kpds7=750 levels=(2,238) grid=218 750 mb 36hr fcst:
JHGvR> TMP=Temp. [K]
JHGvR> timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS
JHGvR> grid 3 num_in_ave 0 missing 0
JHGvR> center 7 subcenter 0 process 84 Table 2 scan: WE:SN
winds(grid)
JHGvR> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
JHGvR> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP
0.000000
JHGvR> North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan 64
mode 8
JHGvR> min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0
BinScale 0
JHGvR> GRIB_CONVERT output:
JHGvR> 21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature
[K]:
JHGvR> ndata=262792:undef=0:mean=273.101:min=248:max=289
JHGvR> grid_template=30:winds(grid):
JHGvR> Lambert Conformal: (614 x 428) input WE:SN output
WE:SN res 8
JHGvR> Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
JHGvR> LatD 60.000000 Latin1 25.000000 Latin2 25.000000
JHGvR> LatSP 0.000000 LonSP 0.000000
JHGvR> North Pole (614 x 428) Dx 12191.000000 m Dy
12191.000000 m mode 8
JHGvR> On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>>
>> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
>> Transaction: Ticket created by muravev at mecom.ru
>> Queue: met_help
>> Subject: METv4.0 problems
>> Owner: Nobody
>> Requestors: muravev at mecom.ru
>> Status: new
>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>>
>>
>> Dear John,
>>
>> I am now in new troubles with MET and would like to ask your
>> help after having changed for the new v4.0 package. Three small
>> directories are attached to better understand my problems.
>>
>> A. RUS_DEMO_METv3.1 directory.
>>
>> The grib1 increment rounding was the previous trouble with
>> METv3.1 though everything else went normally, including
>> ‘gen_poly_mask’, ‘plot_point_obs’ and ‘point_stat’ steps.
>>
>> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>>
>> Station positioning (very tiny red points though) near Sochi
>> region in MET_01_TA_2012-08-01.pdf and numbers in
>> ‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
>> showed that we are on the right track.
>>
>> B. RUS_DEMO_METv4.0 directory.
>>
>> The grib1 lat-lon increments were unacceptable for 1x1-km
>> model, and we decided to reformat all files to grib2 with the ECMWF
>> grib_convert procedure. Some additional testing showed that this
>> procedure gave identical results on both data formats of our COSMO
>> model.
>>
>> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>>
>>
>> 1. No lat-lon INNER boarders were processed correctly by the
>> ‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
>> collected points inside the region as shown in
>> ‘MET_01_TA_2012-08-01.plot_point.log’.
>>
>> 2. That something goes wrong is seen in the coastline drawn by
>> ‘plot_point_obs’. It seems as if lats and lons have been mixed up
>> somewhere (MET_01_TA_2012-08-01.pdf).
>>
>> 3. Together with the grid specification trouble I cannot
>> understand the forecast lead time definition in ‘PointStatConfig’,
>> which was no problem in v3.1 with hourly forecast fields gathered
>> in separate variable-files.
>>
>> C. USA_DEMO_METv3.1-4.0 directory.
>>
>> Since several attempts to realize the error source failed, I
>> decided to test the presentation WRF data in the METv4.0 data
>> storage.
>>
>> I took one model data file, reformatted it to grib2 and ran two
>> parallel calculations with METv4.0/point_stat tool. Table
>> statistics in ‘test_point_stat_01’ refer to grib1 and in
>> ‘test_point_stat_01’ refer to grib2. THEY ARE DIFFERENT, so I
>> realized that I missed something very important from the very
>> beginning.
>>
>> It is worth noting that all formatted RUS and US data are
>> diagnosed with wgrib and wgrib2 procedures and included into
>> corresponding directories. Data files in directories attached are
>> given as links.
>>
>> I would be very much obliged if you could explain my faults and
>> misunderstanding in METv4.0 application for the Sochi region.
>>
>> Thank you in advance.
>>
>> Best regards,
>>
>> Anatoly
>>
>> ........................................
>> Dr. Anatoly Muravyev
>> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
>> OF THE RUSSIAN FEDERATION
>> 11-13, Bolshoi Predtechensky Per.
>> 123242, Moscow, Russia
>> Phone: + 7-499-7952127
>> Fax: + 7-499-2551582
>> E-mail: muravev at mecom.ru
>> ........................................
>>
------------------------------------------------
More information about the Met_help
mailing list