[Met_help] [rt.rap.ucar.edu #58172] History for METv4.0 problems
John Halley Gotway via RT
met_help at ucar.edu
Mon Oct 8 05:50:12 MDT 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear John,
I am now in new troubles with MET and would like to ask your help after having changed for the new v4.0 package. Three small directories are attached to better understand my problems.
A. RUS_DEMO_METv3.1 directory.
The grib1 increment rounding was the previous trouble with METv3.1 though everything else went normally, including ‘gen_poly_mask’, ‘plot_point_obs’ and ‘point_stat’ steps.
Some results are given in RUS_DEMO_METv3.1 with the corresponding run-script.
Station positioning (very tiny red points though) near Sochi region in MET_01_TA_2012-08-01.pdf and numbers in ‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory showed that we are on the right track.
B. RUS_DEMO_METv4.0 directory.
The grib1 lat-lon increments were unacceptable for 1x1-km model, and we decided to reformat all files to grib2 with the ECMWF grib_convert procedure. Some additional testing showed that this procedure gave identical results on both data formats of our COSMO model.
But all METv4.0 steps were discouraging as given in RUS_DEMO_METv4.0.
1. No lat-lon INNER boarders were processed correctly by the ‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly) collected points inside the region as shown in ‘MET_01_TA_2012-08-01.plot_point.log’.
2. That something goes wrong is seen in the coastline drawn by ‘plot_point_obs’. It seems as if lats and lons have been mixed up somewhere (MET_01_TA_2012-08-01.pdf).
3. Together with the grid specification trouble I cannot understand the forecast lead time definition in ‘PointStatConfig’, which was no problem in v3.1 with hourly forecast fields gathered in separate variable-files.
C. USA_DEMO_METv3.1-4.0 directory.
Since several attempts to realize the error source failed, I decided to test the presentation WRF data in the METv4.0 data storage.
I took one model data file, reformatted it to grib2 and ran two parallel calculations with METv4.0/point_stat tool. Table statistics in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’ refer to grib2. THEY ARE DIFFERENT, so I realized that I missed something very important from the very beginning.
It is worth noting that all formatted RUS and US data are diagnosed with wgrib and wgrib2 procedures and included into corresponding directories. Data files in directories attached are given as links.
I would be very much obliged if you could explain my faults and misunderstanding in METv4.0 application for the Sochi region.
Thank you in advance.
Best regards,
Anatoly
........................................
Dr. Anatoly Muravyev
THE HYDROMETEOROLOGICAL RESEARCH CENTRE
OF THE RUSSIAN FEDERATION
11-13, Bolshoi Predtechensky Per.
123242, Moscow, Russia
Phone: + 7-499-7952127
Fax: + 7-499-2551582
E-mail: muravev at mecom.ru
........................................
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: METv4.0 problems
From: John Halley Gotway
Time: Mon Sep 10 08:43:33 2012
Anatoly,
Very good questions indeed. After reading your email, I stepped
through the same process you went through, using the MET test data to
see if the results differed for GRIB1 vs GRIB2. However, the big
difference is that I used the "cnvgrib" utility to do the conversion
from GRIB1 to GRIB2 (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/).
But that conversion errored out after processing the first 36
GRIB records. It died on the 37th record, which contains UGRD data.
So I ran the Point-Stat test script on both the GRIB1 and GRIB2
versions of the file, but only after removing the UGRD/VGRD
verification from the Point-Stat config file. And the output of
Point-Stat
was identical for GRIB1 and GRIB2.
But some obvious questions remain:
(1) Why did cnvgrib error out on record number 37?
I'm not sure and we are not funded to provide support for that
utility.
(2) Why do you see the difference when converting from GRIB1 to 2
using grib_convert?
I downloaded and built the ECMWF GRIB tools. And I ran the same
conversion you did using the grib_convert tool. And I was able to
reproduce the problem you're seeing in the output of Point-Stat.
So I took a step back and just plotted the same record from the
GRIB1 and GRIB2 files, TMP at 750mb. Listed below if the command I
used for plot_data_plane. I've attached two images showing those
plots.
$MET_BASE/bin/plot_data_plane \
nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
'name="TMP"; level="P750";'
Please take a look at those files and you'll notice a big discrepancy
in how the grid is defined. For some reason, the grib_convert utility
is doing something odd to the grid specification. Listed
below is the output of wgrib (for GRIB1) and wgrib2 (for the output of
grib_convert). As you can see, there is something very wrong with the
"LoV -2052.483648" setting in the wgrib2 output.
So there's your answer. The problem is coming from the GRIB 1 -> 2
conversion tool that you're using.
Hope that helps.
John
GRIB1 output:
rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100 kpds7=750
levels=(2,238) grid=218 750 mb 36hr fcst:
TMP=Temp. [K]
timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS grid 3 num_in_ave
0 missing 0
center 7 subcenter 0 process 84 Table 2 scan: WE:SN winds(grid)
Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP 0.000000
North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan 64 mode 8
min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0 BinScale
0
GRIB_CONVERT output:
21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature [K]:
ndata=262792:undef=0:mean=273.101:min=248:max=289
grid_template=30:winds(grid):
Lambert Conformal: (614 x 428) input WE:SN output WE:SN res 8
Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
LatD 60.000000 Latin1 25.000000 Latin2 25.000000
LatSP 0.000000 LonSP 0.000000
North Pole (614 x 428) Dx 12191.000000 m Dy 12191.000000 m
mode 8
On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>
> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
> Transaction: Ticket created by muravev at mecom.ru
> Queue: met_help
> Subject: METv4.0 problems
> Owner: Nobody
> Requestors: muravev at mecom.ru
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>
>
> Dear John,
>
> I am now in new troubles with MET and would like to ask your help
after having changed for the new v4.0 package. Three small directories
are attached to better understand my problems.
>
> A. RUS_DEMO_METv3.1 directory.
>
> The grib1 increment rounding was the previous trouble with METv3.1
though everything else went normally, including ‘gen_poly_mask’,
‘plot_point_obs’ and ‘point_stat’ steps.
>
> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>
> Station positioning (very tiny red points though) near Sochi region
in MET_01_TA_2012-08-01.pdf and numbers in
‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
showed that we are on the right track.
>
> B. RUS_DEMO_METv4.0 directory.
>
> The grib1 lat-lon increments were unacceptable for 1x1-km model, and
we decided to reformat all files to grib2 with the ECMWF grib_convert
procedure. Some additional testing showed that this procedure gave
identical results on both data formats of our COSMO model.
>
> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>
>
> 1. No lat-lon INNER boarders were processed correctly by the
‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
collected points inside the region as shown in ‘MET_01_TA_2012-08-
01.plot_point.log’.
>
> 2. That something goes wrong is seen in the coastline drawn by
‘plot_point_obs’. It seems as if lats and lons have been mixed up
somewhere (MET_01_TA_2012-08-01.pdf).
>
> 3. Together with the grid specification trouble I cannot understand
the forecast lead time definition in ‘PointStatConfig’, which was no
problem in v3.1 with hourly forecast fields gathered in separate
variable-files.
>
> C. USA_DEMO_METv3.1-4.0 directory.
>
> Since several attempts to realize the error source failed, I decided
to test the presentation WRF data in the METv4.0 data storage.
>
> I took one model data file, reformatted it to grib2 and ran two
parallel calculations with METv4.0/point_stat tool. Table statistics
in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’
refer to grib2. THEY ARE DIFFERENT, so I realized that I missed
something very important from the very beginning.
>
> It is worth noting that all formatted RUS and US data are diagnosed
with wgrib and wgrib2 procedures and included into corresponding
directories. Data files in directories attached are given as links.
>
> I would be very much obliged if you could explain my faults and
misunderstanding in METv4.0 application for the Sochi region.
>
> Thank you in advance.
>
> Best regards,
>
> Anatoly
>
> ........................................
> Dr. Anatoly Muravyev
> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
> OF THE RUSSIAN FEDERATION
> 11-13, Bolshoi Predtechensky Per.
> 123242, Moscow, Russia
> Phone: + 7-499-7952127
> Fax: + 7-499-2551582
> E-mail: muravev at mecom.ru
> ........................................
>
------------------------------------------------
Subject: Re[2]: [rt.rap.ucar.edu #58172] METv4.0 problems
From: muravev at mecom.ru
Time: Tue Sep 11 02:30:51 2012
Dear John,
Thank you very much for your detailed letter. I see several problems
remain.
Now I'll take a closer look at your points in the letter and I hope we
will find proper decisions with the help of your targeted answers.
Best regards,
Anatoly
JHGvR> Anatoly,
JHGvR> Very good questions indeed. After reading your email,
JHGvR> I stepped through the same process you went through, using the
JHGvR> MET test data to see if the results differed for GRIB1 vs
JHGvR> GRIB2. However, the big
JHGvR> difference is that I used the "cnvgrib" utility to do
JHGvR> the conversion from GRIB1 to GRIB2
JHGvR> (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/). But that
JHGvR> conversion errored out after processing the first 36
JHGvR> GRIB records. It died on the 37th record, which contains UGRD
data.
JHGvR> So I ran the Point-Stat test script on both the GRIB1
JHGvR> and GRIB2 versions of the file, but only after removing the
JHGvR> UGRD/VGRD verification from the Point-Stat config file. And
JHGvR> the output of Point-Stat
JHGvR> was identical for GRIB1 and GRIB2.
JHGvR> But some obvious questions remain:
JHGvR> (1) Why did cnvgrib error out on record number 37?
JHGvR> I'm not sure and we are not funded to provide support for
that utility.
JHGvR> (2) Why do you see the difference when converting from
JHGvR> GRIB1 to 2 using grib_convert?
JHGvR> I downloaded and built the ECMWF GRIB tools. And I
JHGvR> ran the same conversion you did using the grib_convert tool.
JHGvR> And I was able to reproduce the problem you're seeing in the
JHGvR> output of Point-Stat.
JHGvR> So I took a step back and just plotted the same
JHGvR> record from the GRIB1 and GRIB2 files, TMP at 750mb. Listed
JHGvR> below if the command I used for plot_data_plane. I've attached
JHGvR> two images showing those
JHGvR> plots.
JHGvR> $MET_BASE/bin/plot_data_plane \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
JHGvR> 'name="TMP"; level="P750";'
JHGvR> Please take a look at those files and you'll notice a
JHGvR> big discrepancy in how the grid is defined. For some reason,
JHGvR> the grib_convert utility is doing something odd to the grid
JHGvR> specification. Listed
JHGvR> below is the output of wgrib (for GRIB1) and wgrib2
JHGvR> (for the output of grib_convert). As you can see, there is
JHGvR> something very wrong with the "LoV -2052.483648" setting in the
JHGvR> wgrib2 output.
JHGvR> So there's your answer. The problem is coming from the
JHGvR> GRIB 1 -> 2 conversion tool that you're using.
JHGvR> Hope that helps.
JHGvR> John
JHGvR> GRIB1 output:
JHGvR> rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100
JHGvR> kpds7=750 levels=(2,238) grid=218 750 mb 36hr fcst:
JHGvR> TMP=Temp. [K]
JHGvR> timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS
JHGvR> grid 3 num_in_ave 0 missing 0
JHGvR> center 7 subcenter 0 process 84 Table 2 scan: WE:SN
winds(grid)
JHGvR> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
JHGvR> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP
0.000000
JHGvR> North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan 64
mode 8
JHGvR> min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0
BinScale 0
JHGvR> GRIB_CONVERT output:
JHGvR> 21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature
[K]:
JHGvR> ndata=262792:undef=0:mean=273.101:min=248:max=289
JHGvR> grid_template=30:winds(grid):
JHGvR> Lambert Conformal: (614 x 428) input WE:SN output
WE:SN res 8
JHGvR> Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
JHGvR> LatD 60.000000 Latin1 25.000000 Latin2 25.000000
JHGvR> LatSP 0.000000 LonSP 0.000000
JHGvR> North Pole (614 x 428) Dx 12191.000000 m Dy
12191.000000 m mode 8
JHGvR> On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>>
>> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
>> Transaction: Ticket created by muravev at mecom.ru
>> Queue: met_help
>> Subject: METv4.0 problems
>> Owner: Nobody
>> Requestors: muravev at mecom.ru
>> Status: new
>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>>
>>
>> Dear John,
>>
>> I am now in new troubles with MET and would like to ask your
>> help after having changed for the new v4.0 package. Three small
>> directories are attached to better understand my problems.
>>
>> A. RUS_DEMO_METv3.1 directory.
>>
>> The grib1 increment rounding was the previous trouble with
>> METv3.1 though everything else went normally, including
>> ‘gen_poly_mask’, ‘plot_point_obs’ and ‘point_stat’ steps.
>>
>> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>>
>> Station positioning (very tiny red points though) near Sochi
>> region in MET_01_TA_2012-08-01.pdf and numbers in
>> ‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
>> showed that we are on the right track.
>>
>> B. RUS_DEMO_METv4.0 directory.
>>
>> The grib1 lat-lon increments were unacceptable for 1x1-km
>> model, and we decided to reformat all files to grib2 with the ECMWF
>> grib_convert procedure. Some additional testing showed that this
>> procedure gave identical results on both data formats of our COSMO
>> model.
>>
>> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>>
>>
>> 1. No lat-lon INNER boarders were processed correctly by the
>> ‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
>> collected points inside the region as shown in
>> ‘MET_01_TA_2012-08-01.plot_point.log’.
>>
>> 2. That something goes wrong is seen in the coastline drawn by
>> ‘plot_point_obs’. It seems as if lats and lons have been mixed up
>> somewhere (MET_01_TA_2012-08-01.pdf).
>>
>> 3. Together with the grid specification trouble I cannot
>> understand the forecast lead time definition in ‘PointStatConfig’,
>> which was no problem in v3.1 with hourly forecast fields gathered
>> in separate variable-files.
>>
>> C. USA_DEMO_METv3.1-4.0 directory.
>>
>> Since several attempts to realize the error source failed, I
>> decided to test the presentation WRF data in the METv4.0 data
>> storage.
>>
>> I took one model data file, reformatted it to grib2 and ran two
>> parallel calculations with METv4.0/point_stat tool. Table
>> statistics in ‘test_point_stat_01’ refer to grib1 and in
>> ‘test_point_stat_01’ refer to grib2. THEY ARE DIFFERENT, so I
>> realized that I missed something very important from the very
>> beginning.
>>
>> It is worth noting that all formatted RUS and US data are
>> diagnosed with wgrib and wgrib2 procedures and included into
>> corresponding directories. Data files in directories attached are
>> given as links.
>>
>> I would be very much obliged if you could explain my faults and
>> misunderstanding in METv4.0 application for the Sochi region.
>>
>> Thank you in advance.
>>
>> Best regards,
>>
>> Anatoly
>>
>> ........................................
>> Dr. Anatoly Muravyev
>> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
>> OF THE RUSSIAN FEDERATION
>> 11-13, Bolshoi Predtechensky Per.
>> 123242, Moscow, Russia
>> Phone: + 7-499-7952127
>> Fax: + 7-499-2551582
>> E-mail: muravev at mecom.ru
>> ........................................
>>
------------------------------------------------
Subject: Re[2]: [rt.rap.ucar.edu #58172] METv4.0 problems
From: muravev at mecom.ru
Time: Wed Oct 03 09:58:15 2012
Dear John,
I would like to inform you about how far we are with METv4.0
experiments. Actually, I would like to make two points:
1) GRIB1 experiments.
Passing several formats (grib1, cnvgrib-grib2, grib_convert-grib2,
netcdf) and map projections (lat-lon and lambert) through
gen_poly_mask, plot_point and point_stat tools yielded the following:
=>only grib1 variant could bring resonable pictures and statistical
tables.
N C D U M P output test example:
netcdf Sochi2_T_2M.grb1 {
dimensions:
lat = 301 ;
lon = 301 ;
variables:
float lat(lat, lon) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:standard_name = "latitude" ;
float lon(lat, lon) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:standard_name = "longitude" ;
int SOCHI(lat, lon) ;
SOCHI:long_name = "SOCHI polyline masking region" ;
// global attributes:
:FileOrigins = "File Sochi2_T_2M.grb1.nc generated 20120929_212037
UTC on host linux-xyba.site by the MET gen_poly_mask tool" ;
:MET_version = "V4.0" ;
:MET_tool = "gen_poly_mask" ;
:Projection = "LatLon" ;
:lat_ll = "42.333000 degrees_north" ;
:lon_ll = "37.700000 degrees_east" ;
:delta_lat = "0.009000 degrees" ;
:delta_lon = "0.013000 degrees" ;
:Nlat = "301 grid_points" ;
:Nlon = "301 grid_points" ;
data:
}
=> ALL grib2 variants were unsatisfactory, revealing lon-lat
distortions in netcdf intermediate files.
N C D U M P output:
netcdf Sochi2_T_2M.grb2 {
dimensions:
lat = 301 ;
lon = 301 ;
variables:
float lat(lat, lon) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:standard_name = "latitude" ;
float lon(lat, lon) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:standard_name = "longitude" ;
int SOCHI(lat, lon) ;
SOCHI:long_name = "SOCHI polyline masking region" ;
// global attributes:
:FileOrigins = "File Sochi2_T_2M.grb2.nc generated 20120930_133104
UTC on host linux-xyba.site by the MET gen_poly_mask tool" ;
:MET_version = "V4.0" ;
:MET_tool = "gen_poly_mask" ;
:Projection = "LatLon" ;
:lat_ll = "38.570500 degrees_north" ;
:lon_ll = "-322.300000 degrees_east" ;
:delta_lat = "0.012500 degrees" ;
:delta_lon = "0.009000 degrees" ;
:Nlat = "301 grid_points" ;
:Nlon = "301 grid_points" ;
data:
}
2) You also noted strange values, suspecting corrupt grib_api
conversion. But the same was observed for NCEP cnvgrib, so I decided
to look closer at the actual and distorted values of our regular lat-
lon grid parameters.
I have found out that the first latitude nc-value satisfied the
equation
38.5705 = 42.333 - 301*0.0125 = (valid)lat - Nlat*delta_lat.
The nc-longitude was simply a negative 2*pi complement,
i.e. -(360-(valid)lon).
It would not take a special effort to conclude that such
transformations may be quite natural for the western hemisphere with
+/- longitudes.
Examining the basic codes and searching with grep-command I have
established the location of possible confusion
for our domestic grid definition:
METv4.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc.
Namely the lines after #719
..........................................................
// build a LatLonData struct with the projection information
LatLonData data;
data.name = latlon_proj_type;
data.delta_lat = (double)gfld->igdtmpl[16] / 1000000.0;
data.delta_lon = (double)gfld->igdtmpl[17] / 1000000.0;
data.Nlat = gfld->igdtmpl[8];
data.Nlon = gfld->igdtmpl[7];
data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0) -
(double)data.Nlat * data.delta_lat;
data.lon_ll = 360 - (double)gfld->igdtmpl[12] / 1000000.0;
..........................................................
Inspired by your remark about giving me a free hand to do with codes
all I wanted, I took the liberty of changing the corresponding lines
to the following ones:
..........................................................
// *********************muravev 20121003
// build a LatLonData struct with the projection information
LatLonData data;
data.name = latlon_proj_type;
data.delta_lon = (double)gfld->igdtmpl[16] / 1000000.0;
data.delta_lat = (double)gfld->igdtmpl[17] / 1000000.0;
data.Nlat = gfld->igdtmpl[8];
data.Nlon = gfld->igdtmpl[7];
data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0);
data.lon_ll = 0.0 - (double)gfld->igdtmpl[12] / 1000000.0;
..........................................................
You know, after the re-installation of the package, everything was OK
and in full accord with what I had in the grib1 case (taking into
account the two grids defined with slightly different delta_lons)!
I would be interested to know what you think about all of the above
and look forward to your feedback.
Best regards,
Anatoly
JHGvR> Anatoly,
JHGvR> Very good questions indeed. After reading your email, I
JHGvR> stepped through the same process you went through, using the
JHGvR> MET test data to see if the results differed for GRIB1 vs
GRIB2. However, the big
JHGvR> difference is that I used the "cnvgrib" utility to do the
JHGvR> conversion from GRIB1 to GRIB2
JHGvR> (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/). But that
JHGvR> conversion errored out after processing the first 36
JHGvR> GRIB records. It died on the 37th record, which contains UGRD
data.
JHGvR> So I ran the Point-Stat test script on both the GRIB1 and
JHGvR> GRIB2 versions of the file, but only after removing the
JHGvR> UGRD/VGRD verification from the Point-Stat config file. And
the output of Point-Stat
JHGvR> was identical for GRIB1 and GRIB2.
JHGvR> But some obvious questions remain:
JHGvR> (1) Why did cnvgrib error out on record number 37?
JHGvR> I'm not sure and we are not funded to provide support for
that utility.
JHGvR> (2) Why do you see the difference when converting from GRIB1 to
2 using grib_convert?
JHGvR> I downloaded and built the ECMWF GRIB tools. And I ran
JHGvR> the same conversion you did using the grib_convert tool. And I
JHGvR> was able to reproduce the problem you're seeing in the output
of Point-Stat.
JHGvR> So I took a step back and just plotted the same record from
JHGvR> the GRIB1 and GRIB2 files, TMP at 750mb. Listed below if the
JHGvR> command I used for plot_data_plane. I've attached two images
showing those
JHGvR> plots.
JHGvR> $MET_BASE/bin/plot_data_plane \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
JHGvR> 'name="TMP"; level="P750";'
JHGvR> Please take a look at those files and you'll notice a big
JHGvR> discrepancy in how the grid is defined. For some reason, the
JHGvR> grib_convert utility is doing something odd to the grid
specification. Listed
JHGvR> below is the output of wgrib (for GRIB1) and wgrib2 (for the
JHGvR> output of grib_convert). As you can see, there is something
JHGvR> very wrong with the "LoV -2052.483648" setting in the wgrib2
output.
JHGvR> So there's your answer. The problem is coming from the GRIB 1
JHGvR> -> 2 conversion tool that you're using.
JHGvR> Hope that helps.
JHGvR> John
JHGvR> GRIB1 output:
JHGvR> rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100
JHGvR> kpds7=750 levels=(2,238) grid=218 750 mb 36hr fcst:
JHGvR> TMP=Temp. [K]
JHGvR> timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS grid 3
num_in_ave 0 missing 0
JHGvR> center 7 subcenter 0 process 84 Table 2 scan: WE:SN
winds(grid)
JHGvR> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov -95.000000
JHGvR> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP
0.000000
JHGvR> North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan 64
mode 8
JHGvR> min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0
BinScale 0
JHGvR> GRIB_CONVERT output:
JHGvR> 21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature
[K]:
JHGvR> ndata=262792:undef=0:mean=273.101:min=248:max=289
JHGvR> grid_template=30:winds(grid):
JHGvR> Lambert Conformal: (614 x 428) input WE:SN output
WE:SN res 8
JHGvR> Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
JHGvR> LatD 60.000000 Latin1 25.000000 Latin2 25.000000
JHGvR> LatSP 0.000000 LonSP 0.000000
JHGvR> North Pole (614 x 428) Dx 12191.000000 m Dy
12191.000000 m mode 8
JHGvR> On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
>> Transaction: Ticket created by muravev at mecom.ru
>> Queue: met_help
>> Subject: METv4.0 problems
>> Owner: Nobody
>> Requestors: muravev at mecom.ru
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>> Dear John,
>> I am now in new troubles with MET and would like to ask your help
after having changed for the new v4.0 package. Three small directories
are attached to better understand my problems.
>> A. RUS_DEMO_METv3.1 directory.
>> The grib1 increment rounding was the previous trouble with METv3.1
though everything else went normally, including ‘gen_poly_mask’,
‘plot_point_obs’ and ‘point_stat’ steps.
>> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>> Station positioning (very tiny red points though) near Sochi region
in MET_01_TA_2012-08-01.pdf and numbers in
‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
showed that we are on the right track.
>> B. RUS_DEMO_METv4.0 directory.
>> The grib1 lat-lon increments were unacceptable for 1x1-km model,
and we decided to reformat all files to grib2 with the ECMWF
grib_convert procedure. Some additional testing showed that this
procedure gave identical results on both data formats of our COSMO
model.
>> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>> 1. No lat-lon INNER boarders were processed correctly by the
‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
collected points inside the region as shown in ‘MET_01_TA_2012-08-
01.plot_point.log’.
>> 2. That something goes wrong is seen in the coastline drawn by
‘plot_point_obs’. It seems as if lats and lons have been mixed up
somewhere (MET_01_TA_2012-08-01.pdf).
>> 3. Together with the grid specification trouble I cannot understand
the forecast lead time definition in ‘PointStatConfig’, which was no
problem in v3.1 with hourly forecast fields gathered in separate
variable-files.
>> C. USA_DEMO_METv3.1-4.0 directory.
>> Since several attempts to realize the error source failed, I
decided to test the presentation WRF data in the METv4.0 data storage.
>> I took one model data file, reformatted it to grib2 and ran two
parallel calculations with METv4.0/point_stat tool. Table statistics
in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’
refer to grib2. THEY ARE DIFFERENT, so I realized that I missed
something very important from the very beginning.
>> It is worth noting that all formatted RUS and US data are diagnosed
with wgrib and wgrib2 procedures and included into corresponding
directories. Data files in directories attached are given as links.
>> I would be very much obliged if you could explain my faults and
misunderstanding in METv4.0 application for the Sochi region.
>> Thank you in advance.
>> Best regards,
>> Anatoly
>> ........................................
>> Dr. Anatoly Muravyev
>> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
>> OF THE RUSSIAN FEDERATION
>> 11-13, Bolshoi Predtechensky Per.
>> 123242, Moscow, Russia
>> Phone: + 7-499-7952127
>> Fax: + 7-499-2551582
>> E-mail: muravev at mecom.ru
>> ........................................
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #58172] METv4.0 problems
From: Paul Oldenburg
Time: Fri Oct 05 12:30:54 2012
Anatoly,
Thank you for bringing this situation to our attention. We have done
some testing and released a patch for METv4.0
which contains, among other changes, the fix that you suggested. I
hope this new version of the GRIB2 works for you.
You can download and install the patch by following the instructions
here:
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
Please let us know if you have any questions.
Thanks,
Paul
On 10/03/2012 09:58 AM, muravev at mecom.ru via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>
> Dear John,
>
> I would like to inform you about how far we are with METv4.0
experiments. Actually, I would like to make two points:
>
> 1) GRIB1 experiments.
>
> Passing several formats (grib1, cnvgrib-grib2, grib_convert-grib2,
netcdf) and map projections (lat-lon and lambert) through
gen_poly_mask, plot_point and point_stat tools yielded the following:
>
> =>only grib1 variant could bring resonable pictures and statistical
tables.
>
> N C D U M P output test example:
>
> netcdf Sochi2_T_2M.grb1 {
> dimensions:
> lat = 301 ;
> lon = 301 ;
> variables:
> float lat(lat, lon) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
> lat:standard_name = "latitude" ;
> float lon(lat, lon) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> lon:standard_name = "longitude" ;
> int SOCHI(lat, lon) ;
> SOCHI:long_name = "SOCHI polyline masking region" ;
>
> // global attributes:
> :FileOrigins = "File Sochi2_T_2M.grb1.nc generated 20120929_212037
UTC on host linux-xyba.site by the MET gen_poly_mask tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "gen_poly_mask" ;
> :Projection = "LatLon" ;
> :lat_ll = "42.333000 degrees_north" ;
> :lon_ll = "37.700000 degrees_east" ;
> :delta_lat = "0.009000 degrees" ;
> :delta_lon = "0.013000 degrees" ;
> :Nlat = "301 grid_points" ;
> :Nlon = "301 grid_points" ;
> data:
> }
>
>
> => ALL grib2 variants were unsatisfactory, revealing lon-lat
distortions in netcdf intermediate files.
>
> N C D U M P output:
>
> netcdf Sochi2_T_2M.grb2 {
> dimensions:
> lat = 301 ;
> lon = 301 ;
> variables:
> float lat(lat, lon) ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
> lat:standard_name = "latitude" ;
> float lon(lat, lon) ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> lon:standard_name = "longitude" ;
> int SOCHI(lat, lon) ;
> SOCHI:long_name = "SOCHI polyline masking region" ;
>
> // global attributes:
> :FileOrigins = "File Sochi2_T_2M.grb2.nc generated 20120930_133104
UTC on host linux-xyba.site by the MET gen_poly_mask tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "gen_poly_mask" ;
> :Projection = "LatLon" ;
> :lat_ll = "38.570500 degrees_north" ;
> :lon_ll = "-322.300000 degrees_east" ;
> :delta_lat = "0.012500 degrees" ;
> :delta_lon = "0.009000 degrees" ;
> :Nlat = "301 grid_points" ;
> :Nlon = "301 grid_points" ;
> data:
> }
>
>
> 2) You also noted strange values, suspecting corrupt grib_api
conversion. But the same was observed for NCEP cnvgrib, so I decided
to look closer at the actual and distorted values of our regular lat-
lon grid parameters.
>
> I have found out that the first latitude nc-value satisfied the
equation
> 38.5705 = 42.333 - 301*0.0125 = (valid)lat - Nlat*delta_lat.
> The nc-longitude was simply a negative 2*pi complement,
> i.e. -(360-(valid)lon).
>
> It would not take a special effort to conclude that such
transformations may be quite natural for the western hemisphere with
+/- longitudes.
>
> Examining the basic codes and searching with grep-command I have
established the location of possible confusion
> for our domestic grid definition:
>
> METv4.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc.
>
> Namely the lines after #719
> ..........................................................
> // build a LatLonData struct with the projection information
> LatLonData data;
> data.name = latlon_proj_type;
> data.delta_lat = (double)gfld->igdtmpl[16] / 1000000.0;
> data.delta_lon = (double)gfld->igdtmpl[17] / 1000000.0;
> data.Nlat = gfld->igdtmpl[8];
> data.Nlon = gfld->igdtmpl[7];
> data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0) -
(double)data.Nlat * data.delta_lat;
> data.lon_ll = 360 - (double)gfld->igdtmpl[12] /
1000000.0;
> ..........................................................
>
> Inspired by your remark about giving me a free hand to do with codes
all I wanted, I took the liberty of changing the corresponding lines
to the following ones:
>
> ..........................................................
> // *********************muravev 20121003
> // build a LatLonData struct with the projection information
> LatLonData data;
> data.name = latlon_proj_type;
> data.delta_lon = (double)gfld->igdtmpl[16] / 1000000.0;
> data.delta_lat = (double)gfld->igdtmpl[17] / 1000000.0;
> data.Nlat = gfld->igdtmpl[8];
> data.Nlon = gfld->igdtmpl[7];
> data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0);
> data.lon_ll = 0.0 - (double)gfld->igdtmpl[12] /
1000000.0;
> ..........................................................
>
> You know, after the re-installation of the package, everything was
OK and in full accord with what I had in the grib1 case (taking into
account the two grids defined with slightly different delta_lons)!
>
> I would be interested to know what you think about all of the above
and look forward to your feedback.
>
>
> Best regards,
> Anatoly
>
>
>
> JHGvR> Anatoly,
>
> JHGvR> Very good questions indeed. After reading your email, I
> JHGvR> stepped through the same process you went through, using the
> JHGvR> MET test data to see if the results differed for GRIB1 vs
GRIB2. However, the big
> JHGvR> difference is that I used the "cnvgrib" utility to do the
> JHGvR> conversion from GRIB1 to GRIB2
> JHGvR> (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/). But that
> JHGvR> conversion errored out after processing the first 36
> JHGvR> GRIB records. It died on the 37th record, which contains
UGRD data.
>
> JHGvR> So I ran the Point-Stat test script on both the GRIB1 and
> JHGvR> GRIB2 versions of the file, but only after removing the
> JHGvR> UGRD/VGRD verification from the Point-Stat config file. And
the output of Point-Stat
> JHGvR> was identical for GRIB1 and GRIB2.
>
> JHGvR> But some obvious questions remain:
> JHGvR> (1) Why did cnvgrib error out on record number 37?
>
> JHGvR> I'm not sure and we are not funded to provide support for
that utility.
>
> JHGvR> (2) Why do you see the difference when converting from GRIB1
to 2 using grib_convert?
>
> JHGvR> I downloaded and built the ECMWF GRIB tools. And I ran
> JHGvR> the same conversion you did using the grib_convert tool. And
I
> JHGvR> was able to reproduce the problem you're seeing in the output
of Point-Stat.
> JHGvR> So I took a step back and just plotted the same record from
> JHGvR> the GRIB1 and GRIB2 files, TMP at 750mb. Listed below if the
> JHGvR> command I used for plot_data_plane. I've attached two images
showing those
> JHGvR> plots.
>
> JHGvR> $MET_BASE/bin/plot_data_plane \
> JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
> JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
> JHGvR> 'name="TMP"; level="P750";'
>
> JHGvR> Please take a look at those files and you'll notice a big
> JHGvR> discrepancy in how the grid is defined. For some reason, the
> JHGvR> grib_convert utility is doing something odd to the grid
specification. Listed
> JHGvR> below is the output of wgrib (for GRIB1) and wgrib2 (for the
> JHGvR> output of grib_convert). As you can see, there is something
> JHGvR> very wrong with the "LoV -2052.483648" setting in the wgrib2
output.
>
> JHGvR> So there's your answer. The problem is coming from the GRIB
1
> JHGvR> -> 2 conversion tool that you're using.
>
> JHGvR> Hope that helps.
>
> JHGvR> John
>
> JHGvR> GRIB1 output:
>
> JHGvR> rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100
> JHGvR> kpds7=750 levels=(2,238) grid=218 750 mb 36hr fcst:
> JHGvR> TMP=Temp. [K]
> JHGvR> timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS grid 3
num_in_ave 0 missing 0
> JHGvR> center 7 subcenter 0 process 84 Table 2 scan: WE:SN
winds(grid)
> JHGvR> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov
-95.000000
> JHGvR> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000 LonSP
0.000000
> JHGvR> North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan
64 mode 8
> JHGvR> min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0
BinScale 0
>
> JHGvR> GRIB_CONVERT output:
>
> JHGvR> 21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature
[K]:
> JHGvR> ndata=262792:undef=0:mean=273.101:min=248:max=289
> JHGvR> grid_template=30:winds(grid):
> JHGvR> Lambert Conformal: (614 x 428) input WE:SN output
WE:SN res 8
> JHGvR> Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
> JHGvR> LatD 60.000000 Latin1 25.000000 Latin2 25.000000
> JHGvR> LatSP 0.000000 LonSP 0.000000
> JHGvR> North Pole (614 x 428) Dx 12191.000000 m Dy
12191.000000 m mode 8
>
> JHGvR> On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>
>>> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
>>> Transaction: Ticket created by muravev at mecom.ru
>>> Queue: met_help
>>> Subject: METv4.0 problems
>>> Owner: Nobody
>>> Requestors: muravev at mecom.ru
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>
>
>>> Dear John,
>
>>> I am now in new troubles with MET and would like to ask your help
after having changed for the new v4.0 package. Three small directories
are attached to better understand my problems.
>
>>> A. RUS_DEMO_METv3.1 directory.
>
>>> The grib1 increment rounding was the previous trouble with METv3.1
though everything else went normally, including ‘gen_poly_mask’,
‘plot_point_obs’ and ‘point_stat’ steps.
>
>>> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>
>>> Station positioning (very tiny red points though) near Sochi
region in MET_01_TA_2012-08-01.pdf and numbers in
‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
showed that we are on the right track.
>
>>> B. RUS_DEMO_METv4.0 directory.
>
>>> The grib1 lat-lon increments were unacceptable for 1x1-km model,
and we decided to reformat all files to grib2 with the ECMWF
grib_convert procedure. Some additional testing showed that this
procedure gave identical results on both data formats of our COSMO
model.
>
>>> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>
>
>>> 1. No lat-lon INNER boarders were processed correctly by the
‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
collected points inside the region as shown in ‘MET_01_TA_2012-08-
01.plot_point.log’.
>
>>> 2. That something goes wrong is seen in the coastline drawn by
‘plot_point_obs’. It seems as if lats and lons have been mixed up
somewhere (MET_01_TA_2012-08-01.pdf).
>
>>> 3. Together with the grid specification trouble I cannot
understand the forecast lead time definition in ‘PointStatConfig’,
which was no problem in v3.1 with hourly forecast fields gathered in
separate variable-files.
>
>>> C. USA_DEMO_METv3.1-4.0 directory.
>
>>> Since several attempts to realize the error source failed, I
decided to test the presentation WRF data in the METv4.0 data storage.
>
>>> I took one model data file, reformatted it to grib2 and ran two
parallel calculations with METv4.0/point_stat tool. Table statistics
in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’
refer to grib2. THEY ARE DIFFERENT, so I realized that I missed
something very important from the very beginning.
>
>>> It is worth noting that all formatted RUS and US data are
diagnosed with wgrib and wgrib2 procedures and included into
corresponding directories. Data files in directories attached are
given as links.
>
>>> I would be very much obliged if you could explain my faults and
misunderstanding in METv4.0 application for the Sochi region.
>
>>> Thank you in advance.
>
>>> Best regards,
>
>>> Anatoly
>
>>> ........................................
>>> Dr. Anatoly Muravyev
>>> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
>>> OF THE RUSSIAN FEDERATION
>>> 11-13, Bolshoi Predtechensky Per.
>>> 123242, Moscow, Russia
>>> Phone: + 7-499-7952127
>>> Fax: + 7-499-2551582
>>> E-mail: muravev at mecom.ru
>>> ........................................
>
>
>
>
------------------------------------------------
Subject: Re[2]: [rt.rap.ucar.edu #58172] METv4.0 problems
From: muravev at mecom.ru
Time: Mon Oct 08 01:52:34 2012
Dear Paul,
Thank you for your swift fixing and the patch.
We really see that MET is an 'evolving software package' and also hope
to get our scientific and operational needs fulfilled with its help.
It is very rewarding to get such prompt and substantial feedbacks from
you and your colleagues. And this is what we like and appreciate about
our cooperation.
Best regards,
Anatoly
POvR> Anatoly,
POvR> Thank you for bringing this situation to our attention. We
POvR> have done some testing and released a patch for METv4.0
POvR> which contains, among other changes, the fix that you
POvR> suggested. I hope this new version of the GRIB2 works for you.
POvR> You can download and install the patch by following the
instructions here:
POvR>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
POvR> Please let us know if you have any questions.
POvR> Thanks,
POvR> Paul
POvR> On 10/03/2012 09:58 AM, muravev at mecom.ru via RT wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>> Dear John,
>> I would like to inform you about how far we are with METv4.0
experiments. Actually, I would like to make two points:
>> 1) GRIB1 experiments.
>> Passing several formats (grib1, cnvgrib-grib2, grib_convert-grib2,
netcdf) and map projections (lat-lon and lambert) through
gen_poly_mask, plot_point and point_stat tools yielded the following:
>> =>only grib1 variant could bring resonable pictures and statistical
tables.
>> N C D U M P output test example:
>> netcdf Sochi2_T_2M.grb1 {
>> dimensions:
>> lat = 301 ;
>> lon = 301 ;
>> variables:
>> float lat(lat, lon) ;
>> lat:long_name = "latitude" ;
>> lat:units = "degrees_north" ;
>> lat:standard_name = "latitude" ;
>> float lon(lat, lon) ;
>> lon:long_name = "longitude" ;
>> lon:units = "degrees_east" ;
>> lon:standard_name = "longitude" ;
>> int SOCHI(lat, lon) ;
>> SOCHI:long_name = "SOCHI polyline masking region" ;
>> // global attributes:
>> :FileOrigins = "File Sochi2_T_2M.grb1.nc generated
20120929_212037 UTC on host linux-xyba.site by the MET gen_poly_mask
tool" ;
>> :MET_version = "V4.0" ;
>> :MET_tool = "gen_poly_mask" ;
>> :Projection = "LatLon" ;
>> :lat_ll = "42.333000 degrees_north" ;
>> :lon_ll = "37.700000 degrees_east" ;
>> :delta_lat = "0.009000 degrees" ;
>> :delta_lon = "0.013000 degrees" ;
>> :Nlat = "301 grid_points" ;
>> :Nlon = "301 grid_points" ;
>> data:
>> }
>> => ALL grib2 variants were unsatisfactory, revealing lon-lat
distortions in netcdf intermediate files.
>> N C D U M P output:
>> netcdf Sochi2_T_2M.grb2 {
>> dimensions:
>> lat = 301 ;
>> lon = 301 ;
>> variables:
>> float lat(lat, lon) ;
>> lat:long_name = "latitude" ;
>> lat:units = "degrees_north" ;
>> lat:standard_name = "latitude" ;
>> float lon(lat, lon) ;
>> lon:long_name = "longitude" ;
>> lon:units = "degrees_east" ;
>> lon:standard_name = "longitude" ;
>> int SOCHI(lat, lon) ;
>> SOCHI:long_name = "SOCHI polyline masking region" ;
>> // global attributes:
>> :FileOrigins = "File Sochi2_T_2M.grb2.nc generated
20120930_133104 UTC on host linux-xyba.site by the MET gen_poly_mask
tool" ;
>> :MET_version = "V4.0" ;
>> :MET_tool = "gen_poly_mask" ;
>> :Projection = "LatLon" ;
>> :lat_ll = "38.570500 degrees_north" ;
>> :lon_ll = "-322.300000 degrees_east" ;
>> :delta_lat = "0.012500 degrees" ;
>> :delta_lon = "0.009000 degrees" ;
>> :Nlat = "301 grid_points" ;
>> :Nlon = "301 grid_points" ;
>> data:
>> }
>> 2) You also noted strange values, suspecting corrupt grib_api
conversion. But the same was observed for NCEP cnvgrib, so I decided
to look closer at the actual and distorted values of our regular lat-
lon grid parameters.
>> I have found out that the first latitude nc-value satisfied the
equation
>> 38.5705 = 42.333 - 301*0.0125 = (valid)lat - Nlat*delta_lat.
>> The nc-longitude was simply a negative 2*pi complement,
>> i.e. -(360-(valid)lon).
>> It would not take a special effort to conclude that such
transformations may be quite natural for the western hemisphere with
+/- longitudes.
>> Examining the basic codes and searching with grep-command I have
established the location of possible confusion
>> for our domestic grid definition:
>> METv4.0/src/libcode/vx_data2d_grib2/data2d_grib2.cc.
>> Namely the lines after #719
>> ..........................................................
>> // build a LatLonData struct with the projection information
>> LatLonData data;
>> data.name = latlon_proj_type;
>> data.delta_lat = (double)gfld->igdtmpl[16] / 1000000.0;
>> data.delta_lon = (double)gfld->igdtmpl[17] / 1000000.0;
>> data.Nlat = gfld->igdtmpl[8];
>> data.Nlon = gfld->igdtmpl[7];
>> data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0)
- (double)data.Nlat * data.delta_lat;
>> data.lon_ll = 360 - (double)gfld->igdtmpl[12] /
1000000.0;
>> ..........................................................
>> Inspired by your remark about giving me a free hand to do with
codes all I wanted, I took the liberty of changing the corresponding
lines to the following ones:
>> ..........................................................
>> // *********************muravev 20121003
>> // build a LatLonData struct with the projection
information
>> LatLonData data;
>> data.name = latlon_proj_type;
>> data.delta_lon = (double)gfld->igdtmpl[16] / 1000000.0;
>> data.delta_lat = (double)gfld->igdtmpl[17] / 1000000.0;
>> data.Nlat = gfld->igdtmpl[8];
>> data.Nlon = gfld->igdtmpl[7];
>> data.lat_ll = ((double)gfld->igdtmpl[11] / 1000000.0);
>> data.lon_ll = 0.0 - (double)gfld->igdtmpl[12] /
1000000.0;
>> ..........................................................
>> You know, after the re-installation of the package, everything was
OK and in full accord with what I had in the grib1 case (taking into
account the two grids defined with slightly different delta_lons)!
>> I would be interested to know what you think about all of the above
and look forward to your feedback.
>> Best regards,
>> Anatoly
>> JHGvR> Anatoly,
>> JHGvR> Very good questions indeed. After reading your email, I
>> JHGvR> stepped through the same process you went through, using the
>> JHGvR> MET test data to see if the results differed for GRIB1 vs
GRIB2. However, the big
>> JHGvR> difference is that I used the "cnvgrib" utility to do the
>> JHGvR> conversion from GRIB1 to GRIB2
>> JHGvR> (http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/). But that
>> JHGvR> conversion errored out after processing the first 36
>> JHGvR> GRIB records. It died on the 37th record, which contains
UGRD data.
>> JHGvR> So I ran the Point-Stat test script on both the GRIB1 and
>> JHGvR> GRIB2 versions of the file, but only after removing the
>> JHGvR> UGRD/VGRD verification from the Point-Stat config file. And
the output of Point-Stat
>> JHGvR> was identical for GRIB1 and GRIB2.
>> JHGvR> But some obvious questions remain:
>> JHGvR> (1) Why did cnvgrib error out on record number 37?
>> JHGvR> I'm not sure and we are not funded to provide support
for that utility.
>> JHGvR> (2) Why do you see the difference when converting from GRIB1
to 2 using grib_convert?
>> JHGvR> I downloaded and built the ECMWF GRIB tools. And I ran
>> JHGvR> the same conversion you did using the grib_convert tool.
And I
>> JHGvR> was able to reproduce the problem you're seeing in the
output of Point-Stat.
>> JHGvR> So I took a step back and just plotted the same record
from
>> JHGvR> the GRIB1 and GRIB2 files, TMP at 750mb. Listed below if
the
>> JHGvR> command I used for plot_data_plane. I've attached two
images showing those
>> JHGvR> plots.
>> JHGvR> $MET_BASE/bin/plot_data_plane \
>> JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2 \
>> JHGvR> nam.t00z.awip1236.tm00.20070330.grib_convert.grb2.ps \
>> JHGvR> 'name="TMP"; level="P750";'
>> JHGvR> Please take a look at those files and you'll notice a big
>> JHGvR> discrepancy in how the grid is defined. For some reason,
the
>> JHGvR> grib_convert utility is doing something odd to the grid
specification. Listed
>> JHGvR> below is the output of wgrib (for GRIB1) and wgrib2 (for the
>> JHGvR> output of grib_convert). As you can see, there is something
>> JHGvR> very wrong with the "LoV -2052.483648" setting in the wgrib2
output.
>> JHGvR> So there's your answer. The problem is coming from the GRIB
1
>> JHGvR> -> 2 conversion tool that you're using.
>> JHGvR> Hope that helps.
>> JHGvR> John
>> JHGvR> GRIB1 output:
>> JHGvR> rec 21:5060624:date 2007033000 TMP kpds5=11 kpds6=100
>> JHGvR> kpds7=750 levels=(2,238) grid=218 750 mb 36hr fcst:
>> JHGvR> TMP=Temp. [K]
>> JHGvR> timerange 0 P1 36 P2 0 TimeU 1 nx 614 ny 428 GDS grid 3
num_in_ave 0 missing 0
>> JHGvR> center 7 subcenter 0 process 84 Table 2 scan: WE:SN
winds(grid)
>> JHGvR> Lambert Conf: Lat1 12.190000 Lon1 -133.459000 Lov
-95.000000
>> JHGvR> Latin1 25.000000 Latin2 25.000000 LatSP 0.000000
LonSP 0.000000
>> JHGvR> North Pole (614 x 428) Dx 12.191000 Dy 12.191000 scan
64 mode 8
>> JHGvR> min/max data 248 289 num bits 6 BDS_Ref 248 DecScale 0
BinScale 0
>> JHGvR> GRIB_CONVERT output:
>> JHGvR> 21:5062554:vt=2007033112:750 mb:36 hour fcst:TMP Temperature
[K]:
>> JHGvR> ndata=262792:undef=0:mean=273.101:min=248:max=289
>> JHGvR> grid_template=30:winds(grid):
>> JHGvR> Lambert Conformal: (614 x 428) input WE:SN output
WE:SN res 8
>> JHGvR> Lat1 12.190000 Lon1 226.541000 LoV -2052.483648
>> JHGvR> LatD 60.000000 Latin1 25.000000 Latin2 25.000000
>> JHGvR> LatSP 0.000000 LonSP 0.000000
>> JHGvR> North Pole (614 x 428) Dx 12191.000000 m Dy
12191.000000 m mode 8
>> JHGvR> On 09/10/2012 05:59 AM, muravev at mecom.ru via RT wrote:
>>>> Mon Sep 10 05:59:40 2012: Request 58172 was acted upon.
>>>> Transaction: Ticket created by muravev at mecom.ru
>>>> Queue: met_help
>>>> Subject: METv4.0 problems
>>>> Owner: Nobody
>>>> Requestors: muravev at mecom.ru
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=58172 >
>>>> Dear John,
>>>> I am now in new troubles with MET and would like to ask your help
after having changed for the new v4.0 package. Three small directories
are attached to better understand my problems.
>>>> A. RUS_DEMO_METv3.1 directory.
>>>> The grib1 increment rounding was the previous trouble with
METv3.1 though everything else went normally, including
‘gen_poly_mask’, ‘plot_point_obs’ and ‘point_stat’ steps.
>>>> Some results are given in RUS_DEMO_METv3.1 with the corresponding
run-script.
>>>> Station positioning (very tiny red points though) near Sochi
region in MET_01_TA_2012-08-01.pdf and numbers in
‘point_stat_030000L_20120801_030000V.stat’ in the ‘out’-directory
showed that we are on the right track.
>>>> B. RUS_DEMO_METv4.0 directory.
>>>> The grib1 lat-lon increments were unacceptable for 1x1-km model,
and we decided to reformat all files to grib2 with the ECMWF
grib_convert procedure. Some additional testing showed that this
procedure gave identical results on both data formats of our COSMO
model.
>>>> But all METv4.0 steps were discouraging as given in
RUS_DEMO_METv4.0.
>>>> 1. No lat-lon INNER boarders were processed correctly by the
‘gen_poly_mask’ and only enveloping coordinates (as in SOCHI.poly)
collected points inside the region as shown in ‘MET_01_TA_2012-08-
01.plot_point.log’.
>>>> 2. That something goes wrong is seen in the coastline drawn by
‘plot_point_obs’. It seems as if lats and lons have been mixed up
somewhere (MET_01_TA_2012-08-01.pdf).
>>>> 3. Together with the grid specification trouble I cannot
understand the forecast lead time definition in ‘PointStatConfig’,
which was no problem in v3.1 with hourly forecast fields gathered in
separate variable-files.
>>>> C. USA_DEMO_METv3.1-4.0 directory.
>>>> Since several attempts to realize the error source failed, I
decided to test the presentation WRF data in the METv4.0 data storage.
>>>> I took one model data file, reformatted it to grib2 and ran two
parallel calculations with METv4.0/point_stat tool. Table statistics
in ‘test_point_stat_01’ refer to grib1 and in ‘test_point_stat_01’
refer to grib2. THEY ARE DIFFERENT, so I realized that I missed
something very important from the very beginning.
>>>> It is worth noting that all formatted RUS and US data are
diagnosed with wgrib and wgrib2 procedures and included into
corresponding directories. Data files in directories attached are
given as links.
>>>> I would be very much obliged if you could explain my faults and
misunderstanding in METv4.0 application for the Sochi region.
>>>> Thank you in advance.
>>>> Best regards,
>>>> Anatoly
>>>> ........................................
>>>> Dr. Anatoly Muravyev
>>>> THE HYDROMETEOROLOGICAL RESEARCH CENTRE
>>>> OF THE RUSSIAN FEDERATION
>>>> 11-13, Bolshoi Predtechensky Per.
>>>> 123242, Moscow, Russia
>>>> Phone: + 7-499-7952127
>>>> Fax: + 7-499-2551582
>>>> E-mail: muravev at mecom.ru
>>>> ........................................
------------------------------------------------
More information about the Met_help
mailing list