[Met_help] Re: MET -- grid conversion on GFS
John Halley Gotway
johnhg at rap.ucar.edu
Mon Feb 11 12:48:46 MST 2008
Luke,
I'm really not familiar with GDAS data. However, a while back I wrote a simple tool to read PrepBufr files and create a plot showing the locations of the messages.
If you'd like to point me to the GDAS file you're trying to use, I'd be happy to run the tool on it and send you a plot of the message locations.
It could be that there aren't any messages over your domain. Or it's possible that you may unintentionally be filtering them out by the way you've set up the PB2NC config file.
So just send me the file (or point me to it on the web), and the version of the PB2NC config file you're using.
Thanks,
John
Luke Peffers wrote:
> Hello John:
>
> I am having some problems using the GDAS PrepBfr files to do global
> verification. I can't seem to get any matches in my domain over Iran. Is
> there a way to see the coverage of these data sets? They are global right?
>
> Luke Peffers
>
> On Feb 6, 2008 6:43 PM, John Halley Gotway <johnhg at rap.ucar.edu> wrote:
>
>> Luke,
>>
>> Glad you were able to figure it out. Let us know if any more questions
>> come up in your use of MET.
>>
>> John
>>
>> Luke Peffers wrote:
>>> FYI John:
>>>
>>> I got it! The raw binary data that I get is indeed written in the
>> reverse
>>> order in the y-direction. When I created the GRIB files from those flat
>>> binary files, the GRADS ctl file that I used to feed into LATS had the
>> line;
>>> "options yrev" which tells GRADS to display the data in the reverse
>> order in
>>> the y-direction while the actual GRIB data are written as you and I see
>> in
>>> MET. I now create the files without the "yrev" option and it works
>> great!!
>>> Thanks for all the help.
>>>
>>> Luke Peffers
>>> FSU Met.
>>>
>>> On Feb 5, 2008 3:59 PM, John Halley Gotway <johnhg at rap.ucar.edu> wrote:
>>>
>>>> Luke,
>>>>
>>>> Good idea using the "mask_missing_flag". I forgot about that.
>>>>
>>>> As for the data being flipped about the equator, I thought it could
>> either
>>>> be a problem with the MET code or a problem with the way the data is
>> packed
>>>> into the Grib file.
>>>>
>>>> So I tried using an "independent" reader of the grib file: NCL (NCAR
>>>> Command Language). I don't know if you've used NCL before, but one of
>> the
>>>> tools associated with it is called "ncl_convert2nc". I
>>>> used it to convert the cmorph file into NetCDF. Then I used "ncview"
>> to
>>>> display that NetCDF file.
>>>>
>>>> I've attached the NetCDF output of "ncl_convert2nc" and an image
>> showing
>>>> how ncview displays it. Based on that, I'd say that the data must not
>> be
>>>> packed correctly in the Grib file. As for what might
>>>> be wrong, it may have to do with the "scanning mode flag". If that's
>> set
>>>> incorrectly in the GDS section of the Grib file, then you might see
>>>> something like this flip about the equator:
>>>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/table8.html
>>>>
>>>> I tried doing the following wgrib command on the cmorph file:
>>>> wgrib -GDS10 cmorph_3h_20070619_00z.grb
>>>>
>> 1:0:d=07061900:PRATE:kpds5=59:kpds6=1:kpds7=0:TR=0:P1=0:P2=0:TimeU=1:sfc:anl:NAve=0:GDS10=
>>>> 0 0 32 0 255 0 5 160 1 224 128 233 227 0 0 125 128 0 233 227 5 125 195
>> 0 250
>>>> 0 250 2 32 50 48 48
>>>>
>>>> The 28th value in this list after "GDS10=" (octet 28) is set to a value
>> of
>>>> 2. That means it's scanning in the "+i" direction and "+j" direction.
>>>> Maybe it should be set to 0 for scanning in the "+i"
>>>> and "-j" direction.
>>>>
>>>> John
>>>>
>>>> Luke Peffers wrote:
>>>>> Ok, I decided that converting my WRF to the cmorph data and using the
>>>>> "mask_missing flag" will do the trick. I did so and was surprised to
>>>> find
>>>>> that there was a large squall line running through the center if Iran,
>>>>> something that doesn't at all exist in either my WRF forecast or the
>>>> cmorph
>>>>> data that we used for comparison. After closer inspection I found
>> that
>>>> the
>>>>> squall line does exist but in the southern hemisphere at the same
>>>>> longitude. Somehow, mode seems to have inverted the observation data
>>>> about
>>>>> the equator! The same happened with the data that you sent me, so If
>>>> you
>>>>> take a look at the obs field, you'll see that the precipitation
>> patterns
>>>>> don't really match up with the terrain. I have attached a GRADS image
>>>> of
>>>>> the way the grid should look.
>>>>>
>>>>> What do you think is the problem here?
>>>>>
>>>>> Luke Peffers
>>>>> FSU Met.
>>>>>
>>>>> On Feb 5, 2008 2:09 PM, John Halley Gotway <johnhg at rap.ucar.edu>
>> wrote:
>>>>>> Luke,
>>>>>>
>>>>>> I was able to get it to work. However, after looking at your data,
>> I'd
>>>>>> strongly recommend that you regrid the cmorph data to your WRF grid
>>>> before
>>>>>> running MODE.
>>>>>> Comparing a global dataset to a relatively small domain, you'll end
>> up
>>>>>> with MANY MANY objects in the global domain and relatively few in
>> your
>>>>>> smaller domain. And it'll take MODE way too long to
>>>>>> handle all of those objects. In the sample case I ran, I had 6
>>>> forecast
>>>>>> objects and 133 observation objects!
>>>>>>
>>>>>> Then I converted the cmorph data to the WRF grid, and ran MODE that
>>>> way.
>>>>>> However, there's a big problem. The copygb command which converts
>>>> cmorph
>>>>>> to the WRF grid runs fine but creates a field of all
>>>>>> zeros. By looking at the cmorph data, I see that it should contain
>>>>>> non-zero values there, but for some reason it's all zeroed out. I
>> have
>>>> no
>>>>>> idea why.
>>>>>>
>>>>>> But here's the commands I ran:
>>>>>> // Convert WRF to cmorph domain
>>>>>> copygb -xg '255 0 1440 480 -59875 125 128 60125 125 250 250 64'
>>>>>> WRFPRS_d01.96 WRFPRS_d01.96_cmorph_grid
>>>>>> // Run MODE on cmporph grid (takes about 4 minutes to run)
>>>>>> METv1.0/bin/mode WRFPRS_d01.96_cmorph_grid cmorph_3h_20070619_00z.grb
>>>>>> WrfModeConfig_PRATE_cmorph_grid -outdir out_cmorph_grid
>>>>>>
>>>>>> // Convert cmorph to WRF grid
>>>>>> copygb -xg '255 3 109 109 23688 37515 8 50000 24000 24000 0 64 36000
>>>>>> 36000' cmorph_3h_20070619_00z.grb cmorph_3h_20070619_00z.grb_wrf_grid
>>>>>> // Run MODE on WRF grid (takes 2 seconds to run)
>>>>>> METv1.0/bin/mode WRFPRS_d01.96 cmorph_3h_20070619_00z.grb_wrf_grid
>>>>>> WrfModeConfig_PRATE_wrf_grid -outdir out_wrf_grid
>>>>>>
>>>>>> I've tarred up the relevant files and posted them to the ftp site:
>>>>>> ftp.rap.ucar.edu/incoming/irap/johnhg/for_luke.tar.gz
>>>>>>
>>>>>> Good luck,
>>>>>> John
>>>>>>
>>>>>> Luke Peffers wrote:
>>>>>>> Ok, you now have WRF forecast valid on June 19th, 2007 and cmorph
>>>>>>> precipitation rate data valid at the same time. Recall that I
>> created
>>>>>> the
>>>>>>> precip data using LATS from a flat binary dataset (that may be where
>> a
>>>>>>> problem resides). If you'd like, I can ftp you the original binary
>>>>>> data,
>>>>>>> otherwise, if you're interested here is the website for those data.
>>>>>>>
>>>>>>>
>> http://www.cpc.ncep.noaa.gov/products/janowiak/cmorph_description.html
>>>>>>> Thank you
>>>>>>>
>>>>>>> Luke Peffers
>>>>>>> FSU Met.
>>>>>>>
>>>>>>> On Feb 5, 2008 12:00 PM, John Halley Gotway <johnhg at rap.ucar.edu>
>>>> wrote:
>>>>>>>> Luke,
>>>>>>>>
>>>>>>>> Would you mind posting on of your forecast files (and the
>>>> corresponding
>>>>>>>> obs) to the ftp site here at NCAR? I'd just like to play around
>> with
>>>>>> it to
>>>>>>>> see if I can figure it out. I imagine that we'll
>>>>>>>> continue getting copygb related questions through MET_HELP and this
>>>>>> would
>>>>>>>> give me a chance to learn more about it.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> John
>>>>>>>>
>>>>>>>> Luke Peffers wrote:
>>>>>>>>> Thank you John:
>>>>>>>>>
>>>>>>>>> I have been using the template from the noaa site but just
>> couldn't
>>>>>> seem
>>>>>>>> to
>>>>>>>>> get the correct numbers to enter into it. I used the parameters
>>>> that
>>>>>>>> you
>>>>>>>>> provided me on my WRF forecast data and when I try to view the new
>>>>>>>> copygb'd
>>>>>>>>> data, all variables are undefined. I think I will just work on
>> this
>>>>>>>> problem
>>>>>>>>> for a while and see if I can figure out what I'm doing wrong.
>> I'll
>>>>>> let
>>>>>>>> you
>>>>>>>>> know what I find if you'd like.
>>>>>>>>>
>>>>>>>>> Thanks for all the help.
>>>>>>>>>
>>>>>>>>> Luke
>>>>>>>>>
>>>>>>>>> On Feb 4, 2008 6:26 PM, John Halley Gotway <johnhg at rap.ucar.edu>
>>>>>> wrote:
>>>>>>>>>> Luke,
>>>>>>>>>>
>>>>>>>>>> There's an example listed on the copygb website for how to
>> declare
>>>> a
>>>>>>>>>> lat/lon grid:
>>>>>>>>>> http://www.cpc.ncep.noaa.gov/products/wesley/copygb.html
>>>>>>>>>>
>>>>>>>>>> Here's the template from the webiste:
>>>>>>>>>> grid='255 0 NX NY LAT0 LON0 128 LAT1 LON1 DX DY 64'
>>>>>>>>>> copygb -g"$grid" -x in.grb out.grb
>>>>>>>>>>
>>>>>>>>>> And I believe that this is what you should use:
>>>>>>>>>> grid='255 0 1440 480 -59875 125 128 60125 125 250 250 64'
>>>>>>>>>>
>>>>>>>>>> Let me know if that does the trick for you.
>>>>>>>>>>
>>>>>>>>>> John
>>>>>>>>>>
>>>>>>>>>> Luke Peffers wrote:
>>>>>>>>>>> Hello John:
>>>>>>>>>>>
>>>>>>>>>>> I ftp'd the grib file to you along with the .ctl and .gmp files
>>>> just
>>>>>>>> in
>>>>>>>>>>> case. I can supply you with the original flat binary file I
>> used
>>>> to
>>>>>>>>>> create
>>>>>>>>>>> this grib file if you would like. As I said, I used LATS to
>>>> create
>>>>>>>>>> these
>>>>>>>>>>> files. They work with GRADS well so I'm sure the grib format is
>>>>>>>>>> correct.
>>>>>>>>>>> Thanks for the help.
>>>>>>>>>>>
>>>>>>>>>>> Luke
>>>>>>>>>>>
>>>>>>>>>>> On Feb 4, 2008 4:01 PM, John Halley Gotway <johnhg at rap.ucar.edu>
>>>>>>>> wrote:
>>>>>>>>>>>> Luke,
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately, that didn't have in it what I was hoping for.
>>>>>>>>>>>>
>>>>>>>>>>>> As for decoding the value in the dump of the grib file GDS
>>>> section,
>>>>>>>>>> it's a
>>>>>>>>>>>> pretty difficult thing to do. If I remember correctly, I think
>>>> we
>>>>>>>>>> actually
>>>>>>>>>>>> ran the Grib file through PCP-Combine to pull out
>>>>>>>>>>>> the grid definition, rather than through Grid-Stat. But the
>>>>>> version
>>>>>>>> of
>>>>>>>>>>>> PCP-Combine you have only runs on grib code 61 (for APCP). I
>> see
>>>>>>>> that
>>>>>>>>>>>> you're running on grib code 59 for precipitation rate.
>>>>>>>>>>>>
>>>>>>>>>>>> If you're grib data has 61 in it, you could run it through
>>>>>>>> PCP-Combine
>>>>>>>>>> and
>>>>>>>>>>>> look at the contents of the output NetCDF file to see the grid
>>>>>>>>>> definition.
>>>>>>>>>>>> If not, go ahead and send me one of your grib
>>>>>>>>>>>> files, and I'll figure out what the definition should be.
>>>>>>>>>>>>
>>>>>>>>>>>> John
>>>>>>>>>>>>
>>>>>>>>>>>> Luke Peffers wrote:
>>>>>>>>>>>>> Sure, here is the log file.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Luke
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Feb 4, 2008 2:58 PM, John Halley Gotway <
>> johnhg at rap.ucar.edu>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Luke,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do me a favor. Run Grid-Stat again using the command line
>>>> option
>>>>>>>> "-v
>>>>>>>>>>>> 3"
>>>>>>>>>>>>>> and redirect the entire output to a log file. Then please
>> send
>>>>>> me
>>>>>>>>>> the
>>>>>>>>>>>> log
>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For example...
>>>>>>>>>>>>>> ./grid_stat fcst_file obs_file config_file -v 3 >&
>>>> grid_stat.log
>>>>>>>>>>>>>> There's another section of the output I'd like to see, and
>> then
>>>> I
>>>>>>>>>> think
>>>>>>>>>>>>>> it'll be pretty easy to help you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Luke Peffers wrote:
>>>>>>>>>>>>>>> Hello John:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have been working on creating a precipitation dataset for
>>>> MET
>>>>>>>>>>>>>> comparison.
>>>>>>>>>>>>>>> I was able to create convert a global satellite-derived
>>>>>>>>>> precipitation
>>>>>>>>>>>>>> set to
>>>>>>>>>>>>>>> GRIB1 format using LATS. I ran it through grid_stat in
>> order
>>>> to
>>>>>>>>>>>> extract
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> grid information so I would be able to regrid my WRF
>> forecast
>>>>>> data
>>>>>>>>>> to
>>>>>>>>>>>>>> match
>>>>>>>>>>>>>>> the precip data grid. However, I can't seem to get copygb
>> to
>>>>>> work
>>>>>>>>>>>>>> correctly
>>>>>>>>>>>>>>> when I input my WRF forecast (i.e., all zero values). I
>>>> recall
>>>>>>>> that
>>>>>>>>>>>> you
>>>>>>>>>>>>>>> were able to help me regrid my WRF to a GFS-like sub domain
>>>>>>>> through
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> above mentioned method. I have attached the information for
>>>> my
>>>>>>>>>> precip
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>> that was output from grid_stat below. Will you be able to
>>>> help
>>>>>> be
>>>>>>>>>> put
>>>>>>>>>>>>>>> together a copygb command that will take my WRF forecast and
>>>> put
>>>>>>>> it
>>>>>>>>>> on
>>>>>>>>>>>> a
>>>>>>>>>>>>>>> grid similar to the precip data. Since the precip data is
>> on
>>>> a
>>>>>>>>>> large
>>>>>>>>>>>>>>> near-global domain, I think I will just use the
>>>> plot_valid_flag
>>>>>>>>>> option
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> zoom in instead of regridding my precip data as well. Please
>>>> let
>>>>>>>> me
>>>>>>>>>>>> know
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> I need to direct this question to another location.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank you for your time.
>>>>>>>>>>>>>>> Luke Peffers
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Output from Grid_Stat:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> length: 32
>>>>>>>>>>>>>>> nv: 0
>>>>>>>>>>>>>>> pvpl: 255
>>>>>>>>>>>>>>> type: 0
>>>>>>>>>>>>>>> nx: 1440
>>>>>>>>>>>>>>> ny: 480
>>>>>>>>>>>>>>> lat1: 8448483
>>>>>>>>>>>>>>> lon1: 125
>>>>>>>>>>>>>>> res_flag: 128
>>>>>>>>>>>>>>> lat2: 59875
>>>>>>>>>>>>>>> lon2: 359875
>>>>>>>>>>>>>>> di: 250
>>>>>>>>>>>>>>> dj: 250
>>>>>>>>>>>>>>> scan_flag: 2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Nov 1, 2007 3:29 PM, Luke Peffers <luke.peffers at gmail.com
>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hello John:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am have some difficulty verifying my WRF model run with
>> GFS
>>>>>>>>>> gridded
>>>>>>>>>>>>>>>> date. I would like to follow the general rule that when
>>>>>>>> comparing
>>>>>>>>>>>> two
>>>>>>>>>>>>>>>> gridded files, one should regrid the forecast file to the
>> obs
>>>>>>>> grid,
>>>>>>>>>>>> in
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> case GFS. When I do that, mode compares my relatively
>> small
>>>>>>>> domain
>>>>>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> entire GFS (global) domain.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am, of course, running my WRF output through WRF Post
>>>>>>>> Processor.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> regrid my files using COPYGB.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have no problem regridding the GFS data to my forecast
>>>> grid
>>>>>>>>>> since
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> know the exact grid dimensions for the user defined grid
>> from
>>>>>>>> WRF.
>>>>>>>>>> I
>>>>>>>>>>>>>> don't,
>>>>>>>>>>>>>>>> however, know the exact grid codes for a GFS-like grid that
>>>> is
>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>> exact
>>>>>>>>>>>>>>>> domain size of my forecast file. Thus, I continue to use
>> MET
>>>>>> with
>>>>>>>>>> obs
>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>> regridded to match my WRF output grid file.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you know how I can resolve this problem?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you!!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Luke Peffers
>>>>>>>>>>>>>>>> FSU Meteorology Department
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>
>> ------------------------------------------------------------------------
>
More information about the Met_help
mailing list