[Met_help] Clarification on data ingest by the MODE tool (METv2.0)

[John Halley Gotway]

Matthew,

Glad to help.  Let us know if any questions come up in your analysis of the data in the Northern Hemisphere.  And when you start looking in to the Southern Hemisphere data, please check back in with
us to find out where we are in supporting Polar Stereographic projections the Southern Hemisphere.

Thanks,
John

Matthew Souders wrote:
> John...
> 
> I just want to thank you for going the extra mile to clarify some of the
> inner workings of MODE and look into some of the issues with polar
> stereographic projections.  I intend to start with the Northern Hemisphere
> first to make sure the method for wave packet tracking actually works...if I
> can complete that part of the analysis quickly enough I'll be looking at the
> southern Hemisphere but we do still have some time to figure out the strange
> behavior of MODE over this area.
> 
> It is a major relief that MODE will support polar stereographic mapping and
> analysis...because that is the only logical way to track wave packets and
> storm track behavior.  I am going to spend some time looking through some of
> the examples you've linked...if I have further questions, I will let you
> know.  I greatly appreciate your efforts in helping me get this project off
> the ground.
> 
> I do notice that there's a bit of miscommunication on the grid spacing I
> have at the moment...but I can look through the NCEP grid definitions and
> find the correct spacing hopefully...I'm actually starting with NCEP
> Reanalysis scales (2.5 by 2.5 degree resolution)...and future research will
> incorporate archived GFS forecasts (1.0 by 1.0 degree resolution)...so for
> the moment, I need to start with Reanalysis grid spacing.  Not a big
> deal...just wanted to clarify the progression.
> 
> Matthew Souders
> Stony Brook University
> School of Marine and Atmospheric Sciences
> 
> On 1/28/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>> Matthew,
>>
>> Please take a look at the attached file "gfs_G027_new.png".  By changing a
>> couple lines of code in the attached "mode.cc" file, I was able to get the
>> plotting to look a lot better.  The plot now
>> matches what is shown for this grid by NCEP:
>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid027.gif
>>
>> Unfortunately though, when I run on NCEP grid 028, I'm not able to get it
>> to match NCEP's plot:
>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid028.gif
>>
>> In talking with the MET developer who works mostly on grid support, I don't
>> think we fully support polar stereographic grids over the southern
>> hemisphere.  However, we are working to add support for
>> them in the release this spring.  It's not clear to me whether the numbers
>> are in the MODE output are correct or not, and the plots are just
>> wrong?  I'd suggest running it, looking at the output, and
>> seeing if it makes sense or not.  Sorry for the trouble!
>>
>>
>> John
>>
>>
>> John Halley Gotway wrote:
>>> Matthew,
>>>
>>> MET is able to read Polar Stereographic grids.  I spent some time on this
>> this morning, and here's what I found...
>>> (1) I ran some GFS 6-hour precip data (NCEP Grid 003) through MODE.  To
>> simplify things, I just compared the same field to itself (i.e. forecast
>> field = observation field).  See attached file
>>> "gfs_G003.png"
>>>
>>> (2) I used the copygb tool to regrid the data to NCEP grids 027 and 028
>> (Northern and Southern Hemisphere polar stereographic) with the following
>> commands:
>>> copygb -xg 27 gfs_G003.grb gfs_G027.grb
>>> copygb -xg 28 gfs_G003.grb gfs_G028.grb
>>>
>>> (3) I ran the G027 data through MODE to compare it to itself: See
>> attached file "gfs_G027.png"
>>> (4) I ran the G028 data through MODE to compare it to itself: See
>> attached file "gfs_G028.png"
>>> So I do think that interpolating to northern and southern hemisphere
>> polar stereographic grids is a good idea.  I'd recommend looking through the
>> pre-define NCEP grids
>>> (http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html) to see if
>> there are any grids that fit your needs.  If not, it easy to manually define
>> your own grids to use.  Take a look at the practical
>>> sessions we've added to the MET tutorial about running copygb:
>>>
>> http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_210/copygb/index.php
>>> In particular, take a look at how you define a polar stereographic grid:
>>>
>> http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_210/copygb/run4.php
>>> Also, I'd suggest reading up about the interpolation options for copygb:
>>> http://www.cpc.noaa.gov/products/wesley/copygb.html
>>> and
>>>
>> http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_210/copygb/copygb.txt
>>> It's up to you how you'd like to perform the interpolations.
>>>
>>> At this point, I think you're set.  Except, you may have noticed that the
>> MODE output plots for G027 and G028 don't look very good.  I think we have a
>> problem in how MODE decided what country outlines
>>> to plot - in particular when the grid includes the poles.  I'll take a
>> look at the code and see if I can figure out a fix.
>>> Hope that helps.
>>>
>>> John
>>>
>>>
>>> Matthew Souders wrote:
>>>> I am studying upper level meridional wind patterns (a phenomenon called
>> wave
>>>> packets)...which, unfortunately, are interesting everywhere,
>>>> longitudinally.  HOWEVER...they aren't interesting in the tropics (the
>> storm
>>>> tracks never cross the equator and rarely get south of 20 N), which does
>>>> provide me with the opportunity to use copygb to regrid the data to a
>> pair
>>>> of polar stereographic projections for each hemisphere (N and S)...I
>> assume
>>>> copygb can be used to accomplish this...what confuses me, however, is
>> that a
>>>> polar stereographic projection is circular...I was under the impression
>> that
>>>> MODE needed a rectangular (regular) grid.  Is there a variable in the
>> config
>>>> file for MODE that lets you tell it it's looking at a circular grid and
>> a
>>>> grid box is more like a chunk of a wedge?  If it can handle polar
>>>> stereographic, then the wrapping problem is completely solved.
>>>>
>>>> I appreciate your assistance in this matter...I just want to be sure I'm
>>>> preparing my data in a beneficial way before I get too far into working
>> with
>>>> MODE.
>>>>
>>>> Matthew Souders
>>>> Stony Brook University
>>>> School of Marine and Atmospheric Sciences
>>>>
>>>> On Wed, Jan 27, 2010 at 4:50 PM, John Halley Gotway <
>> johnhg at rap.ucar.edu>wrote:
>>>>> Matthew,
>>>>>
>>>>> You don't need to cut it in half along a longitude yourself.  I was
>> just
>>>>> pointing out the fact that you already knew about - objects straddling
>> the
>>>>> prime meridian will be split in two.  That's all I was pointing out.
>>>>>
>>>>> I believe the answer to your question about expanding by 50 degrees is
>> no,
>>>>> the code couldn't handle that.  I can't be sure without playing with a
>>>>> sample data file, but I think we'd get an error in the subroutine that
>>>>> converts lat/lon's to x/y values.  Since there would be duplicate
>>>>> lat/lon's on the edges, the x/y values wouldn't be well-defined.
>>>>>
>>>>> I'm not sure exactly what you're studying, but what other people have
>> done
>>>>> in the past is chosen a longitude value that isn't very interesting for
>>>>> their problem - perhaps the center of the pacific if you're studying
>>>>> land-based phenomena, or the center or africa if you're studying
>>>>> water-based phenomena.  Then, you could use the copygb tool to regrid
>> the
>>>>> data from NCEP Grid 003 to one you define that wraps at the longitude
>> you
>>>>> select rather than the prime meridian.
>>>>>
>>>>> Another option would be to use the copygb tool to regrid the data into
>>>>> several smaller grids - perhaps a polar stereographic one for the
>> northern
>>>>> hemisphere and another one for the southern hemisphere.  Run MODE on
>> all
>>>>> the subdomains, and try to aggregate the results.
>>>>>
>>>>> But how you attack this problem really depends on what you're
>>>>> investigating.
>>>>>
>>>>> Hope that helps.
>>>>>
>>>>> John
>>>>>
>>>>>> Thanks for the quick reply.  This confirms my basic understanding that
>>>>>> distance calculations are in grid box counts within the tool
>> itself...and
>>>>>> that I'm going to run into a big problem near the poles where the dx
>>>>>> between
>>>>>> two longitudes begins to shrink considerably (which will have the
>> affect
>>>>>> of
>>>>>> rapidly increasing the "importance" of objects at polar latitudes.
>>>>>>
>>>>>> Now I am curious...why does a circumpolar grid need to be split in
>> half
>>>>> by
>>>>>> longitude?  I was going to run into the problem of split objects
>> anyway
>>>>>> even
>>>>>> without cutting the planet in half (because an object crossing the
>> prime
>>>>>> meridian goes out the right hand side and back into the left hand side
>> so
>>>>>> may appear to be two objects)...my proposed solution to this problem
>>>>> would
>>>>>> be falsely expand the grid by adding (say) 50 degrees of duplicated
>> data
>>>>>> to
>>>>>> one side or the other (instead of going -180 to 0, perhaps go -180 to
>> 50)
>>>>>> and when I later process the objects, throw out duplicates.  Can the
>>>>>> program
>>>>>> read in a data set like that, though?  Stretching beyond the 180
>> degree
>>>>>> limit?
>>>>>>
>>>>>> On 1/27/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>>>>>>> Matthew,
>>>>>>>
>>>>>>> It sounds to me like there's a few questions here...
>>>>>>>
>>>>>>> (1) How does MODE handle distances between grid points?
>>>>>>>
>>>>>>> MODE does all it's computations in the grid units, not actual
>> physical
>>>>>>> distances.  So the area of an object is just a count of the grid
>> boxes
>>>>>>> which
>>>>>>> comprise it.  And the distance between two objects
>>>>>>> is just a count of grid units between the centers of the
>> objects.  The
>>>>>>> parameter you're seeing in the configuration file "grid_res" actually
>>>>>>> isn't
>>>>>>> used all that much.  Actually, the only place it's
>>>>>>> used is further down in the configuration file to set reasonable
>>>>>>> defaults
>>>>>>> for other parameters.  But you could read about those other
>> parameters
>>>>>>> and
>>>>>>> set them how you'd like.
>>>>>>>
>>>>>>> (2) What type of projections can the MET tools read?
>>>>>>>
>>>>>>> MODE (as well of the other MET tools) can read data that's on
>> lat/lon,
>>>>>>> lambert conformal, polar stereographic, or mercator projections.  It
>>>>>>> sounds
>>>>>>> like your data is on a global lat/lon projection,
>>>>>>> which may work.  There are a couple of big issues with this though.
>>>>>>> First,
>>>>>>> the size of each grid box on a global lat/lon grid will vary
>>>>>>> considerably.  However, MODE will count them all the same.
>>>>>>> Depending on your data, if you want to treat objects near the poles
>> the
>>>>>>> same as objects nears the equator, this will be problematic.  Second,
>>>>>>> you'll
>>>>>>> need to choose a longitude at which to cut the grid
>>>>>>> in half - perhaps the prime meridian for example.  But if you have an
>>>>>>> object that straddles that line, MODE will treat it as two separate
>>>>>>> objects.  It isn't set up to handle true global data in that way.
>>>>>>>
>>>>>>> I'm guessing the NCEP/NCAR reanalysis 1.0x1.0 degree GFS data is on
>> NCEP
>>>>>>> Grid 3:
>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/grids/grid003.gif
>>>>>>> If that's the case, MODE should be able to read data on that grid.
>>>>>>>
>>>>>>> (3) What data format should I use?
>>>>>>>
>>>>>>> You should use GRIB.  That'd be easiest.  And I'm guessing you'd be
>> able
>>>>>>> to
>>>>>>> get this data in GRIB format.
>>>>>>>
>>>>>>> Hope that helps.  Please let us know if more questions come up, and
>> if
>>>>>>> you're having difficulty, feel free to send some sample data files
>> and
>>>>>>> your
>>>>>>> MODE config file, and we'd be happy to take a look.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> John Halley Gotway
>>>>>>> johnhg at ucar.edu
>>>>>>>
>>>>>>>
>>>>>>> Matthew Souders wrote:
>>>>>>>> I am seeking a bit of clarification about how exactly the MODE tool
>>>>>>> ingests
>>>>>>>> gridded data and accounts for distances between grid points. My data
>>>>>>> sets
>>>>>>>> are both regular lat/lon grids (not regular dx/dy with constant
>>>>>>> spacing)
>>>>>>> and
>>>>>>>> have global coverage (not a region like the WRF might)...I am
>> working
>>>>>>> with
>>>>>>>> NCEP/NCAR Reanalysis and GFS forecast fields from the archived
>> 1.0X1.0
>>>>>>>> degree grids. I need to know whether MODE can correctly interpret
>> real
>>>>>>>> distances from GRIB files in regular lat/lon format or whether I'd
>>>>>>> somehow
>>>>>>>> need to convert regular lat/lon to regular dx/dy (a process which
>>>>>>> would
>>>>>>> be
>>>>>>>> impossible if I were covering the entire globe since dx changes with
>>>>>>>> increasing latitude).
>>>>>>>>
>>>>>>>> >From what I can tell from the User's Guide for METv2.0, I can use
>> any
>>>>>>>> gridded data, but I have to make some kind of rough estimate as to
>> the
>>>>>>>> typical grid spacing. But lat/lon grids aren't regularly spaced
>> enough
>>>>>>> for
>>>>>>>> that to make sense. Any clarification you might be able to provide
>> on
>>>>>>> how
>>>>>>>> best to work with globally-covering regular lat/lon grids would be
>>>>>>>> appreciated.
>>>>>>>>
>>>>>>>> Matthew Souders
>>>>>>>> Stony Brook University
>>>>>>>> School of Marine and Atmospheric Sciences
>>>>>>>>
>>>>>>>>
>>>>>>>>
>> ------------------------------------------------------------------------
>>>>>>>> _______________________________________________
>>>>>>>> Met_help mailing list
>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>
>