[Met_help] [Fwd: Re: Points of clarification for data conversion/usage in MODE]

[John Halley Gotway]

Matt,

I'm forwarding your message to the "met_help at ucar.edu" email address. 
Unfortunately, I'm out of the office at all week.  I'm with my son's who's
recovering from a burst appendix.  He's recovering nicely, but we'll be
here with him all week.

Perhaps Tressa, Randy, or Paul could help you debug the issues you're
experiencing.  I should be available to help out sometime next week.

Thanks,
John

---------------------------- Original Message ----------------------------
Subject: Re: Points of clarification for data conversion/usage in MODE
From:    "Matthew Souders" <matt.souders at gmail.com>
Date:    Tue, March 16, 2010 1:29 pm
To:      "John Halley Gotway" <johnhg at ucar.edu>
--------------------------------------------------------------------------

OK...never mind about the last problem...my apologies for bothering you.  I
got PCP_Combine to run, but there is definitely something wrong with the
copygb-converted GRIB file, because I am not getting all of the northern
hemisphere when I use NCView to look at the resulting .nc file...it should
produce a 91 by 360 pole-centered grid...instead it produces a 65X65 square
centered on Italy (roughly) that only reveals most of Europe and north
Africa (and the Middle East).

The precip data looks odd too...3-hourly values with little areas of 5+
inches...hard to fathom that.

On 3/15/10, Matthew Souders <matt.souders at gmail.com> wrote:
>
> I'm probably missing something ridiculously obvious, but I am trying to run
> a quick experiment using a GFS .grb file I acquired today.  I ran the
copygb
> utility on  grid 003 GFS global domain (gave it the following command:
> copygb -xg 27 gfs_3_20100313_0000_003.grb gfs_27_20100313_0000_003.grb),
and
> that may or may have worked properly (it produced a file, but the size of
> that file was an order of magnitude smaller...which I don't think is
> right...and it gave me a handful of messages about remaining memory being
> allocated to some file called polatev0.f)...but I decided to see what it
> looked like by running the PCP_Combine utility...I got into the METv2.0
main
> directory and issued this command:
>
> bin/pcp_combine \ -sum 20100313_030000 3 20100313_030000 3 \
> test_ncfile_gfs.nc
>
> I only have a single time step in my .grb, so all this should do is add up
> one time step and re-output the data in .nc format.
>
> It's giving me this error though:
>
> ERROR: process_sum_args() -> The init_time and valid_time command line
> arguments must be in the form YYYYMMDD_HHMMSS
>
> What confuses me...is that my arguments ARE in that format.  Not sure why
> it's giving me that error.  Any ideas?
>
>
> On 3/12/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>>
>> Matt,
>>
>> Yeah, polar stereographic is fine.  Actually, *northern* polar
>> stereographic is currently supported in MET - not southern yet for some
>> reason.  You just need to figure out what to put in the NetCDF
>> header to define it.  If you can get your hands on a GRIB file, you could
>> run it through the copygb tool to interpolate it a polar stereographic
>> projection.  Then run it through the PCP-Combine tool
>> (the operation you do with PCP-Combine doesn't really matter).  And then
>> look at the NetCDF output file.
>>
>> Just let me know if you get stuck anywhere.
>>
>>
>> John
>>
>>
>> Matthew Souders wrote:
>> > Boy...it seems like no matter what alley I drive down, there's an acme
>> anvil
>> > rigged to fall on my head.  I should change my last name to Coyote.
>> >
>> > In all seriousness, I do appreciate you going through the various tools
>> and
>> > ideas with me...at this point...I am thinking it would be a less painful
>> > task to change map projection of my data BEFORE I try to create the
>> netCDF
>> > than it would be to convert it later.  I al going to have to write some
>> kind
>> > of MATLAB script to convert every piece of data I have to a projection I
>> can
>> > use...at which point I can then convert to netCDF and read the thing
>> into
>> > MODE, I assume?  Will MODE recognize a netCDF file in a polar
>> stereographic
>> > projection?
>> >
>> > On 3/12/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>> >> Matt,
>> >>
>> >> Here's a link regarding that:
>> >> http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#grib
>> >>
>> >> Converting GRIB to NetCDF is pretty easy.  The tool "ncl_convert2nc"
>> does
>> >> it for you.  Going the other way is much more difficult.  And really,
>> I've
>> >> never done it before.
>> >>
>> >>
>> >> John
>> >>
>> >> Matthew Souders wrote:
>> >>> Unfortunately, I have not used NCL before.  Given my time limmitations
>> in
>> >>> completely this analysis, I am thinking that will be
>> >> impractical.  regular
>> >>> lat/lon doesn't really work because I have to split the grid in half
>> and
>> >>> then track objects that straddle the divide somehow...which causes
>> lots
>> >> of
>> >>> problems.  There is software out there that can convert a netCDF to a
>> >> GRIB,
>> >>> isn't there?  I seem to recall that being out there in some form.
>> >>>
>> >>> On 3/12/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>> >>>> Matt,
>> >>>>
>> >>>> There is no general tool for regridding NetCDF files from one grid to
>> >>>> another.  You may be able to write an NCL (NCAR Command Language)
>> script
>> >> to
>> >>>> do so, but there's a learning curve if you haven't
>> >>>> used NCL before.
>> >>>>
>> >>>>
>> >>>> John
>> >>>>
>> >>>>
>> >>>> Matthew Souders wrote:
>> >>>>> One reason I started down the long and scary road of trying to
>> create
>> >>>> GRIB
>> >>>>> files was to gain access to the copygb utility in order to convert
>> my
>> >>>>> regular lat/lon grid to polar stereographic without too much
>> >>>> hardship.  Is
>> >>>>> there a way to convert a netCDF file in regular lat/lon to polar
>> >>>>> stereographic simply?
>> >>>>>
>> >>>>> It does seem like using netCDF is a better way to go...I believe
>> even
>> >>>> MATLAB
>> >>>>> (my main dta interface tool) has a netCDF writer...it definitely has
>> >> (and
>> >>>> I
>> >>>>> use) a netCDF reader.  But I need to figure out how to change map
>> >>>>> projections and how exactly to set up the netCDF file so that MODE
>> can
>> >>>>> understand it.
>> >>>>>
>> >>>>> I will take a look at the output from P2P_Combine...that file
>> >> definitely
>> >>>>> created successfully when I ran the test script...I can see it in my
>> >>>>> directory.
>> >>>>>
>> >>>>> On 3/12/10, John Halley Gotway <johnhg at ucar.edu> wrote:
>> >>>>>> Matt,
>> >>>>>>
>> >>>>>> Creating a GRIB file is a rather ambitious task I have never
>> actually
>> >>>> done
>> >>>>>> myself.  If I were in your position, I'd actually reformat your
>> binary
>> >>>> data
>> >>>>>> into a NetCDF file that looks like the output of
>> >>>>>> the PCP-Combine tool.  MODE can read either GRIB data or the NetCDF
>> >>>> output
>> >>>>>> of PCP-Combine.  There are a lot more utilities out there for
>> writing
>> >>>> NetCDF
>> >>>>>> files than there are for writing GRIB files.
>> >>>>>>
>> >>>>>> Use the "ncdump" utility to view the NetCDF output from PCP-Combine
>> in
>> >>>> the
>> >>>>>> "METv2.0/out/pcp_combine" directory.  These files are created when
>> you
>> >>>> run
>> >>>>>> the test script included with MET.  And just
>> >>>>>> mimick the file structure with your data.
>> >>>>>>
>> >>>>>> If you choose to go that route, you'll probably need some help
>> >> choosing
>> >>>> the
>> >>>>>> variable name and setting up the grid definition information in
>> NetCDF
>> >>>>>> header.  We're working on expanding our support for
>> >>>>>> NetCDF files for the next release of MET.  They're a lot easier to
>> >> work
>> >>>>>> with than GRIB files are.
>> >>>>>>
>> >>>>>> Let me know how you plan to proceed and what other questions you
>> have.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> John
>> >>>>>>
>> >>>>>>
>> >>>>>> Matthew Souders wrote:
>> >>>>>>> Hello John,
>> >>>>>>>
>> >>>>>>> This is Matthew Souders (Stony Brook University) - I apologize for
>> >>>> taking
>> >>>>>> up
>> >>>>>>> a bit more of your time, but I am running into some trouble
>> getting
>> >>>> data
>> >>>>>>> read into MODE for which I thought you might have a solution on
>> hand
>> >> to
>> >>>>>>> help.  My basic data is a single variable on a 2.5 by 2.5 degree
>> >>>> regular
>> >>>>>>> lat/lon grid in fortran direct access BINARY format.  I have been
>> >>>> trying
>> >>>>>> to
>> >>>>>>> get it intelligibly read into GRIB format (the form MODE needs)
>> for a
>> >>>>>> month
>> >>>>>>> now using Wesley Ebusizaki's gribw utility; however, you need a
>> >> rather
>> >>>>>>> significant background in C scripting to rewrite his basic code
>> and
>> >>>> make
>> >>>>>>> proper use of the PDS file writer.  As I understand it...MODE
>> needs
>> >>>> this
>> >>>>>> PDS
>> >>>>>>> file to operate properly.  I can create a GRIB using the default
>> PDS
>> >>>>>> file,
>> >>>>>>> but that doesn't accurately describe the data in the GRIB.
>> >>>>>>>
>> >>>>>>> I tried e-mail Wesley on this matter but he is understandably
>> >> extremely
>> >>>>>> busy
>> >>>>>>> and limits support to installation and debugging issues.  He
>> >> basically
>> >>>>>> just
>> >>>>>>> directed me to the GRIB standard release.  I was wondering if you
>> had
>> >>>> any
>> >>>>>>> experience getting binary data read into MODE or if you knew
>> someone
>> >>>> who
>> >>>>>>> might be able to help?
>> >>>>>>>
>> >>>>>>> I also would like to know how exactly the MODE tool uses the PDS
>> >>>> file?  I
>> >>>>>>> ask because I am trying to figure out how picky it is w/r/t what
>> is
>> >> in
>> >>>>>> the
>> >>>>>>> PDS file vs. what is in the actual GRIB file.
>> >>>>>>>
>> >>>>>>> Thanks in advance for your time.
>> >>>>>>>
>> >>>>>>> Matt Souders.
>> >>>>>>>
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/met_help/attachments/20100316/4c62f9e0/attachment.html