[Met_help] Fwd: Fwd: pcp-combine
John Halley Gotway
johnhg at ucar.edu
Tue Mar 2 11:57:47 MST 2010
Jeff,
I was able to retrieve the files and take a look at them. And we've got some issues here.
I ran the following wgrib command to look at the Product Description Section of your GRIB file, and got the output below:
wgrib LIS_APCP -PDS10
1:0:d=10021918:APCP:kpds5=61:kpds6=1:kpds7=0:TR=133:P1=3:P2=0:TimeU=1:sfc:3-0hr fcst:ave at 1yr:NAve=0:
PDS10= 0 0 28 128 57 88 255 192 61 1 0 0 10 2 19 18 0 1 3 0 133 0 0 0 21 2 0 0
(1) First, the time range indicator in your forecast file (LIS_APCP) is set pretty weird. It's set to a value of 133 which means:
"Climatological Average of N forecasts, each a year apart, starting from initial time R and for the period from R+P1 to R+P2"
Here's a table for how to interpret time range indicator values: http://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html
For precipitation, this is typically set to a value of 4 to mean "Accumulation".
(2) Second, the time values (the two numbers before 133 in the wgrib output) are set to 3 and 0. These should probably be reverse and set to 0 and 3. For an accumulation, this would mean that the
precip is accumulated from forecast hour 0 to forecast hour 3. Going in reverse, from 3 to 0 doesn't make sense.
(3) Third, I see that this forecast data is on a global grid. I've attached a jpeg showing a plot of the data which I generated using the IDV tool. However your stage4 observations are only valid
over a much smaller CONUS domain. See the other attached image. Really, I'd suggest doing the interpolation the other way... interpolate the global LIS data to the CONUS stageIV domain. Once these
other issues are resolved, what your doing would work. But it'd run a lot faster if you interpolated to the CONUS domain rather than the global one.
It looks to me like there's probably some issues in how these LIS GRIB files are being produced. I'd suggest trying to resolve those first.
Hope that helps.
John
Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN wrote:
> That did it, thanks. I put those three files that I described in the
> directory you gave me and once again the problem I am having is that the
> time range indicator. In the LIS_APCP forecast file it is listed as 133
> and in the ST4_20100219_18 observation file it is 4, and MODE halts when
> it comes across this. I appreciate your help and taking the time to look
> at this for me.
>
> Jeff
>
> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at ucar.edu]
> Sent: Tuesday, March 02, 2010 11:20 AM
> To: Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN
> Subject: Re: Fwd: [Met_help] Fwd: pcp-combine
>
> Jeff,
>
> I'm not sure if this will help, but try using the '-p' option for a
> "passive" ftp like this:
> ftp -p ftp.rap.ucar.edu
>
> I have had ftp problems in the past fixed when I used '-p', but I don't
> know if it'll do the trick for you or not.
>
> Thanks,
> John
>
> Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN wrote:
>> Hey John,
>>
>> I hate to keep bugging you with this but I'm having trouble posting
>> sample files to you ftp site again, even after I have significantly
> cut
>> down the size of the forecast file. I'll show you a step by step
> process
>> of what I have been doing to see if perhaps I have misinterpreted the
>> proper procedure for the ftp site.
>>
>> The three files I am trying to send to you are:
>> Forecast file- LIS_APCP (320,704 bytes)
>> Observation file- ST4_20100219_18 (3,457,012 bytes)
>> Config file- LISQMODEconfig (16,401 bytes)
>>
>> These three are all located in the same directory, so initially I go
> to
>> that directory, then type:
>> ftp ftp.rap.ucar.edu
>> username= anonymous
>> password= Jeffrey.Hamilton at offutt.af.mil
>> cd incoming/irap/met_help
>> put LIS_APCP
>>
>> at this point I receive this output for all the files when attempted:
>> 200 PORT command successful.
>> 425 Can't build data connection: Connection timed out.
>>
>> I'm curious if this is a problem you have come across before, and if
> you
>> have any ideas as to what a solution might be.
>>
>> Thanks,
>>
>> Jeff
>>
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>> Sent: Friday, February 26, 2010 5:25 PM
>> To: Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN; tressa at ucar.edu
>> Subject: RE: Fwd: [Met_help] Fwd: pcp-combine
>>
>> Jeff,
>>
>> Yeah, I think it'd be helpful if you posted some sample data to our
> ftp
>> site for me to take a look at. I'm not entirely sure about your
>> question.
>> The output of PCP-Combine is a NetCDF file. The time range indicator
>> is
>> something specific to GRIB files. It is fine to compare a forecast
> GRIB
>> file with the NetCDF output of PCP-Combine. MODE should be able to
>> handle
>> that fine.
>>
>> Go ahead and post a sample forecast file and observation file. Also
>> include any MODE configuration file you may be using. I'll take a
> look
>> and see if I can better understand your question.
>>
>> Just as a reminder...
>> ftp ftp.rap.ucar.edu
>> username=anonymous
>> password="your email address"
>> cd incoming/irap/met_help
>> put "your files"
>> bye
>>
>> Thanks and have a good weekend,
>> John
>>
>>> John,
>>>
>>> I'm having another time range indicator problem similar to the one I
>>> initially asked you and Tressa about, but this time it is with the
>> MODE
>>> tool. I have an forecast file with a time range indicator of 133 and
>> an
>>> observation file, resulting from PCP-combine output, with an
> indicator
>>> of 4. I am curious if there is a process by which I could still
>> compare
>>> these two files with differing indicators, because I know you
>> mentioned
>>> that there may be differences in the grib files themselves. (and
> would
>>> you prefer me to put the files on your ftp website so that you can
>> take
>>> a look at them?)
>>>
>>> Thanks,
>>> Jeff
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>>> Sent: Tuesday, February 23, 2010 11:30 AM
>>> To: Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN
>>> Cc: Tressa Fowler
>>> Subject: Re: Fwd: [Met_help] Fwd: pcp-combine
>>>
>>> Jeff,
>>>
>>> Great. Glad you figured it out. Just let us know if any more
>> questions
>>> come up.
>>>
>>> Tressa had mentioned you guys were having problems viewing the
>>> PostScript output of MODE. I'd like to help you figure that out if
>>> possible. Do you guys have the "convert" command available on your
>>> machines?
>>>
>>> Thanks,
>>> John
>>>
>>> Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN wrote:
>>>> John,
>>>>
>>>> Oops! Turns out that I was trying to combine the incorrect grib
>> files,
>>>> and trying with the correct ones I was able to generate a combined
>>> file
>>>> on my first try. But I really appreciate the info and I'm sure it'll
>>>> come in handy in the future!
>>>>
>>>> Thanks again,
>>>> Jeff
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>>>> Sent: Tuesday, February 23, 2010 10:17 AM
>>>> To: Tressa Fowler
>>>> Cc: Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN
>>>> Subject: Re: Fwd: [Met_help] Fwd: pcp-combine
>>>>
>>>> Jeff (and Tressa),
>>>>
>>>> I'm not sure why your ftp is timing out. If you think that the file
>>> is
>>>> too large, you could try just cutting out only the GRIB record(s) of
>>>> interest and send us those. Here's how you'd do it using
>>>> the "wgrib" utility - hopefully you have that on your system...
>>>>
>>>> # First, figure out what GRIB record you want (I assume it's
>>> accumulated
>>>> precip)
>>>> wgrib in_file.grib | grep APCP
>>>>
>>>> # The first number in the output tells you the GRIB record number
>>>> corresponding to that record. Suppose, APCP has record number 12 in
>>>> your file. Here's how you'd extract record number 12:
>>>> wgrib in_file.grib -d 12 -grib -o out_file.grib
>>>>
>>>> # The 12-th GRIB record will be written to the output file
>>>> "out_file.grib". If you'd also like to append additional records to
>>>> that output GRIB file, say record number 25, here's how you'd do
>> that:
>>>> wgrib in_file.grib -d 25 -grib -append -o out_file.grib
>>>>
>>>> So now that output GRIB file contains 2 records. Once you'd picked
>>> out
>>>> the record(s) you'd like to send us, try the ftp again - or maybe
> the
>>>> file would be small enough to email. Hope that helps.
>>>> Let me know if you have more problems transferring it.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> Tressa Fowler wrote:
>>>>> John,
>>>>>
>>>>> I don't know what to tell Jeff about our ftp site. Any ideas?
>>>>>
>>>>> Tressa
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: "Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN"
>>>>>> <Jeffrey.Hamilton at offutt.af.mil>
>>>>>> Date: February 23, 2010 7:25:52 AM MST
>>>>>> To: "Tressa Fowler" <tressa at ucar.edu>
>>>>>> Subject: RE: [Met_help] Fwd: pcp-combine
>>>>>>
>>>>>> Tressa,
>>>>>>
>>>>>> I have been able to connect to the ucar anonymous ftp site, but
>> when
>>>> I
>>>>>> try to put the grib file in the incoming/irap/met_help directory,
>>> the
>>>>>> connection times out. Admittedly it is a large file, so I was
>>> curious
>>>> if
>>>>>> there were any options to get around this and what would be the
>> best
>>>>>> path for you?
>>>>>>
>>>>>> Thanks,
>>>>>> Jeff
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Tressa Fowler [mailto:tressa at ucar.edu]
>>>>>> Sent: Monday, February 22, 2010 9:30 AM
>>>>>> To: Hamilton, Jeffrey A CIV USAF AFWA 16 WS/WXN
>>>>>> Subject: Fwd: [Met_help] Fwd: pcp-combine
>>>>>>
>>>>>> Jeff,
>>>>>>
>>>>>> I've forwarded what John says about your error. Can you put a
> file
>>>> on
>>>>>> our anonymous ftp site? Instructions below.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Tressa
>>>>>>
>>>>>>
>>>>>>
>>>>>> cd "the directory that contains the files you want to send"
>>>>>>
>>>>>>
>>>>>> ftp ftp.rap.ucar.edu <http://ftp.rap.ucar.edu/>
>>>>>>
>>>>>>
>>>>>> username = anonymous
>>>>>>
>>>>>>
>>>>>> password = "your email address"
>>>>>>
>>>>>>
>>>>>> cd incoming/irap/met_help
>>>>>>
>>>>>>
>>>>>> put file
>>>>>>
>>>>>> bye
>>>>>>
>>>>>>
>>>>>> Begin forwarded message:
>>>>>>
>>>>>>
>>>>>> From: John Halley Gotway <johnhg at ucar.edu>
>>>>>> Date: February 22, 2010 8:13:27 AM MST
>>>>>> To: Tressa Fowler <tressa at ucar.edu>
>>>>>> Cc: met_help at ucar.edu
>>>>>> Subject: Re: [Met_help] Fwd: pcp-combine
>>>>>>
>>>>>> Tressa (and Jeff),
>>>>>>
>>>>>> I have seen that type of error before. It usually means there
>>>>>> some weirdness in the GRIB files being used. In the Product
>>>> Description
>>>>>> Section (PDS) of each GRIB record, there is a header value that
>>>>>> tells you how to interpret the time information in the file.
>>>>>> That value is called the "Time Range Indicator". Here's a table
>>> that
>>>>>> describes the options:
>>>>>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html
>>>>>>
>>>>>> When summing data up in PCP-Combine, we expect that the time
>>>>>> range indicator will be set to 4 (meaning an accumulation time)
> for
>>>>>> precipitation. For some reason, it's not set to a value of 4.
> And
>>>>>> this really may just be a symptom of larger problems reading
>>>>>> this particular type of GRIB file.
>>>>>>
>>>>>> It'd probably be easiest to just have Jeff send us a few
> sample
>>>>>> GRIB files from this dataset. I could run them through MET using
>>> the
>>>>>> debugger and try to really diagnose the problem. It would be
>>>>>> helpful to know where the data is from and how the GRIB files
>>>>>> were created.
>>>>>>
>>>>>> Thanks,
>>>>>> John
>>>>>>
>>>>>> Tressa Fowler wrote:
>>>>>>
>>>>>>
>>>>>> John,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Any help for Jeff? This looks to me like it should work.
>>>>>> I even tried it
>>>>>>
>>>>>>
>>>>>> on pigpen, and my error was just that I don't have these
>>>>>> input files.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Tressa
>>>>>>
>>>>>>
>>>>>>
>>>>>> Begin forwarded message:
>>>>>>
>>>>>>
>>>>>>
>>>>>> From: "Hamilton, Jeffrey A CIV USAF AFWA 16
>>>>>> WS/WXN"
>>>>>>
>>>>>>
>>>>>> <Jeffrey.Hamilton at offutt.af.mil>
>>>>>>
>>>>>>
>>>>>> Date: February 19, 2010 3:04:04 PM MST
>>>>>>
>>>>>>
>>>>>> To: "Tressa Fowler" <tressa at ucar.edu>
>>>>>>
>>>>>>
>>>>>> Subject: pcp-combine
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Tressa,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hope you enjoyed your trip out here yesterday,
>>>>>> Omaha is so enticing
>>>>>>
>>>>>>
>>>>>> this time of year isn't it? J Anyway, I have a
>>>>>> MET issue about the
>>>>>>
>>>>>>
>>>>>> PCP-combine tool that I wanted to run by you. I
>>>>>> am trying to sum two 3
>>>>>>
>>>>>>
>>>>>> hour obs accumulated precip grib files with the
>>>>>> following code line:
>>>>>>
>>>>>>
>>>>>>
>>>>>> /METv2.0/bin/pcp_combine -sum 00000000_000000 3
>>>>>> 20100215_180000 6
>>>>>>
>>>>>>
>>>>>> /METv2.0/out/pcp_combine/LIS_2010021518_06
>>>>>> -pcpdir \ /METv2.0/data/LIS/
>>>>>>
>>>>>>
>>>>>>
>>>>>> And I am coming up with an error:
>>>>>>
>>>>>>
>>>>>>
>>>>>> ERROR: check_file_time() -> time range indicator
>>>>>> of
>>>>>>
>>>>>>
>>>>>> not currently supported. Time range indicator
>>>>>> must be accumulation
>>>>>>
>>>>>>
>>>>>> (Grib Code 4)
>>>>>>
>>>>>>
>>>>>>
>>>>>> I have tried changing the various times in my
>>>>>> command line
>>>>>>
>>>>>>
>>>>>> (highlighted above), however, I keep coming up
>>>>>> with this. I'm
>>>>>>
>>>>>>
>>>>>> wondering if I am not changing the right thing
>>>>>> or if my code is
>>>>>>
>>>>>>
>>>>>> incorrect? Any suggestions?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>>
>>>>>> // SIGNED //
>>>>>>
>>>>>>
>>>>>> JEFF HAMILTON, Civ SCEP
>>>>>>
>>>>>>
>>>>>> Model Analysis Team
>>>>>>
>>>>>>
>>>>>> 16 WS/ WXN
>>>>>>
>>>>>>
>>>>>> Air Force Weather Agency
>>>>>>
>>>>>>
>>>>>> Offutt AFB, NE 68113
>>>>>>
>>>>>>
>>>>>> (402) 232-7207
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
> ------------------------------------------------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>>
>>>>>>
>>>>>> Met_help mailing list
>>>>>>
>>>>>>
>>>>>> Met_help at mailman.ucar.edu
>>>>>>
>>>>>>
>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>>
>>>>>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LIS_APCP.jpg
Type: image/jpeg
Size: 524341 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20100302/9730b8c5/attachment-0002.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ST4_20100219_18.jpg
Type: image/jpeg
Size: 9721 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20100302/9730b8c5/attachment-0003.jpg
More information about the Met_help
mailing list