[Met_help] [rt.rap.ucar.edu #73523] History for MET v 5.0: duplicate observations

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I am using point_stat in METv5.  I have an observation dataset which unfortunately contains duplicated records for some stations.  Short of manually editing those files, is there any way to detect and remove duplicated obs in MET? I see there is a duplicate_flag entry in the point_stat configuration file, but I’m not sure how to alter it, or whether it would accomplish this task.

Thanks,
Rob

--
Robert Fovell
Professor, Atmospheric & Environmental Sciences
University at Albany - SUNY






----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MET v 5.0: duplicate observations
From: John Halley Gotway
Time: Wed Sep 23 14:58:33 2015

Rob,

Yes, the "duplicate_flag" should accomplish what you're after.  Please
search for "duplicate_flag" in this README file:
   met-5.0/data/config/README

I would suggest setting duplicate_flag = SINGLE;

And then run Point-Stat at verbosity level 3 (-v 3), and use the "-
log"
option to create a log file.  That'll print out some information about
duplicate observations that looks something like this:

DEBUG 3:   [lat: 32.630  lon: -108.150  lev: 834.40  elv: 1655.70] -
used
point ob with valid time offset of 001001 (HHMMSS)
DEBUG 3:     [sid:   KSVC  vld: 20070331_113000  ob_val: 274.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_114959  ob_val: 275.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_121001  ob_val: 274.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_123000  ob_val: 273.1500]

For this station, it found 4 observations falling within the time
window
around the forecast valid time.  And it used the one closest in time
to the
forecast valid time.  It was 10 minutes and 1 second away in time.

Just take a look at the log to make sure point_stat is doing what you
expect.

Hopefully that does the trick for you.

Thanks,
John



On Wed, Sep 23, 2015 at 2:01 PM, Fovell, Robert via RT
<met_help at ucar.edu>
wrote:

>
> Wed Sep 23 14:01:12 2015: Request 73523 was acted upon.
> Transaction: Ticket created by rfovell at albany.edu
>        Queue: met_help
>      Subject: MET v 5.0: duplicate observations
>        Owner: Nobody
>   Requestors: rfovell at albany.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=73523 >
>
>
> Hi,
>
> I am using point_stat in METv5.  I have an observation dataset which
> unfortunately contains duplicated records for some stations.  Short
of
> manually editing those files, is there any way to detect and remove
> duplicated obs in MET? I see there is a duplicate_flag entry in the
> point_stat configuration file, but I’m not sure how to alter it, or
whether
> it would accomplish this task.
>
> Thanks,
> Rob
>
> --
> Robert Fovell
> Professor, Atmospheric & Environmental Sciences
> University at Albany - SUNY
>
>
>
>
>
>

------------------------------------------------
Subject: MET v 5.0: duplicate observations
From: Fovell, Robert
Time: Thu Sep 24 09:02:07 2015

John, that did it.  Thanks!

Best,
Rob

On Sep 23, 2015, at 4:58 PM, John Halley Gotway via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:

Rob,

Yes, the "duplicate_flag" should accomplish what you're after.  Please
search for "duplicate_flag" in this README file:
  met-5.0/data/config/README

I would suggest setting duplicate_flag = SINGLE;

And then run Point-Stat at verbosity level 3 (-v 3), and use the "-
log"
option to create a log file.  That'll print out some information about
duplicate observations that looks something like this:

DEBUG 3:   [lat: 32.630  lon: -108.150  lev: 834.40  elv: 1655.70] -
used
point ob with valid time offset of 001001 (HHMMSS)
DEBUG 3:     [sid:   KSVC  vld: 20070331_113000  ob_val: 274.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_114959  ob_val: 275.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_121001  ob_val: 274.1500]
DEBUG 3:     [sid:   KSVC  vld: 20070331_123000  ob_val: 273.1500]

For this station, it found 4 observations falling within the time
window
around the forecast valid time.  And it used the one closest in time
to the
forecast valid time.  It was 10 minutes and 1 second away in time.

Just take a look at the log to make sure point_stat is doing what you
expect.

Hopefully that does the trick for you.

Thanks,
John



On Wed, Sep 23, 2015 at 2:01 PM, Fovell, Robert via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:


Wed Sep 23 14:01:12 2015: Request 73523 was acted upon.
Transaction: Ticket created by
rfovell at albany.edu<mailto:rfovell at albany.edu>
      Queue: met_help
    Subject: MET v 5.0: duplicate observations
      Owner: Nobody
 Requestors: rfovell at albany.edu<mailto:rfovell at albany.edu>
     Status: new
Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=73523 >


Hi,

I am using point_stat in METv5.  I have an observation dataset which
unfortunately contains duplicated records for some stations.  Short of
manually editing those files, is there any way to detect and remove
duplicated obs in MET? I see there is a duplicate_flag entry in the
point_stat configuration file, but I’m not sure how to alter it, or
whether
it would accomplish this task.

Thanks,
Rob

--
Robert Fovell
Professor, Atmospheric & Environmental Sciences
University at Albany - SUNY








--
Robert Fovell
Professor, Atmospheric & Environmental Sciences
University at Albany - SUNY





------------------------------------------------