[ncl-talk] bandpass filter issue

[Dennis Shea]

Whenever you are performing an FFT on time series data, you should
taper the data. I also prefer explicitly removing the mean

[a] Remove overall mean
[b] taper to zero.
[c] perform fft

If you have a very long time series, you may want to 'segment average'

[d] break series into N segments
[e] remove segment average/taper to zero
[e] average at each frequency

===
I think you should look at complex demodulation.

===



On Tue, Oct 6, 2015 at 12:58 PM, Maria Gehne - NOAA Affiliate
<maria.gehne at noaa.gov> wrote:
> Hi Chang,
>
> I would probably try a larger frequency window for the bandpass filter. As
> you were saying, the narrow bandpass can induce artificial signals in your
> data. The other thing that could be causing issues is the type of window you
> use when you filter in Fourier space. If you simply keep the Fourier
> coefficients you are interested in and set all others to zero you will see
> some evidence of the Gibbs phenomenon in your filtered data. In general it
> is better to use a window that is equal to 1 at the frequencies you want to
> keep and then goes to zero over a range of frequencies. The squared cosine
> usually works well for me. The same holds for filtering in wavenumber!
>
> If you use a larger frequency range and are not convinced that what you are
> seeing are the waves you are interested in, I would suggest doing and EOF
> analysis on your filtered data to extract the mode with the most variance in
> the filtered data. In this case it would be better to start with a much
> larger range of frequencies.
>
> Hope this helps,
> Maria
>
> On Tue, Oct 6, 2015 at 10:39 AM, Xi Chang <xi.chang01 at gmail.com> wrote:
>>
>> Hi Maria,
>>
>> Yes, i actually want to analyze 16-day planetary waves  (k=1) waves using
>> daily data. But in this case, i dont need to decompose it into eastward and
>> westward propgating waves. I tried to bandpass filter the data, but since
>> the frequency bands are too narrow (1/15) and (1/17).. the results are bad.
>> any suggestion? does it mean that i need higher temporal resolution?
>> the steps that i do are:
>> 1. filter data using forward and backward fourier to retain k=1
>> 2. bandpass this filtered data to retain only freq at 1/16day
>>
>> Chang
>>
>>
>> On Tuesday, 6 October 2015, Maria Gehne - NOAA Affiliate
>> <maria.gehne at noaa.gov> wrote:
>>>
>>> Can you explain in more detail what you are trying to do? Are you trying
>>> to only look at phenomena that have a period of 16 days?
>>>
>>> Maria
>>>
>>> On Tue, Oct 6, 2015 at 12:31 AM, Xi Chang <xi.chang01 at gmail.com> wrote:
>>>>
>>>> Thanks for the reply. No i dont use it, since what i need is how to
>>>> extract/filter the daily data having frequency 1/16 day. is it possible to
>>>> only use bandpass filter?
>>>>
>>>>
>>>>
>>>>
>>>> On Tuesday, 6 October 2015, Dennis Shea <shea at ucar.edu> wrote:
>>>>>
>>>>> Are you looking a 'complex demodulation" ???
>>>>>
>>>>> If so, that is in the 6.3.1 release:
>>>>>   http://www.ncl.ucar.edu/future_release.shtml
>>>>>
>>>>> Click 'demod_cmplx'  =>
>>>>>
>>>>> http://www.ncl.ucar.edu/Document/Functions/Contributed/demod_cmplx.shtml
>>>>>
>>>>> See Examples at the bottom of
>>>>> http://www.ncl.ucar.edu/Applications/spec.shtml
>>>>>
>>>>>
>>>>> On Mon, Oct 5, 2015 at 4:32 PM, Xi Chang <xi.chang01 at gmail.com> wrote:
>>>>> > Hi NCL,
>>>>> >
>>>>> > I have daily data 10 years. I'd like to filter the data at frequency
>>>>> > (1/16)
>>>>> > day. To do so what the most appropiate  frquency ranges that i have
>>>>> > to use
>>>>> > ?, is it domething like fa=1/15 and fb=1/17 ?
>>>>> > I aware if the band is too narrow the filtered signals are fake
>>>>> > /artificial
>>>>> > (i tesed with ncl butterwooth and the filtered results are weird) ?
>>>>> > is there
>>>>> > any idea how to do that appropriately? do i need to use hourly data
>>>>> > instead
>>>>> > of daily?
>>>>> >
>>>>> > Thanks
>>>>> >
>>>>> > _______________________________________________
>>>>> > ncl-talk mailing list
>>>>> > ncl-talk at ucar.edu
>>>>> > List instructions, subscriber options, unsubscribe:
>>>>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>> >
>>>>
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>
>> On Tuesday, 6 October 2015, Maria Gehne - NOAA Affiliate
>> <maria.gehne at noaa.gov> wrote:
>>>
>>> Can you explain in more detail what you are trying to do? Are you trying
>>> to only look at phenomena that have a period of 16 days?
>>>
>>> Maria
>>>
>>> On Tue, Oct 6, 2015 at 12:31 AM, Xi Chang <xi.chang01 at gmail.com> wrote:
>>>>
>>>> Thanks for the reply. No i dont use it, since what i need is how to
>>>> extract/filter the daily data having frequency 1/16 day. is it possible to
>>>> only use bandpass filter?
>>>>
>>>>
>>>>
>>>>
>>>> On Tuesday, 6 October 2015, Dennis Shea <shea at ucar.edu> wrote:
>>>>>
>>>>> Are you looking a 'complex demodulation" ???
>>>>>
>>>>> If so, that is in the 6.3.1 release:
>>>>>   http://www.ncl.ucar.edu/future_release.shtml
>>>>>
>>>>> Click 'demod_cmplx'  =>
>>>>>
>>>>> http://www.ncl.ucar.edu/Document/Functions/Contributed/demod_cmplx.shtml
>>>>>
>>>>> See Examples at the bottom of
>>>>> http://www.ncl.ucar.edu/Applications/spec.shtml
>>>>>
>>>>>
>>>>> On Mon, Oct 5, 2015 at 4:32 PM, Xi Chang <xi.chang01 at gmail.com> wrote:
>>>>> > Hi NCL,
>>>>> >
>>>>> > I have daily data 10 years. I'd like to filter the data at frequency
>>>>> > (1/16)
>>>>> > day. To do so what the most appropiate  frquency ranges that i have
>>>>> > to use
>>>>> > ?, is it domething like fa=1/15 and fb=1/17 ?
>>>>> > I aware if the band is too narrow the filtered signals are fake
>>>>> > /artificial
>>>>> > (i tesed with ncl butterwooth and the filtered results are weird) ?
>>>>> > is there
>>>>> > any idea how to do that appropriately? do i need to use hourly data
>>>>> > instead
>>>>> > of daily?
>>>>> >
>>>>> > Thanks
>>>>> >
>>>>> > _______________________________________________
>>>>> > ncl-talk mailing list
>>>>> > ncl-talk at ucar.edu
>>>>> > List instructions, subscriber options, unsubscribe:
>>>>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>> >
>>>>
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>
>