last day (16 days later) » 

18:19
0
A: Savitzky–Golay filter vs. IIR or FIR linear filter

Fat32IMPORTANT NOTE: It seems according to MattL. that the wikipedia link prvoivded by the OP is not correctly defining a Savitzky-Golay filter. [MattL here: I do not have any opinion concerning the correctness of the wikipedia article, the only thing I claim is that Savitzky-Golay filters are simpl...

@DanBoschen you are not missing something... Probably Matt refers to subblock stage processing as fixed, whereas I referred to the whole signal processing as data dependent, nonlinear, time varying filter. wiki page is quite clear (I mean not necessarily correct but at least quite clear). Yet, as I said, I'm not a big fan SG filters and if its definition is something else, I would be quite the happiest to acknowledge it too...
@g6kxjv1ozn , the E&EE definition of adaptive filter is this: if any characteristics of a system (filter) is changed, adjusted, or set according to the specific input it has, then that's called an adaptive system (filter). So the mathematical procedure of polynomial fit is by very definition input dependent and hence an adaptive system. Yet if those SG filters have another definition, then things can change. So it's up to how you define them ;-)
@DanBoschen oh sorry for that misleading thing, I mean I'm saying it now. I have no kind of +/- bias towards those filters. I just felt myself as if I were promoting them :-)
Yes @DanBoschen , adding your engineering expertise would be great again! yet I fear the gang! :-) will downvote you too ! :-)))
@DanBoschen :-))) that's definetely not you Dan... they know themselves ! just a real joke ;-))
@MattL. Yes that's long for a comment. Why not just put your explanation in answer here? My answer is based on the entrance of the wiki link provided by OP. Considering the amount of pseudo science charlatanizm nowadays it can of course be wrong, nevertheless that describes a SG filter as a block based polynomial data fitting procedure which makes it a time varying, nonlinear, data dependent adaptive filter. I dont know why you refrain from commenting on wikiLink but rather insist on pointing your reference? I dont care your reference what I & OP care is the wiki link he provided. :-)
@DanBoschen Hi! what is so sure I cannot see.. So is a SG filter time varying nonlinear adaptive one (a block based polynomial fit) or just an ordinary FIR :-)) which one is sure? So since matt says so makes it sure ? Btw as you can see the gang downvoted me (voice of truth) again ;-)
@MattL. So the summary: My (downvoted) answer is correctly describing the very wiki link provided by the OP. What is that filter called? Savitzky Golay? Bose-Einstein? Mozart-Vialdi? I dont know. All I know is it's a block based polynomial fit and I dont care if it actually a SG filter or not. So I will edit my answer in line with it. If a SG filter is as you insist to be so, then either OP doesnt know what he refers to or the wiki link doesnt know what they rerer to, or you dont know what you are refering to. So I will edit my answer to declare this.
@DanBoschen Now I have edited my answer to reflect the optimal choice. So whether the wiki link is correctly defining a SG filter or not is removed from the discussion and the content is focused...
@MattL. This is really not a good bahaviour... So then I am now explicitly asking, pleae read the wiki article and state your opinion wheter it describes a SG filter correctly or not according to your understanding? Cause if that link is wrong I will delete my answer, othwersie I have to add your objection to it explicitly..
@PeterK Here is an clear example where a conflict is deliberatly locked by the very intruder... This is my 10th comment asking MattL to read the wiki link and state his opinin. As he is the one who started the fuss about my post being compleletly wrong. Now he refrains from declaring his opinion about the link that I based my answer on... So how can we avoid this deadlock?
@MattL. why don't you explicitly state your conclusivie decision on the wiki link? just why? And then why you insist on my post being wrong? And why you refrain from referring to the wiki links as correct or not ? That's what I refer to as not a good behaviour. If you will keep your opinion secret on the wiki links, then this seems like a poker game to me... And this is not what I was given my education about...
@MattL. I really should move all these comments to chat, but these answers may save you the trouble of explaining (dsp.stackexchange.com/a/9494/35, dsp.stackexchange.com/a/9512/35).
@MattL. I (skim) read the link you have provided. It simply says that you design a local polynomial filter which is fit to it input data and converts it into a FIR coeffs for processing the samples in that block. It does not say that you can use the same filter with different data? Am I missing something? So given a 2nd order fitting SG filter coeffs, will you use it for every set of data of arbitrary length you have? is it independent from input data ?
@Fat32 Moving to chat doesn't mean they'll be removed--just transplanted to a setting that's more suitable for discussion. The short answer for SG filters, is that the local polynomial fit is dependent on the sample spacing, which happens to be FIR for equally-spaced samples. The filter coefficients serve to evaluate the least-squares polynomial at 0 using a dot product with the signal values.
@datageist I (accidentally) deleted my comment. It's extremely surprising that a fixed set of FIR coefficients perform a LSE fit to arbitrary data. May I kindly ask you to provide the FIR coeffs of a SG filter for a 3rd order LSE fit to data ? So I will evaluate it and delete my answer completely if that's right...
18:19
@Fat32. Yes, it's extremely surprising, and that's part of the reason people got so excited about them :). Check the two answers in the comment I posted to MattL (one of them has source code to do the design using numpy).
@MattL. I think I made a big sh*t so now to delete my whole answer, may I kindly ask you to provide the coeff of a SG filter for 3rd order polynomial LSE fit purpose ? I will try it and delete my answer then...
@datageist ohh numpy? phyton? I'm too bad at it. Any matlab code? Cannot you just make me a favor and comment (post) just the FIR coefficients for a 3rd order LSE fit SG?
Hi data geist
@Fat32 I think this answer explains it pretty clearly. dsp.stackexchange.com/a/9512/35. It should be clear how to experiment with the designs in Matlab after going through it.
So the SG filter si LTI ?
can you please post the coefficeints, As I cannot decipher the numpy code...
datageist ???
19:20
@MattL I am genuinely confused, reading datageist link he provided earlier and explanation there of the implementation, I see that the coefficients of the filter are indeed determined from the local samples (using a least squared solution). Since it is the local samples, new coefficients would be determined for each subset in the series. This appears very similar to the operation of adaptive equalizers that I am familiar with. Do you see where I am getting confused?
@datageist Since you have some familiarity with the S-G filter, do I interpret your links properly that the coefficients for the filter are determined from the local data samples, and therefore for a longer sequence it would be optimum to recompute the coefficients (and perhaps this is typically done?) for the samples in proximity? This is what I would refer to as an adaptive filter, and there is confusion on this point which was one specific question of the OP's.
 
1 hour later…
20:44
@DanBoschen great shock!!! But they turned out to be LTI. Indeed very simple to see, but surprising to imagine. I will put some clarification. Strictly speaking, the information is processed according to a polynomial fit adaptive manner, but a simple trick (that I will describe) enables a fixed filter (LTI FIR) to process the data, hence data is processed in LTI manner with a fixed FIR impulse response $h[n]$.
21:16
@Fat32 Interesting! So to clarify, are you saying the coefficients of the LTI filter are NOT determined from the data itself? If in contrast the data is used to find the coefficients, I personally would call that adaptive, but perhaps that is my mistake in doing so. If the coefficients are determined from independent polynomial equations that are not effected by the data, then I would agree that it is not an adaptive filter.
yes that's true... I'm in complete shame but I will describe it clearly instead of sitting silent :-)) At the moment I'm trying to implement the filter, once finished I will update my complete answer (or probably delete it). In the mean time; S-G is an adaptive polyfit numerical method algorithm that's degenrated into a state that allows an LTI processing. I will mention that in the editted answer.
@MattL. I'm quite sorry that I refused your correct comment. But since you have refused to describe your reasoning I had no choice but defend myself. Now I'm reading the very nice article you've linked and it's quite clear that the filter is envisioned to be an LTI FIR by cleverly modifying teh poly fit procedure. I will edit my answer as soon as I make an implementation of the filter...
@DanBoschen yes that's true...
@DanBoschen the coefficients of the polynomial are clearly (without doubt data dependent), but the S-G does not find them at all ;-) it finds only one them. That's the trick what enables it to be LTI as I will show.
21:44
@DanBoschen As I read it further I see that you dont even need a trick... I should best stop premature commenting...
22:13
@Fat32 ok thanks for the research and I look forward to your updated answer-- please do post what you find out. I am particularly interested in what the recommended approach is in a streaming application (meaning the data continues for all time); with a fixed non-adaptive filter you would proceed and use the same coefficients, in an adaptive solution you would update the coefficients over time based on changing channel / noise conditions.
The update rate can be extremely slow but if it is doing that based on the samples as received then I would refer to that as adaptive. I haven't read through the links Matt provided yet so am very interested in what you discovered in your own research. Thank you.
@DanBoschen Now I'm doing it... I hope I can be clear enough.
22:34
Woah, good discussion in here! Glad to see the community be really constructive here and hash something out fully! I really enjoyed reading through these comments. This is one of the main reasons I got excited about signal processing: the wide range of applications that one can make with different data. Looking forward to the updated Fat32 answer! thanks for taking the time to do the research, I always appreciate your opinion and point of view on things

  last day (16 days later) »