last day (15 days later) » 

11:51 AM
5
A: Least Squares Solution Using the DFT vs Wiener-Hopf Equations

orchi_dFollowing Royi's derivation, we want to show that, $$\begin{align} \hat{h} = \arg \min_h||Xh - y||^2 = (X^T X)^{-1} X^H y = IDFT(Y \oslash X) \end{align}$$ where $X$ is a circular convolution matrix that is circulant. Deriving the Wiener-Hopf solution is simple, $$\begin{align} \hat{h} &= \arg \m...

 
nice! This is what I was looking for - so your conclusion is the operations are identical (I assume if I zero pad out the DFTs to 2x to be linear convolution)? i thought Royi had stated to me in our chat that they are not identical, so I want to check that. I am going to confirm with a simulation but this answers my questions if they are indeed equivalent.
 
This is essentially what I was saying in the edit to my answer, but more fleshed out (+1).
 
@orchi_d see my simulation results which suggest the two methods aren’t identical — there must be some additional steps involved I am not doing; I am interested in your insights in case you see something I missed.
 
I don't think they will be identical, I can't think of a mathematical equivalence. If x is N samples and y is N+M-1 samples, you can zero pad x, and take the first M points from the IDFT (since the length of h is M samples), and compare that to the result of the least squares solution. Can you share that plot?
 
@orchi_d your statement "the equivalence of the Wiener-Hopf equations to the DFT convolution theorem" with a concluding formula IDFT(DFT(y)/DFT(x)) makes it sound identical. I had tried what you suggested previously, thinking along those same lines, and it wasn't pretty. There are other artifacts when you look at the time response that also didn't make sense to me but might illuminate something. I'll expand my answer at the bottom to include those details. Thanks for your thoughts! (Use @ to tag my name as otherwise I may miss your comment)
 
11:51 AM
@DanBoschen, This has nothing added on top of my answer. Actually I find this answer to be not fair. My operator $ \hat{\boldsymbol{h}} = {\mathscr{D}}^{-1} \left( \left( \mathscr{D} \boldsymbol{y} \right) \oslash \left( \mathscr{D} \boldsymbol{x} \right) \right) $ is just IDFT(DFT() / DFT()). I don't see the added value in the answer.
@DanBoschen, Again, if you're using the regular tools for Wiener Filter they use different model for the convolution.
 
@Royi I think it says essentially the same thing as your answer, but I find it much more clear. It's an explicit derivation, with clear definitions.
 
@Royi I upvoted both answers and appreciated both of them! I had specific questions to be answered and the conclusions for both answers still do not yet match my simulation. In the end they are formulas predicting the channel using the same signals. My signals in this case had no noise (no stochastics) yet the Wiener approach still excelled and the FFT approach failed badly. I was looking for the explanation for that (and still looking!). But I did like Orchi_d's clear presentation, but this is not to say I didn't like your answer!
 
I wrote, this question is about the derivation. If you want to show it numerically, would you open a question with code? I'd be happy to see what's going on.
@Gillespie, I think the way improve others questions is not by copying them and improving but by editing or suggesting to improve some parts. Anyhow, it is water under the bridge.
 
@Royi my question as stated remains unanswered - if the conclusion is they are numerically identical, I am finding that to not be the case. I haven’t selected an answer yet so there still is an opportunity to improve this one; removing points of confusion and to show how this relate a to practical use of one approach over the other.
 
@DanBoschen, hey won't be identical in your code since your code uses Toeplitz Matrix for the convolution method while the DFT assumes Circulant matrix. Again, the question all the answers answer is that, given the convolution model is periodic convolution is the Least Squares solution the inverse DFT of the element wise division of the data and input. The answer is yes. Nobody claimed the Inverse DFT solution is the solution for different convolution model. For discrete signals with finite support the convolution has many types.
@DanBoschen, Your question has nothing to do with Wiener Filter, etc... Your question is, will different convolution models yield different results? The answer is yes. The reason is because the model matrix of the LS problem is different hence the solutions are different.
 
11:51 AM
@Royi- ok. My question is on the two approaches used for channel estimation. The math you show suggests they are identical. The results aren’t so something is missing on one side or the other. My question includes insight into the practical use of each. These questions haven’t been answered yet. I thought I avoided the circulant difference by zero padding the FFT?
 
@DanBoschen, The Math I showed doesn't say they are identical because they are not if you utilize different convolution model. What you are asking for is why $ \arg \min_{\boldsymbol{x}} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} $ is different from $ \arg \min_{\boldsymbol{x}} {\left\| B \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} $? Well, of course they are different because $ A $ and $ B $ are different. As you can see, both are LS solutions, but no one says they should be the same.
 
@Royi This is a second time I see you complaining about others not editing literally complete answers into your answer. I don't know what makes you think this is an acceptable expectation. Edits are for minor changes, not ideas that take personal time to develop. People don't work for you here.
 
@OverLordGoldDragon, Please refrain from such non constructive remarks. In my opinion people shouldn't not explain what others said as a different answer. The can expand / extend, etc. In this case, in my opinion, the answer adds nothing as it is the same as mine. I expressed my opinion. I am not setting the policy here, I'm just communicating. Try it :-).
 
@Royi Your wording goes beyond a passive suggestion, particularly in the other case I saw. It's unnecessary signaling to users, with a rather outlandish idea - you could try Meta.SE, I wager their backlash will be greater. My motivation is also what I see as you being greedy, in other ways - you are already overcompensated on this network, I'm unsure you realize it. Others don't want to engage in giving such feedback, so I am. Could be a bit softer, and generally is.
 
11:51 AM
@OverLordGoldDragon, I will explain what I expect (Example to follow) . This is my expectation. Yet, we're all grown up's we can live when our expectations don't fulfill. Look at the question here: dsp.stackexchange.com/questions/87254. I gave my answer and derivation. At some point RodrigodeAzevedo pointed that I got a specific equation. He didn't took my answer and posted on top of it (He had better case than above in my opinion) . He pinged (He deleted his comment) to improve my answer. I feel this is a good policy. I encourage it. People thinks differently, it is OK as well.
@OverLordGoldDragon, What I don't like is your tone suggesting that I implied people are working for me. This is not constructive. You may share you opinion that you won't and don't think people should do it and earth will keep spinning, all is OK.
 
@Royi I think the problem is, some are too sensitive to directness on this network. What I said isn't that harsh in substance, and I've gotten worse in substance but not in wording, and I think latter matters more. This tone is far from end of the world on StackOverflow. I think it's that, DSP takes the "academic colleagues" perspective, while SO "random strangers on internet". To me, coating feedback in politeness is extra work and sometimes dishonest.
@Royi So, I don't know where your first comment is but to me it read as "how dare you not write your answer into mine", and it was said politely. So, the tone issue didn't start with me in my mind. Anyway in this case I didn't mean my comment in its worst interpretation toward you. I agree it's generally in bad taste, and don't do such.
 
That's not the tone I intended. All I said is that instead of writing my answer differently I'd be happy if orchi_d would have given me a feedback (It is clear my choice of symbols was not clear to many). Again, expectation / suggestion. He is free to do what he wants. Anyhow, I think I explained my opinion enough. It is not a big deal. You suggested I think people work for me. This is not being direct, this is just looking for a conflict. I come from the most direct country, believe me :-).
 
@Royi Ok, let me show it. "Others are entitled to post their own work and take credit for it - "alternative answers" are common and encouraged on StackExchange, without expectation to invest labor in significantly developing others' answers." This took four minutes, and says the same thing, but it's certainly much harder to take insult from it. Anyway I do take your point, and you're not due strong backlash for passively suggesting it - but really, I suggest reconsidering where to draw the line on edit vs answer.
 

last day (15 days later) »