@Johannes L: Let us discuss your question here.

thanks

so now we have Y_k=S_k-E(S_k|X_k) with S_k=f(X_k, Z_k)

Z_k is iid, X_k and Z_k are independent and want to show as a start E(Y_2|Y_1)=0

E(Y_2) is still 0 also with the conditional expectation

But $X_{k+1}=X_k+ f(X_k, Z_k)$ so there is a connection between the X_k and intuitively $X_k$ wouldn't be independent.. On the other hand it would be intuitivly valid to say f(X_k, Z_k) - E(f(X_k, Z_k)|X_k) is a independent sequence (in k) (with the idea that "given we are at the point X_k=x_k" the difference of f and its expectation depends only on Z_k which is a idd sequence)

@Gortaur I am not sure about the etiquette here - but I am off for today, I will look here tomorrow, thanks a lor for your effort in helping me

@JohannesL - about the etiquette everything's ok. It's highly unlikely to have $X_k$ iid unless $f(x,z) = z-x$, say. Though in that case I showed that $M$ is a martingale. I wonder what is your goal: are you trying to prove that $M$ is always a martingale, or you want to raise sufficient conditions on $f$ to ensure that $M$ is a martingale? Because in the first case we can just look for the counterexample.

I know that it is a martingale, I have to include the details of a proof from a paper in my diploma thesis about stochastic gradient descent, and I have a bunch of conditions, and for understanding the proof I have to figure out why, which conditions make it a martingale. The background: X_k is the "present state" of the system, Z_k are generated pseudo random numbers and f(X_k, Z_k) gives a finite differences gradient approximation of a loss function.