last day (15 days later) » 

Feeds
Feeds
13:02
1
A: Why does this notation $\mathbb{E}\left[ p(x \mid z) \right]$, when $x$ is given, make sense?

user10354138Didn't the authors make it clear expectations are taken with respect to which variables? For example, in section 2.2 right after equation (11), "all expectations are taken with respect to $q(z)$", so there is no problem with $\mathbb{E}[p(x\mid z)]$ at the end of the page.

nbro
nbro
Expectations are taken WITH RESPECT TO probability distributions (p.d.f.s or p.m.f.s), but they are taken OF random variables, not p.d.f.s or likelihoods, i.e. the inputs to expectations should be random variables, as far as I know. If you think that's not the case, please, explain in detail, and answer this question: math.stackexchange.com/q/3769903/168764. So, I don't think that comment in the paper explains anything.
user10354138
user10354138
For fixed $x$, $p(x\mid z)$ is a function of $z$ and so is a random variable.
nbro
nbro
No, it's a likelihood. Random variables are specific functions that need to take inputs from the sample space. The likelihood does not have the sample space as the domain.
According to whom is a random variable? Now, you're going to say that p.m.f.s, p.d.f.s and likelihoods are random variables? Explain why you say that rather than stating without explanation. It's not just a random variable because you think it is. Do you even know the definition of a random variable?
user10354138
user10354138
It is a random variable in the probabilty space where $(x,z)$ are random variables (as set up in the first sentence of section 2.1).
Yes we know what a random variable is, and I think you are the one that doesn't understand. You have some model probability space $(\Omega,\mathcal{F},\mathbb{P})$ for which $x,z$ are, say $\mathbb{R}^n$-valued random variables, and now transfer it to underlying space $\mathbb{R}^{2n}\supseteq x(\Omega)\times z(\Omega)$, so now we can speak of joint probability density $p(x,z)$. Then we have $p(x,z)$ and $p(x)$ are real-valued random variables on $\mathbb{R}^{2n}$, $p(x\mid z)$ and $p(z\mid x)$ are defined (up to null sets) and measurable (quotient of two random variables).
nbro
nbro
Yes, $p(x, z)$ is a joint p.d.f., but how does that make it a random variable? I don't understand how your explanation makes $p(x, z)$ a random variable.
Meaning of $L^1$ random variable? Also, the relationship between the joint being a derivative and being a random variable is not clear from your comment. How does that follow? Let's start like this: what is your definition of a random variable? Isn't a random variable a function that maps from $\Omega$ to a measurable space? How does your definition imply that?
user10354138
user10354138
13:02
$p(x,z)$ is the Radon-Nikodym derivative of $\mathbb{P}^{x,z}$ with respect to the Lebesgue measure on $\mathbb{R}^{2n}$ and so is by definition measurable function on our probability space $(\mathbb{R}^{2n},\mathcal{F},\mathbb{P}^{x,z})$.
nbro
nbro
So, are you saying that p.d.f.s, p.m.f.s, likelihoods, marginals, evidences (denominator of Bayes theorem), and so on, are all random variables?? Because they are measurable by definition?
user10354138
user10354138
They are if you work on a suitable probability space.
nbro
nbro
So, the authors of that paper just use $\mathbb{E} \left [ p(x \mid z) \right]$, and then pretend that the arguments of the expectations are random variables because we can SOMEHOW make them random variables? If yes, why do they call $\mathbf{x}$ or $\mathbf{z}$ just variables and not RANDOM variables?
George Moutsopoulos
George Moutsopoulos
Even if an abuse of notation: E(p(x|z)) is E(p(x|Z)) where Z is measured by q(z) and the authors say that. It makes sense because conditioning on Z=z filters by Z. In contrast p(X,z) or p(x,Z) would not make sense even if you can always compose these functions with X or Z.
nbro
nbro
@GeorgeMoutsopoulos Where do they say that? Also, they use $p(x, z)$ there. Now, you're talking something completely different than the other guy above said.
George Moutsopoulos
George Moutsopoulos
13:02
"..and the authors say that" I misread this. In equation (11), the KL divergence, you take expectations of the log difference of two probabilities. Compose with Z and take the expectation, or integrate over q(z) as in wikipedia. In (17) where they start with $p(z_j|z_{−j},x)$, they say the expectation is with respect to $\prod_{i\neq j}q_i$ and use $E_{-j}$. When they use the joint $p(z_j,z_{-j},x)$ in (18) I understand to take the expectation as before, even if this is not as natural as equation (17).
nbro
nbro
@GeorgeMoutsopoulos You didn't get my point I think. If you look at Wikipedia: en.wikipedia.org/wiki/Expected_value, you will see that expectations are taken OF r.v.s $X$ (big capital letters), but WITH RESPECT TO densities (in case of continuous r.v.s). The "with respect to" part is fine, it's the "of" part that is not fine. If r.v.s are taken of r.v.s, how can a likelihood (or density in general), such as $p(x \mid z)$, be the argument of the expected value operator $\mathbb{E}\left[ \cdot \right]$ and thus be a r.v. (assuming that expected values only take r.v.s as inputs)?
nomen
nomen
@nbro: I don't see the problem. The expected value operator is just an integral restricted to the sample space. The reality is that every real function from the sample space is a random variable. (Heck, there are "generalized" random variables, like random matrices, which are reified by mappings from the sample space to the space of matrices)
nbro
nbro
@nomen Suppose that you initially use upper case letters to denote r.v.s. For example, you could say "consider the r.v.s $X_1, \dots, X_n$ and $Z_1, \dots, Z_m$ where the $Z_i$s give rise to $X_j$s", would it make sense to use then $\mathbb{E}\left[ p(x_1, \dots, x_n \mid z_1, \dots, z_m \right)]$? Why or why not? Or should I say instead "consider the r.v.s $x_1, \dots, x_n$ and $z_1, \dots, z_m$ where the $z_i$s give rise to $x_j$s"? Why or why not?
nomen
nomen
I mean, the notation is getting "ugly", but yes it's valid. You can definitely fix points in the domain, condition on them, and take the expectation of the pdf you got by conditioning. In fact, you might want to pick an "easy" continuous distribution and see what you get if you take E(f(X)) (excluding all the conditioning stuff, which is only relevant as a process -- once you "un-condition" a conditional density (by "assuming" the condition), you have a "regular" density).

last day (15 days later) »