4:42 AM
0
Consider the random variable $Z=X+Y$ where $X$ and $Y$ are possibly dependent. $Z$ is distributed according to law $\mu$. Say we information project $Z$ onto the space of random variables of the form $c+Y$ for constant $c$. By this, I mean we find $\nu=\arg\min_{\mu'} D_{KL}(\mu'||\mu)$. Is it tr...
In mathematical statistics, the Kullback–Leibler divergence (also called relative entropy) is a measure of how one probability distribution is different from a second, reference probability distribution. Applications include characterizing the relative (Shannon) entropy in information systems, randomness in continuous time-series, and information gain when comparing statistical models of inference. In contrast to variation of information, it is a distribution-wise asymmetric measure and thus does not qualify as a statistical metric of spread - it also does not satisfy the triangle inequality. In...
12 hours later…
5:06 PM
@MartinSleziak The tag was edited away and then again edited back: math.stackexchange.com/posts/3730974/revisions
« first day (2933 days earlier) ← previous day next day → last day (1408 days later) »