Governor

This seems quite helpful: math.stackexchange.com/questions/3763458/… but I still struggle to justify an appropriate bound

My attempt was to show the event $\{F_X(X) \le \alpha\} \subseteq $ an appropriate set including a quantile function ideally $\{X \le Q^+(\alpha)\}$ where $Q^+(\alpha) = \sup\{x : P(X < x) \le \alpha\}$ but I can't justify it...

$D(P||Q) = \sum_x P(x) \log(\frac{P(x)}{Q(x)})$ - probably want to use that the sum of the pmfs is 1 somehow but I can't see it...

Other way I think is to basically show that $S_1$ and $S_2$ can't be infinite, so the partial sums can't be conditionally convergent - however I am unsure of how to start this. I think there is a simpler way...any advice??

Defining the set $A^+ = \{x \in A | \frac{P(x)}{Q(x)} \ge 1\}$ and taking $A^-$ to be the complement I can split $D(P||Q)$ into two sums $S_1$ & $S_2$ over each disjoint support. It suffiices to show that $S_1$ (positive sum) or $S_2$ (negative sum) is finite, the easiest way I think is to apply jensens inequality in the form of the log-sum inequality but I can't do that since I don't know if the sum converges absolutely.

How would I prove for two PMFs $P$ and $Q$ on countably infinite support $A$, if $D(P||Q) < \infty \implies \sum_{x} P(x) |\log(\frac{(P(x)}{Q(x)})| < \infty$. I tried to split them over the positive and negative part sums and show each one is finite, but I can't apply the log-sum inequality to do so because I have an infinite support

nvm disregard everything i said

I can still calculate $X|X=x$, surely its density is just all concentrated at one point

Maybe this is helpful? math.stackexchange.com/questions/1857196/…

nvm how can a polynomial of degree $n$ have $n+1$ zeros, isnt that FTA?

consider a polynomial of degree $0$

yeah I do

@TedShifrin I struggle to understand what it means

@TedShifrin If I wanted to find the distribution $f_{X^2}(u)$, I can rewrite it as $E_X[f_{X^2 | X}(u|x)]$. How do I interpret whats inside the expectation

Can I not derive the distribution of $f(X) | X$?

$p(f(X) | X)$ doesn't make sense?

I guess im struggling to interpret $p(f(X) | X)$, if I condition on $X$ and $f(X)$ is wholly dependent on $X$ would that not remove most of the randomness in $f(X)$

@TedShifrin ah yeah it is 0

Lets say $X \sim N(0, 1)$

This might sound a little silly, but is $p(f(X) | X = x)$ where $x \in \mathbb{R}$ deterministic?

When you normally marginalise its just a conditional probability distribution - cs.toronto.edu/~duvenaud/distill_bayes_net/public here it is mentioned it is a likelihood

In a posterior predictive distribution, marginalising over $\theta$, $p(x|\vec{X}) = \int p(x|\theta) p(\theta | \vec{X}) d \theta$. I know $p(\theta | \vec{X})$ is the posterior, is $p(x|\theta)$ the likelihood? I would think $x$ is varying & $\theta$ is fixed as we are modelling the distribution of $x$, therefore it is the conditional distribution?

yes it is

Is $e^x$ quasi concave?

In this thread: stackoverflow.com/questions/5148744/… the person answering says there is $0$ probability of obtaining the centre point. Is this correct?

The centre point of the hexagon is assigned density $0$, I know it's a continuous variable but I want to say something mathematically correct along the lines that it is impossible that we play the centre strategy

I guess the best I can say is the probability of achieving values close to $\alpha$ is smaller than any other $\epsilon$ neighbourhood of another point.

The integral represents the probability of being in that interval

Yeah

Can I say anything special about the values close to $\alpha$

and 0 otherwise

What if my pdf was $f(x) = |x-\alpha|$ $x \in [\alpha -1, \alpha + 1]$

Yeah I know that, it's just can I say that the probability of taking values in an $\epsilon$ neighbourhood of this point is almost surely $0$

yes

@TedShifrin derivative of the cdf

Can I say that my random variable $X$ with density $f$ will never take on a value $\alpha$

$f$ is not equal to 0 on an interval about $\alpha$, but solely at the point $\alpha$ itself

Say I have a probability density function $f$, and a point $\alpha$ whereby $f(\alpha) = 0$. Can I say anything probabilistically about $\alpha$

How do I choose the $\alpha$ & $\beta$ for the line search accordingly?

I want to minimise $||Au-b||^2 + c \sum_{i=1}^{40} L_{\epsilon}(u_i)$ where $L_{\epsilon}(u_i) = 1/2 * u_i^2$ for $|u_i| \le \epsilon$ & $L_{\epsilon}(u_i) = \epsilon(|u_i| - 1/2 \epsilon)$ otherwise. Using a gradient descent method with backtracking.

Oh assume $\lambda > 0$ and $A$ is non trivial

$Null(A) \cap Null(D) = \{0\}$

Is this condition sufficient for invertibility of $A^T A + \lambda D^T D$

<3 thank you

ohhh it works

I'll have a go, I do want to find an expression for $||x||^2$

I know the value of $c$ and $d$

Any techniques to "invert" this non-invertible matrix $\begin{bmatrix} 0 & 0 & 0 &\dots & 0 & 0 \\ -a & 1 & 0 & \dots & 0 & 0 \\ \dots & \dots & \dots & \dots & \dots & \dots \\ 0 & 0 & 0 & \dots & -a & 1 \end{bmatrix} * \begin{bmatrix} c \\ x_1 \\ \vdots \\ x_N \end{bmatrix} = d \begin{bmatrix} 0 \\ u_1 \\ \vdots \\ u_N \end{bmatrix}$. The goal is to find an expression for $\vec{x}$ in terms of $\vec{u}$. But the use of that constant in my recurrence relation starts messing things up.