Mathematics - 2020-01-20 (page 3 of 3)

Let me see if I can find a concise way to answer that :)

Or is it different because you don't actually write down the symbols?

But you're still doing the same computations either way it feels like

Well

If you want to do $D(f(g(h(x))))$

and you already have $h(x),g(h(x)),f(g(h(x)))$ in memory

then symbolically finding the derivative of $D(f\circ g\circ h)$ feels like a mess

but just doing $f(g(h(x)))D(g(h(x)))$ is basically "this thing from memory times this simpler thing"

so that feels quicker

Yeah that has something to do with it (reverse-mode automatic differentiation).

Symbolic differentiation blows up expressions.

A neat thing is, if you define a ring with elements $x$ and $y$ such that $xy-yx=1$

From a practical standpoint I use AD all the time when I use PyTorch. It's almost magical: You write a computer program that computes whatever it is you want, and the backend efficiently handles the propagation of derivatives through the computation graph. You just have to say which final value whose gradients you want to obtain.

8:07 PM

(i.e. $\Bbb R(x,y)/\langle xy-yx-1\rangle$)

then $f(x)y-yf(x)$ turns out to equal $f'(x)$ for polynomials $f$

And you get the derivative of that value with respect to any intermediate value or input to the program.

Like, $x^2y-yx^2=x^2y-xyx+xyx-yx^2=x(xy-yx)+(xy-yx)x=2x$

@user76284 That sounds really useful

I think there's also a technique where you work in the ring $\Bbb R[\epsilon]/\langle\epsilon^2\rangle$, i.e. the ring where you append an $\epsilon$ with $\epsilon^2=0$

Yeah that sounds like forward-mode automatic differentiation.

If you define $f(a+b\epsilon)=f(a)+f'(b)\epsilon$ then it's all consistent

@AkivaWeinberger Regarding AD I think it's single-handedly been one of the greatest boons to the field of machine learning.

8:10 PM

That's basically what backprop is, right?

Beforehand you had to clumsily propagate the derivatives yourself for any new architecture you came up with.

Yep.

(Which is about as far as my machine learning knowledge goes)

Backprop is the chain rule!

Yeah

(computed efficiently)

8:11 PM

I know e.g. recurrent neural networks exist, I don't know how they work

Topology and computer science feel really far away from each other

Continuous versus discrete

With AD you can also e.g. compute derivatives through fixed points/roots with implicit differentiation.

Obviously if you have enough discrete points, they kinda approximate some abstract smooth surface

which is the idea behind topological data analysis I think

Dual numbers is what you were referring to. Just remembered the name.

Ah yep

The ring $K[\varepsilon]/(\varepsilon^2)$ essentially gives you all derivations on a field $K$ (the derivations on $K$ are in bijection with the ringhomomorphisms $K\rightarrow K[\varepsilon]/(\varepsilon^2)$ that are the identity $\mod\varepsilon$). This probably isn't relevant.

8:14 PM

That's interesting

I don't know a whole lot about the set of derivations of an arbitrary field $K$

@AkivaWeinberger I remember reading about some interesting connections between topology and computability theory.

I guess if $K$ is, e.g. the polynomial ring, then you can have $K\to K_\epsilon$ defined by $f(x)\mapsto f(x+\epsilon)=f(x)+f'(x)\epsilon$?

(Making up notation)

Oh and I forgot there's the differential lambda calculus.

@user76284 That sounds interesting

So I'm wondering what concept unifies all these seemingly distinct notions of differentiation/derivatives.

8:19 PM

How do I add LaTeX to a comment?

Like en.wikipedia.org/wiki/Product_(category_theory) for products.

"a notion designed to capture the essence behind constructions in other areas of mathematics such as the Cartesian product of sets, the direct product of groups or rings, and the product of topological spaces"

@Student you use dollar signs around the text, e.g. $x$. but you have to enable mathjax in chat first: see the link in the room description for instructions

I tried $\frac{a}{b}$ and $$\frac{a}{b}$$, but this outputs an empty message.

Derivation (differential algebra)

What about this?

In mathematics, a derivation is a function on an algebra which generalizes certain features of the derivative operator. Specifically, given an algebra A over a ring or a field K, a K-derivation is a K-linear map D : A → A that satisfies Leibniz's law: D ( a b ) = a D ( b ) + D ( a ) b . {\displaystyle D(ab)=aD(b)+D(a)b.} More generally, if M is an A-bimodule, a K-linear map D : A → M that satisfies the Leibniz law is also...

A derivation is a linear map that satisfies the product rule

@student both of those work just fine to me with chatjax enabled, so once you have that going it should be fine

8:21 PM

Yeah, that's the tautologous example

@user76284 en.wikipedia.org/wiki/Scott_continuity

Stuff related to this comes up in relation to topology/computability theory

@anakhro Yeah that's one of them.

Especially with the Curry-Howard-Lambek isomorphism.

@user76284 Here's something neat

@Semiclassical For me, this works in questions and answers, but not in comments.

8:24 PM

weird

Let $D$ be the derivative function from the set of polynomials to itself

inb4 write it as a matrix

and $E$ be the shift operator, e.g. $(Ef)(x)=f(x+1)$

phew

Then $E=e^D$

Meaning

8:24 PM

@Semiclassical Yes, this is true.

$E=I+D+\frac12D^2+\frac16D^3+\dotsb$

This is equivalent to Taylor's formula (which is a finite sum for polynomials)

@Semiclassical I even searched on meta for this information, but couldn't find it.

@user76284 But there's more

Let $\underline x$ be the operator that multiplies by $x$

Oh yes I love that one. I've asked a few questions about that
https://math.stackexchange.com/questions/902514/exponential-of-a-polynomial-of-the-differential-operator
https://math.stackexchange.com/questions/1341495/exponential-of-powers-of-the-derivative-operator
https://math.stackexchange.com/questions/2281246/geometric-series-of-the-derivative-operator

(underline to distinguish it from the function $x$)

8:25 PM

the physics version of the above is the translation operator $e^{-i x p/\hbar}$

Then $f(D)\underline x-\underline xf(D)=f'(D)$

where $p$ is the momentum operator $p=-i\hbar D$

Basically the same as the $xy-yx$ thing I mentioned earlier

so nothing really that strange

I write $\frac{a}{b}$ in the comment and save it, but there is nothing there. Only when editing I see $\frac{a}{b}$.

8:26 PM

this sounds like $[x,p]=i\hbar$

@user76284 Also, an operator commutes with $D$ iff it commutes with $E$

@Student huh, that's goofy

iff it's a polynomial (or formal infinite series) in $D$

@Semiclassical Why?

iff it's a series in $E-I=:\Delta$

8:27 PM

that you can only see it registering as a latex expression when editing

Yep.

You can also invert differential operators like the Laplacian (the Green's function).

that's something you use a bunch in physics, yeah

In the case of numerical computations on a lattice you literally invert the matrix of finite differences representing the derivative. That's kind of cool.

@Semiclassical What should I do to insert LaTeX in a comment?

There's a neat way to find the Taylor series of $W(x)$ (the Lambert W function, the inverse of $y=xe^x$) through this

8:29 PM

like I said, it's just putting dollar signs around it

Define $A=De^D=DE$

if that's not working, I don't know why

Then you can try to write $D$ as an infinite series on $A$

@AkivaWeinberger which branch of W(x), lol

The one around $0$?

Are there multiple there?

8:30 PM

@Semiclassical I noticed this problem in other participants as well

@Semiclassical In any case you can think of this as formally inverting the infinite series $xe^x$

You know how, if you define $d_n=x^n/n!$, then $Dd_n=d_{n-1}$, $d_0=1$, and $d_n(0)=0$ (for $n\ne0$)?

Basically $d_n$ is the $D^{-n}1$ (or that's the idea anyway)

You can define $a_n$ similarly with $A$ instead of $D$

lambert-w function has infinitely many branches, analogously to how the log function does

After computing some examples you can find a pattern and write $a_n$ explicitly

and the branching is kinda strange

and once you have that, it's not too hard to write $D$ in terms of $A$ by seeing what it does to $a_n$

8:32 PM

Reminds me of the umbral calculus.

And that's how you find the Taylor series of the Lambert W function through operators on the set of polynomials

@user76284 Indeed that's where this is from

I didn't read the entire article on the umbral calculus,

but I read far enough to realize that this was possible

(This also means I still don't quite understand the umbral calculus or know what it's about)

source for the branch structure: en.wikipedia.org/wiki/…

This looks interesting: ksda.ccny.cuny.edu/PostedPapers/Keigher022015.pdf

note that the branching is different for the principal branch than all the others

@user76284 Unfortunately was not recorded

The talk, I mean

8:39 PM

Reminds me of the weird category-theoretical definition of integration

If we look back at the definition of differentiation

$\lim\frac{f(x+h)-f(x)}h$

This basically wants some notion of topology ($\lim$)

(In the algebraic setting, is this the Zariski topology?)

@Thorgott This one? math.stackexchange.com/questions/728069/…

and some way of dividing

That said, this isn't the definition we use in, say, multivariable calculus

I don't know. The one I saw was in a short note from Tom Leinster

@user76284 I don't know if you're familiar with nonstandard analysis

8:43 PM

A bit.

but the techniques generalize pretty directly to "nonstandard topology"

There's a notion of metric differential

Basically, we construct a set called $\Bbb R^*$ called the hyperreals, which is in a sense "the reals plus infinitesimals"

and it's constructed in such a way that things in $\Bbb R^*$ can tell us about things in $\Bbb R$

The dual numbers are kind of like a simplified version of that

Yep.

I wonder if you can "simplify" $X^*$ for other objects $X$ to get things that are useful for computation

If I remember right, $X$ is a compact topological space if, for all $x^*\in X$, there is a $y\in X$ such that $x\approx y$

Like take $X=(0,1)$ and let $x=1-\epsilon$, say

Then the only real that $x$ is infinitely close to is $1$, but that's not in $X$

Or if $X=\Bbb R$ and $x=N$ where $N$ is an infinitely large hyperreal

@Thorgott Does that assume $r\to0^+$?

8:52 PM

I always forget how the hyperreals work

"A space $X$ is compact if its hyperreal extension $^*X$ has the property that every point of $^*X$ is infinitely close to some point of $X \subset {^*}X$."

Sounds right.

Are they just an ultrapower of $\Bbb R$ over a countable index?

Yeah

For general topological spaces it can't just be countable, I don't remember how it works in general

but for the reals yeah

I'm confused by the general topological spaces thing

I mean a topological space isn't a first order structure in a nice way

Second-order statements translate similarly, you just need to translate, eg $\forall a\in A\subset X$ as $\forall a\in A^*\subset X^*$

or $^*A\subset{^*X}$, whichever

Add stars to all the sets

8:55 PM

Hm. Is ${^\ast }X$ elementarily equivalent to $X$? Is there a Los's theorem for second order stuff?

Probably, not sure

Yes, it does.

If you do the hyperreal extension of the set-theoretic universe, you get Nelson's "IST" approach to nonstandard analysis

which is equivalent to adding a predicate to the language of ZFC and some axioms

I'm very skeptical of a second order Los, being finite can be written in second order logic, but ultraproducts of finite things are often infinite

$|^*X|$ can be a hyperinteger

which is "hyperfinite" in a sense

In other words, when you translate the second order sentence, you go from claiming things have a bijection with a prefix of $\Bbb N$ to claiming things have a bijection with a prefix of $^*\Bbb N$

9:00 PM

Oh this is nice

(and the bijection has to be internal in a sense)

Instead of taking an ultrapower of my structure $N$ I take an ultrapower of the universe, and then the $j\colon V\prec M\simeq V^I/U$ coming from it gives me also an elementary embedding of $N$ into an ultrapower

What are $V$, $M$, and $U$

$V$ is the von neumann universe, $M$ is $V^I/U$ because I wanted to give a shorter name and in set theory the codomain of an elementary embedding of the universe is always called $M$, $U$ is an ultrafilter over $I$

Ah

Q: $ f(x) f’(x) f’’(x) \neq \exp(P(x)) $?

9:06 PM

Taking ultrapowers of the universe is a very common operation in set theory since most large cardinals can be characterized as critical points of the associated elementary embedding

mick

0

Let $P(x) $ be a nonconstant polynomial. Then I assume $$ f(x) f’(x) f’’(x) \neq \exp(P(x)) $$ Using complex analysis , I argue that every entire function $ f , f’ $ or $ f” $ must have a zero. And ofcourse $ \exp(P(x)) $ is never zero. I’m not so good at complex analysis though. And I wo...

@AlessandroCodenotti So is this a small universe inside a larger universe, or something?

@mick If $f(x)=e^x$, then $f(x)f'(x)f''(x)=\exp(3x)$, no?

So $\exp(P(x))$ with $P(x)=3x$

@AkivaWeinberger Almost but there is a potential issue (well a small issue is that the equivalence classes of $V^I/U$ are class sized, but we just pick representative of minimal rank and get away with it). A more serious issue is that $V^I/U$ also has a membership relation $\in_U$ and that doesn't have to be well-founded

In fact $\in_U$ is well founded iff $U$ is $\omega_1$-complete

That means $U$ knows what $\omega_1$ is?

In that case though we can take the Mostowski collapse of $V^I/U$ to get a transitive model $M_U$ which is indeed an inner model of $V$

@AkivaWeinberger It means that $U$ is closed under countable intersections

9:13 PM

OK

If there a cardinal with a nonprincipal $\omega_1$-complete ultrafilter there is also a measurable cardinal though

Conversely if $j\colon V\prec M$ is a nontrivial elementary embedding of the universe in an inner model (nontrivial means that there is an $x$ with $j(x)\neq x$, and looking at the rank of $x$ we can assume that an ordinal is moved), then $\mathrm{crit}(j)$, that is the least ordinal moved by $j$, is a measurable cardinal

Is that the definition of measurable?

Usually the definition is "a cardinal $\kappa$ is measurable if it has a nontrivial $\kappa$-additive two valued measure $\mu\colon\kappa\to\{0,1\}$" (nontrivial meaning that the measure on singletons is zero)

This is equivalent to having a nonprincipal $\kappa$-complete ultrafilter over $\kappa$

Which is equivalent to being the critical point of an elementary embedding of the universe in an inner model

So you can take the one that works the best for your purposes as definition :P (there's also surely more equivalent characterizations)

The nice thing about the characterization in terms of elementary embeddings is that we this gives a natural starting point to build more of the large cardinals hierarchy. Since $V$ is already maximal we can ask for "largeness" conditions on $M$

For example we can prove that if $\kappa$ is measurable and $j\colon V\prec M$ witnesses that, then ${^\kappa}M\subseteq M$

And we can ask for a bigger exponent and get so called supercompact cardinals

Here's a homework problem that I feel like was probably a mistake

Let $A=\{2^n+3^m:n,m\in\Bbb Z\}$. What is the set of accumulation points of $A$, and what is the set of isolated points?

The set of accumulation points is $\{0\}\cup\{2^n:n\in\Bbb Z\}\cup\{3^m:m\in\Bbb Z\}$, if I'm right

The problem is the set of isolated points

'cause these aren't disjoint

The cool thing is that the natural upper limit of this hierachy would be the critical point of an elementary embedding $j\colon V\prec V$. This is called a Reinhardt cardinal

9:25 PM

$\{2,3,4,9\}\subset A'\cap A$

because, e.g. $9=3^2=2^3+3^0$

It is a theorem (Kunen's inconsistency) that Reinhardt cardinals are inconsistent with ZFC

So that means, what, there are no elementary embeddings $j:V\prec V$?

Not in ZFC

Well there's the trivial one

But no interesting ones

(Cont'd: so the set of accumulation points is $A\setminus\{2,3,4,9,\dots\}$)

(where I don't know if the $\dots$ continues)

(We need $|2^n-3^m|=1$)

Kunen's inconsistency actually shows that for no ordinal $\delta$ there can a nontrivial elementary embedding $V_{\delta+2}\prec V_{\delta+2}$. So the next strongest thing is a so called $I_1$ cardinal: the critical point of a nontrivial elementary embedding $V_{\delta+1}\prec V_{\delta+1}$

(In this case $\delta$ actually has to be the least fixed point of $j$ above the critical point, given by $\sup j^n(\mathrm{crit}(j)$)

And of course we can ask for the critical point of an elementary embedding $V_\delta\prec V_\delta$. This is called an $I_3$ cardinal

(where's $I_2$ you might ask? In between, but it's a bit more technical. There's also $I_0$ which is between $I_1$ and inconsistency but I don't know anything about that)

9:37 PM

And I'm guessing it's an open question whether these are consistent

(relative to the consistency of other things)

Just finished a particularly annoying problem set @AlessandroCodenotti

they are they own level in consistency strength pretty much, so unless they're proven downright inconsistent (which was the original opinion, but now they are believed to be consistent) there's no hope in proving them consistent relative to something nice I'm afraid

@AkivaWeinberger In which subject?

Real analysis

So one question was, what's the closure of $\{\langle x,\sin1/x\rangle:x\ne0\}$

Easy enough

Except I realized I didn't quite know how to prove my answer

So that took a while

Basically had to remind myself why the graph of a continuous function is closed

You get an extra segment I'd say?

Yeah

The annoying bit was showing you don't get anything else

The other annoying question on it was the $2^n+3^m$ one

I see

9:43 PM

What's the set of isolated points of $A=\{2^n+3^m:n,m\in\Bbb Z\}$? I think it's $A\setminus\{2,3,4,9\}$

I'm confused by that one. Which topology are you thinking about?

Oh, wait negative exponents, I see now

Looks awful, I'm out

10:32 PM

twitter.com/robertghrist/status/1219377496447242248

Henry Wente died today

I only learned about Wente tori about a month ago

$user image$

Wow, that makes me feel old. He wasn't too much older than me.

I remember hearing him talk about 'em.

Well, I guess he was a bit older than I remembered. Crazy.

10:49 PM

@TedShifrin hey Ted!

A cool image:

From here: https://twitter.com/NathanielVirgo/status/1219367199422517248

> A photography technique I invented: take a series of time lapse photos, colour each one with a different hue using colours from the spectrum of light, then recombine them. The colour of each pixel depends on how its brightness changed over time. - Nathaniel Virgo, Twitter

@AkivaWeinberger that's so cool

NoName

11:04 PM

Does Scribd pay people for their lecture notes? They have put every lecture note on earth behind a paywall.

11:21 PM

TIL that size 12 pt font is (supposed to be) 1/6 inch

Hey is $x \notin A - B$ equivalent to $x \notin A$ or $x \notin B$ or $x \notin A$ or $x \in B$?

Hi @Stan

Yikes, @Abwatts. Too many ors here.

Have you drawn a Venn diagram to see what it looks like?

@TedShifrin Srry :P not really, I should probably.

Either that or write the definition of $x\in A-B$ carefully.

But just to be clear, aren't we negating this statement since $\notin$ indicates negation?

11:29 PM

Well, how can $x\in A-B$ fail? You can see that from the Venn diagram and then write a proof.

@TedShifrin For class, we had to find the directions of the major and minor axes of an ellipse and I decided to use eigenvectors to do it. But I'm kind of getting mixed up.

Of course you should use eigenvectors.

But we haven't got to that yet :D

I drew it out and got that it's equivalent to $x∉A$ or $x∈B$ or alternatively $A ∩ B$. Does that seem right?

No.

@TedShifrin yeah none of my classmates did that, but i wanted to do it cuz i figured it might tie into what i will learn next from ur problems

My equation is of the form

$$\mathbf{(x-v)}^T A^{-1}\mathbf{(x-v)} = 1$$

So did eigendecomposition of this and got like

$A^{-1} = RLR^{-1}$ where R is a rotational matrix and $L$ is a 2x2 matrix with the eigenvalues.

11:34 PM

What do you mean "or alternatively"?

But i'm having trouble because i don't know whether the eigenvalues are the eigenvalues of $A$ or $A^{-1}$

@Stan: So $A$ is symmetric?

Yes, its the covariance matrix

You want eigenvalues of $A^{-1}$ here, since it's defining the conic. So spectral theorem tells you that when you use the eigenvector coordinates it becomes $\sum \lambda_i x_i^2 = 1$. So what are the semi-axes?

@TedShifrin somehow my current solution has skipped the concept of a semi-axis

11:36 PM

@Abwatts: If we negate $x\in A$ and $x\notin B$, we do get $x\notin A$ or $x\in B$. I didn't understand your "alternatively."

:'D

You said you wanted the major and minor axes, @Stan?

I need the direction of the major and minor axes. I thought the eigenvectors would give me that

Oh, just directions. So those are eigenvectors. Those are the same for $A$ and $A^{-1}$, of course.

@TedShifrin I'm sorry, I just confused myself. Nevermind that. I am just a bit unsure how does the $\notin$ symbol works - is that equivalent to negating the expression?

11:39 PM

$x\notin A$ is the negation of $x\in A$, yes.

@TedShifrin well i need the magnitudes too...i thought those were the inverse of the square root of the eigenvalues

Well, look at the equation I wrote up there. Yes, that's right.

It's correct, provided your RHS is $1$.

I don't like memorizing things like that because you have to keep track of exactly what the conditions must be.

@TedShifrin yes, i normalized my right hand side by dividing and included that into the magnitudes for the major and minor axes

@TedShifrin does this generalize to higher dimensions

like ellipsoids

Well, putting the quadratic in the form of sum of squares works in any number of dimensions.

that's so cool. yeah, when I sat down to do this hw problem, I asked myself "how would Ted suggest doing it? whichever that is, i should put in some extra effort to learn"

11:42 PM

Remembering that we're using the eigenvectors as a basis, you want the value of $x_i$ when all the others are $0$ to get the $i$th semi-axis.

it took a lot more time but i figured it out after a lot of reading

LOL, you're silly.

yeah well i need some justification cuz my friends thought i was just working too hard

:'D they were like "why do it that way?"

How did they do it?

@TedShifrin So, would $x \notin A - B$ be equivalent to $x \notin A$ or $x\notin B$ since we can simply replace $\notin$ with ¬?

11:43 PM

So I wrote that out for you above somewhere, @Abwatts.

No, that's not right.

NOT ($x\in A$ AND $x\notin B$).

@TedShifrin one guy wrote out like Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0 or something. and then borrowed the formulas from wikipedia and then just assumed they were true

Oh, right, using horrible formulas from Wiki is obviously the way to know what you're doing.

You get my vote.

also, if I never use the concepts I learned, how will I ever apply them?

The Wiki approach is very special to 2 dimensions.

So, can we just distribute the NOT, thus getting $x \notin A$ or $x \in B$? I'm really confused..

11:45 PM

It's super easy to find eigenvectors of $2\times 2$ matrices, anyhow.

@TedShifrin yeah exactly. and i thought this one with the eigenvectors would work if i were given beyond 2 dimensions

which seems likely in other cases

plus another reason is my professor last quarter kept referencing this approach

@Abwatts: Surely you've studied this? NOT ($X$ and $Y$) is (either) not $X$ OR not $Y$. Make sure you understand that completely.

and i couldn't follow what she was saying, so i thought i should take the time to learn it to make sure i can understand professors going forwards

Of course, they should play up all the linear algebra in statistics, not ignore it.

Gotta understand de Morgan

11:47 PM

@Abwatts: Understand it with common sense. If I say you are wearing a green shirt and blue pants, how do you show that I'm wrong?

And same question later with or.

@TedShifrin yeah and i thought it actually made the intuitions clearer too

@TedShifrin can you expand on what you mean by "defines the conic"?

A conic (or quadric, in higher dimensions) has the equation $x^\top Ax = c$ for some symmetric matrix $A$. That's all I meant.

I'm just confused about what would be the case if we have NOT ($X$ and NOT $Y$).. Does the NOT and NOT cancel each other out when we distribute the NOT?

Yes, NOT NOT is "yes" :P

On the note of stats and linear algebra / geometry, I’m disappointed this question hasn’t gotten more attention :

11:50 PM

@TedShifrin super! ok gtg eat dinner. later amigo!

Later, @Stan.

Q: A geometric interpretation (angles) of mathematical expectation and covariance.

Oops

2

I have a habit of understanding everything geometrically. In this case, the victim is probability theory. I will assume that we are working on $L^2 (\mathcal{F})$, the space of integrable random variables $X^2$, a Hilbert Space. Here we have a well defined inner product given by: $$<X,Y> = E[X...

probability-theory geometric-probability

So, just to clarify $x \notin A - B$ can be re-written as NOT ($x\in A -B$) <=> NOT ($x \in A$ and $x\notin B$) <=> $x\notin A$ or $x\in B$ (because NOT * $\notin$ = $\in$?)

Yes, that's correct.

It finally clicked, thanks!