the no-normie zone - 2017-08-30 (page 2 of 2)

Balarka Sen

20:00

Maybe do this on the Pujas :P It's really actually formal mathematics so I doubt you'd need it in that level of rigor in your life

Anonymous

@BalarkaSen Ah, that's a good idea!

Anonymous

:D

Anonymous

Sure

Anonymous

Ok...next thing that I'm confused with:

Balarka Sen

Mhm?

Anonymous

20:03

Could you teach me how to find maxima minima of multivariable functions by the method of partial derivatives in brief? It seems very hand-wavy...taking partial derivatives w.r.t x and y and then...

Anonymous

And the Lagrange multiplier stuff...I couldn't understand it properly

Balarka Sen

Oh absolutely

It's my favorite thing

Anonymous

There's something called Taylor expansion in two variables

Anonymous

They just gave the formula in my book

Anonymous

I couldn't understand the derivation

Anonymous

20:05

@BalarkaSen Okay!

Balarka Sen

Man it'd be so cool if you got hold of Ted's multivariable book

Anonymous

@BalarkaSen $$$ :P

Balarka Sen

yeah

Anonymous

You bought it?

Anonymous

It's Rs.25,000 or something

Balarka Sen

20:06

nah he gave a soft copy to me.

Anonymous

Ummm, maybe he could lend it to me ? :P

Anonymous

Or could you?

Anonymous

(Take his permission perhaps)

Balarka Sen

Yeah but it wouldn't be ethically the right thing to do, given that I was sent on the basis of not further propagation of the soft copy. I am sure he'd give it to him if you interacted with him on calculus on a regular basis but...

On the other hand I could just teach you whatever he's written :P

Anonymous

@BalarkaSen Alrighty...you teach me :P

Balarka Sen

20:10

So max/min, right?

Anonymous

@BalarkaSen Yes!

Balarka Sen

Do you just have two variable functions, or several variables?

Anonymous

@BalarkaSen Let's start with 2. We can surely extend that concept later I guess

Anonymous

We have upto 3 I think

Balarka Sen

Right. Ok, let's do it.

So suppose I have a two variable function $f : \Bbb R^2 \to \Bbb R$ and suppose $(x_0, y_0)$ is a local maximum slash minimum.

Anonymous

20:13

Okay

Balarka Sen

That means there is a small disk $D \subset \Bbb R^2$ around $(x_0, y_0)$ such that $(x_0, y_0)$ is the unique maximum slash minimum of $f$ on $D$

Anonymous

Agreed

Balarka Sen

I then claim $\displaystyle \frac{\partial f}{\partial x}(x_0, y_0) = \displaystyle \frac{\partial f}{\partial y}(x_0, y_0) = 0$.

Anonymous

Alright...we then need to prove the validity of the claim

Anonymous

Couldn't it be an inflection point or something also

Balarka Sen

20:16

Carefully notice my statement. I said if it's a maximum slash minimum then the partials vanish at that point.

Not the converse.

Indeed, the converse is false. The partials can vanish even if $(x_0, y_0)$ is not maximum slash minumums. Those are called saddle points, indeed generalizations of inflection points.

Anonymous

Oh...gotcha

Anonymous

Go on :)

Balarka Sen

Anyway, the proof is not to hard. Consider the function $f(x, y_0) : \Bbb R \to \Bbb R$ (I fixed the $y$ variable to $y = y_0$). That has a local max/min at $x = x_0$.

What does Fermat's theorem from one variable calculus tell you?

Anonymous

What is Fermat's theorem? I never heard...lemme check

Balarka Sen

Oh maybe you don't know it by that name

Like, the first derivative test

If a function $f(x) : \Bbb R \to \Bbb R$ has local max/min at $x = x_0$, then $df/dx(x_0) = 0$ right?

Anonymous

20:20

Oh...f'(x)=0 at extremum points...that one?

Anonymous

Yes

Balarka Sen

Yup

So you apply that to $f(x, y_0)$.

That tells you $f'(x, y_0) = 0$ at $x = x_0$

Namely, $\displaystyle \frac{\partial f}{\partial x}(x_0, y_0) = 0$

Anonymous

Okay...that seems correct

Anonymous

I think if we could see a 3D graph

Anonymous

That'd be better

Anonymous

20:22

Lemme try to find a 3D graph like that....

Anonymous

one min

Balarka Sen

Let me upload one from Ted's book.

The first two pictures

Anonymous

$\delta x$ can be thought of as a small nudge nudge in the $x$ direction, keeping $y=y_o$. $\delta f$ gives the corresponding change in the value of $f$

Balarka Sen

Right

Anonymous

Ah, makes sense now :)

Anonymous

20:25

go on :D

Balarka Sen

All you're doing is restricting $f$ to the line $y = y_0$; in which case you get a function from that line to $\Bbb R$ (a one variable function) which has a max/min at $x = x_0$

So you apply the first derivative criteria

Anyway, that proves the big theorem

Anonymous

@BalarkaSen Right...gotcha!

Anonymous

And what would be the saddle point criteria?

Balarka Sen

I'm getting to that :)

Anonymous

Okaies :d

Balarka Sen

20:28

So, the question is if it's true that for a point $(x_0, y_0)$ on $\Bbb R^2$, $\partial f/\partial x(x_0, y_0) = \partial f/\partial y (x_0, y_0) = 0$ means $(x_0, y_0)$ is a maximum or a minimum of $f$

As you know, that's not even true in one variable calculus ($y = x^3$ at $x = 0$)

Anonymous

Right

Balarka Sen

A counterexample to this statement is given in the 3rd figure above; the saddle point

This is the graph of $z = xy$ by the way

Anonymous

Yes

Anonymous

That has a saddle point

Anonymous

The xy graph

Balarka Sen

20:31

The reason 1st derivative test fails is because along $y = y_0$, one axis of the saddle, $(x_0, y_0)$ has a maximum, and along $x = x_0$, the other axis of the saddle, $(x_0, y_0)$ is a minimum

So you pick up $f'_x = f'_y = 0$ in both cases

But it's not a global maximum or minimum or any o' that shit

Anonymous

Yep

Anonymous

"In mathematics, a saddle point or minimax point is a point on the surface of the graph of a function where the slopes (derivatives) of orthogonal function components defining the surface become zero (a stationary point) but are not a local extremum on both axes."

Anonymous

What they mean by orthogonal function components...I couldn't understand that...but I guess you're coming to that :)

Balarka Sen

Orthogonal means perpendicular. It means the two axes of the saddle along which (x0, y0) is a minimum and a maximum respectively

it's some stupid terminology they made up

eff that

Anonymous

Here it's the x and the y axes

Anonymous

20:34

Isn't it?

Balarka Sen

Ah, right

I agree

Anonymous

Gotcha :D

Balarka Sen

So uh

turns out it's more complicated than that

There are counterexamples to the converse of the 1st derivative test other than saddles in the multivariable world

In fact lots of lots of strange counterexamples

Anonymous

Interesting :d

Balarka Sen

whereas in the single variable world f'(c) = 0 immediately just means c is a max/min or a saddle like thing (y = x^n for odd n)

@Blue here's an oddball counterexample that haunts mathematics

Monkey saddle

In mathematics, the monkey saddle is the surface defined by the equation z = x 3 − 3 x y 2 . {\displaystyle z=x^{3}-3xy^{2}.\,} It belongs to the class of saddle surfaces and its name derives from the observation that a saddle for a monkey requires three depressions: two for the legs, and one for the tail. The point (0,0,0) on the monkey saddle corresponds to a degenerate critical point...

You see, saddle is two hills two valleys, right?

Moneky saddle is three hills three valleys

Anonymous

20:38

One sec...I can spot two saddle points on either side

Anonymous

Where's the third?

Balarka Sen

There are no saddle points.

At the center there is one point where $f'_x = f'_y = 0$ happens. Exactly one.

Surrounding it are three mountains

Anonymous

Oh...yes

Anonymous

Right

Balarka Sen

It's called a monkey saddle because it's a horse saddle for a monkey to sit in. Google for images, it's too cringy to post here :P

Anonymous

20:41

One...min...I want to point out something in that picture with the snipping tool

Anonymous

That part is confusing

Anonymous

You see the two perpendicular axes I drew (in blue)

Anonymous

It looks exactly like a saddle point

Anonymous

Along one axis that part seems like a minima

Anonymous

20:44

And along the other it seems like a maxima

Anonymous

@BalarkaSen

Anonymous

Atleast locally that should be a saddle point, isn't it?

Balarka Sen

@Blue Ah, but the issue is $f'_x \neq 0$ there

To be a saddle point the tangent plane at that point needs to be parallel to the xy plane

Which doesn't happen there

But yeah I think if you tilt it a little it should be a saddle point

Anonymous

A tilted saddle point :P

Anonymous

We had things like that in 2D also iirc...tilted inflection point

Balarka Sen

20:47

It's not a saddle point of the given function, nonetheless

Yup

@Blue But really the thing that's annoying is the thing in the middle

There you get $f'_x = f'_y = 0$ (the tangent plane is parallel to the xy plane)

But it's not a saddle.

You can check it even. The formula is $f(x, y) = x^3 - 3xy^2$, right?

$f'_x(0, 0) = f'_y(0, 0) = 0$

Anonymous

That type of thing occurs for $f(x,y)=x^3$ also I think...that $f_x'=f_y'=0$ but it's not a saddle

Balarka Sen

Ah... that works.

Oh by the way points $(x_0, y_0)$ such that $f'_x(x_0, y_0) = f'_y(x_0, y_0) = 0$ are called critical points of the function $f$ by the way.

@Blue In this case it's even more anomalous, because there is no isolated critical point. Namely, every point along the $x = 0$ axis is a critical point.

Anonymous

@BalarkaSen Right!

Balarka Sen

Strange, strange business.

Do you have second derivative tests?

Anonymous

Actually that thing didn't seem very surprising to me...probably we can check along two perpendicular axes

Anonymous

20:54

@BalarkaSen I'm checking

Anonymous

One sec

Anonymous

No...doesn't seem so

Anonymous

But could you say in short?

Anonymous

Seems analogous to 2D

Anonymous

Just cut along an axis and perpendicular to xy plane

Anonymous

20:56

And check the nature of graph along that cross section

Anonymous

If one comes out to be a minima and the perpendicular one comes out to be a maxima then it's a saddle point.

Balarka Sen

Yes, excellent!

Anonymous

Or it could be an inflection point like $x^3$ along that slice

Balarka Sen

So I guess for saddle you have to check $f'_x = f'_y = 0$

and if the sign of $f''_x$ and $f''_y$ are different

Anonymous

@BalarkaSen Right...pretty intuitive

Balarka Sen

20:58

because then you'll get max along one cross section and min along the other

@Blue Second derivative test is a way to identify which critical points are max, min and saddles.

It uses an analogue of second derivative in single variable calculus, namely, Hessians

A matrix of mixed partial derivatives. Beautiful little monster.

Anonymous

@BalarkaSen Yes. We just need to slice along a plane perpendicular to xy plane and apply the rules of our good old 2D :)

Anonymous

Well, now the Taylor theorem stuff

Anonymous

:P

Anonymous

Before that:

Anonymous

$$\frac{\partial^2 f}{\partial x \partial y}=\frac{\partial^2 f}{\partial y \partial x}$$...when is this true?

Anonymous

21:01

I lack intuition for this

Balarka Sen

It's called Clairaut's theorem. It's true whenever the second order partial derivatives of $f$ are continuous.

Namely, whenever $f$ is "$C^2$"

Continuously 2ice differentiable

It's not too hard to prove

Anonymous

I see

Anonymous

And would that hold for n number of variables too?

Anonymous

If their second order partial derivatives are continuous

Balarka Sen

Sure.

it extends to n-order partials too, if the function is C^n

Anonymous

21:06

$\frac{\partial^3 f}{\partial x \partial y \partial z}=\frac{\partial^3 f}{\partial y \partial x \partial z}$....this?

Anonymous

and any random combo of denominators

Anonymous

@BalarkaSen Aha

Balarka Sen

You need to assume $f$ is $C^3$ for that.

Anonymous

Gotcha! :D

Balarka Sen

So in particular if $f$ is smooth, so all the partials ever exist, you need not keep track of the order of the things

Anonymous

21:07

I starred that...we can get back to the proof another day (soon) :)

Anonymous

Pinned :D

Balarka Sen

OK. I can also give you a counterexample as an exercise.

Anonymous

@BalarkaSen Okay?

Balarka Sen

Consider $f : \Bbb R^2 \to \Bbb R$, given by $f(x, y) = xy \cdot (x^2 - y^2)/(x^2 + y^2)$ when $(x, y) \neq (0, 0)$ and $f(0, 0) = (0, 0)$.

Compute $\partial^2 f/\partial x \partial y$ and $\partial^2 f/\partial y \partial x$ at $(0, 0)$ and show that these do not agree.

It's a long computation, do it when you have nothing better to do :P

Anonymous

I guess $f$ is $C^2$ ?

Anonymous

21:09

in that question

Anonymous

Or no?

Balarka Sen

Oh, it isn't! If it was then the mixed partials would agree by Clairaut's theorem.

So it's a corollary of that exercise I gave you that $f$ is not $C^2$

Anonymous

@BalarkaSen Then I think counter-example is not the right word :p...sounds like you are giving an exception to the rule

Balarka Sen

Oh. Maybe you're right.

Anonymous

Anyway...got your point :)

Anonymous

21:11

Alright...so coming to the taylor theorem for 2 or more variables

Anonymous

How to derive it?

Balarka Sen

I was giving a counterexample to "For any second differentiable $f$, the mixed partials satisfy $f_{xy} = f_{yx}$".

@Blue Eh, how about we do Langrange optimization instead of Taylor now? I feel it's more exciting :P

Anonymous

@BalarkaSen Alrighty :D

Balarka Sen

Deriving multivariable Taylor is basically a long ass computation. I can tell you, but maybe not in the middle of the night :P

Mean value theorem everywhere

@Blue OK

Anonymous

This is really exciting...I don't feel like sleeping anymore :P

Anonymous

21:14

:D

Balarka Sen

loool

I need to go afk for 5 minutes, do you want to wait up?

Lagrange is too exciting to not talk about today

Anonymous

Sure sure sure....a hundred times :D

Balarka Sen

Great

Anonymous

Meanwhile let me sneakily get some food from the fridge

Anonymous

:P

Balarka Sen

21:20

lol

Anonymous

Got some ice-cream :P

Balarka Sen

I always have emergency cookies saved for myself

Anonymous

My mom had hidden it deep inside the fridge

Anonymous

behind other items

Anonymous

lol

Balarka Sen

21:21

kek

Ok, Lagrange

Anonymous

okies :D

Balarka Sen

So now we know how to find (potential) maximums or minimums of a function $f : \Bbb R^2 \to \Bbb R$; by looking at the points $p$ where the vector $\nabla f(p)= \begin{bmatrix} f'_x(p) \\ f'_y(p) \end{bmatrix}$ is the zero vector (those are the critical points), and checking if $p$ really is a max/min

Anonymous

Yes....agreed

Balarka Sen

$\nabla f$ is called the gradient operator by the way. In FISIKS notation, it's the same as $\displaystyle \nabla f = \frac{\partial f}{\partial x} \hat{i} + \frac{\partial f}{\partial y} \hat{j}$

Anonymous

Right

Balarka Sen

21:26

Anyhow, the goal of Lagrange optimization is to maximize slash minimize a function $f : \Bbb R^2 \to \Bbb R$ with respect to the constraint equation $g = 0$ where $g : \Bbb R^2 \to \Bbb R$ is another function

That means, find maximum/minimum(s) $(x_0, y_0)$ of $f$ such that $g(x_0, y_0) = 0$

Anonymous

Yep!

Balarka Sen

Like, a basic example is maximizing the area of a closed loop formed by a piece of wire

where the constraint equation is length of the wire being fixed

so this is much more useful in real life

Anonymous

Say.....maximizing $x^2+y^2$ given $x+y=1$ or smh

Anonymous

yes

Balarka Sen

yeah, for sure

Anonymous

21:30

Alright

Anonymous

Then?

Balarka Sen

So Lagrange optimization says

@Blue Actually I should clarify this thing a bit more

Anonymous

ok?

Balarka Sen

If $p$ is a point such that $g(p) = 0$, $f$ is max at $p$ constraint to $g = 0$ if $f(p)$ is the maximum among all values $f(x)$ of $f$ nearby $p$ where $g(x) = 0$.

That means, like, $g(x, y) = 0$ is a (implicitly written) curve in $\Bbb R^2$

You draw the curve, and look at the values of $f$ at each point of the curve

$p$ is a local maximum of $f$ on that curve. If $D$ is a small disk around $p$, $f$ attains maximum at $p$ on $D \cap \{(x, y) \in \Bbb R^2 : g(x, y) = 0\}$

No, that's a surface in R^3!

$g(x, y) = 0$ is a curve in $\Bbb R^2$. Think about $x^2 + y^2 - 1 = 0$

Anonymous

Gotcha

Anonymous

21:37

@BalarkaSen Right...right

Balarka Sen

Ok, here's the statement of Lagrange optimization

$f, g : \Bbb R^2 \to \Bbb R$ be $C^1$ functions. Suppose $g(p) = 0$ and suppose $\nabla g(p) \neq (0, 0)$ (not the zero vector). If $p$ is a local maximum of $f$ subject to the constraint $g = 0$, then $\nabla f(p) = \lambda \nabla g(p)$ for some scalar $\lambda$.

The vector $\nabla f(p)$ is a multiple of $\nabla g(p)$. And the multiplier, $\lambda$, is called the Lagrange multiplier :)

There's an intuitive way to understand this. Let me know if you grok the statement.

Anonymous

Got the statement...but I'm lacking intuition :P

Anonymous

Go on

Balarka Sen

Beautiful picture coming in:

This is a lake. A boat is on the origin $(0, 0)$, meeting point of the two axes.

The shoreline of the lake is given by the curve $g(x, y) = 0$

Anonymous

mmm...hmm...great pic

Balarka Sen

21:46

The guy in the boat wants to locate the closest point to shoreline from the origin, so he can row his way out of the lake

Before that, notice it's a constraint optimization problem

Let the distance from the point $(x, y)$ on the shoreline to the boat be $f(x, y)$

We want to minimize $f(x, y)$ constraint to the shoreline constraint $g(x, y) = 0$

Do you agree?

Well, uh, here $f(x, y)$ is nothing but $\sqrt{x^2 + y^2}$ I guess... it's the distance from $(x, y)$ to $(0, 0)$.

Anonymous

Right. Here the circles represent the level set? Or no?

Balarka Sen

Right, exactly, let me get to that

The guy on the boat drops a rock at the origin on the lake

So a ripple forms in circles and expands slowly

Those circles are the ripples. It is of course nothing but $f(x, y) = c$, where $c$ is the radius of the ripple after some time.

At some point it has to hit the shoreline. When it hits the shoreline the first time, the point $(x_0, y_0)$ of the shoreline it hits at is exactly the solution to the optimization. It is the minimum of $f(x, y) = 0$ with the constraint equation $g(x, y) = 0$.

Anonymous

@BalarkaSen I don't get this statement. Why should f(x,y) be the distance function...can't it be any function from R^2 to R ?

Balarka Sen

This is the point $\mathbf{a}$ in the picture.

Anonymous

I'm missing something

Balarka Sen

21:52

@Blue Well, in this context, we're thinking about distance from a point on the shoreline to the boat, right?

The boat is at (0, 0)

that's why the function of interest is $f(x, y) = \sqrt{x^2 + y^2}$

Anonymous

Ok. So you defined $f$ like that

Anonymous

I see

Balarka Sen

But in general this is a nice intuitive picture

Yeah

You can keep this picture in mind for general cases too

Anonymous

Okay, got it till here!

Balarka Sen

So the final ripple that touches the shoreline first has to be tangential to it, right?

Anonymous

21:55

Yes. For sure

Balarka Sen

So $f(x, y) = c_0$ is tangential to $g(x, y) = 0$ for some $c_0$ that's the optimum level set

Anonymous

Agreed

Balarka Sen

If two curves are tangential, what can be said about their normals at the point of tangency?

Anonymous

Same direction...the normals

Anonymous

(?)

Balarka Sen

21:56

right

i.e., normal of one is multiple of normal of the other

Anonymous

Yep!

Anonymous

So we can extend this concept to other functions too

Anonymous

Using the level set concept

Balarka Sen

Fact: For a $C^1$ function $f : \Bbb R^2 \to \Bbb R$ such that $0$ is a regular value of $f$ (that is to say, there is no $p$ for which $f(p) = 0$ and $\nabla f(p) = 0$), then $\nabla f(a, b)$ is the normal to the curve $f(x, y) = 0$ at $(a, b)$.

Once you have that, "normal of one is multiple of normal of the other" translates as

$\nabla f(a) = \lambda \nabla g(a)$

$\lambda$ being the multiplier

@Blue Absolutely correct.

Anonymous

I couldn't understand why the $\nabla$ operator gives the normal though

Anonymous

22:02

$\nabla f(x,y)$

Balarka Sen

That's a thing, yeah

It's not super easy to see.

Basically, $\nabla f(x, y)$ points towards the direction $v$ along which the directional derivative $D_v f(x, y)$ is maximum.

Anonymous

Dot product maximization or smh...I forgot

Balarka Sen

Yeah, this is a story for another day :) It's used in algorithms like gradient descent and so on and so forth

Quickest way to find critical points etc of a function

Anonymous

Why the gradient is the direction of steepest ascent

►

Anonymous

I think this explains it

Anonymous

22:06

I should watch it once

Balarka Sen

Ah yeah probably it does

Anonymous

But okay

Balarka Sen

You should

Uh so lol

Anonymous

I'll watch it tomorrow

Balarka Sen

we boosted through lots of multicalc

in like an hour

Anonymous

22:07

I got the gist of the concept though :)

Anonymous

Alright....so we are probably done with analysis for today :P

Anonymous

LA? lol

Balarka Sen

Hehe, yup

loool

Anonymous

3:37

Anonymous

I can stay up whole night...too excited :P

Anonymous

22:08

But please let me know if you are sleepy

Anonymous

Or have some work

Balarka Sen

Actually there's a neat linear algebra conclusion of what we just discussed; it says every symmetric nxn matrix has n distinct eigenvalues

Anonymous

Or if you wish we can continue tomorrow

Balarka Sen

This can be proved using Lagrange maximization (!!!!)

Anonymous

wow

Anonymous

22:09

Sounds cool

Balarka Sen

@Blue yeah I'm probably going to sleep in a couple minutes or so and before that jam up to a song or two. It's not the best idea to teach mathematics at this hour of the night lol

Anonymous

@BalarkaSen Sure...:) We can get back to this tomorrow :D

Anonymous

Thank you sooooooooo much :)

Balarka Sen

Sure thing

No problem heh

Anonymous

I learnt a lot in 2-3 weeks

Anonymous

22:11

Okies...so goodnight! :D

Balarka Sen

Night

Anonymous

Cya!

Transcript for

♦ $Mathematics$ the no-normie zone

Transcript for

♦ the no-normie zone

♦ $Mathematics$ the no-normie zone