The h Bar

DanielSank

6:00 AM

Drag the "start chatjax" link into the tab for this page.

John Rennie

@DanielSank that's what I use. Enabling MathJax makes formulae easier to read.

user228700

Crap, I'm having some trouble.

DanielSank

Ok no big deal.

Yes, 2^N is correct.

Let's establish some terms:

user228700

Yeah, okay...

DanielSank

Suppose our set of 0's and 1's represents a physical system.

This could be a row of lightbulbs where 0 means off or 1 means on.

It could be a row of electrons and 1 could mean "spin up" while 0 means "spin down". If you don't know about spin that's ok, just forget it.

user228700

6:04 AM

Okay, so, 0 and 1 represent the states of the system..?

DanielSank

Yes exactly.

More precisely, 0 and 1 represent the states of each sub-part of the system.

The "system" usually means the whole row of elements.

user228700

@DanielSank I graduated with 94% in the Indian CBSE exams. I know some stuff :P

DanielSank

So yes, we have a system with N elements, and each one has two possible states.

A common symbol for the number of possible states is $\Omega$.

user228700

Okay...

DanielSank

Oh crap, you can't see that symbol can you?

user228700

6:05 AM

No, that's okay.

user228700

I'll manage :) Go on...

DanielSank

You can read TeX notation?

By the way, I don't think you're dumb. When I want to explain something to a physicist I obey a rule given to me years ago:

"With physicists, assume zero knowledge but infinite intelligence."

Shing

um... I got it now. I miss the point that x,y,z completely describe position, so I don't need to worry about it.

user228700

I can read the notation...but to a very small extent :(

user228700

@DanielSank That's some great advice :D Go on...

DanielSank

6:07 AM

Ok anyway...

Shing

well....... zero knowledge but infinite intelligence

sounds to me how to deal with extreme arrogant ppl :P

DanielSank

Entropy is defined as the logarithm of Omega.

S = log Omega

user228700

Right. Okay...

DanielSank

For now let's use base 2 logarithm (it doesn't actually matter but base 2 is convenient for us right now).

So the entropy of our system is N.

Now suppose we add one element to the system. Then the number of states is 2^(N+1) and the entropy is N+1.

user228700

Okay...

user228700

6:09 AM

And wait, what do you mean it doesn't actually matter? The base, I mean.

DanielSank

@KaumudiHarikumar I mean exactly what I said... in any problem you do the base you use in the logarithm won't matter in the end.

Let's come to that later.

user228700

Yeah, I was wondering why it doesn't matter. But okay...

DanielSank

@KaumudiHarikumar Ok it doesn't matter because the only thing you wind up caring about is differences in entropy.

user228700

Okay...

user228700

Please go on...

DanielSank

6:13 AM

About the bases or the original line of discussion?

user228700

The original line :)

DanielSank

Ok

So we see that the entropy just tells us the number of elements in the system.

user228700

Oh, yes...

user228700

And this works out exactly like this for every case?

DanielSank

Ok so from the absolute basic point of view, that's what entropy means physically. Now, we can go on to discuss some more interesting aspects.

@KaumudiHarikumar Yes, more or less.

user228700

6:15 AM

Alright!

DanielSank

Let's make contact with something @JohnRennie said a few minutes ago.

Consider a single molecule in an empty box.

user228700

Okay...

DanielSank

This molecule can float around. It can occupy any possible position in this box.

Think of each point in the box as a little pixel. Each pixel can be either on or off, where it's on if the molecule is in that pixel and otherwise it's off.

Since we only have one molecule, the only allowed states of our system are those where exactly one pixel is on.

user228700

Right, yes, okay...

DanielSank

So let's say we've divided the box into V pixels. Then there are V possible states in the system. I'm using V because it reminds us that we're really talking about the volume of the box.

Now suppose we have two molecules.

Now we can have any state in which exactly two pixels are on.

How many states are there?

user228700

6:19 AM

VC2?

DanielSank

What is the C?

user228700

The binomial coefficient I mean. Sorry I dunno how to type all this in MathJax yet :(

DanielSank

Heheh, you're too clever for your own good here. I was afraid of this.

user228700

Huh? What do you mean? :P

DanielSank

Well, can you explain why there's a binomial coefficient, and exactly what you mean?

user228700

6:23 AM

Okay, if there are V pixels for the two molecules to occupy, then the number of ways for them to occupy those V pixels would be the number of ways of selecting 2 out of V pixels...no?

user228700

Assuming the molecules are identical ie.

DanielSank

Yes, but by that reasoning shouldn't it be V^2?

user116211

@yuggib: Did you check the criteria of substitutions in Bourbaki: Set Theory?

user116211

I'm having a hard time with those.

user116211

Bourbaki didn't tell why they are meant for.

user228700

6:26 AM

@DanielSank Um, the number of ways of selecting two out of V possible states is V^2?

user116211

I have not found them in any other literatures :(

DanielSank

@KaumudiHarikumar When we select the position of the first molecule, we have V choices.

John Rennie

@KaumudiHarikumar can both molecules be in the same box/pixel?

DanielSank

Then we have again V choices for the second.

user116211

I asked a question on this at MSE.... and got a paper of Mathias.

user228700

6:27 AM

But we only have V-1 choices for the second, since one has already occupied one of the pixels no?

DanielSank

@KaumudiHarikumar Ah ok. Yes you're right, but assuming V is really large that -1 just doesn't matter at all.

user116211

He always seems to be ranting on Bourbaki for their treatment of logic and neglecting Gödel's works.

user228700

So VC2 would give V(V-1)/2 since both molecules are identical too...

user228700

OH. Alright :) Go on...

DanielSank

You're right about the /2 part though!

So wait a minute, you really do know a lot of quantum mechanics then?

user116211

6:29 AM

I have postponed my plan to further study Bourbaki after I complete my conventional semester works.

user228700

@DanielSank :P What makes you think so?

user228700

I really don't. I just graduated high school!

user228700

The syllabus is different here in India...

user228700

I've to know all this stuff to get into college, for eg.

DanielSank

@KaumudiHarikumar Well, the fact that you're talking about identical particles. This is a rather... advanced topic, in a sense.

John Rennie

6:30 AM

@KaumudiHarikumar Daniel's point is ging to be that to get the enropy we take logs, and if V is large the difference between log(V^2) and log(V(V-1)/2) is negligible.

user228700

But no, honestly, I don't.

DanielSank

Ok, whatever you say. By the way, I've written a well-received post about why particles are identical.

Anyway, so yes, for the two particles we get V^2/2 possible states.

user228700

@JohnRennie Okay...

user228700

@DanielSank Yes, okay...

DanielSank

The number of states for N particles?

user228700

6:33 AM

N is small..?

user228700

Compared to V, I mean.

DanielSank

yeah.

user228700

Okay, then, by the same logic, it would be (V^N)/N! no..?

DanielSank

Yes.

Exactly.

Ok now that we've done some easy stuff let's get going for real!

I mentioned all this stuff about particles in a box because it connects to what @JohnRennie called "phase space".

A given particle has two degrees of freedom, its position and its momentum.

If we had only position, then we could characterize the states of a system of many particles as we did just now.

user228700

Okay...

user228700

6:35 AM

Yeah, ok...

DanielSank

Each particle has a set of possible positions which in our case is the box and we can describe the state of the system by stating where the particle is. In other words, the state of the system is given by a single position vector r.

If you have two particles, then you need two position vectors r.

Each one can range over the volume V.

user228700

Okay...

DanielSank

This space, this set of multiple position vectors, one for each molecule, plus the set of the momenta of each particle, is called the system's "phase space"

Note something interesting:

If we just look at the size of the phase space (just taking position for now), it should be V^N.

However, you correctly noted that particles can be identical and so the number of possible states is V^N/N!.

user228700

Okay...

DanielSank

That /N! is very important and it took physicists a long time to figure out that we need it.

The size of phase space that you compute naively is not the correct set of states for the system.

Also, note that if you take the log of the size of phase space, i.e. compute the entropy, you get S = N log(V) - log(N!). Of course, the second term is only there if you include the /N! for identical particles.

user228700

6:40 AM

Oh, wow. I should tell you; I only assumed that N! because I didn't even think about it too much! I just assumed that the particles are bound to be identical

DanielSank

Hahaha! Well it turns out to be a rather subtle and in a sense "game changing" factor!

You will see why later in your studies. If you ignore the /N! then statistical mechanics makes impossible predictions. The /N! is essential.

user228700

Riight. OK...

DanielSank

Anyway, as I was saying, the entropy is S = N log(V). This is another example where the entropy scales linearly with the number of elements in the system.

user228700

Another example?

user228700

Where did we see this before?

DanielSank

6:42 AM

You had asked a few minutes ago whether it's always true that S is proportional to the number of elements. We did this example to show you that the answer is essentially always "yes".

We saw it before with the string of 0's and 1's.

We had S = N log(2).

user228700

Yeah, the string. OK :) Go on...

DanielSank

Ok so we've learned that entropy is the number of elements in the system multiplied by the log of the number of states of each element (V is the number of positions allowed for each molecule).

Ok now let's do something really interesting.

user228700

Wait just one second please.

DanielSank

Ok.

user228700

Let me digest all this (:P)

DanielSank

6:44 AM

ok

I will continue my Russian lesson meanwhile.

user228700

I'm done :P You can go on...

DanielSank

Do you know about Lagrange multipliers?

user228700

Oh God, no :P

DanielSank

that's ok.

user228700

There was something about that in that lecture I watched.

DanielSank

6:47 AM

Let me think for a few minutes.

user228700

Sure :)

DanielSank

Alright let's go back to the string of 0's and 1's.

Suppose you can't see the state of every single element.

Suppose all you can see is the total number of 1's.

user228700

Why can I only see the total number of 1's?

DanielSank

For example, if each element is a microscopic bar magnet, pointing either up or down, perhaps you cannot measure the individual microscopic magnetic field of each magnet. However, you can measure the total magnetization because there are 10^23 of them.

user228700

Okay..?

DanielSank

6:50 AM

Why the question mark? Is something unclear?

user228700

No, the question mark was my way of asking you to go on. Sorry :P

DanielSank

oh ok

Ok suppose each magnet is free to point up or down randomly.

user228700

So we're talking about a bunch of bar magnets free to point up/down...Right, okay..

DanielSank

Yes, and we can represent that as 00011101001010111011101010

user228700

0 being up and 1 being or possibly the other way around?

DanielSank

6:53 AM

Yes.

Can you compute the probability distribution for the total magnetic field? In other words, can you compute the probability distribution of the number of 1's?

Or perhaps you just know the answer.

user228700

No, I don't, sorry.

DanielSank

Think about binomials!

user116211

@yuggib: And then, there is formative criteria.

DanielSank

@KaumudiHarikumar I have faith that you actually can compute this!

(which is awesome for a high school student)

user228700

Wait, I dunno what you mean.

DanielSank

6:57 AM

How many ways can we get a total of zero 1's?

user228700

Oh, THAT.

user228700

Yup, can do :P

user228700

But how many do we have?

user228700

I'm a little confused.

user228700

Clearly :P

DanielSank

6:59 AM

Well let's say we have N elements.

user228700

N magnets?

DanielSank

yep

user228700

And each can have 0/1. Sure, yeah, I can calculate the probability distribution.

DanielSank

If it's easier, think of a magnet being either -1 or +1 instead of 0 or 1.

user228700

But, I mean, it's not like I'll get values or anything right? I'll get expressions involving N...

DanielSank

7:01 AM

Using -1 and +1 is a bit simpler because the mean is zero.

@KaumudiHarikumar Yes, expressions are good.

user228700

Okay...

user228700

Wait, you want me to type the expressions?

DanielSank

Describe it if you prefer.

Or I'm happy to trust you and we can just write it down.

I don't mean to put you through tough exercises. It's just that, by thinking through this in a few minutes we'll have a really good understanding of what entropy means.

user228700

Trust me, please :P I don't know how to type it out.

user228700

Is it necessary for us to talk about the expressions ?

DanielSank

7:04 AM

well, yeah!

You already gave some before, remember?

Number of states of N particles in a box is V^N.

That's an expression.

user228700

I'm not sure if I'm on the same page as you, but wouldn't it take time to type out all the possible expressions? I'm thinking of the table here...

DanielSank

ohhhh

Well, in the end you don't need to type every single case!

We can ask, what is the number of ways to get m 1's in a chain of N elements?

user228700

Wait, am I not thinking of the same thing as you are?

user228700

(That's what I gather from the "ohhhh" :P)

user228700

Oh, okay, so that would be NCm(0.5)^m(0.5)^N-m?

user228700

7:09 AM

So that's just NCm(o.5)N

DanielSank

...trying to understand notation...

user228700

Oh, crap.

user228700

Okay, hold on, I'll try to use MathJax.

user116211

0

Q: Difference between wave of probability and wave function

What is exactly the difference between wave of probability and wave function?

user116211

No research effort but I think some sort of that has been asked before.

user116211

7:12 AM

Wavefunction is probability amplitude ;( This is not probability.

user228700

@DanielSank This is taking too long :(

DanielSank

@KaumudiHarikumar What browser?

user228700

Just one sec.

John Rennie

@MAFIA36790 I'm not sure I would have VTC'd as insufficient effort, because it's far from clear to me what the question is supposed to mean. Unless the OP clarifies it i will VTC as unclear.

user116211

@JohnRennie hmm.

user116211

7:18 AM

@yuggib: o/

user228700

user228700

God, the effort it took to upload that picture :P

user228700

THAT'S the expression.

user228700

I'm pretty sure I'm not correct though. @DanielSank right?

DanielSank

thinking

user228700

7:21 AM

Okay. Sorry for the delay :(

user116211

@KaumudiHarikumar You need help in MathJax?

user228700

Yup. Not right now though. First entropy! :P

DanielSank

I always forget what that C symbol means. Give me a moment. I was educated in the US and we're less handy with that stuff ;)

user228700

Oh, crap. Okay, the NCm basically denotes the number of ways to select m objects out of N.

DanielSank

I think the solution is this:

(1/2)^m (1/2)^(N-m) N! / (m! (N-m)!)

user228700

7:26 AM

That's the expansion of my expression so I'm glad we're on the same page :)

DanielSank

cool.

user228700

Phew. Okay, so now?

DanielSank

It turns out that this function you've written is actually very well approximated by the Gaussian function.

user228700

What do you mean?

DanielSank

You can approximate the expression you've written as:
(1/sqrt(2 pi sigma^2)) exp(-(m - pN)^2 / (2 pi sigma^2))

user228700

7:32 AM

Wtf? Okay, wow. Over my head.

DanielSank

where p is the probability of a 1, q is the probability of a 0 (both are 1/2 in our example) and sigma = N p q

Ok, then let me just tell you the point:

user228700

Oh, yeah, okay, thought sigma was some other complicated thing.

user228700

Sorry, go on :P

DanielSank

If you ask for the average probability that any of the magnet is pointing up, you get a function which looks like this

Gaussian function

In mathematics, a Gaussian function, often simply referred to as a Gaussian, is a function of the form: f ( x ) = a e − ( x − b ) 2 2 c ...

The width of this function is proportional to 1/sqrt(N).

user228700

Wow, okay, I see...

DanielSank

7:34 AM

So, the larger your system, the narrower is the probability distribution of (the number of magnets pointing up / N).

Consider a real life system with 10^23 particles.

John Duffield

@MAFIA36790 : I think you should answer the question. Explaining things helps you to understand things and get them clear in your head. Then you see that some things you took for granted are pseudoscience nonsense. A wave of probability, FFS.

DanielSank

In this case, the function is insanely narrow.

user228700

@DanielSank Yup, okay...

DanielSank

The probability distribution is very sharply peaked near the case of half of the magnets pointing up and half pointing down.

user228700

Okay...

DanielSank

7:36 AM

So in the real system, you can basically just say that the number of magnets pointing up is N/2.

Now

user116211

@JohnDuffield I do think QM can only be learnt reading actual literatures and not layman books like those of Mr. Greene.

DanielSank

What we basically just said is that the system has an overwealming probability to be one of the states where the number of magnets pointing up is N/2.

user228700

@DanielSank Wait, what? I don't get that last part.

DanielSank

:)

Ok, what part is unclear?

user228700

How is the number N/2?

DanielSank

7:38 AM

Oh, that function we got is peaked at N/2.

You can plot it on your computer if you want.

user228700

Oh, so that's just how it is. Okay, alright. Go on...

DanielSank

Although beware of trying to evaluate the factorial function for modestly large arguments ;)

Well it's not just how it is. Use some thinking here:

How many ways are there to get all 0's?

Only one way.

user228700

Yeah.

DanielSank

How many ways to get two 0's? Approximately N^2. How many to get N/2 0's?

A lot.

How many to get all 0's? Just one.

user228700

Yeah, okay. So it peaks at N/2 okay :)

DanielSank

7:41 AM

So you can see that the function must start at zero, go to a large number in the middle, i.e. at N/2, and then go to zero again.

Yeah you can see it peaks at N/2 by symmetry.

user228700

Yes, okay...

DanielSank

As we said, we found out that the system has an overwealming probability to be found in one of the states such that the total number of 1's is N/2.

user228700

Yes, okay...

DanielSank

This is the second law of thermodynamics.

user228700

Wow. Okay...

DanielSank

7:43 AM

The second law is often stated as "The system goes to the state of maximum entropy".

user116211

i.e., maximum multiplicity.

DanielSank

We just have to be a little careful about the wording and then we see that we actually just proved it.

user228700

Yeah, wow.

DanielSank

See, "state" in that sentence means "macrostate" or "state which I describe using a macroscopic variable, even though there are lots of microscopic variables that I'm not specifying".

Please consider what I just said there. It is the essence of understanding entropy.

In our case, the macroscopic variable is the "number of 1's" and the microscopic variables are the states of individual element.

user228700

Yes, I understand :) Like I said, I watched that lecture but I didn't really understand any of the Math involved and I was a bit confused. I only understood about this microstates and number of ways thing and figured that that should be enough.

user228700

7:46 AM

@DanielSank Wait, can you explain about the macroscopic variable being the number of 1's and all?

DanielSank

Wait, did I just waste all your time :(

You already knew all this?

user116211

Hey @ManishEarth! so, your graduation is complete?

user228700

Oh, no, no! This was sooo informative! I mean, you, sir, are a freaking genius! I would NEVER have been able to figure out how to explain all this to a person who doesn't know quantum mechanics.

user116211

It's good to see you back.

DanielSank

Actually, the only way quantum mechanics really changes anything is with that 1/N! factor from way in the beginning of our discussion ;)

@KaumudiHarikumar You are very kind to say this.

user228700

7:48 AM

Yeah, ok :) Wait, can you explain about the macroscopic variable being the number of 1's and all?

DanielSank

I value teaching very highly. It is a good day when someone says something like that to me.

user228700

@DanielSank Are you kidding me?! You're the kind one, explaining so much!

DanielSank

@KaumudiHarikumar Sure.

From a mathematical point of view, "macroscopic variable" just means "variable which is a function of all the microscopic variables, but which has the same value for many different values of the microscopic variables".

Physically, it's something "big" that you can measure without having to measure the state of every single e.g. atom in your system.

In our case, "the number of 1's" has the same value for:
0010101101
1111100000
0001111100
0101010101

In each case, the number of 1's is five.

But see, we have many microscopic arrangements of the system which yield the same macroscopic value.

user228700

Riiight. Okay.

DanielSank

In a real system, like a gas of particles, the macroscopic variables are things like pressure, volume, etc.

Great!

user228700

7:52 AM

And the microscopic variables are...(?)

DanielSank

We've now gone through one of the two major concepts of statistical mechanics! Wow... with a high school student!

@KaumudiHarikumar The microscopic variables are the states of each element, i.e. the individual 1's and 0's.

user228700

Element=Molecule?

user228700

In the real world, I mean.

user228700

Not our 1's and 0's :)

DanielSank

yes

I used the "word" element to avoid talking specifically about any particular system.

user228700

7:56 AM

Okay :)

DanielSank

The ideas of statistical mechanics apply not only to physics. In fact, the study of signal processing and encryption use these very same ideas.

user228700

Nice! :D

DanielSank

It is not an accident that you see things that look like the memory register in a computer in our discussion above.

E.g. here

user228700

I dunno how to thank you sir. Seriously, I wouldn't have had ANY clue how to explain all this to a high school student like me. (Okay, ex-high school student not yet in college :P)

DanielSank

I should write this up as a blog post...

@KaumudiHarikumar Where will you attend college?

user228700

7:57 AM

But you did and it's been amazing.

user116211

@DanielSank You have a blog? I thought only 0celot and Slereah have blog.

user228700

@DanielSank Yeah, well, have you heard of the Indian Institute of Technology a.k.a IIT?

DanielSank

@KaumudiHarikumar yes

user116211

@KaumudiHarikumar Good!!

user228700

Oh no! There's no certainty that I'll be going there yet!

user116211

7:59 AM

@KaumudiHarikumar Which one?

user116211

@KaumudiHarikumar ohh ;P

user228700

@MAFIA36790 Yeah :P I've taken the year off to try and get into the college.

DanielSank

I see.

Transcript for