The h Bar

enumaris

12:03 AM

lol

@danielunderwood can I test a interview-y question on you?

danielunderwood

uh oh

go for it

enumaris

So say you have a weighted six sided die such that 1,2,3 have double the probability of showing up as 4,5,6. You roll this die many times and observe the average. The average, then, is itself a random variable. What distribution describes this random variable?

jeopardy music plays

jeopardy music plays longer

danielunderwood

well

there exists some transformation that maps it to a normal distribution

But really I'm not sure since I'm not really familiar with skewed distributions

enumaris

hmmm

The answer is that the average is normally distributed

It's just the central limit theorem dressed up to look a bit confusing...

danielunderwood

oh the mean is just shifted, the distribution itself isn't skewed...that should have been obvious -_-

enumaris

12:14 AM

Yes, the mean is < 3.5

but the distribution itself is just a normal

it's not a skewed normal

hmmm

maybe not a great question lol

danielunderwood

well my other statement that there was a map between the two was certainly correct!

enumaris

the identity map yes

I could just ask for a description/definition of the central limit theorem

but I was trying to think of a more clever way to ask it

danielunderwood

It may also be worth noting that I'm not all that familiar with the central limit theorem

Shame, I know

enumaris

:O

but this question doesn't seem to have many ways for me to give any sort of "hints" if the person doesn't just know it off the bat

like the hint would have to be "think about the central limit theorem"

since there's no way I'd ask someone to derive the CLT during an interview

hmmm

maybe I'll just ask for a description of the central limit theorem lol

@danielunderwood can I try another one? :D

last one I got

if it ain't good maybe I'll just stick to linear alg questions

danielunderwood

12:31 AM

sure!

Props to you testing your questions. I've tested mine on whatever candidates I'm interviewing

enumaris

lol

Given x and y are unit Gaussian distributions centered on 0. Given y>x, what is the probability that x>0?

b degnan

@enumaris The proof for the dice is the mean value therom.

one die has a perfectly random distribution

enumaris

I'm not familiar with a proof of the CLT using the mean value theorem

b degnan

As you add dice you, you push the average toward the middle. This is one of those proofs we do in cryptography classes

enumaris

but I definitely would not ask anyone to prove it during an interview lol

b degnan

12:36 AM

That's how I got my cryptography job. :P I'm rather good at statistics.

enumaris

@bdegnan what you just described sounds more like the law of large numbers though?

b degnan

anyone who does quantum mechanics is good at statistics, which makes you good at cryptography, etc. Math is math.

enumaris

the mean value theorem just says if you have a continuous function over some interval you have to pass through the mean at some point right

b degnan

Yes, however, if you get into the actual function definitions, you can start doing probabilities for behaviors of randomness.

one die will have a flat distribution and be perfectly random.

As you increase the number, the probability of the "mean" repeating increases.

(on a throw)

enumaris

the way it's described on wikipedia actually talks about the slope...

lol it's been too long since I've dealt with this stuff

b degnan

12:40 AM

if you tried to use wikipedia to model a transistor, life would... not go well. It's a good primer.

enumaris

oh I was thinking of the intermediate value theorem loool

danielunderwood

@enumaris 0.5? I feel like I could be missing something stupid though

enumaris

So x>0 is probability 0.5. Do you think that given the extra information that y>x, that the probability should not change?

So your contention is that P(x>0)=P(x>0|y>x)=0.5?

@bdegnan my statistics and probability theory is mediocre at best lol

@danielunderwood hint: think of the joint probability P(x,y)

What does p(x,y) look like?

danielunderwood

That was a bit of a gut feeling answer...but they should be independent, correct?

enumaris

x and y are independent yes

danielunderwood

12:46 AM

Shouldn't p(x, y) be flat?

er, uniform

enumaris

no, p(x,y) is not the uniform distribution

it can't be right, because x and y are both Gaussians

so if p(x,y) is uniform you're saying you're just as likely to get (10,000,10,000) as you are to get (0,0)

when clearly it's much more likely for x and y to be around 0 right?

danielunderwood

Yep, there's dumb mistake #1

enumaris

But actually if you assumed a uniform distribution, you'd get the same answer...due to the nature of this question...

but a uniform distribution is not well defined on non-compact support...

other than that...

Anyways, hint #2, the probability distribution p(x,y) is simply a (outer) product of gaussians

so along any line through the origin in the (x,y) plane, the probability distribution is a Gaussian...in other words p(x,y) exhibits axial-symmetry

danielunderwood

Yeah...it's just a 2d gaussian, right?

enumaris

yes

So how would you get p(x>0|y>x) given that information?

danielunderwood

12:53 AM

Then you cut the distribution along y=x and you get 0.25? (1/4 in first quadrant, 2/4 in second, 1/4 in third, but first is only place where x>0)

enumaris

Correct :)

Maybe this question is ok...

danielunderwood

Nice!

enumaris

I think the first question should be "what's the probability that y>2x?"

in which case the answer is 0.5 and maybe a bit easier to get "intuitively" without having to resort to this picture of a 2d gaussian

danielunderwood

I like thinking of the picture of a 2d gaussian...no idea of that's what a statistician would do or not though

enumaris

hmmm

you can get a what's the probability y>2x by noting that y and x are normally distributed and therefore is is y-x and y-2x

and so y>2x -> y-2x>0 and y-2x is still normally distributed around 0

so the probability is still 0.5

not sure if that's any easier...

For a physicist that comes up when doing error propagation...if you don't do error propagation...I'm not sure you see that result commonly

in fact one of the proofs of that is...using the 2d gaussian...lol

skullpetrol

1:05 AM

stats is the youngest of the maths

danielunderwood

From a physics perspective, it makes me like the power of visualization and symmetry

and the scenario y>2x makes me want to say 0.125

or are you just saying y>2x and not x>0|y>2x?

enumaris

if y~N(0,1) and x~N(0,1) then y-2x~N(0,sqrt(5))...but you only need to know that the mean adds and so stays at 0

y>2x gives probability 0.5

y>nx is all probability 0.5 you just tilt that line

danielunderwood

Ah I see what you're saying if you're leaving out the x>0 part

enumaris

the n doesn't affect the probability. Intuitively you can think of it like x can be negative half the time. When x is negative, then 2x is more negative so it's "easier" for y>x to hold. When x is positive, then it's "harder" for y>x to hold...the two effects cancel out.

danielunderwood

next ask them y>|x|

enumaris

1:07 AM

lol

should I ask this one or just ask what the CLT is? lol

alternatively I could ask neither

and just stick to more "data science" rather than stats/probability

danielunderwood

I mean I like this one now that I have the picture in my head. May also be one that you could see what someone's thought process is if they have to work through it. I imagine asking what the CLT is would be a relatively straight answer?

enumaris

yes...relatively straight or more tricky and have to think through things...

skullpetrol

isn't stats a data science?

enumaris

stats is one part of data science

it's not all of it though

skullpetrol

right

enumaris

1:10 AM

a lot of data science is also linear algebra and differential calc

integral calc not so much...unless you head on over to reinforcement learning

but I mean...to be a good data scientist, it's not really like "hardcore stats" is what you need to focus on. The more important pieces, imo, are more like you know how to set up a fair experiment and execute on it.

skullpetrol

hmmm

enumaris

a lot more "applied stats" than foundational stats

skullpetrol

right

enumaris

The person I'm gonna interview is a math minor in undergrad...this is the only reason I feel like I could get away with asking these types of questions.

If all the DS experience was gained through work experience rather than any formal education in stats/probability theory, I probably wouldn't bother with these kinds of questions

danielunderwood

I was a math minor in undergrad, but that didn't really include any probability/stats

enumaris

1:15 AM

my main concern is not really that they know these math principles, more like they know how to think through the problem

cry

I think your thought process is good over the second question though so I would be OK if the person went through the question in that way.

The first question was way too "you know it or you don't" kind of gotcha...

danielunderwood

well I think you answered your own question of which to ask between those two if you want to know more of how they think through the problem

enumaris

XD

I'll keep this in reserve

danielunderwood

One that I've liked is what happens to a neural net with identity activations since then you get $y = A_1 \cdots A_Nx = Ax$

Though I guess answering that correctly doesn't really tell you much besides they have a small familiarity with neural nets and linear algebra

or "at what point do you throw out your model and stay away from neural networks for a month?"

:D

enumaris

oh like a neural net of multiple layers without a non-linear activation function is equivalent to a single linear transformation?

danielunderwood

Yep, exactly!

enumaris

1:26 AM

on those kinds of fronts I could also ask things like explain gradient descent and like if you have a linear model why you may use gradient descent rather than the normal equation to solve for the weights.

so there's a question for you, given a model y=Wx+b, what are some benefits to using gradient descent over the normal equation?

danielunderwood

BIG DATA ™

enumaris

That's one reason

I also look for 1 more

danielunderwood

Well kind of in the same sense, a streaming implementation is fairly easy when you're training it in batches

And you gotta get on that streaming train

enumaris

There's a math reason though

hint: under what conditions are the normal equations solvable?

at least strictly speaking

danielunderwood

I'm not sure, but I imagine it may be something to do with underdetermined systems and something something degenerate eigenvalues

enumaris

1:34 AM

you got a word right

but

under what conditions is a matrix invertible?

danielunderwood

Non-zero determinant

Which would be no good if you had an eigenvalue that was 0 I'd think

Man my linear algebra is rusty

enumaris

so you have the normal equation $W=(X^T X)^{-1}X^T y$

This means $X^{-1}$ must exist right

danielunderwood

yes and $({X^T})^{-1}$

enumaris

So your answer is right that $X$ needs to have non-zero determinant. This means (in terms of eigenvalues) that there can't be eigenvalues which are 0 (this is just another way of saying non-degenerate). The Eigenvalue part isn't really all that important.

What's a really easy way to make $X$ degenerate?

danielunderwood

It means the columns and rows have to be independent (since we have $X$ and $X^T$)?

So if you had duplicate training examples, it'd be bad

enumaris

1:41 AM

hooray :D

danielunderwood

wooo I've pulled something out of the depths of my memory!

enumaris

you get a marginal pass. I can hire you, but I can only pay $3.50

danielunderwood

$3.50 a minute? Sure!

enumaris

total

forever

danielunderwood

:(

when you have a job and grad students look rich

enumaris

1:45 AM

my power has gone off...

annoying

danielunderwood

My power went out a couple weeks ago right as I was stepping in after work...I was not pleased

enumaris

It's gonna get hot here soon probably...

no AC

danielunderwood

2:12 AM

that's when you just run away to somewhere that does have AC

bolbteppa

3:47 AM

@RyanUnger you're joking about $\nabla^2$ right

Ryan Unger

4:16 AM

@bolbteppa $\nabla^2$ is the Hessian for me, so no

kyle campbell

...isn't that usually reserved for the Laplacian?

Ryan Unger

I've never seen a mathematician use that for the Laplacian

you'd get laughed out of town

skullpetrol

mathematicians-ville?

or would that be a "villa" :-)

skullpetrol

4:40 AM

Top of the mornin' @JohnRennie

John Rennie

@skullpetrol morning :-)

skullpetrol

:-)

kyle campbell

wot m8 @RyanUnger what would a mathemagician use? physicists use it (including ol' Feynman)

(honestly asking out of curiousity)

Ryan Unger

@kylecampbell $\Delta$, universally

kyle campbell

ah okay yeah I've seen that in differential equations

fair enough you can laugh at me

enumaris

4:44 AM

wikipedia also lists both $\Delta$ and $\nabla^2$

Ryan Unger

I'm not laughing

Like I said above Wiki can't decide which one to use

enumaris

also $\nabla \cdot \nabla$

Ryan Unger

they change from line to line

enumaris

Wiki is the arbiter of all truth

if it's on wiki it's true

and if it's not on wiki it's false

skullpetrol

simple fact^

kyle campbell

4:45 AM

$\nabla \cdot \nabla$ would be incredibly inefficient wow

enumaris

When noone is looking I use $\mathfrak{Laplacian}()$

@danielunderwood I did indeed run away

Ryan Unger

@enumaris federer writes $\mathrm{Lap}\,f$ (5.2.10)

Roger Federer?

Herbert

lol

4:48 AM

he uses $\Delta$ for the modular function of a Haar measure

I don't think that explains it

he's just weird

enumaris

I use $\Delta$ for differentials to tick people off e.g. $v(t)=\Delta x/\Delta t$

skullpetrol

hmmm

kyle campbell

lol

Ryan Unger

@enumaris we mostly just hit the "ignore" button and move on

skullpetrol

it's great having 0celo back :D

enumaris

4:53 AM

he got unbanned?

Ryan Unger

uhhhh

did I?

enumaris

o

celo

Ryan Unger

did you really not know it was me

enumaris

did not lol

skullpetrol

did to

enumaris

5:01 AM

o.o

skullpetrol

:-)

May 16 at 19:01, by Loong

@RyanUnger welcome back

enumaris

ain't seen it

skullpetrol

ya, you were gone for awhile also

note: his ban ended 19:00

bolbteppa

6:05 AM

@RyanUnger haha it's not some iamverysmart mathematician notation, $\nabla^2$ is literally the definition of the Laplacian even on the wiki, $\Delta$ is an abuse of notation

Laplace operator

In mathematics, the Laplace operator or Laplacian is a differential operator given by the divergence of the gradient of a function on Euclidean space. It is usually denoted by the symbols ∇·∇, ∇2. The Laplacian ∇·∇f(p) of a function f at a point p, is (up to a factor) the rate at which the average value of f over spheres centered at p deviates from f(p) as the radius of the sphere grows. In a Cartesian coordinate system, the Laplacian is given by the sum of second partial derivatives of the function with respect to each independent variable. In other coordinate systems such as cylindrical a...

Ryan Unger

you don't know anything man

bolbteppa

$\Delta f = f(x+h) - f(x) = \nabla^2 f$

lol

enumaris

if it's on wiki, it must be true

kyle campbell

6:36 AM

$\nabla \cdot \nabla$ = $\nabla^2$ makes more sense because if you just say $\nabla$ then it's unclear whether you're taking the gradient of $f$ or the Laplacian of $f$ when you say $\nabla f$

nawmean

oops nvm

got my shapes inverted

John Rennie

Accepted notation is what it is, and I'm not sure arguing about its logicality or otherwise is very productive. I must admit I've never seen $\Delta$ used for the Laplacian, but if it's a standard notation in (some?) areas of maths then it is.

enumaris

6:54 AM

upsidedownface

don't mess with upsidedownface

bolbteppa

7:17 AM

Also $\nabla_{\mu}$ is covariant derivative notation and $\nabla_{\mu} (\nabla^{\mu} \phi) = \nabla^2 \phi$ gives the (curved) Laplacian in 3D and beyond

enumaris

7:43 AM

abuse of notation

Loong

@enumaris ISO 80000-2 also has both

PM 2Ring

8:05 AM

Speaking of unfortunate notation, it's hard to beat n!! for the double factorial, aka the semifactorial, and !n for the subfactorial.

bolbteppa

8:33 AM

Does this really mean the Dirac equation is not the right equation in 2+1D

Sebastiano

10:38 AM

@Semiclassical and other users. Excuse me for last night. :-( because I have leaved the chat for my a little health problem due to the excessive heat. We're at 41 degrees now. I checked this morning and obviously there are some mistakes. I will edit my question.

Sebastiano

10:53 AM

Excuse me for my post. But is there a reason for a downvote here? Just a curiosity. Thanks. physics.stackexchange.com/questions/494698/…

0

Q: A best definition of proper acceleration

In this link of Wikipedia https://it.wikipedia.org/wiki/Moto_iperbolico there is a definition of proper acceleration: The proper acceleration to a particle is defined as the acceleration that a particle "feels" while accelerating from one inertial reference system to another. $$\alpha=\frac...

Avnish Kabaj

11:15 AM

Y'all wait I'll be learning that fancy Hamilton operator and stuff soon enough

Lemma

Tuple

Echelon

Fancy stuff^

@blue isn't pingable

Sad times

skillpatrol

ya, he changed his username and hasn't been around for about two months :(

skillpatrol

11:29 AM

from the classical channel

Ryan Unger

1:44 PM

@JohnRennie do you just not have a notation for $\nabla_\mu\nabla_\nu f$?

danielunderwood

2:31 PM

@Loong of course there's a standard for mathematical symbols...though I assume mathematicians and physicists happily ignore anything ISO/NIST

@enumaris I thought of something stupid regarding that question of p(x>0|y>x)...p(x>0) = p(y>x) = 0.5 and they're independent conditions, so you trivially get 0.25 from there, right?

Ryan Unger

@danielunderwood I've never seen a mathematician talk about it

danielunderwood

I never thought about standards until I had "engineer" in my job title...I'll just assume they're an engineer thing

skillpatrol

2:49 PM

@RyanUnger they're always talking about some "standard form" in school?

ax + by = c

danielunderwood

I pesonally like to use $\Delta$ as a function and $\nabla$ as a variable, so we have $\frac{d \Delta}{d \nabla}$

or better yet, $\nabla_\nabla^2 \Delta$

skillpatrol

:D

danielunderwood

Apparently I don't adhere to spelling conventions either

vzn

@RyanUnger very interesting! btw speaking of fluids + cutting edge research into black holes + unresolved mysteries... 44 refs to "wave" in that paper, with special focus on nonlinearities. zero refs to fluids. doncha just luv physics compartmentalization. and experts supposedly hardcore into physics who dont read/ cite any papers :| :/ :( (zen question, is toothpaste a fluid? can one do real research on it?) :P

Ryan Unger

@vzn it's not a physics paper.

vzn

3:03 PM

@RyanUnger lol!

skillpatrol

I searched my .bash_history for the line with the highest ratio of special characters to regular alphanumeric characters, and the winner was: cat out.txt | grep -o "[[(].*[])][^)]]*$" ... I have no memory of this and no idea what I was trying to do, but I sure hope it worked.

danielunderwood

3:23 PM

The real gem is when you're escaping backslashes that get put into somewhere else, so you have `\\\\` for a single backslash

turns out putting 4 backslashes into markdown is...difficult

skillpatrol

~~\o\~~

looks like a star wars fighter

~~|o|~~

~~|o|~~ ~~|o|~~

PM 2Ring

@danielunderwood Yep. Here's a Python example I wrote a while ago:

in Python on Stack Overflow Chat, Jun 9 '16 at 11:39, by PM 2Ring

Note that if we don't use a raw string for the regex we need 8 backslashes in a row: '\\\\\\\\(\d+),(\d+)' That's hard to read and too easy to mess up.

in Python on Stack Overflow Chat, Jun 9 '16 at 11:35, by PM 2Ring

import re
pat = re.compile(r'\\\\(\d+),(\d+)')
data = r"bla bla bla\\123,456hey hey hey\\789,987bye bye"
a = pat.findall(data)
print(a)
#output
[('123', '456'), ('789', '987')]

danielunderwood

3:56 PM

Yeah always gotta use raw strings for the regex. I actually thought for a while that the r-strings stood for regex strings since pycharm automatically highlighted the regex

And if you don't use raw strings, you can't just copy/paste from regex101, which breaks my whole regex flow

PM 2Ring

Definitely! Doing regex in Python without r-strings is rarely a good idea.

Raw strings are one of the language elements that made me come to love Python. They aren't deep or fancy, but they are so useful when you need them. Simple, elegant, and effective.

danielunderwood

4:12 PM

Ruby actually has regex literals that are pretty neat

PM 2Ring

4:38 PM

I feel a bit sorry for Ruby. It was chugging along nicely, and then Python blew it out of the water.

PM 2Ring

4:49 PM

awk also has regex literals, delimited by forward slashes.

Ryan Unger

@EmilioPisanty ok so I want to do something perhaps stupid in TeX

I know that I can hyperlink equations using \eqref as long as I have \begin{equation}\label{foo}\end{equation}

but suppose I don't want the equation to be numbered, but instead by labeled by something else

oops nvm I was getting an error for unrelated reasons

TeX is amazing wow

so you can put \tag on an equation and \eqref is smart enough to grab that instead of the equation number

vzn

6:06 PM

← ruby aficionado since early 2000s, slowly dying breed o_O

Physics Meta

7:10 PM

0

Q: Dimension of the candela unit: What does J stand for?

The J symbol can represent the unit of energy but it's also the symbol for the dimension of the candela (or luminous intensity). For the energy unit, it clearly comes from the family name of the English physicist James Prescott Joule (1818–1889), but for the luminous intensity dimension, what's ...

discussion history

user76284

When I try to match the Weyl metric with a diagonal stress-energy tensor, I always get that the stress-energy tensor must be zero. Is this supposed to happen?

Emilio Pisanty

8:48 PM

@RyanUnger yes ;-)

user76284

Does this look like the expected stress-energy tensor $T_{\mu \nu}$ for the Weyl metric?

Note that if I let $T_{rr} = T_{zz} = p$ to get a perfect fluid in its rest frame, they must both be zero (since they're negatives of each other).

For the contravariant tensor I get the above. Same problem.

Transcript for