@Phonon OMG OMG... I figured it out... the Covariance matrix of our data matrix X in ML-class's regression IS the Hessian!! (aka the second derivative of the cost function) ...mind...blown...

@Mohammad lol. I'm not that far ahead yet, but I have a very good idea of what you're talking about. Just didn't know that they called it a covariance matrix.

@Phonon Mornin'!! :-)

@IvoFlipse Hello Mr Ivo

Hello all :)

@Phonon @IvoFlipse Weather is getting much nicer so I have to start going out to run again!

Peter Norvig gave me quite the workout this morning, trying to do CS212's homework

@Phonon BTW I meant to ask you on here not SigProc channel about TDOA hehe.

@Mohammad If you have any questions, be sure to drop by Fitness

I haven't run for almost a month, too busy :P

@IvoFlipse Haha, well, at least he isnt bad for your knees allegedly as my friends say running is!

@IvoFlipse Ah, Ill have to check it out! :-)

@Mohammad Simply run slower (which I hate doing), you should be fine

@IvoFlipse That might be my first question! :-)

fitness.stackexchange.com/users/8/ivo-flipse?tab=answers

Go ahead and pick

@IvoFlipse I see.

@Mohammad We already had some about those I think

fitness.stackexchange.com/search?q=running+knee

@IvoFlipse Hehe true

@Phonon Yeah, I think it is called the covariance matrix, because we would essentially be computing the variance across all the features of the data matrix

@Phonon And only way to do that is via X^T X

darn those CS101 exercises are a real black box

@IvoFlipse nice

can't get the last one right :(

@IvoFlipse lol

well the exercises are kind of funny :P

but its really hard to figure out what the right answer is

its just twiddling with the modifiers

@IvoFlipse LOL is this the udacity class?

cs-101 from Coursera

@Phonon I asked about the upsample TDOA, should be interesting

@IvoFlipse Ah! Didnt know they had one lol - seems image processing centric?

naah, its just that the code samples are really visual

its just to show how a for loop works

and what if statements can do

we basically loop through all pixels and modify them based on their rgb value

@IvoFlipse Ah nice. Well at least you are learning Python through it though ya?

That's Javascript

@IvoFlipse Oh doh!

I learn Python on Udacity, man that Peter Norvig really puts me to work

really interesting exercises

@IvoFlipse CS101 thru udacity with norvig?

today we covered depth first search applied to several riddles

CS101 was given by Dave ... (can't remember) who was really good too

CS212 = Design of Computer Programs by Norvig

oh and I got my certificate from Algorithms

@IvoFlipse Nice!

got 7 certificates now

I bet I'll get more certificates in one year than regular students get in a complete master program :P

@IvoFlipse LoL I wonder about all of those - are we going to 'get' something out of all of them vis-a-vis a degree or something. I know they are toying with the idea.

@Mohammad Are you looking at Ph.D programs at all?

@IvoFlipse Haha yeah. Really you need two things. 1) Learn something. 2) Proof you learned it.

@Phonon Umm... open to the possibility in some sense...why?

@Mohammad well I don't really care about a degree or anything, but just goes to show that students in university are a bunch of slackers :P

@Mohammad No, just curious. I started inclining towards looking at that lately.

@IvoFlipse Indeed

@Phonon have you considered doing it yourself? I mean, what benefits would a phd program offer you?

@IvoFlipse Yeah all I care about is learning frankly. However 'showing' you learned to employer can be important too. However I think that might not be so important the more experience you have, ...and if you have code to back it up on github...def a nonlinear function.

@Mohammad I'm self employed, though obviously I need to prove to clients that I'm worth their money

@Phonon Heres why I hesistate about phds. First of all if you asked me 5 years ago I would have said yes, but at that time I was fresh out of BSEE. The thing is its SO costly in terms of money spent, AND opportunity cost of getting Phd. If that by itself wasnt a barrier, the education system is (as we can see!) going through an earthquake right now. I think when the dust settles we are going to be in much better shape to learn at the phd level and even more.

@Phonon Third thing is that I am not convinced that I cant learn on my own/through internet/colleagues what I can learn through a phd.

@Mohammad If you realize that if you kept all the money a phd woud cost you and simply paid yourself a salary from that, suddenly things look a lot better

@IvoFlipse Hehe yup. Honestly what I would like to see is a sort of custom-mentorship program. I would not mind paying for one-on-one tutors/mentors. Mentors are very hard to come by though. Good ones.

@Mohammad I tried to get something like that, but they wanted me to come back when I had the grant proposal process going. Basically: you do all the work and then we'll consider helping you

@IvoFlipse Yeah, I mean a no-strings attached mentorship. I dont think they currently exist unfortunately! :)

Which is perhaps quite reasonable, though the process obviously goes a lot smoother if at least you have some 'proof' that you have their support

@Mohammad in Holland we have external Phds, basically you pay like 5000€ for the professors time, that's all you get

man those CS101 questions are just too easy

how many bits are there in a byte

how many values can be stored in a bit...

@IvoFlipse Yeah, Ive gone through 3 universities, and its always the same story: Proff interested in his/her research, not your learning. You pay in time and money. Give them slave labour. Get degree. Leave with gaps in knowledge. heh.

that's really CS001

lol

@IvoFlipse lol. I like that. CS001

CS1e-3

hehe :P

@Mohammad Not sure what exactly It'll give me other than opportunities to get great jobs. Most jobs I've considered applying to as long ass I can remember all required Ph.D degrees.

@Mohammad in that case its honestly better to just go for it on your own (provided you don't require a very expensive lab)

@Mohammad Additionally, I would probably disagree that you can learn as much from books as you can learn from good professors and mandatory research projects.

@Mohammad I have an entire bookshelf of mathematics, physics and DSP books. I've read most of them. But I still learned most of the stuff I know (other than coding) from professors at the university,

@Phonon That's true for sure, but I must say these online courses do make it more accessible

@IvoFlipse That's definitely true

I don't claim I can apply everything we have to implement during PA's (especially not for PGM), but I surely learned a lot of new things in the last few months

@IvoFlipse But they still teach you things on a very easy-going level. I looked at the ML curriculum on Coursera, then I looked at the one in Poly NYU (where I went to grad school), and they cover the entire ML class there in two lectures.

6 months ago I mean didn't know much about C++/Qt/Rails/Algorithms/Search algorithms/data structures etc

@IvoFlipse Online courses give you a good taste for what these things are for real, but they don't make you a good practitioner of the art.

@IvoFlipse But can you claim that you know them well now?

@IvoFlipse That's my concern.

@Phonon I think the ML course helped me understand how the things work or are you supposed to work

Well enough to be dangerous with them :P

@IvoFlipse Yeah, I guess.

@Phonon Well, its a mixed bag of nuts - Im with you on the books aspect - its great on your own time but there is finite time and the best thing is for someone to explain it to you, and involve you, get your hands dirty.

For example with Udacity it often feels like the exercises are reasonably easy, but I think that's mostly because they explained the steps to get to the goal very well

@Phonon That being said, I think this is the direction things are going with the online courses we are seeing

@Phonon The one thing I don't think any of the current courses has nailed, is applying the learned material on something "real"

in all cases, they've supplied so much boilerplate and preprocessing, that its hard to translate it to real world examples

@Phonon Also theoretically speaking yes, a good project, a good professor, a good program, etc etc are great - but realistically how popular are they? How accessible are they? This is what is fueling the current online courses IMO

I'm not sure how they could 'fix' it, but it gives a skewed view of what solving real problems is like

@IvoFlipse My tactic for that is actually doing a 'deep dive' with the classes, and I am writing my own personal library of all those functions as we learn them and have them 'in my pocket'. I think that is really useful

@IvoFlipse Yeah, I'm definitely not arguing against online courses. I think they are the most fantastic thing that happened to education in decades.

@IvoFlipse @Phonon Example just on lecture 1 of machine learning, I deep dove and learned about Hessians, convexity, etc etc.

But in the current job market, especially for people with ambition, "getting your hands dirty" just isn't good enough.

@Phonon Hehe, had this existed in 2006, I would have simply invested my time in full time studying like this rather than grad school :P

@Mohammad That library is a great idea

We should make a website out of that

@IvoFlipse Yup - try it - in fact just my gradient descent library now has stochastic gradient descent, batch gradient descent, and the normal equations. And many other bells and whistles I put in there. I can make it poly, or not, etc etc. Very nice, and I learned a lot, AND I can use it whenever now.

None of these classes teach you how to come up with things that haven't been thought of yet. I guess that's what I've been trying to say all along.

@Mohammad I'm planning to convert some of the Python stuff to C++ because I need to get more practice

@IvoFlipse Thats a good idea.

@Phonon true, though in a lot of cases you're simply applying old ideas to new problems. Getting more tools on your toolbelt definitely helps there

@Phonon Hmm, yes you are right on that... but I would say that being creative/innovative in this regard needs time and familiarity - ...

For example I think I could come up with a reasonable solution to my paw problem now, which I couldn't last year

And now its time for some relaxing, I'll be back in a bit

@Phonon The way I see it, when I have a big tool shack, I can start to get creative. If my took shack is empty, my mind is limited as to the possibilities. I think the online classes (or any class really), helps build my tool box, get VERY familiar with it, and then I can be creative with all of them

@IvoFlipse @Mohammad I'm not putting down the online learning stuff. I find it tremendously useful myself. I just don't think that for me personally it's quite enough.

@Phonon so what can we do to fix that? ;)

@Phonon Im with you on that! Its not enough for me either! :-)

@IvoFlipse That's why I started the Ph.D. conversation = )

@IvoFlipse Was just trying to explain why I'm considering the idea

@Phonon You know, the thing is, I just dont trust the PhD programs too much... if someone is in a good one, and have a good teacher, and good everything, then it might work... and even if it does work, it costs a lot. Although now you are a Dr and that is certainly worth it esp for contracts/grants/etc. But from knowledge aspect..not sure...

@Mohammad Not necessarily. Some Ph.D programs actually give you a stipend if you teach, etc.

@Phonon Yeah, but I mean opportunity cost and the stipend is not a lot compared to salary. Depends on what one is comfortable with I suppose.

@Phonon Interestingly enough, one thing that falls out of this online business, is that if I was to start a PhD, I would feel much better armed and ready... and know where I want to focus on. hehe.

@Mohammad Sure. It's more of a long-term investment. The salaries after the degree can be great though.

In the netherlands a phd is an entry level job

@Mohammad Yeah definitely

@Phonon True. And if you have a company etc you can charge the govt more. ;-)

@IvoFlipse 0_o

@Mohammad @IvoFlipse I'd even go as far as saying g_G (c)

You get like 2400€ pretax. So thats about 1500€ per month. But you probably have to TA and your basically a slave :p

But @phonon I wonder what you would make if you did research level dsp work as a contractor

@IvoFlipse Despite all the things that I know, I'm not sure that I'm truly ready to advertize myself as a research level contractor.

@Phonon What PHD program do you think about? To be completely honest, I actually do think I will do PhD someday... how or when are a different story. I actually think it would be in applied math too... that seems to be the common denominator in ALL DSP related things I see... I also want to get into the neuroscience aspect of things, vis-a-vis biologically inspired AI. But I think applied math is a good basis. What do you think?

@IvoFlipse Additionally, I don't have that much to show for it.

@Mohammad I'm actually learning applied math right now, so that I could go ahead and apply it in research. Nothing helped me more than studying graduate level linear algebra, real analysis, functional analysis and so on.

@Mohammad There's absolutely no difficulty reading all the books you couldn't handle prior to learning those things

@Phonon Yeah. So you are considering applied math phd?

@Mohammad And a lot of things make sense much more quickly. Like this Hessian matrix example. I knew that before you mentioned it.

@Mohammad No, DSP or something like that.

@Mohammad Didn't mean to make it sound like I'm bragging about it. I meant to say that it's very helpful to know these things before you dive into all these applied sciences.

@Mohammad You get "more bang for the minute" of your reading. Your intuition allows you to be one step ahead.

@Phonon Yeah. Interestingly, (and I think you touched on this as far as innovating/ being creative), I noticed from a lot of IEEE papers that a lot of innovation in the DSP came from using insert_some_obscure_property_here that they pulled out of applied stats/math, and so that is why I said to myself, maybe I should go to the heart of the matter - applied math/stats, but with an eye on DSP.

@Phonon Yeah, definitely. That 'second eye'/intuition is what I am also after as you mention being one step ahead, etc.

@phonon compared to me you surely know more math and dsp. Youd be surprised how useful that is in other fields. In my field nobody has those skills for example

@Phonon Id be interested to see what videos/books you have covered in the real analysis/fuinctional analysis etc that you recommend. Ill def take a look at them.

That's one series @Phonon recommended

@IvoFlipse Yep. Francis Su is the man

@Mohammad For functional analysis, I just bought a book.

@Mohammad I can get back to you with the name once I get home.

I'll try to watch the video's once I survive the current batch of courses

@Phonon @IvoFlipse Thanks

@IvoFlipse Yeah, you should do the courses first = )

@Phonon Yeah please do, Id be interested in the book if you think highly of it. How long are Su's lectures btw?

@Phonon How do you suppose, in light of Jims answer, the utility of simply sampling at a faster rate squares with interpolating a low sampling rate signal ?

@Phonon If upsampling is 'good' in this regard, then why bother having a high sample rate to begin with?

@Phonon Still seems to me that a higher sampling rate is the 'true' way to get a better peak location no?

@Mohammad Twenty-six lectures. Around one hour each.

@Mohammad surely there's a point at which upsampling won't be able to be able to recover the content from the original signal

@Phonon ah, so I only need one day to watch them all :P

@IvoFlipse One Dutch day perhaps. We Earthlings run on 24-hour days. = )

@Phonon I'll put the speed on 1.25 :P

@Phonon Hmm not too bad I suppose...

@IvoFlipse It took me mo[N]ths to absorb all that stuff. = )

@IvoFlipse Yeah, thats what I am trying to get at.. :-/

@Phonon @IvoFlipse The thing is I usually pause and think on many points so it will take me a lot of time to integrate all that material

@Phonon I don't have any illusions about actually understanding this stuff

I made manual notes during ml-class/db-class/ai-class but that took forever

@IvoFlipse Sure, I don't think anyone can get it quickly enough. But if you actually compare something like this to the online courses were taking now, you'll definitely feel the difference. Additionally, you won't even be doing any homework. Imagine how much his actual students learned.

now I copy paste screenshots into Evernote and try to summarize stuff, but even that costs a lot of time

@IvoFlipse Have you looked into mind-maps?

@IvoFlipse I use them to summarize my understanding in the very end. In fact since they are visual its very hard to forget main things

nope, but I'm starting to consider to take less notes actually

@IvoFlipse @Phonon One strategy that I have found useful is the following: I will watch a video once on fast, without note taking, or anything. Just watch and move along. This gives me the overall jist and flavour. (In fact I do like 3 videos like this). I look up things that I may have not understood here and there. THEN I go back and 'really' watch the videos one by one.

@IvoFlipse @Phonon Its really made a lot of difference for me. I get things a lot better this way. Your brain knows what to expect and can focus on integration instead of guessing.

The biggest problem with trying to take so many notes is that you can't watch video's leisurely at the 'risk' of missing something. Only to have to watch the video's again while making PAs anyway

@Mohammad Hehe, well that basically describes what I'm planning on doing in the future

@IvoFlipse Yup - Ive found it useful. Might take 1.5 or double the time, but I get 4 to 4 times as much understanding? hehe

@IvoFlipse 3-4*

Though watching every video twice would take way too much time in some cases

@IvoFlipse Surprisingly (or not so surprisingly) if I teach it to someone (or even talk aloud to myself about it) you end up learning it more! Is why I am very interested in teaching as well, although thats not my only (ulterior) motive! :-)

@Mohammad I don't have anyone to teach it to

and honestly I'm in to many courses to be more active on the forums and help others

@IvoFlipse Yeah. I will sometimes just 'prove' it to myself on paper or something just to integrate it into my head some more.

@IvoFlipse I feel that way, too. I wish I had more time to spend on those forums.

Well I once I've learned how to make a blog on Udacity, I should put up some of the stuff I've learned

given this weeks assignment is adding accounts, I can make you an account too :P

@IvoFlipse Feel free to make an account for me too. lool! :P

@IvoFlipse Seriously I hope to take that web course soon too

well I can put up the material from CS253 as well, that should get you started :P

Actual wiring of the human brain.

@Mohammad Yeah. Screw Cartesian dualism!

time for bed for me, I'll see you guys tomorrow!

@Phonon Hahah! Yup! :)

@IvoFlipse Good night bud

@IvoFlipse Good night

@Phonon Its amazing - you look at that and you see order and beauty

@Phonon Its not just this jumble of connections we might think it is.

@Mohammad Yeah. Especially with all these colors it looks like one trippy neuro-mohawk.

@Phonon Hehe yeah

@Phonon Theres actually a well proven and rich understanding of how much of 'thinking' and 'learning' happens, and the key turns out to be heirarchy

@Phonon Analogous to wavelets I suppose - basically looking at an object to be analyzed at different scales

@Phonon Alright dude, gotta run - catch ya later! :-)

@Phonon Email me your thoughts when u get a chance