Coursera Study Group

Ivo Flipse

09:51

@Phonon did you manage to get working on PGM?

Phonon

12:36

@IvoFlipse Hi! Yes.

@IvoFlipse I got LogZ working, and stopped there. Will try and finish the rest tonight.

Ivo Flipse

@Phonon I'm curious how you calculated logZ, because mine doesn't feel very general

I've been scouring the forums to figure out how people did stuff

I think I have an idea of how they create the cliquetree

but I'm not sure how the pieces fit together just yet

think I'll first try to wrap up NLP, so at least I don't have to worry about that one anymore

Phonon

There was one forum response that I read that lead me to the answer

@IvoFlipse Clique tree calibration makes every cluster (i.e. joint factor of a clique) to agree on all of its variables with every other cluster.

Ivo Flipse

temp = FactorProduct(P.cliqueList(1), unnormalizedMessages(2));
logZ = log(sum(temp.val));

That's rather naive, don't you think :P

Phonon

Are you getting a correct value this way?

Ivo Flipse

yup

Phonon

12:41

Sure

It makes good sense

(now that I read the forums that is)

Ivo Flipse

yeah, but I'm worried about the hardcoded 1 and 2

Phonon

K, so here's a sketch of why that works

suppose you have a factor f(a,b)

and a factor f(b,c)

And they talk to each other. The sepset is b

Ivo Flipse

so 2 is what they have in common

Phonon

yes

Ivo Flipse

right what's a sepset anyway, it always felt to me like that its just a subset :P

Phonon

12:43

That's what two cliques have in common. Exactly what you said, just with a fancy name attached to it.

So

Ivo Flipse

this one only works for our 3 word network, so to make it more general, I should find all sepsets I reckon?

Phonon

This is why your method works

@IvoFlipse Your message passing does that already. The scope of every message from factor i to factor j is the sepset of i and j.

Suppose that if you have four cliques in general, you can get a joint factor f(a,b,c,d)

Ivo Flipse

@Phonon is does it, because I hardcoded it in there :P

Phonon

How do you find partition function then?

►

Ivo Flipse

@Phonon how are they connected? like a markov model, all in one line?

Phonon

12:47

@IvoFlipse Doesn't matter. All we have to know right now is that f(a,b) talks to f(b,c)

@IvoFlipse Yes, it's a tree

Ivo Flipse

Hmmm wait I think I didn't fully grasp what the unnormalized message really is, I was now assuming it was just the factors from b

so if you have f(a,b,c,d) then we'd have to take 2 factor products, because we would have to do the same with c

Phonon

Ok, how would you find message delta(1->2)?

from f(a,b) to f(b,c)

Ivo Flipse

actually, I have no clue, I got that we were doing it (on the slides), but not in the code :P

Phonon

Well, the sepset is b

Ivo Flipse

@Phonon but b would be P.cliqueList(1) with a marginalized out?

Phonon

12:53

so you marginalize out a (sum over all a's for every b). That's your message. It's the factor's belief about what b is.

So msg(1,2) = sum_a ( f(a,b) )

or something like that

for the lack of better notation

Ivo Flipse

well in our current case, unnormalizedMessages(1) is empty, so I can't really use that

Phonon

and msg(2,1) = sum_c ( f(b,c) )

Ivo Flipse

2,1? not 2,3?

Phonon

2,1

Ivo Flipse

oh wait the message goes backwards

Phonon

12:57

message from f(b,c) back to f(a,b)

yes!

So

Ivo Flipse

I got what she was talking about, just no clue how it translates to code :P

Phonon

The message back to f(a,b) summarized beliefs about b of the entire remaining network

Ivo Flipse

oh so, unnormalizedMessages is the message coming back?

Phonon

No, the difference between normalized and unnormalized is exactly what it sounds like

One is normalized at every step ( sum( message.val ) == 1 )

Ivo Flipse

@Phonon lol, but why would you multiple those two, that seems awkward

Phonon

12:59

The other one isn't

Ivo Flipse

unless you're trying to figure out with what you have to normalize :P

Phonon

If you're normalizing messages, you're changing your normalization constant for every clique

If you don't the partition function stays the same everywhere

And that's the key

When you multiply those two, your factor now agrees with the rest of the network on everything.

So

Ivo Flipse

So I'll leave my code as is ^^

Phonon

f(a,b) * msg(2->1) = sum_{c,d}f(a,b,c,d)

If you sum over a and b, you get sum_{a,b}[f(a,b)*msg(2->1)] = sum_{a,b,c,d}f(a,b,c,d) = Z

That's why it works

So yeah

You can leave it

I think

As long as it follows ^^

If your unit tests pass, then it does

Ivo Flipse

well the rest of the assignment also uses a 3 letter word

so I'll be fine

Phonon

13:26

cool

Ivo Flipse

14:12

@Phonon how do I create empty factors to which I can assign?

I can make an empty struct, but then I need to add like 3300 empty factors

Ivo Flipse

14:24

All the stuff I see so far only creates a single factor, I need to create the struct that contains all of them :S

Ivo Flipse

14:43

ah @Phonon I 'realized' my mistake, I should be initializing the factor for every i-th element of the struct, d'oh

I could move that work outside of the loop, but that shouldn't be an issue

but I think I have to review exactly what val should look like, some people mentioned each factor has 26 vals, where only values to which we assign theta are non-zero

ah, wait now I get what's wrong, I should get two trees

the singleton one and the one with 2 factors/features whatever :P

there are 1352 2 factor features, which happens to be 26*26*2

The way I went about doing it is:

Build all 2 variable factors (that is 1-2 and 2-3 in the sample case)
Build all 1 variable factors (that is 1, 2 and 3 in the sample case)
Multiply the 1 variable Factors into the 2 variable ones (each factor only once).

I'm getting different values for my output than the sampleUncalibratedTree, probably because I'm not assigning values correctly (cardinalities / variables are correct).

The featureSet contains 3926 features. That is 676 * 2 for the assignments to [1 2] and [2 3], but also 2574 single variable features... that is 858 for each variable [1 …

class.coursera.org/pgm/forum/thread?thread_id=2035

Mohammad

15:23

@Phonon @IvoFlipse Hello hello hello!

Phonon

17:51

@Mohammad Hey dude

Mohammad

18:02

@Phonon Hey! How r ya?

Phonon

@Mohammad Crazy week at work + one day to complete PGM PA. Fun = ) You?

Mohammad

@Phonon LoL - same with work - boss tells me to make a presentation for tomorrow. "Sorry for no notice." Me: ಠ_ಠ

Phonon

@Mohammad Wow. Definitely don't envy you there.

Mohammad

@Phonon Heh yeah. Digging pretty deep in ML class though

Phonon

@IvoFlipse I'm not up to that part yet. = )

Mohammad

18:05

@Phonon Will start neural / nonlinear classifiers this week

Ivo Flipse

@Phonon well that's the part after logZ

and hi @Mohammad

Phonon

18:20

@IvoFlipse Oh, I didn't realize that

@IvoFlipse I thought you said you had it working at some point

Ivo Flipse

I had logZ working :P

but after that we have to get the negative log likelihood thing working

Phonon

@IvoFlipse Yeah, I'm not up to that part = )

Ivo Flipse

but I was think I was going in the right direction with making the cliquetree

Phonon

I only passed the LogZ test

stopped there

Ivo Flipse

well its right after logz

there's only 2 more things, which are quite large (relatively speaking)

Phonon

18:24

@IvoFlipse Yeah

@IvoFlipse As soon as my LogZ test passed, I went to sleep lol = )

Ivo Flipse

you get those features, turn them into factors, then into a clique tree, calibrate it and then do some mojo stuff I don't get

Phonon

@IvoFlipse That'll have to be tomorrow's conversation = )

Mohammad

18:44

@Phonon @IvoFlipse Do yall think there is any inherent advantage of doing matching filtering in the time-frequency domain (as 2D) VS just in the the 1-D time domain?

@Phonon @IvoFlipse GOing to ask a q but want to soundboard it a little first

Phonon

@Mohammad For signal detection?

Mohammad

@Phonon Yeah

Phonon

@Mohammad How would you do it in time-frequency domain?

Image matching?

Mohammad

@Phonon I would design a match filter, (2D of course), only now its width and legnth (and shape) are related to the bandwidth of the pulse I am after, its duration in time, and general shape.

@Phonon Yeah exactly.

@Phonon VS in time-domain which just has duration and shape information

@Phonon Then just convolve 2d 'filter'/template against the spectrogram of incoming signal

@Phonon Then sum across all frequencies, to get back a 1-D signal

@Phonon Whatcha think?

Ivo Flipse

19:31

No clue :P

Transcript for

Coursera Study Group