And then in order to normalize for length you can take the length's root (which, as you said with using logs, just means divide the summed logs by the length)
So your log score is (log(p("T")) + log(p("E" | "T") ) + log(("S" | "E")) + log(p("T" | "S"))) / 4
Since you're dealing with emails, you might want to take the max of (score with numbers removed, score with numbers transliterated)
And then tune that threshold so that it grabs the addresses you want
i'm not quite following why that should be the case - here's what i did - i took the "Bigram Frequencies" from that site, which has a list of all 26*26 digrams and a count of how many times they appeared in the source text, i then normalized them to percentages such that the sum of the 26*26 percentages summed to 1
ooooh ok, applied to markov chains, where you already know it's an E, you need to use a normalized to unity version of the 26 probabilities or the next character!
ok so that changes the values of what i posted above - i can fix that, let me read through the rest of your steps now...
ok makes sense - out of curiousity, what do you expect to happen when scoring "test" versus "testtest", does having more text to analyze make it better at identifying, and so "testtest" would be expected to score higher?
maybe this is where chain length comes in - having twice as many "decent" english letter combinations should probably outweight a single p(t|t) penalty