Transition probabilities from selected texts

From Derek
Revision as of 22:42, 13 July 2009 by A1133050 (talk | contribs) (New data using new punctuation filter)
Jump to navigation Jump to search

The Somerton Man's code (without the extra line) is 44 characters long. So, if the text is purely random (1/26 chance of each letter appearing) then the probability of attaining this particular string of 44 is (1/26)^44 = 5.51027E-63. This is a good initial comparison.

For transitions that have p=0, corrections to p=0.0001 have been performed to attain a non-zero Markov probability.

HMMER score[1] is the log (base 2) of Markov probability / null probability (1/26^44)

First order

All letters


(..\Texts\1984 - George Orwell.txt) All Letters
Markov Probability: 1.4822672916815308E-71
Corrected Zeroes: 1
HMMER Score: -28.46974151192516

(..\Texts\Les Orientales - Victor Hugo.txt) All Letters
Markov Probability: 7.955726018472886E-79
Corrected Zeroes: 2
HMMER Score: -52.620978304803444

(..\Texts\Traumdeutung - Sigmund Freud.txt) All Letters
Markov Probability: 3.749298888974187E-77
Corrected Zeroes: 1
HMMER Score: -47.062494868234964

(..\Texts\Vigenere - 1984.txt) All Letters
Markov Probability: 1.646391769425068E-70
Corrected Zeroes: 0
HMMER Score: -24.99631136880728

Initial letters


(..\Texts\1984 - George Orwell.txt) Initial Letters
Markov Probability: 2.0136596296001355E-56
Corrected Zeroes: 0
HMMER Score: 21.80119412864152

(..\Texts\Les Orientales - Victor Hugo.txt) Initial Letters
Markov Probability: 3.3267604806714393E-60
Corrected Zeroes: 0
HMMER Score: 9.237779904103608

(..\Texts\Traumdeutung - Sigmund Freud.txt) Initial Letters
Markov Probability: 3.820168061668581E-68
Corrected Zeroes: 1
HMMER Score: -17.138126745609156

Second order

All letters


(..\Texts\1984 - George Orwell.txt) All Letters
Markov Probability: 3.9262648017739784E-100
Corrected Zeroes: 15
HMMER Score: -123.40030441377017

(..\Texts\Les Orientales - Victor Hugo.txt) All Letters
Markov Probability: 2.1087630055723357E-106
Corrected Zeroes: 18
HMMER Score: -144.22863349364334

(..\Texts\Traumdeutung - Sigmund Freud.txt) All Letters
Markov Probability: 3.731464295941246E-119
Corrected Zeroes: 21
HMMER Score: -186.5903538114491

(..\Texts\Vigenere - 1984.txt) All Letters
Markov Probability: 1.669944098510842E-92
Corrected Zeroes: 8
HMMER Score: -98.05823732223358

Initial letters


(..\Texts\1984 - George Orwell.txt) Initial Letters
Markov Probability: 7.555198589304339E-61
Corrected Zeroes: 2
HMMER Score: 7.0992034873802545

(..\Texts\Les Orientales - Victor Hugo.txt) Initial Letters
Markov Probability: 1.0973476039668194E-80
Corrected Zeroes: 9
HMMER Score: -58.80087939586076

(..\Texts\Traumdeutung - Sigmund Freud.txt) Initial Letters
Markov Probability: 1.457883499720296E-103
Corrected Zeroes: 18
HMMER Score: -134.7953707374809


References

See also

Back