Information Theory (1948)

Claude E. Shannon’s work “A Mathematical Theory of Commu- nication” created the field of information theory.
Idea: any source transmitting data is a Markov process.
In a communication system with symbols such as letters, one must determine transition probabilities for a letter following anyother letter (e.g., prob. of “u” following “q” is high).
Shannon used these chains to:
- predict communication channel load and optimize utilization of bandwidth.
- generate text using n-grams. First attempt with tri-grams resulted in sentence:
"IN NO IST LAT WHEY CRATICT FOURE BIRS GROCID"


Markov chains have since been used to generate text, images, and music, as well as in speech recognition.

Shannon and his Machines

1) In 1950 he made mechanical mouse Theseus , remote-controlled complex electronic circuit. This mouse learned to find a way out of labyrinth. In honor of his invention, IEEE has established an international competition "mini mouse", which still involve thousands of students of technical universities.

2) In the same 50-s Shannon created a machine that "read thoughts" in the game of "coin": the person guessing "heads " or "tails", and guessed the machine with a probability higher than 50% because that people can not avoid any patterns that the machine can be used.
Philipp’s applet.

It was the first realization of a Markov chain in an electric form.