Information Theory

  • Pictogram: a drawing that resembles the physical object that it represents, eg. a happy face or a human
  • Ideogram: a conceptual picture of an abstract idea, like the symbol of peace
  • Rebus theory: sound + sound = new meaning
  • Proto language: instead of letters using icons

Morse code

  • bits and dashes to compose letters
  • send through electric impulses
  • uses the binary combinations like the flag system
  • the most used letters were the easier to type

information transmission

Channel Capacity

  • Symbol rate (n): how many symbols transferred per second eg. a bitrate. Also known as Baud.
  • Difference (s): how many differences per symbol, eg. a bit would be two differences per symbol.
  • the symbol rate would dictate the depth of the decision tree
  • Message space: the width of the base of the decision tree or $$s^n$$. In other words, how many different symbols or letters can be sent in a specific amount of time.

Measure amount of information transmitted in a unit of time by

log(s^n)