Archive for the ‘Information Theory’ Category

Is there more information in a rock than in the human genetic DNA code?

August 18, 2007

How much information is in here:

  • It is a fine day.

Let’s measure the information by the number of characters. There are 17 characters, so the amount of information is 17.

How much information is in here:

  • It is a fine day. It is a fine day. It is a fine day. It is a fine day. It is a fine day.

The same sentence is repeated five times. Is the amount of information 17 x 5 = 85? Answer: No. The extra four sentences don’t say anything new. The information that is present is this:

  • Repeat 5 times: It is a fine day.

The number of characters in this is: 33

How much information is in here:

  • 3.1415 …

This is the value of pi. Suppose one million digits are displayed. None of the digits repeat, so you might be tempted to say that the amount of information is one million. Not so. The information can be represented succinctly as:

  • Pi to one million digits

Now the amount of information is just 24.

How much information is in here:

  • dkdl;eekrkpeosfdlzdmc;dsfkdopkfospkfs;dlkflas;krwe0q0–03ospaaj

This is just a random sequence of 63 characters. If any random characters will do, then the information can be represented simply as:

  • Random sequence of 63 characters

There are 33 characters.

Something is “information” if it is meaningful, non-random, and unpredictable [1].

How much information is in a rock? If we were to characterize all the properties (location, angular momentum, spin, velocity, and so on) of every atom in the rock, we would have a vast amount of information. A one-kilogram rock has 100000000000000000000000000000 (29 zeros) atoms. That’s one hundred million billion times more information than the genetic code of a human race. But for most common purposes, the bulk of this information is largely random and of little consequence. So we characterize the rock for most purposes with far less information just by specifying its shape, location, and the type of material of which it is made. Thus, it is reasonable to consider the information of an ordinary rock to be far less than that of a human even though the rock theoretically contains vast amounts of information.

[1] If you know what’s going to be said (i.e. it’s predictable) then it’s not information.

– – Extracted from The Singularity is Near by Ray Kurzweil

Life without “E”

July 30, 2007

“In counting the frequency of usage of the letter E in all English prose we will find that E constitutes 13% of all letters appearing, while W, for instance, constitutes only about 2% of all letters appearing.”

“In 1939, Ernest Vincent Wright published a 267-page novel, Gadsby, in which no use is made of the letter E.  I quote from a paragraph below:

Upon this basis I am going to show you how a bunch of bright young folks did find a champion: a man with boys and girls of his own; a man of so dominating and happy individuality that Youth is drawn to him as a fly is to a sugar bowl.  It is a story about a small town.  It is not a gossipy yarn; nor is it a dry, monotonous account, full of such customary ‘fill-ins’ as ‘romantic moonlight casting murky shadows down a long, winding country road.’  Nor will it say anything about twinklings lulling distant folds; robins carolling at twilight, nor any ‘warm glow of lamplight’ from a cabin window.  No.  It is an account of up-and-doing activity; a vivid portrayal of Youth as it is today; and a practical discarding of that wornout notion that a ‘child don’t know anything.'”

– An Introduction to Information Theory by John R. Pierce