Biased coin: p = 0.9, 0.1 — Shannon entropy
intropredictability costs information
Answer
0.4690 bits
Why this example is worth doing
A heavily biased coin carries less than half a bit per flip, because most outcomes are unsurprising. The page plots entropy against p across the whole range so the maximum at one half and the zeros at both ends are visible as a curve, which conveys the shape of the function far better than any single number. It also gives each outcome's surprisal, showing that the rare event carries more information and occurs less often.
Try your own input in the Shannon entropy. Bits per symbol for any distribution, with the surprisal of each symbol shown.
How the answer is reached
Shannon entropy of 2 symbols
2 symbols, 10 total occurrences. Divide by the total to get probabilities, then take −Σ p·log₂ p.
| Symbol | p | Surprisal −log₂ p (bits) | Contribution p·(−log₂ p) |
|---|---|---|---|
| H | 0.900000 | 0.152003 | 0.136803 |
| T | 0.100000 | 3.321928 | 0.332193 |
H0.468996 bits/symbol— the probability-weighted mean of the surprisal column
Total information4.689956 bits— 10 symbols × H
Maximum possible H1.000000 bits/symbol— log₂(2), reached only by the uniform distribution
H / log₂(m)0.468996— how close to uniform this source is
A symbol of probability 0 contributes exactly 0: the convention 0·log₂0 = 0 is a definition (the limit as p → 0), not an approximation. Evaluating it instead returns NaN, which is the usual bug on this page.
Source: C. E. Shannon, Bell System Technical Journal 27:379–423 (1948), Theorem 2