Fair six-sided die — Shannon entropy
corea non-integer answer
Answer
2.5850 bits
Why this example is worth doing
Six equally likely outcomes give log₂6, which is not a whole number — and that is the point. Entropy is the average number of bits needed per outcome in the limit of long sequences, not the number of bits needed for one; you cannot transmit a die roll in 2.585 bits, but a thousand rolls fit in about 2585. The page makes that asymptotic reading explicit, because it is the most common misunderstanding of the quantity.
Try your own input in the Shannon entropy. Bits per symbol for any distribution, with the surprisal of each symbol shown.
How the answer is reached
Shannon entropy of 6 symbols
6 symbols, 6 total occurrences. Divide by the total to get probabilities, then take −Σ p·log₂ p.
| Symbol | p | Surprisal −log₂ p (bits) | Contribution p·(−log₂ p) |
|---|---|---|---|
| 1 | 0.166667 | 2.584963 | 0.430827 |
| 2 | 0.166667 | 2.584963 | 0.430827 |
| 3 | 0.166667 | 2.584963 | 0.430827 |
| 4 | 0.166667 | 2.584963 | 0.430827 |
| 5 | 0.166667 | 2.584963 | 0.430827 |
| 6 | 0.166667 | 2.584963 | 0.430827 |
H2.584963 bits/symbol— the probability-weighted mean of the surprisal column
Total information15.509775 bits— 6 symbols × H
Maximum possible H2.584963 bits/symbol— log₂(6), reached only by the uniform distribution
H / log₂(m)1.000000— how close to uniform this source is
A symbol of probability 0 contributes exactly 0: the convention 0·log₂0 = 0 is a definition (the limit as p → 0), not an approximation. Evaluating it instead returns NaN, which is the usual bug on this page.
Source: C. E. Shannon, Bell System Technical Journal 27:379–423 (1948), Theorem 2