Fair coin: p = 0.5, 0.5 — Shannon entropy
introthe definition of a bit
Answer
1.0000 bits
Why this example is worth doing
One fair binary choice is one bit — this is the definition the unit is named for, and the page starts here so that every later number has a reference point. The tool shows the sum term by term, each symbol contributing its probability times its surprisal, because the formula is more convincing when the two halves are visibly identical.
Try your own input in the Shannon entropy. Bits per symbol for any distribution, with the surprisal of each symbol shown.
How the answer is reached
Shannon entropy of 2 symbols
2 symbols, 2 total occurrences. Divide by the total to get probabilities, then take −Σ p·log₂ p.
| Symbol | p | Surprisal −log₂ p (bits) | Contribution p·(−log₂ p) |
|---|---|---|---|
| H | 0.500000 | 1.000000 | 0.500000 |
| T | 0.500000 | 1.000000 | 0.500000 |
H1.000000 bits/symbol— the probability-weighted mean of the surprisal column
Total information2.000000 bits— 2 symbols × H
Maximum possible H1.000000 bits/symbol— log₂(2), reached only by the uniform distribution
H / log₂(m)1.000000— how close to uniform this source is
A symbol of probability 0 contributes exactly 0: the convention 0·log₂0 = 0 is a definition (the limit as p → 0), not an approximation. Evaluating it instead returns NaN, which is the usual bug on this page.
Source: C. E. Shannon, Bell System Technical Journal 27:379–423 (1948), Theorem 2