How this calculator works
Shannon entropy averages the information carried by an outcome, weighted by its probability. A zero-probability category contributes zero by continuity. Counts are normalised to probabilities; probability mode checks their total. Maximum entropy assumes equal probability across all entered categories, including zero-weight categories. Text mode counts code points and preserves internal spaces and case, but does not model symbol order or language. The engine trims outer whitespace before calculation.
Formula
H = -Σ pᵢ log_b(pᵢ); Hmax = log_b(k); effective outcomes = b^H. Define 0 log(0) = 0.
Worked example
Probabilities 0.5, 0.25, 0.25 give entropy 1.5 bits. Four equally likely categories give 2 bits. Text aabb has two equally frequent symbols and entropy 1 bit per symbol.
Advertisement
Frequently asked questions
Does changing the logarithm base change uncertainty?
It changes the unit. Divide entropy in nats by ln(2) for bits or ln(10) for decimal digits.
Is text entropy a compression guarantee?
This frequency calculation ignores order and dependencies. A language-aware compressor can exploit more structure.
Why keep zero-weight categories in the maximum?
They are part of the alphabet you supplied. Entropy uses their zero contribution, while the alphabet maximum considers all categories equally likely.