Shannon Entropy Calculator

Shannon entropy measures the average uncertainty in a set of outcomes. For probabilities p1 through pn it equals the negative sum of each probability times the log of that probability, written H = -sum(p * log(p)). Higher entropy means the outcomes are harder to predict, while a value of zero means one outcome is certain.

The only difference is the logarithm base. Bits (also called shannons) use log base 2, nats use the natural log (base e), and dits (also called hartleys or bans) use log base 10. The values scale by a fixed factor, so 1 bit equals about 0.693 nats or 0.301 dits, and the efficiency and perplexity come out the same in every unit.

Either works. The calculator divides every value by their total to form a probability distribution, so the counts 2, 2, 1 give the same entropy as the probabilities 0.4, 0.4, 0.2. If you enter probabilities that do not add up to 1, they are rescaled, and the input total shows you what they summed to.

Maximum entropy is the entropy of a uniform distribution over the same number of outcomes, equal to log(n), and it is the largest entropy those outcomes can reach. Efficiency, also called normalized entropy, is the actual entropy divided by that maximum, shown as a percentage from 0% to 100%. Redundancy is 100% minus the efficiency, so a near-uniform distribution has high efficiency and low redundancy.

Perplexity is the effective number of equally likely outcomes, found by raising 2 to the entropy in bits. A fair coin has a perplexity of 2 and a fair six-sided die has a perplexity of 6. It does not depend on the unit you pick, because 2 to the power of H in bits equals e to the power of H in nats.

Text mode counts how often each character appears and treats those counts as symbol frequencies. It is case sensitive and includes spaces and punctuation, and it reads emoji and other characters by Unicode code point so each one counts as a single symbol. The result is the Shannon entropy of that character distribution, a rough measure of how varied or compressible the text is.