About this document
Understanding Shannon by rose marine is a document available to read on EtoBox.
The document introduces concepts related to information theory including Shannon entropy and the source coding theorem. It defines Shannon entropy as the expected information content or uncertainty of a random variable. Entropy is always non-negative and maximized by a uniform distribution. The source coding theorem states that the optimal number of bits needed to compress data is the entropy of the source. Independent random variables have additive entropy.
- Author
- rose marine
- Language
- EN