Mutual information
Also known as: transinformation content · transferred information · transmitted information
- 1Quantifies the mutual dependence between two random variables.
- 2Measures the information gained about one variable by observing another.
- 3Intimately linked to the concept of entropy.
- 4Functions as a core metric in probability theory and information theory.
In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables.
Connections2
Measure of statistical dependence
This quantity measures the amount of information obtained about one random variable through observing another, serving as a key metric in information theory.
Measure of shared information between variables
Derived from entropy, this quantity measures the amount of information obtained about one random variable through observing another.
Mutual information (MI) is a measure in probability and information theory that quantifies the dependence between two random variables. It specifically measures the amount of information that can be obtained about one variable by observing another.
The relationships between information theoretic quantitiesWikimedia Commons: File:Figchannel2017ab.svgVenn diagram showing additive and subtractive relationships of various information measures associated with correlated variables X {\displaystyle X} and Y {\displaystyle Y}. [ 1 ] The area contained by either circle is the joint entropy H ( X, Y ) {\displaystyle \mathrm {H} (X,Y)}. The circle on the left (red and viol...Wikimedia Commons: File:Entropy-mutual-information-relative-entropy-relation-diagram.svg This concept is closely related to entropy, which is a fundamental measure of the expected amount of information held within a random variable. Together, these metrics allow researchers to analyze the statistical relationships and information flow within complex systems.