Property 1:- Mutual Information is Non-Negative
Mutual Information is given by equation
we know that
Substitute Equation (II) in Equation (I)
The above Equation can be written as
we knew that
This result can be applied to Mutual Information , If and be , Both and are two probability distributions on same alphabet , then Equation (III) becomes
i.e, , Which implies that Mutual Information is always Non-negative (Positive).