## 3.13 Covariance

SW 2.3

The covariance between two random variables $$X$$ and $$Y$$ is a masure of the extent to which they “move together”. It is defined as

$\mathrm{cov}(X,Y) := \mathbb{E}[(X-\mathbb{E}[X])(Y-\mathbb{E}[Y])]$ A natural first question to ask is: why does this measure how $$X$$ and $$Y$$ move together. Notice that covariance can be positive or negative. It will tend to be negative if big values of $$X$$ (so that $$X$$ is above its mean) tend to happen at the same time as big values of $$Y$$ (so that $$Y$$ is above its mean) while small values of $$X$$ (so that $$X$$ is below its mean) tend to happen at the same time as small values of $$Y$$ (so that $$Y$$ is below its mean).

An alternative and useful expression for covariance is $\mathrm{cov}(X,Y) = \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]$ Relative to the first expression, this one is probably less of a natural definition but often more useful in mathematical problems.

One more thing to notice, if $$X$$ and $$Y$$ are independent, then $$\mathrm{cov}(X,Y) = 0$$.