Cross-covariance

See also: Cross-correlation

In probability and statistics, given two stochastic processes $\left\{X_{t}\right\}$ and $\left\{Y_{t}\right\}$, the cross-covariance is a function that gives the covariance of one process with the other at pairs of time points. With the usual notation $\operatorname {E}$; for the expectation operator, if the processes have the mean functions $\mu _{X}(t)=\operatorname {\operatorname {E} } [X_{t}]$ and $\mu _{Y}(t)=\operatorname {E} [Y_{t}]$, then the cross-covariance is given by

$\operatorname {K} _{XY}(t_{1},t_{2})=\operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} [(X_{t_{1}}-\mu _{X}(t_{1}))(Y_{t_{2}}-\mu _{Y}(t_{2}))]=\operatorname {E} [X_{t_{1}}Y_{t_{2}}]-\mu _{X}(t_{1})\mu _{Y}(t_{2}).\,$

Cross-covariance is related to the more commonly used cross-correlation of the processes in question.

In the case of two random vectors $\mathbf {X} =(X_{1},X_{2},\ldots ,X_{p})^{\rm {T}}$ and $\mathbf {Y} =(Y_{1},Y_{2},\ldots ,Y_{q})^{\rm {T}}$, the cross-covariance would be a $p\times q$ matrix $\operatorname {K} _{XY}$ (often denoted $\operatorname {cov} (X,Y)$) with entries $\operatorname {K} _{XY}(j,k)=\operatorname {cov} (X_{j},Y_{k}).\,$ Thus the term cross-covariance is used in order to distinguish this concept from the covariance of a random vector $\mathbf {X}$, which is understood to be the matrix of covariances between the scalar components of $\mathbf {X}$ itself.

In signal processing, the cross-covariance is often called cross-correlation and is a measure of similarity of two signals, commonly used to find features in an unknown signal by comparing it to a known one. It is a function of the relative time between the signals, is sometimes called the sliding dot product, and has applications in pattern recognition and cryptanalysis.

Cross-covariance of random vectors

Definition

For random vectors $\mathbf {X}$ and $\mathbf {Y}$, each containing random elements whose expected value and variance exist, the cross-covariance matrix of $\mathbf {X}$ and $\mathbf {Y}$ is defined by

$\operatorname {K} _{\mathbf {X} \mathbf {Y} }=\operatorname {cov} (\mathbf {X} ,\mathbf {Y} ){\stackrel {\mathrm {def} }{=}}\ \operatorname {E} [(\mathbf {X} -\mathbf {\mu _{X}} )(\mathbf {Y} -\mathbf {\mu _{Y}} )^{\rm {T}}]$ (Eq.1)

where $\mathbf {\mu _{X}} =\operatorname {E} [\mathbf {X} ]$ and $\mathbf {\mu _{Y}} =\operatorname {E} [\mathbf {Y} ]$ are vectors containing the expected values of $\mathbf {X}$ and $\mathbf {Y}$. The vectors $\mathbf {X}$ and $\mathbf {Y}$ need not have the same dimension, and either might be a scalar value. Any element of the cross-covariance matrix is itself a "cross-covariance".

Example

For example, if $\mathbf {X} =\left(X_{1},X_{2},X_{3}\right)^{\rm {T}}$ and $\mathbf {Y} =\left(Y_{1},Y_{2}\right)^{\rm {T}}$ are random vectors, then $\operatorname {cov} (\mathbf {X} ,\mathbf {Y} )$ is a $3\times 2$ matrix whose $(i,j)$-th entry is $\operatorname {cov} (X_{i},Y_{j})$.

Properties of cross-covariance matrix

For the cross-covariance matrix, the following basic properties apply:

1. $\operatorname {cov} (\mathbf {X} ,\mathbf {Y} )=\operatorname {E} [\mathbf {X} \mathbf {Y} ^{\rm {T}}]-\mathbf {\mu _{X}} \mathbf {\mu _{Y}} ^{\rm {T}}$
2. $\operatorname {cov} (\mathbf {X} ,\mathbf {Y} )=\operatorname {cov} (\mathbf {Y} ,\mathbf {X} )^{\rm {T}}$
3. $\operatorname {cov} (\mathbf {X_{1}} +\mathbf {X_{2}} ,\mathbf {Y} )=\operatorname {cov} (\mathbf {X_{1}} ,\mathbf {Y} )+\operatorname {cov} (\mathbf {X_{2}} ,\mathbf {Y} )$
4. $\operatorname {cov} (A\mathbf {X} +\mathbf {a} ,B^{\rm {T}}\mathbf {Y} +\mathbf {b} )=A\,\operatorname {cov} (\mathbf {X} ,\mathbf {Y} )\,B$
5. If $\mathbf {X}$ and $\mathbf {Y}$ are independent (or somewhat less restrictedly, if every random variable in $\mathbf {X}$ is uncorrelated with every random variable in $\mathbf {Y}$), then $\operatorname {cov} (\mathbf {X} ,\mathbf {Y} )=0_{p\times q}$

where $\mathbf {X}$, $\mathbf {X_{1}}$ and $\mathbf {X_{2}}$ are random $p\times 1$ vectors, $\mathbf {Y}$ is a random $q\times 1$ vector, $\mathbf {a}$ is a $q\times 1$ vector, $\mathbf {b}$ is a $p\times 1$ vector, $A$ and $B$ are $q\times p$ matrices of constants, and $0_{p\times q}$ is a $p\times q$ matrix of zeroes.

Definition for complex random vectors

Main article: Complex random vector § Cross-covariance matrix and pseudo-cross-covariance matrix

If $\mathbf {Z}$ and $\mathbf {W}$ are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by hermitan transposition:

$\operatorname {K} _{\mathbf {Z} \mathbf {W} }=\operatorname {cov} (\mathbf {Z} ,\mathbf {W} ){\stackrel {\mathrm {def} }{=}}\ \operatorname {E} [(\mathbf {Z} -\mathbf {\mu _{Z}} )(\mathbf {W} -\mathbf {\mu _{W}} )^{\rm {H}}]$

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:

$\operatorname {J} _{\mathbf {Z} \mathbf {W} }=\operatorname {cov} (\mathbf {Z} ,{\overline {\mathbf {W} }}){\stackrel {\mathrm {def} }{=}}\ \operatorname {E} [(\mathbf {Z} -\mathbf {\mu _{Z}} )(\mathbf {W} -\mathbf {\mu _{W}} )^{\rm {T}}]$

Uncorrelatedness

Two random vectors $\mathbf {X}$ and $\mathbf {Y}$ are called uncorrelated if their covariance matrix $\operatorname {K} _{\mathbf {X} \mathbf {Y} }$ matrix is zero.

Complex random vectors $\mathbf {Z}$ and $\mathbf {W}$ are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if $\operatorname {K} _{\mathbf {Z} \mathbf {W} }=\operatorname {J} _{\mathbf {Z} \mathbf {W} }=0$.

Cross-covariance of stochastic processes

The definition of cross-covariance of random vector may be generalized to stochastic processes as follows:

Definition

Let $\{X(t)\}$ and $\{Y(t)\}$ denote stochastic processes. Then the cross-covariance function of the processes $K_{XY}$ is defined by:

$\operatorname {K} _{XY}(t_{1},t_{2}){\stackrel {\mathrm {def} }{=}}\ \operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} \left[\left(X(t_{1})-\mu _{X}(t_{1})\right)\left(Y(t_{2})-\mu _{Y}(t_{2})\right)\right]$ (Eq.2)

where $\mu _{X}(t)=\operatorname {E} \left[X(t)\right]$ and $\mu _{Y}(t)=\operatorname {E} \left[Y(t)\right]$.

Definition for jointly WSS processes

If $\left\{X_{t}\right\}$ and $\left\{Y_{t}\right\}$ are a jointly wide-sense stationary, then the following are true:

$\mu _{X}(t_{1})=\mu _{X}(t_{2})\triangleq \mu _{X}$ for all $t_{1},t_{2}$,

$\mu _{Y}(t_{1})=\mu _{Y}(t_{2})\triangleq \mu _{Y}$ for all $t_{1},t_{2}$

and

$\operatorname {K} _{XY}(t_{1},t_{2})=\operatorname {K} _{XY}(t_{2}-t_{1},0)$ for all $t_{1},t_{2}$

By setting $\tau =t_{2}-t_{1}$ (the time lag, or the amount of time by which the signal has been shifted), we may define

$\operatorname {K} _{XY}(\tau )=\operatorname {K} _{XY}(t_{2}-t_{1})\triangleq \operatorname {K} _{XY}(t_{1},t_{2})$.

The autocovariance function of a WSS process is therefore given by:

$\operatorname {K} _{XY}(\tau )=\operatorname {cov} (X_{t},Y_{t-\tau })=\operatorname {E} [(X_{t}-\mu _{X})(Y_{t-\tau }-\mu _{Y})]=\operatorname {E} [X_{t}Y_{t-\tau }]-\mu _{X}\mu _{Y}$ (Eq.3)

which is equivalent to

$\operatorname {K} _{XX}(\tau )=\operatorname {cov} (X_{t+\tau },Y_{t})=\operatorname {E} [(X_{t+\tau }-\mu _{X})(Y_{t}-\mu _{Y})]=\operatorname {E} [X_{t+\tau }X_{t}]-\mu _{X}\mu _{Y}$.

Cross-covariance of deterministic signals

The cross-covariance is also relevant in signal processing where the cross-covariance between two wide-sense stationary random processes can be estimated by averaging the product of samples measured from one process and samples measured from the other (and its time shifts). The samples included in the average can be an arbitrary subset of all the samples in the signal (e.g., samples within a finite time window or a sub-sampling of one of the signals). For a large number of samples, the average converges to the true covariance.

Cross-covariance may also refer to a "deterministic" cross-covariance between two signals. This consists of summing over all time indices. For example, for discrete-time signals $f[k]$ and $g[k]$ the cross-covariance is defined as

$(f\star g)[n]\ {\stackrel {\mathrm {def} }{=}}\ \sum _{k\in \mathbb {Z} }{\overline {f[k]}}g[n+k]=\sum _{k\in \mathbb {Z} }{\overline {f[k-n]}}g[k]$

where the line indicates that the complex conjugate is taken when the signals are complex-valued.

For continuous functions $f(x)$ and $g(x)$ the (deterministic) cross-covariance is defined as

$(f\star g)(x)\ {\stackrel {\mathrm {def} }{=}}\ \int {\overline {f(t)}}g(x+t)\,dt=\int {\overline {f(t-x)}}g(t)\,dt$.

Properties

The (deterministic) cross-covariance of two continuous signals is related to the convolution by

$(f\star g)(t)=({\overline {f(-\tau )}}*g(\tau ))(t)$

and the (deterministic) cross-covariance of two discrete-time signals is related to the discrete convolution by

$(f\star g)[n]=({\overline {f[-k]}}*g[k])[n]$.

See also

• Autocovariance
• Autocorrelation
• Correlation
• Convolution
• Cross-correlation

External links

• Cross Correlation from Mathworld
• http://scribblethink.org/Work/nvisionInterface/nip.html
• http://www.phys.ufl.edu/LIGO/stochastic/sign05.pdf
• http://www.staff.ncl.ac.uk/oliver.hinton/eee305/Chapter6.pdf