Chap - 2 - Discrete-Time and Continuous-Time AWGN Channels PDF
Chap - 2 - Discrete-Time and Continuous-Time AWGN Channels PDF
(f) df.
In particular, [[x(t)[[
2
=[[ x(f)[[
2
.
Asignalspaceisanysubspaceo L
2
. Forexample,thesetofL
2
signalsthataretime-limited
to an interval [0, T] (have support [0, T ]) is a signal space, as is the set of L
2
signals whose
Fourier transforms are nonzero only in B (have frequency support B).
Every signal space o L
2
has an orthogonal basis
k
(t), k 1, where 1 is some discrete
index set, such that every x(t) o may be expressed as
x(t),
k
(t))
x(t) =
[[
k
(t)[[
2
k
(t),
kI
up to L
2
equivalence. This is called an orthogonal expansion of x(t).
Of course this expression becomes particularly simple if
k
(t) is an orthonormal basis with
[[
k
(t)[[
2
= 1 for all k 1. Then we have the orthonormal expansion
x(t) = x
k
k
(t),
kI
wherex=x
k
=x(t),
k
(t)), k 1 isthecorrespondingsetoforthonormalcoecients. From
this expression, we see that inner products are preserved in an orthonormal expansion; i.e.,
x(t), y(t))=x,y)= x
k
y
k
.
kI
In particular, [[x(t)[[
2
=[[x[[
2
.
2.3 The sampling theorem
The sampling theorem allows us to convert a continuous signal x(t) with frequency support
[W,W] (i.e., a baseband signal with bandwidth W) to a discrete-time sequence of samples
x(kT), k Z at a rate of 2W samples per second, with no loss of information.
The sampling theorem is basically an orthogonal expansion for the space L
2
[0, W ] of sig-
nals that have frequency support [W,W]. If T = 1/2W, then the complex exponentials
exp(2ifkT), k ZformanorthogonalbasisforthespaceofFouriertransformswithsupport
[W,W]. ThereforetheirscaledinverseFouriertransforms
k
(t) = sinc
T
(tkT), k Zform
an orthogonal basis for L
2
[0, W], where sinc
T
(t) = (sin t/T)/(t/T). Since [[sinc
T
(t)[[
2
= T,
every x(t) L
2
[0, W] may therefore be expressed up to L
2
equivalence as
1
x(t) = x(t),sinc
T
(tkT))sinc
T
(tkT).
T
kZ
1
Moreover, evaluating this equation at t = jT gives x(jT) =
T
x(t),sinc
T
(tjT)) for all j
Z (provided that x(t) is continuous at t = jT), since sinc
T
((j k)T) = 1 for k = j and
sinc
T
((jk)T) = 0 for k, =j. Thus if x(t) L
2
[0, W ] is continuous, then
x(t) = x(kT)sinc
T
(tkT).
kZ
This is called the sampling theorem.
1
coecients are x
k
=
x(t),sinc
T
(tkT)) = Tx(kT), we have
T
x(t), y(t)) =x,y) =T x(kT)y(kT).
kZ
The following exercise shows similarly how to convert a continuous passband signal x(t) with
bandwidth W (i.e., with frequency support [f
c
W/2, f
c
+W/2] for some center frequency
f
c
> W/2) to a discrete-time sequence of sample pairs (x
c,k
, x
s,k
), k Z at a rate of W pairs
per second, with no loss of information.
Exercise 2.1 (Orthogonal bases for passband signal spaces)
(a) Show that if
k
(t) is an orthogonal set of signals in L
2
[0, W ], then
k
(t)cos2f
c
t,
k
(t) sin 2f
c
t is an orthogonal set of signals in L
2
[f
c
W,f
c
+W], the set
of signals in L
2
that have frequency support [f
c
W,f
c
+W], provided that f
c
W.
[Hint: use the facts that T(
k
(t)cos2f
c
t) = (
k
(f f
c
) +
k
(f + f
c
))/2 and
T(
k
(t) sin 2f
c
t) = (
k
(f f
c
)
k
(f +f
c
))/2i, plus Parsevals theorem.]
(b) Show that if the set
k
(t) is an orthogonal basis for L
2
[0, W ], then the set
k
(t)cos2f
c
t,
k
(t) sin 2f
c
t is an orthogonal basis for L
2
[f
c
W,f
c
+W], provided that
f
c
W.
[Hint: show that every x(t) L
2
[f
c
W,f
c
+W] may be written as x(t) = x
c
(t)cos2f
c
t+
x
s
(t) sin 2f
c
t for some x
c
(t), x
s
(t) L
2
[0, W ].]
(c) Conclude that every x(t) L
2
[f
c
W,f
c
+W] may be expressed up to L
2
equivalence as
1
x(t) = (x
c,k
cos2f
c
t+x
s,k
sin2f
c
t) sinc
T
(tkT), T =
2W
,
kZ
for some sequence of pairs (x
c,k
, x
s,k
), k Z, and give expressions for x
c,k
and x
s,k
.
2.4 White Gaussian noise
The question of how to dene a white Gaussian noise (WGN) process N(t) in general terms is
plaguedwithmathematicaldiculties. However,whenwearegivenasignalspaceo L
2
with
anorthonormal basis ashere, then dening WGNwith respecttoo isnotsoproblematic. The
following denition captures the essential properties that hold in this case:
Denition 2.1 (White Gaussian noise with respect to a signal space o) Let o L
2
be a signal space with an orthonormal basis
k
(t), k 1. A Gaussian process N(t) is de-
ned as white Gaussian noise with respect to o with single-sided power spectral density N
0
if
(a) The sequence N
k
=N(t),
k
(t)), k 1 is a sequence of iid Gaussian noise variables with
mean zero and variance N
0
/2;
(b) Denethein-bandnoiseastheprojectionN
|S
(t) = N
k
k
(t)ofN(t)ontothesignal
kI
spaceo,andtheout-of-bandnoiseasN
|S
(t) = N(t)N
|S
(t). ThenN
|S
(t)is a process
which is jointly Gaussian with N
|S
(t), has sample functions which are orthogonal to o, is
uncorrelated with N
|S
(t), and thus is statistically independent of N
|S
(t).
k
(t) and Y
|S
(t) = Y (t)Y
|S
(t). Since Y (t) = X(t) + N(t) and
k
X(t) = X(t),
k
(t))
k
(t),
k
since all sample functions of X(t) lie in o, we have
Y (t) = Y
k
k
(t) + N
|S
(t),
k
where N
|S
(t) = N(t) N(t),
k
(t))
k
(t). By Denition 2.1, N
|S
(t) is independent of
k
N
|S
(t) = N(t),
k
(t))
k
(t), and by hypothesis it is independent of X(t). Thus the proba-
k
bility distribution of X(t) given Y
|S
(t) = Y
k
k
(t) and Y
|S
(t) = N
|S
(t) depends only on
k
Y
|S
(t), so without loss of optimality in detection of X(t) from Y (t) we can disregard Y
|S
(t);
i.e., Y
|S
(t)isasucientstatistic. Moreover,sinceY
|S
(t)isspeciedbythesamplesY
k
,these
samples equally form a set of sucient statistics for detection of X(t) from Y (t).
The sucient statistic Y
|S
(t) may alternatively be generated by ltering out the out-of-band
noise N
|S
(t). For example, for the signal space L
2
(B) of signals with frequency support B,
we may obtain Y
|S
(t) by passing Y (t) through a brick-wall lter which passes all frequency
1
components in B and rejects all components not in B.
1
Theorem 2.1 may be extended to any model Y(t) = X(t) + N(t) in which the out-of-band noise N
|S
(t) =
N(t)N
|S
(t)isindependentofboththesignalX(t)andthein-bandnoiseN
|S
(t) = N
k
k
(t);e.g., tomodels
k
inwhichtheout-of-bandnoisecontainssignalsfromotherindependentusers. IntheGaussiancase,independence
of the out-of-band noise is automatic; in more general cases, independence is an additional assumption.
16 CHAPTER 2. DISCRETE-TIME AND CONTINUOUS-TIME AWGN CHANNELS
CombiningDenition2.1andTheorem2.1,weconcludethatforanyAWGNchannelinwhich
the signals are conned to a sample space o with orthonormal basis
k
(t), k 1, we may
without loss of optimality reduce the output Y (t) to the set of samples
Y
k
=Y (t),
k
(t))=X(t),
k
(t))+N(t),
k
(t))=X
k
+N
k
, k 1,
whereN
k
, k 1 isasetofiidGaussianvariableswithmeanzeroandvarianceN
0
/2. Moreover,
if x
1
(t), x
2
(t) o are two sample functions of X(t), then this orthonormal expansion preserves
their inner product:
x
1
(t), x
2
(t))=x
1
, x
2
),
where x
1
and x
2
are the orthonormal coecient sequences of x
1
(t) and x
2
(t), respectively.
2.5 Continuous time to discrete time
We now specialize these results to our original AWGN channel model Y (t) = X(t) + N(t),
where the average power of X(t) is limited to P and the sample functions of X(t) are required
tohavepositivefrequencysupportinabandB ofwidthW. Forthetimebeingweconsiderthe
baseband case in which B = [0, W].
The signal space is then the set o = L
2
[0, W] of all nite-energy signals x(t) whose Fourier
transformhassupportB. Thesamplingtheoremshowsthat
k
(t) =
1
sinc
T
(tkT), k Z
T
is an orthonormal basis for this signal space, where T = 1/2W, and that therefore without loss
of generality we may write any x(t) o as
x(t) = x
k
k
(t),
kZ
where x
k
is the orthonormal coecient x
k
= x(t),
k
(t)), and equality is in the sense of L
2
equivalence.
Consequently, if X(t) is a random process whose sample functions x(t) are all in o, then we
can write
X(t) = X
k
k
(t),
kZ
where X
k
= X(t),
k
(t)) = X(t)
k
(t) dt, a random variable that is a linear functional of
X(t). Inthiswaywecanidentifyanyrandomband-limited processX(t)ofbandwidthW with
adiscrete-timerandomsequenceX=X
k
at a rate of 2W realvariablespersecond. Hereafter
the input will be regarded as the sequence X rather than X(t).
Thus X(t) may be regarded as a sum of amplitude-modulated orthonormal pulses X
k
k
(t).
By the Pythagorean theorem,
[[X(t)[[
2
= [[X
k
k
(t)[[
2
= X
k
2
,
kZ kZ
whereweusetheorthonormalityofthe
k
(t). Thereforetherequirementthattheaveragepower
(energy per second) of X(t) be less than P translates to a requirement that the average energy
ofthesequenceXbelessthanP per2W symbols,orequivalentlylessthanP/2W persymbol.
2
2
The requirement that the sample functions of X(t) must be in L2 translates to the requirement that the
sample sequences x of X must have nite energy. This requirement can be met by requiring that only nitely
many elements of x be nonzero. However, we do not pursue such niteness issues.
k
(t) + N
|S
(t)
kZ
where N = N
k
= N(t),
k
(t)) is the sequence of orthonormal coecients of N(t) in o,
and N
|S
(t) = N(t) N
k
k
(t) is out-of-band noise. The theorem of irrelevance shows
k
that N
|S
(t) may be disregarded without loss of optimality, and therefore that the sequence
Y=X+N is a set of sucient statistics for detection of X(t) from Y(t).
Insummary,weconcludethatthecharacteristicsofthediscrete-timemodelY=X+Nmirror
those of the continuous-time model Y(t) = X(t) + N(t) from which it was derived:
The symbol interval is T = 1/2W; equivalently, the symbol rate is 2W symbols/s;
The average signal energy per symbol is limited to P/2W;
The noise sequence N is iid zero-mean (white) Gaussian, with variance N
0
/2 per symbol;
The signal-to-noise ratio is thus SNR = (P/2W)/(N
0
/2) = P/N
0
W, the same as for the
continuous-time model;
A data rate of bits per two dimensions (b/2D) translates to a data rate of R=Wb/s,
or equivalently to a spectral eciency of (b/s)/Hz.
This important conclusion is the fundamental result of this chapter.
2.5.1 Passband case
Suppose now that the channel is instead a passband channel with positive-frequency support
band B = [f
c
W/2, f
c
+W/2] for some center frequency f
c
> W/2.
The signal space is then the set o = L
2
[f
c
W/2, f
c
+W/2] of all nite-energy signals x(t)
whose Fourier transform has support B.
In this case Exercise 2.1 shows that an orthogonal basis for the signal space is a set of signals
of the form
k,c
(t) = sinc
T
(tkT)cos2f
c
t and
k,s
(t) = sinc
T
(tkT) sin 2f
c
t, where the
symbolintervalisnowT = 1/W. SincethesupportoftheFouriertransformofsinc
T
(tkT) is
[W/2, W/2], the support of the transform of each of these signals is B.
The derivation of a discrete-time model then goes as in the baseband case. The result is that
the sequence of real pairs
(Y
k,c
, Y
k,s
) = (X
k,c
, X
k,s
) + (N
k,c
, N
k,s
)
is a set of sucient statistics for detection of X(t) from Y(t). If we compute scale factors
correctly, we nd that the characteristics of this discrete-time model are as follows:
The symbol interval is T = 1/W, or the symbol rate is W symbols/s. In each symbol
intervalapairoftworealsymbolsissentandreceived. Wemaythereforesaythattherate
is 2W = 2/T real dimensions per second, the same as in the baseband model.
The average signal energy per dimension is limited to P/2W;
18 CHAPTER 2. DISCRETE-TIME AND CONTINUOUS-TIME AWGN CHANNELS
The noise sequences N
c
and N
s
are independent real iid zero-mean (white) Gaussian se-
quences, with variance N
0
/2 per dimension;
The signal-to-noise ratio is again SNR=(P/2W)/(N
0
/2)=P/N
0
W;
A data rate of b/2D again translates to a spectral eciency of (b/s)/Hz.
Thus the passband discrete-time model is eectively the same as the baseband model.
Inthepassbandcase,itisoftenconvenienttoidentifyrealpairswithsinglecomplexvariables
k
(t), where X=X
k
is a random sequence and
k
k
(t) = p(t kT) is an orthonormal sequence of time shifts p(t kT) of a basic modulation
pulse p(t) L
2
by integer multiples of a symbol interval T. This is called orthonormal pulse-
amplitude modulation (PAM).
Thesignalspaceo isthenthesubspaceofL
2
spannedbytheorthonormalsequencep(tkT);
i.e., o consists of all signals in L
2
that can be written as linear combinations
k
x
k
p(tkT).
Again,theaveragepowerofX(t) = X
k
p(tkT)willbelimitedtoP iftheaverageenergy
k
of the sequence X is limited to PT per symbol, since the symbol rate is 1/T symbol/s.
The theorem of irrelevance again shows that the set of inner products
Y
k
=Y(t),
k
(t)) =X(t),
k
(t)) +N(t),
k
(t)) =X
k
+N
k
is a set of sucient statistics for detection of X(t) from Y(t). These inner products may be
obtained by ltering Y(t) with a matched lter with impulse response p(t) and sampling at
integer multiples of T as shown in Figure 1 to obtain
Z(kT) = Y()p( kT) d =Y
k
,
Thus again we obtain a discrete-time model Y = X+N, where by the orthonormality of the
p(tkT) the noise sequence N is iid zero-mean Gaussian with variance N
0
/2 per symbol.
-
Orthonormal
X=X
k
X(t) = X
k
p(tkT)
k
PAM
modulator
- -
( ) N t
?
+
Y(t)
p(t)
sample at
t=kT
Y=
-
Y
k
T)) = g((k k
)T).
The fact that (b) (c) follows from the aliasing theorem, which says that the discrete-time
Fourier transform of the sample sequence g(kT) is the aliased response
1
g(f m/T).
T m
20 CHAPTER 2. DISCRETE-TIME AND CONTINUOUS-TIME AWGN CHANNELS
ItisclearfromtheNyquistcriterion(c)thatifp(t)isabasebandsignalofbandwidthW, then
(i) The bandwidth W cannot be less than 1/2T;
(ii) If W = 1/2T, then g(f) = 0; i.e., g(t) = sinc
T
(t); g(f) = T, W f W, else
(iii) If 1/2T < W 1/T, then any real non-negative power spectrum g(f) that satises
g(1/2T +f) + g(1/2T f) = T for 0 f 1/2T will satisfy (c).
For this reason W = 1/2T is called the nominal or Nyquist bandwidth of a PAM system with
symbol interval T. No orthonormal PAM system can have bandwidth less than the Nyquist
bandwidth,andonlyasysteminwhichthemodulationpulsehasautocorrelationfunctiong(t) =
p(t)p(t) = sinc
T
(t)canhaveexactlytheNyquistbandwidth. However,by(iii),whichiscalled
theNyquistband-edgesymmetrycondition,theFouriertransform[p(f)[
2
maybedesignedtoroll
o arbitrarily rapidly for f > W, while being continuous and having a continuous derivative.
Figure 2 illustrates a raised-cosine frequency response that satises the Nyquist band-edge
symmetryconditionwhilebeingcontinuousandhavingacontinuousderivative. Nowadaysitis
no great feat to implement such responses with excess bandwidths of 510% or less.
T
f
2
T [p(
1
f)[
2T
[p(
1
+f)[
2T
0
2
1
T
Figure 2. Raised-cosine spectrum g(f) = [p(f)[
2
with Nyquist band-edge symmetry.
We conclude that an orthonormal PAM system may use arbitrarily small excess bandwidth
beyond the Nyquist bandwidth W = 1/2T, or alternatively that the power in the out-of-band
frequency components may be made to be arbitrarily small, without violating the practical
constraint that the Fourier transform p(f) of the modulation pulse p(t) should be continuous
and have a continuous derivative.
Insummary,ifweletW denotetheNyquistbandwidth1/2T ratherthantheactualbandwidth,
then we again obtain a discrete-time channel model Y = X+N for any orthonormal PAM
system, not just a system with the modulation pulse p(t) =
1
sinc
T
(t), in which:
T
The symbol interval is T = 1/2W; equivalently, the symbol rate is 2W symbols/s;
The average signal energy per symbol is limited to P/2W;
The noise sequence N is iid zero-mean (white) Gaussian, with variance N
0
/2 per symbol;
The signal-to-noise ratio is SNR=(P/2W)/(N
0
/2)=P/N
0
W;
Adatarateofbitspertwodimensions(b/2D)translatestoadatarateofR=/W b/s,
or equivalently to a spectral eciency of (b/s)/Hz.
2
2.7. SUMMARY 21
Exercise 2.4 (Orthonormal QAM modulation)
Figure 3 illustrates an orthonormal quadrature amplitude modulation (QAM) system with
symbol interval T in which the input and output variables X
k
and Y
k
are complex, p(t) is a
complexnite-energymodulationpulsewhosetimeshiftsp(tkT)areorthonormal(theinner
product of two complex signals is x(t), y(t)) = x(t)y
p (t), and f
c
>1/2T is a carrier frequency. The box marked 2' takes twice the real part
of its input i.e., it maps a complex signal f(t) to f(t) + f
X=X
k
X(t) =
k -
-
X
k
p(tkT)
-
2'
QAM
modulator
?
N(t)
+
2ifct
e
@ ?
Hilbert
Y=Y
k
p (t)
sample at
lter
t=kT
Figure 3. Orthonormal QAM system.
(a) Assume that p(f) = 0 for [f[ f
c
. Show that the Hilbert lter is superuous.
(b) Show that Theorem 2.2 holds for a complex response p(t) if we dene the composite
response (autocorrelation function) as g(t) = p(t)p
p (t)resultsinlinearintersymbolinterferencei.e.,intheabsenceofnoiseY
k
=
j
X
j
h
kj
for some discrete-time response h
k
other than the ideal response
k0
(Kronecker delta).
(e)Showthataphaseerrorofinthereceivecarrieri.e.,demodulationbye
2ifct+i
rather
than by e
2ifct
results (in the absence of noise) in a phase rotation by of all outputs Y
k
.
(f) Show that a sample timing error of i.e., sampling at times t = kT + results in
linear intersymbol interference.
2.7 Summary
To summarize, the key parameters of a band-limited continuous-time AWGN channel are its
bandwidth W inHzandits signal-to-noise ratio SNR, regardless ofother details likewhere the
bandwidth is located (in particular whether it is at baseband or passband), the scaling of the
signal, etc. The key parameters of a discrete-time AWGN channel are its symbol rate W in
two-dimensionalrealorone-dimensionalcomplexsymbolspersecondanditsSNR,regardlessof
otherdetailslikewhetheritisrealorcomplex,thescalingofthesymbols,etc. Withorthonormal
PAMorQAM,thesekeyparametersarepreserved,regardlessofwhetherPAMorQAMisused,
the precise modulation pulse, etc. The (nominal) spectral eciency (in (b/s)/Hz or in b/2D)
is also preserved, and (as we will see in the next chapter) so is the channel capacity (in b/s).