Linear Algebra, Vectors, and Matrices

Appendix
A.1 Linear Algebra, Vectors, and Matrices

Vectors provide an efficient way to compute with multidimensional data.
An ordered pair of numbers can be regarded as a point or a vector in
two-dimensional space, as shown in Figure A.1.
Similarly, a point in three-dimensional space is represented by a vector
with three components. A vector with n dimensions is written using the
notation xZ [x1, x2, ., xn]. For example, an input vector with values for n
variables is an n-dimensional vector. A vector can be written in row format
or column format. For example:
x Z ½ 3; 2; 1
is in row format, while

2 3
3
6 7
y Z425
1
is in column format. Because the individual values and their order are the
same for both x and y, y is called the transpose of x, written as y Z xT, or x
is the transpose of y, written as x Z yT.
A.1.1 Addition of Vectors

Vectors can be added or subtracted following the usual way addition or
subtraction is carried out on an element-by-element basis. Consider two
555
q 2006 by Taylor & Francis Group, LLC

556 & Neural Networks for Applied Sciences and Engineering
X2 X2
(2, 4) (2, 4)
X = [2, 4]
X1 X1
Figure A.1 A point in two-dimensional space (left) and the same point viewed as a
vector (right).
vectors, x and y:
x Z ½ 2; 3; 1
y Z½1 6 5
xCy Z½3 9 6
xK y Z ½ 1 K3 K4 :
A.1.2 Multiplication of a Vector by a Scalar

A vector can be multiplied by a scalar (a constant value). For example,
multiplying x Z [3 2 1] by 2 produces a vector twice as long, 2x Z [6 4 2].
A.1.3 The Norm of a Vector

The length (also called norm or magnitude) of a vector denotes the distance
to the point referred to by its coordinates. The norm or magnitude of a
vector x with n components is
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
kxk Z x12 C x22 C /C xn2 :
Therefore, for the vector x Z [3, 2, 1], the norm is

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffi
kxk Z 32 C 22 C 12 Z 14:

Appendix & 557
A.1.4 Vector Multiplication: Dot Products

The dot product provides an efficient method of multiplying two vectors.
The dot product of two vectors xZ[x1, x2, ., xn] and wZ[w1, w2, ., wn] is
x$w Z x1 w1 C x2 w2 C . C xn wn ;
which is the sum of products of vector components. The dot product can
also be written as
x$w Z kxkkwkðcos qÞ;
where q represents the angle between the two vectors. This is presented in
Figure A.2.
Example. Find the dot product of the vectors [K1 3 6 K2] and
[1 2 2 K3]. Find the angle between the two vectors.
Solution. The dot product is
½1 3 6 2$½1 2 2 3

Z 1 !1 C 3 !2 C 6 !2 C 2 !3 Z 23:
The angle between the two vectors is (from the above equation)
23
cos q Z pffiffiffiffiffipffiffiffiffiffi Z 39:948 :
50 18
pffiffiffiffiffi pffiffiffiffiffi
The 50 and 18 in the above expression are norms or the lengths of
the two vectors x and w. When two vectors are perpendicular or
orthogonal to each other, their dot product is zero. This is because when
q Z908, cos q Z0.
X2
x
X1
Figure A.2 Two vectors, x and w, and the angle between them.

A.2 Matrices
Operating with many vectors can be greatly simplified by organizing them
in matrices. A matrix is a rectangular array of numbers. An m ! n matrix is a
matrix with m rows and n columns that represent vectors. Each element of a
matrix can be indexed by its row and column position. The order of
elements is therefore written as
2 3
a11 a12 a13 . a1n
6a a22 a23 a2n 7
6 21 . 7
6 7
A Z6
6 a31 a32 a33 . a3n 7:
7
6 7
4 « 5
am1 am2 am3 . amn
For the 3 ! 2 matrix B, given below, the element b32 ZK5 and b21 Z 2:
2 3
K3 3
6 7
B Z4 2 9 5:
1 K5
A.2.1 Matrix Addition

Matrices can be added by summing their corresponding elements. For
example, if CZ ACB, then cij Z aij C bij. For example, if
2 3 2 3
3 1 K2 1
6 7 6 3 27
A Z42 45 and B Z 4 5;
1 5 1 4
then
2 3 2 3
3K 2 1C1 1 2
6 7 6 7
C ZACB Z42C3 4 C 2 5 Z 4 5 6 5:
1C1 5C4 2 9
A.2.2 Matrix Multiplication

To multiply two matrices, A and B (written as AB), the number of columns in
A must match the number of rows in B. If A is an m ! n matrix and B is an

Appendix & 559
n ! s matrix, then the matrix product AB is an m ! s matrix. If C Z AB,

then an element of this product is defined as
Cij Z ðith row vector of AÞ$ð jth column vector of BÞ
X
n
Z aik bkj :
kZ1
Example. Compute the product

2 3
" # 4 2 1 1
K2 3 1 6 7
CZ 4K3 K1 4 35
1 2 4
1 4 5 1
Solution. The elements c11 and c23 are calculated as

" #
K2 !4 C 3 !K 3 C 1 !1. . .
CZ ;
. .1 !1 C 2 !4 C 4 !5 .
and the complete solution is

" #
K16 K3 15 8
CZ :
2 16 29 11
Using this procedure, any number of matrices can be multiplied together

by sequentially carrying out the computation.
A.2.3 Multiplication of a Matrix by a Vector

A matrix can be multiplied by a vector in a similar manner to the
multiplication of two matrices. For example, for a matrix w of size 2 ! 2 and
a vector x of size 2 ! 1, given as
" # " #
1 2 5
wZ ; xZ ;
3 4 6
multiplication gives a 2 ! 1 vector, u, as

" #" # " # " #
1 2 5 5 C 12 17
uZ Z Z :
3 4 6 15 C 24 39
In this manner, many matrices and vectors can be multiplied together.

A.2.4 Matrix Transpose

Similar to vector transpose, matrices are transposed by arranging original
rows as columns, or vice versa. If matrix B is the transpose of matrix A, then
the element bij is the same as element aji.
Example. Write the matrix B if B Z AT and
" #
K2 3 1
AZ :
1 2 4
Solution.
2 3
K2 1
6 7
B Z AT Z 4 3 2 5
1 4
References
1. Callan, R. The Essence of Neural Networks, Pearson Education, London, UK,
1999.

Linear Algebra, Vectors, and Matrices

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Linear Algebra, Vectors, and Matrices

Uploaded by

Copyright:

Available Formats

Appendix

A.1 Linear Algebra, Vectors, and Matrices

is in row format, while

A.1.1 Addition of Vectors

q 2006 by Taylor & Francis Group, LLC

A.1.2 Multiplication of a Vector by a Scalar

A.1.3 The Norm of a Vector

Therefore, for the vector x Z [3, 2, 1], the norm is

q 2006 by Taylor & Francis Group, LLC

A.1.4 Vector Multiplication: Dot Products

x$w Z kxkkwkðcos qÞ;

½1 3 6 2$½1 2 2 3

q 2006 by Taylor & Francis Group, LLC

A.2.1 Matrix Addition

A.2.2 Matrix Multiplication

q 2006 by Taylor & Francis Group, LLC

n ! s matrix, then the matrix product AB is an m ! s matrix. If C Z AB,

Cij Z ðith row vector of AÞ$ð jth column vector of BÞ

Example. Compute the product

Solution. The elements c11 and c23 are calculated as

and the complete solution is

Using this procedure, any number of matrices can be multiplied together

A.2.3 Multiplication of a Matrix by a Vector

multiplication gives a 2 ! 1 vector, u, as

In this manner, many matrices and vectors can be multiplied together.

q 2006 by Taylor & Francis Group, LLC

A.2.4 Matrix Transpose

q 2006 by Taylor & Francis Group, LLC

You might also like