7_01_3A_Eigenvalues_and_Eigenvectors_of_a_Matrix

> 来源: LibreTexts

> 原页: https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/07%3A_Spectral_Theory/7.01%3A_Eigenvalues_and_Eigenvectors_of_a_Matrix

Skip to main content

##### Outcomes

1. Describe eigenvalues geometrically and algebraically.

2. Find eigenvalues and eigenvectors for a square matrix.

Spectral Theory refers to the study of eigenvalues and eigenvectors of a matrix. It is of fundamental importance in many areas and is the subject of our study for this chapter.

Definition of Eigenvectors and Eigenvalues

In this section, we will work with the entire set of complex numbers, denoted by \\\mathbb{C}\\. Recall that the real numbers, \\\mathbb{R}\\ are contained in the complex numbers, so the discussions in this section apply to both real and complex numbers.

To illustrate the idea behind what will be discussed, consider the following example.

##### Example \\\PageIndex{1}\\: Eigenvectors and Eigenvalues

Let \\A = \left$$ \begin{array}{rrr} 0 & 5 & -10 \\ 0 & 22 & 16 \\ 0 & -9 & -2 \end{array} \right$$\nonumber\\ Compute the product \\AX\\ for \\X = \left$$ \begin{array}{r} -5 \\ -4 \\ 3 \end{array} \right$$, X = \left$$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right$$\nonumber\\ What do you notice about \\AX\\ in each of these products?

###### Solution

First, compute \\AX\\ for \\X =\left$$ \begin{array}{r} -5 \\ -4 \\ 3 \end{array} \right$$\nonumber\\

This product is given by \\AX = \left$$ \begin{array}{rrr} 0 & 5 & -10 \\ 0 & 22 & 16 \\ 0 & -9 & -2 \end{array} \right$$ \left$$ \begin{array}{r} -5 \\ -4 \\ 3 \end{array} \right$$ = \left$$ \begin{array}{r} -50 \\ -40 \\ 30 \end{array} \right$$ =10\left$$ \begin{array}{r} -5 \\ -4 \\ 3 \end{array} \right$$\nonumber\\

In this case, the product \\AX\\ resulted in a vector which is equal to \\10\\ times the vector \\X\\. In other words, \\AX=10X\\.

Let’s see what happens in the next product. Compute \\AX\\ for the vector \\X = \left$$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right$$\nonumber\\

This product is given by \\AX = \left$$ \begin{array}{rrr} 0 & 5 & -10 \\ 0 & 22 & 16 \\ 0 & -9 & -2 \end{array} \right$$ \left$$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right$$ = \left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$ =0\left$$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right$$\nonumber\\

In this case, the product \\AX\\ resulted in a vector equal to \\0\\ times the vector \\X\\, \\AX=0X\\.

Perhaps this matrix is such that \\AX\\ results in \\kX\\, for every vector \\X\\. However, consider \\\left$$ \begin{array}{rrr} 0 & 5 & -10 \\ 0 & 22 & 16 \\ 0 & -9 & -2 \end{array} \right$$ \left$$ \begin{array}{r} 1 \\ 1 \\ 1 \end{array} \right$$ = \left$$ \begin{array}{r} -5 \\ 38 \\ -11 \end{array} \right$$\nonumber \\ In this case, \\AX\\ did not result in a vector of the form \\kX\\ for some scalar \\k\\.

There is something special about the first two products calculated in Example $\PageIndex{1}$. Notice that for each, \\AX=kX\\ where \\k\\ is some scalar. When this equation holds for some \\X\\ and \\k\\, we call the scalar \\k\\ an eigenvalue of \\A\\. We often use the special symbol \\\lambda\\ instead of \\k\\ when referring to eigenvalues. In Example $\PageIndex{1}$, the values \\10\\ and \\0\\ are eigenvalues for the matrix \\A\\ and we can label these as \\\lambda_1 = 10\\ and \\\lambda_2 = 0\\.

When \\AX = \lambda X\\ for some \\X \neq 0\\, we call such an \\X\\ an eigenvector of the matrix \\A\\. The eigenvectors of \\A\\ are associated to an eigenvalue. Hence, if \\\lambda_1\\ is an eigenvalue of \\A\\ and \\AX = \lambda_1 X\\, we can label this eigenvector as \\X_1\\. Note again that in order to be an eigenvector, \\X\\ must be nonzero.

There is also a geometric significance to eigenvectors. When you have a nonzero vector which, when multiplied by a matrix results in another vector which is parallel to the first or equal to 0, this vector is called an eigenvector of the matrix. This is the meaning when the vectors are in \\\mathbb{R}^{n}.\\

The formal definition of eigenvalues and eigenvectors is as follows.

##### Definition \\\PageIndex{1}\\: Eigenvalues and Eigenvectors

Let \\A\\ be an \\n\times n\\ matrix and let \\X \in \mathbb{C}^{n}\\ be a nonzero vector for which

\\AX=\lambda X \label{eigen1}\\ for some scalar \\\lambda .\\ Then \\\lambda\\ is called an eigenvalue of the matrix \\A\\ and \\X\\ is called an eigenvector of \\A\\ associated with \\\lambda\\, or a \\\lambda\\-eigenvector of \\A\\.

The set of all eigenvalues of an \\n\times n\\ matrix \\A\\ is denoted by \\\sigma \left( A\right)\\ and is referred to as the spectrum of \\A.\\

The eigenvectors of a matrix \\A\\ are those vectors \\X\\ for which multiplication by \\A\\ results in a vector in the same direction or opposite direction to \\X\\. Since the zero vector \\0\\ has no direction this would make no sense for the zero vector. As noted above, \\0\\ is never allowed to be an eigenvector.

Let’s look at eigenvectors in more detail. Suppose \\X\\ satisfies \\\eqref{eigen1}\\. Then \\\begin{array}{c} AX - \lambda X = 0 \\ \mbox{or} \\ \left( A-\lambda I\right) X = 0 \end{array}\nonumber\\ for some \\X \neq 0.\\ Equivalently you could write \\\left( \lambda I-A\right)X = 0\\, which is more commonly used. Hence, when we are looking for eigenvectors, we are looking for nontrivial solutions to this homogeneous system of equations!

Recall that the solutions to a homogeneous system of equations consist of basic solutions, and the linear combinations of those basic solutions. In this context, we call the basic solutions of the equation \\\left( \lambda I - A\right) X = 0\\ basic eigenvectors. It follows that any (nonzero) linear combination of basic eigenvectors is again an eigenvector.

Suppose the matrix \\\left(\lambda I - A\right)\\ is invertible, so that \\\left(\lambda I - A\right)^{-1}\\ exists. Then the following equation would be true. \\\begin{aligned} X &= IX \\ &= \left( \left( \lambda I - A\right) ^{-1}\left(\lambda I - A \right) \right) X \\ &=\left( \lambda I - A\right) ^{-1}\left( \left( \lambda I - A\right) X\right) \\ &= \left( \lambda I - A\right) ^{-1}0 \\ &= 0\end{aligned}\\ This claims that \\X=0\\. However, we have required that \\X \neq 0\\. Therefore \\\left(\lambda I - A\right)\\ cannot have an inverse!

Recall that if a matrix is not invertible, then its determinant is equal to \\0\\. Therefore we can conclude that \\\det \left( \lambda I - A\right) =0 \label{eigen2}\\ Note that this is equivalent to \\\det \left(A- \lambda I \right) =0\\.

The expression \\\det \left( \lambda I-A\right)\\ is a polynomial (in the variable \\x\$ called the characteristic polynomial of \\A\\, and \\\det \left( \lambda I-A\right) =0\\ is called the characteristic equation. For this reason we may also refer to the eigenvalues of \\A\\ as characteristic values, but the former is often used for historical reasons.

The following theorem claims that the roots of the characteristic polynomial are the eigenvalues of \\A\\. Thus when [eigen2] holds, \\A\\ has a nonzero eigenvector.

##### Theorem \\\PageIndex{1}\\: The Existence of an Eigenvector

Let \\A\\ be an \\n\times n\\ matrix and suppose \\\det \left( \lambda I - A\right) =0\\ for some \\\lambda \in \mathbb{C}\\.

Then \\\lambda\\ is an eigenvalue of \\A\\ and thus there exists a nonzero vector \\X \in \mathbb{C}^{n}\\ such that \\AX=\lambda X\\.

Proof

For \\A\\ an \\n\times n\\ matrix, the method of Laplace Expansion demonstrates that \\\det \left( \lambda I - A \right)\\ is a polynomial of degree \\n.\\ As such, the equation \\\eqref{eigen2}\\ has a solution \\\lambda \in \mathbb{C}\\ by the Fundamental Theorem of Algebra. The fact that \\\lambda\\ is an eigenvalue is left as an exercise.

Finding Eigenvectors and Eigenvalues

Now that eigenvalues and eigenvectors have been defined, we will study how to find them for a matrix \\A\\.

First, consider the following definition.

##### Definition \\\PageIndex{2}\\: Multiplicity of an Eigenvalue

Let \\A\\ be an \\n \times n\\ matrix with characteristic polynomial given by \\\det \left( \lambda I - A\right)\\. Then, the multiplicity of an eigenvalue \\\lambda\\ of \\A\\ is the number of times \\\lambda\\ occurs as a root of that characteristic polynomial.

For example, suppose the characteristic polynomial of \\A\\ is given by \\\left( \lambda - 2 \right)^2\\. Solving for the roots of this polynomial, we set \\\left( \lambda - 2 \right)^2 = 0\\ and solve for \\\lambda \\. We find that \\\lambda = 2\\ is a root that occurs twice. Hence, in this case, \\\lambda = 2\\ is an eigenvalue of \\A\\ of multiplicity equal to \\2\\.

We will now look at how to find the eigenvalues and eigenvectors for a matrix \\A\\ in detail. The steps used are summarized in the following procedure.

##### Procedure \\\PageIndex{1}\\: Finding Eigenvalues and Eigenvectors

Let \\A\\ be an \\n \times n\\ matrix.

1. First, find the eigenvalues \\\lambda\\ of \\A\\ by solving the equation \\\det \left( \lambda I -A \right) = 0\\.

2. For each \\\lambda\\, find the basic eigenvectors \\X \neq 0\\ by finding the basic solutions to \\\left( \lambda I - A \right) X = 0\\.

To verify your work, make sure that \\AX=\lambda X\\ for each \\\lambda\\ and associated eigenvector \\X\\.

We will explore these steps further in the following example.

##### Example \\\PageIndex{2}\\: Find the Eigenvalues and Eigenvectors

Let \\A = \left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array} \right$$\\. Find its eigenvalues and eigenvectors.

###### Solution

We will use Procedure $\PageIndex{1}$. First we find the eigenvalues of \\A\\ by solving the equation \\\det \left( \lambda I - A \right) =0\nonumber \\

This gives \\\begin{aligned} \det \left( \lambda \left$$ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array} \right$$ - \left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array} \right$$ \right) &= 0 \\ \\ \det \left$$ \begin{array}{cc} \lambda +5 & -2 \\ 7 & \lambda -4 \end{array} \right$$ &= 0 \end{aligned}\\

Computing the determinant as usual, the result is \\\lambda ^2 + \lambda - 6 = 0\nonumber\\

Solving this equation, we find that \\\lambda_1 = 2\\ and \\\lambda_2 = -3\\.

Now we need to find the basic eigenvectors for each \\\lambda\\. First we will find the eigenvectors for \\\lambda_1 = 2\\. We wish to find all vectors \\X \neq 0\\ such that \\AX = 2X\\. These are the solutions to \$2I - A)X = 0\\. \\\begin{aligned} \left( 2 \left$$ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right$$ - \left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array}\right$$ \right) \left$$ \begin{array}{c} x \\ y \end{array}\right$$ &= \left$$ \begin{array}{r} 0 \\ 0 \end{array} \right$$ \\ \\ \left$$ \begin{array}{rr} 7 & -2 \\ 7 & -2 \end{array}\right$$ \left$$ \begin{array}{c} x \\ y \end{array}\right$$ &= \left$$ \begin{array}{r} 0 \\ 0 \end{array} \right$$ \end{aligned}\\

The augmented matrix for this system and corresponding reduced row-echelon form are given by \\\left$$ \begin{array}{rr\|r} 7 & -2 & 0 \\ 7 & -2 & 0 \end{array}\right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rr\|r} 1 & -\frac{2}{7} & 0 \\ 0 & 0 & 0 \end{array} \right$$\nonumber\\

The solution is any vector of the form \\\left$$ \begin{array}{c} \frac{2}{7}s \\ s \end{array} \right$$ = s \left$$ \begin{array}{r} \frac{2}{7} \\ 1 \end{array} \right$$\nonumber \\

Multiplying this vector by \\7\\ we obtain a simpler description for the solution to this system, given by \\t \left$$ \begin{array}{r} 2 \\ 7 \end{array} \right$$\nonumber\\

This gives the basic eigenvector for \\\lambda_1 = 2\\ as \\\left$$ \begin{array}{r} 2\\ 7 \end{array} \right$$\nonumber\\

To check, we verify that \\AX = 2X\\ for this basic eigenvector.

\\\left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array}\right$$ \left$$ \begin{array}{r} 2 \\ 7 \end{array} \right$$ = \left$$ \begin{array}{r} 4 \\ 14 \end{array}\right$$ = 2 \left$$ \begin{array}{r} 2\\ 7 \end{array} \right$$\nonumber\\

This is what we wanted, so we know this basic eigenvector is correct.

Next we will repeat this process to find the basic eigenvector for \\\lambda_2 = -3\\. We wish to find all vectors \\X \neq 0\\ such that \\AX = -3X\\. These are the solutions to \$(-3)I-A)X = 0\\. \\\begin{aligned} \left( (-3) \left$$ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right$$ - \left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array}\right$$ \right) \left$$ \begin{array}{c} x \\ y \end{array}\right$$ &= \left$$ \begin{array}{r} 0 \\ 0 \end{array} \right$$ \\ \left$$ \begin{array}{rr} 2 & -2 \\ 7 & -7 \end{array}\right$$ \left$$ \begin{array}{c} x \\ y \end{array}\right$$ &= \left$$ \begin{array}{r} 0 \\ 0 \end{array} \right$$ \end{aligned}\\

The augmented matrix for this system and corresponding reduced row-echelon form are given by \\\left$$ \begin{array}{rr\|r} 2 & -2 & 0 \\ 7 & -7 & 0 \end{array}\right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rr\|r} 1 & -1 & 0 \\ 0 & 0 & 0 \end{array} \right$$\nonumber \\

The solution is any vector of the form \\\left$$ \begin{array}{c} s \\ s \end{array} \right$$ = s \left$$ \begin{array}{r} 1 \\ 1 \end{array} \right$$\nonumber\\

This gives the basic eigenvector for \\\lambda_2 = -3\\ as \\\left$$ \begin{array}{r} 1\\ 1 \end{array} \right$$\nonumber\\

To check, we verify that \\AX = -3X\\ for this basic eigenvector.

\\\left$$ \begin{array}{rr} -5 & 2 \\ -7 & 4 \end{array}\right$$ \left$$ \begin{array}{r} 1 \\ 1 \end{array} \right$$ = \left$$ \begin{array}{r} -3 \\ -3 \end{array}\right$$ = -3 \left$$ \begin{array}{r} 1\\ 1 \end{array} \right$$\nonumber\\

This is what we wanted, so we know this basic eigenvector is correct.

The following is an example using Procedure $\PageIndex{1}$ for a \\3 \times 3\\ matrix.

##### Example \\\PageIndex{3}\\: Find the Eigenvalues and Eigencectors

Find the eigenvalues and eigenvectors for the matrix \\A=\left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$\nonumber \\

###### Solution

We will use Procedure $\PageIndex{1}$. First we need to find the eigenvalues of \\A\\. Recall that they are the solutions of the equation \\\det \left( \lambda I - A \right) =0\nonumber \\

In this case the equation is \\\det \left( \lambda \left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ - \left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$ \right) =0\nonumber \\ which becomes \\\det \left$$ \begin{array}{ccc} \lambda - 5 & 10 & 5 \\ -2 & \lambda - 14 & -2 \\ 4 & 8 & \lambda - 6 \end{array} \right$$ = 0\nonumber \\

Using Laplace Expansion, compute this determinant and simplify. The result is the following equation. \\\left( \lambda -5\right) \left( \lambda ^{2}-20\lambda +100\right) =0\nonumber \\

Solving this equation, we find that the eigenvalues are \\\lambda_1 = 5, \lambda_2=10\\ and \\\lambda_3=10\\. Notice that \\10\\ is a root of multiplicity two due to \\\lambda ^{2}-20\lambda +100=\left( \lambda -10\right) ^{2}\nonumber \\ Therefore, \\\lambda_2 = 10\\ is an eigenvalue of multiplicity two.

Now that we have found the eigenvalues for \\A\\, we can compute the eigenvectors.

First we will find the basic eigenvectors for \\\lambda_1 =5.\\ In other words, we want to find all non-zero vectors \\X\\ so that \\AX = 5X\\. This requires that we solve the equation \\\left( 5 I - A \right) X = 0\\ for \\X\\ as follows. \\\left( 5\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ - \left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$ \right) \left$$ \begin{array}{r} x \\ y \\ z \end{array} \right$$ =\left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$\nonumber \\

That is you need to find the solution to \\ \left$$ \begin{array}{rrr} 0 & 10 & 5 \\ -2 & -9 & -2 \\ 4 & 8 & -1 \end{array} \right$$ \left$$ \begin{array}{r} x \\ y \\ z \end{array} \right$$ =\left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$\nonumber \\

By now this is a familiar problem. You set up the augmented matrix and row reduce to get the solution. Thus the matrix you must row reduce is \\\left$$ \begin{array}{rrr\|r} 0 & 10 & 5 & 0 \\ -2 & -9 & -2 & 0 \\ 4 & 8 & -1 & 0 \end{array} \right$$\nonumber \\ The reduced row-echelon form is \\\left$$ \begin{array}{rrr\|r} 1 & 0 & - \frac{5}{4} & 0 \\ 0 & 1 & \frac{1}{2} & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ and so the solution is any vector of the form \\\left$$ \begin{array}{c} \frac{5}{4}s \\ -\frac{1}{2}s \\ s \end{array} \right$$ =s\left$$ \begin{array}{r} \frac{5}{4} \\ -\frac{1}{2} \\ 1 \end{array} \right$$\nonumber \\ where \\s\in \mathbb{R}\\. If we multiply this vector by \\4\\, we obtain a simpler description for the solution to this system, as given by \\t \left$$ \begin{array}{r} 5 \\ -2 \\ 4 \end{array} \right$$ \label{basiceigenvect}\\ where \\t\in \mathbb{R}\\. Here, the basic eigenvector is given by \\X_1 = \left$$ \begin{array}{r} 5 \\ -2 \\ 4 \end{array} \right$$\nonumber \\

Notice that we cannot let \\t=0\\ here, because this would result in the zero vector and eigenvectors are never equal to 0! Other than this value, every other choice of \\t\\ in \\\eqref{basiceigenvect}\\ results in an eigenvector.

It is a good idea to check your work! To do so, we will take the original matrix and multiply by the basic eigenvector \\X_1\\. We check to see if we get \\5X_1\\. \\\left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$ \left$$ \begin{array}{r} 5 \\ -2 \\ 4 \end{array} \right$$ = \left$$ \begin{array}{r} 25 \\ -10 \\ 20 \end{array} \right$$ =5\left$$ \begin{array}{r} 5 \\ -2 \\ 4 \end{array} \right$$\nonumber \\ This is what we wanted, so we know that our calculations were correct.

Next we will find the basic eigenvectors for \\\lambda_2, \lambda_3=10.\\ These vectors are the basic solutions to the equation, \\\left( 10\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ - \left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$ \right) \left$$ \begin{array}{r} x \\ y \\ z \end{array} \right$$ =\left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$\nonumber \\ That is you must find the solutions to \\\left$$ \begin{array}{rrr} 5 & 10 & 5 \\ -2 & -4 & -2 \\ 4 & 8 & 4 \end{array} \right$$ \left$$ \begin{array}{c} x \\ y \\ z \end{array} \right$$ =\left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$\nonumber \\

Consider the augmented matrix \\\left$$ \begin{array}{rrr\|r} 5 & 10 & 5 & 0 \\ -2 & -4 & -2 & 0 \\ 4 & 8 & 4 & 0 \end{array} \right$$\nonumber \\ The reduced row-echelon form for this matrix is \\\left$$ \begin{array}{rrr\|r} 1 & 2 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ and so the eigenvectors are of the form \\\left$$ \begin{array}{c} -2s-t \\ s \\ t \end{array} \right$$ =s\left$$ \begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right$$ +t\left$$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\ Note that you can’t pick \\t\\ and \\s\\ both equal to zero because this would result in the zero vector and eigenvectors are never equal to zero.

Here, there are two basic eigenvectors, given by \\X_2 = \left$$ \begin{array}{r} -2 \\ 1\\ 0 \end{array} \right$$ , X_3 = \left$$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\

Taking any (nonzero) linear combination of \\X_2\\ and \\X_3\\ will also result in an eigenvector for the eigenvalue \\\lambda =10.\\ As in the case for \\\lambda =5\\, always check your work! For the first basic eigenvector, we can check \\AX_2 = 10 X_2\\ as follows. \\\left$$ \begin{array}{rrr} 5 & -10 & -5 \\ 2 & 14 & 2 \\ -4 & -8 & 6 \end{array} \right$$ \left$$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right$$ = \left$$ \begin{array}{r} -10 \\ 0 \\ 10 \end{array} \right$$ =10\left$$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\ This is what we wanted. Checking the second basic eigenvector, \\X_3\\, is left as an exercise.

It is important to remember that for any eigenvector \\X\\, \\X \neq 0\\. However, it is possible to have eigenvalues equal to zero. This is illustrated in the following example.

##### Example \\\PageIndex{4}\\: A Zero Eigenvalue

Let \\A=\left$$ \begin{array}{rrr} 2 & 2 & -2 \\ 1 & 3 & -1 \\ -1 & 1 & 1 \end{array} \right$$\nonumber \\ Find the eigenvalues and eigenvectors of \\A\\.

###### Solution

First we find the eigenvalues of \\A\\. We will do so using Definition $\PageIndex{1}$.

In order to find the eigenvalues of \\A\\, we solve the following equation. \\\det \left(\lambda I -A \right) = \det \left$$ \begin{array}{ccc} \lambda -2 & -2 & 2 \\ -1 & \lambda - 3 & 1 \\ 1 & -1 & \lambda -1 \end{array} \right$$ =0\nonumber \\

This reduces to \\\lambda ^{3}-6 \lambda ^{2}+8\lambda =0\\. You can verify that the solutions are \\\lambda_1 = 0, \lambda_2 = 2, \lambda_3 = 4\\. Notice that while eigenvectors can never equal \\0\\, it is possible to have an eigenvalue equal to \\0\\.

Now we will find the basic eigenvectors. For \\\lambda_1 =0\\, we need to solve the equation \\\left( 0 I - A \right) X = 0\\. This equation becomes \\-AX=0\\, and so the augmented matrix for finding the solutions is given by \\\left$$ \begin{array}{rrr\|r} -2 & -2 & 2 & 0 \\ -1 & -3 & 1 & 0 \\ 1 & -1 & -1 & 0 \end{array} \right$$\nonumber \\ The reduced row-echelon form is \\\left$$ \begin{array}{rrr\|r} 1 & 0 & -1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ Therefore, the eigenvectors are of the form \\t\left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$\\ where \\t\neq 0\\ and the basic eigenvector is given by \\X_1 = \left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\

We can verify that this eigenvector is correct by checking that the equation \\AX_1 = 0 X_1\\ holds. The product \\AX_1\\ is given by \\AX_1=\left$$ \begin{array}{rrr} 2 & 2 & -2 \\ 1 & 3 & -1 \\ -1 & 1 & 1 \end{array} \right$$ \left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$ = \left$$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right$$\nonumber \\

This clearly equals \\0X_1\\, so the equation holds. Hence, \\AX_1 = 0X_1\\ and so \\0\\ is an eigenvalue of \\A\\.

Computing the other basic eigenvectors is left as an exercise.

In the following sections, we examine ways to simplify this process of finding eigenvalues and eigenvectors by using properties of special types of matrices.

Eigenvalues and Eigenvectors for Special Types of Matrices

There are three special kinds of matrices which we can use to simplify the process of finding eigenvalues and eigenvectors. Throughout this section, we will discuss similar matrices, elementary matrices, as well as triangular matrices.

We begin with a definition.

##### Definition \\\PageIndex{3}\\: Similar Matrices

Let \\A\\ and \\B\\ be \\n \times n\\ matrices. Suppose there exists an invertible matrix \\P\\ such that \\A = P^{-1}BP\nonumber \\ Then \\A\\ and \\B\\ are called similar matrices.

It turns out that we can use the concept of similar matrices to help us find the eigenvalues of matrices. Consider the following lemma.

##### Lemma \\\PageIndex{1}\\: Similar Matrices and Eigenvalues

Let \\A\\ and \\B\\ be similar matrices, so that \\A=P^{-1}BP\\ where \\A,B\\ are \\n\times n\\ matrices and \\P\\ is invertible. Then \\A,B\\ have the same eigenvalues.

Proof

We need to show two things. First, we need to show that if \\A=P^{-1}BP\\, then \\A\\ and \\B\\ have the same eigenvalues. Secondly, we show that if \\A\\ and \\B\\ have the same eigenvalues, then \\A=P^{-1}BP\\.

Here is the proof of the first statement. Suppose \\A = P^{-1}BP\\ and \\\lambda\\ is an eigenvalue of \\A\\, that is \\AX=\lambda X\\ for some \\X\neq 0.\\ Then \\P^{-1}BPX=\lambda X\nonumber \\ and so \\BPX=\lambda PX\nonumber \\

Since \\P\\ is one to one and \\X \neq 0\\, it follows that \\PX \neq 0\\. Here, \\PX\\ plays the role of the eigenvector in this equation. Thus \\\lambda\\ is also an eigenvalue of \\B\\. One can similarly verify that any eigenvalue of \\B\\ is also an eigenvalue of \\A\\, and thus both matrices have the same eigenvalues as desired.

Proving the second statement is similar and is left as an exercise.

Note that this proof also demonstrates that the eigenvectors of \\A\\ and \\B\\ will (generally) be *different*. We see in the proof that \\AX = \lambda X\\, while \\B \left(PX\right)=\lambda \left(PX\right)\\. Therefore, for an eigenvalue \\\lambda\\, \\A\\ will have the eigenvector \\X\\ while \\B\\ will have the eigenvector \\PX\\.

The second special type of matrices we discuss in this section is elementary matrices. Recall from Definition 2.8.1 that an elementary matrix \\E\\ is obtained by applying one row operation to the identity matrix.

It is possible to use elementary matrices to simplify a matrix before searching for its eigenvalues and eigenvectors. This is illustrated in the following example.

##### Example \\\PageIndex{5}\\: Simplify Using Elementary Matrices

Find the eigenvalues for the matrix \\A = \left$$ \begin{array}{rrr} 33 & 105 & 105 \\ 10 & 28 & 30 \\ -20 & -60 & -62 \end{array} \right$$\nonumber \\

###### Solution

This matrix has big numbers and therefore we would like to simplify as much as possible before computing the eigenvalues.

We will do so using row operations. First, add \\2\\ times the second row to the third row. To do so, left multiply \\A\\ by \\E \left(2,2\right)\\. Then right multiply \\A\\ by the inverse of \\E \left(2,2\right)\\ as illustrated. \\\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array} \right$$ \left$$ \begin{array}{rrr} 33 & 105 & 105 \\ 10 & 28 & 30 \\ -20 & -60 & -62 \end{array} \right$$ \left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{array} \right$$ =\left$$ \begin{array}{rrr} 33 & -105 & 105 \\ 10 & -32 & 30 \\ 0 & 0 & -2 \end{array} \right$$\nonumber \\ By Lemma $\PageIndex{1}$, the resulting matrix has the same eigenvalues as \\A\\ where here, the matrix \\E \left(2,2\right)\\ plays the role of \\P\\.

We do this step again, as follows. In this step, we use the elementary matrix obtained by adding \\-3\\ times the second row to the first row. \\\left$$ \begin{array}{rrr} 1 & -3 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ \left$$ \begin{array}{rrr} 33 & -105 & 105 \\ 10 & -32 & 30 \\ 0 & 0 & -2 \end{array} \right$$ \left$$ \begin{array}{rrr} 1 & 3 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ =\left$$ \begin{array}{rrr} 3 & 0 & 15 \\ 10 & -2 & 30 \\ 0 & 0 & -2 \end{array} \right$$ \label{elemeigenvalue}\\ Again by Lemma $\PageIndex{1}$, this resulting matrix has the same eigenvalues as \\A\\. At this point, we can easily find the eigenvalues. Let \\B = \left$$ \begin{array}{rrr} 3 & 0 & 15 \\ 10 & -2 & 30 \\ 0 & 0 & -2 \end{array} \right$$\nonumber \\ Then, we find the eigenvalues of \\B\\ (and therefore of \\A\$ by solving the equation \\\det \left( \lambda I - B \right) = 0\\. You should verify that this equation becomes \\\left(\lambda +2 \right) \left( \lambda +2 \right) \left( \lambda - 3 \right) =0\nonumber \\ Solving this equation results in eigenvalues of \\\lambda_1 = -2, \lambda_2 = -2\\, and \\\lambda_3 = 3\\. Therefore, these are also the eigenvalues of \\A\\.

Through using elementary matrices, we were able to create a matrix for which finding the eigenvalues was easier than for \\A\\. At this point, you could go back to the original matrix \\A\\ and solve \\\left( \lambda I - A \right) X = 0\\ to obtain the eigenvectors of \\A\\.

Notice that when you multiply on the right by an elementary matrix, you are doing the column operation defined by the elementary matrix. In \\\eqref{elemeigenvalue}\\ multiplication by the elementary matrix on the right merely involves taking three times the first column and adding to the second. Thus, without referring to the elementary matrices, the transition to the new matrix in \\\eqref{elemeigenvalue}\\ can be illustrated by \\\left$$ \begin{array}{rrr} 33 & -105 & 105 \\ 10 & -32 & 30 \\ 0 & 0 & -2 \end{array} \right$$ \rightarrow \left$$ \begin{array}{rrr} 3 & -9 & 15 \\ 10 & -32 & 30 \\ 0 & 0 & -2 \end{array} \right$$ \rightarrow \left$$ \begin{array}{rrr} 3 & 0 & 15 \\ 10 & -2 & 30 \\ 0 & 0 & -2 \end{array} \right$$\nonumber \\

The third special type of matrix we will consider in this section is the triangular matrix. Recall Definition 3.1.6 which states that an upper (lower) triangular matrix contains all zeros below (above) the main diagonal. Remember that finding the determinant of a triangular matrix is a simple procedure of taking the product of the entries on the main diagonal.. It turns out that there is also a simple way to find the eigenvalues of a triangular matrix.

In the next example we will demonstrate that the eigenvalues of a triangular matrix are the entries on the main diagonal.

##### Example \\\PageIndex{6}\\: Eigenvalues for a Triangular Matrix

Let \\A=\left$$ \begin{array}{rrr} 1 & 2 & 4 \\ 0 & 4 & 7 \\ 0 & 0 & 6 \end{array} \right$$ .\\ Find the eigenvalues of \\A\\.

###### Solution

We need to solve the equation \\\det \left( \lambda I - A \right) = 0\\ as follows \\\begin{aligned} \det \left( \lambda I - A \right) = \det \left$$ \begin{array}{ccc} \lambda -1 & -2 & -4 \\ 0 & \lambda -4 & -7 \\ 0 & 0 & \lambda -6 \end{array} \right$$ =\left( \lambda -1 \right) \left( \lambda -4 \right) \left( \lambda -6 \right) =0\end{aligned}\\

Solving the equation \\\left( \lambda -1 \right) \left( \lambda -4 \right) \left( \lambda -6 \right) = 0\\ for \\\lambda \\ results in the eigenvalues \\\lambda_1 = 1, \lambda_2 = 4\\ and \\\lambda_3 = 6\\. Thus the eigenvalues are the entries on the main diagonal of the original matrix.

The same result is true for lower triangular matrices. For any triangular matrix, the eigenvalues are equal to the entries on the main diagonal. To find the eigenvectors of a triangular matrix, we use the usual procedure.

In the next section, we explore an important process involving the eigenvalues and eigenvectors of a matrix.

---

7_02_3A_Diagonalization

> 来源: LibreTexts

> 原页: https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/07%3A_Spectral_Theory/7.02%3A_Diagonalization

Skip to main content

##### Outcomes

1. Determine when it is possible to diagonalize a matrix.

2. When possible, diagonalize a matrix.

Similarity and Diagonalization

We begin this section by recalling the definition of similar matrices. Recall that if \\A,B\\ are two \\n\times n\\ matrices, then they are similar if and only if there exists an invertible matrix \\P\\ such that \\A=P^{-1}BP\nonumber \\

In this case we write \\A \sim B\\. The concept of similarity is an example of an equivalence relation.

##### Lemma \\\PageIndex{1}\\: Similarity is an Equivalence Relation

Similarity is an equivalence relation, i.e. for \\n \times n\\ matrices \\A,B,\\ and \\C\\,

1. \\A \sim A\\ (reflexive)

2. If \\A \sim B\\, then \\B \sim A\\ (symmetric)

3. If \\A \sim B\\ and \\B \sim C\\, then \\A \sim C\\ (transitive)

Proof

It is clear that \\A\sim A\\, taking \\P=I\\.

Now, if \\A\sim B,\\ then for some \\P\\ invertible, \\A=P^{-1}BP\nonumber \\ and so \\PAP^{-1}=B\nonumber \\ But then \\\left( P^{-1}\right) ^{-1}AP^{-1}=B\nonumber \\ which shows that \\B\sim A\\.

Now suppose \\A\sim B\\ and \\B\sim C\\. Then there exist invertible matrices \\P,Q\\ such that \\A=P^{-1}BP,\\ B=Q^{-1}CQ\nonumber \\ Then, \\A=P^{-1} \left( Q^{-1}CQ \right)P=\left( QP\right) ^{-1}C\left( QP\right)\nonumber \\ showing that \\A\\ is similar to \\C\\.

Another important concept necessary to this section is the trace of a matrix. Consider the definition.

##### Definition \\\PageIndex{1}\\: Trace of a Matrix

If \\A=$$a\_{ij}$$\\ is an \\n\times n\\ matrix, then the trace of \\A\\ is \\\mathrm{trace}(A) = \sum\_{i=1}^n a\_{ii}.\nonumber \\

In words, the trace of a matrix is the sum of the entries on the main diagonal.

##### Lemma \\\PageIndex{2}\\: Properties of Trace

For \\n\times n\\ matrices \\A\\ and \\B\\, and any \\k\in\mathbb{R}\\,

1. \\\mathrm{trace}(A+B)=\mathrm{trace}(A) + \mathrm{trace}(B)\\

2. \\\mathrm{trace}(kA)=k\cdot\mathrm{trace}(A)\\

3. \\\mathrm{trace}(AB)=\mathrm{trace}(BA)\\

The following theorem includes a reference to the characteristic polynomial of a matrix. Recall that for any \\n \times n\\ matrix \\A\\, the characteristic polynomial of \\A\\ is \\c_A(x)=\det(xI-A)\\.

##### Theorem \\\PageIndex{1}\\: Properties of Similar Matrices

If \\A\\ and \\B\\ are \\n\times n\\ matrices and \\A\sim B\\, then

1. \\\det(A) = \det(B)\\

2. \\\mathrm{rank}(A) = \mathrm{rank}(B)\\

3. \\\mathrm{trace}(A)= \mathrm{trace}(B)\\

4. \\c_A(x)=c_B(x)\\

5. \\A\\ and \\B\\ have the same eigenvalues

We now proceed to the main concept of this section. When a matrix is similar to a diagonal matrix, the matrix is said to be diagonalizable. We define a diagonal matrix \\D\\ as a matrix containing a zero in every entry except those on the main diagonal. More precisely, if \\d\_{ij}\\ is the \\ij^{th}\\ entry of a diagonal matrix \\D\\, then \\d\_{ij}=0\\ unless \\i=j\\. Such matrices look like the following. \\D = \left$$ \begin{array}{ccc} \ast & & 0 \\ & \ddots & \\ 0 & & \ast \end{array} \right$$\nonumber \\ where \\\ast\\ is a number which might not be zero.

The following is the formal definition of a diagonalizable matrix.

##### Definition \\\PageIndex{2}\\: Diagonalizable

Let \\A\\ be an \\n\times n\\ matrix. Then \\A\\ is said to be diagonalizable if there exists an invertible matrix \\P\\ such that \\P^{-1}AP=D\nonumber \\ where \\D\\ is a diagonal matrix.

Notice that the above equation can be rearranged as \\A=PDP^{-1}\\. Suppose we wanted to compute \\A^{100}\\. By diagonalizing \\A\\ first it suffices to then compute \\\left(PDP^{-1}\right)^{100}\\, which reduces to \\PD^{100}P^{-1}\\. This last computation is much simpler than \\A^{100}\\. While this process is described in detail later, it provides motivation for diagonalization.

Diagonalizing a Matrix

The most important theorem about diagonalizability is the following major result.

##### Theorem \\\PageIndex{2}\\: Eigenvectors and Diagonalizable Matrices

An \\n\times n\\ matrix \\A\\ is diagonalizable if and only if there is an invertible matrix \\P\\ given by \\P=\left$$\begin{array}{cccc} X\_{1} & X\_{2} & \cdots & X\_{n} \end{array} \right$$\nonumber\\ where the \\X\_{k}\\ are eigenvectors of \\A\\.

Moreover if \\A\\ is diagonalizable, the corresponding eigenvalues of \\A\\ are the diagonal entries of the diagonal matrix \\D\\.

Proof

Suppose \\P\\ is given as above as an invertible matrix whose columns are eigenvectors of \\A\\. Then \\P^{-1}\\ is of the form \\P^{-1}=\left$$\begin{array}{c} W\_{1}^{T} \\ W\_{2}^{T} \\ \vdots \\ W\_{n}^{T} \end{array} \right$$\nonumber \\ where \\W\_{k}^{T}X\_{j}=\delta \_{kj},\\ which is the Kronecker’s symbol defined by \\\delta \_{ij}=\left\\ \begin{array}{c} 1 \text{ if }i=j \\ 0\text{ if }i\neq j \end{array} \right.\nonumber \\

Then \\\begin{aligned} P^{-1}AP & = \left$$\begin{array}{c} W\_{1}^{T} \\ W\_{2}^{T} \\ \vdots \\ W\_{n}^{T} \end{array} \right$$ \left$$\begin{array}{cccc} AX\_{1} & AX\_{2} & \cdots & AX\_{n} \end{array} \right$$ \\ & = \left$$\begin{array}{c} W\_{1}^{T} \\ W\_{2}^{T} \\ \vdots \\ W\_{n}^{T} \end{array} \right$$ \left$$\begin{array}{cccc} \lambda \_{1}X\_{1} & \lambda \_{2}X\_{2} & \cdots & \lambda \_{n}X\_{n} \end{array} \right$$ \\ &= \left$$\begin{array}{ccc} \lambda \_{1} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n} \end{array} \right$$ \end{aligned}\\

Conversely, suppose \\A\\ is diagonalizable so that \\P^{-1}AP=D.\\ Let \\P=\left$$\begin{array}{cccc} X\_{1} & X\_{2} & \cdots & X\_{n} \end{array} \right$$\nonumber \\ where the columns are the \\X\_{k}\\ and \\D=\left$$\begin{array}{ccc} \lambda \_{1} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n} \end{array} \right$$\nonumber \\ Then \\AP=PD=\left$$\begin{array}{cccc} X\_{1} & X\_{2} & \cdots & X\_{n} \end{array} \right$$ \left$$\begin{array}{ccc} \lambda \_{1} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n} \end{array} \right$$\nonumber\\ and so \\\left$$\begin{array}{cccc} AX\_{1} & AX\_{2} & \cdots & AX\_{n} \end{array} \right$$ =\left$$\begin{array}{cccc} \lambda \_{1}X\_{1} & \lambda \_{2}X\_{2} & \cdots & \lambda \_{n}X\_{n} \end{array} \right$$\nonumber\\ showing the \\X\_{k}\\ are eigenvectors of \\A\\ and the \\\lambda \_{k}\\ are eigenvectors.

Notice that because the matrix \\P\\ defined above is invertible it follows that the set of eigenvectors of \\A\\, \\\left\\ X_1, X_2, \cdots, X_n \right\\\\, form a basis of \\\mathbb{R}^n\\.

We demonstrate the concept given in the above theorem in the next example. Note that not only are the columns of the matrix \\P\\ formed by eigenvectors, but \\P\\ must be invertible so must consist of a wide variety of eigenvectors. We achieve this by using basic eigenvectors for the columns of \\P\\.

##### Example \\\PageIndex{1}\\: Diagonalize a Matrix

Let \\A=\left$$\begin{array}{rrr} 2 & 0 & 0 \\ 1 & 4 & -1 \\ -2 & -4 & 4 \end{array} \right$$\nonumber\\ Find an invertible matrix \\P\\ and a diagonal matrix \\D\\ such that \\P^{-1}AP=D\\.

###### Solution

By Theorem $\PageIndex{2}$ we use the eigenvectors of \\A\\ as the columns of \\P\\, and the corresponding eigenvalues of \\A\\ as the diagonal entries of \\D\\.

First, we will find the eigenvalues of \\A\\. To do so, we solve \\\det \left( \lambda I -A \right) =0\\ as follows. \\\det \left( \lambda \left$$\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ - \left$$\begin{array}{rrr} 2 & 0 & 0 \\ 1 & 4 & -1 \\ -2 & -4 & 4 \end{array} \right$$ \right) = 0\nonumber \\

This computation is left as an exercise, and you should verify that the eigenvalues are \\\lambda_1 =2, \lambda_2 = 2\\, and \\\lambda_3 = 6\\.

Next, we need to find the eigenvectors. We first find the eigenvectors for \\\lambda_1, \lambda_2 = 2\\. Solving \\\left(2I - A \right)X = 0\\ to find the eigenvectors, we find that the eigenvectors are \\t\left$$\begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right$$ +s\left$$\begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\ where \\t,s\\ are scalars. Hence there are two basic eigenvectors which are given by \\X_1 = \left$$\begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right$$, X_2 = \left$$\begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$\nonumber \\

You can verify that the basic eigenvector for \\\lambda_3 =6\\ is \\X_3 = \left$$\begin{array}{r} 0 \\ 1 \\ -2 \end{array} \right$$\\

Then, we construct the matrix \\P\\ as follows. \\P= \left$$\begin{array}{rrr} X_1 & X_2 & X_3 \end{array} \right$$ = \left$$\begin{array}{rrr} -2 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & -2 \end{array} \right$$\nonumber \\ That is, the columns of \\P\\ are the basic eigenvectors of \\A\\. Then, you can verify that \\P^{-1}=\left$$\begin{array}{rrr} - \frac{1}{4} & \frac{1}{2} & \frac{1}{4} \\ \frac{1}{2} & 1 & \frac{1}{2} \\ \frac{1}{4} & \frac{1}{2} & - \frac{1}{4} \end{array} \right$$\nonumber \\ Thus, \\\begin{aligned} P^{-1}AP &=\left$$\begin{array}{rrr} - \frac{1}{4} & \frac{1}{2} & \frac{1}{4} \\ \frac{1}{2} & 1 & \frac{1}{2} \\ \frac{1}{4} & \frac{1}{2} & - \frac{1}{4} \end{array} \right$$ \left$$\begin{array}{rrr} 2 & 0 & 0 \\ 1 & 4 & -1 \\ -2 & -4 & 4 \end{array} \right$$ \left$$\begin{array}{rrr} -2 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & -2 \end{array} \right$$ \\ &=\left$$\begin{array}{rrr} 2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 6 \end{array} \right$$ \end{aligned}\\

You can see that the result here is a diagonal matrix where the entries on the main diagonal are the eigenvalues of \\A\\. We expected this based on Theorem $\PageIndex{2}$. Notice that eigenvalues on the main diagonal *must* be in the same order as the corresponding eigenvectors in \\P\\.

Consider the next important theorem.

##### Theorem \\\PageIndex{3}\\: Linearly Independent Eigenvectors

Let \\A\\ be an \\n\times n\\ matrix, and suppose that \\A\\ has distinct eigenvalues \\\lambda_1, \lambda_2, \ldots, \lambda_m\\. For each \\i\\, let \\X_i\\ be a \\\lambda_i\\-eigenvector of \\A\\. Then \\\\ X_1, X_2, \ldots, X_m\\\\ is linearly independent.

The corollary that follows from this theorem gives a useful tool in determining if \\A\\ is diagonalizable.

##### Corollary \\\PageIndex{1}\\: Distinct Eigenvalues

Let \\A\\ be an \\n \times n\\ matrix and suppose it has \\n\\ distinct eigenvalues. Then it follows that \\A\\ is diagonalizable.

It is possible that a matrix \\A\\ cannot be diagonalized. In other words, we cannot find an invertible matrix \\P\\ so that \\P^{-1}AP=D\\.

Consider the following example.

##### Example \\\PageIndex{2}\\: A Matrix which cannot be Diagonalized

Let \\A = \left$$\begin{array}{rr} 1 & 1 \\ 0 & 1 \end{array} \right$$\nonumber\\ If possible, find an invertible matrix \\P\\ and diagonal matrix \\D\\ so that \\P^{-1}AP=D\\.

###### Solution

Through the usual procedure, we find that the eigenvalues of \\A\\ are \\\lambda_1 =1, \lambda_2=1.\\ To find the eigenvectors, we solve the equation \\\left(\lambda I - A \right) X = 0\\. The matrix \\\left(\lambda I -A \right)\\ is given by \\\left$$\begin{array}{cc} \lambda - 1 & -1 \\ 0 & \lambda - 1 \end{array} \right$$\nonumber\\

Substituting in \\\lambda = 1\\, we have the matrix \\\left$$\begin{array}{cc} 1 - 1 & -1 \\ 0 & 1 - 1 \end{array} \right$$ = \left$$\begin{array}{rr} 0 & -1 \\ 0 & 0 \end{array} \right$$\nonumber\\

Then, solving the equation \\\left(\lambda I - A\right) X = 0\\ involves carrying the following augmented matrix to its reduced row-echelon form. \\\left$$\begin{array}{rr\|r} 0 & -1 & 0 \\ 0 & 0 & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$\begin{array}{rr\|r} 0 & -1 & 0 \\ 0 & 0 & 0 \end{array} \right$$\nonumber\\

Then the eigenvectors are of the form \\t\left$$\begin{array}{r} 1 \\ 0 \end{array} \right$$\nonumber\\ and the basic eigenvector is \\X_1 = \left$$\begin{array}{r} 1 \\ 0 \end{array} \right$$\nonumber\\

In this case, the matrix \\A\\ has one eigenvalue of multiplicity two, but only one basic eigenvector. In order to diagonalize \\A\\, we need to construct an invertible \\2\times 2\\ matrix \\P\\. However, because \\A\\ only has one basic eigenvector, we cannot construct this \\P\\. Notice that if we were to use \\X_1\\ as both columns of \\P\\, \\P\\ would not be invertible. For this reason, we cannot repeat eigenvectors in \\P\\.

Hence this matrix cannot be diagonalized.

The idea that a matrix may not be diagonalizable suggests that conditions exist to determine when it is possible to diagonalize a matrix. We saw earlier in Corollary $\PageIndex{1}$ that an \\n \times n\\ matrix with \\n\\ distinct eigenvalues is diagonalizable. It turns out that there are other useful diagonalizability tests.

First we need the following definition.

##### Definition \\\PageIndex{3}\\: Eigenspace

Let \\A\\ be an \\n\times n\\ matrix and \\\lambda\in\mathbb{R}\\. The eigenspace of \\A\\ corresponding to \\\lambda\\, written \\E\_{\lambda}(A)\\ is the set of all eigenvectors corresponding to \\\lambda\\.

In other words, the eigenspace \\E\_{\lambda}(A)\\ is all \\X\\ such that \\AX = \lambda X\\. Notice that this set can be written \\E\_{\lambda}(A) = \mathrm{null}(\lambda I - A)\\, showing that \\E\_{\lambda}(A)\\ is a subspace of \\\mathbb{R}^n\\.

Recall that the multiplicity of an eigenvalue \\\lambda\\ is the number of times that it occurs as a root of the characteristic polynomial.

Consider now the following lemma.

##### Lemma \\\PageIndex{3}\\: Dimension of the Eigenspace

If \\A\\ is an \\n\times n\\ matrix, then \\\dim(E\_{\lambda}(A))\leq m\nonumber\\ where \\\lambda\\ is an eigenvalue of \\A\\ of multiplicity \\m\\.

This result tells us that if \\\lambda\\ is an eigenvalue of \\A\\, then the number of linearly independent \\\lambda\\-eigenvectors is never more than the multiplicity of \\\lambda\\. We now use this fact to provide a useful diagonalizability condition.

##### Theorem \\\PageIndex{4}\\: Diagonalizability Condition

Let \\A\\ be an \\n \times n\\ matrix \\A\\. Then \\A\\ is diagonalizable if and only if for each eigenvalue \\\lambda\\ of \\A\\, \\\dim(E\_{\lambda}(A))\\ is equal to the multiplicity of \\\lambda\\.

Complex Eigenvalues

In some applications, a matrix may have eigenvalues which are complex numbers. For example, this often occurs in differential equations. These questions are approached in the same way as above.

Consider the following example.

##### Example \\\PageIndex{3}\\: A Real Matrix with Complex Eigenvalues

Let \\A=\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & -1 \\ 0 & 1 & 2 \end{array} \right $$\nonumber \\ Find the eigenvalues and eigenvectors of \\A\\.

###### Solution

We will first find the eigenvalues as usual by solving the following equation.

\\\det \left( \lambda \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right $$ - \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & -1 \\ 0 & 1 & 2 \end{array} \right $$ \right) =0\nonumber \\ This reduces to \\\left( \lambda -1\right) \left( \lambda^{2}-4 \lambda +5\right) =0.\\ The solutions are \\\lambda_1 =1,\lambda_2 = 2+i\\ and \\\lambda_3 =2-i.\\

There is nothing new about finding the eigenvectors for \\\lambda_1 =1\\ so this is left as an exercise.

Consider now the eigenvalue \\\lambda_2 =2+i.\\ As usual, we solve the equation \\\left(\lambda I -A \right) X = 0\\ as given by \\\left( \left( 2+i\right) \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right $$ - \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & -1 \\ 0 & 1 & 2 \end{array} \right $$ \right) X =\left $$ \begin{array}{r} 0 \\ 0 \\ 0 \end{array} \right $$\nonumber \\ In other words, we need to solve the system represented by the augmented matrix \\\left $$ \begin{array}{crr\|r} 1+i & 0 & 0 & 0 \\ 0 & i & 1 & 0 \\ 0 & -1 & i & 0 \end{array} \right $$\nonumber \\

We now use our row operations to solve the system. Divide the first row by \\\left( 1+i\right)\\ and then take \\-i\\ times the second row and add to the third row. This yields \\\left $$ \begin{array}{rrr\|r} 1 & 0 & 0 & 0 \\ 0 & i & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber\\ Now multiply the second row by \\-i\\ to obtain the reduced row-echelon form, given by \\\left $$ \begin{array}{rrr\|r} 1 & 0 & 0 & 0 \\ 0 & 1 & -i & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber \\ Therefore, the eigenvectors are of the form \\t\left $$ \begin{array}{r} 0 \\ i \\ 1 \end{array} \right $$\nonumber\\ and the basic eigenvector is given by \\X_2 = \left $$ \begin{array}{r} 0 \\ i \\ 1 \end{array} \right $$\nonumber\\

As an exercise, verify that the eigenvectors for \\\lambda_3 =2-i\\ are of the form \\t\left $$ \begin{array}{r} 0 \\ -i \\ 1 \end{array} \right $$\nonumber\\ Hence, the basic eigenvector is given by \\X_3 = \left $$ \begin{array}{r} 0 \\ -i \\ 1 \end{array} \right $$\nonumber\\

As usual, be sure to check your answers! To verify, we check that \\AX_3 = \left(2 - i \right) X_3\\ as follows. \\\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & -1 \\ 0 & 1 & 2 \end{array} \right $$ \left $$ \begin{array}{r} 0 \\ -i \\ 1 \end{array} \right $$ = \left $$ \begin{array}{c} 0 \\ -1-2i \\ 2-i \end{array} \right $$ =\left( 2-i\right) \left $$ \begin{array}{r} 0 \\ -i \\ 1 \end{array} \right $$\nonumber \\

Therefore, we know that this eigenvector and eigenvalue are correct.

Notice that in Example $\PageIndex{3}$, two of the eigenvalues were given by \\\lambda_2 = 2 + i\\ and \\\lambda_3 = 2-i\\. You may recall that these two complex numbers are conjugates. It turns out that whenever a matrix containing real entries has a complex eigenvalue \\\lambda\\, it also has an eigenvalue equal to \\\overline{\lambda}\\, the conjugate of \\\lambda\\.

---

7_03_3A_Applications_of_Spectral_Theory

> 来源: LibreTexts

> 原页: https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/07%3A_Spectral_Theory/7.03%3A_Applications_of_Spectral_Theory

Skip to main content

##### Outcomes

1. Use diagonalization to find a high power of a matrix.

2. Use diagonalization to solve dynamical systems.

Raising a Matrix to a High Power

Suppose we have a matrix \\A\\ and we want to find \\A^{50}\\. One could try to multiply \\A\\ with itself 50 times, but this is computationally extremely intensive (try it!). However diagonalization allows us to compute high powers of a matrix relatively easily. Suppose \\A\\ is diagonalizable, so that \\P^{-1}AP=D\\. We can rearrange this equation to write \\A=PDP^{-1}\\.

Now, consider \\A^{2}\\. Since \\A=PDP^{-1}\\, it follows that \\A^{2} = \left( PDP^{-1}\right) ^{2}=PDP^{-1}PDP^{-1}=PD^{2}P^{-1}\nonumber\\

Similarly, \\A^3 = \left( PDP^{-1}\right) ^{3}=PDP^{-1}PDP^{-1}PDP^{-1}=PD^{3}P^{-1}\nonumber\\

In general, \\A^n = \left( PDP^{-1}\right) ^{n}=PD^{n}P^{-1}\nonumber\\

Therefore, we have reduced the problem to finding \\D^{n}\\. In order to compute \\D^{n}\\, then because \\D\\ is diagonal we only need to raise every entry on the main diagonal of \\D\\ to the power of \\n\\.

Through this method, we can compute large powers of matrices. Consider the following example.

##### Example \\\PageIndex{1}\\: Raising a Matrix to a High Power

Let \\A=\left $$ \begin{array}{rrr} 2 & 1 & 0 \\ 0 & 1 & 0 \\ -1 & -1 & 1 \end{array} \right $$.\\ Find \\A^{50}.\\

###### Solution

We will first diagonalize \\A\\. The steps are left as an exercise and you may wish to verify that the eigenvalues of \\A\\ are \\\lambda_1 =1, \lambda_2=1\\, and \\\lambda_3=2\\.

The basic eigenvectors corresponding to \\\lambda_1, \lambda_2 = 1\\ are \\X_1 = \left $$ \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \right $$ , X_2 = \left $$ \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \right $$\nonumber\\

The basic eigenvector corresponding to \\\lambda_3 = 2\\ is \\X_3 = \left $$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right $$\nonumber\\

Now we construct \\P\\ by using the basic eigenvectors of \\A\\ as the columns of \\P\\. Thus \\P= \left $$ \begin{array}{rrr} X_1 & X_2 & X_3 \end{array} \right $$ = \left $$ \begin{array}{rrr} 0 & -1 & -1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right $$\nonumber\\ Then also \\P^{-1}=\left $$ \begin{array}{rrr} 1 & 1 & 1 \\ 0 & 1 & 0 \\ -1 & -1 & 0 \end{array} \right $$\nonumber\\ which you may wish to verify.

Then, \\\begin{aligned} P^{-1}AP &=\left $$ \begin{array}{rrr} 1 & 1 & 1 \\ 0 & 1 & 0 \\ -1 & -1 & 0 \end{array} \right $$ \left $$ \begin{array}{rrr} 2 & 1 & 0 \\ 0 & 1 & 0 \\ -1 & -1 & 1 \end{array} \right $$ \left $$ \begin{array}{rrr} 0 & -1 & -1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right $$ \\ &=\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$ \\ &= D\end{aligned}\\

Now it follows by rearranging the equation that \\A=PDP^{-1}=\left $$ \begin{array}{rrr} 0 & -1 & -1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right $$ \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$ \left $$ \begin{array}{rrr} 1 & 1 & 1 \\ 0 & 1 & 0 \\ -1 & -1 & 0 \end{array} \right $$\nonumber\\

Therefore, \\\begin{aligned} A^{50} &=PD^{50}P^{-1} \\ &=\left $$ \begin{array}{rrr} 0 & -1 & -1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right $$ \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$ ^{50}\left $$ \begin{array}{rrr} 1 & 1 & 1 \\ 0 & 1 & 0 \\ -1 & -1 & 0 \end{array} \right $$ \end{aligned}\\

By our discussion above, \\D^{50}\\ is found as follows. \\\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$ ^{50}=\left $$ \begin{array}{rrr} 1^{50} & 0 & 0 \\ 0 & 1^{50} & 0 \\ 0 & 0 & 2^{50} \end{array} \right $$\nonumber\\

It follows that \\\begin{aligned} A^{50} &=\left $$ \begin{array}{rrr} 0 & -1 & -1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right $$ \left $$ \begin{array}{rrr} 1^{50} & 0 & 0 \\ 0 & 1^{50} & 0 \\ 0 & 0 & 2^{50} \end{array} \right $$ \left $$ \begin{array}{rrr} 1 & 1 & 1 \\ 0 & 1 & 0 \\ -1 & -1 & 0 \end{array} \right $$ \\ &=\left $$ \begin{array}{ccc} 2^{50} & -1+2^{50} & 0 \\ 0 & 1 & 0 \\ 1-2^{50} & 1-2^{50} & 1 \end{array} \right $$ \end{aligned}\\

Through diagonalization, we can efficiently compute a high power of \\A\\. Without this, we would be forced to multiply this by hand!

The next section explores another interesting application of diagonalization.

Raising a Symmetric Matrix to a High Power

We already have seen how to use matrix diagonalization to compute powers of matrices. This requires computing eigenvalues of the matrix \\A\\, and finding an invertible matrix of eigenvectors \\P\\ such that \\P^{-1}AP\\ is diagonal. In this section we will see that if the matrix \\A\\ is symmetric (see Definition 2.5.2), then we can actually find such a matrix \\P\\ that is an orthogonal matrix of eigenvectors. Thus \\P^{-1}\\ is simply its transpose \\P^T\\, and \\P^TAP\\ is diagonal. When this happens we say that \\A\\ is orthogonally diagonalizable

In fact this happens if and only if \\A\\ is a symmetric matrix as shown in the following important theorem.

##### Theorem \\\PageIndex{1}\\: Principal Axis Theorem

The following conditions are equivalent for an \\n \times n\\ matrix \\A\\:

1. \\A\\ is symmetric.

2. \\A\\ has an orthonormal set of eigenvectors.

3. \\A\\ is orthogonally diagonalizable.

Proof

The complete proof is beyond this course, but to give an idea assume that \\A\\ has an orthonormal set of eigenvectors, and let \\P\\ consist of these eigenvectors as columns. Then \\P^{-1}=P^T\\, and \\P^TAP=D\\ a diagonal matrix. But then \\A=PDP^T\\, and \\A^T=(PDP^T)^T = (P^T)^TD^TP^T=PDP^T=A\nonumber\\ so \\A\\ is symmetric.

Now given a symmetric matrix \\A\\, one shows that eigenvectors corresponding to different eigenvalues are always orthogonal. So it suffices to apply the Gram-Schmidt process on the set of basic eigenvectors of each eigenvalue to obtain an orthonormal set of eigenvectors.

We demonstrate this in the following example.

##### Example \\\PageIndex{2}\\: Orthogonal Diagonalization of a Symmetric Matrix

Let \\A=\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right $$ .\\ Find an orthogonal matrix \\P\\ such that \\P^{T}AP\\ is a diagonal matrix.

###### Solution

In this case, verify that the eigenvalues are 2 and 1. First we will find an eigenvector for the eigenvalue \\2\\. This involves row reducing the following augmented matrix. \\\left $$ \begin{array}{ccc\|c} 2 - 1 & 0 & 0 & 0 \\ 0 & 2- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 2- \frac{3}{2} & 0 \end{array} \right $$\nonumber\\ The reduced row-echelon form is \\\left $$ \begin{array}{rrr\|r} 1 & 0 & 0 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber\\ and so an eigenvector is \\\left $$ \begin{array}{c} 0 \\ 1 \\ 1 \end{array} \right $$\nonumber\\ Finally to obtain an eigenvector of length one (unit eigenvector) we simply divide this vector by its length to yield: \\\left $$ \begin{array}{c} 0 \\ \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right $$\nonumber\\

Next consider the case of the eigenvalue \\1\\. To obtain basic eigenvectors, the matrix which needs to be row reduced in this case is \\\left $$ \begin{array}{ccc\|c} 1-1 & 0 & 0 & 0 \\ 0 & 1- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 1- \frac{3}{2} & 0 \end{array} \right $$\nonumber\\ The reduced row-echelon form is \\\left $$ \begin{array}{rrr\|r} 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber\\ Therefore, the eigenvectors are of the form \\\left $$ \begin{array}{c} s \\ -t \\ t \end{array} \right $$\nonumber\\ Note that all these vectors are automatically orthogonal to eigenvectors corresponding to the first eigenvalue. This follows from the fact that \\A\\ is symmetric, as mentioned earlier.

We obtain basic eigenvectors \\\left $$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right $$ \text{ and }\left $$ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right $$\nonumber\\ Since they are themselves orthogonal (by luck here) we do not need to use the Gram-Schmidt process and instead simply normalize these vectors to obtain \\\left $$ \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \right $$ \text{ and }\left $$ \begin{array}{c} 0 \\ -\frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right $$\nonumber\\ An orthogonal matrix \\P\\ to orthogonally diagonalize \\A\\ is then obtained by letting these basic vectors be the columns. \\P= \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$\nonumber\\ We verify this works. \\P^{T}AP\\ is of the form \\\left $$ \begin{array}{ccc} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ 1 & 0 & 0 \\ 0 & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{array} \right $$ \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right $$ \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$\nonumber\\ \\= \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$\nonumber\\ which is the desired diagonal matrix.

We can now apply this technique to efficiently compute high powers of a symmetric matrix.

##### Example \\\PageIndex{3}\\: Powers of a Symmetric Matrix

Let \\A=\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right $$ .\\ Compute \\A^7\\.

###### Solution

We found in Example $\PageIndex{2}$ that \\P^TAP=D\\ is diagonal, where

\\P= \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$ \text{ and } D = \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$\nonumber \\

Thus \\A=PDP^T\\ and \\A^7=PDP^T \\ PDP^T \\ \cdots PDP^T = PD^7P^T\\ which gives:

\\\begin{array}{rr} A^7 & = \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$ \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right $$ ^7 \left $$ \begin{array}{ccc} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\ 1 & 0 & 0 \\ 0 & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{array} \right $$ \\ & = \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$ \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2^7 \end{array} \right $$ \left $$ \begin{array}{ccc} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\ 1 & 0 & 0 \\ 0 & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{array} \right $$ \\ & = \left $$ \begin{array}{ccc} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right $$ \left $$ \begin{array}{ccc} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\ 1 & 0 & 0 \\ 0 & \frac{2^7}{\sqrt{2}} & \frac{2^7}{\sqrt{2}} \end{array} \right $$ \\ & = \left $$ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & \frac{2^7+1}{2} & \frac{2^7-1}{2}\\ 0 & \frac{2^7-1}{2} & \frac{2^7+1}{2} \end{array} \right $$ \\ \end{array}\nonumber\\

Markov Matrices

There are applications of great importance which feature a special type of matrix. Matrices whose columns consist of non-negative numbers that sum to one are called Markov matrices. An important application of Markov matrices is in population migration, as illustrated in the following definition.

##### Definition \\\PageIndex{1}\\: Migration Matrices

Let \\m\\ locations be denoted by the numbers \\1,2,\cdots ,m.\\ Suppose it is the case that each year the proportion of residents in location \\j\\ which move to location \\i\\ is \\a\_{ij}\\. Also suppose no one escapes or emigrates from without these \\m\\ locations. This last assumption requires \\\sum\_{i}a\_{ij}=1\\, and means that the matrix \\A\\, such that \\A = \left $$ a\_{ij} \right $$\\, is a Markov matrix. In this context, \\A\\ is also called a migration matrix.

Consider the following example which demonstrates this situation.

##### Example \\\PageIndex{4}\\: Migration Matrix

Let \\A\\ be a Markov matrix given by \\A = \left $$ \begin{array}{rr} .4 & .2 \\ .6 & .8 \end{array} \right $$\nonumber\\ Verify that \\A\\ is a Markov matrix and describe the entries of \\A\\ in terms of population migration.

###### Solution

The columns of \\A\\ are comprised of non-negative numbers which sum to \\1\\. Hence, \\A\\ is a Markov matrix.

Now, consider the entries \\a\_{ij}\\ of \\A\\ in terms of population. The entry \\a\_{11} = .4\\ is the proportion of residents in location one which stay in location one in a given time period. Entry \\a\_{21} = .6\\ is the proportion of residents in location 1 which move to location 2 in the same time period. Entry \\a\_{12} = .2\\ is the proportion of residents in location 2 which move to location 1. Finally, entry \\a\_{22} = .8\\ is the proportion of residents in location 2 which stay in location 2 in this time period.

Considered as a Markov matrix, these numbers are usually identified with probabilities. Hence, we can say that the probability that a resident of location one will stay in location one in the time period is \\.4\\.

Observe that in Example $\PageIndex{4}$ if there was initially say 15 thousand people in location 1 and 10 thousands in location 2, then after one year there would be \\.4 \times 15 + .2 \times 10 = 8\\ thousands people in location 1 the following year, and similarly there would be \\.6 \times 15 + .8 \times 10 = 17\\ thousands people in location 2 the following year.

More generally let \\X_n=\left $$ x\_{1n} \cdots x\_{mn}\right $$ ^{T}\\ where \\x\_{in}\\ is the population of location \\i\\ at time period \\n\\. We call \\X_n\\ the state vector at period \\n\\. In particular, we call \\X_0\\ the initial state vector. Letting \\A\\ be the migration matrix, we compute the population in each location \\i\\ one time period later by \\AX_n\\. In order to find the population of location \\i\\ after \\k\\ years, we compute the \\i^{th}\\ component of \\A^{k}X.\\ This discussion is summarized in the following theorem.

##### Theorem \\\PageIndex{2}\\: State Vector

Let \\A\\ be the migration matrix of a population and let \\X_n\\ be the vector whose entries give the population of each location at time period \\n\\. Then \\X_n\\ is the state vector at period \\n\\ and it follows that \\X\_{n+1} = A X_n\nonumber\\

The sum of the entries of \\X_n\\ will equal the sum of the entries of the initial vector \\X\_{0}\\. Since the columns of \\A\\ sum to \\1\\, this sum is preserved for every multiplication by \\A\\ as demonstrated below. \\\sum\_{i}\sum\_{j}a\_{ij}x\_{j}=\sum\_{j}x\_{j}\left( \sum\_{i}a\_{ij}\right) =\sum\_{j}x\_{j}\nonumber\\

Consider the following example.

##### Example \\\PageIndex{5}\\: Using a Migration Matrix

Consider the migration matrix \\A = \left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$\nonumber\\ for locations \\1,2,\\ and \\3.\\ Suppose initially there are \\100\\ residents in location \\1\\, \\200\\ in location \\2\\ and \\400\\ in location \\3\\. Find the population in the three locations after \\1,2,\\ and \\10\\ units of time.

###### Solution

Using Theorem $\PageIndex{2}$ we can find the population in each location using the equation \\X\_{n+1} = AX_n\\. For the population after \\1\\ unit, we calculate \\X_1 = AX_0\\ as follows. \\\begin{aligned} X_1 &= AX_0 \\ \left $$ \begin{array}{r} x\_{11} \\ x\_{21} \\ x\_{31} \end{array}\right $$ &= \left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$ \left $$ \begin{array}{r} 100 \\ 200 \\ 400 \end{array} \right $$ \\ &= \left $$ \begin{array}{r} 100 \\ 180 \\ 420 \end{array}\right $$\end{aligned}\\ Therefore after one time period, location \\1\\ has \\100\\ residents, location \\2\\ has \\180\\, and location \\3\\ has \\420\\. Notice that the total population is unchanged, it simply migrates within the given locations. We find the locations after two time periods in the same way. \\\begin{aligned} X_2 &= AX_1 \\ \left $$ \begin{array}{r} x\_{12} \\ x\_{22} \\ x\_{32} \end{array}\right $$ &= \left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$ \left $$ \begin{array}{r} 100 \\ 180 \\ 420 \end{array} \right $$ \\ &= \left $$ \begin{array}{r} 102 \\ 164 \\ 434 \end{array}\right $$\end{aligned}\\

We could progress in this manner to find the populations after \\10\\ time periods. However from our above discussion, we can simply calculate \\\left( A^{n}X_0\right) \_{i}\\, where \\n\\ denotes the number of time periods which have passed. Therefore, we compute the populations in each location after \\10\\ units of time as follows. \\\begin{aligned} X\_{10} &= A^{10}X_0 \\ \left $$ \begin{array}{r} x\_{1 10} \\ x\_{2 10} \\ x\_{3 10} \end{array} \right $$ &= \left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$ ^{10}\left $$ \begin{array}{r} 100 \\ 200 \\ 400 \end{array} \right $$ \\ &= \left $$ \begin{array}{c} 115.\\ 085\\829\\22 \\ 120.\\ 130\\672\\44 \\ 464.\\ 783\\498\\34 \end{array} \right $$\end{aligned}\\ Since we are speaking about populations, we would need to round these numbers to provide a logical answer. Therefore, we can say that after \\10\\ units of time, there will be \\115\\ residents in location one, \\120\\ in location two, and \\465\\ in location three.

A second important application of Markov matrices is the concept of random walks. Suppose a walker has \\m\\ locations to choose from, denoted \\1, 2, \cdots, m\\. Let \\a\_{ij}\\ refer to the probability that the person will travel to location \\i\\ from location \\j\\. Again, this requires that \\\sum\_{i=1}^{k}a\_{ij}=1\nonumber\\ In this context, the vector \\X_n=\left $$ x\_{1n} \cdots x\_{mn}\right $$ ^{T}\\ contains the probabilities \\x\_{in}\\ the walker ends up in location \\i, 1\leq i \leq m\\ at time \\n\\.

##### Example \\\PageIndex{6}\\: Random Walks

Suppose three locations exist, referred to as locations \\1, 2\\ and \\3\\. The Markov matrix of probabilities \\A = $$a\_{ij}$$\\ is given by \\\left $$ \begin{array}{rrr} 0.4 & 0.1 & 0.5 \\ 0.4 & 0.6 & 0.1 \\ 0.2 & 0.3 & 0.4 \end{array} \right $$\nonumber\\ If the walker starts in location \\1\\, calculate the probability that he ends up in location \\3\\ at time \\n = 2\\.

###### Solution

Since the walker begins in location \\1\\, we have \\X\_{0} = \left $$ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right $$\nonumber\\ The goal is to calculate \\x\_{32}\\. To do this we calculate \\X\_{2}\\, using \\X\_{n+1} = AX\_{n}\\. \\\begin{aligned} X\_{1} &= A X\_{0} \\ &= \left $$ \begin{array}{rrr} 0.4 & 0.1 & 0.5 \\ 0.4 & 0.6 & 0.1 \\ 0.2 & 0.3 & 0.4 \end{array} \right $$ \left $$ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right $$ \\ &= \left $$ \begin{array}{r} 0.4 \\ 0.4 \\ 0.2 \end{array} \right $$ \\\end{aligned}\\ \\\begin{aligned} X\_{2} &= A X\_{1} \\ &= \left $$ \begin{array}{rrr} 0.4 & 0.1 & 0.5 \\ 0.4 & 0.6 & 0.1 \\ 0.2 & 0.3 & 0.4 \end{array} \right $$ \left $$ \begin{array}{c} 0.4 \\ 0.4 \\ 0.2 \end{array} \right $$ \\ &= \left $$ \begin{array}{r} 0.3 \\ 0.42 \\ 0.28 \end{array} \right $$ \\\end{aligned}\\ This gives the probabilities that our walker ends up in locations 1, 2, and 3. For this example we are interested in location 3, with a probability on \\0.28\\.

Returning to the context of migration, suppose we wish to know how many residents will be in a certain location after a very long time. It turns out that if some power of the migration matrix has all positive entries, then there is a vector \\X_s\\ such that \\A^{n}X\_{0}\\ approaches \\X_s\\ as \\n\\ becomes very large. Hence as more time passes and \\n\\ increases, \\A^{n}X\_{0}\\ will become closer to the vector \\X_s\\.

Consider Theorem $\PageIndex{2}$. Let \\n\\ increase so that \\X_n\\ approaches \\X_s\\. As \\X_n\\ becomes closer to \\X_s\\, so too does \\X\_{n+1}\\. For sufficiently large \\n\\, the statement \\X\_{n+1} = AX_n\\ can be written as \\X_s = AX_s\\.

This discussion motivates the following theorem.

##### Theorem \\\PageIndex{3}\\: Steady State Vector

Let \\A\\ be a migration matrix. Then there exists a steady state vector written \\X_s\\ such that \\X_s = AX_s\nonumber \\ where \\X_s\\ has positive entries which have the same sum as the entries of \\X_0\\.

As \\n\\ increases, the state vectors \\X_n\\ will approach \\X_s\\.

Note that the condition in Theorem $\PageIndex{3}$ can be written as \$I - A)X_s=0\\, representing a homogeneous system of equations.

Consider the following example. Notice that it is the same example as the Example $\PageIndex{5}$ but here it will involve a longer time frame.

##### Example \\\PageIndex{7}\\: Populations over the Long Run

Consider the migration matrix \\A = \left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$\nonumber \\ for locations \\1,2,\\ and \\3.\\ Suppose initially there are 100 residents in location 1, 200 in location 2 and 400 in location 4. Find the population in the three locations after a long time.

###### Solution

By Theorem $\PageIndex{3}$ the steady state vector \\X_s\\ can be found by solving the system \$I-A)X_s = 0\\.

Thus we need to find a solution to \\\left( \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right $$ -\left $$ \begin{array}{rrr} .6 & 0 & .1 \\ .2 & .8 & 0 \\ .2 & .2 & .9 \end{array} \right $$ \right) \left $$ \begin{array}{c} x\_{1s} \\ x\_{2s}\\ x\_{3s} \end{array} \right $$ =\left $$ \begin{array}{c} 0 \\ 0 \\ 0 \end{array} \right $$\nonumber \\ The augmented matrix and the resulting reduced row-echelon form are given by \\\left $$ \begin{array}{rrr\|r} 0.4 & 0 & -0.1 & 0 \\ -0.2 & 0.2 & 0 & 0 \\ -0.2 & -0.2 & 0.1 & 0 \end{array} \right $$ \rightarrow \cdots \rightarrow \left $$ \begin{array}{rrr\|r} 1 & 0 & -0.25 & 0 \\ 0 & 1 & -0.25 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber\\ Therefore, the eigenvectors are \\t\left $$ \begin{array}{c} 0.25 \\ 0.25 \\ 1 \end{array} \right $$\nonumber\\

The initial vector \\X_0\\ is given by \\\left $$ \begin{array}{r} 100 \\ 200 \\ 400 \end{array} \right $$\nonumber\\

Now all that remains is to choose the value of \\t\\ such that \\0.25t+0.25t+t=100+200+400\nonumber\\ Solving this equation for \\t\\ yields \\t= \\ \frac{1400}{3}\\. Therefore the population in the long run is given by \\ \\ \frac{1400}{3}\left $$ \begin{array}{c} 0.25 \\ 0.25 \\ 1 \end{array} \right $$ = \left $$ \begin{array}{c} 116. 666\\666\\666\\ 666\\7 \\ 116. 666\\666\\666\\ 666\\7 \\ 466. 666\\666\\666\\ 666\\7 \end{array} \right $$\nonumber\\

Again, because we are working with populations, these values need to be rounded. The steady state vector \\X_s\\ is given by \\\left $$ \begin{array}{c} 117 \\ 117 \\ 466 \end{array} \right $$\nonumber\\

We can see that the numbers we calculated in Example $\PageIndex{5}$ for the populations after the \\10^{th}\\ unit of time are not far from the long term values.

Consider another example.

##### Example \\\PageIndex{8}\\: Populations After a Long Time

Suppose a migration matrix is given by \\A = \left $$ \begin{array}{ccc} \\ \frac{1}{5} & \\ \frac{1}{2} & \\ \frac{1}{5} \\ \\ \frac{1}{4} & \\ \frac{1}{4} & \\ \frac{1}{2} \\ \\ \frac{11}{20} & \\ \frac{1}{4} & \\ \frac{3}{10} \end{array} \right $$\nonumber\\ Find the comparison between the populations in the three locations after a long time.

###### Solution

In order to compare the populations in the long term, we want to find the steady state vector \\X_s\\. Solve \\\left( \left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right $$ -\left $$ \begin{array}{ccc} \\ \frac{1}{5} & \\ \frac{1}{2} & \\ \frac{1}{5} \\ \\ \frac{1}{4} & \\ \frac{1}{4} & \\ \frac{1}{2} \\ \\ \frac{11}{20} & \\ \frac{1}{4} & \\ \frac{3}{10} \end{array} \right $$ \right) \left $$ \begin{array}{c} x\_{1s} \\ x\_{2s} \\ x\_{3s} \end{array} \right $$ =\left $$ \begin{array}{c} 0 \\ 0 \\ 0 \end{array} \right $$\nonumber\\ The augmented matrix and the resulting reduced row-echelon form are given by \\\left $$ \begin{array}{rrr\|r} \\ \frac{4}{5} & - \\ \frac{1}{2} & - \\ \frac{1}{5} & 0 \\ - \\ \frac{1}{4} & \\ \frac{3}{4} & - \\ \frac{1}{2} & 0 \\ - \\ \frac{11}{20} & - \\ \frac{1}{4} & \\ \frac{7}{10} & 0 \end{array} \right $$ \rightarrow \cdots \rightarrow \left $$ \begin{array}{rrr\|r} 1 & 0 & - \\ \frac{16}{19} & 0 \\ 0 & 1 & - \\ \frac{18}{19} & 0 \\ 0 & 0 & 0 & 0 \end{array} \right $$\nonumber\\ and so an eigenvector is \\\left $$ \begin{array}{c} 16 \\ 18 \\ 19 \end{array} \right $$\nonumber\\

Therefore, the proportion of population in location 2 to location 1 is given by \\ \\ \frac{18}{16}\\. The proportion of population 3 to location 2 is given by \\ \\ \frac{19}{18}\\.

Eigenvalues of Markov Matrices

The following is an important proposition.

##### Proposition \\\PageIndex{1}\\: Eigenvalues of a Migration Matrix

Let \\A=\left $$ a\_{ij}\right $$\\ be a migration matrix. Then \\1\\ is always an eigenvalue for \\A.\\

Proof

Remember that the determinant of a matrix always equals that of its transpose. Therefore, \\\det \left( \lambda I - A\right) =\det \left( \left( \lambda I - A\right) ^{T}\right) =\det \left( \lambda I - A^T\right)\nonumber\\ because \\I^{T}=I.\\ Thus the characteristic equation for \\A\\ is the same as the characteristic equation for \\A^{T}\\. Consequently, \\A\\ and \\A^{T}\\ have the same eigenvalues. We will show that \\1\\ is an eigenvalue for \\A^{T}\\ and then it will follow that \\1\\ is an eigenvalue for \\A\\.

Remember that for a migration matrix, \\\sum\_{i}a\_{ij}=1.\\ Therefore, if \\A^{T}=\left $$ b\_{ij}\right $$\\ with \\b\_{ij}=a\_{ji},\\ it follows that \\\sum\_{j}b\_{ij}=\sum\_{j}a\_{ji}=1\nonumber\\

Therefore, from matrix multiplication, \\A^{T}\left $$ \begin{array}{r} 1 \\ \vdots \\ 1 \end{array} \right $$ =\left $$ \begin{array}{c} \sum\_{j}b\_{ij} \\ \vdots \\ \sum\_{j}b\_{ij} \end{array} \right $$ =\left $$ \begin{array}{r} 1 \\ \vdots \\ 1 \end{array} \right $$\nonumber\\

Notice that this shows that \\\left $$ \begin{array}{r} 1 \\ \vdots \\ 1 \end{array} \right $$\\ is an eigenvector for \\A^{T}\\ corresponding to the eigenvalue, \\\lambda =1.\\ As explained above, this shows that \\\lambda =1\\ is an eigenvalue for \\A\\ because \\A\\ and \\A^{T}\\ have the same eigenvalues.

Dynamical Systems

The migration matrices discussed above give an example of a discrete dynamical system. We call them discrete because they involve discrete values taken at a sequence of points rather than on a continuous interval of time.

An example of a situation which can be studied in this way is a predator prey model. Consider the following model where \\x\\ is the number of prey and \\y\\ the number of predators in a certain area at a certain time. These are functions of \\n\in \mathbb{N}\\ where \\n=1,2,\cdots\\ are the ends of intervals of time which may be of interest in the problem. In other words, \\x \left( n \right)\\ is the number of prey at the end of the \\n^{th}\\ interval of time. An example of this situation may be modeled by the following equation \\\left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ =\left $$ \begin{array}{rr} 2 & -3 \\ 1 & 4 \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber\\ This says that from time period \\n\\ to \\n+1\\, \\x\\ increases if there are more \\x\\ and decreases as there are more \\y\\. In the context of this example, this means that as the number of predators increases, the number of prey decreases. As for \\y,\\ it increases if there are more \\y\\ and also if there are more \\x\\.

This is an example of a matrix recurrence which we define now.

##### Definition \\\PageIndex{2}\\: Matrix Recurrence

Suppose a dynamical system is given by \\\begin{aligned} x\_{n+1} &= a x_n + b y_n \\ y\_{n+1} &= c x_n + d y_n\end{aligned}\\

This system can be expressed as \\V\_{n+1} = A V\_{n}\\ where \\V\_{n} = \left $$ \begin{array}{r} x_n \\ y_n \end{array} \right $$\\ and \\A = \left $$ \begin{array}{rr} a & b \\ c & d \end{array} \right $$\\.

In this section, we will examine how to find solutions to a dynamical system given certain initial conditions. This process involves several concepts previously studied, including matrix diagonalization and Markov matrices. The procedure is given as follows. Recall that when diagonalized, we can write \\A^{n} = PD^{n}P^{-1}\\.

##### Procedure \\\PageIndex{1}\\: Solving a Dynamical System

Suppose a dynamical system is given by \\\begin{aligned} x\_{n+1} &= a x_n + b y_n \\ y\_{n+1} &= c x_n + d y_n\end{aligned}\\

Given initial conditions \\x_0\\ and \\y_0\\, the solutions to the system are found as follows:

1. Express the dynamical system in the form \\V\_{n+1} = AV_n\\.

2. Diagonalize \\A\\ to be written as \\A = PDP^{-1}\\.

3. Then \\V\_{n} = PD^{n} P^{-1} V\_{0}\\ where \\V\_{0}\\ is the vector containing the initial conditions.

4. If given specific values for \\n\\, substitute into this equation. Otherwise, find a general solution for \\n\\.

We will now consider an example in detail.

##### Example \\\PageIndex{9}\\: Solutions of a Discrete Dynamical System

Suppose a dynamical system is given by \\\begin{aligned} x\_{n+1} &= 1.5 x_n - 0.5y_n\\ y\_{n+1} &= 1.0 x_n\end{aligned}\\

Express this system as a matrix recurrence and find solutions to the dynamical system for initial conditions \\x_0=20, y_0=10\\.

###### Solution

First, we express the system as a matrix recurrence. \\\begin{aligned} V\_{n+1} &= AV\_{n}\\ \left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ &=\left $$ \begin{array}{rr} 1.5 & -0.5 \\ 1.0 & 0 \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\end{aligned}\\

Then \\A = \left $$ \begin{array}{rr} 1.5 & -0.5 \\ 1.0 & 0 \end{array} \right $$\nonumber \\ You can verify that the eigenvalues of \\A\\ are \\1\\ and \\.5\\. By diagonalizing, we can write \\A\\ in the form \\P^{-1} D P = \left $$ \begin{array}{rr} 1 & 1 \\ 1 & 2 \end{array} \right $$ \left $$ \begin{array}{rr} 1 & 0 \\ 0 & .5 \end{array} \right $$ \left $$ \begin{array}{rr} 2 & -1 \\ -1 & 1 \end{array} \right $$\nonumber\\

Now given an initial condition \\V_0 = \left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$\nonumber\\ the solution to the dynamical system is given by \\\begin{aligned} V_n &= P D^n P^{-1} V_0\\ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$ &=\left $$ \begin{array}{rr} 1 & 1 \\ 1 & 2 \end{array} \right $$ \left $$ \begin{array}{rr} 1 & 0 \\ 0 & .5 \end{array} \right $$ ^{n}\left $$ \begin{array}{rr} 2 & -1 \\ -1 & 1 \end{array} \right $$ \left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$ \\ &=\left $$ \begin{array}{rr} 1 & 1 \\ 1 & 2 \end{array} \right $$ \left $$ \begin{array}{rr} 1 & 0 \\ 0 & \left( .5\right) ^{n} \end{array} \right $$ \left $$ \begin{array}{rr} 2 & -1 \\ -1 & 1 \end{array} \right $$ \left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$ \\ &=\left $$ \begin{array}{c} y\_{0}\left( \left( .5\right) ^{n}-1\right) -x\_{0}\left( \left( .5\right) ^{n}-2\right) \\ y\_{0}\left( 2\left( .5\right) ^{n}-1\right) -x\_{0}\left( 2\left( .5\right) ^{n}-2\right) \end{array} \right $$ \end{aligned}\\

If we let \\n\\ become arbitrarily large, this vector approaches \\\left $$ \begin{array}{c} 2x\_{0}-y\_{0} \\ 2x\_{0}-y\_{0} \end{array} \right $$\nonumber\\

Thus for large \\n,\\ \\\left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$ \approx \left $$ \begin{array}{c} 2x\_{0}-y\_{0} \\ 2x\_{0}-y\_{0} \end{array} \right $$\nonumber\\

Now suppose the initial condition is given by \\\left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$ = \left $$ \begin{array}{r} 20 \\ 10 \end{array} \right $$\nonumber\\

Then, we can find solutions for various values of \\n\\. Here are the solutions for values of \\n\\ between \\1\\ and \\5\\ \\n=1: \left $$ \begin{array}{r} 25.0 \\ 20.0 \end{array} \right $$, n=2: \left $$ \begin{array}{r} 27.5 \\ 25.0 \end{array} \right $$, n=3: \left $$ \begin{array}{r} 28.75 \\ 27.5 \end{array} \right $$\nonumber\\ \\n=4: \left $$ \begin{array}{r} 29.375 \\ 28.75 \end{array} \right $$, n=5: \left $$ \begin{array}{r} 29.688 \\ 29.375 \end{array} \right $$\nonumber\\

Notice that as \\n\\ increases, we approach the vector given by \\\left $$ \begin{array}{c} 2x\_{0}-y\_{0} \\ 2x\_{0}-y\_{0} \end{array} \right $$ = \left $$ \begin{array}{r} 2\left(20\right)- 10\\ 2\left( 20 \right)-10 \end{array} \right $$ = \left $$ \begin{array}{r} 30\\ 30 \end{array} \right $$\nonumber\\

These solutions are graphed in the following figure.

Figure $\PageIndex{1}$

The following example demonstrates another system which exhibits some interesting behavior. When we graph the solutions, it is possible for the ordered pairs to spiral around the origin.

##### Example \\\PageIndex{10}\\: Finding Solutions to a Dynamical System

Suppose a dynamical system is of the form \\\left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ =\left $$ \begin{array}{rr} 0.7 & 0.7 \\ -0.7 & 0.7 \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber \\ Find solutions to the dynamical system for given initial conditions.

###### Solution

Let \\A = \left $$ \begin{array}{rr} 0.7 & 0.7 \\ -0.7 & 0.7 \end{array} \right $$\nonumber\\ To find solutions, we must diagonalize \\A\\. You can verify that the eigenvalues of \\A\\ are complex and are given by \\\lambda_1 = .7+.7i\\ and \\\lambda_2 = .7-.7i\\. The eigenvector for \\\lambda_1 = .7+.7i\\ is \\\left $$ \begin{array}{r} 1 \\ i \end{array} \right $$\nonumber\\ and that the eigenvector for \\\lambda_2 = .7-.7i\\ is \\\left $$ \begin{array}{r} 1 \\ -i \end{array} \right $$\nonumber\\

Thus the matrix \\A\\ can be written in the form \\\left $$ \begin{array}{rr} 1 & 1 \\ i & -i \end{array} \right $$ \left $$ \begin{array}{cc} .7+.7i & 0 \\ 0 & .7-.7i \end{array} \right $$ \left $$ \begin{array}{rr} \frac{1}{2} & - \frac{1}{2}i \\ \frac{1}{2} & \frac{1}{2}i \end{array} \right $$\nonumber\\ and so, \\\begin{aligned} V_n &= PD^nP^{-1}V_0 \\ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$ &=\left $$ \begin{array}{rr} 1 & 1 \\ i & -i \end{array} \right $$ \left $$ \begin{array}{cc} \left( .7+.7i\right) ^{n} & 0 \\ 0 & \left( .7-.7i\right) ^{n} \end{array} \right $$ \left $$ \begin{array}{rr} \frac{1}{2} & - \frac{1}{2}i \\ \frac{1}{2} & \frac{1}{2}i \end{array} \right $$ \left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$\end{aligned}\\

The explicit solution is given by \\\left $$ \begin{array}{c} x\_{0}\left( \frac{1}{2}\left( 0.7-0.7i \right) ^{n}+ \frac{1}{2} \left( 0.7+0.7i\right) ^{n}\right) + y\_{0}\left( \frac{1}{2} i\left( 0.7-0.7i\right) ^{n}-\frac{1}{2}i \left( 0.7+0.7i\right) ^{n}\right) \\ y\_{0}\left( \frac{1}{2} \left( 0.7-0.7i \right) ^{n}+ \frac{1}{2} \left( 0.7+0.7i\right) ^{n}\right) - x\_{0}\left( \frac{1}{2} i\left( 0.7-0.7i\right) ^{n}- \frac{1}{2}i\left( 0.7+0.7i\right) ^{n}\right) \end{array} \right $$\nonumber\\

Suppose the initial condition is \\\left $$ \begin{array}{c} x\_{0} \\ y\_{0} \end{array} \right $$ =\left $$ \begin{array}{r} 10 \\ 10 \end{array} \right $$\nonumber\\ Then one obtains the following sequence of values which are graphed below by letting \\n=1,2,\cdots ,20\\

Figure $\PageIndex{2}$

In this picture, the dots are the values and the dashed line is to help to picture what is happening.

These points are getting gradually closer to the origin, but they are circling the origin in the clockwise direction as they do so. As \\n\\ increases, the vector \\\left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\\ approaches \\\left $$ \begin{array}{r} 0 \\ 0 \end{array} \right $$\\

This type of behavior along with complex eigenvalues is typical of the deviations from an equilibrium point in the Lotka Volterra system of differential equations which is a famous model for predator-prey interactions. These differential equations are given by \\\begin{aligned} x^{\prime } &=x\left( a-by\right) \\ y^{\prime } &=-y\left( c-dx\right)\end{aligned}\\ where \\a,b,c,d\\ are positive constants. For example, you might have \\X\\ be the population of moose and \\Y\\ the population of wolves on an island.

Note that these equations make logical sense. The top says that the rate at which the moose population increases would be \\aX\\ if there were no predators \\Y\\. However, this is modified by multiplying instead by \\\left( a-bY\right)\\ because if there are predators, these will militate against the population of moose. The more predators there are, the more pronounced is this effect. As to the predator equation, you can see that the equations predict that if there are many prey around, then the rate of growth of the predators would seem to be high. However, this is modified by the term \\-cY\\ because if there are many predators, there would be competition for the available food supply and this would tend to decrease \\Y^{\prime }.\\

The behavior near an equilibrium point, which is a point where the right side of the differential equations equals zero, is of great interest. In this case, the equilibrium point is \\x=\frac{c}{d}, y=\frac{a}{b}\nonumber\\ Then one defines new variables according to the formula \\x+\frac{c}{d}=x,\\ y=y+\frac{a}{b}\nonumber\\ In terms of these new variables, the differential equations become \\\begin{aligned} x^{\prime } &=\left( x+\frac{c}{d}\right) \left( a-b\left( y+\frac{a}{b} \right) \right) \\ y^{\prime } &=-\left( y+\frac{a}{b}\right) \left( c-d\left( x+\frac{c}{d} \right) \right)\end{aligned}\\ Multiplying out the right sides yields \\\begin{aligned} x^{\prime } &=-bxy-b\frac{c}{d}y \\ y^{\prime } &=dxy+\frac{a}{b}dx\end{aligned}\\ The interest is for \\x,y\\ small and so these equations are essentially equal to \\x^{\prime }=-b\frac{c}{d}y,\\ y^{\prime }=\frac{a}{b}dx\nonumber\\

Replace \\x^{\prime }\\ with the difference quotient \\\frac{x\left( t+h\right) -x\left( t\right) }{h}\\ where \\h\\ is a small positive number and \\y^{\prime }\\ with a similar difference quotient. For example one could have \\h\\ correspond to one day or even one hour. Thus, for \\h\\ small enough, the following would seem to be a good approximation to the differential equations. \\\begin{aligned} x\left( t+h\right) &=x\left( t\right) -hb\frac{c}{d}y \\ y\left( t+h\right) &=y\left( t\right) +h\frac{a}{b}dx\end{aligned}\\ Let \\1,2,3,\cdots\\ denote the ends of discrete intervals of time having length \\h\\ chosen above. Then the above equations take the form \\\left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ =\left $$ \begin{array}{cc} 1 & - \frac{hbc}{d} \\ \frac{had}{b} & 1 \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber\\ Note that the eigenvalues of this matrix are always complex.

We are not interested in time intervals of length \\h\\ for \\h\\ very small. Instead, we are interested in much longer lengths of time. Thus, replacing the time interval with \\mh,\\ \\\left $$ \begin{array}{c} x\left( n+m\right) \\ y\left( n+m\right) \end{array} \right $$ =\left $$ \begin{array}{cc} 1 & - \frac{hbc}{d} \\ \frac{had}{b} & 1 \end{array} \right $$ ^{m}\left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber\\ For example, if \\m=2,\\ you would have \\\left $$ \begin{array}{c} x\left( n+2\right) \\ y\left( n+2\right) \end{array} \right $$ =\left $$ \begin{array}{cc} 1-ach^{2} & -2b \frac{c}{d}h \\ 2 \frac{a}{b}dh & 1-ach^{2} \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber\\ Note that most of the time, the eigenvalues of the new matrix will be complex.

You can also notice that the upper right corner will be negative by considering higher powers of the matrix. Thus letting \\1,2,3,\cdots\\ denote the ends of discrete intervals of time, the desired discrete dynamical system is of the form \\\left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ =\left $$ \begin{array}{rr} a & -b \\ c & d \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$\nonumber\\ where \\a,b,c,d\\ are positive constants and the matrix will likely have complex eigenvalues because it is a power of a matrix which has complex eigenvalues.

You can see from the above discussion that if the eigenvalues of the matrix used to define the dynamical system are less than 1 in absolute value, then the origin is stable in the sense that as \\n\rightarrow \infty ,\\ the solution converges to the origin. If either eigenvalue is larger than 1 in absolute value, then the solutions to the dynamical system will usually be unbounded, unless the initial condition is chosen very carefully. The next example exhibits the case where one eigenvalue is larger than 1 and the other is smaller than 1.

The following example demonstrates a familiar concept as a dynamical system.

##### Example \\\PageIndex{11}\\: The Fibonacci Sequence

The Fibonacci sequence is the sequence given by \\1, 1, 2, 3, 5, \cdots\nonumber\\ which is defined recursively in the form \\x\left( 0\right) =1=x\left( 1\right) ,\\ x\left( n+2\right) =x\left( n+1\right) +x\left( n\right)\nonumber\\ Show how the Fibonacci Sequence can be considered a dynamical system.

###### Solution

This sequence is extremely important in the study of reproducing rabbits. It can be considered as a dynamical system as follows. Let \\y\left( n\right) =x\left( n+1\right) .\\ Then the above recurrence relation can be written as \\\left $$ \begin{array}{c} x\left( n+1\right) \\ y\left( n+1\right) \end{array} \right $$ =\left $$ \begin{array}{rr} 0 & 1 \\ 1 & 1 \end{array} \right $$ \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$ ,\\ \left $$ \begin{array}{c} x\left( 0\right) \\ y\left( 0\right) \end{array} \right $$ =\left $$ \begin{array}{r} 1 \\ 1 \end{array} \right $$\nonumber \\

Let \\A = \left $$ \begin{array}{rr} 0 & 1 \\ 1 & 1 \end{array} \right $$\nonumber \\

The eigenvalues of the matrix \\A\\ are \\\lambda_1 = \frac{1}{2}-\frac{1}{2}\sqrt{5}\\ and \\\lambda_2 = \frac{1}{2}\sqrt{5}+\frac{1}{2}\\. The corresponding eigenvectors are, respectively, \\X_1 = \left $$ \begin{array}{c} - \frac{1}{2}\sqrt{5}- \frac{1}{2} \\ 1 \end{array} \right $$ , X_2 = \left $$ \begin{array}{c} \frac{1}{2}\sqrt{5}- \frac{1}{2} \\ 1 \end{array} \right $$\nonumber \\

You can see from a short computation that one of the eigenvalues is smaller than 1 in absolute value while the other is larger than 1 in absolute value. Now, diagonalizing \\A\\ gives us \\\left $$ \begin{array}{cc} \frac{1}{2}\sqrt{5}- \frac{1}{2} & - \frac{1}{2}\sqrt{5}- \frac{1}{2} \\ 1 & 1 \end{array} \right $$ ^{-1}\left $$ \begin{array}{rr} 0 & 1 \\ 1 & 1 \end{array} \right $$ \left $$ \begin{array}{cc} \frac{1}{2}\sqrt{5}- \frac{1}{2} & - \frac{1}{2}\sqrt{5}- \frac{1}{2} \\ 1 & 1 \end{array} \right $$\nonumber \\ \\=\left $$ \begin{array}{cc} \frac{1}{2}\sqrt{5}+ \frac{1}{2} & 0 \\ 0 & \frac{1}{2}- \frac{1}{2}\sqrt{5} \end{array} \right $$\nonumber \\

Then it follows that for a given initial condition, the solution to this dynamical system is of the form \\\begin{aligned} \left $$ \begin{array}{c} x\left( n\right) \\ y\left( n\right) \end{array} \right $$ &=\left $$ \begin{array}{cc} \frac{1}{2}\sqrt{5}- \frac{1}{2} & - \frac{1}{2}\sqrt{5}- \frac{1}{2} \\ 1 & 1 \end{array} \right $$ \left $$ \begin{array}{cc} \left( \frac{1}{2}\sqrt{5}+ \frac{1}{2}\right) ^{n} & 0 \\ 0 & \left( \frac{1}{2}- \frac{1}{2}\sqrt{5}\right) ^{n} \end{array} \right $$ \cdot \\ &\left $$ \begin{array}{cc} \frac{1}{5}\sqrt{5} & \frac{1}{10}\sqrt{5}+ \frac{1}{2} \\ - \frac{1}{5}\sqrt{5} & \frac{1}{5}\sqrt{5}\left( \frac{1}{2}\sqrt{5}- \frac{1 }{2}\right) \end{array} \right $$ \left $$ \begin{array}{r} 1 \\ 1 \end{array} \right $$\end{aligned}\\ It follows that \\x\left( n\right) =\left( \frac{1}{2}\sqrt{5}+\frac{1}{2}\right) ^{n}\left( \frac{1}{10}\sqrt{5}+\frac{1}{2}\right) +\left( \frac{1}{2}-\frac{1}{2}\sqrt{5}\right) ^{n}\left( \frac{1}{2}-\frac{1}{10}\sqrt{5}\right)\nonumber \\

Here is a picture of the ordered pairs \\\left( x\left( n\right) ,y\left( n\right) \right)\\ for \\n=0,1,\cdots ,n\\.

Figure $\PageIndex{3}$

There is so much more that can be said about dynamical systems. It is a major topic of study in differential equations and what is given above is just an introduction.

The Matrix Exponential

The goal of this section is to use the concept of the matrix exponential to solve first order linear differential equations. We begin by proving the matrix exponential.

Suppose \\A\\ is a diagonalizable matrix. Then the matrix exponential, written \\e^{A}\\, can be easily defined. Recall that if \\D\\ is a diagonal matrix, then \\P^{-1}AP=D\nonumber \\ \\D\\ is of the form \\\left $$ \begin{array}{ccc} \lambda \_{1} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n} \end{array} \right $$ \label{diagonalmatrix}\\ and it follows that \\D^{m}=\left $$ \begin{array}{ccc} \lambda \_{1}^{m} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n}^{m} \end{array} \right $$\nonumber \\

Since \\A\\ is diagonalizable, \\A=PDP^{-1}\nonumber \\ and \\A^{m}=PD^{m}P^{-1}\nonumber \\

Recall why this is true. \\A=PDP^{-1}\nonumber \\ and so \\\begin{aligned} A^{m} &=\overset{ \text{m times}}{\overbrace{PDP^{-1}PDP^{-1}PDP^{-1}\cdots PDP^{-1}}} \\ &=PD^{m}P^{-1}\end{aligned}\\

We now will examine what is meant by the matrix exponental \\e^{A}\\. Begin by formally writing the following power series for \\e^{A}\\: \\e^{A} = \sum\_{k=0}^{\infty }\frac{A^{k}}{k!}=\sum\_{k=0}^{\infty }\frac{PD^{k}P^{-1}}{k!}=P \left( \sum\_{k=0}^{\infty }\frac{D^{k}}{k!} \right)P^{-1}\nonumber \\ If \\D\\ is given above in \\\eqref{diagonalmatrix}\\, the above sum is of the form \\P \left( \sum\_{k=0}^{\infty }\left $$ \begin{array}{ccc} \frac{1}{k!}\lambda \_{1}^{k} & & 0 \\ & \ddots & \\ 0 & & \frac{1}{k!}\lambda \_{n}^{k} \end{array} \right $$ \right) P^{-1}\nonumber \\ This can be rearranged as follows: \\e^{A}=P\left $$ \begin{array}{ccc} \sum\_{k=0}^{\infty }\frac{1}{k!}\lambda \_{1}^{k} & & 0 \\ & \ddots & \\ 0 & & \sum\_{k=0}^{\infty }\frac{1}{k!}\lambda \_{n}^{k} \end{array} \right $$ P^{-1}\nonumber \\ \\=P\left $$ \begin{array}{ccc} e^{\lambda \_{1}} & & 0 \\ & \ddots & \\ 0 & & e^{\lambda \_{n}} \end{array} \right $$ P^{-1}\nonumber \\

This justifies the following theorem.

##### Theorem \\\PageIndex{4}\\: The Matrix Exponential

Let \\A\\ be a diagonalizable matrix, with eigenvalues \\\lambda_1, ..., \lambda_n\\and corresponding matrix of eigenvectors \\P\\. Then the matrix exponential, \\e^{A}\\, is given by \\e^{A} = P\left $$ \begin{array}{ccc} e^{\lambda \_{1}} & & 0 \\ & \ddots & \\ 0 & & e^{\lambda \_{n}} \end{array} \right $$ P^{-1}\nonumber \\

##### Example \\\PageIndex{12}\\: Compute \\e^A\\ for a Matrix \\A\\

Let \\A=\left $$ \begin{array}{rrr} 2 & -1 & -1 \\ 1 & 2 & 1 \\ -1 & 1 & 2 \end{array} \right $$\nonumber \\ Find \\e^{A}\\.

###### Solution

The eigenvalues work out to be \\1,2,3\\ and eigenvectors associated with these eigenvalues are \\\left $$ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right $$ \leftrightarrow 1, \left $$ \begin{array}{r} -1 \\ -1 \\ 1 \end{array} \right $$ \leftrightarrow 2,\left $$ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right $$ \leftrightarrow 3\nonumber \\ Then let \\D=\left $$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{array} \right $$, P=\left $$ \begin{array}{rrr} 0 & -1 & -1 \\ -1 & -1 & 0 \\ 1 & 1 & 1 \end{array} \right $$\nonumber \\ and so \\P^{-1}=\left $$ \begin{array}{rrr} 1 & 0 & 1 \\ -1 & -1 & -1 \\ 0 & 1 & 1 \end{array} \right $$\nonumber \\

Then the matrix exponential is \\e^{At} = \left $$ \begin{array}{rrr} 0 & -1 & -1 \\ -1 & -1 & 0 \\ 1 & 1 & 1 \end{array} \right $$ \left $$ \begin{array}{ccc} e^{1} & 0 & 0 \\ 0 & e^{2} & 0 \\ 0 & 0 & e^{3} \end{array} \right $$ \left $$ \begin{array}{rrr} 1 & 0 & 1 \\ -1 & -1 & -1 \\ 0 & 1 & 1 \end{array} \right $$\nonumber \\ \\\left $$ \begin{array}{ccc} e^{2} & e^{2}-e^{3} & e^{2}-e^{3} \\ e^{2}-e & e^{2} & e^{2}-e \\ -e^{2}+e & -e^{2}+e^{3} & -e^{2}+e+e^{3} \end{array} \right $$\nonumber \\

The matrix exponential is a useful tool to solve autonomous systems of first order linear differential equations. These are equations which are of the form \\X^{\prime }=AX, X(0) = C\nonumber \\ where \\A\\ is a diagonalizable \\n\times n\\ matrix and \\C\\ is a constant vector. \\X\\ is a vector of functions in one variable, \\t\\: \\X = X(t) = \left $$ \begin{array}{c} x_1(t) \\ x_2(t) \\ \vdots \\ x_n(t) \end{array} \right $$\nonumber \\ Then \\X^{\prime }\\ refers to the first derivative of \\X\\ and is given by \\X^{\prime} = X^{\prime}(t) = \left $$ \begin{array}{c} x_1^{\prime}(t) \\ x_2^{\prime}(t) \\ \vdots \\ x_n^{\prime}(t) \end{array} \right $$, \\ x_i^{\prime}(t) = \text{the derivative of}\\ x_i(t)\nonumber \\

Then it turns out that the solution to the above system of equations is \\X\left( t\right) =e^{At}C\\. To see this, suppose \\A\\ is diagonalizable so that \\A=P\left $$ \begin{array}{ccc} \lambda \_{1} & & \\ & \ddots & \\ & & \lambda \_{n} \end{array} \right $$ P^{-1}\nonumber \\ Then \\e^{At}=P\left $$ \begin{array}{ccc} e^{\lambda \_{1}t} & & \\ & \ddots & \\ & & e^{\lambda \_{n}t} \end{array} \right $$ P^{-1}\nonumber \\ \\e^{At}C=P\left $$ \begin{array}{ccc} e^{\lambda \_{1}t} & & \\ & \ddots & \\ & & e^{\lambda \_{n}t} \end{array} \right $$ P^{-1}C\nonumber \\

Differentiating \\e^{At}C\\ yields \\X^{\prime} = \left( e^{At}C\right) ^{\prime }=P\left $$ \begin{array}{ccc} \lambda \_{1}e^{\lambda \_{1}t} & & \\ & \ddots & \\ & & \lambda \_{n}e^{\lambda \_{n}t} \end{array} \right $$ P^{-1}C\nonumber \\ \\=P\left $$ \begin{array}{ccc} \lambda \_{1} & & \\ & \ddots & \\ & & \lambda \_{n} \end{array} \right $$ \left $$ \begin{array}{ccc} e^{\lambda \_{1}t} & & \\ & \ddots & \\ & & e^{\lambda \_{n}t} \end{array} \right $$ P^{-1}C\nonumber \\ \\\begin{aligned} &=P\left $$ \begin{array}{ccc} \lambda \_{1} & & \\ & \ddots & \\ & & \lambda \_{n} \end{array} \right $$ P^{-1}P\left $$ \begin{array}{ccc} e^{\lambda \_{1}t} & & \\ & \ddots & \\ & & e^{\lambda \_{n}t} \end{array} \right $$ P^{-1}C \\ &=A\left( e^{At}C\right) = AX\end{aligned}\\ Therefore \\X = X(t) = e^{At}C\\ is a solution to \\X^{\prime }=AX\\.

To prove that \\X(0) = C\\ if \\X(t) = e^{At}C\\: \\X(0) = e^{A0}C=P\left $$ \begin{array}{ccc} 1 & & \\ & \ddots & \\ & & 1 \end{array} \right $$ P^{-1}C=C\nonumber \\

##### Example \\\PageIndex{13}\\: Solving an Initial Value Problem

Solve the initial value problem \\\left $$ \begin{array}{c} x \\ y \end{array} \right $$ ^{\prime }=\left $$ \begin{array}{rr} 0 & -2 \\ 1 & 3 \end{array} \right $$ \left $$ \begin{array}{c} x \\ y \end{array} \right $$ ,\\ \left $$ \begin{array}{c} x(0)\\ y(0) \end{array} \right $$ =\left $$ \begin{array}{c} 1 \\ 1 \end{array} \right $$\nonumber \\

###### Solution

The matrix is diagonalizable and can be written as \\\begin{aligned} A &= PDP^{-1} \\ \left $$ \begin{array}{rr} 0 & -2 \\ 1 & 3 \end{array} \right $$ &=\left $$ \begin{array}{rr} 1 & 1 \\ -\frac{1}{2} & -1 \end{array} \right $$ \left $$ \begin{array}{rr} 1 & 0 \\ 0 & 2 \end{array} \right $$ \left $$ \begin{array}{rr} 2 & 2 \\ -1 & -2 \end{array} \right $$\end{aligned}\\ Therefore, the matrix exponential is of the form \\e^{At} = \left $$ \begin{array}{rr} 1 & 1 \\ -\frac{1}{2} & -1 \end{array} \right $$ \left $$ \begin{array}{cc} e^{t} & 0 \\ 0 & e^{2t} \end{array} \right $$ \left $$ \begin{array}{rr} 2 & 2 \\ -1 & -2 \end{array} \right $$\nonumber \\ The solution to the initial value problem is \\\begin{aligned} X(t) &= e^{At}C \\ \left $$ \begin{array}{c} x\left( t\right) \\ y\left( t\right) \end{array} \right $$ &= \left $$ \begin{array}{rr} 1 & 1 \\ -\frac{1}{2} & -1 \end{array} \right $$ \left $$ \begin{array}{cc} e^{t} & 0 \\ 0 & e^{2t} \end{array} \right $$ \left $$ \begin{array}{rr} 2 & 2 \\ -1 & -2 \end{array} \right $$ \left $$ \begin{array}{c} 1 \\ 1 \end{array} \right $$ \\ &=\left $$ \begin{array}{c} 4e^{t}-3e^{2t} \\ 3e^{2t}-2e^{t} \end{array} \right $$\end{aligned}\\ We can check that this works: \\\begin{aligned} \left $$ \begin{array}{c} x\left( 0\right) \\ y\left( 0\right) \end{array} \right $$ &= \left $$ \begin{array}{c} 4e^{0}-3e^{2(0)} \\ 3e^{2(0)}-2e^{0} \end{array} \right $$ \\ &= \left $$ \begin{array}{c} 1 \\ 1 \end{array} \right $$\end{aligned}\\

Lastly, \\X^{\prime} = \left $$ \begin{array}{c} 4e^{t}-3e^{2t} \\ 3e^{2t}-2e^{t} \end{array} \right $$ ^{\prime }=\left $$ \begin{array}{c} 4e^{t}-6e^{2t} \\ 6e^{2t}-2e^{t} \end{array} \right $$\nonumber \\ and \\AX = \left $$ \begin{array}{rr} 0 & -2 \\ 1 & 3 \end{array} \right $$ \left $$ \begin{array}{c} 4e^{t}-3e^{2t} \\ 3e^{2t}-2e^{t} \end{array} \right $$ =\left $$ \begin{array}{c} 4e^{t}-6e^{2t} \\ 6e^{2t}-2e^{t} \end{array} \right $$\nonumber \\ which is the same thing. Thus this is the solution to the initial value problem.

---

7_04_3A_Orthogonality

> 来源: LibreTexts

> 原页: https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/07%3A_Spectral_Theory/7.04%3A_Orthogonality

Skip to main content

Orthogonal Diagonalization

We begin this section by recalling some important definitions. Recall from Definition 4.11.4 that non-zero vectors are called orthogonal if their dot product equals \\0\\. A set is orthonormal if it is orthogonal and each vector is a unit vector.

An orthogonal matrix \\U\\, from Definition 4.11.7, is one in which \\UU^{T} = I\\. In other words, the transpose of an orthogonal matrix is equal to its inverse. A key characteristic of orthogonal matrices, which will be essential in this section, is that the columns of an orthogonal matrix form an orthonormal set.

We now recall another important definition.

##### Definition \\\PageIndex{1}\\: Symmetric and Skew Symmetric Matrices

A real \\n\times n\\ matrix \\A,\\ is symmetric if \\A^{T}=A.\\ If \\ A=-A^{T},\\ then \\A\\ is called skew symmetric.

Before proving an essential theorem, we first examine the following lemma which will be used below.

##### Lemma \\\PageIndex{1}\\: The Dot Product

Let \\A=\left ( a\_{ij} \right )\\ be a real symmetric \\n \times n\\ matrix, and let \\\vec{x}, \vec{y} \in \mathbb{R}^n\\. Then \\A\vec{x} \cdot \vec{y} = \vec{x} \cdot A \vec{y}\nonumber\\

Proof

This result follows from the definition of the dot product together with properties of matrix multiplication, as follows: \\\begin{aligned} A\vec{x} \cdot \vec{y} &= \sum\_{k,l}a\_{kl}x\_{l}y\_{k} \\ &=\sum\_{k,l} (a\_{lk})^Tx\_{l}y\_{k} \\ &= \vec{x}\cdot A^{T}\vec{y} \\ &= \vec{x}\cdot A \vec{y}\end{aligned}\\

The last step follows from \\A^T = A\\, since \\A\\ is symmetric.

We can now prove that the eigenvalues of a real symmetric matrix are real numbers. Consider the following important theorem.

##### Theorem \\\PageIndex{1}\\: Orthogonal Eigenvectors

Let \\A\\ be a real symmetric matrix. Then the eigenvalues of \\A\\ are real numbers and eigenvectors corresponding to distinct eigenvalues are orthogonal.

Proof

Recall that for a complex number \\a+ib,\\ the complex conjugate, denoted by \\\overline{a+ib}\\ is given by \\\overline{a+ib}=a-ib.\\ The notation, \\\overline{\vec{x}}\\ will denote the vector which has every entry replaced by its complex conjugate.

Suppose \\A\\ is a real symmetric matrix and \\A\vec{x}=\lambda \vec{x}\\. Then \\\overline{\lambda \vec{x}}^{T}\vec{x}=\left( \overline{A \vec{x}}\right) ^{T}\vec{x}=\overline{\vec{x}}^{T}A^{T}\vec{x}= \overline{\vec{x}}^{T}A\vec{x}=\lambda \overline{\vec{x}}^{T} \vec{x}\nonumber \\ Dividing by \\\overline{\vec{x}}^{T}\vec{x}\\ on both sides yields \\\overline{\lambda }=\lambda\\ which says \\\lambda\\ is real. To do this, we need to ensure that \\\overline{\vec{x}}^{T}\vec{x} \neq 0\\. Notice that \\\overline{\vec{x}}^{T}\vec{x} = 0\\ if and only if \\\vec{x} = \vec{0}\\. Since we chose \\\vec{x}\\ such that \\A\vec{x} = \lambda \vec{x}\\, \\\vec{x}\\ is an eigenvector and therefore must be nonzero.

Now suppose \\A\\ is real symmetric and \\A\vec{x}=\lambda \vec{x}\\, \\A \vec{y}=\mu \vec{y}\\ where \\\mu \neq \lambda\\. Then since \\A\\ is symmetric, it follows from Lemma $\PageIndex{1}$ about the dot product that \\\lambda \vec{x}\cdot \vec{y}=A\vec{x}\cdot \vec{y}=\vec{x}\cdot A\vec{y}=\vec{x}\cdot \mu \vec{y}=\mu \vec{x}\cdot \vec{y}\nonumber \\ Hence \\\left( \lambda -\mu \right) \vec{x}\cdot \vec{y}=0.\\ It follows that, since \\\lambda -\mu \neq 0,\\ it must be that \\\vec{x}\cdot \vec{y}=0\\. Therefore the eigenvectors form an orthogonal set.

The following theorem is proved in a similar manner.

##### Theorem \\\PageIndex{2}\\: Eigenvalues of Skew Symmetric Matrix

The eigenvalues of a real skew symmetric matrix are either equal to \\0\\ or are pure imaginary numbers.

Proof

First, note that if \\A=0\\ is the zero matrix, then \\A\\ is skew symmetric and has eigenvalues equal to \\0\\.

Suppose \\A=-A^{T}\\ so \\A\\ is skew symmetric and \\A\vec{x}=\lambda \vec{x}\\. Then \\\overline{\lambda \vec{x}}^{T}\vec{x}=\left( \overline{A \vec{x}}\right) ^{T}\vec{x}=\overline{\vec{x}}^{T}A^{T}\vec{x}=- \overline{\vec{x}}^{T}A\vec{x}=-\lambda \overline{\vec{x}}^{T} \vec{x}\nonumber \\ and so, dividing by \\\overline{\vec{x}}^{T}\vec{x}\\ as before, \\\overline{\lambda }=-\lambda .\\ Letting \\\lambda =a+ib,\\ this means \\a-ib=-a-ib\\ and so \\a=0.\\ Thus \\\lambda\\ is pure imaginary.

Consider the following example.

##### Example \\\PageIndex{1}\\: Eigenvalues of a Skew Symmetric Matrix

Let \\A=\left$$ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right$$ .\\ Find its eigenvalues.

###### Solution

First notice that \\A\\ is skew symmetric. By Theorem $\PageIndex{2}$, the eigenvalues will either equal \\0\\ or be pure imaginary. The eigenvalues of \\A\\ are obtained by solving the usual equation \\\det (\lambda I - A ) = \det \left$$ \begin{array}{rr} \lambda & 1 \\ -1 & \lambda \end{array} \right$$ =\lambda ^{2}+1=0\nonumber \\

Hence the eigenvalues are \\\pm i,\\ pure imaginary.

Consider the following example.

##### Example \\\PageIndex{2}\\: Eigenvalues of a Symmetric Matrix

Let \\A=\left$$ \begin{array}{rr} 1 & 2 \\ 2 & 3 \end{array} \right$$ .\\ Find its eigenvalues.

###### Solution

First, notice that \\A\\ is symmetric. By Theorem $\PageIndex{1}$, the eigenvalues will all be real. The eigenvalues of \\A\\ are obtained by solving the usual equation \\\det (\lambda I - A) = \det \left$$ \begin{array}{rr} \lambda - 1 & -2 \\ -2 & \lambda - 3 \end{array} \right$$ = \lambda^2 -4\lambda -1=0\nonumber \\ The eigenvalues are given by \\\lambda_1 =2+ \sqrt{5}\\ and \\\lambda_2 =2-\sqrt{5}\\ which are both real.

Recall that a diagonal matrix \\D=\left ( d\_{ij} \right )\\ is one in which \\d\_{ij} = 0\\ whenever \\i \neq j\\. In other words, all numbers not on the main diagonal are equal to zero.

Consider the following important theorem.

##### Theorem \\\PageIndex{3}\\: Orthogonal Diagonalization

Let \\A\\ be a real symmetric matrix. Then there exists an orthogonal matrix \\U\\ such that \\U^{T}AU = D\nonumber \\ where \\D\\ is a diagonal matrix. Moreover, the diagonal entries of \\D\\ are the eigenvalues of \\A\\.

We can use this theorem to diagonalize a symmetric matrix, using orthogonal matrices. Consider the following corollary.

##### Corollary \\\PageIndex{1}\\: Orthonormal Set of Eigenvectors

If \\A\\ is a real \\n\times n\\ symmetric matrix, then there exists an orthonormal set of eigenvectors, \\\left\\ \vec{u}\_{1},\cdots ,\vec{u} \_{n}\right\\ .\\

Proof

Since \\A\\ is symmetric, then by Theorem $\PageIndex{3}$, there exists an orthogonal matrix \\U\\ such that \\U^{T}AU=D,\\ a diagonal matrix whose diagonal entries are the eigenvalues of \\A.\\ Therefore, since \\A\\ is symmetric and all the matrices are real, \\\overline{D}=\overline{D^{T}}=\overline{U^{T}A^{T}U}=U^{T}A^{T}U=U^{T}AU=D\nonumber \\ showing \\D\\ is real because each entry of \\D\\ equals its complex conjugate.

Now let \\U=\left$$ \begin{array}{cccc} \vec{u}\_{1} & \vec{u}\_{2} & \cdots & \vec{u}\_{n} \end{array} \right$$\nonumber \\ where the \\\vec{u}\_{i}\\ denote the columns of \\U\\ and \\D=\left$$ \begin{array}{ccc} \lambda \_{1} & & 0 \\ & \ddots & \\ 0 & & \lambda \_{n} \end{array} \right$$\nonumber \\ The equation, \\U^{T}AU=D\\ implies \\AU = UD\\ and \\\begin{aligned} AU &=\left$$ \begin{array}{cccc} A\vec{u}\_{1} & A\vec{u}\_{2} & \cdots & A\vec{u}\_{n} \end{array} \right$$ \\ &=\left$$ \begin{array}{cccc} \lambda \_{1}\vec{u}\_{1} & \lambda \_{2}\vec{u}\_{2} & \cdots & \lambda \_{n}\vec{u}\_{n} \end{array} \right$$ \\ &= UD\end{aligned}\\ where the entries denote the columns of \\AU\\ and \\UD\\ respectively. Therefore, \\A\vec{u}\_{i}=\lambda \_{i}\vec{u}\_{i}\\. Since the matrix \\U\\ is orthogonal, the \\ij^{th}\\ entry of \\U^{T}U\\ equals \\\delta \_{ij}\\ and so \\\delta \_{ij}=\vec{u}\_{i}^{T}\vec{u}\_{j}=\vec{u}\_{i}\cdot \vec{u} \_{j}\nonumber \\ This proves the corollary because it shows the vectors \\\left\\ \vec{u} \_{i}\right\\\\ form an orthonormal set.

##### Definition \\\PageIndex{2}\\: Principal Axes

Let \\A\\ be an \\n \times n\\ matrix. Then the principal axes of \\A\\ is a set of orthonormal eigenvectors of \\A\\.

In the next example, we examine how to find such a set of orthonormal eigenvectors.

##### Example \\\PageIndex{3}\\: Find an Orthonormal Set of Eigenvectors

Find an orthonormal set of eigenvectors for the symmetric matrix \\A = \left$$ \begin{array}{rrr} 17 & -2 & -2 \\ -2 & 6 & 4 \\ -2 & 4 & 6 \end{array} \right$$\nonumber \\

###### Solution

Recall Procedure 7.1.1 for finding the eigenvalues and eigenvectors of a matrix. You can verify that the eigenvalues are \\18,9,2.\\ First find the eigenvector for \\18\\ by solving the equation \$18I-A)X = 0\\. The appropriate augmented matrix is given by \\\left$$ \begin{array}{ccc\|c} 18-17 & 2 & 2 & 0 \\ 2 & 18-6 & -4 & 0 \\ 2 & -4 & 18-6 & 0 \end{array} \right$$\nonumber \\ The reduced row-echelon form is \\\left$$ \begin{array}{rrr\|r} 1 & 0 & 4 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ Therefore an eigenvector is \\\left$$ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right$$\nonumber \\ Next find the eigenvector for \\\lambda =9.\\ The augmented matrix and resulting reduced row-echelon form are \\\left$$ \begin{array}{ccc\|c} 9-17 & 2 & 2 & 0 \\ 2 & 9-6 & -4 & 0 \\ 2 & -4 & 9-6 & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 1 & 0 & - \frac{1}{2} & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ Thus an eigenvector for \\\lambda =9\\ is \\\left$$ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right$$\nonumber \\ Finally find an eigenvector for \\\lambda =2.\\ The appropriate augmented matrix and reduced row-echelon form are \\\left$$ \begin{array}{ccc\|c} 2-17 & 2 & 2 & 0 \\ 2 & 2-6 & -4 & 0 \\ 2 & -4 & 2-6 & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 1 & 0 & 0 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ Thus an eigenvector for \\\lambda =2\\ is \\\left$$ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right$$\nonumber \\

The set of eigenvectors for \\A\\ is given by \\\left\\ \left$$ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right$$, \left$$ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right$$, \left$$ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right$$ \right\\\nonumber \\ You can verify that these eigenvectors form an orthogonal set. By dividing each eigenvector by its magnitude, we obtain an orthonormal set: \\\left\\ \frac{1}{\sqrt{18}}\left$$ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right$$ ,\frac{1}{3}\left$$ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right$$ ,\frac{1}{\sqrt{2}}\left$$ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right$$ \right\\\nonumber \\

Consider the following example.

##### Example \\\PageIndex{4}\\: Repeated Eigenvalues

Find an orthonormal set of three eigenvectors for the matrix \\A = \left$$ \begin{array}{rrr} 10 & 2 & 2 \\ 2 & 13 & 4 \\ 2 & 4 & 13 \end{array} \right$$\nonumber \\

###### Solution

You can verify that the eigenvalues of \\A\\ are \\9\\ (with multiplicity two) and \\18\\ (with multiplicity one). Consider the eigenvectors corresponding to \\\lambda =9\\. The appropriate augmented matrix and reduced row-echelon form are given by \\\left$$ \begin{array}{ccc\|c} 9-10 & -2 & -2 & 0 \\ -2 & 9-13 & -4 & 0 \\ -2 & -4 & 9-13 & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 1 & 2 & 2 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ and so eigenvectors are of the form \\\left$$ \begin{array}{c} -2y-2z \\ y \\ z \end{array} \right$$\nonumber \\ We need to find two of these which are orthogonal. Let one be given by setting \\z=0\\ and \\y=1\\, giving \\\left$$ \begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right$$\\.

In order to find an eigenvector orthogonal to this one, we need to satisfy \\\left$$ \begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right$$ \cdot \left$$ \begin{array}{c} -2y-2z \\ y \\ z \end{array} \right$$ =5y+4z=0\nonumber \\ The values \\y=-4\\ and \\z=5\\ satisfy this equation, giving another eigenvector corresponding to \\\lambda=9\\ as \\\left$$ \begin{array}{c} -2\left( -4\right) -2\left( 5\right) \\ \left( -4\right) \\ 5 \end{array} \right$$ =\left$$ \begin{array}{r} -2 \\ -4 \\ 5 \end{array} \right$$\nonumber \\ Next find the eigenvector for \\\lambda =18.\\ The augmented matrix and the resulting reduced row-echelon form are given by \\\left$$ \begin{array}{ccc\|c} 18-10 & -2 & -2 & 0 \\ -2 & 18-13 & -4 & 0 \\ -2 & -4 & 18-13 & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 1 & 0 & - \frac{1}{2} & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ and so an eigenvector is \\\left$$ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right$$\nonumber \\

Dividing each eigenvector by its length, the orthonormal set is \\\left\\ \frac{1}{\sqrt{5}} \left$$ \begin{array}{r} -2\\ 1 \\ 0 \end{array} \right$$ , \frac{\sqrt{5}}{15} \left$$ \begin{array}{r} -2 \\ -4 \\ 5 \end{array} \right$$ , \frac{1}{3}\left$$ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right$$ \right\\\nonumber \\

In the above solution, the repeated eigenvalue implies that there would have been many other orthonormal bases which could have been obtained. While we chose to take \\z=0, y=1\\, we could just as easily have taken \\y=0\\ or even \\y=z=1.\\ Any such change would have resulted in a different orthonormal set.

Recall the following definition.

##### Definition \\\PageIndex{3}\\: Diagonalizable

An \\n\times n\\ matrix \\A\\ is said to be non defective or diagonalizable if there exists an invertible matrix \\P\\ such that \\P^{-1}AP=D\\ where \\D\\ is a diagonal matrix.

As indicated in Theorem $\PageIndex{3}$ if \\A\\ is a real symmetric matrix, there exists an orthogonal matrix \\U\\ such that \\U^{T}AU=D\\ where \\D\\ is a diagonal matrix. Therefore, every symmetric matrix is diagonalizable because if \\U\\ is an orthogonal matrix, it is invertible and its inverse is \\U^{T}\\. In this case, we say that \\A\\ is orthogonally diagonalizable. Therefore every symmetric matrix is in fact orthogonally diagonalizable. The next theorem provides another way to determine if a matrix is orthogonally diagonalizable.

##### Theorem \\\PageIndex{4}\\: Orthogonally Diagonalizable

Let \\A\\ be an \\n \times n\\ matrix. Then \\A\\ is orthogonally diagonalizable if and only if \\A\\ has an orthonormal set of eigenvectors.

Recall from Corollary $\PageIndex{1}$ that every symmetric matrix has an orthonormal set of eigenvectors. In fact these three conditions are equivalent.

In the following example, the orthogonal matrix \\U\\ will be found to orthogonally diagonalize a matrix.

##### Example \\\PageIndex{5}\\: Diagonalize a Symmetric Matrix

Let \\A=\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right$$ .\\ Find an orthogonal matrix \\U\\ such that \\U^{T}AU\\ is a diagonal matrix.

###### Solution

In this case, the eigenvalues are \\2\\ (with multiplicity one) and \\1\\ (with multiplicity two). First we will find an eigenvector for the eigenvalue \\2\\. The appropriate augmented matrix and resulting reduced row-echelon form are given by \\\left$$ \begin{array}{ccc\|c} 2-1 & 0 & 0 & 0 \\ 0 & 2- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 2- \frac{3}{2} & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 1 & 0 & 0 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ and so an eigenvector is \\\left$$ \begin{array}{r} 0 \\ 1 \\ 1 \end{array} \right$$\nonumber \\ However, it is desired that the eigenvectors be unit vectors and so dividing this vector by its length gives \\\left$$ \begin{array}{c} 0 \\ \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right$$\nonumber \\ Next find the eigenvectors corresponding to the eigenvalue equal to \\1\\. The appropriate augmented matrix and resulting reduced row-echelon form are given by: \\\left$$ \begin{array}{ccc\|c} 1-1 & 0 & 0 & 0 \\ 0 & 1- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 1- \frac{3}{2} & 0 \end{array} \right$$ \rightarrow \cdots \rightarrow \left$$ \begin{array}{rrr\|r} 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right$$\nonumber \\ Therefore, the eigenvectors are of the form \\\left$$ \begin{array}{r} s \\ -t \\ t \end{array} \right$$\nonumber \\ Two of these which are orthonormal are \\\left$$ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right$$\\, choosing \\s=1\\ and \\t=0\\, and \\\left$$ \begin{array}{c} 0 \\ - \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right$$\\, letting \\s=0\\, \\t= 1\\ and normalizing the resulting vector.

To obtain the desired orthogonal matrix, we let the orthonormal eigenvectors computed above be the columns. \\\left$$ \begin{array}{rrr} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right$$\nonumber \\

To verify, compute \\U^{T}AU\\ as follows: \\U^{T}AU = \left$$ \begin{array}{rrr} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ 1 & 0 & 0 \\ 0 & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{array} \right$$ \left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right$$ \left$$ \begin{array}{rrr} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right$$\nonumber \\ \\=\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right$$ = D\nonumber \\ the desired diagonal matrix. Notice that the eigenvectors, which construct the columns of \\U\\, are in the same order as the eigenvalues in \\D\\.

We conclude this section with a Theorem that generalizes earlier results.

##### Theorem \\\PageIndex{5}\\: Triangulation of a Matrix

Let \\A\\ be an \\n \times n\\ matrix. If \\A\\ has \\n\\ real eigenvalues, then an orthogonal matrix \\U\\ can be found to result in the upper triangular matrix \\U^T A U\\.

triangulation

This Theorem provides a useful Corollary.

##### Corollary \\\PageIndex{2}\\: Determinant and Trace

Let \\A\\ be an \\n \times n\\ matrix with eigenvalues \\\lambda_1, \cdots, \lambda_n\\. Then it follows that \\\det(A)\\ is equal to the product of the \\\lambda_i\\, while \\trace(A)\\ is equal to the sum of the \\\lambda_i\\.

Proof

By Theorem $\PageIndex{5}$, there exists an orthogonal matrix \\U\\ such that \\U^TAU=P\\, where \\P\\ is an upper triangular matrix. Since \\P\\ is similar to \\A\\, the eigenvalues of \\P\\ are \\\lambda_1, \lambda_2, \ldots, \lambda_n\\. Furthermore, since \\P\\ is (upper) triangular, the entries on the main diagonal of \\P\\ are its eigenvalues, so \\\det(P)=\lambda_1 \lambda_2 \cdots \lambda_n\\ and \\trace(P)=\lambda_1 + \lambda_2 + \cdots + \lambda_n\\. Since \\P\\ and \\A\\ are similar, \\\det(A)=\det(P)\\ and \\trace(A)=trace(P)\\, and therefore the results follow.

The Singular Value Decomposition

We begin this section with an important definition.

##### Definition \\\PageIndex{4}\\: Singular Values

Let \\A\\ be an \\m\times n\\ matrix. The singular values of \\A\\ are the square roots of the positive eigenvalues of \\A^TA.\\

Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary \\m\times n\\ matrix. This decomposition is the focus of this section.

The following is a useful result that will help when computing the SVD of matrices.

##### Proposition \\\PageIndex{1}\\: Same Nonzero Eigenvalues

Let \\A\\ be an \\m \times n\\ matrix. Then \\A^TA\\ and \\AA^T\\ have the same nonzero eigenvalues.

Proof

Suppose \\A\\ is an \\m\times n\\ matrix, and suppose that \\\lambda\\ is a nonzero eigenvalue of \\A^TA\\. Then there exists a nonzero vector \\X\in \mathbb{R}^n\\ such that \\\label{nonzero} (A^TA)X=\lambda X.\\

Multiplying both sides of this equation by \\A\\ yields: \\\begin{aligned} A(A^TA)X & = A\lambda X\\ (AA^T)(AX) & = \lambda (AX).\end{aligned}\\ Since \\\lambda\neq 0\\ and \\X\neq 0_n\\, \\\lambda X\neq 0_n\\, and thus by equation \\\eqref{nonzero}\\, \$A^TA)X\neq 0_m\\; thus \\A^T(AX)\neq 0_m\\, implying that \\AX\neq 0_m\\.

Therefore \\AX\\ is an eigenvector of \\AA^T\\ corresponding to eigenvalue \\\lambda\\. An analogous argument can be used to show that every nonzero eigenvalue of \\AA^T\\ is an eigenvalue of \\A^TA\\, thus completing the proof.

Given an \\m\times n\\ matrix \\A\\, we will see how to express \\A\\ as a product \\A=U\Sigma V^T\nonumber \\ where

• \\U\\ is an \\m\times m\\ orthogonal matrix whose columns are eigenvectors of \\AA^T\\.
• \\V\\ is an \\n\times n\\ orthogonal matrix whose columns are eigenvectors of \\A^TA\\.
• \\\Sigma\\ is an \\m\times n\\ matrix whose only nonzero values lie on its main diagonal, and are the singular values of \\A\\.

How can we find such a decomposition? We are aiming to decompose \\A\\ in the following form:

\\A=U\left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$ V^T\nonumber \\ where \\\sigma\\ is of the form \\\sigma =\left$$ \begin{array}{ccc} \sigma \_{1} & & 0 \\ & \ddots & \\ 0 & & \sigma \_{k} \end{array} \right$$\nonumber \\

Thus \\A^T=V\left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$ U^T\\ and it follows that \\A^TA=V\left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$ U^TU\left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$ V^T=V\left$$ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right$$ V^T\nonumber \\ and so \\A^TAV=V\left$$ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right$$ .\\ Similarly, \\AA^TU=U\left$$ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right$$ .\\ Therefore, you would find an orthonormal basis of eigenvectors for \\AA^T\\ make them the columns of a matrix such that the corresponding eigenvalues are decreasing. This gives \\U.\\ You could then do the same for \\A^TA\\ to get \\V\\.

We formalize this discussion in the following theorem.

##### Theorem \\\PageIndex{6}\\: Singular Value Decomposition

Let \\A\\ be an \\m\times n\\ matrix. Then there exist orthogonal matrices \\U\\ and \\V\\ of the appropriate size such that \\A= U \Sigma V^T\\ where \\\Sigma\\ is of the form \\\Sigma = \left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$\nonumber \\ and \\\sigma\\ is of the form \\\sigma =\left$$ \begin{array}{ccc} \sigma \_{1} & & 0 \\ & \ddots & \\ 0 & & \sigma \_{k} \end{array} \right$$\nonumber \\ for the \\\sigma \_{i}\\ the singular values of \\A.\\

Proof

There exists an orthonormal basis, \\\left\\ \vec{v}\_{i}\right\\ \_{i=1}^{n}\\ such that \\A^TA\vec{v}\_{i}=\sigma \_{i}^{2}\vec{v}\_{i}\\ where \\\sigma \_{i}^{2}\>0\\ for \\i=1,\cdots ,k,\left( \sigma \_{i}\>0\right)\\ and equals zero if \\i\>k.\\ Thus for \\i\>k,\\ \\A\vec{v}\_{i}=\vec{0}\\ because \\A\vec{v}\_{i}\cdot A\vec{v}\_{i} = A^TA\vec{v}\_{i} \cdot \vec{v}\_{i} = \vec{0} \cdot \vec{v}\_{i} =0.\nonumber \\ For \\i=1,\cdots ,k,\\ define \\\vec{u}\_{i}\in \mathbb{R}^{m}\\ by \\\vec{u}\_{i}= \sigma \_{i}^{-1}A\vec{v}\_{i}.\nonumber \\

Thus \\A\vec{v}\_{i}=\sigma \_{i}\vec{u}\_{i}.\\ Now \\\begin{aligned} \vec{u}\_{i} \cdot \vec{u}\_{j} &= \sigma \_{i}^{-1}A \vec{v}\_{i} \cdot \sigma \_{j}^{-1}A\vec{v}\_{j} = \sigma\_{i}^{-1}\vec{v}\_{i} \cdot \sigma \_{j}^{-1}A^TA\vec{v}\_{j} \\ &= \sigma \_{i}^{-1}\vec{v}\_{i} \cdot \sigma \_{j}^{-1}\sigma \_{j}^{2} \vec{v}\_{j} = \frac{\sigma \_{j}}{\sigma \_{i}}\left( \vec{v}\_{i} \cdot \vec{v}\_{j}\right) =\delta \_{ij}.\end{aligned}\\ Thus \\\left\\ \vec{u}\_{i}\right\\ \_{i=1}^{k}\\ is an orthonormal set of vectors in \\\mathbb{R}^{m}.\\ Also, \\AA^T\vec{u}\_{i}=AA^T\sigma \_{i}^{-1}A\vec{v}\_{i}=\sigma \_{i}^{-1}AA^TA\vec{v}\_{i}=\sigma \_{i}^{-1}A\sigma \_{i}^{2}\vec{v} \_{i}=\sigma \_{i}^{2}\vec{u}\_{i}.\nonumber \\ Now extend \\\left\\ \vec{u}\_{i}\right\\ \_{i=1}^{k}\\ to an orthonormal basis for all of \\\mathbb{R}^{m},\left\\ \vec{u}\_{i}\right\\ \_{i=1}^{m}\\ and let \\U= \left$$ \begin{array}{ccc} \vec{u}\_{1} & \cdots & \vec{u}\_{m} \end{array} \right $$\nonumber \\ while \\V= \left( \vec{v}\_{1}\cdots \vec{v}\_{n}\right) .\\ Thus \\U\\ is the matrix which has the \\\vec{u}\_{i}\\ as columns and \\V\\ is defined as the matrix which has the \\\vec{v}\_{i}\\ as columns. Then \\U^TAV=\left$$ \begin{array}{c} \vec{u}\_{1}^T \\ \vdots \\ \vec{u}\_{k}^T \\ \vdots \\ \vec{u}\_{m}^T \end{array} \right$$ A\left$$ \vec{v}\_{1}\cdots \vec{v}\_{n}\right$$\nonumber \\ \\=\left$$ \begin{array}{c} \vec{u}\_{1}^T \\ \vdots \\ \vec{u}\_{k}^T \\ \vdots \\ \vec{u}\_{m}^T \end{array} \right$$ \left$$ \begin{array}{cccccc} \sigma \_{1}\vec{u}\_{1} & \cdots & \sigma \_{k}\vec{u}\_{k} & \vec{0} & \cdots & \vec{0} \end{array} \right$$ =\left$$ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right$$\nonumber \\ where \\\sigma\\ is given in the statement of the theorem.

The singular value decomposition has as an immediate corollary which is given in the following interesting result.

##### Corollary \\\PageIndex{3}\\: Rank and Singular Values

Let \\A\\ be an \\m\times n\\ matrix. Then the rank of \\A\\ and \\A^T\\equals the number of singular values.

Let’s compute the Singular Value Decomposition of a simple matrix.

##### Example \\\PageIndex{6}\\: Singular Value Decomposition

Let \\A=\left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$\\. Find the Singular Value Decomposition (SVD) of \\A\\.

###### Solution

To begin, we compute \\AA^T\\ and \\A^TA\\. \\AA^T = \left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$ \left$$\begin{array}{rr} 1 & 3 \\ -1 & 1 \\ 3 & 1 \end{array}\right$$ = \left$$\begin{array}{rr} 11 & 5 \\ 5 & 11 \end{array}\right$$.\nonumber \\

\\A^TA = \left$$\begin{array}{rr} 1 & 3 \\ -1 & 1 \\ 3 & 1 \end{array}\right$$ \left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$ = \left$$\begin{array}{rrr} 10 & 2 & 6 \\ 2 & 2 & -2\\ 6 & -2 & 10 \end{array}\right$$.\nonumber \\

Since \\AA^T\\ is \\2\times 2\\ while \\A^T A\\ is \\3\times 3\\, and \\AA^T\\ and \\A^TA\\ have the same *nonzero* eigenvalues (by Proposition $\PageIndex{1}$), we compute the characteristic polynomial \\c\_{AA^T}(x)\\ (because it’s easier to compute than \\c\_{A^TA}(x)\$.

\\\begin{aligned} c\_{AA^T}(x)& = \det(xI-AA^T)= \left\|\begin{array}{cc} x-11 & -5 \\ -5 & x-11 \end{array}\right\|\\ & = (x-11)^2 - 25 \\ & = x^2-22x+121-25\\ & = x^2-22x+96\\ & = (x-16)(x-6)\end{aligned}\\

Therefore, the eigenvalues of \\AA^T\\ are \\\lambda_1=16\\ and \\\lambda_2=6\\.

The eigenvalues of \\A^TA\\ are \\\lambda_1=16\\, \\\lambda_2=6\\, and \\\lambda_3=0\\, and the singular values of \\A\\ are \\\sigma_1=\sqrt{16}=4\\ and \\\sigma_2=\sqrt{6}\\. By convention, we list the eigenvalues (and corresponding singular values) in non increasing order (i.e., from largest to smallest).

To find the matrix \\V\\:

To construct the matrix \\V\\ we need to find eigenvectors for \\A^TA\\. Since the eigenvalues of \\AA^T\\ are distinct, the corresponding eigenvectors are orthogonal, and we need only normalize them.

\\\lambda_1=16\\: solve \$16I-A^TA)Y= 0\\. \\\left$$\begin{array}{rrr\|r} 6 & -2 & -6 & 0 \\ -2 & 14 & 2 & 0 \\ -6 & 2 & 6 & 0 \end{array}\right$$ \rightarrow \left$$\begin{array}{rrr\|r} 1 & 0 & -1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right$$, \mbox{ so } Y=\left$$\begin{array}{r} t \\ 0 \\ t \end{array}\right$$ =t\left$$\begin{array}{r} 1 \\ 0 \\ 1 \end{array}\right$$, t\in \mathbb{R}.\nonumber \\

\\\lambda_2=6\\: solve \$6I-A^TA)Y= 0\\. \\\left$$\begin{array}{rrr\|r} -4 & -2 & -6 & 0 \\ -2 & 4 & 2 & 0 \\ -6 & 2 & -4 & 0 \end{array}\right$$ \rightarrow \left$$\begin{array}{rrr\|r} 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right$$, \mbox{ so } Y=\left$$\begin{array}{r} -s \\ -s \\ s \end{array}\right$$ =s\left$$\begin{array}{r} -1 \\ -1 \\ 1 \end{array}\right$$, s\in \mathbb{R}.\nonumber \\

\\\lambda_3=0\\: solve \$-A^TA)Y= 0\\. \\\left$$\begin{array}{rrr\|r} -10 & -2 & -6 & 0 \\ -2 & -2 & 2 & 0 \\ -6 & 2 & -10 & 0 \end{array}\right$$ \rightarrow \left$$\begin{array}{rrr\|r} 1 & 0 & 1 & 0 \\ 0 & 1 & -2 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right$$, \mbox{ so } Y=\left$$\begin{array}{r} -r \\ 2r \\ r \end{array}\right$$ =r\left$$\begin{array}{r} -1 \\ 2 \\ 1 \end{array}\right$$, r\in \mathbb{R}.\nonumber \\

Let \\V_1=\frac{1}{\sqrt{2}}\left$$\begin{array}{r} 1\\ 0\\ 1 \end{array}\right$$, V_2=\frac{1}{\sqrt{3}}\left$$\begin{array}{r} -1\\ -1\\ 1 \end{array}\right$$, V_3=\frac{1}{\sqrt{6}}\left$$\begin{array}{r} -1\\ 2\\ 1 \end{array}\right$$.\nonumber \\

Then \\V=\frac{1}{\sqrt{6}}\left$$\begin{array}{rrr} \sqrt 3 & -\sqrt 2 & -1 \\ 0 & -\sqrt 2 & 2 \\ \sqrt 3 & \sqrt 2 & 1 \end{array}\right$$.\nonumber \\

Also, \\\Sigma = \left$$\begin{array}{rrr} 4 & 0 & 0 \\ 0 & \sqrt 6 & 0 \end{array}\right$$,\nonumber \\ and we use \\A\\, \\V^T\\, and \\\Sigma\\ to find \\U\\.

Since \\V\\ is orthogonal and \\A=U\Sigma V^T\\, it follows that \\AV=U\Sigma\\. Let \\V=\left$$\begin{array}{ccc} V_1 & V_2 & V_3 \end{array}\right$$\\, and let \\U=\left$$\begin{array}{cc} U_1 & U_2 \end{array}\right$$\\, where \\U_1\\ and \\U_2\\ are the two columns of \\U\\.

Then we have \\\begin{aligned} A\left$$\begin{array}{ccc} V_1 & V_2 & V_3 \end{array}\right$$ &= \left$$\begin{array}{cc} U_1 & U_2 \end{array}\right$$\Sigma\\ \left$$\begin{array}{ccc} AV_1 & AV_2 & AV_3 \end{array}\right$$ &= \left$$\begin{array}{ccc} \sigma_1U_1 + 0U_2 & 0U_1 + \sigma_2 U_2 & 0 U_1 + 0 U_2 \end{array}\right$$ \\ &= \left$$\begin{array}{ccc} \sigma_1U_1 & \sigma_2 U_2 & 0 \end{array}\right$$\end{aligned}\\ which implies that \\AV_1=\sigma_1U_1 = 4U_1\\ and \\AV_2=\sigma_2U_2 = \sqrt 6 U_2\\.

Thus, \\U_1 = \frac{1}{4}AV_1 = \frac{1}{4} \left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$ \frac{1}{\sqrt{2}}\left$$\begin{array}{r} 1\\ 0\\ 1 \end{array}\right$$ = \frac{1}{4\sqrt 2}\left$$\begin{array}{r} 4\\ 4 \end{array}\right$$ = \frac{1}{\sqrt 2}\left$$\begin{array}{r} 1\\ 1 \end{array}\right$$,\nonumber \\ and \\U_2 = \frac{1}{\sqrt 6}AV_2 = \frac{1}{\sqrt 6} \left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$ \frac{1}{\sqrt{3}}\left$$\begin{array}{r} -1\\ -1\\ 1 \end{array}\right$$ =\frac{1}{3\sqrt 2}\left$$\begin{array}{r} 3\\ -3 \end{array}\right$$ =\frac{1}{\sqrt 2}\left$$\begin{array}{r} 1\\ -1 \end{array}\right$$.\nonumber \\ Therefore, \\U=\frac{1}{\sqrt{2}}\left$$\begin{array}{rr} 1 & 1 \\ 1 & -1 \end{array}\right$$,\nonumber \\ and \\\begin{aligned} A & = \left$$\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right$$\\ & = \left(\frac{1}{\sqrt{2}}\left$$\begin{array}{rr} 1 & 1 \\ 1 & -1 \end{array}\right$$\right) \left$$\begin{array}{rrr} 4 & 0 & 0 \\ 0 & \sqrt 6 & 0 \end{array}\right$$ \left(\frac{1}{\sqrt{6}}\left$$\begin{array}{rrr} \sqrt 3 & 0 & \sqrt 3 \\ -\sqrt 2 & -\sqrt 2 & \sqrt2 \\ -1 & 2 & 1 \end{array}\right$$\right).\end{aligned}\\

Here is another example.

##### Example \\\PageIndex{7}\\: Finding the SVD

Find an SVD for \\A=\left$$\begin{array}{r} -1 \\ 2\\ 2 \end{array}\right$$\\.

###### Solution

Since \\A\\ is \\3\times 1\\, \\A^T A\\ is a \\1\times 1\\ matrix whose eigenvalues are easier to find than the eigenvalues of the \\3\times 3\\ matrix \\AA^T\\.

\\A^TA=\left$$\begin{array}{ccc} -1 & 2 & 2 \end{array}\right$$ \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$ =\left$$\begin{array}{r} 9 \end{array}\right$$.\nonumber \\

Thus \\A^TA\\ has eigenvalue \\\lambda_1=9\\, and the eigenvalues of \\AA^T\\ are \\\lambda_1=9\\, \\\lambda_2=0\\, and \\\lambda_3=0\\. Furthermore, \\A\\ has only one singular value, \\\sigma_1=3\\.

To find the matrix \\V\\: To do so we find an eigenvector for \\A^TA\\ and normalize it. In this case, finding a unit eigenvector is trivial: \\V_1=\left$$\begin{array}{r} 1 \end{array}\right$$\\, and \\V=\left$$\begin{array}{r} 1 \end{array}\right$$.\nonumber \\

Also, \\\Sigma =\left$$\begin{array}{r} 3 \\ 0\\ 0 \end{array}\right$$\\, and we use \\A\\, \\V^T\\, and \\\Sigma\\ to find \\U\\.

Now \\AV=U\Sigma\\, with \\V=\left$$\begin{array}{r} V_1 \end{array}\right$$\\, and \\U=\left$$\begin{array}{rrr} U_1 & U_2 & U_3 \end{array}\right$$\\, where \\U_1\\, \\U_2\\, and \\U_3\\ are the columns of \\U\\. Thus \\\begin{aligned} A\left$$\begin{array}{r} V_1 \end{array}\right$$ &= \left$$\begin{array}{rrr} U_1 & U_2 & U_3 \end{array}\right$$\Sigma\\ \left$$\begin{array}{r} AV_1 \end{array}\right$$ &= \left$$\begin{array}{r} \sigma_1 U_1+0U_2+0U_3 \end{array}\right$$\\ &= \left$$\begin{array}{r} \sigma_1 U_1 \end{array}\right$$\end{aligned}\\ This gives us \\AV_1=\sigma_1 U_1= 3U_1\\, so \\U_1 = \frac{1}{3}AV_1 = \frac{1}{3} \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$ \left$$\begin{array}{r} 1 \end{array}\right$$ = \frac{1}{3} \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$.\nonumber \\

The vectors \\U_2\\ and \\U_3\\ are eigenvectors of \\AA^T\\ corresponding to the eigenvalue \\\lambda_2=\lambda_3=0\\. Instead of solving the system \$0I-AA^T)X= 0\\ and then using the Gram-Schmidt process on the resulting set of two basic eigenvectors, the following approach may be used.

Find vectors \\U_2\\ and \\U_3\\ by first extending \\\\ U_1\\\\ to a basis of \\\mathbb{R}^3\\, then using the Gram-Schmidt algorithm to orthogonalize the basis, and finally normalizing the vectors.

Starting with \\\\ 3U_1 \\\\ instead of \\\\ U_1 \\\\ makes the arithmetic a bit easier. It is easy to verify that \\\left\\ \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$, \left$$\begin{array}{r} 1 \\ 0 \\ 0 \end{array}\right$$, \left$$\begin{array}{r} 0 \\ 1 \\ 0 \end{array}\right$$\right\\\nonumber \\ is a basis of \\\mathbb{R}^3\\. Set \\E_1 = \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$, X_2 = \left$$\begin{array}{r} 1 \\ 0 \\ 0 \end{array}\right$$, X_3 =\left$$\begin{array}{r} 0 \\ 1 \\ 0 \end{array}\right$$,\nonumber \\ and apply the Gram-Schmidt algorithm to \\\\ E_1, X_2, X_3\\\\.

This gives us \\E_2 = \left$$\begin{array}{r} 4 \\ 1 \\ 1 \end{array}\right$$ \mbox{ and } E_3 = \left$$\begin{array}{r} 0 \\ 1 \\ -1 \end{array}\right$$.\nonumber \\

Therefore, \\U_2 = \frac{1}{\sqrt{18}} \left$$\begin{array}{r} 4 \\ 1 \\ 1 \end{array}\right$$, U_3 = \frac{1}{\sqrt 2} \left$$\begin{array}{r} 0 \\ 1 \\ -1 \end{array}\right$$,\nonumber \\ and \\U = \left$$\begin{array}{rrr} -\frac{1}{3} & \frac{4}{\sqrt{18}} & 0 \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & \frac{1}{\sqrt 2} \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & -\frac{1}{\sqrt 2} \end{array}\right$$.\nonumber \\

Finally, \\A = \left$$\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right$$ = \left$$\begin{array}{rrr} -\frac{1}{3} & \frac{4}{\sqrt{18}} & 0 \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & \frac{1}{\sqrt 2} \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & -\frac{1}{\sqrt 2} \end{array}\right$$ \left$$\begin{array}{r} 3 \\ 0 \\ 0 \end{array}\right$$ \left$$\begin{array}{r} 1 \end{array}\right$$.\nonumber \\

Consider another example.

##### Example \\\PageIndex{8}\\: Find the SVD

Find a singular value decomposition for the matrix \\A= \left$$ \begin{array}{ccc} \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \end{array} \right$$\nonumber \\

###### Solution

First consider \\A^TA\\ \\\left$$ \begin{array}{ccc} \frac{16}{5} & \frac{32}{5} & 0 \\ \frac{32}{5} & \frac{64}{5} & 0 \\ 0 & 0 & 0 \end{array} \right$$\nonumber \\ What are some eigenvalues and eigenvectors? Some computing shows these are \\\left\\ \left$$ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right$$ ,\left$$ \begin{array}{c} -\frac{2}{5}\sqrt{5} \\ \frac{1}{5}\sqrt{5} \\ 0 \end{array} \right$$ \right\\ \leftrightarrow 0,\left\\ \left$$ \begin{array}{c} \frac{1}{5}\sqrt{5} \\ \frac{2}{5}\sqrt{5} \\ 0 \end{array} \right$$ \right\\ \leftrightarrow 16\nonumber \\ Thus the matrix \\V\\ is given by \\V=\left$$ \begin{array}{ccc} \frac{1}{5}\sqrt{5} & -\frac{2}{5}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{5} & \frac{1}{5}\sqrt{5} & 0 \\ 0 & 0 & 1 \end{array} \right$$\nonumber \\ Next consider \\AA^T\\ \\\left$$ \begin{array}{cc} 8 & 8 \\ 8 & 8 \end{array} \right$$\nonumber \\ Eigenvectors and eigenvalues are \\\left\\ \left$$ \begin{array}{c} -\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \end{array} \right$$ \right\\ \leftrightarrow 0,\left\\ \left$$ \begin{array}{c} \frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \end{array} \right$$ \right\\ \leftrightarrow 16\nonumber \\ Thus you can let \\U\\ be given by \\U=\left$$ \begin{array}{cc} \frac{1}{2}\sqrt{2} & -\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \end{array} \right$$\nonumber \\ Lets check this. \\U^TAV=\\ \\\left$$ \begin{array}{cc} \frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \\ -\frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \end{array} \right$$ \left$$ \begin{array}{ccc} \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \end{array} \right$$ \left$$ \begin{array}{ccc} \frac{1}{5}\sqrt{5} & -\frac{2}{5}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{5} & \frac{1}{5}\sqrt{5} & 0 \\ 0 & 0 & 1 \end{array} \right$$\nonumber \\ \\=\left$$ \begin{array}{ccc} 4 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right$$\nonumber \\

This illustrates that if you have a good way to find the eigenvectors and eigenvalues for a Hermitian matrix which has nonnegative eigenvalues, then you also have a good way to find the singular value decomposition of an arbitrary matrix.

Positive Definite Matrices

Positive definite matrices are often encountered in applications such mechanics and statistics.

We begin with a definition.

##### Definition \\\PageIndex{5}\\: Positive Definite Matrix

Let \\A\\ be an \\n \times n\\ symmetric matrix. Then \\A\\ is positive definite if all of its eigenvalues are positive.

The relationship between a negative definite matrix and positive definite matrix is as follows.

##### Lemma \\\PageIndex{2}\\: Negative Definite Matrix

An \\n\times n\\ matrix \\A\\ is negative definite if and only if \\-A\\ is positive definite

Consider the following lemma.

##### Lemma \\\PageIndex{3}\\: Positive Definite Matrix and Invertibility

If \\A\\ is positive definite, then it is invertible.

Proof

If \\A\vec{v}=\vec{0},\\ then \\0\\ is an eigenvalue if \\\vec{v}\\ is nonzero, which does not happen for a positive definite matrix. Hence \\\vec{v}=\vec{0}\\ and so \\A\\ is one to one. This is sufficient to conclude that it is invertible.

Notice that this lemma implies that if a matrix \\A\\ is positive definite, then \\\det(A) \> 0\\.

The following theorem provides another characterization of positive definite matrices. It gives a useful test for verifying if a matrix is positive definite.

##### Theorem \\\PageIndex{7}\\: Positive Definite Matrix

Let \\A\\ be a symmetric matrix. Then \\A\\ is positive definite if and only if \\\vec{x}^T A \vec{x}\\ is positive for all nonzero \\\vec{x} \in \mathbb{R}^n\\.

Proof

Since \\A\\ is symmetric, there exists an orthogonal matrix \\U\\ so that \\U^{T}AU=diag(\lambda_1,\lambda_2,\ldots,\lambda_n)=D,\nonumber \\ where \\\lambda_1,\lambda_2,\ldots,\lambda_n\\ are the (not necessarily distinct) eigenvalues of \\A\\. Let \\\vec{x}\in\mathbb{R}^n\\, \\\vec{x}\neq \vec{0}\\, and define \\\vec{y}=U^T\vec{x}\\. Then \\\vec{x}^TA\vec{x}=\vec{x}^T(UDU^T)\vec{x} = (\vec{x}^TU)D(U^T\vec{x}) =\vec{y}^TD\vec{y}.\nonumber \\

Writing \\\vec{y}^T=\left$$\begin{array}{cccc} y_1 & y_2 & \cdots & y_n\end{array}\right$$\\, \\\begin{aligned} \vec{x}^TA\vec{x} & = \left$$\begin{array}{cccc} y_1 & y_2 & \cdots & y_n\end{array}\right$$ diag(\lambda_1,\lambda_2,\ldots,\lambda_n) \left$$\begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_n\end{array}\right$$\\ & = \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots \lambda_n y_n^2.\end{aligned}\\

\$\Rightarrow)\\ First we will assume that \\A\\ is positive definite and prove that \\\vec{x}^T A \vec{x}\\ is positive.

Suppose \\A\\ is positive definite, and \\\vec{x}\in\mathbb{R}^n\\, \\\vec{x}\neq\vec{0}\\. Since \\U^T\\ is invertible, \\\vec{y}=U^T\vec{x}\neq \vec{0}\\, and thus \\y_j\neq 0\\ for some \\j\\, implying \\y_j^2\>0\\ for some \\j\\. Furthermore, since all eigenvalues of \\A\\ are positive, \\\lambda_i y_i^2\geq 0\\ for all \\i\\ and \\\lambda_jy_j^2\>0\\. Therefore, \\\vec{x}^TA\vec{x}\>0\\.

\$\Leftarrow)\\ Now we will assume \\\vec{x}^T A \vec{x}\\ is positive and show that \\A\\ is positive definite.

If \\\vec{x}^TA\vec{x}\>0\\ whenever \\\vec{x}\neq \vec{0}\\, choose \\\vec{x}=U\vec{e}\_j\\, where \\\vec{e}\_j\\ is the \\j^{\mbox{th}}\\ column of \\I_n\\. Since \\U\\ is invertible, \\\vec{x}\neq\vec{0}\\, and thus \\\vec{y}=U^T\vec{x}=U^T(U\vec{e}\_j) =\vec{e}\_j.\nonumber \\ Thus \\y_j=1\\ and \\y_i=0\\ when \\i\neq j\\, so \\\lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots \lambda_n y_n^2 =\lambda_j,\nonumber \\ i.e., \\\lambda_j=\vec{x}^TA\vec{x}\>0\\. Therefore, \\A\\ is positive definite.

There are some other very interesting consequences which result from a matrix being positive definite. First one can note that the property of being positive definite is transferred to each of the principal submatrices which we will now define.

##### Definition \\\PageIndex{6}\\: The Submatrix \\A_k\\

Let \\A\\ be an \\n\times n\\ matrix. Denote by \\A\_{k}\\ the \\k\times k\\ matrix obtained by deleting the \\k+1,\cdots ,n\\ columns and the \\k+1,\cdots ,n\\ rows from \\A.\\ Thus \\A\_{n}=A\\ and \\A\_{k}\\ is the \\k\times k\\ submatrix of \\A\\ which occupies the upper left corner of \\A.\\

##### Lemma \\\PageIndex{4}\\: Positive Definite and Submatrices

Let \\A\\ be an \\n\times n\\ positive definite matrix. Then each submatrix \\A\_{k}\\ is also positive definite.

Proof

This follows right away from the above definition. Let \\\vec{x}\in \mathbb{R}^{k}\\ be nonzero. Then \\\vec{x}^{T}A\_{k}\vec{x}=\left$$ \begin{array}{cc} \vec{x}^{T} & 0 \end{array} \right$$ A\left$$ \begin{array}{c} \vec{x} \\ 0 \end{array} \right$$ \> 0\nonumber \\ by the assumption that \\A\\ is positive definite.

There is yet another way to recognize whether a matrix is positive definite which is described in terms of these submatrices. We state the result, the proof of which can be found in more advanced texts.

##### Theorem \\\PageIndex{8}\\: Positive Matrix and Determinant of \\A_k\\

Let \\A\\ be a symmetric matrix. Then \\A\\ is positive definite if and only if \\\det \left( A\_{k}\right)\\ is greater than \\0\\ for every submatrix \\A\_{k}\\, \\k=1,\cdots ,n\\.

Proof

We prove this theorem by induction on \\n.\\ It is clearly true if \\n=1.\\ Suppose then that it is true for \\n-1\\ where \\n\geq 2\\. Since \\\det \left( A\right) =\det \left( A\_{n}\right) \>0,\\ it follows that all the eigenvalues are nonzero. We need to show that they are all positive. Suppose not. Then there is some even number of them which are negative, even because the product of all the eigenvalues is known to be positive, equaling \\\det \left( A\right)\\. Pick two, \\\lambda \_{1}\\ and \\\lambda \_{2}\\ and let \\A \vec{u}\_{i}=\lambda \_{i}\vec{u}\_{i}\\ where \\\vec{u}\_{i}\neq \vec{0}\\ for \\i=1,2\\ and \\\vec{u}\_{1}\cdot \vec{u}\_{2}=0.\\ Now if \\\vec{y}\equiv \alpha \_{1}\vec{u}\_{1}+\alpha \_{2}\vec{u}\_{2}\\ is an element of \\span\left\\ \vec{u}\_{1},\vec{u}\_{2}\right\\ ,\\ then since these are eigenvalues and \\\\ \vec{u}\_{1}\cdot \vec{u}\_{2}=0,\\ a short computation shows \\\left( \alpha \_{1}\vec{u}\_{1}+\alpha \_{2}\vec{u}\_{2}\right) ^{T}A\left( \alpha \_{1}\vec{u}\_{1}+\alpha \_{2}\vec{u}\_{2}\right)\nonumber \\ \\=\left\vert \alpha \_{1}\right\vert ^{2}\lambda \_{1} \vec{u} \_{1} ^{2}+\left\vert \alpha \_{2}\right\vert ^{2}\lambda \_{2}\vec{u}\_{2}^{2}\<0.\nonumber \\ Now letting \\\vec{x}\in \mathbb{R}^{n-1},\\ we can use the induction hypothesis to write \\\left$$ \begin{array}{cc} x^{T} & 0 \end{array} \right$$ A\left$$ \begin{array}{c} \vec{x} \\ 0 \end{array} \right$$ =\vec{x}^{T}A\_{n-1}\vec{x}\>0.\nonumber \\ Now the dimension of \\\left\\ \vec{z}\in \mathbb{R}^{n}:z\_{n}=0\right\\\\ is \\n-1\\ and the dimension of \\span\left\\ \vec{u}\_{1},\vec{u} \_{2}\right\\ =2\\ and so there must be some nonzero \\\vec{x}\in \mathbb{R} ^{n}\\ which is in both of these subspaces of \\\mathbb{R}^{n}\\. However, the first computation would require that \\\vec{x}^{T}A\vec{x}\<0\\ while the second would require that \\\vec{x}^{T}A\vec{x}\>0.\\ This contradiction shows that all the eigenvalues must be positive. This proves the if part of the theorem. The converse can also be shown to be correct, but it is the direction which was just shown which is of most interest.

##### Corollary \\\PageIndex{4}\\: Symmetric and Negative Definite Matrix

Let \\A\\ be symmetric. Then \\A\\ is negative definite if and only if \\\left( -1\right) ^{k} \det \left( A\_{k}\right) \>0\nonumber \\ for every \\k=1,\cdots ,n\\.

Proof

This is immediate from the above theorem when we notice, that \\A\\ is negative definite if and only if \\-A\\ is positive definite. Therefore, if \\\det \left( -A\_{k}\right) \>0\\ for all \\k=1,\cdots ,n,\\ it follows that \\A\\ is negative definite. However, \\\det \left( -A\_{k}\right) =\left( -1\right) ^{k}\det \left( A\_{k}\right) .\\

The Cholesky Factorization

Another important theorem is the existence of a specific factorization of positive definite matrices. It is called the Cholesky Factorization and factors the matrix into the product of an upper triangular matrix and its transpose.

##### Theorem \\\PageIndex{9}\\: Cholesky Factorization

Let \\A\\ be a positive definite matrix. Then there exists an upper triangular matrix \\U\\ whose main diagonal entries are positive, such that \\A\\ can be written \\A= U^TU\nonumber \\ This factorization is unique.

The process for finding such a matrix \\U\\ relies on simple row operations.

##### Procedure \\\PageIndex{1}\\: Finding the Cholesky Factorization

Let \\A\\ be a positive definite matrix. The matrix \\U\\ that creates the Cholesky Factorization can be found through two steps.

1. Using only type \\3\\ elementary row operations (multiples of rows added to other rows) put \\A\\ in upper triangular form. Call this matrix \\\hat{U}\\. Then \\\hat{U}\\ has positive entries on the main diagonal.

2. Divide each row of \\\hat{U}\\ by the square root of the diagonal entry in that row. The result is the matrix \\U\\.

Of course you can always verify that your factorization is correct by multiplying \\U\\ and \\U^T\\ to ensure the result is the original matrix \\A\\.

Consider the following example.

##### Example \\\PageIndex{9}\\: Cholesky Factorization

Show that \\A=\left$$\begin{array}{rrr} 9 & -6 & 3 \\ -6 & 5 & -3 \\ 3 & -3 & 6 \end{array}\right$$\\ is positive definite, and find the Cholesky factorization of \\A\\.

###### Solution

First we show that \\A\\ is positive definite. By Theorem $\PageIndex{8}$ it suffices to show that the determinant of each submatrix is positive. \\A\_{1}=\left$$\begin{array}{c} 9 \end{array}\right$$ \mbox{ and } A\_{2}=\left$$\begin{array}{rr} 9 & -6 \\ -6 & 5 \end{array}\right$$,\nonumber \\ so \\\det(A\_{1})=9\\ and \\\det(A\_{2})=9\\. Since \\\det(A)=36\\, it follows that \\A\\ is positive definite.

Now we use Procedure $\PageIndex{1}$ to find the Cholesky Factorization. Row reduce (using only type \\3\\ row operations) until an upper triangular matrix is obtained. \\\left$$\begin{array}{rrr} 9 & -6 & 3 \\ -6 & 5 & -3 \\ 3 & -3 & 6 \end{array}\right$$ \rightarrow \left$$\begin{array}{rrr} 9 & -6 & 3 \\ 0 & 1 & -1 \\ 0 & -1 & 5 \end{array}\right$$ \rightarrow \left$$\begin{array}{rrr} 9 & -6 & 3 \\ 0 & 1 & -1 \\ 0 & 0 & 4 \end{array}\right$$\nonumber \\

Now divide the entries in each row by the square root of the diagonal entry in that row, to give \\U=\left$$\begin{array}{rrr} 3 & -2 & 1 \\ 0 & 1 & -1 \\ 0 & 0 & 2 \end{array}\right$$\nonumber \\

You can verify that \\U^TU = A\\.

##### Example \\\PageIndex{10}\\: Cholesky Factorization

Let \\A\\ be a positive definite matrix given by \\\left$$ \begin{array}{ccc} 3 & 1 & 1 \\ 1 & 4 & 2 \\ 1 & 2 & 5 \end{array} \right$$\nonumber \\ Determine its Cholesky factorization.

###### Solution

You can verify that \\A\\ is in fact positive definite.

To find the Cholesky factorization we first row reduce to an upper triangular matrix. \\\left$$ \begin{array}{ccc} 3 & 1 & 1 \\ 1 & 4 & 2 \\ 1 & 2 & 5 \end{array} \right$$ \rightarrow \left$$ \begin{array}{ccc} 3 & 1 & 1 \\ 0 & \frac{11}{3} & \frac{5}{3} \\ 0 & \frac{5}{3} & \frac{14}{5} \end{array} \right$$ \rightarrow \left$$ \begin{array}{ccc} 3 & 1 & 1 \\ 0 & \frac{11}{3} & \frac{5}{3} \\ 0 & 0 & \frac{43}{11} \end{array} \right$$\nonumber \\

Now divide the entries in each row by the square root of the diagonal entry in that row and simplify. \\U = \left$$ \begin{array}{ccc} \sqrt{3} & \frac{1}{3}\sqrt{3} & \frac{1}{3}\sqrt{3} \\ 0 & \frac{1}{3}\sqrt{3}\sqrt{11} & \frac{5}{33}\sqrt{3}\sqrt{11} \\ 0 & 0 & \frac{1}{11}\sqrt{11}\sqrt{43} \end{array} \right$$\nonumber \\

QR Factorization

In this section, a reliable factorization of matrices is studied. Called the \\QR\\ factorization of a matrix, it *always* exists. While much can be said about the \\QR\\ factorization, this section will be limited to real matrices. Therefore we assume the dot product used below is the usual dot product. We begin with a definition.

##### Definition \\\PageIndex{7}\\: \\QR\\Factorization

Let \\A\\ be a real \\m\times n\\ matrix. Then a \\QR\\ factorization of \\A\\ consists of two matrices, \\Q\\ orthogonal and \\R\\ upper triangular, such that \\A=QR.\\

qrfactorization

The following theorem claims that such a factorization exists.

##### Theorem \\\PageIndex{10}\\: Existence of \\QR\\Factorization

Let \\A\\ be any real \\m\times n\\ matrix with linearly independent columns. Then there exists an orthogonal matrix \\Q\\ and an upper triangular matrix \\R\\ having non-negative entries on the main diagonal such that \\A=QR\nonumber\\

The procedure for obtaining the \\QR\\ factorization for any matrix \\A\\ is as follows.

##### Procedure \\\PageIndex{2}\\: \\QR\\Factorization

Let \\A\\ be an \\m \times n\\ matrix given by \\A = \left$$ \begin{array}{cccc} A_1 & A_2 & \cdots & A_n \end{array} \right$$\\ where the \\A_i\\ are the linearly independent columns of \\A\\.

1. Apply the Gram-Schmidt Process 4.11.1 to the columns of \\A\\, writing \\B_i\\ for the resulting columns.

2. Normalize the \\B_i\\, to find \\C_i = \frac{1}{ B_i } B_i\\.

3. Construct the orthogonal matrix \\Q\\ as \\Q=\left$$ \begin{array}{cccc} C_1 & C_2 & \cdots & C_n \end{array} \right$$\\.

4. Construct the upper triangular matrix \\R\\ as \\R = \left$$ \begin{array}{ccccc} B_1 & A_2 \cdot C_1 & A_3 \cdot C_1 & \cdots & A_n \cdot C_1 \\ 0 & B_2 & A_3 \cdot C_2 & \cdots & A_n \cdot C_2 \\ 0 & 0 & B_3 & \cdots & A_n \cdot C_3 \\ \vdots & \vdots & \vdots & & \vdots \\ 0 & 0 & 0 & \cdots & B_n \end{array} \right$$\nonumber \\

5. Finally, write \\A=QR\\ where \\Q\\ is the orthogonal matrix and \\R\\ is the upper triangular matrix obtained above.

Notice that \\Q\\ is an orthogonal matrix as the \\C_i\\ form an orthonormal set. Since \\ B_i \> 0\\ for all \\i\\ (since the length of a vector is always positive), it follows that \\R\\ is an upper triangular matrix with positive entries on the main diagonal.

Consider the following example.

##### Example \\\PageIndex{11}\\: Finding a \\QR\\Factorization

Let \\A = \left$$ \begin{array}{rr} 1 & 2 \\ 0 & 1 \\ 1 & 0 \end{array} \right$$\nonumber \\ Find an orthogonal matrix \\Q\\ and upper triangular matrix \\R\\ such that \\A=QR\\.

###### Solution

First, observe that \\A_1\\, \\A_2\\, the columns of \\A\\, are linearly independent. Therefore we can use the Gram-Schmidt Process to create a corresponding orthogonal set \\\left\\ B_1, B_2 \right\\\\ as follows: \\\begin{aligned} B_1 &= A_1 = \left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$ \\ B_2 &= A_2 - \frac{A_2 \cdot B_1}{ B_1 ^2} B_1 \\ &= \left$$ \begin{array}{r} 2 \\ 1 \\ 0 \end{array} \right$$ - \frac{2}{2} \left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$ \\ &= \left$$ \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \right$$\end{aligned}\\

Normalize each vector to create the set \\\left\\ C_1, C_2 \right\\\\ as follows: \\\begin{aligned} C_1 &= \frac{1}{ B_1 } B_1 = \frac{1}{\sqrt{2}} \left$$ \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \right$$ \\ C_2 &= \frac{1}{ B_2 } B_2 = \frac{1}{\sqrt{3}} \left$$ \begin{array}{r} 1 \\ 1 \\ -1 \end{array}\right$$\end{aligned}\\

Now construct the orthogonal matrix \\Q\\ as \\\begin{aligned} Q &= \left$$ \begin{array}{cccc} C_1 & C_2 & \cdots & C_n \end{array} \right$$ \\ &= \left$$ \begin{array}{rr} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{3}} \\ 0 & \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{2}} & - \frac{1}{\sqrt{3}} \end{array} \right$$\end{aligned}\\

Finally, construct the upper triangular matrix \\R\\ as \\\begin{aligned} R &= \left$$ \begin{array}{cc} B_1 & A_2 \cdot C_1 \\ 0 & B_2 \end{array} \right$$ \\ &= \left$$ \begin{array}{cc} \sqrt{2} & \sqrt{2} \\ 0 & \sqrt{3} \\ \end{array} \right$$\end{aligned}\\

It is left to the reader to verify that \\A=QR\\.

The \\QR\\ Factorization and Eigenvalues

The \\QR\\ factorization of a matrix has a very useful application. It turns out that it can be used repeatedly to estimate the eigenvalues of a matrix. Consider the following procedure.

##### Procedure \\\PageIndex{3}\\: Using the \\QR\\ Factorization to Estimate Eigenvalues

Let \\A\\ be an invertible matrix. Define the matrices \\A_1, A_2, \cdots\\ as follows:

1. \\A_1 = A\\ factored as \\A_1 = Q_1R_1\\

2. \\A_2 = R_1Q_1\\ factored as \\A_2 = Q_2R_2\\

3. \\A_3 = R_2Q_2\\ factored as \\A_3 = Q_3R_3\\

Continue in this manner, where in general \\A_k = Q_kR_k\\ and \\A\_{k+1} = R_kQ_k\\.

Then it follows that this sequence of \\A_i\\ converges to an upper triangular matrix which is similar to \\A\\. Therefore the eigenvalues of \\A\\ can be approximated by the entries on the main diagonal of this upper triangular matrix.

Power Methods

While the \\QR\\ algorithm can be used to compute eigenvalues, there is a useful and fairly elementary technique for finding the eigenvector and associated eigenvalue nearest to a given complex number which is called the *shifted inverse power method*. It tends to work extremely well provided you start with something which is fairly close to an eigenvalue.

Power methods are based the consideration of powers of a given matrix. Let \\\left\\ \vec{x}\_{1},\cdots ,\vec{x}\_{n}\right\\\\ be a basis of eigenvectors for \\\mathbb{C}^{n}\\ such that \\A\vec{x}\_{n}=\lambda \_{n}\vec{x}\_{n}.\\ Now let \\\vec{u}\_{1}\\ be some nonzero vector. Since \\\left\\ \vec{x}\_{1},\cdots ,\vec{x}\_{n}\right\\\\ is a basis, there exists unique scalars, \\c\_{i}\\ such that \\\vec{u}\_{1}=\sum\_{k=1}^{n}c\_{k}\vec{x}\_{k}\nonumber \\ Assume you have not been so unlucky as to pick \\\vec{u}\_{1}\\ in such a way that \\c\_{n}=0.\\ Then let \\A\vec{u}\_{k}=\vec{u}\_{k+1}\\ so that \\\vec{u}\_{m}=A^{m}\vec{u}\_{1}=\sum\_{k=1}^{n-1}c\_{k}\lambda \_{k}^{m}\vec{x} \_{k}+\lambda \_{n}^{m}c\_{n}\vec{x}\_{n}. \label{20maye1}\\ For large \\m\\ the last term, \\\lambda \_{n}^{m}c\_{n}\vec{x}\_{n},\\ determines quite well the direction of the vector on the right. This is because \\\left\vert \lambda \_{n}\right\vert\\ is larger than \\\left\vert \lambda \_{k}\right\vert\\ for \\k\scaling factor. Then \\\vec{u}\_{2}\\ will not be just \\A\vec{u}\_{1}\\ but \\A\vec{u}\_{1}/S\_{2}.\\ Next let \\S\_{3}\\ denote the entry of \\A\vec{u}\_{2}\\ which has largest absolute value and define \\\vec{u}\_{3}\equiv A\vec{u}\_{2}/S\_{3}.\\ Continue this way. The scaling just described does not destroy the relative insignificance of the term involving a sum in \\\eqref{20maye1}\\. Indeed it amounts to nothing more than changing the units of length. Also note that from this scaling procedure, the absolute value of the largest element of \\\vec{u}\_{k}\\ is always equal to 1. Therefore, for large \\m,\\ \\\vec{u}\_{m}= \frac{\lambda \_{n}^{m}c\_{n}\vec{x}\_{n}}{S\_{2}S\_{3}\cdots S\_{m}}+\left( \text{relatively insignificant term}\right) .\nonumber \\ Therefore, the entry of \\A\vec{u}\_{m}\\ which has the largest absolute value is essentially equal to the entry having largest absolute value of \\A\left( \frac{\lambda \_{n}^{m}c\_{n}\vec{x}\_{n}}{S\_{2}S\_{3}\cdots S\_{m}}\right) = \frac{\lambda \_{n}^{m+1}c\_{n} \vec{x}\_{n}}{S\_{2}S\_{3}\cdots S\_{m}}\approx \lambda \_{n}\vec{u}\_{m}\nonumber \\ and so for large \\m,\\ it must be the case that \\\lambda \_{n}\approx S\_{m+1}.\\ This suggests the following procedure.

##### Procedure \\\PageIndex{4}\\: Finding the Largest Eigenvalue with its Eigenvector

1. Start with a vector \\\vec{u}\_{1}\\ which you hope has a component in the direction of \\\vec{x}\_{n}.\\ The vector \\\left( 1,\cdots ,1\right) ^{T}\\ is usually a pretty good choice.

2. If \\\vec{u}\_{k}\\ is known, \\\vec{u}\_{k+1}=\frac{A\vec{u}\_{k}}{S\_{k+1}}\nonumber \\ where \\S\_{k+1}\\ is the entry of \\A\vec{u}\_{k}\\ which has largest absolute value.

3. When the scaling factors, \\S\_{k}\\ are not changing much, \\S\_{k+1}\\ will be close to the eigenvalue and \\\vec{u}\_{k+1}\\ will be close to an eigenvector.

4. Check your answer to see if it worked well.

The shifted inverse power method involves finding the eigenvalue closest to a given complex number along with the associated eigenvalue. If \\\mu\\ is a complex number and you want to find \\\lambda\\ which is closest to \\\mu ,\\ you could consider the eigenvalues and eigenvectors of \\\left( A-\mu I\right) ^{-1}\\. Then \\A\vec{x}=\lambda \vec{x}\\ if and only if \\\left( A-\mu I\right) \vec{x}=\left( \lambda -\mu \right) \vec{x}\nonumber \\ If and only if \\\frac{1}{\lambda -\mu }\vec{x}=\left( A-\mu I\right) ^{-1}\vec{x}\nonumber \\ Thus, if \\\lambda\\ is the closest eigenvalue of \\A\\ to \\\mu\\ then out of all eigenvalues of \\\left( A-\mu I\right) ^{-1},\\ you would have \\\frac{1}{ \lambda -\mu }\\ would be the largest. Thus all you have to do is apply the power method to \\\left( A-\mu I\right) ^{-1}\\ and the eigenvector you get will be the eigenvector which corresponds to \\\lambda\\ where \\\lambda\\ is the closest to \\\mu\\ of all eigenvalues of \\A\\. You could use the eigenvector to determine this directly.

##### Example \\\PageIndex{12}\\: Finding Eigenvalue and Eigenvector

Find the eigenvalue and eigenvector for \\\left$$ \begin{array}{rrr} 3 & 2 & 1 \\ -2 & 0 & -1 \\ -2 & -2 & 0 \end{array} \right$$\nonumber \\ which is closest to \\.9+.9i\\.

###### Solution

\\\left ( \left$$ \begin{array}{rrr} 3 & 2 & 1 \\ -2 & 0 & -1 \\ -2 & -2 & 0 \end{array} \right$$ - (.9+.9i)\left$$ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right$$ \right )^{-1}\nonumber \\

\\= \left$$ \begin{array}{ccc} -0.619\\19-10.\\ 545i & -5.\\ 524\\9-4.\\ 972\\4i & -0.370\\57-5.\\ 821\\3i \\ 5.\\ 524\\9+4.\\ 972\\4i & 5.\\ 276\\2+0.248\\62i & 2.\\ 762\\4+2.\\ 486\\2i \\ 0.741\\14+11.\\ 643i & 5.\\ 524\\9+4.\\ 972\\4i & 0.492\\52+6.\\ 918\\9i \end{array} \right$$\nonumber \\

Then pick an initial guess an multiply by this matrix raised to a large power. \\= \left$$ \begin{array}{ccc} -0.619\\19-10.\\ 545i & -5.\\ 524\\9-4.\\ 972\\4i & -0.370\\57-5.\\ 821\\3i \\ 5.\\ 524\\9+4.\\ 972\\4i & 5.\\ 276\\2+0.248\\62i & 2.\\ 762\\4+2.\\ 486\\2i \\ 0.741\\14+11.\\ 643i & 5.\\ 524\\9+4.\\ 972\\4i & 0.492\\52+6.\\ 918\\9i \end{array} \right$$^{15}\left$$ \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right$$\nonumber \\

This equals \\\left$$ \begin{array}{c} 1.\\ 562\\9\times 10^{13}-3.\\ 899\\3\times 10^{12}i \\ -5.\\ 864\\5\times 10^{12}+9.\\ 764\\2\times 10^{12}i \\ -1.\\ 562\\9\times 10^{13}+3.\\ 899\\9\times 10^{12}i \end{array} \right$$\nonumber \\ Now divide by an entry to make the vector have reasonable size. This yields \\\left$$ \begin{array}{c} -0.999\\99-3.\\ 614\\0\times 10^{-5}i \\ 0.499\\99-0.499\\99i \\ 1.0 \end{array} \right$$\nonumber\\ which is close to \\\left$$ \begin{array}{c} -1 \\ 0.5-0.5i \\ 1.0 \end{array} \right$$\nonumber\\ Then \\\left$$ \begin{array}{rrr} 3 & 2 & 1 \\ -2 & 0 & -1 \\ -2 & -2 & 0 \end{array} \right$$ \left$$ \begin{array}{c} -1 \\ 0.5-0.5i \\ 1.0 \end{array} \right$$ =\left$$ \begin{array}{c} -1.0-1.0i \\ 1.0 \\ 1.0+1.0i \end{array} \right$$\nonumber \\ Now to determine the eigenvalue, you could just take the ratio of corresponding entries. Pick the two corresponding entries which have the largest absolute values. In this case, you would get the eigenvalue is \\1+i\\ which happens to be the exact eigenvalue. Thus an eigenvector and eigenvalue are \\\left$$ \begin{array}{c} -1 \\ 0.5-0.5i \\ 1.0 \end{array} \right$$, 1+i\nonumber \\

Usually it won’t work out so well but you can still find what is desired. Thus, once you have obtained approximate eigenvalues using the \\QR\\ algorithm, you can find the eigenvalue more exactly along with an eigenvector associated with it by using the shifted inverse power method.

Quadratic Forms

One of the applications of orthogonal diagonalization is that of quadratic forms and graphs of level curves of a quadratic form. This section has to do with rotation of axes so that with respect to the new axes, the graph of the level curve of a quadratic form is oriented parallel to the coordinate axes. This makes it much easier to understand. For example, we all know that \\x_1^2 + x_2^2=1\\ represents the equation in two variables whose graph in \\\mathbb{R}^2\\ is a circle of radius \\1\\. But how do we know what the graph of the equation \\5x_1^2 + 4x_1x_2 + 3x_2^2=1\\ represents?

We first formally define what is meant by a quadratic form. In this section we will work with only *real* quadratic forms, which means that the coefficients will all be real numbers.

##### Definition \\\PageIndex{8}\\: Quadratic Form

A quadratic form is a polynomial of degree two in \\n\\ variables \\x_1, x_2, \cdots, x_n\\, written as a linear combination of \\x_i^{2}\\ terms and \\x_ix_j\\ terms.

Consider the quadratic form \\q = a\_{11}x_1^2 + a\_{22}x_2^2 + \cdots + a\_{nn}x_n^2 + a\_{12}x_1x_2 + \cdots\\. We can write \\\vec{x} = \left$$ \begin{array}{r} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right$$\\ as the vector whose entries are the variables contained in the quadratic form.

Similarly, let \\A = \left$$ \begin{array}{rrrr} a\_{11} & a\_{12} & \cdots & a\_{1n} \\ a\_{21} & a\_{22} & \cdots & a\_{2n} \\ \vdots & \vdots & & \vdots \\ a\_{n1} & a\_{n2} & \cdots & a\_{nn} \end{array} \right$$\\ be the matrix whose entries are the coefficients of \\x_i^2\\ and \\x_ix_j\\ from \\q\\. Note that the matrix \\A\\ is not unique, and we will consider this further in the example below. Using this matrix \\A\\, the quadratic form can be written as \\q = \vec{x}^T A \vec{x}\\.

\\\begin{aligned} q &= \vec{x}^T A \vec{x} \\ &= \left$$ \begin{array}{rrrr} x_1 & x_2 & \cdots & x_n \end{array} \right$$ \left$$ \begin{array}{rrrr} a\_{11} & a\_{12} & \cdots & a\_{1n} \\ a\_{21} & a\_{22} & \cdots & a\_{2n} \\ \vdots & \vdots & & \vdots \\ a\_{n1} & a\_{n2} & \cdots & a\_{nn} \end{array} \right$$ \left$$ \begin{array}{r} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right$$ \\ &= \left$$ \begin{array}{rrrr} x_1 & x_2 & \cdots & x_n \end{array} \right$$ \left$$ \begin{array}{c} a\_{11}x_1 + a\_{21}x_2 + \cdots + a\_{n1}x_n \\ a\_{12}x_1 + a\_{22}x_2 + \cdots + a\_{n2}x_n \\ \vdots \\ a\_{1n}x_1 + a\_{2n}x_2 + \cdots + a\_{nn}x_n \end{array} \right$$ \\ &= a\_{11}x_1^2 + a\_{22}x_2^2 + \cdots + a\_{nn}x_n^2 + a\_{12}x_1x_2 + \cdots\end{aligned}\\

Let’s explore how to find this matrix \\A\\. Consider the following example.

##### Example \\\PageIndex{13}\\: Matrix of a Quadratic Form

Let a quadratic form \\q\\ be given by \\q = 6x_1^2 + 4x_1x_2 + 3x_2^2\nonumber \\ Write \\q\\ in the form \\\vec{x}^TA\vec{x}\\.

###### Solution

First, let \\\vec{x} = \left$$ \begin{array}{r} x_1 \\ x_2 \end{array} \right$$\\ and \\A = \left$$ \begin{array}{rr} a\_{11} & a\_{12} \\ a\_{21} & a\_{22} \end{array} \right$$\\.

Then, writing \\q = \vec{x}^TA\vec{x}\\ gives \\\begin{aligned} q &= \left$$ \begin{array}{rr} x_1 & x_2 \end{array} \right$$ \left$$ \begin{array}{rr} a\_{11} & a\_{12} \\ a\_{21} & a\_{22} \end{array} \right$$ \left$$ \begin{array}{r} x_1 \\ x_2 \end{array} \right$$ \\ &= a\_{11}x_1^2 + a\_{21}x_1x_2 + a\_{12}x_1x_2 + a\_{22}x_2^2\end{aligned}\\

Notice that we have an \\x_1x_2\\ term as well as an \\x_2x_1\\ term. Since multiplication is commutative, these terms can be combined. This means that \\q\\ can be written \\q = a\_{11}x_1^2 + \left( a\_{21}+ a\_{12}\right) x_1x_2 + a\_{22}x_2^2\nonumber \\

Equating this to \\q\\ as given in the example, we have \\a\_{11}x_1^2 + \left( a\_{21}+ a\_{12}\right) x_1x_2 + a\_{22}x_2^2 = 6x_1^2 + 4x_1x_2 + 3x_2^2\nonumber \\

Therefore, \\\begin{aligned} a\_{11} &= 6 \\ a\_{22} &= 3 \\ a\_{21}+a\_{12} &= 4\end{aligned}\\

This demonstrates that the matrix \\A\\ is not unique, as there are several correct solutions to \\a\_{21}+a\_{12} = 4\\. However, we will *always* choose the coefficients such that \\a\_{21} = a\_{12} = \frac{1}{2} (a\_{21}+a\_{12})\\. This results in \\a\_{21} = a\_{12} = 2\\. This choice is key, as it will ensure that \\A\\ turns out to be a symmetric matrix.

Hence, \\A = \left$$ \begin{array}{rr} a\_{11} & a\_{12} \\ a\_{21} & a\_{22} \end{array} \right$$ = \left$$ \begin{array}{rr} 6 & 2 \\ 2 & 3 \end{array} \right$$\nonumber \\

You can verify that \\q = \vec{x}^T A \vec{x}\\ holds for this choice of \\A\\.

The above procedure for choosing \\A\\ to be symmetric applies for any quadratic form \\q\\. We will *always* choose coefficients such that \\a\_{ij}=a\_{ji}\\.

We now turn our attention to the focus of this section. Our goal is to start with a quadratic form \\q\\ as given above and find a way to rewrite it to eliminate the \\x_ix_j\\ terms. This is done through a change of variables. In other words, we wish to find \\y_i\\ such that \\q = d\_{11}y_1^2 + d\_{22}y_2^2 + \cdots + d\_{nn}y_n^2\nonumber \\ Letting \\\vec{y} = \left$$ \begin{array}{r} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right$$\\ and \\D = \left$$ d\_{ij} \right$$\\, we can write \\q = \vec{y}^T D \vec{y}\\ where \\D\\ is the matrix of coefficients from \\q\\. There is something special about this matrix \\D\\ that is crucial. Since no \\y_iy_j\\ terms exist in \\q\\, it follows that \\d\_{ij} = 0\\ for all \\i \neq j\\. Therefore, \\D\\ is a diagonal matrix. Through this change of variables, we find the principal axes \\y_1, y_2, \cdots, y_n\\ of the quadratic form.

This discussion sets the stage for the following essential theorem.

##### Theorem \\\PageIndex{11}\\: Diagonalizing a Quadratic Form

Let \\q\\ be a quadratic form in the variables \\x_1, \cdots, x_n\\. It follows that \\q\\ can be written in the form \\q = \vec{x}^T A \vec{x}\\ where \\\vec{x} = \left$$ \begin{array}{r} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right$$\nonumber \\ and \\A = \left$$ a\_{ij} \right$$\\ is the symmetric matrix of coefficients of \\q\\.

New variables \\y_1, y_2, \cdots, y_n\\ can be found such that \\q = \vec{y}^T D \vec{y}\\ where \\\vec{y} = \left$$ \begin{array}{r} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right$$\nonumber \\ and \\D=\left$$ d\_{ij} \right$$\\ is a diagonal matrix. The matrix \\D\\ contains the eigenvalues of \\A\\ and is found by orthogonally diagonalizing \\A\\.

While not a formal proof, the following discussion should convince you that the above theorem holds. Let \\q\\ be a quadratic form in the variables \\x_1, \cdots, x_n\\. Then, \\q\\ can be written in the form \\q = \vec{x}^T A \vec{x}\\ for a symmetric matrix \\A\\. By Theorem $\PageIndex{3}$ we can orthogonally diagonalize the matrix \\A\\ such that \\U^TAU = D\\ for an orthogonal matrix \\U\\ and diagonal matrix \\D\\.

Then, the vector \\\vec{y} = \left$$ \begin{array}{r} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right$$\\ is found by \\\vec{y} = U^T \vec{x}\\. To see that this works, rewrite \\\vec{y} = U^T \vec{x}\\ as \\\vec{x} = U\vec{y}\\. Letting \\q = \vec{x}^TA\vec{x}\\, proceed as follows: \\\begin{aligned} q &= \vec{x}^T A \vec{x}\\ &= (U\vec{y})^T A (U\vec{y})\\ &= \vec{y}^T (U^TAU) \vec{y} \\ &= \vec{y}^T D \vec{y}\end{aligned}\\

The following procedure details the steps for the change of variables given in the above theorem.

##### Procedure \\\PageIndex{5}\\: Diagonalizing a Quadratic Form

Let \\q\\ be a quadratic form in the variables \\x_1, \cdots, x_n\\ given by \\q = a\_{11}x_1^2 + a\_{22}x_2^2 + \cdots + a\_{nn}x_n^2 + a\_{12}x_1x_2+\cdots\nonumber \\ Then, \\q\\ can be written as \\q = d\_{11}y_1^2 + \cdots + d\_{nn}y_n^2\\ as follows:

1. Write \\q = \vec{x}^T A \vec{x}\\ for a symmetric matrix \\A\\.

2. Orthogonally diagonalize \\A\\ to be written as \\U^TAU=D\\ for an orthogonal matrix \\U\\ and diagonal matrix \\D\\.

3. Write \\\vec{y} = \left$$ \begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right$$\\. Then, \\\vec{x} = U \vec{y}\\.

4. The quadratic form \\q\\ will now be given by\\q = d\_{11}y_1^2 + \cdots + d\_{nn}y_n^2 = \vec{y}^T D \vec{y}\nonumber \\ where \\D = \left$$ d\_{ij} \right$$\\ is the diagonal matrix found by orthogonally diagonalizing \\A\\.

Consider the following example.

##### Example \\\PageIndex{14}\\: Choosing New Axes to Simplify a Quadratic Form

Consider the following level curve \\6x_1^2 + 4x_1x_2 + 3x_2^2 = 7\nonumber \\ shown in the following graph.

Figure $\PageIndex{1}$

Use a change of variables to choose new axes such that the ellipse is oriented parallel to the new coordinate axes. In other words, use a change of variables to rewrite \\q\\ to eliminate the \\x_1x_2\\ term.

###### Solution

Notice that the level curve is given by \\q = 7\\ for \\q = 6x_1^2 + 4x_1x_2 + 3x_2^2\\. This is the same quadratic form that we examined earlier in Example $\PageIndex{13}$. Therefore we know that we can write \\q = \vec{x}^T A \vec{x}\\ for the matrix \\A = \left$$ \begin{array}{rr} 6 & 2 \\ 2 & 3 \end{array} \right$$\nonumber \\

Now we want to orthogonally diagonalize \\A\\ to write \\U^TAU=D\\ for an orthogonal matrix \\U\\ and diagonal matrix \\D\\. The details are left to the reader, and you can verify that the resulting matrices are \\\begin{aligned} U &= \left$$ \begin{array}{rr} \frac{2}{\sqrt{5}} & - \frac{1}{\sqrt{5}} \\ \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}} \end{array} \right$$ \\ D &= \left$$ \begin{array}{rr} 7 & 0 \\ 0 & 2 \end{array} \right$$\end{aligned}\\

Next we write \\\vec{y} = \left$$ \begin{array}{c} y_1 \\ y_2 \end{array} \right$$\\. It follows that \\\vec{x} = U \vec{y}\\.

We can now express the quadratic form \\q\\ in terms of \\y\\, using the entries from \\D\\ as coefficients as follows: \\\begin{aligned} q &= d\_{11}y_1^2 + d\_{22}y_2^2 \\ &= 7y_1^2 + 2y_2^2 \end{aligned}\\

Hence the level curve can be written \\7y_1^2 + 2y_2^2 =7\\. The graph of this equation is given by:

Figure $\PageIndex{2}$

The change of variables results in new axes such that with respect to the new axes, the ellipse is oriented parallel to the coordinate axes. These are called the principal axes of the quadratic form.

The following is another example of diagonalizing a quadratic form.

##### Example \\\PageIndex{15}\\: Choosing New Axes to Simplify a Quadratic Form

Consider the level curve \\5x_1^{2}-6x_1x_2+5x_2^{2}=8\nonumber\\ shown in the following graph.

Figure $\PageIndex{3}$

Use a change of variables to choose new axes such that the ellipse is oriented parallel to the new coordinate axes. In other words, use a change of variables to rewrite \\q\\ to eliminate the \\x_1x_2\\ term.

###### Solution

First, express the level curve as \\\vec{x}^TA\vec{x}\\ where \\\vec{x} = \left$$ \begin{array}{r} x_1 \\ x_2 \end{array} \right$$\\ and \\A\\ is symmetric. Let \\A = \left$$ \begin{array}{rr} a\_{11} & a\_{12} \\ a\_{21} & a\_{22} \end{array} \right$$\\. Then \\q = \vec{x}^T A \vec{x}\\ is given by \\\begin{aligned} q &= \left$$ \begin{array}{cc} x_1 & x_2 \end{array} \right$$ \left$$ \begin{array}{rr} a\_{11} & a\_{12} \\ a\_{21} & a\_{22} \end{array} \right$$ \left$$ \begin{array}{r} x_1 \\ x_2 \end{array} \right$$\\ &= a\_{11}x_1^2 + (a\_{12} + a\_{21})x_1x_2 + a\_{22}x_2^2\end{aligned}\\

Equating this to the given description for \\q\\, we have \\5x_1^2 -6x_1x_2 + 5x_2^2 = a\_{11}x_1^2 + (a\_{12} + a\_{21})x_1x_2 + a\_{22}x_2^2\nonumber \\ This implies that \\a\_{11} = 5, a\_{22} = 5\\ and in order for \\A\\ to be symmetric, \\a\_{12} = a\_{22} = \frac{1}{2} (a\_{12}+a\_{21}) = -3\\. The result is \\A = \left$$ \begin{array}{rr} 5 & -3 \\ -3 & 5 \end{array} \right$$\\. We can write \\q = \vec{x}^TA\vec{x}\\ as \\\left$$ \begin{array}{cc} x_1 & x_2 \end{array} \right$$ \left$$ \begin{array}{rr} 5 & -3 \\ -3 & 5 \end{array} \right$$ \left$$ \begin{array}{c} x_1 \\ x_2 \end{array} \right$$ =8\nonumber \\

Next, orthogonally diagonalize the matrix \\A\\ to write \\U^TAU = D\\. The details are left to the reader and the necessary matrices are given by \\\begin{aligned} U &= \left$$ \begin{array}{rr} \frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} & - \frac{1}{2}\sqrt{2} \end{array} \right$$ \\ D &= \left$$ \begin{array}{rr} 2 & 0 \\ 0 & 8 \end{array} \right$$\end{aligned}\\

Write \\\vec{y} = \left$$ \begin{array}{r} y_1 \\ y_2 \end{array} \right$$\\, such that \\\vec{x} = U \vec{y}\\. Then it follows that \\q\\ is given by \\\begin{aligned} q &= d\_{11}y_1^2 + d\_{22}y_2^2 \\ &= 2y_1^{2}+8y_2^{2}\end{aligned}\\ Therefore the level curve can be written as \\2y_1^{2}+8y_2^{2}=8\\.

This is an ellipse which is parallel to the coordinate axes. Its graph is of the form

Figure $\PageIndex{4}$

Thus this change of variables chooses new axes such that with respect to these new axes, the ellipse is oriented parallel to the coordinate axes.

---

7_E_3A_Exercises

> 来源: LibreTexts

> 原页: https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/07%3A_Spectral_Theory/7.E%3A_Exercises

Skip to main content

Exercise \\\PageIndex{1}\\

If \\A\\ is an invertible \\n\times n\\ matrix, compare the eigenvalues of \\A\\ and \\A^{−1}\\. More generally, for \\m\\ an arbitrary integer, compare the eigenvalues of \\A\\ and \\A^m\\.

Answer

\\A^mX = λ^mX\\ for any integer. In the case of \\−1,\\ A^{−1}λX = AA^{−1}X = X\\ so \\A^{−1}X = λ^{−1}X\\. Thus the eigenvalues of \\A^{−1}\\ are just \\λ^{-1}\\ where \\λ\\ is an eigenvalue of \\A\\.

Exercise \\\PageIndex{2}\\

If \\A\\ is an \\n\times n\\ matrix and \\c\\ is a nonzero constant, compare the eigenvalues of \\A\\ and \\cA\\.

Answer

Say \\AX = λX\\. Then \\cAX = cλX\\ and so the eigenvalues of \\cA\\ are just \\cλ\\ where \\λ\\ is an eigenvalue of \\A\\.

Exercise \\\PageIndex{3}\\

Let \\A,\\ B\\ be invertible \\n\times n\\ matrices which commute. That is, \\AB = BA\\. Suppose \\X\\ is an eigenvector of \\B\\. Show that then \\AX\\ must also be an eigenvector for \\B\\.

Answer

\\BAX = ABX = AλX = λAX\\. Here it is assumed that \\BX = λX\\.

Exercise \\\PageIndex{4}\\

Suppose \\A\\ is an \\n\times n\\ matrix and it satisfies \\A^m = A\\ for some \\m\\ a positive integer larger than \\1\\. Show that if \\λ\\ is an eigenvalue of \\A\\ then \\\|λ\|\\ equals either \\0\\ or \\1\\.

Answer

Let \\X\\ be the eigenvector. Then \\A^mX = λ^mX,\\ A^mX = AX = λX\\ and so \\\lambda^m=\lambda\nonumber\\ Hence if \\\lambda\neq 0\\, then \\\lambda^{m-1}=1\nonumber\\ and so \\\|\lambda\|=1\\.

Exercise \\\PageIndex{5}\\

Show that if \\AX = λX\\ and \\AY = λY\\, then whenever \\k,\\ p\\ are scalars, \\A(kX+pY)=\lambda (kX+pY)\nonumber\\ Does this imply that \\kX+pY\\ is an eigenvector? Explain.

Answer

The formula follows from properties of matrix multiplications. However, this vector might not be an eigenvector because it might equal \\0\\ and eigenvectors cannot equal \\0\\.

Exercise \\\PageIndex{6}\\

Suppose \\A\\ is a \\3\times 3\\ matrix and the following information is available. \\\begin{aligned}A\left$$\begin{array}{r}0\\-1\\-1\end{array}\right$$&=0\left$$\begin{array}{r}0\\-1\\-1\end{array}\right$$ \\ A\left$$\begin{array}{c}1\\1\\1\end{array}\right$$&=-2\left$$\begin{array}{c}1\\1\\1\end{array}\right$$ \\ A\left$$\begin{array}{r}-2\\-3\\-2\end{array}\right$$&=-2\left$$\begin{array}{r}-2\\-3\\-2\end{array}\right$$\end{aligned}\\ Find \\A\left$$\begin{array}{r}1\\-4\\3\end{array}\right$$\\.

Exercise \\\PageIndex{7}\\

Suppose \\A\\ is a \\3\times 3\\ matrix and the following information is available. \\\begin{aligned}A\left$$\begin{array}{r}-1\\-2\\-2\end{array}\right$$&=1\left$$\begin{array}{r}-1\\-2\\-2\end{array}\right$$ \\ A\left$$\begin{array}{c}1\\1\\1\end{array}\right$$&=0\left$$\begin{array}{c}1\\1\\1\end{array}\right$$ \\ A\left$$\begin{array}{r}-1\\-4\\-3\end{array}\right$$&=2\left$$\begin{array}{r}-1\\-4\\-3\end{array}\right$$\end{aligned}\\ Find \\A\left$$\begin{array}{r}3\\-4\\3\end{array}\right$$\\.

Exercise \\\PageIndex{8}\\

Suppose \\A\\ is a \\3\times 3\\ matrix and the following information is available. \\\begin{aligned}A\left$$\begin{array}{r}0\\-1\\-1\end{array}\right$$&=2\left$$\begin{array}{r}0\\-1\\-1\end{array}\right$$ \\ A\left$$\begin{array}{c}1\\1\\1\end{array}\right$$&=1\left$$\begin{array}{c}1\\1\\1\end{array}\right$$ \\ A\left$$\begin{array}{r}-3\\-5\\-4\end{array}\right$$&=-3\left$$\begin{array}{r}-3\\-5\\-4\end{array}\right$$\end{aligned}\\ Find \\A\left$$\begin{array}{r}2\\-3\\3\end{array}\right$$\\.

Exercise \\\PageIndex{9}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}-6&-92&12 \\ 0&0&0\\-2&-31&4\end{array}\right$$\nonumber\\ One eigenvalue is \\-2\\.

Exercise \\\PageIndex{10}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}-2&-17&-6 \\ 0&0&0\\1&9&3\end{array}\right$$\nonumber\\ One eigenvalue is \\1\\.

Exercise \\\PageIndex{11}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}9&2&8 \\ 2&-6&-2 \\ -8&2&-5\end{array}\right$$\nonumber\\ One eigenvalue is \\-3\\.

Exercise \\\PageIndex{12}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}6&76&16 \\ -2&-21&-4 \\ 2&64&17\end{array}\right$$\nonumber\\ One eigenvalue is \\-2\\.

Exercise \\\PageIndex{13}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}3&5&2 \\ -8&-11&-4 \\ 10&11&3\end{array}\right$$\nonumber\\ One eigenvalue is \\-3\\.

Exercise \\\PageIndex{14}\\

Is it possible for a nonzero matrix to have only \\0\\ as an eigenvalue?

Answer

Yes. \\\left$$\begin{array}{cc}0&1\\0&0\end{array}\right$$\\ works.

Exercise \\\PageIndex{15}\\

If \\A\\ is the matrix of a linear transformation which rotates all vectors in \\\mathbb{R}^2\\ through \\60^{\circ}\\, explain why \\A\\ cannot have any real eigenvalues. Is there an angle such that rotation through this angle would have a real eigenvalue? What eigenvalues would be obtainable in this way?

Exercise \\\PageIndex{16}\\

Let \\A\\ be the \\2\times 2\\ matrix of the linear transformation which rotates all vectors in \\\mathbb{R}^2\\ through an angle of \\θ\\. For which values of \\θ\\ does \\A\\ have a real eigenvalue?

Answer

When you think of this geometrically, it is clear that the only two values of \\θ\\ are \\0\\ and \\π\\ or these added to integer multiples of \\2π\\.

Exercise \\\PageIndex{17}\\

Let \\T\\ be the linear transformation which reflects vectors about the \\x\\ axis. Find a matrix for \\T\\ and then find its eigenvalues and eigenvectors.

Answer

The matrix of \\T\\ is \\\left$$\begin{array}{rr}1&0\\0&-1\end{array}\right$$\\. The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}0\\1\end{array}\right$$\right\\↔ -1,\\\left\\\left$$\begin{array}{c}1\\0\end{array}\right$$\right\\↔1\nonumber\\

Exercise \\\PageIndex{18}\\

Let \\T\\ be the linear transformation which rotates all vectors in \\\mathbb{R}^2\\ counterclockwise through an angle of \\π/2\\. Find a matrix of \\T\\ and then find eigenvalues and eigenvectors.

Answer

The matrix of \\T\\ is \\\left$$\begin{array}{rr}0&-1\\1&0\end{array}\right$$\\. The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-i\\1\end{array}\right$$\right\\↔ -i,\\\left\\\left$$\begin{array}{c}i\\1\end{array}\right$$\right\\↔i\nonumber\\

Exercise \\\PageIndex{19}\\

Let \\T\\ be the linear transformation which reflects all vectors in \\\mathbb{R}^3\\ through the \\xy\\ plane. Find a matrix for \\T\\ and then obtain its eigenvalues and eigenvectors.

Answer

The matrix of \\T\\ is \\\left$$\begin{array}{ccc}1&0&0\\0&1&0\\0&0&-1\end{array}\right$$\\. The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}0\\0\\1\end{array}\right$$\right\\↔-1,\\\left\\\left$$\begin{array}{c}1\\0\\0\end{array}\right$$,\\\left$$\begin{array}{c}0\\1\\0\end{array}\right$$\right\\↔1\nonumber\\

Exercise \\\PageIndex{20}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}5&-18&-32\\0&5&4\\2&-5&-11\end{array}\right$$\nonumber\\ One eigenvalue is \\1\\. Diagonalize if possible.

Answer

The eigenvalues are \\−1,−1, 1\\. The eigenvectors corresponding to the eigenvalues are: \\\left\\\left$$\begin{array}{r}10&-2&3\end{array}\right$$\right\\↔-1,\\\left\\\left$$\begin{array}{r}7\\-2\\2\end{array}\right$$\right\\↔1\nonumber\\ Therefore this matrix is not diagonalizable.

Exercise \\\PageIndex{21}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}-12&-28&28\\4&9&-8\\-4&-8&9\end{array}\right$$\nonumber\\ One eigenvalue is \\3\\. Diagonalize if possible.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}2\\0\\1\end{array}\right$$\right\\↔1,\\\left\\\left$$\begin{array}{r}-2\\1\\0\end{array}\right$$\right\\↔1,\\\left\\\left$$\begin{array}{r}7\\-2\\2\end{array}\right$$\right\\↔3\nonumber\\ The matrix \\P\\ needed to diagonalize the above matrix is \\\left$$\begin{array}{rrr}2&-2&7\\0&1&-2\\1&0&2\end{array}\right$$\nonumber\\ and the diagonal matrix \\D\\ is \\\left$$\begin{array}{ccc}1&0&0\\0&1&0\\0&0&3\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{22}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}89&38&268\\14&2&40\\-30&-12&-90\end{array}\right$$\nonumber\\ One eigenvalue is \\-3\\. Diagonalize if possible.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-6\\-1\\-2\end{array}\right$$\right\\↔6,\\\left\\\left$$\begin{array}{r}-5\\-2\\2\end{array}\right$$\right\\↔-3,\\\left\\\left$$\begin{array}{r}-8\\-2\\3\end{array}\right$$\right\\↔2\nonumber\\ The matrix \\P\\ needed to diagonalize the above matrix is \\\left$$\begin{array}{rrr}-6&-5&-8\\-1&-2&-2\\2&2&3\end{array}\right$$\nonumber\\ and the diagonal matrix \\D\\ is \\\left$$\begin{array}{rrr}6&0&0\\0&-3&0\\0&0&-2\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{23}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}1&90&0\\0&-2&0\\3&89&-2\end{array}\right$$\nonumber\\ One eigenvalue is \\1\\. Diagonalize if possible.

Exercise \\\PageIndex{24}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}11&45&30\\10&26&20\\-20&-60&-44\end{array}\right$$\nonumber\\ One eigenvalue is \\1\\. Diagonalize if possible.

Exercise \\\PageIndex{25}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}95&25&24\\-196&-53&-48\\-164&-42&-43\end{array}\right$$\nonumber\\ One eigenvalue is \\5\\. Diagonalize if possible.

Exercise \\\PageIndex{26}\\

Suppose \\A\\ is an \\n\times n\\ matrix and let \\V\\ be an eigenvector such that \\AV = λV\\. Also suppose the characteristic polynomial of \\A\\ is \\\det (xI-A)=x^n+a\_{n-1}x^{n-1}+\cdots +a_1x+a_0\nonumber\\ Explain why \$A^n+a\_{n-1}A^{n-1}+\cdots +a_1A+a_0I)V=0\nonumber\\ If \\A\\ is diagonalizable, give a proof of the Cayley Hamilton theorem based on this. This theorem says \\A\\ satisfies its characteristic equation \\A^n+a\_{n-1}A^{n-1}+\cdots +a_1A+a_0I=0\nonumber\\

Exercise \\\PageIndex{27}\\

Suppose the characteristic polynomial of an \\n\times n\\ matrix \\A\\ is \\1−X^n\\. Find \\A^{mn}\\ where \\m\\ is an integer.

Answer

The eigenvalues are distinct because they are the \\n\\th roots of \\1\\. Hence if \\X\\ is a given vector with \\X=\sum\limits\_{j=1}^na_jV_j\nonumber\\ then \\A^{nm}X=A^{nm}\sum\limits\_{j=1}^na_jV_j=\sum\limits\_{j=1}^na_jA^{nm}V_j=\sum\limits\_{j=1}^na_jV_j=X\nonumber\\ so \\A^{nm}=I\\.

Exercise \\\PageIndex{28}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}15&-24&7\\-6&5&-1\\-58&76&-20\end{array}\right$$\nonumber\\ One eigenvalue is \\−2\\. Diagonalize if possible. Hint: This one has some complex eigenvalues.

Exercise \\\PageIndex{29}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}15&-25&6\\-13&23&-4\\-91&155&-30\end{array}\right$$\nonumber\\ One eigenvalue is \\2\\. Diagonalize if possible. Hint: This one has some complex eigenvalues.

Exercise \\\PageIndex{30}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}-11&-12&4\\8&17&-4\\-4&28&-3\end{array}\right$$\nonumber\\ One eigenvalue is \\1\\. Diagonalize if possible. Hint: This one has some complex eigenvalues.

Exercise \\\PageIndex{31}\\

Find the eigenvalues and eigenvectors of the matrix \\\left$$\begin{array}{rrr}14&-12&5\\-6&2&-1\\-69&51&-21\end{array}\right$$\nonumber\\ One eigenvalue is \\−3\\. Diagonalize if possible. Hint: This one has some complex eigenvalues.

Exercise \\\PageIndex{32}\\

Suppose \\A\\ is an \\n\times n\\ matrix consisting entirely of real entries but \\a + ib\\ is a complex eigenvalue having the eigenvector, \\X +iY\\ Here \\X\\ and \\Y\\ are real vectors. Show that then \\a−ib\\ is also an eigenvalue with the eigenvector, \\X − iY\\. Hint: You should remember that the conjugate of a product of complex numbers equals the product of the conjugates. Here \\a+ib\\ is a complex number whose conjugate equals \\a−ib\\.

Answer

\\AX = (a+ib)X\\. Now take conjugates of both sides. Since \\A\\ is real, \\A\overline{X}=(a-ib)\overline{X}\nonumber\\

Exercise \\\PageIndex{33}\\

Let \\A=\left$$\begin{array}{cc}1&2\\2&1\end{array}\right$$\\. Diagonalize \\A\\ to find \\A^{10}\\.

Answer

First we write \\A=PDP^{-1}\\. \\\left$$\begin{array}{cc}1&2\\2&1\end{array}\right$$=\left$$\begin{array}{rr}-1&1\\1&1\end{array}\right$$\left$$\begin{array}{rr}-1&0\\0&3\end{array}\right$$=\left$$\begin{array}{rr}-\frac{1}{2}&\frac{1}{2} \\ \frac{1}{2}&\frac{1}{2}\end{array}\right$$\nonumber\\ Therefore \\A^{10}=PD^{10}P^{-1}\\. \\\begin{aligned}\left$$\begin{array}{cc}1&2\\2&1\end{array}\right$$^{10}&=\left$$\begin{array}{rr}-1&1\\1&1\end{array}\right$$\left$$\begin{array}{rr}-1&0\\0&3\end{array}\right$$^{10}\left$$\begin{array}{rr}-\frac{1}{2}&\frac{1}{2}\\ \frac{1}{2}&\frac{1}{2}\end{array}\right$$ \\ &=\left$$\begin{array}{rr}-1&1\\1&1\end{array}\right$$\left$$\begin{array}{rr}(-1)^{10}&0\\0&3^{10}\end{array}\right$$\left$$\begin{array}{rr}-\frac{1}{2}&\frac{1}{2}\\ \frac{1}{2}&\frac{1}{2}\end{array}\right$$ \\ &=\left$$\begin{array}{rr}29525&29524 \\ 29524&29525\end{array}\right$$\end{aligned}\\

Exercise \\\PageIndex{34}\\

Let \\A=\left$$\begin{array}{ccc}1&4&1\\0&2&5\\0&0&5\end{array}\right$$\\. Diagonalize \\A\\ to find \\A^{50}\\.

Exercise \\\PageIndex{35}\\

Let \\A=\left$$\begin{array}{rrr}1&-2&-1\\2&-1&1\\-2&3&1\end{array}\right$$\\. Diagonalize \\A\\ to find \\A^{100}\\.

Exercise \\\PageIndex{36}\\

The following is a Markov (migration) matrix for three locations \\\left$$\begin{array}{ccc}\frac{7}{10}&\frac{1}{9}&\frac{1}{5} \\ \frac{1}{10}&\frac{7}{9}&\frac{2}{5} \\ \frac{1}{5}&\frac{1}{9}&\frac{2}{5}\end{array}\right$$\nonumber\\

1. Initially, there are \\90\\ people in location \\1,\\ 81\\ in location \\2\\, and \\85\\ in location \\3\\. How many are in each location after one time period?

2. The total number of individuals in the migration process is \\256\\. After a long time, how many are in each location?

Answer

1. Multiply the given matrix by the initial state vector given by \\\left$$\begin{array}{c}90\\81\\85\end{array}\right$$\\. After one time period there are \\89\\ people in location \\1\\, \\106\\ in location \\2\\, and \\61\\ in location \\3\\.

2. Solve the system given by \$I − A)X_s = 0\\ where \\A\\ is the migration matrix and \\X_s=\left$$\begin{array}{c}x\_{1s} \\ x\_{2s} \\ x\_{3s}\end{array}\right$$\\ is the steady state vector. The solution to this system is given by \\\begin{aligned}x\_{1s}&=\frac{8}{5}x\_{3s} \\ x\_{2s}&=\frac{63}{25}x\_{3s}\end{aligned}\\ Letting \\x\_{3s} = t\\ and using the fact that there are a total of \\256\\ individuals, we must solve \\\frac{8}{5}t+\frac{63}{25}t+t=256\nonumber\\ We find that \\t = 50\\. Therefore after a long time, there are \\80\\ people in location \\1,\\ 126\\ in location \\2\\, and \\50\\ in location \\3\\.

Exercise \\\PageIndex{37}\\

The following is a Markov (migration) matrix for three locations \\\left$$\begin{array}{ccc}\frac{1}{5}&\frac{1}{5}&\frac{2}{5} \\ \frac{2}{5}&\frac{2}{5}&\frac{1}{5} \\ \frac{2}{5}&\frac{2}{5}&\frac{2}{5}\end{array}\right$$\nonumber\\

1. Initially, there are \\130\\ individuals in location \\1,\\ 300\\ in location \\2\\, and \\70\\ in location \\3\\. How many are in each location after two time periods?

2. The total number of individuals in the migration process is \\500\\. After a long time, how many are in each location?

Exercise \\\PageIndex{38}\\

The following is a Markov (migration) matrix for three locations \\\left$$\begin{array}{ccc}\frac{3}{10}&\frac{3}{8}&\frac{1}{3} \\ \frac{1}{10}&\frac{3}{8}&\frac{1}{3} \\ \frac{3}{5}&\frac{1}{4}&\frac{1}{3}\end{array}\right$$\nonumber\\ The total number of individuals in the migration process is \\480\\. After a long time, how many are in each location?

Answer

We solve \$I −A)X_s = 0\\ to find the steady state vector \\X_s=\left$$\begin{array}{c}x\_{1s} \\ x\_{2s} \\ x\_{3s}\end{array}\right$$\\. The solution to the system is given by \\\begin{aligned}x\_{1s}&=\frac{5}{6}x\_{3s} \\ x\_{2s}&=\frac{2}{3}x\_{3s}\end{aligned}\\ Letting \\x\_{3s} = t\\ and using the fact that there are a total of \\480\\ individuals, we must solve \\\frac{5}{6}t+\frac{2}{3}t+t=480\nonumber\\ We find that \\t = 192\\. Therefore after a long time, there are \\160\\ people in location \\1,\\ 128\\ in location \\2\\, and \\192\\ in location \\3\\.

Exercise \\\PageIndex{39}\\

The following is a Markov (migration) matrix for three locations \\\left$$\begin{array}{ccc}\frac{3}{10}&\frac{1}{3}&\frac{1}{5} \\ \frac{3}{10}&\frac{1}{3}&\frac{7}{10} \\ \frac{2}{5}&\frac{1}{3}&\frac{1}{10}\end{array}\right$$\nonumber\\ The total number of individuals in the migration process is \\1155\\. After a long time, how many are in each location?

Exercise \\\PageIndex{40}\\

The following is a Markov (migration) matrix for three locations \\\left$$\begin{array}{ccc}\frac{2}{5}&\frac{1}{10}&\frac{1}{8} \\ \frac{3}{10}&\frac{2}{5}&\frac{5}{8} \\ \frac{3}{10}&\frac{1}{2}&\frac{1}{4}\end{array}\right$$\nonumber\\ The total number of individuals in the migration process is \\704\\. After a long time, how many are in each location?

Exercise \\\PageIndex{41}\\

A person sets off on a random walk with three possible locations. The Markov matrix of probabilities \\A = $$a\_{ij}$$\\ is given by \\\left$$\begin{array}{ccc}0.1&0.3&0.7 \\ 0.1&0.3&0.2\\0.8&0.4&0.1\end{array}\right$$\nonumber\\ If the walker starts in location \\2\\, what is the probability of ending back in location \\2\\ at time \\n = 3\\?

Answer

\\X_3=\left$$\begin{array}{c}0.38\\0.18\\0.44\end{array}\right$$\nonumber\\ Therefore the probability of ending up back in location \\2\\ is \\0.18\\.

Exercise \\\PageIndex{42}\\

A person sets off on a random walk with three possible locations. The Markov matrix of probabilities \\A = $$a\_{ij}$$\\ is given by \\\left$$\begin{array}{ccc}0.5&0.1&0.6\\0.2&0.9&0.2\\0.3&0&0.2\end{array}\right$$\nonumber\\ It is unknown where the walker starts, but the probability of starting in each location is given by \\X_0=\left$$\begin{array}{r}0.2\\0.25\\0.55\end{array}\right$$\nonumber\\ What is the probability of the walker being in location \\1\\ at time \\n = 2\\?

Answer

\\X_2=\left$$\begin{array}{r}0.367\\0.4625\\0.1705\end{array}\right$$\nonumber\\ Therefore the probability of ending up in location \\1\\ is \\0.367\\.

Exercise \\\PageIndex{43}\\

You own a trailer rental company in a large city and you have four locations, one in the South East, one in the North East, one in the North West, and one in the South West. Denote these locations by SE, NE, NW, and SW respectively. Suppose that the following table is observed to take place.

| | SE | NE | NW | SW |

|--------|-----------------|------------------|------------------|------------------|

| SE | \\\frac{1}{3}\\ | \\\frac{1}{10}\\ | \\\frac{1}{10}\\ | \\\frac{1}{5}\\ |

| NE | \\\frac{1}{3}\\ | \\\frac{7}{10}\\ | \\\frac{1}{5}\\ | \\\frac{1}{10}\\ |

| NW | \\\frac{2}{9}\\ | \\\frac{1}{10}\\ | \\\frac{3}{5}\\ | \\\frac{1}{5}\\ |

| SW | \\\frac{1}{9}\\ | \\\frac{1}{10}\\ | \\\frac{1}{10}\\ | \\\frac{1}{2}\\ |

Table \\\PageIndex{1}\\

In this table, the probability that a trailer starting at NE ends in NW is \\1/10\\, the probability that a trailer starting at SW ends in NW is \\1/5\\, and so forth. Approximately how many will you have in each location after a long time if the total number of trailers is \\413\\?

Answer

The migration matrix is \\A=\left$$\begin{array}{cccc}\frac{1}{3}&\frac{1}{10}&\frac{1}{10}&\frac{1}{5} \\ \frac{1}{3}&\frac{7}{10}&\frac{1}{5}&\frac{1}{10} \\ \frac{2}{9}&\frac{1}{10}&\frac{3}{5}&\frac{1}{5} \\ \frac{1}{9}&\frac{1}{10}&\frac{1}{10}&\frac{1}{2}\end{array}\right$$\nonumber\\ To find the number of trailers in each location after a long time we solve system \$I − A)X_s = 0\\ for the steady state vector \\X_s=\left$$\begin{array}{c}x\_{1s} \\ x\_{2s} \\ x\_{3s} \\ x\_{4s}\end{array}\right$$\\. The solution to the system is \\\begin{aligned} x\_{1s}&=\frac{9}{10}x\_{4s} \\ x\_{2s}&=\frac{12}{5}x\_{4s} \\ x\_{3s}&=\frac{8}{5}x\_{4s}\end{aligned}\\ Letting \\x\_{4s} = t\\ and using the fact that there are a total of \\413\\ trailers we must solve \\\frac{9}{10}t+\frac{12}{5}t+\frac{8}{5}t+t=413\nonumber\\ We find that \\t = 70\\. Therefore after a long time, there are \\63\\ trailers in the SE, \\168\\ in the NE, \\112\\ in the NW and \\70\\ in the SW.

Exercise \\\PageIndex{44}\\

You own a trailer rental company in a large city and you have four locations, one in the South East, one in the North East, one in the North West, and one in the South West. Denote these locations by SE, NE, NW, and SW respectively. Suppose that the following table is observed to take place.

| | SE | NE | NW | SW |

|--------|-----------------|-----------------|------------------|------------------|

| SE | \\\frac{1}{7}\\ | \\\frac{1}{4}\\ | \\\frac{1}{10}\\ | \\\frac{1}{5}\\ |

| NE | \\\frac{2}{7}\\ | \\\frac{1}{4}\\ | \\\frac{1}{5}\\ | \\\frac{1}{10}\\ |

| NW | \\\frac{1}{7}\\ | \\\frac{1}{4}\\ | \\\frac{3}{5}\\ | \\\frac{1}{5}\\ |

| SW | \\\frac{3}{7}\\ | \\\frac{1}{4}\\ | \\\frac{1}{10}\\ | \\\frac{1}{2}\\ |

Table \\\PageIndex{2}\\

In this table, the probability that a trailer starting at NE ends in NW is 1/10, the probability that a trailer starting at SW ends in NW is 1/5, and so forth. Approximately how many will you have in each location after a long time if the total number of trailers is 1469.

Exercise \\\PageIndex{45}\\

The following table describes the transition probabilities between the states rainy, partly cloudy and sunny. The symbol p.c. indicates partly cloudy. Thus if it starts off p.c. it ends up sunny the next day with probability \\\frac{1}{5}\\. If it starts off sunny, it ends up sunny the next day with probability \\\frac{2}{5}\\ and so forth.

| | rains | sunny | p.c. |

|-----------|-----------------|-----------------|-----------------|

| rains | \\\frac{1}{5}\\ | \\\frac{1}{5}\\ | \\\frac{1}{3}\\ |

| sunny | \\\frac{1}{5}\\ | \\\frac{2}{5}\\ | \\\frac{1}{3}\\ |

| p.c. | \\\frac{3}{5}\\ | \\\frac{2}{5}\\ | \\\frac{1}{3}\\ |

Table \\\PageIndex{3}\\

Given this information, what are the probabilities that a given day is rainy, sunny, or partly cloudy?

Exercise \\\PageIndex{46}\\

The following table describes the transition probabilities between the states rainy, partly cloudy and sunny. The symbol p.c. indicates partly cloudy. Thus if it starts off p.c. it ends up sunny the next day with probability \\\frac{1}{10}\\. If it starts off sunny, it ends up sunny the next day with probability \\\frac{2}{5}\\ and so forth.

| | rains | sunny | p.c. |

|-----------|------------------|-----------------|-----------------|

| rains | \\\frac{1}{5}\\ | \\\frac{1}{5}\\ | \\\frac{1}{3}\\ |

| sunny | \\\frac{1}{10}\\ | \\\frac{2}{5}\\ | \\\frac{4}{9}\\ |

| p.c. | \\\frac{7}{10}\\ | \\\frac{2}{5}\\ | \\\frac{2}{9}\\ |

Table \\\PageIndex{4}\\

Given this information, what are the probabilities that a given day is rainy, sunny, or partly cloudy?

Exercise \\\PageIndex{47}\\

You own a trailer rental company in a large city and you have four locations, one in the South East, one in the North East, one in the North West, and one in the South West. Denote these locations by SE, NE, NW, and SW respectively. Suppose that the following table is observed to take place.

| | SE | NE | NW | SW |

|--------|------------------|------------------|------------------|------------------|

| SE | \\\frac{5}{11}\\ | \\\frac{1}{10}\\ | \\\frac{1}{10}\\ | \\\frac{1}{5}\\ |

| NE | \\\frac{1}{11}\\ | \\\frac{7}{10}\\ | \\\frac{1}{5}\\ | \\\frac{1}{10}\\ |

| NW | \\\frac{2}{11}\\ | \\\frac{1}{10}\\ | \\\frac{3}{5}\\ | \\\frac{1}{5}\\ |

| SW | \\\frac{3}{11}\\ | \\\frac{1}{10}\\ | \\\frac{1}{10}\\ | \\\frac{1}{2}\\ |

Table \\\PageIndex{5}\\

In this table, the probability that a trailer starting at NE ends in NW is 1/10, the probability that a trailer starting at SW ends in NW is 1/5, and so forth. Approximately how many will you have in each location after a long time if the total number of trailers is 407?

Exercise \\\PageIndex{48}\\

The University of Poohbah offers three degree programs, scouting education (SE), dance appreciation (DA), and engineering (E). It has been determined that the probabilities of transferring from one program to another are as in the following table.

| | SE | DA | E |

|--------|--------|--------|--------|

| SE | \\.8\\ | \\.1\\ | \\.3\\ |

| DA | \\.1\\ | \\.7\\ | \\.5\\ |

| E | \\.1\\ | \\.2\\ | \\.2\\ |

Table \\\PageIndex{6}\\

where the number indicates the probability of transferring from the top program to the program on the left. Thus the probability of going from DA to E is \\.2\\. Find the probability that a student is enrolled in the various programs.

Exercise \\\PageIndex{49}\\

In the city of Nabal, there are three political persuasions, republicans (R), democrats (D), and neither one (N). The following table shows the transition probabilities between the political parties, the top row being the initial political party and the side row being the political affiliation the following year.

| | R | D | N |

|-------|-----------------|-----------------|-----------------|

| R | \\\frac{1}{5}\\ | \\\frac{1}{6}\\ | \\\frac{2}{7}\\ |

| D | \\\frac{1}{5}\\ | \\\frac{1}{3}\\ | \\\frac{4}{7}\\ |

| N | \\\frac{3}{5}\\ | \\\frac{1}{2}\\ | \\\frac{1}{7}\\ |

Table \\\PageIndex{7}\\

Find the probabilities that a person will be identified with the various political persuasions. Which party will end up being most important?

Exercise \\\PageIndex{50}\\

The following table describes the transition probabilities between the states rainy, partly cloudy and sunny. The symbol p.c. indicates partly cloudy. Thus if it starts off p.c. it ends up sunny the next day with probability \\\frac{1}{5}\\. If it starts off sunny, it ends up sunny the next day with probability \\\frac{2}{7}\\ and so forth.

| | rains | sunny | p.c. |

|-----------|-----------------|-----------------|-----------------|

| rains | \\\frac{1}{5}\\ | \\\frac{2}{7}\\ | \\\frac{5}{9}\\ |

| sunny | \\\frac{1}{5}\\ | \\\frac{2}{7}\\ | \\\frac{1}{3}\\ |

| p.c. | \\\frac{3}{5}\\ | \\\frac{3}{7}\\ | \\\frac{1}{9}\\ |

Table \\\PageIndex{8}\\

Given this information, what are the probabilities that a given day is rainy, sunny, or partly cloudy?

Exercise \\\PageIndex{51}\\

Find the solution to the initial value problem \\\begin{aligned}\left$$\begin{array}{c}x\\y\end{array}\right$$'&=\left$$\begin{array}{rr}0&-1\\6&5\end{array}\right$$\left$$\begin{array}{c}x\\y\end{array}\right$$\\ \left$$\begin{array}{c}x(0) \\ y(0)\end{array}\right$$&=\left$$\begin{array}{c}2\\2\end{array}\right$$\end{aligned}\\ Hint: form the matrix exponential \\e^{At}\\ and then the solution is \\e^{At}C\\ where \\C\\ is the initial vector,

Answer

The solution is \\e^{At}C=\left$$\begin{array}{c}8e^{2t}-6e^{3t} \\ 18e^{3t}-16e^{2t}\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{52}\\

Find the solution to the initial value problem \\\begin{aligned}\left$$\begin{array}{c}x\\y\end{array}\right$$'&=\left$$\begin{array}{rr}-4&-3\\6&5\end{array}\right$$\left$$\begin{array}{c}x\\y\end{array}\right$$\\ \left$$\begin{array}{c}x(0) \\ y(0)\end{array}\right$$&=\left$$\begin{array}{c}3\\4\end{array}\right$$\end{aligned}\\ Hint: form the matrix exponential \\e^{At}\\ and then the solution is \\e^{At}C\\ where \\C\\ is the initial vector.

Exercise \\\PageIndex{53}\\

Find the solution to the initial value problem \\\begin{aligned}\left$$\begin{array}{c}x\\y\end{array}\right$$'&=\left$$\begin{array}{rr}-1&2\\-4&5\end{array}\right$$\left$$\begin{array}{c}x\\y\end{array}\right$$\\ \left$$\begin{array}{c}x(0) \\ y(0)\end{array}\right$$&=\left$$\begin{array}{c}2\\2\end{array}\right$$\end{aligned}\\ Hint: form the matrix exponential \\e^{At}\\ and then the solution is \\e^{At}C\\ where \\C\\ is the initial vector.

Exercise \\\PageIndex{54}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. \\A=\left$$\begin{array}{rrr}11&-1&-4 \\ -1&11&-4\\-4&-4&14\end{array}\right$$\nonumber\\ Hint: Two eigenvalues are \\12\\ and \\18\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\frac{1}{\sqrt{3}}\left$$\begin{array}{c}1\\1\\1\end{array}\right$$\right\\\leftrightarrow 6,\left\\\frac{1}{\sqrt{2}}\left$$\begin{array}{r}-1\\1\\0\end{array}\right$$\right\\\leftrightarrow 12,\left\\\frac{1}{\sqrt{6}}\left$$\begin{array}{r}-1\\-1\\2\end{array}\right$$\right\\\leftrightarrow 18\nonumber\\

Exercise \\\PageIndex{55}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. \\A=\left$$\begin{array}{rrr}4&1&-2\\1&4&-2\\-2&-2&7\end{array}\right$$\nonumber\\ Hint: One eigenvalue is \\3\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\frac{1}{\sqrt{2}}\left$$\begin{array}{r}-1\\1\\0\end{array}\right$$,\frac{1}{\sqrt{3}}\left$$\begin{array}{c}1\\1\\1\end{array}\right$$\right\\\leftrightarrow 3,\left\\\frac{1}{\sqrt{6}}\left$$\begin{array}{r}-1\\-1\\2\end{array}\right$$\right\\\leftrightarrow 9\nonumber\\

Exercise \\\PageIndex{56}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{rrr}-1&1&1\\1&-1&1\\1&1&-1\end{array}\right$$\nonumber\\ Hint: One eigenvalue is \\-2\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 1,\left\\\left$$\begin{array}{c}-\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \\ 0\end{array}\right$$,\left$$\begin{array}{c}-\frac{1}{6}\sqrt{6} \\ -\frac{1}{6}\sqrt{6} \\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow -2\nonumber\\ \\\left$$\begin{array}{c}\sqrt{3}/3&-\sqrt{2}/2&-\sqrt{6}/6 \\ \sqrt{3}/3&\sqrt{2}/2&-\sqrt{6}/6 \\ \sqrt{3}/3&0&\frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$^T\left$$\begin{array}{rrr}-1&1&1\\1&-1&1\\1&1&-1\end{array}\right$$\nonumber\\ \\\left$$\begin{array}{c}\sqrt{3}/3&-\sqrt{2}/2&-\sqrt{6}/6 \\ \sqrt{3}/3&\sqrt{2}/2&-\sqrt{6}/6 \\ \sqrt{3}/3&0&\frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\nonumber\\ \\=\left$$\begin{array}{rrr}1&0&0\\0&-2&0\\0&0&-2\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{57}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{rrr}17&-7&-4 \\ -7&17&-4 \\ -4&-4&14\end{array}\right$$\nonumber\\ Hint: Two eigenvalues are \\18\\ and \\24\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 6,\left\\\left$$\begin{array}{c}-\frac{1}{6}\sqrt{6} \\ -\frac{1}{6}\sqrt{6} \\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 18,\left\\\left$$\begin{array}{c}-\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \\ 0\end{array}\right$$\right\\\leftrightarrow 24\nonumber\\ The matrix \\U\\ has these as its columns.

Exercise \\\PageIndex{58}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{rrr}13&1&4\\1&13&4\\4&4&10\end{array}\right$$\nonumber\\ Hint: Two eigenvalues are \\12\\ and \\18\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-\frac{1}{6}\sqrt{6} \\ -\frac{1}{6}\sqrt{6} \\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 6,\left\\\left$$\begin{array}{c}-\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \\ 0\end{array}\right$$\right\\\leftrightarrow 12, \left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 18.\nonumber\\ The matrix \\U\\ has these as its columns.

Exercise \\\PageIndex{59}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{ccc}-\frac{5}{3}&\frac{1}{15}\sqrt{6}\sqrt{5}&\frac{8}{15}\sqrt{5} \\ \frac{1}{15}\sqrt{6}\sqrt{5}&-\frac{14}{5}&-\frac{1}{15}\sqrt{6} \\ \frac{8}{15}\sqrt{5}&-\frac{1}{15}\sqrt{6}&\frac{7}{15} \end{array}\right$$\nonumber\\ Hint: The eigenvalues are \\-3,-2,1\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}\frac{1}{6}\sqrt{6} \\ 0\\ \frac{1}{6}\sqrt{5}\sqrt{6}\end{array}\right$$\right\\\leftrightarrow 1,\left\\\left$$\begin{array}{c}-\frac{1}{3}\sqrt{2}\sqrt{3} \\ -\frac{1}{5}\sqrt{5} \\ \frac{1}{15}\sqrt{2}\sqrt{15}\end{array}\right$$\right\\ -2,\left\\\left$$\begin{array}{c}-\frac{1}{6}\sqrt{6} \\ \frac{2}{5}\sqrt{5} \\ \frac{1}{30}\sqrt{30}\end{array}\right$$\right\\\leftrightarrow -3\nonumber\\ These vectors are the columns of \\U\\.

Exercise \\\PageIndex{60}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{ccc}3&0&0 \\ 0&\frac{3}{2}&\frac{1}{2} \\ 0&\frac{1}{2}&\frac{3}{2}\end{array}\right$$\nonumber\\

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}0\\-\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2}\end{array}\right$$\right\\\leftrightarrow 1,\left\\\left$$\begin{array}{c}0\\ \frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2}\end{array}\right$$\right\\\leftrightarrow 2,\left\\\left$$\begin{array}{c}1\\0\\0\end{array}\right$$\right\\\leftrightarrow 3.\nonumber\\ These vectors are the columns of the matrix \\U\\.

Exercise \\\PageIndex{61}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{ccc}2&0&0\\0&5&1\\0&1&5\end{array}\right$$\nonumber\\

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}1\\0\\0\end{array}\right$$\right\\\leftrightarrow 2,\left\\\left$$\begin{array}{c}0\\ -\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2}\end{array}\right$$\right\\\leftrightarrow 4, \left\\\left$$\begin{array}{c}0\\ \frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2}\end{array}\right$$\right\\\leftrightarrow 6.\nonumber\\ These vectors are the columns of \\U\\.

Exercise \\\PageIndex{62}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{ccc}\frac{4}{3}&\frac{1}{3}\sqrt{3}\sqrt{2}&\frac{1}{3}\sqrt{2} \\ \frac{1}{3}\sqrt{3}\sqrt{2}&1&-\frac{1}{3}\sqrt{3} \\ \frac{1}{3}\sqrt{2}&-\frac{1}{3}\sqrt{3}&\frac{5}{3}\end{array}\right$$\nonumber\\ Hint: The eigenvalues are \\0,2,2\\ where \\2\\ is listed twice because it is a root of multiplicity \\2\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-\frac{1}{5}\sqrt{2}\sqrt{5} \\ \frac{1}{5}\sqrt{3}\sqrt{5} \\ \frac{1}{5}\sqrt{5}\end{array}\right$$\right\\\leftrightarrow 0,\left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ 0\\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$,\left$$\begin{array}{c}\frac{1}{5}\sqrt{2}\sqrt{5} \\ \frac{1}{5}\sqrt{3}\sqrt{5} \\ -\frac{1}{5}\sqrt{5}\end{array}\right$$\right\\\leftrightarrow 2.\nonumber\\ The columns are these vectors.

Exercise \\\PageIndex{63}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for \\A\\. Diagonalize \\A\\ by finding an orthogonal matrix \\U\\ and a diagonal matrix \\D\\ such that \\U^TAU = D\\. \\A=\left$$\begin{array}{ccc}1&\frac{1}{6}\sqrt{3}\sqrt{2}&\frac{1}{6}\sqrt{3}\sqrt{6} \\ \frac{1}{6}\sqrt{3}\sqrt{2}&\frac{3}{2}&\frac{1}{12}\sqrt{2}\sqrt{6} \\ \frac{1}{6}\sqrt{3}\sqrt{6}&\frac{1}{12}\sqrt{2}\sqrt{6}&\frac{1}{2}\end{array}\right$$\nonumber\\ Hint: The eigenvalues are \\2,1,0\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-\frac{1}{3}\sqrt{3} \\ 0\\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow 0,\left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ -\frac{1}{2}\sqrt{2} \\ \frac{1}{6}\sqrt{6}\end{array}\right$$\right\\\leftrightarrow 1,\left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ \frac{1}{2}\sqrt{2} \\ \frac{1}{6}\sqrt{6}\end{array}\right$$\right\\\leftrightarrow 2.\nonumber\\ The columns are these vectors.

Exercise \\\PageIndex{64}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for the matrix \\A=\left$$\begin{array}{ccc}\frac{1}{3}&\frac{1}{6}\sqrt{3}\sqrt{2}&-\frac{7}{18}\sqrt{3}\sqrt{6} \\ \frac{1}{6}\sqrt{3}\sqrt{2}&\frac{3}{2}&-\frac{1}{12}\sqrt{2}\sqrt{6} \\ -\frac{7}{18}\sqrt{3}\sqrt{6}&-\frac{1}{12}\sqrt{2}\sqrt{6}&-\frac{5}{6}\end{array}\right$$\nonumber\\ Hint: The eigenvalues are \\1,2,-2\\.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-\frac{1}{3}\sqrt{3} \\ \frac{1}{2}\sqrt{2} \\ \frac{1}{6}\sqrt{6}\end{array}\right$$\right\\\leftrightarrow 1, \left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ 0\\ \frac{1}{3}\sqrt{2}\sqrt{3}\end{array}\right$$\right\\\leftrightarrow -2, \left\\\left$$\begin{array}{c}\frac{1}{3}\sqrt{3} \\ \frac{1}{2}\sqrt{2} \\ -\frac{1}{6}\sqrt{6}\end{array}\right$$\right\\\leftrightarrow 2.\nonumber\\ Then the columns of \\U\\ are these vectors.

Exercise \\\PageIndex{65}\\

Find the eigenvalues and an orthonormal basis of eigenvectors for the matrix \\A=\left$$\begin{array}{ccc}-\frac{1}{2}&-\frac{1}{5}\sqrt{6}\sqrt{5}&\frac{1}{10}\sqrt{5} \\ -\frac{1}{5}\sqrt{6}\sqrt{5}&\frac{7}{5}&-\frac{1}{5}\sqrt{6} \\ \frac{1}{10}&\sqrt{5}&-\frac{1}{5}\sqrt{6}&-\frac{9}{10}\end{array}\right$$\nonumber\\ Hint: The eigenvalues are \\-1,2,-1\\ where \\-1\\ is listed twice because it has multiplicity \\2\\ as a zero of the characteristic equation.

Answer

The eigenvectors and eigenvalues are: \\\left\\\left$$\begin{array}{c}-\frac{1}{6}\sqrt{6} \\ 0 \\ \frac{1}{6}\sqrt{5}\sqrt{6}\end{array}\right$$,\left$$\begin{array}{c}\frac{1}{3}\sqrt{2}\sqrt{3} \\ \frac{1}{5}\sqrt{5} \\ \frac{1}{15}\sqrt{2}\sqrt{15}\end{array}\right$$\right\\\leftrightarrow -1,\left\\\left$$\begin{array}{c}\frac{1}{6}\sqrt{6} \\ -\frac{2}{5}\sqrt{5} \\ \frac{1}{30}\sqrt{30}\end{array}\right$$\right\\\leftrightarrow 2.\nonumber\\ The columns of \\U\\ are these vectors. \\\left$$\begin{array}{ccc}-\frac{1}{6}\sqrt{6}&\frac{1}{3}\sqrt{2}\sqrt{3}&\frac{1}{6}\sqrt{6} \\ 0&\frac{1}{5}\sqrt{5}&-\frac{2}{5}\sqrt{5} \\ \frac{1}{6}\sqrt{5}\sqrt{6} &\frac{1}{15}\sqrt{2}\sqrt{15}&\frac{1}{30}\sqrt{30}\end{array}\right$$^T\left$$\begin{array}{ccc}-\frac{1}{2}&-\frac{1}{5}\sqrt{6}\sqrt{5}&\frac{1}{10}\sqrt{5} \\ -\frac{1}{5}\sqrt{6}\sqrt{5}&\frac{7}{5}&-\frac{1}{5}\sqrt{6} \\ \frac{1}{10}\sqrt{5}&-\frac{1}{5}\sqrt{6}&-\frac{9}{10}\end{array}\right$$.\nonumber\\ \\\left$$\begin{array}{ccc}-\frac{1}{6}\sqrt{6}&\frac{1}{3}\sqrt{2}\sqrt{3}&\frac{1}{6}\sqrt{6} \\ 0&\frac{1}{5}\sqrt{5}&-\frac{2}{5}\sqrt{5} \\ \frac{1}{6}\sqrt{5}\sqrt{6}&\frac{1}{15}\sqrt{2}\sqrt{15}&\frac{1}{30}\sqrt{30}\end{array}\right$$=\left$$\begin{array}{rrr}-1&0&0\\0&-1&0\\0&0&2\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{66}\\

Explain why a matrix \\A\\ is symmetric if and only if there exists an orthogonal matrix \\U\\ such that \\A = U^TDU\\ for \\D\\ a diagonal matrix.

Answer

If \\A\\ is given by the formula, then \\A^T=U^TD^TU=U^TDU=A\nonumber\\ Next suppose \\A = A^T\\. Then by the theorems on symmetric matrices, there exists an orthogonal matrix \\U\\ such that \\UAU^T=D\nonumber\\ for \\D\\ diagonal. Hence \\A=U^TDU\nonumber\\

Exercise \\\PageIndex{67}\\

Show that if \\A\\ is a real symmetric matrix and \\λ\\ and \\µ\\ are two different eigenvalues, then if \\X\\ is an eigenvector for \\λ\\ and \\Y\\ is an eigenvector for \\µ\\, then \\X •Y = 0\\. Also all eigenvalues are real. Supply reasons for each step in the following argument. First \\\lambda X^T\overline{X}=(AX)^T\overline{X}=X^TA\overline{X}=X^T\overline{AX}=X^T\overline{\lambda X}=\overline{\lambda}X^T\overline{X}\nonumber\\ and so \\\lambda=\overline{\lambda}\\. This shows that all eigenvalues are real. It follows all the eigenvectors are real. Why? Now let \\X,\\ Y,\\µ\\ and \\λ\\ be given as above. \\\lambda (X\bullet Y)=\lambda X\bullet Y=AX\bullet Y=X\bullet AY=X\bullet\mu Y=\mu (X\bullet Y)=\mu (X\bullet Y)\nonumber\\ and so \$\lambda -\mu )X\bullet Y=0\nonumber\\ Why does it follow that \\X\bullet Y=0\\?

Answer

Since \\\lambda\neq\mu\\, it follows \\X\bullet Y=0\\.

Exercise \\\PageIndex{68}\\

Find the Cholesky factorization for the matrix \\\left$$\begin{array}{ccc}1&2&0 \\ 2&6&4\\0&4&10\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{69}\\

Find the Cholesky factorization for the matrix \\\left$$\begin{array}{rrr}4&8&0\\8&17&2\\0&2&13\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{70}\\

Find the Cholesky factorization for the matrix \\\left$$\begin{array}{rrr}4&8&0\\8&20&8\\0&8&20\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{71}\\

Find the Cholesky factorization for the matrix \\\left$$\begin{array}{rrr}1&2&1\\2&8&10\\1&10&18\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{72}\\

Find the Cholesky factorization for the matrix \\\left$$\begin{array}{rrr}1&2&1\\2&8&10\\1&10&26\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{73}\\

Suppose you have a lower triangular matrix \\L\\ and it is invertible. Show that \\LL^T\\ must be positive definite.

Exercise \\\PageIndex{74}\\

Using the Gram Schmidt process or the \\QR\\ factorization, find an orthonormal basis for the following span: \\span\left\\\left$$\begin{array}{c}1\\2\\1\end{array}\right$$,\left$$\begin{array}{r}2\\-1\\3\end{array}\right$$,\left$$\begin{array}{c}1\\0\\0\end{array}\right$$\right\\\nonumber\\

Answer

Using the \\QR\\ factorization, we have: \\\left$$\begin{array}{rrr}1&2&1\\2&-1&0\\1&3&0\end{array}\right$$=\left$$\begin{array}{ccc}\frac{1}{6}\sqrt{6}&\frac{3}{10}\sqrt{2}&\frac{7}{15}\sqrt{3} \\ \frac{1}{3}\sqrt{6}&-\frac{2}{5}\sqrt{2}&-\frac{1}{15}\sqrt{3} \\ \frac{1}{6}\sqrt{6}&\frac{1}{2}\sqrt{2}&-\frac{1}{3}\sqrt{3}\end{array}\right$$\left$$\begin{array}{ccc}\sqrt{6}&\frac{1}{2}\sqrt{6}&\frac{1}{6}\sqrt{6} \\ 0&\frac{5}{2}\sqrt{2}&\frac{3}{10}\sqrt{2} \\ 0&0&\frac{7}{15}\sqrt{3}\end{array}\right$$\nonumber\\ A solution is then \\\left$$\begin{array}{c}\frac{1}{6}\sqrt{6} \\ \frac{1}{3}\sqrt{6} \\ \frac{1}{6}\sqrt{6}\end{array}\right$$,\left$$\begin{array}{c}\frac{3}{10}\sqrt{2} \\ -\frac{2}{5}\sqrt{2} \\ \frac{1}{2}\sqrt{2}\end{array}\right$$,\left$$\begin{array}{c}\frac{7}{15}\sqrt{3} \\ -\frac{1}{15}\sqrt{3} \\ -\frac{1}{3}\sqrt{3}\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{75}\\

Using the Gram Schmidt process or the \\QR\\ factorization, find an orthonormal basis for the following span: \\span\left\\\left$$\begin{array}{c}1\\2\\1\\0\end{array}\right$$,\left$$\begin{array}{r}2\\-1\\3\\1\end{array}\right$$,\left$$\begin{array}{c}1\\0\\0\\1\end{array}\right$$\right\\\nonumber\\

Answer

\\\left$$\begin{array}{rrr}1&2&1\\2&-1&0\\1&3&0\\0&1&1\end{array}\right$$=\left$$\begin{array}{cccc}\frac{1}{6}\sqrt{6}&\frac{1}{6}\sqrt{2}\sqrt{3}&\frac{5}{111}\sqrt{3}\sqrt{37}&\frac{7}{111}\sqrt{111} \\ \frac{1}{3}\sqrt{6}&-\frac{2}{9}\sqrt{2}\sqrt{3}&\frac{1}{333}\sqrt{3}\sqrt{37}&-\frac{2}{111}\sqrt{111} \\ \frac{1}{6}\sqrt{6}&\frac{5}{18}\sqrt{2}\sqrt{3}&-\frac{17}{333}\sqrt{3}\sqrt{37}&-\frac{1}{37}\sqrt{111} \\ 0&\frac{1}{9}\sqrt{2}\sqrt{3}&\frac{22}{333}\sqrt{3}\sqrt{37}&-\frac{7}{111}\sqrt{111}\end{array}\right$$\nonumber\\ \\\left$$\begin{array}{ccc}\sqrt{6}&\frac{1}{2}\sqrt{6}&\frac{1}{6}\sqrt{6} \\ 0&\frac{3}{2}\sqrt{2}\sqrt{3}&\frac{5}{18}\sqrt{2}\sqrt{3} \\ 0&0&\frac{1}{9}\sqrt{3}\sqrt{37} \\ 0&0&0\end{array}\right$$\nonumber\\ Then a solution is \\\left$$\begin{array}{c}\frac{1}{6}\sqrt{6} \\ \frac{1}{3}\sqrt{6} \\ \frac{1}{6}\sqrt{6}&0\end{array}\right$$,\left$$\begin{array}{c}\frac{1}{6}\sqrt{2}\sqrt{3} \\ -\frac{2}{9}\sqrt{2}\sqrt{3} \\ \frac{5}{18}\sqrt{2}\sqrt{3} \\ \frac{1}{9}\sqrt{2}\sqrt{3}\end{array}\right$$,\left$$\begin{array}{c}\frac{5}{111}\sqrt{3}\sqrt{37} \\ \frac{1}{333}\sqrt{3}\sqrt{37} \\ -\frac{17}{333}\sqrt{3}\sqrt{37} \\ \frac{22}{333}\sqrt{3}\sqrt{37}\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{76}\\

1. \\\left$$\begin{array}{ccc}1&2&3\\0&3&4\\0&0&1\end{array}\right$$\\

2. \\\left$$\begin{array}{cc}2&1\\2&1\end{array}\right$$\\

3. \\\left$$\begin{array}{rr}1&2\\-1&2\end{array}\right$$\\

4. \\\left$$\begin{array}{cc}1&1\\2&3\end{array}\right$$\\

5. \\\left$$\begin{array}{rrr}\sqrt{11}&1&3\sqrt{6} \\ \sqrt{11}&7&-\sqrt{6} \\ 2\sqrt{11}&-4&-\sqrt{6}\end{array}\right$$\\ Hint: Notice that the columns are orthogonal.

Exercise \\\PageIndex{77}\\

Using a computer algebra system, find a QR factorization for the following matrices.

1. \\\left$$\begin{array}{rrr}1&1&2\\3&-2&3\\2&1&1\end{array}\right$$\\

2. \\\left$$\begin{array}{rrrr}1&2&1&3\\4&5&-4&3\\2&1&2&1\end{array}\right$$\\

3. \\\left$$\begin{array}{rr}1&2\\3&2\\1&-4\end{array}\right$$\\ Find the thin QR factorization of this one.

Exercise \\\PageIndex{78}\\

A quadratic form in three variables is an expression of the form \\a_1x^2 + a_2y^2 + a_3z^2 + a_4xy+a_5xz+a_6yz\\. Show that every such quadratic form may be written as \\\left$$\begin{array}{ccc}x&y&z\end{array}\right$$A\left$$\begin{array}{c}x\\y\\z\end{array}\right$$\nonumber\\ where \\A\\ is a symmetric matrix.

Answer

\\\left$$\begin{array}{ccc}x&y&z\end{array}\right$$\left$$\begin{array}{ccc}a_1&a_4/2&a_5/2 \\ a_4/2&a_2&a_6/2 \\ a_5/2&a_6/2&a_3\end{array}\right$$\left$$\begin{array}{c}x\\y\\z\end{array}\right$$\nonumber\\

Exercise \\\PageIndex{79}\\

Given a quadratic form in three variables, \\x, y,\\ and \\z\\, show there exists an orthogonal matrix \\U\\ and variables \\x′ , y ′ ,z ′\\ such that \\\left$$\begin{array}{c}x\\y\\z\end{array}\right$$=U\left$$\begin{array}{c}x'\\y'\\z'\end{array}\right$$\nonumber\\ with the property that in terms of the new varaibles, the quadratic form is \\\lambda_1(x')^2+\lambda_2(y')^2+\lambda_3(z')^2\nonumber\\ where the numbers, \\\lambda_1\\, \\\lambda_2\\, and \\\lambda_3\\ are the eigenvalues of the matrix \\A\\ in Exercise $\PageIndex{78}$.

Answer

The quadratic form may be written as \\\vec{x}^TA\vec{x}\nonumber\\ where \\A = A^T\\. By the theorem about diagonalizing a symmetric matrix, there exists an orthogonal matrix \\U\\ such that \\U^TAU=D,\\A=UDU^T\nonumber\\ Then the quadratic form is \\\vec{x}^TUDU^T\vec{x}=(U^T\vec{x})^TD(U^T\vec{x})\nonumber\\ where \\D\\ is a diagonal matrix having the real eigenvalues of \\A\\ down the main diagonal. Now simply let \\\vec{x}'=U^T\vec{x}\nonumber\\

Exercise \\\PageIndex{80}\\

Consider the quadratic form \\q\\ given by \\q = 3x_1^2 −12x_1x_2 −2x_2^2\\.

1. Write \\q\\ in the form \\\vec{x}^TA\vec{x}\\ for an appropriate symmetric matrix \\A\\.

2. Use a change of variables to rewrite \\q\\ to eliminate the \\x_1x_2\\ term.

Exercise \\\PageIndex{81}\\

Consider the quadratic form \\q\\ given by \\q = −2x_1^2 +2x_1x_2 −2x_2^2\\.

1. Write \\q\\ in the form \\\vec{x}^TA\vec{x}\\ for an appropriate symmetric matrix \\A\\.

2. Use a change of variables to rewrite \\q\\ to eliminate the \\x_1x_2\\ term.

Exercise \\\PageIndex{82}\\

Consider the quadratic form \\q\\ given by \\q = 7x_1^2 +6x_1x_2 −x_2^2\\.

1. Write \\q\\ in the form \\\vec{x}^TA\vec{x}\\ for an appropriate symmetric matrix \\A\\.

2. Use a change of variables to rewrite \\q\\ to eliminate the \\x_1x_2\\ term.

---