Why Eigenval/vec When Solving ODE? (中文)

2026-11-10

This post aims at an audience of freshmen and sophomores taking differential equations and getting confused by the appearance of linear algebra in linear ODEs. It is rooted in the fact that linear algebra is not a bunch of neat tricks for solving linear equations or building deep learning models (while, of course, they certainly do), but it has two components: "linear" and "algebra". "Linear" is important since it behaves nicely and widely, while "algebra" aims to generalize "linear" as much as possible. However, for whatever reason, "algebra" linear algebra is an elective course for a lot of STEM majors, and a lot of engineers or even math people don't get any insights when encountering it. This article tries to bridge the gap and assumes prior study of "ordinary" linear algebra. There are, of course, much more nuances than this essay, and you really should take the upper-div linear algebra lecture just for the fun of it.

"Non-trivial" Linear Algebra

A lower-division course usually approaches the subject through concrete computations and examples in \(\mathbb{R}^n\), and we'll skip that part here. In a formal algebra course, linear algebra is all about vector space that can be defined on arbitrary structures (see below) and arbitrary fields (instead of \(\mathbb{R}\), but still mainly \(\mathbb{R}\) and \(\mathbb{C}\) in practice).

Previously we only used \(\mathbb{R}^n\) as our vector space, but under this general view, functions, sequences, and random variables, etc. can also be vectors. Consider these:

• Limit: \(\lim : \{\text{all convergent real sequences}\} \to \mathbb{R}\)
  \[a\lim_{n\to\infty} f_n + b\lim_{n\to\infty} g_n = \lim_{n\to\infty}(af_n + bg_n)\]
• Integral: \(\int_0^1 : \{\text{continuous functions}\} \to \mathbb{R}\)
  \[a\int_0^1 f(x)\,\mathrm{d}x + b\int_0^1 g(x)\,\mathrm{d}x = \int_0^1 (af(x) + bg(x))\,\mathrm{d}x\]
• Differentiation: \(\frac{\mathrm{d}}{\mathrm{d}x} : \{\text{differentiable functions}\} \to \{\text{functions}\}\)
  \[a\frac{\mathrm{d}f}{\mathrm{d}x} + b\frac{\mathrm{d}g}{\mathrm{d}x} = \frac{\mathrm{d}}{\mathrm{d}x}(af + bg)\]
• Expected value: \(\mathbb{E} : \{\text{random variables}\} \to \mathbb{R}\)
  \[a\mathbb{E}[X] + b\mathbb{E}[Y] = \mathbb{E}[aX + bY]\]

Any \(T\) s.t. the form of \(T(a\mathbf{u} + b\mathbf{v}) = aT(\mathbf{u}) + bT(\mathbf{v})\) is known as a linear map, where \(\mathbf{u}\) and \(\mathbf{v}\) live in the vector space and \(a\) and \(b\) live in the field.

Differentiation and Eigenfunctions

Without going too much into the rigorous and formal details, by that differentiation is a linear operator, a natural thing to ask is "what is its eigenvector and eigenfunction"?

Exponential functions \(e^{\lambda t}\) are literally the eigenvectors (often called eigenfunctions). A nice way to think about it is that (I) we construct it and inspect it indeed is an eigenfunction; (II) by Picard's Theorem, there are no more solutions.

Coupled ODEs and Why We Want Diagonalization

Now let's look at a system of first-order linear ODEs with a constant matrix \(\mathbf{A}\). Here, our vectors are differentiable functions \(\mathbf{x}: \mathbb{R} \to \mathbb{R}^n\) with pointwise addition \((\mathbf{x}+\mathbf{y})(t) = \mathbf{x}(t)+\mathbf{y}(t)\) and scalar multiplication \((a\mathbf{x})(t) = a\mathbf{x}(t)\). Consider the concrete system:

What makes this hard to solve directly is that \(x_1\) and \(x_2\) are coupled: \(x_1'\) depends on \(x_2\), and \(x_2'\) depends on \(x_1\). However, imagine if the matrix were diagonal instead, say:

In a diagonal system, since the variables are decoupled, it reduces to the previously discussed single variable case.

Recall from "ordinary" linear algebra that we can translate a matrix, linear combination in the language of basis coefficients, into another basis system, and if the new basis is an eigenbasis, the matrix is a diagonal matrix. Specifically, if \(\mathbf{A}\) has enough linearly independent eigenvectors, we can diagonalize it as \(\mathbf{A} = \mathbf{P}\mathbf{D}\mathbf{P}^{-1}\), where \(\mathbf{D}\) is the diagonal matrix of eigenvalues and \(\mathbf{P}\) has the corresponding eigenvectors as columns.

For \(\mathbf{A} = \begin{bmatrix}5&3\\2&0\end{bmatrix}\), we solve the characteristic equation:

Finding the null space of \(\mathbf{A}-\lambda\mathbf{I}\) for each eigenvalue gives the eigenvectors:

Now substitute \(\mathbf{A} = \mathbf{P}\mathbf{D}\mathbf{P}^{-1}\) into \(\mathbf{x}' = \mathbf{A}\mathbf{x}\) and multiply both sides on the left by \(\mathbf{P}^{-1}\):

Define \(\mathbf{u} = \mathbf{P}^{-1}\mathbf{x}\), which represents \(\mathbf{x}\) in the eigenvector basis. Since \(\mathbf{P}\) is constant, differentiation commutes with \(\mathbf{P}^{-1}\), that is \(\mathbf{u}' = \mathbf{P}^{-1}\mathbf{x}'\). The coupled ODE turns into the decoupled system:

Finally, we transform back to the standard coordinates using \(\mathbf{x}(t) = \mathbf{P}\mathbf{u}(t)\):

Because \(D\) and \(\mathbf{A}\) are linear, any linear combination of solutions is still a solution as proven in the course.