ogbofall fatherly morton

[Cristoforo Kanoy]

The purpose of this document is to quickly refresh (presumably) knownnotions in linear algebra. It contains a collection of facts related tovectors, matrices, and geometric entities they represent that we willuse heavily in our course. Even though this quick reminder may seemredundant or trivial to most of you (I hope), I still suggest at leastto skim through it, as it might present less common ways ofinterpretation of very familiar definitions and properties. And even ifyou discover nothing new in this document, it will at least be useful tointroduce notation.

Though the notion of an inner product is more general than this, we willlimit our attention exclusively to this particular case (and its matrixequivalent defined in the sequel). This is sometimes called the standardor the Euclidean inner product. The inner product defines or induces avector norm

A more general notion of a norm can be introduced axiomatically. We willsay that a non-negative scalar function$ \cdot : \RR^n \rightarrow \RR_+$ is a norm if it satisfies thefollowing axioms for any scalar $a \in \RR$ and any vectors$\bbu,\bbv \in \RR^n$

The latter can beexpressed compactly using the matrix-vector product notation$\bbv = \bbA \bbu$, where $\bbA$ is the matrix with the elements$a_ij$. In other words, a matrix $\bbA \in \RR^m \times n$ is acompact way of expressing a linear map between $\RR^m$ and $\RR^n$. Anmatrix is said to be square of $m=n$; such a matrix defines anoperator mapping $\RR^n$ to itself. A symmetric matrix is a squarematrix $\bbA$ such that $\bbA^\Tr = \bbA$.

In this course, we will often encounter the trace of a square matrix,which is defined as the sum of its diagonal entries,\trace\, \bbA = \sum_i=1^n a_i. The following property of thetrace will be particularly useful:

In particular, the squared norm $ \bbu ^2 = \bbu^\Tr \bbu$ canbe written as $\trace(\bbu^\Tr \bbu) = \trace(\bbu\bbu^\Tr)$. Wewill see many cases where such an apparently weird writing is veryuseful. The above property can be generalized to the product of $k$matrices $\bbA_1 \dots \bbA_k$ by saying that$\trace(\bbA_1 \dots \bbA_k)$ is invariant under a cyclicpermutation of the factors as long as their product is defined. Forexample,$\trace(\bbA\bbB^\Tr\bbC) = \trace(\bbC\bbA\bbB^\Tr) = \trace(\bbB^\Tr\bbC\bbA)$(again, as long as the matrix dimensions are such that the products aredefined).

If the matrix $\bbA$ is symmetric, it can be shown that itseigenvectors are orthonormal, i.e.,$\langle \bbu_i, \bbu_j \rangle = \bbu_i^\Tr \bbu_j = 0$ forevery $i \ne j$ and, since the eigenvectors have unit length,$\bbu_i^\Tr \bbu = 1$. This can be compactly written as$\bbU^\Tr \bbU = \bbI$ or, in other words,$\bbU^-1 = \bbU^\Tr$. Matrices satisfying this property are calledorthonormal or unitary, and we will say that symmetric matricesadmit unitary eigendecomposition$\bbA = \bbU \bb\Lambda\bbU^\Tr$.

Eigendecomposition is a very convenient way of performing various matrixoperations. For example, if we are given the eigendecomposition of$\bbA = \bbU\bb\Lambda\bbU^-1$, its inverse can be expressedas$\bbA^-1 = (\bbU\bb\Lambda\bbU^-1)^-1 = \bbU\bb\Lambda^-1\bbU^-1$;however, since $\bb\Lambda$ is diagonal,$\bb\Lambda^-1 =\diag1/\lambda_1,\dots,1/\lambda_n$. (This doesnot suggest, of course, that this is the computationally preferred wayto invert matrices, as the eigendecomposition itself is a costlyoperation).

A similar idea can be applied to the square of a matrix:$\bbA^2 = \bbU\bb\Lambda\bbU^-1 \bbU\bb\Lambda\bbU^-1 = \bbU\bb\Lambda^2\bbU^-1$and, again, we note that$\bb\Lambda^2 =\diag\lambda_1^2,\dots,\lambda_n^2$. By usinginduction, we can generalize this result to any integer power $p \ge 0$:$\bbA^p = \bbU\bb\Lambda^p\bbU^-1$. (Here, if, say, $p=1000$,the computational advantage of using eigendecomposition might be welljustified).

The above procedure is a standard way of constructing a matrix function (thisterm is admittedly confusing, as we will see it assuming another meaning); forexample, matrix exponential and logarithm are constructed exactly like this.Note that the construction is sharply different from applying the function$\varphi$ element-wise!

Symmetric square matrices define an important family of functions calledquadratic forms that we will encounter very often in this course.Formally, a quadratic form is a scalar function on $\RR^n$ given by\bbx^\Tr \bbA \bbx = \sum_i,j=1^n a_ij x_i x_j, where$\bbA$ is a symmetric $n \times n$ matrix, and $\bbx \in \RR^n$.

Since $\bbU^\Tr$ is full rank, the vector $\bby$ is also anarbitrary non-zero vector in $\RR^n$ and the only way to make the lattersum always positive is by ensuring that all $\lambda_i$ are positive.The very same reasoning is also true in the opposite direction.

Geometrically, a quadratic form describes a second-order (hence the namequadratic) surface in $\RR^n$, and the eigenvalues of the matrix$\bbA$ can be interpreted as the surface curvature. Very informally,if a certain eigenvalue $\lambda_i$ is positive, a small step in thedirection of the corresponding eigenvector $\bbu_i$ rotates the normalto the surface in the same direction. The surface is said to havepositive curvature in that direction. Similarly, a negative eigenvaluecorresponds to the normal rotating in the opposite direction of the step(negative curvature). Finally, if $\lambda_i = 0$, a step in thedirection of $\bbu_i$ leave the normal unchanged (the surface is saidto be flat in that direction). A quadratic form created by a positivedefinite matrix represents a positively curved surface in alldirections. Such a surface is cup-shaped (if you can imagine an$n$-dimensional cup) or, formally, is convex; in the sequel, we willsee the important consequences this property has on optimizationproblems.

The course combines academic-level material with industry insights, leveraging resources and textbooks. You'll see how university concepts seamlessly translate into practical applications. The course features practical examples, including a detailed one-hour walkthrough of solving systems of linear equations with Gaussian elimination by hand, a core technique in linear algebra.

This course offers a solid foundation in linear algebra, serving as a fantastic warm-up for anyone looking to explore generative AI in our upcoming courses. Watch the full course on the freeCodeCamp.org YouTube channel (6-hour watch).

This crash course is targeted to Imaging Science Graduate Students and affiliated students, and researchers (McKelvey School of Engineering, Radiology). However, all WashU graduate students and researchers are welcome to participate in the courses. Feel free to reach out to us (imsci.wus...@gmail.com) if you have any questions!

Concepts covered in this crash course will be tailored toward becoming familiar with the mathematical concepts. Therefore, complete mastery of these skills is not expected due to the 9-week time constraint. This crash course is divided into the following three sections:

Disclaimer: In a typical university setting, these concepts would be taught over the course of many semesters. For this reason, the goal of the crash course is familiarity with these concepts and not mastery!

Tutoring sessions will be held in-person with Zoom option 1-2 times per week, and students will be encouraged to bring questions to these sessions. There will be one beginner and one advanced problem session, during which tutors will walk you through key concepts and go over example problems. Feel free to join as many sessions as you like.

Making the decision to register for the math crash course is entirely up to you. We have provided the following self-assessments for each of the sections to allow you to determine if you are prepared for the concepts that will be appearing in your imaging science courses. The course is designed to cover introduction-level material so no math background is required.

I'm reading Linear Algebra book and it's completely foreign to me. The author starts out with solving linear equations (two equations, two variables) and I remember learning about that in school, but almost immediately he jumps into other things that seem to be taken directly from the Necronomicon.

Not a book, but you should check out the math curriculum over at the Khan Academy. I'm in the process of using these videos to brush up on my own math skills. They cover an extremely broad range of material, and the author has been praised for his teaching style.

I have Mathematics for 3D Game Programming and Computer Graphics 2nd edition by Eric Lengyel and it's a great refresher on math related to games. It starts with a very basic introduction to linear algebra with vectors and matrices and works through more advanced things like illumination, visibility determination, computational geometry, and game physics systems.

The one thing you need to make sure to do regardless of the book you choose is to do the exercises! You can't learn unless you actually do them. For fun you could even implement them to allow you to see how the math maps to code.

Frank D. Luna provides a brief but comprehensive introduction to the applicable concepts of linear algebra in his books on game development with Direct3D. You might want to take a look at his latest book, Introduction to 3D Game Programming with DirectX 10.

Concepts covered in this crash course will be tailored toward becoming familiar with the mathematical concepts. Therefore, complete mastery of these skills is not expected due to the 11-week time constraint. This crash course is divided into the following three sections:

c01484d022