Linear And Multilinear Algebra

[Basa Benejan]

Multilinearalgebra is the study of functions with multiple vector-valued arguments, with the functions being linear maps with respect to each argument. It involves concepts such as matrices, tensors, multivectors, systems of linear equations, higher-dimensional spaces, determinants, inner and outer products, and dual spaces. It is a mathematical tool used in engineering, machine learning, physics, and mathematics.[1]

While many theoretical concepts and applications involve single vectors, mathematicians such as Hermann Grassmann considered structures involving pairs, triplets, and multivectors that generalize vectors. With multiple combinational possibilities, the space of multivectors expands to 2n dimensions, where n is the dimension of the relevant vector space.[2] The determinant can be formulated abstractly using the structures of multilinear algebra.

Multilinear algebra appears in the study of the mechanical response of materials to stress and strain, involving various moduli of elasticity. The term "tensor" describes elements within the multilinear space due to its added structure. Despite Grassmann's early work in 1844 with his Ausdehnungslehre, which was also republished in 1862, the subject was initially not widely understood, as even ordinary linear algebra posed many challenges at the time.

The concepts of multilinear algebra find applications in certain studies of multivariate calculus and manifolds, particularly concerning the Jacobian matrix. Infinitesimal differentials encountered in single-variable calculus are transformed into differential forms in multivariate calculus, and their manipulation is carried out using exterior algebra.[3]

Following Grassmann, developments in multilinear algebra were made by Victor Schlegel in 1872 with the publication of the first part of his System der Raumlehre[4] and by Elwin Bruno Christoffel. Notably, significant advancements came through the work of Gregorio Ricci-Curbastro and Tullio Levi-Civita,[5] particularly in the form of absolute differential calculus within multilinear algebra. Marcel Grossmann and Michele Besso introduced this form to Albert Einstein, and in 1915, Einstein's publication on general relativity, explaining the precession of Mercury's perihelion, established multilinear algebra and tensors as important mathematical tools in physics.

In 1958, Nicolas Bourbaki included a chapter on multilinear algebra titled "Algbre Multilinaire" in his series lments de mathmatique, specifically within the algebra book. The chapter covers topics such as bilinear functions, the tensor product of two modules, and the properties of tensor products.[6]

This semester I'm taking a course in linear algebra and now at the end of the course we came to study the tensor product and multilinear algebra in general. I've already studied this theme in the past through Kostrikin's "Linear Algebra and Geometry", but I'm not sure this is enough.

My teacher didn't know what to recommend as textbook for this part of the course and he could just recommend one book that does everything in modules. Now, it's not that I'm not interested in modules, it's just that until today I've never dealt with them, so it's a little confusing to study the tensor product on that book.

For basic introduction, I strongly recommend Chern's "Lecture on Differential Geometry". I think it really inspiring. For extension, you may be interested in Clifford or geometric algebra which extends (multi-)linear algebra and everything doesn't care about its coordinate. Then Hestenes's book is quite appropriate.

For beginners, "Tensor Calculus for Physics: A Concise Guide" by Dwight E. Neuenschwander is a great choice. It covers the basics of tensors and multilinear algebra in a clear and concise manner with plenty of examples and exercises.

Yes, "Tensors, Differential Forms, and Variational Principles" by David Lovelock and Hanno Rund is a highly recommended text for advanced study. It delves into the geometric and differential aspects of tensors and multilinear algebra.

"Applications of Tensor Analysis" by A.J.M. Spencer is a great resource for those interested in the practical applications of tensors and multilinear algebra. It covers topics such as elasticity, continuum mechanics, and general relativity.

"Tensor Algebra and Tensor Analysis for Engineers: With Applications to Continuum Mechanics" by Mikhail Itskov is a self-contained text that is suitable for self-study. It provides a thorough introduction to tensors and multilinear algebra with a focus on engineering applications.

Yes, "Introduction to Tensor Calculus and Continuum Mechanics" by John H. Heinbockel is a free online textbook that covers the fundamentals of tensors and multilinear algebra. There are also various lecture notes and video lectures available on sites like YouTube and Coursera.

Over the last century, linear algebra has proven to be one of the most powerful and flexible tools for problem solving in a huge number of scientific fields including signal processing, statistics, machine learning and data science. The central objects in linear algebra are vectors and matrices, i.e., 1-way and 2-way arrays of numbers. The great power of vectors and matrices comes from their ability to represent and provide insight into signals, differential equations, dynamical systems, linear and bilinear forms, etc.

Furthermore, while matrices and vectors are convenient for recording data, many datasets depend on more than two parameters. For example, consider a dataset arising from ratings of movies on a stream service such as Netflix. The ratings of course depend on the user and movie giving two clear dependencies. However, to develop a more refined understanding of the ratings given, one could ask that the user rate the movie on a number of different aspects, e.g., plot, acting, and cinematography. In this manner, one arrives at a dataset that has a total of three dependencies: user x movie x aspect.

A major advancement in the last few decades has been to move beyond linear to the multilinear world. Though in one sense a natural extension of linear algebra, multilinear algebra is an important and powerful new framework which can be used to address an even more general collection of problems than its predecessor.

One of the fundamental tools in multilinear algebra is tensor decompositions. In a nave sense, tensor decompositions are similar to the matrix decompositions which are fundamental in linear algebra. Both matrix and tensor decompositions are used to express an array in terms of some collection of basic or elementary components. However, this is where the similarities end.

Just as higher-order tensors are more structured than their flat counterparts, tensor decompositions are significantly more structured than matrix decompositions. One of the most fundamental tensor decompositions is the Canonical Polyadic Decomposition (CPD), which expresses a given tensor as a sum of elementary components. The structure present in the CPD is responsible for one of its most important properties: low-rank CPDs are unique with very mild assumptions. Intuitively, the CPD being unique means that there is only one way to write the tensor as a sum of elementary components. The uniqueness of low-rank CPDs is in stark contrast to the matrix setting where decompositions are only unique with strong assumptions such as orthogonality or positivity of the factors.

This uniqueness is of great importance in applied settings. It means that when component information is extracted from a signal or dataset via a CPD, the components that one arrives at are the true components that are responsible for the data at hand. That is, one can be sure they that have not erroneously arrived at some other components. Uniqueness of the CPD has already been shown to have many applications, including telecommunications, radioastronomy, and biomedical data analysis.

In addition to enabling recovery of component information, tensor decompositions can be used for a multilinear version of PCA. This allows one to perform dimensionality reduction in a manner that is more sensitive to higher-order structure than traditional PCA. For example, it may be the case that a dataset is more easily compressed along a given parameter than another. Multilinear PCA is easily able to capture this behaviour. In contrast, traditional PCA can struggle to account for such behaviour since the row rank of a matrix is always equal to the column rank. That is, the amount of compression in the two parameters of a matrix must be the same. Tensors, on the other hand, allow for different compression ratios for different parameters.

Tensor techniques and decompositions also provide numerically reliable means to break the so-called Curse of Dimensionality. The curse refers to issues that are caused by the exponential dependence of computational and memory costs on the number of parameters in a given dataset. This can quickly lead to prohibitive computational costs for nave methods. However, appropriate tensor methods can be effectively employed to break this curse. As such, tensor methods have revolutionized scientific computing in high dimensions, enabling a fast and reliable computation of functions whose number of unknown values exceeds the number of atoms in the observable universe.

Another important aspect of multilinear algebraic techniques is that they bridge the gap between the linear and nonlinear worlds. A notable shortcoming of linear algebra is that, while it is highly useful in solving linear problems, many problems of interest are nonlinear. As previously mentioned, linear algebraic approaches can often be applied in nonlinear settings by using local linear approximations; however, local techniques necessarily lead to local solutions. This means that linear algebraic approaches can be left wanting in situations where one desires to find all or many solutions to a polynomial or nonlinear system of equations.

3a8082e126