Linear Algebra ============== There is another commercial package for doing math which is very popular, particularly among engineers, that you should know about: `MATLAB `_ *Matlab* stands for "*Matrix Laboratory*", which gives you some indication of its purpose. **Linear Algebra** is the mathematics of matrices and vectors, but that's almost the simplest way to think about it. It's the mathematics of *systems of linear equations* (which can be written in matrix form, no matter how many, and *many* other related topics. In fact, we will be not be using *Matlab*, but rather a free "clone" with much of *Matlab*'s functionality, called *Octave*. More on that below. | Back to *Linear Algebra*. Believe me, linear algebra is a *FREEKING DEEP* subject. It should be required of all physics and engineering students, and included in PACS. Almost everything you have learned in advanced math has a linear algebra connection. This is very useful for many things: + Many data sets can be thought of as a long vector. Calculations on these data can be done by MATLAB. An example might be Stock Market data over some period of time. + Systems of linear equations. Many other problems, such as partial differential equations, statics problems, fluid dynamics, etc. can be given a matrix/linear algebra form. See: `Finite Element Methods `_ + Many things, like the set of all periodic functions :math:`f(x) = f(x+L)` can be represented as a set of vectors in an *infinite dimensional vector space*. That's *Fourier Analysis*... + Signal Processing. MATLAB has many tools for signal analysis, Fourier transforms, audio and image processing (after all an image is just a matrix of pixel values--see what I mean about most problems having a linear algebra representation?). + Representing *rotations*, *tranlations* (both from one language to another as well as the physics meaning: movement from place to another in space. Actually, if you think about Heisenberg's `Matrix Formulation of Quantum Mechanics `_, in fact *Change* in general can be represented by an infinite dimensional matrix in the space of all possibilities... + Representing *symmetry*. ...just to name a few. We are fortunately going to take baby steps into linear algebra. | Whereas Mathematica was a tool whose primary use is for **symbolic** (algebraic) calculations, MATLAB is fundamentally **numerical**. That means that it manipulates matrices and vectors as tables and arrays of numbers (like **[1, -1, 3]**), not as algebraic symbols (like *a, b, g*, and *x* ) as Mathematica did. Matrix Algebra -------------- First, let's make sure you remember what *matrix* :math:`\times` *matrix*, and *matrix* :math:`\times` *vector multiplication* is. If you are rusty on multiplying *matrices* (the plural of *matrix*), have a look at the Wikipedia page on `Matrix Multiplication `_. Like many Wikipedia pages, it is rather dense. But it's pretty complete. Anothere place to brush up on your *matrix* :math:`\times` *matrix multiplication* is `Kahn Academy's page `_ on matrix algebra. .. warning:: Make sure you understand the basics of matrix algebra before going on. | ------------------------------- In particular, you need to be able to understand the problems in this quick little quiz. You can check your answers `below `_. | If .. math:: {\bf M} = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) What is :math:`{\bf M}^2 = ?` *i.e.*, what is the *matrix* :math:`\times` *matrix* product? .. math:: {\bf M}^2 = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) \times \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) = ? | Go that one? Good. | ----------------------------------------------------------- What about this *matrix* :math:`\times` *vector* product? We will use a *single-column matrix* for the *vector* :math:`\vec v` is .. math:: \vec v = \left( \begin{array}{c} 2 \\ 1 \\ -1 \end{array} \right) What is :math:`{\bf M} \times \vec v`? (Also, we will usually leave the :math:`\times` symbol out) .. math:: {\bf M} \vec v = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) \left( \begin{array}{c} 2 \\ 1 \\ -1 \end{array} \right) = ? Check your answers `here `_ before going on. | ------------------------- | A Linear System of Equations ---------------------------- Here's a typical linear algebra problem. Let's say you need to solve these three coupled *linear* equations for the variables :math:`x, y`, and :math:`z`. .. math:: \begin{array}{ccccc} x & +2y & -z & = & 4 \\ 2x & ~ & -2z & = & 1 \\ x & +y & ~ & = & -4 \end{array} These are *linear* equations because the variables :math:`x, y`, and :math:`z` appear in linear form, *i.e.* raised to the 1st power, :math:`x^1, y^1, z^1`, as opposed to quadratic :math:`x^2`, cubic :math:`y^3`, or other *non-linear* (:math:`\neq=1`) powers. You could also write this in matrix form. Let .. math:: {\bf A} = \left( \begin{array}{ccc} 1 & +2 & -1 \\ 2 & 0 & -2 \\ 1 & +1 & 0 \end{array} \right) .. math:: {\vec r} = \left( \begin{array}{c} x \\ y \\ z \end{array} \right) and .. math:: \vec b = \left( \begin{array}{c} 4 \\ 1 \\ -4 \end{array} \right) | Remembering the rules of *matrix* :math:`\times` *vector* multiplication. We can write our three linear equations *very* succinctly as :math:`{\bf A} \vec r = \vec b`. | .. math:: {\bf A} \vec r = \left( \begin{array}{ccc} 1 & +2 & -1 \\ 2 & 0 & -2 \\ 1 & +1 & 0 \end{array} \right) \left( \begin{array}{c} x \\ y \\ z \end{array} \right) = \left( \begin{array}{ccc} x & +2y & -z \\ 2x & ~ & -2z \\ x & +y & ~ \end{array} \right) = \left( \begin{array}{c} 4 \\ 1 \\ -4 \end{array} \right) = {\vec b} | where :math:`{\bf A}` is the matrix on the left, :math:`\vec r` is the (column-matrix) vector of unknowns [ :math:`x, y, z` ], and :math:`\vec b` is the vector on the right side. | Let's now define another (unknown) matrix: :math:`{\bf A}^{-1}` which will be the **inverse of A**. | If :math:`{\bf A}` was a normal *scalar* variable (like 5, or :math:`\pi`), let's call it :math:`a`, then the *inverse* of :math:`a` would be :math:`a^{-1} = \frac{1}{a}`. Then :math:`a^{-1} a = 1` For *matrices*, given :math:`{\bf A}`, we want to find another matrix :math:`{\bf A}^{-1}`, defined such that if you multiply :math:`{\bf A}^{-1} \times {\bf A}` you get something like :math:`{\bf 1}` (One). | In matrix algebra, :math:`{\bf 1}` (One) is called **The Identity Matrix**. It is the matrix that you can multiply times *ANY* matrix :math:`{\bf M}`, and get the same matrix :math:`{\bf M}` as the result. That's what :math:`1~~` (One) does for *scalar* numbers: for any :math:`a`, :math:`1 \times a = a`. | The *Identity Matrix*, :math:`{\bf I}`, looks like this (in 3-dimensions): .. math:: {\bf I} = \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right) Try it yourself: Multiply .. math:: {\bf I} \times {\bf M} = \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right) \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) = {\bf M} Please check that this is true. | The **Inverse of matrix** :math:`{\bf A}` is another matrix, which we write as :math:`{\bf A}^{-1}`, such that if you multiply :math:`{\bf A}^{-1} \times {\bf A}` you get :math:`{\bf I}`, the Identity matrix. .. math:: {\bf A}^{-1} {\bf A} = {\bf I} | *Now*, if we knew the *inverse*, :math:`{\bf A}^{-1}`, of :math:`{\bf A}`, we could use this to find the vector of unknowns :math:`\vec r` in our original problem---the linear set of equatoins we started with. Because, if .. math:: {\bf A}^{-1} {\bf A} = {\bf I} then, multiply both sides of the equation above, :math:`{\bf A} {\vec r} = {\vec b}`, by :math:`{\bf A}^{-1}` .. math:: {\bf A} {\vec r} &= {\vec b} \\ {\bf A}^{-1}{\bf A} {\vec r} &= {\bf A}^{-1} {\vec b} \\ {\bf I} {\vec r} &= {\bf A}^{-1} {\vec b} \\ {\vec r} &= {\bf A}^{-1} {\vec b} *Wow*! All we have to do is find the matrix :math:`{\bf A}^{-1}` and we can use it to find our *unknown* vector :math:`{\vec r}`, and *solve the set of linear equations*. The solution is just .. math:: {\vec r} = {\bf A}^{-1} {\vec b} | -------------------------------- On to Octave. | A Taste of **Octave** --------------------- We will use a free "clone" of *Matlab*, called **Octave**. It doesn't have all the fanciest features of *Matlab*--it actually uses *Gnuplot* for its graphics, but it does have much of the numerical linear algrebra capability. Octave comes standard on *Fedora Scientific*., so you can run it on your own computer; you don't have to log on to another computer. To start **Octave**, simply open a terminal and type: ``octave``. You should see this: :: GNU Octave, version 3.0.3 Copyright (C) 2008 John W. Eaton and others. This is free software; see the source code for copying conditions. ... octave:1> Once again, you get a prompt: **octave:1>** waiting for input. | At the **octave:1>** prompt, type: :: octave:1> A = [1, 2, -1; 2, 0, -2; 1, 1, 0] When you hit ENTER, *octave* responds with :: A = 1 2 -1 2 0 -2 1 1 0 You have just entered the *matrix form of :math:`{\bf A}`. | Notice the *octave syntax* for matrices: + elements in a *row* are separated by *commas* [1 **,** 2 **,** -1; 2 **,** 0...] + *rows* are separated by semicolons; [1, 2, -1 **;** 2, 0, -2 **;** 1, 1, 0] | Next enter the :math:`{\vec b}` *vector*. :math:`{\vec b}` is a *single-column matrix*, with only one element per *row*, so each entry is separated by a semicolon "**\;**" :: octave:2> b = [4; 1; -4] b = 4 1 -4 | Now, to find :math:`{\bf A}^{-1}`, the *inverse of :math:`{\bf A}`, type :: octave:3> Ainv = inv(A) Ainv = -0.50000 0.25000 1.00000 0.50000 -0.25000 -0.00000 -0.50000 -0.25000 1.00000 We have defined a new matrix named **Ainv** and called octave's *matrix inverse* function: **inv()**. | Octave uses the star **\*** for matrix multiplication. | Let's check that :math:`{\bf A}^{-1} {\bf A} = {\bf I}`. Mathematically, we are asking if .. math:: {\bf A}^{-1} {\bf A} = \left( \begin{array}{ccc} -0.5 & 0.25 & 1.0 \\ 0.5 & -0.25 & 0.0 \\ -0.5 & -0.25 & 1.0 \end{array} \right) \left( \begin{array}{ccc} 1 & +2 & -1 \\ 2 & 0 & -2 \\ 1 & +1 & 0 \end{array} \right) = {\bf I}~~~? You can check this "*by hand*", or use *Octave*. | At the **octave:>** prompt type :: octave:4> Ainv * A ans = 1 0 0 0 1 0 0 0 1 We have shown that indeed, **Ainv** = :math:`{\bf A}^{-1}`. | Now let's solve those `coupled linear equations `_. Type: :: octave:5> Ainv * b ans = -5.7500 1.7500 -6.2500 There you go: The unknowns, :math:`x, y`, and :math:`z` are .. math:: \vec r = \left( \begin{array}{c} x \\ y \\ z \end{array} \right) = {\bf A}^{-1} {\vec b} = \left( \begin{array}{ccc} -0.5 & 0.25 & 1.0 \\ 0.5 & -0.25 & 0.0 \\ -0.5 & -0.25 & 1.0 \end{array} \right) \left( \begin{array}{c} 4 \\ 1 \\ -4 \end{array} \right) = \left( \begin{array}{c} -5.75 \\ 1.75 \\ -6.25 \end{array} \right) In other words, the solution of the original set of equations is .. math:: \left( \begin{array}{ccc} x & = & -5.75 \\ y & = & 1.75 \\ z & = & -6.25 \end{array} \right) Pretty easy. You can verify that these solve the above `equations `_. .. math:: \begin{array}{ccccc} x & +2y & -z & = & 4 \\ 2x & ~ & -2z & = & 1 \\ x & +y & ~ & = & -4 \end{array} Consider the last equation of the set: :math:`x + y = -4` Indeed: :math:`-5.75 + 1.75 = -4`. Probably this doesn't seem like we did rocket science, but imagine if we had a system of 500 coupled linear equations! It would be just as easy, except for typing in the matrix and vector, which you probably would program Octave to construct the vector for you. At least now, you have a little taste of what Octave can do. | --------------------------- Homework ======== Homework 11 is `here `_ | | | | | | | | | | ------------------------ Answers to practice quiz ~~~~~~~~~~~~~~~~~~~~~~~~ .. math:: {\bf M}^2 = {\bf M M} = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) = \left( \begin{array}{ccc} 1 & 0 & -4 \\ -4 & 5 & 0 \\ -1 & 1 & 8 \end{array} \right) And .. math:: {\bf M} \vec v = \left( \begin{array}{ccc} 2 & -1 & 0 \\ 3 & -2 & 4 \\ -1 & 1 & 2 \end{array} \right) \left( \begin{array}{c} 2 \\ 1 \\ -1 \end{array} \right) = \left( \begin{array}{c} 3 \\ 0 \\ -3 \end{array} \right) Click `here to jump `_ back to the quiz