Course Announcements

4/18/24: 1. Set 16 of the class notes have been posted.
4/12/24: 1. Some more codes for Program 3a have been posted.
4/10/24: 1. Study Question Set 6 and the solutions have been posted.
4/9/24: 1. Some clarifications on Given's rotations have been added to Set 7. Set 14 on the symmetric eigenvalue problem QR algorithm has had some information duplicated from Set 7 to make it more self-contained.
4/8/24: 1. Sets 13, 14, and 15 of the class notes have been posted. We will discuss Sets 14 and 15 in class (QR for symmetric eigenvalue problems and computing the SVD). Set 13 is for your reference.; 2. Several papers on the symmetric eigenvalue problem, computing the SVD and specifically Jacobi's method have been posted.; 3. Code for the Schur rank 2 and rank 4 displacement coding assignment has been posted.
4//5/24: 1. The preconditioning section of Set 10 has been modified to include more detail on the production, effects and removal of fill-in in the incomplete Cholesky discussion.
3/25/24: 1. Sets 10, 11 and 12 of the class notes have been posted. Lectures and the next programming assignment will concentrate on the CD/CG portion of Set 10. The DFT/FFT lectures will follow CD/CG. Set 11 contains a rigorous contraction-based analysis of Generalized Descent and Steepest Descent convergence rates. Set 12 contains polynomial-based proofs of the fundamental convergence theorems for CG. Sets 11 and 12 will not be covered in the lectures but are covered partly in Study Question Set 5.; 2. Study Question Set 5 and associated solutions have been posted. It covers iterative methods for symmetric positive definite linear systems. This includes material from Sets 10 and 11.
3/25/24: 1. Several papers have been posted on the various topics for the semester project talk/paper. This set is not exhaustive. Consult the references in the most recent and see me if you need help identifying the most relevant and the key original papers.; 2. Study Question Set 4 and the solutions have been posted.
3/19/24: 1. Set 8 on sparse matrix primitives has been moved to Set 9 and a new Set 8 that summarizes the theory of all rank linear least squares problem and its solution by SVD along with related topics. Computation of the SVD and related rank decompositions will be discussed later.
3/18/24: 1. Program 3a and 3b have been posted with instructions on who must do which.
3/8/24: 1. The Schur algorithm for displacement rank 4 has been added to Set 5 using the example of Cholesky factorization of the normal equations for a Toeplitz least squares problem. The approach is applicable to any symmetric matrix with low displacement rank.
3/6/24: 1. The formulas for a Given's rotation and Stewart's version have been made consistent by fixing a type in Stewart's algorithm slide.
3/5/24: 1. An additional question has been added to Study Question Set 3. It is related to an alternative method of computing the vector that defines a Householder reflector.; 2. The solutions to Study Question Set 3 has been posted.
3/4/24: 1. Study Question Set 3 has been posted.
3/2/24: 1. Set 7 of the class notes on incremental, weighted, and regularized least squares solutions has been posted.; 2. Set 8 of the class notes on sparse matrix computational primitives has been posted.
2/28/24: 1. Incomplete list of potential project/talk topics has been posted.
2/27/24: 1. Code and comments for Program 2 have been posted.; 2. A referece to discussion of hyperbolic rotations in Golub and Van Loan and a summary of the formulas have been added to Set 5.; 3. The paper by Alexander, Pan and Plemmons that discusses numerics of hyperbolic transformations has been posted. Also, note that the papers posted earlier by M. Stewart and P. Van Dooren, and M. Stewart and G. W. Stewart discuss the numerics of hyperbolic transformations.

2/17/24

1. Set 5 has been updated to reflect the discussions during the lecture on the Cholesky factorization's derivation.

2/10/24

1. Study Question Set 2 and its solutions have been posted. Note this is a large set of questions covering a wide range of topics. So you are encouraged to start reading it immediately.

2. Comments on the solutions for Program 1 have been posted.

3. Program 2 has been posted.

2/9/24

1. Sets 5 and 6 of the class notes have been posted.

2. Several papers related to Toeplitz matrices and low-rank displacement matrices have been posted for your reference. You are not responsible for the content this semester.

2/2/24

1. Matlab codes for the various versions of multiplication and solvin with triangular matrices has been posted. Two experiment drivers are included. One for LU, Lv, Uv matrix-matrix and matrix-vector multiplication codes and one for solving Lz=v and Uz-v. Column and row oriented versions of each primitive are provided. The drivers require setting parameters and selections to control the routines, problem generation, and output form used in the particular experiment set. An option is included to summarize the results for a the set of experiments in terms of a plot of the sizes of the problems run and the mean and max relative error of each set of problems that share the same value of n.

2/2/24

1. Typos in Set 4 on slides 21, 24, and 43 have been fixed and the set reposted.

1/31/24

1. Slide 34 of Set 4 has had a minor typo fixed and some clarification added.

1/30/24

1. The solutions for Graded Homework 1 have been posted.

1/29/24

1. The typo in the list of subroutines required has been fixed. (Old item 4 removed.) Two triangular solvers and two triangular matrix times a vector, and a matrix product M = L*U are all that are required.

1/25/24

1. The LAPACK user manual has been posted. It contains information and references on the current standard numerical linear algebra library.

2. Several papers on high-performance numerical linear algebra have been posted. The span several decades and include the "product form" lower triangular system solver discussed in class. In addition to high performance and parallel aspects of the algorithms these are also a good source of information about the structure of the BLAS hierarchy from a performance analysis point of view.
3. A section on data structures has been added to Set 4 and the set has been reposted.

1/24/24

1. Program 1 has been posted.

1/23/24

1. Graded Homework 1 submission link has been added to Canvas.

2. Set 4 of the class notes has been posted. It is strongly recommended that you read ahead in this set.
2/17/24: 1. Set 5 has been updated to reflect the discussions during the lecture on the Cholesky factorization's derivation.
2/10/24: 1. Study Question Set 2 and its solutions have been posted. Note this is a large set of questions covering a wide range of topics. So you are encouraged to start reading it immediately.; 2. Comments on the solutions for Program 1 have been posted.

3. Program 2 has been posted.
2/9/24: 1. Sets 5 and 6 of the class notes have been posted.

2. Several papers related to Toeplitz matrices and low-rank displacement matrices have been posted for your reference. You are not responsible for the content this semester.
2/2/24: 1. Matlab codes for the various versions of multiplication and solvin with triangular matrices has been posted. Two experiment drivers are included. One for LU, Lv, Uv matrix-matrix and matrix-vector multiplication codes and one for solving Lz=v and Uz-v. Column and row oriented versions of each primitive are provided. The drivers require setting parameters and selections to control the routines, problem generation, and output form used in the particular experiment set. An option is included to summarize the results for a the set of experiments in terms of a plot of the sizes of the problems run and the mean and max relative error of each set of problems that share the same value of n.
2/2/24: 1. Typos in Set 4 on slides 21, 24, and 43 have been fixed and the set reposted.
1/31/24: 1. Slide 34 of Set 4 has had a minor typo fixed and some clarification added.
1/30/24: 1. The solutions for Graded Homework 1 have been posted.
1/29/24: 1. The typo in the list of subroutines required has been fixed. (Old item 4 removed.) Two triangular solvers and two triangular matrix times a vector, and a matrix product M = L*U are all that are required.
1/25/24: 1. The LAPACK user manual has been posted. It contains information and references on the current standard numerical linear algebra library.

2. Several papers on high-performance numerical linear algebra have been posted. The span several decades and include the "product form" lower triangular system solver discussed in class. In addition to high performance and parallel aspects of the algorithms these are also a good source of information about the structure of the BLAS hierarchy from a performance analysis point of view.
3. A section on data structures has been added to Set 4 and the set has been reposted.
1/24/24: 1. Program 1 has been posted.
1/23/24 1. Graded Homework 1 submission link has been added to Canvas. 2. Set 4 of the class notes has been posted. It is strongly recommended that you read ahead in this set.
1/22/24: 1. Graded Homework 1 due date is now 11:59 PM Friday January 26, 2024.
1/18/24: 1. Set 3 of the class notes has been posted.
1/16/24: 1. Graded Homework 1 has been posted.
1/15/24: 1. Solutions for Study Question Set 1 have been posted.
1/12/24: 1. The definition of an eigenvalue, eigenvector pair has been added to the end of Set 1 of the class notes and the pdf file has been reposted.; 2. Study Question Set 1 has been posted.
1/9/24: 1. Set 2 of the class notes has been modified and reposted.
1/7/24: 1. The class syllabus has been posted.; 2. Sets 1 and 2 of the class notes have been posted.

Course Announcements

4/18/24

1. Set 16 of the class notes have been posted.

4/12/24

1. Some more codes for Program 3a have been posted.

4/10/24

1. Study Question Set 6 and the solutions have been posted.

4/9/24

1. Some clarifications on Given's rotations have been added to Set 7. Set 14 on the symmetric eigenvalue problem QR algorithm has had some information duplicated from Set 7 to make it more self-contained.

4/8/24

1. Sets 13, 14, and 15 of the class notes have been posted. We will discuss Sets 14 and 15 in class (QR for symmetric eigenvalue problems and computing the SVD). Set 13 is for your reference.

2. Several papers on the symmetric eigenvalue problem, computing the SVD and specifically Jacobi's method have been posted.

3. Code for the Schur rank 2 and rank 4 displacement coding assignment has been posted.

4//5/24

1. The preconditioning section of Set 10 has been modified to include more detail on the production, effects and removal of fill-in in the incomplete Cholesky discussion.

3/25/24

2. Study Question Set 5 and associated solutions have been posted. It covers iterative methods for symmetric positive definite linear systems. This includes material from Sets 10 and 11.

3/25/24

1. Several papers have been posted on the various topics for the semester project talk/paper. This set is not exhaustive. Consult the references in the most recent and see me if you need help identifying the most relevant and the key original papers.

2. Study Question Set 4 and the solutions have been posted.

3/19/24

1. Set 8 on sparse matrix primitives has been moved to Set 9 and a new Set 8 that summarizes the theory of all rank linear least squares problem and its solution by SVD along with related topics. Computation of the SVD and related rank decompositions will be discussed later.

3/18/24

1. Program 3a and 3b have been posted with instructions on who must do which.

3/8/24

1. The Schur algorithm for displacement rank 4 has been added to Set 5 using the example of Cholesky factorization of the normal equations for a Toeplitz least squares problem. The approach is applicable to any symmetric matrix with low displacement rank.

3/6/24

1. The formulas for a Given's rotation and Stewart's version have been made consistent by fixing a type in Stewart's algorithm slide.

3/5/24

1. An additional question has been added to Study Question Set 3. It is related to an alternative method of computing the vector that defines a Householder reflector.

2. The solutions to Study Question Set 3 has been posted.

3/4/24

1. Study Question Set 3 has been posted.

3/2/24

1. Set 7 of the class notes on incremental, weighted, and regularized least squares solutions has been posted.

2. Set 8 of the class notes on sparse matrix computational primitives has been posted.

2/28/24

1. Incomplete list of potential project/talk topics has been posted.

2/27/24

1. Code and comments for Program 2 have been posted.

2. A referece to discussion of hyperbolic rotations in Golub and Van Loan and a summary of the formulas have been added to Set 5.

2/17/24

1. Set 5 has been updated to reflect the discussions during the lecture on the Cholesky factorization's derivation.

2/10/24

1. Study Question Set 2 and its solutions have been posted. Note this is a large set of questions covering a wide range of topics. So you are encouraged to start reading it immediately.

2. Comments on the solutions for Program 1 have been posted.

3. Program 2 has been posted.

2/9/24

1. Sets 5 and 6 of the class notes have been posted.

2. Several papers related to Toeplitz matrices and low-rank displacement matrices have been posted for your reference. You are not responsible for the content this semester.

2/2/24

2/2/24

1. Typos in Set 4 on slides 21, 24, and 43 have been fixed and the set reposted.

1/31/24

1. Slide 34 of Set 4 has had a minor typo fixed and some clarification added.

1/30/24

1. The solutions for Graded Homework 1 have been posted.

1/29/24

1. The typo in the list of subroutines required has been fixed. (Old item 4 removed.) Two triangular solvers and two triangular matrix times a vector, and a matrix product M = L*U are all that are required.

1/25/24

1. The LAPACK user manual has been posted. It contains information and references on the current standard numerical linear algebra library.

3. A section on data structures has been added to Set 4 and the set has been reposted.

1/24/24

1. Program 1 has been posted.

1/23/24

1. Graded Homework 1 submission link has been added to Canvas.

2. Set 4 of the class notes has been posted. It is strongly recommended that you read ahead in this set.

1/22/24

1. Graded Homework 1 due date is now 11:59 PM Friday January 26, 2024.

1/18/24

1. Set 3 of the class notes has been posted.

1/16/24

1. Graded Homework 1 has been posted.

1/15/24

1. Solutions for Study Question Set 1 have been posted.

1/12/24

1. The definition of an eigenvalue, eigenvector pair has been added to the end of Set 1 of the class notes and the pdf file has been reposted.

2. Study Question Set 1 has been posted.

1/9/24

1. Set 2 of the class notes has been modified and reposted.