Course Announcements

10/30/26

1. Four papers about LBFGS in stochastic and deterministic settings have been posted along with an early excellent discussion of more advanced line search methods by Barzilai and Ben-Tal.

10/26/26

1. A large amount of material for convex functions and the minimization has been added to Set 2 as mentioned in class. This provides sufficient conditions for the descent methods covered by the bounded condition and sufficient decrease theory to converge to local/global minimizers. You are not responsible for the material in any detail. It does, however, have details on the definition and important properties of convex functions that are crucial to understanding the performance of descent methods and their rates of convergence. Some extensions to it will be used in the section on Gradient Projection methods this semester.

10/26/26

1. Sets 14 and 15 have been posted.

10/22/26

1. Two new papers posted: Li and Wan on an adaptive BB stepsize method for large problems; and Kroll and Keyes on matrix-free Newton-Krylov iterations for solving nonlinear systems of equations. This is more general than the systems solved in optimization but it can be adapted to that situation.

10/21/26

1. Sets 10, 11, 12, and 13 of the class notes have been posted.

2. Local convergence rates for descent and Inexact Newton (boosting and iterative method-based) have been added to Set 7. These complement the Quasi-Newton convergence results of Set 10.

3. The dissertation of Shuguang Zhang has been posted. This gives a good survey of the literature on various forms of limited memory Quasi-Newton methods and extensions to the full Broyden family in deterministic and stochastic forms for manifolds and Euclidean spaces. It represents the state-of-the-art.

10/19/26

The Set 7 proof of descent in Truncated CG has been modified to add the details mentioned in the last lecture.

10/8/26

1. The university is closing at 12:00 on Friday. Rather than lecture remotely, I will be available on zoom in the meeting room linked on my homepage d uring class time to answer any questions you may have.

10/7/26

1. Sets 8 and 9 of the class notes have been posted.

10/6/26

1. Typos have been fixed in Set 6.

2. Set 7 is now the inexact Newton discussion (was Set 8).

3. Convexity, Coercive and L-smooth function implications in old Set 7 will be discussed after some sets on local model-based methods along with the related convergence theorems.

10/5/26

1. Sets 7 and 8 of the class notes have been posted.

9/24/26

1. A typo in the convergence rate theorem at the end of Set 4 of the class notes has been corrected and the set reposted.

9/22/26

1. Set 6 of the class notes has been posted.

9/3/26

1. Set 5 of the class notes has been posted.

9/4/26

1. Minor changes made to Set 2 and the set has been reposted.

2. There will be no office hours today.

8/30/26

1. Some clarifications have been added to Set 3 of the class notes and the set has been reposted.

2. Set 4 of the class notes has been posted.

8/26/26

1. Set 3 of the class notes has been posted.

8/24/26

1. Security information have been posted on the canvas course page.

2. The syllabus and a description of the format for submitting program solutions has been posted.

3. Sets 1 and 2 of the class notes and many reference papers have been posted.