## CIM: Numerical optimization

This course is organized by the Graduate school at the Center for Interdisciplinary Mathematics (CIM)
If you intend to participate in the course, please send an email to maya.neytcheva@it.uu.se. Thank you.

### General description

This course gives an overview of different numerical optimization methods that are common in various applications. The material is split into three modules as follows.
• Multi-objective and model-based optimization problems. Stochastic methods (4hp)
This module explores techniques from artificial intelligence and machine learning for solution of \u2018black-box\u2019 optimization problems. In particular, evolutionary algorithms will be studied as means to solve single and multi-objective optimization problems. Although heuristic-based, they typically perform very well in practice. However, they usually require a large number of objective function evaluations to converge. For computationally expensive problems, this limits their applicability. Machine learning methods will be studied as tools to solve such problems. Model-driven optimization techniques such as Bayesian optimization and surrogate modelling will be discussed in depth. In general, the aim of the module is to introduce state-of-the-art methods that can be quickly applied to solve a wide variety of optimization problems. Towards that end, the strengths, weaknesses and applicable use-cases of each method will be discussed.
• PDE-constrained optimization problems. Solution methods based on first order necessary conditions (3hp)
Optimal design, optimal control and parameter estimation of systems governed by partial differential equations (PDE) give rise to a class of problems referred to as 'PDE-constrained optimization' (OPT-PDE).OPT-PDE pursues the idea to influence phenomena and processes, governed by PDEs, by use of a control function. As PDEs describe almost every aspect of physics, chemistry, engineering, biology, finance etc., that fit into a continuum framework, OPT-PDE can be regarded as the ultimate/farthermost goal of any application problem to steer the underlying systems in a desired way. In this module we discuss the problem formulation, various forms of the cost functional and regularization techniques, discuss the interplay between regularization, stabilization and discretization parameters in order to understand their mutual dependence and the need to balance the corresponding errors. To this end we consider some typical PDEs as constraints. The algebraic problems that arize in OPT-PDE problems lead to matrices that possess a particular structure. We will also touch the question how to solve these algebraic systems efficiently by iterative solution methods.
Inverse problems constitute one very important class of OPT-PDE problems. As an example, think about the need to determine an uncnoun parameter (or a set of parameters) in a PDE model, so that certain observations (measurements) are met. We discuss the difficulties arising within the above framework and various solution approaches.

Module 1 will require a work on a predefined problem that has to be presented at a seminar occasion. Modules 2 and 3 require to fulfill an assignment consisting of implementing a problem numerically in Matlab and a written report on that.
It is allowed to choose all or some of the modules. Information on the assignments to be done in order to pass the course follows.

### Course prerequisites

Module 1: Knowledge of linear algebra and probability theory. Familiarity with a programming environment (e.g., MATLAB/Python).
Modules 2 and 3: Knowledge on PDEs and their numerical solution using some discretization methods is necessary. Knowledge on numerical linear algebra, nonlinear solution methods and iterative solution methods is beneficial.
The lectures will be given at the Department Information Technology, Uppsala University.

### Lecturers

Prashant Singh (PS) Maya Neytcheva (MN) Ken Mattsson (KM)

### Additional course materials (to be updated)

Module 1: Lecture 1   Lecture 2   Lecture 3   Assignment   Dropbox link

### Detailed time schedule

Nov 28 10:15-12:00
2344
KM
Date Topic(s) Time
Location
Lecturer
Sep 18 Lecture 1: Introduction to and motivation behind black-box optimization, real-world applications, simulation-based design, parameter-tuning, evolutionary optimization 10:00-12:00
2244
PS
Sep 21 Lecture 2: 'Model-driven optimization: Bayesian optimization and surrogate modelling 10:15-12:00
2345
PS
Sep 26 Lecture 3: Multi-objective and constrained optimization 10:15-12:00
2344
PS
Oct 3 Seminar: Presentation of projects and discussions 13:15-15:00
2344
PS

Oct 11 Lecture 1: Introduction to optimization problems, constrained by PDEs. Problem formulation, general theory. The method of Lagrange Multipliers. 10:00-12:00
2247
MN
Oct 12 Lecture 2: Solution approaches for OPT-PDE paroblems. Model problems. Optimality conditions, the KKT system. Discretization strategies. 10:15-12:00
2345
MN
Oct 19 Lecture 3: Model problems, continued. Time-dependent constraints: general case, time-periodic constraints 10:15-12:00
1213
MN
Oct 20 Lecture 4: Box constraints and non-smooth Newton method. Solving large scale OPT-PDE problems - VERY brief on accellerated iterative solution methods. Discussion on project work 10:15-12:00
2344
MN

Nov 14 Introduction to the adjoint method (the continuous case) 10:00-12:00
2344
KM
Nov 16 The adjoint method (the semi-discrete case) 10:15-12:00
2344
KM
Nov 28 Presentation of projects 10:15-12:00
2344
KM

Recommended books:

1. Module 1:
1. Evolutionary Optimization, Dan Simon (Wiley)
2. Engineering Design via Surrogate Modelling: A Practical Guide, Alexander Forrester (Wiley)
3. Taking the Human Out of the Loop: A Review of Bayesian Optimization, B. Shahriari et. al., Proceedings of the IEEE
2. Module 2: There will be slides provided as course material. Details can be found in, for instance, Juan Carlos De los Reyes, Numerical PDE-Constrained Optimization, Springer 2015. (Available as e-book via the UU library). Another source to use are the slides by Matthias Heinkenschloss at https://www.siam.org/meetings/op08/Heinkenschloss.pdf
3. Module 3: To come

Organization issues:
Some instructions how to find us in Uppsala are to be found here .

Last changed on September 18, 2017.
Mail to: Maya dot Neytcheva "at" it dot uu dot se "