A Design Optimization Course Layout and Syllabus
textbook can be used for a variety of course formats.
The template below is offerred for a typical audience
for entry graduate/advanced senior engineering students,
similar to the course Design Optimization (ME 555) that has been taught at the University of
Michigan in Ann Arbor, first introduced in 1980.
The course aims at providing the student with a first
exposure to a rational integration of traditional
design methodologies with concepts and techniques
of modern optimization theory and practice. "Design" is defined in a broad context and students from diverse disciplines, including outside engineering, can benefit from the course. The student
learns to create appropriate mathematical optimization
models and to use analytical and computational techniques
to solve them. Availability of software analysis models, such as finite element analysis, makes possible optimization of increasingly complex designs.
The exposition is designed for students without prior
knowledge of optimization who wish to apply optimization
in practical design situations or who may pursue advanced
research work. The treatment of topics is mathematically
rigorous but with an emphasis on practical use.
Topics relate primarily to nonlinear
models with a finite number of variables (nonlinear
programming) including the following: mathematical
modeling for design optimization, boundedness and
monotonicity analysis of models, differential theory
and optimality conditions, numerical methods of nonlinear
programming, hierarchical system design, partitioning
and coordination, and industrial case studies. Knowledge
of basic differential calculus and linear algebra
There are three parts in the course work: (i) A set
of homework assignments and three in-class exams;
these are intended as aids to understanding the theoretical
content of the course; (ii) an individual project
where a design problem chosen by each student is formulated,
analyzed and solved, as a independent subsystem of the larger system; (iii) a team project where
2-4 students combine their individual subsystem design
problems into a system design synthesis. The teams for
Part (iii) are formed early on so that the individual
design projects can be selected to fit into the final
system design problem. There is no final exam other than project presenations. A typical
grade distribution can be
The project grading can be based on the model's originality
and on the successful application of the course theory
for design optimization model analysis, numerical
solution, and interpretation of results.
alternative is to omit Part III above in order to
simplify project work or to de-emphasize team work.
For large classes a single project can be assigned
to a group, while for smaller classes the projects
can be conducted individually.
Several state-of-the-art optimization software packages
should be available for student use in assignments and design
project work, including commercial and in-house products.
are expected to program the analysis models needed to compute objective and constraint functions and to interface them with the available optimization
packages. However, students are
not typically expected to code their own optimization
algorithms, except as practice in homework. Modeling in matlab is quite common and interfacing matlab/simulink models with optimization packages is quite easy.
The course uses mathematics from several classical
areas typically taught at the first or second year in typical engineering or scheince curricula. This mathematical knowledge is required
at a working level: the student must have a good understanding
of the concepts and techniques rather than superficial
familiarity. Mathematical understanding is needed
because design optimization is treated as a mathematical
problem that can be solved by numerical algorithms.
The problem itself (nonlinear optimization) is a very
difficult one to solve and the available techniques
(commercial codes or otherwise) can fail to give an
answer or, worse, give an answer that is wrong. A
deeper knowledge of the mathematical reasoning behind
the algorithms is important for two reasons: one,
so that the design engineer can formulate a problem
is a way that some of the potential difficulties are
alleviated; two, so that when numerical results are
generated, proper interpretation can be given to them
and steps taken to achieve solution robustness.
Although some of math concepts can be reviewed in
the course, lack of familiarity will increase the
student’s workload, at least the first few weeks
of the course. However, this knowledge is what will
distinguish an informed user from a casual one (who
may get wrong answers and not even know it).
The main mathematical concepts needed are as follows.
Calculus and Linear Algebra
addition, multiplication, dot products, euclidean
Matrix addition, multiplication, product, inversion,
Linear combinations, linear dependence/independence
Basis of a vector space, subspace, null space
Solution of linear systems of equations
closed sets, neighborhoods, sequences, limits
Continuity and differentiability of functions
Partial derivatives, gradient vector
brief summary of mathematical concepts needed for
the course can be obtained below.
For most programs of study, the book contains much more material than would be necessary for three lecture hours a week for one semester.
Any course that requires an optimal design project should include Chapters 1, 2, and 9. Placing emphasis on problem formulation should include Chapter 3. A strong theme on gradient-based solution methods would include material from Chapters 4, 5, and 6. A selection from the nongradient approaches in Chapter 7 would round out the basic ideas for all algorithms in use today. Chapter 8 on systems optimization would be the basis for a course that emphasizes what has become known as multidisciplinary design optimization strategies.
Linear programming for problems with purely linear functions is included in Chapter 5 on boundary optima, as a special case of boundary-tracking, active set strategy algorithms, thus avoiding the overhead of the specialized terminology traditionally associated with
the subject. Problems with discrete variables first encountered in Chapter 3 can be
addressed with the methods described in Chapter 7.
An approximate allocation of topics throughout a very ambitious semester is as follows.
1 - 2
Optimization Process Overview
formulations and engineering design. Basic concepts
of models. Optimization and Quality-Function-Deployment.
Preference structures and Pareto optimality. Boundedness
of models. Parametric solutions. Optimality conditions.
Numerical methods. Computer software introduction.
Simple case studies. [Chapter 1]
3 - 4
and Monotonicity Analysis
||Building optimization models. Simulation-based models, surrogate models. Constraint
activity. Monotonicity Principles. Well-bounded
models. Monotonicity table. Model transformations.
Activity maps. [Chapters 2, 3]
5 - 7
Theory: Interior and Boundary Optima
Approximations of functions. Necessary and sufficient
conditions. Convexity. Gradient and Newton's methods.
Line search. Trust regions. [Chapter 4]
directions. Regular points. Necessary and sufficient
conditions for equality constrained problems.
Lagrange multipliers. Generalized reduced gradient
method. Karush-Kuhn-Tucker conditions. Sensitivity
analysis. Linear programming as a special case of nonlinear programming [Chapter 5]
8 - 9
||Gradient-based Numerical Methods: Basic Strategies
and Global Convergence. Termination criteria.
Exact and inexact line search. Quasi-Newton Methods.
Penalty methods. Sequential Quadratic Programming.
Trust region algorithms. [Chapter 6]
||Nongradient Numerical Methods
Coordinate Search. Generalized Pattern Search. Nelder–Mead Algorithm. Simulated Annealing. Genetic Algorithm. Multiobjective Genetic Algorithm. Particle Swarm Optimization. Black-Box Methods. Dividing Rectangles (DIRECT).
Efficient Global Optimization. [Chapter 7}
approximation. Automatic differentiation. Scaling.
Active set strategies. Degeneracy. Model transformations. [Chapter 9]
||System interactions. Distributed optimization. Partitioning
and coordination strategies .Hierarchical and nonhierarchical methods. MDF and IDF. Target cascading.
Modular and shared component architectures. Design
optimization and product development. [Chapter 8]
of several case studies and student projects.