Machine Learning for Engineering

Instructor: Daning Huang

Introduction

This website hosts the slides used for the AERSP/ME 597 course taught at Penn State.

This course has a strong focus on regression (almost no classification). It

• covers the machine learning techniques for the data-driven modeling and data analysis with emphasis on aerospace engineering applications, and
• exposes the students to the latest advances in the data-driven modeling studies that would be beneficial to their research.

The topics are accompanied by case studies representing the application of machine learning techniques in aerospace engineering research. Knowledge of multivariate calculus and basic linear algebra is required, and some familiarity with probability would be helpful.

Specifically, the course consists of following topics:

• Mathematical background
• Linear regression
• Gaussian process regression
• Neural networks
• High-Dimensional systems
• Dynamical and differential systems

See following slides for more details

Mathematical background

• Linear Algebra
• Probability Theory, Supp. notes
• Optimization
• Differential Equations

Linear regression

• Basics
• Regularization
• Probability Formulation
• Model Selection
• Kernel Method
• More Variants draft

Gaussian process regression

• Basics
• Formulation
• Markov Chain Monte Carlo
• Sampling
• Bayesian Optimization (PDF) slides (draft)
• Variants (PDF) slides (draft)

Neural networks

• Basics
• Training
• Automatic Differentiation
• Common Architectures
• Model Embedding
• Physics-Informed NN

High-Dimensional systems

• Basics
• Spatial Decomposition
• Autoencoders
• Clustering
• Temporal Decomposition

Dynamical and differential systems

• Basics
• Linear System
• Nonnormality, DMD Revisited
• Tackling Nonlinearity
• Sparse Regression
• Koopman Operator Theory
• Differentially-constrained problems
• Neural ODEs
• Diffusion Models

References

1. [PRML] Pattern Recognition and Machine Learning, Christopher Bishop, 2006.
2. [MLaPP] Machine Learning: A Probabilistic Perspective, Kevin P. Murphy, 2012.
3. [GPML] Gaussian Processes for Machine Learning, C. E. Rasmussen and C. K. I. Williams, 2006.
4. [DMSC] Data-driven Modeling and Scientific Computation, Nathan Kutz, 2013.

Relevant Courses

1. More basic ones: Berkeley CS 189, Stanford CS 229
2. CNN-specific: Stanford CS 231n
3. ML with a Linear Algebra flavor: MIT 18.065

Credits:

1. UMich EECS 545
2. @evislomo for generating the homework solutions