Machine Learning for Engineering

Dynamical System: Characterization of Dynamics

Instructor: Daning Huang

$$ \newcommand\norm[1]{{\left\lVert #1 \right\rVert}} \newcommand{\bR}{\mathbb{R}} \newcommand{\vf}{\mathbf{f}} \newcommand{\vg}{\mathbf{g}} \newcommand{\vv}{\mathbf{v}} \newcommand{\vw}{\mathbf{w}} \newcommand{\vx}{\mathbf{x}} \newcommand{\vy}{\mathbf{y}} \newcommand{\vA}{\mathbf{A}} \newcommand{\vB}{\mathbf{B}} \newcommand{\vU}{\mathbf{U}} \newcommand{\vV}{\mathbf{V}} \newcommand{\vW}{\mathbf{W}} \newcommand{\vX}{\mathbf{X}} \newcommand{\vtf}{\boldsymbol{\phi}} \newcommand{\vty}{\boldsymbol{\psi}} \newcommand{\vtF}{\boldsymbol{\Phi}} \newcommand{\vtS}{\boldsymbol{\Sigma}} \newcommand{\vtL}{\boldsymbol{\Lambda}} \newcommand{\cF}{\mathcal{F}} \newcommand{\cK}{\mathcal{K}} $$

TODAY: Dynamical System - I

References:

• [DMSC] Part 3

Motivation

In this module we would like to work with a general non-linear dynamical system: $$ \begin{align} \dot{x} &= f(x,t,u;\mu) + d,\quad x(0)=x_0 \\ y_k &= g(x_k,t_k,u_k;\mu) + \epsilon \end{align} $$

• $x$: State vector, possibly very high dimensional
• $y$: Observation vector, possibly also high-dim and over discrete time
• $t$: Time, there could be multiple time scales
• $x_0$: Initial conditions, sensitivity to $x_0$ - chaos?
• $u$: Control inputs
• $\mu$: System parameters
• $d$, $\epsilon$: Disturbance, Noise

What to do with dynamical systems

• Modeling for control applications
  • Design controllers
  • Trajectory optimization
  • Control co-design optimization
  • etc.
• System diagnostics and prognostics
  • Future prediction and decision making
  • Feature extraction and interpretation
  • Health monitoring and anomally detection
  • etc.
• Discovery of non-linear dynamics
  • Learn physics from data
  • Coupling to multi-disciplinary analysis
  • etc.

In this module we are mainly concerned with how to learn the dynamical model from data (and its interpretation)

Posing the question

Given

• Trajectories $D=\{(t_i,u_i,\mu_i,y_i)\}_{i=1}^N$
• Possibly also knowledge about the problem
  • Physical properties: conservativeness, stability, etc.
  • Part of governing equations

Identify $$ \begin{align} \dot{x} &= \hat{f}(x,t,u;\mu),\quad x(0)=x_0 \\ y_k &= \hat{g}(x_k,t_k,u_k;\mu) \end{align} $$

• To reproduce the given data and generalize to unseen data
• If possible provide an error estimate on the prediction

Features of Dynamical systems

Examples

What are dynamical features in these examples?

• Flutter test
• Turbulent cylinder flow at Mach 3
• Kelvin-Helmholtz instability
• Hopf bifurcation supercritical, subcritical

For example,

• Time response
  • Transient dynamics
  • Asymptotic behavior
• Spectral components
  • Frequency and phase
  • Time scales
  • Modes/eigenvectors
• Stability
  • Equilibrium points
  • Attractors (and "strange attractors")
  • Bifurcation

Need to understand what we are fitting, before we fit anything...

Phase plane representation (2D)

If we have two scalar states $x$ and $y$, and $$ \begin{align} \dot{x} &= f_1(x,y) \\ \dot{y} &= f_2(x,y) \end{align} $$ Then at $(x,y)$ one can compute $(\dot{x},\dot{y})$ as the direction to evolve - so called vector field

A limit cycle example

We will be using this example as a toy problem later as well

pltQuiver([-3,3], 31, np.pi/4, [-0.1,-0.2], 1)

# Trajectories on phase plane
pltPhasePlaneNL([-2.5,1.5],15,[1,1],1,np.pi/4,[-0.1,-0.2],1, ifLin=False)

# Trajectories as regular time response (Plotting only the first state)
x0s = [-2.5, -0.9, 1.5]
pltRespSweep(x0s, [1], [1], np.pi/4, [-0.1,-0.2], label='x0', tf=25, ifLin=False)

# How about now - unstable limit cycle?
pltQuiver([-3,3], 31, np.pi/4, [-0.1,-0.2], -1)

Stability Analysis (Linear)

Regular procedure for linearization

Consider the time-invariant case

• Linearization
  • Find an equilibrium point $$0=f(x^*)$$
  • Perturb the equilibrium point $$x=\delta x + x^*$$
  • Taylor series expansion of original system, note $\dot{x}^*=0$ $$\delta\dot{x} = A\delta x + \mbox{H.O.T.}$$ where $$A=\frac{\partial f}{\partial x^*}$$

• Diagonalization
  • Perform eigendecomposition, $\Lambda=\mbox{diag}[\lambda_1,\cdots,\lambda_n]$ $$A=V\Lambda V^{-1}$$
  • Check stability $$Re(\lambda_i)\leq 0$$

For linear stability analysis, when it does not work, we might intuitively blame on the linearization, but is that always the case?

What about diagonalization?

A 2D example

Let's consider the linearized system from previous example with $x^*=0$.

The eigenvectors and eigenvalues of $A$ are $$ V=\begin{bmatrix} \cos(\pi/4-\delta) & \cos(\pi/4+\delta) \\ \sin(\pi/4-\delta) & \sin(\pi/4+\delta) \end{bmatrix},\quad \Lambda=\begin{bmatrix} -0.1 & 0 \\ 0 & -0.2 \end{bmatrix} $$

So the linear system should be stable, at least when $t\rightarrow \infty$

However, when $\delta\rightarrow 0$, $V$ becomes ill-conditioned. What happens then?

Let's start from $x_0=[1.0, 0.1]$ and solve the linear system response for $\delta=\pi/4, 0.1, 0.05, 0.02$

X0 = [1, 0.1]
ds = [np.pi/4, 0.1, 0.05, 0.02]
pltPhasePlaneSS(X0, ds)

Depending on $\delta$, the transient growth may throw the states back to nonlinear region and possibly experience new instability.

# Taking delta=0.05 for example - Look at the overshoot in linear response in upper right
pltPhasePlaneNL([-0.2,1.0],15,[0.1,0.1],1,0.05,[-0.1,-0.2],1)

# Sample time responses - again note the overshoot
xs = [0.4, 0.8, 1.2]
pltRespSweep(xs, [0.1], [1], 0.05, [-0.1,-0.2], label='x0', tf=25)

For a linear system to exhibit stable dynamics, we require all eigenvalues of its operator $A$ to satisfy $Re(\lambda_i(A))<0$.

• Although this criterion guarantees the solution $x(t)$ to be stable for large $t$, i.e. $t\rightarrow\infty$ ...
• It does not provide insights into the transient behavior of $x(t)$.

We will dive into this characteristics in the next lecture - which bridges to the general nonlinear dynamics