CHAPTER 6. STOCHASTIC OPTIMAL CONTROL APPROACH

In this chapter, the stochastic optimal control method employed to solve the stochastic formation mission design problem is introduced. First, the main sources of uncertainty in air traffic management are described. Later, the formulation of the stochastic switched optimal control problem is introduced. After that, the methodology to convert the stochastic constraints into deterministic constraints and the stochastic objective functional into a deterministic objective functional is described. This methodology is based on nonintrusive generalized polynomial chaos stochastic collocation. Finally, the method employed to conduct the sensitivity analysis is described, the aim of which is to identify the random variables that have more influence on the variability of a component of the solution of the stochastic formation mission design problem.

6.1. Introduction

An OCP of a stochastic switched dynamical system is an OCP in which the continuous dynamics of the system is represented by stochastic differential equations, the objective functional is a stochastic functional, and the constraints, which are defined by means of stochastic functions, must be satisfied almost surely, i.e., with probability 1. The set of possible exceptions in which the constraints are not satisfied may be non-empty but must have probability 0. Therefore, in this thesis, the adjective stochastic means that the functions and the solutions of the differential equations depend on a vector of random variables. This problem is referred to as the stochastic switched optimal control problem (SSOCP).

Modeling, control, and optimal control of stochastic switched systems are addressed in [Alwan and Liu, 2018, Zhu, 2019], in which the random factors acting on the continuous dynamic models of the switched system in each discrete state are represented by some idealized processes such as the Wiener process, and tools such as stochastic calculus have been employed to obtain solutions.

Another approach to model uncertainties in switched dynamical systems is to treat uncertainties as random variables or random processes and recast the original deterministic switched dynamical system as a stochastic switched dynamical system. This type of stochastic systems are different from those represented by classical stochastic differential equations, where the random inputs are idealized processes. Consider a stochastic optimal control problem in which only random variables are present in its formulation like in the formation mission design problem studied in this thesis. One of the most commonly used methods to solve OCPs of stochastic dynamical systems of this type is the Monte Carlo sampling [Shapiro, 2003], in which independent realizations of the random variables are generated based on their probability distributions. For each realization, the OCP becomes deterministic. After solving the deterministic instances of the problem that correspond to these realizations of the random variables, an ensemble of solutions is obtained, from which statistical information can be extracted. Although Monte Carlo sampling is straightforward to apply as it only requires iterative resolutions of deterministic instances of the OCP, it requires a large number of solutions, because the solution statistics converge relatively slowly. For example, the mean value typically converges as $1/\sqrt{N}$, where N is the number of realizations. The need for large number of solutions for accurate results can lead to an excessive computational cost, especially for OCPs that are already computationally intensive in the deterministic settings, as the mission design problem considered in this thesis. The Generalized Polynomial Chaos (gPC) expansion can contribute to alleviating this drawback.

A systematic and coherent presentation of numerical strategies for uncertainty quantification and stochastic computing is given in [Xiu, 2010], with a focus on the methods based on the gPC approach. Both the intrusive stochastic Galerkin and the nonintrusive stochastic collocation approaches to gPC are illustrated in detail. The nonintrusive stochastic collocation approach based on regression is described in [Sudret, 2008] and [Blatman and Sudret, 2010].

In this thesis, a stochastic collocation method has been selected. With this approach, a small number of sample points of the random variables are used to jointly solve particular instances of the SSOCP. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random variables using these sample points. This is a nonintrusive methodology because the model equations are not altered. Depending on the distributions of the random variables, different types of orthogonal polynomials can be chosen to achieve better numerical precision. This technique allows statistical and sensitivity analysis of the stochastic solutions to be conducted at a low computational cost. Thus, the gPC method converts the SSOCP into an augmented deterministic SOCP, in which particular instances of the SSOCP are solved together as a single deterministic OCP. It is important to point out that the gPC method is applicable to solve the SSOCP when the solution depends smoothly on the random variables. This means that the solutions obtained for each combination of sample points of the random variables must give rise to the same discrete solution, i.e., to the same sequence of discrete states of the switched dynamical system that represents the formation mission.

Sources of uncertainty in ATM

In [Cook et al., 2015], the different sources of uncertainty that affect ATM have been classified according to the following five major groups:

•Data uncertainty. This type of uncertainty is due to the presence of some level of uncertainty in the data or some degree of inaccuracy in the models.

•Data unavailability. In this case, there is lack of information due to managerial or technological barriers.

•Operational uncertainty. In this case, uncertainty is due to decisions taken by humans, e.g. air traffic controllers, flight dispatchers, and pilots.

•Equipment uncertainty. This type of uncertainty is due to problems with aircraft or with the communications, navigation and surveillance equipment.

•Weather uncertainty. This type of uncertainty is due to the presence of winds, thunderstorms, snowfalls, and fog.

From all this sources of uncertainty, it is well known that uncertainty in flight departure times is among the main causes of trajectory uncertainty, which generates inefficiency in the ATM system [Rivas and Vazquez, 2016]. Timing is not only crucial in general ATM operations, but also in formation missions. Indeed, considering the usual cruise speed of most long-haul commercial aircraft, missing the rendezvous location by, for instance, ten minutes means spatially missing the partner aircraft by 150 km. Such cases require catch-up maneuvers, which result in a loss of performance compared to the planned formation mission.

Additionally to the former general ATM-related uncertainties, there are uncertainties inherent to formation flight.

As seen in Chapter 2, in extended formations, due to the great distance between the leader and the follower aircraft, instabilities such as meandering and external factors, may affect the motion of the vortices. It is therefore important to maintain the relative position between the follower aircraft and the leader’s wake vortices precisely, as the potential formation benefits are very sensitive to that relative positioning. However, the relative positioning between the follower aircraft and the leader’s wake vortices can not be kept precisely. Therefore, the induced drag reduction factor and the fuel savings achieved in formation flight are actually uncertainties of the formation mission design problem. Fig. 6.1 gives a schematic overview of both, general uncertainties related to ATM and uncertainties inherent to formation flight.

A great number of research studies focused on stochastic modeling. In [Shone et al., 2021] a review of the literature on stochastic modeling with applications to ATM is provided, include literature on stochastic optimal control.

Most of these studies focused on the conflict detection and resolution problem. In [Hu et al., 2005], the problem of aircraft conflict prediction is studied for two-aircraft midair encounters. First, a model is presented for prediction of the aircraft positions along a time horizon, during which each aircraft is following a prescribed flight plan in the presence of additive wind perturbations on its velocity. Then, a method for estimating the probability of conflict is proposed. This method is based on a Markov chain approximation of the stochastic processes that models the aircraft flight. In [Prandini and Hu, 2009], the aircraft conflict prediction problem is formulated as a reachability problem in a stochastic hybrid system framework. Specifically, a switching diffusion model is employed to predict the future positions of an aircraft following a given flight plan, and the probability that the aircraft enters an unsafe region of the airspace is estimated using a numerical algorithm for reachability computation.

In [Li et al., 2014], an approach to aircraft trajectory optimization in the presence of uncertainties based on gPC is presented, where only one random variable has been considered, which represents an uncertain aerodynamic parameter of the dynamic model of the aircraft. This random variable has been modeled as a uniformly distributed random variable. In [Matsuno et al., 2015], a stochastic optimal control method based on gPC is developed for determining conflict-free aircraft trajectories under wind uncertainty. The random processes that represent the components of the wind speed are approximated as a linear combination of deterministic functions multiplied by independent random variables using the Karhunen-Loève expansion.

Although there is a growing interest in addressing uncertainties in the field of ATM, there are very few studies that study formation flight for commercial aircraft in the presence of uncertainties. One of these few research studies is [Kent and Richards, 2014], in which the impact of ground delays on formation flight is studied using stochastic dynamic programming.

Specification of the stochastic formation mission design problem

In this thesis, the formation mission design problem for commercial aircraft in the presence of uncertainties in some of the parameters and boundary conditions of the problem is studied. Given several commercial flights, the stochastic formation mission design problem consists in establishing how to organize them in formation or solo flights and in finding the trajectories that minimize the expected value of the DOC of the formation mission in the presence of uncertainties. Since each aircraft can fly solo or in various positions within a formation, the mission is modeled as a stochastic switched dynamical system, in which the flight modes of the aircraft are described by sets of stochastic ordinary differential equations, the discrete state describes the combination of flight modes of the individual aircraft, and logical constraints establish the switching logic among the discrete states of the system. Stochastic switched dynamical systems inherit all the features of the deterministic switched dynamical systems seen in Chapter 4.

Both the discrete and the continuous dynamics are affected by uncertainties. This general definition can encompass a wide range of stochastic phenomena. In this thesis, only the stochastic hybrid systems in which random variables only affect the continuous dynamics have been considered. Specifically, random variables represent uncertain in the departure times of the aircraft and in the fuel burn savings for the trailing aircraft. The probability distribution functions of these random variables are assumed to be known.

In the following section, the formulation of an OCP of a stochastic switched dynamical system is introduced.

6.2. The stochastic switched optimal control problem

The presence of random variables in the SOCP converts it into an SSOCP. In the formation mission design problem studied in this thesis, not all the elements of the SSOCP contain random variables. Specifically, the fuel consumption reduction factor ℜ_fuel in the mass flow rate equation (3.14) of the aircraft flying in formation as a trailing or an intermediate aircraft is a random variable. In contrast, the equations of motion associated with the state variables ϕ, λ, χ, and V do not contain random variables. The departure times of the aircraft are also assumed to be random variables of the SSOCP. One of these departure times coincides with t_I, the initial time of the formation mission. These random variables are assumed to be independent and characterized by probability density functions.

Let θ = (θ₁,θ₂,…,θ_N) be the vector of random variables that represent the random parameters of the SOCP. For sake of clarity, hereinafter the subindex “SS” will be employed in the formulation of the OCP for stochastic switched dynamical systems. It is assumed that the control vector u_ss(t) is deterministic and, therefore, it is a function of time only, whereas the state vector x_ss(t, θ) is stochastic and, therefore, it is a function of both time and the vector of random variables, θ [Li et al., 2014].

The continuous-time SSOCP has been formulated as in [Li et al., 2014], namely all the elements of the problem depend on θ and the dynamic equations, the path constraints, and the boundary conditions must be satisfied almost surely. In the formulation of the SSOCP, the dynamical equations and endpoint constraints are:

$$\begin{array}{lll}{\dot{x}}_{SS}(t,\theta )\hfill & =\hfill & f_{SS,v_{SS}(t)}\left(t,x_{SS}(t,\theta ),u_{SS}(t),\theta \right)\text{a}\text{.s}\text{.,}\hfill \\ x_{SS}\left(t_I,\theta \right)\hfill & =\hfill & x_I\in {ℝ}^n\text{a}\text{.s}\text{.}\hfill \\ x_{SS}\left(t_F,\theta \right)\hfill & =\hfill & x_F\in {ℝ}^n\text{a}\text{.s}\text{.}\hfill \\ v_{SS}(t)\hfill & \in \hfill & \{0,1\},t_I\le t\le t_F.\hfill \end{array}\mathrm{ (6.1)}$$

The performance index of the SSOCP has the following form:

$$\begin{array}{l}{J}_{SS}\left(t,x_{SS}(t,\theta ),u_{SS}(t),v_{SS}(t),\theta \right)=\hfill \\ M_{SS}\left(t_F,x_F,\theta \right)+{\displaystyle {\int }_{t_I}^{t_F}{L}_{SS,v_{SS}(t)}\left(t,x_{SS}(t,\theta ),u_{SS}(t),\theta \right)dt}.\hfill \end{array}\mathrm{ (6.2)}$$

The SSOCP is thus stated as follows

$$\underset{u_{SS}\in \text{Ω},v_{SS}\in \{0,1\}}{\min }{J}_{SS}\left(t,x_{SS}(t,\theta ),u_{SS}(t),v_{SS}(t),\theta \right),\mathrm{ (6.3)}$$

subject to dynamical equations and endpoint constraints from the set of Eqs. (6.1). It can be seen that the difference between the formulations of the deterministic SOCP and the SSOCP is that the vector random variables θ is included in the latter.

It is important to point out that probabilistic constraints, also known as chance constraints, could be introduced in the formulation of the SSOCP [Matsuno et al., 2015]. However, in this thesis, chance constraints have not been considered in the formulation of the SSOCP employed to solve the stochastic formation mission design problem.

6.2.1. The generalized polynomial chaos expansion

In this section, following [Li et al., 2014, Matsuno et al., 2015], the gPC expansion is introduced and the method for determining the stochastic solution of the SSOCP and computing its statistical information is described.

Using the gPC expansion, the stochastic solution of the SSOCP can be approximated by a finite sum of orthogonal polynomials. The P-th order gPC approximation of the stochastic solution of the SSOCP, z(t, θ), can be expressed as

$$z_P(t,\theta )={\displaystyle \sum }_{m=1}^M{C}_m(t){\text{Φ}}_m(\theta ),\mathrm{ (6.4)}$$

where θ = (θ₁,θ₂, …, θ_N) is a vector of independent random variables, C_m(t) are the corresponding coefficients of the expansion, which can be obtained using either a nonintrusive or an intrusive approach, and Φ_m(θ) are the multivariate orthogonal polynomial basis functions, which are calculated from the l_i-th order one-dimensional polynomial basis function ${\varphi }^{\left(l_i\right)}\left({\theta }_i\right)$ of the random variable θ_i by means of the tensor product rule as follows:

$${\text{Φ}}_m(\theta )={\displaystyle \prod }_{i=1}^N{\varphi }_i^{\left(l_i\right)}\left({\theta }_i\right)\mathrm{ (6.5)}$$

Notice that, in Eq. (6.5), a unique combination of l_i, i = 1, 2, …, N, corresponds to each subscript m, which satisfy the condition ${\displaystyle \sum }_{i=1}^N{l}_i\le P$. The number of tensor product basis functions is $M=(\begin{array}{c}N+P\\ N\end{array})$.

Orthogonal polynomials basis functions

The orthonormal polynomials in Eq. (6.5) satisfy the following orthogonality condition

$$E\left[{\varphi }_i^{(j)}\left({\theta }_i\right){\varphi }_i^{(k)}\left({\theta }_i\right)\right]={{\displaystyle \int }}^{\text{}}{\varphi }_i^{(j)}\left({\theta }_i\right){\varphi }_i^{(k)}\left({\theta }_i\right){\rho }_i\left({\theta }_i\right)d{\theta }_i={\delta }_{jk},\mathrm{ (6.6)}$$

where E is the expected value operator, ρ_i(θ_i) is the probability density function (PDF) of the random variable θ_i, and θ_jk is the Kronecker delta function.

To improve convergence, the choice of the orthogonal polynomials should be made on the basis PDF ρ_i(θ_i). For instance, the Legendre polynomials are the best choice for uniform random variables, whereas Hermite polynomials are the best option for Gaussian random variables. Some relationships between continuous and discrete probability distributions and their correspondent gPC basis polynomials are shown in Tables 6.1 and 6.2, respectively.

6.2.2. The stochastic collocation approach

The expansion coeficientes, C_m(t), can be obtained using intrusive or nonintrusive approaches.

In the stochastic Galerkin method, the gPC expansions in terms of the random variables that represent the sources of uncertainty such as parameters and boundary conditions, are first determined and then the gPC expansions are substituted into the model equations. The Galerkin projection is then performed to project the model equations onto the random space spanned by the polynomial basis. This projection transforms each stochastic equation of the model into a set of coupled deterministic equations. Finally, the resulting system of equations is solved with a suitable numerical method. This is an intrusive methodology in the sense that the model equations are altered. The stochastic Galerkin method can be challenging when the stochastic model equations take complicated forms. In this case, the derivation of the set of deterministic equations can be very difficult or even impossible [Li et al., 2014].

Within the nonintrusive methods, the regression approach and the stochastic collocation method can be employed. The nonintrusive stochastic collocation approach based on regression is described in [Sudret, 2008] and [Blatman and Sudret, 2010]. This method is specially appropriate when the number of random variables considered is high, in general, larger than 10 variables [Matsuno et al., 2015]. In stochastic collocation methods the stochastic model equations are satisfied at a discrete set of points, called nodes, in the corresponding random space. From this point of view, all classical sampling methods like Monte Carlo sampling are collocation methods. However, in stochastic collocation, polynomial approximation theory is employed to locate the nodes, known as collocation points, strategically to increase the numerical accuracy and significantly reduce the sample points.

In this thesis, the coefficients C_m(t) of the expansion (6.4) are calculated using a nonintrusive gPC based stochastic collocation method as follows

$$C_m=E\left[z(t,\theta ){\text{Φ}}_m(\theta )\right]={{\displaystyle \int }}^{\text{}}z(t,\theta ){\text{Φ}}_m(\theta )\rho (\theta )d\theta ,\mathrm{ (6.7)}$$

where $\rho (\theta )={\displaystyle \prod }_{i=1}^N{\rho }_i\left({\theta }_i\right)$ is the joint probability density function of the vector of random variables θ.

A Gaussian quadrature can be employed to approximate the integral in Eq. (6.7) using a set of q collocation points ${\theta }_i^{\left(j\right)}$ for each random variable θ_i and calculating the corresponding set of quadrature weights ${\alpha }_i^{\left(j\right)}$, j = 1,2,…, q.

Higher precision in the quadrature can be achieved by increasing the number of collocation points q. Using the tensor product rule, the total number of collocation points is q^N. This number, which could become large when N, the number of random variables in θ, grows, can be reduced to Q using the sparse grid quadrature based on the Smolyak rule [Xiu, 2010].

Let θ^(j) be the collocation points and α^(j), j = 1,…,Q, the corresponding weights. Then, the coefficients of the expansion C_m in Eq. (6.7) are approximated by:

$$C_m\approx {\displaystyle \sum }_{j=1}^{Q}z\left(t,{\theta }^{(j)}\right){\text{Φ}}_m\left({\theta }^{(j)}\right){\alpha }^{(j)},\mathrm{ (6.8)}$$

where z(t, θ^(j)) denotes the solution of the augmented SOCP obtained using the j-th collocation point θ^(j) and Φ_m(θ^(j)) represents the multivariate orthogonal polynomial basis function evaluated at the j-th collocation point θ^(j). Thus, the SSOCP is converted into an augmented deterministic SOCP, in which particular instances of the SSOCP, which correspond to the collocation points of the random variables, are combined into a single optimal control problem. The resulting augmented SOCP is solved by means of the numerical methods described in chapters 4 and 5. The expected value and variance of the stochastic solution z_P(t, θ) are calculated using the coefficients of the expansion as follows:

$$E\left[z_P(t,\theta )\right]=C_1(t),\mathrm{ (6.9)}$$

$$VAR\left[z_P(t,\theta )\right]={\displaystyle \sum }_{m=2}^M{C}_m^2(t),\mathrm{ (6.10)}$$

where VAR denotes the variance operator. As stated in Eqs. (6.9) and (6.10), the expected value coincides with the first coefficient, whereas the variance is the sum of the squares of the other coefficients of the gPC expansion. Using the former expressions, statistical information of the solutions of the SSOCP can be easily computed. Additionally, the mean and the standard deviation of some component of the stochastic solution can be expressed as a function of the variables of the SSOCP and included in the objective functional and the constraints of the problem.

A complete treatment of the gPC expansion can be found in the monography [Xiu, 2010], whereas the procedure followed to determine the stochastic solution of the SSOCP and to compute its statistical information is schematically represented in Fig. 6.2. Further details can be found in [Li et al., 2014, Matsuno et al., 2015].

6.2.3. Specification of the stochastic switched optimal control problem

Following the technique described in [Li et al., 2014], the stochastic constraints from Eqs. (6.1) have been converted into deterministic constraints. Likewise, the stochastic objective functional of the SSOCP from Eq. (6.2) has been converted into a deterministic functional by taking its expected value. The new formulation of the SSOCP can be expressed as follows

$$\begin{array}{ll}\underset{\begin{array}{c}{u}_{SSS}\in \text{Ω},v_{SS}\in \{0,1\}\end{array}}{\min }\hfill & J_{SS}=E\left[M_{SS}\left(t_F,x_F,\theta \right)+\underset{t_I}{\overset{t_F}{{\displaystyle \int }}}{L}_{SS,v_{SS}(t)}\left(t,x_{SS}(t,\theta ),u_{SS}(t),\theta \right)dt\right]\hfill \\ \hfill & \text{s}\text{.t}\text{.}\hfill \\ \hfill & {\dot{x}}_{1_{SS}}\left(t,{\theta }_1\right)=f_{SS,v_{SS}(t)}\left(t,x_{SS}\left(t,{\theta }_1\right),u_{SS}(t),{\theta }^{(1)}\right),\hfill \\ \hfill & {\dot{x}}_{2_{SS}}\left(t,{\theta }_2\right)=f_{SS,v_{SS}(t)}\left(t,x_{SS}\left(t,{\theta }_2\right),u_{SS}(t),{\theta }^{(2)}\right),\hfill \\ \hfill & ⋮\hfill \\ \hfill & {\dot{x}}_{N_{SS}}\left(t,{\theta }_N\right)=f_{SS,v_{SS}(t)}\left(t,x_{SS}\left(t,{\theta }_N\right),u_{SS}(t),{\theta }^{(Q)}\right),\hfill \\ \hfill & E\left[x_{SS}\left(t_I,\theta \right)\right]=x_I\in {ℝ}^n,\hfill \\ \hfill & E\left[x_{SS}\left(t_F,\theta \right)\right]=x_F\in {ℝ}^n,\hfill \\ \hfill & v_{SS}(t)\in \{0,1\},t_I\le t\le t_F,\hfill \end{array}\mathrm{ (6.11)}$$

where θ⁽¹⁾, θ⁽²⁾,…, 0^(Q) are the Q collocation points of the vector random variable θ.

Solution of the stochastic switched optimal control problem

As already seen in Chapter 4, deterministic switched dynamical systems are described by both a continuous and a discrete dynamics, in which the transitions among discrete states are not established in advance. Stochastic switched dynamical systems inherit all the features of the deterministic switched dynamical systems.

The stochastic collocation method converts the SSOCP into an augmented deterministic SOCP. With this approach, a small number of sample points of the random parameters are employed to jointly solve particular instances of the SOCP. The same methodology as the one explained in Chapter 4 for deterministic SOCP can be followed, obtaining an augmented deterministic EOCP with inequality and equality constraints which can be solved using the optimal control techniques explained in Chapter 5.

The solution of the SSOCP includes both the state and control variables of the system that represent the formation mission. Due to the presence of the vector of random variables in the SSOCP, the solution is stochastic, i.e., it depends on the actual realization of the vector random variable. In particular, it is a random function of state or time. Therefore, the solution is actually a random process, characterized by the mean function and the corresponding 95% confidence envelope. Specifically, the observed values of a state variable at a given time t is a random variable. In this case, the random process is a random function of time. Likewise, the time at which a given value of the state variable is reached is a random variable. This means that the timing of the trajectories, which is defined as the time at which a state is reached, is also a random process. In this case, the random process is a random function of the state. In this paper, the timing of the trajectories is represented as a random function of the orthodromic distance from the departure location. For the sake of ease of exposition, the stochastic solution of the SSOCP is denoted in the following sections as a random function of time. All the random variables observed at specific time instants or states are characterized by their expected values and by the corresponding 95% confidence intervals, including the arrival times and the fuel consumption of the aircraft of the formation mission.

6.3. Sensitivity analysis of the solution

The purpose of the sensitivity analysis is to identify the random variables that have more influence on the variability of a component of the stochastic solution with the aim of reducing its variability acting on the source. Therefore, in this thesis, a global sensitivity analysis of the stochastic solution is also conducted [Saltelli et al., 2008] in the numerical experiment that involves two random variables such as Experiment B presented in Chapter 8, in which a two-aircraft transoceanic mission design problem with uncertainty in the departure times of both flights is addressed.

In particular, the aim of the variance-based sensitivity analysis is to quantify what proportion of the variance of each component of the solution z(t, θ) of the formation mission design problem is due to the variance of each component θ_j,j = 1,2,…,N of the random vector θ.

In this thesis, this analysis relies on the computation of the so-called Sobol’ indices, under the assumption that the random variables in θ are independent [Saltelli et al., 2008]. Sobol’ indices can be directly derived from the coefficients C_m(t) of the gPC expansion (6.4) as explained in [Trucchia et al., 2019].

Following [Trucchia et al., 2019], the Sobol’ index S_i,j(t) for the i-th component of the solution z(t, θ) and the j-th component of the random variable θ can be expressed as

$$S_{i,j}(t)=\frac{1}{VA{R}_i}{\displaystyle \sum }_{m\in I}{C}_m^2(t),\mathrm{ (6.12)}$$

where VAR_i is the variance calculated in Eq. (6.10) for the i-th component of the solution and I is the set of multi-indices such that the computation of S_i,j(t) only includes terms that depend on the random variable θ_j, j = 1,2,…,N.

6.4. Conclusions

In this chapter, the formulation of the stochastic optimal control problem employed to model the stochastic formation mission design problem is presented and the approach followed to solve it is described. It is a nonintrusive polynomial chaos based stochastic collocation method, which converts the stochastic switched optimal control problem into an augmented deterministic switched optimal control problem. With this approach, a small number of sample points of the random parameters are used to jointly solve particular instances of the switched optimal control problem. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random parameters using these sample points. The technique allows statistical and global sensitivity analysis of the stochastic solutions to be conducted at a low computational cost.

Bibliography

[Abeyratne, 2020] Abeyratne, R. (2020). The outcome of the 40th ICAO assembly: A new look at ICAO? Air and Space Law, 45(1).

[Abrantes et al., 2021] Abrantes, I., Ferreira, A. F., Silva, A., and Costa, M. (2021). Sustainable aviation fuels and imminent technologies-CO₂ emissions evolution towards 2050. Journal of Cleaner Production, 313:127937.

[Airbus, a] Airbus. https://www.airbus.com/en/newsroom/press-releases/2021-11-airbus-and-its-partners-demonstrate-how-sharing-the-skies-can-save. Accessed: 2021-11-18.

[Airbus, b] Airbus. https://www.airbus.com/en/innovation/disruptive-concepts/biomimicry/fellofly. Accessed: 2022-01-10.

[Alwan and Liu, 2018] Alwan, M. S. and Liu, X. (2018). Theory of Hybrid Systems: Deterministic and Stochastic. Springer.

[Ansari et al., 2020] Ansari, M. Y., Ahmad, A., Khan, S. S., Bhushan, G., et al. (2020). Spatiotemporal clustering: A review. Artificial Intelligence Review, 53(4):2381–2423.

[Assembly 40th, 2019] Assembly 40th, I. https://www.icao.int/Meetings/a40/Documents/WP/wp_317_en.pdf. Accessed: 2021-11-21.

[Aung and Tan, 2010] Aung, H. H. and Tan, K.-L. (2010). Discovery of evolving convoys. In Gertz, M. and Ludäscher, B., editors, International Conference on Scientific and Statistical Database Management, pages 196–213. Springer.

[Bengea et al., 2011] Bengea, S., Uthaichana, K., Zěfran, M., and DeCarlo, R. A. (2011). Optimal control of switching systems via embedding into continuous optimal control problem. In Levine, W. S., editor, The Control Handbook: Advanced Methods, chapter 31, pages 31–1 – 31–23. CRC Press.

[Bengea and DeCarlo, 2005] Bengea, S. C. and DeCarlo, R. A. (2005). Optimal control of switching systems. Automatica, 41:11–27.

[Betts, 2010] Betts, J. T. (2010). Practical Methods for Optimal Control and Estimation using Nonlinear Programming. SIAM.

[Bieniawski et al., 2014] Bieniawski, S. R., Clark, R. W., Rosenzweig, S. E., and Blake, W. E. (2014). Summary of flight testing and results for the formation flight for aerodynamic benefit program. In Proceedings of the 52nd Aerospace Sciences Meeting, National Harbor, MD, USA.

[Blake and Flanzer, 2016] Blake, W. B. and Flanzer, T. C. (2016). Optimal routing for dragreducing formation flight: A restricted case. Journal of Guidance, Control, and Dynamics, 39(1):173–176.

[Blatman and Sudret, 2010] Blatman, G. and Sudret, B. (2010). An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis. Probabilistic Engineering Mechanics, 25(2):183–197.

[Bower et al., 2009] Bower, G. C., Flanzer, T. C., and Kroo, I. M. (2009). Formation geometries and route optimization for commercial formation flight. In Proceedings of the 27th AIAA Applied Aerodynamics Conference, San Antonio, TX, USA.

[Buhmann, 2003] Buhmann, M. D. (2003). Radial Basis Functions: Theory and Implementations. Cambridge University Press.

[Camilleri, 2018] Camilleri, M. A. (2018). Aircraft operating costs and profitability. In Travel Marketing, Tourism Economics and the Airline Product, chapter 12, pages 191–204. Springer.

[Caprace et al., 2019] Caprace, D.-G., Winckelmans, G., Chatelain, P., and Eldredge, J. D. (2019). Wake vortex detection and tracking for aircraft formation flight. In Proceedings of the AIAA Aviation 2019 Forum, Dallas, TX, USA.

[Cavalier et al., 1990] Cavalier, T. M., Pardalos, P. M., and Soyster, A. L. (1990). Modeling and integer programming techniques applied to propositional calculus. Computers and Operations Research, 17(6):561–570.

[Cook et al., 2015] Cook, A., Blom, H. A., Lillo, F., Mantegna, R. N., Micciche, S., Rivas, D., Vázquez, R., and Zanin, M. (2015). Applying complexity science to air traffic management. Journal of Air Transport Management, 42:149–158.

[Dee et al., 2011] Dee, D. P., Uppala, S. M., et al. (2011). The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quarterly Journal of the Royal Meteorological Society, 137(656):553–597.

[Durango et al., 2016] Durango, G. J., Lawson, C., and Shahneh, A. Z. (2016). Formation flight investigation for highly efficient future civil transport aircraft. The Aeronautical Journal, 120(1229):1081–1100.

[Eurocontrol, 2013] Eurocontrol (2013). User manual for the Base of Aircraft Data (BADA). Technical Report 130416, Eurocontrol.

[Fahroo and Ross, 2000] Fahroo, F. and Ross, I. M. (2000). A spectral patching method for direct trajectory optimization. Journal of the Astronautical Sciences, 48(2):269–286.

[Fahroo and Ross, 2008] Fahroo, F. and Ross, I. M. (2008). Advances in pseudospectral methods for optimal control. In Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, Honolulu, HI, USA.

[Falck et al., 2021] Falck, R., Gray, J. S., Ponnapalli, K., and Wright, T. (2021). Dymos: A Python package for optimal control of multidisciplinary systems. Journal of Open Source Software, 6(59):2809.

[Flanzer et al., 2014] Flanzer, T. C., Bieniawski, S. R., and Blake, W. B. (2014). Operational analysis for the formation flight for aerodynamic benefit program. In Proceedings of the 52nd Aerospace Sciences Meeting, National Harbor, MD, USA.

[Flanzer et al., 2020] Flanzer, T. C., Bieniawski, S. R., and Brown, J. A. (2020). Advances in cooperative trajectories for commercial applications. In Proceedings of the AIAA SciTech 2020 Forum, Orlando, FL, USA.

[Garg, 2011] Garg, D. (2011). Advances in Global Pseudospectral Methods for Optimal Control. PhD thesis, University of Florida, Gainesville, FL, USA.

[Garg et al., 2011] Garg, D., Patterson, M. A., Francolin, C., Darby, C. L., Huntington, G. T., Hager, W. W., and Rao, A. V. (2011). Direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using a Radau pseudospectral method. Computational Optimization and Applications, 49(2):335–358.

[Garg et al., 2009] Garg, D., Patterson, M. A., Hager, W., Rao, A. V., Benson, D. A., and Huntington, G. T. (2009). An overview of three pseudospectral methods for the numerical solution of optimal control problems. Advances in the Astronautical Sciences, 135:1–17.

[Haalas et al., 2014] Haalas, D. J., Bieniawski, S. R., Whitehead, B., Flanzer, T., and Blake, W. B. (2014). Formation flight for aerodynamic benefit simulation development and validation. In Proceedings of the 52nd Aerospace Sciences Meeting, National Harbor, MD, USA.

[Hallock and Holzapfel, 2018] Hallock, J. N. and Holzäpfel, F. (2018). A review of recent wake vortex research for increasing airport capacity. Progress in Aerospace Sciences, 98:27–36.

[Hart et al., 2012] Hart, W. E., Laird, C. D., Watson, J.-P., Woodruff, D. L., Hackebeil, G. A., Nicholson, B. L., and Siirola, J. D. (2012). Pyomo-Optimization Modeling in Python. Springer.

[Hartjes et al., 2018] Hartjes, S., van Hellenberg Hubar, M. E., and Visser, H. G. (2018). Multiple-phase trajectory optimization for formation flight in civil aviation. In Dolega, B., Glebocki, R., Kordos, D., and Zugaj, M., editors, Advances in Aerospace Guidance, Navigation and Control, pages 389–405. Springer.

[Hartjes et al., 2019] Hartjes, S., Visser, H. G., and van Hellenberg Hubar, M. E. (2019). Trajectory optimization of extended formation flights for commercial aviation. Aerospace, 6(9):100.

[Héder, 2017] Héder, M. (2017). From NASA to EU: the evolution of the TRL scale in Public Sector Innovation. The Innovation Journal, 22(2):1–23.

[Hu et al., 2005] Hu, J., Prandini, M., and Sastry, S. (2005). Aircraft conflict prediction in the presence of a spatially correlated wind field. IEEE Transactions on Intelligent Transportation Systems, 6(3):326–340.

[Hull, 2007] Hull, D. G. (2007). Fundamentals of Airplane Flight Mechanics. Springer.

[Huntington, 2007] Huntington, G. T. (2007). Advancement and analysis of a Gauss pseudospectral transcription for optimal control problems. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA.

[Huntington and Rao, 2008] Huntington, G. T. and Rao, A. V. (2008). Comparison of global and local collocation methods for optimal control. Journal of Guidance, Control, and Dynamics, 31(2):432–436.

[IATA, 2013] IATA (2013). Technology roadmap, 4th Edition. Technical report, International Air Transport Association.

[IATA, 2016] IATA (2016). Press Release No. 59. IATA forecasts passenger demand to double over 20 years. https://www.iata.org/en/pressroom/pr/2016-10-18-02/. Accessed: 2021-12-19.

[ICAO, 1944] ICAO (1944). Convention on international civil aviation. Annex 2. Technical report, International Civil Aviation Organization.

[ICAO, 2016] ICAO (2016). Environmental report 2016. Aviation and climate change. Technical report, International Civil Aviation Organization.

[ICAO, 2017] ICAO (2017). Airline operating costs and productivity. Technical report, International Civil Aviation Organization.

[IPCC, 2018] IPCC (2018). Global warming of 1.5C. Technical report, The Intergovernmental Panel on Climate Change.

[Jeung et al., 2010] Jeung, H., Yiu, M. L., Zhou, X., Jensen, C. S., and Shen, H. T. (2010). Discovery of convoys in trajectory databases. Proceedings of the VLDB Endowment, 1(1):1068–1080.

[Kent and Richards, 2014] Kent, T. and Richards, A. (2014). Accounting for the effect of ground delay on commercial formation flight. In Proceedings of the 2014 UKACC International Conference on Control, Loughborough, United Kingdom.

[Kent and Richards, 2015] Kent, T. E. and Richards, A. G. (2015). Analytic approach to optimal routing for commercial formation flight. Journal of Guidance, Control, and Dynamics, 38(10):1872–1884.

[Kent and Richards, 2021] Kent, T. E. and Richards, A. G. (2021). Potential of formation flight for commercial aviation: Three case studies. Journal of Aircraft, 58(2):1–14.

[Kim and Braatz, 2012] Kim, K. K. K. and Braatz, R. D. (2012). Probabilistic analysis and control of uncertain dynamic systems: Generalized polynomial chaos expansion approaches. In Proceedings of the 2012 American Control Conference, Montreal, QC, Canada.

[Klavins, 2004] Klavins, E. (2004). Communication complexity of multi-robot systems. In Boissonnat, J., Burdick, J., Goldberg, K., and Hutchinson, S., editors, Algorithmic Foundations of Robotics V, pages 275–291. Springer.

[Kless et al., 2013] Kless, J. E., Aftosmis, M. J., Ning, S. A., and Nemec, M. (2013). Inviscid analysis of extended-formation flight. AIAA Journal, 51(7):1703–1715.

[Klower et al., 2021] Klower, M., Allen, M., Lee, D., Proud, S., Gallagher, L., and Skowron, A. (2021). Quantifying aviation’s contribution to global warming. Environmental Research Letters, 16(10):104027.

[Lai et al., 2022] Lai, Y. Y., Christley, E., Kulanovic, A., Teng, C.-C., Bjorklund, A., Nordensvard, J., Karakaya, E., and Urban, F. (2022). Analysing the opportunities and challenges for mitigating the climate impact of aviation: A narrative review. Renewable and Sustainable Energy Reviews, 156:111972.

[Lee et al., 2009] Lee, D. S., Fahey, D. W., Forster, P. M., Newton, P. J., Wit, R. C., Lim, L. L., Owen, B., and Sausen, R. (2009). Aviation and global climate change in the 21st century. Atmospheric Environment, 43(22–23):3520–3537.

[Li et al., 2014] Li, X., Nair, P. B., Zhang, Z., Gao, L., and Gao, C. (2014). Aircraft robust trajectory optimization using nonintrusive polynomial chaos. Journal of Aircraft, 51(5):1592–1603.

[Lissaman and Shollenberger, 1970] Lissaman, P. B. S. and Shollenberger, C. A. (1970). Formation flight of birds. Science, 168(3934):1003–1005.

[Liu et al., 2019] Liu, C., Jiang, B., Patton, R. J., and Zhang, K. (2019). Integrated fault-tolerant control for close formation flight. IEEE Transactions on Aerospace and Electronic Systems, 56(2):839–852.

[Marks and Gollnick, 2016] Marks, T. and Gollnick, V. (2016). Influence of aircraft type and order on fuel savings gained by two aircraft formations. In Proceedings of the 30th Congress of the International Council of the Aeronautical Science, Deajeon, South Korea.

[Matsuno et al., 2015] Matsuno, Y., Tsuchiya, T., Wei, J., Hwang, I., and Matayoshi, N. (2015). Stochastic optimal control for aircraft conflict resolution under wind uncertainty. Aerospace Science and Technology, 43:77–88.

[Michel et al., 2020] Michel, D.-T., Dolfi-Bouteyre, A., Goular, D., Augere, B., Planchat, C., Fleury, D., Lombard, L., Valla, M., and Besson, C. (2020). Onboard wake vortex localization with a coherent 1.5 μm Doppler LiDAR for aircraft in formation flight configuration. Optics Express, 28(10):14374–14385.

[Murphy, 2012] Murphy, K. P. (2012). Machine learning: A probabilistic perspective. The MIT Press.

[Ning et al., 2011] Ning, A., Flanzer, T. C., and Kroo, I. M. (2011). Aerodynamic performance of extended formation flight. Journal of Aircraft, 48(3):855–865.

[Ning, 2011] Ning, S. A. (2011). Aircraft Drag Reduction Through Extended Formation Flight. PhD thesis, Stanford University, Standord, CA, USA.

[Patterson and Rao, 2014] Patterson, M. A. and Rao, A. V. (2014). GPOPS-II: A MATLAB software for solving multiple-phase optimal control problems using hp-adaptive Gaussian quadrature collocation methods and sparse nonlinear programming. ACM Transactions on Mathematical Software, 41(1):1–37.

[Prandini and Hu, 2009] Prandini, M. and Hu, J. (2009). Application of reachability analysis for stochastic hybrid systems to aircraft conflict prediction. IEEE Transactions on Automatic Control, 54(4):913–917.

[Rao et al., 2009] Rao, A. V., Benson, D., Huntington, G. T., Francolin, C., Darby, C. L., and Patterson, M. (2009). User’s Manual for GPOPS Version 2.1: A MATLAB Package for Dynamic Optimization Using the Gauss Pseudospectral Method.

[Rivas and Vazquez, 2016] Rivas, D. and Vazquez, R. (2016). Uncertainty. In Cook, A. and Rivas, D., editors, Complexity Science in Air Traffic Management. Routledge.

[Ross, 2020] Ross, I. (2020). Enhancements to the DIDO optimal control toolbox. arXiv preprint arXiv:2004.13112.

[Ross and Fahroo, 2004] Ross, I. M. and Fahroo, F. (2004). Pseudospectral knotting methods for solving optimal control problems. Journal of Guidance, Control, and Dynamics, 27(3):397–405.

[Saltelli et al., 2008] Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., and Tarantola, S. (2008). Global Sensitivity Analysis. The Primer. John Wiley & Sons.

[Seo et al., 2017] Seo, J., Kim, Y., Kim, S., and Tsourdos, A. (2017). Collision avoidance strategies for unmanned aerial vehicles in formation flight. IEEE Transactions on Aerospace and Electronic Systems, 53(6):2718–2734.

[Shapiro, 2003] Shapiro, A. (2003). Monte Carlo Sampling Methods. In Ruszczynski, A. and Shapiro, A., editors, Stochastic Programming, volume 10 of Handbooks in Operations Research and Management Science, pages 353–425. Elsevier.

[Shone et al., 2021] Shone, R., Glazebrook, K., and Zografos, K. G. (2021). Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty. European Journal of Operational Research, 292:1–26.

[Slotnick, 2014] Slotnick, J. P. (2014). Computational aerodynamic analysis for the formation flight for aerodynamic benefit program. In Proceedings of the 52nd Aerospace Sciences Meeting, National Harbor, MD, USA.

[Sudret, 2008] Sudret, B. (2008). Global sensitivity analysis using polynomial chaos expansions. Reliability Engineering and System Safety, 93:964–979.

[Tierno et al., 2012] Tierno, M. Á. G., Cortés, M. P., and Márquez, C. P. (2012). Mecánica del vuelo. Ibergaceta.

[Torre et al., 2019] Torre, E., Marelli, S., Embrechts, P., and Sudret, B. (2019). A general framework for data-driven uncertainty quantification under complex input dependencies using vine copulas. Probabilistic Engineering Mechanics, 55:1–16.

[Trucchia et al., 2019] Trucchia, A., Egorova, V., Pagnini, G., and Rochoux, M. (2019). On the merits of sparse surrogates for global sensitivity analysis of multi-scale nonlinear problems: Application to turbulence and fire-spotting model in wildland fire simulators. Communications in Nonlinear Science and Numerical Simulation, 73:120–145.

[Tu et al., 2008] Tu, Y., Ball, M. O., and Jank, W. S. (2008). Estimating flight departure delay distributions—a statistical approach with long-term trend and short-term pattern. Journal of the American Statistical Association, 103(481):112–125.

[Vachon et al., 2002] Vachon, M. J., Ray, R., Walsh, K., and Ennix, K. (2002). F/A-18 aircraft performance benefits measured during the autonomous formation flight project. In Proceedings of the AIAA Atmospheric Flight Mechanics Conference and Exhibit, Monterey, CA, USA.

[Vechtel et al., 2018] Vechtel, D., Fischenberg, D., and Schwithal, J. (2018). Flight dynamics simulation of formation flight for energy saving using LES-generated wake flow fields. CEAS Aeronautical Journal, 9(4):735–746.

[Vieira et al., 2009] Vieira, M. R., Bakalov, P., and Tsotras, V. J. (2009). On-line discovery of flock patterns in spatio-temporal data. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA.

[Voskuijl, 2017] Voskuijl, M. (2017). Cruise range in formation flight. Journal of Aircraft, 54(6):2184–2191.

[Wachter and Biegler, 2006] Wachter, A. and Biegler, L. T. (2006). On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming, 106(1):25–57.

[Wei et al., 2008] Wei, S., Zěfran, M., and DeCarlo, R. A. (2008). Optimal control of robotic systems with logical constraints: Application to UAV path planning. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA.

[Wieselsberger, 1914] Wieselsberger, C. (1914). Beitrag zur Erklärung des Winkelfluges einiger Zugvögel. Zeitschrift für Flugtechnik und Motorluftschiffahrt., 5:225–229.

[Xiu, 2010] Xiu, D. (2010). Numerical Methods for Stochastic Computations. A Spectral Method Approach. Princeton University Press.

[Xu et al., 2014] Xu, J., Ning, S. A., Bower, G., and Kroo, I. (2014). Aircraft route optimization for formation flight. Journal of Aircraft, 51(2):490–501.

[Zhu, 2019] Zhu, Y. (2019). Uncertain Optimal Control. Springer.