Introduction to the Pontryagin Maximum Principle for Quantum Optimal Control

Optimal Control Theory is a powerful mathematical tool, which has known a rapid development since the 1950s, mainly for engineering applications. More recently, it has become a widely used method to improve process performance in quantum technologies by means of highly efficient control of quantum dynamics. This tutorial aims at providing an introduction to key concepts of optimal control theory which is accessible to physicists and engineers working in quantum control or in related fields. The different mathematical results are introduced intuitively, before being rigorously stated. This tutorial describes modern aspects of optimal control theory, with a particular focus on the Pontryagin Maximum Principle, which is the main tool for determining open-loop control laws without experimental feedback. The different steps to solve an optimal control problem are discussed, before moving on to more advanced topics such as the existence of optimal solutions or the definition of the different types of extremals, namely normal, abnormal, and singular. The tutorial covers various quantum control issues and describes their mathematical formulation suitable for optimal control. The connection between the Pontryagin Maximum Principle and gradient-based optimization algorithms used for high-dimensional quantum systems is described. The optimal solution of different low-dimensional quantum systems is presented in detail, illustrating how the mathematical tools are applied in a practical way.

In the majority of quantum control protocols, the control law is computed in an open-loop configuration without experimental feedback. In this context, a powerful tool is Optimal Control Theory (OCT) [2] which allows a given process to be carried out, while minimizing a cost (e.g., the control time). This approach has key advantages. Its flexibility makes it possible to adapt to experimental constraints or limitations and its optimal character leads to the physical limits of the driven dynamics. OCT can be viewed as a generalization of the classical calculus of variations for problems with dynamical constraints [26]. Its modern version was born with the Pontryagin maximum principle (PMP) in the late 1950s [27]. Since the pioneering study of Pontryagin and co-workers, OCT has undergone rapid development and is nowadays a recognized field of mathematical research. Recent tools from differential geometry have been applied to control theory, making these methods very effective in dealing with problems of growing complexity. Many reference textbooks have been published on the subject both on mathematical results and engineering applications [28,29,30,31,26,32,33,34,36,35]. Originally inspired by problems of space dynamics, OCT was then applied in a wide spectrum of applications such as robotics or economics. OCT was first used for quantum processes [37,38] in the context of physical chemistry, the goals being to steer chemical reactions [39,40,4,41,42] or to control spin dynamics in Nuclear Magnetic Resonance [43,44,45,46]. A lot of results have recently been established for quantum technologies, as for example the minimum duration to generate highfidelity quantum gates [2]. Two types of approach based on the PMP have been used to solve optimal control problems in low-and high-dimensional systems, respectively. In the first situation called geometric optimal control theory, the equations for optimality are solved by using geometric and analytical tools. The results can be determined analytically or at least with a very high numerical precision. The PMP allows to deduce the structure of the optimal solutions and, in some cases, a proof of their global optimality can be established. In this context, a series of low-dimensional quantum control problems has been rigorously solved in recent years for both closed [47,48,49,50,51,52,53,54,55,56] and open quantum systems [57,58,59,60,61,62,63,64]. Specific numerical optimization algorithms have been developed and applied to design control fields in larger quantum systems [45,65,25,66,67]. Due to the complexity of control landscape, only local optimal solutions are found with this numerical optimal control approach.
However, despite the recent success of quantum optimal control theory, the situation is still not completely satisfactory. The difficulty of the concepts used in this field does not allow a non-expert to understand and apply easily these techniques. The mathematical textbooks use a specialized and sophisticated language, which makes these works difficult to access. Very few basic papers for physicists are available in the literature, while having a minimum grasp of these tools will be an important skill in the future of quantum technologies. The purpose of this tutorial is to provide an introduction to the core mathematical concepts and tools of OCT in a rigorous but understandable way by physicists and engineers working in quantum control and in related fields. A deep analogy can be carried out between OCT and finding the minima of a real function of several variables. This parallel is used throughout the text to qualitatively describe the key aspects of the PMP. The tutorial is based on an advanced course for PhD students in physics taught at Saarland University in Spring of 2019. It assumes a basic knowledge of standard topics in quantum physics, but also of mathematical techniques such as linear algebra or differential calculus and geometry. Finally, we hope that this paper will give the reader the prerequisites to access a more specialized literature and to apply optimal control techniques to their own control problems.

Structure of the paper
A tutorial about optimal control is a difficult task because a large number of mathematical results have been obtained and many techniques have been developed over the years for specific applications. Among others, we can distinguish the following problem classes: finite or infinite-dimensional systems, open or closed-loop control, linear or nonlinear dynamical systems, geometric or numerical optimal control, PMP or Hamilton-Jacobi-Bellmann (HJB) approach. . . We briefly recall that the HJB method which is the result of the dynamic programming theory leads to necessary and sufficient conditions for optimality in which the optimal cost is solution of a nonlinear partial differential equation [26]. Unfortunately, this equation is generally very difficult to solve numerically.
This means making choices about which topics to include in this paper. We have deliberately selected specific aspects of OCT that are treated rigorously, while others are only briefly mentioned. The choice fell on basic mathematical concepts which are the most useful in quantum control. We limit our focus on the optimal control of open-loop finite-dimensional system by using the PMP. In particular, we consider only analytical and geometric techniques to solve lowdimensional control problems. To ensure overall consistency and limit the length of the paper, we do not discuss in depth numerical optimization methods and the infinite-dimensional case [68,69], which are also key issues in quantum control. In order to connect this tutorial with the current applications of optimal control to high-dimensional quantum system, we describe the link between the PMP and the most current implementation of the gradient-based optimization algorithm (the GRAPE algorithm [45]). Finally, we stress that a precise knowledge of the PMP is an essential skill for numerical optimization, and that the scope of the material of this paper is much broader than the examples presented.
The paper is built on three reading levels. A first level corresponds to the main text and explains the main concepts necessary to describe and apply the theory of optimal control. Some key ideas in optimal control are first introduced qualitatively for a simple quantum system in Sec. 2. In addition to the two examples solved in Sec. 8 and 9, the different notions are described rigorously and then systematically illustrated by examples. This establishes a direct link between the mathematical concept and its practical application. A second reading level is given by footnotes, which recall a mathematical definition or correspond to a more specific comment which can be skipped on a first reading. A final reading level is available in the appendices. These different paragraphs explain in detail the mathematical origin of the theorems used for the controllability and the existence problem and some standard counter-examples or specificities the reader should have in mind. We point out that these sections are not mathematical proofs of theorems, but rather a description of the formalism introduced in a language accessible to a physicist. In order to facilitate the reading of the paper, a list of the main notations used is given in Sec. 12 with the place of the text where they are first introduced.
Although the paper is thought for a physics audience and the mathematical details are kept as simple as possible, our objective is to stick to rigorous statements and claims, since this aspect becomes crucial while implementing optimal control ideas in numerical simulations or in experiments.
The paper is organized as follows. We first introduce the main ideas used in optimal control in the case of a simple quantum system in Sec. 2. We then show how to formulate an optimal control problem from a mathematical point of view in Sec. 3. Closed and open quantum systems illustrate this discussion. The different steps to solve such a problem are presented in Sec. 4 by using the analogy with finding a minimum of a function of several variables. The tutorial continues with a point which is crucial, but often overlooked in quantum control studies, namely the existence of optimal solutions. We present in Sec. 5 some results based on the Filippov test, which is one of the most important techniques to address this question. The first-order conditions are described in Sec. 6, with a specific attention on the different types of extremals and on the statement of the PMP. The connection between the PMP and gradient-based optimization algorithms is described in Sec. 7. Sections 8 and 9 are dedicated to the presentation of two examples in three and two-level quantum systems, respectively. Recent advances in the application of OCT to quantum technologies are briefly described in Sec. 10, where we mention some of the current directions that are being followed for the development of these techniques. A conclusion is given in Sec. 11. Mathematical details about the controllability and existence problems are postponed, respectively, to Appendices A and B.
2 Introduction to the optimal control concepts: The case of a two-level quantum system In quantum control, a general problem is to prepare a given quantum state by means of a specific time-dependent electromagnetic pulse. This leads to some questions such as which states can be achieved or which shape of control is required to realize this objective. These aspects, which are addressed rigorously in the rest of the tutorial, are first introduced qualitatively in this section.
To this aim, we consider the control driving a two-level quantum system from the ground to the excited state. The system is described by a wave function ψ(t) whose dynamics are governed by the Schrödinger equation where units such that = 1 have been chosen. The parameters E 0 and E 1 denote respectively the energies of the ground and excited states, while Ω(t) corresponds (up to a multiplicative factor) to a complex external field whose real and imaginary parts are, e.g., the components of two orthogonal linearly polarized laser fields. We consider resonant fields for which the carrier frequency ω of the laser is equal to the energy difference E 1 − E 0 , namely, where the amplitude u(t) represents the control and is assumed to be real. We now apply a time-dependent change of variables corresponding to the choice of a rotating frame. The time evolution ofψ = Υ −1 ψ, with Υ = diag(e −iE0t , e −iE1t ), satisfies the differential equation We denote by c 1 = x 1 + iy 1 and c 2 = x 2 + iy 2 the two complex coordinates ofψ in a basis of the Hilbert space C 2 where the indices 1 and 2 correspond respectively to the ground and excited states. Since ψ is a state of norm 1 and Υ a unitary operator, we deduce that x 2 1 + y 2 1 + x 2 2 + y 2 2 = 1. Starting from the state x 1 = 1, the goal of the control is to bring the system to a target for which x 2 2 + y 2 2 = 1. The Schrödinger equation is equivalent to the following set of equations for the coefficients x k and y k : Since u is a real control, we immediately see that the first and the last equations are coupled to each other and decoupled from the two others. In other words, the initial state of the dynamics is only connected to states for which y 1 = x 2 = 0, i.e., such that x 2 1 + y 2 2 = 1. The system thus evolves on a circle. For our control objective, the only interesting states correspond therefore to y 2 = ±1. It is also straightforward to verify that such target states can be reached at least with a constant control u. In control theory, this formulation of the control problem and the analysis of the reachable set from the initial state constitute a basic prerequisite before deriving a specific control procedure. This step is detailed in Sec. 3.
We now explore the optimal control of this system. We first use the circular geometry of the dynamics to simplify the corresponding equations. We introduce the angle θ such that x 1 = cos θ and y 2 = sin θ, with θ(0) = 1. We arrive aṫ where the target state is here defined as θ fi = ± π 2 . By symmetry, we can fix without loss of generality θ fi = − π 2 . Many control solutions u exist to reach this state and a specific protocol can be selected by minimizing at the same time a functional of the state of the system and of the control, called a cost. Here, an example is given by the control time. To summarize in this example, the goal of the optimal control procedure is then to find the control u steering the system to the target state in minimum time. Consider first constant controls u(t) = u 0 with u 0 ∈ R. The duration of the process is thus T = π 2u0 . This solution reveals a key problem in optimal control which corresponds to the existence of a minimum. In this example, arbitrarily fast controls can be achieved by considering larger and larger amplitudes u 0 and an optimal trajectory minimizing the transfer time does not exist. The analysis of the existence of optimal solutions which is a building block of any rigorous description of an optimal control problem is discussed in Sec. 5. It can be shown with the results presented in Sec. 5 that an optimal solution exists if the set of available controls is restricted to a bounded interval, e.g., u(t) ∈ [−u m , u m ] where u m is the maximum amplitude. In this case, the optimal pulse is the control of maximum amplitude, the minimum time being equal to π/2u m .
In order to illustrate the method of solving an optimal control problem, we consider the same transfer but in a fixed time T , the goal being to minimize the energy associated with the control, i.e., the term T 0 u(t) 2 2 dt. There is no additional constraint on the control and we have u(t) ∈ R. The target state is reached if Introducing the Lagrange multiplier λ ∈ R, this constrained optimization problem can be transformed into the minimization of the functional If we denote by H the function H = − u 2 2 − λu, the Euler-Lagrange principle leads to ∂H ∂u = 0, i.e., u(t) = −λ. Using the constraint on the dynamics, we finally arrive at the optimal control u(t) = π 2T . In this simple example, the optimal solution can be derived without the complete machinery of the Pontryagin Maximum Principle presented below. However, in the example we have introduced the main tools used in the PMP, such as the Lagrange multiplier, the Pontryagin Hamiltonian H, and the maximization condition ∂H ∂u = 0. A few comments are in order here. The dynamical constraint is quite simple since the dimension of the state space is the same as the number of controls, the dynamics can be exactly integrated and the set of controls satisfying the constraint T 0 u(t)dt = π 2 is regular. This is not the case for a general nonlinear control system for which (1) the Lagrange multiplier (which usually is not constant but a function of time) is not easily found, (2) abnormal Lagrange multipliers appear if the set of controls satisfying the constraint is not regular. We observe that H can be rewritten as which corresponds to the general formulation of the Pontryagin Hamiltonian in the normal case. The maximization condition remains the same in a general setting if there is no constraint on the available control. These aspects are discussed in details in Sec. 6.

Formulation of the control problem
The dynamics. A finite-dimensional control system is a dynamical system governed by an equation of the forṁ where q : I → M represents the state of the system, I is an interval in R and M is a smooth manifold whose dimension is denoted by n [70]. We recall that a manifold is a space that locally (but possibly not globally) looks like R n . Manifolds appear naturally in quantum control to describe, for instance, the (2N − 1)-dimensional sphere S 2N −1 , which is the set of wave functions of a N -level quantum system. The control law is u : I → U ⊂ R m and f is a smooth function such that f (·,ū) is a vector field on M for everyū ∈ U . An example of set U of possible values of u(t) is given by U = [−1, 1] m , meaning that the size of each of the coordinates of u is at most one. The set U can be the entire R m if there is no control constraint. To be sure that Eq. (1) is well-posed from a mathematical viewpoint, we consider the case in which I = [0, T ] for some T > 0 and u belongs to a space of regular enough functions U called the class of admissible controls (see [71] for a precise definition). Piecewise continuous controls form a subset of admissible controls, and in experimental implementations in quantum control they are the only control laws that can be reasonably applied. However, the class of piecewise constant controls is not suited to prove existence of optimal controls [72].
Given an admissible control u(·) and an initial condition q(0) = q in ∈ M , there exists a unique solution q(·) of Eq. (1), defined at least for small times [74]. A continuous curve q(·) for which there exists an admissible control u(·) such that Eq. (1) is satisfied is said to be an admissible trajectory.
Let us present some typical situations encountered in quantum control. Consider the time evolution of the wave function of a closed N -level quantum system. In this case, under the dipolar approximation [10,75,76], the dynamics are governed by the Schrödinger equation (in units where = 1) where ψ, the wave function, belongs to the unit sphere in C N and H 0 , . . . , H m are N × N Hermitian matrices. The control parameters u j (t) ∈ R are the components of the control u(·). This control problem has the form (1) with Note that the uncontrolled part corresponding to the H 0 -term is called the drift. The solution of the Schrödinger equation can also be expressed in terms of the unitary operator U(t, t 0 ), which connects the wave function at time t 0 to its value at t: ψ(t) = U(t, t 0 )ψ(t 0 ). The propagator U(t, t 0 ) also satisfies the Schrödinger equation with initial condition U(t 0 , t 0 ) = I N . In quantum computing, the control problem is generally defined with respect to the propagator U. Equation (3) has the form (1) with M = U (N ) ⊂ C N ×N and q = U. The wave function formalism is well adapted to describe pure states of isolated quantum systems, but when one lacks information about the system the correct formalism is the one of mixed-state quantum systems. The state of the system is then described by a density operator ρ, which is a N × N positive semi-definite Hermitian matrix of unit trace. For a closed quantum system, the density operator is a solution of the von Neumann equatioṅ For an open N -level quantum system interacting with its environment, the dynamics of ρ are governed in some cases [80,81] by the following first-order differential equation, called the Kossakowski-Lindblad equation [77,78]: This equation differs from the von Neumann one in that a dissipation operator L acting on the set of density operators has been added. This linear operator which describes the interaction with the environment cannot be chosen arbitrarily. Its expression can be derived from physical arguments based on a Markovian regime and a small coupling with the environment [79,81]. From a mathematical point of view, the problem of finding dynamical generators for open systems that ensure complete positivity of the dynamical evolution was solved in finite-and infinite-dimensional Hilbert spaces [77,78]. The operator L is a linear operator acting on the space of density matrices that can be expressed for a N -level quantum system as where the matrices V 1 , . . . , V N 2 −1 are trace-zero and orthonormal. The linear mapping L is completely positive if and only if the matrix a = (a kk ) is positive [82,83]. The density operator ρ can be represented as a vector ρ by stacking its columns. The corresponding time evolution is generated by superoperators in the Schrödinger-like form Here B N 2 −1 denotes the ball of radius 1 in R N 2 −1 [84].
The initial and final states. When considering a quantum control problem, the goal in most situations is not to bring the system from an initial state q in to a final state q fi , but rather to reach at time T a smooth submanifold T of M (see [85] for a precise definition), called target: This issue arises, for instance, in the population transfer from a state ψ in to an eigenstate ψ fi of the field-free Hamiltonian H 0 . In this case, since the phase of the final state is not physically relevant, T is characterized by {e iθ ψ fi | θ ∈ [0, 2π]}. It can also happen that the initial condition q(0) = q in is generalized to q(0) ∈ S, where S is a smooth submanifold of M . However, for the sake of presentation, we will not treat this case here, the changes to be made to the method being straightforward. Finally, note that the time T can be fixed or free, as, for instance, in a time-minimum control problem.
The optimal control problem. Two different optimal control approaches can be used to steer the system from q in to a target T .
• Approach A: Prove that the target T is reachable from q in (in time T if the final time is fixed or in any time otherwise) and then find the best possible control realizing the transfer. This approach requires to solve the preliminary step of controllability. Essentially, we need to show that:  The manifold M is R 2 and the coordinates of q are (q 1 , q 2 ). The intersection between the two sets R T (q in ) and T is non-empty for a long enough time T . Note that the initial point of the dynamics does not belong to the reachable set at time T . This is due to a specific choice of the dynamical system with a drift.
or that and then solve the minimization problem where f 0 : M × U → R is a smooth function, which in many quantum control applications depends only on the control. An example is given by the functional T 0 u 2 (t)dt which represents the energy used in the control process. The control time T is fixed or free. This integral is generally called the cost functional. A schematic illustration of the reachable set R T (q in ) and the target T is given in Fig. 1. The test of controllability is sometimes easy (as, for instance, for low-dimensional closed quantum systems [87,88]) and sometimes more difficult. We recall that a closed quantum system is controllable if the matrix Lie algebra generated by the matrices H 0 , . . . , H m is su(N ). For general systems, a useful sufficient condition for controllability is described in Appendix A. When the test of controllability can be performed, this approach is to be preferred since it permits to reach exactly the final state. • Approach B: Find a control that brings the system as close as possible to the target, while minimizing the cost. This approach is used for systems for which the controllability step cannot be easily verified. In this case, the initial point Table 1: Summary of the different optimal control approaches.
Approach Aq(t) = f (q(t), u(t)) When controllability can be verified, i.e., one can prove that: When controllability cannot be verified , u(t)) dt + d(T , q(T )) → min T fixed or free is fixed and the final point is free, but the cost contains a term (denoted d(·, ·) in the next formula) depending on the distance between the final state of the dynamics and the target: where T is fixed or free.
Example 1. An example is given by open quantum systems governed by the Kossakowski-Lindblad equation, for which the characterization of the reachable set is quite involved [90,91]. If we denote by ρ fi the target state, a cost functional to minimize penalizing the energy of the control and the distance to the target can be where · is the norm corresponding to the scalar product of density matrices ρ 1 |ρ 2 = Tr[ρ † 1 ρ 2 ]. Optimization problem in these two approaches should be of course considered together with the dynamics (1) and the initial and final conditions. They are summarized in Tab. 1.

The different steps to solve an optimal control problem
The steps to determine a solution to the minimization problems (7) and (8) are similar to finding the minimum of a smooth function f 0 : R → R.
0. Find conditions which guarantee the existence of solutions. We recall that among smooth functions f 0 : R → R, it is easy to find examples not admitting a minimum (e.g., the function x → e −x and the function x → x do not have minima). This step is crucial. If it is skipped, firstorder conditions may give a wrong candidate for optimality (see below for details) and numerical optimization schemes may either not converge or converge towards a solution which is not a minimum. For optimal control problems, there exist several existence tests, but they are not always applicable or easy to use. In Sec. 5, we present the Filippov test.
1. Apply first-order necessary conditions. For a smooth function f 0 : R → R, this means that ifx is a minimum then d dx f 0 (x) = 0. This condition gives candidates for minima, i.e., identifies local minima, local maxima, and saddles. Note that if one does not verify a priori existence of minima, first-order conditions could give wrong candidates. Think for instance to the function x → (x 2 +1/2)e −x 2 . This function has a single local minimum, obtained at x = 0, whose value is 1/2, which is well identified by first-order conditions. However its infimum is zero (for x → ±∞, the function tends to zero). For optimal control problems, first-order necessary conditions should be given in an infinite-dimensional space (a space of curves) and they are expressed by the PMP, which is presented in Sec. 6.2. In Approach A, note that the condition that the system reaches exactly the target is a constraint leading to the appearance of Lagrange multipliers (normal and abnormal). This point is discussed in details in Sec. 6.1.

2.
Apply second-order conditions. For instance, for a smooth function f 0 : R → R, among the points at which we have d dx f 0 (x) = 0, a necessary condition to have a minimum is d 2 dx 2 f 0 (x) ≥ 0. This step is generally used to reduce further the candidates for optimality. For optimal control problems, there are several second-order conditions, such as higher-order Pontryagin Maximum Principles or Legendre-Clebsch conditions (see for instance [28,31,32]). In some cases, this step is difficult and it could be more convenient to go directly to the next one.
3. Selection of the best solution among all candidates. Among the set of candidates for optimality identified in step 1 and (possibly) further reduced in step 2, one should select the best one. This step is often done by hand if the previous steps have identified a finite number of candidates for optimality. For optimal control problems, one often ends up with infinitely many candidates for optimality and this step is generally very difficult.
There are of course specific examples for which the solution is particularly simple. This is the case of convex problems, for which only first-order conditions should be applied, since the existence step is automatic and first-order conditions are both necessary and sufficient for optimality. This situation is however rare in quantum control and we will not discuss it further.

Existence of solutions for Optimal Control Problem: the Filippov test
The existence theory for optimal control is difficult and, unfortunately, there is no general procedure that can be applied in any situation. In this section, we present the most important technique, the Filippov test that allows to tackle several types of problems. In order to keep this paragraph as accessible as possible, we present below only the main ideas and some propositions derived from the Filippov test. These results can be directly applied to quantum systems. A complete statement of the Filippov test is provided in Appendix B. We emphasize that it is fundamental to verify the existence of optimal controls before applying first-order conditions (i.e., the PMP). Otherwise, as discussed in the finite-dimensional case, it may occur that the PMP has solutions, but none of them is optimal. Let us consider the problem in Approach A with T fixed. Problem P1q In order to tackle the existence problem, we define a new variable q 0 obtained as the value of the cost during the time-evolution, that is, and we denoteq = (q 0 , q). The dynamics of the new stateq in R × M are given byq This control system is called the augmented system. The minimization problem in integral form, min T 0 f 0 (q(t), u(t)) dt, becomes a problem of minimization of one of the coordinates at the final time, i.e., min q 0 (T ).
We denote byR T (0, q in ) the reachable set at time T starting from (0, q in ) for the augmented system. The key observation on which optimal control is based is expressed by the following proposition. Proposition 1. If q(·) is an optimal trajectory for problem (P1), thenq(T ) ∈ ∂R T (0, q in ).
is nonempty and compact, an optimal trajectory for problem (P1) exists [92]. We deduce the following property.
is nonempty, then there exists a solution to problem (P1).
Hence the compactness ofR T (0, q in ) is a key point. A similar reasoning allows to relate the compactness of the reachable set within time T and the existence of solutions for the minimum time problem. A sufficient condition for compactness of the reachable set is given by Filippov's theorem, which is stated in Appendix B. A consequence of Filippov's theorem is the following (see also Propositions 11 and 12 in Appendix B).
where ψ(t) evolves in the unit sphere S 2N −1 of C N , the matrices H 0 , . . . , H m are N × N Hermitian, and u(t) ∈ U . Let ψ 0 be the initial condition for ψ(·) and the final target T be closed. Assume that the set U is convex and compact. Then each of the following two optimal control problems admits a solution: 1. T is fixed, R T (q in ) intersects T , and the cost function f 0 : Example 2. The compactness of U is a key assumption to ensure the existence of an optimal solution. We come back to the example of Sec. 2 and we consider the control of a two-level quantum system whose dynamics is governed by the Hamiltonian H = u ( 0 1 1 0 ) with u real-valued [50]. The goal of the control is to steer the wave function from ψ in = (1, 0) to the target T = {(0, e iθ ) | θ ∈ R} in minimum time. If U = R then there is no optimal solution since the target state can be reached in an arbitrary small time, while the control cannot be realized in a zero time. When U = (−1, 1), the transfer can be achieved in a time larger than π/2, but not exactly in time T = π/2. It is only in the case where U is compact, e.g. U = [−1, 1], that an optimal solution exists. We find for the constraint |u(t)| ≤ 1 a pulse which allows to bring the population from one level to another in a time T = π/2. We show in Sec. 8 and Sec. 9 how to use these results in two standard quantum control examples.

First-order conditions
For a smooth real-valued function of one variable f 0 : R → R, first-order optimality conditions are obtained from the observation that, at points where df 0 dx = 0, the function f 0 , which is well approximated by its first-order Taylor series, cannot be optimal since it behaves locally as a non-constant affine function. In this way, one obtains the necessary condition: Ifx is minimal for f 0 then df 0 dx (x) = 0. First-order conditions in optimal control are derived in the same way. We have to require that for a small control variation, there is no cost variation at first order.
More precisely, if J(u(·)) is the value of the cost for a reference admissi- and v(·) is another admissible control, one would like to consider a condition of the form But difficulties may arise for the following reasons.
We work in an infinite-dimensional space (the space of controls) and hence condition (10) should be required for infinitely many v(·). It may very well happen that if u(·) and v(·) are admissible controls then u(·) + hv(·) is not admissible for every h close to 0. Think, for instance, to the case in which is not admissible for any non-zero perturbation v(·) when hv(t) is strictly positive. Hence, one should be very careful in choosing the admissible variations in order to fulfill the control restrictions. In Approach A, one should restrict only to control variations for which the corresponding trajectory reaches the target. More precisely, ifq(·) is the trajectory corresponding to the controlũ(·) := u(·) + hv(·), one should add the conditioñ with T either free or constrained to be the fixed final time depending on the problem under study. Condition (11) should be considered as a constraint for the minimization problem, which results in the use of Lagrange multipliers (normal and abnormal).
The occurrence of Lagrange multipliers in optimal control is therefore not due to the fact that the optimization takes place in an infinite-dimensional space, but is rather a general feature of constrained minimization problems, as explained in Sec. 6.1.

Why Lagrange multipliers appear in constrained optimization problems
We first recall how to find the minimum of a function of n variables f 0 (x), where x = (x 1 , . . . , x n ), under the constraint f (x) = 0, with the method of Lagrange multipliers.
Here f 0 and f are two smooth functions R n → R. We have two cases. • Ifx is a point such that f (x) = 0 with ∇f (x) = 0, then the implicit function theorem guarantees that {x | f (x) = 0} is a smooth hypersurface in a neighborhood ofx. In this case, a necessary condition for f 0 to have a minimum atx is that the level set of f 0 (i.e., the set on which f 0 takes a constant value) is not transversal to the set {x | f (x) = 0} atx. See Figure 3. More precisely, this means that This statement can be proved by assuming, for instance, that ∂ xn f (x) = 0.
The set {x | f (x) = 0} can then be expressed locally aroundx as x n = g(x 1 , . . . , x n−1 ). The requirement that . Notice that λ could be equal to zero. This case corresponds to the situation in which f 0 has a critical point atx even in the absence of the constraint.
• Ifx is a point such that f (x) = 0 with ∇f (x) = 0 then the set {x | f (x) = 0} could be very complicated in a neighborhood ofx (typical examples are a single point, two crossing curves, . . . but it could be any closed set). In general the value of f 0 at these points cannot be compared with neighboring points by requiring that a certain derivative is zero (think for instance to the case in which {x | f (x) = 0} is an isolated point). However, they are candidates to optimality. As an illustrative example, consider the case where n = 2, These results can be rewritten in the following form.
Theorem 4 (Lagrange multiplier rule in R n ). Let f 0 and f be two smooth functions from R n to R. If f 0 has a minimum atx on the set {x | f (x) = 0}, To show that this statement is equivalent to what we just discussed, we observe that the second equality in (13) gives the constraint f (x) = 0. For the first equation, we have two cases. Ifλ 0 = 0 then we can normalizeλ 0 = −1 and we getλ∇ x f (x) − ∇ x f 0 (x) = 0, i.e., Eq. (12) with the change of notation λ →λ. Ifλ 0 = 0 thenλ = 0 and we get ∇ x f (x) = 0, that is, the second case studied above.
The quantitiesλ andλ 0 are respectively called Lagrange multiplier and abnormal Lagrange multiplier. If (x,λ 0 ,λ) is a solution of Eq. (13) withλ 0 = 0 (resp.,λ 0 = 0) thenx is called a normal extremal (resp., abnormal extremal). An abnormal extremal is a candidate for optimality and occurs, in particular, when we cannot guarantee (at first order) that the set {x | f (x) = 0} is a smooth curve. Abnormal extremals are candidates for optimality regardless of cost f 0 . Note that ifx is such that ∇ x f (x) = 0 and ∇ x f 0 (x) = 0 thenx is both normal and abnormal. This is the case in whichx satisfies the first-order condition for optimality even without the constraint, but we cannot guarantee that the constraint is a smooth curve.
In the (infinite-dimensional) case of an optimal control problem, normal and abnormal Lagrange multipliers appear in a very similar way.

Statement of the Pontryagin Maximum Principle
In this section, we state the first-order necessary conditions for optimal control problems, namely the PMP. The basic idea is to define a new object (the pre-Hamiltonian, see Eq. (14) below) which allows to formulate the Lagrange multiplier conditions in a simple and direct way.
The theorem is stated in a more general form that unifies and slightly generalizes the optimal control problems of Approaches A and B. In particular, we add to the cost T 0 f 0 (q(t), u(t))dt a general terminal cost φ(q(T )). In Approach A, we have φ = 0, while in Approach B, φ represents the distance from q(T ) to the target T . We allow the target T to coincide with M . This corresponds to leaving the final point q(T ) free in Approach B.
Theorem 5. Consider the optimal control probleṁ • T is a (non-empty) smooth submanifold of M . It can be reduced to a point (fixed terminal point) or coincide with M (free terminal point), • f , f 0 are smooth, Define the function (called pre-Hamiltonian) with which corresponds to the maximization condition.
where dφ is the differential of the function φ.
Some comments are in order.
• A proof of the PMP can be found, for instance, in [28,27]. An intuitive derivation based on the Lagrange multiplier rule is presented in Section 7.1 in the case in which T is fixed, the final point is free, and U = R m .
• The covector p is called adjoint state in the control theory literature [70], while p 0 is the abnormal multiplier. The quantities p(·) and p 0 play the role of Lagrange multipliers for the constrained optimization problem. We point out the similarity between the expressions of H and of Λ in Th. 4 (with the change of notation q → x and p → λ).
• A trajectory q(·) for which there exist p(·), u(·), and p 0 such that (q(·), p(·), u(·), p 0 ) satisfies all the conditions given by the PMP is called an extremal trajectory and the 4-uple (q(·), p(·), u(·), p 0 ) an extremal or, equivalently, an extremal lift of q(·). Such an extremal is called normal if p 0 = 0 and abnormal if p 0 = 0. It may happen that an extremal trajectory q(·) admits both a normal extremal lift (q(·), p 1 (·), u(·), p 0 ) and an abnormal one (q(·), p 2 (·), u(·), 0). In this case, we say that the extremal trajectory q(·) is a non-strict abnormal trajectory. Note that (as in the finite-dimensional case) abnormal trajectories are candidates for optimality regardless of the cost. In the finite-dimensional case, they correspond to singularities of the constraint function, while here they correspond to singularities of the functional associating with a control v(·) the endpoint at time T of the solution ofq(t) = f (q(t), v(t)), q(0) = q in . It is worth noticing that abnormal extremals do not only appear in pathological cases, but they are often present in real-world applications, as for instance in the two examples presented in Sec. 8 and 9.
• The PMP is only a necessary condition for optimality. It may very well happen that an extremal trajectory is not optimal. The PMP can therefore provide several candidates for optimality, only some of which are optimal (or even none of them if the step of existence has not been verified, see Sec. 5).
As a consequence, some useful normalizations are possible. A typical normalization for normal extremals is to require p 0 = − 1 2 but other choices are also possible.
• When there is no final cost (φ = 0), the transversality condition (v) simplifies to: When the final point is fixed (T = {q fi }), T q(T ) T is a zero-dimensional manifold and hence condition (15) is empty. When the final point is free (T = M ) the transversality condition simplifies to p(T ) = p 0 dφ(q(T )). In local coordinates, we recover that p(T ) is proportional to the gradient of φ evaluated at the point q(T ). Notice that, since (p(T ), p 0 ) = 0, in this case one necessarily has p 0 = 0. Table 2 gives a list of the possible extremal solutions of the PMP. Example 3. As a general example in quantum control, we consider a dynamical system governed by Eq. (2) where the goal is to minimize at the fixed final time T the cost −| ψ fi |ψ(T ) | 2 + 1 2 T 0 m j=1 u 2 j (t)dt, where ψ fi is a target state towards which we want to drive the system (up to a global phase) [93]. A direct application of the PMP shows that the pre-Hamiltonian H can be expressed as: where the adjoint state, denoted here by χ, is an abstract wave function which can be chosen so that it belongs to the unit sphere in C N , χ|χ = 1. Since ψ and χ are complex-valued functions, the pre-Hamiltonian is defined through the real part of the scalar product between χ andψ. The standard definition used in Th. 5 can be found by introducing the real and imaginary parts of the wave functions. Using Eq. (2), we deduce that which leads to i.e., χ also satisfies the Schrödinger equation. We stress that this condition is only true in the bilinear case. A specific equation has to be computed for other dynamics, such as, e.g., the Gross-Pitaevskii equation [94]. The final condition χ(T ) is given by the transversality condition v) of the PMP: The maximization condition of the PMP leads to the constraints ∂H ∂u j (ψ, χ, u j , p 0 ) = 0 for j = 1, . . . , m. A direct computation from Eq. (16) gives For the normal extremal with p 0 = −1, we finally get

Use of the PMP
The application of the PMP is not so straightforward. Indeed, there are many conditions to satisfy and all of them are coupled. This section is aimed at describing how to use it in practice.
The following points should be followed first for normal extremals (p 0 < 0) and then for abnormal extremals (p 0 = 0). In the first case, p 0 can be normalized to −1/2 since p 0 is defined up to a multiplicative positive factor. In the different steps, several difficulties (that are briefly mentioned) may arise. Most of them should be solved case by case, since they can be of different nature depending on the problem under study.
Step 1. Use the maximization condition iii) to express, when possible, the control as a function of the state and of the covector, i.e., u = w(q, p). Note that if we have m controls (e.g., if U is an open subset of R m ) then the firstorder maximality conditions give m equations for m unknowns. When the maximization condition permits to express u as a function of q and p, we say that the control is regular, otherwise the control is said to be singular. We may have regions where the control is regular and regions where it is singular. For singular controls, finer techniques have to be used to derive the expression of the control. These different points are discussed in the examples in Sec. 8 and 9.
Step 2. Insert the control found in the previous step into the Hamiltonian equations i) and ii): In case the previous step provides a smooth w(·, ·), this is a well-defined set of 2n equations for 2n unknowns. Note, however, that the boundary conditions are given in a non-standard form since we know q(0) but not p(0). Instead of p(0), we have a partial information on q(T ) and p(T ) depending on the dimension of T (see the next step to understand how these final conditions are shared between q(T ) and p(T )). We then solve Eq. (19) for fixed q(0) = q in and any p(0) = p in ∈ T * qin M . Let us denote the solution as q(t; p in , p 0 ), p(t; p in , p 0 ).  When the maximization condition of the PMP gives u = ω(q, p) Singular control When the control is not regular.
We stress that when w(·, ·) is not regular enough, solutions to the Cauchy problem (19) with q(0) = q in and p(0) = p in may fail to exist or to be unique.
Step 3. Find p in such that Note that if T is reduced to a point and T is fixed, we get n equations for n unknown (the components of p in ). If T is free then an additional equation is needed. This condition is given by the relation iv) in the PMP. If T is a k-dimensional submanifold of M (k ≤ n) then Eq. (20) provides only n − k equations and the remaining ones correspond to the transversality condition v) of the PMP.
Step 4. If Eq. (20) (together with the transversality condition and condition iv) of the PMP if T is free) has a unique solution p in and if we have verified a priori the existence step, then the optimal control problem is solved. Unfortunately, in general there is no reason for Eq. (20) to provide a unique solution. Indeed, the PMP is only a necessary condition for optimality. If several solutions are found, one should choose among them the best one by a direct comparison of the value of the cost. This is, in general, a non-trivial step, complicated by the difficulty of solving explicitly Eq. (20). For this reason, several techniques have been developed to select the extremals. Among others, we mention the sufficient conditions for optimality given by Hamilton-Jacobi-Bellman theory and synthesis theory. We refer to [Boscain and Piccoli(2015)] for a discussion. In Example 1 (Section 8) we are able to select the optimal solution without the use of sufficient conditions for optimality, while this is not the case in Example 2 (Section 9).
Example 4. We come back to the case of Example 3. We have shown with Eq. (18) that the maximization condition allows to express the m controls u j as functions of ψ and χ in the normal case. This situation therefore corresponds to Step 1 above where the control is regular. For abnormal extremals for which p 0 = 0, we get ( χ|H j |ψ ) = 0, and the control is singular because this relation does not give directly the expression of u j . Applying Step 2, we obtain in the regular situation the following coupled equations for ψ and χ: with the boundary conditions ψ(0) = ψ in and Eq. (17). In Step 3, we then solve Eq. (22) to find the initial condition χ(0) such that the final state χ(T ) satisfies Eq. (17) at time T . The numerical procedures used to select the initial condition χ(0), called shooting methods in control theory, are based on suitable adaptations of the Newton algorithm [35,36].
Step 4 consists finally in comparing the cost of the different solutions found in Step 3.
In the abnormal case, we use the fact that Eq. (21) is satisfied in a non-zero time interval so the time derivatives of ( χ|H j |ψ ) are also zero. The first time derivative leads to the m relations . We deduce that the control u is given as a function of ψ and χ as u = R −1 s. If this system is singular then the second time derivative has to be used. This is the case, e.g., for m = 1, when a further constraint has to be fulfilled, namely, ( χ|[H 0 , H 1 ]|ψ ) = 0. From the derivation of u, we then apply Steps 2 and 3 to the abnormal extremals.

Gradient-based optimization algorithm
The aim of this section is to introduce a first order gradient-based optimization algorithm based on the PMP. We first derive the necessary conditions of the PMP in the case of a fixed control time without any constraint on the final state and on the control. This construction is known in control theory as the weak PMP. Note that we consider only the case of regular control. Iterative algorithms can be deduced from these conditions. In a second step, we apply this idea to quantum control and we show how a gradient-based optimization algorithm, GRAPE [45], can be designed from this approach.

The Weak Pontryagin Maximum Principle
We consider a control system whose dynamics are governed by Eq. (1), when the final state is free and the control is unconstrained. The objective is to solve a control problem in the Approach B, as defined in Sec. 3 with a fixed control time T . We recall that the cost functional to minimize can be expressed as J(u(·)) = T 0 f 0 (q(t), u(t)) dt + d(T , q(T )).
Considering the evolution equation (1) as a dynamical constraint (in infinite dimension), in order to apply (formally) the Lagrange multiplier rule for normal extremals, we introduce the functional Λ(p(·), u(·)) = d(T , q(T )) + T 0 f 0 (q(t), u(t)) dt We stress that the Lagrange multiplier p(·) is here a function on [0, T ]. Integrating by parts Eq. (23), we obtain Λ(p(·), u(·)) = d(T , q(T )) + p(T ), q(T ) Note that the scalar function H has the same expression (with p 0 = −1) as the pre-Hamiltonian H introduced in Sec. 6.2 for the statement of the PMP. Since there is no constraint on the control law, i.e., u(t) ∈ R m for any time t, we consider the variation δΛ in Λ at first order due to the variation δu of u. This change of control induces a variation of the trajectory δq(t) with δq(0) = 0, the initial point being fixed. Note that the adjoint state p is not modified. We arrive at: A necessary condition for Λ to be an extremum is δΛ = 0 for any variation δu. A solution is given by taking an adjoint state p satisfyinġ the final boundary condition and requiring ∂H ∂u = 0 on [0, T ].
As it could be expected, we find here a weak version of the equations of the PMP introduced in Sec. 6.2 where the maximization of the pre-Hamiltonian is replaced by an extremum condition given by the partial derivative with respect to u. We point out that this approach works if the set U is open.

Gradient-based optimization algorithm
The set of nonlinear coupled differential equations can be solved numerically by an iterative algorithm. The basic idea used in such algorithms can be formulated as follows. Assume that a control u(·) sufficiently close to the optimal solution is known. If p(·) satisfies Eq. (26) and (27)  where is a small positive parameter. The iterative algorithm is then described by the following steps.
2. Propagate forward the state of the system q fromq = f (q, u), with the initial condition q(0) = q 0 .  6. Go to step 2 and repeat until a given accuracy is reached. This algorithm is an example of first-order gradient-based optimization algorithm. By construction, it converges towards an extremal control of J which is not, in general, a global minimum solution of the control problem, but only a local one. Note that several numerical details are hidden in the description of this method. Among others, we mention the choice of the guess control which allows to reach a good local solution and the determination of the parameter . This latter must be sufficiently small to remain in the first order approximation, but large enough to reduce the number of iterations and therefore the computational time. We refer the reader to standard numerical optimization textbooks to address these issues [36].
This approach can be directly applied to quantum systems. The bilinearity of quantum dynamics allows to simplify the different terms used in the algorithm. We consider a quantum system whose dynamics are governed by the Schrödinger equation The goal of the control process is to bring the system from ψ in towards ψ fi in a fixed time T . The control problem aims at minimizing the cost In the normal case, the pre-Hamiltonian H can be expressed as: where the wave function χ is the adjoint state of the system. We thus deduce that the gradient on which the iterative algorithm is based is given by We find with Eq. (28) the standard control correction used in the GRAPE algorithm in quantum control [45].

Example 1: A three-level quantum system with complex controls
In this section, we mainly use the results of [50], see also [95,49,96]. Note, however, that original results concerning the selection of the best extremal among all the possible solutions are presented.

Formulation of the quantum control problem
We consider a three-level quantum system whose dynamics are governed by the Schrödinger equation. The system is described by a pure state ψ(t) belonging to a three-dimensional complex Hilbert space. The system is characterized, in the absence of external fields, by three energy levels E 1 , E 2 , and E 3 and is controlled by the Pump and the Stokes pulses which couple, respectively, states one and two and states two and three [97]. Note that there is no direct coupling between levels one and three. The time evolution of ψ(t) is given by where Here Ω 1 (t), Ω 2 (t) ∈ C are the two time-dependent complex control parameters. We denote by ψ 1 (t), ψ 2 (t), and ψ 3 (t) the coordinates of ψ(t), that is, ψ(t) = (ψ 1 (t), ψ 2 (t), ψ 3 (t)). They satisfy |ψ 1 (t)| 2 + |ψ 2 (t)| 2 + |ψ 3 (t)| 2 = 1, leading to M = S 5 , a manifold of real dimension 5. The goal of the control process is to transfer population from the first eigenstate to the third one in a fixed time T . In other words, the aim is to find a trajectory in M going from the submanifold |ψ 1 | 2 = 1 to the one with |ψ 3 | 2 = 1. The system is completely controllable thanks to Point 2 in Proposition 9 and the approach A can be chosen. The optimal control problem is defined through the cost functional to be minimized. The cost C can be interpreted as the energy of the control laws used in the control process. We consider the specific case in which the control parameters are in resonance with the energy transition. More precisely, we assume that the pulses Ω 1 (t) and Ω 2 (t) can be expressed as with u 1 (t), u 2 (t) ∈ R to be optimized. Note that this assumption is not restrictive since it can be shown that the resonant case corresponds to the optimal solution [50,95]. The uncontrolled part, called the drift, together with the imaginary unit in the Schrödinger equation, can be eliminated through a unitary transformation Y (t) given by Defining a new wave function x such that ψ(t) = Y (t)x(t), we obtain that x(t) solves the Schrödinger equation Since Y only modifies the phases of the coordinates, ψ(t) and x(t) correspond to the same population distribution, i.e., setting Computing explicitly H we arrive at Without loss of generality, the optimal control problem can be restricted to the submanifold . This statement is trivial if the initial condition belongs to S 2 (i.e., if x(0) is real). Otherwise, a straightforward change of phase of the coordinates ψ k allows to come back to this condition. Equation (29) can be expressed in a more compact form asẋ where Notice that F 1 and F 2 are two vector fields defined on the sphere S 2 representing, respectively, a rotation along the x 3 axis and along the x 1 axis. Since Ω 1 (t) and Ω 2 (t) differ from u 1 (t) and u 2 (t) only for phase factors, the minimization problem becomes with T fixed. Concerning initial and final conditions, since the goal is to go from the submanifold |ψ 1 | 2 = 1 to the submanifold |ψ 3 | 2 = 1, we can assume without loss of generality that x(0) = (1, 0, 0) (again, a straightforward change of coordinates allows to come back to this condition otherwise). Since we are now restricted to real variables, the target is T = {(0, 0, +1), (0, 0, −1)}. Now, being the target made of two points only, one should compute separately the optimal trajectories going from (1, 0, 0) to (0, 0, 1) and those going from (1, 0, 0) to (0, 0, −1). Finally between all these trajectories, one should take the ones having the smallest cost. Because of the symmetries of the system, the two families of optimal trajectories have precisely the same cost (this will be clear in the explicit computations later on). As a consequence, without loss of generality, we can fix the final condition as x(T ) = (0, 0, 1). The problem (30)-(31) with fixed initial and final conditions is actually a celebrated problem in OCT called the Grushin model on the sphere [98,49,99].

Existence
For convenience, let us re-write the optimal control problem as follows: To prove the existence of solutions to P Grushin (T ), one could be tempted to use Propositions 3 and 11. However u 1 and u 2 take values in R and, hence, the second hypothesis of the proposition is not verified.
Instead, we are going to use the following fact.
To prove this proposition we first state a general lemma for driftless systems.
In the following, for the optimal control problem under study, it is convenient to normalize T in such a way that u 1 (t) 2 + u 2 (t) 2 = 1. Usually, when one makes this choice the trajectories are said to be parameterized by arc length. If the objective is to reach the target at time T , it is sufficient to use the controls (α u 1 (α t), α u 2 (α t)), where α = T T . When T is fixed in such a way that u 1 (t) 2 + u 2 (t) 2 = 1, we call problem P Grushin (T ) simply P Grushin .
Notice that, as a consequence of Lemma 7, if T is not fixed then P Grushin (T ) has no solution. Indeed, assume by contradiction that x(·), defined on [0, T 0 ] and corresponding to controls u j (·), j = 1, 2, is a minimizer for T free. Let c ∈ (0, 1). The trajectory corresponding to controlsū j (·) = u j (c ·)c, j = 1, 2, is a reparameterization of x(·) reaching the final point at timeT = T 0 /c with a cost Proof of Proposition 6. Let us define We are going to use the Cauchy-Schwarz inequality f, g 2 ≤ f 2 g 2 (with equality holding if and only if f and g are proportional), which holds in any Hilbert space. This inequality simply tells that the scalar product of two vectors is less than or equal to the product of the norms of the two vectors (with equality holding iff the two vectors are collinear). In particular, this can be used in the (with equality holding iff f ∝ g, a.e.). Now, let f (t) = u 1 (t) 2 + u 2 (t) 2 and g(t) = 1 for t ∈ [0, T ]. Notice that f, g ∈ U ⊂ L 2 ([0, T ], R). We have L(u(·)) 2 ≤ E(u(·))T (with equality holding iff u 1 (t) 2 + u 2 (t) 2 = const a.e.). Now, let u(·) be a minimizer of E defined on [0, T ]. Assume by contradiction that u 1 (t) 2 + u 2 (t) 2 is not a.e. constant. Then L(u(·)) 2 < E(u(·))T . Letx(·) be an admissible trajectory defined on [0, T ] corresponding to controls satisfying ū 1 (t) 2 +ū 2 (t) 2 = L(u(·))/T a.e., of which x(·) is a reparameterization as in Lemma 7. One immediately checks that L(u(·)) = L(ū(·)). For this trajectory we have E(ū(·))T = L(ū(·)) 2 = L(u(·)) 2 < E(u(·))T, contradicting the fact that u(·) is a minimizer of E.

Now we have the following.
Proposition 8. The problem P Grushin is equivalent to the problem of minimizing time with the constraint on the controls u 1 (t) 2 + u 2 (t) 2 ≤ 1.
Proof. Since for P Grushin we have u 1 (t) 2 + u 2 (t) 2 = 1 a.e., then Hence P Grushin is equivalent to the problem of minimizing T with the constraint on the controls u 1 (t) 2 + u 2 (t) 2 = 1. To conclude the proof, let us show that a trajectory corresponding to controls for which the condition is not satisfied cannot be optimal for the time-optimal control problem mentioned in the statement. Actually if (32) is not satisfied then L = T 0 u 1 (t) 2 + u 2 (t) 2 < T and an arc length reparameterization of the trajectory reaches the target in time exactly L.
Let us now go back to the problem of existence of optimal trajectories for P Grushin . Thanks to Proposition 8, P Grushin can be equivalently recast as a time-optimal control problem with controls in the convex and compact set U = {(u 1 , u 2 ) ∈ R 2 | u 2 1 + u 2 2 ≤ 1}. We can then apply Propositions 3 and 12 and deduce the existence of an optimal trajectory for P Grushin (as well as P Grushin (T ) for every T > 0).

Application of the PMP
Before applying the PMP, it is convenient to reformulate the problem in spherical coordinates. Indeed, one can prove the following statement.
Claim. Consider an optimal control problem as in the statement of the PMP (Theorem 5). If all admissible trajectories starting from q in are contained in a submanifold of M of dimension strictly smaller than n, then each admissible trajectory has an abnormal extremal lift. This property can be qualitatively justified as follows. We have already mentioned that abnormal trajectories correspond to singularities of the functional associating with a control law the endpoint of the corresponding controlled trajectory. If all admissible trajectories starting from q in are contained in a proper submanifold of M , then the endpoint functional is everywhere singular, meaning that each admissible trajectory is abnormal.
As a consequence, since in our case all trajectories are contained in the sphere S 2 , if we apply the PMP in R 3 all optimal trajectories admit an abnormal extremal lift. This creates additional difficulties that can be avoided working directly on S 2 in spherical coordinates.
In these coordinates, the starting point x(0) = (1, 0, 0) and the final point x(T ) = (0, 0, 1) become (θ, ϕ)(0) = (π/2, 0), (θ, ϕ)(T ) = (π/2, π/2). Notice that these coordinates are singular for θ = 0 and θ = π but such a singularity does not create any problem, as can be checked by using a second system of coordinates around the singularity. The control system takes then the form It can be simplified by using the controls v 1 and v 2 defined by which do not modify the expression of the cost C since u 2 1 + u 2 2 = v 2 1 + v 2 2 . The control system becomes now Let us now apply the PMP. Set q = (θ, ϕ) and let p = (p θ , p ϕ ). The pre-Hamiltonian (14) has the form We consider the steps of Section 6.3 first for abnormal (p 0 = 0) and then for normal (p 0 = − 1 2 ) extremals.
Step 1. In this step, we have to apply the maximization condition to find the control as a function of q and p. Since the controls are unbounded and the Hamiltonian is concave, the maximization condition is equivalent to: For abnormal extremals, we obtain These conditions do not permit to obtain the control as a function of q and p. Hence, for this problem, abnormal extremals correspond to singular controls. Since p and p 0 cannot be simultaneously zero, the only possibility to have an abnormal extremal is that θ(t) ≡ π/2 on [0, T ]. In this case, from Eq. (33), we deduce that ϕ(t) should be constant. As a consequence, an abnormal extremal trajectory starting from the initial condition (θ, ϕ)(0) = (π/2, 0) will never reach the final condition (θ, ϕ)(T ) = (π/2, π/2) and we can disregard these trajectories.
For normal extremals, condition (35) gives Hence, we obtain the controls as a function of q and p and we can conclude that normal extremals correspond to regular controls.
Step 2. Let us insert (36) into the Hamiltonian equations i) and ii) of Theorem 5. We have to consider the case p 0 = −1/2 only. We obtain: Equation (40) tells us that p ϕ is a constant of the motion, denoted by a in the following. We are then left with the differential equationṡ θ = p θ ,ṗ θ = a 2 cot(θ)(1 + cot 2 (θ)).
Once these are solved, ϕ is obtained by integrating in time equation (39) which now has the formφ = a cot 2 (θ).
Equations (41) and (42) should be solved for every value of a ∈ R with the initial conditions The last condition comes from the property that the maximized Hamiltonian is now fixed to 1 2 (corresponding to the choice of taking T in such a way that optimal trajectories are parameterized by arc length). More precisely: (p θ (t) 2 + a 2 cot 2 (θ(t))).
From expression (43), requiring x 2 (T ) = 0 we get sin( √ 1 + a 2 T ) = 0. Then from (44), requiring x 1 (T ) = 0 we arrive at cos(a T ) = 0. We have then the conditions 1 + a 2 T = n 1 π, aT = π 2 + n 2 π, with n 1 , n 2 ∈ Z. Notice that n 1 > 0 since T > 0, and hence from which we deduce that |n 2 + 1 2 | < n 1 . It follows that a = (n 2 + 1 2 )/n 1 1 − ( The target is reached at time Step 4. The previous step provided a discrete set of trajectories reaching the target. The ones arriving in shorter time corresponds to n 1 = 1 and n 2 = 0 or n 2 = −1, for which T = π √ 3 2 and a = ±1/ √ 3. The final expression of the optimal trajectories and optimal controls are where ε = ±1 is the sign of a. We point out that the two trajectories arriving at time T at the point (0, 0, 1) are those corresponding to n 2 = 0 (i.e., to a = 1/ √ 3) and to p θ (0) = ∓1 (i.e., to the sign + in formula (43)), while the two trajectories arriving at time T at the point (0, 0, −1) are associated with n 2 = −1 (i.e., to a = −1/ √ 3). See Section 8.1. The four trajectories obtained in this way are optimal since we know that optimal trajectories exist and the ones that we have selected are the best among all trajectories satisfying a necessary condition for optimality. In this example, note that we can select the optimal trajectories by applying the PMP and finding by hand the best extremals, without using second order conditions nor other sufficient conditions for optimality. Figures 5 and 6 display, respectively, the two symmetric extremal trajectories reaching the target state (0, 0, 1) at time T and the time evolution of the corresponding controls. 9 Example 2: A minimum time two-level quantum system with a real control

Formulation of the control problem
We consider the control of a spin-1/2 particle whose dynamics are governed by the Bloch equation in a given rotating frame [100,101]: where M = (M x , M y , M z ) is the magnetization vector and ω the offset term. The system is controlled through a single magnetic field along the x axis that satisfies the constraint |ω x | ≤ ω max . We introduce normalized coordinates (x, y, z) = M/M 0 , where M 0 is the thermal equilibrium magnetization, a normalized control u = ω x /ω max which satisfies the constraint |u| ≤ 1, and a  normalized time τ = ω max t (denoted t below). Dividing the previous system by ω max M 0 , we get that the time evolution of the normalized coordinates is given by the equations  ẋ ẏ z where ∆ = ω ωmax is the normalized offset. The trajectories of the system lie on the Bloch sphere defined by the equation x 2 + y 2 + z 2 = 1. The manifold M is here the sphere S 2 . The differential system can be written in a more compact form asq where q = (x, y, z) is the state of the system, u(t) ∈ U = [−1, 1], and F and G are the skew-symmetric 3 × 3 matrices The vector fields Fq and Gq generate rotations around, respectively, the z and the x axis.
Existence of time-optimal trajectories. By Point 1 in Proposition 9, any initial point q 0 on the Bloch sphere M can be connected by an admissible trajectory of the control system to any other point q 1 on the Bloch sphere (see also [88]). In this case the existence of a time-optimal trajectory connecting q 0 to q 1 is a direct consequence of Proposition 12 or Proposition 3. Indeed, U = [−1, 1] is compact and the set {Fq + uGq | u ∈ U } is convex for any point q of the Bloch sphere.

Application of the PMP
The goal of the two control processes that we are considering is to steer the system in minimum time from the north pole (0, 0, 1) of the Bloch sphere to, respectively, the south pole (0, 0, −1) or the state (1, 0, 0). We follow here the results established in [48,102] (see also [103] for an experimental implementation).
The time-optimal control problem is solved by the application of the PMP. Here the cost C to minimize is where T is free. The pre-Hamiltonian can be expressed as H(q, p, u, p 0 ) = p(F + uG)q + p 0 , where p = (p x , p y , p z ) ∈ R 3 is the covector and p 0 is a nonpositive constant such that p and p 0 are not simultaneously equal to 0. (Here p is seen as a row vector and q as a column vector.) The value of H is constantly equal to 0 since the final time is free. The PMP states that the optimal trajectories are solutions of the equationsq The dynamics of the adjoint state p are given bẏ Note that d dt p(t) 2 = 0, so that p(t) does not depend on time. Its constant value is nonzero since H = 0 and (p, p 0 ) = 0.
Steps 1 and 2. Since the only term of the pre-Hamiltonian depending on the control is upGq, the maximization condition of the PMP leads to the introduction of the switching function In the regular case in which Φ(t) = 0, we deduce from the maximization condition that the optimal control is given by the sign of Φ, u(t) = sign[Φ(t)]. The corresponding trajectory is called a bang trajectory. If Φ has an isolated zero in a given time interval, then the control function may switch from −1 to 1 or from 1 to −1. A bang-bang trajectory is a trajectory obtained after a finite number of switches. Using the relation (49), we havė where [·, ·] denotes the matrix commutator operator. In particular, Φ is a C 1 function and, for almost every t, Since where Ω = √ 1 + ∆ 2 and the second equality follows from the identity H = 0. Abnormal extremals. Abnormal extremals are characterized by the equality p 0 = 0, from which, together with H = 0, we deduce that Φ(t) = −p(t)Fq(t)/u(t). If Φ(t) = 0 at time t, then p(t) is orthogonal both to Fq(t) and Gq(t). If, moreover, t were not an isolated zero of Φ, thenΦ(t) = 0, since Φ is C 1 . It would follow from (50) that p(t) is orthogonal also to [G, F]q(t). Since for every q ∈ M the vectors Fq, Gq, and [G, F]q span the tangent plane T q M to the sphere M , we would deduce that p(t) = 0, contradicting the PMP. This means that abnormal extremals are necessarily bang-bang. Moreover, we deduce from (51) that the switching times are the zeros of a nontrivial solution of the equation The length of an arc between any two successive switching times is then equal to π/Ω. Singular arcs. When the trajectory is normal, there might exist extremals for which Φ is zero on a nontrivial time interval. The control is singular on such an interval, since it cannot directly be obtained from the maximization condition. The restriction of the trajectory to an interval on which Φ ≡ 0 is called a singular arc. Singular arcs are characterized by the fact that the time derivatives of Φ at all orders are zero. Since p(t) is different from zero, the only possibility to have simultaneously Φ(t) = 0 andΦ(t) = 0 is that the vectors Gq(t) and [G, F]q(t) are parallel. Since Gq(t) and [G, F]q(t) generate, respectively, the rotations around the x and the y axis, we deduce that singular arcs are contained in the equator z = 0 of the sphere. The singular control law u s can be calculated from (51) by imposing that Φ and its second time derivative are zero, yielding u s (t) = 0. As it could be expected, this control law generates a rotation along the equator. It is admissible because |u s | ≤ 1. Normal bang-bang extremals. Consider a normal extremal and an interior bang arc of duration T between the switching times t 0 and t 0 + T on which the control u is constantly equal to +1 or −1. An interior arc is an arc which is neither at the beginning nor at the end of the extremal. We normalize p 0 to −1. According to (51), the function Ψ = Φ − u Ω 2 is a solution ofΨ + Ω 2 Ψ = 0. Moreover, since Φ is non-constant then Ψ is nontrivial. Hence, Ψ(t) = ν cos(Ωt+ θ 0 ) for some ν > 0 and θ 0 ∈ R. Moreover, ν > 0 is uniquely identified by u anḋ Φ(t 0 ) through the equalities Ψ(t 0 ) = − u Ω 2 andΨ(t 0 ) =Φ(t 0 ). Switchings occur if Φ = Ψ + u Ω 2 vanishes and changes sign. Since, moreover, sign[u] = sign[Φ], it follows that Ψ is larger than the negative value − 1 Ω 2 on (t 0 , t 0 +T ) when u = +1 and smaller than the positive value 1 Ω 2 on (t 0 , t 0 + T ) when u = −1. Hence T is larger than π/Ω andΦ (t 0 + T ) = −Φ(t 0 ).
IfΦ(t 0 ) = 0, we deduce that T = 2π/Ω. Notice that if u is constantly equal to +1 or −1, then the solutions ofq = (F + uG)q are 2π/Ω periodic rotations around the axis spanned by (u, 0, ∆). Since a time-optimal trajectory cannot self-intersect, we conclude that T < 2π/Ω andΦ(t 0 ) = 0. If t 0 + T is the starting time of another internal bang arc, then by the above considerations the duration of such internal bang arc is also equal to T . Given a normal bang-bang trajectory, there exists then T ∈ (π/Ω, 2π/Ω) such that the trajectory is the concatenation of bang arcs of duration T , except possibly for the first and last bang arc, whose length can be smaller than T . General extremals. As we have seen in the previous paragraphs, if a trajectory contains an internal bang arc, then it is bang-bang. Otherwise the set of zeros of Φ is connected, that is, either Φ has a single zero or it vanishes on a nontrivial singular arc and is different from zero out of it.
To summarize, extremal trajectories are of two types: • Bang-bang trajectories whose internal bang arcs have all the same length T ∈ [π/Ω, 2π/Ω) (the case T = π/Ω corresponding to abnormal extremals) and for which the first and last bang arcs have length at most T .
• Concatenations of a (possibly trivial) bang arc of length smaller that 2π/Ω, a singular arc on which u s = 0, and another (possibly trivial) bang arc of length smaller than 2π/Ω.
Steps 3 and 4. We solve in this paragraph two time-optimal control problems. Starting from the north pole (0, 0, 1), the goal is to reach in minimum time the points (0, 0, −1) (problem (P1)) and (1, 0, 0) (problem (P2)). To simplify the derivation of the optimal solutions, we assume that |∆| ≤ 1 [48]. Before solving (P1) and (P2), we first derive analytical results describing the dynamics of the system. Consider a bang extremal trajectory starting from the north pole at time t = 0 with control u(t) = ε = ±1. The corresponding trajectory is given by The first two times for which z(t) = 0 are t 1 = 1 Ω (π − arccos(∆ 2 )) and t 2 = 1 Ω (π + arccos(∆ 2 )). Notice that all other times for which z(t) = 0 are larger than 2π/Ω and cannot be the duration of a bang arc of an optimal trajectory. The optimal solution of (P1) is a bang-bang trajectory with a first switch on the equator at t = t 1 . The total duration of the process is t 1 + t 2 = 2π Ω . A symmetric configuration is possible with a first switch at t = t 2 . The two trajectories are displayed in Fig. 7. Let us discuss how the optimality of such a trajectory can be asserted. The proposed trajectory is clearly extremal and connects the chosen initial and final points. Since a bang-bang trajectory with at least two internal bang arcs has duration larger than 2π/Ω, it follows that any bang-bang trajectory with four or more bang arcs has a duration larger than the candidate optimal trajectory (that is, 2π Ω ). One is then left to compare 2π Ω with finitely many types of trajectories: bang-bang trajectories with two or three bang arcs, and trajectories obtained by concatenation of a bang, a singular, and a bang arc. By setting the initial and final points, this leaves few competitors to the optimal trajectory, which can be easily excluded by enumeration. As an example, we consider the concatenation of a bang arc with u = −1, a singular arc and a new bang arc with u = +1. The two bang arcs last for a time t 1 , while the duration of the singular arc is 2 ∆ arctan( ). It is then straightforward to deduce that the duration of this extremal solution is larger than t 1 + t 2 = 2π Ω , except in the case ∆ = 1 for which the singular arc is of length zero.
Let us now discuss the solution of (P2). Using the results of Sec. 9.2, we consider the concatenation of a bang extremal with u = +1 during the time t 1 and of a singular extremal with u = 0 during the time t s . At time t = t 1 , the trajectory reaches the point (∆, − √ 1 − ∆ 2 , 0). We deduce that t s = 1 ∆ arctan( ). The total duration of the control process is t 1 + t s . The corresponding trajectory is represented in Fig. 8. One can check that all other candidates for optimality join (1, 0, 0) in a longer time. The situation is more complicated than for (P1), since here t s → ∞ as ∆ → 0, so the candidate trajectory should be compared with trajectories with more and more bangs as ∆ → 0. A proof of the optimality of the trajectory described above can be obtained, for instance, using optimal synthesis theory, i.e., describing all the optimal trajectories starting from the north pole, as done in [48].

Applications of quantum optimal control theory
We discuss in this section the link between the material and results presented in this tutorial and the current research objectives in this field, both from a theoretical and an experimental points of view. The ability to quickly and accurately perform operations in a quantum device is a key task in quantum technologies. Quantum optimal control theory addresses this challenge by combining analytical and numerical tools to design procedures adapted to the experimental setup under study. This approach has the key advantage of being based on a rigorous theoretical framework on which all developments have been based since the 1980s. In the case of a low-dimensional quantum system as the two examples presented in Sec. 8 and 9, the optimal control problem may be solved analytically or at least with a very high numerical accuracy. For highdimensional systems, an efficient alternative is provided by numerical optimal control procedures, which include first and second-order gradient ascent algorithms [45,66,104,105] whose structure is based on the PMP. The connection between the PMP and gradient-based optimization algorithms is described in Sec. 7. Their flexibility allows them to adapt to many experimental situations for which precise modeling of the dynamics is known. As concrete and recent examples, we mention the experimental implementation of such techniques in Rydberg atoms [106,107], in spin-wave states of atomic ensembles [108], in electron spin resonance [109] or in superconducting cavity resonator [110]. The diversity of these examples shows the key role that optimal control now plays in quantum technologies. We describe below some examples of recent applications which are based on the mathematical formalism described in this tutorial. We also indicate some open issues in this field. We stress that our aim is not to provide an exhaustive list of all optimal control studies in this very active area (we refer the interested reader to recent reviews on this subject [2,5,6,4]), but rather to give an overview of the different aspects that can be analyzed. The analytical techniques have been applied recently to a series of fundamental and practical issues in quantum technologies such as robustness [111,112,113,114,115,116,117] and selectivity [118,119] of the control process with respect to parameters of the system Hamiltonian. The robustness property is a key factor for the experimental implementation of open-loop optimal pulses. The basic idea consists in controlling simultaneously an ensemble of identical systems which differ only by the value of the unknown parameter. Robust or selective optimal pulses are obtained respectively in the case where the target states are the same or are different according to the systems. In the same direction, numerical optimal control has been also employed to improve the estimation of Hamiltonian parameters [120,121]. Here, from a specific cost functional, the control is used to optimize the discrimination between the system dynamics, which leads to a precise characterization of the Hamiltonian [122]. Using the same type of approach, optimal control has proven to be very effective for quantum sensing [123,23,124].
A key aspect of quantum technologies is the minimum time to perform a given operation. Lower bounds on the time can be established in the framework of quantum speed limits [13,125] where the time is expressed as a ratio between the distance to the target state and the dynamical speed of evolution.
Time-optimal trajectories correspond by definition to the speed limit of the processes. The solutions of the two examples solved in this tutorial give therefore the quantum speed limits of the problems under study. However, in a higher dimensional quantum system, the many local minima of the control landscape make it very difficult to find a good approximation of the optimal trajectory and lead generally to an upper bound of the minimum time. Recent studies have explored the link between these two formalisms [54,126]. A major step forward would be to rigorously describe the concept of speed limit in terms of the Pontryagin Maximum Principle [127,128].
Quantum thermodynamics is nowadays a very useful tool to describe the dynamics of open quantum systems used in quantum technologies [129]. Optimal control and the PMP can be applied to analyze many issues in this field such as the enhancement of quantum engine performances [130,131], work extraction [132,133] or qubit purification [136,134,135]. The quantification of the ressources to control a quantum system has been also discussed and could be optimized from the PMP [137].
An intense effort is being made to develop new quantum optimal control algorithms, which are well suited to experimental limitations and constraints [138,139,140,141,142] or which are particularly effective in designing optimal pulses [67,143,144]. In particular, hybrid optimal control using feedback from the experiment has been developed to avoid problems due to inaccuracies in the modeling of the dynamics. The structure of the control landscape for the preparation of quantum many-body systems has been identified and exhibits a spin-glass-like phase transition in the space of control protocols close to the optimal fidelity [145]. This structure explains the difficulty to find numerically the optimal solution in this region. Combining optimal control and machine learning approaches as proposed in [146] could be a way to solve this problem and to improve the efficiency of optimal algorithms [147,148]. However, we stress that a rigorous mathematical description of these numerical methods and results will be necessary in the future to systematically apply these approaches to other quantum devices.
We conclude this section by pointing out that the PMP may also play a more unexpected role in quantum computing. It has recently been shown that Grover's quantum search problem can be mapped to a time-optimal control problem, and then described through the PMP [149]. A connection between optimal control theory and variational quantum algorithms [150] has been established in [151]. A general description of this intrinsic link is given in [152] where optimal control is used to precisely adjust the parameters of a quantum circuit. These results highlight that the PMP as a general optimization tool is not only interesting for the computation of time-dependent control pulses, but also in other optimization problems of interest in quantum technologies.

Conclusion
In this tutorial, we have attempted to give the reader a minimal background on the mathematical techniques of OCT. In our opinion, this is a fundamental prerequisite to rigorously and correctly apply these tools in quantum control.
The objectives of the tutorial are twofold. First, we have highlighted the key concepts of the PMP using ideas based on the finding of extrema of func-

A Test of controllability
We give in this section a general and useful sufficient condition for controllability. This section completes the discussion of Sec. 3.
Proposition 9. Let M be a smooth manifold and U be a subset of R m containing a neighborhood of the origin. Consider a control system of the forṁ q = F 0 (q) + m j=1 u j (t)F j (q), with F 0 , . . . , F m smooth vector fields on M and u(t) = (u 1 (t), . . . , u m (t)) ∈ U . Let L 0 be the Lie algebra generated by the vector fields F 0 , . . . , F m and L 1 be the Lie algebra generated by the vector fields F 1 , . . . , F m . The system is controllable if at least one of the following conditions is satisfied: 1. F 0 is a recurrent vector field and dim(L 0 (q)) is equal to the dimension of M at every q ∈ M ; 2. U = R m and dim(L 1 (q)) is equal to the dimension of M at every q ∈ M .
For a definition of Lie algebra generated by a set of vector fields and for the notion of recurrent vector field we refer to [28,89]. For our purposes, it is sufficient to recall that: -if F 0 (q) = A 0 q, . . . , F m (q) = A m q are linear vector fields then G ∈ L 0 if and only if G(q) = Bq with B in the matrix Lie algebra (for the commutator product) generated by A 0 , . . . , A m ; -if a vector field F is such that every solution ofq = F (q) is periodic, then F is recurrent. With these sufficient conditions, we recover the standard controllability conditions introduced in quantum control for closed systems [87,88] and recalled in Sec. 3.

B Filippov's theorem
We state here Filippov's theorem, which gives a sufficient condition for the compactness of the reachable set [26]. This section completes the discussion of Sec. 5.
Theorem 10 (Filippov). Fix T > 0. Consider the control systemq(t) = f (q(t), u(t)), q ∈ M , u ∈ U, where M is a n-dimensional manifold and U ⊂ R m . Fix an initial condition q in ∈ M . Assume the following conditions: • the set U is compact, • the set F(q) = f (q, u) | u ∈ U is convex for every q ∈ M , • for every u ∈ U, the solution ofq(t) = f (q(t), u(t)), q(0) = q in , is defined on the whole interval [0, T ].
Then the sets R T (q in ) and R ≤T (q in ) are compact.
Here R ≤T (q in ) denotes the reachable set from q in within time T , i.e., the set of pointsq in M such that there exist T ∈ [0, T ] and an admissible trajectory q : [0, T ] → M such that q(0) = q in and q(T ) =q.
Note that the third hypothesis of Th. 10 is automatically satisfied when M is compact.
By applying Filippov's theorem to the augmented system for problem (P1) introduced in Sec. 5, one obtains: Proposition 11. Fix T > 0. Assume that • T is closed and R T (q in ) ∩ T = ∅, • the set U is compact, • the setF(q) = f 0 (q, u) f (q, u) | u ∈ U is convex for every q ∈ M , • for every u ∈ U the solution ofq(t) = f (q(t), u(t)), q(0) = q in , is defined on the whole interval [0, T ].
Then there exists a solution to problem P1.
Example 5. The conclusion of Proposition 11 may fail to hold if we drop the convexity assumption on the set F(q) of admissible velocities, as illustrated by the following example. Take M = R 2 , U = {−1, 1}, and f (q, u) = A u q with A 1 = −1 1 −1 0 and A −1 = −1 −1 1 0 . We point out that the corresponding control system satisfies all the assumptions of Th. 10 except for the convexity of F(q), since the control can only take two discrete values −1 and 1. Pick q in = (1, 0) and any T > 0. Then q T := e −T q in is in the closure of R T (q in ), since we can end up arbitrarily close to q T at time T by applying a control that switches fast enough between −1 and 1. On the other hand, ifq(t) = f (q(t), u(t)) at time t and q(t) has a nonzero vertical component, then d dt q(t) > − q(t) . Since every trajectory of the control system starting at q in necessarily leaves the horizontal axis, its final point at time T has norm larger than e −T . Consequently q T ∈ R T (q in ). This proves that R T (q in ) is not closed, hence the conclusion of Th. 10 does not hold. This result is illustrated in Fig. 9 which shows numerically that the distance to the target state tends to zero when the number of switches goes to +∞. This example also highlights possible numerical problems when the existence of the optimal solution is not verified. Here, an optimization algorithm cannot converge towards a well-defined control law.
The idea of reducing the problem of existence of an optimal control to the compactness of the reachable set of the augmented system can be used for more general problems. For instance, if we add a terminal cost φ(q(T )) to the cost T 0 f 0 (q(t), u(t)) dt, where φ is a smooth function (as for instance in Approach B, or in the general formulation given in Sec. 6.2), we get a similar result adding to f 0 (q(t), u(t)) the directional derivative of φ along f (q(t), u(t)), that is, replacing f 0 (q(t), u(t)) by f 0 (q(t), u(t)) + dφ(q(t)), f (q(t), u(t)) . We recall that here dφ(q(t)) denotes the differential of φ evaluated at q(t) and that ·, · is the duality product between covectors of T * M and vectors of T M .
When the final time is free, it is more difficult to get the existence of optimal trajectories. However, the compactness of R ≤T (q in ) in the Filippov theorem can be used to find conditions for the existence of optimal controls in minimum time. We state this result in the case where M is compact. Note that the problem of minimizing time can be written in the form of problem P1 with T free and f 0 = 1. Proposition 12. Consider problem P1 with T free, f 0 = 1, and M compact. Assume that • T is closed and R(q in ) ∩ T = ∅, • the set U is compact, • the set F(q) = {f (q, u) | u ∈ U } is convex for every q ∈ M .
Then there exists a solution to the problem.
A straightforward application of Propositions 11 and 12 to closed quantum systems yields Proposition 3 of the main text.
[70] When not specified differently, we consider in this paper manifolds without boundary. Recall that the tangent space T q M at a point q of a smooth manifold M is the vector space of all the vectors tangent to M at q. The tangent bundle T M is the set of all the tangent spaces at any q ∈ M . At every point q ∈ M one can also attach the cotangent space T * q M , which is the dual space to T q M . An element p of T * q M is called a covector and is a scalar-valued linear function on T q M , whose value at a vector v ∈ T q M is denoted by p, v and referred to as the duality product between p and v. The collection of the cotangent spaces at all points of M forms the cotangent bundle T * M . If M has dimension n, p ∈ T * q M , and v ∈ T q M , in local coordinates, one can think of p as a row vector and v as a column vector, so that the duality product p, v is equal to pv. [72] The reason is that a minimizing sequence of piecewise constant controls may fail to have a cluster point in the class (intervals on which the control is constant may have zero duration at the limit). For this reason, optimal control problems are set for a larger class of controls in which optimal solutions can be shown to exist (under rather general conditions that we discuss in the main text) and can be characterized through suitable necessary conditions. Such conditions often allow to ensure that the optimal controls are indeed piecewise continuous. We should stress, however, that there exist some examples, such as the Fuller problem [73], for which the optimal control law has infinitely many switches between two values of U in finite time. Such a control law is not piecewise continuous, but belongs to L ∞ ([0, T ], U ).
[74] The unique solution q(·) of Eq. (1) is locally Lipschitz continuous. A locally Lipschitz continuous curve q(·) for which there exists an admissible control u(·) such that Eq. (1) is satisfied is said to be an admissible trajectory.
Recall that, given an interval I ⊂ R, a function q : I → M is said to be Lipschitz continuous if there exists C > 0 such that for every t 1 , t 2 ∈ I the distance in M between q(t 1 ) and q(t 2 ) is at most C|t 1 − t 2 |, while q(·) is said to be locally Lipschitz continuous if for every t ∈ I there exists ε > 0 such that the restriction of q to I ∩ (t − ε, t + ε) is Lipschitz continuous.