Examples of nonclassical feedback control problems

Transkript

1 Noliear Differ. Equ. Appl. 13, c 1 Spriger Basel AG 11-97/13/49-3 DOI 1.17/s Noliear Differetial Equatios ad Applicatios NoDEA Examples of oclassical feedback cotrol problems Alberto Bressa ad Delig Wei Abstract. We cosider a cotrol system with oclassical dyamics: ẋ = ft, x, u, D xu, where the right had side depeds also o the first order partial derivatives of the feedback cotrol fuctio. Give a probability distributio o the iitial data, we seek a feedback u = ut, x which miimizes the expected value of a cost fuctioal. Various relaxed formulatios of this problem are itroduced. I particular, three specific examples are studied, showig the equivalece or o-equivalece of these approximatios. Mathematics Subject Classificatio. 49N5, 49N Itroductio Cosider a cotrolled system whose state x IR evolves accordig to ẋ = ft, x, u, Du. 1.1 Here u = ut, x is a feedback cotrol, takig values i a set U IR m, while the upper dot deotes a derivative w.r.t. time. The depedece of the right had side of 1.1 o the Jacobia matrix Du = u i / x j makes the problem oclassical. Cotrol systems with this geeralized dyamics arise aturally i coectio with closed-loop Stackelberg solutios to differetial games [5, 6, 8]. Asremarkedi[8], it is useful to compare 1.1 with the relaxed system ẋ = ft, x, u, v, 1. where u IR m, v IR m are ow regarded as idepedet cotrols. Clearly, every solutio of 1.1 yields a solutio of 1., simply by choosig v = Du. O the other had, give a iitial data x = x, 1.3 Dedicated to Arrigo Cellia: teacher, metor ad fried.

2 5 A. Bressa ad D. Wei NoDEA let t x t be the solutio of the Cauchy problem correspodig to the ope-loop measurable cotrols ut, vt. If we choose u t, x =ut+vt x x t 1.4 for all x i a eighborhood of x t, the x satisfies also the origial equatio 1.1. Give a cost fuctioal such as J =. L t, xt, ut, xt dt, 1.5 for a fixed iitial coditio 1.3 the ifimum of the cost over all admissible cotrols is thus the same for trajectories of 1.1 or1.. The mai difficulty i the study of this miimizatio problem lies i the fact that the cotrol v ca be arbitrarily large ad comes at zero cost. Therefore, optimal trajectories may well have impulsive character. For optimizatio problems with impulsive cotrols we refer to [7,1,11]. Aim of the preset paper is to uderstad what happes i the case where the iitial data is ot assiged i advace, ad oe seeks a feedback u = ut, x that is optimal i coectio with a whole collectio of possible iitial data. Motivated by problems related to differetial games [5, 6, 8], we cosider a system whose state is described by a pair of scalar variables x, ξ IR IR. For simplicity, we also assume that the cotrol variable ut IR is oe-dimesioal. Let the system evolve i time accordig to the ODEs ẋ = ft, x, ξ, u, 1.6 ξ = gt, x, ξ, u, u x, where f,g are Lipschitz cotiuous fuctios of their argumets. We assume that the iitial data for the variable x is distributed accordig to a probability distributio μ o IR, while ξ is determied by a costrait of the form ξ = hx. 1.7 Here h : IR IR is some cotiuous fuctio. A feedback cotrol u = ut, x is sought, i order to miimize the cost fuctioal [ ] Ju, μ =. T E μ L t, xt, ξt, ut, xt dt. 1.8 Here E μ deotes the coditioal expectatio w.r.t. the probability distributio μ o the set of iitial data. I geeral, a optimal feedback may ot exist, withi the class of C fuctios. Ideed, it is quite possible that the optimal cotrol will be discotiuous w.r.t. the space variable x, or eve measure-valued. To bypass all difficulties stemmig from the possible lack of regularity, we cosider the family U of all C fuctios u :[,T] IR IR. For each feedback cotrol u Uthe equatio 1.6 uiquely determies a flow o IR. We deote by x ξ xt t =Ψ u ξt t

3 Vol. 13 Examples of oclassical feedback cotrol problems 51 the solutio of the Cauchy problem d xt f t, xt,ξt,ut, xt =, 1.9 dt ξt g t, xt,ξt,ut, xt,u x t, xt with iitial data x ξ x x = =. 1.1 ξ h x Here x IR is a radom variable, distributed accordig to the probability measure μ. Letμ t be the correspodig probability distributio at time t, defied as the push-forward of μ through the flow Ψ u t. This meas μ t Ψ u t A = μa for every Borel set A IR. The cost fuctioal i 1.8 ca be equivaletly writte as Ju, μ = E μt L t, x, ξ, ut, x dt, 1.11 where E μt deotes expectatio w.r.t. the probability distributio μ t. We the cosider Problem 1. Determie Jμ =if. Ju, μ. 1.1 u U Describe a sequece of feedback cotrols u 1 achievig the ifimum i 1.1. As it will be show by some examples, the ifimum i 1.1 may ot be stable w.r.t. perturbatios of the probability distributio μ. A related questio is to determie the value where { d μ, μ =sup J w μ. = lim if d μ,μ ϕd μ if u U ϕdμ ; Ju, μ, 1.13 } ϕ C1, ϕ 1 is the Katorovich Wasserstei distace betwee two probability measures. Oe ca thik of J w as the lower semicotiuous regularizatio of J w.r.t. the topology of weak covergece of measures. I the case where μ is absolutely cotiuous with desity φ w.r.t. Lebesgue measure, it is also meaigful to cosider J s μ. = lim if φ φ L 1 if u U Ju, μ, 1.14 where μ is a probability measure havig desity φ. I other words, J s is the lower semicotiuous regularizatio of J w.r.t. a strog topology, correspodig to L 1 covergece of the desities.

4 5 A. Bressa ad D. Wei NoDEA As it will be show by specific examples, the three values i may well be differet. I additio, by replacig u x with a idepedet cotrol fuctio v, from1.9 oe obtais the relaxed system d dt xt ξt = f t, xt,ξt,ut, xt g t, xt,ξt,ut, xt,vt, xt We shall deote by J μ, u, v be the correspodig cost 1.8, with dyamics give at Remark 1. I geeral, the optimal cotrol u = ut, x which miimizes the expected cost 1.8 subject to the dyamics 1.9 will strogly deped o the probability distributio μ o the iitial data. O the other had, sice the dyamics 1.15 does ot ivolve derivatives of the cotrol fuctios u, v, the optimal value ca be achieved poitwise for each iitial data x,ξ. I this case, the same pair of feedback cotrols u,v ca be optimal for every probability distributio μ o the iitial data. We ow itroduce the set V of all C fuctios v :[,T] IR IR, ad cosider Problem. Determie the optimal value for the relaxed problem J relax μ. = if u,v U V J u, v, μ Describe a sequece of feedback cotrols u,v 1 achievig the ifimum i From the defiitios, it is immediately clear that J relax μ Jμ, J w μ J s μ Jμ I this paper we aalyze three specific examples. showig the differeces betwee the origial ad relaxed formulatios, ad the possible obstructios ecoutered i the approximatio of solutios 1.15 with solutios of 1.9. Motivated by these examples, geeral results o the equivalece betwee the various values i 1.17 will be proved i the forthcomig paper []. The uderlyig motivatio for the preset aalysis comes from the theory of Stackelberg solutios i closed-loop form, for differetial games. I oe space dimesio, this leads to a problem of the form 1.6, where x is the state of the system, u = ut, x is the feedback cotrol implemeted by the leader, ad ξ is the adjoit variable i the optimality equatio determiig the strategy of the follower. For a differetial game, it is atural to put a probability distributio o the state x at the iitial time t =, ad a costrait of the type ξt =hxt o the adjoit variable at the termial time T. The preset research, dealig with the Cauchy problem where all data are give at time t =, is iteded to be a itermediate step toward the uderstadig of this boudary value problem, more relevat for game-theoretic applicatios.

5 Vol. 13 Examples of oclassical feedback cotrol problems 53. A case of shrikig fuels Example 1. Cosider the optimizatio problem [ ] T miimize: Ju =E μ [x t+ξ t+u t] dt..1 for the system with dyamics { ẋ = u,. ξ = ξu x. Here u = ut, x ca rage over the etire real lie IR. As iitial coditio, assume that ξ 1 while x is uiformly distributed o the iterval [, 1]. Of course, this meas that μ is the measure with desity φ = χ [,1] the characteristic fuctio of the uit iterval w.r.t. Lebesgue measure. I this case, the correspodig relaxed problem, with u x replaced by a idepedet cotrol fuctio v, is decoupled. Ideed, it yields two idepedet problems: miimize: J 1 u = miimize: J v = [x t+u t] dt, with dyamics ẋ = u,.3 ξ t dt, with dyamics ξ = ξv..4 The first is a stadard liear-quadratic optimal cotrol problem. The optimal feedback is liear w.r.t. x, amely Fig. 1, left u t, x = et T e T t e t T x..5 + et t The secod problem is solved by a ubouded impulsive cotrol v that istatly steers the compoet ξ to the origi. Returig to the origial problem.1., call φt, the desity of the probability distributio μ t. This fuctio satisfies φ t + uφ x = φu x. Callig ξt, x the value of ξt alog a characteristic, i.e. at the poit t, xt, the secod equatio i. yields Together, these two equatios yield ξ t + uξ x = ξu x. φξ t + uφξ x =..6 I the followig, for y [, 1] we shall deote by t xt, y the particular solutio of the Cauchy problem ẋ = ut, x, x = y..7

6 54 A. Bressa ad D. Wei NoDEA 1 x x x T t t t Figure 1. Left the optimal trajectories for the stadard liear-quadratic optimizatio problem with dyamics.9 ad cost.3 idepedet of ξ. Ceter the presece of a cost depedig o ξ reders more profitable a cotrol where u x is large ad egative. Hece the optimal solutio, obtaied by solvig.16, should be supported o a smaller iterval. Right if we allow small gaps i the support of the probability distributio μ o the iitial data, the the miimum cost becomes arbitrarily close to the miimum cost for relaxed problem.3.4 Expressig the feedback cotrol i terms of this ew variable: ut, y =. ut, xt, y, we obtai u x t, xt, y = u y t, y φt, y = u yt, y ξt, y. The problem ca thus be reformulated as 1 [ ] miimize: x t, y+ξ t, y+u t, y dy dt.8 subject to { ẋ = u, ξ = u y, { x,y=y, ξ,y=1..9 Sice the evolutio equatio does ot deped explicitly o x, ξ, the adjoit equatios are { { λ1 = L/ x = x, λ1 T,y=,.1 λ = L/ ξ = ξ, λ T,y=. Hece λ 1 t, y = t xτ,y dτ, λ t, y = The maximality coditio yields Fig. 1, ceter ut, = argmi ω 1 t ξτ,y dτ..11 [ ] λ 1 t, y ωy+λ t, yω y y+ω y dy..1

7 Vol. 13 Examples of oclassical feedback cotrol problems 55 Assume that, for a fixed time t, the fuctio u = ut, y provides the miimum i.1. The, for every smooth fuctio ϕ :[, 1] IR, settig u ε y =ut, y+εϕy oe should have = d dε = = 1 1 [ ] λ 1 t, y u ε y+λ t, yu ε y y+u ε y dy [ ] ε= λ 1 t, y ϕy+λ t, yϕ y y+ut, yϕy dy [ ] λ 1 t, y λ,y t, y+ut, y ϕy dy + λ t, 1ϕ1 λ t, ϕ. 1 Sice the fuctio ϕ ca be arbitrary, this yields the Euler Lagrage equatios ut, y = λ 1t, y+λ,y t, y..13 together with the boudary coditios λ t, = λ t, 1 =..14 Differetiatig.13 w.r.t. t ad usig.1, we obtai u t t, y = λ,tyt, y λ 1,t t, y = x ξ y..15 Usig the idetities x tt t, y =u t t, y, x y t, y = 1 = ξt, y, φt, y we evetually obtai the PDE x tt + x yy x =..16 This is a liear elliptic equatio, to be solved o the rectagle [,T] [, 1]. From.9 ad the termial coditios i.1, usig.13 oe obtais the boudary coditios x,y=y, x t T,y=uT,y=..17 Moreover, x y t, = ξt, = 1 λ,tt, =, x y t, 1 = ξt, 1 = 1 λ,tt, 1 =,.18 because of.11 ad.14. By stadard PDE theory, the liear elliptic boudary-value problem.16,.17,.18 has a uique solutio. Particular solutios of.16 satisfyig.18 ca be obtaied by separatio of variables. For every iteger k ad coefficiets A k,b k, oe has the solutio xt, y =X k tz k y where X k t =A k e 1+k π t + B k e 1+k π t, Z k y = cos kπy,.19

8 56 A. Bressa ad D. Wei NoDEA Imposig the additioal boudary coditios.17 oe obtais a represetatio of the solutio as a Fourier series: xt, y = et T + e t e T +1 + e 1+k π t T + e 1+k π t e 1+k π T +1 k=1 1k 1 k π cos kπy, e 1+k π t T + e 1+k π t ξt, y =x y t, x = e 1+k π T +1 k=1 1 1k kπ si kπy.. Estimates o this solutio ca be obtaied by compariso with upper ad lower solutios [9]. Differetiatig.16 w.r.t. y oe obtais a boudary value problem for x y, amely x y tt +x y yy x y = t, y [,T] [, 1],.1 with the boudary coditios { xy,y=1, x y t T,y=, { xy t, =, x y t, 1 =, A stadard compariso argumet here yields the lower boud x y t, y for all t, y.. Oe ca also cosider the above problem o the half lie, for t [, + [. I this case,. reduces to xt, y = e t + e 1+k π t 1 k 1 k π cos kπy..3 k=1 As t, this solutio approaches zero. Ideed, xt, L [,1], ξt, L [,1], ut, L [,1]..4 We ow study what happes if we allow arbitrarily small gaps i the support of the iitial probability distributio μ. For ay, let μ be the probability distributio with desity 1 φ x = if x [a i,b i ] =. [ i 1, i 1 for some i =1,...,, otherwise. ],.5 Clearly, lim φ φ L 1 =. We claim that, as, the costs of the correspodig optimal solutios approach the miimum cost for.3.

9 Vol. 13 Examples of oclassical feedback cotrol problems 57 To prove this, cosider ay of the above itervals [a i,b i ] [, 1]. Let x i t, y be the solutio of the liear elliptic boudary value problem x tt + x yy x = t, y [,T] [a i,b i ],.6 with boudary coditios { x,y= y, x t T,y=, { xy t, a i =, x y t, b i =..7 The solutio to this boudary value problem ca agai be expressed as a Fourier series: e t T + e t bi + a i e λ k t T + e λ k t x i t, y = e T + +1 e λ k T +1 k=1 1 k 1 b i a i k π cos kπy a i,.8 b i a i where λ k = 1+ k π b i a i 1. I coectio with the itervals [a i,b i ] defied at.5, for i =1,..., cosider the fuels Fig. 1, right. Γ i = {t, x; t [,T], x = xi t, y for some y [a i,b i ]}..9 We claim that these fuels are pairwise disjoit. Ideed, cosider the fuctio zt, y = et T + e t e T +1 y.3 where t zt, y is determied as the uique solutio to the two-poit boudary value problem zt zt = fort [,T], z = y, żt =..31 The z y provides a solutio to the elliptic boudary value problem w tt + w yy w = t, y [,T] [a i,b i ],.3 with boudary coditios { w,y= 1, w t T,y=, wt, a i =wt, b i = et T + e t e T O the other had, the partial derivative x i,y provides a solutio to the same equatio.3, but with boudary coditios { w,y= 1, w t T,y=, wt, a i =t, b i =..34 By compariso, we obtai x y t, y z y t, y for all t, y [,T] [a i,b i ]..35

10 58 A. Bressa ad D. Wei NoDEA Whe y =a i + b i /, from.8 it follows x i t, y =zt, y for all t [,T]. Sice the estimates.35 hold for every i =1,...,, we coclude x i 1 t, b i 1 zt, b i 1 < zt, a i x i t, a i, provig that the fuels Γ 1,...,Γ remai disjoit. We ca ow defie a feedback cotrol u by settig u t, x =x i,t t, y if t, x Γ i for some i {1...,},.36 ad extedig u i a smooth way to the etire domai [,T] IR. Propositio 1. The above costructio yields lim Jμ,u =J relax μ..37 Therefore, for this example oe has J s μ =J relax μ. Proof. Writig the Euler Lagrage equatios for.3 ad observig that the ifimum cost for.4 is zero, we compute 1 J relax μ = z t, y+żt t, y dt dy,.38 where z is the fuctio i.3. O the other had, Jμ,u = bi x 1 i t, y+x i,tt, y+x i,yt, y dtdy,.39 i=1 a i where, for y [a i,b i ], the quatity x i t, y is give by.8. We observe that each fuctio x i provides the global miimizer to the variatioal problem miimize: J i w =. bi w t, y+wt t, y+wyt, y dt dy.4 a i amog all fuctios w W 1, [,T] [a i,b i ] such that w,y=y for all y [a i,b i ]..41 Cosider the fuctios w i defied as follows. For y [a i,b i ], let 1 w i t, y =. t z, a i + b i +1 t y, if t [, 1 ], z t, a i + b.4 i, if t [ 1,T]. For every 1adi {1,...,}, it is easy to check that these fuctios satisfy the uiform bouds w i t, y [, 1], w i,y t, y [, 1], w i,t t, y M,.43 for some uiform costat M. Hece the followig estimates hold: bi a i bi 1/ a i w i,yt, y dt dy w i t, y+w i,tt, y bi 1/ a i dt dy = b i a i,.44 dt dy 1 + M b i a i..45

11 Vol. 13 Examples of oclassical feedback cotrol problems 59 Usig the above iequalities ad recallig that i=1 b i a i = 1/, sice each x i :[,T] [a i,b i ] IR provides a global miimizer, we obtai Jμ,u 1 i=1 bi a i w i t, y+w i,tt, y+w i,yt, y dt dy + M b i a i 1 i=1 + b i a i z t, a i + b i + zt t, a i + b i dt 1 i=1 1/ + M + a i+1 a i z t, a i + b i i=1 + zt t, a i + b i dt =. A + B..46 Lettig, we clearly have A. O the other had, B is a approximate Riema sum for the itegral.38. Hece lim B = J relax μ. From.46 it follows lim if Jμ,u J relax μ, The coverse iequality is clear. Remark. I this example, the presece of gaps i the probability distributios μ is essetial. Ideed, if we used the feedback cotrols u i coectio with the origial probability μ, uiformly distributed o [, 1], the cost Jμ, u would be very large. This is because, for iitial data b i <x <a i+1, alog the trajectory t xt oe ca have u x t, xt >> 1. This forces t ξt = exp u x s, xs ds to be very large, producig a large cost i.1. Although the probability of the iitial data fallig outside the itervals 1 i [a i,b i ] is very small, these few iitial data determie a big icrease i the expected cost i.1. This example illustrates a case where J relax μ =J s μ, but J s μ Jμ. The problems.3 ad.8.9 both have regular solutios, but the miimizatio problem 1.14 does ot. A miimizig sequece u should have the form u t, x =u t, x+ũ t, x where u is the optimal liear feedback i.5, while ũ C, ũ C 1 = O1. Remark 3. This first example suggests a geeral strategy for provig the equivalece J relax μ =J s μ. Namely:

12 6 A. Bressa ad D. Wei NoDEA i Let u ε,v ε be C feedbacks which achieve a almost optimal cost, i coectio with the relaxed problem 1.11, I other words, assume J μ; u ε,v ε J relax + ε. ii Split the support of the iitial distributio μ ito several small, disjoit itervals [a i,b i ], separated by small gaps. For each i, lett x i t bethe solutio of 1.15 with iitial data x i = a i + b i /. iii Defie the liear feedback cotrol u i εt, x =. u ε t, x i t + v ε t, x i t x x i t, ad let F i be the set of all solutios to the ODE ẋ = f t, x i t, ξt, u i εt, x, ξ = g t, xt, ξt, u i εt, x, v ε t, x i t, with iitial data x [a i,b i ]. iv If the fuels Γ i. = { t, xt; t [,T], x F i }, i =1,...,, do ot overlap, the oe ca defie a ew feedback by settig ũt, x =. u i εt, x if t, x Γ i,.47 ad extedig ũ i a smooth way o IR \ i Γ i. By choosig the itervals [a i,b i ] sufficietly small, the cost provided by this feedback cotrol u ca be redered arbitrarily close to Jμ, u ε,v ε. Here the fact that the fuels Γ i remai disjoit is essetial. I the ext sectio we look at a case where this property fails, ad the two ifimum costs J relax ad J s do ot coicide. 3. A case of expadig fuels Example. We ow cosider the problem of miimizig the same quadratic fuctioal as i.1, but subject to the dyamics { ẋ = u, 3.1 ξ = ξu x. Because of the egative sig i the secod equatio, we ow have φt, x = ξt, x for all t, x. Usig agai the variable y to label characteristics, cosider the problem.8, for a system with dyamics { xt = u, ξ t = ξ u y, { x,y=y, ξ,y=1. The evolutio of the dual variables is determied by λ 1,t = x, λ 1 T,y=, λ,t =ξλ u y ξ, λ T,y=

13 Vol. 13 Examples of oclassical feedback cotrol problems 61 Therefore λ 1 t, y = xτ,y dτ, λ t, y = exp t t τ ξs, y u y s, yds ξτ,y dy. 3.4 t The maximality coditio takes the form 1 [ ] ut, = argmi λ 1 t, y ωy λ t, yξ t, yω y y+ω y dy. 3.5 ω Assume that, for a fixed time t, the fuctio u = ut, y provides the miimum i.1. The, for every smooth fuctio ϕ :[, 1] IR, settig u ε y =ut, y+εϕy oe should have = d 1 [ ] λ 1 t, y u ε y λ t, yξ t, yu ε y y+u ε y dy dε ε= 1 [ ] = λ 1 t, y ϕy λ t, yξ t, yϕ y y+ut, yϕy dy 1 [ ] = λ 1 t, y+λ ξ y t, y+ut, y ϕy dy + λ t, 1ξ t, 1ϕ1 λ t, ξ t, ϕ. Sice the fuctio ϕ ca be arbitrary, this yields the Euler Lagrage equatios ut, y = λ 1t, y+λ ξ y t, y. 3.6 together with the boudary coditios λ ξ t, = λ ξ t, 1 =. 3.7 Observe that 3. ad 3.3 yield λ ξ t =ξλ u y ξξ λ ξ u y = ξ Differetiatig both sides of 3.6 w.r.t. t ad usig 3.8 oe obtais u t t, y = λ 1,tt, y+λ ξ t,y t, y = x +ξ 3 y. 3.9 Usig the idetities x tt t, y =u t t, y, x y t, y = 1 φt, y = 1 ξt, y, 3.1 we thus recover the PDE x tt + 3x yy x = x y 4 This is a oliear elliptic equatio, to be solved o the rectagle [,T] [, 1]. From 3. ad the termial coditios i 3.3, usig 3.6 oe obtais the boudary coditios x,y=y, x t T,y=uT,y=. 3.1

14 6 A. Bressa ad D. Wei NoDEA x 1 x 1 x 1 T t T T Figure. Left the optimal trajectories for the stadard liear-quadratic optimizatio problem with dyamics.9 ad cost.3 idepedet of ξ. Ceter for the system 3.1, the presece of a cost depedig o ξ reders more profitable a cotrol where u x is positive. Hece the optimal solutio should be supported o a larger iterval. Right allowig gaps i the support of μ does ot provide a way to achieve a lower cost, because i this case the fuels Γ i determied by earoptimal feedback cotrols would overlap Moreover, x y t, = 1 ξt, =+, x y t, 1 = 1 ξt, y =+, 3.13 because = λξ t t, = ξ 3 t,, = λξ t t, 1 = ξ 3 t, 1 Fig.. I cotrast with the optimal solutio i Example 1, lettig T the optimal trajectories do ot coverge to zero. Rather tha.4, we expect that the solutio xt, y,ξt, y will approach a steady state xy, ξy. Because of the idetity ξ x y 1, the fuctio x should provide a global miimizer to the variatioal problem [ 1 ] 1 miimize: x y+ x dy, 3.14 y the miimum beig sought amog all absolutely cotiuous, o-decreasig fuctios x :[, 1] IR, with free boudary coditios. The Euler Lagrage equatios for this problem yield x 3x x =, lim y + x y = lim y 1 x y = A solutio to the above equatios is foud to be implicitly determied by xy 3 3 πxy π x y+ π arcsi = 3 y

15 Vol. 13 Examples of oclassical feedback cotrol problems 63 Observe that this fuctio satisfies xy = x1 y fory [, 1]. As t +, the measures μ t approach a probability distributio μ which is symmetric w.r.t. the origi. Notice that, as T +, i Example 1 the miimum cost remais bouded, while i Example it approaches +. I this case, we could agai cosider the fuels Γ i, defied as i.9, where ow x = xt, y is the solutio of 3.11 o[,t] [a i,b i ], with boudary coditios { x,y=y, x t T,y=, { xy a i,t=+, x y b i,t= However, these fuels ow overlap with each other Fig., ad the defiitio.36 is ot meaigful. For this example, we thus expect Jμ =J s μ, but J s μ J relax μ. 4. A case where the width of the fuels ca be cotrolled Example 3. We ow cosider a case where f depeds also o ξ, ad oe ca use this additioal variable i order to cotrol the width of the fuels Γ i, prevetig their overlap. Cosider agai the optimizatio problem.1, but assume that the state of the system evolves accordig to { ẋ = u + ξ, 4.1 ξ = u x, with iitial data x = y, ξ = hy. 4. As before, we assume that y is a radom variable uiformly distributed o the iterval [, 1]. Otherwise stated, the probability measure μ has desity φ = χ [,1] w.r.t. Lebesgue measure. The correspodig relaxed system is { ẋ = u + ξ, 4.3 ξ = v. By.1 ad 4.3, to achieve a global miimum oe should have u = ξ = x t. 4.4 For each fixed y, writig the Euler Lagrage equatios we fid that the optimal solutio t xt, y to the relaxed problem solves the two-poit boudary value problem ẍ x = fort [,T], x,y=y, ẋt,y=. 4.5 The optimal solutio is thus foud to be ˆxt, y = e t T + e t y e T

16 64 A. Bressa ad D. Wei NoDEA For this relaxed solutio, the correspodig variables u, ξ are give by ût, y =ˆξt, y = ˆx tt, y = e t T e t 1+e T y. 4.7 O the other had, for a fixed y, the map ˆv,y should formally be give by the distributioal derivative of the map t ˆξt, y. This is a measure cotaiig a poit mass of size ˆξ+,y hy = e T 1 e T +1 y hy at the origi, while its restrictio to the ope set {t >} is absolutely cotiuous w.r.t. Lebesgue measure, with desity ˆvt, y = 1 ˆx ttt, y =ˆxt, y = e t T + e t y e T From the above aalysis, we coclude that the ifimum amog all costs Jμ, u, v, with u, v C, is provided by 1 J relax μ = ˆx t, y+ 1 ˆx t t, y dt dy. 4.9 A miimizig sequece of feedback cotrols u ν,v ν ν 1 is provided by ν v ν t, x = x u ν t, x =ut, x = e t T e t e t T + e t e T 1 e T +1 x hx x, if t [, ν 1 ], if t [ν 1,T]. By performig a suitable cut-off, followed by a mollificatio, we achieve u ν,v ν C. This prelimiary aalysis shows that, for ay ε>, there exists smooth feedback cotrols u = u t, x adv = v t, x such that J μ, u,v J relax μ+ε, 4.1 ad, callig x = x t, y, ξ = ξ t, y the correspodig solutios of , oe has x C [,T ] [,1] M, ξ C [,T ] [,1] M, u C [,T ] IR M 1, v 4.11 C [,T ] IR M 1, x yt, y ρ >, hy C [,1] M, 4.1 for some costats M,M 1,M,ρ, possibly depedig o ε.

17 Vol. 13 Examples of oclassical feedback cotrol problems 65 Propositio. I the above example oe has J s μ =J relax μ. Proof. Give ε>, let u,v be a pair of geeralized feedback cotrols for which all the estimates hold. To prove Propositio, we eed to show that there exists a measure μ with desity φ satisfyig φ φ L 1 ε ad a feedback cotrol ũ C such that Jũ, μ Ju,v,μ+ε Cosider the augmeted system of ODEs ẋ = u + ξ, x,y=y, ξ = v, ξ,y=hy, η = ηv + z, η,y=1, ż = wη, z,y=h y Here we thik of η = x y ad z = ξ y as a additioal variable, while v = u x, w = v x = u xx are additioal cotrols. Notice that the last two ODEs i 4.14 follow from x y t = u y + ξ y = u x x y + ξ y, ξ y t = v y = v x x y.. For large, cosider the probability distributio μ havig desity [ i 1 if x. 1, i 1 ].=[ai φ x =,b i ], 4.15 for some i {1,...,}, otherwise. For i =1,...,, deote by t x i t. = x t, a i, t ξ i t. = ξ t, a i the compoets of the solutio of with y = a i. As a first attempt, oe may costruct the feedback ũ by settig ũt, x. = u t, x i t + v t, x i t x x i t 4.16 for x x i t. For y [a i,b i ], call t xt, y, ξt, y the solutio of , with u = ũ give by Observe that this costructio yields xt, a i =x i t t [,T], 1 i. Itroducig the tubes. = x t, a i, ξt, ai =ξ i t. = ξ t, a i, Γ i. = {t, xt, y; t [,T], y [ai,b i ]}, 4.17 oe may hope to defie ũt, x by4.16 fort, x Γ i, ad exted ũ i a smooth way o the complemetary set [,T] IR \ i Γ i. Notice that 4.1 implies x 1 t <x t < <x t, so that the ceters of these tubes do ot cross each other. Ufortuately, i the preset situatio there is o guaratee that the tubes Γ i remai disjoit for all t [,T]. We thus eed to refie our costructio, relyig o a global cotrollability property of the system 4.14.

18 66 A. Bressa ad D. Wei NoDEA Γj b i y a i x it Γi δ T t Figure 3. O the iitial time iterval [,δ] a feedback cotrol is implemeted such that all iitial poits y [a i,b i ] are steered iside a very small eighborhood of the poit x i δ =xδ, a i. Sice at time t = δ we have x y ad ξ y iside each tube Γ i, this guaratees that for t [δ, T ] the tubes Γ i remai mutually disjoit O a small time iterval [, δ], we will costruct a feedback ũ such that the correspodig solutio of satisfies xδ, y x i δ < ɛ, ξδ, y ξ i δ < ɛ, for all y [a i,b i ] If 4.18 holds, with ɛ<< 1 suitably small, the for t [δ, T ], ad x x i t the defiitio 4.16 will provide a feedback with the desired properties. To achieve 4.18 we shall costruct a feedback such that x y δ, y ad ξ y δ, y, for all y [a i,b i ] Fig Let δ>begive. Relyig o the cotrollability of the ODE 4.14, for t δ ad i {1,...,}, we costruct cotrol fuctios u i,v i,w i such that the solutio of the Cauchy problem ẋ = u i + ξ, x = a i, ξ = v i, ξ = ha i, η = ηv i + z, η = 1, ż = ηw i, z = h a i. satisfies xt =x i t =. x t, a i for all t [, δ] ad moreover ξδ =ξ i δ =. ξ δ, a i, ηδ, zδ. Solutios of 4.19 are more coveietly foud usig the variables 4.19 X =lη, Y = z η, 4.

19 Vol. 13 Examples of oclassical feedback cotrol problems 67 which evolve accordig to Ẋ = η η = v i + Y, Ẏ = ż η z η η η = w i Y. Yv i = wi, X =, Y = h a i. 4.1 We regard w i as a idepedet cotrol fuctio. Clearly, we ca assig the cotrols v i, w i arbitrarily, the compute the solutio of 4.1 ad defie the cotrol w i t = w. i t+y t+ytv i t. To achieve 4.18, we use the cotrols v i t =. 1+ξ i δ ξ i δ 1+ξ i δ ξ i δ t< δ, δ t δ, 4. w i t =. 1, t δ, 4.3 δ3 while the cotrol u i is defied as u i t =. [ u t, x i t + ξ i t ξ i + t ] v i s ds t [, δ]. 4.4 The correspodig solutio of 4.1 is t Xt = v i s ds + Y t =h a i t δ 3. I particular, at t = δ oe has t h a i s δ 3 ds, 4.5 Xδ = ξ i δ ξ i + h a i δ 1 δ, Yδ = h a i 1 δ. 4.6 Goig back to origial variables η, z, oe obtais ηδ =expxδ = exp ξ i δ ξ i + h a i δ 1, δ zδ =Y δηδ = h a i 1δ exp ξ i δ ξ i + h a i δ 1. δ 4.7

20 68 A. Bressa ad D. Wei NoDEA Moreover, by the defiitio of w i 4.1, the cotrol w i = w i + Y + Yv i is give by 1 δ 3 + h a i t 1+ξi δ ξ i + δ 3 h a i t δ δ 3, t< δ, w i t = 1 δ 3 + h a i t 1+ξi δ ξ i + δ 3 h a i t 4.8 δ δ 3, δ t δ. By 4., 4.3, ad 4.5, for δ> sufficietly small we have the estimates v i t δ, Y t δ, w it 5 δ 4, for all t [,δ]. 4.9 Moreover, by 4.4, the solutio x, ξ of 4.19 satisfies ẋt = u i t+ξt =u t, x i t + ξ i t =ẋ i t, for all t [,δ]. 4. O a suitable eighborhood of each trajectory t x i t, we the defie the feedback cotrol ũ as ũt, x. = 4.3 u i t+x x i t v i t+ x x it w i t, t [,δ], u t, x i t + x x i t v t, x i t, t [δ, T ]. The correspodig solutio of 4.1 with iitial data 4. will be deoted by t xt, y, ξt, y. We ca the exted ũ i a smooth way w.r.t. the x- variable o the complemet of the set 1 i Γ i. Notice that, by choosig = δ >> δ 1, we ca achieve the covergece ũ u L [δ,t ] IR as δ For every i {1,...,}, the above costructio yields xt, a i =x i t t [,T]. ξt, ai t = ξ i + v i s ds M 1 + δ+1 t [,δ], ξδ, a i =ξ i δ. 4.3 We claim that, by choosig δ> sufficietly small oe ca achieve < x y δ, a i < δ, ξy δ, a i < δ, 4.33 < x y t, a i < x yt, a i for all t [,δ] To prove 4.33 we observe that the fuctios ηt = x y t, a i adzt = ξ y t, a i satisfy the system of ODEs Hece the bouds 4.33 area immediate cosequece of 4.7, because e 1/δ << δ for δ> small.

21 Vol. 13 Examples of oclassical feedback cotrol problems 69 Next, by 4.11 wehave x yt C x C M. Sice x y,a i = 1, this implies x yt, a i 1 M t. For t [, δ/] by 4.5 wehave Xt = 1+ξ iδ ξ i δ t + h a i t t δ 3 M t l x yt, a i, 4.35 provided that δ>issmall eough. O the other had, for y [δ/, δ], we obtai δ/ t t Xt = v i s ds + v i s ds + h a i s δ/ δ 3 1+ ξ i δ ξ i + M δ 1 8δ M t Therefore, always assumig that δ>is sufficietly small, for all t [,δ]we have x y t, a i = exp Xt x yt, a i. A etirely similar estimate ca be proved for every iitial poit y [a i,b i ]. Provided that δ> is sufficietly small, we thus coclude < x y δ, y < δ, ξy δ, y < δ, for all y [a i,b i ], 4.37 x y t, y x yt, y for all t [,δ], y [a i,b i ] The estimate 4.38 shows that o the iitial time iterval [,δ]the tubes Γ i defied as i 4.17 do ot overlap with each other. Next, we check that the tubes Γ i remai disjoit also for t [δ, T ]. The first iequality i 4.1 implies x i+1 t x i t ρ for all i {,..., 1} Observe that, by 4.1 ad 4.3, x y t, y + ξ y t, y v t, x i t + 1 x y t, y + ξ y t, y. t Therefore the boud 4.11 ov together with 4.33 yields x y t, y e M1+1T x y δ, y + ξ y δ, y for all t [δ, T ]. 4.4 For y [a i,b i ]adt [δ, T ]wehave xt, y x i t provided that y a i x y t, z d z b i a i sup x y t, y ρ y [,1], 4.41 e M1+1T sup x y δ, y + ξδ, y y ρ.

22 7 A. Bressa ad D. Wei NoDEA Recallig 4.33 we ca ow choose δ>small eough so that e M1+1T 4δ < ρ. The we choose = δ >> 1/δ large eough so that, by cotiuity, the estimates x y δ, y + ξδ, y < 4δ < ρ e M1+1T remai valid for every y [a i,b i ], i =1,...,.By4.39 ad 4.41, this implies that the tubes Γ i remai mutually disjoit also for t [δ, T ]. 6. It is clear that the sequece of desities φ i 4.15 coverges to φ = χ [,1] as. Havig chose δ>ad = δ >> δ 1 as before, let ũ =ũt, x be a feedback cotrol satisfyig 4.3 oeachtubeγ i, exteded i a smooth way outside the uio i=1 Γ i. It remais to show that, as δ, the expected cost for the feedback ũ approaches J u,v,μ. Ideed, o the iitial iterval [,δ], by 4.3 all fuctios x, ξ,ũ remai uiformly bouded as δ. Therefore 1 i=1 δ bi a i x t, y+ ξ t, y+ũ t, xt, y dy dt Cδ. 4.4 To see what happes o the remaiig iterval [δ, T ], cosider the quatities. bi A = 1 x t, y+ ξ t, y+ũ t, xt, y dy dt i=1 δ a i. 1 B = x t, a i +ξ t, a i +u t, x i t dy dt. i=1 δ Recallig 4.31wehave A B asδ ad = δ.moreover, lim B = 1 x t, y+ξ t, y+u t, x t, y dy dt = J u,v,μ. This completes the proof. Remark 4. We poit out a fudametal differece betwee the first two examples ad this last oe. Namely, cosider the system of four ODEs, obtaied by addig to 1.15 two additioal equatios for the variables αt =x y t, y ad βt =ξ y t, x. ẋ = f, ξ = g, 4.43 α =f x + f u v α + f ξ β, β =g x + g u v + g ux wα + g ξ β. Here v = u x ad w = u xx are regarded as idepedet cotrol fuctios. I the first two examples this system is ot cotrollable. Ideed, o matter what cotrols are implemeted, i Example 1 we always have ξt αt, while i Example oe has ξt αt 1. O the other had, i Example 3 there

23 Vol. 13 Examples of oclassical feedback cotrol problems 71 is o fuctioal relatio betwee ξ ad α. We expect that this cotrollability property of the exteded system 4.43 should play a key role, determiig the equality betwee the miimal costs J s μ adj relax μ. Refereces [1] Basar, T., Olsder, G.J.: Dyamic Nocooperative Game Theory. Reprit of the d ed. SIAM, Philadelphia 1999 [] Bressa, A., Wei, D.: No classical problems of optimal feedback cotrol. J. Differ. Equ. 53, [3] Cesari, L.: Optimizatio: Theory ad Applicatios. Spriger, Berli 1983 [4] Docker, E.J., Jorgese, S., Log, N.V., Sorger, G.: Differetial Games i Ecoomics ad Maagemet Sciece. Cambridge Uiversity Press, Cambridge [5] Jugers, M., Trélat, E., Abou-Kadil, H.: Mi-max ad mi-mi Stackelberg strategies with closed-loop iformatio structure. J. Dyam. Cotrol Syst. 17, [6] Medaic, J.: Closed-loop Stackelberg strategies i liear-quadratic problems. I: IEEE Tras. Autom. Cotrol 3, [7] Miller, B.M., Rubiovich, Y.E.: Impulsive Cotrol i Cotiuous ad Discrete- Cotiuous Systems. Kluwer, New York 3 [8] Papavassilopoulos, G.P., Cruz, J.B.: Noclassical cotrol problems ad Stackelberg games. IEEE Tras. Autom. Cotrol 4, [9] Protter, M.H., Weiberger, H.F.: Maximum Priciples i Differetial Equatios. Pretice Hall 1967 [1] Rishel, R.W.: A exteded Potryagi priciple for cotrol systems whose cotrol laws cotai measures. SIAM J. Cotrol 3, [11] Silva, G.N., Viter, R.B.: Measure drive differetial iclusios. J. Math. Aal. Appl., Alberto Bressa ad Delig Wei Departmet of Mathematics Pe State Uiversity Uiversity Park PA 168 USA bressa@math.psu.edu Delig Wei wei@math.psu.edu Received: 1 December 11. Accepted: 14 March 1.