UDU Factored Discrete-Time Lyapunov Recursions Solve Optimal Reduced-Order LQG Problems

Transkript

1 European Journal of Control (24)1: # 24 EUCA UDU Factored Dscrete-Tme Lyapunov Recursons Solve Optmal Reduced-Order LQG Problems L.G. Van Wllgenburg 1, and W.L. De Konng 2,y 1 Systems and Control Group, Wagenngen Unversty, Technotron, P.O. Box 17, 67 AA Wagenngen, The Netherlands; 2 Faculty of Electrcal Engneerng, Mathematcs and Computer Scence, Delft Unversty of Technology, Mekelweg 4, 2628 CD Delft A new algorthm s presented to solve both the fntehorzon tme-varyng and nfnte-horzon tmenvarant dscrete-tme optmal reduced-order lnear quadratc Gaussan (LQG) problem. In both cases the frst order necessary optmalty condtons can be represented by two non-lnearly coupled dscrete-tme Lyapunov equatons, whch run forward and backward n dscrete tme. The algorthm terates these two equatons forward and backward n dscrete tme, respectvely, untl they converge. In the fnte-horzon tme varyng case the teratons start from boundary condtons and the forward and backward n tme recursons are repeated untl they converge. The dscrete-tme recursons are sutable for UDU factorsaton. It s llustrated how UDU factorsaton may ncrease both the numercal effcency and accuracy of the recursons. By means of several numercal examples and the benchmark problem proposed by the European Journal of Control, the results obtaned wth the new algorthm are compared to results obtaned wth algorthms that terate the strengthened dscrete-tme optmal projecton equatons forward and backward n tme. The convergence propertes are llustrated to be comparable. Especally when the reduced compensator dmensons are sgnfcantly smaller than those of the controlled system, the algorthm presented n ths paper s more effcent. Keywords: Non-lnearly Coupled Dscrete-Tme Lyapunov Equatons; Optmal Reduced-Order LQG Control; Restrcted Complexty Controllers; UDU Factorsaton 1. Introducton Controller reducton s a vtal practcal ssue. Two approaches to controller reducton may be dstngushed, drect versus ndrect desgn [1]. Indrect desgn s characterzed by the fact that the desgn s performed n two steps, nstead of one. The two steps concern ether model-reducton followed by full-order controller desgn or full-order controller desgn followed by controller-reducton. A major dsadvantage of these ndrect approaches s that stablty of the closed loop system, n general, cannot be guaranteed, and optmalty, n general, s lost. Drect desgn on the other hand ncorporates both stablty of the closed loop system and optmalty. Therefore f, gven the desgn crtera, a drect desgn method s feasble, t should always be preferred. Ths paper deals wth the drect desgn of optmal reduced-order lnear quadratc Gaussan (LQG) controllers for tme-nvarant and tme-varyng dscrete-tme systems. Ths desgn often s a fundamental part of the desgn of a dgtal optmal control systems [2], sampled ether conventonally [9], or unconventonally [6,16,17]. So far, two approaches to compute nfnte-horzon optmal reduced-order LQG compensators may be E-mal: gerard.vanwllgenburg@wur.nl y E-mal: w.l.dekonng@ew.tudelft.nl Receved 26 March 22. Accepted 24 June 24. Recommended by H. Hjalmarsson and I.D. Landau.

2 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 589 dstngushed. The frst one uses parameter optmsaton to optmse all the parameters of the compensator. Gven a compensator the assocated costs can be computed by solvng a Lyapunov equaton [7,18]. Unless the dmensons of the compensator are very small, ths method becomes nfeasble, due to the large number of parameters, and the non-lnear nature of the optmsaton. Other approaches use the so-called optmal projecton equatons (OPEs), ntally presented by Hyland and Bernsten [1] and Bernsten et al. [3]. In the dscrete-tme case the OPE were strengthened [8,18,19] resultng n so-called strengthened dscrete-tme optmal projecton equatons (SDOPEs). These equatons represent frst-order necessary optmalty condtons. Intally the algorthms based on the OPE were homotopy algorthms [14,15]. These algorthms solve a seres (famly) of problems and are therefore usually not very effcent. In the dscrete-tme case, based on results obtaned for full-order compensaton of dscrete-tme systems wth whte parameters [7], algorthms were presented that terate the frst-order necessary optmalty condtons forward and backward n dscrete tme. These algorthms consttute a generalsaton of the algorthm whch solves the dscrete-tme estmaton and control Rccat equatons of full-order LQG control through forward and backward n tme recurson. As demonstrated n [18] these algorthms are generally much more effcent than homotopy algorthms. Apart from these algorthms, to the best knowledge of the authors, no other algorthms have been presented n the lterature that use forward and backward n tme recurson of dscrete-tme equatons representng frst order necessary optmalty condtons. In ths paper, a thrd approach to compute dscretetme optmal reduced-order LQG compensators s presented. Instead of the SDOPE the formulaton of the necessary optmalty condtons n terms of two non-lnearly coupled recursve dscrete-tme Lyapunov equatons s used. Lke the SDOPE these equatons are terated forward and backward n dscrete tme to solve both the fnte horzon tme-varyng and nfnte horzon tme-nvarant optmal reducedorder LQG problem. As demonstrated n ths paper the Lyapunov equatons allow for UDU factorsaton of the recursons, as opposed to the SDOPE. Ths paper llustrates how UDU factorsaton enhances both the numercal effcency and accuracy of the recursons. Furthermore the forward and backward n tme recursons of the two Lyapunov equatons guarantee the desred nonnegatve defnteness of the matrces, whch determne the soluton, durng the recurson, as opposed to recursons of the SDOPE. Fnally t wll become clear that f the dmenson of the LQG compensator s small compared to that of the system the forward and backward n tme teraton of the two Lyapunov equatons s more effcent than those of the SDOPE. Accordng to [5], some homotopy algorthms that may be used to solve the contnuous-tme nfnte horzon optmal reduced-order LQG problem perform teratons durng whch lnearly coupled contnuous-tme Lyapunov equatons are solved. These teratons dffer sgnfcantly from the forward and backward dscrete-tme recursons performed n our paper. In our case the forward and backward n tme recursons apply to two non-lnearly coupled dscretetme recursve Lyapunov equatons whch consttute necessary optmalty condtons. In our case the algorthm conssts solemnly of these forward and backward n tme recursons. The mentoned homotopy algorthms, among other thngs, solve a set of lnearly coupled Lyapunov equatons durng each sngle teraton. Fnally our approach solves both the tme-nvarant nfnte horzon reduced-order LQG problem as well as the fnte horzon tme-varyng reduced-order LQG problem. Solvng the latter problem enables optmal reduced-order LQG feedback desgn for non-lnear systems [2]. Because they are generally much more effcent than homotopy algorthms we compare the results obtaned wth our new algorthm wth results obtaned by algorthms that terate the SDOPE forward and backward n dscrete-tme [18,19]. In the fntehorzon tme-varyng case one example, consdered n [19], s used to make the comparson. An mportant feature of the fnte-horzon tme-varyng case s that both the system and compensator may have a state, nput and output the dmenson of whch vares wth tme. Although the dmensons of the state, nput and output could n prncple be fxed to the hghest one occurrng, we delberately choose not to do so. The reasons for ths are threefold: (1) allowng for tmevaryng dmensons may result n sgnfcantly smaller mnmal realzatons of compensators [2]; (2) systems wth tme-varyng dmensons arse naturally f the samplng wthn a dgtal control system s not performed synchronously [16,17]; (3) wth fxed dmensons, the nverse of some matrces occurrng n the equatons whch determne the optmal compensator, no longer exst [2]. In the nfnte horzon tme-nvarant case the dmenson of the state, nput and output s fxed and constant and the algorthm s compared wth the one n [18] by means of some characterstc examples taken from the 34 whch were consdered n [18]. In addton we make a comparson by consderng the European Journal of Control benchmark problem.

3 59 L.G. Van Wllgenburg and W.L. De Konng In Sectons 2 5 the paper treats the fnte-horzon tme-varyng case. In Secton 6 the nfnte-horzon tme-nvarant case s treated as a specal case. Secton 7 concludes the paper. 2. The Optmal Fxed-Order Dscrete-Tme LQG Problem Consder the tme-varyng dscrete-tme system, x þ1 ¼ x þ u þ v, ¼, 1,..., N 1, ð1:1þ y ¼ C x þ w, ¼, 1,..., N, ð1:2þ where x 2 R n s the state, u 2 R m s the control, y 2 R l s the observaton, v 2 R n þ1 the system nose, w 2 R l the observaton nose and 2 R n þ1n, 2 R n þ1m, and C 2 R l n are real matrces. Note that the dmenson n of the system state, the dmenson m of the nput and the dmenson l of the output, whch are all bounded, may vary over tme. As a result may not be square. The processes fv g and fw g are zero-mean whte nose sequences wth covarance V 2 R n þ1n þ1, W 2 R l l respectvely and cross covarance Y 2 R n þ1l, whch satsfy, V, W > ) V Y Y T : ð1:3þ W The ntal condton x 2 R n s a stochastc varable wth mean x 2 R n and covarance X 2 R n n and s ndependent of fv g and fw g. As controller the followng tme-varyng dynamc compensator s chosen: ^x þ1 ¼ F ^x þ K y, ¼, 1,..., N 1, ð2:1þ a compensator (2.1) and (2.2), wth prescrbed dmensons n c, ¼, 1,..., N, whch mnmzes the crteron, J N ^x,f N,K N,L N ( ¼ E x T N Zx N þ XN 1 Q M ) x ¼ x T u T M T R u ð3:1þ and to fnd the mnmum value of J N where Q 2 R n n, R 2 R m m, M 2 R n m, Z 2 R n Nn N satsfy, Q, R > ) Q M M T, R Z : ð3:2þ 3. Mnmalty and Frst-Order Necessary Optmalty Condtons The mnmalty of compensators plays a crucal role n optmal fxed-order LQG compensator desgn. One reason s that, gven the desre to reduce the compensator dmensons, mnmal compensators are the nterestng ones. The results to be presented n ths paper requre a sutable combnaton of the results presented by Van Wllgenburg and De Konng [18,19,2]. Ths sutable combnaton of results s presented n ths secton and therefore proofs are omtted. The dmensons of a fnte-horzon mnmal compensator satsfy [2], n c l n c þ1 nc þ m ¼, 1,..., N 1, u ¼ L ^x, ¼,1,..., N 1, ð2:2þ where ^x 2 R nc s the compensator state. The dmenson n c, ¼, 1,..., N of the compensator state s bounded and may be tme-varyng. F 2 R nc þ1 nc, K 2 R nc þ1 l and L 2 R m n c are real matrces. Note that F may not be square. The ntal condton ^x 2 R nc s determnstc. Compensator (2.1) (2.2) s denoted by ð^x, F N, K N, L N Þ where F N ¼fF, ¼, 1,..., N 1g, K N ¼fK, ¼, 1,..., N 1g, L N ¼ fl, ¼, 1,..., N 1g Problem Formulaton Gven the system (1.1) and (1.2) the optmal fxedorder dynamc compensaton problem s to fnd n c ¼ 1, nc N ¼ : ð4:1þ On the other hand f a fnte-horzon compensator has dmensons satsfyng (4.1) then t can always be chosen such that t s mnmal. If n addton to (4.1) the prescrbed compensator dmensons satsfy n c n m, ð4:2þ where n m, ¼, 1,..., N are the dmensons of a mnmal realsaton of the optmal full-order LQG compensator, then, f the conjecture n [2] holds, the global optmal reduced-order compensator s mnmal. Defnton 1. Prescrbed compensator dmensons that satsfy the condtons (4.1) and (4.2) are called max mn compensator dmensons.

4 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 591 In [2] t s explaned how to compute a mnmal realsaton of a fnte-horzon compensator and how to select prescrbed compensator dmensons whch satsfy (4.1) and (4.2). Let P, S, ¼, 1,..., N denote the second moment of the closed loop system and the dual closed loop system respectvely whch satsfy, Pþ1 þ V 2 R ðn þ1þn c þ1 Þðn þ1þn c þ1 Þ, ¼, 1,..., N 1, " # ð5:1þ P ¼ x x T x ^x T ^x x T ^x ^x T Theorem 1. A compensator ð^x, F N, K N, L N Þ satsfes the frst-order necessary optmalty condtons assocated to the optmal fxed-order compensaton problem and s mnmal f and only f F ¼ H þ1 ½ K C L ŠGT 2 R nc þ1 nc, ¼, 1,..., N 1, K ¼ H þ1 K 2 R nc þ1 l, ¼, 1,..., N 1, ð8:1þ ð8:2þ L ¼ L GT 2 R m n c, ¼, 1,..., N 1, ð8:3þ S ¼ T S þ1 þ Q 2 R ðn þn c Þðn þn c Þ, ¼, 1,..., N 1, " SN ¼ Z #, ð5:2þ ^x ¼ H x 2 R nc, where K ¼ð P C T þ Y ÞðC P C T þ W Þ 1, ¼, 1,..., N 1 ð8:4þ ð9:1þ where ¼ L 2 R ðn þ1þn c þ1 Þðn þn c Þ, ð6:1þ K C F Q Q M L ¼ L T MT L T R 2 R ðn þn c Þðn þn c Þ, L ¼, 1,..., N 1 ð6:2þ " # V V Y K T ¼ 2 R ðn þ1þn c K Y T K W K T þ1 Þðn þ1þn c þ1 Þ, ¼, 1,..., N 1 ð6:3þ wth L ¼ð T S þ1 þ R Þ 1 ð T S þ1 þ M T Þ, ¼, 1,..., N 1 ð9:2þ G ¼ P 2þ P 12T 2 R nc n, ¼, 1,..., N ð9:3þ H ¼ S 2þ S 12T 2 R nc n, ¼, 1,..., N ð9:4þ P ¼ P 1 ^P 2 R n n, ¼, 1,..., N ð1:1þ S ¼ S 1 ^S 2 R n n, ¼, 1,..., N ð1:2þ To state the man theorem consder the parttonngs, " # P ¼ P1 P 12, P 1 2 R n n, P 12 2 R n n c, and S P 12T P 2 P 2 2 R nc nc, ¼, 1,..., N ð7:1þ " # ¼ S1 S 12, S 1 2 R n n, S 12 2 R n n c, S 12T S 2 S 2 2 R nc nc, ¼, 1,..., N ð7:2þ and furthermore let P þ denote the Moore Penrose pseudo nverse of P. ^P ¼ P 12 ^S ¼ S 12 P 2þ P 12T 2 R nc nc, ¼,1,...,N ð1:3þ S 2þ S 12T 2 R nc nc, ¼,1,...,N ð1:4þ rankð ^P Þ¼rankð ^S Þ¼n c, ¼,1,...,N ð1:5þ Observe from (5.1) and (5.2) that the condtons n Theorem 1 represent a two pont boundary value problem (TPBVP). Furthermore from (5) and (1) P 1, P2, S1, S2, P, S, ^P, ^S, ¼, 1,..., N are all symmetrc nonnegatve matrces snce P, S, ¼, 1,..., N are nonnegatve symmetrc because they are second moment matrces.

5 592 L.G. Van Wllgenburg and W.L. De Konng 4. The UDU Factored Numercal Algorthm 4.1. Descrpton of the Algorthm Equatons (5.1) and (5.2) are coupled recursve Lyapunov equatons that run forward and backward n tme respectvely. They apply for arbtrary compensators. A mnmal optmal compensator has to satsfy Eqs (8.1) (8.4) whch through Eqs (9.1) (1.5) ntroduces addtonal non-lnear couplngs n Eqs (5.1) and (5.2). The algorthm starts wth a choce, descrbed n Secton 4.4, of nonnegatve symmetrc ntal values for P, ¼, 1,..., N and S, ¼, 1,..., N 1. Note from (5.2) that SN s fully determned by Z. Then the backward n tme recurson of Eq. (5.2) and forward n tme recurson of Eq. (5.1) can be represented as follows: P, S ð5þ;ð8þ;ð9þ;ð1þ ð6:1þ;ð6:2þ F, K, L! þ1! ¼, 1,..., N 1, S, x ð8:4þ;ð9:4þ! ^x ð5:1þ! P,, Q ð5:2þ! S, ð11:1þ ð11:2þ P ð5þ;ð8þ;ð9þ;ð1þ ð6:1þ;ð6:3þ,s þ1! F,K,L!,V ð5:1þ! Pþ1, ¼,1,...,N 1: ð11:3þ From the matrces at the left of each arrow, usng the equatons mentoned above the arrow, the matrces on the rght of the arrow are computed. The matrces on the rght of an arrow replace prevous values that have been computed. The algorthm conssts of repeated applcaton of (11.1) (11.3) respectvely, untl convergence s reached, that s, when new values of P, S, ¼, 1,..., N become dentcal to the prevous ones. In that case f the rank condtons (1.5) are satsfed a soluton of the TPBVP (5.1) (1.5) s found. These rank condtons are satsfed f durng the recursons of the coupled Lyapunov equatons no loss of rank has occurred. Remark 1. The recursons of the coupled Lyapunov equatons guarantee the desred postve nonnegatve defnteness of the matrces P, S, ^P, ^S durng recurson, as opposed to recursons of the SDOPE. On the other hand recursons of the SDOPE can recover the rank of ^P, ^S whch s desrable f, durng the recursons, a loss of rank occurs. It follows from the frstorder necessary optmalty condtons that these two desred propertes exclude one another. From the results obtaned for a large number of examples n [18,19] and the results presented n Sectons 5 and 6 of ths paper, t seems that, n practce, the lack of one of these two propertes s hardly ever a problem Advantages and Implementaton of UDU Factorsaton Gven, Q, V, ¼, 1,..., N 1 the computaton of one recurson of the Lyapunov equatons (5.1) and (5.2) s dentcal to the computaton of a tme-update of the Kalman flter. Therefore two sequental UDU factored computatonal schemes of Berman [4], namely the one on p. 53 and the one on pp , ntended for Kalman flterng, can be used to perform these computatons. Observe that to mplement Eqs (8.1) (1.5) ordnary and pseudo nverses of symmetrc matrces have to be computed. From Eqs (16.1) (17.6) t becomes clear that the computaton of the pseudo nverses s completely crcumvented by employng UDU factorsaton. Furthermore the UDU factorsaton enables a very effcent computaton of the ordnary nverses of the symmetrc matrces n (9.1) and (9.2). Fnally from the extensve numercal analyss n [4] t follows that UDU factorsaton enhances the numercal accuracy and effcency of the teratons of the Lyapunov equatons. The ncreased accuracy becomes apparent only when the teratons tend to be ll condtoned [4]. The computatonal schemes n [4] are only sutable for postve defnte covarance matrces havng constant dmensons. In our case the covarance matrces have tme-varyng dmensons and may be nonnegatve. Fortunately the algorthms can be easly adapted to deal wth both. The followng three equatons represent the computatons performed by the two algorthms, where a tlde denotes a symmetrc nonnegatve matrx: ~P ¼ U ~ P ~D ~ P U T ~P, ð12:1þ ~V ¼ U ~ V ~D ~ V U T ~V, ð12:2þ ~P þ1 ¼ ~P T þ ~V ¼ U ~ P þ1 ~D ~ P þ1 U T ~P þ1 : ð12:3þ Equaton (12.1) represents an UDU factorsaton of ~P n whch U P ~, ~D P ~ are a unt upper trangular and a nonnegatve dagonal matrx respectvely. Smlarly Eq. (12.2) represents an UDU factorsaton of ~V. Our algorthm computes U P ~, ~D P ~ from ~P and U V ~, ~D V ~ from ~V. Equaton (12.3) represents one recurson

6 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 593 of a dscrete Lyapunov equaton. From the matrces U P ~, ~D P ~, U V ~, ~D V ~ and the algorthm computes U P ~ þ1, ~D P ~ þ1 whch represents an UDU factorsaton of ~P þ1. The fact that the nverse U 1 of a non-sngular ~P upper trangular matrx U P ~ can be computed wth hgh accuracy and effcency usng another algorthm from [4], that s, the one on p. 65, wll also be exploted. The algorthm apples to non-sngular upper trangular matrces. Explotng the fact that our matrces are unt upper trangular enables us to further smplfy ths algorthm. In addton we wll explot the fact that the nverse of a postve defnte dagonal matrx ~D P ~ s a dagonal matrx ~D 1 wth dagonal ~P elements that are one over the correspondng ones n ~D P ~. In partcular we may compute the nverse ~P 1 of ~P n (12.1) as, ~P 1 ¼ U T ~P ~D 1 ~P U 1 ~P : ð13þ The actual mplementaton of these algorthms nsde Matlab [12] was performed by the authors through mex fles, the source of whch s wrtten n C-code [13]. The C-code of these mex fles s avalable upon request UDU Factorsaton of the Algorthm Havng computed UDU factorsatons U P, D P, ¼, 1,..., N these can be parttoned n the P same way as P n (7.1), " U 1 P U 12 # U P ¼ P, U 1 U 2 P 2 R n n, U 12 P 2 R n n c P, 2 R nc nc, ¼, 1,..., N, ð14:1þ D P U 2 P " D 1 # P ¼, D 2 P D 1 P 2 R n n, D 2 P 2 R nc, ¼, 1,..., N, ð14:2þ where U 1 P, U 2 P, ¼, 1,..., N are also unt upper trangular and D 1 P, D 2 P, ¼, 1,..., N are also dagonal and nonnegatve. From [11] note that, rankð ^P Þ¼n c, P 2 >, D 2 P rankð ^S Þ¼n c, S 2 >, D 2 S of > ð15:1þ > : ð15:2þ Usng (15.1), (15.2), (1.1) (1.5) after multplyng out the parttoned UDU factorsaton (14.1) and (14.2) we obtan P 1 ¼ U 1 P P 12 ¼ U 12 P P 2 ¼ U 2 P P ¼ U 1 P ^P ¼ U 12 P G ¼ U 2 T P D 1 P D 2 P D 2 P D 1 P D 2 P U 1T P U 2T P U 2T P U 1T P þ U 12 P D 2 P U 12T P, ð16:1þ, ð16:2þ, ð16:3þ, ð16:4þ U 12T P, ð16:5þ U 12T P : ð16:6þ Dually after parttonng and multplyng out the UDU factorsatons U S, D S of S, we obtan S 1 ¼ U 1 S S 12 ¼ U 12 S S 2 ¼ U 2 S S ¼ U 1 S ^S ¼ U 12 S D 1 S D 2 S D 2 S D 1 S D 2 S H ¼ U 2 T S U 1T S U 2T S U 2T S U 1T S þ U 12 S D 2 S U 12T S, ð17:1þ, ð17:2þ, ð17:3þ, ð17:4þ U 12T S, ð17:5þ U 12T S : ð17:6þ Gven (16.1) (17.6) the repeated computaton of (11.1) s performed as follows. To compute F, K, L n (11.1) frst K, L are computed accordng to Eqs (9.1) and (9.2). In (9.1) an UDU factorsaton of ðc P C T þ W Þ s computed frst. As an nput ths computaton uses the matrx C, the UDU factorsaton (16.4) of P and an UDU factorsaton of W. Next an UDU factorsaton of the nverse of ths matrx, that s, ðc P C T þ W Þ 1, s computed accordng to Eq. (13). Smlar arguments hold for the computaton of ð T S þ R Þ 1 n Eq. (9.2). From the UDU factorsatons of these matrces the matrces

7 594 L.G. Van Wllgenburg and W.L. De Konng themselves are recovered and next K, L n (9.1) and (9.2) are computed through standard matrx calculatons. The same holds for the calculaton of G, H accordng to (16.6) and (17.6) and fnally for the calculaton of F, K, L accordng to Eqs (8.1) (8.3). Havng computed F, K, L n (11.1), va standard matrx calculatons we calculate, Q accordng to Eqs (6.1) and (6.2). Next an UDU factorsaton of Q s computed. Ths UDU factorsaton s used together wth and the UDU factorsaton of Sþ1 to compute an UDU factorsaton of S accordng to equaton (5.2). Ths completes the computaton of (11.1). The computaton of (11.2) s completely performed usng standard matrx calculatons. Fnally the computaton of (11.3) s dual to the computaton of (11.1) and so s performed n the same manner Intalsaton of the Algorthm The nonnegatve values of P, ¼ 1, 2,..., N and, ¼, 1,..., N 1, wth whch the algorthm s S ntalsed, are selected based on the followng consderatons. Equatons (9.1) (1.5) descrbe the lnk between the parttoned matrces whch determne the Lyapunov equatons and the SDOPE. The latter are lnked to the standard estmaton and control Rccat equatons of full-order LQG control and so s the ntalsaton of the algorthms to solve these equatons [6,19]. Therefore the ntalsaton of the Lyapunov equatons s performed smlar to the ntalsaton of the SDOPE, that s, ^P, ¼ 1, 2,..., N 1 are chosen to be random nonnegatve whle P, ¼ 1, 2,..., N are chosen to be zero matrces. Accordng Eqs (16.4) and (16.5) ths comes down to settng D 1 to zero, whch makes the choce of U 1 rrelevant, and pckng the free elements of U 12, U 2, D2 random and postve n the case of D 2. Through Eqs (16.1) (16.3) ths fully determnes P. The matrces S, ¼, 1,..., N 1 are ntalsed the same way. 5. Numercal Issues, Examples and an Algorthm Comparson 5.1. Numercal Dampng and Convergence Detecton The convergence of algorthms to compute optmal reduced-order LQG compensators, based on the SDOPE, n crtcal stuatons, s enhanced sgnfcantly by the ntroducton of a numercal dampng [18,19]. The same holds for the algorthms presented here. In the fnte horzon tme-varyng case the numercal dampng s realsed by computng: P jþ1 S jþ1 :¼ ð1 ÞP jþ1 ;ð11þ þ Pj, ¼, 1,..., N, <1, ð18:1þ :¼ ð1 ÞS jþ1, ð11þ þ Sj, ¼, 1,..., N, <1, ð18:2þ where P j, Sj represents P, S after the jth recurson of the algorthm and where P j ;ð11þ, Sj ;ð11þ represents P, S obtaned after executon of (11.1) (11.3) durng the jth recurson of the algorthm. The parameter <1 s the numercal dampng factor. Based on the fact that the scalng (scalar multply) of an UDU factored matrx comes down to the scalng of the dagonal elements of the dagonal matrx we can agan explot the adapted algorthms from Berman [4] to perform ths computaton. Lke n [19] convergence s assumed when the relatve dfference of consecutve values of traceðp N þ S Þ falls repeatedly below a certan tolerance Numercal Examples and an Algorthm Comparson Example 1. Consder the fnte-horzon dscrete-tme LQG compensaton problem (1.1) (2.2), (4.1) and (4.2) wth :9653 :7942 ¼ð1þsnðÞÞ, :7942 :9653 ¼ :4492, C ¼½:6171 :3187Š, :1784 V ¼ dagð:7327 :8612Þ, Y ¼½ :677 :536Š T, W ¼ :9334, Q ¼ dagð:437 :118Þ, M ¼½ :859 :17Š T, R ¼ :3311, x ¼½11Š T, X ¼ dagð:1 :1Þ, Z ¼ dagð:1 :1Þ The spectral radus of equals ð1 þ :2 snðþþ:1:25 so the system s unstable. The followng prescrbed compensator dmensons are max mn dmensons, n c ¼ 1, ¼, 1, 2, 3, 6, 7, 8, n c ¼ 2, ¼ 4, 5, n c ¼, ¼ 9, N ¼ 9 ð19þ

8 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 595 Table 1. Solutons of Example 1 for dfferent prescrbed compensator dmensons. Equaton Costs assocated to each soluton (25) (26) (27) The frst column specfes the equaton whch determnes the prescrbed compensator dmensons n c, ¼, 1,..., N. The remanng elements n each row represent the costs assocated to each soluton. Table 2. Speed of convergence of the Lyapunov (L) equatons and the SDOPE (P) for Example 1 wth prescrbed compensator dmensons (23) and a convergence tolerance of 1 8. Algorthm Costs Number of recursons before convergence/experment ndex L /4 44/1 44/11 46/9 47/2 47/13 48/6 53/12 71/7 9/15 P /7 14/8 17/3 17/12 146/2 157/11 162/6 175/13 197/15 27/14 218/1 224/1 274/4 L /8 196/5 219/1 228/14 P /9 L /3 P /5 Wth these prescrbed max mn compensators dmensons the problem s dentcal to the one n [19] when ¼. In [19] two dfferent solutons were found usng the algorthm based on the SDOPE. In our case, besdes these two solutons, a thrd soluton was found (see Table 1). The rank condtons (1.5) n each case were verfed to hold. The prescrbed compensator dmensons n c ¼ 1, ¼, 1,...,8, n c ¼, ¼ 9, N ¼ 9 ð2þ are also max mn dmensons. In ths case our algorthm fnds sx solutons (see Table 1). Although the dfferences n costs are rather small, nspecton of the matrces P, S, ¼, 1,..., N 1 reveals that they are truly dfferent solutons. Agan the rank condtons (1.5) n each case were verfed to hold. Fnally, on purpose, the followng prescrbed compensator dmensons whch are not max mn dmensons have been selected: n c ¼ 1, ¼, 1,...,6, n c ¼ 2, ¼ 7, 8, n c ¼, ¼ 9, N ¼ 9: ð21þ In ths case the global optmal compensator s not mnmal because n c, ¼ 8,9 volate the condtons n Theorem 1. Then our algorthm should preferably produce non-mnmal compensators, the mnmal realsatons of whch have dmensons n c mn, ¼, 1,..., N gven by n c mn ¼ 1, ¼, 1, 2, 3, 4, 5, 6, 8, n c mn ¼ 2, ¼ 7, n c ¼, ¼ 9, N ¼ 9, ð22þ snce these are the maxmal values whch satsfy the condtons n Theorem 1 and n c mn n c, ¼, 1,..., N. The rank of both ^P, ^S, ¼ 1, 2,..., N 1 determne the dmensons of a mnmal realsaton of a compensator satsfyng the frst-order necessary optmalty condtons [2]. For all three solutons (see Table 1) the rank of both ^P, ^S equals n c mn, ¼ 1, 2,..., N 1. Table 2 compares the speed of convergence of the algorthm based on the Lyapunov equatons and the SDOPE. Both algorthms were executed 15 tmes. Each tme the ntal values were taken randomly, as descrbed n Secton 4.4, and were used for both algorthms. For each soluton and both algorthms the number of recursons necessary to reach convergence s tabulated n ascendng order followed by the experment number whch runs from 1 to 15. Observe that despte the equal ntalsaton of both algorthms they may generate dfferent (locally) optmal solutons. For ths example the number of recursons requred by the Lyapunov equatons, on average, equals 96.6 aganst for the SDOPE. On the other hand one sngle recurson (one backward and one forward recurson) takes approxmately.563 seconds for the Lyapunov equatons and.43 seconds for the SDOPE usng a pentum 866 MHz PC, and Matlab verson 6. under Wndows 2. Although for ths example one recurson of the SDOPE s computatonally faster, for hgher orders of the system and small orders of the compensator, the stuaton s the opposte, as demonstrated n the next secton.

9 596 L.G. Van Wllgenburg and W.L. De Konng 6. The Infnte-Horzon Tme-Invarant Case 6.1. Descrpton of the Algorthm Assume that the state, output and nput of the system and the compensator have a constant fxed dmenson, that s, n ¼ n, m ¼ m, l ¼ l, n c ¼ n c, ¼, 1,.... Also assume that all the matrces n the problem formulaton are tme-nvarant, that s, ¼, ¼, C ¼ C, V ¼ V, W ¼ W, Y ¼ Y, Q ¼ Q, R ¼ R, M ¼ M, F ¼ F, K ¼ K, L ¼ L, ¼, 1,..., N 1. Consder the crteron (3.1) and (3.2) dvded by N, that s, dvded by the fnal tme. Then f the fnal tme N!1, the tme-nvarant nfnte-horzon fxedorder LQG compensaton problem s obtaned. Wthn the tme-nvarant nfnte-horzon fxed-order LQG compensaton problem, the matrx Z and the boundary condtons, play no role anymore, and steady state solutons of the Lyapunov equatons (5.1) and (5.2) determne optmal fxed-order compensators. As n the tme-varyng fnte-horzon case, these solutons can be obtaned through forward and backward n tme recurson of Eqs (5.1) and (5.2). In ths case however only a sngle recurson of each Lyapunov equaton s performed smultaneously. Usng the same notaton as n (11.1) (11.3), and dsregardng the tme ndces and boundary condtons n the equatons, these computatons are represented by: P,S ð5þ;ð8þ;ð9þ;ð1þ ð6:1þ;ð6:2þ;ð6:3þ! F,K,L!,Q,V ð5:1þ;ð5:2þ! P,S : ð23þ Startng from ntal values P, S, computed as descrbed n Secton 4.4, the algorthm conssts of repeated applcaton of the computatons (23), untl convergence s reached, that s, when new values of P, S become dentcal to the prevous ones. If P 2 >, S 2 > then a mnmal compensator s obtaned whch satsfes the frst-order necessary optmalty condtons. The numercal dampng and the convergence detecton are mplemented accordng to the descrpton n Secton 5.1 n whch the tme ndces should now be removed. In the nfnte horzon case stablty of the closed loop systems becomes an ssue. Condtons whch guarantee closed loop stablty have been stated n [18] Numercal Examples and an Algorthm Comparson Algorthms for nfnte-horzon tme-nvarant optmal reduced-order LQG compensaton based on the SDOPE have been presented n [18]. In that paper the results of two computer experments contanng a total of 34 examples were presented. In [18] the numercal dampng appled n the algorthms was fxed. Wth ths fxed numercal dampng the algorthm faled to converge for a number of problems. If we select the numercal dampng automatcally, by ncreasng t f dvergence or oscllatons occur durng the teratons, t turns out that the algorthms n [18] can be made to converge for all the 34 examples. Usng the same automatc selecton of the numercal dampng we verfed that, apart from one example, ths also apples to the algorthm presented n ths paper. In Table 3 we compare the speed of convergence of the Lyapunov and SDOPE algorthms on four characterstc examples out of the 34 examples consdered n [18]. Lke n [18] the algorthms were executed 1 tmes. Each tme the ntal values were randomly selected, as descrbed n Secton 4.4, and used for both algorthms. The number of recursons necessary to reach convergence are tabulated n ascendng order. For n ¼ 5 one out of the ten Lyapunov algorthm runs dd not converge. The Lyapunov equatons consttute two equatons of dmenson n þ n c whle the SDOPE consttute four equatons of dmenson n. As a result, when n c s sgnfcantly smaller than n, Lyapunov recursons are faster. When on the other hand, n c s close to n, then recursons of the SDOPE are faster. Furthermore the computaton tme of the SDOPE s more or less ndependent of the value of n c. In Fg. 1 the computaton tme of a sngle recurson of each algorthm s plotted, as a functon of n and n c. Agan these computaton tmes are measured on a pentum 866 MHz PC, usng Matlab verson 6. under Wndows 2. Table 3. Speed of convergence of the Lyapunov equatons (L) and the SDOPE (P) wth a convergence tolerance of 1 6. Algorthm n Number of recursons before convergence L P L P L P

10 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 597 Fg. 1. Computaton tme [s] of one recurson of the Lyapunov equatons and the SDOPE for n ¼ 1, 2, 4, 8 versus n c ¼ 1, 2,..., n The European Journal of Control Benchmark Problem The benchmark problem for reduced-order controller desgn concerns an actve suspenson system used for dsturbance attenuaton n a large frequency band n the presence of load varatons. Two lnear dscretetme black box models n polynomal form have been dentfed separately, namely one descrbng the transfer functon C/D from the sngle dsturbance

11 598 L.G. Van Wllgenburg and W.L. De Konng descrbed by x þ1 ¼ x þ u þ v, ¼, 1,..., ð24:1þ y ¼ Cx þ w, ð24:2þ Fg. 2. Block dagram of the European Journal of Control benchmark. nput to the sngle measured output, and one descrbng the transfer functon B/A from the sngle control nput to the sngle measured output, see Fg. 2. Due to the separate and black box nature of the models both ther states do not have a physcal meanng and do not nteract. If an attempt would be made to model the system n state-space form and n contnuous-tme, usng frst prncples from mechancs, then the full system state would be nfluenced by both the control nput and the dsturbance. Also then the dsturbance attenuaton could be formulated n terms of the full state of the system by means of a quadratc ntegral crteron rather then n terms of nput and output senstvty functons, as s done n the benchmark problem formulaton. After computng the equvalent dscrete-tme optmal control problem [6] our dscrete-tme optmal reduced-order LQG controller desgn method would be very well suted to solve ths problem. Unfortunately, the proposed state-space model s not avalable. We can however also apply our dscrete-tme optmal reduced-order LQG controller desgn when the dsturbance attenuaton s formulated n terms of a quadratc measure y T Q y y penalzng the magntude of the resdual force whch s the sngle measured output y and when we convert the prmary and secondary dscretetme transfer functons nto two state space realzatons ð p, p, C p Þ and ð s, s, C s Þ respectvely. The states of these two state-space realzatons are denoted by x p and x s respectvely, and do not nteract. Ths s the approach that we wll adopt here. Note that our desgn penalzes the resdual force equally over all frequences. Therefore our desgn wll not necessarly satsfy the desgn specfcatons of the benchmark whch are mposed through bounds on the nput and output senstvty functons. Our problem formulaton takes the followng form. The system s where x 2 R n s the state, u 2 R m s the control y 2 R l s the output,,, C are the system matrces. v, ¼, 1,... s a dscrete-tme whte nose process wth a covarance matrx V 2 R nn representng the dsturbances actng on the system and w, ¼, 1,... s a dscrete-tme whte nose process wth a covarance matrx W 2 R ll representng the measurement errors. The quadratc crteron s gven by ( ) 1 X N J ¼ E lm x T N!1 N Qx þ u T Ru : ð25þ ¼1 The system state x and the matrces,, C, Q, R, V, W whch make up the LQG problem equal x ¼ x p, ¼ p, ¼, x s s s C ¼ ½C p C s Š, V ¼ T p V p p, ð26þ W ¼ W y, Q ¼ C T Q y C, R ¼ R u where V p 2 R 11 s the scalar covarance matrx of the scalar whte nose process v p, ¼, 1,..., the dsturbance nput to the prmary model, W y 2 R 11 s the scalar covarance matrx of the scalar whte measurement nose affectng the sngle output y. Furthermore Q y 2 R 11 determnes the scalar quadratc penalty y T Q y y on the sngle system output reflectng the dsturbance attenuaton and R u determnes the scalar quadratc penalty u T R u u on the sngle control varable u. In Eq. (26) the scalars V p, W y, Q y, R u are desgn parameters related to the magntude and attenuaton of dsturbances. Assumng the measurement errors to be small compared to the dsturbances and allowng for a reasonably large control to attenuate dsturbances the followng choce of the desgn parameters was made: V p ¼ 1, W y ¼ 1 4, Q y ¼ 1, R u ¼ 1 4 : ð27þ The optmal reduced-order LQG problem obtaned n ths manner, especally for ntermedate and small compensator dmensons, suffers from several local mnma. We suspect that these local mnma relate to

12 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson 599 the large number of complex conjugate egenvalues of the stable system matrx, several of whch have absolute values close to 1. Also the spectral radus of the closed loop system, n each case, s very close to 1. Roughly speakng the closed loop system s only just stable and hghly oscllatory. These propertes slow down the convergence of the teratve algorthms and also make them senstve to numercal errors. Despte these propertes all solutons were obtaned from recursons of both the SDOPE and the Lyapunov equatons, except for n c ¼ 1, 2. For the compensator dmensons n c ¼ 1, 2 constraned non-lnear parameter optmzaton outperforms our algorthms [18]. Otherwse the drawbacks assocated to hgh dmensonal constraned non-lnear parameter optmzaton preval and soon renders ths approach practcally mpossble. For n c ¼ 1, 2 the optmal compensator was computed usng constraned non-lnear parameter optmsaton and a controllablty canoncal representaton of the compensator. Then for n c ¼ 1 two parameters need to be optmzed and for n c ¼ 2 four parameters. Fgure 3 shows the results of four computatons of the compensator order n c versus the costs. Two of them are computed as follows. For each value n c ¼ 27, 26,..., 4, 3 the Lyapunov and SDOPE algorthm were executed fve tmes wth dfferent random ntalzatons and the best soluton was selected. A monotone decrease of the cost of the global optmal compensator should occur f the conjecture n [2] holds. Due to the exstence of several local mnma the costs are not dentcal everywhere and also not monotoncally decreasng everywhere. Two other attempts were made to fnd the global solutons and to obtan a monotonc decrease. In ths case for each value of n c the frst of the fve algorthm runs of both the SDOPE and the Lyapunov equatons was ntalzed wth the best soluton obtaned from the prevous fve algorthm runs for compensator order n c þ 1. In case of the SDOPE ths ntalzaton s straghtforward snce the dmensons of P, S, ^P, ^S 2 R nn do not depend on n c whle rankð ^PÞ ¼ rankð ^StÞ ¼n c s automatcally reduced by the algorthm. In the case of the Lyapunov equatons however the dmensons of P, S 2 R ðnþn cþðnþn c Þ do depend on n c. From [18] we adopt the followng approach. Frst P, S 2 R ðnþn cþ1þðnþn c þ1þ are translated nto P, S, ^P, ^S 2 R nn usng Eqs (1.1) (1.4). Then a Fg. 3. Four computatons of the costs versus the compensator dmenson n c ¼ 1, 2,..., 28 for the European Journal of Control benchmark problem.

13 6 L.G. Van Wllgenburg and W.L. De Konng ðg, M, HÞ factorzaton of ^P ^S s computed based on the largest n c postve egenvalues of ^P ^S [18]. Then G, H 2 R n cn and from [18, Eq. (A22)] " # P ¼ P þ ^P ^PH T 2 R ðnþn cþðnþn c Þ, H ^P H^PH T " # S ¼ S þ ^S ^SG T 2 R ðnþn cþðnþn c Þ, ð28þ G ^S G^SG T When the frst of the fve algorthm runs s ntalsed n ths way a monotone decrease and the lowest costs n Fg. 3 for each value of n c are obtaned wth the Lyapunov algorthm. The results obtaned wth the SDOPE are dentcal except for the costs for n c ¼ 3, 12. Note that the results obtaned wth the Lyapunov algorthm stll do not guarantee that we have found the global mnmum for each value of n c although t s more lkely. 7. Conclusons A new algorthm has been presented to compute fnte horzon tme-varyng and nfnte horzon tme-nvarant dscrete-tme optmal reduced-order LQG compensators. The algorthm terates two nonlnearly coupled recursve dscrete-tme Lyapunov equatons forward and backward n tme untl convergence. Compared to the computaton of optmal reduced-order compensators va forward and backward n tme recurson of the SDOPE, ths method has the followng advantages. Snce the Lyapunov equatons consttute two matrx equatons of dmenson n þ n c, ¼, 1,..., N (n þ n c n the tme-nvarant nfnte horzon case) whle the SDOPE consttute four equatons of dmenson n, ¼, 1,..., N, ðnþ f the dmensons of the compensator state n c ðn cþ are small compared to those of the system, that s, n ðnþ, the Lyapunov equatons are more effcent, assumng that the convergence propertes are comparable. The results of ths paper ndcate that the latter s the case. Also the Lyapunov equatons guarantee nonnegatve defnteness of the matrces durng the recursons, as opposed to the SDOPE. Furthermore the Lyapunov equatons do not requre the computaton of the oblque projecton, whch s fundamental to the SDOPE, and whch requres computatonally expensve egenvalue decompostons and nversons of non symmetrc square matrces. As demonstrated n ths paper the Lyapunov equatons are sutable for UDU factorzaton, as opposed to the SDOPE, whch cannot be guaranteed to be nonnegatve durng the recursons. It has been llustrated how UDU factorsaton can mprove the relablty and effcency of the computatons. Recursons of the SDOPE on the other hand, are able to ncrease the rank of ^P, ^S ð ^P, ^SÞ durng the recursons, as opposed to recursons of the Lyapunov equatons. The rank of both ^P, ^S ð ^P, ^SÞ should be as hgh as possble and s constraned to be less or equal to the prescrbed dmensons of the compensator state n c ðn cþ. The ncrease of ths rank s mportant f, durng the recursons, for some reason, a temporary loss of rank occurs. The results n ths paper obtaned wth the Lyapunov equatons ndcate that ths stuaton hardly ever occurs. The capablty of the SDOPE to ncrease the rank s precsely the one that prevents that nonnegatve defnteness of the matrces durng the recursons can be guaranteed. The results of ths paper also ndcate that the possble negatve defnteness durng the recursons hardly ever presents a problem. The mathematcal proof of the convergence propertes of forward and backward n tme recursons of both the SDOPE and the Lyapunov equatons remans an open area of research, except for the SDOPE assocated to full-order compensaton of systems wth determnstc and whte parameters. Only the latter two problems do not suffer from the possble exstence of local mnma whch hamper the mathematcal proof [18,19]. Despte ths lack of mathematcal proof, the practcal experence obtaned by the authors, wth a large number of examples some of whch have been presented n ths paper, ndcates that the algorthms have very mpressve and comparable convergence propertes, f automatc selecton of the numercal dampng s appled. Lkely ncreasng the numercal dampng makes the problem more convex locally. References 1. Anderson BDO, Lu Y. Controller reducton: concepts and approaches. IEEE Trans Autom Control 1989; 34(8): Athans M. The role and use of the stochastc lnear quadratc gaussan problem n control system desgn. IEEE Trans Autom Control 1971; 16(6): Bernsten DS, Davs LD, Hyland DC. The optmal projecton equatons for reduced-order dscrete-tme modelng, estmaton and control. J Gudance Control Dyn 1986; 9(3): Berman GJ. Factorzaton methods for dscrete sequental estmaton. Acadamc Press, New York; Collns EG, Hodel AS. Effcent soluton of lnearly coupled Lyapunov equatons. SIAM J Matrx Anal Appl 1997; 18(2):

14 Optmal Reduced-Order LQG Desgn Through Lyapunov Recurson De Konng WL. Equvalent dscrete optmal control problem for randomly sampled dgtal control systems. Int J Syst Sc 198; 11: De Konng WL. Compensatablty and optmal compensaton of systems wth whte parameters. IEEE Trans Autom Control 1992; 37(5): De Konng WL, Van Wllgenburg LG. Numercal algorthms and ssues concernng the dscrete-tme optmal projecton equatons for systems wth whte parameters. In: Proceedngs UKACC Control 98 Conference, Vol. 2, Swansea UK, 1 4 September 1998; pp Halyo N, Caglayan AK. A seperaton theorem for the stochastc sampled-data LQG problem. Int J Control 1976; 23(2): Hyland DC, Bernsten DS. The optmal projecton equatons for reduced-order dynamc compensaton. IEEE Trans Autom Control 1984; 29: Krendler E, Jameson A. Condtons for nonnegatveness of parttoned matrces. IEEE Trans Autom Control 1972; 17: Matlab 6 User s Gude. The MathWorks Inc Matlab 6 External Interface Gude. The MathWorks Inc Rchter S. A homotopy algorthm for solvng the optmal projecton equatons for reduced-order dynamc compensaton: exstence, convergence and global optmalty. In: Proceedngs Amercan Control Conference, Mnneapols, MN, June 1987; pp Rchter SL, Collns EG. A homotopy algorthm for reduced-order compensator desgn usng the optmal projecton equatons. In: Proceedngs 28th Conference on Decson and Control, Tampa, FL Van Wllgenburg LG, De Konng WL. Dervaton and computaton of the dgtal LQG regulator and tracker n case of asynchronous and aperodc samplng. Control Theory Adv Tech 1995; 1(4): Van Wllgenburg LG. Dgtal optmal control and LQG compensaton of asynchronous and aperodcally sampled non-lnear systems. In: Proceedngs of the European Control Conference, Vol. 1, Rome, 5 8 September 1995; pp Van Wllgenburg LG, De Konng WL. Numercal algorthms and ssues concernng the dscrete-tme optmal projecton equatons. Eur J Control 2; 6: Van Wllgenburg LG, De Konng WL. Optmal reduced-order compensators for tme-varyng dscretetme systems wth determnstc and whte parameters. Automatca 1999; 35(1): Van Wllgenburg LG, De Konng WL. Mnmal and non-mnmal reduced-order compensators for tmevaryng dscrete-tme systems. Automatca 22; 38(1):