Evaluating Parallel Algorithms for Solving Sylvester-Type Matrix Equations

Transkript

1 Evaluating Parallel Algorithms for Solving Sylvester-ype Matrix Equations Direct ransformation-based versus Iterative Matrix-Sign-Function-Based Methods Robert Granat and Bo Kågström, Umeå University and HPC2N, Sweden

2 Outline Sylvester-ype Matrix Equations Direct ransformation-based Methods Matrix-Sign-Function-Based Methods Matrix Equations as Linear Systems Condition Estimation ScaLAPACK Environment Parallel Implementations Experimental Evalutation Summary Future/Ongoing Work References

3 Sylvester-ype Matrix Equations Sylvester-type matrix equations arise in many applications in science and engineering: block diagonalization of matrices in Schur form, condition estimation of eigenvalue problems, control theory etc. he continuous-time Sylvester equation (SYC): AX M M N N XB = C, A R, B R, Solution is unique iff λ( A) λ( = If we can solve SYC, we can solve many other similar equations We consider and compare two different methods C R M N

4 Direct ransformation-based Methods () o solve SYC apply Bartels-Stewart s method: ransform A and B to real Schur form: A = Q AQ, B = P BP Update the matrix C with respect to the transformations ~ C = Q CP Solve the reduced triangular system A ~ X ~ X B ~ = C ransform the solution back to the original coordinate system X ~ = QXP No extra conditions is imposed on A or B by the method

5 Direct ransformation-based Methods (2) riangular problem is solved by blocking AX XB = C A ii X ij X ij B ij In parallel, we traverse the matrix C/X along its block diagonals and solve SYC subsystems and do GEMM-updates w r t the subsolutions on the nodes = C ij ( D a k= i+ A ik X kj j k= X ik B kj )

6 Matrix-Sign-Function-Based Methods () Let have the Jordan decomposition where and contain the Jordan blocks with eigenvalues in the open left and right half planes, respectively. he matrix sign function is defined as he matrix sign function can be computed via Newton iteration for the equation : R Z R Z p p ), λ( + = S J J S Z ) ( ) (, k p k p k k C J C J + ) ( = S I I S Z sign k p k Z = I 2 + = = + 2 ) / ( k k k Z Z Z Z Z J + J

7 Matrix-Sign-Function-Based Methods (2) [J.D. Roberts, ]: sign( Z) = lim Zk, and k A C X sign + I m+ n = 2 B I Sign-function method can be applied to SYC iff A and B are c-stable: Re(λi ) <, i Parallel implementation by Benner and Quitana-Orti (in PSLICO)

8 Matrix Equations as Linear Systems All linear matrix equations can be represented as a linear system of equations: op ( A) X Xop( = C Z SYC x = y Z SYC = I op( A) op( N I M x = vec( X ), y = vec(c) In blocked algorithms, Zx = y representation is used in kernels for solving small-sized matrix equations.

9 Condition Estimation An important quantity in the perturbation theory for Sylvestertype equations is the separation between two matrices: sep( A, o compute sep(a, exactly costs O(M 3 N 3 ) flops (only of interest in theory). We want to compute a reliable but low cost estimate (serially as well as in parallel). We apply a general method (Hager 84, Higham 88, Kågström- Poromaa 92) for estimating A which only uses matrix vectors products: = inf op( A) X Xop( = σ min ( Z SYC ) = Z SYC X = F F A x A we can estimate sep(a, for SYC by solving the equation itself to an O(M 2 N+ MN 2 ) cost. Notice that when sep(a, is tiny, the SYC equation is close to singular, i.e., ill-conditioned (compare with the scalar case x = c / (a b) ). x 2

10 ScaLAPACK Environment HPC library for dense linear algebra on distributed memory machines Buildt on LAPACK, BLAS, PBLAS, BLACS Fortan 77 SPMD object-oriented programming style 2D processor grid All matrices are blockpartitioned by rows and columns and distributed using 2D block-cyclic mapping

11 Parallell Implementations () ScaLAPACK-style implementation of Bartels- Stewart s method (Granat-Kågström-Poromaa): Reduction to triangular form Hessenberg reduction PDGEHRD QR-algorithm PDLAHQR ransforming rhs and solution to tri. problem PDGEMM Solving the triangular problem Kernel SYC-solver DRSYL GEMM-updates DGEMM Resulting routine PGESYCD

12 Parallell Implementations (2) ScaLAPACK-style implementation of the Matrix-sign-function-based method (Benner and Quintana-Orti): LU decomposition PDGERF Solving linear systems of equations PDGERS Inversion based on LU decomposition PDGERI Solving triangular systems with multiple rhs PDRSM Pivoting of a distributed matrix - PDLAPIV Resulting routine from PSLICO psb4md

13 Experimental Evaluation () wo target parallel computers: IBM SP system 64 thin 2MHz nodes with 28MB RAM 5 Mbyte/sec peak network bandwidth Well-balanced: t_flop/t_comm =. Linux Super Cluster 2 dual.667mhz nodes with GB RAM 667 Mbyte/sec peak network bandwidth Less well-balanced: t_flop/t_comm =.25

14 Experimental Evaluation (2) est problem matrices: We present three performance ratios Measured parallel execution time Accuracy: Frobenius norm of absolute error: Frobenius norm of absolute residual: A = Q( α D + βm ) Q If any ratio >, psb4md shows better results, otherwise PGESYCD performs equal or better Condition estimation by computing a lower bound of sep ( A, A A X q, q, q R ~ X X F AX ~ XB ~ C

15 Experimental Evaluation (3) Wellconditioned problems on IBM SP sep est ( A, Exec.time mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n= Error mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n=496.5 Residual mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n= q t q x 2 q r

16 Experimental Evaluation (4).8.6 Illconditioned problems on IBM SP Exec.time mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n=496 2 Error mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n=496 sep est ( A, Residual mb=nb=64 m=n=52 m=n=24 m=n=248 m=n=372 m=n=496 q t log (q x ) log (q r )

17 Experimental Evaluation (5) Wellconditioned problems on Linux Super Cluster Exec.time mb=nb= Error mb=nb=32 m=n=24 m=n=248 m=n=496 m=n= sep est ( A, 3 Residual mb=nb=32 m=n=24 m=n=248 m=n=496 m=n= q t q x.8 q r m=n=24 m=n=248 m=n=496 m=n=

18 Experimental Evaluation (6) Illconditioned problems on Linux Super Cluster Exec.time mb=nb=32 m=n=24 m=n=248 m=n=496 m=n= Error mb=nb=32 sep est ( A, Residual mb=nb= q t log (q x ) 8 2 log (q x ) m=n=24 m=n=248 m=n=496 m=n= m=n=24 5 m=n=248 m=n=496 m=n=

19 Experimental Evaluation (7) Wellconditioned triangular problems on Linux Super Cluster sep est ( A, Exec.time mb=nb=28 m=n=24 m=n=248 m=n=496 m=n= Error mb=nb=28 m=n=24 m=n=248 m=n=496 m=n= Residual mb=nb=28 m=n=24 m=n=248 m=n=496 m=n= q t.4 q x q x

20 Summary Routine Generality Reliability Speed Accuracy PGESYCD A and B must have no eigenvalues in common Always delivers a result Up to four times faster on the most balanced parallel platform Always much better for illconditioned problems Always much faster for triangular problems (even on less balanced platform) ends to deliver the smallest residual norm psb4md A and B must have no eigenvalues in common A and -B must be c-stable Did not always converge for illconditioned problems Always faster for general problems on the less balanced platform when converging Slightly better absolute error norm for wellconditioned problems

21 Ongoing/Future Work New implementations for all transpose and sign-variants of the non-generalized standard matrix equations Software package SCASY will also contain generalized solvers and parallel condition estimators Ongoing investigation of hybrid algorithms with fast kernels from HPC library RECSY op ( A) X ± Xop( = C op( A) X + Xop( A ) = C op ( A) Xop( ± X = C op( A) Xop( A ) X = C op( A) X ± Yop( = C, op( D) X ± Yop( E) = F op ( A) Xop( ± op( D) Xop( E) = C op( A) Xop( A ) op( E) Xop( E ) = C SYC LYC SYD LYD GCSY GSYL GLYC op( A) X ( E ) + op( E) Xop( A ) = C GLYD

22 References [] R.H. Bartels and G.W. Stewart. Algorithm 432: Solution of the Equation AX+XB = C, Comm. ACM, 5(9):82-826, 972. [2] P. Benner, E.S. Quintana-Orti. Solving Stable Generalized Lyapunov Equations with the matrix sign functions, Numerical Algorithms, 2(), pp. 75-, 999. [3] P. Benner, E.S. Quintana-Orti, G. Ouintana-Orti. Numerical Solution of Discrete Schur Stable Linear Matrix Equations on Multicomputers, Parallel Alg. Appl., Vol. 7, No, pp , 22. [4] R. Granat, B. Kågström, P. Poromaa. Parallel ScaLAPACK-style Algorithms for Solving Continuous-ime Sylvester Equations. In H. Kosch et al. (eds), Euro-Par 23 Parallel Processing. Lecture Notes in Computer Science, Vol. 279, pp. 8-89, 23. [5] W.W. Hager, Condition Estimates, SIAM J. Sci. Statist. Comput. 5, pp. 3-36,984. [6] N. J. Higham, FORAN codes for Estimating the One-Norm of a Real of Complex Matrix, ACM rans. Of Math. Software, Vol. 4, No. 4, pp , December 988 [7] I. Jonsson and B. Kågström, Recursive Blocked Algorithms for Solving riangular Matrix Equations - Part I: One-Sided and Coupled Sylvester-ype Equations, ACM rans. Math. Software, Vol. 28, No. 4, pp , 22. [8] I. Jonsson and B. Kågström, Recursive Blocked Algorithms for Solving riangular Matrix Equations - Part II:wo-Sided and Generalized Sylvester and Lyapunov Equations, ACM rans. Math. Software, Vol. 28, No. 4, pp , 22. [9] B. Kågström and P. Poromaa, Distributed and shared memory block algorithms for the triangular Sylvester Equation with Sep estimators, SIAM J. Matrix Anal. Appl., 3 (992), pp [] P. Poromaa. Parallel Algorithms for riangular Sylvester Equations: Design, Scheduling and Scalability Issues. In Kågström et al. (eds), Applied Parallel Computing. Large Scale Scientific and Industrial Problems, Lecture Notes in Computer Science, Vol. 54, pp , Springer- Verlag, 998. [] J.D. Roberts. Linear model reduction and solution of the algebraic Ricatti Equation by use of the sign function, Intern. J. Control, 32: , 98.