Industrial Problem Solving Workshop on Medical Imaging. Problem # 2 Rapid Modeling of Internal Structures of Deformable Organs (i.e.

Transkript

1 Industrial Problem Solving Workshop on Medical Imaging Problem # 2 Rapid Modeling of Internal Structures of Deformable Organs (i.e. Liver) Edward Xishi Huang and James Drake The Hospital for Sick Children

2 Problem definition Accurate estimation of deformation of the soft organ s internal structures between two imagesacquired acquired at different conditions What information are used?! Point landmarks [1] Vessel segments and curves [1] Surface information

3 Problem definition Application: Tumor treatment using high intensity focused ultrasound (HIFU) thermalprocedures procedures. Real time image guidance is required for targeting. Intraoperative imaging systems: Low resolution and quality images Long acquisition time Compatibility with other equipments in the operation room. Solution: Updating preoperative treatment plans and physical deformation models based on intraoperatively acquired images. The idea: Employing boundary conditions (i.e. curves of vessels and bifurcations of vessels, landmarks, and surface information ) to constrain the solution of the deformable model of organ.

4 Problem definition Registering internal structures and surface information of the two set of images acquired from the soft organ. Image Registration Given a reference image R and a template image T, find a reasonable transformation y, so that the transformed image T[y] is similar to R [2].

5 Problem 1 Modeling internal structures in 2D images using landmarks and vessel segments with known end points C C Sx ( ) B D S ( x ) B A 1 2 A D

6 Problem 1 Solution ( ) ( ) Landmarks: d f l x, x and x, y for l = 1,2,..., n l l l l Vessel Curves: y = S ( x ) and y = S ( x ) Displacement Function: ( uxy (, ), vxy (, ) ) Landmarks: ( x, y ) ( x + u( x, y ), y + v( x, y )) l l l l l l l l Any point on the curve: ( x, y ) ( x+ u( x, y), y+ v( x, y) )

7 Problem 1 Solution The cost function to be minimized i i dfor landmarks matching: n {( ) 2 ( ) 2 } l l l l l l l l J (, ) (, ) (, ) 1 u v = x x u x y + y y v x y l=1 The cost function to be minimized for curve matching: ( ) ( xsx, ( ) ) x + uxsx (, ( ) ), Sx ( ) + vxsx (, ( ) ) This point corresponds to ( ( ) ) ( x, S( x) ) x+ u( x, S( x) ), S x+ u( x, S( x) ) x 2 1 ( ( )) ( ) J (, ), ( ) ( ), ( ) 2 uv = S x+ u xsx Sx v xsx dx x 2

8 Problem 1 Solution The total cost function to be minimized i i dfor landmarks and curve matching: Juv (, ) = wj( uv, ) + wj( uv, ) Having the variational model in optimization model form: u = Ax+ By + x 0 + a i ϕ i ( x, y) 0 N i= 1 N v= Cx+ Dy+ y + bϕ ( x, y) ϕ i ( i = 1,2,..., N) are radial basis functions. The cost function in terms of unknown parameters i= 1 ( ) min J A, B, C, D, x, y, a,..., a, b,..., b N 1 i i N

9 Problem 2 Modeling internal structures in 2D images using landmarks and vessel segments withone end point. 9 8 Sx ( ) B ( x, y ) e e S ( x ) ( x, y ) e e 4 3 A ( x 1, y 1 ) A A 1 B?! 4 3 ( x, y ) 1 1 B 1 A

10 Problem 2 Solution ( xe, ye) is unknown point to be found and needs to be added d to the optimization function as follows: 2 ( ( )) ( ) ( ) 2 2 J ( x, u, v) = x x u x, S( x ) + y y v x, S( x ) e e e e e e e e e xe ( (, ( ) ) ) ( ) (, ( ) ) + S x + uxsx Sx vxsx dx x 1 2

11 Problem 3 Modeling internal structures in 3D image Landmarks: The same as 2D images J ( u, v, w ) = 1 n { ( x x u( x, y, z )) 2 + ( y y v( x, y, z ) 2 + ( z z v( x, y, z ) 2 } l l l l l l l l l l l l l l l l= 1 Curves: Parametric cubic representation: piecewise spline x t a t b t c t d 3 2 () = x + x + x + x yt () = at + bt + ct + d 3 2 y y y y zt () = at + bt + ct + d 3 2 z z z z

12 Problem 4 Registration based on surface matching of the object in two set of images ( xysxy,, (, )) ( x, y, S ( x, y ) ) ( uxyz (,, ), vxyz (,, ), wxyz (,, ) )

13 Problem 4 The cost function to be minimized: ( xysxy,, (, )) ( x, y, S ( x, y )) Juvw (,, ) = Ω ( ( ) ( )) ( ) S x+ u x, y, S ( xy, ), y+ vxys,, ( xy, ) S ( xy, ) wxys,, ( xy, ) dx dy Another solution: deformable surface modeling using active contours [3] 2

14 Impact FEM based image registration. Intensity based d[4] 1 min ( ), T Ju = DTu ( [ ] R) + α uku; α R+ 2 Multimodality Local minima Ju ( ) u [ ] (, R) DTu = + α Ku = u [ ] (, R) 1 DTu Ku = f ( u) = α u 0

15 Impact FEM based image registration. Feature based Internal structures and surface information of the deformable organprovides necessary boundary conditiontoto solve the FEM based deformation equation. Example: Linear finite element model of the brain shift and deformation [3].

16 FEM based image registration. Impact

17 References [1] Dietlind Zühlke, Sven Arnold, Gernoth Grunst, Peter Wißkirchen, Intra interventional registration of 3Dultrasound to models of the vascular system of the liver GMS CURAC 2007, Vol. 2(1) [2] J. Modersitzki, Numerical Methods for Image Registration. New York:Oxford, o [3] M. Ferrant, A. Nabavi, B. Macq, F. Jolesz, R. Kikinis, and S. Warfield, Registration of 3D Intraoperative MR Images of the Brain Using a Finite Element Biomechanical Model, IEEE Trans. on Medical Imaging, g, vol. 20, no. 12, pp , [4] B. Marami, S. Sirouspour, and D. Capson, Model Based Deformable Registration of Preoperative 3D to Intraoperative Low Resolution 3D and 2D Sequences of MR Images, in MICCAI 2011, pp , 2011.

18 Group Members Edward Xishi Huang C. Sean Bohun Tian Chen Jamil Jabbour Ken Jackson Iain Moyles Cartihk Sharma Yongji Tan Bahram Marami

19 Blah Motivation Modelling a three-dimensional image of the liver is computationally expensive (and has challenging physics) Image data arrives in several two-dimensional cross sections (slices) as the body is scanned Each slice is identified by unique markers (landmarks and curves) If all of the markers from one slice map to all be on a transformed slice then we can consider a series of two-dimensional transformations

20 Blah Setup Landmarks The N distinguishing points on image Curves The κ distinguishing curves (vessels) on image Assume a transformation from pre-image (X, Y ) to post-image ( X, Ỹ ) via a transformation ( X, Ȳ ) X = X + j a j φ j (X, Y ), Ȳ = Y + j b j φ j (X, Y ), φ j are radial basis functions (RBF) φ j (X, Y ) = exp ( (X X j) 2 + (Y Y j ) 2 ) σ (X j, Y j ) are the L allocation points to form a basis set (L N)

21 Blah Setup For curves we assume there exists a mapping for curve points on the pre-image (xx, yy) given by yy = S(xx) Similarly there exists a map yy = S( xx) for the curve points on the post-image ( xx, yy) Often the form of S and S will be through an interpolation (cubic splines) The curves have the requirement that the endpoints are landmarks

22 Blah Pre Image

23 Blah Post Image

24 Blah Optimization We wish to determine the coefficients for the RBF that transforms the pre-image into the post-image We determine these by minimizing an error function composed of matching landmarks and curves Therefore we consider 1 Landmark Error We wish all terms of the form X X to be small so that the landmarks are close 2 Curve Error We wish the curve points to be close as well i.e. xx xx and S( xx) S(xx) are small The predicted curve S(xx) = S(xx) + j b jφ j (xx, S(xx))

25 Blah Solution

26 Blah Oops

27 Blah Constrained Optimization There are many local minima to this problem with large energies In fact, there are many local minima with small energies that don t represent the true solution (lack of global minimizer?) Idea: Build in a constraint for each curve that forces the area under the post-image curve to match that of the predicted curve xx2 xx 1 ( S( xx) S(xx) j b j φ j ) dxx = 0

28 Blah Solution

29 Blah Conclusions Created image data Performed an optimization to recover RBF coefficients that transform pre-image to post-image Included integral constraints to reduce set of minimizers Future Work Apply to real landmark data Consider weightings carefully Center of mass type penalty system Voronoi diagrams Allow for curve endpoints to move