Notes

Outline

Knowledge Based 3D  Medical Image Segmentation Tina Kapur MIT Artificial Intelligence Laboratory http://www.ai.mit.edu/~tkapur

Outline Goal of Segmentation Applications Why is segmentation difficult? My method for segmentation of MRI Future Work

The Goal of Segmentation

The Goal of Segmentation

Applications of Segmentation Image Guided Surgery

Applications of Segmentation Image Guided Surgery

Applications of Segmentation Image Guided Surgery Surgical Simulation

Applications of Segmentation Image Guided Surgery Surgical Simulation

Applications of Segmentation Image Guided Surgery Surgical Simulation Neuroscience Studies Therapy Evaluation

Limitations of Manual Segmentation slow (up to 60 hours per scan) variable (up to 15% between experts) [Warfield 95, Kaus98]

The Automatic Segmentation Challenge An automated segmentation method needs to reconcile Gray-level appearance of tissue Characteristics of imaging modality Geometry of anatomy

How to Segment? i.e. Issues in Segmentation of Anatomy

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models Parametric [Vannier] Non-Parametric [Gerig] Point distribution Models [Cootes] Texture [Mumford]

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models Imaging Modality Models

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models Imaging Modality Models MRI inhomogeneity [Wells]

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models Imaging Modality Models Anatomy Models: Shape, Geometric/Spatial

How to Segment? i.e. Issues in Segmentation of Anatomy Tissue Intensity Models Imaging Modality Models Anatomy Models: Shape, Geometric/Spatial PCA [Cootes and Taylor, Gerig, Duncan, Martin] Landmark Based [Evans] Atlas [Warfield]

Typical Pipeline for Segmentation of Brain MRI

Typical Pipeline for Segmentation of Brain MRI

Typical Pipeline for Segmentation of Brain MRI

Contributions of Thesis Developed an integrated Bayesian Segmentation Method for MRI that incorporates de-noising and global geometric knowledge using priors into EM-Segmentation Applied integrated Bayesian method to segmentation of Brain and Knee MRI.

Contributions of Thesis The Priors de-noising: novel use of a Mean-Field Approximation to a Gibbs random field in conjunction with EM-Segmentation (EM-MF) geometric: novel statistical description of global spatial relationships between structures, used as a spatially varying prior in EM-Segmentation

Background to My Work Expectation-Maximization Algorithm EM-Segmentation

Expectation-Maximization Relevant Literature: [Dempster, Laird, Rubin 1977] [Neal 1998]

Expectation-Maximization (what?) Search Algorithm for Parameters of a Model to Maximize Likelihood of Data Data: some observed, some unobserved

Expectation-Maximization (how?) Initial Guess of Model Parameters Re-estimate Model Parameters: E Step: compute PDF for hidden variables, given observations and current model parameters M Step: compute ML model parameters assuming pdf for hidden variables is correct

Expectation-Maximization (how exactly?) Notation Observed Variables: Hidden Variables : Model Parameters:

Expectation-Maximization (how exactly?) Initial Guess: Successive Estimation of E Step: M Step:

Expectation-Maximization Summary/Intuition: If we had complete data, maximize likelihood Since some data is missing, approximate likelihood with its expectation Converges to local maximum of likelihood

EM-Segmentation [Wells 1994] Observed Signal is modeled as a product of the true signal and a corrupting gain field due to the imaging equipment Expectation-Maximization is used on log-transformed observations for iterative estimation of tissue classification corrupting bias field (inhomogeneity correction)

Slide 32

Slide 33

EM-Segmentation [Wells 1994] Observed Variables log transformed intensities in image Hidden Variables indicator variables for classification Model Parameters the slowly varying corrupting bias field (                      refer to variables at voxel s in image)

EM-Segmentation [Wells 1994] Initial Guess: Successive Estimation of E Step: M Step:

EM-Segmentation [Wells 1994] Initial Guess: Successive Estimation of E Step: M Step:

Situating My Work Prior in EM-Segmentation: Independent and Spatially Stationary My contribution is addition of two priors: a spatially stationary Gibbs prior to model local interactions between neighbors (thermal noise) spatially varying prior to model global relationships between geometry of structures

The Gibbs Prior Gibbs Random Field (GRF) natural way to model piecewise homogeneous phenomena used in image restoration [Geman&Geman 84] Probability Model on a lattice Partially Relaxes independence assumption to allow interactions between neighbors

EM-MF Segmentation: EM + Gibbs Prior We model tissue classification W as a Gibbs random field:

EM-MF Segmentation: Gibbs Prior on Classification We model tissue classification W as a Gibbs random field:

EM-MF Segmentation: Gibbs Prior on Classification To fully specify the Gibbs model: define neighborhood system         as a first order neighborhood system i.e. 6 closest voxels use       to define

EM-MF Segmentation: Gibbs form of Posterior Gibbs prior and Gaussian Measurement Models lead to Gibbs form for Posterior:

EM-MF Segmentation: Gibbs form of Posterior Gibbs prior and Gaussian Measurement Models lead to Gibbs form for Posterior:

EM-MF Segmentation For E-Step: Need values for

EM-MF Segmentation For E-Step: Need values for Cannot compute directly from Gibbs form

EM-MF Segmentation For E-Step: Need values for Cannot compute directly from Gibbs form Note

EM-MF Segmentation For E-Step: Need values for Cannot compute directly from Gibbs form Note Can approximate      Mean-Field Approximation to GRF

Mean-Field Approximation Deterministic Approximation to GRF [Parisi84] the mean/expected value of a GRF is obtained as a solution to a set of consistency equations Update Equation is obtained using derivative of partition function with respect to the external field g. [Elfadel 93] Used in image reconstruction [Geiger, Yuille, Girosi 91]

Mean-Field Approximation to Posterior GRF

Summary of EM-MF Segmentation Modeled piecewise homogeneity of tissue using a Gibbs prior on classification Lead to Gibbs form for Posteriors Posterior Probabilities in E-Step are approximated as a Mean-Field solution

EM-MF Results Application: Brain MRI white matter, gray matter, fluid/air, skin/scalp Results Comparison with Manual Segmentation

Some Results

Some Results

More Results

Posterior Probabilities (EM)

Posterior Probabilities (EM-MF)

Results

Modeling Global Geometric Relationships between Structures

Modeling Global Geometric Relationships between Structures Relative Geometry Models Motivate Using Knee MRI Brain MRI Example

Segmented Knee MRI

Motivation Primary Structures image well easy to segment Secondary Structures image poorly relative to primary

Relative Geometric Prior Approach Select primary/secondary structures Measure geometric relation between primary and secondary structures from training data Given novel image segment primary structures use geometric relation as prior on secondary structure in EM-MF Segmentation

Segment Primary Structures:  Femur, Tibia

Status Have Bone Want Cartilage

Measure Geometric Relationship between Primary and Secondary Structures Using primitives such as distances between surfaces local normals of primary structures local curvature of primary structures etc.

Measure Geometric Relationship between Primary and Secondary Structures

Estimate of

Status Have Bone Have spatial relation between Bone and Cartilage Need Cartilage

Use Relative Geometric Prior in EM Segmentation Replace stationary prior with relative geometric prior:

Results: Segmentation of Femoral & Tibial Cartilage

Relative Geometric Priors for Brain Tissue Prior Estimation Select primary structures (boundary of skin, ventricles) Estimate Using Prior in Segmentation Segment primary structures: skin, ventricles Use                                                       as geometric prior

Estimate

Resultant Segmentation

Posterior Probabilities

In Summary Incorporated robustness to thermal noise by using Mean-Field Approximation to Gibbs model in conjunction with EM Segmentation.  Applied to Brain MRI. Introduced Relative-Geometry Models and applied to Brain and Knee MRI.

Future Work Further development of Relative-Geometry Models: Automatic selection of primary/secondary structures Additional primitives for Spatial Relationships

STOP

Class Conditional Density

Unified Bayesian Segmentation Method

Rest of Talk: 1. The Unified Segmentation Method 2. Two Priors 3. Results on Brain, Knee Segmentation 4. Conclusions

The Gibbs Prior To fully specify the Gibbs model: define neighborhood system         as a first order neighborhood system i.e. 6 closest voxels use       to define

Components of EM Framework Measurement models for tissue Prior models for tissue Model for bias field piecewise smooth

Addition of Two Priors

Gibbs Model

Gibbs Model probability model on  a lattice independence assumption is partially relaxed spatial range of interaction is local neighborhood

Slide 86

Gibbs Model probability model on  a lattice independence assumption is partially relaxed spatial range of interaction is local neighborhood

Contributions of Thesis

Proposed MRI Segmentation Method Bayesian Statistical Classification Scheme that uses Expectation-Maximization Replaces pipeline with Priors on intensity and geometry

Proposed MRI Segmentation Method Previous Work [Wells 1994, 1996] derived as a special case spatially stationary, independent priors piecewise smooth inhomogeneity model This Work: locally interacting prior for intensity spatially varying prior for geometry

Next Background on Expectation-Maximization

Computation of 1. Use Bayes’ rule, and independence between     and      to write: 2. Specify Measurement Models                            as Gaussian

Computation of 3. Specify Prior on Classification           as a Gibbs Random Field. 4. Gibbs Prior + Gaussian Measurement Model imply                 is also a GRF. 5. Approximate                         using Mean-Field Solution for the GRF.

Computation of Recap:    1. Bayes’ Rule to rewrite …    2. Gaussian Measurement Models Ö 3. Gibbs form of Prior Ö 4. Gibbs form of Posterior Ö 5. Mean-Field Approximation to Gibbs form

The Gibbs Prior Tissue-class interaction Matrix Stationary Prior

The Mean-Field Solution Gibbs models can be solved using [Metropolis 1953], [Geman and Geman 1984], [Besag 1986]  etc. We use Mean-Field Approximation to estimate the expected value of the posterior GRF as a solution to a set of consistency equations.

The Mean-Field Approximation Deterministic Approximation Update Equation is obtained using derivative of partition function with respect to the external field g.

The Mean-Field Solution

EM-MF Summary Initial Guess: Successive Estimation of E Step:  Estimate                                as Mean-Field Solution to a Gibbs Random Field M Step:  Compute           same as [Wells 1996]