Good Morning.  I’m Tina Kapur. I’m a graduate student here at  MIT, at the  Artificial Intelligence  Lab.  I work with Eric Grimson, Sandy Wells, and Ron Kikinis.  Today I will present an overview of some of my thesis research.

The goal of the segmentation problem is to start with grayscale images such as this stack of slices in an MRI scan, and create labeled images from which 3D models such as this can be generated.

Let’s start by looking at an MRI scan and what we would like to segment from it. Here is a knee scan which has about 100 images in it that were acquired sagitally.  Typically knee MRI scans consist of 50-100 cross-sectional images that are acquired either sagitally or axially.

The belief that is underlying my work is that medical image segmentation is a task in which domain knowledge should be exploited as much as possible. Specifically, I think that three different kinds of knowledge are necessary for sucessful segmentation, in general.

The belief that is underlying my work is that medical image segmentation is a task in which domain knowledge should be exploited as much as possible. Specifically, I think that three different kinds of knowledge are necessary for sucessful segmentation, in general.

The belief that is underlying my work is that medical image segmentation is a task in which domain knowledge should be exploited as much as possible. Specifically, I think that three different kinds of knowledge are necessary for sucessful segmentation, in general.

For example, we need intensity models for the different tissue classes present in the image.  Typically these models are: - parametric eg. Gaussian; white matter in gradient echo brain MRI

For example, we need intensity models for the different tissue classes present in the image.  Typically these models are: - parametric eg. Gaussian; white matter in gradient echo brain MRI

Several of us have also discovered, that in many cases, intensity and imaging models are not sufficient for segmentation of the structures we are interested in. - sometimes we use tools from low lever computer vision: morphological opening and closing, and region-growing or connectivity analysis to encode topological or geometric information about anatomy.  Again Guido Gerig, Brummer, and our own group has used these tools often in brain isolation from MRI images.  Mike Brady has used them for chest-wall detection in breast MRI. -Cootes and Taylor use PCA to create shape models -- Active Shape Models -- to encode the variation in shape over a population. - Some of the talks earlier today refer to geometric anatomical info incorporation in the segmentation process

Several of us have also discovered, that in many cases, intensity and imaging models are not sufficient for segmentation of the structures we are interested in. - sometimes we use tools from low lever computer vision: morphological opening and closing, and region-growing or connectivity analysis to encode topological or geometric information about anatomy.  Again Guido Gerig, Brummer, and our own group has used these tools often in brain isolation from MRI images.  Mike Brady has used them for chest-wall detection in breast MRI. -Cootes and Taylor use PCA to create shape models -- Active Shape Models -- to encode the variation in shape over a population. - Some of the talks earlier today refer to geometric anatomical info incorporation in the segmentation process

Several of us have also discovered, that in many cases, intensity and imaging models are not sufficient for segmentation of the structures we are interested in. - sometimes we use tools from low lever computer vision: morphological opening and closing, and region-growing or connectivity analysis to encode topological or geometric information about anatomy.  Again Guido Gerig, Brummer, and our own group has used these tools often in brain isolation from MRI images.  Mike Brady has used them for chest-wall detection in breast MRI. -Cootes and Taylor use PCA to create shape models -- Active Shape Models -- to encode the variation in shape over a population. - Some of the talks earlier today refer to geometric anatomical info incorporation in the segmentation process

Several of us have also discovered, that in many cases, intensity and imaging models are not sufficient for segmentation of the structures we are interested in. - sometimes we use tools from low lever computer vision: morphological opening and closing, and region-growing or connectivity analysis to encode topological or geometric information about anatomy.  Again Guido Gerig, Brummer, and our own group has used these tools often in brain isolation from MRI images.  Mike Brady has used them for chest-wall detection in breast MRI. -Cootes and Taylor use PCA to create shape models -- Active Shape Models -- to encode the variation in shape over a population. - Some of the talks earlier today refer to geometric anatomical info incorporation in the segmentation process

The pipeline in which it is used at the Brigham starts with a preprocssing step for noise removal, then a run of the EM segmenter to get a classification of the image into typically four tissue classes (white, gray, csf, bg), and then a post-processing step which may be morpholigcal opening/closing+connectivity for brain isolation, or semi-automatic outlining of structures such as the caudate or thalamus.

The pipeline in which it is used at the Brigham starts with a preprocssing step for noise removal, then a run of the EM segmenter to get a classification of the image into typically four tissue classes (white, gray, csf, bg), and then a post-processing step which may be morpholigcal opening/closing+connectivity for brain isolation, or semi-automatic outlining of structures such as the caudate or thalamus.

The pipeline in which it is used at the Brigham starts with a preprocssing step for noise removal, then a run of the EM segmenter to get a classification of the image into typically four tissue classes (white, gray, csf, bg), and then a post-processing step which may be morpholigcal opening/closing+connectivity for brain isolation, or semi-automatic outlining of structures such as the caudate or thalamus.

Given a novel scan, segmenting it involves a pixelwise classification of all the images.  The classes or labels that we are interested in are: bone (femur, tibia, patella, fibula), cartilage, ligaments, tendons, menisci,  and muscles. What we have developed so far is a system that segments femur, tibia, femoral cartilage and tibial cartilage.

Motivation for this approach to segmentation is fairly simple, and can be thought of as "coarse-to-fine" in feature space. Some structures in the knee (such as the femur and the tibia) are  large and have a fairly unique response to low-level processing techniques.  While other structures (such as cartilage) are better discernable in context of other structures like bone, so we use a “model” to detect those after we have detected the bones.

Motivation: bootstrap.

We define the  two largest bones, the femur and the tibia, as the two features to be detected using low-level processing of the image. Feature detection is done in three steps.  First, a seed for each feature is placed in the image.  Second, the seed region is grown adaptively.  And finally, the boundary of the grown region is localized using regularizing constraints.

We observed that the spatial relationship between the femur and femoral cartilage seems to be well described by two parameters: Going along with this assumption, we compute the joint probability density of these two quantities (d & n) for femoral cartilage. That is our statistical “model” for femoral cartilage.  It is a spatial prior for cartilage, when the location of the bone is known.

We observed that the spatial relationship between the femur and femoral cartilage seems to be well described by two parameters: Going along with this assumption, we compute the joint probability density of these two quantities (d & n) for femoral cartilage. That is our statistical “model” for femoral cartilage.  It is a spatial prior for cartilage, when the location of the bone is known.

Motivation: bootstrap.

This work indicates that spatial priors are a natural mechanism for introducing additional information into the segmentation process.

This work indicates that spatial priors are a natural mechanism for introducing additional information into the segmentation process.

What I would like to propose today, is a method for incorporating the pre- as well as post-processing steps into the EM Segmentation framework.

The first prior is a local spatial prior which is a Markov Random Field with a Mean-Field solver.  It models the piece-wise homegeneity of tissue, which helps reduce noise. The second prior model  is a relative geometric prior that models the spatial relationships between structures.  For eg. It can be used to model the fact that gray matter is outside ventricles and inside the skull.

An MRF is a probability model on a lattice, where the standard independence assumption is relaxed in such a way that the spatial range over which sites interact, in the sense of conditional probability, is limited to a local neighborhood.

An MRF is a probability model on a lattice, where the standard independence assumption is relaxed in such a way that the spatial range over which sites interact, in the sense of conditional probability, is limited to a local neighborhood.