High-Level Perceptual Contours From A Variety Of Low-Level Physical Features

Brian Scassellati

The Problem:

Extraction of robust contour information from real-world images using many different visual modalities such as disparity, luminance, color, and texture.

Motivation:

For years, researchers in machine vision have focused on extracting object boundary information from luminance derivatives, color contrast, depth information, and textural patterns. However, no single one of these methods is sufficient for detecting all of the contour information that humans perceive. Humans have little difficulty discriminating depth-based contours from the fused images in a stereogram, extracting illusory contours from a Kanizsa square, or even in completing contours through the blind spot.

This project reformulates the problem of “edge detection” into the recognition of high-level perceptual features which we call “contours”. It involves the development of a novel visual processing framework that deals with contours as high-level image features. We will demonstrate the operation of this new model on real-world images using a prototype active vision head designed to support this visual architecture.

Previous Work:

The problem of contour detection has been focused on by a variety of machine vision researchers. The best known luminance-based contour detectors arose from the work of John Canny [1], while spectral-based contour detection was the focus of Mollon [2]. The psychophysical aspects of texture were first explored by Julesz [3], and followed by the computational work of Bergen and Landy [4], Malik and Perona [5], and Voorhees and Poggio [6]. The use of disparity as a contour detector was first explored by psychophysicists with the use of random-dot stereograms and later Marr, Poggio, and Grimson investigated the computational aspects [7, 8].

The psychophysics of contour identification has been explored for centuries [9]. However, the most relevant work to this project concerns the work of Gaetano Kanizsa and the Gestalt psychologists. Kanizsa demonstrated a large class of illusory contours that were psychologically highly salient, yet could not be identified using luminance, color, or disparity information [10]. The Kanizsa square and triangle are perhaps the most widely known examples of these phenomena.

Approach:

Our model of contour processing allows for the integration of independent low-level edge detection modules (such as luminance and texture) with other higher-level sources of information (such as Gestalt effects and expectations). The central idea behind the integrated contour model is the proposition that complex, highlevel perceptions can be created by using simple compositions of low-level routines coupled with input from other high-level processes.

The model itself is relatively simple: allow for each of the low-level edge detectors to operate independently, accept information from higher level processes, and combine all of the information at a single integration point. The integrated contour model also allows for the results from the integration to be passed back to each of the sub-modules. In this way, building a contour is an iterative process that can rely on multiple stages of processing; that is, the integration box does not perform one-shot processing on an input image, but rather allows for successive refinements in the contour image produced.

We are currently constructing an active vision platform to allow for continuous real-time testing of the model presented above. The system will perform simple eye-tracking for identified contours.

Impact:

The robust object segmentation available through this method will allow for better continuous-time processing of visual input for robotic systems. The difficulties of dealing with real-world, noisy images are mediated by the presence of multiple, overlapping detectors that combine positively to create a single unified mapping. Robot navigation, automated contour processing for medical and military applications, and automated visual searches for large image databases should all profit from this approach. References:

[1] John F. Canny. Finding edges and lines in images. Memo 720, Massachusetts Institute of Technology Artificial Intelligence Laboratory, Cambridge, Massachusetts, June 1983.

[2] J. D. Mollon and L. T. Sharpe, editors. Color Vision: Physiology and Psychophysics. Academic Press, 1983.

[3] Bela Julesz. Experiments in the visual perception of texture. Scientific American, 232:34–43, April 1975.

[4] M. S. Landy and J. R. Bergen. Texture segregation and orientation gradient. Vision Research, 31, 1991.

[5] Jitendra Malik and Pietro Perona. Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America, 7, 1990.

[6] H. Voorhees and T. Poggio. Computing texture boundaries from images. Nature, 333, 1988.

[7] T. Poggio and D. Marr. Cooperative computation of stereo disparity. Science, 194:283–287, 1976.

[8] W.E.L. Grimson. From Images to Surfaces. MIT Press, 1981.

[9] Susan Petry and Glenn E. Meyer, editors. The Perception of Illusory Contours. Springer-Verlag, 1987.

[10] Gaetano Kanizsa. Organization in Vision: Essays on Gestalt Perception. Praeger Publishers, 1979.