\documentstyle[12pt]{article} \begin{document} \begin{center} \Large Planning for the Future\\ of the IUE\\\ \\ \large Original Draft by:\\ {\em J.L. Mundy\\} {\em Chair: IUE Technical Committee} \\\ \\ Revised by:\\ {\em W.E.L. Grimson\\} {\em Chair: IUE Advisory Board} \\\ \\ Draft 2.1\\ \today \end{center} \section{Introduction} \subsection{Purpose} Now that Version 1.0 of the IUE Core system is nearing completion, it is useful to identify a number of stages for the future evolution of the IUE. The purpose of this document is to identify the main issues associated with the IUE release and areas where future development is needed to enhance its success. \subsection{Overview} The goals of the IUE are: \begin{enumerate} \item provide a common software environment for image understanding research and development; \item provide a standard format for data and computational structures to facilitate the exchange of datasets and algorithms; \item provide a common demonstration platform to facilitate technology transfer; \item provide a mechanism for system integration, both within single research sites, and across sites. \end{enumerate} Of these goals, the last is perhaps of lesser importance than the first three, although it is still of relevance to considerations of the future health of the IUE. The IUE was designed by the IUE Technical Committee and is being implemented by Amerinex AI. The IUE is an object-oriented system implemented in C++ which is partially generated directly from the LaTex IUE class specification. The implementation is based on the goal of using free, public domain software where possible. The initial release of the IUE is targeted at the UNIX operating system, using LINUX to support PC hardware. It is planned to support the major UNIX hardware platforms, such as SUN, SiliconGraphics and HP. Currently, SUNOS, Solaris, and LINUX are supported by AAI. The IUE is being developed using the gnu C++ compiler, gcc version 2.7.0 and libg++ version 2.7.0a. In addition to it being free, the decision to use gcc is based on its broad availability across platforms and its relatively good compliance with the current ANSI C++ standard. The IUE is based on the C++ Standard Template Library (STL) which is a set of base classes, such as list and set. STL is now part of the C++ standard and provides a style of iteration which is used in IUE container classes. The IUE provides an ASCII data exchange format called DEX (Data EXchange). DEX encodes C++ class attributes with a self-documenting format specification which is part of the DEX file. The IUE provides DEX readers and writers for each IUE class. A library of C++ support utilities for reading and writing DEX format is also available. \section{User Models} \label{sec:user-modes} In order to plan for the dissemination and evolution of the IUE, it is important to identify the likely modes of utilization of the IUE and the requirements which must be met to effectively support these user modes. In particular, it is important to consider what additional elements of the IUE are necessary to gain wide usage and acceptance by the user community. It is the view of the Advisory Board that wide spread usage by a range of IU sites is essential to the future growth of the IUE. A (non-exhaustive) list of important user models is as follows. The user models include both applications as well as focused research activities. \subsection{Image Display and Image Processing} This mode is similar to that provided by XV or by KHOROS. On the one hand, there is no strong motivation to provide yet another implementation of this mode of use. On the other hand, it is such a common mode that an IUE user would want such operations in conjunction with data browsing and educational uses. This mode should support chaining a series of image processing stages, including user-coded processes. Furthermore, there should be a relatively seamless interface between this component and the current structures of the IUE. There are many applications of IU work that require the ability on the part of the programmer to gain efficient access to primitive image elements (such as pixels) with minimum overhead. The lack of a seamless interface stands as a potential barrier to wide acceptance of the IUE within this important part of the user community. \subsection{2D Image Feature Extraction} This user mode introduces the use of the {\em spatial-object} which is the root of the central portion of the IUE. The spatial object hierarchy encodes the geometric and topological attributes and operations which are at the center of image understanding algorithms. It seems feasible to restrict the use to 2D geometry where research is focused on the extraction of image features and the development of feature grouping algorithms. It is very important, however, to provide a GUI which supports image display with 2D spatial object overlays. In conjunction with such a GUI, new feature extraction algorithms can be built on the standard IUE image feature classes, and then easily compared with standard segmentation operations, such as the Canny edge detector. \subsection{3D Scene Modeling} Here, the 3D portions of the hierarchy are added along with various sensor models which define the projection of 3D objects into 2D. The 3D hierarchy involves additional classes of surfaces and curves which do not exist in 2D. Geographic coordinate systems and transforms are required. The GUI is more extensive than for 2D where manipulation and construction of 3D objects must be supported. This mode is similar to that used in RADIUS by RCDE for the manual construction of 3D site models. The main use is to produce site models and experiment with IU algorithms for 3D object construction. It is necessary to support NITF 2.0 in order to effectively work on military applications in this area. It is also the case that the IUE should support Photogrammetry mechanisms for the required sensors. Although this capability is not part of the core IUE, it is important to add it if this user mode, and especially if the photogrammetry community and its associated applications users, are to be widely supported by the IUE. \subsection{Image Exploitation} This application is very broad, but again extensive use of most of the IUE hierarchy is expected to incorporate scene context as well as various levels of recognition and matching algorithms. The main goal is to define change of military significance in an archive or incoming stream of intelligence imagery. Again NITF 2.0 and sensor photogrammetry support is essential. \subsection{Automatic Target Recognition (ATR)} ATR applications, as currently practiced, are centered around interpretation of targets in an image as primarily signal events therefore the emphasis is on image processing. Statistically-derived matched filters are extensively used. Some model-based simulation is used to predict the appearance of objects, particularly in SAR and limited use has been made of model-based recognition. More advanced forms of ATR algorithms are beginning to use spatial context provided by site models and therefore will make good use of the geometric and geographic aspects of the IUE. \subsection{Model-based object recognition} Model-based object recognition, particularly the recognition of 3D objects from 2D intensity images is perhaps the most demanding user mode for the IUE. A tight integration of 2D image-features, sensors and 3D object models is required. Most of the IUE class hierarchy is required to support state-of-the-art research in object recognition. In this mode it is also expected that the IUE would import object models from other modeling systems, such as CAD modelers or existing target databases. \subsection{Physics-Based Vision} The goal here is accurate graphics rendering of 3D scenes, including color. The more advanced classes and methods associated with the class {\bf Sensor} are required. The physical models of light scattering and the effects of optical systems are incorporated into a realistic simulation of image formation. Again, extensive use is made of the 3D spatial object class hierarchy and 3D coordinate systems and transforms. \subsection{DEX Browser} In this mode, the IUE is acting as a central data exchange format switch. This mode is patterned after the successful XV tool which has the goal of displaying and transforming a large number of image formats. In the DEX browser, this idea is extended to the exchange and display of 2D image features and 3D scene models. There is little processing done on the data items but a reasonable portion of IUE class hierarchy is required to import and construct various formats of 2D and 3D spatial data. There appear to be no cohesive formats for image features, such as edgels, so the IUE class format will serve as a target for users wishing to exchange such data. For 3D objects, it is necessary to import IGES and AutoCad's DXF as a minimum. Standard sensor model formats are just now emerging. The IUE follows the notation and semantics of the Photogrammetry handbook which is a good start. The DEX browser API should be consistent with HTML/Netscape interfaces so the browser can be used to support Internet data viewing. \vskip 20pt We do not expect that all of these user models will immediately be supported by the IUE, nor do we believe that this is an exhaustive list of potential users. Nonetheless, we feel it is very important that likely user models be considered in developing plans for the near term and long term future of the IUE. If none of these user groups finds the IUE to be an important leveraging tool for their research and development, then the longterm outlook for utility and thus survivability of the IUE is poor, in the view of the Advisory Board. \section{Programming Issues} \label{sec:programming} Beyond the user application models just described, it is necessary to define issues related to the IUE as a programming environment per se. In the rapidly-evolving universe of programming languages, Internet operability and graphics support, it is crucial that the IUE provides an up-to-date interface to these mechanisms and at the same time maintains a stable class library for IU algorithms. Planning for the future of the IUE must also take these issues into account. In this section issues related to the IUE as programming environment are discussed, particularly those issues which represent new extensions or new developments which affect the long-term evolution of the IUE. \subsection{Graphical User Interface} The recent explosion of Internet applications using new inter-operability mechanisms such as Netscape and Java now place requirements on the IUE which were not envisioned at the time the Spec was developed. Fortunately, the choice of FRESCO, an object-oriented GUI standard, for the IUE is likely to provide Java support and is also targeted to operate under the CORBA object broker standard. It remains to be seen if CORBA evolves to be a real object exchange standard. In any case, the original idea of the role of the GUI as a viewer for IU structures must now be extended to include Internet operability. \subsection{The C++ Language} It is clear that C++ has achieved dominance as the object-oriented language of choice for most new large object-oriented systems. At the same time, the language is still evolving including the relationship of the Standard Template Library(STL) to the C++ standard. It is still not the case that all C++ compilers fully handle, at least efficiently, the requirements of STL. Nevertheless, it was a good choice to base the IUE design on STL and to build the implementation on an existing implementations of STL, even though there are still serious limitations and bugs -- this is where C++ is headed. The IUE is committed to the use of gcc, a public domain C++ compiler. For a time, this compiler was as good as any under UNIX, if not the best available, in terms of supporting advanced features of the language, including support for templates. Now, there is some question about the continuing evolution of gcc. It is still our best choice, since it allows operability over a large number of hardware platforms. It may be the case that we will have limit the use of some C++ features or provide patches for gcc as part of the IUE release. There is also an issue in that many IU labs have not yet caught up with the commercial world in their use of C++. It is the case that there are many C programmers and new C implementations of IU algorithms being developed. At the same time most leaders of IU labs see that they should move to C++ and would appreciate a low-cost entry into C++ development. It is important that the IUE provide levels of access for both the experienced and new C++ programmer. Currently, it is necessary to be a fairly sophisticated programmer to work with the IUE. \subsection{IUE Partitioning} The IUE class hierarchy is large, and thus compiling and linking a full IUE is a lengthy process. Similarly, a full executable is also large. The use of shared libraries has reduced this problem considerably. However, for the IUE to be broadly accepted, it will be necessary to partition the class libraries so that programmers only need compile and link to reasonably related functionality. This issue was not a primary consideration in the design of the class hierarchy in the initial design, particularly in the choice of sharing pointers to object instances. By just passing pointers it is often possible to separate libraries which are initially interdependent. As the IUE moves towards broader use by the IU community, however, this issue becomes much more important. Initial attempts to use the IUE by a number of experienced programmers was met mostly by frustration with the apparently inherent slowness of compiling, loading and executing code. Thus, as we work on the next version of the IUE core, this issue should be a central consideration. The desired result is to be able to support the various user modes described in section~\ref{sec:user-modes} with small library partitions and executables when the desired functionality is also limited. \subsection{Memory Management} The IUE implementation does not currently support the reference counting and memory management mechanisms necessary for clean pointer-based programming styles and avoidance of memory leaks. Additional effort must be applied to defining and adapting the IUE to use these mechanisms. This iteration is a natural evolution in the maturation of the design. \subsection{Image Processing} \label{sec:image-processing} There has been some effort in 1995 to define a basic set of image processing operators which the IUE will support. Further work on the specification of these operators and the general design of how an operator interacts with the IUE image class is needed. This library is important to implement perhaps jointly with KHOROS, but resources have not permitted the attention this area deserves. There may be other avenues by which image processing mechanisms can be seamlessly integrated into the IUE, the key issue is that this needs to be addressed in the near future. \subsection{Hierarchy Evaluation} \label{sec:hierarchyeval} Many areas of the IUE use novel designs. The IUE Technical Committee provided these designs in order to give the IUE a rational and expressive hierarchy. When using new designs, however, it is crucial to evaluate the designs in use, and where necessary, revise them. Matricies, image features, and user extensibility in general, are three areas that could benefit from design review. Ideally such a review should occur after we have more experience using the existing hierarchy, but before the IUE is so widely used that interface changes become a significant burden. \subsection{Documentation} It has always been recognized that for the IUE to be broadly accepted it will be necessary to encode the practice of IU algorithm theory and evaluation in terms of the programming structures of the IUE. Right from the start, the documentation rigor of the IUE has been one of its strong points. Nonetheless, it is very important that the documentation be maintained in an up-to-date manner, especially given the size and complexity of the full system. Carrying forward to a well-thought-out set of examples in the IUE primer, and perhaps the long-planned IUE textbook, is also a key priority. \subsection{Independent Evaluation} The IUE Summer Camp program has been successful in encouraging the use and evaluation of the IUE by outside organizations. Originally presented as a six week course for six students, Summer Camp was repeated this year as a two week course for ten students, with a follow-up session to occur two months later. The follow-up session is intended to encourage the students to continue working with the IUE at their home sites and to ensure that the IUE committee receives their feedback. This program provides for direct feedback over a significant period of time in a face-to-face environment. This feedback is much more thorough than that gained through typical e-mail forums. This valuable tool should be continued throughout the development of the IUE. \section{Evolution of the IUE} \subsection{The ideal} How should the IUE evolve over the next five years? The original plan called for the IUE to be released and during 1996, at which point the IU community would work together to assemble IU algorithm libraries on top of the core. This is exactly the plan formulated by the UK in their effort to adopt the IUE in Britain. A central hub, the University of Manchester, will take in code from various UK research centers and adapt it to the IUE environment and then redistribute it to the participating users. Funding is identified to support this activity, both at Manchester and at the ``spokes''. There is no such organized plan, nor funding for such a plan in the US. In the current funding climate it does not seem likely that US IU researchers will have the resources to learn about and adopt the IUE in such a direct manner. Initial insertion will more likely happen at the research centers which are already participating in the design and development of the IUE, e.g. IUE committee members. For example, Terry Boult's lab and other groups at AAI are already developing in the IUE. For the long term health of the IUE, however, it is important that the IUE quickly reach a stage at which other user groups, without the vested interest and thus deep commitment to the IUE, can easily begin to use the IUE for development purposes. These means that the IUE development team must ascertain what constitutes a minimum threshold of acceptance, and then work to achieve that as quickly as possible. \subsection{Possible paths of development} There are at least two different models for how further IUE development can proceed. The first is to use applications to pull the future development of the IUE, the second is to perform more general tool building, and let that push the integration of the IUE into the broader IU community. The Advisory Board feels strongly that it is very important at this stage to the IUE to build strongs ties to the IU community beyond the Technical committee, and thus feels that it is probably best to pick a small number of applications areas, and build (perhaps together with the IU community) a base set of library routines needed to support these applications. Some examples of possible avenues are detailed below, but input from both DARPA and the Technical Committee on other possibilities should also be sought. \subsection{Co-Development With DARPA Programs} As noted, we believe that the IUE should be extended along lines which are appropriate to some set of applications. This model might be called the testbed model in analogy to the development of SRI's CME lisp-based environment, now called RCDE. Its development has been driven over the last few years primarily by requirements of RADIUS. Assuming that IUE funding and development will be structured around specific program requirements, two aspects can be identified: \begin{enumerate} \item enhancements and maintenance of the core IUE; \item application algorithms and interoperability application system architectural elements. \end{enumerate} Item 1 is probably best carried out by AAI, the original developer of the IUE. Some examples of type 1 work are given in section~\ref{sec:programming}. To maintain stability and configuration of the code it is essential that AAI continue to play a key role in developing and maintaining the IUE core. AAI is also well positioned to adapt the IUE to new application areas, but it is essential that other application contractors be used as well to disseminate the IUE and broaden the community skills in its use. For type 2 items it is assumed that funding will be earmarked from application programs to support the necessary developments and enhancements of the IUE to support the application requirements~\footnote{There is an existing model of this approach at GE CRD where a precursor to the IUE, TargetJr is maintained and developed in exactly this manner.}. The IUE Technical Committee has begun investigating whether an initial application of the IUE within MSTAR would be profitable to both the IUE and to MSTAR. This needs to be pursued aggressively. \section{MSTAR} \subsection{Overview} The MSTAR(Moving and Stationary Target Acquisition and Recognition) project is an extensive DARPA project to advance the state of the art in model-based ATR. The focus of the program is on SAR. The processing within MSTAR is divided into several stages: \begin{itemize} \item Modeling - Deriving suitable geometric models of ATR targets from a CAD model, selecting features such as corner reflectors. \item Focus of Attention - Using simple CFAR algorithms detect likely target indications. \item Indexing - Selecting an initial set of target hypotheses for each region of interest, prior to executing the PEMS loop (Predict, Extract, Match, Search). \item Prediction - Predicting the appearance of a target in SAR by modeling the scattering of electro-magnetic radiation. \item Extraction - Extracting features from input SAR imagry. \item Matching - Finding a match between predicted features from stored models of targets and extracted features from input SAR imagry. \item Search - Narrowing the search for targets and managing recognition hypotheses. \end{itemize} During the first quarter of 1996, AAI and Ross Beveridge of the IUE TC investigated the MSTAR project with the goal of finding a good fit to applying the IUE. After some discussion it has been decided to work on the Matching function within MSTAR. \subsection{The IUE and MSTAR Matching} There are two contractors currently working on the Matching function: TASC/MIT, and NYU/ERIM. The TASC/MIT algorithm is already written in C++ while the NYU/ERIM algorithm is written in a LISP derivative, SETL. The MIT algorithm is based on maximizing mutual information and the NYU algorithm exploits geometric hashing. The general idea is to integrate, both the MIT matching algorithm and the NYU algorithm into a common framework within the IUE. There are many common elements which can be shared and enhance the IUE class libraries and thus provide a unification of matching machinery. In this approach, the IUE development can be done in parallel and produce a more flexible and comprehensive matcher, perhaps combining the strengths of both. Ideally, the implementation of the IUE matcher would be carried out in close collaboration with the MSTAR matcher developers. Bob Hummel of NYU is enthusiastic and has offered a student to work with AAI to define the geometric hashing algorithm in terms of generic IUE structures. MIT is also willing to pursue methods for integrating their algorithm into the IUE, and Gil Ettinger of TASC should be approached directly about this. The rough timescale is that some initial implementation of these algorithms should be demonstrable by the end of 1996 to gain credibility for the IUE and to insure future evolution of the IUE implementation during the remainder of the MSTAR project. Thus, there is a clear need to push ahead in this direction as soon as possible. \section{The IUE and TargetJr} Another aspect of planning the evolution and development of the IUE is its relationship to TargetJr. \subsection{What is TargetJr?} TargetJr is a C++ software environment (~500K lines of code) for carrying out Image Understanding research and applications. TargetJr's class libraries have been developed over the last seven years with emphasis on a geometric approach to image analysis. The system was initiated at GE's Corporate Research and Development (CRD) with initial emphasis on photo-interpretation and X-ray image analysis. TargetJr has been used in research on object recognition based on geometric invariants under DARPA funding. TargetJr is also used in a project called PINPOINT which is developing a system for simulating weapon sensor-derived imagery. In this application, TargetJr plays a similar role as RCDE in RADIUS and provides site modeling and photogrammetry support. TargetJr also is used in RADIUS as an environment for developing and integrating C and C++ IU algorithm implementations. To support this algorithm integration a small and lightweight blackboard architecture has been implemented which includes an in-memory relational database to support queries to the blackboard. During 1996 the RADIUS applications developed in TargetJr are being applied to a internal R\&D project with Lockheed Martin Corporation to apply MSE concepts to the retrieval of images from an image archive. The project will use event triggers to select relevant images for use in intelligence analysis. The TargetJr libraries are divided into Packages, which are groups of class libraries with partitions to minimize cross-library dependency. Packages are available for the following general functions: \begin{itemize} \item[] {\bf Image} - Libraries to support reading and writing various image formats, e.g., PGM, TIFF, MIT etc., as well as image processing. \item[] {\bf Spatial Object} - Representation of geometry and topology including digital curves which are derived from image segmentation. \item[] {\bf Viewing} - Support for graphical display of entities and menu driven interaction. TargetJr's graphics is based on the Interviews class library. \item[] {\bf Photogrammetry} - Models for various types of image projection such as perspective and graphics display cameras. \item[] {\bf Segmentation} - Classes and methods for extracting geometric descriptions from an image. Includes various Canny edge detectors and curve fitting algorithms. \item[] {\bf Database} - A simple relational database implementation for storing in-memory temporary data to be shared between classes. The database also supports a blackboard architecture for encapsulating large-grained processes which communicate via database relations and queries. \item[] {\bf Morse} - A suite of classes to support research in invariant object representation. Included is a recognition system, called Lewis, which represents planar shapes using algebraic invariants. \item[] {\bf GeneralUtility} - General programming support including: \begin{itemize} \item[]{\bf DEX} - Several libraries to support ASCII import and export of TargetJr classes. \item[]{\bf Solver} - Various numerical methods for solving systems of equations, including support for polynomials. \item[]{\bf Stat} - Classes to support statistical analysis \item[]{\bf Basics} - Really basic classes, e.g. UNIX directory parsing, reference counting, ..etc. \end{itemize} \item[] {\bf COOL} - A templated container class library called C++ Object Oriented Library or COOL. COOL was originally developed the AI group at Texas Instruments in the late 1980's as a base for AI programming in C++. \end{itemize} \subsection{What does TargetJr have to do with the IUE?} Currently TargetJr is being used by a consortium of vision researchers in object recognition, including: Oxford University, University of Leuven(Belgium) and INRIA(France). These groups and GE-CRD as well plan to migrate this recognition system development to the IUE with the goal of incorporating TargetJr functionality into the IUE over the next several years. At the end of the transition period, TargetJr would be adapted completely within the IUE. \subsection{How will the transition be carried out?} During March 19-24 a meeting of this consortium was held on Martha's Vineyard. In attendance were 17 participants from the US and Europe. Representatives of AAI and the IUE TC were also present as well as a representative of the UK IUE consortium. The main focus of the meeting was to define a plan for the evolution of TargetJr towards the IUE. The plan which emerged is as follows: \begin{itemize} \item[]{\bf Phase I }(3Q 1996) - Replace the current TargetJr image classes with the IUE image classes. Work jointly with AAI to maintain consistency. \item[]{\bf Phase II }(1Q 1997) - Implement the FRESCO graphical user interface design to be used in the IUE and integrate and replace TargetJr's current Interviews GUI with IUE interface components. \item[]{\bf Phase III }(4Q 1997) - Replace the SpatialObject library in TargetJr with the IUE SpatialObject library. Emulate COOL with IUE base classes. \end{itemize} Some additional considerations which emerged from the Martha's Vineyard discussions were: \\ \paragraph{Image Processing}~~The University of Manchester has developed an extensive C++ library for 2D image processing and image geometry called RADIAL\footnote{Jeff Hunter, of AAI, was a key participant in the RADIAL project while he was at Manchester.}. As mentioned earlier, Manchester is also the hub for adapting vision research software in the UK to the IUE. One possibility is that the image processing library in RADIAL can be adapted to the IUE and to TargetJr as an initial phase of working with the IUE image classes. \paragraph{Multiview 3D reconstruction}~~ There are three European projects, Vanguard and Realise with the goal of advancing the state of 3D reconstruction from image sequences using recent developments in projective camera theory, and Improofs with the goal of advancing the state of forensic image analysis using also recent developments in projective camera theory and in the theory of partial differential equations. The algorithms are being developed in TargetJr and these applications will be good testbeds for the new IUE/TargetJr GUI. \section{Overall Plan} The Advisory Board reiterates that in their view the most pressing immediate need is for the IUE to pass the minimum threshold necessary to become usable by portions of the community outside of those with a committed stake in the system, i.e., the Technical Committee and AAI. This, in our view, should be the overarching theme throughout considerations of detailed plans. This implies a need: \begin{itemize} \item to define a small number of target applications of interest to DARPA, \item to develop a basic set of library routines needed to support these applications, \item to ensure that system development work on the IUE infrastructure proceeds along lines that will enhance performance characteristics, so that the system is usable, i.e., reasonably fast processing, compilation, loading, etc., \item to connect, through well designed interfaces, with other utilities that are needed, but not supported by the IUE, e.g., image processing routines, graphics, etc. \end{itemize} \subsection{Overview} From these considerations, the proposed five-year plan for further evolution of the IUE (1996 - 2000) is as follows: \vskip 10pt \noindent {\bf Year 1 - 1996} ~~Primary driving focus should be on demonstrating utility and usability of IUE in application domains. Thus: \begin{itemize} \item Identify and define two applications for demonstration of the IUE (possibilities include MSTAR match modules; if these are not appropriate then other applications should be defined by working with Tom Strat to identify applications of clear immediate benefit to his IU program). \item Task and complete library implementations necessary to support these applications. This will couple with the next point. \item Outline strategy for integration of IUE with Khoros, with completion of first pass at this integration by end of the year. \item Complete infrastructure necessary for efficient usability, i.e. library partitioning, compilation efficiency, GUI. \item End of year demo: an integrated model matching library and matching application to support MSTAR or some equivalent system as proof of operation. \end{itemize} \vskip 10pt \noindent {\bf Year 2 - 1997} ~~ Primary driving focus should be on expanding base of applicability of IUE, and on extending IUE to support distributed development. Thus: \begin{itemize} \item Develop and extend the IUE to support the evolution of the targeted applications such as the MSTAR matcher. Work in collaboration with the system integrator to integrate the IUE more extensively. If MSTAR is the selected application, this would be LMC Denver. \item It is essential that the IUE be developed to the point where IU groups can easily begin working with it, and can use it as part of their everyday development environment. Thus, the IUE must be deployed as a working laboratory infrastructure component in at least 5 sites. \item Collect additions to the IU libraries from these sites, from the European consortium, and from the IUE/TargetJr transition project. \item By the end of 1997, the spatial object core of TargetJr should be replaced by the IUE spatial object classes, and TargetJr 3D libraries supplementing the IUE. \item Monitor and evaluate the use of the IUE within the targeted distribution sites. Based on analysis of this evaluation, complete extensions of the IUE needed to support distributed development between mutlitple sites. This will include library development as well data exchange mechanisms. The following two points follow from this. \item Monitor the developments of SEDRIS, VRML, STEP, JAVA and CORBA to insure IUE compatibility with these rapidly evolving interchange mechanisms. Look for opportunities to exploit the IUE SpatialObject hierarchy in data exchange and browsing applications -- the XV of spatial data. LMC Valley Forge has successfully demonstrated import of CMU building models using DEX. Perhaps this role can be extended within SAIP. \end{itemize} \vskip 20pt \noindent{\bf Year 3 - 1998} ~~ Primary driving focus should be to install collaborative development, and to ensure broadly based user support. Thus: \begin{itemize} \item Coordinate with the ATD contractors for the IUBA initiative. Establish methods for using the IUE as integrative mechanisms within those programs. Establish methods for incorporating feedback and methods from those programs into the IUE, especially in fleshing out the libraries, and filling in holes in the classes. \item Develop a protocol for incorporating new library contributions into the IUE, including methods for testing and validating new methods. This will require an oversight committee of some sort, as well as established protocols for inserting new contributions, disseminating to the broader community, and ensuring compatibility of new insertions. \item In addition to operational demonstrations, the IUE is extended to provide support for context-based IU. The representation of context is going to place increasing demands on the semantics of the IUE which will go beyond the current emphasis on geometry. The IUE hierarchy should also be extended to incorporate reasoning and planning mechanisms. In any case, even in current developments for RADIUS at CRD it has proven useful to have a general blackboard architecture for supporting the integration and control of IU algorithms. \end{itemize} \vskip 20pt \noindent {\bf Year 4 - 1999} Primary focus should be to establish self-sufficiency for the IUE. Thus: \begin{itemize} \item Complete the development of advanced representations in the IUE. This will depend heavily on having good feedback from implementers using the IUE, especially within large collaborative projects. \item Install protocols for maintaining and updating the IUE from the user community. \end{itemize} \vskip 20pt \noindent {\bf Year 5 - 2000} Transition to self-support. \begin{itemize} \item At this stage a number of major contractors should be using the IUE and it should be in place at many universities with web-based access and interoperability. The model here is perhaps the one exhibited by LINUX. After a critical mass of users was achieved, the further development of the environment has become a world-wide shared enterprise. It will require about \$1-2M/year to fund AAI to maintain the IUE and incorporate enhancements to the core. This amount will still have to be provided by a combination of application contracts and direct financial support from users wanting rapid bug fixes and enhancements. This latter model is currently one mechanism which is funding maintenance support for gcc. \end{itemize} \subsection{Detailed Plan for Year 1} \paragraph{Define the target applications}~~ A key set of applications much be determined, in conjunction with DARPA. The basic libraries needed to support these applications should be defined, implemented. tested and deployed. This will further serve to support the next topic. \paragraph{Complete and strengthen IUE core}~~ In its current state of implementation the IUE core is ready to evaluate by implementing IU library algorithms. It is already apparent that additional effort is required to provide more efficiency and to streamline the interaction between classes. For example, the initial implementation of the graph class to support coordinate transforms is too complex and general. A simpler graph implementation is required, perhaps to support chain and star networks. As another example, Ross Beveridge, Chris Graves and Keith Price have asked for a more efficient image region representation, based on run length encoding, based on initial experience in coding image segmentation algorithms. Continue this evolution and hardening of the core by implementing the initial IUE algorithm library. \paragraph{Improve code compilation and linking efficiency}~~ It is recognized that improvements in GCC are required to produce effective code for the IUE, particularly in the case of templated classes involving STL. Rapid progress is underway in reducing code bloat and run time computational speed. However, currently the libraries are too large and too interconnected to permit small applications to be built on the IUE without linking a large unwieldy executable. The IUE must be more finely partitioned and dependencies between libraries reduced. Such an evolution will likely require some class design and implementation changes. \paragraph{Implement the IUE GUI}~~ Initial experience with FRESCO is promising and there seems to be considerable interest in FRESCO as a portable object-oriented structured graphics library. The plan is to integrate FRESCO with the IUE image classes to provide a 2D image and geometry GUI. The display of 3D will be provided by an overlay plane supporting a public domain version of open-gl called mesa-graphics. The plan is to share the development of FRESCO between GE-CRD and AAI and to produce a common base GUI between TargetJr and the IUE. TargetJr will migrate to use the IUE image classes. \paragraph{Implement the MSTAR Matcher Library}~~ If the MSTAR Match Module is chosen as the initial application for demonstration, then a collaboration with the developers must be established quickly. Since there is apparently a downselect in MSTAR expected in November 1996, it might be prudent to wait until after that event. Whichever developer remains after downselect, the goal is still the same: to realize the matching algorithm in terms of basic IUE classes and to extend the class hierarchy if necessary to support the needs of a generic interpretation of the algorithm. \paragraph{Integration with KHOROS}~~ Work with KRI to define the data-level integration of the IUE with KHOROS. Use the requirements of the MSTAR Matcher algorithms to define an initial level of data exchange between the IUE-Matcher and other modules in MSTAR. \paragraph{Integrate with European efforts}~~ Two avenues are worth exploring here. The first opportunity is to work closely with Manchester to integrate the image processing algorithms from RADIAL into the IUE. Manchester aims to do this anyway and a closer collaboration will insure successful re-use of these image processing libraries. A first task is to finalize the agreement on the design of image processing classes initially reached at the Martha's Vineyard meeting. The second opportunity is to work with the EC funded ``network of excellence'' ECVNET (European Computer Vision NETwork, http://afrodite.lira.dist.unige.it/fullservice.html). This network has a subcommittee on software environments which has been a bit dormant but could easily be reactivated. The network is going up for new funding next year, thus this could be a great opportunity to start something big jointly between the EC and the US. \paragraph{Complete IUE Contract Commitments}~~Complete the other commitments for the final year of the initial IUE contract, e.g. documentation, new code releases and libraries. \section{1997 Statement of Work} Based on the five year plan outline above, the following items should be completed during 1997. \begin{itemize} \item {\bf Target applications} Two target applications must be quickly defined and detailed. If one or more of the MSTAR modules are to be included, arrangements must be quickly worked out with those contractors, including a clear definition of algorithmic needs, data structure needs and interaction needs. \item {\bf Library implementation} Based on such identification of target applications, implement and install library routines to support them. \item {\bf Khoros integration} Implement and install at least a first prototype integration with Khoros, enabling the user to easily utilize Khoros routines within an IUE framework, and vice versa. This will include ensuring efficient interchanges of data structures between the two systems. \item {\bf Efficiency issues} Structuring of libraries for focused target applications, efficient compilation of selected IUE components are both essential and must be completed this year. A prototype GUI for easy use, especially one that integrates Khoros and IUE should be developed. \end{itemize} %In order to ensure the maintainence of a critical team at AAI, and %insuring Rick Lerner as project leader it will be necessary to %insure a base-level funding of \$500K during 1997. This funding would %support evolution and maintenance of the core IUE and provide resources %to fix bugs and follow the changes in compilers and C++ language features. %The demands of library implemenatations will also require a basic level %of support for enhancements by AAI. Additional resources at AAI would %be be applied to the IUE in the context of IUE applications, for example %further tasks in MSTAR or new projects funded by new IU BAA's. \section{The Future of the IUE Technical Committee} %\begin{verbatim} %This section is sensitive and the proposal should not %be disseminated until consensus is reached between JLM, %Tom Strat and AAI with regard to the role and membership %of the IUE UGC. %\end{verbatim} The initial purpose of the IUE Technical Committee (IUE TC) has been largely completed, namely the definition of basic classes and functions required for IU algorithm development. In the next phases of the evolution of the IUE, the focus should shift to a governing body which is concerned with the application of the IUE class hierarchy and the implementation of IU libraries. To reflect this change in emphasis the following new IUE Users Group Committee is proposed. The new membership would take place after the completion of existing DARPA-funded TC support contracts. Members would no longer be indivdually funded, but there still should be a small amount of annual committee funding, say \$50K to support IUE UGC meetings and workshops. Membership would involve individuals working with the IUE in the context of DARPA-funded projects or other national funding. New members will be added as dissemination and use of the IUE expands. \end{document}