Undergraduate Research in the IVILAB

The Interdisciplinary Visual Intelligence group has an active program for involving undergraduates in research. We strongly encourage capable and interested undergraduate students to become involved with research early on. Doing so is becoming increasingly critical for preparing for grad school and has also helped many students get jobs in industry.

Link to current opportunities and the application process.

Due to overwhelming interest in the IVILAB we need to limit the number undergraduate researchers. Unfortunately, we have to turn away many undergraduates looking for research experience.

Publishing (needs updating).

Undergraduate researchers working with IVI lab have a great record of contributing sufficiently to research projects that they become authors on papers. So far, twelve undergraduates have been authors on nineteen vision lab papers and three abstracts (needs updating). Click here for the list.

IVILAB undergraduate researchers past and present (needs updating).

Students who have participated in the IVILAB as undergraduates include Matthew Johnson (honor's student, graduated December 2003), Abin Shahab (honor's student, graduated May 2004), Ekaterina (Kate) Taralova (now at CMU), Juhanni Torkkola (now at Microsoft), Andrew Winslow (now at Tufts), Daniel Mathis, Mike Thompson, Sam Martin, Johnson Truong (now at SMU), Andrew Emmott (headed to Oregon State), Ken Wright, Steve Zhou, Phillip Lee, James Magahern, Emily Hartley, Steven Gregory, Bonnie Kermgard, Gabriel Wilson, Alexander Danehy, Daniel Fried, Joshua Bowdish, Lui Lui, Ben Dicken, Haziel Zuniga, Mark fischer, Matthew Burns, Racheal Gladysz, Salika Dunatunga (honors, now at U. Penn), Kristle Schulz (honors), and Soumya Srivastava.

Examples of undergraduate research in the IVILAB (needs updating)

Understanding Scene Geometry

The image(*) to the right shows undergraduate Emily Hartley determining the geometry of an indoor scene and the parameters of the camera that took the picture of the scene. Such data is critical for both training and validating systems that automatically infer scene geometry, the camera parameters, the objects within the scene, and their location and pose.

Funding for undergraduates provided by REU supplement to NSF CAREER grant IIS-0747511.

(*) Photo credit Robert Walker Photography.

Semantically Linked Instructional Content (SLIC)

The image shows a screen shot of the SLIC educational video browsing system which is an excellent project for undergraduates interested in multimedia and web design. For more information see the SLIC project page, or contact Yekaterina (Kate) Kharitoova (ykk AT email DOT arizona DOT edu) for more information. SLIC has led to several publications with undergraduate authors, and currently several undergraduate students are working on it.

SLIC is partly supported by NSF grant EF-0735191.

Aligning image caption words with image elements

There are now millions of images on-line with associated text (e.g., captions). Information in captions is either redundant (e.g., the word dog occurs, and the dog is obvious) or complementary (e.g., there is sky above the dog, but it is not mentioned). Redundant information allows us to train machine learning methods to predict one of these modalities from the other. Alternatively, complementary information in the modalities can disambiguate uncertainty (see "Word Sense Disambiguation with Pictures" below), or provide for combined visual and textual searching and data mining. Under the guidance of PHD student Luca del Pero, undergraduates Phil Lee, James Magahern, and Emily Hartley have contributed to research on using object detectors to improve the alignment of natural language captions to image data, which has already led to a publication for them. For more information on this project, contact Luca del Pero (delpero AT cs DOT arizona DOT edu).

Funding for undergraduates provided by ONR and REU supplement to NSF CAREER grant IIS-0747511.

On the left is the baseline result; the image on the right shows the result with detectors.

Simultaneously tracking many objects with applications to biological growth

Simultaneously tracking many objects with overlapping trajectories is hard because you do not know which detections belong to which objects. The vision lab has developed a new approach to this problem and has applied it to several kinds of data. For example, the image to the right shows tubes that are growing out of pollen specs (not visible) towards ovules (out of the picture) in an in vitro plant fertilization experiment. The paths of the pollen tubes are green, and the tracks automatically found are superimposed in red. This work has led to a publication for undergraduate researcher Phil Lee. This project is being led by PHD student Ernesto Brau. For more information contact Ernesto (ernesto AT cs DOT arizona DOT edu).

This project is in collaboration with the Palanivelu lab.

Funding for this project provided by NSF grant IOS-0723421.

Identifying machine parts using CAD models

CAD models provide the 3D structure of many man made objects such as machine parts. This projects aims to find objects in images based on these models. However, since the data is most readily available as triangular meshes, 3D features that are useful for matching 2D images must be extracted from mesh data. Undergraduate Emily Hartley has contributed software for this task, and undergraduate Andrew Emmott has contributed software for matching extracted 3D features to 2D images. They have been mentored by PHD student Luca del Pero. For more information, contact him (delpero AT cs DOT arizona DOT edu).

Funding for undergraduates provided by NSF Grant 0758596 and REU supplement to NSF CAREER grant IIS-0747511.

Inferring Plant Structure from Images

Quantifying plant geometry is critical for understanding how subtle details in form are caused by molecular and environmental changes. Developing automated methods for determining plant structure from images is motivated by the difficulty of extracting these details by human inspection, together with the need for high throughput experiments where we can test against a large number of variables.

To get numbers for structure we fit geometric models of plants to image data. The picture on the right shows multiple views of an Arabidopsis plant (top), two views (bottom, left), and fits of the skeleton to the image data, projected using camera models corresponding to those two views. Undergraduates Sam Martin and Emily Hartley have helped collect the image data, arrange feature extraction, and create ground truth data fits to it for training and evaluation. This project is led by Kyle Simek. For more information, contact him (ksimek AT cs DOT arizona DOT edu).

This project is in collaboration with the Palanivelu and Wing labs.

Funding for undergraduates provided by the NSF funded iPlant project, via the University of Arizona UBRP program and an REU supplement to NSF CAREER grant IIS-0747511.

Modeling and visualizing Alternaria

To the right is a labeled model of the fungus Alternaria generated by a stochastic L-system built by undergraduate researcher Kate Taralova. For more information, follow this link.

This project is in collaboration with the Pryor lab.

Support for undergraduates provided by TRIFF and a REU supplement to a department of computer science NSF research infrastructure grant

Inactive and Subsumed Projects

Word sense disambiguation with pictures

Many words in natural language are ambiguous as illustrated here by the word "bank". Typically, resolving such ambiguity is attempted by looking at nearby words in the passage. Computer vision lab undergraduate researcher Matthew Johnson played a key role in the development of a novel method for adding information from accompanying illustrations to help reduce the ambiguity. The system learns from a data base of images that certain word senses (e.g., meanings of bank found with outdoor photos), are associated with certain kinds of image features. This association is then used to incorporate information in illustrations to help disambiguate the word under consideration. This work led to two publications for Matthew.

Support for undergraduates provided by TRIFF and a REU supplement to a department of computer science NSF research infrastructure grant.

Vision system for flying robots

These three images illustrate work by computer science students on a UA multi-department effort to compete in international aerial robotics competition which is largely an event for undergraduates. Here computer controlled planes and/or helicopters work towards accomplishing a mission specified by the contest organizers. Part of the current task is to find a building having a particular symbol on it (left), and identify the doors and windows of that building, and then identify which doors and windows are open so that a sub-vehicle can be launched through the portal. The middle figure shows the symbol identification software being tested from a moving vehicle to simulate flight. The far right figure shows a view from the computer science department with lines found in this image and the matching lines found in a companion image. The students use the shift (shown in green) between matching edges to estimate the distance to the edge, which is used to help analyze the structures. Images provided by undergraduate researcher Ekaterina (Kate) Taralova (now at CMU).

Browsing large image collections

A screen shot of a program for browsing large digital art image databases that is being developed by undergraduate students in computer science at the U of A. (Art images courtesy of the Fine Arts Museum of San Francisco). Contributions to this project have been made by undergraduates Matthew Johnson and John Bruce.

Evaluation of image segmentation algorithms

Two images which have been segmented by three different methods. U of A undergraduate students in computer science are involved in research to evaluate the quality of such methods. Segmentation quality is quantified by the degree to which the regions are useful to programs which automatically recognize what is in the images. Contributions have been made by Abin Shahab.