Undergraduate Research in the IVILAB
The Interdisciplinary Visual Intelligence group has an active program for involving undergraduates in research. We strongly encourage capable and interested undergraduate students to become involved with research early on. Doing so is becoming increasingly critical for preparing for grad school and has also helped many students get jobs in industry.
Due to overwhelming interest in the IVILAB we need to limit the number undergraduate researchers. Unfortunately, we have to turn away many undergraduates looking for research experience.
Publishing (needs updating).Undergraduate researchers working with IVI lab have a great record of contributing sufficiently to research projects that they become authors on papers. So far, twelve undergraduates have been authors on nineteen vision lab papers and three abstracts (needs updating). Click here for the list.
IVILAB undergraduate researchers past and present (needs updating).Students who have participated in the IVILAB as undergraduates include Matthew Johnson (honor's student, graduated December 2003), Abin Shahab (honor's student, graduated May 2004), Ekaterina (Kate) Taralova (now at CMU), Juhanni Torkkola (now at Microsoft), Andrew Winslow (now at Tufts), Daniel Mathis, Mike Thompson, Sam Martin, Johnson Truong (now at SMU), Andrew Emmott (headed to Oregon State), Ken Wright, Steve Zhou, Phillip Lee, James Magahern, Emily Hartley, Steven Gregory, Bonnie Kermgard, Gabriel Wilson, Alexander Danehy, Daniel Fried, Joshua Bowdish, Lui Lui, Ben Dicken, Haziel Zuniga, Mark fischer, Matthew Burns, Racheal Gladysz, Salika Dunatunga (honors, now at U. Penn), Kristle Schulz (honors), and Soumya Srivastava.
Examples of undergraduate research in the IVILAB (needs updating)
Understanding Scene GeometryThe image(*) to the right shows undergraduate Emily Hartley determining the geometry of an indoor scene and the parameters of the camera that took the picture of the scene. Such data is critical for both training and validating systems that automatically infer scene geometry, the camera parameters, the objects within the scene, and their location and pose.
Funding for undergraduates provided by REU supplement to NSF CAREER grant IIS-0747511.
(*) Photo credit Robert Walker Photography.
Semantically Linked Instructional Content (SLIC)The image shows a screen shot of the SLIC educational video browsing system which is an excellent project for undergraduates interested in multimedia and web design. For more information see the SLIC project page, or contact Yekaterina (Kate) Kharitoova (ykk AT email DOT arizona DOT edu) for more information. SLIC has led to several publications with undergraduate authors, and currently several undergraduate students are working on it.
SLIC is partly supported by NSF grant EF-0735191.
Aligning image caption words with image elementsThere are now millions of images on-line with associated text (e.g., captions). Information in captions is either redundant (e.g., the word dog occurs, and the dog is obvious) or complementary (e.g., there is sky above the dog, but it is not mentioned). Redundant information allows us to train machine learning methods to predict one of these modalities from the other. Alternatively, complementary information in the modalities can disambiguate uncertainty (see "Word Sense Disambiguation with Pictures" below), or provide for combined visual and textual searching and data mining. Under the guidance of PHD student Luca del Pero, undergraduates Phil Lee, James Magahern, and Emily Hartley have contributed to research on using object detectors to improve the alignment of natural language captions to image data, which has already led to a publication for them. For more information on this project, contact Luca del Pero (delpero AT cs DOT arizona DOT edu).
Funding for undergraduates provided by ONR and REU supplement to NSF CAREER grant IIS-0747511.
On the left is the baseline result; the image on the right shows the result with detectors.
Simultaneously tracking many objects with applications to biological growthSimultaneously tracking many objects with overlapping trajectories is hard because you do not know which detections belong to which objects. The vision lab has developed a new approach to this problem and has applied it to several kinds of data. For example, the image to the right shows tubes that are growing out of pollen specs (not visible) towards ovules (out of the picture) in an
This project is in collaboration with the Palanivelu lab.
Funding for this project provided by NSF grant IOS-0723421.
Identifying machine parts using CAD modelsCAD models provide the 3D structure of many man made objects such as machine parts. This projects aims to find objects in images based on these models. However, since the data is most readily available as triangular meshes, 3D features that are useful for matching 2D images must be extracted from mesh data. Undergraduate Emily Hartley has contributed software for this task, and undergraduate Andrew Emmott has contributed software for matching extracted 3D features to 2D images. They have been mentored by PHD student Luca del Pero. For more information, contact him (delpero AT cs DOT arizona DOT edu).
Funding for undergraduates provided by NSF Grant 0758596 and REU supplement to NSF CAREER grant IIS-0747511.
Inferring Plant Structure from ImagesQuantifying plant geometry is critical for understanding how subtle details in form are caused by molecular and environmental changes. Developing automated methods for determining plant structure from images is motivated by the difficulty of extracting these details by human inspection, together with the need for high throughput experiments where we can test against a large number of variables.
To get numbers for structure we fit geometric models of plants to image data. The picture on the right shows multiple views of an Arabidopsis plant (top), two views (bottom, left), and fits of the skeleton to the image data, projected using camera models corresponding to those two views. Undergraduates Sam Martin and Emily Hartley have helped collect the image data, arrange feature extraction, and create ground truth data fits to it for training and evaluation. This project is led by Kyle Simek. For more information, contact him (ksimek AT cs DOT arizona DOT edu).
This project is in collaboration with the Palanivelu and Wing labs.
Modeling and visualizing AlternariaTo the right is a labeled model of the fungus Alternaria generated by a stochastic L-system built by undergraduate researcher Kate Taralova. For more information, follow this link.
This project is in collaboration with the Pryor lab.
Support for undergraduates provided by TRIFF and a REU supplement to a department of computer science NSF research infrastructure grant
Inactive and Subsumed Projects
Word sense disambiguation with picturesMany words in natural language are ambiguous as illustrated here by the word "bank". Typically, resolving such ambiguity is attempted by looking at nearby words in the passage. Computer vision lab undergraduate researcher Matthew Johnson played a key role in the development of a novel method for adding information from accompanying illustrations to help reduce the ambiguity. The system learns from a data base of images that certain word senses (e.g., meanings of bank found with outdoor photos), are associated with certain kinds of image features. This association is then used to incorporate information in illustrations to help disambiguate the word under consideration. This work led to two publications for Matthew.
Support for undergraduates provided by TRIFF and a REU supplement to a department of computer science NSF research infrastructure grant.