Mines computer vision research: Making the Wii look easy

The technology that makes the XBox Kinect or other motion sensing games is a complex field in computer science. Professor Bill Hoff, along with five graduate students, is heavily involved with research in this field, specifically in the areas of computer vision and pattern recognition.

“Computer vision is a process of using computers to interpret images from cameras and do things like recognize objects or measure the location of things.” Hoff and his team mainly work with off-the-shelf cameras and sensors, focusing on software development. They write algorithms using high-level programming like MATLAB and C++, or in the case of developing applications for tablets and phones, Java.

Beyond video games, the research into computer vision has numerous practical applications. For example, two years ago Hoff was on sabbatical working at Lockheed Martin. “They were working on autonomous convoy technology. The idea is you would have a driver in a lead truck, with driverless trucks following it. I helped them develop mapping and landmark recognition localization using computer vision to help the following trucks follow the lead truck.”

Computer vision technology is also versatile and easily applied in inter-disciplinary research. For example, ten years ago Hoff worked with a research group in Denver and used medical imaging technology, like X-rays, in combination with computer vision technology to evaluate how knee implants degraded over time. “We were trying to understand how these implants function and why they wear out so quickly. Your typical knee implant has a piece of plastic in-between the bones which tends to wear out in ten to fifteen years. We were studying, essentially, X-ray movies of people with these implants to see exactly how they were moving and using computer vision technology to quantify that motion. We found, as a result of this analysis, that one type of implant was better than the others.”

Hoff is currently working in the developing field of augmented reality. He said, “I’m really excited about augmented reality, it’s an area I’m really interested in, and I’d love to get students involved. Augmented reality is the process of augmenting the real world with virtual object. There are some applications on phones, like when you take a picture of a landmark and it is labeled. There is a lot of potential for many applications, like navigation, training, or education. Generally, when you want to help a person understand what they’re looking at and guide them through a task.”

Outside of computer vision is the broader area of pattern recognition. Hoff, along with a graduate student have developed a project using pattern recognition which will hopefully lead to a number of published journal articles.

“Pattern recognition is a bit more general, where you try to classify patterns from information which is more general than vision information. For example, I have a project in activity recognition, where we are trying to recognize group activities of people in Brown Building, with the purpose of predicting where people are going to be throughout the day for building energy efficiency purposes. The sensors detect the presence of people, and then patterns can be created about room use throughout the day, as well as recognizing specific patterns like evening exams, snow days, etc,” he said.

Hoff and his projects are a part of the larger research group on campus which is led by Professor Tracy Camp. The research group makes up CARDI, the Center for Automated Robotics and Distributed Intelligence, which brings together professors with similar research areas for collaboration, seminars, and occasionally joint projects.
Hoff mentioned that the technology behind the Wii was simple, covered within the first few days of lecture in his computer vision class. The research he conducts may produce practical solutions, but it may also find its way into the next big video game.