'Minority Report' software hits the real world

'Minority Report' software hits the real world

  8:00:00 AM    Science Lover

---

The software behind the film "Minority Report" -- where Tom Cruise speeds through video on a large screen using only hand gestures -- is making its way into the real world.

The interface developed by scientist John Underkoffler has been commercialized by the Los Angeles firm Oblong Industries as a way to sift through massive amounts of video and other data.

And yes, the software can be used by law enforcement and intelligence services. But no, it is not the "pre-crime" detection program illustrated in the 2002 Steven Spielberg sci-fi film.

Kwin Kramer, chief executive of Oblong, said the software can help in searching through "big data" for information. It can also create souped-up video-conference capabilities where participants share data from multiple devices like smartphones and tablets, integrated into a large video display.

"We think the future of computing is multiuser, multiscreen, multidevice," Kramer told AFP.

"This system helps with big workflow problems."

A key part of the system is the gesture interface, which the company calls the "g-speak" spatial operating environment.

That grew out of a project by Underkoffler -- then a scientist at the prestigious Massachusetts Institute of Technology -- for "Minority Report," before he became chief scientist at startup Oblong.

"We have demo versions of this kind of software which show exactly the 'Minority Report' user experience, allowing you to move back and forth in time, or to zoom in to look at details," Kramer said.

He said the same software can help businesses to "allow better collaboration, visualization and analysis of large amounts of data.

"You can have a lot of data but it's hard to make use of that," Kramer said.

"It can be on different machines and hard to access. This allows multiple people to look at that."

An employee demonstrates the use of a data glove to navigate a map on a computer screen at Los Angeles-based software company Oblong Industries' offices in Washington in June 2012. The software behind the film "Minority Report" -- where Tom Cruise speeds through video on a large screen using only hand gestures -- is making its way into the real world.

Gestural interfaces have been developed for other firms including Microsoft's Kinect but Oblong says it has far more sophisticated systems which can use Kinect and more.

Some highly sensitive systems use a data glove which can be more precise than ordinary hand movements.

Oblong has contracts with firms such as Boeing, General Electric and Saudi Aramco to help in analyzing large amounts of data. It is also developing a gestural interface for onboard computers with automaker Audi.

It has raised an unspecified amount of venture capital from investors including Foundry Group, Energy Technology Ventures and Morgan Stanley Alternative Investment Partners.

Brad Feld, managing director at Foundry Group, said Oblong offers "a path to fundamentally change the way we interact with computers."

Yet the question Oblong often gets is how users can get the "Minority Report" software.

David Schwartz, the company's vice president for sales, said "We get calls from people in the military who say, 'I want the 'Minority Report' interface."

He said the systems could be used for a realistic version of high-tech software interfaces on TV shows like "CSI."

"They would like to get it for free," he added.

What makes the real-life version of the software different from the one seen on film is that Oblong does not supply the analytics of the futuristic "pre-crime" division.

That does not prevent a company or law enforcement agency from using the software and adding its own analytics.

"We think law enforcement and intelligence are big data users and we think our technology is the leader," Kramer said.

He said Oblong currently has no government customers in the United States or abroad but offers itself as "a core technology provider."

Still, Oblong leverages its role in the movies to get in the door, even if the software is not quite the same.

"I think most people look at those 'Minority Report' interfaces and imagine how they could use that flexible system in their own office or designs studio," Kramer said.

"It isn't science fiction, it's real."

Read More

A Search Engine for the Human Body

A Search Engine for the Human Body

  2:12:00 AM    Science Lover

---

Microsoft software recognizes organs and other structures in medical images.

A new search tool developed by researchers at Microsoft indexes medical images of the human body, rather than the Web. On CT scans, it automatically finds organs and other structures, to help doctors navigate in and work with 3-D medical imagery.

Inside out: A close up of a CT processed by new software from Microsoft.
Credit: Microsoft Research

CT scans use X-rays to capture many slices through the body that can be combined to create a 3-D representation. This is a powerful tool for diagnosis, but it's far from easy to navigate, says Antonio Criminisi, who leads a group at Microsoft Research Cambridge, U.K., that is attempting to change that. "It is very difficult even for someone very trained to get to the place they need to be to examine the source of a problem," he says.

When a scan is loaded into Criminisi's software, the program indexes the data and lists the organs it finds at the side of the screen, creating a table of hyperlinks for the body. A user can click on, say, the word "heart" and be presented with a clear view of the organ without having to navigate through the imagery manually.

Once an organ of interest has been found, a 2-D and an enhanced 3-D view of structures in the area are shown to the user, who can navigate by touching the screen on which the images are shown. A new scan can also be automatically and precisely matched up alongside a past one from the same patient, making it easy to see how a condition has progressed or regressed.

Criminisi's software uses the pattern of light and dark in the scan to identify particular structures; it was developed by training machine-learning algorithms to recognize features in hundreds of scans in which experts had marked the major organs. Indexing a new scan takes only a couple of seconds, says Criminisi. The system was developed in collaboration with doctors at Addenbrookes Hospital in Cambridge, U.K.

The Microsoft research group is exploring the use of gestures and voice to control the system. They can plug in the Kinect controller, ordinarily used by gamers to control an Xbox with body movements, so that surgeons can refer to imagery in mid-surgery without compromising their sterile gloves by touching a keyboard, mouse, or screen.

Body search: This CT image shows organs and other features identified by the Microsoft software. A list of these features appears at left.
Credit: Microsoft Research

Kenji Suzuki an assistant professor at the University of Chicago, whose research group works on similar tools, says the Microsoft software has the potential to improve patient care, providing it really does make scans easier to navigate. "As medical imaging has advanced, so many images are produced that there is a kind of information overload," he explains. "The workload has grown a lot."

Suzuki says Microsoft's approach is a good one, but that medical professionals might be more receptive to the design if it indexed signs of disease, not just organs. His own research group has developed software capable of recognizing potentially cancerous lung nodules; in trials, it made half as many mistakes as a human expert.

Criminisi sticks by the notion of using organs as a kind of navigation system but says that disease-spotting capability is also under development. He says, "We are working to train it to detect differences between different grades of glioma tumor"—a type of brain tumor.

The Microsoft group also intends the tool to be used at large scales. It could automatically index a collection of 3-D scans or other images, making possible new ways of tracking medical records, says Criminisi. Today, records are kept as text that describes scans and other information. A search tool that finds the word "heart", for example, would not know if that meant it appeared in a scan or was mentioned in another context. If a hospital's computer system indexed new scans, the Microsoft software could automatically record what was imaged in a person's records and when.

Read More

Microsofts 3-D Strategy Microsoft's Craig Mundie describes how the company's vision of 3-D gaming could extend to all computer interactions.

Microsofts 3-D Strategy Microsoft's Craig Mundie describes how the company's vision of 3-D gaming could extend to all computer interactions.

  10:01:00 AM    Science Lover

---

Microsoft has joined the wave of companies betting that 3-D is the next big thing for computing. At a recent talk at MIT, chief research and strategy officer Craig Mundie said he sees the technology as an innovation that "will get people out of treating a computer as a tool" and into treating the device as a natural extension of how they interact with the world around them. Microsoft plans to introduce consumers to the change through its gaming products, but Mundie outlined a vision that would eventually have people shopping and searching in 3-D as well.

The future of 3-D: During a talk at MIT last week, Craig Mundie, Microsoft's chief research and strategy officer, showed how a natural 3-D interface could let users manipulate and examine products--like the disassembled motorcycle in the background.
Credit: Microsoft/Technology Review

The combination of better chips, better displays, and better sensors, Mundie said, is finally making it possible to move computing from today's graphical user interfaces to the "natural user interface," by allowing people to interact with 3-D content through the gestures they normally use. Today's interfaces require users to learn about menu bars and double-clicks, but Mundie believes natural user interfaces, which work through gesture and voice, will be faster and easier to learn, and will prove more flexible in the long run.

Mundie also argued that natural user interfaces would reduce the mental effort required for people to operate software. Even people who are good at using controllers, keyboards, and mouses might find that a natural interface frees up attention and concentration so that they can focus better on the task at hand, he said. He believes that natural interfaces will make it easier to introduce software to people unfamiliar with computers, as well as make software generally easier to use, and therefore more attractive to consumers.

He also noted that today many programs come with what is essentially "an application-specific prosthetic"--for example, some driving games come with a steering-wheel device. Natural user interfaces may require some peripherals, such as depth-sensing cameras that can detect users' movements, but Mundie sees these as ultimately having broader purpose than most of today's devices.

The first step in this strategy, Mundie said, is Microsoft's release next month of the Kinect sensor for the Xbox 360 gaming console; Kinect incorporates a depth-sensing camera and voice recognition and will cost about $150. It will allow users to play games by gesturing, without the need for a controller or additional equipment. This opens the way to 3-D interaction with games that Mundie hopes will lead to broader use of 3-D displays.

Mundie demonstrated how Kinect would allow a user to interact with 3-D game content through hand gestures, virtually picking up clues to examine them or show them to friends. "We're trying to create a genre of games where you don't have to think about how what you would do naturally would map to the controls," Mundie said.

He also showed a concept video for a real-time 3-D multiplayer game called "The Spy from the 2080s" that included a TV show and a game that players could interact with using multiple devices. For example, they might watch an episode in 3-D on TV, then log in through a gaming console to work with friends to solve clues from the show. Mobile devices might provide additional updates. In the video, the outcome of gameplay even influenced the course of the TV show.

But while the company may plan to start with gaming, Mundie envisions 3-D eventually becoming a key part of many computer interfaces and online content. In one example, he demonstrated shopping using a 3-D natural interface; his hand gestures spun a 3-D image of a product, displaying it from a variety of angles, and opened it up to look at the parts inside.

He acknowledged, however, that there are challenges that need to be solved before 3-D can become ubiquitous. "We need a lot more computer than we currently have," Mundie said, noting that processing high-definition, 3-D video in real time would strain the capabilities of most home computers today. He also admitted that companies still need to refine how users would interact with computers through gesture and voice--for example, distinguishing between when a gamer is issuing commands to the computer and when the same user is conversing with another player.

Microsoft is wise to focus on games initially, says Norbert Hildebrand, business development manager for Insight Media, a marketing research firm that covers emerging display technologies. With 3-D technologies, providing enough content is a huge issue today, he explains. Games are already created in 3-D and then rendered to work on a 2-D screen, which makes it easy to convert them for 3-D displays and other types of interfaces.

For other types of 3-D interaction, such as shopping or advertising, Hildebrand says content creators will need to be persuaded to invest the necessary money and resources. As far as Mundie's vision of 3-D shopping, Hildebrand says, "at this point, it's marketing talk only." He points out that today's 3-D displays don't display text well, so marketers would have to come up with a hybrid approach to display both product images and information.

For now, Hildebrand believes that the average person views 3-D technology as something used on special occasions, not as a day-to-day technology. Some people have interpreted sales figures for 3-D-enabled televisions as a sign that consumers are adopting the technology, he adds, but these can be misleading, since most high-end televisions today have 3-D capabilities. It's much harder to determine whether people are actually using 3-D and how often, he says.

3-D is on the way, Hildebrand says, but before Mundie's vision of day-to-day 3-D becomes viable, "a lot of things have to come together." This includes more 3-D content, better bandwidth for delivering it to users, faster processors to render it, and particularly, he believes, the next generation of display technology--one that doesn't require special glasses.

Read More