Collaborative Working

Minority Report UI creator's first collaborative system

“It’s purely a historical accident that computers are intended for only one person at a time. That’s what characterises them. We’ve never built, as a technological species, a collaborative computer. So Mezzanine is not intended as a replacement for the technology that we have today, it is a compliment for it,” says John Underkoffler, CEO of Oblong Industries.

The “Mezzanine” that Underkoffler is referring to is an “architectural computer” that is part of the room and multiple people can use at the same time. It is this very technology that was used in an iconic scene in the Spielberg movie Minority Report and Underkoffler is the man behind it. In the scene, you see Tom Cruise interacting with the screens in front of him using just his hands with not a keyboard or mouse in sight. The impact of this for viewers was huge.

The 2002 movie was so influential that despite being set in 2054, it managed to spookily predict other types of technology that we are already using today: driverless cars, predictive policing, and gestural interface are just some of them.

But back to present day, 2015 and I am sitting opposite Underkoffler and Steve Smith, UK director for Oblong Industries in their Mezzanine room at their office in Shoreditch, London. The room is laid out in a conference-room style format but it’s the three large screens on the wall in front of me and two vertically-aligned screens on a wall to the left that stand out the most. There is also a white board on one of the walls and the desk in the middle of the room is full of Mac laptops. The whole feel of the room is very slick and stylish.

I have been invited to see the technology in action and Underkoffler is calm yet clearly passionate about his creation. Underkoffler is no ordinary interface designer. Not only did he advise the filmmakers on Minority Report, he also showed off his user interface at a Ted presentation that demonstrated the future of computers.


How did he come up with it?

Underkoffler says that around 1999 or early 2000 he was at Media Lab at MIT busy finishing up his dissertation work when the production designer and the prop master for Minority Report dropped by the lab. He says they were on a “shopping spree” flying around the US visiting academic institutions and R&D labs looking for all of the emerging technologies they could possibly import into the film.

“The brief from Spielberg was that he wanted to depict a completely believable 2054. And the production designer was a really amazing thinker who correctly surmised that the way to paint this believable picture was to populate it with bits from the present that he could extrapolate forward,” Underkoffler tells me.

“So when they saw a bunch of the UI work that I and the others were doing at MIT I think they felt that one of their biggest design problems could potentially be solved because Spielberg wanted to know what computers would look like in 50 years. And he was really adamant that his set not be littered with mice and keyboards and so forth. Which is gratifying to hear!” Underkoffler adds jokingly.

Being a long-time film aficionado Underkoffler fell into long conversations with the production designer (he says he had no motive behind it) and then some months later before he knew it….

“I had one of those mysterious middle of the night calls when a British voice on the other end says: ‘John would you like to help us build minority report?’ Of course you say yes! And then he said, ‘Good, get on the plane!’”

Underkoffler says that the user interface we see Tom Cruise using in the movie is “derived very directly” from the work he had been doing at MIT, although then it was in an academic setting. He says that it then had to be refined for the movie which he found to be a very valuable process. Following the enthusiastic reception those scenes had in the movie, Underkoffler started receiving calls from big companies showing interest in the technology and wondering if it was real.

“I decided in time that it was important to build it for real and then we started Oblong. And the very first thing we did was to build precisely the Minority Report system that included gloves and the rest of it. So what you see in the film is today completely real.”


Minority Report aside, how will this technology apply to real-use cases?

Smith picks up a wand to control the screens and starts showing me how it all works. The two large screens in front of us are what he describes as a “virtual workspace” that is shared and distributed amongst all the Mezzanines that people work with in a shared digital space. Smith points and clicks at the smaller icons at the bottom of the screen which is a portfolio - a storage area for assets. This could contain anything from a bunch of slides for a presentation to a large engineering diagram.

“It’s different from traditional presenting. PowerPoint has been around for about 25 years and hasn’t actually changed that much in all that time. In management meetings, traditionally we have to use three static slides but then what if you have to shift some numbers around? With a Mezzanine system it’s much easier to use real-time, real-content,” Smith tells me.

The vertically-aligned screens on the left are “corkboards” that are more “static in nature”. Smith says they are like a parking lot for ideas and the key to a lot of their customer use cases. Smith then points the wand at an idea from one of the portfolios and drags it seamlessly to the corkboard and drops it.

Underkoffler and Smith see this technology being most useful for presentations, everyday meetings and real-time decision-making. I am told that productivity can be increased by at least 30%. But I am wondering about space. Not all offices have the full capacity to build Mezzanine rooms like this one.

“In terms of the sorts of rooms this goes into it’s incredible flexible. We are not interested in screen technology per say, we work with the customer to establish what the right format is for them,” Smith tells me.

At the moment you are using a wand but eventually will we be able to use our hands?

Underkoffler says that’s ultimately up to their customers but they do have all the gestural stuff in their ‘back-pocket” so they can always put that back in. He smiles and jokes that it would be fun. Smith thinks that the world is not ready for that kind of depth of interaction from the hands yet.

“I could train you to use this (picks up the wand) in about 30 seconds flat – but I could not train you to do the same thing with a fully gestural-based system because essentially you are learning a limited form of sign language and that’s not a simple thing to do. I think in the future people will be ready for it,” says Smith.

We then connect to the team in Barcelona in an “infopresence” style meeting joining the two Mezzanine systems together. Smith and Underkoffler are keen to emphasise that this is not like a standard video call so they show things like expanding the size of spreadsheets and asking colleagues to make changes.

Oblong Industries’ customer base is diverse and I am told that IBM is a very enthusiastic adopter of this technology. What does Underkoffler make of augmented and virtual reality systems?

“Our position along that spectrum is real reality. So [our position] is sort of the opposite of VR – we are going to take the pixels that have always been part of the computational experience and make them part of the room so that everyone can see them.”

Since Underkoffler has worked on a movie himself, how influential does he think movies are in inspiring these types of technologies?

“Of course scientists and engineers have always been susceptible to the appeal and lure of science fiction and science fiction is great at testing out ideas but sometimes it’s not nearly critical enough,” says Underkoffler. “For me the social aspects are as interesting and as important as the pure technological aspects. Which is why Philip K Dick was such a great sci-fi writer as he was saying let’s suppose we have this technology and suppose we have this capability – now look at all the ways that it could go wrong.”

“It’s the unintended consequences that are so important to at least try to think about a little bit. It’s a dangerous point of view to assume that the more technology we pump into the world the better off things will be.”

It’s time for me to leave. The Mezzanine system is every bit as it’s packaged to be - stylish, slick, and very cool. I would have liked to have seen the system work with hands instead of the wand – but maybe that’s the movie-me talking.


« Box CEO will double-down on security


C-suite career advice: Bharath Rangarajan, Pertino »
Ayesha Salim

Ayesha Salim is Staff Writer at IDG Connect

  • twt
  • Mail


Do you think your smartphone is making you a workaholic?