WATCH: New technology allows phone and computer screens to interact seamlessly

WATCH: New technology allows phone and computer screens to interact seamlessly

Researchers in the US have invented a new technology that combines images on multiple screens, allowing your smartphone to interact with what's happening on your laptop, tablet, and television screens in some truly amazing ways. Researchers from MIT's Tangible Media Group, which exists to make the digital world feel more physical, and Fluid Interface Group, which aims to make human-computer interactions more seemless, are working on a project called THAW to enable your screens to interact with each other in entirely new, and really exciting, ways.  Imagine if you wanted to transfer a file from one device to another, and didn’t have to bother with bluetooth connections, or uploading it to the dreaded, privacy-threatening cloud. Imagine if all you had to do to transfer a song file or a digital photo from your laptop to your smartphone is hover your phone over your laptop screen. Or what about if you paused a video game you've been playing on your Playstation console, transferred it to your phone by hovering it over your television screen, and continued playing it as you left the house. "We live in an increasingly digital world, but that world is fractured between many screens and interfaces,” Philipp Schoessler from the Tangible Media Group told John Brownlee at Fast Company. "The question we wanted to try to answer with THAW was how can we combine these computer interfaces and screens into a single seamless experience.” According to Brownlee, THAW works by first having a coloured grid placed on a computer screen. The lens of an iPhone’s camera is then hovered over the grid, and a sensor figures out where on the computer screen it’s positioned, sort of like how sensors tell a computer mouse where to put a curser on your computer screen. "But THAW is much smarter than your average mouse: it can see what's underneath it, and use the phone's screen to interact with it. To not distract users, the team devised a way to hide the colour grid everywhere except directly underneath the iPhone camera,” he says. Not only could you theoretically transfer the same video game between multiple screens, but you could also use your screens to completely change its game-play mechanics. As a die-hard Super Mario Bros. fan, one of the team, Sang-won Leigh of the Fluid Interface Group, has figured out how to use THAW to turn a smartphone into both an obstacle and an aid for a character in a video game that's being played on a laptop. Brownlee explains at Fast Company: "In Leigh's game, the goal is to move a polar bear to a flag at the end of the level, but each level has a different mechanism. In one level, you might have to cross a pit full of spikes by using your iPhone as a physical platform for your bear to jump across; in the next, you might capture the bear in your iPhone, physically shake the device, then shoot him over to an island across the world like a champagne cork."