The Future of Print Media?

Work:

Vuforia's been around the block a while. As apple and google begin throw their hats into the mobile AR ring, people are beginning to look at augmented reality as a real possibility for the first time. But for those in the know, the biggest name in the AR game is Vuforia. The image tracking software has been around since late 2012, and now that the AR SDK was officially integrated with the ubiquitous Unity game engine in 2017, it's going to see more use than ever.

Its latest iteration, 'Vuforia 7' introduced some new and exciting features. Namely, the ability to project AR content based on SLAM tracking, as apple and google's offerings do. For this experiment, I wanted to re-introduce myself to their SDK, explore some of their new features, as well as attempt to create a spec piece for the 'newspaper of the future', an AR-enabled paper integrated with live webpages and video content.

In doing so, I found that, while much has changed in the last five years, some things have stayed the same. The quality of their image tracking still depends very much on the quality of the image you're asking it to track - while the front of a newspaper is visually complex enough to perform well as the equivalent of a QR code, individual images on the surface of the paper don't fare as well, leading to a bit of a dillemma. Projecting individual video pieces over specific images allows for the paper to be more reactive - as the surface of the paper bends, the individually tracked images allows the AR content to be projected at various angles, providing an enjoyable tactile response and leading the user to perceive the AR content as contiguous with the tracked surface. 

However, the size and quality of the images on the surface of the paper don't make for good tracking points, so all of the AR-enabled content will not be projected onto the surface unless you get really close and focus on one image at a time.

Therefore I opted to use the entire front page, back page, and front/back spread as the 3 tracked 'images', and arranged the models and videos to be rendered accordingly. This allowed the application to render all the AR-enabled content at once, at the cost of some flexibility and tracking accuracy - You can see in the above  video that all the AR content sometimes spills over the edge of the page or becomes missplaced. When the paper bends, the 'flat' virtual surface does not bend with it, which would not be the case with individually tracked images.

Above all, I had fun creating this 'harry potter' style video-enabled paper, and I see a lot of use-cases for this sort of print media experience. Look forward to more soon!

---

Next Steps

• find HQ scans of McClatchy papers for higher-res image trackers.
• Explore new features in Vuforia 7 - enable SLAM tracking
• Create "pop-up theater" style video viewer within paper
• Enable live web pages and virtual buttons to view links embedded in paper
• Create "weather visualizer" to display live meteorological data overlay

Feedback

"It's like a harry potter paper!"