Programs like Sketchup have allowed amateur artists and architects to sculpt and design in three dimensions for years, but this app facilitates far more informal works that really look like sketches in 3D.

Mental Canvas is part of a suite of software for Microsoft Surface that takes more conventional sketched forms and allows users to navigate and expand them in three-dimensional space, retaining the character of rough drawings.

The infinite-depth approach of this digital canvas software lets users zoom in and rotate their view, adding layers along the way and inspecting their work from various angles. The effect is remarkable as these illustrations show.

From its creators: CAD modelling “comes at a cost. The cost is that one, you have to have a fully resolved three-dimensional model, even if it’s just a cube; second it’s very hard to edit a CAD model, it’s not fast and fluid like a sketch. At any given moment when you’re sketching, the designer or artist has full control over everything in that representation, but with a CAD model, that’s defined by the computer.”

This clever approach liberates an artist from the ordinary constraints of a physical page or even a drawing tablet, letting them shape worlds that maintain the character of a sketch but can be used to tell and frame stories in new ways.

As with programs like Sketchup, a bookmarking tool allows users to save and store various views then retrace their steps, creating potential for 3D storytelling. It also lets a user pick the perfect shot, navigating three-dimensional space to select an ideal perspective. A dynamic dial tool, meanwhile, lets the artist rotate between colors and line qualities without backing out of the drawing.

As any good architect knows, an unfinished-looking drawing can be an asset: “A sketch has a quality when it’s unfinished; there’s so much possibility, there’s so much room for interpretation, and [Mental Canvas] expands the idea of a sketch. It makes it less flat. In architecture you always want that “Wow” design that gets everyone excited in the room, and [Mental Canvas] enhances it.”

One can see a lot of similarities here between Mental Canvas and tools being developed for augmented and virtual reality platforms. In a way, this program closes the gap, allowing users with more ordinary tools, devices and interfaces to accomplish something wonderful in 3D space.

Using a series of networked camera-and-projector units, this working demonstration turns an entire living room into and interactive gaming environment. Dubbed RoomAlive (a successor to IllumiRoom, featured previously), this augmented reality expansion from Microsoft could transform not just the way we game but ultimately the way to interact with everyday spaces both private and public.

Existing objects and furniture within the space are automatically mapped and integrated into a given game or experience. The constituent procam units are auto-calibrating and can self-localize so long as there is some overlap in their fields of view. The entire array is designed to be simple to install and use – it effectively sets itself up once you put the technology in place.

The system creates a depth-aware 3D model of the room, identifying vertical and horizontal surfaces, which is plugged into a Unity gaming engine. It then populates the space with whatever the user desires, from passive environments to active games. Virtual characters can navigate along the floor then up along the walls, for instance, but users can also dodge and shoot, interacting as characters themselves. Items in the room can also become props, toys or tools within a programmed scenario.

This particular demo integrates six Kinect cameras to show various scenes and scenarios – just the tip of the iceberg. These could, of course, also be deployed in other settings to turn public or institutional spaces into fully interactive environments. For now, this is just a demonstration, but as these technologies become less expensive and more accessible it is easy to imagine commercial applications of all kinds.

From from the Microsoft development team: “RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented entertainment experience. Our system enables new interactive projection mapping experiences that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment.”

“The basic building blocks of RoomAlive are projector-depth camera units, which can be combined through a scalable, distributed framework. The projector-depth camera units are individually auto-calibrating, self-localizing, and create a unified model of the room with no user intervention. We investigate the design space of gaming experiences that are possible with RoomAlive and explore methods for dynamically mapping content based on room layout and user position. Finally we showcase four experience prototypes that demonstrate the novel interactive experiences that are possible with RoomAlive and discuss the design challenges of adapting any game to any room.”

What ever happened to in-home 3D technology and virtual reality? Perhaps futurists got a bit ahead of themselves, and maybe the way our eyes and minds focus means that many entertainment experiences must be focused on a screen … but that does not mean they are limited to it.

From Microsoft: “IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to combine our virtual and physical worlds. For example, our system can change the appearance of the room, induce apparent motion, extend the field of view, and enable entirely new game experiences.”

Beyond a cool gimmick or special effect, on the one hand, or a practical extension of screen space on the other, this is about enhancing your experience – tapping into the background, like surround sound does.

In some cases, context can be added – snowflakes sifting downward as you race on a MarioKart winter level. In other cases, useful peripheral elements can show you the broader built environment beyond your main area of focus – skies and streets above, below or off to the site in wireframe format.