Crazy Sophisticated Virtual Reality Camera Released by Lytro
Lytro has clear ambitions to be a leader in virtual reality. Now it has the hardware to prove it. The company just took the wraps off of an entirely new kind of camera, the Lytro Immerge, designed specifically to capture footage for virtual-reality experiences.
Looking like an alien beach ball on a sleek tripod, the camera is equipped with hundreds of tiny lenses and image sensors, all built to capture the entire “light field” — the company’s signature optical tech that records not just the color and intensity of light, but also the direction it’s traveling.
The Immerge isn’t just a camera, though. Lytro has created an entire end-to-end VR production system that encompasses a camera, a server, editing tools, streaming from the cloud and a consumer app for VR headsets.
“This is the first light-field-powered system for creating cinematic quality live-action VR,” Lytro CEO Jason Rosenthal told Mashable. “Nobody has thought about this end to end — from camera to storage to post-production to playback.”
Based on the scale and specs of the Immerge, Lytro has created something that has no equal in the field of 360-degree video capture. The closest thing is Google Jump, which debuted earlier this year as a platform for creating spherical videos with 3D-like depth. But Jump remains unreleased, and it cobbles together existing cameras (the first units will employ 16 individual GoPros) in a custom rig; Lytro designed the Immerge for VR from scratch.
“Google Jump is a great step forward,” Rosenthal says, “but to get true live-action presence in VR, existing systems were never going to get you there. To really do this, you need to re-think it from the ground up.”
For the person experiencing the Immerge’s footage, not only will it look sharp and three-dimensional, but they’ll also be able to move around within the environment, Rosenthal says. This is something that’s currently possible with completely computer-generated footage — such as in the short film Henry for Oculus Rift — but the Immerge brings it to live action.
The freedom of movement will only be a cubic meter or so (the same size as the camera sphere), but when wearing a VR headset, you often don’t want to move that much anyway.
The camera houses a frightening amount of optical technology. Each of the five horizontal “layers” within the sphere has an array of 360-degree cameras. Still, the controls are streamlined enough to run on an iPad, letting the operator select frame rate, white balance, ISO and more.
The operator can even choose where to weigh the visual data. Often in VR you want to direct a viewer’s attention in one direction, so the Immerge can be set to capture 270- or 180-degree video rather than the full 360 degrees.
The amount of data coming off the camera is enormous, and the server will hold up to one hour of footage. Lytro created custom editing software and plug-ins for existing workflows, and it even made a playback app for everything from Oculus Rift to Google Cardboard to Microsoft HoloLens.
“We have broad distribution across the landscape of VR — this part has been a nightmare [for content creators], says Rosenthal. “We’re going to dramatically simplify that for them. But to get the best experience, you’ll need a headset with positional tracking.”
Given the size and scope of the Immerge, it’s not surprising that it will have a price tag to match. When it becomes available in early 2016 (Lytro is targeting January), the whole system will cost in the “hundreds of thousands” of dollars, Rosenthal said, so it’s definitely something for studios and companies serious about creating VR content, not consumers.
However, Rosenthal says a leasing program is part of the plan as well, so a company would theoretically be able to rent the hardware and tools for a limited time at a lower cost.
Lytro collaborated with budding VR studios Vrse, WEVR and Felix & Paul to develop the Immerge system. Rosenthal says he was taken aback by the studios’ enthusiasm when the company first began exploring the possibility of creating a VR camera.
“They told us, ‘We cannot do what we need to do in VR without capturing the light field.’ I had the same conversation 3-4 times. About 14 months ago we started getting really deep on this, talking to these guys a lot more.”
Lytro had trouble generating consumer interest in its light-field technology via consumer cameras, and its hard detour into VR has yet to prove itself. But if the Immerge is the hit with content creators that the company wants it to be, Lytro may actually start delivering on its initial hype.
