Microsoft sheds some light on its mysterious holographic processing unit

Since it was first unveiled, we’ve learned bits and pieces about the hardware inside Microsoft’s HoloLens augmented reality headset. But Microsoft’s custom Holographic Processing Unit (HPU) has always posed something of a mystery. At Hot Chips this week, the company finally shed a little light on what its special chip is doing. Ever since we first used HoloLens, we knew that it had some special hardware. Our first units weren’t the sleek all-in-one devices that are now available to developers and corporations for $3,000. Instead, each of our devices had a bulky chest-mounted unit that contained an FPGA (a kind of chip that can be rewired on-the-fly to change its behavior), fans to keep it cool, and an umbilical cord to provide power. That FPGA was the precursor to the HPU that the HoloLens headsets now contain. The HPU integrates data from the HoloLens’s sensors (accelerometers to detect motion and a Kinect-like camera system to detect depth). The chip uses those sensors to recognize gestures, maintain a map of the environment, and ensure that virtual 3D objects retain their position in the real world.”