New Google Brain research brings the ‘zoom and enhance’ trope to reality

Re-creating a pixelated image is the holy grail of impossible CSI technology. And while zooming in on grainy footage, enhancing the image, and seeing the culprit reflected in the victim’s iris is still the stuff of fantasy, some new research from Google Brain may have done the next best thing, using a pair of neural networks to process a 8 pixel x 8 pixel image and generate an approximation of the original. To be clear right off the bat: this isn’t some magical sharpening of the image — the faces it generates aren’t those of the initial person, but rather a fictional creation that represents the computer’s best guess at approximating the original. As Ars Techinca breaks down, the software utilizes a two different neural networks to generate the images. The first is a “conditioning network,” which maps the pixels of the low-resolution picture to a similar high-resolution one that gets used as a rough skeleton of how the face or room should look. The second is a “prior network” analyzes the pixelated image image and tries to add details based on existing images with similar pixel locations. The two outputs are then combined to create the (fairly impressive) final image. Full details on the process can be found in the published research paper here, for those looking for a deeper dive into the methodology.”