AI Makes Google Plus Photos Sharper

Google Plus Photos is an excellent service for storing, editing, and sharing pics taken with your phone. Unlimited free storage for compressed files, adequate for most smartphone cameras, along with instant upload, and in app editing and sharing make using it a no-brainer. If you use a dSLR or otherwise wish to store super sized files, you can dip into your free storage or purchase more. (I’ve never noticed quality problems with my photos, and I allow my files to be compressed so as to qualify for free, unlimited storage.)

Google’s announcement that it will use AI to enhance compressed photos by 75% is interesting. Its easy to go from a crisp photo to a grainy, pixelated image, but its hard to go the other way. But that’s exactly what Google is doing. Unfortunately it’s not available to Google Photos users writ large yet, however is offered for select Google Plus users.

Less Bandwidth

Photos take are, at least compared to text, large files requiring more data and time to download. Where a user has a poor connection or limited data plan, compressing photos makes a lot of sense as smaller images equate to smaller file size. But such an approach sacrifices quality for speed and size.

Downsampling

Downsampling is the process through which a large image is compressed. It works by taking several very small pieces of the image and combining them. Imagine a checkerboard where, in full resolution, each cell is rendered either black or white. In downsampling several squares will be combined to yield fewer, larger blocks of some intermediate shade. Through this process, the file shrinks in size as it is called upon to store fewer pieces of information. The cost is blurred lines and muted colors.

Crime dramas on TV may make ‘enhancing’ grainy images look easy, but it’s not. Doing so requires figuring out what the underlying, downsampled pixels were.

Upsampling

In a crime drama, an investigator may ‘enhance’ a pixelated license plate image, for example, with ease to yield crisp numbers. This makes for a great show, but in reality, it’s more likely that the human eye interprets the license plate number from a larger picture. As downsampling is taking fewer ‘samples’ of an image so as to represent it in fewer pixels, upsampling (interpolation) is the process of going from a low quality image to a higher quality rendering.

Example of image compressed and processed by RAISR.
Example photo compressed and then enhanced through RAISR. Compression reduces the amount of data necessary to transmit the photo by 75%. Photo by Google.

Humans can (somewhat) follow the lines of the image, block by block, to fill in the missing curves and sharpen colors in the mind. Asking a computer to do so is a taller order.

RAISR

Computers lack the human intuition to say that a fuzzy figure is a ‘3’ or an ‘8’ in a grainy picture. But what if computers could be trained to recognize the patterns that result from downsampling various shapes? Then could they backfill the missing detail to sharpen up those compressed pictures? Enter machine learning.

Diagram and images explaining RAISR.
Google’s RAISR (Rapid and Accurate Image Super-Resolution) process. The steps of the process are shown on the top and a RAISR processed image below. Photo by Google.

Google is training its brain to recognize just such patterns so that it can fill in detail missing from compressed images. Its process is RAISR or Rapid and Accurate Image Super-Resolution. Pairs of images, a high resolution and low resolution, are used to train Google’s computers. The computers search for a function that will, pixel by pixel, convert the low resolution image back to (or close to) the original high resolution image. After training, when their computers see a low resolution photo, they hash it. In hashing, each piece of information is combined through a mathematical operation to come up with a unique value, the hash value, that can be compared against the hash values of other, known images which were computed similarly. From this comparison, Google’s computers ascertain which function is required to convert the particular image (or perhaps piece of image) back to high resolution.

We can imagine a schema where low resolution image is downloaded to a user device and hashed locally on the phone, etc. The device could then send the hash value back to the Google mother ship, retrieve the required formulas, and implement them locally, generating a very high quality picture. Google says the process will be something along these lines, cutting file size by 75%.

The Next Step

What could Google have in mind with this technology? Clearly they are deploying it to allow full resolution Google Photo image download with lower data burden. But is there anything else? Perhaps they see it used more universally with Chrome, whereby any picture on the web is compressed, downloaded, and then upsampled, making webpages load faster. Or perhaps they will pair it with their unlimited photo storage option, allowing users to store a ‘pseudo’ high resolution photo that exists in the ether as a compressed file, but appears on the screen as full size.

Time will tell.

© Peter Roehrich, 2017

Author: Peter Roehrich

I'm a railroader, business analyst, and trained scientist.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s