You’ve seen the movie where a crappy surveillance camera picks up an image of a distant car. The boss says “improve”, and suddenly the license plate is readable. I thought it was Hollywood hyperbole. But for better or for worse, this is an incredibly active area of research with amazing results.
The last refinement is detailed in an article – Single image super-resolution via a holistic attention network – researchers from Northwestern U and the State Key Laboratory of Information Security of the Chinese Academy of Sciences and others.
The most successful process to date uses convolutional neural networks (CNNs) to learn a mapping function from the low-resolution image to the high-resolution image. Deep CNN is made up of multiple layers of artificial neurons, with each node in one layer connected to each node in the adjacent layer. A deep network allows CNN to learn a complex mapping between low and high resolution image.
However, the architecture introduces a problem: each layer can only respond to the layer above. All the details of the shallower layers are encoded into what the deeper layers see in the only adjacent layer. The detail is lost.
Which is a problem. It’s like going to a committee that had many meetings, with only access to the notes from the last meeting. There is a lot of history – details – that you won’t be aware of.
Likewise, SISR CNNs lose detail as the network sinks. The contribution of the article is to detail a method to overcome this loss of history in deep CNNs.
The document presents an architecture that uses two new modules to examine the correlations between multiple layers and to learn the interdependencies of the functionality in each layer. Essentially, the new modules give you a summary of all the committee meetings you missed with the most important details highlighted.
But the proof is in the pudding. Here is an example of the quality of SISR results using the new modules compared to the old architectures.
Leaving aside the implications of turning every surveillance camera into a super-resolution device, this is some pretty amazing technology. Yesmask wearing becomes an ongoing part of pandemic management I expect similar concepts can be used to “see” through masks to identify individual faces.
Yet I also see the article as a reminder of the youth of AI as a discipline and the development we can expect over the next 50 years.
Adding a vertical component to the horizontal CNN layer stack doesn’t seem like a huge conceptual leap. Yet here we are.