Seeing in the dark with AI

Roberto Saracco May 26, 2018 Blog 839 Views

The 120 Mpixel sensor in APS format. Credit: Canon

Progress continues in better and better sensors. Canon announced a 120 Mpixels sensor in the APS format that will be available to third parties in a variety of fields, where such high resolution is desirable.

They have also announced availability of a low resolution chip, just 2.2 Mpixels (sufficient for HD video recording) that can operate in very low light situation. This is possible by the bigger size of each pixel, 19µm x 19µm vs the 2.2µm x 2.2µm of the high resolution chip. By having a bigger bucket (pixel) you can harvest the fewer photons flying around when it is dark.

The problem in getting just a few photons is that the noise in the electronic circuitry, as well as the ambient noise (photons fluctuations – read this very nice article on noise to understand the reasons for noise), becomes so big with respect to the few photons you capture that the image quality rapidly degrades.

Here is where Artificial Intelligence come to the rescue. Researchers at the University of Illinois Urbana-Champaign have published an article describing how they developed a deep neural network system that can brighten up dark images without letting the noise to ruin the image. You can take a look at the clip to see the (amazing) results obtained.

The deep neural network has been trained using 5094 pairs of images, once taken with a long exposure (hence having a good image plus electronic noise due to the long exposure) and once taken with short exposure (resulting in very dark images with low electronic noise but high photons fluctuations noise).

Artificial Intelligence is playing an increased role in image processing with some AI algorithm already being embedded in the camera and others being used in post processing.

Self driving cars and safety cameras are already using AI to perform their magic and we can expect AI to support Augmented Reality in the next decade, allowing us a better vision, be it at the wheel of our car during a foggy day or at night or may be looking in a forest to spot nocturnal animals…