A chip that “sees”

Roberto Saracco May 21, 2017 Blog 241 Views

The IMX382 chip is a digital image sensor that processes the image to detect and identify objects at 1,000 frames per second. Credit: Sony

The digital image sensors that we have in our smart phones and digital cameras capture photons and sends the data about each of their pixels with a string of digits expressing the luminosity and colour. The sensor does not know if a pixel is related to a house or to a dog…

In these last five years digital cameras (including the ones embedded in smart phones) have become smarter and several of them are able to recognise a face, a smile, the relevance of an object in the frame to provide the best possibile exposure or to track it as it moves. However, this ability is not embedded in the sensor but it is part of a special chip processing the image.

Now Sony has announced a chip, the IMX382, that has the capability to recognise objects directly in the image chip. The chip has a (relatively) low resolution, just 1.27Mpixels, way below the kind of resolution we got used to in our smartphones and digital cameras (most of the time over 10Mpixels) but it has a very high frame rate, 1,000 frames per second.

It has been designed with robots in mind, to provide industrial robots with the capability to detect objects around them, take a look at the clip.

The chip consists of stacked layers, photoreceptors, backlit pixel array and signal processing circuit. This latter comprises the image processing and a programmable column-parallel processor. Actually, this organisation looks a bit like the one we have in our retina, where some image processing takes place before the data are sent to the brain.