IEEE Future Directions
Related Articles
We are so used at doing most of our daily activity that we seldom stop to reflect on what goes on in our head. Our senses feed our brain with data and it is up to the brain to make “sense” and take action (actually this is not always true, our spinal ganglia are the ones taking action when you touch a burning stove…). And our brain is really good in making sense of the world (as long as it pertains to our usual daily life). So it can seamlessly spot when we drive our car that the road turns left and send appropriate commands to our arms to turn the steering wheel just … right! The light may be very different, noon and sunshine, sunset, dusk… Yet the brain compensates the differences honing on the meaning of what is “out there”.
Self driving cars have been made possible by huge progress in understanding images. This understanding comes from analyses made by software using deep learning (DL) and convolutional neural networks (CNN) technologies. What is fascinating, but scaring, in these technologies is that we do not know how they work! It is not like coding an algorithm where you can follow step by step what is going on. Here the processing keeps changing and it is elusive. With this technology the machine learns over time, so in principle it gets better. On the other hand one cannot be sure of the outcome because one does not know what has been learnt and what has not been learnt that cannot be inferred from what has been learnt so far.
In a machine working through algorithm you can program the concept: “you can pull a rope but you cannot push it”. Using CNN or DL a machine can come across a situation where a rope is being pulled but it is unlikely it can come across a situation where a rope is being pushed. On the other hand it may likely come across situations where poles are pulled and pushed. Will it associate a rope to a pole and hence “believe” that one can push a rope? This is just an example to give you a feeling of the diversity between an algorithmic based decision and an artificial intelligence based decision.
Clearly this difficulty in understanding how a machine think is a huge problem when you need to assure that its reasoning is correct.
Today testing is made by submitting different situations to the analyses of the machine and check versus the expected, correct, outcome. The problem is that as the machine is responding it is also changing its “experience” and that might result in a different outcome next time. Do you see the problem?
Researchers at Columbia and Lehigh universities have created a tool, DeepXplore, designed to test Deep Learning systems. The tool creates a variety of situations to test the Deep Learning system using world test images to expose logical errors in the neural networks. In the figure you see an example: the same image is shown with two different illumination. The system should derive the same understanding and therefore take the same decision but in this case it is not happening. The darker image is interpreted in a different way resulting in the car crashing against the guardrail.
The DeepXplore generates thousands of images simulating partial darkening resulting from dust on the camera lens or from an object partially blocking the sight. It has been able to spot a number of bugs that were not intercepted with usual testing systems.
The big question, how does a machine think remains, and it remains because machines are now mimicking, in a way, more and more our brain and the truth is, we do not know, yet, how we think!
