A hand that can see

A prosthetic hand with an embedded camera and computer that “sees” the environment and determines what are the appropriate actions to perform. Credit: Newcastle University

Creating a prosthetic hand that can move like a “hand” and operate like a “hand” is much more difficult that it would seem. It is not just about being able to move and create a shape that fits the purpose, as an example keeps the fingers sufficiently open to go around a egg and then closing them to pick it up. It is also about “knowing” that you cannot squeeze an egg unless you want to break it.

People having a prosthetic hand need to look at what the hand is doing and signal the hand what to do. Even this is quite difficult since you need to have the sensation of the pressure on the fingers to finely tune the pressure on the object.

Advances in this area have led to embed pressure sensors in the artificial fingers and in the joints and to relate their signals, electrically, to the muscles of the arm to send the sensation to the brain. Even with this advanced prosthetics the person wearing them needs to pay close attention, quite differently from the seamless actions we perform continuously with our hands.

Now a study at Newcastle University published in an open paper on the Journal of Neural Engineering reports on the embedding of a video camera in a prosthetic hand whose images can be interpreted (using a deep learning software) to understand what the hand is supposed to do and activate the required movements.  The goal is to achieve a seamless operation of the hand, like we are used to with our hands.

It is yet another example of a symbioses between human and machine. Actually, one may see that this innovation can be used to improve the performances of an autonomous robot.

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.