Skipping the eyes, seeing with the brain

Brain-wide activity in a zebrafish when it sees and tries to pursue prey. Credit: Ehud Isacoff lab/UC Berkeley

DARPA has awarded 26M$ to the University of Berkeley as part of its NESD – Neural Engineering System Design – program.

The overall program, announced in January 2016, calls for the development of an implantable system supporting precise communications between the digital world and the brain, in practice converting electrochemical signals produced by the brain into bits and bits into signals that can be processed by neurones.

The University of Berkeley, the research team led by Ehud Isacoff, aim is a first step in this direction. They want to create a system able to interact with the brain of a small animal (thus having a small brain) proving that it is possible to establish a direct communications with a brain.

They chose the zebrafish, an animal that has already being used to show the activity of neurones in real time, and aim at reading, and writing up to a million neurones. So far, after one year, they have been able to read/write up to one thousands neurones, a small fraction of the final target and an infinitesimal number if compared with the 86 billion neurones in a human brain, but still something that just in the last decade would have looked like science fiction.

To read/write they insert two genes inside the neurones (using viruses as vectors), one that will create a protein that fluoresce when the neurone is active and the other that activates the neurone when hit by a flash of light (optogenetics). To record the light from neurones the researchers are using a light field microscope that has the potential resolution to pick up light from one million neurones.

The long term goal is to understand the “coding” rules to be able to code any content, like an image, into parallel stimulation of neurones in the visual cortex thus creating the desired image in the brain, without the need for an eye. This would clearly be an amazing result for people who have lost their sight for an impossibility of the optical nerve to transport signal from the eye (the electronic eye, like Argus II, requires a fully functional optical nerve).

We are still far from this possibility but just the fact that research teams have this objective is amazing. It means that it is no longer considered an impossibile task, just a very difficult one.

Notice that our vision system is very, very complex. Stimulating the visual cortex to create the sensation of an image is addressing just a part of the visual capabilities. We see, and we understand what we see. We “know” if what we see may be dangerous, it is something that gives specific emotions and so on. These aspects of vision are not located in the visual cortex and creating an image just there would create a partial vision, like the one experienced by people suffering from blindsight. Because of a damage to the visual cortex they cannot “see” but they feel the meaning of what they are “not” seeing. If you throw a punch to a person affected by blindsight he will move to avoid the blow but when asked why he moved he won’t know the answer.  Here it would be the reverse: you see perfectly the blow coming to you but you won’t move since you will not perceive it as a danger to avoid.

I am mentioning these complexity just to emphasise how difficult is the problem, not to downplay in any way the efforts made, and results already achieved. They are amazing.

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.