IEEE Future Directions
Related Articles
The fading boundary between bits and atoms is “fading” more and more as technology finds better and better ways to trick our eyes, actually our brain.
Varjo, a Finnish company, has presented a new visor that from a technological point of view is a marvel and from the experience point of view seems to make a significant step towards tricking our brain.
Our brain has developed, through million of years of evolution (we have inherited most of it from our ancestors), very sophisticated ways to recreate an inner image of reality. It is more than what our eyes see, although this is an important part. It is about how the eyes move, how “we” move as we look around and it is about what we expect to see.
As strange as it might seem, a short sighted person perceives images as well as the one of us with the highest visual acuity. It does not mean that that short sighted person actually see all the details that the other sees, but if you ask him he would tell you that he sees perfectly well. His brain is familiar with what those eyes see and makes the most of it. Besides, our eyes have different acuity in different part of the retina: if one were to take a snapshot of what the eye sees the image would be very clear (high resolution) in one spot and will get fuzzy (low resolution) in the other parts. The brain works out a perfect, high resolution image, by having the eye(s) scanning the world and merging the images into a single high resolution one.
Varjo does something similar. It uses 2 screens, one to provide a broad view of the surrounding with low resolution, and the other much smaller to provide high resolution. The two merge smoothly and trick the brain to see reality as it normally does. Notice that having a screen providing a broad view is essential to trick the brain into “feeling” that it is part of the whole. Without the feeling of immersion the perception of reality would disappear.
As mentioned before our perception of reality is also dependent on the movement of our eyes (saccadic movements) and on the movement of our head and body. The brain integrates those movement sensations into the images harvested by the eyes and they have to be consistent (some people get nausea when reading as they ride a car: that is a case where the movement sensation is not in synch with the images and the result is nausea).
In the case of Varjo, the software rendering the images that go onto the two screens takes care of integrating the movement effect. This is highly processing demanding and that is the reason to have the two screens. Having a broad high resolution screen would be too demanding in terms of image processing, having only a smaller screen would not trick the brain into feeling to be part of the scene.
Is this the end of the line? Not quite. The Varjo’s Bionic Display is bulky (and tethered) so it cannot really fool the brain although it gets pretty close in terms of images. It will probably take two decades for mixed reality/augmented reality to become a seamless part of our perception (by the way: mixed reality means to create an image where real objects and artefacts are present and are basically indistinguishable from one another, augmented reality means overlaying on real objects data/images to provide additional information – the distinction sometimes blurs).
A seamless perception will go through an ambient that is able to re-create artefacts and mix them with the objects present in the ambient or through the implant of some device that can communicate with the brain (like an electronic lens). This latter would be much more effective and probably will become available in the second part of this century, leading to a true human augmentation.
