| The goal of Workpackage 2 is to establish the neuroscientific (brain functional, neurophysiological, behavioural and cognitive) basis for haptic enhanced multimodal interaction. It is generally known that we can only create effective devices to provide high fidelity input for our senses if we understand at least the underlying basics of perception. Typical examples are colour movies (why three color channels? Why displaying between 25 and 100 Hz?) or audio compression, which is perfectly adapted to the underlying perceptional mechanisms. These individual human sensory systems are now studied for decades and we have an incredible wealth of neuroscientific information about them to allow the development of highly advanced feedback technologies. This is clearly not the case for dynamic haptic-enhanced, multi-modal manual interaction. Our knowledge about the underlying neural and perceptual mechanisms is still very limited, which simply does not allow selecting the right technological solutions to the arising problems. For instance, neuroscience research will provide information about perceptual adaptation and recalibration processes - a way to perceptually overcome asynchrony introduced by the computer system, which is essential for the correct technological design of haptic systems. Mechanism for perceptual prediction of interaction are also central in design of technology – if the essential cues needed for the brain to predict interaction are known, then these can be integrated in the technology for optimal interaction. In summary, neuroscience should deliver a psychophysical and cognitive model that defines the boundary conditions for the design of the new interfaces and provide information about the dynamics of human perception allowing implementing appropriate control strategies. This should be done by identifying regularities of our multimodal perception for interaction with passive and active partners allowing the simplification of simulated signals mimicking the real world to something manageable by our technology.
|