Human collaborative tasks through object (POP)

A paradigm example is two people (remotely) manipulating a large object (imagine a table) from one room to another within a virtual or mixed reality environment. The application of this scenario would be training for construction, or testing the ergonomics of a scene to be actually constructed such as on a building site, or indeed remote manipulation of real objects in a teleoperator system. People can perform a variety of collaborative motor tasks such as pulling a rope, lifting a gurney, passing a ring through a wire, playing piano, or dancing, without communicating verbally. Haptics plays an important role in collaborative manipulation of objects in industry, e.g. object handling, assembly/disassembly, and system maintenance. Surgical situations are normally based on two collaborating surgeons where each adapts the own pressure to the pressure exerted by the collaborating surgeon. In these situations, synchronization, mutual adaptation and reflex actions are involved in performing the task. These require haptic sensing and haptic induced actions.

There is evidence that a type of haptic communication exists between humans performing collaborative tasks physically. This alleged communication is performed via haptic interactions (i.e. force and touch) and is automatically interpreted by the human performers. Our goal is to decipher this haptic communication system, sort of a “haptic language”, and study the characteristics of haptic collaboration between people and their ability to adaptively collaborate and interact with the environment. A model of haptic interaction with the environment is crucial to further understanding of design of haptic presence, a topic that is hardly studied.

Our hypothesis is that unlike the actions of two preprogrammed “robots” performing a task after being “fed” with all the system parameters and variants, the human subjects will have to go through a learning phase in which they will gradually adapt to the collaborative system, and converge into a mode of energy-optimal performance. We will conduct a series of experiments to explore the optimal collaborative tasks. The following are examples: We wish to conduct a simple experiment in which two people remotely located in a distributed environment. Some examples of typical collaborative tasks are:

1) Play music together such as: press the piano keys with the correct force exerted, play a violin together; 2) Create a specific harmonic motion by controlling a mass connected to two springs at its ends; 3) Hold a tray on both ends while balancing a thin cylinder on top of the tray; 4) Catch falling objects in a basket (the objects fall in various directions and the basket needs to be held in accordance); 5) Balance a permanent magnet between two electromagnets (one at the top, and the other at the bottom, similar to a magnetic monorail train); 6) Row a boat in a straight line on a river with current; 7) Assembly a windshield in a virtual car.

Of course only a few examples will be investigated (which will be determined early into the project), but they are mentioned here for insight and to explain that POP is of broad interest and huge potential impact in research, society, and industry. This task will be performed with/out any verbal interaction between the subjects. We shall examine and compare the task performance of the subjects under several parametric changes and compare a combined two-subject performance to a bimanual performance of the task by one person. We will also inspect and compare between the learning curves of the subjects in the one-subject and two-subject paradigms. We will explore the effect and adaptation to latency.

In addition we will examine the presence of an after-effect in the performance of the task unaccompanied after a period of learning to perform it in collaboration. The presence of such an after-effect would suggest the existence of different learning mechanism for each of the cases. In addition to the above, we would also like to discover and isolate fragments of the “haptic language”, which are the building blocks of haptic communication. Haptic collaboration in a “presence’ environment allows two collaborators to sense the exact same haptic simultaneously, an act that cannot be done in the physical world. This allows an expert to teach a novice what haptic feel corresponds to e.g. a fat lump vs. a more serious lump. Until now, novice surgeon for instance, had to experience the physical case. We will explore this new capability to test the learning curve of novices, in the selected collaborative task.

Haptic collaboration through local or distant distributed environments is still an open problem that is highly interesting for the industry (e.g. assembly of a windshield in cars), medical applications (e.g. collaborative telesurgery) and arts (e.g. collaborative sculpting). In such cases, collaborative haptics may shape into collaboration with an avatar. The avatar may be programmed according to particular behaviours that are consistent with optimal collaborative performance. Record-and-replay analysis provides a database for identifying tacit problems obstacles for optimal distributed performance, by that improve distributed presence.