Virtual Hand with Ambiguous Movement between the Self and Other Origin: Sense of Ownership and 'Other-Produced' Agency

Summary

While previous research on bodily self-consciousness assumed that self- and other-origin movements were perceptually distinguishable, this protocol allows them to be ambiguous on a virtual hand with unintentional slight movements. This enables us to observe one’s experience formed by SoO and other-produced SoA, rather than the absence of SoA.

Abstract

The feeling that a body part is one’s own body (sense of ownership; SoO) and the feeling based on the causal relationship between one’s will and action (sense of agency; SoA) have been recognized as the basis of our bodily self-consciousness. Previously, the illusory SoO over a fake body part (e.g., rubber hand) was introduced as the rubber hand illusion (RHI). Furthermore, it was determined that one could also evoke a SoA over an object with movements linked to the one’s prior intention. On the other hand, the postdictivity of our spontaneity implies that it is essentially inseparable whether actions originate from self or others. In other words, our SoA or daily experiences are obtained in such as inseparable scenario. Previous research, however, has maintained the premise that self- and other-origin movements are perceptually distinguishable. Here, we implement a protocol to make these aspects ambiguous for the participants and to estimate whether they can feel SoO and/or SoA and how. To this end, we employ an experiment using virtual reality, under which participants observe virtual fingers moving very slowly (or quickly or not moving) while their own fingers do not move. For evaluation of the illusory SoO, measurements of skin conductance responses against a knife threat are adopted. Additionally, we introduce face-to-face interviews to determine whether the feelings regarding the slow movement match the conventional SoA definition. Our representative results suggest that the SoO is evoked over the hand, and various attitudes to accept its movement as the participant’s own with awareness that they did not originate it are reported by the majority. As the results show, the novelty of this protocol is discovering that in such a situation, the SoO cooperates with an externally produced SoA to establish one’s own bodily experience rather than the independence of the SoO and SoA.

Introduction

How does one perceive one’s body or the actions one has performed as one’s own? These sensations constitute our bodily self-consciousness and our own experience. Such an issue, particularly in terms of “minimal self”, has been considered to comprise a sense of ownership (SoO), the sense to feel the body as one’s own, and a sense of agency (SoA), the sense of authorship over bodily movement¹. Many studies on SoO/SoA have been based on the comparator model², in which one’s actions are subjected to internal comparison processes^3,4,5,6. The comparator model describes a feed-forward loop linking intention with a prediction of one’s motion and a feedback loop relating sensory feedback to the prediction; these loops have been recognized as the SoA and SoO, respectively^1,6.

As a means of experimental investigation, body transfer illusions have been employed. A typical approach is the rubber hand illusion (RHI), in which synchronous stroking of participants’ hidden hands and fake hands located in front of them evokes illusory ownership of the fake hand⁷. While the original RHI refers only to the SoO, recent implementations of the RHI equipped with a movable hand have allowed assessment of the mechanism of SoA as giving experiences of feed-forward loop matching to the participants. Although the SoO and SoA coincide in normal behavior¹, they can be independent of each other, and both can be evoked separately. For example, if participants are shown an avatar, they feel only a weak SoO but not an SoA, or if participants see their hand hiding their thumb from a third person’s perspective, they feel only an SoA but not an SoO⁸. An SoO elicited by an SoA has also been reported⁹, and the relationship between the two and how both together produce bodily consciousness are still open questions.

Although SoA is a broad term, it is commonly regarded as a sensation of a generated action while self-detached from the external world. In this sense, in order to evaluate the SoA experimentally, it is assumed that the participants themselves manipulate the real body or the fake body, for example, using a hand image projected on a screen¹⁰, a fake hand located on a wooden pole¹¹, or a hand located on a cover above the participant’s hand^12,13. In these studies, when the visible fake hand was synchronized with participant-generated movement, the emergence of the SoA was confirmed. Additionally, when the movements were externally generated not by a participant but by an experimenter using a physical connection to the hand, it was confirmed that the absence of the SoA could occur, while the SoO was retained. In those studies, a pair of absence and presence of the agency was implemented by a pair of other- and self-origin movements for the participant.

The question arises: does the absence of the SoA imply that body movement is made by others except for the one? Is there a clear separation between me and others? However, myself could involve unconsciousness which could be others in myself. Concerning unconsciousness, one cannot determine whether a movement originates from the one’s self or others. Although the sensation of autonomous convulsion is destined to be relevant for unconsciousness, the previous experimental setups have not paid attention to the ambiguity of the self and others. To involve unconscious movement in an experiment, one has to implement a virtual move that cannot be readily determined as originating from the participant or others. Such ambiguity of the participant and others is also observed in the discussion of “readiness potential”¹⁴. The readiness potential preceding free will and intentional consciousness can be compared to others and me in a brain, respectively¹⁵. In other words, the self and others in one’s self are inseparable from each other. In fact, behaviorally, it has been pointed out that SoA could be acquired in a postdictive manner^16,17. Considering that our SoA or bodily self-consciousness could be acquired in such a self/other ambiguity, while previous research on SoO/SoA has been carried out after making them perceptually distinguishable, it is necessary to investigate what kind of subjective experience occurs when both parties are ambiguous.

Here, we introduce a protocol that enables us to evaluate SoO/SoA effects on a virtual hand with slight movements such that it is unclear whether the movement is self- or other-originated. Specifically, using virtual reality (VR), our participants were made to see a virtual hand from the first person’s perspective through a head-mounted display (HMD) without telling who the hand belongs to. Their hand was given a tactile stimulus (stroking with a paintbrush) in sync with the brush movement on the hand shown in HMD, and then, after one minute, the fingers in the video suddenly start moving slowly. For comparison, we conducted six trials for each participant, following three conditions (in duplicate): no hand movement, slow movement (ambiguity between self- and other-origin), and fast movement (clearly not self- but other-origin). To evaluate the SoO, skin conductance response (SCR) measurements were adopted. The change in SCR during a presentation of a threat was used as an indicator of bodily self^18,19,20, as it provides an online and objective estimation. In this study, we defined SCR arising from a knife threat following hand movement as the intensity of the SoO^18,19. Moreover, for evaluating the SoA, subjective reports were collected using face-to-face interviews after all RHI procedures. Here, it could be supposed that such an ambiguous movement between self- and other-origin would cause the bodily movement originating from another to be perceived as the participant’s own. Such a rewriting of the causal relationship in SoA cannot be captured by the pre-established method to simply evaluate the consistency between one's intention and one’s action. Therefore, while in conventional RHI studies, the participants were required to score given statements describing the sensation¹³, this protocol allows participants to describe their sensations in their own words. After the subjective reports are collected in such a manner, the steps for classifying and evaluating them are prepared. These series enable us not only to determine whether SoA was present or absent but also to observe how their sensations, which were accepted as their own experience, differed from the stereotypical SoA.

Protocol

All methods described here were approved to follow the Code of Ethics of the World Medical Association (Declaration of Helsinki) and approved by the Independent Ethics Committee of Nagaoka University of Technology. No identifying information was collected from the undergraduates. Participants were given detailed information about the experiments before participating and were required to sign a written acceptance statement. Written informed consent was obtained from all subjects and was stored at Nagaoka University of Technology.

1. Preparing seven kinds of videos

NOTE: In total, create seven videos for a pre-experimental session (one video) and for experimental sessions in the “stable”, “slow” and “fast” conditions (Figure 1) for each gender (three times two videos).

1. In an experiment room, set a table and two chairs, a hand rest (approximately 35 x 60 cm white cushion tilted approximately 25 degrees) and a display on the table (Figure 2). Mark these positions to be able to reproduce accurately in the experimental session.
2. Place a 360° camera at eye level when the participants are sitting in the chair (Figure 2).
3. Start to record the camera with nobody in the video for approximately 20–30 s for a pre-experimental session to ensure that there is reasonable visibility for each participant.
4. For the video to show male participants, let a male experimenter (Experimenter A) wear a lab coat, sit in the chair and put on his left hand on the hand rest with the palm up, taking care that the fingers do not touch each other nor touch the hand rest.
5. Play a 60 bpm 4-beat reference tone using a metronome application and a smartphone.
6. Let another experimenter (Experimenter B) sit facing Experimenter A, with a paint brush in Experimenter B’s hand (Figure 3).
7. Start to record the camera for the video to use the “stable” condition.
8. Using the reference tone, let Experimenter B stroke all of Experimenter A’s fingers for 80 s. At the same time, let Experimenter B memorize the trajectory to reproduce it later.
9. By the third experimenter, show an action that an approximately 30 cm kitchen knife (20 cm blade) appears, slashes Experimenter A’s left wrist and disappears, taking 5 s (Figure 4).
10. After 5 s of margin duration, stop the camera.
11. Start to record the camera for the video to use the “slow” condition.
12. Repeat step 1.8 while changing the brush trajectory to avoid monotony.
13. 60 s after the start, let Experimenter A close and open the five fingers at a consistent slow speed for 20 s (20 s/close and open cycle, approximately 1.3 mm/s; Figure 5(A)).
14. Repeat steps 1.9 and 1.10.
15. Start to record the camera for the video to use the “fast” condition.
16. Repeat steps 1.12 to 1.14 while changing the finger speed from slow to fast (2 s/open and close cycle, approximately 35 mm/s; Figure 5(B)).
17. Repeat steps 1.4 to 1.16 while changing from a male Experimenter A to a female Experimenter A to produce another three videos for the female participants.
18. Remove the camera. Check that the videos do not diverge from the real world, using the HMD and sitting the chair and putting left hand on the hand rest.
19. If necessary, practice reproducing the trajectory of the brush accurately.

2. Experimental procedure

1. Use more than 18 participants who are healthy adults (approximate ages of 18–25 and a gender ratio of 50% are better) without history of major illness and naïve to the experiment and speaking the same native language to collect subjective reports. Experiment with each participant separately.
2. RHI with/without finger movement sessions and SCR measurements
   1. Conduct the following steps that include the role of interacting with the participant by the same experimenter with the same looks as Experimenter B in the videos to use. Use assistance if necessary.
   2. Ask a participant to wear a lab coat and sit the chair. Attach Ag–AgCl electrodes connected with an SCR data acquisition device to the index and ring fingers of the participant’s right hand after polishing those fingers. Keep the hand down while not touching anything. Set the sampling rate at 500 samples/s, and start to record the SCR data.
   3. Give to the participant the following three instructions:
      1. “Your task is put your left hand on the hand rest with the palm facing up and keeping the hand stable”
      2. “Please try to relax.”
      3. “When wearing the HMD, please look at your left hand”.
   4. Have the participant wear an HMD with a black screen. Mirror the screen on the HMD to the display on the table until the end of the experiment.
   5. Show the video for the pre-experimental session on the HMD after announcing that. Ask the participant to look around and ensure that the view is natural. If something is wrong, adjust the HMD or the seat height until the participant is comfortable.
   6. Show a black screen on the HMD, and ask the participant to put his/her left hand on the hand rest with the palm up. Help the participant, and take care that the fingers are not touching each other or the hand rest.
   7. Sit facing the participant with headphones and a paintbrush. Prepare for one video for one of the three conditions with the left hand of the same gender as the participant.
   8. Begin the first trial with one condition randomly selected from the three. After announcing that the session is beginning, start the video. Brush the participant’s left hand with the same timing and position as s/he is watching, checking the mirroring display on the desk and the reference tone, until the knife cuts in (Figure 3(B)).
   9. Show a black screen on the HMD 2 minutes to allow SCR to return to normal.
   10. Repeat steps 2.2.8 to 2.2.10 for the remaining five times (a total of six times; two trials of each of the three conditions) in random order. Ask the participant after every two trials whether anything is wrong with the physical condition.
   11. Stop recording the SCR data. Remove the HMD and SCR data acquisition device.
3. Face-to-face interviews: Pose the following questions to the participant and collect the answers orally while recording them.
   1. Ask “Did you see that the hand was moved?” [Q1: For confirmation]
      If the answer is not “Yes”, go to step 2.3.4.
   2. Ask “Did you see the movement of the two speeds? [Q2: For confirmation]
      If the answer is not “Yes”, go to step 2.3.4.
   3. Ask “What did you think about the slower hand movement?” [Q3: For SoA evaluation]
   4. Ask “Could you tell me if you felt anything during the experiment.” [Q4: Free report]

3. Data analysis

1. SCR data
   1. Identify the SCRs for each trial in the following manner^19,21,22: the difference in amplitude between the maximal and minimal value that occurred during all of 2–10 s in the last 5 s (knife threat; for evaluation of SoO) and immediately after 5 s.
   2. Calculated the magnitude of amplitudes according to a pre-established method²³ as:
      
   3. Conduct the Shapiro–Wilk test on all data. If it is confirmed that some data were not normally distributed, use nonparametric statistical tests to process the data; otherwise, use parametric statistical tests. In all statistical tests, set alpha to 5%.
   4. Evaluate the difference among the three conditions with a multiple comparison.
2. Face-to-face interviews
   1. Divide the data of those who answered “No” to Q1 and/or 2.
   2. Compare the answers to Q3 with the following three statements that have been used to measure participants’ SoA in the RHI study¹³.
      Statement 1: The hand moved just like I wanted it to, as if it was obeying my will.
      Statement 2: I felt as if I could control the movements of the hand.
      Statement 3: I felt as if I could cause the movement I saw.
   3. Classify the answers to Q3 from a summary of each, and analyze the overall trend.

Representative Results

The participants were 21 healthy students (eleven males; mean age 21.1 years, range 20–23; 18 right-handed) recruited at the Nagaoka University of Technology. All participants were naïve to the experimental tasks and native Japanese speakers. Data from 19 participants, including those exhibiting no response (a “null response”), were used, but data from 2 participants were excluded because of equipment problems.

As the result of our study, SCR values with a knife threat showed no significant differences between the three conditions (slow vs stable: p = 0.829, t = 0.581; slow vs fast: p = 0.872, t = 0.499; and fast vs stable: p = 0.988, t = 0.145, Steel–Dwass test; Figure 7(A)). This result should be recognized as negative to our hypothesis that the fast movement of the shown hand would eliminate SCRs, namely, the intensity of an illusional SoO. However, when the SCR of the first and second duplicates were compared for each condition, a significant decrease in the SCR was observed only in the condition with fast movement (slow: p = 0.984, z = 0.0197; stable: p = 0.922, z = 0.0983; and fast: p = 0.0181, z = 2.36, Wilcoxon signed-rank tests; Figure 7(B)). This indicates that if obvious unpredicted (fast) movements are repeated, the illusory SoO would disappear, but if the movements are slow, then the SoO would be maintained at the same level as that for no movement.

For the face-to-face interviews, all the participants answered “Yes” to both Q1 and 2; therefore, it was confirmed that all of them observed slow movements by their eyes. After confirming this, it could be evaluated that their answers to Q3 did not include Statements 1-3. The responses can be divided into the following four types (Table 1). Type 1: six participants answered “I did not move” their own hand. Type 2: Another six answered with “I thought that my hand was moving without my will” or the equivalent. Type 3: eight answered with “I was drawn to be moved.” or the equivalent. Type 4: three claimed “a feeling of moving my fingers.” One participant’s answer included both Types 2 and 3, another included both Types 2 and 4, another included both Types 2, 3 and 4, and the rest contained only one of Types 1-4. Types 1, 2 and 3 are clearly not consistent with any of Statements 1-3 in terms of denying the will to move the hand. Moreover, Type 4 does not include causality; thus, it is also incompatible with any of Statements 1-3. Overall, the results suggest that none of the participants evoked the SoA in the conventional sense. Moreover, it can be considered that while Type 1 simply denied their SoA, Types 2-4 indicated their attitudes to overlap themselves with the observed bodily movement that was generated externally. For example, Type 2 accepted the observed movement as their own, recognizing that it was not accompanied by their will. Type 3 mentioned their motivation to bring consistency with the observed other-origin movement, that is, such external movement was likely to cause their action. Type 4 claimed a feeling of sensory feedback, despite being aware of the lack of their will or their action. Taken together, 13 out of 19 participants described the observed event as their own experience, which was formed by the other originated SoA rather than rejection of the SoA.

Figure 1: Procedural flow chart for one trial in the RHI experiment. In all three conditions, one trial was completed within 90 s. The SCRs were identified from 80 s to 90 s in each trial. Please click here to view a larger version of this figure.

Figure 2: Prepared setup. The setup for steps 1.1 to 1.17, preparing the videos. A 360° camera is located in the participants’ point of view. Please click here to view a larger version of this figure.

Figure 3: A frame from a video shown to participants. Each participant is shown a virtual hand that is located in the same place as his/her own real hand. Please click here to view a larger version of this figure.

Figure 4: The knife threat event. For the SCR measurement, 80 s after the beginning of the trial, a kitchen knife appears, slides across the shown hand, and is removed. Please click here to view a larger version of this figure.

Figure 5: The close and open movements of the virtual hand. 60 s after the beginning of the trial, the open hand shown to the participants starts to close at a consistent speed, taking 10 s in the slow condition (A) or 1 s in the fast condition (B). Soon after that, it opens at the same speed. In the fast condition, this action is repeated 10 times. Please click here to view a larger version of this figure.

Figure 6: Experimental setup. The setup for conducting the trials. Participants face the same background and the experimenter in the actual setup as in the video. Please click here to view a larger version of this figure.

Figure 7: Boxplot of the mean SCR following the knife threat for trials involving no hand movement (stable) or slow or fast hand movement. (A) Division into three groups and (B) into six groups (separated the first and second trials). Error bars indicate standard deviations. When the results of the first and second rounds were combined and multiple comparisons were performed under each of the three conditions, no significant difference was found in any pair. This result contradicts our hypothesis that the fast movement of the shown hand would eliminate SCR (the intensity of illusional SoO) because it was the same level as the other two. However, when tested between the first and second trials within each condition, a significant difference was observed only in the fast condition (the first vs second trial in the fast condition; p < 0.05, Wilcoxon signed-rank tests). Moreover, in the trials with slow movement and no movement (stable), both the first and second trials showed high SCR values that were not significantly different. This suggests that while in the fast condition, the SCR of the second trial was not maintained to the same degree as that of the first, in the slow and stable condition, the SCR was maintained even though the event was repeated two times. Please click here to view a larger version of this figure.

	Summary of the answer		number	overrup
Type 1	I did not move.		6
Type 2	I thought that my hand was moving without my will.		6	a	b	c
Type 3	I was drawn to be moved.		8	a		c
Type 4	I had a feeling of moving my fingers.		3		b	c

Table 1: The result of typing answers to Q3 from a summary of each. Q3, the question to evaluate illusory SoA, “What did you think about the slower hand movement?”, was posed after confirming that the participants had observed the slow movement of the hand. Three participants (a, b and c) made multiple descriptions. Overall, there is no answer type that includes the statements that matched the definition of the SoA. Moreover, Types 2-4 show various attitudes towards accepting the observed movement as their own movement while recognizing it as originating from elsewhere.

Discussion

The purpose of this protocol is to highlight the inherent inseparability of whether the origin of action is the participant’s self or another. It could be supposed that such inseparability causes the participants to accept the movement of alien origin as their own experiences. Here, we proposed a method to collect the sensations in such a situation, referring to the conventional discussion of the SoO/SoA. As a means of achieving this, step 1.13 (preparing the video) and step 2.2.8 (showing it under plausible circumstances) are achieved by appropriately shifting the behavior of the observed body part from that of the actual. Some other steps support this aim from the following two points. The first point is to make the virtual image closer to reality. Thus, the world in the video is visually aligned to the real world regarding the hand features (steps 1.4, 1.17 and 2.2.3), laboratory landscapes (step 1.1), and the facing experimenter (step 2.2.1). In addition, establishing cross-modal interaction would also be effective, as it is considered to be essential, especially in the full body-transfer illusion^24,25. Therefore, it would be effective to use a 360-degree image (step 1.2; matching the visual information and head movement control) and to accurately trace the trajectories (step 2.2.8; matching of visual and tactile sensation). The second point is to broaden the interpretation of (virtual) visual information. This point was also emphasized in another study that made participants uncertain of whether their view in HMD was virtual or real²⁶. While the authors achieved this goal by reducing the virtual and real visual resolution equally, in this protocol, we applied the following two methods in step 2.2.3, to give no information about what would be shown and to make themselves relax. These cares seemed to work considering the reaction to a question “Guess what hand you were looking at?” that we asked to all participants after Q3 to confirm the validity of this protocol (this is an extra question that does not need to appear in the protocol). Then, rather than having no one correctly answer that it was a prerecorded video, it was found that the majority of them were experimenting with uncertainties and allowing various possibilities.

Additionally, step 2.3 is considered to be important in evaluating such feelings that the bodily movement originating from others is perceived as one’s own and that may also contain contradictions when viewed objectively. While in the typical method using a questionnaire about SoO/SoA, participants are presented the preprepared expressions about the feelings first, in this protocol, they are requested to describe their feelings in words of their own. This way enables us to avoid reinterpretation of their experience by the given expressions by the experimenter and to collect contradictory answers. Actually, as shown above, the majority of our participants reported that the observed move was compatible with what was relevant to them. Moreover, we could collect the answers “it was like mine and not mine” from three participants in step 2.3.4. In addition, it is also notable that illusory sensory feedback was reported in Type 3, although the number was small. Those points that had not been observed in previous RHI studies might offer us new insight.

Regarding possibilities to improve and troubleshoot this method, a common problem is that accurately tracing the paintbrush trajectory is important but difficult. Indeed, the illusion disappears as soon as the movement of the brush is asynchronous^7,27. Our subjects also reported that the sense of ownership suddenly disappeared at the moment when the experimenter failed to trace by accident. While it has been reported that the illusion intensity increases with increasing complexity and randomness of the brush movement, we recommend taking a simple trajectory that is not easily mistaken rather than taking the risk. In addition, regarding SCR measurement, one could be concerned that the participants may get used to the repeated knife threat. In some previous studies^19,22, however, the experiment was conducted many more times (for example, thirty times¹⁹). This protocol proposes two trials for each condition, for a total of six times per participant to reduce their burden, as the minimum number of times that would reveal the difference between the three conditions. Furthermore, it is possible to consider increasing the number of trials for more statistically reliable data collection and to study the effect of iteration of the trial. Another point, our materials table shows the set of equipment that we used to obtain our representative results. However, especially since the head-mounted display manufacturers are frequently updated to new products, the catalog number (Oculus DK2) in the table is now discontinued. Currently, the same experiment can be performed by using the latest model (Oculus Rift S or Quest 2; in this case the corresponding display is a PC with Windows OS instead of Mac OS), and we recommend carrying out with the product available at that time.

A considerable limitation of this protocol is that the estimation of SoA depends on the retrospective method. The reason that the face-to-face interview is set as the last step of the experimental procedure part and not just after the target trials (the trail in the “slow” condition), such as general methods¹³, is to prevent the participants from focusing their attention on the question in the subsequent trials. Considering that this risk had a greater impact on the entire experiment, we set the current order. On the other hand, this case in which other trials are inserted between the target trial and the interview arises the risks that the acquired feeling would be mixed or forgotten. Here, that issue was mitigated by using questions 1 and 2. Furthermore, the use of general questionnaires is still retrospective. Therefore, developing a measurement method that can be performed in parallel with the experiment using physiological indexes such as SCR could be said as a common issue. Moreover, it may be possible to observe some differences when comparing the trends of the subjective reports against slow and fast conditions. While repeating questions has a risk of affecting participants’ retrospective subjective reports, a possible approach is a comparison between groups, to ask about only one condition to each.

In summary, this protocol has implemented a situation in which the SoO coexists with the SoA that is inconsistent on one body part, creating a good balance of difference and consistency between the shown state and the actual state. This time, for the sake of simplification, the participants’ real hand is made to be stable and the shown hand is made to move slightly, but the opposite case is also applicable. In that case, it is important for each participant to move their fingers at a uniform speed and distance; thus, it is necessary to confirm and practice this process well in advance instructions. Moreover, a wide range of applications can be imagined, such as the case with a change in the shape and/or the number of shown fingers or the case with intervention and/or interaction with the obvious hands of others.

Materials

Name	Company	Catalog Number	Comments
display	Apple Inc.	MacBookAir	for mirroring the screen which is shown to the participants in the HMD
head mounted display	Oculus VR Inc.	Oculus DK2	for dispaying the 360° videos
metronome apptication	ihara-product	Smart Metronome	for generate reference tones
SCR data acquisition device	Biopac Systems inc.	Biopac System MP150	for SCR data mesurement
SCR data processing software	Biopac Systems inc.	AcqKnowledge 4.1	for SCR data processing
smart phone	Apple Inc.	iPhone SE	for playing reference tones
statistical analyzing software	The R Foundation	R 3.4.2.	for statistical analyzing
360° camera	RICOH Company Ltd.	RICOH THETA S	for preparing the videos