Place:
Linz, AT;
Site:
Time's Up Laboratories
Date:
-
Video and circular movement, cross correlations with sound
Report 7/26
First of all, I've never had so much stomach-problems as last week. After setting up the ' Get Going' on the first day and pedaling a few rounds I thought, how will I ever be able to last in the upcoming week of experiments. But...after some exercise you get at least that much used to the spinning that your stomach-contents will stay in. And if the experiments' aim 'nonmotion within motion' would work out, the nausea would decrease along the week. Note that there's some guy running around that every once and a while jumps up the bike to do a few rounds for fun, even without any further equipment!
So... there's the testperson on the Get Going wearing video-goggles displaying several moving video-images. All the used video-material suggests oneway or the other motion. The testperson should try to hook up with the video to get a feeling of non-motion within motion.
In the following lines I'll just jump from one experiment to the other, considering you've read the exact setup in the other piece by Tim.
1st exp:
Setup: Rotating camera, speed-control in hands of the experimenter.
Target: Will the test-person be able to get the bike in the same speed as the cam and will he have an experience of telepresense.
Result: It's quite easy to get the bike-speed equal with the speed of the cam, but the setup isn't very inviting to do so.
2nd exp.
Setup: Same as exp.1 with reference object (pine-apple) mounted before the cam.
Target: Will the test-person be able to look at the pine-apple as an extension of the self and will that increase the scores
Result: Nobody claimed the pineapple to become an extension of the self although it helped the controlability.
3rd exp.
Setup: Same as exp.1with reference object (pine-apple) turning in opposite direction before the cam.
Target: Let's do some extremes
Result: Absolutely uncontrollable situation.
4th exp.
Setup: Same as exp.1 but with an animation based on scrolling stripes.
Target: Working of an abstract image; to investigate pure speed without place.
Result: It's possible to catch up with a single stripe as long as it's on screen. But it's quite frustrating once the stripe runs off. So you would like to have a series of monitors in a line. That's probably why in this experiment the surface of the screen inside the goggles didn't function as an extension of the eye anymore. After stopping the experiment the looking at the scrolling stripes seem to help your stomach recover.
5th & 6th exp.
I recognize the experiments in Tim's report, but just don't remember much about them. So...no comment.
Experiment 7.I &7:II
Setup: similar animations based on scrolling stripes blended into each other with the fade function of a videomixer, animation 1 is scrolling in a constant speed. Animation 2 is scrolling conform the bike-speed. The constantly moving stripes are a bit lighter. The test-person should try to hook up with the constantly scrolling animation so the two will then fall together into one picture. The experiment is done with and without correlated sound.
Target: Working of an abstract image; to investigate pure speed without place, and the influence of correlated sound on the test.
Result: Most of testpersons find it quit easy to hook up without the sound on. The working of the sounds is interpreted in various ways. One testperson thinks the sound is changing if he performs better (not true). An other asks if the sounds are made to be distractive. And somebody even saw the stripes becoming 3-dimensional.
Experiment 7.III & 7.IV
Setup: Same as 7.I & 7.II in this case the animations are from the rotating camera pictures.
Target: Will the test work if the element of place (position) is introduced, and the influence of correlated sound on the test.
Results: All the testpersons claim to have big trouble to see which animation they control. But... very nice images on the check monitor.
Experiment 8
Setup: Same as 7.III & 7.IV in this case one of the animation is set in negative to improve self-recognition.
Target: Same
Results: The testperson that's, by the way, quit common with the Get Going manages to get the two layers almost sync which gave again very beautiful images. A very exiting moment! The added sound seemed to help the testperson to concentrate.
All together, a very satisfying working-period in which I did play-wise, looking back by now, a lot of experiments of which at least some of the experiences will have efforts in upcoming projects. Marnix de Nijs
Report 7/26
Field of Investigation:
Video and circular movement, cross correlations with sound. The effects of rotational movement on visual perception via HMD.
Equipment:
Get Going cycle device, a rotating platform.
I-glasses HMD with radio link.
Rotating speed-controlled camera.
Video replay control.
Surround-sound directional loudspeaker array.
Multi-channel midi-controlled mixer.
Preliminary experiments and background information:
In all cases the experimentee is on the Get Going, and is wearing the HMD. They have control over their own rotation via pedaling. There is, in the case of pedaling the Get Going with no extra interfaces, a certain degree of nausea experienced by most people. There are two observed aspects of this nausea, a stomach-based feeling and a dizzy, head-based feeling. The nausea also falls into two phases, one during pedaling, when the experimentee is in motion, one following the cessation of motion. Often a long-term lingering effect in the stomach is observed. It has been previously observed that interfaces can alleviate these feelings, or can move this feeling around. For instance a small screen on the handlebars captures the attention and focus of the experimentee, allowing them to achieve greater rotational velocity, although an extreme feeling of nausea arises when the motion stops. In some sense the nausea is shifted from the peddling phase into the later phase.
---
Experiment 1:
The experimentee received an image from the rotating speed-controlled camera. The speed of the camera's rotation was controlled by the experimenter. This was referred to as an 'invitation' to match speed with the camera. The experimentee found it, in general, very difficult to match speeds in a way that felt as though they were having a telepresence experience.
Experiment 2:
Similar to experiment 1, except that a fixed object, an Ananas (pineapple) was positioned upon the rotating platform with the camera, giving a fixed reference point for the experimentee. This inflated the 'invitation' feeling, the perceived degree of involvement of the experimentee in the situation was increased and their control over their movement was increased.
Experiment 3:
The Ananas was hung in the visual field and was spun by the experimenter as the camera rotated. The most notable effect was a massive increase in the feelings of nausea, the experience became "physically upsetting."
The first phase of experiments came to an end with a feeling that the feedback missing in these experiments, or the fact that the feedback was very abstract, transmission of information via movement in the visual field, was a major problem. The dislocation, probably related to the phenomenon known as "VR-sickness", between the perceived visual movement and the physical movement that the experimentee is involved in, was suggested as a basis for the problem with these experiments.
So we proceeded to attempt to close the loop.
Another aspect was the use of visually very rich camera views. This relates to the feelings of nausea reported in the Baginsky experiments, where one experimentee reported a feeling of extreme nervousness and resulting nausea from the closeness and busyness of the objects before the camera. Thus a series of experiments were made using simpler, more abstract pictures.
Experiment 4:
The use of a single-line image, an animation of a vertical line moving across the visual field, at a fixed speed. In essence the experiment was the same as those previously, but with a more abstract image. The same experiment was also performed with a multiple-line image, 6 evenly-spaced vertical lines moving synchronously across the visual field. The multi-line image was found to be more interesting.
Experiment 5:
Using the multi-line image, which became the standard for later experiments, the speed of movement of the image across the visual field was correlated with the speed of rotation of the experimentee. This was more of an experiment in the technologies behind this experiment, though it was found to be interesting whether the images moved in the direction of rotation, or against it. If the image moved against the direction of motion, there was a feeling that they were part of a background that was fixed that one was moving against, although they were not actually fixed relative to the physical background. The interpretation of the image by the visual processing parts of the brain were easily tricked into making the most simple interpretation of events.
Experiment 6:
The same abstract image was used, but the rotation was made to start off at a fixed speed which then decreased towards zero as the experimentee accelerated, the image finally becoming still (relative to the experimentee's field of vision), or even travelling in the reversed direction if the experimentee pedaled fast enough. Once again the velocities were not directly bound to the speed of rotation, but were only correlated; the experimentee made this "catching up" perception naturally.
It would seem, based upon the past two experiments, that the experimentee needs only a limited degree of correlation to lose a portion of the motion or VR-sickness that was previously apparent. After this experiment, a statistical and mathematical analysis of the correlation between the signals we had been analysing and their "true" or "objective" values found that the correlations were very nonlinear and bizarre. However the experimentees were able to work through them.
Experiments 7.x:
These experiments were variations upon a theme. An image sequence was taken and made into a loop. One of the loops was played continuously with constant speed, while another copy of the loop was played at a speed based upon the movement of the experimentee. The movement of the images was in the same direction as the movement of the experimentee (but see notes later about experimentee "DM"). The two images were then faded together, one being slightly darker than the other. In one version of the experiments, synchronous to the movement of the image controlled by the experimentee, a sound source was moved around the experimentee. There were also two loops, one taken from the abstract multiple lines image, one from the rotating video camera. Thus there were 4 classes of experiments, video and abstract image on one axis, silence and with correlated sound on the other axis. Thus the experiments were:
I: abstract image with sound
II: abstract image without sound
III: video image without sound
IV: video image with sound
Some notes were of interest here. The experiments are listed in order of their occurrence.
Experimentee DM:
I:Reported that the sound seemed to change when the two images met one another. This is not the case, this phenomenon is purely subjective.
II: The differing grey levels of the two loops induced a 3D effect.
III: Had problems with the relative levels of the video imagery. Could not find a way to get the images to superimpose. "Komme ich nicht rein."
IV: Technical hitch stopped the experiment.
In general found the abstract images better ("laessig!"). Interestingly, DM pedalled in the opposite direction to what is normal. As the Get Going had been modified specifically to allow this, actually to increase the degree of control by allowing pedal breaking by locking the freewheel unit, it might be interesting to repeat the experiment with him when he was forced to pedal in the "standard" direction. He declared that this was the correct direction, however.
Experimentee G:
II: No particular notes. Some time was needed to come to terms with the hardware, as she has never used such devices before. All other experimentees had had some experience with the Get Going.
I: Claimed that the sound didn't make much difference. Asked whether it was meant to help or distract. Noticed that it was becoming easier to make the lines superimpose.
II: Now with the comparison, perceived that the sound movement helped with the matching of speeds, but not sure as to why.
I: No particular notes.
Discussion about the possible sources of improvement in matching capacity with sound ensued. Most likely hypothesis was that the sound helped with speed maintenance; if the experimentee's speed remained constant, then the subjective positioning of the sound source also remained constant. Since the positioning of the sound source has a much finer granularity, 384 positions in a complete round as opposed to 63 frames for the image loop (58 in the video loop), it is proposed that this positioning helps maintain a smoother movement.
After a pause, the following experiments took place:
III: Questions of relative speed arose.
IV: Telling the two pictures apart was too hard. Attempts were made with the contrast in the pictures, or to raise the contrast between them. G stopped suddenly, "emergency brake," due to extreme nausea.
Experimentee KHM:
II: No notes.
I: Simply cannot get the images together. Sound or lack thereof didn't seem to make a difference.
KHM was suffering a damaged foot, possibly this limits the amount of fine control possible.
Experiment 8:
The problem with the differentiation of the video images in the previous experiments was countered by negating one of the images.
Experimentee M9:
III: Recognition of "self" made easier with negation.
IV: When the images superimpose, a grey mess results, very reminiscent of cheap 70s video effects. Sound helps with the concentration possibilities. Distraction becomes less.
---
Analysis and Ideas:
The mass of experiments preformed this week working with rotating video is a part of a series of researches that have been sitting in the to-do book for quite some time. Marnix De Nij's previous work with rotating video was the interface point for this lab phase.
It seems that imagery suffers in movement by its more discrete nature, or the fact that the discretisation is at a level that is relevant for our perceptions. When we begin to incorporate movement into video imagery, we begin to notice the jumps and discontinuities that exist. Probably one of the most important observations here is the importance of feedback in increasing the intuition of movement in virtual or telepresence spaces. The sense of a "self" in such spaces was also apparent. Such place-signifiers as ridiculous as an Ananas became important and valuable. Even the observation by experimentee M9 that the negation of the image allowed a perception of a self indicates that this perception is necessary. The possible abstractness of usable feedback was, however, interesting. There need not be complex force-feedback relationships, simple self-objects and a closed loop of action and perceptual changes appeared sufficient. The analysis made after experiment 6 shows that relationships in these loops can be extremely non-linear and still be understandable and usable by the experimentee.
First of all, I've never had so much stomach-problems as last week. After setting up the ' Get Going' on the first day and pedaling a few rounds I thought, how will I ever be able to last in the upcoming week of experiments. But...after some exercise you get at least that much used to the spinning that your stomach-contents will stay in. And if the experiments' aim 'nonmotion within motion' would work out, the nausea would decrease along the week. Note that there's some guy running around that every once and a while jumps up the bike to do a few rounds for fun, even without any further equipment!
So... there's the testperson on the Get Going wearing video-goggles displaying several moving video-images. All the used video-material suggests oneway or the other motion. The testperson should try to hook up with the video to get a feeling of non-motion within motion.
In the following lines I'll just jump from one experiment to the other, considering you've read the exact setup in the other piece by Tim.
1st exp:
Setup: Rotating camera, speed-control in hands of the experimenter.
Target: Will the test-person be able to get the bike in the same speed as the cam and will he have an experience of telepresense.
Result: It's quite easy to get the bike-speed equal with the speed of the cam, but the setup isn't very inviting to do so.
2nd exp.
Setup: Same as exp.1 with reference object (pine-apple) mounted before the cam.
Target: Will the test-person be able to look at the pine-apple as an extension of the self and will that increase the scores
Result: Nobody claimed the pineapple to become an extension of the self although it helped the controlability.
3rd exp.
Setup: Same as exp.1with reference object (pine-apple) turning in opposite direction before the cam.
Target: Let's do some extremes
Result: Absolutely uncontrollable situation.
4th exp.
Setup: Same as exp.1 but with an animation based on scrolling stripes.
Target: Working of an abstract image; to investigate pure speed without place.
Result: It's possible to catch up with a single stripe as long as it's on screen. But it's quite frustrating once the stripe runs off. So you would like to have a series of monitors in a line. That's probably why in this experiment the surface of the screen inside the goggles didn't function as an extension of the eye anymore. After stopping the experiment the looking at the scrolling stripes seem to help your stomach recover.
5th & 6th exp.
I recognize the experiments in Tim's report, but just don't remember much about them. So...no comment.
Experiment 7.I &7:II
Setup: similar animations based on scrolling stripes blended into each other with the fade function of a videomixer, animation 1 is scrolling in a constant speed. Animation 2 is scrolling conform the bike-speed. The constantly moving stripes are a bit lighter. The test-person should try to hook up with the constantly scrolling animation so the two will then fall together into one picture. The experiment is done with and without correlated sound.
Target: Working of an abstract image; to investigate pure speed without place, and the influence of correlated sound on the test.
Result: Most of testpersons find it quit easy to hook up without the sound on. The working of the sounds is interpreted in various ways. One testperson thinks the sound is changing if he performs better (not true). An other asks if the sounds are made to be distractive. And somebody even saw the stripes becoming 3-dimensional.
Experiment 7.III & 7.IV
Setup: Same as 7.I & 7.II in this case the animations are from the rotating camera pictures.
Target: Will the test work if the element of place (position) is introduced, and the influence of correlated sound on the test.
Results: All the testpersons claim to have big trouble to see which animation they control. But... very nice images on the check monitor.
Experiment 8
Setup: Same as 7.III & 7.IV in this case one of the animation is set in negative to improve self-recognition.
Target: Same
Results: The testperson that's, by the way, quit common with the Get Going manages to get the two layers almost sync which gave again very beautiful images. A very exiting moment! The added sound seemed to help the testperson to concentrate.
All together, a very satisfying working-period in which I did play-wise, looking back by now, a lot of experiments of which at least some of the experiences will have efforts in upcoming projects. Marnix de Nijs
Report 7/26
Field of Investigation:
Video and circular movement, cross correlations with sound. The effects of rotational movement on visual perception via HMD.
Equipment:
Get Going cycle device, a rotating platform.
I-glasses HMD with radio link.
Rotating speed-controlled camera.
Video replay control.
Surround-sound directional loudspeaker array.
Multi-channel midi-controlled mixer.
Preliminary experiments and background information:
In all cases the experimentee is on the Get Going, and is wearing the HMD. They have control over their own rotation via pedaling. There is, in the case of pedaling the Get Going with no extra interfaces, a certain degree of nausea experienced by most people. There are two observed aspects of this nausea, a stomach-based feeling and a dizzy, head-based feeling. The nausea also falls into two phases, one during pedaling, when the experimentee is in motion, one following the cessation of motion. Often a long-term lingering effect in the stomach is observed. It has been previously observed that interfaces can alleviate these feelings, or can move this feeling around. For instance a small screen on the handlebars captures the attention and focus of the experimentee, allowing them to achieve greater rotational velocity, although an extreme feeling of nausea arises when the motion stops. In some sense the nausea is shifted from the peddling phase into the later phase.
---
Experiment 1:
The experimentee received an image from the rotating speed-controlled camera. The speed of the camera's rotation was controlled by the experimenter. This was referred to as an 'invitation' to match speed with the camera. The experimentee found it, in general, very difficult to match speeds in a way that felt as though they were having a telepresence experience.
Experiment 2:
Similar to experiment 1, except that a fixed object, an Ananas (pineapple) was positioned upon the rotating platform with the camera, giving a fixed reference point for the experimentee. This inflated the 'invitation' feeling, the perceived degree of involvement of the experimentee in the situation was increased and their control over their movement was increased.
Experiment 3:
The Ananas was hung in the visual field and was spun by the experimenter as the camera rotated. The most notable effect was a massive increase in the feelings of nausea, the experience became "physically upsetting."
The first phase of experiments came to an end with a feeling that the feedback missing in these experiments, or the fact that the feedback was very abstract, transmission of information via movement in the visual field, was a major problem. The dislocation, probably related to the phenomenon known as "VR-sickness", between the perceived visual movement and the physical movement that the experimentee is involved in, was suggested as a basis for the problem with these experiments.
So we proceeded to attempt to close the loop.
Another aspect was the use of visually very rich camera views. This relates to the feelings of nausea reported in the Baginsky experiments, where one experimentee reported a feeling of extreme nervousness and resulting nausea from the closeness and busyness of the objects before the camera. Thus a series of experiments were made using simpler, more abstract pictures.
Experiment 4:
The use of a single-line image, an animation of a vertical line moving across the visual field, at a fixed speed. In essence the experiment was the same as those previously, but with a more abstract image. The same experiment was also performed with a multiple-line image, 6 evenly-spaced vertical lines moving synchronously across the visual field. The multi-line image was found to be more interesting.
Experiment 5:
Using the multi-line image, which became the standard for later experiments, the speed of movement of the image across the visual field was correlated with the speed of rotation of the experimentee. This was more of an experiment in the technologies behind this experiment, though it was found to be interesting whether the images moved in the direction of rotation, or against it. If the image moved against the direction of motion, there was a feeling that they were part of a background that was fixed that one was moving against, although they were not actually fixed relative to the physical background. The interpretation of the image by the visual processing parts of the brain were easily tricked into making the most simple interpretation of events.
Experiment 6:
The same abstract image was used, but the rotation was made to start off at a fixed speed which then decreased towards zero as the experimentee accelerated, the image finally becoming still (relative to the experimentee's field of vision), or even travelling in the reversed direction if the experimentee pedaled fast enough. Once again the velocities were not directly bound to the speed of rotation, but were only correlated; the experimentee made this "catching up" perception naturally.
It would seem, based upon the past two experiments, that the experimentee needs only a limited degree of correlation to lose a portion of the motion or VR-sickness that was previously apparent. After this experiment, a statistical and mathematical analysis of the correlation between the signals we had been analysing and their "true" or "objective" values found that the correlations were very nonlinear and bizarre. However the experimentees were able to work through them.
Experiments 7.x:
These experiments were variations upon a theme. An image sequence was taken and made into a loop. One of the loops was played continuously with constant speed, while another copy of the loop was played at a speed based upon the movement of the experimentee. The movement of the images was in the same direction as the movement of the experimentee (but see notes later about experimentee "DM"). The two images were then faded together, one being slightly darker than the other. In one version of the experiments, synchronous to the movement of the image controlled by the experimentee, a sound source was moved around the experimentee. There were also two loops, one taken from the abstract multiple lines image, one from the rotating video camera. Thus there were 4 classes of experiments, video and abstract image on one axis, silence and with correlated sound on the other axis. Thus the experiments were:
I: abstract image with sound
II: abstract image without sound
III: video image without sound
IV: video image with sound
Some notes were of interest here. The experiments are listed in order of their occurrence.
Experimentee DM:
I:Reported that the sound seemed to change when the two images met one another. This is not the case, this phenomenon is purely subjective.
II: The differing grey levels of the two loops induced a 3D effect.
III: Had problems with the relative levels of the video imagery. Could not find a way to get the images to superimpose. "Komme ich nicht rein."
IV: Technical hitch stopped the experiment.
In general found the abstract images better ("laessig!"). Interestingly, DM pedalled in the opposite direction to what is normal. As the Get Going had been modified specifically to allow this, actually to increase the degree of control by allowing pedal breaking by locking the freewheel unit, it might be interesting to repeat the experiment with him when he was forced to pedal in the "standard" direction. He declared that this was the correct direction, however.
Experimentee G:
II: No particular notes. Some time was needed to come to terms with the hardware, as she has never used such devices before. All other experimentees had had some experience with the Get Going.
I: Claimed that the sound didn't make much difference. Asked whether it was meant to help or distract. Noticed that it was becoming easier to make the lines superimpose.
II: Now with the comparison, perceived that the sound movement helped with the matching of speeds, but not sure as to why.
I: No particular notes.
Discussion about the possible sources of improvement in matching capacity with sound ensued. Most likely hypothesis was that the sound helped with speed maintenance; if the experimentee's speed remained constant, then the subjective positioning of the sound source also remained constant. Since the positioning of the sound source has a much finer granularity, 384 positions in a complete round as opposed to 63 frames for the image loop (58 in the video loop), it is proposed that this positioning helps maintain a smoother movement.
After a pause, the following experiments took place:
III: Questions of relative speed arose.
IV: Telling the two pictures apart was too hard. Attempts were made with the contrast in the pictures, or to raise the contrast between them. G stopped suddenly, "emergency brake," due to extreme nausea.
Experimentee KHM:
II: No notes.
I: Simply cannot get the images together. Sound or lack thereof didn't seem to make a difference.
KHM was suffering a damaged foot, possibly this limits the amount of fine control possible.
Experiment 8:
The problem with the differentiation of the video images in the previous experiments was countered by negating one of the images.
Experimentee M9:
III: Recognition of "self" made easier with negation.
IV: When the images superimpose, a grey mess results, very reminiscent of cheap 70s video effects. Sound helps with the concentration possibilities. Distraction becomes less.
---
Analysis and Ideas:
The mass of experiments preformed this week working with rotating video is a part of a series of researches that have been sitting in the to-do book for quite some time. Marnix De Nij's previous work with rotating video was the interface point for this lab phase.
It seems that imagery suffers in movement by its more discrete nature, or the fact that the discretisation is at a level that is relevant for our perceptions. When we begin to incorporate movement into video imagery, we begin to notice the jumps and discontinuities that exist. Probably one of the most important observations here is the importance of feedback in increasing the intuition of movement in virtual or telepresence spaces. The sense of a "self" in such spaces was also apparent. Such place-signifiers as ridiculous as an Ananas became important and valuable. Even the observation by experimentee M9 that the negation of the image allowed a perception of a self indicates that this perception is necessary. The possible abstractness of usable feedback was, however, interesting. There need not be complex force-feedback relationships, simple self-objects and a closed loop of action and perceptual changes appeared sufficient. The analysis made after experiment 6 shows that relationships in these loops can be extremely non-linear and still be understandable and usable by the experimentee.
Links:
References:
Related publications: