Move

Iteration #3

lab partner: Salma Kashani

Introduction:

Previously on iteration II, we compared the perception of the force impact using solenoid/s and vibration using tactors to convey move messages. Our experiment shows that having the haptic feedback at the back part of the leg can convey class of propelling movement signals such as lift the leg or step forward,  while on the opposite direction was more stopping and alerting (step back, stop, obstacle). During our experiments we reached to some initial findings that worths studying more. We saw that users can better perceive one site vibration compared to one force impact.However having two force impacts works better than two sites vibration.In additon having redundant stmiuli seems to be more exoressive and better to convey our messages.

Objectives:

Before exploring more about the effect of adding more vibrotactile stimulus or other methods of applying the haptic feedback, we decided to first study the psychophysical properties of these signals which would help us to better design our system to convey our messages. We are aiming to study both temporal and spatial properties of the signals to have more conclusive results. This will help us to understand the difference between having two sites of stimulus compared to only one, and the intensive or extensive nature of vibrotactile feedback.

Research question:

In our study in iteration 3 we are trying to answer the following questions:

1. Is there a difference between the perceptual experience of a stimulus that it is applied to the leg through using one tactor versus two?
2. What psychophysical characteristics of the stimulus (e.g., intensive/extensive, temporal or spatial) influences one's haptic perception of it?
3. How the properties of the stimulus can be customized in order to optimize the variance of the conveyed message?

Expected outcomes:

Having two sites of stimulus can convey the message more direct and transparent compared to only one site.The vibrotactile stimulus has intensive properties which means that the perception and interpretation of the signal by the user is independent of the magnitude of the stimuli; however, the timing of stimulation (temporal) and spatial properties  of the signal not only affect the feeling, but can change the meaning of the signal. Having asynchronous stimulus and placing them far apart will help to have better feeling and variance of the signal.The special sequence of two stimulus with different spatial position can convey motion vector messages(spin, forward, backward).

Approach:

To answer our research questions we design an experiment with variety stimuli parameters and different conditions. Each participant will participate in a 30 minutes long experiment in which he/she will put on a Velcro band with two mounted tactors. We ask participant to move the Velcro so that we can apply the vibration stimuli on the back part of their upper legs.

Fig1,Two tactors were mounted on a piece of Velcro by 12 cm distance. 

Experimental design:

We design our experiments in 3 different phases.

• Phase I: We get the first impression and reactions of our participants.

We exposed our participants to vibrotactile stimuli and ask them about their immediate perception. This study will help us to measure the cognitive load of our signals. 

• Phase II: Thoughtful consideration with suggested move messages.

We are changing the physical properties of the signals to study the temporal and spatial effects and change of amplitude.

• Phase III:Customization of the stimuli based on their experiences. 

We ask our participants to help us designing more transparent signals by changing the signal's parameters for given messages.

Design Space:

To measure the physical properties of our signals we needed to make a setup to fulfill following requirements :

-- One Site stimuli (one tactor):

• Change of amplitude
• Repeated tap with time delay.

-- Two sites stimulus (two tactors):

• Change of amplitude
• Repeated tap with time delay.
• Sequential tap with different order.

Based on our study requirements we came up with a design to individually control each tactor. Our setup consists of a software part which was coded in Processing and a hardware part which was implemented in Arduino.

• Software:

 Our software part includes a GUI which is depicted in Fig2. By pressing each button on the GUI our predefined vibration signal will be played on a PC. We get the output signal from the audio jack and connect it to our hardware design. Meanwhile the controlling signal is transferred through the serial port to Arduino.  This controlling signal will manage which of the tactors should be activated and which one should remain passive. Our timing for delays and the sequential order of activation were implemented in processing. The amplitude of the signal is also controlled by means of the PC volume level.

Fig 2,System GUI

Below is one example that shows how the Two_Tactor_delay works.

public void Two_Tactors(int theValue) {

  println("a button event from TwoTactors: "+theValue);

   myport.write('e'); //send impulse to Arduino to activate one tactor

      delay(100); //communication delay

      Boom.play(0); // Audio signal plays

   delay(500);        // Duration of the signal

   delay(Delay_Time);// Delay between two signal

   myport.write('e'); //send impulse to Arduino to activate the other tactor

    Boom.play(0); // Audio signal plays

   delay(500);        // Duration of the signal

  myport.write('a');// Disable both tactors and get back to initial condition  

}

The delay time between activation can be controlled by means of a sliding bar. This configuration will help us to run our experiments faster, and  also users can better play with different conditions to customise their Vibrotactile stimuli.

• Hardware:

On our hardware side, first we get the vibrotactile signal from the audio jack and amplify it using a 3W amplifier. The outputs of the amplifier were connected to two mechanical relays, and then were wired to our tactors. The reason that we used mechanical relays instead of any electronic switches was that, relays have gain of 1 over the audio range of frequency, but this gain may change for our electrical switches. In addition to have the electrical switches we need to bias them first which add a DC part to our signal, which then needed to be filtered, while mechanical switches simply don’t have any of these problems. We couldn’t directly connect our relays to Arduino since they took much more than maximum output current of Arduino,so  when both were activated one of them was not working perfectly. To solve this problem we used transistors before relays to generate more current. Now when the Arduino receive the serial signal from the processing it activate one/ two of the relay/s and then the amplified signals will be played on tactors. Fig 3 shows the relay control using Arduino output.

Fig3.  a)Shows our electrical circuit on breadboard.b)  The circuit schematic to drive each relay (Swagatam Innovation)

• Participants:  We asked 5 people among our friends and classmates to attend to our study with no payment. 

Study:

We asked our participants to imagine the scenario of walking in a short range from our system while the tactors were mounted on the back part of their upper leg. In case of using one tactor,  we wanted them to make sure that the active tactor was placed at the center of the leg and in the case when both of them were active, move them on the sides. We also ensured  that our participants can feel the signal. We randomized the order of the experiments to reduce the effect of learning on our results.

In phase I, we played the conditions three times to our participants, and then asked them the following questions regarding their experiences.

1.  Did you feel the stimulus?
2.  How easy was is to feel the stimulus? 1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard
3. Did the stimulus convey a message to you?
4. If yes, what was the message?
5. How easy was it to perceive that the stimulus intended to convey a message? 1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard
6. How easy was it to interpret and understand the message?  1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard

In phase II, We defined group of messages and ask participants to choose among their options while similar conditions were applied. We informed them that they have no limitation on the number of playing signal at each condition and encouraged  them to think about the conveyed message from the given list:

• Walk forward
• Walk backward
• Lift your leg 
• Bend your knee back
• Stop
• Other...

Similar questions of phase I, were asked while they have limited options for the conveyed message.

In Phase III: After our participants were familiarized with our system we asked them to help us customizing the vibrotactile feedback in order to have more transparent signal for the following messages:

• Walk forward
• lift your leg

our system enable them to manipulate the number of sites, amplitude, delay time and order of sequence(left to right/right to left).

Discussion:

• Phase I: Immediate perception 

---One Tactor, Conditions:(Low amplitude, High Amplitude, Delay ), 5 participants

     Low Amplitude:Our participants could feel the signal of one tap with average difficulty of (2.6/5).Three of them could perceive a general message which was not from the movement vocabulary or was hard to explain.

     High Amplitude: Increasing the volume of the signal to maximum help participant feel the signal more easily with difficulty average of (1.6/5).This time only two of them could feel the message, one of them could feel the move message of lift up your leg, while the other person just perceive the alert signal.

     Taps With Delay: Except one, most of the participant could easily feel the signal  with average of (1.2/5). 4 out of 5 participant perceive the message which was rather too complex or not from the move vocabulary.

---Two Tactors, Conditions:(Low amplitude Delay ), 5 participants

    Low Amplitude: The difficulty average to feel the signal was (2/5). Two participants could perceive the signal to be alerting or urging and one found it similar to one tap case with no message.   
    Taps With Delay:Participants could feel the signal with the average of difficulty of (2.6/5). Three of them could perceive a signal with different messages including:"Pay attention", rhythm of steps" and you have a message. 
  

The initial perception of the signals showed the required cognitive process required  to convey a signal. This phase showed us that the transparency of our signals is still not high enough to directly convey a move message. For most of the cases our participants identified the signal to be more alerting with no consistent messages. They could realize that there is a convey message with average of (2.6/5); however, it was difficult for them to interpret the message (Average of 3.6/5).

• Phase II: Thoughtful perception

---One Tactor, Conditions:(Low amplitude, High Amplitude, Delay ), 5 participants

     Low Amplitude: After we familiarized our participants with the signals, they could feel the signal of one tap with average difficulty of (2/5).Three of them could perceive walk forward message and one selected both lift your leg and walk forward, the last one didn't identify any messages. Among those who received the message, the average of understanding that there was a signal was a message was (1.5/5) with interpretation difficulty average of (2.5/5).

     High Amplitude: Increasing the volume of  improved the difficulty average of feeling to (1.4/5).Here all of the participant could identify a message, but with lower variance in which they add other options to move forward.Interpretation difficulty average of (2.6/5).

     Taps With Delay: Similar to Phase I, most of the participant could easily feel the signal  with average of (1.2/5). They also could easily recognize that there is a message in the signal (1.4/5) with high variance and transparency(1.8/5)

---Two Tactors, Conditions:(High amplitude Delay ), 5 participants

    High Amplitude: The difficulty average to feel the signal was (1.6/5). There was more variation on the conveyed messages.Additionally they found it not easy to perceive(2.4/5) and interpret the message. interpretation difficulty(3.2).    
    Taps With Delay:Participants could detect the signal with the average of difficulty of (1.5/5). The message was more consistent and was completely different from previous experiences. 3 out of 5 selected "lift your leg" and interpretation difficulty average of (1.5/5).

Similar to Phase I, the feeling rate of one site of stimuli was better than two sites. Participants could more easily interpret the conveyed signal in one site stimuli.Our predefined list of movement narrowed down their options and help them to understand the signal more easily. "Walking forward" was selected many times as a conveyed messages with high variance. Except the two sites stimulus with delay where participants were thinking about higher information density signal and picked "lift your leg" with uncertainty. 

• Phase II: Customization of signal

The data that we get from customization was not consistent between participants.When we asked them to design a signal for "Walk forward" Two of them selected two sites synchronous signal which they believed can work like a gentle push. The rest of the participants came up with one tap and after changing the amplitude and delay time, two of them prefered one tap without delay and the last one picked up one tap with delay.

For the message of "lift your leg up"4 out of 5 participants selected two sites stimulus with short delay. If the delay was too long they would feel it as two different messages such as " stop and do sth else". We found the following comments interesting, when they were designing the signal.

"-Two tactor taps without a delay was perceived as one.
- Shorter delay between the taps indicates urgency.
- Higher amplitude means urgency.
- The message interpretation depends on the walking's state."

Sources of Error:

• Users cloths; Jeans and other physical layers between the tactors and skin work as a damper and reduce the amplitude of the signal.
• Velcro tightness:The amount of pre-stress on tactors, significantly can change the amount of vibration. We provide our participants with some advices and make sure that they could feel the vibration.

Conclusion:

This iteration shows us, how the affordances of one or two stimulus can be exploited to convey move messages more efficiently. We could find answers for our research objectives and study the physical behaviour of our stimulus . Our empirical findings shows that the amplitude of signals plays a crucial role in conveying a message. We find out that,the nature of vibration signals is excessive and the reason that two participants prefer the 2 tactors vs one was that, they could feel more push by having two.Accordingly in our experiments they prefer one strong actuation compared to two weaker stimulus, but increasing the amplitude of the signal can increase the interpretation costs.
We conclude that for low information density, we should use one tactor and as we want to support more complex messages we need to increase the number of stimulus. Studying the temporal behaviour of vibrotactile feedback shows that, adding a delay to our signal improves the feeling rate and increases the complexity of the signal (harder to interpret).Participants can distinguish the spatial properties of the signal  and draw an imaginary vector based on the sequence of activation of the stimulus; however if they are not provided with relative messages, they would ignore the differences.  

Next Step: Portable vibrotactile feedback 

Some of the users were complaining about the wires and preferred to freely walk with the tactors and sense the signals more realistically . To do that we need to make a wireless communication. We have two options:1. WiFi  2. Bluetooth  

First we thought of using Wi-Fi and play the signals using our phones. We also needed to have a microcontroller to make the communication and switch the tactors if needed. Since Wi-Fi was consuming too much energy we decided to move on to Bluetooth.

Our second option was using a Bluetooth module and then connecting it to a small Arduino, however we might need to use a SD Card module to play the signal from that due to small flash of Arduino board Nano.

Our final decision was using a new series of Bluetooth module with built in micro controller. Among three options we opt out RFduino with Bluetooth low energy communication and an embedded micro-controller with 8 Kbyte flash memory. This module can provide up to 5 mAmp output which was compatible to our amplifiers.  Fig4 shows the RFduino and its programer beside two audio amplifiers and one tactor. Now we need to use the pulse width modulation (PWM) output of our controller to make our digital audio player.  We mapped our wave signal to range of 0 to 256 using  Wav2C convertor to make an array of our sound data. We converted our Audio signal to a file with 8-bit, 8000 Hz which can be uploaded to our microcontroller flash.

So whenever we call the signal on our Bluetooth module it plays the array of sound data on one of the PWM output pins. We make the further amplification of the signals by means of our amplifiers and then connect them to the tactor. We use a 1000 mAmp rechargeable battery to power our system. We also need to add a power regulator since RFduino is working with 3.3V power supply.

Fig 4, Portable prototype

Contribution:

After brainstorming about our experiment's structure and direction, Salma focused on designing the study part, and I made the software and hardware. Meanwhile we were discussing our designs and giving feedbacks to each other . After she conducted the experiments, we analyzed our results, and now we are preparing the next step for the demo.