Move

Iteration #3

lab partner: Salma Kashani

Introduction:

Previously on iteration II, we compared the perception of the force impact using solenoid/s and vibration using tactors to convey move messages. Our experiment shows that having the haptic feedback at the back part of the leg can convey class of propelling movement signals such as lift the leg or step forward,  while on the opposite direction was more stopping and alerting (step back, stop, obstacle). During our experiments we reached to some initial findings that worths studying more. We saw that users can better perceive one site vibration compared to one force impact.However having two force impacts works better than two sites vibration.In additon having redundant stmiuli seems to be more exoressive and better to convey our messages.

Objectives:

Before exploring more about the effect of adding more vibrotactile stimulus or other methods of applying the haptic feedback, we decided to first study the psychophysical properties of these signals which would help us to better design our system to convey our messages. We are aiming to study both temporal and spatial properties of the signals to have more conclusive results. This will help us to understand the difference between having two sites of stimulus compared to only one, and the intensive or extensive nature of vibrotactile feedback.

Research question:

In our study in iteration 3 we are trying to answer the following questions:

1. Is there a difference between the perceptual experience of a stimulus that it is applied to the leg through using one tactor versus two?
2. What psychophysical characteristics of the stimulus (e.g., intensive/extensive, temporal or spatial) influences one's haptic perception of it?
3. How the properties of the stimulus can be customized in order to optimize the variance of the conveyed message?

Expected outcomes:

Having two sites of stimulus can convey the message more direct and transparent compared to only one site.The vibrotactile stimulus has intensive properties which means that the perception and interpretation of the signal by the user is independent of the magnitude of the stimuli; however, the timing of stimulation (temporal) and spatial properties  of the signal not only affect the feeling, but can change the meaning of the signal. Having asynchronous stimulus and placing them far apart will help to have better feeling and variance of the signal.The special sequence of two stimulus with different spatial position can convey motion vector messages(spin, forward, backward).

Approach:

To answer our research questions we design an experiment with variety stimuli parameters and different conditions. Each participant will participate in a 30 minutes long experiment in which he/she will put on a Velcro band with two mounted tactors. We ask participant to move the Velcro so that we can apply the vibration stimuli on the back part of their upper legs.

Fig1,Two tactors were mounted on a piece of Velcro by 12 cm distance. 

Experimental design:

We design our experiments in 3 different phases.

• Phase I: We get the first impression and reactions of our participants.

We exposed our participants to vibrotactile stimuli and ask them about their immediate perception. This study will help us to measure the cognitive load of our signals. 

• Phase II: Thoughtful consideration with suggested move messages.

We are changing the physical properties of the signals to study the temporal and spatial effects and change of amplitude.

• Phase III:Customization of the stimuli based on their experiences. 

We ask our participants to help us designing more transparent signals by changing the signal's parameters for given messages.

Design Space:

To measure the physical properties of our signals we needed to make a setup to fulfill following requirements :

-- One Site stimuli (one tactor):

• Change of amplitude
• Repeated tap with time delay.

-- Two sites stimulus (two tactors):

• Change of amplitude
• Repeated tap with time delay.
• Sequential tap with different order.

Based on our study requirements we came up with a design to individually control each tactor. Our setup consists of a software part which was coded in Processing and a hardware part which was implemented in Arduino.

• Software:

 Our software part includes a GUI which is depicted in Fig2. By pressing each button on the GUI our predefined vibration signal will be played on a PC. We get the output signal from the audio jack and connect it to our hardware design. Meanwhile the controlling signal is transferred through the serial port to Arduino.  This controlling signal will manage which of the tactors should be activated and which one should remain passive. Our timing for delays and the sequential order of activation were implemented in processing. The amplitude of the signal is also controlled by means of the PC volume level.

Fig 2,System GUI

Below is one example that shows how the Two_Tactor_delay works.

public void Two_Tactors(int theValue) {

  println("a button event from TwoTactors: "+theValue);

   myport.write('e'); //send impulse to Arduino to activate one tactor

      delay(100); //communication delay

      Boom.play(0); // Audio signal plays

   delay(500);        // Duration of the signal

   delay(Delay_Time);// Delay between two signal

   myport.write('e'); //send impulse to Arduino to activate the other tactor

    Boom.play(0); // Audio signal plays

   delay(500);        // Duration of the signal

  myport.write('a');// Disable both tactors and get back to initial condition  

}

The delay time between activation can be controlled by means of a sliding bar. This configuration will help us to run our experiments faster, and  also users can better play with different conditions to customise their Vibrotactile stimuli.

• Hardware:

On our hardware side, first we get the vibrotactile signal from the audio jack and amplify it using a 3W amplifier. The outputs of the amplifier were connected to two mechanical relays, and then were wired to our tactors. The reason that we used mechanical relays instead of any electronic switches was that, relays have gain of 1 over the audio range of frequency, but this gain may change for our electrical switches. In addition to have the electrical switches we need to bias them first which add a DC part to our signal, which then needed to be filtered, while mechanical switches simply don’t have any of these problems. We couldn’t directly connect our relays to Arduino since they took much more than maximum output current of Arduino,so  when both were activated one of them was not working perfectly. To solve this problem we used transistors before relays to generate more current. Now when the Arduino receive the serial signal from the processing it activate one/ two of the relay/s and then the amplified signals will be played on tactors. Fig 3 shows the relay control using Arduino output.

Fig3.  a)Shows our electrical circuit on breadboard.b)  The circuit schematic to drive each relay (Swagatam Innovation)

• Participants:  We asked 5 people among our friends and classmates to attend to our study with no payment. 

Study:

We asked our participants to imagine the scenario of walking in a short range from our system while the tactors were mounted on the back part of their upper leg. In case of using one tactor,  we wanted them to make sure that the active tactor was placed at the center of the leg and in the case when both of them were active, move them on the sides. We also ensured  that our participants can feel the signal. We randomized the order of the experiments to reduce the effect of learning on our results.

In phase I, we played the conditions three times to our participants, and then asked them the following questions regarding their experiences.

1.  Did you feel the stimulus?
2.  How easy was is to feel the stimulus? 1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard
3. Did the stimulus convey a message to you?
4. If yes, what was the message?
5. How easy was it to perceive that the stimulus intended to convey a message? 1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard
6. How easy was it to interpret and understand the message?  1-Very easy, 2-easy, 3-medium ,4-hard ,5- Very hard

In phase II, We defined group of messages and ask participants to choose among their options while similar conditions were applied. We informed them that they have no limitation on the number of playing signal at each condition and encouraged  them to think about the conveyed message from the given list:

• Walk forward
• Walk backward
• Lift your leg 
• Bend your knee back
• Stop
• Other...

Similar questions of phase I, were asked while they have limited options for the conveyed message.

In Phase III: After our participants were familiarized with our system we asked them to help us customizing the vibrotactile feedback in order to have more transparent signal for the following messages:

• Walk forward
• lift your leg

our system enable them to manipulate the number of sites, amplitude, delay time and order of sequence(left to right/right to left).

Discussion:

• Phase I: Immediate perception 

---One Tactor, Conditions:(Low amplitude, High Amplitude, Delay ), 5 participants

     Low Amplitude:Our participants could feel the signal of one tap with average difficulty of (2.6/5).Three of them could perceive a general message which was not from the movement vocabulary or was hard to explain.

     High Amplitude: Increasing the volume of the signal to maximum help participant feel the signal more easily with difficulty average of (1.6/5).This time only two of them could feel the message, one of them could feel the move message of lift up your leg, while the other person just perceive the alert signal.

     Taps With Delay: Except one, most of the participant could easily feel the signal  with average of (1.2/5). 4 out of 5 participant perceive the message which was rather too complex or not from the move vocabulary.

---Two Tactors, Conditions:(Low amplitude Delay ), 5 participants

    Low Amplitude: The difficulty average to feel the signal was (2/5). Two participants could perceive the signal to be alerting or urging and one found it similar to one tap case with no message.   
    Taps With Delay:Participants could feel the signal with the average of difficulty of (2.6/5). Three of them could perceive a signal with different messages including:"Pay attention", rhythm of steps" and you have a message. 
  

The initial perception of the signals showed the required cognitive process required  to convey a signal. This phase showed us that the transparency of our signals is still not high enough to directly convey a move message. For most of the cases our participants identified the signal to be more alerting with no consistent messages. They could realize that there is a convey message with average of (2.6/5); however, it was difficult for them to interpret the message (Average of 3.6/5).

• Phase II: Thoughtful perception

---One Tactor, Conditions:(Low amplitude, High Amplitude, Delay ), 5 participants

     Low Amplitude: After we familiarized our participants with the signals, they could feel the signal of one tap with average difficulty of (2/5).Three of them could perceive walk forward message and one selected both lift your leg and walk forward, the last one didn't identify any messages. Among those who received the message, the average of understanding that there was a signal was a message was (1.5/5) with interpretation difficulty average of (2.5/5).

     High Amplitude: Increasing the volume of  improved the difficulty average of feeling to (1.4/5).Here all of the participant could identify a message, but with lower variance in which they add other options to move forward.Interpretation difficulty average of (2.6/5).

     Taps With Delay: Similar to Phase I, most of the participant could easily feel the signal  with average of (1.2/5). They also could easily recognize that there is a message in the signal (1.4/5) with high variance and transparency(1.8/5)

---Two Tactors, Conditions:(High amplitude Delay ), 5 participants

    High Amplitude: The difficulty average to feel the signal was (1.6/5). There was more variation on the conveyed messages.Additionally they found it not easy to perceive(2.4/5) and interpret the message. interpretation difficulty(3.2).    
    Taps With Delay:Participants could detect the signal with the average of difficulty of (1.5/5). The message was more consistent and was completely different from previous experiences. 3 out of 5 selected "lift your leg" and interpretation difficulty average of (1.5/5).

Similar to Phase I, the feeling rate of one site of stimuli was better than two sites. Participants could more easily interpret the conveyed signal in one site stimuli.Our predefined list of movement narrowed down their options and help them to understand the signal more easily. "Walking forward" was selected many times as a conveyed messages with high variance. Except the two sites stimulus with delay where participants were thinking about higher information density signal and picked "lift your leg" with uncertainty. 

• Phase II: Customization of signal

The data that we get from customization was not consistent between participants.When we asked them to design a signal for "Walk forward" Two of them selected two sites synchronous signal which they believed can work like a gentle push. The rest of the participants came up with one tap and after changing the amplitude and delay time, two of them prefered one tap without delay and the last one picked up one tap with delay.

For the message of "lift your leg up"4 out of 5 participants selected two sites stimulus with short delay. If the delay was too long they would feel it as two different messages such as " stop and do sth else". We found the following comments interesting, when they were designing the signal.

"-Two tactor taps without a delay was perceived as one.
- Shorter delay between the taps indicates urgency.
- Higher amplitude means urgency.
- The message interpretation depends on the walking's state."

Sources of Error:

• Users cloths; Jeans and other physical layers between the tactors and skin work as a damper and reduce the amplitude of the signal.
• Velcro tightness:The amount of pre-stress on tactors, significantly can change the amount of vibration. We provide our participants with some advices and make sure that they could feel the vibration.

Conclusion:

This iteration shows us, how the affordances of one or two stimulus can be exploited to convey move messages more efficiently. We could find answers for our research objectives and study the physical behaviour of our stimulus . Our empirical findings shows that the amplitude of signals plays a crucial role in conveying a message. We find out that,the nature of vibration signals is excessive and the reason that two participants prefer the 2 tactors vs one was that, they could feel more push by having two.Accordingly in our experiments they prefer one strong actuation compared to two weaker stimulus, but increasing the amplitude of the signal can increase the interpretation costs.
We conclude that for low information density, we should use one tactor and as we want to support more complex messages we need to increase the number of stimulus. Studying the temporal behaviour of vibrotactile feedback shows that, adding a delay to our signal improves the feeling rate and increases the complexity of the signal (harder to interpret).Participants can distinguish the spatial properties of the signal  and draw an imaginary vector based on the sequence of activation of the stimulus; however if they are not provided with relative messages, they would ignore the differences.  

Next Step: Portable vibrotactile feedback 

Some of the users were complaining about the wires and preferred to freely walk with the tactors and sense the signals more realistically . To do that we need to make a wireless communication. We have two options:1. WiFi  2. Bluetooth  

First we thought of using Wi-Fi and play the signals using our phones. We also needed to have a microcontroller to make the communication and switch the tactors if needed. Since Wi-Fi was consuming too much energy we decided to move on to Bluetooth.

Our second option was using a Bluetooth module and then connecting it to a small Arduino, however we might need to use a SD Card module to play the signal from that due to small flash of Arduino board Nano.

Our final decision was using a new series of Bluetooth module with built in micro controller. Among three options we opt out RFduino with Bluetooth low energy communication and an embedded micro-controller with 8 Kbyte flash memory. This module can provide up to 5 mAmp output which was compatible to our amplifiers.  Fig4 shows the RFduino and its programer beside two audio amplifiers and one tactor. Now we need to use the pulse width modulation (PWM) output of our controller to make our digital audio player.  We mapped our wave signal to range of 0 to 256 using  Wav2C convertor to make an array of our sound data. We converted our Audio signal to a file with 8-bit, 8000 Hz which can be uploaded to our microcontroller flash.

So whenever we call the signal on our Bluetooth module it plays the array of sound data on one of the PWM output pins. We make the further amplification of the signals by means of our amplifiers and then connect them to the tactor. We use a 1000 mAmp rechargeable battery to power our system. We also need to add a power regulator since RFduino is working with 3.3V power supply.

Fig 4, Portable prototype

Contribution:

After brainstorming about our experiment's structure and direction, Salma focused on designing the study part, and I made the software and hardware. Meanwhile we were discussing our designs and giving feedbacks to each other . After she conducted the experiments, we analyzed our results, and now we are preparing the next step for the demo.

             

MOVE

Iteration #2

Project partner: Salma Kashani

Introduction:

Previously on iteration one, different actuation mechanisms were tested to design a haptic feedback in order to communicate  step commands.They include, Static pressure, direct force (impact) and vibration. We reached our paradigm, when we were applying poke stimuli using long sticks, which could also explain the reason that it was being used in martial art  and military training for gait modification. None of our experiments using pressure and vibration at constant frequency was as informative as poke experiment. We decided to make a similar vibration by exploiting the force impact feeling.

Motivation:

Our initial findings showed that having two body sites works more effectively compared to single body site. In addition one of our main objective is to make our haptic stimuli as close as we can to the poke feeling. These results have motivated us to expand our study on single vs two tractors and explore whether we can have more expressive force feedback using solenoids.

Objectives:

1) Tailoring the force impact by means of solenoid

Some of the main limitations of tractors such as small displacement and low produced force encourage us to see the effect of direct force on haptic perception. Although this force is much smaller than required force to move a leg but at least it is from the same family of poke.

2) One site vs two sites stimuli

 The effect of having redundancy in spatial stimuli mechanism is investigated for both vibration and force impact.

Step 1,

Solenoid vs Voice coil,

Although both of them are working based on electromagnetism forces there are differences in their mechanisms . A solenoid consists of a piece of ferromagnetic material which moves inside of a coil. When coil is energized using electrical current, the ferromagnetic part moves in order to increase the magnetic flux by filling the air gaps. A biased spring, will move back the metallic core when the coil is deenergized. They often are used  to create large displacement and forces very fast(1 time) but not at high frequencies.On the other hand voice coil is permanent magnet which is enclosed in a coil. Their main application is to create vibration so they work at high frequency and generate lower force and displacement compared to solenoids.

• Implementation of Solenoid:

To install our solenoid on site, We needed to make a holder that mechanically connect the solenoid to the velcro. After trying different flexible and rigid materials with different hole sizes we found out that mounting any flexible or rigid material at the head of solenoid can damp the mechanical impact and would not be as effective if we have other contacts to skin close the point of impact. TO have more free of contact space around our impact point we made the box as depicted in fig.1 a.

    

fig.1 a) Implementation of solenoid b)Electrical circuit for driving solenoid (http://electronics.stackexchange.com/)

First,we used a piece of balloon as a biased spring to bring the core of solenoid to initial position and later we removed that and benefit from the body stiffness to do the same job. We derived our solenoid by using a MOSFET and a push button circuit as depicted in fig.1.b.This system can be controlled both by arduino and push button. fig.2 shows the complete setup for one solenoid. In order to have more force,we slightly increased the maximum voltage since we were applying short pulses. (20 V and 500 mA).

Fig.2. Solenoid setup 

Step 2

At this part of our study we run some experiments on our friends to have more feedback on our system and have more unbiased results . Similar to iteration one we conduct our experiments on 6 differents points of the leg fig.3.

Fig.3. Stimulation points on the left leg  (photo courtesy: Salma Kashani)

for each point we tried different actuation and number of sites.the same signal of  vibration in iteration 1 was selected. Both vibration and force impact signals' length were less than a second. in each experiment we mount solenoid or tactor on velcro and make sure that there is enough initial stress that they can work perfectly.  Fig 4. shows the installation of tactor and solenoid on velcro.

fig.4.Mounting Solenoid and tactor on velcro 

In our first experiment we compared the behaviour of one site vibration to solenoid impact. We mounted both of actuators on velcro so the participant can switch between the force feedback immediately and compare them. we asked participant to step back and forth and bend and release her knee while we were  manually triggering the feedback system. A noise cancelling headphone was also used to eliminate the  effect of solenoid sound when being activated.

                                        

fig.5 Applying force feedback to participant's knee

We explained our experiment to the participants and asked them to tell us what was the message of the signal they had felt. if it took them so long to explain the message we provided them with some options and if they couldn't identify the message we considered that signal had no message. at each level we also checked whether they could feel the force signal or not. 

We  tested our two sites  stimuli on a new participant, similar question were asked. In addition we asked our participant to tell us her preference between vibration and force impact and which one she found more effective in conveying the gate modification message. fig.6 shows the two sites stimuli in action. 

fig6. a) Two sites force impact stimuli b)Two sites vibration stimuli 

we ran all of the experiments on our third participant, however as we got closer to the end of experiment our participant seems to be bored and gave the same answers. Due to our lengthy experiment, we decided to just study one subject on each participant in future.

Discussion:

Although our initial experiments on us were suggesting that force impact is more informative than vibration , running the same  experiments on the participant gave unpredicted results. It is still too soon to generalize our results due to limited number of participants and subjectivity of our study, but we reach to following findings based on common answers of our participant.

•  Force impact vs vibration

Two of our participants were mentioning that for single point of contact they could barely feel the force impact and found vibration to be more informative. We expected that our force impact to be at least more sensible because it has higher force and displacement amplitude, but in practice, since vibration was carefully selected from a library of vibration. it could evoke the feeling by working in certain range of frequency, on the contrary,  force impact is working similar to impulse signal and resonant all of the frequencies; however, this full spectrum of resonance can  cause ambiguity in receptors and may have smaller amplitude at certain range of frequency.

• One vs two sites of stimuli

Two of our participants in addition to us found two sites of stimuli to be more transparent in conveying a message signal compared to one stimuli,regardless of the nature of it.  This means that, they got stronger messages of the same type when using two stimuli.  Sometimes  when they were using only one force feedback the message was vague and they could only feel that there was a vibration or impact in comparison with two stimuli that they could realize the message. The conveyed messages were not consistent from one participant to the other but for most of them the back of the leg stimulations had more propelling meaning while in the front leg was more dragging and stopping.  

fig.7. Conveyed messages at different points and different states (standing/walking)

As shown in fig.7 both position and timing of stimuli are important in cognition part. These massages can vary based on position of stimuli and state of the leg/activity. This is one example of different messages during Standing/Walking state.The question mark means no message. 

Why conveying a message using two stimulus is more direct and effective?

Based on our experience and asking our participants we believe that having just one stimuli can be precepted more accidental compared to two stimulus. Having two sites of vibration or impact make a redundancy to our signal and in most of the cases make the user think "for sure there is a message and what should I do? What is that message? " 
The further question that we should respond is the intensive or extensive nature of each of these stimulus. As we know the  intensive property does not depend on the stimuli amplitude and amount, nor the size and extent of actuator , while we have additive relationship in an extensive property.

Source of errors

• Actuators placement
• number of participants 
• Solid design of  experiment
• physical limitation

Conclusion 

In the iteration two, we tried to replicate the similar poking feeling by means of solenoid and expand our study on direct haptic messaging using local stimuli/stimulus. We designed studies to find the differences between 1) Vibration and force impact and 2) Single and two sites of stimuli.
Although we couldn't realize the communication of exact messages to each individual, but our findings show that having vibration or impact at the back part of the leg is more propelling while on the opposite site is more dragging and prohibiting. Perception of vibration is more stronger than force impact when we are dealing with only one site of stimuli, while two sites of force impact is overtaking two vibrators. We could also discover that having having redundancy in our stimuli can help better precepting of the signal and enhance expressiveness of our commands. 

In the next step, instead of exploring different actuation and sequence of activation that have great transparency and conveying a solid message to every user, we decided to study more the intensive and extensive properties of  each stimulus to find the answer that can explain why having two sites of stimuli works better than single site. Both temporal and spatial properties should be tested by means of  designing studies in which placement, amplitude and timing varies. The next question that we are curious about is to know if this is just because we are doubling the amplitude by having two stimulus or we have sensory integration similar to multimodal system.

My contribution

We were developing each part of the project together after having conversation and sharing ideas with each other, so it is hard for me to draw line between my work and Salma's.
I think because of my background, I was mainly involved in hardware developing and experimenting the best design of the solenoids, includes different mounting setups, different solenoid head sizes and mechanisms to bring the solenoid back to its initial position.
I was also involved in designing our study to meet our objectives, plus, synthesizing the results of our study.In addition, I was helping participants to mount the setup on their legs during study.