Move  
Iteration #1

Project partner- Salma Kashani

Introduction

Human locomotion is mainly characterized  by the sequence of lifting and releasing of the limbs in which  propel the center of gravity forward at each step. There are 6 possible configurations of limbs for a biped walker. (lift left leg, release left leg/ lift right leg, release right leg / lift both legs and release both legs together). Human can move naturally or under specialized training. These set of various moves are called human gaits. The natural gaits include 5 main categories of  walk, jog, run, skip and sprint are common between cultures. On the other hand the specialized gaits are being used for medical, training or entertainment purposes and require direct commands to start or modify the movement .

Displaying on a screen or using voice commands are among the most common ways, which are exploited to direct to specific gait position, although they may overuse the auditory and visual sensory input  and sometimes be misleading.

In this project we offer to use haptic feedback to convey the gait commands directly to the limbs that are involved in movement to have a fast and efficient communication. Physical perception and haptic feedback seems to work more transparent compared to auditory and visual feedback and need lower  level of cognitive processing. Unlike conventional haptic feedback which are placed on person's wrist or hand, we believe that the most effective communication will occur only when they target the designated limb.

Objectives

As mentioned earlier, in this project we will implement a local haptic feedback to provide highly transparent signal for gate modification. In our first iteration we will answer the following question:

1. What is the best way to measure the position of the leg accurately ?
2. Given various types of actuation, which one will be the most perceivable actuation by the user? 
3. Which method of actuation can better convey the message and lead to special movement without any training?
4. Where are the effective points of stimulation on the leg for certain movement and what  is the proper number of actuators?

First we will try to implement the position sensor  to measure the knee bending. Second, we explore different actuation both in case of feeling and the message that is conveyed. In the next step, based on the conclusion we got from the second step we will investigate the appropriate actuation to guide to the determined move.

STEP 1 

             Knee positioning:

Before starting any actuation and producing haptic feedback, we need to know the current position of the limb. It is necessary that this measurement doesn't interfere with the ongoing activity. We had two different types of sensors which can give us the amount of knee bending by changing their resistivity. Our first option was using a conductive stretchable fabric and sewing that to a knee pad. We observed that during the leg lift we have stretch of the knee pad in both horizontal and vertical direction. We observe 30% change of resistivity on fully lifted leg with full scale resistance of 3Ω. Our second option was using a flex sensor with 40 kΩ resistance. we also observed more than 40% change of resistivity for fully lifted and fully released leg. We opt out to use the flex sensor due to simpler readout circuit.

STEP 2

             Actuation principles:

We divided the actuation techniques based on their mechanism into 3 main groups:

2-1 Static pressure with compressibility.
2-2 Direct force and impact  by means of rigid actuator.
2-3 Vibration.  

To perform uniform experiments we defined three location on upper limb, lower limb and knee and exposed them to various stimulation. Our main focuses in this study were the feeling and the message which was conveyed. Since the magnitude of actuation is largely dependent on the leg situation and  the  amount of pre-stress during installation we will not discuss that in this project.

Fig1, Stimulation points on the left leg  (photo courtesy: Salma Kashani)

2-1 Compressible inflated balloon with static pressure behaviour.

Similar to the mechanism of compression socks, we made a static pressure by placing an inflated balloon between knee pad and the leg on 6 different points as depicted in Fig1. The reason that we haven't tried the dynamic pressure was the slow response of pneumatic pumps in small scale and other implementation difficulties. Another challenge that we faced during our experiment was the  high amount of prestress of knee pad which was dominant compared to amount of pressure caused by inflated balloon. Later we decided to replace that with a loose fabric.

 Fig2, Static pressure applied to the leg (photo courtesy: Salma Kashani)

Results:We tried these experiments on 3 different people (We ask one of our friends to join us) and observed that by placing the static pressure at any point except the knee it is hardly noticeable. The only feeling that one of us has experienced was in the situation where we put the balloon behind the knee and it prevents the full bend of knee during walking and guiding to stop.


2-2 Direct force and impact  by means of rigid actuator.

Being inspired by the gait modification methods in military and martial arts trainings, we decided to apply direct force to different parts of leg manually by means of wood sticks. At this point we could not only feel the impact and force, but also it directs us to different movements. By poking at different points of legs we observed that different messages such as lifting or releasing the leg or stepping back and forth can be conveyed, however these messages were not consistent from one person to another. We also asked another friend to join our experiment without telling her the meaning of each signal and wanted her to explain the task she wanted to do for each poke. 

Video 1, Applying single direct force

Results:This method seems to give us more reliable outputs with enough transparency to follow the instructions without thinking about them. However we didn't have a general perception for each impact.After a discussion we agreed on the following instruction:

Fig3, Different impact points and their suggested actions (photo courtesy: Salma Kashani)

• Front of the leg
• Upper: State:lifted ,Action: release
• Knee:  bent the knee. 
• Lower: stepping back. 
• Back of the leg
• Upper: State:released   Action:lift 
• Knee: Sit. 
• Lower: stepping forward.

In our next experiment we increased the number of impact points to two. We noticed that at some level, The user needs to think about the action and the message is ambiguous. Our intuition was that just by using a single impact, we are just sending alert signals and it can be interpreted that it happened accidentally.  By having two points of stress, the message was more clear and consistent between different users and they don't need to think before taking actions. Just by poking at the lower limbs the user starts to walk by evoking the push feeling and poking at the sides we could stop the motion by creating gripping feeling.

Video 2, Applying direct forces at two points

2-3 Vibration.  

By adding an unbalanced weight to a DC motor we made a very powerful vibration system. At this point, the frequency and magnitude of vibration were kept constant.We mounted the vibration system on different points of the leg and vibration continued until the full action is fulfilled.

                                         Video 3, Mounting vibration system on upper limb.

Results:

Vibration is shown to be a powerful techniques to alert the user, but at this level was unable to convey a motion message unless the user is being told about the desired action. It appeared that for some cases, the feeling was unpleasant and causes pain or stress so user was just obeying to relieve from that.


STEP 3

In step 2, we studied different mechanisms of actuation and whether they can convey a certain message. Among the plausible actuation (inflation of balloon, poking and vibration) we selected vibration and poking based on their perception by the user. In step 3 we will study more deeply on different variables of vibration and we will try to deliver a message by applying a designed class of vibration.
Our first experience of applying direct force by means of wooden sticks was successful in conveying the desired motion messages. We consider this effect in our selection of vibration signals.First we started by using a vibration generator app called HIVExport designed by Oliver Schneider from SPIN lab to generate vibration with different frequencies and time lengths. We connect the output signals to an amplifier which drives a tractor and tried different vibration. Although we could tell the difference between them but none of them could really convey a motion message. So we decided to use more complex vibration signals by using a predesigned vibration library (VibViz) by Hasti Seifi from SPIN lab. We selected 8 different vibrations based on the feeling and closeness to poking experiments.

                            Fig 4, Selection of 8 vibration signals.(photo courtesy: Salma Kashani)

Then we mounted our tactor/tactors on a piece of Velcro fabric strip and rounded it around different parts of the leg so the tactor be placed on predefined stimulation points with moderate tightness. Similar to our direct force experiment we tried effect of one and two points of actuation.

Fig 5, Mounting tactor/tactors on a peach of Velcro fabric

Fig 6, In situ testing of vibration tactors

By in situ testing of vibration signals on our legs, we narrowed down the list of vibration to 3 and make them just a single beat.

Fig 7,  Final selection of vibration signals

Results:

1. One Tactor :    While we expected to see similar effect of one impact in direct force, we could only feel the vibration which didn't lead to a specific motion. At this level vibration was not unpleasant or causing stress but it was hard to tell the message by its own.                                            
2. Two Tactors at 10 cm distance  : At this level we could feel the vibration not only in two parts but also in the area between two tactors. These vibrations seems to be able to resonant the skin and create the haptic illusion. Even the larger area of vibration couldn't lead us to certain movement.
3. Two Tactors at 15 cm distance : By increasing the distance between two tactors we could better distinguish two separate vibration which sounds to be more expressive such that by feeling them, user will try to change the state of leg to the other.For example if the leg is in lifted position and vibrations are being applied it will direct to release and vice versa.

Conclusion

In our first iteration we studied different actuation mechanisms to grant natural feeling for gait modification. Among different methods of actuation(pressure, impact and vibration), applying direct force and vibration are considered as good candidates to communicate our gait commands.While we could only recognize the alert signal from vibration, applying direct force and impact could lead us to specific moves more voluntarily and seems to be processed at lower cognitive level. later, a class of vibration signals is introduced to mimic the same effect of impact. Further we investigate the effect of having two vibrations at different distances. We observed that by having 2 vibrations at far distances relative to each other, our vibrations are more expressive for the purpose of motion so they can lead to change the state of the gait;however there is still large gap between the feelings which are evoked by vibration compared to impact signal. 

Lab 3: Communicate Something.

Waitperson assistant; A smart coaster that communicates [Beware, Attend, and Resolve]

Lab partner: Salma Kashani.

Problem:

In order to appeal to the new customers and keep the old clients, the customer service plays a crucial role in all the small and large businesses. In this project, we came up with an intelligent coaster to assist waitpersons manage and provide better services to the diners during busy hours. The main duties of our smart coaster are to communicate the following words:

•  Attend [uh-atend]: to take care of; to pay attention  to
• Resolve [ri-zolv]: to settle; to deal with 
• Beware [bih-wait]: to be cautious; be careful 

We have narrowed down our focus to deal with a situation when the waitperson needs to provide warm tea to the diners in a restaurant. Imagine a busy day at a sushi place full of people who not only need to be served with food but also wish to be treated with tea at the right temperature. So the waitpersons need to keep their eyes on the consumers' cups to fill and refill them.

By communicating Attend we aimed to show the waitperson when:

1.              There is a diner with no teacup;

2.              Teacup is more than half empty and; 

3.              Tea is too cold and needs to be changed.

In the Resolve situation there would be an interaction between waitperson and system when:

1.             A waitperson has seen the request and is about take an action. This will reduce the traffic of requests and avoid distracting other waitpersons.

2.             System should work on the standby mode (no need of help).

Beware is a word of communication between the system and diner, and the system and waitperson when:

1.             Teacup is too hot and should be touched carefully.

2.             Tea is in the normal range and safe to drink.

3.             Tea is cold and needs to be replenished.

Realization 

In order to communicate the mentioned words with our hardware, we used a group of sensors, an actuator and a light indicator. To measure the temperature a TMP36  sensor was used and directly connected to the analog pin 0. Assuming that there is a linear relation between temperature and output voltage we use analogread with default frequency of arduino (50-200 kHz) temperature can be calculated by simple subtraction and multiplication. More details about wiring and functionality can be found here.

A RGB LED was connected to digital pins of 3, 5, 6 for our light indication purposes (these pins were not used by motor shield).

Fig1, Implementation of Temp Sensor and RGB indicator (photo courtesy Salma Kashani )

A force sensing resistor 0.5" circle with a range of 0-10 kg was wired to analog pin 1. We also considered a linear relation between pressure and change of resistance and put it in series with a 10k  resistor. So by using voltage dividing technique we can find the resistance change of the FSR and read the voltage accordingly, however it shows near infinite resistance in no touch case. Here is more details about FSR. 

Fig 2, A force resistive sensor is mounted on the surface of coaster (photo courtesy Salma Kashani )

To control the movements of our DC motor we used Adafruit motor shield V2 kit.  The motor was connected to  M1 pins so we can change the polarity of voltage and control the speed. Using "Forward" and "Backward" we will change the polarity of voltage and the amount of driving voltage by mapping [0V-5V]to [0-255.] The motor is connected to our coaster by means of 2 stages 3:1 gear train so when the motor is engaged it can produce enough torque to rotate the teacup. We made two types of movement by rotating the coaster back and forth.

 a) Turning move by making a speed ramp between 0:128 .

 b) Shaking move by ramping the speed between 100:200,

Video1, Hardware test using serial input (Video courtesy Salma Kashani )

The following conditions were used that our hardware communicates the predefined words:

• If there is no cup on the coaster or need a refill:
•  No keystroke; shake the coaster
• If there is a cup on the coaster:
• Temperature < 30 C;  turn on the the LED light to blue
•  30 C <temperature <70 C; change the LED light to green
• Temperature>70 C; change the LED light to red & start turning the coaster  

• Keystroke; Standby mode to stop the movement turn off LED

here you can find the link to the code!

Results:

As Video 2 shows, three words of communication is implemented in our hardware to serve a warm tea to diners by informing the waitpersons without distracting them .If the tea temperature is too hot and may burn the diners, the LED will turn red and we can see the slight turn movements of the cup (Beware) and if the tea is in right temperature the LED will turn green and if it is too cold, it will turn blue to ask for warmer tea.

If there is a customer with no teacup or it is less than half empty, the coaster will shake to inform the waitperson (Attend). When the wait person received the system's message and is going to do the proper action (Resolve).By pressing the keystroke the system will go on standby.

Video2, Communication of  3 words of Attend, Beware and shake.(Video courtesy Salma Kashani )

Future plan:

The movement of the system both in shake and turn need more improvement. There are more conditions that need to be considered in real situation and need to be added for example when someone is lifting the cup to drink or put it somewhere else. There should be a history of requests and responses per each customer to avoid many unpredicted situation.

Lab 2:Arduino Intro + Actuator Basics

lab partner: Salma Kashani.



Smart haptic panel using multiprobe surface scanner with tunable turning disc.

Chances are so high, that we have seen or at least have heard about a mechanical recording and reproducing device,it's called phonograph or later known as gramophone. A song can be recorded by engraving or etching grooves and incisions on a rotating disc. As the  result when the probe is tracing for any groove and incisions, while the disc is rotating, it will be vibrated and creates the recorded sound.

Being made by Edison at 1890s, gramophone is still fascinating enough to inspire us to use it in our next variation in lab2. We exploited this mechanism to generate different local vibration patterns on an smart haptic panel. The haptic panel can provide users with different experiences, when they are trying to press or drag their finger in it. Our system consists of 3 main parts,

• Tunable turning disc
• Multiprobe surface scanner
• Smart haptic panel

The rest of this post will continue as follow. First, I will explain each part in more details and it's functionality, principle mechanism and science behind it. Then, I will explain the the iteration and improvement for each section. Later, I will describe the system as whole and how these parts work together, and finally I will come with some problems and future plan.

1. Tunable turning disc

In first step, we needed to make a rotating disc with premade vibration patterns. We have used a DC motor to produce our rotational movement. In order to reduce the speed and enhancing the torque, we mounted two stages of 3:1 gear train between motor and disc. A stepper motor is also tried, but due to vibration and lower speed we prefered to use a DC motor.

To control the rotational speed of the motor we changed the driving voltage of motors by means of pulse width modulation method. Each time Arduino  will read the voltage of variable resistor (Potentiometer) using AnalogRead with 10 bit resolution and  accordingly, will map it to 1 to 255 ratio of duty cycle. Arduino will generate pulses with different duty cycles and switch the voltage across the motor using a Mosfet so that by changing the potentiometer we can control the speed of rotation.

Fig 1) Arduino kit and motor driving circuit (photo courtesy Salma Kashani)

Fig 2) Rotational speed control using potentiometer (photo courtesy Salma Kashani)

To create the vibration pattern we tested different cavity and bump structures and got into the conclusion that for our purpose a simple ramp which is cut through the disc will work just fine.

We observed that our probes will got stuck in some ramp edges so we covered the disc with a layer of plastic sheet. By having different ramp heights we can control the magnitude of vibration.

Fig 3) Cutting ramps through the rotating disc to generate vibration (photo courtesy Salma Kashani)

          2-Multiprobe surface scanner

As described in previous post a multiprobes surface scanner (MSS) can detect the roughness and surface properties including concave and convex areas. As depicted in Fig 4, we mounted the MSS system over the turning table so it can scan the surface of the rotating disc.Fig 5 shows how the rotational movement of the disc is transformed in linear vibration of probes.  

Fig 4) Multiprobe surface scanner is mounted on turning bench.  (photo courtesy Salma Kashani)

Fig 5) Transformation of rotational movement to linear displacement of probes is generating vibration on each probe.  (photo courtesy Salma Kashani)

(photo courtesy Salma Kashani)

           3-Smart Haptic panel

The mystery behind our smart haptic panel is a non-Newtonian fluid called Oobleck; These group of materials  has properties of both liquids and solids at the same time. You can slowly dip your hand into it like a liquid, but you will feel it solid if you squeeze or punch it.  We have experienced that just by vibrating this material it can create different feeling of pressing or friction when you are dragging your finger into it. In another word, the viscosity of this material will change based on frequency and amplitude of vibration which is analogous to  change of damping ratio or friction. Basicly you will feel the dragging force while pressing or moving your finger on the panel. Higher frequency ==> higher viscosity==> feels like moving your finger in a honey jar, Lower frequency==> lower viscosity==> feels like moving your finger into glass of water. Bellow you can see a movie of the effect of vibration on changing the viscosity of Oobleck. The trace behind the finger is relative to the viscosity of liquid.

         Results:

Now assume that you have a liquid panel of Oobleck  with a number of local vibrators. At each segment you can control the stiffness/viscosity by changing the vibration's frequency. The magnitude of vibration also controls the size of each segment and it's range. To generate many vibrations with just one motor, we have used the MSS which is connected to the rotating disc. by controlling the speed of rotation and size of the ramps we can control the frequency and magnitude of vibration. 

Future plan:

More flexible material is required instead of the plastic plate that can transform the vibration to the Oobleck (maybe ziploc!). The coupling between rotational and linear motion needs further improvements.