Injury Preventing Smart Shoe
This shoe detects poor pronation to prevent injuries and provides useful biometric data, including heart rate and distance. It also has a navigation function, eliminating the need to check a phone. Finally, it has a bright LED strip that lights up how and when the wearer wants.
| Engineer | School | Area of Interest | Grade |
|---|---|---|---|
|
Lucas M. |
Manhasset High School |
Mechanical Engineering |
Incoming Senior |
DEMO AND REFLECTION
Although I had some basic experience in Arduino and programming before this program, I did not truly know what it meant to be an engineer. While I still may not comprehend it completely, I feel like I understand the engineering process and thinking much better. I also learned a lot about independent learning/research, getting used to sifting through forums and appreciating all the documentation already available from the engineering community. Additionally, I finally understand how engineering is not always simply straightforward numbers or right and wrong. All my previous experience with STEM classes had led me to believe that almost everything is black and white, with not much room for creativity or thinking outside the box. Here, though, I had to do quite a bit of that with my project, whether fitting and attaching components to my shoe in creative ways or figuring out new ways to debug/approach a problem. Finally, I am less resistant to failure or spending a lot of time on seemingly small bugs (while making sure not to neglect other parts of the project), as I recognize the value in these processes. This experience definitely strengthened my interest engineering, and I’m excited to use what I learned here in the future.
FOURTH MILESTONE
For my fourth milestone, I added a pulse sensor to measure heart rate, a Neopixel LED strip with buttons, a potentiometer, and a piezo pressure sensor, and put all my hardware on my shoe. I first added the pulse sensor. When a heartbeat occurs, blood is gets squeezed into the capillary tissues, which increases their volume, and between heartbeats, the volume inside the capillary tissues decreases. This change in volume between the heartbeats affects the amount of light that will transmit through these tissues, which is measured by the pulse sensor. I had to place a transparent sticker on the sensor’s front and a covering of hot glue on its back to prevent any possible damage from sweat or oily skin. A person’s heart beat can be felt on the top of the foot near the ankle, so I put the sensor on the underside of the shoe’s tongue. I’ve had a lot of trouble getting accurate beats per minute calculations, even though the beats can be visualized pretty clearly using Arduino’s built in serial plotter. The sensor gives values to the amount of light it measures, and counts a heartbeat when the measurement is above a threshold specified by the user. The sensor then uses this to calculate the beats per minute. However, very slight movements or even just environmental factors can change what this threshold should be, and it is very difficult to write a code that will automatically account for these changes in the threshold. The heartbeat counter can be improved upon by also measuring a lower threshold, but it is still not perfect. Also, the optimal place for this sensor to measure heart rate is on the finger tip, and it is not as accurate on the foot.
I next added the LED strip, which can act both as an aesthetic boost and as a safety measure if exercising at night. I used a second Arduino Nano for this, as I had very little space on my perfboard and the memory on the other Nano was running low. I have a piezo vibration sensor on the back of the shoe, which detects if pressure is placed on it. If the wearer taps it with their other foot, the LED strip will turn on and off. When there is no pressure or vibration, the sensor measures 1023, and lower values when touched. The sensitivity is pretty high, so the sensor measures 0 pretty much every time a decent amount of pressure is put on it, which is what I use to activate the lights, but sometimes keeps reading 0 a few seconds after pressure is released. I had to add a wait time of three seconds between the possibility to turn the strip on or off to avoid this, but this causes some lag for the user. I also have five different colored buttons: white, red, blue, yellow, and green. The LED strip will turn the color of whatever color button is pressed. All these buttons are connected to a single analog pin, but each connected by a resistor of a different value. When a button is pressed, the Arduino separates the buttons by reading a different analog value for each, due to the resistors, and displays a color accordingly. I also have a black button that calls a blink function, which involves the potentiometer. Depending on how the wearer sets the potentiometer, the blink will be faster or slower. I also had to add a very short delay after the reading of this blink button, since each press was often counted more than once and would flicker between solid and blink mode.
Finally, I soldered everything together and put it on my shoe. I soldered the second Nano, the LED strip, and the other pieces onto a perfboard, and connected both my Nano’s together via wires on the 5v and ground pins. I am powering the boards using a power bank without the auto-off feature, which is being connected directly to my second Nano. I used velcro, double sided adhesive, and hot glue to attach the two perf boards, the LED strip, and the other smaller pieces in a pretty compact and organized way on the shoe. I LED strip was surprisingly hard to attach, since electrical tape, the double sided adhesive, and hot glue all slid right off of its rubber exterior. I had to wrap it in a thin velcro at critical connection points and then glue the velcro to an adhesive attached to the shoe. A couple of solder joints including small wires, like the vibration motor and the vibration sensor, fell apart, so I put some hot glue on after I re-soldered to strengthen the joint. Finally, I did some minor debugging and tested everything.
I tried to add a bluetooth module so the user could control the LED’s from their phone or computer. However, after wiring up the module and powering the Arduino, it started smoking and killed the Arduino’s diode, so it can no longer be powered through the USB. I decided to give up bluetooth to make sure there was no further damage. Luckily, I can still power the Arduino through the 5v pin and I can still upload code through the USB port.
THIRD MILESTONE
For my third milestone, I incorporated a GPS module and then soldered most of my parts together. I first tried to add wifi capabilities to the Arduino Nano, but this proved too difficult and not worth the amount of time I spent on it. My first approach was to use an ESP8266 wifi microcontroller along with my nano; I couldn’t switch entirely to the ESP8266 since there were not enough analog pins (I am using at least 7; the nano has eight and the ESP8266 has one). I was able to get a connection to the computer, but kept having trouble uploading code to the ESP8266 to receive the data from the Arduino, and gave up after a couple days. I then tried using the Particle Photon, since I had already used it earlier in my project. Using the photon had another advantage, besides wifi; my code was extremely close to the maximum flash storage of the Arduino Nano, and shifting some things to the photon would save space on the nano. I connected the Photon to my laptop through my wifi, and powered it through the nano. I succeeded in using serial communication between the two microcontrollers, and realized that only char types could be sent, which greatly complicated the process of sending four digit ints. I also had problems with sending multiple numbers from different Arduino functions immediately after the other and then receiving and displaying those numbers the right way on the Photon. I eventually decided the real time dashboard (which is what wifi would give me) was not worth it, and the local SD card would be enough.
I then added a GPS, which I used to add three main components to my project with the help of the Tiny GPS library made by Mikal Hart. I was unable to use the Tiny GPS++ library because of storage issues. GPS works by communicating with satellites, and determines its exact location by measuring the time it takes to receive signals from multiple satellites. The first thing I added was basic metrics, including distance ran/walked, the current and average overall mile pace, and the cumulative elevation gain. I decided to measure speed with the mile pace, and not miles per hour, since that’s the standard with runners, although I used the miles per hour to convert into mile pace. I calculated the average mile pace by finding the average of each current mile pace measured every second. Since there wasn’t a built in function that found the total distance moved, I estimated it by using the current mile pace throughout the activity. Finally, I created a function to find the cumulative elevation gain throughout the activity (whenever the elevation increases, the increase is added to the cumulative elevation gain).
The second part was a function that helps the wearer navigate to a predetermined point without a phone or other navigation device by using two small piezo vibration motors on the left and right side of the shoe. If the wearer has to go left, the left vibration motor goes off, and it vibrates more for a sharper turn, and vice versa. The right vibration motor works the same way. There is no vibration to go straight, and both have a mild vibration to turn around. When the destination is reached, both buzzers go off at maximum vibration. I found some example code for the navigation part of this, but it was written with the TinyGPS++ library, so I had to adapt it for TinyGPS. The third addition was a feature that allows the wearer to see where he/she went during their activity. Every second, the latitude and longitude of the wearer is saved onto the SD card. This data can then be imported onto a platform called QGIS, where I layered a Google Maps satellite hybrid map. The lat/longs are plotted on top of the map, showing the route taken. When testing this with a power bank, I noticed the “auto-off” feature since the Arduino takes in so little power, which is a small issue. I also later realized I could just import the data straight into Google Maps, and I made it so if you click on a specific point along your route, you will be shown not just the lat/long but also the altitude and mile pace at that point.
I also created a step counter function, since many people like to keep track of their steps in an activity/day. It works by detecting if the force on one of the force sensors is very close to zero (lifting up your foot), and reevaluates a step every 1/5 of a second. If the person lifts up their foot, they are taking step, and then I multiply this by two to calculate the total steps.
Finally, I soldered a bunch of the components together. First, I soldered the force sensors together and put them on the underside of the sole of my shoe. I connected all the ground and power wires of the force sensor so it would take less space when soldered onto the perfboard to connect to the Arduino Nano, and covered all exposed connections with electrical tape. Then I inserted the sole into the shoe and fed the wires out through the top. The force sensors are very sensitive to heat, and I accidentally destroyed one of them while soldering, so I ended up with four force sensors instead of the five that I had wanted. I then slowly soldered the nano, gyroscope, SD card breakout board, GPS module, and slide switch onto the perfboard in the most space efficient way I could. I also soldered the vibration buzzers together and taped them to the front left and front right of the shoe.
SECOND MILESTONE
For my second milestone, I created a real-time dashboard using a ThingSpeak channel, incorporated a gyroscope and slide switch, and included a micro-SD card reader to log and later view data on my computer. I integrated a ThingSpeak channel by basically “pushing” the data to the specific API key of my channel over Wifi, although I had a lot of trouble with this and looked at a number of different example codes before the integration worked. On ThingSpeak, I created a different graph for each force sensor and the force values read at a specific time interval were graphed. However, after I successfully connected to ThingSpeak, I found out that the free version would only update every 15 seconds, so it would miss all of the force readings that happened in between those 15 second intervals. I realized in the context of my running/walking shoe application, the most important readings were the maximum readings, so I changed the code so ThingSpeak would display the maximum force reading from each 15 second interval.
Next, I incorporated a gyroscope. This will let me see the rotation of the shoe in two or three different planes. I had tried to incorporate a gyroscope earlier without soldering, but was unable to; luckily I had just recently learned how to solder and this was the first soldered part that I included in my project. I first tested the gyroscope on Arduino using a custom library I found, which worked well. The only problem was due to the “noise” of the gyroscope, where the values are often slightly off and changing even when the gyroscope is not moving. In my application, though, this should not be a problem. However, I kept having trouble running this library with the photon, and wasn’t able to get the gyroscope working on the photon. Around the same time, the Particle Web IDE and Particle Cloud went down and had some major bugs, so I decided to just transition back to Arduino and plan to add a Wifi module to connect it to ThingSpeak. I also incorporated a slide switch that allows the wearer to pause the collection of data, if, for example, they are taking a water break, and then want to resume their activity.
Finally, I added a SD card reader so the wearer will be able to log data without staying in Wifi range of a computer. I used the millis() function in my code to avoid using a delay and also to calculate and print out the number of seconds since the start, as this will be the x-axis of my graphs. I then formatted the time in seconds and force sensor and gyroscope data onto a file on the SD card that could be organized by Google Sheets. Lastly, I imported this data onto Sheets, where I made graphs for the force and rotational data as a function of time. I next want to add a GPS and explore the possibilities it offers, and a pulse sensor to measure heart rate, and at some point integrate everything onto a shoe.
FIRST MILESTONE
For my first milestone, I got force sensitive resistors (FSR) to output readings to an OLED screen. I first set up an FSR on an Arduino Uno board. FSR’s are technically both resistors and sensors. Their resistance changes when more or less pressure/force is placed on them, which is how they work. I then replaced the Arduino board with a Particle Photon. This allowed me to connect to my phone and laptop, and I can use Wifi later on. I then connected an OLED screen to display the force sensor readings, which will be helpful for debugging any problems with force sensors I use. Finally, I added two more force sensors and outputted everything in a easily readable way.
I used the Particle Web IDE to code the Particle Photon, and I ran into a couple problems. One was that the code used to include a library could not be copy and pasted, even though it looks exactly the same to the user. Another problem was that an example code had a break in it, and when I tried to flash new code to the Particle Photon, for some reason the break in the code prevented the update. This happened even when the Web IDE was telling me the flash was successfully happening, and I had to set the photon to safe mode, which makes the current firmware on the board stop running. One problem I had with my fsr was with a connecting resistor. I started with a 10k resistor, but this restricted the maximum reading of the FSR to a value lower than what will normally be exerted while running, so I downgraded to a 680R resistor, which increased the range and works much better.
