Object Detection with Deep Learning on Raspberry Pi



Taiga A.

Area of Interest

Aerospace Engineering


Rising Senior


Monta Vista High School

Reflection: This project was… 

Final Milestone

Social hosted video URL is empty.

For my final milestone…

Second Milestone

Second Milestone Video

With my second milestone, I was able to detect cups using the model I trained on NanoNets. To get to this point, I first had to train the model using NanoNets, which is an online service that does machine learning on their end. I had to gather 50 images of cups of various sizes and angles to annotate. Then, I used a python file which uses the NanoNets API to communicate with the model on the cloud. Object detection using NanoNets puts a lot less strain on the raspberry pi compared to the first milestone because the comparison of tensors is done on the cloud. That being said, the speed of response is still slow because I am using the free NanoNets plan which caps the speed intentionally.

The process of training the model was pretty straightforward because NanoNets provided a very straightforward with their online user interface. However, implementing the model with python code was confusing because I did not know which links to reference. Finally, it was very frustrating that NanoNets forces a subscription of $99 per month to download the model using docker. Overall, I learned that machine learning is not an inexpensive task, whether you use cloud services, TPUs, or GPUs.

First Milestone

First Milestone

With my first milestone, I was able to setup the raspberry pi and perform object detection using tensorflow, protobuf, and the ssd mobilenet pre-trained tensorflow zoo model. To get to this point, I first needed to setup the raspberry pi with raspbian via NOOBS. Then, I had to install Tensorflow, OpenCV, and Protobuf using the command prompt. Finally, I had to compile the python file which initiates the object detection.

The object detection which takes place in my setup is somewhat complicated. First, the executed python file takes in the image from the pi camera to process. Then, Tensorflow and Protobuf are used to communicate the tensors of the image with the pre-trained model. The pre-trained model returns a confidence value of how sure something is that object. The python program then draws boxes on parts of the image where the confidence threshold is greater than 0.3. A tensor is a multi-dimensional array which is used to compare the image to the categories defined in the model.

Setting up the raspberry pi with Raspbian, and connecting the camera was straightforward. However, the steps which came after were different. I messed up the steps of installing Tensorflow and ended up corrupting the ‘pip’ versions when troubleshooting. My pip and pip3 had to compatibility issues which could not be fixed easily. I had to get reinstall Raspbian again on the SD card. Additionally, the process of installing TensorFlow and Protobuf was painfully slow for it took more than an hour for some commands. I also ran into a problem with the python file, which I took from an online tutorial because I did not have the matplotlib module to import. However, the program turned out to to work without the import so I commented those imports out. Through this milestone, I learned more about tensors, protocol buffers, and openCV which perform crucial back-end tasks to make object detection possible.

Starter Project

Starter Project Video

For my starter project, I chose the Big Time Watch because I wanted to practice soldering, and I was interested in how time was kept on electronic devices. The Big Time Watch is a big acrylic watch which is capable of telling the time. The watch uses a pre-programmed ATmega328 processor, a 32 kHz crystal oscillator, a 10kΩ resistor, two 10μF capacitors, a button, and a 7-segment display. The time in this watch is kept by the quartz in the crystal oscillator, which oscillates at a constant frequency of 32,000 times per second when electricity flows through. The hardest part of the project was soldering the display and microprocessor, which had many vertical pins. I made a mistake when soldering the crystal oscillator because I did not leave enough space for the oscillator to bend down, but because it did not have a significant effect on the whole design, I left it as is.

Leave a Comment

Start typing and press Enter to search