Ball Tracking Robot
My project the ball tracking robot that uses a raspberry pi connected to a pi camera to track a red ball. It uses image processing to make all of the red pixels of the ball on the screen white and all of the non-red pixels black. If it sees the ball on the left or right side of the screen it turns to face it then goes towards it. I also added a servo mount to the Pi cam as a modification as well as some LEDs one to iluminate what the camera is seeing and the other to blink when the code is running as a saftey feature.
| Engineer | School | Area of Interest | Grade |
|---|---|---|---|
| Hrehaan M | Skyline High School | Mechanical Engineering | Incoming Junior |
Demo Video
Final Milestone
For my final milestone I have added 2 modifications I added a servo Mount for my Pi cam and 2 LEDs on that iluminates what is in front of the car and the other blinks when the code is running as a saftey feature. One problem when implementing the servo mount was the mount came with its own driver board to run the servos but to use it in code you need to use some special libraries which i did not know and none ove my code worked. eventually i plugged the servos strait into the raspberry Pi and they worked. One of the main things I have learned from BSE is how to solve a problem on my own through google or other reasources instead of always asking for help.
Second Milestone
Since my last milestone I completed the my base project. I got the image proccessing working so the robot only sees the ball. I the ball is in the center of the screen it goes forward and if it is on the left or right it turns so it is in the center then goes forward. It also checks if the ball takes up most of the screen if it does then it stops. Then if the ultrasonic sensor detects an object it stops. If the car does not see the ball it starts to spin right slowly until it detects the ball. When I was writing the code for this project there were many errors and problems but the biggest one happened after everything started working. The next day it all sudenly stoped working and it was even more frustrating because everything had been working. I eventualy figured out that some of the ultrasonic sensor wires got randomly got unplugged so the code was getting stuck. Next I plan to add my modifications since my base project is complete. I plan on adding a gimbel for the raspberry pi so it can turn to the left right up and down. I also plan on adding LED lights to the front to improve the image that the pi cam is getting and an LED light that turns on only when the program is running so people know not to go near it when is is running.
First Milestone
My project is the ball tracking robot which uses ultrasonic sensors and a raspberry pi camera to track a red ball. The camera an ultrasonic sensors are all connected to a raspberry pi 4. For my first milestone I finished building the base of my car with the motors and got them connected to the raspberry pi through an H bridge. After that i wrote code to test if the motors worked and the car could move forward, backwars, right, and left. Then i wired up my Ultrasonic sensors to the raspberry pi and wrote code to test them to make sure they worked. Lastly I wrote code to combine the movement of the motors with the what the ultrasonic sensor detects so if the left ultrasonic sensor detects an object less than 5 cm away the car turns to the left if the right ultrasonic sensor detects an object less than 5 cm away the car turns to the right and if the middle one detects an object less than 5 cm away it stops. For my next milestone my plan is to attach the pi camaera to the car and get it to be able to detect the ball.
Schematics
Code
import RPi.GPIO as GPIO
import time
import cv2 #OpenCV
from picamera2 import Picamera2, Preview
import numpy as np
GPIO.setmode(GPIO.BCM)
GPIO.setup(17, GPIO.OUT)
GPIO.setup(22, GPIO.OUT)
GPIO.setup(23, GPIO.OUT)
GPIO.setup(24, GPIO.OUT)
GPIO.setup(14, GPIO.OUT)
GPIO.setup(15, GPIO.OUT)
GPIO.setup(4,GPIO.OUT)
bottom = GPIO.PWM(4,50)
GPIO.setup(3,GPIO.OUT)
top = GPIO.PWM(3,50)
def right(sec):
GPIO.output(17, True)
GPIO.output(22, False)
GPIO.output(23, True)
GPIO.output(24, False)
time.sleep(sec)
stop()
def left(sec):
GPIO.output(17, False)
GPIO.output(22, True)
GPIO.output(23, False)
GPIO.output(24, True)
time.sleep(sec)
stop()
def stop():
GPIO.output(17, False)
GPIO.output(22, False)
GPIO.output(23, False)
GPIO.output(24, False)
def reverse(sec):
GPIO.output(17, True)
GPIO.output(22, False)
GPIO.output(23, False)
GPIO.output(24, True)
time.sleep(sec)
stop()
def forward():
GPIO.output(17, False)
GPIO.output(22, True)
GPIO.output(23, True)
GPIO.output(24, False)
#time.sleep(sec)
#stop()
def top_set_angle(angle):
duty = angle / 18.0 + 2.5
GPIO.output(15, GPIO.HIGH)
GPIO.output(3, True)
top.ChangeDutyCycle(duty)
time.sleep(0.5)
GPIO.output(3, False)
GPIO.output(15, GPIO.LOW)
top.ChangeDutyCycle(0)
def light_blink(LED_PIN):
GPIO.output(LED_PIN, GPIO.HIGH)
time.sleep(1)
GPIO.output(LED_PIN, GPIO.LOW)
GPIO.cleanup()
def bottom_set_angle(angle):
duty = angle / 18.0 + 2.5
GPIO.output(4, True)
bottom.ChangeDutyCycle(duty)
time.sleep(1)
GPIO.output(4, False)
bottom.ChangeDutyCycle(0)
# GPIO pins setup for Ultrasonic Sensor 1
TRIG1 = 16 # GPIO 16 (pin 36)
ECHO1 = 26 # GPIO 26 (pin 37)
# GPIO pins setup for Ultrasonic Sensor 2
TRIG2 = 27 # GPIO 27 (pin 13)
ECHO2 = 25 # GPIO 25 (pin 22)
# GPIO pins setup for Ultrasonic Sensor 3
TRIG3 = 6 # GPIO 6 (pin 31)
ECHO3 = 5 # GPIO 5 (pin 29)
def setup():
GPIO.setmode(GPIO.BCM)
# Setup for Ultrasonic Sensor 1
GPIO.setup(TRIG1, GPIO.OUT)
GPIO.setup(ECHO1, GPIO.IN)
# Setup for Ultrasonic Sensor 2
GPIO.setup(TRIG2, GPIO.OUT)
GPIO.setup(ECHO2, GPIO.IN)
# Setup for Ultrasonic Sensor 3
GPIO.setup(TRIG3, GPIO.OUT)
GPIO.setup(ECHO3, GPIO.IN)
def distance(trig, echo):
# Ensure the trigger pin is low initially
GPIO.output(trig, False)
time.sleep(0.1)
# Generate a short pulse to trigger the sensor
GPIO.output(trig, True)
time.sleep(0.00001)
GPIO.output(trig, False)
# Measure the duration of the echo pulse
while GPIO.input(echo) == 0:
pulse_start = time.time()
while GPIO.input(echo) == 1:
pulse_end = time.time()
pulse_duration = pulse_end - pulse_start
# Speed of sound at sea level is 343m/s, or 34300 cm/s
# Divide by 2 because we're measuring to and from the object
distance = (pulse_duration * 34300) / 2
return distance
def segment_colour(frame):
hsv_roi = cv2.cvtColor(frame, cv2.COLOR_RGB2HSV)
mask_1 = cv2.inRange(hsv_roi, np.array([130, 150,40]), np.array([190,255,255]))
mask = mask_1
kern_dilate = np.ones((12,12),np.uint8)
kern_erode = np.ones((6,6),np.uint8)
mask= cv2.erode(mask,kern_erode) #Eroding
mask= cv2.dilate(mask,kern_dilate) #Dilating
(h,w) = mask.shape
cv2.namedWindow('mask',cv2.WINDOW_NORMAL)
cv2.resizeWindow('mask',800,600)
cv2.imshow('mask', mask) # Shows mask (B&W screen with identified red pixels)
return mask
def find_blob(blob):
largest_contour=0
cont_index=0
contours, hierarchy = cv2.findContours(blob, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
for idx, contour in enumerate(contours):
area=cv2.contourArea(contour)
if (area >largest_contour):
largest_contour=area
cont_index=idx
r=(0,0,2,2)
if len(contours) > 0:
r = cv2.boundingRect(contours[cont_index])
return r,largest_contour
def get_centerX(x,w):
centerX = (w/2) + x
return centerX
def get_centerY(y,h):
centerY = (h/2) + y
return centerY
def draw_center(x,center_x,center_y):
cv2.circle(x,center_x,center_y,10,(255,0,0),-1)
if __name__ == '__main__':
try:
picam2 = Picamera2()
camera_config = picam2.create_still_configuration(main={"size": (1920, 1080)}, lores={"size": (480, 270)}, display="lores")
picam2.configure(camera_config)
picam2.start_preview(Preview.QTGL)
picam2.start()
time.sleep(2)
top.start(0)
bottom.start(0)
angle = 80
top_set_angle(angle)
GPIO.output(14,GPIO.HIGH)
bottom_set_angle(90)
while(True):
GPIO.output(15,GPIO.HIGH)
im = picam2.capture_array()
height = im.shape[0]
width = im.shape[1]
setup()
hsv1 = cv2.cvtColor(im, cv2.COLOR_RGB2HSV)
#fucntion 1
mask_red=segment_colour(im[:,:,[0,1,2]])
#fucntion 2
#time.sleep(0.1)
loct,area=find_blob(mask_red)
x,y,w,h=loct
#print(area)
centerx = get_centerX(x,w)
centery = get_centerY(y,h)
#print(centery)
#print("x = " + str(get_centerX(x,w)))
#print("y = " + str(get_centerY(y,h)))
#print(top_duty)
#print(angle)
GPIO.output(15,GPIO.LOW)
if(centery > 700 and angle < 175):
#top_duty += 0.5
#top.start(0)
angle -= 3
top_set_angle(angle)
#time.sleep(0.5)
#top.stop()
elif(1 < centery < 300 and angle > 5):
#top_duty += 0.5
#top.start(0)
angle += 3
top_set_angle(angle)
#time.sleep()
if(centerx > 1200):
right(0.1)
elif(centerx < 600):
left(0.1)
else:
if(centerx > 500 and centerx < 1300 and area < 1000000):
forward()
else:
stop()
'''
dist1 = distance(TRIG1, ECHO1)
#print(f"Distance from Sensor 1: {dist1:.1f} cm")
# Measure distance for Ultrasonic Sensor 2
dist2 = distance(TRIG2, ECHO2)
#print(f"Distance from Sensor 2: {dist2:.1f} cm")
# Measure distance for Ultrasonic Sensor 3
dist3 = distance(TRIG3, ECHO3)
#print(f"Distance from Sensor 3: {dist3:.1f} cm")
if(dist1 <= 5):
right(0.2)
elif(dist2 <= 5):
left(0.2)
elif(dist3 <= 2):
stop()
'''
#print()
#cv2.circle(im,(get_centerX(x,w),get_centerY(y,h)),100,(255,0,0),-1)
#cv2.namedWindow('x',cv2.WINDOW_NORMAL)
#cv2.resizeWindow('x',800,600)
#cv2.imshow('x',loct)
except KeyboardInterrupt:
stop()
GPIO.cleanup()
top.stop()
# Measure distance for Ultrasonic Sensor 1
Bill of Materials
| Part | Note | Price | Link |
|---|---|---|---|
| Raspberry Pi Kit | controls everything | 95.91 | Link |
| Robot Chassis | base of the car | $18.99 | Link |
| Screwdriver Kit | What the item is used for | $5.94 | Link |
| Ultrasonic Sensor | detect how close an object is to the car | $9.99 | Link |
| H Bridge | control the motors | $7.79 | Link |
| Pi Cam | record a live feed to see the ball | $12.86 | Link |
| Electronics Kit | wire ultrasonic sensors LEDs and H-bridge | $11.98 | Link |
| Motors | move the car | $11.98 | Link |
| DMM | check if there is a wiring problem | $11 | Link |
| AA batteries | power the motors and ultrasonic sensors | $14.89 | Link |
| Champion sports ball | item for robot to track | $12.34 | Link |
| USB power bank | power the raspberry pi | $16.91 | Link |
| Camera tilt platform | move the pi cam up down left and right | $26.99 | Link |