Firing Up (BeagleBone Black, WebCam, Missile Launcher and OpenCV)

This post was originally posted on Firing Up (BeagleBone Black, WebCam, Missile Launcher and OpenCV).

If you are an SAP Employee, please follow us on Jam.

My good friend John Astill came to visit us at d-shop some time ago…and he saw the Thunder Missile Launcher that I bought as a Christmas present for my team dinner. He told me…”Blag…you should build that app that fires up missiles when you put a color or object in front of the camera”…that got me thinking of course -;)

So John…this blog is for you 

So, let’s start…

You will need OpenCV installed…so the regular way should work…

sudo apt-get install opencv

And OpenCV should be installed and ready to use now…

Next…I needed to hack the Missile Launcher in order to be able to make it fire…so this is where the fun begins…I hooked the Missile Launcher to my Linux box on VMWare and made a lsusb -vv command to get the information about the USB device…

Here we can see that the IdVendor is 0x2123 and the idProduct is 0x1010. This doesn’t really give us anything as we don’t know to pass the information we need in order to make it work…so next step was to sniff the device in order to get the USB commands…I did this using SnoopyPro a USB Sniffer Tools for Windows…and just notice that I couldn’t make it work properly on Windows 7, so I need to use my old and nice Windows XP VMWare Image…and install the Dream Cheeky software...

Once we have SnoopyPro running, need to connect the Missile Launcher and click on File --> Unpack Drivers…

Next, we need to look for the correct IdVendor and IdProduct and right-click to select Install and Restart.

This will allow us to log everything that happens while using the device. By default, at least for the Missile Launcher…it will log 14 lines that don’t matter much to us…but when we hit for example the Up button…then we get more interesting info…

This is basically saying that to make the launcher move up, we need to pass the following…

0x02 0x02 0x00 0x00 0x00 0x00 0x00 0x00

Now…the part that interests us is the Fire action…so let’s explore it…

This one is

0x02 0x10 0x00 0x00 0x00 0x00 0x00 0x00

Cool…now we’re getting closer…but…there’s a piece that it’s missing…how do we send this command? Easy…just take a look at the Transfer command…

This command tells us that we need to pass the information like this…

0x21 0x09 0 0

Followed by the action we want to perform…

In order to use the device on our BeagleBone…we need to install something first…

pip install pysub

This library will allow us to connect and more important, send commands to our device.

As we’re going to use a Camera and the Missile Launcher both at the same time…we need a USB Hub that it’s powered…otherwise, it will never work…I got this one from Amazon.

So…power up the USB Hub and connect both WebCam and Missile Launcher, connect the Hub to your BeagleBone Black and then connect the BeagleBone to your laptop…

Use Nano or any other editor of your choice and create a file called Thunder.py

Thunder.py
import cv2 import numpy as np import usb.core cam = cv2.VideoCapture(-1) cam.set(3, 960) cam.set(4,720) s, img = cam.read() dev = usb.core.find(idVendor=0x2123,idProduct=0x1010) try: dev.detach_kernel_driver(0) except Exception, e: pass winName = "Thunder" cv2.namedWindow(winName, cv2.CV_WINDOW_AUTOSIZE) while s: s, img = cam.read() key = cv2.waitKey(10) if key == 27: cv.destroyWindow(winName) break hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) lower_green = np.array([38, 100, 50]) upper_green = np.array([75, 255, 255]) mask = cv2.inRange(hsv, lower_green, upper_green) res = cv2.bitwise_and(img,img, mask=mask) cv2.imshow(winName, res) if cv2.countNonZero(mask) > 61000: dev.ctrl_transfer(0x21, 0x09, 0, 0, [0x02, 0x010, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00])

This code will basically open up the camera connection, and turn the display black so only the color that we want to track is shown. In this case, we want to fire up the missile launcher only when we see something “green”. As you can see we’re passing the transfer command as single parameters but the fire command as an array.

Now…the BeagleBone doesn’t have a display…so you could attach a monitor and a keyboard or you could do something more interesting and smart…

We’re going to install Tight VNC Server

sudo apt-get install tightvncserver

The first time you run it, it will ask you for a password for the current user, so go ahead and provide one…

And it will start running…so you will be able to open a Remote Desktop connection to your BeagleBone…now you only need a client to connect to it...

If you’re using Windows you can download the TightVNC Client from here…

http://www.tightvnc.com/download/2.7.10/tightvnc-2.7.10-setup-64bit.msi

http://www.tightvnc.com/download/2.7.10/tightvnc-2.7.10-setup-32bit.msi

If you’re using Linux you can simply download it using apt-get install like this…

sudo apt-get install xtightvncviewer

Run it on the terminal

Just provide the IP for your BeagleBone which should be 192.168.7.2:1 on the small box that will appear, provide the password that you setup for the VNCServer on the next screen and you’re all set…

If you’re using Windows…simply double click the tightvnc-jviewer.jar file and enter the following information…

In order to test this, we need to simply print a nice green box…

Then we can run the app…

And put the green box in front of the camera -;)

Sorry for the flaky picture…but the Missile Launcher was firing off and I had my phone on one hand and the green box paper in the other one -;)

Greetings,

Blag.

Development Culture.