Pong Assist Graphic Representation

So, it occurred to me, after watching the velocity-tracking program I wrote spit out endless values at me, that it might be useful to have a computer animation of the ball, Pong Assist paddle, and starting and ending trajectories, so that, in the future, I can differentiate between hardware problems and programming problems. As a result, I've begun work on a Processing program that will produce a visual of the intended interaction of paddle and ball. Below, I've posted a drawing of what I want the representation to resemble eventually and a video of what I have so far. Needless to say, there's a long way to go, but I'm pretty happy with how things look so far.

Intended Visual Representation Layout:

Progress So Far (loooong way to go)

Ping Pong Ball Velocity Program (Video)

Ping Pong Ball Velocity Program

So, with the lag in Kinect imaging finally sorted out, and with a program (thanks to Making Things See by Greg Boorenstein) that successfully tracks the point closest to the Kinect I decided it was time to create a program capable of tracking the velocity of that closest point. The first step in doing so was to compensate for the fact that the closest-point program I'm using tracks x and y-values based upon pixels. Obviously, a movement of 5 pixels three feet from the Kinect is smaller than a movement of 5 pixels twenty feet from the Kinect, so some sort of calibration was necessary to attain consistent velocity values. To accomplish this calibration, I first determined the field of view of the Kinect by taking specs. I found online and expanding upon them with some trig. functions, as shown below:

Field of View Calculations

Next, I needed to write an algorithm that would take those field of vision calculations, as well as the Kinect's generated depth values (ranging from 0 to 2048), and determine the ball's velocity based upon them. Below is a picture of the calculations I used, as well as an explanation of variable and my procedure.

Velocity Algorithm Calculations

- First, delta x and delta y (the difference between the Kinect's horizontal and vertical fields of view at 6m and at .7m) were computed.

- Delta x and y were then divided by 2048 (the maximum depth value the Kinect returns) to determine the x-constant (Xc) and y-constant (Yc)--the amount in meters that the horizontal and vertical fields of vision of the Kinect grow by for each unit of depth.

- Xc was then multiplied by D (the depth value of the closest point as returned by the Kinect). The resulting product was then added to the minimum horizontal field of view (.7601m) to determine the size of the horizontal field of view at depth D. The resulting sum was then divided by 640 (the number of pixels making up the horizontal field of view) to yield the value Ux (the distance in meters between each pixel at a given depth D in the x-dimension).

- The same process was undertaken to determine Uy, though a different value (.5515m) was used for the minimum field of view and a different number of pixels (480) was used as a divisor, in accordance with the y-field's smaller size.

- Next, the velocity in the x direction (Vx), was determined by multiplying Ux by delta pixelsx (the change in x value (in pixels) from one Kinect measurement to the next) and by 24 (the number of Kinect measurements (as determined by Kinect frame rate) per second). By multiplying by 24 in this manner, the Vx value returned is in m/s, rather than in m/((1/24)s), which would be a really irritating unit to have to deal with.

- The velocity in the y direction (Vy) was determined in the same manner.

 

At this point, I realized that I'd forgotten I needed the ball's velocity in the z-direction as well, so I began another set of similar calculations:

  

- First, delta z was determined by subtracting the minimum functioning Kinect distance from the maximum (6m-.7m).

- This value was then divided by 2048 (the max depth value returned by the Kinect) to determine a z-constant (Zc)

-  Zc was then multiplied by D (the depth value). The resulting product was then added to the minimum depth (.7m) to determine the true depth at returned depth D. The resulting sum was then divided by 2048 (the max depth value) to yield the value Uz.

- Uz was then multiplied by delta D (the change in depth from one Kinect measurement to the next) and multiplied by 24 to yield Vz in m/s.

FINAL VELOCITY CALCULATION:

With these myriad values determined, I then needed to calculate the overall velocity:

- The magnitude of the overall velocity (Mv) was calculated by taking the square root of (Vz)^2 + (Vx)^2 + (Vy)^2.

- The angle of the overall velocity (with respect to the z-axis) was then determined by taking arcsin(Vx/Mv)

This algorithm has been used to produce a functional Processing program in conjunction with the Kinect. A video of this program in action will be uploaded in the near future (ie. as soon as I find a program that allows me to take video of what's happening on my computer screen)

Lag Solved!

As it turns out, the only reason I was having so many issues with lag in my Kinect program was my choice of computer. I was running simpleopenni and processing on an Acer netbook with one gig of RAM which, as it turns out, is not really enough to support a Processing program of any great complexity. When I downloaded and configured simpleopenni for my family's brand new iMac (8 gigs of RAM), the lag disappeared completely. So, problem solved! Although it's significantly more difficult to position a 21" iMac next to a ping pong table than it is a 10.1" netbook....

Paddle Mechanisms

While I talked about the paddle mechanisms and shared some pictures in an earlier post, I didn't have any actual video of them, so here are a few:

 Paddle with Motor Attachment

Paddle without Motor Attachment (Range of Motion Displayed)

Individual Mechanisms

As soon as I get my motor shield up and running (I'm waiting on a couple extra parts I had to order), I'll also post a video of the mechanism actually being driven by the motors.

Kinect Tripod Mount

I designed and built a tripod mount for the Kinect so it'll be a little easier to actually position by a physical ping pong table when I get to that point in the project. It's also necessary because simpleopenni doesn't, for whatever reason, allow you to control the Kinect's motor and adjust the angle of view, so it's helpful to be able to do that manually with a tripod. Here's a video of the mount: 

Pong Assist T-Shirt

So, whenever I get fed up with actually try to accomplishing things on my project, I design something related to it instead (ie. a logo, a blog, etc.) This time I've decided to design a t-shirt and have, as of yesterday, formatted the lettering for the back (though I've yet to cut out a stencil of it) and cut the stencils for the logo on the front. I've done a few tests to determine the best way to arrange those stencils and I think I've come up with an arrangement I really like. There's a picture of it below, though the colors are not the ones I plan to use on the actual shirt (they're just ones that could effectively be layered on top of each other using marker). I've also included a picture of the lettering for the back below. Hopefully I'll get a chance to cut out the the stencils for the back and actually print a shirt soon!

Back of Shirt Part 1 (will be on one line)

Back of Shirt Part 2 (last word will ultimately be in-line with the others for both captions)

Stenciled Logo for Front of Shirt

Other Misc. Logo (maybe useful somewhere else eventually)

Stencils Used to Produce Logos

My "Brand Mark," which I'll put in the area where the tag is in the collar of the shirt

Wall Wart Power

As I described in an earlier post, I've decided to use external power with my motor shield to drive my servo motors. The question that left was what exactly I should use as that external power source. Unfortunately, the servos require 5v power, which essentially rules out batteries as an option. AA, AAA, C and D batteries are only 1.5v, which is far too little, and 9v batteries, which might feasibly work voltage-wise, have far too little capacity to drive the motors for any length of time whatsoever. Any larger batteries will have both too much voltage and way, way, way too much weight. The best option would probably be to order a 5v battery online, but that would be both expensive and time-consuming and, for all I know, ultimately unsuccessful, so instead I plan to use a 5v wall wart from some phone long ago lost to the ages.

While electronics draw only what power they need from batteries, wall warts function differently, and will feed the full amount of voltage to the components regardless of the actual power needing to be drawn. This is an issue because although the wall wart I'm re-purposing is rated for 5.1v, the load on the wall wart in amps was likely to be different than the load the phone required. If the load were less, the wall wart would feed more than 5v to my engines and possibly destroy them in the process. To make sure this wouldn't happen, I cut off the end of the wall wart cord that would have plugged into the phone, stripped the positive and negative wire inside, and tested the voltage passing through them with no load using a multimeter. To my surprise, the multimeter indicated that the wall wart was still providing almost exactly 5.1v of power. As a result, I've decided to go ahead and try it with the servo motors, as the worst that can happen is that the load from the motors will be too great for the wall wart to provide enough power to run them. That'll keep my motors from running, but at least it won't damage them, and I can then get another wall wart and test that, hopefully with better results.

Below are a couple images of the wall wart I plan to use:

Kinect, Processing, Arduino, and a whole lot of lag

Well, the good news is that I have a tracking program that, when a point is at a certain depth, communicates with an Arduino through processing and turns on an LED. That's exciting. What isn't exciting is the huge amount of lag I'm dealing with. From troubleshooting and writing out various sections of code, I've determined that the lag is a result of the communication between processing and Arduino and not as a result of any of the actual program content, as little lag is present when the Arduino communication is stripped from the code. I'm using a library called "Firmata" as a means of communicating with the Arduino. It involves loading an unchanging Firmata code onto the Arduino and then using a Firmata library to actually control the Arduino through code in Processing. It's very cool. It is, however, also a very big source of lag for some reason. I previously tried communicating with the Arduino by sending communication between Processing and the Arduino coding interface using serial commands, but this also created lag. I hoped using Firmata would fix the problem, but it just really hasn't at all. It's a problem I'm still looking into and one that, unfortunately, makes my program pretty much useless for the moment.

Plan for Kinect Ball-Tracking program

The good news is that I believe I have a method for tracking the ping pong ball using the Arduino. I'll post code and a more detailed explanation shortly, but in the meantime, I'll just provide a basic idea of the system I plan to use. It will consist of, in essence, a closest-point tracking system. I plan to position the Kinect off to the side of the ping pong table, facing it horizontally. From this angle, the Kinect will be programmed to ignore objects such as the net or the edge of the table, and to only recognize objects within a depth range as wide as the table. After these changes to the program are enacted, the moving table tennis ball will be the closest point to the Kinect within its frame of view. From this point, it is a simple matter to track the closest point using an application of code I learned in Making Things See. By then comparing the x, y, and depth positions of the closest point from each frame to the next, the velocity and trajectory of the ball could be roughly calculated. The main issue that could arise with this system would be the straying of people's hands or paddles into the Kinect's viewing frame. This would introduce a new moving closest point, confusing the system and ruining ball tracking. I believe, however, that a relatively simple piece of code could introduce a limiting case that would eliminate this problem. As the trajectory of the ball after it is hit by the Pong-Assist-equipped paddle has no bearing on the motorized actions of the paddle, by recognizing only objects entering from the field of view opposite the player, it could be ensured that only the ball would be tracked.

Adafruit Motor Shield Purchased and Soldered

I've been really busy lately doing a variety of things so I haven't gotten a chance to make any posts about my progress, but that doesn't mean there hasn't been progress. As a result, I'll be making a flurry of posts very shortly about the variety of things I've accomplished over the last month or so. Just before I went on winter break the Adafruit Motor Shield I'm going to use to drive my servo motors was delivered and over the break, I soldered it together (it comes as a bag of assorted components, which is a bit of a pain, but also was a good opportunity to learn how to solder). This is the finished product:

After plugging it in and hooking up my two servos as a test, I found that the motor shield was functional (the power light turned on) and that it was capable of driving the two servos without an external power source, running purely off of Arduino power. However, I'm concerned that once I begin hooking up other components to the Arduino, the motors will fail to function, just as they did previously when I powered them through the Arduino without  use of the motor shield while using other components. Because of this concern, I've decided to use external power with the motor shield, even though it involves cutting some trace on the shield, a process I'm a little concerned about doing, as a small slip would pretty much ruin the entire component. Nevertheless, I'm convinced it's necessary so I'll be doing that shortly and then attempting to run the motors using a 5v wall wart. More posts on how exactly I'm hoping to do that will follow shortly.

There's also one more issue I've discovered with the motor shield--it fits directly on top of the Arduino and hooks up to every single pin, making them impossible to access for use with other components, such as buttons, LEDs, potentiometers, etc. However, after researching I have found that the analog pins on the Arduino are still available and can be accessed through the shield. However, as I don't want to solder wires directly to the board and there were no female headers included in the components I received from Adafruit, I'm in the process of ordering a set of female headers that I can solder on to the shield and use as serial ports. As soon as they've arrived and been soldered, I should be able to begin testing the shield in earnest, using potentiometers to control the two motors while the kinect program is still being developed.