The Eyes, They be Moving!

Here's a short video that shows the eyes on Exuro moving. You can see Mac moving around in the screen on the lower right of the video and the eyes moving to track the closest part of him. I'll post more details soon but right now I'm just psyched that they're moving and tracking pretty well!

Kinect, Python and Pushing Through

There's tangent project that I'm working on that involves robotics, arduinos, kinect and fire. It's a small robot called Exuro that has a pair of stainless steel eyes that are meant to track a person coming up to a donation box and when they make a donation, set off a small poofer. The idea is to track people using a kinect and have the eyes move as if they're watching the closest person to the donation box. Working with an arduino to control external systems is pretty straight forward for me, it's something that I've done before. But pulling sensor data from something like a kinect and interpreting the data is something I've never done. It's rather intimidating. Processing video data at something like 30 frames per second, not something I'm used to do. But it sounds like fun!

There's an open source driver to access the kinect called libfreenect that's available from openkinect.org. Included are wrappers for using the library from Python which most definitely my preferred programming language. That works.

Getting libfreenect to build on a Ubuntu 10.10 system was pretty straight forward. Just follow the instructions in the README.asciidoc file. Getting the Python wrappers to work took a bit more effort. cython is used to create the bindings between libfreenect and Python. Unfortunately, the version that's currently included with Ubuntu 10.10 isn't up to the task. Once I removed the Ubuntu and installed from the latest source, the Python bindings built and worked as just fine. I'm sure the fine folks maintaining Ubuntu will make a newer version available at some point, I'm just not willing to put this project on hold till they do ;-)

There's a few demo files that are included with the wrapper so you can start to play with the interface, library and the kinect data. Two of them, demo_cv_sync.py and demo_cv_thresh_sweep.py, make for demo. The first opens two windows and shows a live video feed of the rgb camera in one and the depth camera in the other. The other demo shows a video of the depth camera but sweeps through the data showing what's seen at different depths. These are really interesting demos to help wrap your head around what's available from the kinect.

I got to wondering about the depth data and if there wasn't a way to combine the two demos to be able to slide through the depth manually to see what's there. The result is demo_cv_threshold.py. It allows you to slide along at any depth to see what's there and then to contract or expand to see what's around that depth. Here's a sample video showing my hand pushing through a virtual wall:



The depth slider sets the focal point for what data to display and the threshold provides a +/- tolerance for how much data to display. A depth of 535 and a threshold of 0 would show just the data at 535 while a depth of 535 and a threshold of 6 would show the data from 529 thru 541.

It's an interesting application to play with the gain a basic understanding of the data being returned and possible ways to use it. I've submitted a pull request on github to the maintainers of libfreenect to see if they're willing to include it in the next release. Here's hoping that they will.

There's a lot more work I need to do for this project. The next steps will be to find the closest person in the data stream and calculate their location in the real world in reference to the location of the kinect. And I have almost no idea how to go about doing that. Time to read up on numpy and opencv...

Arduino and LEDs

I’m a software developer by training and one aspect of the software that I’ve always developed is that it just lives inside a computer or network. It doesn’t interact with the real world. That’s been well and good for the last 20 years but of late I’ve been wanting to break out and have software that I develop effect some change in the physical world.

There have been fits and starts where I’ve tried writing code for some embedded system or chip but the mountain of basic knowledge thats needed to get started has always been overwhelming. I would read some book (like Embedded System Design on a Shoestring ) and while the books make it seem easy to get going, the projects would always die a quick and quiet death. There’s nothing wrong with Embedded System Design on a Shoestring or any of the other books I tried, many people have used them quite successfully. Unfortunately it just never clicked for me.

But that’s all changed.

I recently discovered the Arduino platform. The system provides a small microprocessor, a bunch of I/O pins, a USB connection and a very easy to use IDE. The basic features of the platform allow for most any type of interaction with the physical world that I can currently envision. I’m sure there will be some scenario that it won’t support but for right now, I’m very happy with the capabilities.

One of the problems that I encountered in the past was connecting an embedded system to a host platform. It may have involved a vendor specific hardware or something else that wasn’t supported on my systems. The Arduino uses a USB port. After installing the needed drivers on my OS X system, I was able to connect the Arduino without any problems. Seems way easier than other solutions.

The place that I see the Arduino really shining is with the IDE. It’s different that any other IDE that I’ve used but the interface is really straight forward and very easy to use. I was able to quickly try out one of the sample C programs that turns an LED on and off again. The thing that really got me was that it compiled, loaded and ran on the first try. This was just brilliant. Easy to use and works the first time. Outstanding.

When a friend asked for help with a project he was doing and told me that he’d like to have 3 LEDs turning on and off in a random fashion, I knew that an Arduino would be the perfect tool to use. I was able to rework the sample code for turning an LED on and off to work on this project.

The idea behind the project is to have an LED inside of a wax pumpkin where the LED would turn on and off like the light on a firefly. There would be three pumpkins on a small platform, hence the three LEDs.

Here’s the code for the project:

/*
* firefly multibutt illumination
*
* Copyright (c) 2008 Peter Kropf. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/


// all times are in milliseconds
#define SPIN_DELAY 10

#define ILLUMINATION_MAX       255
#define ILLUMINATION_MIN       0
#define ILLUMINATION_INCRIMENT 5

#define MAX_ILLUMINATION_TIME 2000
#define MIN_ILLUMINATION_TIME 500 

#define MAX_DARK_TIME 10000
#define MIN_DARK_TIME 2000


#define LED_COUNT    6
#define LED_OFF      1
#define LED_ON       2
#define LED_SPINUP   3
#define LED_SPINDOWN 4

int led_pin[LED_COUNT]   = {3, 5, 6, 9, 10, 11};
int led_state[LED_COUNT] = {LED_OFF, LED_OFF, LED_OFF, LED_OFF, LED_OFF, LED_OFF};
int led_spin[LED_COUNT]  = {0, 0, 0, 0, 0, 0};
int led_level[LED_COUNT] = {0, 0, 0, 0, 0, 0};


void setup()
{
   randomSeed(analogRead(0));
}


void loop()
{
   for (int led = 0; led < LED_COUNT; led++) {
       switch (led_state[led]) {
       case LED_OFF:
           if (led_spin[led] > SPIN_DELAY) {
               led_spin[led] -= SPIN_DELAY;
           } else {
               led_state[led] = LED_SPINUP;
           }
           break;

       case LED_ON:
           if (led_spin[led] > SPIN_DELAY) {
               led_spin[led] -= SPIN_DELAY;
           } else {
               led_state[led] = LED_SPINDOWN;
           }
           break;

       case LED_SPINUP:
           if (led_level[led] < ILLUMINATION_MAX) {
               led_level[led] += ILLUMINATION_INCRIMENT;
           } else {
               led_level[led] = ILLUMINATION_MAX;
               led_state[led] = LED_ON;
               led_spin[led]  = random(MIN_ILLUMINATION_TIME, MAX_ILLUMINATION_TIME);
           }
           break;

       case LED_SPINDOWN:
           if (led_level[led] > ILLUMINATION_MIN) {
               led_level[led] -= ILLUMINATION_INCRIMENT;
           } else {
               led_level[led] = ILLUMINATION_MIN;
               led_state[led] = LED_OFF;
               led_spin[led]  = random(MIN_DARK_TIME, MAX_DARK_TIME);
           }
           break;
       }

       analogWrite(led_pin[led], led_level[led]);
   }

   delay(SPIN_DELAY);
}

The code is pretty straight forward. Each of the LEDs has a particular state that it’s in: off, on, spinning up or spinning down. That’s tracked in the led_state array. The led_level array holds the current level of illumination for each of the LEDs and the led_spin array holds the random amount of time that the LED will remain on or off. The rest of the code manages the arrays, sets the current level of the LEDs and waits for a small delay before doing it all over again.

Note that the code is currently setup to illuminate up to 6 LEDs since that’s the number of digital I/O lines on the Arduino that supports pulse width modulation, PWM. PWM is how the Arduino is able to vary the light level produced by the LEDs.

Here's a short video I took while working on the LED timing. It shows only one LED instead of the three that were used in the finished project.

With this project successfully completed, I’m looking forward to my next project: having a flashing light turned on when a specific voicemail box on a Asterisk based phone system has messages waiting. This’ll involve connecting an Arduino to ethernet and controlling a 120v circuit. I can’t wait.