Features:

• Autonomous Operation
• Remote Control
• Stereo Vision
• Gyro Balancing
• Tilt Sensing
• Proximity Sensing
• Bluetooth, Remote Proessing

Movie Captions

Watch or comment this on YouTube.

Solid blue lines are wired connections. Dashed blue lines are wireless connections. Items above the dashed red line are on the robot, items below it are by the operator.

Not too much of a rat's nest. All of the wires were cut and re-molexed to a proper length.

The Bluetooth is covered in bubble wrap and placed on the very back. Almost no bulging when the cover is on!

Too see pictures of the sensors go here.

This software gives an operator a view of the robots vision and duplicates the functionality of the remote control. The head can be controlled too (remote control cant do this becuase head is attached to KCB not RCB).

RoboCom also does machine vision. It currently can figure out depth perception using the stereo cameras and track the location of faces in it's vision.

Check out the videos for a demonstration.

A pain of stereo vision is software calibration of the cameras. To calibrate, the robot is shown a series of checkerboards. Becuase the software knows what size and shape the checkerboard squares are supposed to be, it can figure out the angle and distance of the chekerboard in its left camera, and then compare that to the apparent angle and distance of the checkerboard in its right camera. This plus the known distance between the 2 cameras tell the software how the cameras are tilted and shifted relative to each other. The checkerboard information also allows to correct for the curvature of the lenses. The output of the calibration is in the screenshots below.

Sound is also received but is just passed through to the speakers for now.

Manual head controller.

Live sensor info.

Here is the output of the facial recognition. It works on my cat too.

See a full size image

The right 3D part of the image is a view from behind and to the right of the robot. The faint box around this view is 1 meter in each dimension. The height of the wall that is visible to the robot is about 50cm.

You can see in the right part of this image that the green pillow is being reconstructed in 3D from the 2 2D input images.

The middle pair of images are calibrated, the bottom pair are outputs of their "disparity."

See a full size image

The [nearly] same input, but the 3D view on the right was slightly rotated to get the robots body out of the way.

See a full size image

This time the robot's head was rotated a little. The pillow's 3D reconstruction to the right is more complete, but of course the door to the left disappears.

Download

Requirements: Xcode 3.1 under OSX.

Download RoboCom (Version 100307, BSD License)

The autonomous software. Master Control Program.

To control the RCB with the KCB, the KCB emulates a remote controller over the low speed serial port of the RCB. The other method would be to send commands over the com ports of the KCB and RCB. I can't do that because I need the com port on the KCB available for Bluetooth for communication with the computer. Take a look at the flowchart and wiring diagram.

Right now the autonomous functions are simple. If the feet sensors detect something near, the robot turns left or right. If the acceleration sensor detects that the robot is on its front or back, it gets up.

Check out the videos for a demonstration.

Most of the complexity has been setting up the scheme of communication between the KCB, RCB, Computer, and sensors. And splitting up the functionality.

The MCP controls the head servos with the KCB PIO ports using PWM. Information from the Kondo support blog (1, 2) helped this.

Download

The MCP is dependent on my modifications to the KCB-1 SDK uart.h, and also rcb3-sio2.h to help control the KCB as a remote over the low speed serial. KCB-SDK must be version 1.1.0 Rev.20090410.

Download the MCP (Version 100307, BSD License)

As far as I could tell, the com port buffer in the KCB SDK is only one byte. This is even in the case of enabling the interrupt driven com port. So if this buffer is not polled frequently enough, data will be lost. This offers almost no advantage over using the non-interrupt driven mode and polling the device. This is no good for communicating with my computer via Bluetooth.

I modified uart.h to contain a FIFO buffer. Now the interrupt driven com port makes sense.

How-To

Sorry, the KCB-SDK is copyrighted. So you have to purchase the kit and patch uart.h yourself becuase I can't provide it for download.

Make sure you are using 1.1.0 Rev.20090410 of the KCB-SDK. In uart.h, insert the following code at line 104

And after that insert the following code at line 607 of uart.h.

Usage

I need communication between the KCB and RCB, but I can't use the com ports becuase I need that for the Bluetooth. Another option is to use a spare serial port on the KCB to communicate with the radio controller low speed serial port on the RCB.

The "spare" serial port on the KCB is meant to communicate with serial servos, which I don't have. If I do start using serial servos from the KCB, there is still another serial port free for this. The problem with this is that the low speed serial on the RCB can only accept input. To get output from the RCB a spare servo port is used by setting its signal to HI or LO. This is attached to an AD input of the KCB. It is set to HI when the RCB is "busy" as described here.

Becuase the radio control method is used, communication from the KCB to RCB must take the format of a Button (up, down, etc) and up two 4 variables. This would be a limitation in some cases, but since I only need the KCB to conrol RCB, it works out idealy. See the wiring chart for more info.

I created a header to make communication with the RCB's low speed serial port easier with the KCB's SIO2 port. It was inspired by this blog entry at the Kondo's site. And here at the Robospot blog.

The following chart shows the pros and cons of different communication methods between the KCB and RCB. The first method is basicly what I do.

Note: If the RCB is not upgraded to 1.1a, it cannot properly communicate with the RCB over the com port.

Usage

The command "code" takes the format described in krc.h. Ie: BTN7B_LU for left up button.
I usually redefine these to the names of my motion routines like this:
#define BTN7B_LU FORWARD.

Download

The KCB-SDK must be version 1.1.0 Rev.20090410 to use rcb3-sio2.h

Get rcb3-sio2.h (Version 090927, BSD License)

This is the software that controls the movement of the robot. I have not really done anything special to the original data provided by Kondo. My get-up routines are original, becuase my robot has a funny center of gravity these days. But they are pretty bad so I don't recommend you using them.

I have two versions of nearly each motion. One version is a single shot execution which is activated by a button. The other version executes the motion continuously until the button is released. The reason for this is becuase it makes testing easier, and also controlling from the KCB by emulating a controller becomes easier. In the MCP KCB-1 App I explain this in greater detail.

The most important modification to these motion routines is that they use a servo output to communicate with the KCB. A high signal tells the KCB that the RCB is executing a motion, and a low signal tells the KCB that the RCB is idle. The KCB side of the software is explained here. Refer to the wiring diagram to see that the RCB servo 24 output signal wire is connected to the signal wire of the AD 7 input on the KCB.

I learned that trick from this post on the Kondo site which includes a picture of how to set it up from Heart2Heart.

Download

Download my motion routines (Version 090927, BSD License)

This affected me after updating the RCB-3 to 1.1a.

The RCB would sporadically execute the motion routine in slot 8. So I setup a dummy motion in slot 8 that simply maintains the robot's current state. I would rather have setup a blank motion, but that is not permitted by H2H.

The motion to fix it is caled BUG.RCB and is available in my pack of motion routines here.

This handles the communication with the computer. It is nice and small after removing the serial port, power plug, and switches.

To set it up:

1. Disable the CTS/RTS by enabling switch #1.
2. [Optional] Then connect to it over the com port with a terminal emulator. Using the documented commands, add to its bond list the Bluetooth address of your computer and set a password for the connection.
3. Then from the computer, scan for Bluetooth devices. Connect to the device and set it up as a serial port with SPP. If you are in Windows, set the connection to be initialized by the computer.
4. [Optional] Enable options to use the security key during communication.

By default, you can send data to it with the KCB by opening a com port connection at 115k baud, no parity, 1 stop bit. Check out the MCP software for some source code examples.

I've had interference problems as described here.

This is a much better solution than the Kondo based one. The Kondo controller only supports PA-1 and PA-2 analog input of the RCB, and those inputs overlap with up/down/left/right button input. With the PS2 controller you can get all 4 analog channels, and the D-Pad does not overlap with them. You also get some additional buttons.

However, the REV-1 is very big because of the PS2 plug on it. The Hori wireless PS2 receiver which plugs into is has the same problem. So I took them apart and soldered them together with no problems. In the picture the REV is on the top.

It is bubble wrapped and crammed in between the KCB and RCB.

It has taken a lot of work to get the vision system right. I started out with these cameras. Their FOV was too narrow for good stereo vision but that could have been remedied with a new lens.

The biggest problem they had was that they were not designed to operate simultainiously. They were both on slightly different signals, but they completely interfered with each other.

Another problem was that they transmitted over 900MHz, which doesn't have enough bandwidth for a crisp signal. Avoid getting these cameras!

Don't get any wireless camera with a receiver that looks like this!

Here is the current setup. The product info is above. 90 degrees seems like a good FOV for stereo vision. After lens distortion correction (to remove the pin cushion effect) the FOV becomes about 70 degrees. The separation of the cameras is 33mm. These cameras are designed to operate up to 4 simultaneously.

Here's the wireless security camera I am currently using. After taking it apart, the camera board and wireless transmitter board are separate. The interface between the two boards are standard composite video and audio wires. So it would be possible to remove the transmitter and wire the camera directly into a capture card.

I power the receivers with the firewire port of my MacBook, so everything is still completely portable.

USB Video Capture

Getting the video capture stuff to work was another challenge.

I wanted to use USB capture devices. Firewire is too expensive and for the purpose of this project I didn't need the speed or quality of firewire. I also wanted to develop in OS X so I needed a device that came with drivers for it.

I bought two of the same capture device becuase I wanted to make sure the video was consistent. It turns out that the low quality drivers wouldn't work like that. They can only support one capture device plugged into the system.

So I bought two more USB capture devices. Same result. I checked out the .plist and binaries of the drivers, and I realized that they were the same driver with just a few string changes! Both of my capture devices actually had the same circuitry and parts in them. So the driver was sold originally by a third company to these two manufactureres. I went to the third company for their (not free) core version of the driver, which supposedly supports multiple devices plugged in, but it didn't work.

The current solution is to use one of each brand capture device. Now I can do two capture devices simultaneously.

Linux/Windows Notes

Because I am investigating purchasing a Roboard I tried using both of the same device in windows and linux too.
The linux drivers for video capture didn't have built in support for the Japanese manufacturers of my capture devices. But since the design of them seems to be generic, I was able to set up a driver for them based on a similar device that was already supported. Unfortunately I can't get 2 simultanious devices working. I'm using the v4l2 interface and I get errors when selecting the device.

On the Windows side, one driver could load both capture devices simultaneously. I don't know if the driver is compatible with the Roboard though. If the driver uses mmx, then the Roboard will not support it in Windows as described here at Robosavvy. This is not a worry in Linux, becuase there we can compile our own driver without mmx.

Check out the RoboCom section to see some images of the stereo vision at work.

The head is controlled with the KCB PIO from the MCP software.

First Attempt: Cover Basics

This is the first head I made. It's just an L-bar attached to the original head of the robot. The power adapter for the cameras is in a Lego block, and I left the original plugs on them.

This head functioned fine as I develeoped the the stereo vision software. But after the software started working it was time to replace it. If it can't look down, it's not going to be very useful.

Second Attempt: 2 DOF, Mini Servos

I needed to make a 2 degree of freedom head. The Kondo servos are too big to build the head externally, and at the time I was using the inside of the torso for the Bluetooth and KRR receiver. So I decided to get some mini 6V servos.

This power adapter gets power from the RCB, and a PWM signal from the KCB. It provides power to the cameras, and power and signal to the servos. It had one power regulator for the cameras, and one for the servos.
It failed.

One small power regulator was not enough for both of the servos. The servos would move more and more sporadically as the regulator overheated, which was very quickly.

I experimentally determined that it took 2 of these power regulators per servo to keep them running well. With the power loss from the high current on this, I may as well use Kondo's 10V servos. Realizing that a more sophisticated power supply was needed, I scrapped this design.

The mini servos were nice, but the horns that I used to assemble the head were too flimsy. Also you may have noticed, I built this one upside-down. The bottom joint should rotate and the top should tilt.

Third Attempt: Final 2 DOF

The next version of the power adapter just passes the power through, because the incoming voltage is reado for Kondo HV servos. Same input and output as the previous version.

The smaller 2 circuit boards are for the cameras IR power. After cuting off the excessively large power plugs from the cameras, I discovered that they had hidden these boards inside the casing of the plug.

This is the completion of the 2 DOF servo assembley for the head. I finally went with standard Kondo brackets and servos.

I also fit the servo inside the torso. To do this, a hole had to be cut on the top plate, and I had to relocate the bulky power switch to the chest of the robot.

I also relocated the microphone from the back of the camera to the side. The bracket for the cameras has many holes so I can determine the best separation. However, the camera's FOV was so narrow that I had to keep them together. These cameras were junk, and soon replaced.

Fourth Attempt: New Cameras and Lenses

Here is the current and hopefully final version. The new cameras a little larger. The cameras and circuit boards are protected by the large L-bar and a piece of plastic. On the other side of the plastic I mounted the transmitters.

The power adapter is much better this time. The power and signal to the servos are handled by a couple of small splitters inside the backpack now. The power adapter has a separate regulator for each camera, and a third one for the infrared lighting. The infrared stuff is not currently in use. Each power circuit is isolated and has its own smoothing with capacitors too. This was to improve the quality of the video transmission. The microphones are currentlty not attached.

Fifth Revision: Quality

A protective plastic casing has been added around everything. The cameras are now fully protected by the new upper L-bar. Stereo microphones were added, and the audio is transmitted.

Acceleration Sensor

The acceleration sensor fits in this hole on the back below the backpack.
It is attached with hot melt glue.
If I add waist a rotation servo to the robot, this will have to move.

I can't get a picture of where the gyro sensors are right now. But they are in the compartment behind this hole.

Distance Sensors

To make it removable I glued a separate Lego block to the foot of the robot and the sensor.

The front Lego with the hole is to guide the incoming signal more strictly. It helps prevent the two sensors from interferring with each other.

Update - I have removed the lego mounts as shown here. Also, I think the signal guiding hole was too narrow.

This is an extra option for the KHR. The KHR-1HV kit comes with the extra parts needed, you just have to buy two more servos.

Here is a link to the Robospot blog on how to assemble it. And here it is on my robot.

Now using these feet from the KHR3.

They are a little larger than before, which is kind of cheating, but it is very helpful. Their curved shape seems good for the tatami and carpet.

I also replaced the lego blocks with some shoddy homemade alumininum brackets.

The cameras, Bluetooth, and PS2 controller all communicate over 2.4Ghz. This is not good.

When the Bluetooth is on, the video suffers slightly. Sometimes if the cameras are turned on before the Bluetooth syncs, the Bluetooth cannot establish a connection. This is inconsistent, I think when the battery is stronger, the cameras transmit a little too loud for the Bluetooth. External interference from the neighbors microwave or the positioning of the moon ;) could play a part too.

If the PS2 receiver is on before the Bluetooth, the same thing happens. But, if I turn the receiver on first, and make sure that it syncs with the PS2 controller (get it out of the searching mode) then the Bluetooth works fine. The PS2 receiver does not seem to affect the video quality.

I used to have the Bluetooth adapter on the top of the robots backpack, parallel to the ground. I have changed it so now the Bluetooth is behind the backpack, and rotated 90 degrees to be perpendicular to the ground. This positioning seems to have reduced the interference problems. I might put a wall of RF sheilding in between each of these components, but I don't think grounding it to the robots body is good enough.

The main reason I am having problems with this is because of the Bluetooth protocol. It marches up and down the spectrum as it transmits. With 802.11 it is possible to set a static frequency, so I could set it to something out of the range of the video and controller. Also, 802.11 checks for other devices transmitting on the same frequency, and tries to wait for them to become quiet before it transmits. Originally I could not find 802.11 to serial adapters, so I thought Bluetooth was my only option. Since then I have found such adapters but they are too bulky anyway.

I have considered getting a Roboard, a miniature PC for robots. Then I could do the video capture on-board and wouldn't have to worry about that transmission. Also, any off-robot processing could be transmitted with 802.11 so Bluetooth woes become moot. Doing this could also replace the KCB and RCB. This would be nice, but I am not ready to start all of the software over.

Update - I did buy and setup a 802.11b adapter, as mentioned in the parts list. But I have since moved on to using the Roboard solution.

The following chart shows the uart voltage, TTL logic, etc of each component on the robot. I found lots of conflicting information about the signals of the Kondo parts, so I checked with an oscilloscope and found the following: