Lab: MapBuilder and Shapes

From Tekkotsu Wiki

Jump to: navigation, search

In this lab you will learn how to invoke the MapBuilder to do vision for you. For this exercise you will be running Tekkotsu in "simulator mode" using a pre-stored camera image called sample-image.png instead of the robot's webcam. In fact, you don't even need a robot at all for this lab; you can install Tekkotsu on your laptop and run it there.


Examining the Sample Image

1. Start Tekkotsu in simulator mode by using the sim.plist configuration file:

cd ~/project
./tekkotsu-CREATE -c sim.plist

In simulator mode, Tekkotsu loads logged data (images and/or sensor values) from the ~/project/logged directory instead of from a live robot.

2. In the ControllerGUI, click on the RawCam button to see the sample image that comes pre-installed in your project/logged directory. Click on RawCam again to deactivate the RawCam viewer.

3. Click on the SegCam button to see the color-segmented version of the image. This is what the MapBuilder will use to find shapes in the image. Click on SegCam again to deactivate the SegCam viewer.

The SeeShapes Demo

1. In the ControllerGUI, go to Root Control > Framework Demos > Vision Demos > SeeShapes.

2. When the SeeShapes demo runs, it will invoke the MapBuilder and then tell you what shapes it found in the image. Were you surprised by the result? How many shapes did you expect it to find?

The SketchGUI

The SketchGUI is a tool for examining the sketches and shapes that make up Tekkotsu's "dual coding" vision system.

  1. With the SeeShapes behavior still active, click on the "C" button in the Sketch row of the ControllerGUI. This calls up a SketchGUI to examine camera space; the window is labeled camGUI.
  2. In the camGUI window click on the rawY sketch to overlay the shapes on an intensity (Y-channel) version of the camera image.
  3. Control-click on the camFrame sketch (i.e., hold down the Control key while clicking on camFrame) to also overlay the color segmented version of the camera image.
  4. Deselect raw-Y by control-clicking on it. The camFrame sketch should still be selected, but now you can't see any of the shapes.
  5. Click on the Invert All Shapes checkbox in the camGUI window; now the shapes are visible again.
  6. In the "cam view" window, click on the "ID" checkbox to display shape ID numbers. Notice that these match the numbers in the camGUI window.
  7. Deselect the camFrame sketch and turn off Invert all shapes.
  8. In the camGUI window, uncheck the Select All Shapes box. Then click on individual line and ellipse shapes.
  9. Use control-click to toggle the selection status of individual shapes without affecting other selections.
  10. Use shift-click to select all items within a range, e.g., click on the first item you want to select, then shift-click on the last item.

Invoking the MapBuilder

Here is some sample code that invokes the MapBuilder to look for blue blobs. You can cut and paste this code into the file ~/project/

#include "Behaviors/StateMachine.h"

$nodeclass MapBuilderExample1 : VisualRoutinesStateNode {

  $nodeclass BuildMap : MapBuilderNode : constructor {

  $nodeclass Reporter : VisualRoutinesStateNode : doStart {
    cout << "There were " << camShS.allShapes().size() << " shapes in the image." << endl;

    BuildMap =MAP=> Reporter


  1. Run this behavior on the sample image, and examine the results in the SketchGUI.
  2. Where is addObjectColor defined? Go to the main page of the online Tekkotsu reference, select the DualCoding name space, and then type addObjectColor in the search box in the upper left to find it.
  3. Modify the definition of BuildMap to also look for green blobs. What blobs do you find?
  4. Use the addMinBlobArea method of MapBuilderRequest to require green blobs to have at least 50 pixels. How does this affect your results?

Running On Live Images

  1. Run the SeeShapes demo on the robot without the -c switch so you can use its webcam for images instead of the sample-image.png file. (If you don't have a robot, plug a webcam into your laptop or workstation and run Tekkotsu with "-c webcam.plist".)
  2. Make some lines from colored masking tape, or lay out some plastic easter egg halves or jar lids, and try to get SeeShapes to recognize line and ellipse shapes.

The World Map

  1. Read the tutorial section or class lecture notes on local and world maps and the differences between camera space and local (egocentric, or body-centered) space and world (allocentric) space.
  2. Run the SeeShapes demo on the sample image again.
  3. This time, click on the "W" button in the Teleop window instead of the "C" button. What do you see?
  4. Uncheck the Select All Shapes box in the localGUI window and select just the two ellipse shapes with the smallest center X coordinate (the first number in parentheses is the X coordinate), using control-Click. Also control-click on the Agent shape to select that. Then click on the Rescale the Image button in the localGUI window.
  5. The robot is represented as the blue triangle at the bottom of the image. The sizes and distances of the ellipse shapes are distorted because the robot's assumptions about the position and tilt of the camera are incorrect, so the coordinate transformation from camera space to local space is off. You will learn how to correct this later.