Skip to content

simonlmn/actracktive

Repository files navigation

Actracktive

README

Actracktive is a highly configurable real-time computer vision/image processing platform written in C++. It provides a framework and runtime environment for developing and executing complex processing graphs which can for example capture images from a camera, pre-process them using various image filters, extract features, track the detected objects of interest over time and finally send information about the objects to another application via network.

Actracktive originally started as a fork of the [CCV] (http://ccv.nuigroup.com/) project used for vision-based finger tracking in the multitouch/interactive tabletop systems. As a more modular and configurable solution was needed, the idea for Actracktive was born and a new architecture and implementation was built from scratch. It aims at providing a general and easily extensible framework for implementing most computer vision tasks (2D and 3D), but also other kinds of (frame-based) signal processing.

The key features are:

  • Freely configurable graphs (as opposed to linear pipelines) of processing nodes via simple XML description files

  • Generic graphical user interface for configuration of arbitrary settings on processing nodes

  • Non-interactive daemon mode, allowing execution of pre-configured graphs in the background with maximum performance

  • Flexible and extensible software framework to develop new and custom processing nodes which integrate well with existing nodes and the generic UI

  • Modern C++ implementation making use of the standard library as well as the boost library

  • Readily available processing nodes, including:

    • camera capture (via Firewire and generic USB)
    • camera calibration (distortion correction and rectification)
    • video playback
    • static image source (JPEG or PNG)
    • various image filters
    • detection temporal tracking of touches and fiducial markers
    • grid based object calibration
    • OSC/TUIO transmission

Using Actracktive

Note: If you use one of the pre-compiled binaries from the project's website, you might still have to install some additional libraries which are not included with Actracktive. Please see BUILDING.txt for further information.

Actracktive is designed to be used as a process running in the background, doing some processing task until it is stopped. However, to tweak some parameters on the fly, it also offers an interactive GUI that shows what is currently happening in the application and allows for adjustment of various parameters and settings.

When starting the application without any command line parameters and/or configuration file, it will by default start up in interactive GUI mode. It will try to load the configured processing graph config file (named processing-graph.xml by default) from the application's data directory. If a processing graph could be successfully loaded, it will immediately start processing.

Graphical User Interface

The interface is divided into four regions:

  • The global control panel at the top allows to control the application as a whole:

    • Quit the application
    • Start/Stop processing
    • Save the settings of all nodes to the configuration file
  • The list of all nodes in the graph at the left allows to view all nodes and adjust settings (if any). The settings of each node can be shown by clicking on the node's name (with the small box in front of it). Some nodes also have a graphical representation (mostly image source/filter nodes), which is shown above the nodes name as a small preview. Clicking on this preview will show this representation in a larger version.

  • The large node display area at the right/center allows to view a larger graphical representation of a node, if it has one.

  • The status bar at the bottom of the application window will show the current status of the application and periodically updated performance data:

    • At which rate frames are actually processed, measured in Hertz (Hz)
    • How long a single frame of execution took, measured in milliseconds (ms)
    • How much time was spent being idle, also measured in milliseconds (ms)

The UI does not offer any way to remove or add any nodes the graph. This has to be done in the graph's XML configuration file. For details see the section 'Configuration' below.

Daemon/Console Mode

For simply executing a given processing graph without any need for adjustments, Actracktive offers also an non-interactive mode, also called daemon mode or headless mode. To start into this mode, either a command line parameter can be passed in or it can be set via the global configuration file in the data directory. On the command line, headless mode can be enabled by using the --headless option, e.g.

$ ./Actracktive --headless

Note: On Mac OS X, the application is packaged as an application bundle, which cannot be directly called with command line parameters, thus you have to go into the directory Actracktive.app/Contents/MacOS where the application binary is located.

The application will then start in the same way as in interactive mode, except that it does not open any GUI windows. Depending on the logging configuration, log messages might be written to the console, including periodical performance data.

A complete list of command line options and their meaning can be obtained by passing in the --help option.

Configuration

Actracktive can be started without any additional configuration files, but at least a non-empty processing-graph.xml should be provided, as it would be quite useless otherwise. All configuration files, log files or any additional files needed for processing are searched and stored in the data directory. This is by default looked for (and created if not found) in the user's home directory, i.e. ~/Actracktive Data. Some example files that would go in this directory can be found in the data directory of the project's sources (see 'Project Structure' below).

There are generally three possible configuration files:

  • Global application configuration (config.xml by default)
  • Processing graph configuration (processing-graph.xml by default)
  • Logging configuration (log4cplus.properties by default)

Global Configuration

The global application configuration is a simple XML file which allows to set application wide options. The default contents would look like this:

<?xml version="1.0" ?>
<config>
    <headless>false</headless>
    <logging-config>log4cplus.properties</logging-config>
    <graph-config>processing-graph.xml</graph-config>
    <timer-output>5</timer-output>
</config>

headless controls if the application starts in headless mode by default. Note that on Mac OS X, setting this to true and starting Actracktive by double clicking the bundle will eventually start the application but will lead to a constantly bouncing app icon in the dock, as no GUI window is opened.

logging-config allows to specify a different configuration file for the logging system. Note that by default, Actracktive first looks for log4cplus.properties in the data directory and if no file is found it falls back to loading the default logging configuration file provided in the application's resource directory (this is packaged with the applications binary, on OS X within the application bundle, on Linux in the resources directory).

graph-config allows to specify a different processing graph configuration file.

timer-output controls how often performance data is logged in headless mode. It specifies the delay in seconds between two log messages. Setting this to zero or a negative value will disable performance data logging.

Processing Graph Configuration

Processing graph configurations are stored in XML files which are also described by a DTD found in the resources directory as well as online on the project's repository.

A processing graph is described as list of all processing nodes needed for a certain task. Each node is minimally described by its type, which is a unique string naming the implementation (usually the class name). To make connections in the graph, each node can be given a unique identifier. This identifier can be used in the configuration of other nodes to wire them together. Additionally, each node can have an arbitrary set of properties, which are defined by the implementing class and are used to control the behavior of the node. There is also a mechanism for configuring binary type properties, which are used to store non-standard data types such as images.

As an example, consider the following configuration file:

<?xml version="1.0" ?>
<!DOCTYPE processing-graph SYSTEM "http://actracktive.googlecode.com/svn/trunk/aux/resources/processing-graph.dtd">
<processing-graph>
    <node id="camera" type="DC1394Source" name="Camera">
        <property name="cameraId">49712223530884286:0</property>
        <property name="mode">MODE_640x480_MONO8</property>
    </node>
    <node id="shading" type="ShadingCorrectionFilter">
        <property name="enabled">true</property>
        <connection name="source">camera</connection>
        <blob name="shading"><![CDATA[QkJBQkFBQD9A...]]></blob>
    </node>
</processing-graph>

The first node is of type DC1394Source (which implements access to firewire cameras supporting the IIDC protocol using libdc1394). It is also given an ID and an arbitrary name, which is purely for displaying purposes. Two properties are configured, in this case the UUID of the camera and the 'mode', controlling image resolution and color channels. Note that implementations will have default values for properties, which allows to leave out those properties where you want to use the default. However, when the graph is saved back to file using the GUI, all properties, even those using default values, will be written to the file.

The second node is of type ShadingCorrectionFilter (which is an image filter for correcting shading differences in images). Again, a property is configured, in this case to make sure the filter is enabled. The next element connects the 'shading' node to the 'camera' node, i.e. the 'shading' node will get passed a pointer to the 'camera' node, thus allowing it to access the images captured by the 'camera' node. Note that connections are typed, i.e. it is not possible to connect arbitrary nodes to each other. The implementation of each node defines which type it accepts as a connection. In this case, the source connection of ShadingCorrectionFilter (actually any ImageFilter) accepts any node that is of type ImageSource.

Finally, a blob (binary large object) is configured. These always consist of a CDATA section containing base64 encoded data. In this case, it is a 640 by 480 pixels large greyscale image describing the shading of the scene. Storing this information in the processing graph's configuration file makes them self contained and easy to move around. Otherwise, if this kind of data was stored in external files, it becomes harder to always keep the files together.

There is no limit to how many properties, connections or blobs a node can have. Also, there is no limit to how many nodes might be connected to some other node. In this example, you could easily add several copies of the 'shading' node (giving each its own, unique ID or just leaving out the ID attribute). It is also not an error to try to configure undefined properties, they will simply be ignored.

Logging Configuration

Actracktive uses the [log4cplus] (http://log4cplus.sourceforge.net/) logging system, which is based on the popular log4j system widely used in the Java world. The configuration is therefore using properties files. For a detailed description how to configure the logging system, see the documentation on the log4cplus and log4j projects website.

Building Actracktive

The build process is using the Eclipse CDT's internal builder. For detailed build instructions, see BUILDING.txt.

Project Structure

The projects files are organized as follows:

aux contains binary or other resources used by the application at runtime. Some of them are typically copied along with the binary output in the post-build scripts.

data contains various data files, mostly examples of processing graphs or configuration files, which are not required to build or run the application.

libs contains all third-party libraries which get statically linked into the application. The libraries are included as precompiled binaries for every target platform together with the required public header files.

src contains all the source code of the application.

Actracktive uses the Eclipse CDT internal builder at the moment. The Eclipse project files with all the build configurations and settings are included in the root directory. There are also some shell scripts used by the build process, namely pre-build-osx.sh, pre-build-linux.sh, post-build-osx.sh, post-build-linux.sh and version.sh.

After building the application, the final application binary or bundle (on OSX) will be placed inside the bin directory, which is created automatically.

License

Copyright (C) 2012 Simon Lehmann simon.lmn@gmail.com

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

Third Party Components

Actracktive incorporates the following statically linked libraries:

  • tinyxml by Lee Thomason
  • oscpack by Ross Bencina
  • fidtrack by Ross Bencina and Martin Kaltenbrunner

The libraries are included as pre-compiled static libraries together with the public interface header files required for compiling against those.

About

A highly configurable real-time computer vision/image processing platform written in C++

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published