Media Processing Framework built with dependence on C++11.. Design is based on GStreamer's documentation but does not follow it completely. The framework is pretty rough at the moment and you will likely need help getting something functional. Currently only handles live image sources, audio is in the works. Don't pull unless you are wanting something to work on.

###Platforms

• Windows only (for the moment)

##The Framework is split across several libraries.

• sdk/Base - provides basics for the plugin system, attribute system, serializer/un, logger, etc...
• sdk/Media - provides the media plugin system, basic media sample, plugin factory, clock, and OpenGl/OpenCL support
• sdk/MediaPipeline - provides pipeline to be used in the frame, currently only a single pipeline avilable
• sdk/QtComponents - area for Qt support items for the Framework
• sdk/Utilities - area for utilities, currently thread helpers, math, etc...
• sdk/AvStream - provides connections into Windows DirectShow API

##Framework also includes some applications and filter plugins

• applications/Conversion - (not functional)
• applications/DebugApp - app for testing
• applications/Sanctuary - basic loading of filters (may not be functional at the moment)
• filters/AudioControls - provides VU meter and Mixer (under development) (Qt based)
• filters/AudioViewer -
• filters/AvStreamOutput - uses AvStream to output from pipeline into DirectShow
• filters/ControlStructures - provides filters to branch, tee, join and synchronize parts of the pipeline
• filters/GStreamerPlugin - start of a filter to allow use of gstreamer plugins, (not functional)
• filters/ImageFilters - provides gpu base ColorConversion and Filters to move images to/from the gpu
• filters/ImageViewer - filter to show images (Qt based)
• filters/ffmpeg - filter using ffmpeg's libraries to read and write to file/streams

##Framework also includes some helper libraries and utility programs

• libraries/cvlib - contais ImageConcept, SimpleImage, slidingWindow function, binary match, and save as ppm
• libraries/medialib - contains spectrogram and audio synchronize using audio finger printing
• tools/Embed - through cmake takes source files and creates c files with the source as a string, mainly used for embedding opencl files

##Application Development Building an application requires creating a pipeline.

#include "MediaPipeline/MediaPipeline.h"

Limitless::SharedMediaPipeline pipeline;

pipeline.reset(new Limitless::MediaPipeline("Pipeline", Limitless::SharedMediaFilter()));

Then creating filters

#include "Media/MediaPluginFactory.h"

Limitless::SharedMediaFilter input=Limitless::MediaPluginFactory::create("FfmpegInput", "Input");
Limitless::SharedMediaFilter inputDecoder=Limitless::MediaPluginFactory::create("FfmpegDecoder", "Decoder");
Limitless::SharedMediaFilter inputViewer=Limitless::MediaPluginFactory::create("ImageViewer", "Viewer");

and then adding them to the pipeline and linking

pipeline->addMediaFilter(input);
pipeline->addMediaFilter(inputDecoder);
pipeline->addMediaFilter(inputViewer);

pipeline->link(input, inputDecoder);
pipeline->link(inputDecoder, inputViewer);

set any attributes required by the filter

input->setAttribute("location", "test.mp4");

then start the pipeline playing

pipeline->play();

There is a lot more functionality but for a simple video player this would be the extent. Later I will provide more detail in the wiki.

##Plugin Development It uses an automatic registering system through a little template voodoo, base class your filter with Limitless::MediaAutoRegister<YourClass, BaseClass> and provide your class as the first template argument and the base class you want to inherient from as the second argument.

#include "Media/MediaPluginFactory.h"
#include "Media/IMediaFilter.h"

class ImageViewer_EXPORT ImageViewer:public Limitless::MediaAutoRegister<ImageViewer, Limitless::IMediaFilter>
{}

as soon as the dll is loaded the class will auto register with the plugin factory. The other tidbits you need to define for your plugin is some traits.

namespace Limitless{namespace traits
{
	template<> struct type<ImageViewer>
	{
		static FilterType get()
		{return Sink;}
	};
	template<> struct category<ImageViewer>
	{
		static std::string get()
		{return "sink";}
	};
}}//Limitless::traits

This allows applications to look up your filter by type and category.

Past the above you need to provide some function overrides to communicate with the pipeline. The main ones being

• onAttributeChanged - notification that one of the plugins attributes was changed
• onAcceptMediaFormat - used to accept or reject upstream formats
• onLinkFormatChanged - notification of the format selected between this plugin and either upstream or down stream plugins
• onReady, onPlay, onPause - notification of state changes
• processSample - sample provided by pipeline for plugin to process

##Building the frame work Works Visual Studio 2015, requires CMake

You will need several libraries to build the frame work. However some of these are handled by a CMake package handler (Hunter) and will be downloaded/compiled when cmake is executed.

Required (manual install)

• Qt 5.X+ - link (todo: make optional)
• Ffmpeg libs - link (todo: make optional)

Required (package manager)

• Boost 1.5X+
• Glew
• Glm
• RapidJson
• opencl
• fftw (todo: make optional)
• dlib (todo: make optional)
• zlib (todo: make optional)
• libpng (todo: make optional)

There is a batch file in the root dir vs2015x64_rebuild.bat, inside it are variables that can be set to the directories for Qt and ffmpeg. Once you set them you can run the bat file which will use CMake to create the VS solution files. Running vs2015x64.bat will build the VS solution file an automatically start VS with that solution.