Alright you just found einstein, einstein is an intergrated library for implmenting deep learning algorithms for control algorithms.
pip install hg+http://bitbucket.org/pygame/pygame
pip install theano
pip install pybrain
pip install matplotlib
pip install git+https://github.com/craffel/nntools.git
pip install pyzmq
conda install -c https://conda.anaconda.org/tlatorre pygame