VFW encoder for AMD VCE h264 encoder. Usable with Virtualdub, Dxtory etc.

https://github.com/jackun/openencodevfw/archive/master.zip

Extra settings are saved to registry under HKCU\Software\OpenEncodeVFW

You may need to install MSVC++ 2013 runtimes. Last MSVC++2010 commit.

NOTE: You need to install x86 version for 32bit codec even if your Windows is 64 bit.

AMD's GCN based cards and APUs. From AMD's blog:

• Unpack the archive somewhere, right click on install.bat and click Run as Administrator.

If it complains about missing files, try the more manual version:

• Unpack the archive somewhere, open command prompt as administrator by typing cmd to start menu or "Metro" and press SHIFT+CTRL+Enter or right click on the icon and click Run as Administrator.
• Go to unpacked folder by typing into opened command prompt cd some\where\OpenEncodeVFW-bin.
• Type install.bat and press enter to run the installer.

If uninstaller fails its job, manually remove these registry keys:

HKLM\SYSTEM\CurrentControlSet\Control\MediaResources\icm\VIDC.H264
HKLM\Software\Microsoft\Windows NT\CurrentVersion\drivers.desc\OPENENCODEVFW.DLL
HKLM\Software\Microsoft\Windows NT\CurrentVersion\Drivers32\VIDC.H264
HKLM\Software\Wow6432Node\Microsoft\Windows NT\CurrentVersion\drivers.desc\OPENENCODEVFW.DLL
HKLM\Software\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Drivers32\VIDC.H264

and OPENENCODEVFW.DLL in %WINDIR%\syswow64 or %WINDIR%\system32

• Fixed QP basically keeps picture quality constant across all frames.
• CBR keeps constant bitrate so picture quality gets worse if there is frequently fast motion in video and bitrate is too low or wastes harddrive space if frame could have been compressed more. Seems to fluctuate too much though.
• VBR uses variable bitrate, tries to keep in target bitrate but rises bitrate a little bit if needed or lowers if frame can be compressed more.
• CABAC is more efficient and resource intensive encoding option.
• Search range is motion vector range. Specifies how wide the codec looks for moved pixels so it can just say that these pixels moved to x,y and just save that. Higher (max 36?) is better and more resource intensive encoding option.
• Profiles / levels: start from http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles . Colorspace is limited to Y'UV420.

Probably not very accurate descriptions :P

Also:

• Send FPS sets encoder framerate properties to video framerate, but not all framerates are supported by encoder. Untick to treat all videos as having 30 fps, but this may make encoding inefficient and increase bitrate more than necessary.
• Speedy Math tries to speed up OpenCL floating point math by making it less accurate, but should be good enough.
• Switch byte order : for the rare case when input bitmap is RGB(A) instead of BGR(A).
• Header insertion : adds SPS/PPS to every frame, may make cutting/splitting video easier. More of a 'debug' feature.

Quickset buttons for speed vs quality:

• Speed : encodes 1080p at 60+ fps (theoretical max 80+)
• Balanced : encodes 1080p at 40+ fps
• Quality : encodes 1080p at 30+ fps (can probably do 720p@60)

With newer AMD cards (hawaii+), seem to support B-frames, though VCE may not actually generate B-frames with OpenVideo, and AVI kinda sucks with these (see). You may need to remux to MKV/MP4 for better audio/video sync. (Also maybe ffmpeg -fflags +genpts)