Wednesday, July 14, 2010

Some Blender Sequence Editor Notes

Currently playing around with the non-linear video editor in Blender, aka the Sequence Editor, as I ran into a bunch of trouble, here some notes (still a work in progress, so some stuff in here might not actually be true).

Convert your videos
The sequence editor works on frames, not time, this means when you simply import a video that has a framerate different from your target framerate, the audio will go out of sync, as blender will handle each frame in the input as one frame and ignore the duration it has on screen. Furthermore the audio will also go out of sync when your video contains duplicate frames, but doesn't actually store the frames in the container, this happens for example when you slowed down a video with mencoder and -speed 0.5, thus every second frame is duplicate and won't be stored or when you used mencoder mf://.. to generate your video. To fix this you have to convert the video with the harddup option:

mencoder input.avi -o output.avi -ofps 30 -vf harddup -ovc lavc -lavcopts vcodec=ljpeg -oac copy

So you basically have decide on the resolution and framerate of your output before you start working and then convert all the material you want to import accordingly. Not sure which codec is best, ffv1 seems to lead to extremely slow seek, ljpeg seems to work so far.

While you can mirror, flip, crop and translate a video easily in the sequence editor, you can't scale it easily, so you might want to do that outside of blender as well.

When you want to cut your videos into chunks beforehand, note that you can't use mencoders -ss and -endpos to cut your videos, as mencoder will cut by keyframes, not exact time, so you are also a random amount of from where you placed the cut, use avidemux instead and convert to ffv1 if needed, as avidemux won't let you cut h.264 video easily either.

Convert your Sound
Blender can import audio directly from video, but that is broken in Blender 2.49b, if you look at the wave forms you will see that Blender cuts out a 0.5sec piece at the start and adds garbage to the end. I haven't tried all possible formats and codecs, but so far I haven't found one that works. Solution is to separate audio and video each into its own file. This can be done with:

mplayer -vc dummy -ao pcm:file=out.wav:fast input.wav

and then add it with "Audio (HD)".

Input Codec Pro/Contra
  • ffv1 seems to lead to extremely slow seek, so it is unusable for practical use
  • ljpeg is more then twice the size as ffv1, but seems to allow fast seeks and encodes 20% faster then ffv1
  • raw is fasted to encode (third of the time ljpeg takes), but four times bigger then ljpeg, raw is also a little faster to seek then ljpeg
  • jpeg image sequence seems to work
Best Encoding
  • raw is fast to encode and fast to use in Blender
  • raw is lossless
  • format=yv12 is needed sometimes because videos might be in a colorspace that blender does not understand
  • harddup to make sure that the video file has a frame for each frame and doesn't ignore duplicates which confuses Blender
mencoder -ovc raw -vf format=yv12:harddup -oac pcm input.avi -o output.avi

Don't forget to...
  • click "Do Sequence" to actually render your videos via the animate button
  • set the output to a video format or else you will just get a bunch of jpegs in your /tmp/ directory
Important keys
  • space - lets you add a strip
  • c - lets you change the type of strip later on
  • m - lets you group strips into a meta strip
  • shift-k soft cuts the selected strip (soft cuts contain the full video, hard cuts only the necessary frames)
  • ctrl-leftclick - lets you place a node in the curve editor
Blue/Green Screen
Effects like color-keying, aka blue/green screen, can be done in node editor, but not in the sequence editor. Not sure if there is any way to use both of those together in a single session or if one has to save the results of the node editor first and then import them as normal avi back into the sequence editor.

Notes about Sound
Using "Audio (RAM)" gives you a nice wave form in the sequence editor, using "Audio (HD)" just gives you a block. Haven't yet tested if that relates to out of sync issues.

Getting Sound to Work
No idea how to get sound to work, all I get is:

Couldn't open audio: No available audio device

If I load a sound block, I can play it, but the sequence editor stays silent, even with scrubbing enabled.

After a serious of murder attempts at pulseaudio and using play 0001_0250.wav before starting blender, sound now works, but that is smudgy and non-reproducible. Using blender -g noaudio might also help in some situations.

Problems with Workarounds/Fixes
  • there doesn't seem to be a way to change the length of an effect-strip, the length is automatically determined from the overlap from the two child strips, splitting the strip is usable as workaround, but adds some complications
  • you can stack normal strips and set their blend mode, but you can't do that with effect strips, their blend mode seems to be limited to "replace", grouping them as meta-strip however gives you all the other blendmodes back
  • the Blender 2.49b that ships with Ubuntu 10.04 is broken, all videos except image sequences come out in very low image quality, looks like the pixels get ordered incorrectly, the official Blender 2.49b release seems to work fine
  • Even with all the conversion done and the movie and sound showing up in Blender with the same length, there is still a audio delay in the end, this seems like a bug, splitting the wav data and importing is separately from the video helps
  • alpha blending seems broken, placing a RGBA image over a sequence sort of works, but only with borked alpha, enabling SequencerButtons/Filter/Premult seems to fix it
  • you can't make meta-strip out of "Audio (RAM)" and video, but you can do a meta strip out of "Audio (HD)" and video, so its probably best to avoid "Audio (RAM)"
Problems
  • marking a strips file location as "Relative" doesn't seem to work
  • What is the best way to handle RGBA movie sequences? PNG files?
  • Is there a way to set the output filename to a fixed one? It is kind of annoying to have the filename switch when the number of frames change
  • Blender is suppossed to be able to play sound, but doesn't play any, even when sound is enabled and "scrub" is activated for the wav, seems to be a issue with pulseaudio, yet again
  • something as simple as a text overlay seems to be quite complicated to do in Blender, as there aren't really any proper 2D editing capabilities in the sequence editor
  • building a proxy for a strip doesn't seem to have any effect, the jpegs for each frame are generated fine, but doesn't seem to speed anything up
  • there seems to be a 2GB limit for AVI RAW files, making them mostly useless

No comments: