Video can be good if you stick to a garden path of simple scenes with tons of examples in the training material and not a ton of motion between overlapping objects in a scene, and don't really care too much about specifics.

As soon as you want something very specific, or something novel, or anything with a lot of moving objects/people, it falls apart.