When the resulting frame is already in a GPU texture then, displaying it has fairly low overhead.
My question is: how wrong am I?
Motion vectors can be large (for example, 256 pixels for VP8), so you wouldn't get much extra parallelism by decoding multiple frames together.
However, even if the worst-case performance is bad, you might see good performance in the average case. For example, you might be able to decode all of a frame's inter blocks in parallel, and that might unlock better parallel processing for intra blocks. It looks like deblocking might be highly parallel. VP9, H.265 and AV1 can optionally split each frame into independently-coded tiles, although I don't know how common that is in practice.
How do you imagine this would interact with try-finally being used to clean up resources, release locks, close files, and so forth?
For this reason, try-finally is at best a tool for enforcing local invariants in your code. When a function like process.exit() completely retires the current JavaScript environment, there’s no harm in skipping `finally` blocks.