At least the opengl-hq VO allocates additional resources when downscaling a lot, which is just a waste. Also see #1547 (although I doubt that this is the cause; if it is, a real fix will be required).