April 2, 2015

What we are working on: Variable High Frame Rates

Written by

We are currently doing experiments on the use of Variable Frame Rates (VFRs) consisting of both High Frames Rates (HFRs) and standard frame rates (as 24fps). As part of our evaluation of HFR and VFR content, it is necessary to know how we can direct the viewers gaze using intra-scene VFR (and if this is possible), through tracking where the viewers gaze pattern flows on a screen with both standard frame rate, HFR, and intra-scene VFR content in order to compare differences.

Intra-scene describes the use of multiple techniques within the same scene (and possibly also the frame) . In this case, the use of frame rates.

Why might this be relevant? 

With “Soul Mates 3D”, we experimented with changing the frame rate from scene to scene. With intra-scene VFR, we propose changing the frame rate within the very frame itself; for example, a foreground of a horse running past a camera may be filmed/animated at 120 fps, and the background consisting of a stadium crowd may be filmed/animated at 24fps. The two scenes are composited together within the same timebase (let’s use 60fps as a choice), and we may appreciate the lack of motion artifacts within the foreground (horse) while enjoying the familiar 24fps aesthetic of the background visuals. The two elements within the scene will provide completely different motion information, and we as viewers would have a choice of where to look within the frame.

In cinema, television and gaming, viewers primarily focus their attention on an area of a moving image that has the most visual information. For example:

  • depth of field/focus (by means of eliminating information in the area that is not to be the focus through blur, we then focus our attention on the area of the frame that is sharp)
  • convergence (in stereoscopic 3D this used to target the viewer’s attention onto a chosen area of the frame by changing the 3D plane of the moving image)
  • composition and camera movement (using image obstruction and reveal, called occlusion)
  • And moving subjects or objects.

We selected a face as the image we would experiment with, because from birth faces are important in our social interaction and how we communicate effectively throughout our lives with one another. We look at faces daily, and we draw a vast amount of information from looking at faces.

We will update the project page as we generate data and undertake this research.