Visual Weight

To understand points of interest, first, we must understand "visual weight". Visual weight is a term used by photographers to describe parts of an image that eye is drawn to when looking at the image. Unlike points of interest, it is independent from narrative significance.

Photographers use this concept to create "balanced" images, where there is a "center of gravity" in the image which is balanced - considering the visual weight of the elements of the images.

I prefer to think of visual weight is not as a literal measurement, but as a model for thinking about how an image is composed. "Why does this thing draw attention to itself?" Do I like that? What can I do about it?

Things have more visual weight tend to be:

  • In focus
  • Big
  • Colorful/Saturated
  • Bright
  • Darker against lighter background (or vise versa).
  • Closer to the camera (foreground elements are more important than background elements)
  • Contrasty
  • On a path of leading lines (such as the center of 1 point perspective)
  • Moving, or suggesting motion
  • Eyes are looking at it (in the frame)
  • On one of the thirds
  • Breaking a pattern (visual or otherwise)

Points Of Interest

Points of Interest (POI's) are initially determined by visual weight, but they are much more than that. Not just "high level" attributes of visual interest, something boring or static can become a POI - if the user is motivated to look at it. Far more attention-getting than an object merely festooned with visual weight is an object that holds an emotional significance for the user - or a narrative significance. A door may be visually boring, but if something on the other side is trying to break out - if all of the characters are wide-eyed, scared, and backing away - unable to break eye contact with that door - guns held high under whispered breath... that door is very likely to hold our gaze.

In short, a point of interest is any point in a frame that a user may be likely to look at, for whatever reason. Whether or not we know why the viewer looks there, or if we even intended the viewer to look there, a POI is still a POI.

Considering abstract static images, it's the same as looking at what has the most visual weight. Which is helpful, except we aren't dealing with abstract static images. There are far too many attributes that influence what makes or doesn't make a POI to list them all, but here are a few so you get the idea:

  • People. Things with eyes
  • Things that are speaking
  • Things that are moving, or one thing moving towards the camera, or anything breaking a pattern of motion (i.e. static person in a sea of commuters)
  • Things that we empathize with/care about (i.e. the main character)
  • Things that intrigue us or make us curious. (i.e. A locked toy chest that a child seems deathly afraid of what's inside, while a parent insists everything is fine)
  • Things that are about to change or reveal something
  • The direction the camera is moving (Even as passengers, we don't sit in cars facing sideways; we like to look where we are going)
  • Things new to the scene
  • Things that are making noise
  • Things that are about to do something (anticipation; a batter wound up and ready to swing, a bomb with a ticking noise)
  • Things that actors are looking at or paying attention to, or have some kind of established relationship with
  • Things that actors aren't paying attention to (but should be/still have some kind of relationship. Like a horror movie villain creeping around a corner)
  • Things that try to hide from us (the viewer can "accepts a challenge" and go hunting for the thing)
  • Things with giant flashing arrows pointed at it

Areas Of Interest

A viewer may not always be watching a scene unfold as they follow POIs. They may be switching between multiple POI's they are trying to keep track of, or searching an area for something. "I wonder what's over here?" so to speak. When the viewer does that, we can consider the area that they are scanning to be an "area of interest". For example, if we put a camera in the stands of a baseball game, the field may be a point of interest, while the nearby stands - people in them, beer/hot dog vendors, someone with a cowbell, and everything else going on (and making the noise the viewer hears) would be an area of interest. If one of those things takes up a lot more interest- say, a hot dog vendor throwing a hot dog past the camera to a fan - then we would consider that to be a new point of interest.

Knowing when you are dealing with points and areas, and when the viewer is motivated to look at these different points and areas, is very important.

We can also create areas of interest by removing any point of interest. If there is nothing that stands out for the viewer to look at, then they will start scanning the scene, looking for something to discover. My instinct is that this is not an ideal way to motivate the user to explore the environment, but perhaps a shot like this - all environment, no one thing of importance - is a good tool to establish mood, setting, or atmosphere. An establishing shot, in other words.

Big Flashing Arrows

If you are filming a shot in front of a school, right as the last bell is rung and hundreds of children leave - how do you make sure the viewer is able to find our character. Well, let's put her in a brighter coat than all the other kids, and have her walk a little slower - let the other kids be blurred out as they run faster to leave the school. Have those other kids run off to the left or the right of the frame. Let's track focus on the girl, so she remains in focus, and play some audio of the girl humming, discourage sounds of other children speaking (outside of a noisy murmur that sells the scene) and a mother going "Charlotte, over here!" - then, on that cue, let's see this girl turn her head and look up - right at the camera, and then start walking right towards it. All of this consideration just for an intro shot of a Mom picking a girl up from school.

Basically, creating POI's is not about putting up some giant flashing arrows, it's about putting up a thousand giant flashing arrows - except any individual arrow isn't really all that giant, or flashing. If it feels too obvious what you want the viewer to look at, then it feels like you are forcing the viewer. Viewers don't like to be forced. This is especially so in spherical video, where the viewer has a choice to make on where to look. So, the strategy is the same one of films that achieve more subtle (think, an actual movie vs. a scene from Dora the Explorer). Instead of big obvious "arrows", use lots and lots of small and subtle arrows. This way, the user can't quite place their finger on what is motivating them, and they don't think about the what that is motivating them, but rather, on the thing, the point of interest.

Strategies For Identifying Points Of Interest

Well, there are 2 ways to identify points of interest. The first is to, well, look at your scene. Watch it. Think about it. Combine intuition with those above "arrows", and imagine where a user will likely look. This isn't all that hard, filmmakers have been doing it since, well, ever. Yet, it is uniquely difficult when editing 360 video in a rectangular projection, and not in an actual VR environment. Also tainted by what all editors are tainted by - the editors own head.

This is fine, and really there is no other way to get started. Yet, it can't be the only method employed when making a film. Spherical video is arguably an interactive system. Open loop, but interactive. The viewer has decisions to make. They look behind them, we show them what's behind them. Like any such interactive system, that means playtesting.

Playtest

Watch a fresh (unfamiliar with the content or footage) viewer watch your scene, and observe where they choose to look. Don't say anything while they are watching the film, and don't tell them you are going to be paying attention to where they look. Just ask them to check out something you are working on. Afterward, ask them non-leading questions about the objects they were interested in, or why they think they looked where they looked. Ask them about which character they cared about, or what they thought was going to happen during the film. Playtesting is so incredibly important - and not a tool that traditional filmmakers have in their playbooks.

Playtesting implies writing towards a generic, averaged audience. This is not ideal. We don't want to make more Marvel movies with unmemorable soundtracks; we want to create something unique and awesome. Yet, playtesting is still going to provide the director or editor with a measurable tool. It's an easy and concrete way to resolve disagreements when editing, and there is no better way to see how a film will do than to put it in the hands of viewers and see how the film does.

Playtestingworkshops.com has plenty of great and applicable information about playtesting.