SDI Blog from 2000 to 2019

Popcorn in action - a dozen different demos

As explored in part one, Popcorn.js and timed metadata are ways to bring the rich interactive and interconnected web into video and audio. As audiences increasingly 'dual-screen' – surf the web and social networks while watching TV or a video in the background – connecting this extra information to a film in a way that is coherent with the look and feel of the film, has lots of potential.

Some Popcorn creations keep the data away from the video, while others add so much data that to some viewers they could be seen as distracting from the documentary. Some go further still and consider the entire web browser as the documentary canvas, and the video elements just one part of that beside any other form of media. Like any new tool, there’s a wealth of possibilities with it, and the most interesting thing about watching the framework’s development over the last few years has been the depth of ingenuity people have applied when mixing audio and video with the potential of the HTML, javascript and the many open web services out there. Here's selection of a wide variety of use-cases.

1. On-screen notations

Inspired by the pop-up info panels that would appear on some music TV shows, pop-ups let you provide text-based commentary during a video and can be styled and coloured as needed – and placed anywhere on screen for as long as desired. Because these aren't encoded into the video, they can be indexed by search engines, translated by screen readers – and changed and added to during the life of the video.

Example: Reblious Pixel’s Buffy vs Edward remix with pop-ups

2. Rich on-screen captions and graphics

By making title cards and on-screen captions HTML rather than rendered in-video, they can be made interactive by linking to a website, streaming recent tweets, being translatable through on-screen translation tools and indexable by search-engines. In this example for Ryerson University, although they are styled to look like they are part of the video, the slugs for each interview are HTML and feature a set of links – to the person’s website, their university department, their books and so forth. This method also lets you update mistakes after the film – or change the job title, surname, or references as the film ages.

Example: SSH500 peer learning experience

3. Transcripts linked to video

Linking video or audio with textual transcripts offers much potential, especially in teaching languages and literature. In the example below, a reading from Macbeth is linked with the related text, with a glossary for challenging words. In Happy Worm’s Hyperaudio and hyper video project (see above) the transcript goes one step further and is linked precisely to the timecode of the related file, letting you click on text on screen to get the video or audio to jump to that particular place. In this example, created as part of the Mozilla-Knight News Fellowship, oMark Boas worked with Al Jazeera to integrate hyper-audio with US election coverage, so you could share a section of video by selecting that section of text in the transcript. One very time-saving use of HyperAudio would be in editing documentaries where huge transcripts of interviews in the the rushes/dailies, could be quickly searched and a rough cut assembled by copying and pasting the text, then imported into an editor

Example: Al Jazeera

4. Interactive crowd-driven narratives

There has been much discussion in the documentary community about crowd-driven stories which pull in video from the audience - with films such as Glastonbury, Life in a Day, 10.10.10 and the forthcoming We Are Northern Lights all built from user-submitted footage. 18 Days in Egypt is an attempt to pool the vast quantity of user-generated media around the uprising in Egypt into a single crowd-built documentary. The filmmakers worked with developer Brian Chirls (@bchirls) experimented with Popcorn in two ways: firstly to add multiple videos of a single event into one space, alongside annotations and further information coming from Twitter at that time.

Example: code.chirls.com/buttercamp

In the second demo, a speech by Mubarak is used to invite people to submit their videos, while creating a stream of further viewer-submitted info and videos alongside the video

Example: code.chirls.com/18days

5. Visuals made up from live Tweets

By pulling in recent data or graphics from the wider web, a Popcorn ‘video’ can render a different viewer experience every time it is watched. This can be particularly powerful with audio where there is no music video or related information. In this example, the music video for Know Your Exit is generated through a number of tweets it can find that use lyrics from the song - which works well for a song about the sense of isolation in the modern world.

Example: Know Your Exit

6. Visuals made up from Flickr photos

7. Lecture recording with accompanying slides

Archived videos of lectures, classes, conference speeches and interviews can be enhanced by connecting and syncing slide presentations with the video itself, and comments or links to resources. The idea has been around for a while, Popcorn is one of the first non-Flash based solutions to do this.

Example: Slideshare

8. Sentiment analysis of dialogue (and response)

Buiilt around the Sentiwordnet language analysis framework, the demo below analyses the content of a speech by newly re-elected President Obama, and plots the negative and positive words on a graph. The applications for this could be interesting when applied to the live tweets accompanying a TV broadcast as it would demonstrate real-time audience feedback – interesting for both analysis and metrics.

Example: Victory Speech Sentiment Tracker  

9. Scrolling references

For filmmakers who don’t want extra data to interfere with the audience’s experience of watching their film – Swedish filmmaker Simon Klose worked with Popcorn hackers at the Mozilla Festival 2011 to create an elegant hidden sidebar where references can appear. After joining Mozilla's WebFwd programme, the director of the anticipated Pirate Bay documentary developed the system into a start-up, LinkLib, launched at this year's MozFest. This provides extra information about everything on screen (who is speaking, Wiki references, and so forth) in a ‘waterfall’ of data that can be hidden or shown to side of the video, not interfering with playback. If you look at the demo below you can see how the right column can be shown or hidden. In a previous version, when the mouse rolls over an annotation, it paused the video while clicking on any of the references would jump the video to that point, making it a method to navigate around the video. LinkLib also lets you, by scanning the QR code, get the extra data through the browser of your smart phone or tablet, letting you have a data-free 'big screen' experience.

Example: Pirate Bay Trial 

10. Remixable stories

Online meme-sharing seems to happen most quickly when a meme can be easily adapted and remixed – be it photos of cats with ever funnier captions or Downfall's Hitler in fury at the latest controversy. For digital natives growing up in a wired world, remixability will probably be assumed with anything they intend to share – from trailers and posters to campaign and activist videos, and features. Of course, it will better suit some projects than others. In the following Popcorn video (produced at MozFest 2012), the video gains more meaning when you make it your own, and share it amongst friends. There's more on memes and Popcorn here. 

Example: popcorn.webmadecontent.org/yh

11. Travelog and location based media

Video and audio can pull in live map references from Google and Open Map, and even street-view level tours. In this example, created by young members of the Bay Area Video Coalition, David Hilliard, former chief of staff for the Black Panthers, gives an audio tour of key parts of Oakland, accompanied with views of the street and annotations. By using Soundcloud, people can comment on the film throughout.

Example: History in these Streets  

12. Rich data to support archivists and researchers

By adding extended metadata to parts of a video – map references, wiki references, transcriptions and so on – the process of cataloguing and finding relevant video is made much easier, as researchers can find specific parts of videos of relevance and connect lots of related resources to a specific timecode. Some tools, like this prototype from Open Images illustrate how much extra info can be added to video – while also illustrating how much more useful it is to have this data outside of the video and not using Flash, because although it is written in Dutch, it is possible to re-render the whole page into another language with Google Translate.

Example: Open Beelden prototype