Making DonahueApril 26, 2011 | Permalink
by Garrett Kalleberg, Lead Architect & Developer, Arc90
What Donahue Is, What It Does
Over the past decade, Donahue’s product leads Tim Meaney & Chris Fahey, watched audiences at conferences turn away from the presenters and increasingly inward, or downward, towards their laptops, phones, and tablets. Their insight was that this apparent disengagement was actually a detached kind of engagement—people were interacting with the substance of the presentation, but this interaction was directed out of the presenter-audience loop and into a network of other connectivities. Their idea was to develop an application that rewired that engagement back into the experience of the presentation, transforming detachment into dialogue.
Donahue, a collaboration between Behavior Design and Arc90, is all about dialogue. During the building of our application, Tim mentioned to me one day that he came to a turning point when he had the epiphany that technology and publishing were both moving towards conversation. As he writes in the Presentation Toss the Projector: Redefining the Presenter-Audience Dynamic: “We are at the beginning of the conversation revolution”. Donahue, an application that premiered at the SXSW conference in Austin on March 13, 2011 with this Presentation, is part of the revolution.
Donahue is an application that provides tools for assembling a Presentation, made of slide-like artifacts called Points, and presenting these Points in real time in 2 different views: 1) a Projector view, which represents the Points in their clearest form on a screen, and 2) a Participant view, which represents the Points along with a streaming “backchannel” area. Donahue uses Twitter to engage participants in retweeting Points, replying to Points that are captured as tweets, and create new tweets using the Presentation's hashtag, and these flow into the backchannel in real time during the course of the Presentation.
What follows is a technical description of our project. I'll walk through the primary concepts and objects of the application, then describe its architecture, then talk about building and presenting a Presentation finally walking through the process of publishing a Point.
Concepts and Objects of Donahue
Donahue is a tool for building and presenting Presentations (and archiving the Presentation dialogue to be replayed at any later date). Each Presentation consists of any number of Points, a Point being an artifact similar to a Keynote or Powerpoint slide, combining text and an optional image, with the following constriction: the text can be no longer that 140 characters (including the Presentation hashtag and a shortened URL for the image). The character constraint of course comes from Twitter, because each Point is posted as a tweet the moment it is published during the Presentation. These tweets are posted in association with the Twitter account of the particular Presenter (a Presentation can theoretically have any number of Presenters), and then captured via the Twitter streaming API and reincorporated in the Presentation itself and represented in the backchannel area. Being a tweet, any user logged in via their Twitter account—thereby considered a de facto Participant in the Presentation—can retweet or reply to any other tweet in the backchannel, or create a new tweet using the Presentation's hashtag.
So Donahue considers two main types of users: Presenters and Participants. But Donahue is not meant to be an exclusive environment: in addition to Presenters and Participants, anyone can view the entire dialogue, without logging into Donahue via a Twitter account.
Before moving on to discuss the application architecture, I'll describe the four Donahue application user interfaces or views, beautifully designed by Behavior Design and then built by my partner on this project, Tyler Gaw. I mentioned two different views for viewing the Presentation, what we call 1) the Projector view (image and text of a Point), and 2) the Participant view (the image and text of a Point, along with a backchannel area of tweets and tweeting tools). In addition to these are 3) the Presenter view—the interface that the Presenter uses to move along the Presentation; and 4) the Builder view—this is the interface for creating and editing a Presentation, and adding, removing, and editing Points.
Architecture and Technology of Donahue
The primary components of Donahue are:
- A pair of shell processes to handle Twitter stream consumption and tell the application to do things with the incoming data as needed; and
- A cluster of Node servers managing data between the application and browser clients via Socket.IO.
1. Donahue application
The back-end Donahue application is a fairly light application written in PHP in MVC style with fat models and minimal controller functionality and an autonomous presentation layer. The models handle persistence, data manipulation, and basic application event behavior, such as logging in via a Twitter account, setting an application to be "open" or "closed", and using the Twitter REST API to post Twitter account status updates with the Presentation hashtag.
2. Stream consumption processes
Stream consumption is handled by two interrelated PHP scripts. The stream consumption script itself captures tweets from the Twitter stream (via the Twitter streaming API) by hashtag; this script runs in its own process in shell.
The stream consumption script is managed by a manager process which maintains the set of track words (search terms) for open Presentations, feeding these track words to the stream process as needed. A Presentation has an open date range; as long as a Presentation is open, its hashtag is in the set of stream track words and tweets are captured for this Presentation. This is the conversational period of the Presentation in Donahue.
The stream process takes incoming tweet data, and by full access to the Donahue application instructs the application to create a Tweet object and make it persist; the application then converts the Tweet object into JSON data and POSTs it to the Node server which then pushes the data to connected clients to be inserted into the DOM (this last part is described below).
3. Node Servers and Sockets
The schematic overview of our use of Node is that we have a Node server running with an API to handle POSTing Points and Tweets and pushing these to clients connected via Socket.IO.
In fact, we have more than one Node server in the application. In order to manage load, and have a fallback in case of critically heavy usage, we set up: 1) a projector server to handle the POSTing of Points to one special, connected client: a client open in Projector view and connected to a physical projector for display on some kind of screen; and 2) a participant manager server which takes requests and delegates these requests to individual worker Node servers. The idea of the cluster is that we'd have no idea how many people might tweet with our hashtag or be connected via Socket.IO in any given period of time; we successfully tested against a load of over 12,000 simultaneous connections and requests with this cluster setup.
Key in our implementation of Node and Socket.IO was another brilliant Arc90 developer, Elijah Insua; Eli, an enthusiast of and contributor to the Node.org community, helped us set up, test, load test, and debug our use of Node, and wrote the cluster code (cluster-socket.io).
From Building to Presenting a Presentation
Building a Presentation
Building a Presentation involves giving it a name, a (unique within Donahue) hashtag, associating one or more Presenters with the Presentation, and setting some other attributes, then adding and editing Points. Points may be reordered or deleted, and images may be associated with Points (in the Projector and Participant views, these images are represented in elements; there is a permanent URL to the image itself represented by a shortened URL in the published tweet form of the Point).
Rehearsing a Presentation
Once the Presentation has been roughed out it can be viewed. Because 1) presenting a Presentation involves publishing tweets with the Point text and 2) Twitter restricts duplicate statuses from the same account, we made a "rehearsal mode" whereby a Presenter can move through a Presentation and view this in the Projector and Participant views without publishing the Points as tweets.
Presenting a Presentation
When the time has come, a Presenter will go to the URL for the Presentation and start it by clicking a button. This first sets the Presentation as open, thereby making the hashtag available to the stream manager process to feed to the stream consumption process. Next, an invitational tweet is constructed and POSTed to Twitter, which is then captured by stream consumption and inserted in the backchannel as the Introductory Point. (I'll describe the full mechanics of this in the section Publishing a Point). From here, the Presenter can use either the right arrow key or a clicker device to advance to the next Point, and so on, until the talk is done. After the last composed Point, the Presenter will click on a button to Set Presentation to Off Air—this constructs and POSTs a concluding tweet to Twitter with a permanent link to stats about the Presentation, including percentage of active Participants, number of tweets generated through the application, number of Points retweeted, etc. These stats are also then displayed as the final or end Point of the Presentation.
Publishing a Point
When the Presenter advances to the next Point:
- The application reassembles the next Point text along with any (shortened) image URL and the Presentation hashtag, and POSTs this data via the Twitter REST API as a status update for the associated Presenter Twitter user account;
- The application takes the POST response (or, since we have stream consumption, the tweet as returned through that process—oddly this often happens before we get a response from the POST) and reconstructs a Tweet object, makes it persist, and associates that with the Point;
- The application repackages the Tweet object as JSON data and POSTs it to the participant Node server;
- The participant server then pushes it to all connected clients (via Socket.IO), which insert the data into the DOM as a tweet element in the backchannel;
- The application repackages the Point as JSON data and POSTs it to the projector Node server;
- The projector pushes this data to the unique client connected as the "projector" client, in Projector view mode, where the data is inserted into the DOM and appears on the screen;
- The application takes the JSON Point data and POSTs it to the participant Node server;
- The participant sends the data to all connected clients, which insert the data into the DOM as a new element, hiding the other Point elements and doing some element management so that a Participant can traverse all currently published Points.
During the Presentation of Toss the Projector: Redefining the Presenter-Audience Dynamic at SXSW, many users were connected simultaneously, retweeting, replying, posting new tweets, while these Points were being published. Each of those events appeared in the users' browsers in what felt like real time, succeeding (to my mind) in representing the medium of a live, ongoing dialogue, which is the spirit of Donahue.