Presentation vs. Discussion (or 'Attendee Evaluation Analytics')

I have a client who pays a lot of attention to their Attendee Evaluations. They collect a ton of data with them - enough, in fact, to do some interesting analysis. For example, for each session at the event, they ask attendees to evaluate each speaker AND the session overall. I expected that the session would usually score in about the same range as the average of the speaker scores.

As it turned out, this wasn't the case at all.

In some instances, the session scored higher than the average of the speakers, like this:

SESSION 1a:
Speaker 1: 4.2 (out of 5)
Speaker 2: 3.8
Speaker 3: 3.7
Speaker 4: 4.3
Speaker Average: 4.0
Session Score: 4.3

In others, the speaker average was higher than the Session Score, thusly:

SESSION 2a:
Speaker 1: 4.2 (out of 5)
Speaker 2: 3.8
Speaker 3: 3.7
Speaker 4: 4.3
Speaker Average: 4.0
Session Score: 3.7

I wanted to try to find some pattern behind this data. So I looked at it relevant to moderator score, track or category, day, N, audience size, and other factors. Finally a pattern emerged when I looked at the data as a function of session format.

When the session was a Panel Presentation (where speakers prepared individual or a collective presentation), the Session Score was almost always higher than the Speaker Average.

When the session was a Roundtable Discussion (where 3-5 speakers conversed without a presentation), the Session Score was almost always lower than the Speaker Average.

Cool, a pattern.

But why? Well, we didn't know the pattern exists so we didn't include any 'Why do you think that?' type questions (although this is a good idea for next year). But I think it's due to two reasons:

1. Increasingly, conference audiences want something tactical they can benefit from immediately upon returning to their desks. Panel presentations - despite because of their uninspired outline-driven, bullet-pointed format DO provide some clear take-aways.

2. Conference attendees warm to a session where the speakers show enough respect for their audience to actually create a physical work product by way of preparation. The audience knows they're not just showing up, speaking their mind, and splitting. Sessions that respect the audience score better.

I think a session that scores higher is more desirable than speakers that score higher. Speakers can 'belong' to other conferences, but a Session is proprietary. It's where a conference producer's unique voice is heard, even if the speakers are regulars on the circuit.

The takeaways?
- Pre-determine a ratio of panel presentations to roundtable discussions.
- Test the different formats on your audience, and collect enough data (as per above) in your evaluations to do some analysis.
- Don't shy away from multiple formats: Panel Presentations, Roundtable Discussions, Town-Halls and other participatory formats, Solo Presentations (that aren't necessarily keynotes). They not only give your show variety - they give you the opportunity to find something (new) that works (better).