The Numbers That Lie

Hello everyone, and hello summer!

We’re back (OK, I’m back – Derek is in Alaska for two weeks) from our usual midyear hiatus and are in the beginning stages of planning for CMEpalooza Fall — that’s Wednesday, October 19 if you are scoring at home. We’ll have our Fall agenda soonish. That means sometime before the last fresh corn of the season leaves your local supermarket. We like to give ourselves plenty of wiggle room with these things.

Anyway, I’m here today to write a little bit about data, or more significantly, data that makes you go, “Hmmmm.” Since the dawn of CMEpalooza, we’ve tracked traffic to our various delivery mechanisms to get a sense of what’s popular, what’s not, how our audience is growing, etc. The usual metrics. And truthfully, we’ve grown about 10-15% in terms of general traffic to just about everything year over year. It’s been a nice, steady climb that we’ve always felt good about.

And then came this Spring, and well, something weird happened. Our YouTube viewership for a handful of sessions went crazy. Prior to this Spring’s event, our most viewed session had accumulated somewhere the neighborhood of 2,500 views according to YouTube analytics. Derek probably has the actual number somewhere, but I’m too lazy to do research (hello, it is summertime!).

As we do following every live event, Derek and I went in to see what our YouTube numbers looked like this Spring. We can see, in real time, how many people are watching each session — that’s one of the best things about Streamyard, the platform we use for our broadcasts — so we knew our live event was well attended but in line with previous iterations. So we were quite surprised a few days later to see what our YouTube metrics looked like for two of our sessions.

Beyond Checking the Box to Achieve Commendation: 6,327 views
Demonstrating the Value of CME to Internal and External Stakeholders: 9,064 views

Every other session from the Spring had relatively normal traffic, but these two significantly outperformed any expected metrics. I joked to some of the presenters of these sessions that perhaps their legions of college exes had been stalking them to see what they were up to. We dug a little deeper into YouTube Analytics to try to figure out what had happened. And while were are some answers there, a lot of questions remain.

From what we could tell, these two specific videos somehow became popular “recommended” suggestions for people watching other YouTube videos. For the most popular of these two CMEpalooza Spring sessions, the most popular linked videos were the Optimist Bahamas Live StreamData Exchange Podcast (Episode 123): Jack Clark; and Day 1 Conference: “The Geopolitical Impact on Talent Acquisition” (Anke Strauss, IOM). All very interesting I’m sure, but I have no idea what any of them have to do with our sessions or what about the title or content or audience may have triggered their inclusion in those videos “recommended” suggestions. Not surprisingly, the number of viewers of these two sessions who watched more than the first 30-60 seconds was quite low, a significantly lower percentage than our typical sessions.

For those of us in the CME planning world, we see these sorts of statistical anomalies from time to time. Maybe it’s pre-test data that looks a bit squirrely or something in the evaluation that has us scratching our heads. It’s often tempting to overlook the potential drivers of these data deceivers because they look so good. I mean, who wouldn’t want to be able to report that 5,732 learners accessed one of our online educational activities or that only 12.3% of learners answered a pre-test question correctly about a key variable tied to a learning objective? But usually, there is enough that looks suspicious (and sometimes, you can figure out the issue) that requires the outlier data to be cast aside.

So no, in our next CMEpalooza sponsor prospectus, you won’t see us crowing that our overall audience for the Spring 2022 event was 400% greater than any other iteration. But say we did. Would that be a boldfaced lie? Technically, maybe not – I mean, the YouTube data shows what it shows. But in an industry where we rely so heavily on data to tell our outcomes stories, it’s the interpretation of the data that often matters most. So, no, we won’t pretend that thousands of people are suddenly interested in Derek’s new haircut or the insightful question from our audience at 34:52 of one of our recent sessions. We’re good, but we’re not that good.

Nudge Nudge: Some CMEpalooza Reminders

Things are always busy in the days leading up to CMEpalooza, and I know you are probably tired of seeing ONE MORE notification in your inbox from us. I promise we’ll leave you alone tomorrow, but just a few reminders from me today:

  1. We still need about 30 more responses to our CMEpalooza Feud survey to reach the magic number of 100. If you have not yet filled out your answers, you can go to this link to access the survey. It’ll be fun. And short. I promise. Just do it while you groove out to a little Gordon Lightfoot. (Note from Derek: Gordon Lightfoot?? Good lord we are old.)
  2. To avoid a last-minute scramble next Wednesday, go ahead and download the Poll Everywhere app now so that you can actively participate in our various sessions. Our friends at Array are sponsoring our audience response system this spring, and we’d hate to let them down.
  3. Go to your email calendar and double check that you have marked off next Wednesday as a “medical emergency” day. We wouldn’t want anyone bothering you while you watch our sessions.
  4. In the spirit of yesterday’s blog post, Derek has asked me to offer a CMEpalooza Twitter Haiku challenge to our audience. I told him no one would probably participate, but he’s persistent. So go onto Twitter today and give us your best, CMEpalooza-themed 5-7-5. Maybe you’ll even get a prize if your haiku is good – I dunno.

Let’s Play the CMEpalooza Feud!!

“We asked 100 left-handed firefighters whose names starts with the letter Q the following question…”

Wait, did they really ask this question of 100 left-handed firefighters whose name starts with the letter Q? Are there really 100 left-handed firefighters whose name starts with the letter Q? Or is there some pencil pusher in the back who just makes up the numbers to the Family Feud surveys?

According to the Google, there is a firm that actually polls random members of the public (or at least they used to), though there doesn’t seem to be much of a trace of people who actually participated in these polls. So I’m skeptical.

But what I’m not skeptical about is that our very special version of CMEpalooza Feud coming up next week during CMEpalooza Spring will include actual survey answers from the CMEpalooza community (texts Derek to remind him that our Spring meeting is on Wednesday, April 6 from 9 am-5 pm ET). But we need your help.

Click here to access our survey that will be used during our CMEpalooza Feud session. We’ve got a bunch of different sorts of questions here, none of which will take a tremendous amount of thought for you to answer. Just write down the first thing that comes to mind. This is all anonymous, so don’t worry about getting into trouble with your boss (or spouse). What we’ll do once we get 100 responses is tabulate the results and use them during next week’s session.

Once again, click here to access our survey that will be used during our CMEpalooza Feud session.

Thanks everyone – you are the best!