Why Arbitron's satellite radio ratings are wrong

Tuesday, April 29, 2008 at 2:43 PM
Tags: Arbitron, Ratings, Satellite Radio, Sirius, XM

"When the only tool you have is a hammer, then every problem begins to look like a nail."
- Abraham Maslow

In the world of statistics and research, methodology is paramount. That's pretty basic. If you're going to draw a conclusion from the polling of a sample, the methods you use to extract those answers had damn well better be solid. Because in the end, the way this data is gathered defines the final outcome.

And this is exactly why I say that the Arbitron ratings for satellite radio listening are wrong. Indeed, they're bullshit.

Maybe that was a bit harsh, I'm sorry. Let's instead say that these ratings are for "information purposes only," because that's how Arbitron refers to them.

Mind you, the criticism that's about to ensue has absolutely nothing to do with terrestrial radio and how its listenership is measured. That's a whole different issue. This is meant to point out the flaws in how Arbitron measures satellite radio listenership and nothing more. So when I say that "Arbitron's ratings are wrong" I mean for Sirius and XM, and not regular radio.

So let's begin.

Arbitron Diary

Reason #1
The Arbitron Diary.

Actually, it's the methodology for terrestrial radio ratings that's screwing it all up. Arbitron's diary is built from the ground up to measure AM/FM. Not Internet Radio. Not Satellite Radio. Not even HD Radio (that is, if anyone was listening to HD Radio). It's meant for good ol' regular radio and nothing else.

Here's a big problem: There is no checkbox for a listener to select "Satellite Radio." None. Nothing for Sirius. Nothing for XM. Nothing. Diarykeepers need to actually physically write in the service and the channel info, in addition to the time they started and stopped listening to the program as well as the location of where they did this. Do you think people will actually write in all that information? Right there, the data becomes tainted.

Follow the jump to keep reading...



Reason #2

Arbitron favors AM/FM.

Yes, the Arbitron's ratings automatically skew away from satellite radio. And Arbitron makes no secret of this:
"In instances where it is not clear if a diarykeeper was attempting to record listening to a terrestrial station or to a satellite radio channel, Arbitron's diary edit procedures presume that the diarykeeper was attempting to record listening to an AM or FM radio station."
(emphasis added)
Read that again. If a diarykeeper doesn't explicitly write down either XM or Sirius, then then data point goes to terrestrial.

So, for instance, if you're in New York and listening to a Yankees game, just writing down "Yankees" automatically gives the rating to WCBS - even if you were listening to XM. Or maybe you write down "Sean Hannity," which could go to either Sirius or XM, but the rating would instead go to WABC. And yes, the same even applies to certain shock jocks (*cough* Opie & Anthony *cough*) that happen to be syndicated between satellite and terrestrial radio.



Reason #3
The numbers don't add up.

Sirius had roughly 14% less subscribers than XM at the time of the Spring 2007 ratings, yet Arbitron's cume showed that there were 36% less Sirius listeners than XM. By the end of the fourth-quarter, Sirius had only 8% less subscribers than XM, but Arbitron's Fall 07 ratings show a difference of 33% in total listeners between Sirius and XM.

Considering how Sirius continues to close the gap in subscribers, the numbers don't add up - not even remotely. So what gives?

Are there more XM diarykeepers than Sirius diarykeepers? Maybe diarykeepers tend to gravitate to XM-equipped vehicles? Oh, I got it, maybe diarykeepers tend to carpool with XM subscribers!

Or maybe, for a nationwide calculation, the sampling is actually pretty small. Maybe, a single diary entry can throw off the ratings. Arbitron even admits that it takes only 15 minutes of listening recorded in one diary to make the cut:
"Estimates are reported for all satellite radio channels that were credited for at least one quarter-hour of unweighted listening in at least one in-tab diary in the specified demo and daypart."


Reason #4
These aren't subscribers, and they don't act like them.

Sirius and XM should attract a similar "type" of listener but with some subtle differences in taste. Afterall, there's only a certain subset of the population that is willing to pay for radio. And a majority of people subscribe because Sirius or XM came pre-installed in their car.

But yet, there's a massive difference in how Sirius listeners and XM listeners tune in to similar channels.

For instance, there should be relatively the same listenership between both Sirius and XM's Hits channels: Sirius Hits 1 vs XM's Top 20 on 20. Yet Sirius Hits 1 brings in 56% less listeners than XM's Top 20 on 20. Classic Vinyl vs Top Tracks: a 35% difference. Let's put Sirius' top news channel against XM's top news channel - and this is identical content mind you - Sirius FOX News vs XM FOX News: nearly a 70% difference.

The answer is that these numbers aren't measuring subscribers, they're measuring terrestrial radio listeners who just happen to hear satellite radio.


XM's Audience
Reason #5
Internal metrics show a completely different picture

At the Deutsche Bank Securities Inc. Leveraged Finance Conference in October 2007, XM Satellite Radio gave a presentation (PDF) that gave a peek into the size of the audience some of their music channels pull in. Here's what they divulged:

  • XM 80s on 8 - Audience: 3,400,000
  • XM 70s on 7 - Audience: 3,200,000
  • XM Top 20 on 20 - Audience: 2,500,000

Yet Arbitron's numbers show a completely different picture:
  • XM 80s on 8 - Cume: 698,300
  • XM 70s on 7 - Cume: 667,400
  • XM Top 20 on 20 - Cume: 1,049,200
The data that XM released was sourced to OTX Ratings Survey Spring 2007. Incidentally, XM dropped Arbitron in favor of OTX in April of 2007 - and later that year Arbitron began to release its satellite radio ratings to the public.

XM's reasoning was replacing Arbitron was: "...XM is a highly specialized media vehicle, and Arbitron's methodology didn't adequately reflect that. After years of trying to work with its methodology, it was still not accepted by most agencies and clients... [OTX] better represents our subscribers' listening habits."



The basic and overall problem with these ratings is that Arbitron's diary process is based on "listener recall" rather than an automated/electronic method. So theoretically, repeating the channel name over and over again can result in a higher rating.

But since Sirius and XM don't care about Arbitron numbers, they don't employ the same tactics that terrestrial radio stations do. Plus, reason #1 above is more than enough to screw everything up, because without the service/channel name, the credit goes to terrestrial (Mark Ramsey was the first to point this out for what it's worth).

An electronic method, such as Arbitron's new Portable People Meter (PPM), would show an entirely different picture. But neither Sirius or XM are encoding their channels for the PPM, so it doesn't matter.

The real question we should be asking is, with all the lapses in methodology, why is Arbitron even releasing this data to begin with?

TrackBack/Ping:

Comments

Well written Ryan.

Ryan, great job. Thanks for drilling down on this charade.

thj

0.0?

The only numbers that matter around here.


Good job Ryan.

Ryan, on point #3("the numbers don't add up")...

I think part of the discrepancy is the way the two services measure "subscribers". Since Sirius counts units in vacant cars (OEM), rather than real paying customers, it stands to reason that those empty cars aren't filling out diaries. just my $.02.

thj

Well done again. I've done the OTX diaries and they definitely are more reliable than the Arbitron guesstimates...

Thanks for writing this up. Looking foward to reading it.

0.0

@waitwhat Factor in for the "car lot subs" and it's still about a 17% spread between XM and Sirius subscribers.

And when you compare the Spring '07 ratings to the Fall '07 rating, Arbitron's cume only grew 574,800. While Sirius and XM - during that same period - grew by 1,956,247 combined subscribers.

Ryan, on point #3("the numbers don't add up")...

I think part of the discrepancy is the way the two services measure "subscribers". Since Sirius counts units in vacant cars (OEM), rather than real paying customers, it stands to reason that those empty cars aren't filling out diaries. just my $.02.

thj

Posted by: waitwhat | April 29, 2008 3:47 PM
-----------------------------------------------------------------------------------------------------------

You took the words out of my mouth.
Parking lot subs don't have listeners.

Excellent Work. :applause:

There's also the fact that SIRIUS & XM, because they have online streaming, both have a very good idea of who listens to what and in what numbers. Obviously, not everything is available online, and an online listener has different habits and so on, but . . .

I love how you Stern zombies spew out 0.0 yet O and A keep getting their contracts renewed.

Keep trying, drones.

"I love how you Stern zombies spew out 0.0 yet O and A keep getting their contracts renewed.

Keep trying, drones."

haha O&A ratings keep going down, and the only reason they keep getting renewed contracts is because management is stupid. Its the same thing with Imus, his ratings are crap but they still rehired him.

Keep sucking E-Lo's dick because you got mentioned on the Virus that is quickly getting cured. Excuses, excuses. Fact of the matter is, Howard is Sirius, and XM will never miss 202. It isn't even in the top third of the channels. All three of the pests can cry when Mel culls that whole stupid fucking channel, but I hope they use the bandwidth from XM to stream Howard TV to my car. What better way to capitalize on their obvious biggest draw than to allow you to not just listen, but to watch the show as it goes down? That is the future.

Opie and Anthony keep getting their contract renewed because in the country of the blind, the one-eyed man is king.

To simplify this analogy for O & A fans (because you're not smart): They're not funny, interesting, or entertaining in the least bit, but their competition on terrestrial radio is even worse. That's why they can get 0.0 and still be on the air.

OH! Well since an XM executive tells prospective investors the number are wrong I guess your right !?!?!? The SPIN is “people write O&A in their diaries and that’s counted for terrestrial radio”. Here's a link to an Arbitron article on how listeners fill out a diary with a % breakdown of how’s its recorded:
http://www.arbitron.com/downloads/Di...r_brochure.pdf
Only 0.5 % of the diary keepers solely use programming (show name) to identify their choices. The three other identifiers Station name, Frequency and Call letters are far and away the most popular methods of identification especially for syndicated programming. All the identifiers need to be written in, now I know O&A fans can’t read or write but that’s not Arbitron’s problem. Arbitron specifically states in their diary instructions to denote internet or satellite radio stations http://www.arbitron.com/downloads/diary.pdf If there is any problem with the ratings it’s caused by the retards filling out the diary not the company counting the results. I thought CBS said O&A were #1 in their demographic according to Arbitrends? But wait Arbitron doesn’t know what they are doing so how can that be right? Here’s a little hint for retards filing out paper diaries:
Station name: The virus, Frequency: XM/ Satellite
Call letters: 202, Programming: O&GAY SUCKFEST

Has it occurred to you that O&A had better numbers when they were suspended for 30 days? If I was Ron and Fez I’d be pissed off they came back! R&F did better without them.


Why is it all the O&A and Stern drones are always arguing over who's better? It seems fairly obvious that neither of them are being given the credit that they deserve. Also, being that O&A are on both satellite and terrestrial, will it ever be a feasible idea that their numbers could ever be properly recorded?

amateur statistics.
fix the polling forms before claiming you have any sort of insight into satellite listenership

How about an Orbitcast article on how all the retards filling out their Arbitron Diaries wrong still didn’t help O&GAY’s terrestrial ratings in NewYork? I mean if Satt ratings unfairly went to WXRK that means they SUCK at testicle radio that much more. Isn’t a 1.7 share David Lee Roth territory?

"Fact of the matter is, Howard is Sirius"

This is why i hope the merger never happens and why i would never want sirius. While the majority of my xm listining is O&A, i know there is a lot more great stuff to listen to. if i had sirius, i guess if i didn't listen to Howard i would be screwed since Howard is Sirius.

back to the post. with all the howie zombie rhetoric aside (we get it O&A suck,and Howard is a radio god, and nobody will ever be as good as him), the only way to really figure out who listens to what is to send every subscriber a diary (that shouldn't be that hard for them to send out) or have them take a survey on what they listen to. let them document if it is listened online or recorded content, or live. maybe instead of specifying certain times, maybe document how much time each person listens to a particular channel during the day. For instance, I record the O&A replay in the evening and listen to it the next day at work. I am away from my desk off and on and hit pause when i get up. while the times i am actually listening vary, the amount of time i listen never changes. I never listen live, but always catch the full show. then when i get done listening to that, i just log online and listen to a couple music channels (mostly Lucy) and the comedy channel.

Ryan, very nice article. One of the reasons I continue to come back to your website is your excellent points and facts.

Keep it up!

so, i guess you howie drones are going to avoid GTA4, considering that O&A are in it

"To simplify this analogy for O & A fans (because you're not smart): "

I'd be willing to put my PhD against anything you have, it's pretty obvious that this whole O&A/Stern thing is way overblown. It's also pretty clear that Stern only has a handful of years left in this business before he retires and has a good time with his billions. I just wonder where these foul mouthed Stern fans will go in 3-5 years.

pests are certified cunts

**********Warning************
this will cause major douche chills
http://wackbag.com/showthread.php?t=89886

Who hands out those certifications? Someone from SternFanNetwork? Which we all know is also a place filled with cunts.

"Who hands out those certifications?"

Common sense does
your welcome

0.0

0.0

0.0

the same cunt that attacked Sterns daughter is waaing like a little bitch that his whore "chick" is being attacked...

KARMA HYPOCRITE KARMA!!!!!!!!!!!!

Wait Ryan you mean a company that gets most of its revenue from the radio market, making it hard for sat radio to be counted......no wai.

thing with stats is that many of the problems effect both sides so the effect is not as profound as one would make it out to be. now if you want to compair sat to FM then the two sides are not equal

also stern sees less negative impact on a "write in" factor because of his insane name recognition and the fact that his vote will hardly ever be confused with FM.

To the guy that wants to watch Howard TV streaming in his car as well as listening to it on the radio. I hope that you get what you wish, and then I hope you drive head-on into a school bus. Dope.

0.0....ok enough with the bullshit. Think for yourself instead of what Howie lies to you about. They were NEVER 0.0

That being said, we get it...some of us like Howie...some like OnA. A matter of taste. Thing is, most OnA fans were once Howie fans but found something funnier in OnA. I'm willing to bet that its a tiny minority that were OnA fans that found Howie. For anyone to say how boring or unfunny OnA are without listening for more than 20 minutes is hypocritical and just dishonest. Probably 90% of all OnA fans have heard Howie in the past and just outgrew him. He hasn't changed with the climate of radio and to many (including myself who used to think he was a god back in the 90s) just is the same ol stale crap driven into the ground. His interviewing skills are WAY overrated and anything funny that does happen on his show is someone else's doing.

OnA, in all likelihood will never have the numbers Howie had on FM. Howie was on air before internet radio...before satellite...before Ipods, etc. There were no alternative sources for entertainment like there are now. Not to mention the moral climate of radio now where you can't even say 'poop' or 'Do you know karate?' without being dumped out of. Listen to old OnA from wnew and you'd think they were clips from the XM show because of what they got away with. They just cannot do the show that most OnA fans are used to. Yet they still adapted to the best of their ability and put on the most entertaining show on radio. Howie...even with the less strict rues before he left couldn't adapt. His steadily dropping numbers reflect that. And even now his show is an uncreative rehash of the same fart and lesbian jokes and interview questions from a decade ago. If thats what you like...more power to ya. But we also know that the arbitron numbers for OnA are way off. Execs at XM have said numerous times that OnA are one of if not THE most listened to show on XM. They don't subscribe to arbitron. The company they do use tells a HUGELY different numbers story.

OnA have a niche audience. They are far edgier and harsh than Howie ever was. Many people won't gravitate to that kind of humor. They will prefer Howie's poopy and fart jokes rather than rape and cancer jokes. But yanno what? In all my years of being a huge Howie fan (from wnbc till 2002) I NEVER laughed out loud when listening to Howie. But from the day I found OnA I have shed tears laughing almost on a daily basis.

The hell with the numbers and Stern fan regurgitation of bullshit Howard tells them...OnA entertain me and millions of others...more than Howie ever did. Thats all that matters to me. If the lies were true and they really were 0.0...I'd still think they are the funniest, edgiest, most clever and entertaining show on radio EVER.

My humble opinion.


Yay, more Stern vs O&A rhetoric clogging the network.

Zombies and Pests amaze me. How they can spend sooo much time argueing.

For the Zombies, if Stern is so good and on so often, how come you're constantly in here talking about O&A. Does Stern's show suck that bad? Do you have tons of free time to complain about two people who "don't matter"? Are your lives that shallow?

For the Pests, why do you care about what Stern still does? Does your show suck bad too?

My suggestion: Enjoy your own show, stop clogging the network with your troll posts, and GET A LIFE!

Thanks.

^ Millions? Don't get too excited, its just a few thousand according to Arbitrons ratings. Even if the terrestrial stations get the credit for the Open&Anus show, it is still putting up horrendous numbers on terrestrial as well. What Ryan says simply does not compute. Howard has millions, 0.0 have a few thousand.

whiny cunts...its to shove the shit O&A feed you mindless sheep back in your faces!

"howards irrelevent" .."hes just washed up"...really?math says otherwise


and if its true or false,the ginger kids meltdown over his whores porno is glorious

thats called karma for fucking with Sterns daughter

"O&A win in the end!"...what did they win????????????

"Common sense does
your welcome"

Hey douche it's YOU'RE welcome as in you ARE welcome.. brilliant post by a Stern fan.

Anonymous Coward = Anonymous Cunt

YOU'RE WELCOME!!!!!!!!!!!!!!!

I love the Stern fans in here calling others "sheep". Makes me chuckle everytime.

I have never seen a bigger mindless mass of minions than Stern fans, ever.

Which is why I have Sirius, and NEVER listen to Stern.

Grow up kids.

Anonymous Coward = Anonymous Cunt

YOU'RE WELCOME!!!!!!!!!!!!!!!


Good, glad you got it right this time. You're a moron by the way.

@ BOC, why are you trolling wackbag? Not even O&A fans go to wackbag anymore. You waste a lot of your life don't you?

If you're a Stern fan, great. The guy is a bona fide legend and still has a popular show on satellite radio. If you're an O&A fan, great. They do mornings in the country's biggest market and also have a highly listened to satellite radio audience. Personally, I prefer the latter. But I'm getting tired of IDIOTS over and over again posting useless shit like "0.0". Hey asshole, despite what either show tells you, both shows are doing well and they both have a lot of fans. But any story that tangentially has anything to do with O&A or Stern, and the dummies come out of the woodwork. And I hate to say it, but it's usually the Zombies that always get their first little 0.0 jab in. God, will you morons stop your infantile babble?

We all know that the real problem here is that ratings were created by the Jews.

Hey JC

“For anyone to say how boring or unfunny OnA are without listening for more than 20 minutes is hypocritical and just dishonest.”

They were on WJFK (Wash. D.C. market) after Howard left testicle radio. I gave them a try for a month because just like CBS radio I had no choice. I figured, “what the hell if they’re good clones I might not need a Satradio”. Well something went horribly wrong with the cloning process so I bought a satradio and listened to RawDog comedy until Howard started. O&GAY sucked sooo hard at WJFK that station overnight unknowns “Big O and Dukes” recently took over their time slot. This is a show and I quote, “hosted by an illegal alien and a beefy stalker” Ar! With a tag line like that you can see why O&GAY had no chance. (snicker,snicker). So much for the Virus spreading in D.C. two near janitors stomped them out.


“Execs at XM have said numerous times that OnA are one of if not THE most listened to show on XM. They don't subscribe to arbitron. The company they do use tells a HUGELY different numbers story.”

Oh well, if an XM suit says so I guess you’re right!??!! I mean an XM exec would never have a reason to “color the truth” for customers and prospective investors in order to protect their precarious position. Every PEST knows that. Arbitron has been a respected name in the rating game for nearly 50 years. Satellite radio doesn’t subscribe to Arbitron because they didn’t like the TRUE numbers the service was generating. XM would rather hire an unknown Internet company “Online Testing Exchange” (OTX) as a ratings mouthpiece OTX has only been in business for 3 years their website looks like a front for a PR firm (how do I look up ratings)? http://home.otxresearch.com/

Most of their press releases point to VERY LITTLE EXPERIENCE in rating radio of any kind. Who certifies OTX results? Neither XM or Sirius are willing to encode their signal for the new Automated PPM because they know the real numbers but they don’t want them reported by a source outside their control. The PPM system will be harder to spin no wonder XM won’t subscribe. Neither service has offered solid rating numbers only illusions to internal research pointing to higher numbers and disparate attempts to buy better numbers from PR FRONTS LIKE OTX. These numbers are only use to fool investors and comfort deluded PESTS!

ZERO ………..POINT…………ZERO!

Hey Ryan How about an article comparing Arbitron to Online Testing Exchange? OTX claims to be a top global marketing research firm. Well they’re not listed in the top 25

http://www.marketingpower.com/content/honointl25.pdf

Arbitron is #10 ranked what is the OTX rating? Here’s an excerpt from a review of OTX at a market research website
“First, OTX gets props for several things. First, they share a lot of their research results If you go to their web site, you can read all about teen shopping habits, teens and cell phones, movie survey results, etc. They seem to do a lot with kids and teens, which isn’t surprising given the cuteness of their interface (which I’ll explain in a moment).”

http://www.marketresearchtech.com/otx-online-testing-exchange.htm#more-113

WOW! Is XM Desperate to buy some numbers or what? They had to sink to web based Kiddy review site to find the numbers they wanted to fight Arbitron LOL! The OTX website smells of PR FIRM in the guise of Market research.

This is all arbitron's fault they expect the people filling out the diaries to be able to read and write. They just overestimated the average O&GAY fan .An X and and M is a lot to expect from someone who can barely manage an O much less an A

whoa whoa whoa!!!!!!!!

lets be fair here,0.0 have earned there right to be called 1.7 in the NY Morning Drive

Sincerely
-David Lee Roth

OTX IS A JOKE!!! http://www.surveyexchange.com/howItWorks.htm You only need a fake free email address to register for the email surveys "Take a survey: get points and entry into sweepstakes, Redeem your points for great prizes" weeee! I can almost hear circus music. Online Testicle Exchange has less credibility than Arbitron period! XM fears the PPM and refuses to encode for it what do they FEAR? Why do they try to use a nobody like OTX to grasp at straws?

3 facts on Dopie and anthony

1. in philly, pweston and steve on WMMR were even beating them...NOW that is AWFUL!!!!

2. 0.0-----admit it o&a fans....dont sugar coat that story!

3. stern put up amazing rating's on FM radio that O&a will NEVER TOUCH!!!!!!! so how are they better???? stern had a huge following when people buy XM for music, not dopie and anthony!....

JC says: Howie...even with the less strict rues before he left couldn't adapt. His steadily dropping numbers reflect that. And even now his show is an uncreative rehash of the same fart and lesbian jokes and interview questions from a decade ago. If thats what you like...more power to ya. But we also know that the arbitron numbers for OnA are way off. Execs at XM have said numerous times that OnA are one of if not THE most listened to show on XM. They don't subscribe to arbitron. The company they do use tells a HUGELY different numbers story.
=======================================================================

you obviously dont listen to stern so dont lie about what content is on the show!
you o&a pests constantly lie to the public about stern's content so dopie and anthony can improve that 0.0 rating ! IF IT WASNT 4 STERN Dopie and anthony would have no SHOW!!!!!!!!!!!!

-STEADY DROPPING NUMBERS????????.....those numbers were still MUCH better then O&As.

--Execs at XM have said numerous times that OnA are one of if not THE most listened to show on XM??????????? .................. i bet jim breuer gets more listeners on sirius raw dog then the virus!

BHWAAH HAHAHA! It is common for the low men on the ratings totem pole to challenge the Arbitron numbers religious stations do it all the time. But for XM to use an unknown unproven company like OTX Online Testicle Exchange to refute an industry standard like Arbitron is a sign of utter desperation! Why did XM break with Arbitron right before the introduction of the PPM system? Why refuse to encode for PPM when everyone else is. I mean the PPM would have removed the possibility of incorrect entries why jump ship (unless your numbers SUCK)?

Notice how Opie is crying about Page Six articles slamming his girlfriend, but its okay when he personally attacks others on the Radio, including howards daughter. Opie Hughes = a hypocrit piece of $h*t

OTX IS A JOKE!!! http://www.surveyexchange.com/howItWorks.htm You only need a fake free email address to register for the email surveys "Take a survey: get points and entry into sweepstakes, Redeem your points for great prizes" weeee! I can almost hear circus music. Online Testicle Exchange has less credibility than Arbitron period! XM fears the PPM and refuses to encode for it what do they FEAR? Why do they try to use a nobody like OTX to grasp at straws?

Post a comment

(or continue the conversation in the Orbitcast Forums)

Recent Entries

From the Forums...
Search Orbitcast:

Recent Readers
Latest Poll
Sponsored Links





Copyright © 2008 Orbitcast Media, LLC.