Perhaps as vexing is who is counting readers and who counts them best. Unlike the methods Arbitron and Nielsen use to develop radio and TV ratings, the science of counting online and digital news consumers has existed only for a short time. At least nine companies have crowded into the business of measuring digital audiences over the past 15 years. Each company employs its own methodology to collect data. And because digital technology seems to leap forward almost every day, measurement techniques that were acceptable yesterday may not be adequate tomorrow.
"In human years, it's somewhere between 15 and 20 years old," Tom Rosenstiel, executive director of the American Press Institute, said of the maturity of Web analytics. But, he added, "In its evolution, I would say it's still actually quite young, probably not yet a teenager."
Eric John, vice president of digital services at the Alliance for Audited Media, is in charge of figuring out how the company which until last year when it rebranded itself was the Audit Bureau of Circulations can certify that the growing explosion of digital data collected by analytics providers is accurate. AAM has been doing digital audits for more than 15 years, but with the rise of new mobile phones and tablets, the company's greatest challenge today is staying in front of the way that data is collected from those devices in order to ensure its reliability.
The need to stay ahead never ends because a lot of advertising money is put at risk every year. In a blog post last week, John cited a study by RadiumOne, a social ad data firm. It said unverified Web traffic cost advertisers $400 million over the last year. AAM's digital auditing group regularly sees phantom traffic, "bots" that try to trick web measurement software, and even "shady outfits trying to game the system," he said.
That makes third-party verification of web analytics "crucial" for the growth of digital advertising, John said.
"It all comes down to comparability. If you have three or four different analytics providers, and an advertiser wants to look at a report of unique devices and content views and time spent [on a website], if all those different providers are counting things in a different way, there's no way an advertiser could ever make a smart decision about which publisher to go with ... or whether the publisher delivered what the advertiser bought," he said.
Clicks matter • At a time when the newspaper industry is facing what amounts to an existential crisis, accurately counting website traffic is central to staying in business. Last month, the Deseret News declared that KSL.com and deseretnews.com received far more visits in February than did The Salt Lake Tribune's sltrib.com site. The News cited two sources Experian, a Dublin, Ireland-based information measuring company, and Omniture, the Lehi-based Web analytics business acquired by Adobe in 2009. (The Omniture name has been retired; the company is a division of Adobe.)
The News didn't tell its readers which figures were provided by Experian and what came from Omniture. But the conflation of two sources made it impossible to know if the rankings were accurate. When The Tribune investigated, it discovered that Experian was the source for one KSL.com number (which included visits to its classified ad pages), and for the deseretnews.com and sltrib.com numbers. A second KSL.com number, which counted only news visits, came from Omniture.
One thing is certain. Experian and Omniture employ wildly different methods for counting Web traffic. Experian computes Web visits and other measurements with panel surveys that are augmented with data it buys from Internet service providers.
By contrast, Adobe puts tags on client Web pages to track site visits. Its method is used by NBC, newspaper giant Gannett Co., The Economist, Time Inc. and The Tribune. Every time someone clicks on a client page, the action is counted as a visit. By that method, sltrib.com received almost 3.5 million visits in February. The number of visits to The Tribune site was three times more than Experian's estimate of 1.1 million visits. Adobe's sltrib.com number was also 60 percent bigger than Experian's estimate of 2.2 million visits to deseretnews.com.
Chris Lee, president of Deseret Digital Media, wouldn't give out comparable Adobe's February numbers for his deseretnews.com website. But he said Adobe's estimate of visits to the LDS Church-owned site was "much higher" than Experian's number. The Adobe number for deseretnews. com also was "higher" than its visits number for sltrib.com, Lee said.
Tribune Editor Nancy Conway is skeptical. "How do we know, really? We are willing to share our numbers. We did, and we will [when asked for them]," she said.
Adobe thinks its measurement yields high levels of detail about Web traffic that panels can't provide. Matt Tatham, an Experian spokesman, says his company doesn't focus on which method may be superior. He suggests instead that both are valid.
"We have a large sample size" of 10 million Internet users. "It's not saying we are better than them or that they are better than us. I don't see Omniture as a competitor. Their service can be complementary to what we do, as ours can be complementary to what they do," Tatham said.
Crowded field • This lack of certainty about who has more visitors is typical of the still-evolving Web analytics business, which is problematic given the high stakes involved. Perhaps a dozen companies have moved into the Web analytics market other names in addition to Adobe and Experian are comScore, Google Analytics and Nielsen and all are jockeying for supremacy and the profits that come with cornering a big piece of it.
"It's a big market, and we obviously believe that the upside in our market is, as more content is digitally enabled, our business is only going to grow," said Bill Ingram, vice president of Adobe Analytics and Adobe Social, based in Lehi. "That's why Adobe bought Omniture."
Adobe is pushing its methodology of providing behavioral data collected by its Web analytics software over that collected by panels. The company hopes that like Nielsen, the longtime gold standard for measuring television viewership, Adobe will become the yardstick by which advertisers and newspaper executives calculate Web traffic and rank the competition. But today there is almost no agreement in the Web analytics industry about what should be measured or how.
"What is really lacking is the maturity around defining and [deciding] these are the metrics that we care about, [that] this is the accuracy of the metrics and getting that implementation of Web analytics to the point where it is certified by an audit, and then leveraging that metric to not only to describe your [newspaper] readership, but also use it as a way to sell your inventory" to potential advertisers and new readers, Ingram said.
Nor is there agreement among advertisers, who are cautious about putting their dollars into online newspaper ads. The share of digital advertising dollars captured by U.S. newspapers fell to 8 percent in the third quarter of 2012 from 10 percent in 2011 and 15 percent in 2007, according to the Internet Advertising Bureau.
"There is no consensus about how advertisers want to buy," Ingram said. "What we do know is that it's changing. Advertisers want more granular [data] buys than they are getting today. They don't just want to do ad buys against panel data. ... They want to know who is getting their message and how receptive they are to that message.
"We know that it is evolving, but there is no standard out there in the market today about which metric is going to win."
Questionable measures • Dana Chinn, a lecturer at the University of Southern California's Annenberg School for Communication and Journalism, said newspapers haven't kept up with other industries that do business online.
"There is a stark contrast between the news industry and e-commerce, in that e-commerce is saying analytics is do or die for us because we are a digital business," Chinn said. "News organizations don't say that, because if they did they would use the right metrics. All the news organizations I know are usually using the wrong metrics to make the decisions that are needed to survive."
"Unique visitors," for example, is a count of computers, not people, Chinn said. If someone accesses a newspaper website from home on a laptop, at work on a desktop and on the go with a smartphone or tablet, he or she is counted as three or four people. On the other hand, if several people share a computer at a library or school, they are counted only as one unique visitor.
"We use 'unique visitors' because it sounds better, and it sounds awfully close to what we are comfortable with, which is total circulation," she said. "It's just a comfortable measurement. But when it comes to really evaluating what impact and how many people we are reaching, it's totally flawed."
Another faulty measure is "time spent on site," Chinn said. More time could mean viewers like the content they see. Or it could mean viewers struggle to find the content they want. Less time spent on a site could mean viewers quickly find what they want and then depart.
Page view counts are ambiguous, too, she said. If the number of page views is high, it can mean people are finding lots of content to read. Or it might mean they can't find what they need. It could even mean the newspaper is gaming its numbers by slicing a story into several pages. What page views don't do is measure quality. "Good" journalism that has lots of impact frequently gets fewer page views than "commodity" journalism that has a short shelf life, said Jennifer Frigault, director of business analytics at Gannett Digital, whose dozens of websites include USAToday.com.
"As recently as a few years ago, page views was king. Everybody wanted to know how many page views did I get on my story [or] on my site. Now, we are moving toward a much more sophisticated view of not just page views, but how much engagement are people getting," Frigault said.
The metrics that many advertisers like most are "impressions," the number of times an ad is served up on a Web page to online readers and "click-through rate" how often the ad is actually opened by the reader, said Cathie DeDaughel, managing director of advertising agency R&R Partners' Salt Lake City office.
The number of impressions is similar to newspaper circulation in that it's based on the volume of traffic to the website. Each time the page is seen, the ad is counted, whether or not it's clicked on, DeDaughel said.
"Impressions basically reflect how many people are seeing your ad, so that's going to measure the awareness of the ad, how many people see it," she said, adding that click-throughs are even more important because they take the online reader directly to the advertisers website.
The question of whether digital advertising works has been answered, she said. It does, and it's becoming increasingly important to advertisers.
"That's where the readers are migrating to, particularly the younger readers," DeDaughel said. "It's pretty standard now; it's part of the overall media mix" that advertisers must consider when planning ad campaigns.
Dueling metrics • Chinn thinks the best measurement of Web traffic is visits or sessions a concept similar to impressions. When the count goes up, it means more people are coming to a site or it means the same group of people is coming, but they are coming more often. And when the number falls, it probably means consumers don't find the site relevant, she said.
"It's a strong metric, in that there is no ambiguity about what it means. It's flawed technically, but not as flawed as either unique visitors, or the really worst measure of all, time on site," Chinn said.
Frigault said measuring viewer traffic isn't easy to do. Gannett Digital has used Omniture's (now Adobe) Web analytics software since 2006, and although Frigault said there are plusses, it isn't perfect. Adobe tends to overcount unique visitors, so Gannett Digital also leans on comScore, which seems to undercount visitors, she said.
"So we use a combination of different data sources to tell the story," Frigault said. "There has not been one company that's been able to crack the nut."
Tribune Editor Conway agreed. "You can't rely on one method," she said. "I think we have to look at all the numbers and weigh them, understanding that there is not a single magic number" that accurately represents any paper's digital readership.
At the American Press Institute, Rosenstiel, thinks the confidence of readers and advertisers in Web analytics is being undermined by what he suggested is an oversupply of measuring companies.
"There is a sense that it's not helpful if different companies have different metrics, and people are skeptical about the value of the core metrics that we have," Rosenstiel said.
"There's an old saying in research that is attributed to [Albert] Einstein. Not everything that matters can be counted, and not everything that can be counted matters."
Twitter: @sltribpaul Who reads newspapers lately?
164 million U.S. adults read newspaper content in print or online in a typical week, or access it on mobile devices in a typical month.
59 percent of adults, ages 18-24, read newspaper content in print or online in a typical week, or access it on mobile devices in a typical month.
The mobile newspaper audience was up 58 percent in an average month in 2012, compared with 2011. There were an estimated 34 million adults in the mobile audience last year.
Total print and online newspaper audience in a typical week fell an estimated 2 percent in 2011. When the mobile audience is included, the decline was less than 1 percent.
Source: Scarborough Research via Newspaper Association of America