Facebook measurement errors

Facebook measurement errors have led some brands to question their investments and agencies to recalibrate their content evaluations.

Soon after Facebook announced in November that it had been overstating Page posts’ organic reach, digital agency Blitz was pitching a client on a Facebook campaign idea to make people more aware of the brand.

That meant measuring how many people were exposed to the campaign, including those who organically saw the ad because a friend shared it or commented on it, even though those impressions were free. According to Blitz’s social director, Kevin Wright, this brand had never been exposed to erroneous organic reach measurements by Facebook, but they were worried.

The brand’s team “came back and said, ‘Hey, I’m very concerned. Should we continue to spend these recommended dollars on Facebook with the awareness objective given that reach may be off?’” said Wright. That was definitely a hurdle.

Facebook Measurement Errors

The aftershocks of Facebook’s measurement errors continue to ripple across the advertising industry. While Facebook has emphasized that the flawed figures — such as average watch time, organic reach and video completion rates — did not affect how much it charged advertisers for their campaigns, that doesn’t mean advertisers and their agencies haven’t been affected.

“Yeah, media buys are safe, but not everything has a media buy,” said Nikki Scoggins, VP and director of social media at creative agency Deutsch.

In addition to paying for distribution, brands also pay for the production of the content, including content published to a brand’s Facebook page that may never be run as an ad or maybe an initial test to see if it performs well enough to justify a paid promotion.

To evaluate that content, the brands and agencies look to stats like average watch time and video completion rate. But, having learned that these stats weren’t accurate in the past and were corrected, marketers are now wrestling with whether to trust them and how soon they can.

“We’ll have to start almost over as far as our evaluation,” said Scoggins. “For me to feel confident going to the [agency’s creative team] and saying, ‘Your video did this or that. Let’s look at changing this now [in the video],’ I’ll feel confident speaking to those points in another quarter. I think.”

Not surprisingly, the less a brand or agency relied on Facebook’s own measurements, the less shaken it was by Facebook’s errors.

For creative agency 180LA and its clients, Facebook’s measurement errors have been a non-story, according to the agency’s new media and communications director, Roy Dank. That’s partly because most measurement errors were specific to videos on Facebook, and 180LA’s clients don’t invest much in Facebook videos.

The one false metric that would have affected 180LA’s clients, organic reach, has ceased to be an essential metric for clients as Facebook has shifted to a pay-to-play platform for brands over the past few years.

“We weren’t dependent on these particular metrics or their metrics in general because Facebook has been de-prioritized for many of our clients,” Dank said, attributing the lowered status to Facebook cutting Pages’ organic reach.

According to Blitz’s Wright, none of the agency’s strategic recommendations to its clients was “greatly affected” by Facebook’s erroneous metrics.” Still, it could and does hinder a client’s trust in these channels” and its willingness to invest in the producing the content to distribute on those channels.

“When we’re asking for dollars to produce video content, and this is in the back of their head — are the measurements my agency is telling me accurate? — that’s a problem,” he said.

A potentially more significant problem is whether these relatively minor errors preclude a major one that directly impacts how advertisers spend their money on Facebook. For example, the metric of Facebook’s store visits estimates how many people see a retailer’s ad on Facebook and stop by one of the retailer’s brick-and-mortar stores. Wright said Blitz considers this metric when recommending large Facebook ad buys to a client that operates a hundred stores.

“Imagine if an error came out a few months later. That will make a much more significant impact than having video completion rate off by 35 percent,” Wright said. “Now it’s always going to be in the back of the head: Is the data accurate? Because there are primary metrics that will greatly affect a brand’s or agency’s investment in the channel. It’s a concern. It’s not going to go away.”

The possibility of Facebook flubbing as a necessary measurement of store visits would have been inconceivable for some marketers before the errors were announced. But then again, Wright said that the aforementioned Blitz client who questioned whether to spend the recommended budget on Facebook ads would never have raised that question before the errors’ disclosure.

Facebook’s string of measurement errors disclosed since September has shaken some marketers, even if those marketers weren’t directly affected by the social network’s messed-up math.

Via Marketing land