Meta's Survey of "Worst Companies in the US" Ranked Meta Last

On the 12th day of FTC v. Meta Platforms, survey king Curtiss Cobb comes back to the stand chastened, and we look at ad load with revenue head John Hegeman.

As the third week of trial neared its end, it was the tale of two witnesses: Curtiss Cobb from Meta, who on Day 11 made his best case that the blue sky was red, and John Hegeman, a straight shooter who serves as Meta’s Chief Revenue Officer and sits on the board of Robinhood. We also saw the first few minutes of a deposition video from another TikTok witness, the trial’s fourth. Trial ended earlier, around 4pm, and started later, after Snap came to court to argue about confidentiality. Longer portions of Hegeman’s testimony took place in a closed court, so we have a shorter summary to close out the week.

Re-Entering the Realm of Reality

Meta’s in-house survey chief Curtiss Cobb returned to the stand at the start of Day 12 a new man. Perhaps Meta’s counsel gave him some high level advice to truthfully admit the obvious (though a witness may not discuss the substance of their testimony with anyone while off the stand before the court dismisses them as a witness).

The first exhibit of Day 12 included a chart showing that users place a high value on “friends and family” updates and rate Facebook as high performance for that task. Cobb gamely accepted that characterization “at the time of the study, yes.” (As a reminder, trial exhibits are being posted to this website, with a two-day lag.)

Later in the same exhibit was a graph plotting what Meta calls Relative “Cares About Users” (RCAU), one of the company’s main consumer sentiment metric. As the name indicates, the metric measures how much consumers believe Meta cares about its users. The chart compared RCAU sentiment about Facebook Blue (that is, the Facebook site/app, as distinct from Instagram, WhatsApp, and other Meta products) to that of other companies, including Comcast. What the graph showed was that after peaking in 2015, Facebook’s RCAU score declined to a low as of November 2020, passing Comcast’s low score. “There were times in 2020 where that was the case,” Cobb admitted.

An update from May 2022 showed that Facebook’s RCAU score continued declining to near zero in October 2021, although Cobb added that the “relative” nature of the score means that a zero RCAU equates to around 40-45% “agreement that a company cares about its users.” Still, Cobb admitted that it was “one of the lower marks that we had for” any company, meaning that, according to Meta’s own survey, Facebook was viewed by consumers as one of the worst brands in the United States, if not the worst.

As Cobb explained the score to executives at that time:

“Remember zero is around the lowest we think a company can go and is ascertained as a lower bound benchmark by how people rate the companies they first identify as the worst companies in the U.S.”

On cross, Meta’s counsel—here, Kellogg Hansen’s Kevin Miller—invited Cobb to explain why surveys don’t actually matter that much for explaining behavior. Cobb testified that “[i]t's a well-known phenomenon in survey research that respondents are actually poor at explaining frequent behaviors and how much they do things through surveys.” People’s views about how Meta’s apps work may simply be “contaminated” by negative press about Meta generally, and respondents can’t “disambiguate” their negative views about the brand from their views about the product.

Cobb separately tried to explain how Meta finds these surveys valuable for tracking “memes” or other “emergent” trends that users are reacting to. But it sounded like these two threads were contrary: surveys are unreliable because people can’t be trusted to tell the truth, and yet Meta invests money into this team headed by Cobb that runs “thousands” of surveys a year.

“Typically we don’t find” that the main consumer sentiment tracking survey "reflect[s] that the quality of Meta’s apps has changed,” Cobb claimed. A couple of times he blamed Tucker Carlson for talking about Meta for the decline in user sentiment. A similar but more credible version of this point came in on cross, when Cobb explained that data on “revealed preferences”—how people actually behave—is more reliable than how people people say they behave. It’s a fair point; I’m sure we all tell the doctor that we have healthier habits than we actually do.

But it’s also a point that’s difficult to disentangle from the FTC’s theory, which is that users stuck with Meta apps not because they believed it was the best quality option, but because they had no other options for friends and family sharing. As user sentiment waned, users held on. Does that consumer stickiness show Meta’s monopoly power? Or is it more likely, as Meta suggests, that everyone is simply being untruthful about not liking Meta, concealing a “revealed preference” that Meta is in fact great?

Cobb also testified that “[n]obody likes to admit that they are not wanting to be productive at some moment,” suggesting again that survey respondents are unreliable. The implication is that people would rather say they use Meta for friends and family sharing because it’s a more palatable response than admitting they browse Meta apps to waste time.

Another line of questioning pointed out that liberal and conservative users have different reactions to current events, which in turn changes how they view Meta. Cobb said that the losing side in U.S. presidential elections tends to view Meta negatively after that cycle. According to Cobb, Facebook competes with TikTok and Snapchat, so Meta also looks at CAU metrics for those apps, which are “around the same” or worse than Meta’s ratings. Shown a demonstrative with more recent data, Cobb said that Meta’s RCAU scores have improved since the 2021 low seen on direct.

On re-direct, Cobb fell into his old bad habits as he was impeached with his deposition testimony three more times. At one point Miller objected that it was improper impeachment “cognizant of [the court’s] ruling earlier.” Chief Judge Boasberg replied, “Cognizant of my ruling, overruled.”

The FTC’s Albert Teng also cleaned Cobb out on his testimony that user sentiment affects user engagement. Cobb had to admit that he’s not responsible for tracking engagement. Another line from a survey cast doubt on Cobb’s spin about politics: “We have found previously that political leanings of a state are not related to sentiment, but education and race are.”

More than that, the “top U.S. sentiment driver” on Facebook is “connecting friends and family.” And that might be the most important takeaway from Cobb’s second turn on the stand.

Feeding Frenzy

John Hegeman was the FTC’s next witness of Day 12. Apart from a year-long stint with Quora, Hegeman has been with Meta since 2007. He started on the product side in engineering, rising to lead ads delivery, then the Facebook news feed, and then became a vice president for ads and monetization before ascending to Chief Revenue Officer. Here’s a video of him dispelling some common “myths” about Facebook:

The FTC started by walking Hegeman through this Q&A blog post. The court started out by calling Hegeman’s answers in this post hearsay, and sustained some objections to questions about them. Chief Judge Boasberg got it wrong: these statements were not hearsay because they qualify as party-opponent admissions. After a recess, he corrected his earlier erroneous ruling on this. The point of the questioning was to show that the News Feed “was created to help people see what was going on in the lives of their friends and family,” as Hegeman said in the interview.

The FTC turned next to “ad load”—the proportion of content that a user sees on Meta apps that is advertising. Meta recognized that ad load can reduce user engagement on its apps; Hegeman agreed that in “some cases,” that’s true. The FTC also used Hegeman to explain some basic concepts around “ad load,” which presumably will be useful later for its expert economist.

Here we learned some key terms. Meta has at least two parameters around ad load that can be tweaked to up the share of ads: the gap rule (how far apart ads need to be on a user’s feed) and the position rule (how early in the feed that an ad can appear). There’s also “dynamic ad load,” which allows Meta to increase or decrease the number of ads a specific user sees. There were some questions around “needy users,” a Meta term, but Hegeman didn’t know how that was defined. I expect we’ll revisit these terms later on, but all in all, no home runs were hit here.

On cross, Meta lawyer Alex Parkinson returned. We covered his prior sarcastic crosses from earlier in trial. And that’s how he started off with a witness from his own client, asking what Hegeman’s hobbies were outside of work. The question drew a rare rebuke from Chief Judge Boasberg, who said that he “appreciate[d] the familiarity with the witness” but instructed Parkinson to address Hegeman with the decorum appropriate for the courtroom.

It was another example of Meta forgetting that even a bench trial is a morality play, and it would be better advised to stop having its witnesses and lawyers play the part of bullies that you instinctively want to rule against.

Hegeman testified that Meta doesn’t add as many ads as it can.

On cross, Meta’s goal was to sever a link that we’ve heard about at trial and that will probably be a focus in testimony from the FTC’s expert economist, Scott Hemphill. The FTC claims that Meta can engage in a form of price discrimination, a sign of market and monopoly power, by showing more ads to users who value friends and family sharing more—hence the questions about “needy users” on direct. Of course, Meta wants to show that it doesn’t do this, as Sheryl Sandberg testified earlier.

Asked whether Meta shows more ads on surfaces where there is more friend sharing like the news feed, Hegeman answered that “[w]e don’t do that and I don’t think it would be effective.” Asked whether Meta can show more ads to people who like friends and family content more, Hegeman said that the algorithm isn’t “designed to do that and I haven’t seen evidence that it’s doing that.”

There’s a natural limit to showing more ads because Meta recognizes that showing too many ads could cause a decline in time spent on the apps, making ads less effective and in turn hurting revenue. Globally, it seemed that introducing dynamic ad load didn’t change any metrics like user feedback, time spent, or engagement, and appeared that ad clicks changed by ~2%. There was also a minimum impact to reducing the space between ads (thus increasing the frequency of ads) from every 5 posts to every 4.

Parkinson also established with Hegeman that its ad auction system doesn’t always award space to the highest bidder, focusing on a variety of metrics and placing importance on the likelihood that a user “converts” by taking the action suggested by the ad.

There was also an email chain about the “competitive threat from TikTok” that began with a late Saturday night email from Zuckerberg to a bunch of executives that continued on Sunday with a reply from Hegeman. On the whole, I thought this was a strange email choice for Meta to introduce since it seemed to support the FTC’s friends and family use case.

Hegeman wrote:

“TikTok is clearly in the lead [i]n critical areas . . . However, we still have some large content advantages including friend/group content and a large base of non-video content . . . . it should be a lot easier to get creators of public content to post on Reels in addition to TikTok than it will be for TikTok to replicate our base of differentiated friend/group content.”

The cross exhibits had some other interesting lines:

• “People Problem: Everyone sees (roughly speaking) 1 ad every 9 stories, no matter their personal preference or the demand from advertisers.”

• “In randomized tests ad load has marginal impact to engagement, but no measurable impact to sentiment.” This cuts the other way from the FTC’s theory that events like Cambridge Analytica reduced sentiment but left usage unchanged.

• Flagged on redirect, a line from a defense exhibit: “Ad-load today is 10.15% on mobile feed and 9.3% on desktop feed.”

Hegeman also testified on cross that while Meta had started with a friends and family content focus, it broadened that mission over time to encompass the other types of content it has today from unconnected accounts and short-form videos. On re-direct, though, the FTC pointed out that Hegeman had said this in a prior exhibit:

“Friend content: While clearly growing slower than other markets I think non-messenger friend content (in particular stories and reshares) is still big and will remain big in the future (even if they will be a smaller percent of the overall pie).”

Hegeman also seemed to agree that engagement can be an input into dynamic ad load that varies how many ads a user sees. There were many other points in a skillful re-direct, but those were the big ones. We’ll have to get to Hemphill, the FTC’s economist, to hear the FTC’s view of truth about dynamic ad load and how it can target users who like friends and family content most.

Coda

With just a few minutes remaining in the court day that ended at 4pm, the FTC played the first six minutes of the deposition of Eric Morrison of TikTok. Morrison explained that TikTok was like meeting someone you didn’t know before at a music festival: content creators and consumers are brought together by a common interest. But Facebook and Instagram are apps that you use with “people you already know, whether that’s family, friends, colleagues,” etc. That, of course, supports the FTC’s proposed market definition of a personal social networking service (“PSN”) market and makes TikTok sound like it’s not part of that market, leaving Meta as the monopolist.

As the day ended, consistent with what we reported on Day 11 that the testimony was starting to sound repetitive, Chief Judge Boasberg asked for updated estimates of how much trial time each side has used and expects to use and invited the parties to propose any changes to trial procedures that might expedite things. He said this all in a respectful and nonjudgmental way, but it seems that the court has now heard a lot of the same points (see the four TikTok witnesses) and would like to get on with it.

Big Tech on Trial will be back for Week 4.