- REUTERS/Kevin Lamarque
Since becoming a regular on television as President Donald Trump’s campaign manager and later top counselor, Kellyanne Conway has raised doubts about the veracity and credibility of her answers.
But while some TV hosts like MSNBC’s Mika Brzezinski say they no longer trust Conway’s answers enough to put her on their shows, they aren’t the first to reject the Trump counselor for a lack of credibility.
In 2007, Conway – then a Republican pollster running the research firm The Polling Company – was hired as an expert witness by Whole Foods in a lawsuit filed by the Federal Trade Commission in Washington.
The FTC sued Whole Foods that year to block the upscale grocery chain’s purchase of Wild Oats, a North American grocery store chain known similarly for its organic products, arguing the merger would reduce competition for organic goods and increase food prices in several markets. To support this, the FTC produced a series of internal company emails and public blog posts that showed the company’s desire for Wild Oats’ “monopoly” over its local markets.
During the case, Whole Foods retained Conway to design a survey to support the testimony of David Scheffman, a former FTC official with experience testifying on behalf of companies in antitrust suits. He argued that Whole Foods and Wild Oats shoppers frequently shopped at other grocery chains and that these other supermarkets competed for the business of crossover customers.
In other words, he was arguing that Whole Foods and Wild Oats couldn’t raise prices without losing customers to competing grocers, voiding the FTC’s concerns about a monopoly.
But it didn’t exactly pan out for Conway or the mega-supermarket chain.
The DC District Court threw out Conway’s survey, saying it would not give “any weight or consideration” to it because of its methodology.
The court found that the FTC’s expert witness, Kent Van Liere, presented compelling evidence about the study’s drawbacks, which the agency said suffered “serious and fatal design flaws, poor execution, and conclusions that are inconsistent with the survey’s results,” and failed to “meet basic standards” of survey research.
“My overall opinion in this matter is that Ms. Conway’s survey methodology and procedures are fundamentally flawed and render her data and results unreliable,” Van Liere testified, according to July 2007 court documents. “In addition, it is my opinion that her survey does not provide a reliable basis to assess the issues associated with consumer perceptions of the substitutability of products and services across food retailers.”
Van Liere said Conway’s response rate was “so low that her results cannot be considered reliable” and included “unqualified respondents” who could not “understand or could not complete the questions accurately.” Her response rate, according to Van Liere, varied “from 6 percent to 0.5 percent” of callers the pollsters attempted to reach, and over 20% did not reside in the zip codes designated in the survey.
The court agreed that the survey was too complicated.
The FTC argued that survey respondents said they shopped mostly at Whole Foods or Wild Oats, but responded to later questions saying they “purchased most or all of the same product categories from other food retailers.”
“Inconsistent survey results reveal the presence of underlying flaws in the survey design itself, such as the presentation of questions that are confusing, complex, and poorly worded, which here led to unreliable results,” the FTC argued in its July memorandum in support of excluding Conway’s testimony.
Further, court documents said that Scheffman didn’t ultimately rely on Conway’s report, despite it being conducted on his behalf and cost Whole Foods over a quarter of a million dollars, according to court documents filed in July 2007.
Despite the dismissal of the survey, some experts said that the court was probably too harsh on Conway’s survey.
“It makes sense that you would have to cast a very wide net to find enough people who were somewhat regular shoppers at these stores, so the response rate doesn’t necessarily bother me,” Tom Jensen, director of Public Policy Polling, told Business Insider. “I’m used to that kind of thing if we’re trying to survey people who listen to a particular radio station, for instance.”
But Jensen said the results suggested the court was correct that the survey was confusing.
He said it was “very curious” that Conway’s survey claimed to have interviewed respondents who were in the designated zip codes, yet included respondents who didn’t live in the zip codes.
“The survey is unreasonably complicated for most poll respondents to be able to reliably respond to,” Jensen said. “There’s no way people know off the top of their head how much they spend at different grocery stores on a monthly basis for different categories of products, so I agree that it’s a stretch to make very broad assertions on that data.”
- Chip Somodevilla/Getty Images
The court’s rejection of the survey wasn’t a rebuke of Whole Foods’ position writ large – the court found the testimony from another Whole Foods expert witness, Jon Stanton, relevant in its description of grocery-store categories.
The case itself became less relevant as other grocery stores cashed in on the organic-food trend and Whole Foods attempted to lower prices on some goods as consumer spending dipped during the beginning of the recession in 2008. The two parties settled in 2009, with Whole Foods agreeing to shut down 12 Wild Oats stores and one Whole Foods and sell the locations that had already been closed.
The settlement may have been a blessing in disguise for Whole Foods – according to a 2009 Reuters report, many of the stores Whole Foods was forced to sell were not high-performing locations.
Conway’s political career has largely been dedicated to polling.
Before managing Trump’s campaign, Conway for decades worked in Republican circles as a pollster, with a focus on female voters. Her group, The Polling Company, made almost $2 million conducting surveys for candidates like Sen. Ted Cruz, former presidential candidate Ben Carson, and Trump during the 2016 election – double the amount than any previous campaign cycle.
Poor polling clearly frustrates Conway – The New Yorker reported that during a presidential debate in September, Conway chided an assistant who was tweeting unscientific polls from the then campaign manger’s Twitter account.
Conway, Whole Foods, and The Polling Company did not respond to Business Insider’s requests for comment.