MIAMI – Put several media industry professionals on a panel and ask them about the challenges of television and video audience measurement and you’re going to get lots of lively conversation. Along with some very blunt observations about just what’s holding back the progress of “advanced TV.”

So it was at the recent Beet Retreat Miami 2017 when representatives of Tru Optik, Inscape, Nielsen Catalina Solutions and Team Arrow Partners gathered on stage with moderator Joanna O’Connell, VP and Principal Analyst at Forrester Research. While there was overall cordiality, there was no mincing of words when discussing the business and technical challenges of audience measurement.

They ranged from agency compensation that accentuates cheap CPM’s over business outcomes to the speed with which campaigns can be analyzed for return on investment. After summarizing the four core ways to measure TV—panel, set-top box data, automatic content recognition (ACR) data and in-app—Tru Optik CEO Andre Swanston offered these observations on device validation and audience de-duplification:

“If I spent a million dollars with Hulu and a million dollars with NBC and a million dollars with Fox and a million dollars with Crackle, that’s great that I had 100 million impressions. How many households did I reach?” De-duplicating across multiple publishers and platforms is “not anything super sexy or exciting, but it’s some of the basic things that people come to expect across linear and digital.”

Matt O’Grady, CEO of Nielsen Catalina Solutions, cited speed to market as a big challenge. “Campaigns are running longer and longer, or somebody is always on in the marketplace, so our traditional methodology of test and control is just too slow quite honestly. So we’re investing very heavily in new models that are based upon artificial intelligence and multiple models running simultaneously so that we can get faster results and so that we can get true in-flight reads that somewhat mimic what the end result is.”

Then there’s the issue of whose data are deemed to be the most useful, according to Jodie McAfee, SVP, Marketing & Business Development at Inscape. “What we hear a lot of is ‘we think that the legacy data sets and specifically Nielsen data is flawed.’ And then people will look at our data and they’ll go ‘well this doesn’t match up with Nielsen data.’ The same people that believe that those legacy data sets are flawed have business systems and operations historically built around those datasets that you literally would have to practically blow up the entire market just to get everybody to change.”

Jason Harrison, President of Team Arrow Partners, GroupM’s dedicated agency for retailer Target, addressed one of many elephants in the room: how some agencies are compensated. “The idea that you want to target tightly to reach an audience to drive a response but you’re being held accountable to declining CPM’s is wrong, obviously. It’s misaligned. It sets up the wrong incentives.”

More than one panelist took issue with the preponderance of companies that claim to be able to do various things—leading to people at other companies burning time just trying to find the truth.

Said Swanston, whose background is in finance (J.P. Morgan), to much laughter: “I’ve never kind of experienced an industry where there’s so much smoke blown. The advertising and tech. A lot of people are full of it in this industry. And I say that in a nice way.”

Added McAfee: “There are an absolute crap-load of companies that are way out over their skies and we think it does a disservice to the market. We spend entirely too many cycles trying to separate the wheat from the chaff and trying to figure out what’s real and what’s not. As opposed to just doing our day jobs.”

This video was produced at the Beet Retreat Miami, 2017 presented by Videology along with Alphonso and 605. For more videos from the event, please visit this page.