Nielsen recently announced new investments in our core media measurement panels in the U.S., including expansion in many local markets and the introduction of mobile measurement in our national people meter (NPM) panel. People have asked me why we’d do this, in a world that is increasingly digital and fragmented. The answer is actually very simple: panels are still fundamental to measurement, though their role is evolving in an increasingly digital world.
In traditional media measurement, panels were used for direct measurement. The panel was recruited to represent the media universe, panelist behavior was recorded (via diary or electronic means), and that usage was projected to the universe to create ratings. That methodology is still used in our TV measurement today. At a national level, the Nielsen panel includes more than 21,000 households and 50,000 people, and in about half of those homes we measure both TV and computer usage electronically. As the core sample for national TV ratings, which are used to transact some $70 billion in advertising annually, it is overseen by the Media Ratings Council (MRC) and audited on an ongoing basis by the public accounting firm Ernst & Young. The NPM panel is widely considered the highest quality sample of media consumption in the world.
That said, in an era of audience fragmentation, direct measurement in a panel is challenged. As consumption moves to digital devices, and advertising is dynamically served (i.e., different people get different ads), the sample size required for direct measurement becomes cost prohibitive. What is required is the application of digital measurement methods, what we call a census-based approach, in which the data needed for measurement is collected from a very large proportion of the full user population. But if the data needed for measurement can be collected from the population, why do we need a panel at all? The reason is that while census methods such as tags or set top box data can tell us what is being consumed, they do not tell us the who. Nielsen has solved for this by partnering with Facebook and Experian for access to large, high-quality datasets that we can access in real time to help determine the audience. But even this is not completely adequate, as all third-party datasets have built in inconsistencies and biases. At the end of the day, even in a world of big data, proper audience measurement requires an observable sample of the population in order to be confident in the who – i.e., a panel.
In other words, while direct measurement means sample size is critical, in census-based measurement quality takes on crucial importance. In this world, what do we mean by quality? Nielsen believes the following are most critical:
- Validation – In line with the key point made above, methods must be employed to ensure that the panelist is actually who the machinery says they are. The Nielsen approach to recruitment and panel management are face-to-face; we are invited into panelists’ homes, we know exactly who they are and a great deal about them, and we keep that data up to date during their term. We also employ our people meter technology, which tells us who is consuming media at any given time on the measured device.
- Representativeness – The sample must accurately represent the measured universe in terms of geography and demographics. Nielsen uses an area probability recruitment approach that ensures, at a city-block level, that the panel is properly representative, and we conduct regular maintenance to ensure strict adherence among panel households to the protocols necessary to maintain that representativeness.
- Consistency – For accurate measurement, panel turnover must be properly managed. Having panelists go in and out of during and across measurement periods is highly problematic. Nielsen carefully monitors panelist participation in the NPM panel, and enforces turnover after two years in the panel to manage “panelist fatigue.”
- Comprehensiveness – In an era of cross-device usage, the panel must capture behavior and usage across all media devices. This is why Nielsen introduced computer measurement in the NPM in 2010, and recently announced the rollout of smartphone and tablet measurement across the NPM over the next 12-18 months.
To enable quality across these dimensions, a convenience recruiting approach such as online intercepts or browser toolbar downloads simply will not do. Only a robust process that directly approaches potential panelists and spans the full tenure of participation can accomplish this.
Finally, it’s important to understand why data quality is so important. As with traditional media, audience data is increasingly being used as a transaction currency for digital and cross-platform brand advertising. Digital display-related advertising reached nearly $13 billion in 2013, and continues to grow at double-digit rates. With this amount of spend on the line, advertisers and agencies will expect the same quality in the digital currency as they have today in traditional media:
- Fidelity – When trading on age and gender, clients need to know that those data are as precise as possible. Even when using third party data providers, as we do with Facebook for our own Online Campaign Ratings (OCR) product, the data needs to be calibrated to a representative, valid, consistent and comprehensive sample.
- Stability – Audience data needs to be consistent within and across campaigns, which is difficult to ensure if the sample has high turnover.
- Accreditation – As in traditional media, proper and ongoing auditing of panels and process and ultimately the review and accreditation of the MRC provide confidence that the data can be used as currency.
As we move into the cross-platform and digital world, delivery of accurate, stable and accredited data takes on increasing importance. To get there, we will continue to need a valid, representative, consistent and comprehensive view of audiences, which is why high quality panels continue to be of primary importance.
