At Nielsen, we believe that our panels make our company stand out. We devote a great deal of time and resources to ensuring that our panels produce high-quality data. By combining big data with smaller data sets from carefully chosen and measured households, we believe that we provide a higher quality of insights than anyone else in the marketplace.

Behavioral methods data scientists care about the panelist experience because a great panelist experience leads to our best data collected, which leads to delighted clients. And to achieve that, we think of what the entire panelist experience is like. Was the registration survey easy to fill out? Is participation rewarding? Does the panelist feel like they are shaping their world? At Nielsen, our behavioral methods data scientists gather insights into our panelists, their experiences and devise ways to optimize data quality.

Many of the behavioral researchers who work in Nielsen’s data science department have in-depth training in survey research, psychology, sociology and anthropology, among other fields. Some of these researchers focus on scientific and systematic collecting of information through methods such as focus groups, in-depth interviews, surveys, observation and usability testing. Using these qualitative methods, or methods focused on words and experiences rather than numbers and large-scale patterns, provides an understanding of panelist experience and identifies ways our data scientists can improve Nielsen’s measurement. Next, we’ll review the variety of ways research is conducted for Nielsen’s world class measurement.

Observation
Behavior researchers use observation to see how people interact with meters, surveys and other measurement tools. Observation is perhaps the most basic data collection method in the behavioral sciences. It’s valuable because it bridges the gap that can exist between what humans say and what they do. Think of when you visit your dentist—have you ever lied about how often you’re flossing? Or perhaps you know someone who stretches the truth about how much they exercise when their doctor asks. People commonly misreport their behavior for reasons such as: having incorrect memories of an event, wanting to paint a positive picture of themselves by omitting “negative” behaviors, and wanting to please others by saying and doing the “right things.” This leads research participants to distort the truth. Observation allows scientists to see what people are doing and overcome the errors of social desirability.

Surveys
Surveys are useful for understanding beliefs, behaviors and attitudes in large groups of people and for collecting characteristic information. They can help to reveal patterns that are not always obvious in smaller data sets. They’re also useful for gathering information on sensitive subjects, such as topics that people might consider private, or for gathering information to compare groups of people and their behaviors over time. Nielsen uses surveys to understand opinions and behaviors of current and former panelists. Among our surveys, we’re most well known for our radio diaries, which survey people about the shows, stations and programming they enjoy. We are also well known for our Scarborough service, which surveys people about their print media, local television, shopping, and radio listening habits, to name a few.

Usability testing
Usability practitioners conduct testing to understand how people use websites, tools, and technology. Think about the last time you paid a bill online. Was the experience pleasant? Was it easy to navigate to the bill payment section of the site? Did you feel like the website looked trustworthy? If you answered yes to all of those questions, you experienced a well-designed site. Unfortunately, many websites are confusing and challenging for regular people to navigate. Usability testing is important because it involves watching real people use products and websites. Observation, combined with researcher questions, help designers understand how to improve their products. Usability testing has been used to upgrade Nielsen’s computer and mobile panel web page, as well as our mobile app.

Interviews
In-depth interviews consist of an interviewer and interview participants and typically last between 30 minutes and an hour. A well-run interview session is like a great first date, with the interviewer making the respondent feel comfortable while getting to know a great deal of detail about one or more areas of their life. Interviewers are trained in active listening, on-the-spot data analysis, and in asking probing questions. Interviewers follow pre-written lists of questions about one or more topics of interest. Data scientists might ask about pain points that panelists experience as part of their participation, as well as about the benefits of participating in a panel and exit interviews with individuals who are no longer in a panel.

Focus groups
Focus groups are designed to gather a range of opinions on one or more topics. Focus groups are typically made up of six to eight participants and a moderator trained in facilitating group dynamics and in asking open-ended questions. We use focus groups when we want to understand how a group of people might talk about an issue. Skilled moderators make participants feel like they’re chatting with friends. The benefit is that the conversation will remain on the topic(s) of interest and can show diversity of opinions or that most people agree. Focus groups typically last between one and two hours and can involve questions, surveys, activities and idea generation. Panelists typically do not participate in focus groups, but they may be invited to participate after their panel participation is over to give feedback on future directions for research.

These methods allow us to better understand why and how the data we gather is generated. By talking to panelists, surveying them and observing their behavior, we can identify patterns that participants may not be able to share with Nielsen. We can also pinpoint sources of measurement errors, such as survey questions that may be unclear to participants, or difficulties with meters that may lead to reduced compliance.

Though the results of these initiatives generate smaller data sets that we don’t always provide to external clients, they shape the choices and decisions we make internally and help ensure that the voice of our panelists affect the measurement decisions we make at Nielsen. This ensures that the data provided to our clients is top notch—the science behind what’s next.