Nielsen is formally announcing its decision to create within its National People Meter panel a “single source” measurement for both television and online consumption of video content. The plan is to roll out Internet measurement to National People Meter households starting December 23, 2009 and finishing by August 31, 2010 (sooner than had originally been proposed to clients).
This move will give them the ability to measure TV Everywhere-type content and other online video business models (however, they are not yet announcing when they will be able to add TV Everywhere into the ratings – this is only about the roll-out of Internet measurement into their TV panel).
Here is the update letter to clients from Sara Erichson, President, Media Client Services North America:
December 1, 2009
Update on TV and Online Video Measurement in Nielsen’s National People Meter Panel
Dear Client,
On October 16th, Nielsen held a special client meeting in New York City at which we discussed plans to measure online viewing to television programs in the National People Meter (NPM) sample. The meeting was designed to gather client input to ensure that our plans were aligned with industry developments.
The purpose of this communication is (1) to provide a summary of the October meeting and (2) to share with you Nielsen’s “moving forward” plan.
As you will read below, a critical element of our moving forward plan is the decision to begin, later this month, the rollout of Internet meters to all households in the National People Meter sample that have Internet access – an initiative we refer to as “TVandPC”. This will enable the measurement of online viewing to television content among the same People Meter households that are the source of Nielsen’s national television ratings. Once the rollout is complete we expect this will result in online measurement from approximately 7,500 National People Meter homes representing about 20,000 people and 12,000 computers. This new single source panel is designed to meet the needs of multiple client business models and will serve as the foundation for a number of Nielsen’s cross platform measurement solutions. It is in addition to the 200,000 computers now measured in our Nielsen Online panel. Accompanying this letter is a document that provides the details of the NPM rollout.
Summary of October 16th Client Meeting
• At the meeting we discussed Nielsen’s “TVandPC” initiative, described above. Since early this year we have been conducting a TVandPC test to determine whether the additional request of Internet measurement had any impact on sample quality metrics such as cooperation and intab rates. The test results have been excellent, with no negative impact seen. Accordingly, we stated our conclusion that we could move forward with the full roll out to the National sample. There was broad client support for this rollout. However, clients stressed that the quality of the National sample could not be meaningfully impacted in a negative way. Discussion focused on the attention that must be placed in balancing speed of deployment with quality. In addition, some clients stated that the original timetable presented at the meeting was too slow and therefore did not meet industry needs. Our moving forward plans, described later in this update, reflect a new, compressed timetable.
• At the meeting we also discussed business models for television content online. In fact, one impetus for the meeting was the recent industry dialogue around initiatives such as “OnDemand Online” and “TV Everywhere”. Many programmers are testing business models that would make television programs available online, at no cost, in the format that the programs were originally presented on television.
These initiatives have received considerable support from cable and broadcast network programmers who see them as a way to successfully monetize the placement of full episode video online.
Importantly, these initiatives are very compatible with Nielsen’s television ratings system; that is, online audiences viewing these programs could be incorporated into the regular Nielsen’s TV ratings (including C3).
We shared with clients Nielsen’s ideas for developing “Extended Screen” reporting. This refers to the development effort that is required to take the online viewing to a particular telecast and merge it with the standard television audience data. This is necessary in order to report a single combined number in
our TV ratings reports and systems. Extended Screen is designed for those telecasts that are presented online with the same program and national commercial content that air on traditional television. It is compatible with much of thinking behind “TV Everywhere” and “OnDemand Online”. A prerequisite for Extended Screen reporting is the full roll out the TVandPC initiative described above.
Many clients expressed support for Extended Screen reporting as it aligned with their business plans. However, we heard that the timetable we initially presented was too slow. Our moving forward plans, described below, will focus on compressing the timetable to the extent possible. At the same time, we heard that not all clients believe in initiatives such as TV Everywhere. Therefore Nielsen cross platform measurement solutions will be designed to support different business models and will be able to capture the multitude of ways in which video content is presented on the web. As previously noted, TVandPC will provide the foundation for many of these applications.
Nielsen’s Moving Forward Plans
• Since the October 16th meeting, Nielsen has been working to adjust and finalize our plans for TVandPC and Extended Screen ensuring that client input has been evaluated and incorporated.
• We are now ready to begin the full roll out of TVandPC in the National People Meter sample. The recruitment process and installation of Internet meters, beyond the 375 test households that already have them, will commence later this month starting December 23rd. Accompanying this letter is a detailed document describing all elements of this roll out including the schedule, the statistical and
methodological research plans, and the field training and recruitment plans. As you will see, the roll out schedule has been compressed. The new schedule reflects a completion date by the end of next August. The document also stresses that the roll out will be monitored very carefully and should quality
concerns arise we will pause to make the necessary adjustments prior to resuming.
• We are near completion of our Extended Screen planning. A second detailed document will be sent to all clients later this month outlining our plans to combine online viewing to national television programs in our television ratings reports. We have been working to compress this timetable as well and will be able to share a new schedule with you at that time.
We appreciate the significant time our clients have spent consulting with us on these initiatives during the last year and particularly during the past few months. We believe it is critical that our measurement services continue to evolve to reflect the changing ways in which people consume media. Client feedback
has been crucial to our planning and decision making process.
Best Regards,
Sara Erichson
President, Media Client Services North America
The Nielsen Company
Excerpts from the rollout details:
TVandPC Initiative –National People Meter Panel Roll Out
TVandPC
The Nielsen Company has historically collected data from sample households within its National People Meter (NPM) panel to measure TV usage and viewing of programs aired over television, delivering, among other data, program and commercial ratings, reach and frequency studies, and demographic composition to clients for program research and advertising buying, selling, and planning purposes. In recent years, technology advancements have enabled consumers to view TV programs on a delayed basis through devices such as DVRs and DVD recorders and also through Video on Demand. Nielsen has enhanced its measurement service over time to include timeshifted viewing as well as viewing that takes places in dorms and off campus housing. Today there are an increasing number of alternate platforms, beyond the traditional television set, where viewers can go to see television content. The most popular of these is the
Internet, where full episode television programs are increasingly common.
The Nielsen Company is now announcing the rollout of its “TVandPC” initiative, which will enable measurement of viewing to television content online among the same National People Meter households that are the source of its national television ratings. Starting later this month Nielsen will begin installing Internet meters within existing People Meter households that have both a computer and Internet access and who agree to include their computer measurement. This measurement will include computer use, URLs visited and video consumption.
The addition of computer measurement will provide clients with a unique and valuable singlesource sample. This sample will enable Nielsen to:
• Enhance currency panels by adding overall computer usage, including video
consumption
• Recognize changing TV consumption patterns
• Proactively monitor and identify shifts in audience demographics
Insights into Internet consumption of traditional TV content, computer-based video viewing, and web surfing will all be available to clients to use in assessing their Internet business models and to help them make informed decisions that will support current and future business objectives.
Schedule
The TVandPC initiative will be rolled out to all National People Meter households starting December 23, 2009 in the field areas used to obtain the 375 panelists who participated in a test of this initiative this year. This represents about 30% of all field areas. The remaining field areas will start recruitment on February 1, 2010 allowing for the proper training of Field personnel following the challenges of the holiday season. The rollout will be completed by the end of next summer – no later than August 31, 2010.
Recruitment
All existing and new specs in the National Sample will be recruited for computer measurement. Households currently metered for TV will be approached at their next Field visit or their normally scheduled six-month Field visit. Participation in computer measurement will be optional. If a household refuses to allow its computer usage to be measured, it will still remain in the panel for television-only measurement.
The NetSight Meter
Computers will be metered with the Nielsen Online NetSight Meter. The NetSight meter is a downloadable utility that tracks web sites visited, pages viewed and time spent as well as video consumed. Another important function of this meter is that it allows panelists to “log in” so their personal usage can be attributed to them, similar to the television People Meter. The NetSight meter has recently been enhanced with the ability to recognize Nielsen’s A/P codes when present, as well as to take audio signatures for comparison to our reference library for video content identification purposes.
Computers Metered
Every attempt will be made to meter all computers with Internet in the home — both laptops and desktops. Any metered computer that is used outside of the home will continue to collect and transmit data for TVandPC measurement. Computers that are not owned by the panelist and, currently, computers with operating systems that are not compatible with the NetSight Meter (Mac, Linux, and older versions of MS Windows, among others) will not be metered. Nielsen is currently developing a Mac-compatible meter and expects to deploy this important enhancement in 2010. Extended Home panelists (i.e., students living in dorms or off-campus housing) will also be recruited for computer measurement and may optionally participate. For extended homes to qualify for measurement, television measurement is required.
Statistical Research Analysis Plan
Since April 2009, Nielsen has been conducting a test of the impact of collecting Internet usage within a randomly selected 375 household panel drawn from the National People Meter sample. During the test, comparisons were made of the quality metrics described below between the Test Group, Control Group and the total sample of Internet households in the NPM sample.
These results have been shared at a number of checkpoints with the MRC, several Nielsen client committees, and at National Client Meetings. The test results (described in detail below) showed no impact on the quality metrics of the NPM sample. Given these results, Nielsen will now begin the rollout of Internet measurement among the remainder of the NPM sample, and will continue to monitor and report on these quality metrics as we proceed.
An analysis plan that will be conducted during the NPM rollout will use two separate sets of analyses. These analyses will be referred to as the 375-Test Household Analysis and the NPM Rollout Analysis.
375-Test Household Analysis Sample Design
An initial sample of approximately 1,465 housing units was selected for TV and PC recruitment in order to achieve a target of approximately 375 installed Internet homes. This sample was a national sub-sample of National People Meter (NPM) housing units, targeting 150 Field Areas with approximately 9-10 housing units each.
Results to Date
Since April 2009, Nielsen has been conducting a test of the impact of collecting Internet usage among the National People Meter (NPM) sample households. One objective of the test was to estimate the effect of recruiting NPM households with Internet access and installing Internet measurement software on each PC in the home. The following metrics were monitored to quantify the effect:
• Set & Person In-tab Rate
• Quit Rate
• Number of Viewers Logged in per Tuned Minute
Each metric was compared among the following groups:
• Test Group: a randomly selected sub-sample of the NPM sample Internet households
• Control Group: remainder of the NPM sample Internet households
• Total Group: combined Test and Control Groups
If, during the test, any deleterious difference had been detected when comparing In-tab Rate and Quit Rate between the Control and Total Groups or Viewers per Tuned Minute between the Test and Control Groups, the test would have been paused. The pause would have allowed Nielsen to determine the cause and develop corrective action before proceeding with the recruitment and installation of the Internet measurement software among the Test Group.
However, a pause was never required since there was no deleterious effect of the test shown.
Another objective of the test was to quantify the accept rates and cancellation rates related to recruitment and installation of the Internet measurement software among the Test Group. Nielsen employed additional analyses to examine any differences in the types of households that participated in the Internet measurement compared to those that did not participate.
As of November 1, 2009, 48.9% of the Test Group households that were recruited for the TV and PC measurement agreed to participate and installed the software. Further, 2.8% installed the software but later asked that the software be removed.
Households installing the software are more likely to be age of head 65+, non-cable, have 3 or more sets, have income less than $75K and a householder that is not a college graduate. Computational adjustments will be employed, if necessary, when reporting their Internet usage.
Analysis Plan Going Forward
As the rollout progresses beyond the initial 375 households, Nielsen will monitor the rollout as a whole and also continue to monitor the 375-household test panelists over time to ensure that there are no longer-term deleterious effects that haven’t shown in our analyses to date.
In monitoring the rollout as a whole, Nielsen will compare quality metrics among households that have been approached for TV and PC measurement (Test Group) and households that have not yet been approached (Control Group). The Test and Control Groups will change through the rollout as each household that is recruited will move from the Control Group into the Test Group. Because the Test Group is not a random sub-sample of the National People Meter sample, additional weighting will be required for the Rollout Analysis when comparing Test vs. Control or Control vs. Total. Weighting will be required to adjust for differences in demographic and geographic distribution between the two sub-groups. This analysis will begin in February 2010.
The following plan describes the analytical approach and statistical testing that will be conducted for both the 375-Test Household Analysis and the NPM Rollout Analysis. Differences in the approach between the 375-Test Household Analysis and the NPM Rollout Analysis will be described for each metric being measured.
Metrics will be compared among the following groups:
• Test Group to Control Group or
• Control Group to Total Group (Combined Test and Control Groups)
As described above the definitions of the Test and Control Groups differ between the 375-Test Household Analysis and the NPM Rollout Analysis.
In-tab Rate
In-tab rate will be computed weekly as the sum of weighted households passing set and person edits divided by sum of weighed installed households. Comparisons will be made between the Total Group and the Control Group to identify meaningful and statistically relevant differences. The 375-Test Households are a random sub-sample of the total NPM sample and can be compared to the total sample using weights to adjust for disproportionate sampling in the local People Meter markets. However, this is not the case for the Test and Control Groups throughout the NPM Rollout Analysis. This analysis will require daily weighting of the Control Group installed Internet homes to adjust for differences in the demographic and geographic distribution compared to the total sample.
The following characteristics will be used for weighting: ADS, Cable, Education of Householder, County Size, Number of Sets, Presence of Non-Adults, Age of Householder, Household Size, Hispanic Ethnicity, and Geography. Controls may be collapsed or eliminated pending sufficient sample sizes.
Quit Rate
For the 375-Test Household Analysis, quit rates will be reported monthly, and will reflect the cumulative quit rates since Internet measurement recruitment began. A significance test will be performed comparing the Test Group to the Control Group. Further, the Total Group will be compared to the Control Group.
Quit rate cannot be compared for the NPM Rollout Analysis due to the Test and Control Groups changing daily as households are recruited throughout rollout. However, changes in the quit rate will be reflected in the SPI. As such, we will monitor overall SPI compared to historical levels.
Beyond monitoring SPI, quit rate for the total Internet sample during the current month will be compared to levels from previous years. For example, to evaluate quit rate for February, a comparison of the metric for February of the current year will be made to the same metric for February 2005-2009. Quit rate will be computed monthly as the percent of Internet homes installed on the first day of the month that left the panel by the last day of the month.
Number of Persons Viewing per Tuned Minute
For the 375 Test Household Analysis, the number of viewers logged per tuned minute by household size will also be monitored. Analysis will be performed comparing the average number of viewers logged between the Test and Control Groups to identify meaningful and statistically relevant differences.
For the NPM Rollout Analysis, the same comparisons will be done. However, the analysis will require daily weighting of the Test Group in-tab homes to the demographic and geographic distribution of the Control Group in-tab homes. The following characteristics will be used in the weighting process: ADS, Cable, Education of Householder, County Size, Number of Sets, Presence of Non-Adults, Age of Householder, Household Size, Hispanic Ethnicity, and Geography. Controls may be collapsed or eliminated pending sufficient sample sizes.
Recruitment Methodology
All homes will be recruited in-person consistent with all NPM recruitment. Already-installed TV homes will be approached by the Field Representative. New Homes will be approached by the Membership Representative with the message that participation will be in Nielsen TV Ratings and in computer research. All training and coaching materials have been updated to reflect the inclusion of computers in our data collection. (Recruitment processes and the training and coaching materials already exist as they were developed and utilized in the 375 household test. These same processes and materials are now being used for the full rollout.)
Methodological Research will provide ongoing quality monitoring (field staff interviews, refusal report forms) throughout the rollout and for the first six months following the full rollout. The monitoring provides the ability to identify and implement necessary real-time changes or enhancements to existing procedures.
Staffing
Based upon knowledge gained from the 375 test, additional resources will be needed to maintain households installed for TV and computers. These additional resources will be added as needed through the rollout.
Training
In April 2009, field and membership representatives from approximately 150 field areas were trained in person on TVandPC procedures for the Test 375. The training approach proved to be very successful and will be used as the basis for training the remaining Field force. The Field has scheduled in-person training sessions for all remaining People Meter Field and Membership Representatives for the two weeks prior to February 1, 2010. (As noted above, until such a time as the remaining field force is trained, all recruitment and installation will continue to be done by the field and membership reps already trained.) Training will include recruitment, installation and maintenance of TVandPC households.
Recruitment
In rolling out this initiative we will recruit households on the next visit or the next scheduled call to the home. To mitigate impacts to holiday recovery, January 2010 will be an optional month for recruiting households with scheduled calls due in that month. The Field will use their professional judgment for recruiting those households with scheduled visits in January.
Otherwise, these households will have computer recruitment completed by their July scheduled visit. The full TVandPC rollout will be guided carefully by the same principles that guided the 375 household test. Should the monitoring of the quality metrics show a material impact on sample quality, the rollout will be paused and adjustments will be made prior to continuing.
