by

Jongpil Hong

Doctoral Student

jphong@mail.utexas.edu

Department of Advertising

College of Communication

The University of Texas at Austin

Austin, Texas 78712

and

John D. Leckenby

Everett D. Collier Centennial Chair

in Communication

john.leckenby@mail.utexas.edu

http://uts.cc.utexas.edu/~tecas/

Department of Advertising

College of Communication

The University of Texas at Austin

Austin, Texas 78712

paper submitted to

1997 Annual Conference

American Academy of Advertising

St. Louis, Missouri

This paper examines the concepts of reach and frequency in the Internet media environment. Data were collected to provide input to three reach/frequency estimation methods and to provide a benchmark via tabulated schedules of Internet vehicles. Resul ts show that for the three models studied (Beta Binomial Distribution, Morganzstern Sequential Aggregation Distribution and the Conditional Beta Distribution) all perform less accurately in the new medium than they have in magazine and television environm ents. In some cases, however, the results are comparable to those found in other media. This study concludes it may be necessary to develop new models or modify existing models to meet the special requirements of the Internet medium. Since the data ba se is small, this study should be viewed as a pilot for larger-scale work (more subjects in audience measurement data base and more test schedules) before definitive conclusions are drawn.

Reach and frequency estimation of media schedules for media types such as magazines, spot, network and cable television, spot and network radio, newspapers, and other standard media types have become the common method of operation for media dir ectors and planners in advertising agencies over the last 50 years or so. These estimations are conducted using probability models of proprietary and non-proprietary natures.

All indications point to an increasing trend in the usage of reach/frequency models as seen in studies of the practices of media directors in the top 200 advertising agencies in the U.S. In 1982, 87.9% of these agencies used estimation models for reac h and 74.5% used them for the frequency distribution estimation. The corresponding figures for 1993 were 90.5% for reach and 87.3% for the frequency distribution (Leckenby and Kim 1994). While these increases in 1993 are not dramatic, it is clear the mo de of operation in media planning continues to rely upon reach/frequency estimation and the trend is upward rather than downward in usage.

With the advent of the new Internet World Wide Web (WWW) as a medium for advertisers, the issue arises as to the applicability of the standard operating approach in media planning as applied to this new medium. Are the twin concepts of reach and frequ ency useful tools for the building of WWW media schedules? If so, do the methods developed over the past 40 years in other standard media types work when used to estimate reach/frequency for this new medium? Or will entirely new methods need to be inven ted? This question assumes importance because of the rapid growth of this medium and its projected consequences.

The Internet has been the fastest growing medium in the last few years. User demand on the Internet has experienced dramatic increase since its launch. The approximate number of total Internet users is 32 million (Forrester Research 1996). The Inter net growth rate in domain and host numbers internationally also supports the notion of dramatic increase in demand on this medium. For example, between July 1995 and July 1996, the Americas experienced 138 percentage growth in domain numbers with a total of 8,224,279 hosts. Other countries experienced a more dramatic increases: 686 percent for Malaysia and 521 percent for Indonesia (CyberAtlas, 1996 at URL: http://www.cyberatlas.com/news.html). The explosive increase of the Internet can also be demonst rated by the number of downloads of the two most popular web browsers: Netscape and Internet Explorer. According to Netcape (1996 at URL: http://www.netscape.com/), its web site receives more than 85 million hits per day.

While the first question is outside the scope of this paper, it is clear that history is on the side of the application of the "old tools" to the "new media." When radio and television came along as new media available to advertise rs, many of the techniques in reach/frequency estimation as well as the terminologies of magazines and newspapers were applied to these new media. As noted earlier, 87.9 percent of media planners apply reach/frequency models to schedules consisting of se veral different media types (Leckenby and Kim 1994). It also worked the other way around; gross rating points (GRP) has come to be applied to magazines and other media when invented for the new medium of television. History points toward an interactive relationship between media planning applications in the "old" and the "new" media.

The second question serves as the focus of this paper. Standard media reach/frequency models developed for magazines, television and other media can be applied to the new WWW medium. The main question concerns how accurate these old methods may be in the new media environment. Since no data are publicly available of the type needed for such a study, original data need to be collected on WWW site audiences first and then used as input and benchmarks for accuracy of the old models in the new medium.

This approach will serve as the basis for this examination of Internet WWW reach/frequency planning.

Three models which have served as the basis for performance comparisons in magazine and television media schedules, among others, will be used as the estimation methods in this study of Internet WWW reach/frequency: (1) Beta Binomial D istribution (BBD); (2) Sequential Aggregation Distribution (SAD); and (3) Conditional Beta Distribution (CBD).

These three models have been studied extensively and are selected to represent the spectrum of methods available for reach/frequency estimation (Chandon 1976; Danaher 1988a,b; Danaher 1989; Headen, Klompmaker and Rust, 1979; Hofmans 1969; Ju 1990; Kishi 1987; Leckenby and Kim 1992; Lee 1988; Rice 1985; Rust 1986; and Rust and Leone 1984). In addition, many of these approaches have been available for use in proprietary formats for several years (Lancaster 1987; and Telmar 1980).

This is one of the oldest known models which was developed by Richard Metheringham in the 1960’s (Metheringham 1964) for use in advertising agency media planning. It is the simplest of the models studied here and, therefore, has been frequentl y used in practice (Leckenby and Kim 1994). It is also the least accurate, generally speaking, of any of the known models except the binomial distribution (Leckenby and Ju 1990). But it serves as a benchmark for more complex models and may be appropriat e in certain media situations. If the Internet exhibits low between-vehicle duplication, for example, then this method may be appropriate. The main problem in the BBD lies in its estimation of between-vehicle duplication (cross-pair duplication) which r esults in low overall estimation accuracy.

This method uses Morgenzstern’s reach formula to estimate reach (Chandon 1976). Then, each vehicle’s marginal probability distribution is developed using the BBD separately for each vehicle. This overcomes the problem of the BBD used alone co ncerning between-vehicle duplication. These marginal distributions are combined sequentially to form a two-dimensional joint exposure distribution which is collapsed at each step along the main diagonals to form a marginal distribution to combine with th e next vehicle’s marginal distribution. It is known that different order of vehicle aggregation produces different results (Lee 1988). This method is often used in practice (Leckenby and Kim 1994) and is very accurate (Leckenby and Ju 1990) if theoretic ally inelegant.

This method was developed by Kim (1994) to overcome some of the theoretical problems of the Sequential Aggregation Distribution method. It combines the use of the Canonical Expansion method of Danaher (1988a) and the Beta Binomial Distribution used as conditional distributions. It uses the BBD as the basis for all estimations but in a model which uses the BBD for separate parts of the modeling for between- and within-vehicle duplication. It has proven very accurate in magazine schedule testi ng (Kim 1994). And, unlike the DMDLK (Leckenby and Kishi 1984), this method is applicable to all schedules whether they have equal or unequal insertions in vehicles. Unlike SAD, this method is a true multivariate probability distribution and, therefore, behaves in predictable ways.

The history of such studies shows that, most frequently, input data for the models and benchmark data for accuracy assessment of those models are syndicated data such as those developed routinely by Simmons Market Research Bureau, A.C. N ielsen or, prior to their demise, Arbitron, Inc. Unfortunately, the data currently available for the Internet WWW do not provide the type of information needed for a full test of reach/frequency model performance.

Currently, data are available on WWW site audiences relating to total hits and its variations (URLs: http://www.ipro.com; http://www.webtrack.com; http://techweb.cmp.com/techweb/ia/features/history.html). In addition, the concept of "cookie s" has provided a glimpse of individual user tracking on sites (Barr 1996; Liberatore 1996; Mills 1996). What is needed for reach/frequency models of the type to be tested here are data on "unique users" of a site on two subsequent occasio ns. These data would allow the establishment of within- and between-vehicle duplication as well as average site audience. Hong and Leckenby (1996) showed how to estimate unique users as a function of total site hits in a given time frame; however, this estimation process cannot take the place of actual unique user measurement since that process provides no estimate of duplications.

Because of this situation, the authors undertook a small pilot study to collect data on WWW site audiences of the type needed to conduct full-scale tests of the models. Because of the limited audience demographics and sample size, these data are m eant to be illustrative rather than definitive as a means for testing the models in this study.

During the Fall of 1996, 92 students in advertising classes at a large southwestern university were asked to complete a group-administered questionnaire which closely follows the ‘Recent Reading’ method of magazine audience measurement as emplo yed by MRI, Inc. (Mediamark Research Inc. 1984).

In this methodology, two questions are asked after the respondents were shown, via computer projection, the "front page" of each of the top 30 WWW sites with respect to current advertising revenues as compiled by Jupiter Communications i n Fall 1996 (URL: http://www.jupiter.com/):

These questions were asked of the sites after each was shown on the large-screen computer projection system for 15 seconds each. They were shown in alphabetical order. This constituted the Phase I data collection process.

Seven days later, the process was repeated for Phase II of the data collection. In this phase, the sites were shown in reverse alphabetical order to control for order effects. The measurement took about 20 minutes in each phase of data collection . Respondents provided their ID number as the basis for questionnaire matching on the two occasions.

Calculation of audience statistics was based upon the definition used by SMRB (Simmons Market Research Bureau 1989). The Average Site Audience was calculated as:

The Cumulative Site Audience was calculated as:

The Between-Vehicle Duplication was calculated as:

These data were divided by 92 (measurement sample size) to provide estimates of the audience proportions which are input to the media reach/frequency estimation models. To qualify as a "viewer," the respondent must have answered & quot;Yes. I am sure I did." to Question #2 above.

The above three media vehicle statistics for each of the 30 sites served as input data for the three model estimations. One site (iWorld) was dropped since its audience was non-existent over the two phases of measurement. Table1 and Table 2 summa rize the results of the average site audience, the cumulative site audience, and the between-vehicle duplication for the 30 sites.

Because this is a pilot study, 28 schedules of WWW site vehicles were developed to show a reasonable test of the models’ efficacy. Table 3 shows the distribution of these 28 schedules by number of vehicles. The vehicles each contained two ins ertions to be compatible with the tabulation and measurement systems over Phase I and Phase II. There were four schedules each of sizes 2, 3, 4,5,6,7 and 8 vehicles, respectively. This provided exposure distributions in range of size from 4 insertions t otal to 16 insertions total.

Tabulation involves, for each schedule, counting person-by-person exposure to each of the vehicles on each of the two measurement phases. This results in the "true" answer for the sample of exposure to the schedule vehicles over two occa sions. This shows, in a two-vehicle schedule, for example, the proportion of the sample exposed no times, one time, two times, three times or four times to the vehicles in the schedule. Notice that measurement is about the general site and not specific renditions of the site.

The reach and frequency distributions for these schedules were then estimated using the three models. The performance of the models is assessed using standard error criteria described below.

Definition of Error

In evaluating performances of different models, their accuracy depends partly on the manner in which error is defined in the study. In this study, two different error factors, error in reach estimation (AER) and error in the exposure distribution (APE ), are adopted from previous studies (Kishi and Leckenby 1982; Leckenby and Kishi 1984). Danaher (1991) also used these definitions of AER and APE (he renamed AER as RER, "relative error in reach," and APE as EPOR, "error in exposure probab ilities over schedule reach").

The error in the reach estimates for the test schedules was defined as the absolute value of the difference between the observed and predicted reach in terms of percentage.

The error in the each exposure level is simply defined as the absolute difference between the observed and the estimate frequencies.

Tables 4 and 5 provide two example schedules (a small—3-vehicle, and a large—8 vehicle are shown) to give some sense of the typical shapes of the exposure distribution and the magnitude of errors for each schedule. Typically, the distributions are more complex than those found in magazines or television. The primary contributing factor is not the high cumulative audience which contributes to bumps at even-numbered frequencies (this is observed in other media types as well), but rather the low between-vehicle or cross-pair duplication because of the tremendous number of sites available to the viewer.

Table 6 shows the Average Error in Reach results for the sample of 28 schedules. Clearly, of the three models, the Beta Binomial Distribution produces the lowest error, followed by CBD and MSAD. This is contrary to previous findings in other medi a types where the BBD usually produces the poorest results in accuracy. Note the BBD and CBD are not very far apart on AER.

The Average Percentage Errors in the Distribution (APE) are shown for the two sample schedules in Tables 4 and 5.

The overall results for APE for the sample of 28 schedules are shown in Table 6. These errors, again, are relatively large compared to other media types. This can be seen by examining Table 7 for other magazine and television study results on som e of the same models studied here. While in other studies, the CBD shows about a 3 percent AER and 14 percent APE, in this study the AER is about 7 percent and APE shown in Table 6 is about 35 percent, considerably higher.

This is the first study of which the authors are aware which formally has studied the estimation characteristics of existing reach/frequency models in the Internet medium environment. In this sense, this study is pioneering.

This study has examined the performance of existing reach/frequency estimation models for a sample of 28 Internet WWW media schedules ranging in size from two to eight vehicles and four to sixteen insertions. Results show that the models do not perfor m as well on Internet data as they do for either magazine or television data.

Within the study, it is clear the Conditional Beta Distribution (CBD), overall, outperforms the Beta Binomial Distribution (BBD) and the Morgenzstern Sequential Aggregation Distribution (MSAD). This is an unusual finding since invariably, the MSAD has performed quite well in other media environments.

The data here are based upon a very small sample size of college students. This group is appropriate for study in this medium since it is known to be upscale educationally. But the small number of respondents in the data collection phase of this study and the small number of schedules examined undoubtedly have led to some bias. Other studies have used larger schedules to serve as the basis for error statistics; all indications point to a larger error than observed had that been done in the curre nt study since, usually, larger schedules are harder to estimate and lead to larger error in other media types.

Results of this study, though preliminary, point to the need to develop new or modify existing reach/frequency methods to accurately model the Internet WWW audience.

Table 1

Top 30 sites selected in this study and Top 30 sites reported by Interactive Age

	Top 30 sites in this study (Average Audience in %)	Top 30 sites based on Number of Unique Users *
1	Netscape (50.5)	Netscape
2	Yahoo (45.1)	ESPNEet SportsZone
3	Lycos (19.6)	Lycos
4	InfoSeek (18.5)	Yahoo
5	Excite (15.8)	Pathfinder
6	Magellan Internet Directory (14.7)	Playboy
7	WebCrawler (12.5)	InfoSeek
8	ESPNET Sports Zone (12.5)	Penthouse
9	CNN Interactive (8.2)	HotWired
10	NBA.com (Starwave) (5.4)	Novell
11	Cool Site of the Day (4.4)	Digital Equipment
12	Playboy (4.3)	Microsoft
13	Discovery Channel Online (3.8)	Global Network Navigator
14	USA Today (3.8)	Electronic Newstand
15	U.S. News Online (3.8)	First Virtual Bank
16	C|net: the computer network (3.3)	Times FAX/New York Times
17	TradeWave Galaxy (3.3)	Internet Shopping Network
18	Netscape World (3.3)	News and Observer Publishing
19	HotWired Magazine (3.2)	Silicon Graphics
20	NewsPage (individual inc.) (2.7)	Beverly Hills Internet
21	Pathfinder (2.7)	NewsPage
22	Wall Street Journal Interactive (2.7)	Netcom Online
23	Cybershop (1.6)	Apple Computer
24	Women's Wire (1.6)	Sony Online
25	GolfWeb (1.1)	eMall
26	Riddler (1.1)	Spyglass
27	Word (1.1)	Spry Inc.
28	NandO.net (0.5)	Mercury Center
29	Zd Net (0.5)	Hollywood Online
30	iWorld (0.0)	1-800 Flowers

* as reported by Interactive Age 1995

(URL: http://techweb.cmp.com/techweb/ia/features/hitstory.html/)

Table 2

Within-vehicle and Between-vehicle Duplication

Table 3

Tabulated Schedule Profiles

(n=28)

Table 4

Sample Exposure Distribution for a Three-vehicle Schedule

(Two inserts in Excite, ESPNET Sports Zone, and Yahoo)

Table 5

Sample Exposure Distribution for an Eight-vehicle Schedule

(Two inserts in Cool Site of the Day, ESPNET Sports Zone, GolfWeb, Magellan,

NBA.com, Playboy, USA Today, and Yahoo)

Table 6

Summary of Average Error Calculations

Internet schedules (n=28)

Table 7

New Zealand 1985 AGB Magazine Data (n=600)

SMRB US 1979 Magazine Data (n=515)

SMRB US 1984 Magazine Data (n=508)

Table 7 (continued)

TV SMRB Data 1984