Stewart J Chow, Masters Candidate, University of Texas at Austin
Qualitative and Quantitative Analysis, ADV F380J Summer 2007

The following report contains statistical analyses of copy-testing research on print ads for three well-known brands of healthy snack bars: Nature Valley, Nutri-Grain, and Powerbar Pria. The purpose of the study was to determine the brand preferences of a random sample of respondents, to examine the perceptions with regard to each brand's attributes, and to determine the effect of advertising on the brand preferences. An online questionnaire was developed and sent via e-mail and data was collected for the span of an entire week.  In total, there were 60 completed surveys.

From there, several research methods were used including: regression analysis, discriminant analysis, ANOVA/MANOVA, factor analysis, and cluster analysis along with other basic statistical methods.

It was shown that there was hardly any difference in the perceptions between the three brands.  Nature Valley was the most favored out of the three.  For a more cohesive analysis, continue on to the full report.

Healthy on-the-go eating is a hot trend these days, as the everyday man or woman wades through their hectic lives of work, family, and the daily grind. Snack bars have been a longtime favorite due to their portability, taste, health benefits, and overall convenience. Perception is always a gray area marketers and advertisers are wary of. Does the advertising make sense? Is it hitting the right audience? The purpose of this study was to test the effectiveness of three different ads for three brands of health snack bars. Between Nature Valley, Nutri-Grain, and Powerbar Pria, this study used copy-testing methods to analyze brand preferences of the respondents. These three brands/ads were chosen because of the similarities between price, ingredients, and image. The questionnaire respondents took (which can be accessed in Appendix B) asked several questions about how the user felt about a brand's attributes. In the extensive analyses after the data collection, statistically significant data was drawn to make predictions about how this sample of respondents would relate to the population it was drawn from.

From this study, we were able to identify which brands respondents prefered before viewing an ad, and then how they felt after they were shown the ads. The numbers from the relationships calculated can be found in the analysis section of the report. All the rest of the formalities are addressed in the methodology section, outlining specific tools, questions, and tactics that were used in compiling the data.

Questionnaire, tools: Adobe Dreamweaver CS3, Cold Fusion, Microsoft Access 2007

The questionnaire was the primary tool of data collection to test perceptions of brand attitudes. Initially, it asked respondents to list different types of snack nuts and fruit snacks to prime their reasoning towards the subject at hand: health snack bars. They had to be reasonably of the same "type" of food without duplicating the actual item in question. Then, respondents were asked to divide 10 points among 3 given well-known brands in a constant sum scale. Three ads were found from various home, variety, and entertainment magazines that had to have a certain commonality between them. The ads were then shown the respondents, as they were told to view them for 30 seconds each. After viewing, respondents were asked again to rate the brands on the same 10 point constant sum scale. The differences in pre-vew and post-view scores would later go on to test upward or downward movement in their impression of the brand. The survey continued with 10 Likert items that measured various brand attributes (how did they feel, would they recommend, is it tasty, etc.) for all the brands. Subsequent sections went on to ask the effectiveness of the ads themselves. Values such as 'was the ad informational?' or 'did you find the ad boring?' were collected and used to measure how well the ads played on their impressions. The final section dealt with general demographic and lifestyle information concerning the respondents.

Once the survey was completed it was sent non-randomly via e-mail and list-servers to fellow students, co-workers, and acquaintances under the assumption that all responses were to remain confidential. They were requested to forward the questionnaire on to as many people they knew to acheive the necessary total to run an adequate analysis. This was the general e-mail that went out:

"Subject: Marketing research survey, assistance wanted

Hello everybody,

You've seen this time and time again, and those of you in the know, you understand what "research" and "survey" in the subject line of an e-mail means. I would greatly appreciate your devoting the time and patience to filling out this survey, if you can.

http://www.ciadvertising.org/SA/summer_07/adv380j/chowsj/Brand%20Survey.html

All the best,"

Design, tools: Adobe Photoshop CS2, Canon Digital Scanner

Moreover, the ads used in the pre-exposure and post-exposure sections had to be specific. One all bleed, full-color magazine ad was used for each snack bar brand. They had to show the packaging and/or the bar itself. The only differences were the amount of copy and the tones of each ad. The pictures had to be scanned and cropped to be proportional to each other, as to maximize commonality across the three brands. The questionnaire was built in Dreamweaver, mentioned before, and data was collected to the class server through Access.

Data Analysis, tools: SPSS v. 15, Microsoft Word 2007, Microsoft Access 2007

Data that all respondents entered and submitted went to a repository on the class server. The data was collected and then imported to SPSS to conduct the many types of analyses and to compute the statistics. A minimum of 60 respondents were required to have a substantial sample to draw from. In both the Analysis section and Appendix A, are the statistical findings computed from SPSS. They go to show the various means, standard deviations, and other values that go on to determine the significance of these numbers.

 

      The nature of this statistical analysis is to show the varying perceptions and differences between 3 health snack bars.  The findings come from a week’s worth of data collection through online surveys which respondents took the time to fill out.  Their data has been processed using various tests, correlations and frequency comparisons to learn about the consumer differences in copy-testing practices for these bars and their advertisements.  For instructive purposes, Brand A = Nature Valley, Brand B = Nutri-Grain, Brand C = Powerbar Pria.  The following tables have been compiled to express the scores between brands.

    Table 1a.

Table 1a. expresses the average of the 10 Likert items for each brand on the survey also known as the brand index score (BIS).  The larger the number, the more favorable the brand is to the respondents. The highest possible brand index score is 50 and the lowest is 10.  The standard deviations are also shown here to represent where approximately 50, 48 and 53 percent of the respondents are from the mean value. From the results, Nature Valley had the highest average favorability with the respondents.

                              Table 1b.

Table 1b. presents “statistically significant” data.  The numbers extrapolated from the sample test group is a good measure to the rest of the population; meaning if 85 of 100 respondents were pulled from the same population as this sample, they would receive similar results allowing this data to be translated to the population as a whole.  The numbers are significant comparing Nature Valley to Nutri-Grain bars and Nature Valley to Powerbar Pria bars.

Table 2a.

Table 2a shows a sizeable difference in the respondents feelings.  Twelve people viewed Nature Valley more favorably after seeing the ad while twice as many felt less favorably. In 85 or more samples out of every 100 drawn from the same population as this sample, we can expect that the Brand Index Score up-movers (those who viewed the brand more favorably after seeing the ad than prior) will have a less favorable perception of the ad than the Nature Valley down-movers (those who viewed the brand less favorably after viewing the ad than prior)      

Table 2b.

Alternately, there is no way to extrapolate that 85 out of 100 samples taken from the same sample population would result in Nature Valley Advertising Index Score up-movers (those who preferred the brand more after viewing the ad than prior to viewing it) to have more of a preference for the Nature Valley brand than down-movers (those who preferred the brand less after viewing the ad than prior).

      Table 3

Table 3 indicates that 27 respondents had above median Brand Index Scores in comparison with 21 below median. To demonstrate, the reading of the first cross tab would show that of these 60 respondents, 3 respondents were: both up-movers and above median in their Brand Index Scores, accounted for 33.3% of all post-ad up-movers, accounted for 11.1% of all of those above median, and accounted for 6.3% of the total sample. There’s little variation where the respondents’ scores stayed the same. There were more respondents who originally started the survey favoring Nature Valley, but thought lower of the brand after viewing the ad.  The Chi-Squared test indicates that these results are significant at the 0.15 level and that in 85 or more samples out of every 100 samples drawn from the same population as this sample, the same results for the Nature Valley ad could be expected.

      Table 4

Table 4 shows the distribution of the 60 respondents’ attitudes toward the three brands before and after the ad viewing. Powerbar Pria had the highest up movers score, with respondents commenting on how well the ad spoke to their time intensive schedules. With Nutri-Grain, most respondents stayed the same, not being either positively or negatively affected after seeing the ad. Most respondents moved down after seeing the Nature Valley, either because the ad said nothing new to the respondent or perhaps since the ad was more female friendly, it turned away the male viewers.

From the 60 collected respondents, 31 people preferred Nature Valley to Nutri-Grain, while 29 prefer Nutri-Grain to Nature Valley. Based on their brand index scores, 51.7% of respondents felt more positive about Nature Valley than Nutri-Grain overall. With the small difference in preference, the data indicates the possibility between the two brands: 1.) there is an undifferentiated place in consumer’s minds, or 2.) they are both as well-liked across the market.

The correlation co-efficient between Nature Valley and Nutri-Grain is a positive .1 (1.0 being a perfect positive correlation), meaning that an increase or decrease in Nature Valley’s BIS has no discernable effect to Nutri-Grain’s BIS.

We used multiple regression analyses to examine the relationships between multiple independent variables and one dependent variable. For each of the three brands that were analyzed we utilized this form of analysis. As the independent variables we used each brand’s 10 Likert items from the survey, and as the dependent variable we used the respondents’ overall change in constant-sum score for that brand after viewing the ads. The objective was to determine the impact each Likert item had on the change in constant-sum score.

Brand A: Nature Valley

Table 1a

In this sample of 60 respondents, there was a low, positive correlation between how respondents thought about the brand (measured by the 10 Likert scale items) and the respondents’ change in purchase intent (measured by the change in constant-sum score). The 10 Likert items could only explain 3.8% of the total variance in the constant-sum score’s change. The standard error was fairly high, meaning the average distance of respondents from the regression equation line was high at 2.4 units. Thusly, as determined from the F-ratio, these results were not significant so they cannot be accurately projected to the population from which the sample was drawn.

Table 1b

The fact that Nature Valley provided ‘better benefits’, made the buyer ‘feel good’, would ‘not recommend’ to friends, and had ‘attractive’ packaging were all positively correlated with the Change Score, meaning that as these item values increased, so did the Change Score, albeit in this case, a very low correlation. The idea that this was a ‘good bar’, that ‘fits their lifestyle’, provided ‘more variety’, was ‘not as flavorful’, and was ‘appetizing’ were all negatively correlated with the Change Score, which means that as the values for these items increase, the Change Score decreases.  The important variables in explaining variance in the Change Score for this sample included those described as having ‘better benefits,’ a product that ‘feels good’ to buy, and “not recommended.”  There were no significant items for this brand, which means that none of these statistics could be projected to a sample drawn from the same population as this.

Brand B: Nutri-Grain

Table 2a

Nutri-Grain also showed a modestly low correlation score between how respondents thought about the brand and respondents’ change in purchase intent.  The 10 Likert items here explained a higher 14.6% of the total variance in the constant-sum score’s change.  But again, as determined from the F-ratio, these results were not significant so they cannot be accurately projected to the population from which the sample was drawn.

Table 2b

‘Feel Good’ pulled a perfect positive correlation from the sample, while other factors ‘Not as Flavorful’, ‘Sates Appetite’, ‘Not Recommended’, and ‘Attractive’ were minimally correlated, meaning that as these item values increased, so did the Change Score, but the connection is not as strong.  The analysis showed that the bar being perceived as ‘not as flavorful,’ satiating your appetite, and making the buyer ‘feel good’ about the purchase were the only important Likert items.  Determined from the t-ratios, the results for ‘Better Benefits’, ‘Not Good Tasting’, and ‘Feeling Good’ were significant. So we can know with some certainty that in 85 or more samples out of 100 drawn from the same population as this sample of 60 people, it would be expected that the results for the three Likert items would be about the same as they are in this sample.

Brand C: Powerbar Pria

Table 3a

In this last sample for Powerbar Pria, there is a moderate correlation between how respondents thought about the brand and respondents’ change in purchase intent. The 10 Likert items explained an improved 26.7% of the total variance in constant-sum score change over the other brands.  Consequently, the standard error of the estimate is figured to be lower (the average distance of respondents from the regression equation line is lower at 1.3 units). According to the F-Ratio, these results were significant so they can be projected to the population from which the sample was drawn.

Table 3b

A multiple linear discriminant analysis was conducted for Nutri-Grain bars to test whether or not the responses to ten Likert items of brand attributes could discriminate between respondents whose brand scores moved up or moved down after viewing the ad. In this test, the ten Likert items were treated as the independent variables, while the up-movers and down-movers were treated as the dependent variables.

Table 1

After comparing the mean scores for the two groups, the analysis shows that on average, down movers have higher scores than up movers across the variables; however, any variation was not drastic.  Only the variable “Nutri-Grain is not a good tasting health snack bar” showed the highest difference with a 0.4 between up and down movers.

                   Table 2

Only the standardized discriminant function coefficients that were equal to or greater than half of the absolute value of the biggest standardized discriminant function coefficient were considered important. Knowing this (1.7/2 = 0.9), only one variable “Nutri-Grain is not a good tasting snack bar” (1.7) was an important variable in discriminating between the two groups.  The more Nutri-Grain was thought as ‘Not Good Tasting’, the more respondents disliked the ad.

        Table 3a                                                                     

Table 3b

The Wilks’ Lambda is a statistic to test the significance of Group Centroids. The Group Centroids are the means of the discriminate function scores; they sit far enough apart to be distinguishable. The Group Centroids were -0.6 for up movers and 0.5 for down movers. In this test, the Group Centroids were not significant, because the Wilks’ Lambda of 0.73 had an insignificant Chi-squared value, which means that in 85 or more samples out of 100 samples drawn from the same population as this sample, the Group Centroid data cannot be projected to the population.

               Table 4

% of original grouped cases are correctly classified = (9 + 13) / (14+16) = 73.3%
to = (.733 – .500) / sqrt[ (.733 x .267 / 30) + (.5 x .5 / 30) ] = 1.91*
*p > 1.04
In this classification matrix, 73.3% of the originally grouped cases were correctly classified. Nine people in the original up-mover group and thirteen people in the original down-mover group moved in a predicated way. The t-observed score calculates out to 1.91, which is greater than 1.04 thus proving significance.  This means that in 85 or more samples out of 100 samples the statistics here can be accurately projected to the population from which the sample was drawn.

ANOVA

We used a two-way factorial ANOVA (analysis of variance) test to determine the relationship between one Likert item “Nutri-Grain is not as flavorful as other bars” and the movement pattern of respondents pre to post advertisement viewing and the respondent’s gender. In this ANOVA test the dependent variable was the Likert item stated before while the two independent variables were the respondents’ move scores and gender.

    Table 1

Among respondents who were male, the brand attribute score for “Nutri-Grain is not as flavorful as other bars” were highest for up movers through the non movers. The difference between male and female respondents was the largest for down movers and smallest for up and non movers. For females the up-movers generally had a lower mean score for the Likert item.  What is of interest, are the means of the change scores, whose direction and respondents’ gender appear inverse.

Since this was a ‘negatively’ posed question, the lower mean scores reflect a more negative view of the answer.  The data showed that a handful of males thought Nutri-Grain bars were not as flavorful as other bars after viewing the ad.  On the other hand, more females disagreed that the bars were not as flavorful after a similar viewing of the ad.  There could have been something visually responsive about seeing the bar itself in the ad that would change a male or female respondent’s perception of flavor.

Table 2

In determining significance of this data, we could conclude that the relationship between the gender of a subject and their mean score was not statistically significant in this ANOVA test. In 85 or more samples out of every 100 samples, drawn from the same population of 60 people, we would not expect the same F-ratio between the Brand Attribute item and the gender of the respondent. This relationship between gender and their mean score cannot be applied to the general population. Furthermore, the F-ratio between the change score and the Lickert item and the F-ratio between both independent variables and the Likert item were not statistically significant. This means that the patterns we observed for both males and females with respect to their change scores relative to the particular Brand Attribute cannot be applied to the general population. 

MANOVA

For the second analysis we performed a two-way factorial MANOVA (Multivariate Analysis of Variance) test to determine the relationship between the ten Likert items for Nutri-Grain and the respondents’ change scores and gender. The chart below shows the results for the MANOVA test:

Table 3