Study Findings

Data Analysis

Data from the pre-test questionnaire, the satisfaction survey and retention quiz were combined and analyzed accordingly. Since there were an unequal number of findings from each of the stimuli, an average of each was taken. From our initial findings, there was not a statistically significant difference in either the retention or satisfaction (t-test: retention = .26; satisfaction = .12). Given the small sample size it may be difficult to detect differences in the means. The mean retention difference was very small, so it is not clear there are differences between passive and interactive presentations for retention. For the satisfaction means, however, there was a more substantial difference (9.95 versus 13.5), which suggests that with a larger sample size it may be possible to detect a statistically significant difference in satisfaction between passive and interactive multimedia presentations (Figure 5).

Figure 5: Average score in the satisfaction surveys and short-term retention quizzes for all test subjects

Overall findings regarding retention rates and satisfaction levels of our subject pool

In order to more closely analyze each set of findings for the two dependent variables, each was considered separately. First, the results from the retention quizzes of all 10 subjects were averaged for each of the stimuli (Figure 6). It is interesting to note that an asychronized interactive site (I2) reported the worst retention rates, whereas an asychronized passive presentation (P1) tied for the best retention rates with a synchronized interactive site (I4). While the researcher cannot confirm the exact reasoning behind these findings, it begs the question if media content impacted retention rates more than that of media display.

Figure 6: Average questions answered correctly in the retention quizzes of all study subjects

Overall findings regarding retention rates of our subject pool

Second, the results from the satisfaction surveys of all 10 subjects were averaged for each of the stimuli (Figure 7). While satisfaction scores widely varied by subject and stimulus, again it must be noted that an asynchronized interactive site (I6) received the lowest satisfaction score, with the asynchronized passive presentation (P1) only receiving one point better. This suggests that users may prefer multimedia considered to be in the middle of the continuum (either 2 or 3 in Figure 1) rather than on the extremes.

In reviewing the findings against the self-reported data from the pre-test questionnaire, I noticed a trend in my findings that align with those of Liu and Shrum (2002) in that a user’s personality largely impacts his/her perceived satisfaction. In this case, after experiencing the asynchronized interactive stimulus (I6), a person who reported him/herself as a “slight introvert” recorded the lowest satisfaction level of all subjects on all stimuli, whereas another subject who reported him/herself as an “extreme extrovert” reported one of the highest scores.

Figure 7: Average score in the satisfaction survey of all study subjects

Overall findings regarding retention rates of our subject pool

Furthermore, data from the post-test questionnaire regarding subject’s most and least favorite presentation supports the hypothesis that users would find interactive presentations more satisfying (Figure 8). It should be noted that a total of four subjects viewed P1, P2, P5, P6, I1, I2, I5, and I6. However, only two subjects viewed P3, P4, I3, and I4. Therefore, both subjects who viewed the asynchronized passive infographic (P3) noted that it was their least favorite.

Figure 8: Self-reported most and least favorite presentation of the four each subject viewed

Subjects most and least favorite multimedia presentation

Limitations of the Study

I acknowledge the difficulty in attempting to directly compare passive and interactive presentations of drastically different content. Depending on the stimuli chosen and the topic of content featured within each, the findings will likely vary. For this reason, I propose the need for a larger study with both more test subjects and multimedia stimuli. It would also be beneficial if there were a way to control for the variety of retention questions given for each stimulus to ensure consistent levels of difficulties for questions across all testing content. Moreover, it is difficult to test retention in interactive stimuli since the site is not fixed and users did not come to the site with a pre-determined task. Future studies may be able to control for this inconsistency by either providing task-specific questions or drawing a user’s attention to a particular area of the presentation for the purpose of testing retention.

Furthermore, this qualitative study only focused on one age group. However, if a quantitative study was proposed to enhance this study, it might be beneficial to recruit test subjects from both Gen Y and Gen X (or any other generational group) to determine if age impacts retention rates and satisfaction levels.

Lastly, the test subjects were informed before viewing the stimuli that they would be given a retention quiz afterward, so they may have viewed the presentation more closely in order to be better prepared for the quiz. Future researchers may wish to not inform the subjects in order to more closely mimic a normal viewing session.

Importance of Study

Researching the implications of multimedia format on an end user is extremely important because it is still a fairly new field of study, and there is no prior research of which I am aware that has assessed the retention and satisfaction of the consumer dependant on whether a multimedia presentation is passive or interactive. While research has suggested numerous benefits of utilizing multimedia over traditional, non-multimedia formats (such as text stories or HTML sites), a gap still exists in the research when comparing across different types of multimedia.

Furthermore, with the rise of social media and digital technology, it is vital that employers can successfully connect with their consumers. This entails making deliberate decisions about what media to utilize during a digital campaign to effectively get their message across. Historically, people have had a tendency to use a medium they are most comfortable with, rather than what would be most effective. Moreover, what is most effective differs depending on who the target audience is, and whether the end goal of the presentation is to inform and educate (academia and journalism) or to promote and sell (marketing and advertising).

The results of this study will hopefully have wide-reaching benefits within the journalism sector, and provide material for further research in the fields of academia, marketing and advertising. Furthermore, the findings will hopefully contribute to several research sectors, including information architecture, usability testing, and human-computer interaction. The user satisfaction portion directly relates to usability and information architecture, while evaluating retention rates would further research in the field of human-computer interaction.

Conclusion

This study aimed to gauge how the presence of interactivity in a multimedia presentation might affect a user’s short-term retention rates and satisfaction levels. The data suggests, although not conclusively, that while users reported higher satisfaction levels after viewing interactive sites, their short-term retention rates were higher after viewing passive sites. I also noticed a trend that suggests the possibility that personality traits are likely to play a role in whether or not users will find an interactive presentation enjoyable, which supports the findings from Liu and Shrum (2002). I did not have a wide enough sampling of data to determine if sex, race, age, or other dependent variables regarding user traits might impact retention rates and satisfaction levels. However, the findings suggest the need for a more in-depth and controlled quantitative study in order to more comprehensively determine the implications on the end user from the decision to produce either a passive or interactive multimedia presentation.



About the Researcher

Tracy Boyer is an award-winning multimedia technology strategist, specializing in the intersection of digital media and interactive technology. Currently, Tracy is the first MBA/MSIS dual master’s candidate at UNC-Chapel Hill, where she is studying General Management at Kenan-Flagler Business School and Human-Computer Interaction in the School’s Information Science program. Boyer is currently the managing editor of Innovative Interactivity, a widely read multimedia blog that she founded in 2007.

Previously, she reported on malnutrition in Honduras with The Pulitzer Center, was a multimedia producer at Roanoke.com, served as the UNC correspondent for CNN.com and interned with The Atlanta Journal-Constitution. In 2007, she was selected to participate in the Poynter Summer Fellowship. Boyer graduated with a multimedia degree from UNC’s School of Journalism and Mass Communication.

Feel free to email Tracy directly with comments, suggestions, and any other feedback related to this research.

View more of her work at www.tracyboyerclark.com.