In this paper, we examine the effects of software that monitors and adapts to user performance. Subjects with high task confidence (TC) showed decreased performance, while task performance for subjects with low TC increased under perceived software monitoring and adaptation. Implications for adaptive interface design are discussed. Keywords: Interfaces, software adaptation, monitoring, task performance, media equation , experimental research.

INTRODUCTION

Adaptive user interfaces are among the most prevalent trends in current interface design, and many consumer software products offer them [10]. The concept of an adaptive user interface involves interface changes based on some characteristic of the user [10]. Examples of adaptive user interfaces include help menus which adapt to the userÕs task, stress level [6], or proficiency level [10]; agent-based interfaces which learn about the user to function as assistants or advisors [5]; and computerized adaptive testing (CAT), a testing procedure in which testing is adapted interactively to match the ability level of the examinee [2]. Advancements in CAT have led to a large-scale computerization of standardized tests [2].

Getting to know the user

Much like a human tutor needs to gain knowledge about a person in order to adapt to her actions, adaptive software relies on user-modeling techniques to monitor and assess user behavior [5]. Computers can generate user models implicitly, by unobtrusively monitoring the user's behavior during regular tasks, or explicitly, by prompting the user to specify preferences or give examples [8].

What happens when a user knows she is being monitored? When adaptive software adjusts its difficulty level to user performance, the user knows that better performance will cause the interface to become more complex, and vice versa. Could this knowledge have a psychological effect on the user, and what would this effect be?

Linking the real world and the media world

Being watched by another person causes humans to act differently than when they are alone. The presence of others even affects peopleÕs task performance. Could these effects also occur when people feel monitored by a computer ? This extension of interface research to the human domain is based on The Media Equation [7], which demonstrates that responses to media are remarkably similar to responses to real life. The effects of being watched by a human can thus provide an indication of what might happen when the observer is a computer.
 

Social Facilitation: The "monitoring explanation"

Social facilitation describes changes in people's behavior which are caused by the presence of others (3). While the mere presence of others can affect task performance [4], the effects are strongest when people feel watched and evaluated by others [4]. Interestingly, the perceived presence of others has differential effects on different types of people: It causes people with high task confidence to perform better ("spectator effect"), while it causes people with low TC to perform less well because of stress [12].

"Choking": The "adaptation explanation"

Conversely, when adaptive interfaces adjust their difficulty, we might expect users with high TC to perform less well in the adaptation condition, since the idea of the task getting increasingly harder as they perform well might cause a "choking"-effect. Low TC subjects should thus perform better since they might feel that the adaptation is "helping" them by making the task easier.

CURRENT STUDY

While electronic performance monitoring (EPM) without adaptation has been linked to increased stress [11] and decreased task performance [9], and social facilitation effects have been demonstrated in EPM [1], the present study attempts to determine whether one of the two proposed explanations applies to adaptive software which uses both monitoring and adaptation.

Method

We chose a CAT situation as the context for this experiment. Two computerized versions of the Graduate Record Exam (GRE) were programmed, both containing the same 15 GRE questions. The versions differed in two aspects: First, the text window appearing after each question differed ("next question ready" in the random condition and "monitoring completed and next question selected based on performance" in the monitoring condition). Second, the introductory text of the first version announced "15 random questions", while the second version claimed to "constantly evaluate user performance and adjust question difficulty accordingly".

47 subjects from a communication course participated in the experiment. One month earlier, they completed a questionnaire about their GRE confidence. Assignment to conditions was random, but balanced for gender and TC. Upon arrival, each subject was read instructions which repeated the manipulation. The experimenter left the room and the subject was given 15 minutes to complete the task. To measure performance, correct answers were counted; possible scores ranged from 0 to 15.

Results

Consistent with the "adaptation explanation", high TC subjects performed less well and low TC subjects performed better in the monitoring and adaptation condition than in the random condition (F (1,43)=6.7, p<.05). When difficulty is adjusted according to performance, the adaptive nature of the interface "helps" users with low TC while it discourages high TC users.

Figure 1. Comparison of mean scores for performance

DISCUSSION

This study addresses social facilitation and adaptation effects in a context in which both monitoring and adaptation are present. The implications for adaptive interface design are many. One suggestion is to emphasize that the interface will become easier if the user has a problem, but not to disclose that the interface might also get more complex. In addition, the softwareÕs adaptive capabilities should be emphasized if the software is geared towards novice users, but not when it is created for expert users. In addition, the user should be alerted whenever the interface is getting easier, but not when it is becoming harder to avoid "choking". More research is needed to assess the effects of other types of interface adaptation, such as emotional adaptation.
 

1. Aiello, J. & Kolb, K. Electronic Performance Monitoring and Social Context. Journal of Applied Psychology, 7 (1993), 537 - 548.
2. de Beer, M. & Visser, D. Comparability of paper and pencil and computerized versions of the GSAT. South African Journal of Psychology, 1 (1998), 37-42.
3. Graydon, J., et al. The effects of personality on social facilitation. Journal of Personal and Individual Differences, 2 (1995), 265-267.
4. Guerin, B. Mere Presence Effects in Humans: A Review. Journal of Experimental and Social Psychology, 22 (1986), 38-77.
5. Minsky, M. A Conversation with Marvin Minsky. Communications of the ACM, 7 (1994), 23 - 29.
6. Picard, R. Does HAL cry Digital Tears: Emotions and Computers. In D.G. Stork (ed.) HAL's Legacy. Cambridge: MIT Press, 1997.
7. Reeves, B. and Nass, C. The Media Equation. How People treat Computers, Television, and New Media like Real People and Places. New York: Cambridge University Press, 1996.
8. Selker, T. (1994). COACH: A Teaching Agent that Learns. Communications of the ACM, 7 (1994) 2Ñ12.
9. Stanton, J. & Barrell, J. Effects of Electronic Performance Monitoring. Journal of Applied Psychology, 6 (1996), 738-745.
10. Trumbly, J.E., Arnett, K.P., & Johnson, P.C. Productivity Gains via an Adaptive User Interface. Journal of human-computer studies, 40 (1994), 63-81.
11. Westin, A. Two key factors that belong in a macroergonomic analysis of electronic monitoring. Applied Ergonomics, 23 (1992), 35-42.
12. Zajonc, R. Social Facilitation. Science, 149 (1965), 269-275.