Adaptive Interfaces:
Effects on User Performance
Department of Communication
October 2000
(Unpublished Manuscript)
Introduction
Computer interfaces are traditionally not custom-tailored towards individual users. Rather, programmers and software designers usually create interfaces for pre-defined, hypothetical, prototypical users (Cooper, 2000) and attempt to make them appropriate for a large and diverse group of users. Because of differences in the experience levels, learning and work styles, cognitive abilities, interests, goals, motivations, and environments of different users, however, traditional interfaces often pose problems for individual users. For example, an interface might be too complex for a novice user, while appearing too simplistic to an expert user.
Adaptive
User Interfaces
Adaptive user interfaces are a promising attempt to overcome such problems resulting from the increasing complexity of human-computer interaction. The basic concept of an adaptive user interface involves interface changes based on some characteristic of the user (Trumbly et al, 1994). Adaptive interfaces hold the promise to alleviate user problems by dynamically modifying the interface based on knowledge about current states, goals, and environments of an individual user. These software artifacts assist users in accomplishing computer-based tasks and, in the process, construct a model of that user's preferences so as to serve him/her better in the future (Langley, 1999). They are designed to tailor a system's interactive behavior with consideration of both the individual needs of human users and altering conditions within an application environment. This interactive approach is believed to have great potential for improving the effectiveness of human-computer interaction (Langley, 1999). Adaptive interfaces - both as desktop-applications and internet-based (e.g., Etzioni & Weld, 1998) - are increasingly adopted by major software vendors: The current movement towards intelligent user interfaces [1] includes adaptive characteristics as a major source of their intelligent behavior. Adaptive strategies are a prevalent trend in current interface design, and a number of consumer software products currently offer adaptive features. Examples of adaptive user interfaces include systems that help users sort electronic mail, filter news stories or web query results, fill out repetitive forms on the internet, select routes and construct schedules. In addition, intelligent help menus adapt to the user’s task, stress level (Picard, 1997), or proficiency level (Trumbly et al, 1994), while interfaces featuring agents (either animated or invisible 'intelligent agents") learn about the user to function as assistants or advisors (Minsky, 1994; Isbister & Layton, 1995; Lashkari et al, 1998). Studies (e.g., Trumbly et al, 1993) provided empirical support for the concept that user performance can be increased when the interface characteristics match the user skill level, emphasizing the importance of adaptive user interfaces.
Computerized
Adaptive Testing
One area in which adaptive interfaces are currently employed is that of computerized testing. Many types of tests and exams, such as the GRE (Graduate Record Examination) and GMAT (Graduate Management Admission Test) are currently being offered in computerized versions because of their dramatic improvements in cost-effectiveness for the test administrator and considerable test length reduction (De Beer & Visser, 1998). This large-scale computerization of standardized tests has gone hand in hand with advances in computerized adaptive testing (CAT) (Barnard, 1990). Unlike their paper-and-pencil counterparts, these on-screen exams do not provide each person with the same set of questions. Rather, computerized adaptive testing comprises a testing procedure in which testing is adapted interactively to match the ability level of the examinee by means of a statistical method called "Item Response Theory (IRT; DeBeer & Visser, 1998). Testing Institutions utilizing this method are citing as its unique advantage to the learner that "everyone is challenged, but not too discouraged" (Rocklin et al, 1995, p.103). When taking an adaptive computerized test, usually questions of increased difficulty are presented as long as an examinee responds correctly. If incorrect responses are entered, the computer presents questions of lesser difficulty. Because the computer takes into account the weighted difficulty of a question when scoring an exam, manufacturers of computer adapted testing software claim that on average, people score as well on CAT exams as they do on paper-and-pencil exams. Several studies support the equivalence of paper-and-pencil and CAT test results (De Ayala, Dodd & Koch, 1990; Henly, Klebe, Mc Bride & Cudeck, 1989). However, studies conducted to test this claim usually compare the average test results of groups of people, rather than compare how the same individuals would do on a paper-and-pencil and an adaptive test.
Therefore, it is unclear whether aspects of adaptive interfaces favor some types of users and disadvantage others while maintaining the same overall point average for the whole subject population, or whether indeed, as testing providers claim, results are nearly identical. In order to better understand how users might react to adaptive technologies, a closer look at the interaction between users and adaptive interfaces appears warranted.
Knowledge Generation in Intelligent User Interfaces (IUIs)
Much as a human needs to have a certain level of information about someone else in order to respond to that person appropriately, adaptive software relies on knowledge gaining techniques which monitor and assess the user's actions (Minsky, 1994). Computers can generate "user models" in two different ways: Implicitly, by unobtrusively monitoring and evaluating the user's behavior during regular tasks, or explicitly, by prompting the user to specify preferences or give examples (Selker, 1994). Recording and evaluating the user's choices during web searches in order to be able to re-rank web query results based on the user's interests and previous choices is an example for implicit user modeling, while an animated help agent asking a user whether he/she needs help formatting a certain document (e.g., Winograd, 1996) exemplifies explicit user modeling. A special case of intelligent user interfaces is adaptive software that adjusts the difficulty level of an application based on electronic monitoring of the user's previous performance, such as in computerized adaptive testing (CAT). In such situations, the user is usually aware that better performance will cause the interface to become more complex, while making mistakes will make the interface simpler or the task easier. Unlike in self-adapted testing (SAT; Rocklin et al, 1995), where users can select the difficulty level for each new question, CAT selects the difficulty level based solely on performance and does not give the user the option to intervene (Roos, Wise & Plake, 1997). Stanton & Barnes-Farrell (1996) indicate that perceived control over the interface appears to lead to better task performance. Such control, however, is not given to the user in CAT. Could the user's knowledge of performance monitoring combined with "forced" adaptation of difficulty level have a psychological effect on the user, and what would this effect be?
The Media Equation
Since the psychological consequences of human-computer interaction (HCI) with Electronic Performance Monitoring (EPM) have not been studied widely yet, we turn to human-human interaction for an indication to what might happen when a user's performance is monitored. This extension of computer interface research to the human domain is based on The Media Equation (Reeves & Nass, 1996), which states that "media = real life" and "computers are social actors". In a series of experiments, Reeves and Nass (1996) demonstrate that individuals' interactions with computers and other media are fundamentally social and natural and follow the same rules as individuals' interactions with other people. Similarly, Rich & Sidner (1998) argue that autonomous agents, in their interaction with people, must be governed by the same principles that underlie human collaboration findings from human-human interaction. Thus, knowledge about interpersonal interaction can give good indications for human-computer interaction. However, could the same effects as in "real life" also occur when computer monitoring is not explicit, but implicit?According to Nass and Reeves (1996), "people don't need much of a cue to respond socially" (p.25) and generally perceive computers as social actors by default. The effects of being watched by a human can thus provide an indication of what might happen when the observer is a computer.
Behavioral
Monitoring in Everyday Interaction
What happens in everyday situations when a person's behavior is monitored? First, being watched by other people causes humans to act differently than when they are alone (Zajonc, 1965). Furthermore, different types of people are affected differently by the presence of others. Monitoring in this context is defined as a broad process of orientation and attention to a subject to assess various parameters (lynn, 1966, cited in Guerin & Innes, 1982). A multitude of studies demonstrates that the mere presence of others can affect task performance (Graydon et al, 1995; Guerin, 1986), and that the effects are strongest and occur most frequently when people feel watched and evaluated by others (Guerin, 1986). In his meta-analysis of over 100 studies, Guerin (1986) found that, while of the studies using mere presence of another person only some found effefcts on performance, all of the studies involving explicit watching and/or evaluating yielded effects. These effects have been linked to controlling social approval (Guerin, 1986), concern over evaluation, self presentation concerns, distraction, and attentional factors (Aiello & Kolb, 1995) and are generally known as "social facilitation effefcts", whereas social facilitation is defined as the presence of others who do not directly interfere, compete, or interact with a person (Guerin, 1986). The study of social facilitation thus concerns the effefcts of the presence of other people on a person's behavior (Guerin & Innes, 1982).
Social Facilitation
Furthermore, Zajonc (1965) demonstrated that being watched by others has differential effects on different types of people. His Social Facilitation Theory (1965) states that monitoring causes people with high confidence for the task they are performing (TC) to perform better than when they are alone (“spectator effect”; a person confident about a task is encouraged to perform better to enhance his/her image in front of others), while it causes people with low task confidence (TC) to perform less well because of stress (Zajonc, 1965) brought on by the fear of performing badly in front of others. In the case of both simple tasks (high TC) and complex or novel tasks (low TC), increased non-directive arousal brought on by the social presence of others as well as distraction and evaluation mark the onset of the social facilitation effect (Guerin & Innes, 1986). Several competing theories attempt to explain whether mere presence, anitcipation of negative or positive evaluation from others based on TC (learned drive), or distraction and its overcompensation (Distraction/Conflict Model) then determine whether performance will improve or decline (Geen, 1981; Sanders, 1981).
Social
Facilitation Effects of Electronic Performance Monitoring (EPM)
As early as 1995, Rocklin et al. noted that
"Although it is widely recognized that students learn differently, there is comparatively little recognition of, or accomodation for, the fact that students respond differently to testing situations",
thus emphasizing the need to study testing effects on different types of students. Most studies to date, however, studied the effects of CAT on student populations which were assumed to be homogenous. Stanton & Barnes-Farrell (1996) cite a number of early EPM studies demonstrating the possibility of increased anxiety and perceived stress in EPM situations (e.g., Smith et al, 1992). In addition to increased stress (Westin, 1992) and decreased task performance (Stanton and Barrell, 1996), EPM without adaptation has been shown to cause social facilitation effects (Aiello & Kolb, 1993): Using a social facilitation framework in the study of EPM, Aiello and Kolb found some support that electronic presence might enhance performance on a simple task (Griffith, 1993) and increase performance on a complex task (Aiello & Svec, 1993). These early studies have, however, been conducted in work environments, and electronic performance monitoring was in the form of surveillance of work progress; thus it is not clear whether results are generalizable to academic testing environments.. No studies have explored the psychological effects of a combination of monitoring and adaptation. This combination is the focus of the present study. From a social facilitation framework, we can thus hypothesize that persons with high task confidence will perform better when monitored by a computer than when performing without monitoring, while persons with low task confidence will perform less well when monitored by a computer than when performing the same task without being monitored.
“Choking”: The adaptation hypothesis
In sharp contrast to the "social
facilitation hypothesis" stands a different prediction
concerning how task confidence might interact with social presence
to affect individuals' task performance while interacting with an
adaptive user interface:
When interfaces adapt their difficulty (as distinct from merely monitoring user performance), the opposite of social facilitation might occur: Users with high TC might actually perform less well when working with adaptive software, since they would expect task difficulty to increase because of the user's expected high initial performance. This might cause a “choking”-effect: A fear of the tasks getting increasingly complex, which might ultimately lead to actual decreased performance. Low TC participants, on the other hand, should perform better when working with adaptive software, since they might expect the adaptation function to “help” them: Since their low task confidence would cause them to expect to perform poorly, they could also reasonably assume that the software must present them with simpler tasks as the test progresses. This more positive expectation should naturally decrease stress, increase confidence, and should lead to higher performance with adaptive testing than with non-adaptive testing. The two suggested explanations about the interaction of task confidence and social presence in adaptive testing situations are thus competing explanations. The present study is designed to determine which (if any) theory is better suited to explain the psychological effects of adaptive interfaces. The study will also determine whether, in addition to affecting task performance, social presence in adaptive interfaces also affects the user's perception of the interaction and of the software used. HCI
vs. CMC
As noted earlier, numerous studies have demonstrated human-computer interaction to be fundamentally social and to follow the same rules as human-human interaction (Reeves & Nass, 1996). While this has been shown to be true in many situations and for many types of interactions, there might conceivably be specific situations in which the nature of computers renders the interaction so different that we might observe different outcomes in the human-computer interaction than in human-human exchanges. Because the psychological dimensions and effects of adaptive, intelligent user interfaces are a largely unexplored area, both HCI and a computer-mediated communication (CMC) based conditions have been included in the current study in order to yield a more complete understanding of the subject.
CURRENT
STUDY
Method
81 subjects from a communication course participated in the 2 (high TC vs. low TC) x 2 (random vs. monitoring+adaptation [M+A]) x 2 (HCI vs. CMC) laboratory experiment. A between-factors full factorial design was used to test the competing hypotheses. One month earlier, participants completed a questionnaire about their confidence for taking the GRE (Graduate Record Exam), the results of which were used to determine task confidence. Subjects were randomly assigned to eight conditions which were balanced for gender.
Procedure
When participants arrived at the laboratory, they were told that they would be taking a 15-question GRE-type computerized test for which they would have 15 minutes to complete. Participants in the "random" (non-adaptive and non-monitoring) conditions were told that the questions were chosen randomly by the computer or by the person; participants in the "monitoring & adaptation" conditions were told that each question would be chosen based on their performance on previous questions: better performance would lead to harder questions while wrong answers would lead to easier questions. In addition, subjects assigned to the "computer" conditions were told that they were working with a computer only which would select and present the questions (and in the M&A condition monitor performance and adjust test difficulty) Subjects assigned to the "human" conditionwere told that the computer was merely a medium to connected them online in real-time to a representative of the GRE testing agency with whom they would be interacting. Subjects in the human-random conditions were told that the person had randomly pre-selected 15 questions for them, and subjects in the human M&A condition were told that the representative would determine whether each answer was right or wrong and send them a new question based on their performance. Everyone actually received identical questions in the identical order and worked with the same interface, which provided radio buttons for answer selection and a "submit" button after each answer. Subjects in the "human" conditions were presented with the exactly same interface and experiment setup; they were not actually connected to another person.
Both the introductory text provided by the research assistant and the on-screen instructionscontained the manipulations, mentioning “15 random questions" vs. “constantly evaluate user performance and adjust question difficulty accordingly”, or giving information such as "the representative of ETS testing is online right now to do this experiment with you". To further instantiate the manipulation of perceived M+A and human vs. computer, a status bar and alert window appeared on the screen after each answer had been submitted, displaying text messages such as “next question ready" vs. “monitoring completed and next question selected based on performance", and "transmitting answer… receiving next question" in the human conditions. Manipulation checks confirmed successful manipulations.
Instrument
The software recorded the test score for each participant which was used as the behavioral measure for task performance. In addition, participants filled out a post-test questionnaire containing items to test subjects' attitudes towards the GRE testing software as well as their perceptions of the quality of their interaction with the software/ the "other person". The questionnaire included standard items commonly used in computer human interaction studies in the social sciences such as "I enjoyed using the software", or "The interaction was cooperative"(see Reeves 7 Nass, 1996). Results
a. Task Performance Regarding the question whether results would be consistent with social facilitation or, alternatvely, with choking based on adaptation, an interesting pattern emerged. Results for HCI differed from results for CMC. While performance was generally higher when subjects were told they were interacting with a human, rather than a computer (F (1,81)=8.0, p<.01), the pattern of the 2-way interactions between TC and random vs. M&A was reversed in HCI and CMC, resulting in a highly significant 3-way interaction between high/low TC, random/M&A and HCI/CMC (F (1,81)=10.1, p<.005). We then conducted separate ANOVAs for HCI and for CMC to test for 2-way effects for TC and random/M&A.
1. HCI For HCI, results were consistent with the “adaptation explanation”. While the expected main effect for task confidence emerged in that people with high task confidence performed better across conditions than people with low task confidence (F (1,40)= 5.5, p=.02), there was a significant interaction such that subjects with high task confidence performed less well, while low TC subjects performed better in the monitoring and adaptation condition than in the random condition (F (1,40)=5.6, p=.02). In fact, while high TC subjects performed much better than low TC subjects when they thought questions were chosen randomly by the computer (11.7 versus 8.9), the performance of the two groups was identical when they were told that the computer would monitor their performance and adapt question difficulty. Thus, when difficulty is adjusted to performance, the adaptive nature of the interface appears to have a reassuring effect on users with low TC, causing them to perform better, while causing a "choking effefct" in users with high TC. For the HCI condition, the "adaptation" hypothesis was therefore confirmed.
2. CMC The reverse was true in the computer mediated conditions. The expected main effefct for TC was found here, too, in that subjects with high task confidence performed slightly better than subjects with low task confidence overall; the effect was borderline significant (F(1,40)=3.9, p=.056). More interestingly, however, was the fact that the 2-way interactions were in the reverse direction as those in HCI. Thus, while the performance of subjects with low and high task confidence was almost identical when questions had been chosen randomly, being told they were being monitored and adapted to by another person caused high task confidence subjects to perform better, and subjects with low task confidence to perform less well (F (1,40)=5.1, p=.03). For CMC, the social facilitation hypothesis was therefore confirmed. The results for CMC replicate findings in interpersonal interaction from the social facilitation literature.
Figure 1. Comparison of mean scores for performance in HCI and CMC.
b. Attitudinal Measures
Few effects were found for the attitudinal measures. Questionnaire items were collapsed into the variables "software likability" (a= .75) and "enjoyment of interaction" (a=.81). Similar results emerged for each of these variables. The clear result patterns for the behavioral variable did not emerge here; no interactions between task confidence, random vs. monitoring & adaptation, and computer vs. human were present. Rather, an interesting main effect for computer vs. human demonstrated that subjects rated both the likability of the GRE testing software (F (1,81)=9.814, p =.002), and their enjoyment of the interaction (F (1,81)=8.53, p=.005) significantly higher when working with a computer, rather than with another person. This finding is interesting given that subjects actually performed better when working with another person than with a computer. Thus, while subjects prefer working with an adaptive computer interface, they perform better in the human interaction they like less.
DISCUSSION
This study demonstrates that perceived adaptation reverses the social facilitation effect of monitoring in adaptive interface-HCI while demonstrating social facilitation effects in CMC. First, the findings are relevant as an extension of social facilitation research to the realm of computer-human interaction and computer-mediated communication, demonstrating that, while social facilitation effects were replicated in computer-mediated communication, social facilitation does not apply to computerized adaptive testing. With regards to the Media Equation (Reeves & Nass, 1996), this study shows that in highly specific situations, such as CAT, HCI can differ from CMC and from non-mediated interpersonal interaction. It would be an interesting extension to this study to test whether the Media Equation holds true for other types of adaptive interfaces which do not specifically adjust the difficulty level based on performance, but adapt to the user in different ways. Finally, the results also have implications for the design of adaptive interfaces. To alleviate the dilemma of high TC users' performance suffering from the awareness that performance is monitored and difficulty is dynamically adjusted, and low TC users benefiting from the same knowledge, one design suggestion might be to emphasize that the interface will become easier when the user has a problem, but not to disclose that the interface might get more complex as long as the user performs well. Thus, the user should be alerted whenever the interface is getting easier, but not when it is becoming harder, to avoid “choking”, or the notion of "competing with oneself". In addition, the software’s adaptive capabilities should be emphasized if the software is geared towards novice users, but not when it is created for expert users who are more likely to perform well and to be faced with more complex interface selections.
Directions
for Further Research
Clearly, this experiment represents a novel field of study, and more research is needed to assess the effects of other types of both real and perceived interface adaptation. Results indicate that the psychological effefcts of intelligent user interfaces are complex and are likely to depend on the type of interaction. Therefore, to fully understand the interaction between humans and intelligent, adaptive interfaces, it will be necessary to study a multitude of different adaptive interfaces representing not only different monitoring and adaptation strategies, but also different tasks, activities, and contexts. Related
to cognitive adaptation, the new field of emotional adaptation (e.g.,
Picard, 1997) where interface adjustments are based on the computer's
assessment of a user's affective state, might yield another promising
related area of study. In addition, the unexpected differences between a human tester (in the CMC situation) and a computer as tester warrants further analysis. Locus of Control theory, for example, has in the past been used to explain differences between effects of human and computer experimenters ( Martin & Knight, 1986) and might be a starting point for additional study. Similarly, social loafing has been linked to social facilitation (Harkins, 1986) and might be a factor in the human interaction with adaptive interfaces in that a human might view the computer as a group member in an adaptive situation. Several other theories from psychological research might be relevant to the study of HCI and intelligent user interfaces and might serve as useful vantage points for further exploration of a type of interaction which in the near future might be as ubiquitous as human-human interaction.
REFERENCES
1.
Aiello, J. & Kolb, K. Electronic
Performance Monitoring and Social Context. Journal of Applied Psychology,
7 (1993), 537 - 548.
2.
Aiello, J. & Svec, C. Computer
Monitoring of work performance: Extending the social facilitation
framework to electronic presence. Journal of Applied Social Psychology,
23 (1993), 537-548.
3.
Barnard, J.J. Using item response
theory for test construction. Pretoria, South Africa: Human Sciences
Research Council, 1990.
4.
Cooper, A. THE INMATES ARE RUNNING
THE ASYLUM: Why High-tech Products Drive Us Crazy and How to Restore
the Sanity. Indianapolis, IN: SAMS Macmillan Computer Publishing,
2000.
5.
De Ayala, R.J., Dodd, B.G., Koch,
W.R. A simulation and
comparison of flexilevel and Bayesian computerized adaptive testing.
Journal of Educational Measurement (1990). 27, 227-229.
6.
De Beer, M. & Visser, D. Comparability
of paper and pencil and computerized versions of the GSAT. South African
Journal of Psychology, 1 (1998), 37-42.
7.
Etzioni O. & Weld, D. A Softbot-based
Interface to the Internet. In: Huhns, M.N., & Singh, M.P. (eds.).
Readings in Agents. San Francisco, CA: Morgan Kauffman Publishers,
1998.
8.
Geen, R.G. Evaluation Apprehension
and Social Facilitation: A Reply to Sanders. Journal of experimental
social psychology, 17 (1981), 252-256.
9.
Graydon, J., et al. The effects
of personality on social facilitation whilst performing a sports related
task. Journal of Personal and Individual Differences, v. 19, n.2 (1995),
265-267.
10.
Griffith, T.L. Monitoring and Performance:
A comparison of computer and supervisor monitoring. Journal of Applied
Social Psychology, 23 (1993), 549-572.
11.
Guerin, B. Mere Presence Effects
in Humans: A Review. Journal of Experimental and Social Psychology,
22 (1986), 38-77.
12.
Guerin, B, & Innes, J. Social
Facilitation and Social Monitoring: A new look at Zajonc's mere presence
hypothesis. British Journal of Social Psychology, 21 (1982), 7-18.
13.
Harkins, S. Social Loafing and
Social Facilitation. Journal of experimental social psychology, 23
(1987), 1-18.
14.
Henly, S., Klebe, K.J., McBride,
J.R. & Cudeck, R. Adaptive and conventional versions of the DAT:
the first complete test battery comparison. Applied Psychological
Measurement, 13 (1989), 363-371.
15.
Hindus, Debby, Mark S. Ackerman,
Scott Mainwaring, and Brian Starr. "Thunderwire: A Field Study
of an Audio-Only Media Space." Proceedings of the ACM Conference
on Computer Supported Cooperative Work (CSCW '96), November, 1996,
pp. 238-247.
16.
Huhns, M.N., & Singh, M.P.
(eds.). Readings in Agents. San Francisco, CA: Morgan Kauffman Publishers,
1998.
17.
Isbister, K., & Layton, T. (1995). Agents: What (or
who) are they? In J. Nielsen (Ed.), Advances in Human-Computer Interaction
5 ,67-86. Norwood, N.J.: Ablex Publishing Corporation.
18.
Langley, P. User modeling in adaptive
interfaces. Proceedings of the Seventh International Conference on
User Modeling (pp. 357-370). Banff, Alberta: Springer, 1997.
19.
Lashkari, Y. et al. Collaborative
interface Agents. In: Huhns, M.N., & Singh, M.P. (eds.). Readings
in Agents. San Francisco, CA: Morgan Kauffman Publishers, 1998.
20.
Martin, S. & Knight, J.M.
Social Facilitation Effects resulting from locus of control using
human and computer experimenters. Computers in Human Behavior v. 1
(1986), 123-130.
21.
Minsky, M. A Conversation with
Marvin Minsky. Communications of the ACM, 7 (1994), 23 - 29.
22.
Picard, R. Does HAL cry Digital Tears: Emotions and
Computers. In D.G. Stork (ed.) HAL's Legacy. Cambridge, MA: MIT Press,
1997.
23.
Reeves, B. and Nass, C. The Media
Equation. How People treat Computers, Television, and New Media like
Real People and Places. New York: Cambridge University Press, 1996.
24.
Rich, Ch. & Sidner, C. COLLAGEN:
When agents collaborate with people. In: Huhns, M.N., & Singh,
M.P. (eds.). Readings in Agents. San Francisco, CA: Morgan Kauffman
Publishers, 1998.
25.
Rocklin, T.R., et al. Effects and
underlying Mechanisms of self-adapted testing. Journal of educational
Psychology v. 87, n. 1 (1995), 103-116.
26.
Roos, L., Wise, S.L., & Plake,
B. The Role of Item Feedback in Self Adapted Testing. Educational
and Psychological Measurement, v. 57 n. 1 (1997), 85-98.
27.
Sanders, G. Driven by distraction:
An integrative review of social facilitation theory and research.
Journal of experimental social psychology, 17 (1981), 227-251.
28.
Selker, T. COACH: A Teaching Agent
that Learns. Communications of the ACM, 7 (1994) 2–12.
29.
Smith, M.J., et al. Employee Stress
and health complaints in jobs with and without electronic performance
monitoring. Applied Ergonomics, 23, (1992), 17-27.
30.
Stanton, J. & Barnes-Farrell,
J. Effects of Electronic Performance Monitoring. Journal of Applied
Psychology, 6 (1996), 738-745.
31.
Trumbly, J.E., Arnett, K.P., &
Johnson, P.C. Productivity Gains via an Adaptive User Interface. Journal
of human-computer studies, 40
(1994), 63-81.
32.
Westin, A. Two key factors that
belong in a macroergonomic analysis of electronic monitoring. Applied
Ergonomics, 23 (1992), 35-42.
33.
Winograd, T. Profile: Microsoft
Bob. In: Winograd, T. (Ed.). Bringing Design to Software. New York,
NY: ACM Press, 1996.
34.
Zajonc, R. Social Facilitation.
Science, 149 (1965), 269-275.
|
