Abstract for workshop at ICLS '98

We argue that web-based collaboration is hard because it calls upon students and teachers to make dramatic changes in their school-related practices (in the sense of the word as used by Bourdieu [1977]). If this is true, understanding the existing practices of students and teachers and the changes these practices must undergo as a result of the introduction of some innovation (either technology-based or otherwise) become paramount issues for research. The paradox of technology assessment is that, while the motivation for many education reform initiatives (e.g., enhancing "termlessness" or lifelong learning [Koschmann, in press]) has more to do with effecting a persistent change in the learner's dispositions and stance toward learning than with the acquisition of any particular content, innovations are often evaluated on the basis of test performance rather than by examining differences in learner practice. Making a shift from the study of learning to the study of learner practice raises a number of interesting questions. How does one go about actually studying practice, in the way in which we are using the term here? How can the results of studies of practice be applied to the design of new educational innovations? In particular, how does what we are proposing here in the context of educational evaluation relate to earlier theories of system design, such as "participatory design" [Suchman & Trigg, 1991]? How can instruments and rubrics of assessment be designed on the basis of learner practice?