The Role of Computational Cognitive Artifacts

in Collaborative Learning and Education

Project Summary

This project addresses ROLE quadrants 2 and 3: It builds bridges from cognitive and social theories of the role of artifacts to research on learning in educational settings, and it develops a methodology for the principled assessment and research-based design of technological artifacts to mediate learning processes. The goal of the project is to refine both a micro-analytic methodology and an artifact-centered theoretical framework that can aid in the principled design of distance learning environments. The project will not only result in a much-needed methodology for future designers of educational technology, it will also deepen our understanding of the role that such computational cognitive artifacts can play in collaborative learning and formal education.

The project studies small groups of students using prototype versions of learning environments to see what the students go through in learning how to use the computer-based artifacts and what problems interfere with the learning goals. The project brings together educational software developers and experts in human-human and human-computer interaction to conduct the analysis of videotaped student interactions and to iterate the design of the educational environments.

Three software systems developed by project team members are studied, gradually advancing from a relatively simple computer simulation, through a semester-long on-line biology lab curriculum, to a distance education version of the labs:

1. SimRocket simulates the launch of rockets having varying characteristics. Five middle school students used the simulation to predict the effects of the different characteristics on the height attained by the rocket. Their sessions working with the simulation were videotaped.
2. VirtualBiologyLab is a series of 10 freshman college biology lab experiments simulated and conducted on the Web. Groups of 2 or 3 students work together to complete the lab – analysis of their successes and difficulties feeds back into the iterative design of the digital curriculum.
3. WebGuide is a knowledge-building environment for supporting collaboration. Functionality from WebGuide will be integrated into the VirtualBiologyLab in the final project year, and used by geographically distributed high school students for a version of the biology labs redesigned for Advanced Placement study.

From the perspective of the project’s theoretical framework, these learning environments are treated as interacting networks of computational cognitive artifacts. For instance, in analyzing students working with SimRocket, the project team looks at how the students talk about and make use of (a) the rocket simulation, (b) a display of rocket characteristics, and (c) a data collection form. Each of these three artifacts is designed in a way that permits it to be used in certain ways to accomplish certain tasks: (a) some artifacts like the simulation are computational and change on their own in response to inputs; (b) others like the display convey knowledge; while (c) yet others like the form provide cognitive support by organizing and preserving information. In each case, the students must learn how to recognize and take advantage of these artifact affordances. Designers of learning environments must design both the affordances of individual artifacts and the curricular context that will make these meaningful to students within a coherent educational experience.

The project analyzes the collaborative efforts of small groups of students in order to determine (a) the extent to which students can understand the use of educational artifacts, (b) where this is problematic, and (c) how student learning can be scaffolded to overcome problems. In collaborative interactions, students must display to each other their beliefs, their questions, their problems, and the resolution of problems. When this process is videotaped and carefully analyzed, it makes the students’ learning visible to researchers as well. The project adopts a micro-analytic form of communication analysis called micro-ethnography to study what is displayed. This is a rigorous method for analyzing both vocal and visible forms of human interaction recorded on video.

The project team has already begun to adapt micro-ethnography and the theory of artifacts to the analysis of student interactions with on-line educational technologies. Members of the project team have collaborated in various combinations in the past, including a semester-long pilot project investigating SimRocket student interaction data and theories of artifacts. The 4 Principal Investigators are experienced in the design of educational technology and/or the micro-ethnographic analysis of people interacting with educational artifacts. All 9 of the Advisory Board consultants assessed educational technologies in their PhD dissertations and/or in their current work. The 2 Graduate Research Assistants are pursuing dissertations closely linked to this project.

Project Description

Overview of Proposal

1. The Problem of Educational Artifacts
2. Three Educational Artifacts for Study
3. A Staged Research Plan
4. Theoretical Framework for Cognitive Artifacts
5. Research Methodology for Studying Interaction
6. Sample Analysis of an Episode of Collaborative Learning
7. Pilot Studies Conducted
8. Results from Prior Support
9. Contributions to ROLE Goals and Potential Impact
10. Project Team – see Biographical Sketches

1. The Problem of Educational Artifacts

As schools across the nation get wired for computer-based learning, the problem of scarcity of effective on-line curriculum and content becomes increasingly urgent. Current research stresses the importance of carefully designed software artifacts that are “student centered, knowledge centered, assessment centered, and community centered” (Bransford et al., 1999) . Yet, with a few notable exceptions, there is little in the way of constructivist curriculum and content that meets these criteria and is also ready to take advantage of computer- and Internet-based media. In fact, there is little systematic knowledge of how to develop such educational artifacts, grounded in a theoretical understanding of the role that such artifacts might play in learning and in a methodology for software testing.

Distance Learning

The general problems of computer-mediated education multiply substantially under the pressure to rush to distance learning. Universities and dot.com’s around the country have jumped on the distance education bandwagon, without necessarily thinking through the complex educational issues involved. It is true that distance education has the potential to address various pressing educational, social, geographic, and economic issues (Keegan, 1986) . It is also true that the technical infrastructure that will enable this revolution in education to proceed is being quickly set in place. However, the design and development of the necessary curriculum and content lags far behind. In the dash to market, providers of distance education are likely to settle on software technologies that were developed for other uses and are inappropriate for educational applications, curricula that implement outmoded approaches like drill-and-practice, and content that has not been tested for its learning effects. We need to develop more new models of computer support for learning that are effective in distance learning.

Designing Computer Support for Learning

Based on our own experiences with software in classrooms, we have found that computer-supported collaborative learning (CSCL) has a vast – and largely untapped – potential. Access to global sources of information is just one facet. In addition, computer simulations can transform conceptual representations into interactive worlds for inquiry. They can transcend real-world barriers of time, expense, geography, scale, expertise, etc. to allow students to engage with and experience phenomena that have until now been unapproachable. Hypertext systems of information can personalize presentations to meet individual learning needs. Communication media can promote collaboration in ways never before possible, as well as among people who could not hitherto interact. Structured curricular databases and shared knowledge-building environments can support student learning processes. However, we have seen that students always use computer artifacts in ways not envisioned by the designers. So, careful study of the artifacts in naturalistic settings is critical to the development of effective educational technology.

Understanding Computational Cognitive Artifacts

It is possible to ground the design and assessment of educational software applications in an understanding of their role as “artifacts” in learning. Our preliminary understanding views various forms of artifacts as absolutely central to human cognition and learning. People construct their understanding through interaction with artifacts; often artifacts extend, amplify, or transform cognition; eventually the artifacts may be internalized as mental procedures (Cole & Griffin, 1980; Donald, 1991; Engelbart, 1995; Hutchins, 1999; Norman, 1993; Papert, 1980; Pea, 1985; Vygotsky, 1930/1978) . We intend to further develop this theoretical framework, which is inherent in theories of situated action and situated learning, in distributed cognition, in activity theory, and in various philosophies.

In particular, we propose to apply this framework to analyze educational technologies as “computational cognitive artifacts” (CCAs). We use this term to refer to computer-based or Internet-based educational artifacts (simulations, data analysis tools, on-line curricular modules, etc.): They are “computational” if they respond interactively to user interactions by changing their display. They are “cognitive” to the extent that they can become part of the user’s thinking, by, for instance, helping the user to visualize some phenomenon, providing an external memory or workspace for manipulating representations, or aiding in conducting a computation. They may also be “cognitive” in the further sense that they can be internalized in the user’s mind so that he or she can make use of them as a mental metaphor or representation in the future when they are no longer even virtually present on a monitor screen. They are “artifacts” in the sense that they are perceptible objects that were designed to serve as some kind of tool – even though today they may not be physical objects that can literally be grasped.

To conceptualize an educational application as a CCA is not to assume a priori that it functions effectively in this role. Rather, it is to raise a set of critical issues:

·        Does it facilitate human-computer interactions or mediate human-human interactions computationally?

·        Does it support and enhance cognitive functions of its users?

·        Does it function as a useful artifact in accordance with its design?

By conceptualizing certain types of educational technology (e.g., computer simulations) as CCAs, we can ask if they are fulfilling this role effectively in specific situations that we observe. We propose to investigate how people develop the understanding required to use CCAs effectively in CSCL settings, and conversely to study the roles these CCAs then play within the collaborative learning and education taking place.

Assessing Collaborative Learning for Iterative Design

Our goal is to contribute to the design of CCAs that are effective for supporting collaborative learning and education. Our theoretical framework does not directly imply criteria for the design of educational technology. Rather, it suggests that we develop prototypes of software applications and look at how students actually relate to them as computational cognitive artifacts – that is, that we look at how students concretely explore, come to understand, and use the software as an artifact for extending their cognitive powers – and then we iteratively revise the design of the software. For us as researchers to look at this, we need a methodology. We believe that micro-ethnography provides such a methodology. Micro-ethnography was designed to look very closely at social interaction processes. This project will adapt micro-ethnography to look at computer-mediated interactions in situations of collaborative learning.

The proposed project is an application of micro-ethnography’s method to the concerns of human-computer interaction. It brings together a team of people from these areas who are experienced in interdisciplinary research (see Biographical Sketches). Our team includes faculty and students from Communication, Computer Science, Cognitive Science, and Education, as well as developers of educational software – within a broader academic community that is supportive of this project. This project is unique in bringing together educational software developers and specialists in the micro-analysis of interaction to develop and systematically test a rigorous methodology and a grounded theoretical framework for the design of distance learning artifacts.

2. Three Educational Artifacts for Study

In our project we will study the use of three software systems that we have developed: SimRocket, VirtualBiologyLab, and WebGuide (see section on Pilot Studies for more details):

A Middle School Computer Simulation

SimRocket is a simulation of rocket launches. We already conducted and videotaped a three-hour trial of it with 5 middle school students and a teacher. We have begun to analyze the data from this trial. We have observed that the simulation artifact played a central role in the interaction: it opened up and defined the whole educational space, providing the narrative context as well as the source of data for collection and interpretation. In the analysis of a specific episode with SimRocket (see below), we will see the collaborative interaction revolving around three inter-related artifacts: the computational simulation of rocket launches, an external memory display of rocket characteristics, and a paper chart of recorded rocket heights. By closely analyzing the interactions among the teacher and students we see: (a) successes and failures of students to grasp the meaning/use of these artifacts, and (b) the teacher’s attempts as an experienced scientist to guide the group to effective use of the artifacts. Because of problems in the interaction that become apparent to the participants, the teacher must make his analytic skills observable and the students must make their adoptions or misunderstandings apparent. We also see that there is not a single simple artifact here, but a subtle network of artifacts with different functions. Furthermore, the artifacts only exercise their cognitive function or activate their meaning when they are being used appropriately. Our observations of the teacher’s patterns of face-to-face interaction suggest forms of scaffolding that could be introduced in distance learning where a teacher is not physically present.

A College On-line Lab

The VirtualBiologyLab is a much more complex network of interdependent artifacts. It is a complete one-semester curriculum on the Web, intended to replace college freshman biology wet labs for non-majors. Each of 10 planned labs takes an estimated three hours for a student to work through – and enables students to conduct seminal experiments from the history of biology that would not be feasible in traditional physical wet labs (see attached letter of support from the developer). One can distinguish multiple kinds of artifacts composing the software: a guiding narrative, animations of lab equipment, simulations of lab procedures, data collection / analysis / graphing / display tools, background materials (theory, history, remedial text), links to related websites, and interactive assessment exercises. The virtual lab is designed to be used by students independent of any teacher guidance, although it is loosely coordinated with a biology lecture course. The on-line system must work as a whole, motivating and guiding students through a sequence of tasks; each of the distinct component artifacts must work effectively on its own and within the whole pedagogical context.

A High School Distributed Education Lab

WebGuide is a knowledge-building environment to support collaborative learning. It provides a collaboratively constructed and shared external memory medium on the Web. The display is dynamically computed to show a hierarchy of notes arranged as a personal or group “perspective” on the persistent, asynchronous discussion. This perspective mechanism is an artifact that people must learn how to use and navigate to mirror and support the interpersonal relationships of collaboration. WebGuide also incorporates a variety of knowledge management functions that must be learned in order to manipulate the ideas stored in the system and to build effective shared knowledge. Certain components of WebGuide will be integrated with a version of the VirtualBiologyLab toward the end of our project to explore a collaborative distance learning biology curriculum at the high school Advanced Placement level. We will also extend the lab software to incorporate educational scaffolding techniques from other knowledge-building environments like CSILE/KnowledgeForum (Scardamalia & Bereiter, 1996) , KIE/WISE (Cuthbert, 1999) , and CoVis (Pea, 1993) .

3. A Staged Research Plan

Collaborative learning is a complex process. Accordingly, our project will build up gradually from our relatively simple pilot study to a full example of collaborative distance education.

Project Schedule

The project will consist of three main stages:

1.            Analysis of the three hours of video tape already collected of five middle school students and a teacher conducting a study of rocket design using the SimRocket computer simulation.

2.a.         A very brief study of college freshmen in a biology wet lab. This will serve as an informal baseline for the next stage.

2.b.         This is the core study for the project. We will videotape small groups of students working together with the on-line VirtualBiologyLab. This software is currently under development at the University of Colorado. The developers are involved in this project and will be iterating the design of the software in response to the analysis of the usage trials. We will focus our analysis on critical steps in the lab, like learning how to use a particular piece of equipment or a data analysis tool.

3.            A distance education version of VirtualBiologyLab will incorporate a collaboration medium based on WebGuide. This will be offered as an Advanced Placement curriculum to geographically distributed high schools students. The curriculum will be designed to be collaborative, and we will log user interactions and use these to study the learning taking place.

Following is a timeline for these stages:

Data Gathering and Analysis

Our gathering and analysis of data involves the PIs working closely with the graduate and undergraduate team members. In addition, our consultants participate in workshops held monthly. The workshops not only review project progress and plan next steps, but they importantly include group data sessions for the analysis of data. The data gathering and analysis process (for instance for the VirtualBiologyLab sessions) will typically proceed through the following steps:

1.        Videotaping of students. Two or three students are gathered around a computer. Cameras and microphones are set up to capture the facial expressions and body movements of all participants. The monitor image is also captured. Microphones are arranged to capture all speech as clearly as possible and to distinguish the speakers.

2.        The video is combined (picture-in-picture) and time-code is burned in to provide a frame-by-frame reference system.

3.        A minute-by-minute record log is created, describing in a sentence or two what takes place each minute. This is typically done by a graduate student and reviewed by a PI. The log may be revised later.

4.        A list of interesting episodes is created. Episodes are meaningful interactions lasting up to several minutes. The list is discussed by the whole project team at a group workshop.

5.        Selected episodes are digitized and made available electronically. This allows them to be replayed easily, looped, freeze-framed, slowed down, and studied by project consultants at distant locations.

6.        A detailed transcript is created. It transcribes both speech and visible behaviors. Speech of different participants is color-coded. The transcripts are printed and posted on the Web with the digitized clips.

7.        Each episode is assigned to a project team member who “owns” that piece of data. The owner watches the clip many times to understand what is happening there.

8.        A data session is conducted with the whole project team at a group workshop. This is a collaborative analysis of the data’s empirical details. Usually, about two hours are spent on a single episode. The session is led by the owner of the data, who presents the episode and raises issues. The owner may audio-tape this session to preserve ideas and interpretations that come up.

9.        The owner of the episode returns to a study of the video clip. At this point, the transcript may be revised and extended to include more details of interaction. The owner may invite other project team members to view and discuss the clip. The owner may present the clip at another data session. Finally, the owner drafts a micro-ethnographic analysis of the episode. This is distributed for comment. The analysis includes:

a.       A detailed description of the actions of all participants and their interactions.

b.      A discussion of what learning is evidenced in the data.

c.      A discussion of the role of any artifacts.

d.      A discussion of problems with the software, learning problems, etc.

10.     The analyses of the episodes are reviewed by the whole project team and various suggestions are made based on this:

a.       Proposed revisions to the software.

b.      Changes to the list of interesting episodes, such as the inclusion of additional episodes.

c.      Alterations to the research plan, such as scheduling additional usage sessions or changing the way they are conducted.

d.      Revisions to the research methodology and theoretical framework.

Project Assessment and Dissemination

We will engage in formative evaluation of our project throughout. That will be an important function of our larger team, which includes assessment experts, and will form a regular part of the monthly workshops. We will check that we are making progress toward our project goals in accordance with the project timeline and are following our data analysis procedures. Specifically, we will check that we are developing our methodology for making learning visible and for iteratively designing software artifacts, as well as disseminating our findings.

The micro-analytic approach that the project will develop provides a built-in assessment process for the project. By videotaping sessions of students working with artifacts, we will derive a formative evaluation of the learning facilitated by the artifacts. By the end of the project, we will be able to compare in a detailed and documented way how well our revised versions of educational software artifacts perform as compared to how they worked in the pilot studies and in earlier phases of the project. In addition, we will assess how successful we were in the course of the project in developing, formulating, and applying micro-ethnographic methodology for studying the educational role of cognitive artifacts and for assessing the ability of students to adopt the computational artifacts into their collaborative learning.

In addition to the micro-ethnographic analysis which examines both how students learn with computer technologies and their learning processes as revealed through their interactions (computer-mediated and face-to-face), it is important to understand how students relate to the technologies, as well as the degree to which students learn. In order to understand this, a triangulated approach to assessment will be adopted. Some students in the core trials of VirtualBiologyLab will be given a set of pre-assignment questions to gauge their prior knowledge and understanding of the concepts. Once they have completed the trial, they will be asked the same questions so that we can calculate their learning gains. In addition, we will interview these students in order to understand their perceptions of the artifacts as effective learning tools. This information will be gathered with each iteration and use of the software under development, and the comments and perceptions will be fed back into the development of software and the articulation of learning processes that involve computer software and computer-mediated collaboration. Understanding student perceptions of their experiences will also enable us to track our progress toward our research goals and to evaluate the effectiveness of the theory and method under development by answering the critical question of, does it work: have we indeed made learning visible in a way that can contribute to iterative design of effective software artifacts?

The PI will be personally responsible for coordinating activities associated with the project. He will supervise the work of students and consultants and ensure that they are working in accordance with the project plan, including the preceding procedure for the collection and analysis of data. The PI will make certain that the plan is followed and the timetable met (taking into account changes adopted during the life of the project). He will also attempt to mediate any conflicts that arise within the diverse and interdisciplinary project staff. The PI will engage project Advisory Board consultants who are assessment specialists to assist in on-going project evaluation and to conduct a quarterly project review for reporting to the Advisory Board.

Data collection and analysis issues including sampling and confidentiality will conform to rigorous research conventions and University of Colorado Human Subjects standards.

We will establish a website for both internal use and broad dissemination. The website will collect and coordinate materials and findings of the project. It will include logs of our videotapes, digitized clips of selected episodes, detailed transcripts, analyses of interactions, etc. It will also include all papers submitted to journals and conferences.

This project and its findings will be broadly disseminated in the CSCL, CSCW, HCI, education, and communication research communities through conferences and journals. It will be particularly prominent at CSCL 2002 and subsequent meetings of CSCL, AERA, CSCW, Group, ICLS, and WebNet. It will also significantly impact the release of a published VirtualBiologyLab curriculum at the college and the high school level.

4. Theoretical Framework for Cognitive Artifacts

In the current Fall 2000 semester, the PI offered an interdisciplinary seminar on the theory of artifacts. Many of the project co-PIs, graduate research assistants, and consultants participated fully in the seminar. The project’s theoretical framework grow out of this seminar. It will be considerably refined through a grounded theory analysis of the data collected in the project. This theoretical framework provides a bridge from selected findings of various cognitive sciences to research on learning in educational settings. It will guide the questions we pose in looking at our data.

Mediated Cognition

We start from three principles enunciated by Vygotsky (1930/1978; 1934/1986) :

1.        Mediated cognition. Modern human cognition is thoroughly mediated by physical and symbolic artifacts such as tools and words. We extend this to the use of computer-based artifacts like simulations, data analysis tools, and collaboration media.

2.        Social cognition. Meanings and practices are first established interpersonally and may then be internalized in individual minds. We take advantage of this by analyzing the interpersonal interactions, which are largely observable to the trained analyst as well as to the participants.

3.        Zone of proximal development. A student learns most productively when guided somewhat beyond his or her current developmental level by peers or a mentor. We use this principle to design experimental situations in which a small group of students is challenged to engage in a scaffolded scientific task.

Collaborative Knowledge Building

We conceptualize our subject matter as the process of “knowledge-building” (Bereiter, 2000) . This is an active collaborative learning process in which a community constructs conceptual meaning. For instance, in our SimRocket pilot study the students came to understand the effect of different variables upon future rocket launches and learned to isolate variables to measure their independent effects. The process of collaborative knowledge-building is interpersonal and observable – primarily through analysis of the communicative interactions through which it takes place.

Collaborative knowledge-building involves an interplay between individuals and the group, with individuals contributing from their personal perspectives and the group accepting these contributions in its own way (Stahl & Herrmann, 1999) . This perspective-taking and perspective-making unfolds in the observable world of signs and artifacts, such as spoken utterances and external memory devices (Boland & Tenkasi, 1995) . The physical and symbolic artifacts mediate between personal and group understandings.

The Role of Artifacts

It is possible to re-conceptualize learning (both individual and collaborative) through a focus on the artifacts that are involved. Artifacts – including software artifacts – embody intentionality, meaning, and experiences of their creators and preserve these for future users (Donald, 1991; Hall, 1996) . The problem is for users of artifacts to know how to reactivate this stored wisdom. This requires complex skills of interpretation (Gadamer, 1960/1988; Stahl, 1993) . Education can be viewed as largely the effort to socialize children and other new-comers into a practical understanding of the artifacts and practices that constitute a society’s or a community’s culture (Lave & Wenger, 1991) . The written word and the symbols of mathematics, for instance, are cognitive artifacts that take years of schooling to master. While people have been producing and using artifacts forever (Donald, 1991; Geertz, 1973) , we have little experience designing and teaching computational artifacts.

Artifacts play an absolutely central role in learning and understanding according to the philosophic roots that underlie contemporary cognitive theories that are influential for CSCL theories (Koschmann, 1996; Koschmann, 1999; Koschmann, in press) , such as situated action (Suchman, 1987) , situated learning (Lave & Wenger, 1991) , activity theory (Engeström et al., 1999) , distributed cognition (Hutchins, 1996) , dialogicality (Bakhtin, 1986) , and critical inquiry (Dewey & Bentley, 1949/1991) .

According to Hegel (1807/1967) , the very basis of self-consciousness and sociality in mutual recognition is thoroughly mediated by the creation and use of artifacts – which embody human consciousness or meaning in their imposed form or design. Marx (1867/1976) argues that the production, circulation, and consumption of artifacts as commodities is both affected by the prevailing social relations and reproduces those relations – and influences how we understand and learn about contemporary artifacts; these commodities are essentially stored labor – physical and intellectual – that comes alive in use. Marx traces the social history of artifacts from simple tools through machinery to computational automated industry. For Husserl (1936/1989) , meaning is established and historically sedimented in the form of artifacts; Heidegger (1927/1996) expands this analysis to argue that the life-world of our everyday involvements is structured as networks of meaningful artifacts. More recently, software is seen as a new form of stored meaning or intentionality (Keil-Slawik, 1992; Stahl, 1993; Winograd & Flores, 1986) . For instance, effects of “artificial intelligence” are accomplished by embedding human intelligence in software procedures and knowledge-bases.

Engelbart (1995) and Norman (1993) claim that it is artifacts that make us smart, by amplifying our very limited native abilities like short-term memory. Others (e.g., Cole & Griffin, 1980; Pea, 1985) counter that these artifacts change our tasks, rather than simply increasing our powers, but this still places artifacts centrally in our attempts to increase our intellectual capabilities. Donald (1991) argues that the entire enterprise of modern knowing and science only became possible with the development of artifacts like books, which provided external memories that could be circulated and that might outlive their creators. Papert (1980) , reflecting on his own learning history, believes that playing with automobile gears as a young child “did more for my mathematical development than anything I was taught in elementary school. Gears, serving as models, carried many otherwise abstract ideas into my head” (p. vi).

If one looks closely at learning – from infancy to kindergarten, formal schooling, and on-the-job – one sees that artifacts (now including computational artifacts) are pervasive. While it is clear that a primary function of education (and socialization into culture generally) is to teach new-comers how to understand and use the available artifacts of one’s society or of its specialties, we have only narrow studies of how this takes place. For instance, Bruner (1990) discusses how children acquire the ability to follow and generate narratives as verbal cognitive artifacts, and Hall (Hall & Stevens, 1995) investigates how young students use design tools.

How Artifacts are Understood

Even in our very preliminary pilot study of the SimRocket data, it has already become clear that the process of coming to understand a computer simulation that models a scientific phenomenon is a complex process, which strains the cognitive abilities of middle school students. Without strong guidance from a teacher, the students would at best have treated the simulation as a video game, perhaps competing to get the highest rocket flight, but not investigating the scientific factors that might lead to success.

Although students often make statements that sound like they understand how to construct certain kinds of knowledge, when one watches them struggling through the steps that are actually required one gains a much more detailed understanding of what is involved for a novice, what supports are helpful, and where problems typically arise. For instance, while the students in the pilot study were proficient at taking averages of sets of numbers in a traditional math lesson, they ran into many problems when averaging their rocket data. A major problem had to do with the organization of the data and of their averages on a data sheet. The two teams of students became very confused about which rocket heights had been observed by which team, and which averages were associated with them. While an adult experienced with scientific experiments can keep these things straight without thinking about it, the students had to learn this skill. They did this partially by negotiating with the teacher, who alerted them to problems and guided them back on track, and partially by collaboratively applying their own intellectual and communicative skills.

Our work and that of our current and past colleagues explores the use of gesture in understanding artifacts and in constructing shared understanding of artifacts. In his seminal example of micro-ethnographic analysis (which studies the interaction of five young children in a school project, and thereby provides a model for us) and subsequently, Streeck (1983; 1993; 1996) focuses on the roles of gesture in making social understanding visible. LeBaron analyzes different forms of gesture that are successively used to build a shared vocabulary of meaningful gestural artifacts (LeBaron, 1998; LeBaron & Hopper, 1997; LeBaron & Koschmann, 1999; LeBaron & Koschmann, 2001; LeBaron & Streeck, 2000) . Koschmann also highlights the role of gesture in educational settings (Koschmann et al., 1997; Koschmann & LeBaron, submitted; Koschmann et al., 1998; Koschmann & Stahl, 1998) . Our micro-ethnographic method (see below) is explicitly adapted to making learning visible by systematically attending to the sorts of gestures and bodily interactions that people use to co-construct the meaning of artifacts.

According to our theoretical framework, learning through interaction with artifacts is an inherently social process, involving either interaction with other people through the artifact or at least interacting with an artifact that was made by other people and that incorporates their intentions. For our research, collaborative interactions have an important characteristic: in order to collaborate, participants must make their ideas and their relationships visible to each other as part of their communication. That is, they make learning visible. As researchers, we can capture this in video or computer logs and analyze it. That way, we can see how students are relating to computational artifacts and what they are learning in the process. This overcomes the traditional problem of educational assessment, where it is assumed that learning is invisible to researchers and must be inferred from learning outcome measures. Thus, our approach avoids the restriction of educational assessment to the kinds of analyses of pre/post-test statistics and after-the-fact interviews that so often lead to “no significant difference” (Russell, 1999) results, which are of little value for design purposes.

Of course, not all learning is made visible, so other methods to indirectly measure learning outcomes are necessary and complementary. But focusing on the visible displays of learning prevents the common tendency to lose track of the learning in favor of secondary phenomena that seem easier to describe or quantify. For instance, much of the traditional literature on cooperative learning focuses on small group facilitation, rather than on cognitive and group learning processes (for a recent review of this literature, see (Brody & Davidson, 1998) reviewed by the PI (Stahl, 2000a) ). Even recent CSCL studies often miss the interesting learning phenomena (e.g., (Hakkarainen & Lipponen, in prep) and (Jong et al., in prep) , reviewed by the PI (Stahl, in prep) ).

Grounded Practical Theory

While we have encountered many suggestive ways of thinking about artifacts in our readings, the roles and functioning of artifacts are most clearly revealed by close observation of our data. We expect to come to a deep understanding of the role of artifacts in education – and conversely of the role of learning in artifacts – through our study of student interactions with educational artifacts.

Glaser & Strauss (1967) have described techniques for deriving theory from qualitative data in sociology. In philosophy, Gadamer (1960/1988) has proposed that hermeneutic understanding can be derived through reflection on life experience and situated interpretation. Schön (Schön, 1983; Schön, 1987) argues for reflective practice in professional activities like design and teaching.

Project co-PI Craig and his colleagues (Craig & Sanusi, 2000; Craig & Tracy, 1995) – building on Glaser & Strauss, Gadamer, Schön, and others – have developed an approach to grounded practical theory within communication analysis. The general idea of this approach is that practical theory – theory designed to inform praxis – involves conceptually reconstructing practice. This can be done on three levels:

1. A problem level that accounts for difficulties or dilemmas typically encountered in the practice.
2. A technical level that describes a repertoire of practical techniques for addressing problems.
3. A philosophic level that formulates normative principles to govern the use of techniques

For example, collaborative learning is a normative principle that can govern the use of practical techniques such as the SimRocket exercise. A problem noted in the pilot project was that middle school students may not collaborate toward certain desirable learning objectives without some guidance by the teacher (level 1). In the SimRocket data, we see a teacher using various interactional techniques that may display his orientation to this problem. To facilitate reflection on those techniques, the problem might be conceptualized theoretically as an instance of the more general dilemma of any pedagogical practice that attempts to be learner-centered while achieving specific learning objectives. "Scaffolding" names a general sort of technique that teachers can use to address this dilemma (level 2), but scaffolding can be, for example, either too directive (becoming teacher centered) or too nondirective (risking failure to achieve prescribed learning objectives). The collaborative learning principle (level 3) suggests a solution to the dilemma: the use of scaffolding techniques that focus the group's attention on a task that both structurally entails the prescribed learning objectives and requires active student collaboration to be completed. This may provide a principled basis for assessing the teacher's techniques in the SimRocket data, and also a principled basis for design revisions in the computational artifact (to better enable preferred forms of scaffolding). By the same token, the micro-ethnographic analysis provides a basis for assessing the relevance and applicability of this or any other theoretical reconstruction of the practice that might be proposed.

Such a grounded practical theory approach will guide us to:

1. Reflect upon problems that arise in the interactions we observe.
2. Define techniques that are responsive to these problems.
3. Formulate principled ways to move from empirical observations to software recommendations.

5. Research Methodology for Studying Interaction

Iterative Software Design

The core of the project is to develop a methodology for driving the iterative development of software for computer-mediated education. The idea is to start with an initial prototype, videotape small groups of students collaborating with the software, analyze the problems that arise as well as the kinds of learning that take place, formulate revisions to the software based on that analysis, and iterate system design (along with any associated recommendations for classroom presentation) toward improved learning.

Iterative design is a well-established approach in software development, particularly when the effectiveness of the software depends upon the ability of people to use it as intended. The problem is how to analyze the quality of usage in successive trials. This is best done by interpreting in a rigorous way how learning is taking place. During the past 25 years, scientific methodologies for interpreting social interaction have been developed. We focus on one particularly promising school of this science, micro-ethnography.

Micro-ethnography

For this project, we adopt a recent tradition of human interaction analysis (Jordan & Henderson, 1995) that we refer to as “micro-ethnography.” This methodology builds on a convergence of conversation analysis (Sacks, 1992) , ethnomethodology (Garfinkel, 1967) , nonverbal communication (Birdwhistell, 1970) , and context analysis (Kendon, 1990) . An integration of these methods has only recently become feasible with the availability of videotaping and digitization that records human interactions and facilitates their detailed analysis. It involves close attention to the role that various micro-behaviors – such as turn-taking, participation structures, gaze, posture, gestures, and manipulation of artifacts – play in the tacit organization of interpersonal interactions. Utterances made in interaction are analyzed as to how they shape and are shaped by the mutually intelligible encounter as a holistic context – rather than being taken as expressions of individuals’ psychological intentions or of external social rules (Streeck, 1983) . At the same time, micro-ethnography addresses larger social concerns, such as criminal justice (LeBaron & Hopper, 1997) , medical education (LeBaron & Koschmann, 1999) , and problem solving in complex technological settings (Hutchins & Palen, 1998) .

Micro-ethnographic research typically involves the following components:

1.        A specific setting, or research site – such as several students gathered around a computer running specific software.

2.        A detailed analysis of both audible and visible micro-behaviors, which are to be understood in terms of their embeddedness within the particular social and material environment – such as a classroom.

3.        A recognition that culture (which includes the meaning and use of shared artifacts) is a product and a process of naturally-occurring communication, simultaneously co-constructed and experienced by participants – and thereby made available for empirical study and interpretation by researchers.

4.        A use of recent technologies, like digitized video, that allow researchers to look at in detail the orderly performance of social life – such as the negotiation of learning between teacher and student or among collaborating peers.

We will build on this micro-ethnographic approach and on the expertise and methodology which has evolved through the micro-ethnographic data sessions conducted by the project co-PIs and their colleagues for several years. We will collect appropriate data and conduct our own data sessions for project staff, as we have already begun to do with our pilot study data.

Micro-ethnography can be adapted from the study of human-human interaction to that of human-computer interaction or computer-mediated collaboration. Our pilot studies suggest that such an adaptation of the methodology can be accomplished effectively. Our past use of micro-ethnography in collaborative educational settings – particularly in medical problem-based learning – has been very insightful and encouraging.

Micro-ethnography and Human-Computer Interaction

Our research approach brings together educational software designers and micro-analytic researchers. We use micro-ethnography to analyze empirical student interactions with educational software artifacts. Techniques related to micro-ethnography, such as video analysis and conversation analysis, have previously been used to analyze human-computer interaction in limited cases (Bødker, 1989; Bødker, 1996; Frohlich & Luff, 1990; Hollan et al., 2000; McIlvenny, 1990; Nardi, 1996; Suchman, 1987; Suchman & Trigg, 1991) . However, these cases typically did not analyze interactions at the micro-behavior level, including such things as gesture and posturing, which are important means of making understandings visible in face-to-face communication (suggestive exceptions from our own community include (Hutchins & Palen, 1998; Streeck, 1996) ). But, most importantly, these studies did not investigate learning technologies. Nor did they investigate learning taking place through the interactions. Those that did look at learning (like Roschelle (1996) ) did not use this to feed back into the design of the technology. Thus, our project is undertaking an approach that is unique in combining all three:

1.        Analysis of interaction at a micro level.

2.        Analysis of the learning taking place.

3.        Application of the analysis to revision of the technology.

Our own past work using micro-ethnography has begun to move this approach toward our project goal. Co-PI LeBaron (1998) shows through micro-ethnography how an architecture teacher goes through four stages of successive abstraction to define meaningful gestures, which the students then gradually adopt in their own presentations. By freezing key video frames and relating them to the speech and bodily behaviors of the teacher and students, LeBaron makes the teaching and learning process – which the participants are only tacitly aware of – visible to researchers.

Co-PI Koschmann and collaborators (including LeBaron) have been engaged for almost 10 years in fine-grained studies of collaboration among medical students in a problem-based learning (PBL) curriculum (Glenn et al., 1999; Koschmann & Glenn, submitted; Koschmann et al., 1997; Koschmann et al., 2000; LeBaron & Koschmann, 2001) . In particular, we have shown how group discussions raise learning issues for further study and how the status of these issues is negotiated by the students and a tutor. While we have investigated the role of a tutor in face-to-face PBL sessions, we have only recently begun to study the role of computer-based artifacts and media in distance-PBL sessions (Koschmann & LeBaron, submitted; LeBaron & Koschmann, 2001) . The proposed project will build upon the isolated pioneering efforts of ourselves and others, and attempt to put these methods together in a systematic way and apply them to the design of educational artifacts.

Focus on Artifacts in Learning

Artifacts frequently play a central role in learning situations as analyzed by micro-ethnography. In the preceding examples, for instance, a student architectural model was the focus of discussion in LeBaron’s example, a medical instrument in Koschmann’s surgical example, and a whiteboard for listing learning issues in Koschmann’s PBL example. In each of these cases, the artifact was germane to the students’ learning. In fact, the students were primarily learning how to use and make sense of the artifact. The architecture teacher was demonstrating how to hold and study an architectural model, how to move through the spaces it creates, and how to critique its design. The surgeon was teaching his student how to manipulate the automated instrument and how to interpret the image on the screen as the organs being surgically treated. In the PBL session, the tutor was subtly guiding the students to formulate issues for the whiteboard and teaching them how to orient their collaborative learning processes around this artifact. In the following section, we see the role of artifacts in our SimRocket pilot study.

6. Sample Analysis of an Episode of Collaborative Learning

To illustrate our micro-ethnographic approach, we analyze a brief moment of classroom interaction taken from our SimRocket data. Our excerpt begins approximately 90 minutes into the videotaped record. The students have already performed multiple launches of seven different rockets, each rocket having a certain combination of features (i.e., engine type, body type, nose shape, and number of fins). By noting the height of each launch, students were able to calculate an average height for each rocket. The students might have also compared these averages to determine the effects of the different features – but the students were having difficulty conducting such a comparison. Through his interaction with students, the teacher was able to assess participants’ understandings and lead them into a discussion of basic scientific procedures.

An Instance of Collaborative Learning with Artifacts

Consider the following transcribed moments, involving the teacher (T) and his various students (the transcript has been simplified for this presentation) :

1      T:     And you don't have anything like that there?

2            (1.4) ((T gestures toward monitor & data))

3      S:     I don't think so.

4      J:     Not with the same engine. Not with the same-

5      T:     With the same engine but with a different nose

6            cone?

Repeatedly (e.g., lines 1 and 5), the teacher directs students’ attention toward their computer monitors (with the simulation and rocket description – see Figure 1 below) and their data sheets, inviting them to recognize these artifacts as having answers to the group’s various questions. One student (C) has just finished explaining how he would like to use computer software to drag various features onto simulated rockets for purposes of comparison. While looking and orienting toward C (line 1), the teacher gestures toward the computer monitors and data sheets (line 2), and thereby encourages the students to recognize their simulation artifacts as already embodying what they need for their questions. Having been prompted by the teacher, the students attend to what is the same or different about the simulated rockets in the description list. The transcription continues as follows:

7      C:     These are both of the same thing o-

8      B:     This one is different ((gestures toward monitor))

9      J:     Yeah but it has - uh

10           (0.4)

11     C:     A pointy nose cone

12     C:     But it's not the same engine.

13     J:     Yeah it is

14     B:     Yes it is

15     J:     Compare two and one.

16     C:     Oh

17     B:     It's the same engine.

18     J:     So if you compared two and one

19     C:     Oh yeah I see I see I see

While the teacher remains silent, the students look toward their computer monitors and data sheets, and talk about the features of particular rockets. When C claims that two rockets are the same (line 7), B corrects by literally pointing out a difference (line 8). In collaboration with J, C notices that the rocket B referred to has a pointy nose cone (line 11) in contrast to rocket one’s rounded nose – still insisting, though, that “it’s not the same engine” (line 12). J and B quickly disagree with C (lines 13 and 14), and J prompts a comparison between rockets “two and one” (line 15). Through a series of discourse markers, C shows that he has a new understanding: the particle “oh” marks a change in his information state, and the recycled words “I see” provide additional evidence.

Through group interaction involving both vocal and visible forms of communication, participants’ shifting understandings were made visible to each other (and to us), and through this process were eventually improved. C showed understanding of the need to compare rockets before he recognized the simulation software as the needed comparison activity. C's displayed vision of experimental design was essentially the same as the teacher's – C simply failed to see that the work of designing and data gathering had already been done. We are not claiming that this knowledge is something that individuals have acquired and firmly mastered; what interests us is the ability that they demonstrate in concrete interactional learning situations. Across videotaped episodes this knowledge is seen to be fragile and inconsistent, leading us to suspect that it may be dependent upon situational details, group dynamics, and knowledge-building processes.

Implications for Artifact Design

This analysis has implications for the design of the educational experience and of the artifacts that comprise it. The rocket simulation designer might have thought that putting the list of rocket characteristics so prominently on the computer screen would automatically enable the users to select rockets for comparison. The teacher actually began the session by having students read this list out loud; he might have thought that the reading and the carefully located list would make the implications unmistakable for the students. But we have just seen that it took a relatively extensive and collaborative effort at comparison before the structure of the information could become meaningful to the students such as C. They had to engage in particular ways with the simulation, rocket list, and data sheet artifacts within a collaborative task in order to learn something that a software designer might have taken for granted.

This interaction offers another insight with (re)design implications. C had offered an alternative to the simulation as he then saw it. In his effort to demonstrate his vision, he was actually very involved with the artifact on the screen, proposing dragging features to create rockets with the right set of features for comparing – even while not yet understanding the affordances that already existed in the given rockets. A designer who takes seriously the possibility that what C proposes might be more effective could consider making the design of the rockets an interactive aspect of the simulation. Rather than presenting rockets with already fixed sets of characteristics (which is what proved so hard for C to focus on), an interactive design would provide "parts" and require students to configure pairs of rockets with sets of characteristics that would allow them to measure the effect of different nose cones, etc. The results would be the same from a mathematical viewpoint, but the process might be more meaningful and insightful for the students and involve them interactively at an earlier stage of experimental design in scientific inquiry. It is also possible, of course, that it might be more confusing for students and chaotic for the group, but that is an empirical question to be decided after another trial and micro-ethnographic analysis.

The log for this data is available at http://GerryStahl.net/readings/simrocket/log.htm. The digitized clip is available at: http://GerryStahl.net/readings/simrocket/clip16.mov. The full detailed transcript is available at: http://GerryStahl.net/readings/simrocket/t16.html.

7. Pilot Studies Conducted

The proposed project builds upon a series of activities that we have already started to work on. These activities – that either grew out of our previous engagements or were conducted to explore the basis for this proposal – have led to the design of our project and are suggestive of its probable success.

SimRocket (http://GerryStahl.net/previous/simrocket)

The PI designed and implemented SimRocket, a computer simulation of a rocket launch. The height attained by a simulated rocket is dependent upon its engine, nose-cone shape, fin configuration, and surface texture – as well as a random noise factor. The simulation was originally designed as part of a larger “Mission to Mars” curriculum built around launching model rockets and used with troubled middle school students in a remedial summer project. The PI was invited to try it in a local Boulder school with five boys engaged in a model rocket science project. The teacher guided the students (grouped in front of two computers) to fire each of 7 virtual rockets with different characteristics six times and to average the resultant heights in order to predict the height of an 8^th rocket. Project staff then engaged in micro-ethnographic analysis of this three-hour interaction – during data sessions in the Communication Department, the seminar on artifact theory, a summer workshop on micro-ethnography, and pilot sessions for this proposal.

WebGuide (http://GerryStahl.net/webguide)

WebGuide is a knowledge-building environment for discussing topics via the Web developed by the PI and colleagues over the past two years. It has been used in a middle school environmental science class and in college seminars on CSCL (Stahl, 1999c) . WebGuide goes beyond similar discussion-based systems by supporting the representation and development of personal and group perspectives (Stahl, 1999a; Stahl, 1999b) .

VirtualBiologyLab (http://www.virtuallaboratory.net)

VirtualBiologyLab is a Web-based curriculum to substitute for biology wet labs in introductory freshman biology courses for non-majors. Currently under development, it will soon consist of ten labs, each of which takes approximately three hours for a student to work through. Project staff has begun to review this software with the designer, Mike Klymkowsky, who is closely involved in this project. While the scientific content of the curriculum has been carefully thought through, the software artifact will cause many problems for students until it is subjected to thorough testing in naturalistic situations.

Medical PBL

Beginning students in the Medical School at Southern Illinois University have the option of taking a Problem-Based Learning (PBL) track for their first two years. Traditional educational assessments indicate that students opting for this are better prepared for the rest of their medical studies than those who attend lectures (Barrows, 1994) . PBL students learn by working collaboratively in teams of five students and a tutor (facilitator) to investigate specific medical cases (problems). Koschmann has undertaken many studies of this approach to education, increasingly relying upon micro-ethnography and collaborating with trained micro-ethnographers. His experience has provided insight into the workings of collaborative learning in this particular successful setting, and has convinced us of the potential of micro-ethnography as a methodology for studying the role of artifacts in collaborative learning.

Artifacts Seminar (http://GerryStahl.net/readings)

The PI organized a seminar on artifact theory, primarily as a pilot project for this proposal. Core members of the project team met along with other faculty and graduate students from Communication, Education, Philosophy, and Computer Science. We reviewed theoretical texts on the nature of artifacts from cognitive science, CSCL, communication, cultural studies, psychology, philosophy, and social theory. We also held data sessions on episodes from the SimRocket tapes. Out-of-class discussions were held in WebGuide and we conducted a SimRocket experiment mediated by WebGuide.

StateTheEssence (http://GerryStahl.net/projects/essence)

StateTheEssence is Web-based software developed by the PI to help middle school students develop their text summarization skills. It was used in an interdisciplinary four-year research project at a local Boulder public school. After undergoing considerable revision and refinement based on testing with students, the software was shown to improve text summarization, particularly in cases where the original text was somewhat difficult for the student to understand (Kintsch et al., 2000; Stahl & dePaula, 2001; Steinhart, 2000) . Evaluation of this software was conducted by means of controlled experiments and teacher ratings.

JIME (http://www-jime.open.ac.uk)

The web-based Journal of Interactive Media in Education, JIME, conducts group reviews of submitted articles on-line, and then includes an edited version of the review discourse with the published version. We are currently analyzing the on-line review discussions to draw conclusions about how the journal software and practices might be improved (Lenell & Stahl, 2001) . This study provides us with some experience in analyzing on-line discourse, which will be important in the final stage of our proposed project.

8. Results from Prior Support

Organizational Memory and Organizational Learning

“Conceptual Frameworks and Computational Support for Organizational Memories and Organizational Learning (OMOL),” PIs: Gerhard Fischer, Gerry Stahl, Jonathan Ostwald, September 1997 – August 2000, $725,000, from NSF CSS Program #IRR-9711951.

This grant led to the current proposal’s focus on Web-based learning environments. The OMOL project started from a model of computer support for organizations as Domain-Oriented Design Environments (DODEs) in which both domain knowledge and local knowledge are stored in the form of artifact designs and associated design rationale. This CSCW model evolved into one of Collaborative Information Environments (CIEs), that emphasized the interactive, asynchronous, persistent discussion of concepts and issues within an organization. Gradually, interest in organizational learning aspects led to involvement in CSCL and the model of collaborative Knowledge-Building Environments (KBEs). A number of software prototypes were developed to explore the use of the Web as a communication and collaboration medium:

Work on this grant led to the focus on KBEs as models of computer support for organizational memory and collaborative learning. In particular, it provided a number of different systems, each with useful functionality. As we tested and deployed these systems, we confronted serious issues of adoption and focused our concerns increasingly on socio-technical and social informatics issues: motivation, media competition, critical mass, social practices, seeding, management, re-seeding, convergence of ideas, peer-to-peer collaboration, deployment strategies. These issues led to a new research agenda (Stahl, 2000b) and this proposal.

Environmental Perspectives in a Middle School Classroom

“Collaborative Web-Based Tools for Learning to Integrate Scientific Results into Social Policy,” PIs: Ray Habermann, Gerry Stahl, November 1998 – July 1999, $89,338, NSF, #EAR-9870934.

This grant funded the initial implementation of WebGuide as an integrated Java applet KBE supporting personal and group perspectives. It was a joint effort between the PI, a middle school teacher, and a research group at the National Oceanographic and Atmospheric Administration (NOAA) labs in Boulder. The teacher taught an environmental science class in which he wanted to spend the year having his students interview various adults and construct a set of contrasting perspectives (conservationist, regulatory, business, community) on a particular local environmental issue that the students had previously been involved in. WebGuide was used by the students to collect notes on their interviews and to formulate personal and team perspectives on the issue. Results of this software trial were analyzed and presented at the AERA, CSCL, ICLS, CILT, WebNet, and Group conferences (Stahl, 1999c) . These findings led to a number of revisions of WebGuide, including the separation of the perspectives mechanism from the Web interface, and recognition of the need for software architectures, standards, and components to support flexible rapid prototyping of KBEs.

Interoperability among Knowledge-Building Environments

“Interoperability Among Knowledge-Building Environments,” PI: Gerry Stahl, September 1999 – August 2000, $9,124, from NSF-funded Center for Innovative Learning Technology (CILT), Subcontract #17-000359 under NSF grant #EIA-9720384.

This was a seed grant whose purpose was to stimulate collaboration among KBE research groups. Part of the intention of the grant was to prepare a proposal for fuller funding, such as the present proposal. This grant resulted in a semester-long student project involving three graduate and three undergraduate students (one collaborating virtually from Germany using WebGuide) creating an XML DTD that defines a data format for data imported from several different KBE prototypes and displayed in a Web browser using XSL. The grant supported a workshop organized by the PI, entitled “Collaborating on the Design and Assessment of KBEs in the 2000's” at CSCL ’99 at Stanford. This workshop attracted over 60 participants and was preceded by an on-line discussion of 28 submitted position papers. This grant led to the emphasis on collaboration among KBE research groups and the need to develop and disseminate theory and methodology for developing educational artifacts, as proposed here.

Incorporating Automated Text Evaluation into Collaborative Software Environments

“Allowing Learners to be Articulate: Incorporating Automated Text Evaluation into Collaborative Software Environments,” PIs: Walter Kintsch, Gerhard Fischer, Thomas Landauer (Stahl served as co-PI and primary software developer), calendar 1997-2000, $1,400,000 for four-years, from the McDonnell Foundation’s Cognitive Science in Education (CSEP) program.

This grant supported the design, implementation, and testing of StateTheEssence, Web-based software to teach middle school students summarization skills (Kintsch et al., 2000; Stahl & dePaula, 2001; Steinhart, 2000) .

New Media to Support Collaborative Knowledge-Building

“New Media to Support Collaborative Knowledge-Building: Beyond Consumption and Chat,” PI: Stahl, November 2000 through March 2001, $19,752, from the Lab for New Media at CU, sponsored by the Omnicom Corporation.

The grant is to test WebGuide in the Artifacts Seminar and to make a number of technical improvements to WebGuide’s functionality.

9. Contributions to ROLE Goals and Potential Impact

Recent research on learning and on technology in education – as surveyed in the Report to the President (Panel on Educational Technology, 1997) and in How People Learn (Bransford et al., 1999) – stresses the potential of innovative constructivist educational approaches to foster deep understanding. The latter document, for instance, concludes that computer technology “has great potential to enhance student achievement and teacher learning, but only if it is used appropriately” (Ch. 9).

Pursuing the Potential of Computer-Supported Collaborative Learning (CSCL)

Our project proposes to investigate at a detailed level the key learning processes in a computer-supported collaborative knowledge-building environment. We believe that collaborative learning has a great potential to foster deep knowledge-building when it brings together the perspectives of multiple students in a productive way. However, this requires a more detailed understanding of how collaborative knowledge-building processes work. We further anticipate that computer support has the potential to facilitate collaboration by removing communication limitations and by helping to manage the complexity of ideas and interactions. However, this requires carefully designed knowledge-management software applications tuned to the needs of collaborative learning. This project will investigate the potential of CSCL through focused inquiry using micro-ethnographic methods to observe how learning takes place in collaborative settings under conditions of computer support.

A Methodology for Assessing Computer Support for Collaborative Learning

We will develop a methodology for assessing the role of computational cognitive artifacts in supporting collaborative knowledge-building. We will start with micro-ethnography as a methodology for analyzing the construction of social organization in small group communicative interactions. We will adapt this methodology to address the concerns of human-computer interaction. This will provide future researchers and designers with an approach for studying CSCL at a level of detail that can usefully drive iterative software design. Our approach will make visible the learning that is taking place within small groups of students engaged in computer-supported tasks, thereby indicating what is needed to help students learn how to take advantage of the computational artifacts and where the design of the artifacts is effective or problematic.

Increased Understanding of the Role of Computational Cognitive Artifacts in Learning and Education

Our project will increase our understanding of the social origins and maintenance of knowledge, how knowledge is embodied in artifacts, how the artifacts transmit this knowledge, and how people learn to use the artifacts to make cognitive use of the embedded knowledge. In complex learning systems like VirtualBiologyLab, such knowledge is presented in many forms: explicit textual knowledge like background information and procedural instructions, animations and other representations of physical processes, simulations that generate data for analysis, interactive tasks, mini-tests of student understanding, interactive tools for manipulating data, and lab results that require guided interpretation. We will study how students in small groups successfully and problematically collaboratively construct these various forms of knowledge through interaction with the software and with each other. This will increase our theoretical understanding as CSCL researchers of the role of computational cognitive artifacts in learning and education.

Impact on the CSCL Research Community

There is a vigorous and growing CSCL research community in the United States and globally, despite rather limited funding opportunities here beyond ROLE. Last year over 700 people attended CSCL ’99 and this year the first European CSCL conference will take place. In January 2002, CSCL will be hosted at the University of Colorado, with all of this project’s staff actively involved. The focus of the conference will be on new methodologies and deepened theoretical frameworks. We anticipate that our project will provide an organizational basis for establishing at CSCL 2002 on-going global research collaborations. Our project will provide an example of a systematic attempt to apply a new methodology within the field and to elaborate a coherent theoretical framework that addresses core issues of CSCL.

ROLE Quadrant 2 and 3

While ROLE projects related to neuroscience may focus quite literally inside the head of an individual learner, this project will look outside at the social interactions through which knowledge is constructed and shared – and at the same time evidenced – in collaborative educational settings. The project is situated in ROLE’s quadrant 2 because it builds bridges from the cognitive sciences to research on learning. Through micro-ethnographic studies of educational environments, it undertakes fundamental research on behavioral, cognitive, affective, and social aspects of human learning as mediated by artifacts. It is also situated in quadrant 3 because it is building a stronger research base to support educational approaches (e.g., scaffolded collaborative small groups), curriculum materials (VirtualBiologyLab), and technological tools (WebGuide) to facilitate the learning process. In particular, the project builds on diverse cognitive theories of the role of artifacts and on methods of micro-ethnography in order to develop and refine new education research and evaluation methods for analyzing the role of computational cognitive artifacts in collaborative learning and education – and investigates these theories and methods in formal science learning settings in middle school, high school and college.

References Cited

Bakhtin, M. (1986) Speech Genres and Other Late Essays, (V. McGee, Trans.), University of Texas Press, Austin, TX.

Barrows, H. (1994) Practice-based Learning: Problem-Based Learning Applied to Medical Education, SIU School of Medicine, Springfield, IL.

Bereiter, C. (2000) Education and Mind in the Knowledge Age. Available at: http://csile.oise.utoronto.ca/edmind/main.html .

Birdwhistell, R. (1970) Kinesics and Context, University of Pennsylvania Press, Philadelphia, PA.

Bødker, S. (1989) A human activity approach to user interfaces, Human-Computer Interaction, 4 , pp. 171-195.

Bødker, S. (1996) Applying activity theory to video analysis: How to make sense of video data in HCI. In B. Nardi (Ed.) Context and Consciousness: Activity Theory and Human-Computer Interaction, MIT Press, Cambridge, MA, pp. 147-174.

Boland, R. J. & Tenkasi, R. V. (1995) Perspective making and perspective taking in communities of knowing, Organization Science, 6 (4), pp. 350-372.

Bransford, J., Brown, A., & Cocking, R. (Eds.) (1999) How People Learn: Brain, Mind, Experience, and School, National Research Council, Washington, DC. Available at: http://books.nap.edu/html/howpeople1/ .

Brody, C. & Davidson, N. (Eds.) (1998) Professional Development for Cooperative Learning: Issues and Approaches, SUNY Press, Albany, NY.

Bruner, J. (1990) Acts of Meaning, Harvard University Press, Cambridge, MA.

Cole, M. & Griffin, P. (1980) Cultural amplifiers reconsidered. In D. Olson (Ed.) The Social Foundations of Language and Thought, Norton, New York, NY.

Craig, R. T. & Sanusi, A. L. (2000) "I'm just saying": Discourse markers of standpoint continuity, Argumentation, 14 (4), pp. 425-445.

Craig, R. T. & Tracy, K. (1995) Grounded practical theory: The case of intellectual discussion, Communication Theory, 5 , pp. 248-272.

Cuthbert, A. (1999) Designs for collaborative learning environments: Can specialization encourage knowledge integration?, In: Proceedings of Computer Supported Collaborative Learning (CSCL '99), Palo Alto, CA, pp. 117-126.

Dewey, J. & Bentley, A. (1949/1991) Knowing and the known. In J. A. Boydston (Ed.) John Dewey: The Later Works, 1925-1953, Volume 16, SIU Press, Carbondale, IL.

Donald, M. (1991) Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition, Harvard University Press, Cambridge, MA.

Engelbart, D. C. (1995) Toward augmenting the human intellect and boosting our collective IQ, Communications of the ACM, 38 (8), pp. 30-33.

Engeström, Y., Miettinen, R., & Punamaki, R.-L. (Eds.) (1999) Perspectives on Activity Theory, Cambridge University Press, New York, NY.

Frohlich, D. & Luff, P. (1990) Applying the technology of conversation to the technology for conversation. In P. Luff, N. Gilbert, & D. Frohlich (Eds.), Computers and Conversation, Academic Press, New York, NY.

Gadamer, H.-G. (1960/1988) Truth and Method, Crossroads, New York, NY.

Garfinkel, H. (1967) Studies in Ethnomethodology, Prentice-Hall, Englewood Cliffs, NJ.

Geertz, C. (1973) The Interpretation of Cultures, Basic Books, New York, NY.

Glaser, B. G. & Strauss, A. L. (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine de Gruyter, New York, NY.

Glenn, P., Koschmann, T., & Conlee, M. (1999) Theory sequences in a problem-based learning group: A case study, Discourse Processes, 27 , pp. 119-133.

Hakkarainen, K. & Lipponen, L. (in prep) Epistemology of inquiry and computer-supported collaborative learning. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL2: Carrying Forward the Conversation, Lawrence Erlbaum Associates, Mahwah, NJ.

Hall, R. (1996) Representation as shared activity: Situated cognition and Dewey's cartography of experience, The Journal of the Learning Sciences, 5 (3), pp. 209-238.

Hall, R. & Stevens, R. (1995) Making Space: A Comparison of Mathematical Work in School and Professional Design Practices. In S. L. Star (Ed.) The Cultures of Computing, Blackwell Publishers, Oxford, UK.

Hegel, G. W. F. (1807/1967) Phenomenology of Spirit, (J. B. Baillie, Trans.), Harper & Row, New York, NY.

Heidegger, M. (1927/1996) Being and Time: A Translation of Sein und Zeit, (J. Stambaugh, Trans.), SUNY Press, Albany, NY.

Hollan, J., Hutchins, E., & Kirsh, D. (2000) Distributed cognition: Toward a new foundation of human-computer interaction research, ACM Transactions on Computer-Human Interaction, 7 (2), pp. 174-196.

Husserl, E. (1936/1989) The origin of geometry. (D. Carr, Trans.), In J. Derrida (Ed.) Edmund Husserl's Origin of Geometry: An Introduction, University of Nebraska Press, Lincoln, NE, pp. 157-180. (Also in Husserl, E. (1970) The Crisis of European Sciences and Transcendental Philosophy, Northwestern University Press, Evanston, IL, pp. 353-378.)

Hutchins, E. (1996) Cognition in the Wild, MIT Press, Cambridge, MA.

Hutchins, E. (1999) Cognitive artifacts. In MIT Encyclopedia of the Cognitive Sciences, MIT Press, Cambridge, MA. Available at: http://cognet.mit.edu/library/MITECS.

Hutchins, E. & Palen, L. (1998) Constructing meaning from space, gesture and speech. In L. B. Resnick, R. Saljo, C. Pontecorvo, & B. Burge (Eds.), Discourse, Tools, and Reasoning: Situated Cognition and Technologically Supported Environments, Springer Verlag, Heidelberg, Germany.

Jong, F. d., Diermanse, E., & Lutgens, G. (in prep) Computer supported collaborative learning in university and vocational education. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL2: Carrying Forward the Conversation, Lawrence Erlbaum Associates, Mahwah, NJ.

Jordan, B. & Henderson, A. (1995) Interaction analysis: Foundations and practice, Journal of the Learning Sciences, 4 (1), pp. 39-103. Available at: http://lrs.ed.uiuc.edu/students/c-merkel/document4.HTM .

Keegan, D. (1986) Foundations of Distance Education, Routledge, London, UK.

Keil-Slawik, R. (1992) Artifacts in software design. In C. Floyd, H. Züllighoven, R. Budde, & R. Keil-Slawik (Eds.), Software Development and Reality Construction, Springer Verlag, Berlin, Germany, pp. 168-188.

Kendon, A. (1990) Conducting Interaction: Patterns of Behavior in Focused Encounters, Cambridge University Press, Cambridge, UK.

Kintsch, E., Steinhart, D., Stahl, G., Matthews, C., Lamb, R., & the LSA Research Group (2000) Developing summarization skills through the use of LSA-backed feedback, Interactive Learning Environments, 8 (2), pp. 87-109.

Koschmann, T. (Ed.) (1996) CSCL: Theory and Practice of an Emerging Paradigm, Lawrence Erlbaum Associates, Hillsdale, NJ.

Koschmann, T. (1999) Toward a dialogic theory of learning: Bakhtin's contribution to learning in settings of collaboration, In: Proceedings of Computer Supported Collaborative Learning (CSCL '99), Palo Alto, CA, pp. 308-313. Available at: http://kn.cilt.org/cscl99/A38/A38.HTM .

Koschmann, T. (in press) A third metaphor for learning: Toward a form of trans-actional inquiry into inquiry. In D. Klahr & S. Carver (Eds.), Cognition and Instruction: 25 Years of Progress, Lawrence Erlbaum Associates, Mahwah, NJ.

Koschmann, T. & Glenn, P. (submitted) A tale of two theories: A sequential analysis of hypothesis generation. Available at: http://edaff.siumed.edu/tk/articles/theories/ .

Koschmann, T., Glenn, P., & Conlee, M. (1997) Analyzing the emergence of a learning issue in a problem-based learning meeting, Medical Education Online, 2 (1). Available at: http://www.utmb.edu/meo/res00003.pdf .

Koschmann, T., Glenn, P., & Conlee, M. (2000) When is a problem-based tutorial not tutorial? Analyzing the tutor's role in the emergence of a learning issue. In D. Evensen & C. Hmelo (Ed.) Problem-Based Learning: A Research Paradigm on Learning Interactions, Lawrence Erlbaum, Mahwah, NJ, pp. 53-74.

Koschmann, T. & LeBaron, C. (submitted) Learner articulation as interactional jointing: Studying gesture conversationally. Available at: http://edaff.siumed.edu/tk/articles/gestures/ .

Koschmann, T., Ostwald, J., & Stahl, G. (1998) Shouldn't we really be studying practice? [panel position paper], International Conference on the Learning Sciences (ICLS '98), Atlanta, GA. Available at: http://GerryStahl.net/publications/conferences/1998/icls98/ICLS Workshop.html.

Koschmann, T. & Stahl, G. (1998) Learning issues in problem-based learning: Situating collaborative information seeking. Workshop on Technologies for Collaborative Information Seeking [workshop position paper], ACM Conference on Computer Supported Cooperative Work (CSCW 98), Seattle, WA.

Lave, J. & Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, Cambridge, UK.

LeBaron, C. (1998) Building Communication: Architectural Gestures and the Embodiment of New Ideas, Ph. D. Dissertation, Department of Communication Studies, University of Texas at Austin, Austin, TX.

LeBaron, C. & Hopper, R. (1997) Social learning during therapy for self: How a preoccupation with administrative matters may constitute putty for social practice, Sixth International Conference on Language and Social Psychology, Ottawa, Canada.

LeBaron, C. & Koschmann, T. (1999) The conversation of gestures: Interaction and learner articulation, 49th Annual Conference of the International Communication Association, San Francisco, CA.

LeBaron, C. & Koschmann, T. (2001) Gesture and the transparency of understanding. In P. Glenn, C. LeBaron, & J. Mandelbaum (Eds.), Excavating the Taken-for-Granted: Essays in Language and Social Interaction, Lawrence Erlbaum, Mahwah, NJ.

LeBaron, C. & Streeck, J. (2000) Gesture, knowledge and the world. In D. McNeill (Ed.) Language and Gesture, Cambridge University Press, Cambridge, UK.

Lenell, E. & Stahl, G. (2001) Evaluating affordance short-circuits by reviewers and authors participating in on-line journal reviews (submitted), In: Proceedings of European Computer Supported Collaborative Learning (ECSCL '01), Maastricht, NL.

Marx, K. (1867/1976) Capital, Volume I, (B. Fowkes, Trans.), Vintage, New York, NY. Available at: http://www.karl-marx.org/archive/marx/works/1867-c1/index.htm .

McIlvenny, P. (1990) Communicative action and computers: Re-embodying conversation analysis? In P. Luff, N. Gilbert, & D. Frohlich (Eds.), Computers and Conversation, Academic Press, New York, NY.

Nardi, B. (Ed.) (1996) Context and Consciousness: Activity Theory and Human-Computer Interaction, MIT Press, Cambridge, MA.

Norman, D. A. (1993) Things That Make Us Smart, Addison-Wesley Publishing Company, Reading, MA.

Panel on Educational Technology (1997) Report to the President on the Use of Technology to Strengthen K-12 Education in the United States, The Whitehouse, Washington, DC. Available at: http://www.whitehouse.gov/WH/EOP/OSTP/NSTC/PCAST/k-12ed.html .

Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas, Basic Books, New York, NY.

Pea, R. (1985) Beyond amplification: Using the computer to reorganize mental functioning, Educational Researcher, 20 (4), pp. 167-182.

Pea, R. (1993) The collaborative visualization project, Communications of the ACM, 36 (5), pp. 60-63.

Roschelle, J. (1996) Learning by collaborating: Convergent conceptual change. In T. Koschmann (Ed.) CSCL: Theory and Practice of an Emerging Paradigm, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 209-248.

Russell, T. (Ed.) (1999) The No Significant Difference Phenomenon, Mindspring Press. Available at: http://cuda.teleeducation.nb.ca/nosignificantdifference/ .

Sacks, H. (1992) Lectures on Conversation, Blackwell, Oxford, UK.

Scardamalia, M. & Bereiter, C. (1996) Computer support for knowledge-building communities. In T. Koschmann (Ed.) CSCL: Theory and Practice of an Emerging Paradigm, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 249-268.

Schön, D. A. (1983) The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York, NY.

Schön, D. A. (1987) Educating the Reflective Practitioner, Jossey-Bass, San Francisco, CA.

Stahl, G. (1993) Interpretation in Design: The Problem of Tacit and Explicit Understanding in Computer Support of Cooperative Design, Ph.D. Dissertation, Department of Computer Science, University of Colorado, Boulder, CO. Available at: http://GerryStahl.net/publications/dissertations/computer .

Stahl, G. (1999a) POW! Perspectives on the Web, In: Proceedings of WebNet World Conference on the WWW and Internet (WebNet '99), Honolulu, HA. Available at: http://GerryStahl.net/publications/conferences/1999/webnet99/webnet99.html.

Stahl, G. (1999b) Reflections on WebGuide: Seven issues for the next generation of collaborative knowledge-building environments, In: Proceedings of Computer Supported Collaborative Learning (CSCL '99), Palo Alto, CA, pp. 600-610. Available at: http://GerryStahl.net/publications/conferences/1999/cscl99/  -- and -- http://kn.cilt.org/cscl99/A73/A73.HTM .

Stahl, G. (1999c) WebGuide: Guiding collaborative learning on the Web with perspectives, Annual Conference of the American Educational Research Association (AERA '99), Montreal, Canada. Available at: http://GerryStahl.net/publications/conferences/1999/aera99/  -- and -- http://www-jime.open.ac.uk/00/stahl/stahl-t.html .

Stahl, G. (2000a) book review: Professional Development for Cooperative Learning: Issues and Approaches, Teaching and Learning in Medicine: An International Journal, 12 (4). Available at: http://GerryStahl.net/publications/journals/medicine/coop_learn.html .

Stahl, G. (2000b) Collaborative information environments to support knowledge construction by communities, AI & Society, 14 , pp. 1-27. Available at: http://GerryStahl.net/publications/journals/ai&society/ .

Stahl, G. (in prep) Rediscovering CSCL. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL2: Carrying Forward the Conversation, Lawrence Erlbaum, Mahway, NJ.

Stahl, G. & dePaula, R. (2001) Evolution of an interactive medium for learning to write summaries (in preparation). Available at: http://GerryStahl.net/publications/journals/ILE/ile.html .

Stahl, G. & Herrmann, T. (1999) Intertwining perspectives and negotiation, In: Proceedings of International Conference on Supporting Group Work (Group '99), Phoenix, AZ. Available at: http://GerryStahl.net/publications/conferences/1999/group99/ .

Stahl, G. & Sanusi, A. (2001) It doesn't take a rocket scientist: Multi-layered perspectives on collaborative learning activities in a middle school rocket simulation project (submitted), In: Proceedings of 22nd Annual Ethnography in Education Research Forum, Philadelphia, PA.

Steinhart, D. (2000) Designing, Testing, and Evaluating Summary Street, an LSA-Based Summarization Tool, Ph. D. Dissertation, Department of Psychology, University of Colorado, Boulder, CO.

Streeck, J. (1983) Social Order in Child Communication: A Study in Microethnography, Benjamins, Amsterdam, NL.

Streeck, J. (1993) Gesture as communication I: Its coordination with gaze and speech, Communication Monographs, 60 (Dec.), pp. 275-299.

Streeck, J. (1996) How to do things with things, Human Studies, 19 , pp. 365-384.

Suchman, L. (1987) Plans and Situated Actions: The Problem of Human-Machine Communication, Cambridge University Press, Cambridge, UK.

Suchman, L. & Trigg, R. (1991) Understanding practice: Video as a medium for reflection and design. In J. Greenbaum & M. Kyng (Eds.), Design at Work: Cooperative Design of Computer Systems, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 65-90.

Vygotsky, L. (1930/1978) Mind in Society, Harvard University Press, Cambridge, MA.

Vygotsky, L. (1934/1986) Thought and Language, MIT Press, Cambridge, MA.

Winograd, T. & Flores, F. (1986) Understanding Computers and Cognition: A New Foundation of Design, Addison-Wesley, Reading, MA.

Go to top of this page