Gerry Stahl’s assembled texts volume #10

Essays in

Group-Cognitive

Science

Gerry Stahl

Gerry Stahl's Assembled Texts

1. Marx and Heidegger

2. Tacit and Explicit Understanding in Computer Support

3. Group Cognition: Computer Support for Building Collaborative Knowledge

4. Studying Virtual Math Teams

5. Translating Euclid: Designing a Human-Centered Mathematics.

6. Constructing Dynamic Triangles Together: The Development of Mathematical Group Cognition

7. Essays in Social Philosophy

8. Essays in Personalizable Software

9. Essays in Computer-Supported Collaborative Learning

10. Essays in Group-Cognitive Science

11. Essays in Philosophy of Group Cognition

12. Essays in Online Mathematics Interaction

13. Essays in Collaborative Dynamic Geometry

14. Adventures in Dynamic Geometry

15. Global Introduction to CSCL

16. Editorial Introductions to ijCSCL

17. Proposals for Research

18. Overview and Autobiographical Essays

19. Theoretical Investigations

20. Works of 3-D Form

21. Dynamic Geometry Game for Pods

Gerry Stahl’s assembled texts volume #10

Essays in

Group-Cognitive

Science

Gerry Stahl

Gerry Stahl

Gerry@GerryStahl.net

www.GerryStahl.net

Copyright © 2015, 2022 by Gerry Stahl

Published by Gerry Stahl at Lulu.com

Printed in the USA

ISBN 978-1-329-59252-0 (paperback)

ISBN 978-1-329-59379-4 (ebook)

Contents

Contents      5

Introduction      6

Part A: Toward a Science of Group Cognition      14

1.      Group Cognition as a Foundation for the New Science of Learning      15

2.      A View of Computer-Supported Collaborative Learning Research and its Lessons      38

Part B: Theories of Group Cognition      60

3.      How I View Learning and Thinking in CSCL Groups      61

4.      Theories of Group Cognition: Foundations for CSCL and CSCW      93

Part C: Analyzing Group Cognition      114

5.      How a Virtual Math Team Structured its Problem Solving      115

6.      Interaction Analysis of a Biology Chat      138

7.      Coding Scheme for Sequential Discourse      170

8.      Coordinating Collaborative Action in Online Math Problem-Solving      181

9.      Methodological issues in developing a multi-dimensional coding procedure for small group chat communication      197

10.      A Multi-Dimensional Coding Scheme for Mathematical Collaboration      216

References      241

Introduction

T

his volume of collected writings looks at the study of group cognition as a scientific enterprise. Although it includes case studies of empirical data of students interacting and although it documents methodological systems, it does not follow the textual rubric that some educational researchers and academic journals consider standard. Instead, it adopts the essay genre. An essay is a literary reflection on a delimited topic, which tries to evoke what is special and interesting about that topic in a way that is uniquely appropriate to expressing the topic.

A scientific approach to the study of group cognition need not be a bureaucratic intellectual prison for inquiry, nor a religion bowing to the traditions, paradigms and expectations of established sciences from quite different domains. Explorations of group cognition need not follow the medical model of statistical analyses of randomized trials. Rather, it can be a creative investigation—as long as it follows the basic principle evoked in the title of the closing chapter of Group Cognition (Stahl 2006): “Thinking at the Small-Group Unit of Analysis.” This unit of analysis is the core of group-cognition science: to grasp what is taking place in the interactions of a collaborating group. Group Cognition was the first of a quartet of four published volumes reporting on the Virtual Math Teams Project.

The VMT Project was proposed and pursued in order to investigate group cognition systematically and rigorously. The project is described from multiple analytic perspectives in Studying Virtual Math Teams (Stahl 2009). The methodology documented in that book included coding and interaction analysis; statistics and interpretation; findings and insights.

Subsequent work on the VMT Project began to focus on a particular domain of learning, targeting the educational topic of collaborative dynamic geometry. This effort is described in Translating Euclid (Stahl 2013d). That book provides eleven distinct lenses on the VMT research, ending with a chapter that characterizes the approach to the group-cognitive science as well as to the mathematical domain: “Design-Based Research: Human-Centered Geometry.”

Finally, a fine-grained longitudinal case study of a virtual math team learning the basics of collaborative dynamic geometry—Constructing Dynamic Triangles Together (Stahl 2015)—offers a concrete and detailed example of analyzing the development of mathematical group cognition. The approach taken there uses interaction analysis to identify group practices that the team adopted in each of its eight online sessions. It shows how the team negotiated, adopted and re-used its group practices as a core component of its collaborative learning.

The VMT Project represents a paradigmatic CSCL effort. It developed innovative technologies, collaborative pedagogies, school-oriented curricular materials, analytic approaches and theoretical formulations. It supported group learning in online small groups of students in school settings. The essays collected below consider methodological approaches, issues, techniques and materials. This is the best source for understanding the VMT data-analysis methodology, as it developed over 12 years. Such an overview was not possible in focused papers or even in thematic books. Yet, it is something that may be of particular interest for CSCL researchers and graduate students. The current volume collects essays and supporting materials from throughout the VMT Project (2002-2015). These were key documents in determining and recording the development of the project’s scientific method, which did not, however, find a place in the volumes mentioned above. Some of the methodological documentation has not been published elsewhere.

Perhaps one of the most significant findings of the VMT Project involved an analysis of the structure of problem-solving discourse. This finding was never discussed in any of the four major publications. It is documented extensively here. It builds on the central discovery of Conversation Analysis: that conversation is driven by (and thereby structured by) “adjacency pairs” of utterances, which elicit and respond to each other (like question/answer pairs). In mathematical problem solving, however, it is important to look at larger argumentative structures than such adjacency pairs—so-called “longer sequences.”

In VMT, we found that the larger structures of online problem-solving discussions were built out of response pairs of chat postings. Furthermore, in their interactions, students tended to structure their discussions implicitly into a hierarchy of structures: events, sessions, themes, discourse moves, response pairs, postings and indexical references. Conversation Analysis had worked out a well-defined understanding of much of this structure. This suggested that a coding scheme could be defined for coding online mathematical problem solving, grounded in such an analysis.

In CSCL, it is common to turn to coding schemes to provide a seemingly “objective,” scientific approach to analysis of data. Various coding schemes may be useful for pursuing specific research questions, such as comparing two experimental conditions quantitatively. However, the coding schemes tend to have various problems: they are not theoretically grounded in the structure of interaction, they are not applicable in a general way, they require indoctrination of coders. Above all, they typically lose the sequentiality of the temporal flow of the discussion, which is perhaps the most important characteristic of collaborative interaction and discourse in general.

A unique approach considered in the VMT Project—but not documented outside of this collection of essays and coding documents—would be to code adjacency pairs (or chat response pairs), rather than individual postings. This would shift the unit of analysis from individual representations to interpersonal interactions. In addition, segments of discourse could be coded based upon the structure of interaction constituted by the student discourse itself. This could resolve the conflict between ethnomethodology and statistical approaches (popularly referred to as the conflict between qualitative and quantitative research).

In this volume, the structure of student interactions is worked out. Then, toward the end of the volume, several coding schemes are defined, based on the response structure of interaction. By coding response pairs rather than single utterances and by situating them in the hierarchical structure of dialog, these approaches focused on the small-group unit of analysis, rather than on individuals. They are illustrated by case studies of VMT data. This is a major contribution of this book.

The VMT Project was an active research collaboration, which I directed during my time on the Drexel University faculty: from September 2002 to September 2014. The project was funded by several federal grants totaling about $7,000,000. The VMT Project is continuing without me under the auspices of the Math Forum.

During the twelve years that I worked on the VMT Project, it combined core elements from a number of different paradigms of group-cognition science:

- Coding of response pairs,

- Design-based research,

- Interaction analysis of longer sequences and

- Identification of group practices.

These are discussed or documented in this volume.

The other important aspect of this manuscript is its introductory nature and the overviews it provides of the VMT research. Through my essays, I have tried to introduce various audiences to my ideas about group cognition and about research in educational technology. Thus, some of the essays present an approach informally, while others go over the same ground in more detail. The volume is divided into three Parts:

Part A of this volume consists of two introductions:

- The first essay introduces the idea of a science of group cognition. Various aspects of the analysis of group cognition are described and related to criteria from traditional paradigms of scientific research. (Stahl 2010)

- The second essay reproduces a presentation for students of computer science not familiar with CSCL, who might be inspired to pursue CSCL research. (Stahl 2012b)

Part B includes two discussions of the theory of group cognition, from different perspectives:

- First, the hierarchy of structures of discourse in group-cognition interactions is explained in a keynote address delivered in Hong Kong. It was then published in RPTEL (Stahl 2011c). A video of the presentation is available on YouTube: http://youtu.be/h5MpUJnTipM.

- Then, group cognition is situated within the various theoretical traditions that are popular in CSCW and CSCL. (Stahl 2013c)

Part C presents two case studies and documents coding schemes relevant to VMT:

- First, the essay that worked out the longer structure of problem solving, building on adjacency pairs. (Stahl 2011b)

- Then a case study in the domain of biology, using the VMT collaboration environment. This essay shows how interaction analysis can be used to provide rapid feedback on student usage of technology. (Stahl 2013a)

- An unpublished coding scheme is given for the analyses conducted in the VMT Project during 2013. This was an attempt to code adjacency pairs as defined in Conversation Analysis and to take into account the hierarchy of structures discussed in the earlier chapters. This coding scheme is applied to a log that has been analyzed in many VMT publications. It is applied to the complete data from SpringFest 2013 Group B Session 4 in a spreadsheet that is available at: http://GerryStahl.net/elibrary/science/codes4b.xls.

- A case study by Alan Zemel looks closely at how typical adjacency pairs operate in the analyzed log. He takes a Conversation Analysis approach to analyzing the interactional structure of problem-solving proposals and similar response structures. The discussion of proposals (and “failed proposals”) was a first finding of the VMT Project, already reported in Group Cognition. Zemel and colleagues later argued for developing a coding approach based on interactional adjacency pairs in Studying VMT. Most coding in CSCL takes place at the individual unit of analysis, coding isolated postings by individual students. This fails to capture much of the collaborative interaction among students. It implicitly assumes that individual minds are the agents of discourse, rather than that postings or utterances are primarily responses to the previous and anticipated utterances of others. The VMT Project, in contrast, assumes that contributions to the meaning-making process of interaction are formed by reference to other contributions, so that the meaning is created at the small-group, interactional unit of analysis, with the group as fundamental agent. Rationalist psychological models attribute the formation of contributions to the thinking of individual minds, but the process of forming discourse contributions is not accessible to the conscious mind, but takes place through hidden semantic references and responses. Therefore, a scientific investigation of meaning making cannot proceed through interviews of participants, but must rely more on analysis of response structures. As Zemel points out in this case study, most of the discourse sequences are collaborative, in that the meaning is constructed in the interplay among contributions by multiple participants.

- Finally, an article by (Strijbos & Stahl 2007) is reproduced. It describes a multi-dimensional coding scheme developed in the first year of the VMT Project. The results of using this coding were discussed in Chapters 22 and 23 of (Stahl 2009). After this effort to apply coding to the VMT context, the project switched to using its own adaptations of Conversation Analysis. The full coding scheme is then documented in the following section.

References

The essays in this volume were originally published as (Stahl 2010b; 2011a; 2011b; 2011c; 2011d; 2011f; 2011g; 2012a; 2012b; 2013a; 2013b; 2013c; Stahl & Strijbos 2004; Strijbos & Stahl 2007).

Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press. Acting with technology series. 510 pages. Web: http://GerryStahl.net/elibrary/gc . Doi: http://mitpress.mit.edu/books/group-cognition .

Stahl, G. (2009). Studying virtual math teams. New York, NY: Springer. Computer-supported collaborative learning series #11. 626 pages. Web: http://GerryStahl.net/elibrary/svmt . Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3 .

Stahl, G. (2010). Group cognition as a foundation for the new science of learning. In M. S. Khine & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education. (pp. 23-44). New York, NY: Springer. Web: http://GerryStahl.net/pub/scienceoflearning.pdf .

Stahl, G. (2011a). Coding scheme for sequential discourse. Web: http://GerryStahl.net/pub/coded4b.pdf .

Stahl, G. (2011b). How a virtual math team structured its problem solving. In the proceedings of the Connecting computer-supported collaborative learning to policy and practice: CSCL 2011 conference proceedings. N. Miyake, H. Spada & G. Stahl. Lulu: ISLS. Proceedings pp. 256-263.  Web: http://GerryStahl.net/pub/cscl2011stahl.pdf , http://GerryStahl.net/pub/cscl2011stahl.ppt.pdf , http://youtu.be/0Dg02YQCQIE .

Stahl, G. (2011c). How I view learning and thinking in CSCL groups. Research and Practice in Technology Enhanced Learning (RPTEL). 6(3), 137-159. Web: http://GerryStahl.net/pub/rptel2011.pdf , http://youtu.be/h5MpUJnTipM .

Stahl, G. (2011d). How to study group cognition. In S. Puntambekar, G. Erkens & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methodologies, approaches and issues. (ch. 5, pp. 107-130). New York, NY: Springer. Web: http://GerryStahl.net/pub/analyzinginteractions.pdf .

Stahl, G. (2011e). The structure of collaborative problem solving in a virtual math team. In the proceedings of the iConference 2011.  Seattle, WA.  Web: http://GerryStahl.net/pub/iconf2011.pdf .

Stahl, G. (2011f). Theories of cognition in CSCW. In the proceedings of the European Computer-Supported Cooperative Work.  Aarhus, Denmark.  Web: http://GerryStahl.net/pub/ecscw2011.pdf .

Stahl, G. (2012a). Theorien des CSCL [theories in CSCL -- in German] (J. Haake & M. Muehlpfordt, Trans.). In J. Haake, G. Schwabe & M. Wessner (Eds.), CSCL-kompendium 2.0. (pp. 16-30). Frankfurt, Germany: Oldenburg. Web: http://GerryStahl.net/pub/theorien.pdf, http://GerryStahl.net/pub/theories.pdf .

Stahl, G. (2012b). A view of computer-supported collaborative learning research and its lessons for future-generation collaboration systems. Web: http://GerryStahl.net/pub/fgcs.pdf .

Stahl, G. (2013a). Interaction analysis of a biology chat. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs & N. Law (Eds.), Productive multivocality in the analysis of group interactions. (Vol. #14 Springer CSCL Book Series, ch. 28, pp. 511-540). New York, NY: Springer. Web: http://GerryStahl.net/pub/multivocal.pdf .

Stahl, G. (2013b). Theories of cognition in collaborative learning. In C. Hmelo-Silver, A. O'Donnell, C. Chan & C. Chinn (Eds.), International handbook of collaborative learning. (ch. 4, pp. 74-90). New York, NY: Taylor & Francis. Web: http://GerryStahl.net/pub/clhandbook.pdf.

Stahl, G. (2013c). Theories of collaborative cognition: Foundations for CSCL and CSCW together. In S. Goggins & I. Jahnke (Eds.), CSCL@work. (Vol. #13 Springer CSCL Book Series, ch. 3, pp. 43-64). New York, NY: Springer. Web: http://GerryStahl.net/pub/collabcognition.pdf .

Stahl, G. (2013d). Translating Euclid: Designing a human-centered mathematics. San Rafael, CA: Morgan & Claypool Publishers. Synthesis lectures on human-centered informatics series #17. 221 pages. Web: http://GerryStahl.net/elibrary/euclid . Doi: http://dx.doi.org/10.2200/S00492ED1V01Y201303HCI017 .

Stahl, G. (2015). Constructing dynamic triangles together: The development of mathematical group cognition. Cambridge, UK: Cambridge University Press. Learning in doing: Social, cognitive and computational perspectives series. 250 pages. Web: http://GerryStahl.net/elibrary/analysis . Doi: http://www.cambridge.org/la/academic/subjects/psychology/educational-psychology/constructing-dynamic-triangles-together-development-mathematical-group-cognition?format=HB

Stahl, G., & Strijbos, J. W. (2004). A multi-dimensional coding scheme for mathematical collaboration. Web: http://GerryStahl.net/pub/multi_dim_codes.pdf .

Strijbos, J. W., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small group chat communication. Learning & Instruction. Special issue on measurement challenges in collaborative learning research. 17(4), 394-404. Web: http://GerryStahl.net/vmtwiki/jw.pdf.

Zemel, A., & Stahl, G. (2010). Coordinating collaborative action in online math problem-solving. In the proceedings of the National Communication Association’s 96th Annual Convention (NCA 2010).  San Francisco, CA. Web: http://GerryStahl.net/pub/nca2010.pdf

Part A: Toward a Science of Group Cognition

1. Group Cognition as a Foundation for the New Science of Learning

In today’s global world, knowledge is primarily produced socially and progressively, not just by spontaneous acts of isolated minds. Individuals participate in this as knowledge learners, knowledge users and knowledge builders, predominantly through their interactions in small groups. Analysis of the cognitive work of small groups can be distinct from that of individual thinking and community knowledge processing.

We need a new science of group cognition to complement our sciences of individual learning and of community knowledge building. In particular, we need a science that will help us to realize the potential of computer networking to foster the formation of productive virtual groups and effective computer-supported collaborative learning.

The construct of group cognition provides a theoretical and practical foundation for developing the needed science, for analyzing the work of small groups and for designing effective collaboration software. The Virtual Math Teams Project provides a model of such scientific research. Along with related explorations of group processes, it has begun to produce concrete analyses of group cognition and to develop a multi-faceted online educational environment.

A science of group cognition can systematically provide findings that are objective, reliable and generalizable through its interpretive case studies. With its focus on the group as the unit of description, group cognition joins other post-cognitive theories, which extend the analysis of cognition beyond the psychological individual. As the pivotal middle ground in which knowledge is primordially co-constructed, the group provides a foundation for sciences of learning at the individual, small group and community levels.

“I

t takes a village to raise a child.” This ancient African proverb reflects the direct bearing of social relations on learning. In pre-industrial society, the individual, family-of-origin, extended family, clan, tribe, village and culture blended into one another almost seamlessly. With the rise of capitalism, the individual was uprooted from its social ground and celebrated as a free spirit—in order to compete unencumbered on the labor market (Marx 1867/1976). With globalization, the forces of production require information-processing tasks that exceed the capabilities of individual minds, necessitating the formation of well-coordinated knowledge-building teams. Thus, Hillary Clinton’s use of the proverb as the title of her book (Clinton 1996) not only looks back nostalgically to a romanticized past of homogeneous villages and neighborly towns, but also reflects the realities of our increasingly interconnected global village.

The nature of learning is transformed—along with other aspects of human social existence—by societal upheavals. However, our thinking about learning lags behind these changes. Furthermore, the evolution of social institutions is uneven, and past forms linger on in confusing mixtures. So our theories of learning, founded upon popular conceptions or “folk theories” (Bereiter 2002), confuse individual, group and community characteristics, while still exalting the individual learner.

It is time for a new science of learning because, as Bob Dylan already announced to the youth social movement of the 1960s, “the times they are a-changin’.” Foremost in our reconceptualization of learning must be a recognition not only of the role of the (post-modern) village, but also of the often ephemeral small groups that mediate between the tangible individual learner and the insubstantial communities within which the learner comes to participate. Imagine the gatherings of friends who listened to Dylan’s lyrics together, forming cadre of the new age awakening around the world a half century ago. The interactions in these peer groups contributed to the new identities of the individuals involved as well as of their generation. Creative ways of thinking, making meaning and viewing the world emerged. The scientific disciplines with their traditional methods are not equipped to analyze the interpenetration of such learning processes at the individual, small-group and community levels.

The Need for a New Science of Group Cognition

The idea of a science of group cognition was originally motivated by issues of software design for collaborative learning. The design of software to support group work, knowledge building and problem solving should be built on the foundation of an understanding of the nature of group interaction and group meaning making. However, previous research in computer-supported collaborative learning (CSCL) is mostly based on an ad hoc collection of incommensurable theories, which are not grounded in an explicit investigation of group interaction. What is needed is a science of group interaction focused on the group level of description to complement psychological theories of individuals and social theories of communities.

CSCL is fundamentally different from other domains of study in the learning sciences (Stahl 2002b). It takes as its subject matter collaborative learning, that is, what takes place when small groups of workers or students engage together in cognitive activities like problem solving or knowledge building (Koschmann 1996b; Stahl 2006a, Ch. 11). On a theoretical level, CSCL is strongly oriented toward the approach of Vygotsky (1930/1978), who stressed that learning and other higher psychological processes originally take place socially, intersubjectively. Piaget (1985), too, pointed to inter-subject processes like conflicting perspectives as a fundamental driver for creativity and cognitive development. Despite this powerful insight, even Vygotsky, Piaget and their followers generally maintain a psychological focus on the individual mind in their empirical studies and do not systematically investigate the intersubjective phenomena of small-group interaction.

A science of group interaction would aim to unpack what happens at the small-group unit of analysis (Stahl 2004b). Thus, it would be particularly relevant for CSCL, but may not be as directly applicable to other forms of learning, where the individual or the community level predominates. As a science of the group, it would complement existing theories of acting, learning and cognition, to the extent that they focus either on the individual or the community or that they reduce group phenomena to these other levels of description.

In the chapters of Studying Virtual Math Teams (Stahl 2009c) and of Group Cognition (Stahl 2006a), my colleagues and I have reviewed some of the research literature on small-group learning, on small-group processes and on collaborative mathematics. We have noticed that small-group studies generally look for quantitative correlations among variables—such as the effect of group size on measures of participation—rather than trying to observe group knowledge-building processes. Studies of small-group processes from psychology, sociology and other social sciences also tend to focus on non-cognitive aspects of group process or else attribute all cognition to the individual minds rather than to group processes. This was true of writings on cooperative learning in the 1970s and 1980s as well, e.g., Johnson & Johnson (1989).

There are some notable exceptions; in particular, we viewed (Barron 2000; 2003; Cohen et al. 2002; Sawyer 2003; Schwartz 1995) as important preliminary studies of group cognition within the learning sciences. However, even theories in cognate fields that seem quite relevant to our concerns—like distributed cognition (Hutchins 1996), actor-network theory (Latour 2007), situated cognition (Lave & Wenger 1991), ethnomethodology (Garfinkel 1967) and activity theory (Engeström 1987)—adopt a different focus. They generally focus on interaction of individuals with artifacts rather than among people, indicating an orientation to the larger community scale of social science.

Recent commentaries on situated cognition (Robbins & Aydede 2009) and distributed cognition (Adams & Aizawa 2008) frame the issues at the individual level, even reducing all cognitive phenomena to neural phenomena. At the other extreme, social theories focus on community phenomena like division of labor, apprenticeship training, linguistic structure, laboratory organization. For all its insight into small-group interaction and its analysis, even ethnomethodology maintains a sociological perspective, concerned with linguistic communities. Similarly, even when activity theory addresses the study of teams—in the most detail in Chapter 6 of (Engeström 2008)—it is mostly concerned with the group’s situation in the larger industrial and historic context; rather than analyzing how groups interactionally build knowledge, it paraphrases how they deal politically with organizational management issues. These theories provide valuable insights into group interaction, but none of them thematizes the small-group level as a domain of scientific study. As sciences, these are sciences of the individual or of the society, not of the collaborative group.

Each of the three levels of description is populated with a different set of phenomena and processes. For instance, individuals in a chat or threaded discussion interpret recent postings and design new postings in response, the group constructs, maintains and repairs a joint problem space and the community evolves its practices and institutions of social organization. The description of the individual level is the province of psychology; that of the community is the realm of sociology or anthropology; the small-group level still has no corresponding science.

A science of group interaction would take its irreducible position between the psychological sciences of the individual and the social sciences of the community—much as biology analyzes phenomena that are influenced by both chemicals and organisms without being reducible to either. The science of group interaction would fill a lacuna in the multi-disciplinary work of the human sciences—including the learning sciences. This science would not be primarily oriented toward the “low level” processes of groups, such as mechanical or rote behaviors, but would be concerned with the accomplishment of creative intellectual tasks. Intellectual teamwork, knowledge work and knowledge-building activities would be prototypical objects of study. The focus would be on group cognition.

The bifurcation of the human sciences into individual and societal creates an irreconcilable opposition between individual creative freedom and restrictive social institutions. A science of group cognition would flesh out the concept of structuration, demonstrating with detailed analyses of empirical data how group interactions can mediate between individual behavior and social practice.

The Construct of Group Cognition

The term group cognition does not signify an object or phenomenon to analyze like brain functions or social institutions (Stahl 2009c, Ch. 11). It is a proposal for a new science or focus within the human sciences. It hypothesizes:

When small groups engage in cooperative problem solving or collaborative knowledge building, there are distinctive processes of interest at the individual, small-group and community levels of analysis, which interact strongly with each other. The science of group cognition is the study of the processes at the small-group level.

The science of group cognition is a human science, not a predictive science like chemistry nor a predominantly quantitative one like physics. It deals with human meanings in unique situations, necessarily relying upon interpretive case studies and descriptions of inter-personal processes.

Processes at the small-group level are not necessarily reducible to processes of individual minds, nor do they imply the existence of some sort of group mind. Rather, they may take place through the weaving of semantic and indexical references within a group discourse (Stahl 2004a). The indexical field (Hanks 1992) or joint problem space (Teasley & Roschelle 1993) co-constructed through the sequential interaction of a group  has the requisite complexity to constitute an irreducible cognitive act in its own right. The rise of cognitive science in the 1980s broadened the definition of “cognition” beyond an activity of human minds in order to include the artificial intelligence of computers. What counts as cognitive is now a matter of computational complexity. Anything that can compute well enough to play chess or prove theorems can be a cognitive agent—whether it is a person, computer or collaborative small group (Stahl 2005).

Largely because of its linguistic form, the noun phrase “group cognition” is often misunderstood as refering to some kind of physical or mental object. However, it is a theoretical construct, not an object, as indicated by the hypothesis stated above. Commonsensical folk theories assume that we generally talk about physical objects. However, if one looks closely, most sciences deal with hypothesized entities, not physical objects; mental representations are a prime example at the individual level and cultural norms or social rules at the community level. Mental entities cannot be physically located—even with the latest MRI machines—and neither can rules.

The group that engages in group cognition is not necessarily a set of physical people who interact together in the present moment. For example, group processes of problem solving, meaning making and knowledge building can be revealed in computer logs of chat or threaded discussion, where the people who contributed are now long gone. The interaction is captured and remains accessible through the log. The “interaction” of textual artifacts is not like physical interactions (the cause-and-effect interactions of instantaneously colliding billiard balls in physics or the face-to-face embodied interactions of people in the present moment), but can bring together references from the distant past or into the future (the mediator interactions of a network of actors). As Latour notes, “a banal conversation may become a terribly complex chain of mediators where passions, opinions and attitudes bifurcate at every turn” (2007, p. 39). The textual interaction itself constitutes the discourse as a group interaction, by, for instance, addressing proposals to the group as a whole (Lerner 1993) and referencing among postings in complex ways. Of course, the subtle textual interactions were originally designed by the people situated in the discourse and typing the postings, but the persistent interaction takes place through the textual medium by means of the meditational agency of the textual elements (Zemel & Çakir 2007).

Rather than taking the number of human bodies as a definition of “small group,” one could consider the complexity of the interaction among postings from different people. Productive knowledge building, creative meaning making and innovative group cognition take place through dense networks of referencing, in which the postings of each participant takes into account the past postings and the potential future responses of all the other participants. This contrasts not only with a case in which someone’s postings only refers to their own other postings (individual cognition), but also with cases where there are clusters of mutual referencing connected by sparse references (community cognition). The group cognition case involves active generation of shared understandings, whereas the community case requires the mediation over time of persistent representations, artifacts and dissemination processes in order to cross the sparse connections.

Not only is it wrong to associate the construct of the “group” in a simple way with a set of people, it is wrong to think of it in terms of a type of physical object or assemblage distinguished from other types. The distinction between individual, group and community might better be conceptualized by developing Vygotsky’s (1930/1978) sketchy and problematic concepts of “internalization” and “externalization.” Vygotsky argued that the higher psychological processes of humans (i.e., cognition) developed through interaction between people. These processes could subsequently (through complicated, extended processes of mediation that are not well understood) be “internalized” as un-vocalized self-talk and mental skills, and “externalized” as artifacts and other elements of a community’s culture. Taking this approach, the theory of group cognition is interested in exploring the pivotal meaning-making processes of group interaction, which can subsequently lead to both individual thought and community norms. Such a view differentiates individual, group and community levels as functionally distinct. It lends a foundational role to group cognition within a new science of learning.

The Group Unit of Description

The theory of group cognition stakes out a new domain for exploration: the domain of group meaning-making processes. Importantly, it distinguishes this domain from the traditional domains of sciences of individual learning and of the development of social practices in communities. Virtually all discussions in the learning sciences have been ambiguous in their terminology when it comes to distinguishing the individual, group and cultural levels of description. My own writings have used the relevant terminology in a loose way. Therefore, it may be helpful to try to codify a set of terms for speaking at the three different levels (see Table 1).

Table 1. Terminology distinguishing the three levels of description.

Individual

Group

Community

Role

Person / student

Group participant

Community member

Adjective

Personal

Collaborative

Social

Object of analysis

Mind

Discourse

Culture

Unit of analysis

Mental representation

Utterance response pair

Mediating artifact

Form of knowledge

Subjective

Intersubjective

Cultural

Form of meaning

Interpretation

Shared understanding, joint meaning making, common ground

Domain vocabulary, artifacts, institutions, norms, rules

Learning activity

Learn

Build knowledge

Science

Way to accomplish cognitive tasks

Skill, behavior, habit

Group method / group practice

Member method / social practice

Communication

Thought

Interaction

Membership

Mode of construction

Constructed

Co-constructed

Socially constructed

Context of cognitive task

Personal problem

Joint problem space

Problem domain

Context of activity

Embodiment

Situation

World

Referential system

Associations

Indexical field

Cultural world

Form of existence

Being there

Being with

Folk

Temporal structure

Subjective experiential internal time

Co-constructed shared temporality

Measurable objective time

Theory of cognition

Constructivist

Post-cognitive

Socio-cultural

Science

Cognitive and educational psychology

Group cognition

Sociology, anthropology, linguistics

Of course, some of this classification of terms is arbitrary and inconsistent with prior usage. In particular, the terms related to groups and cultures have not been kept distinct in the past. Even Vygotsky, who pioneered in distinguishing the social from the individual, would sometimes use terms like “social” and “intersubjective” to apply to anything from a dyad to all of society. Within the learning sciences, “knowledge building” has been used at every level, resulting in controversy about whether classrooms are communities-of-practice, for instance. The characteristics of scientific research communities were projected onto classrooms, project groups and individuals without carefully distinguishing their different ways of building knowledge. Table 1 therefore suggests, for instance, using “intersubjective” at the group level and “social” at the community level. It suggests using “joint,” “shared,” “collaborative” and “co-constructed for group phenomena.

Such ambiguity of terminological usage even led to pseudo-problems, which can now be resolved by the theory of group cognition, showing how small groups mediate between the individual and the community phenomena. To take one example, the seeming irreconcilability of subjective and objective time can be bridged by considering how small groups co-construct their shared temporal reference system. Significantly, the co-construction can be observed in logs of interaction and analyzed in detail—which cannot be done for either the subjective sense of internal time (Husserl 1917/1991) or the abstract dimension of scientifically measured time (Heidegger 1927/1996).

The move from the individual to the group level of description as foundational entails an important philosophical step: from cognitivism to post-cognitivism. This step has its basis in philosophy (Hegel 1807/1967; Heidegger 1927/1996; Marx 1867/1976; Merleau-Ponty 1945/2002; Wittgenstein 1953), in social science (Bourdieu 1972/1995; Geertz 1973; Giddens 1984b) and in analytic methods of ethnomethodology and conversation analysis (Garfinkel 1967; Livingston 1987; Sacks 1965/1995; Schegloff 2007). Post-cognitive theories influential in CSCL and the learning sciences include: the critique of cognitivism (Dreyfus 1972; Polanyi 1962; Schön 1983; Winograd & Flores 1986), situated action (Suchman 1987), situated learning (Lave & Wenger 1991), activity theory (Engeström 1987), distributed cognition (Hutchins 1996), actor-network theory (Latour 2007) and knowledge building (Scardamalia & Bereiter 1996).

In seminal statements of post-cognitivist theory, Hutchins has explicitly pointed to group-cognitive phenomena: “Cognitive processes may be distributed across the members of a social group” (Hollan, Hutchins & Kirsh 2000, p. 176). “The cognitive properties of groups are produced by interaction between structures internal to individuals and structures external to individuals” (Hutchins 1996, p. 262). “The group performing the cognitive task may have cognitive properties that differ from the cognitive properties of any individual” (Hutchins 1996, p. 176). However, rather than focusing on these group phenomena in detail, he analyzes socio-technical systems and the cognitive role of highly developed artifacts (airplane cockpits, ship navigation tools). Certainly, these artifacts have encapsulated past cultural knowledge (community cognition), and Hutchins’ discussions of this are important. But in focusing on what is really the community level—characteristically for a cultural anthropologist—he does not analyze the cognitive meaning making of the group itself (the active navigational team).

In general, the related literature on small groups and on post-cognitivist phenomena provide some nice studies of the pivotal role of small groups, but do not account for this level of description theoretically. In the final analysis, they are usually based on either a psychological view of individuals or a sociological view of rules, etc. at the community level. None of them have a foundational conception of small groups as a distinct level. They confuse talk at the group level and at the social level, and they lack a developed account of the relationships between individual, group and community.

If we take group phenomena seriously as “first-class objects” of our theory, then we can study: interpersonal trains of thought, shared understandings of diagrams, joint problem conceptualizations, common references, coordination of problem-solving efforts, planning, deducing, designing, describing, problem solving, explaining, defining, generalizing, representing, remembering and reflecting as a group. In our empirical case studies, we will see the group-cognitive accomplishments emerge from the network of meaningful references built up by, for instance, textual postings in online chat. We will see how the group and its cognitive accomplishments are enacted in situated interaction.

A Model of the New Science

Having motivated the development of a science of group cognition as future work, let us see how the Virtual Math Teams (VMT) Project (Stahl 2009c) may have begun to prepare the way. Preparing for a new science requires three major undertakings:

(a) The domain of the science must not only be defined, it must be explored and captured in the form of a data corpus.

(b) Methods for analyzing the data must be selected, adapted, refined and mastered.

(c) Analytic findings must be organized in terms of a framework of theoretical conceptualizations.

The Virtual Math Teams Project at Drexel University has approached these tasks by:

(a) Creating a synchronous online service in which small groups of students engaged in problem-solving work in mathematics (and generated data),

(b) Conducting chat interaction analysis of a number of case studies from the data recorded in that service (using modified methods) and

(c) Conceptualizing some of the features of the small-group interactions that were observed (and publishing theoretical accounts).

The first step in the VMT design-based research process was to start simply and see what issues came up. We had seen in face-to-face case studies that there were problems with (i) recording and transcribing the verbal interaction, (ii) capturing the visual interaction and (iii) knowing about all the influences on the interaction. We decided to form groups of students who did not know each other and who only interacted through text chat. Students were recruited through the Math Forum at Drexel University, an established online resource center. We used AIM, AOL’s Instant Messaging system, which was freely available and was already familiar to many students. We included a researcher in the chat room with each small group of students. The facilitator told the students their math task, dealt with any technical difficulties, posted drawings from the students on a web page where they could be seen by all the students, notified the group when the session was over and saved an automatically generated log of the chat. In this way, we obtained a complete and objective log of the interaction, captured everything that the students shared on their computers and excluded any unknown influences from affecting the interaction.

The issue of including everything affecting the interaction is a subtle issue. Of course, the interaction is influenced by the life histories, personalities, previous knowledge and physical environment of each student. A student may have windows other than AIM open on the computer, including Internet browsers with math resources. A student may be working out math problems on a piece of paper next to the computer. Also, a student may leave the computer for some time to eat, listen to music, talk on the phone, and so on without telling anyone in the chat. In such ways, we do not have information about everything involved in a particular student’s online experience. We do not even know the student’s gender or age. We do not know if the student is shy or attractive, speaks with an accent or stutters. We do not know if the student usually gets good grades or likes math. We do not know what the student is thinking or feeling. We only know that the students are in an approximate age group and academic level—because we recruited them through teachers. However, the VMT Project is only concerned with analyzing the interaction at the group unit of analysis. Notice that the things that are unknown to us as researchers are also unknown to the student group as a whole. The students do not know specifics about each other’s background or activities—except to the extent that these specifics are brought into the chat. If they are mentioned or referenced in the chat, then we can be aware of them to the same extent as are the other students.

The desire to generate a complete record for analysis of everything that was involved in a team’s interaction often conflicted with the exploration of technology and service design options. For instance, we avoided speech-based interaction (VOIP, Skype, WIMBA) and support for individual work (e.g., whiteboards for individual students to sketch ideas privately), because these would complicate our review of the interactions. We tried to form teams that did not include people who knew each other or who could interact outside of the VMT environment. These decisions had pros and cons: The use of text-based communication provided persistent access to what had been said—for the students as well as their teachers and the researchers. The scarcity of private workspaces encouraged joint attention and collaboration.

In addition to personal influences, the chat is responsive to linguistic and cultural matters. Of course, both students and researchers must know English to understand the chats. In particular, forms of English that have evolved with text chat and cell-phone texting have introduced abbreviations, symbols and emoticons into the online language. The linguistic subculture of teenagers also shows up in the VMT chats. An interdisciplinary team of researchers comes in handy for interpreting the chats. In our case, the research team brought in experience with online youth lingo based on their backgrounds as Math Forum staff, teachers or parents.

The early AIM chats used simple math problems, taken from standardized math tests and Math Forum Problems-of-the-Week. One experiment to compare individual and group work used problems from a standardized multiple-choice college-admissions test. These problems had unique correct answers. While these provided a good starting point for our research, they were not well suited for collaborative knowledge building. Discourse around them was often confined to seeing who thought they knew the answer and then checking for correctness. For the VMT Spring Fests in 2005, 2006 and 2007, we moved to more involved math topics that could inspire several hours of joint inquiry.

Even with straightforward geometry problems, it became clear that students needed the ability to create, share and modify drawings within the VMT environment. We determined that we needed an object-oriented draw program, where geometric objects could be manipulated (unlike a pixel-based paint program). We contracted with the developers of ConcertChat to use and extend their text chat and shared whiteboard system, which is now available in Open Source. This system included a graphical referencing tool as well as social awareness and history features (Mühlpfordt & Stahl 2007). In order to help students find desirable chat rooms and to preserve team findings for all to see, we developed the VMT Lobby and integrated a Wiki with the Lobby and chat rooms (Stahl 2008b). Gradually, the technology and the math topics became much more complicated in response to the needs that were revealed when we analyzed the trials of the earlier versions of the VMT service. As the system matured, other research groups began to use it for their own trials, with their own math topics, procedures, analytic methods or even new technical features. These groups included researchers from Singapore, Rutgers, Hawai’i, Romania and Carnegie-Mellon.

The Nature of the New Science

The approach to chat interaction analysis that emerged in the VMT Project will now be discussed in terms of a number of issues (which correspond to general issues of most research methodologies, as indicated in parentheses).

Group cognition in a virtual math team (research question)

Learning—whether in a classroom, a workplace or a research lab—is not a simplistic memorization or storage of facts or propositions, as traditional folk theories had it. The term learning is a gloss for a broad range of phenomena, including: the development of tacit skills, the ability to see things differently, access to resources for problem solving, the discursive facility to articulate in a new vocabulary, the power to explain, being able to produce arguments or the making of new connections among prior understandings. We can distinguish these phenomena as taking place within individual minds, small-group interactions or communities of practice. The analysis of learning phenomena at these various levels of analysis requires different research methodologies, appropriate to corresponding research questions. The VMT Project was intended to explore the phenomena of group cognition and accordingly pursued the research question:

How does learning take place in small groups, specifically in small groups of students discussing math in a text-based online environment? What are the distinctive mechanisms or processes that take place at the small-group level of description when the group is engaged in problem-solving or knowledge-building tasks?

While learning phenomena at the other levels of analysis are important and interact strongly with the group level, we have tried to isolate and make visible the small-group phenomena and to generate a corpus of data for which the analysis of the group-level interactions can be distinguished from the effects of the individual and community levels.

The methods used to gather and analyze one’s data should be appropriate to one’s research question. To support such research, one must generate and collect data that are adequate for the selected kinds of analysis. Because we were interested in the group processes that take place in virtual math teams, we had to form teams that could meet together online. In the Spring Fests, students had to be able to come back together in the same teams on several subsequent occasions. The VMT environment had to be instrumented to record all messages and activities that were visible to the whole team in a way that could be played back by the analysts. The math problems and the feedback to the teams had to be designed to encourage the kinds of math discussions that would demonstrate processes of group cognition, such as formulating questions and proposals, coordinating drawings and textual narratives, checking proposed symbolic solutions, reviewing the team’s work and so on. A sense of these desirable group activities and the skill of designing problems to encourage them had to develop gradually through the design-based research iterations.

Non-laboratory experimental design (validity)

Of course, to isolate the small-group phenomena we do not literally isolate our subject groups from individuals and communities. The groups consist of students, who are individuals and who make individual contributions to the group discourse based on their individual readings of the discourse. In addition, the groups exist and operate within community and social contexts, drawing upon the language and practices of their math courses and of their teen and online subcultures. These are essential features of a real-world context and we would not wish to exclude them even to the extent possible by confining the interaction to a controlled laboratory setting. We want the students to feel that they are in a natural setting, interacting with peers. We do not try to restrict their use of language in any way (e.g., by providing standardized prompts for chat postings or scripting their interactions with each other).

We are designing a service that can be used by students and others under a broad array of scenarios: integrated with school class work, as extra-curricular activities, as social experiences for home-schooled students, as cross-national team adventures or simply as opportunities (in a largely math-phobic world) to discuss mathematics. To get a sense of how such activities might work, we have to explore interactions in naturalistic settings, where the students feel like they are engaged in such activities rather than being laboratory subjects.

Data collection at the group level of description (unit of analysis)

Take the network of references in a chat-threading diagram (see Figure 1) as an image of meaning making at the group level (Stahl 2007). One could almost say that the figure consists entirely of contributions from individuals (the chat postings and whiteboard drawings) and resources from the math community; that everything exists on either the individual or community level, not on the group level. Yet, what is important in the figure is the network of densely interwoven references, more than the objects that are connected by them. This network exists at the group level. It mediates the individual and the community by forming the joint problem space (Sarmiento 2007; Teasley & Roschelle 1993), indexical ground (Hanks 1992), referential network (Heidegger 1927/1996) or situation (Suchman 2007) within which meanings, significant objects and temporal relations are intersubjectively co-constructed. On the individual level, these shared group meanings are interpreted and influence the articulation of subsequent postings and actions (Dourish 2001). On the community level, the meanings may contribute to a continually evolving culture through structuration processes (Giddens 1984a). The VMT Project is oriented toward the processes at the group unit of analysis, which build upon, connect and mediate the individual and community phenomena.

Figure 1. The network of references in a chat log excerpt.

Elements from the individual and community levels only affect the group level if they are referenced in the team’s interaction. Therefore, we do not need to gather data about the students or their communities other than what appears in the interaction record. We do not engage in surveys or interviews of the students or their teachers. For one thing, the design of the VMT Project prohibits access to these sources of data, because the students are only available to the project team through the chat sessions. External sources of data would be of great interest for other research questions having to do with individual learning or cultural changes, but for our research question, they are unnecessary and might even form a distraction or skew our analysis because it would cause our readings of the postings to be influenced by information that the group had not had.

By moving to the disembodied online realm of group cognition in virtual math teams, it is easier for us to abandon the positivist metaphors of the mechanistic worldview. Not only is it clear that the virtual group does not exist in the form of a physical object with a persistent memory akin to a computer storage unit, but even the individual participants lack physical presence. All that exists when we observe the replayed chats are the traces of a discourse that may have taken place years ago. Metaphors that might come naturally to an observer of live teamwork in a workplace or classroom—personalities, the group, learning, etc.—no longer seem fundamental. What exist immediately are the textual, graphical and symbolic inscriptions. These are significant fragments, whose meaning derives from the multi-layered references to each other and to the events, artifacts and agents of concern in the group discourse. This meaning is as fresh now as when the discourse originated, and can still be read off the traces by an analyst, much as by the original participants. This shows that the meanings shared by the groups are not dependent upon mental states of the individual students—although the students may have had interpretations of those meanings in mind, external to the shared experience. The form of our data reinforces our focus on the level of the shared-group-meaning making as an interactional phenomenon rather than a psychological one.

Instrumentation and data formats (objectivity)

It was noted above that when one videotapes small-group interactions a number of practical problems arise. Data on face-to-face classroom collaboration runs into issues of (i) recording and transcribing the verbal interaction, (ii) capturing the visual interaction and (iii) knowing about all the influences on the interaction. The data is in effect already partially interpreted by selective placement of the microphone and camera. It is further interpreted by transcription of the talk, and is restricted by limited access to facial expressions and bodily gestures. Much happens in a classroom influencing the student teams that is not recorded.

The online setting of the VMT sessions eliminates many of these problems. As already described, the automatic computer log of the session captures everything that influences the group as a whole. This includes all the postings and whiteboard activity, along with their precise timing. They are captured at the same granularity as they are presented to the students. Chat postings appear as complete messages, defined by the author pressing the Enter button. Whiteboard textboxes appear as complete, when the author clicks outside of the textbox. Whiteboard graphics appear gradually, as each graphical element is positioned by the author. Computer-generated social-awareness messages (when people enter or exit the chat room, begin or end typing, move a graphical object, etc.) are also accurately recreated. The precision of the log recording is assured because it consists of the original actions (as implemented by the computer software) with their timestamps. The original display to the students is generated from the server using the same log data that is used by the VMT Replayer. There is no selectivity or interpretation imposed by the analysts in the preparation of the full session record.

For our analysis of chats, we use a VMT Replayer. The Replayer is simply an extended version of the Java applet that serves as the chat/whiteboard room in the VMT environment. The reproduced chat room is separated by a thin line at the bottom from a VCR-like interface for replaying the session (see Figure 2). The session can be replayed in real time or at any integral multiple of this speed. It can be started and stopped at any point. An analyst can drag the pointer along the timeline to scroll both the whiteboard history and the chat history in coordination. One can also step through the recorded actions, including all the awareness messages. In addition, spreadsheet logs can be automatically generated in various useful formats.

Figure 2. The VMT Replayer

The data analyzed in the VMT Project is recorded with complete objectivity. There is no selectivity involved in the data generation, recording or collecting process. Furthermore, the complete recording can be made available to other researchers as a basis for their reviews of our analyses or the conducting of their own analyses. For instance, there have been multiple published analyses of the VMT data by other research groups following somewhat different research questions, theories and methods (Koschmann & Stahl 2009; Stahl 2009c). While collaborative sessions are each unique and in principle impossible to reproduce, it is quite possible to reproduce the unfolding of a given session from the persistent, comprehensive and re-playable record.

Collaborative data sessions (reliability)

Interpretation of data in the VMT Project first begins with an attempt to describe what is happening in a chat session. We usually start this process with a data session (Jordan & Henderson 1995) involving six to twelve researchers. A typical data session is initiated by a researcher who is interested in having a particular segment of a session log discussed by the group. Generally, the segment seems to be both confusing and interesting in terms of a particular research question.

For our data sessions, we sit around a circle of tables and project an image of the VMT Replayer onto a screen visible to everyone. Most of us have laptop computers displaying the same Replayer, so that we can scan back and forth in the segment privately to explore details of the interaction that we may want to bring to the attention of the group. The group might start by playing the segment once or twice in real time to get a feel for how it unfolds. Then we typically go back to the beginning and discuss each line of the chat sequentially in some detail.

The interpretation of a given chat line becomes a deeply collaborative process. Generally, one person will make a first stab at proposing a hypothesis about the interactional work that line is doing in the logged discourse. Others will respond with suggested refinements or alternatives to the proposal. The group may then engage in exploration of the timing of chat posts, references back to previous postings or events, etc. Eventually the data analysis will move on to consider how the student group took up the posting. An interesting interpretation may require the analysts to return to earlier ground and revise their tentative previous understandings (Stahl 2009c, Ch. 10).

The boundaries of a segment must be considered as an important part of the analysis. When does the interaction of interest really get started and when is it resolved? Often, increasingly deep analysis drives the starting point back as we realize that earlier occurrences were relevant.

It is usually first necessary to clarify the referential structure of the chat postings and how they relate to events in the whiteboard or to the comings and goings of participants. The threading of the chat postings provides the primary structure of the online, text-based discourse in much the same way that turn taking provides the core structure of spoken informal conversation. Because of the overlap in the typing of chat postings, it is sometimes tricky to figure out who is responding to what. Looking at the timestamps of posts and even at the timestamps of awareness messages about who is typing can provide evidence about what was visible when a posting was being typed. This can often suggest that a given post could or could not have been responding to a specific other post, although this is sometimes impossible to determine. When it is hard for the analyst to know the threading, it may have also been hard for most of the chat participants (other than the typist) to know; this may result in signs of trouble or misunderstandings in the subsequent chat.

The test of correctness of chat interaction analysis is not a matter of what was in individuals’ minds, but of how postings function in the interaction. Most of the multi-layered referencing takes place without conscious awareness by the participants, who (as speakers of the language) are experts at semantic, syntactic and pragmatic referencing and can design utterances in response to local resources without formulating explicit plans (Suchman 2007). Thus, inspection of participants’ memories—whether with interviews or fMRI scans—would not reveal causes. Of course, participants could retroactively tell stories about why they posted what they did, but these stories would be based upon their current (not their original) interpretations using their linguistic competence and upon their response to their current (not original) situation, including their sense of what the person interviewing them wants to hear. Thus, interpretations by the participants are not in principle privileged over those of the analyst and others with the relevant interpretive competence (Gadamer 1960/1988). The conscious memories that a participant may have of the interaction are, according to Vygotsky’s theory, just more interaction—but this time sub-vocal self-talk; if they were brought into the analysis, they would be in need of interpretation just as much as the original discourse.

Since our research question involves the group as the unit of analysis, we do not raise questions in the data session about what one student or another may have been doing, thinking or feeling as an individual. Rather, we ask what a given posting is doing interactionally within the group process, how it responds to and takes up other posts and what opportunities it opens for future posts. We look at how a post is situated in the sequential structure of the group discourse, in the evolving social order and in the team’s meaning making. What is this posting doing here and now in the referential network? Why is it “designed to be read” (Livingston 1995) in just this way? How else could it have been phrased and why would that not have achieved the same effect in the group discourse?

We also look at how a given posting positions (Harré & Moghaddam 2003) both the author and the readers in certain ways. We do not attribute constant personalities or fixed roles to the individuals, but rather look at how the group is organized through the details of the discourse. Perhaps directing a question toward another student will temporarily bestow upon her a form of situated expertise (Zhou, Zemel & Stahl 2008) such that she is expected to provide an extended sequence of expository postings (Mercer & Wegerif 1999).

The discussion during a data session can be quite un-orderly. Different people see different possible understandings of the log and propose alternative analyses. Generally, discussion of a particular posting continues until a consensus is tentatively established or someone agrees to look into the matter further and come back next week with an analysis. Notes are often taken on the data session’s findings, but the productive result of the discussion most often occurs when one researcher is inspired to write about it in a conference paper or dissertation section. (Almost 300 analyses from the VMT Project are available at: http://gerrystahl.net/vmt/pubs.html.) When ideas are taken up this way, the author will usually bring the more developed analysis back for a subsequent data session and circulate the paper.

In coding analysis, it is conventional to train two people to code some of the same log units and to compare their results to produce an inter-rater reliability measure (Strijbos & Stahl 2007). In our chat interaction analysis, we do not pretend that the log can be unproblematically partitioned into distinct units, which can be uniquely assigned to a small number of unambiguous codes. Rather, most interesting group discourse segments have a complex network of interwoven references. The analysis of such log segments requires a sophisticated human understanding of semantics, interpersonal dynamics, mathematics, argumentation and so on. Much is ultimately ambiguous and can be comprehended in multiple ways—sometimes the chat participants were intentionally ambiguous. At the same time, it is quite possible for analysts to make mistakes and to propose analyses that can be shown to be in error. To attain a reasonable level of reliability of our analyses, we make heavy use of data sessions. This ensures that a number of experienced researchers agree on the analyses that emerge from the data sessions. In addition, we try to provide logs—or even the entire session data with the Replayer—in our papers so that readers of our analyses can judge for themselves the interpretations that are necessarily part of chat analysis.

Describing group practices (generalizability)

The research question that drives the VMT Project is: What are the distinctive mechanisms or processes that take place at the small-group level of description when the group is engaged in problem-solving or knowledge-building tasks? Therefore, we are interested in describing the inter-personal practices of the groups that interact in the VMT environment. There are, of course, many models and theories in the learning sciences describing the psychological practices of individuals involved in learning. At the opposite extreme, Lave & Wenger’s (1991) theory of situated learning describes social practices of communities of practice, whereby a community renews itself by moving newcomers into increasingly central forms of legitimate peripheral participation. However, there are few descriptions specifically of how small groups engage in learning practices.

Vygotsky (1930/1978) argued that learning takes place inter-subjectively (in dyads or groups) before it takes place intra-subjectively (by individuals). For instance, in his analysis of the infant and mother (p. 56), he outlines the process through which an infant’s unsuccessful grasping at some object becomes established by the mother-child dyad as a pointing at the object. This shared practice of pointing subsequently becomes ritualized by the dyad (LeBaron & Streeck 2000) and then mediated and “internalized” by the infant as a pointing gesture. The pointing gesture—as a foundational form of deictic reference—is a skill of the young child, which he can use for selecting objects in his world and learning about them. The gesture is understood by his mother because it was intersubjectively established with her. In this prototypical example, Vygotsky describes learning as an inter-subjective or small-group practice of a dyad.

While we can imagine that Vygotsky’s description is based on a concrete interaction of a specific infant and mother in a particular time and place, the pointing gesture that he analyzed is ubiquitous in human culture. In this sense, the analysis of a unique interaction can provide a generalizable finding. The science of ethnomethodology (the study of the methods used in cultures) (Garfinkel 1967) is based on the fact that people in a given culture or linguistic community share a vast repertoire of social practices for accomplishing their mundane tasks. It is only because we share and understand this stock of practices that we can so quickly interpret each other’s verbal and gestural actions, even in novel variations under unfamiliar circumstances. The analysis of unique case studies can result in the description of group practices that are generalizable (Maxwell 2004). The methods developed in specific situated encounters are likely to be typical of a broad range of cases under similar conditions.

In our data sessions, we find the same kinds of moves occurring in case after case that we analyze. On the one hand, group methods are extremely sensitive to changes in the environment, such as differences in features and affordances of the communication media. On the other hand, groups of people tend to adapt widespread methods of interaction to changing circumstances in similar ways—to support general human and social needs. Group methods are not arbitrary, but draw on rich cultural stocks of shared behavior and adapt their outward appearances in order to maintain the underlying structure under different conditions.

By describing the structure of group methods in detailed case studies, we can characterize general methods of group behavior, group learning or group cognition. Findings from analyses of case studies can lead to the proposal of theoretical categories, conceptualizations, structures or principles—in short, to a science of group interaction.

The Foundational Role of Group Cognition

As discussed above, students in virtual math teams are active as individuals, as group participants and as community members. They each engage in their own, private individual activities, such as reading, interpreting, reflecting upon and typing chat messages. Their typed messages also function as group actions, contributing to the on-going problem solving of the team. Viewed as community events, the chats participate in the socialization process of the society, through which the students become increasingly skilled members of the community of mathematically literate citizens.

A core thesis of the theory of group cognition is, “Small groups are the engines of knowledge building. The knowing that groups build up in manifold forms is what becomes internalized by their members as individual learning and externalized in their communities as certifiable knowledge” (Stahl 2006a, p. 16). Despite their centrality, small groups have not been theorized or studied extensively.

Some small-group literature has been produced from either the methodological perspective of psychology or that of sociology, primarily since World War II. Traumatized by the mass-culture horrors of fascism and by extreme forms of mentalist pseudo-science, these predominantly behaviorist studies focused on the negative aspects of “group think” and caricatured the notion of “group mind”—which had a well-respected history before the rise of positivism (Wegner 1986). These studies miss the pivotal role of small groups in processes of learning.

More recent theories like distributed cognition, situated action or activity theory actually conduct case studies of small-group interaction, but they do not theorize the small group as their unit of analysis and therefore they do not produce descriptions of small-group methods as such. Even Hutchins (1996), in studying distributed cognition in the wild, does not thematize the interpersonal interactions, but focuses on the cognitive unit of analysis, simply broadening it to include the external computational and physical representational artifacts that an individual worker uses. Furthermore, the cognitive accomplishments he studies are fundamentally routine, well scripted procedures that do not involve creative solutions to ill-structured problems; the coordination of the navigational team is fixed by naval protocol, not co-constructed through the interaction, although it must still be enacted in concrete situations.

The VMT studies provide a model for describing the small-group methods as distinct from individual behaviors and community practices. They look at rich interactions in groups larger than dyads, where individual identities play a smaller role. They analyze group efforts in high-order cognition such as mathematical problem solving and reflection on the group problem-solving trajectory. They investigate groups that meet exclusively online, where the familiar visual, physical and aural modes of communication are unavailable, and where communication is mediated by designed technological environments.

Understanding how a collaborative group as a whole constructs knowledge through joint activity in a CSCL setting is what sets the science of group cognition apart from other approaches to the study of learning. Successful collaboration involves not only the incorporation of contributions of individuals into the group discourse, but also the effort to make sure that participating individuals understand what is taking place at the group level. The contributions of individuals to the group and of understandings from the group to the individuals cannot be studied by analyses at the individual unit of analysis, but only be studying the interactions at the group level. The group knowledge-construction process synthesizes innumerable resources from language, culture, the group’s own history, individual backgrounds, relevant contexts and the sequential unfolding of the group discourse in which the individuals participate. Although the group process is dependent upon contributions and understanding of individuals, their individual cognition is essentially situated in the group process. Group cognition is the science of cognitive processes at the group unit of analysis. These group processes—such as the sequential flow of proposals, questioning, building common ground, maintaining a joint problem space, establishing intersubjective meanings, positioning actors in evolving roles, building knowledge collaboratively and solving problems together—are not analyzable as individual behaviors.

There is a scientific lacuna within the learning sciences between sciences of the individual and sciences of communities. There are important cognitive achievements at the small-group level of description, which should be studied by a science of groups.

Online small groups are becoming increasingly possible and important in the global networked world, and a post-cognitive science of virtual groups could help the design of collaborative software for working and learning. It could provide an effective foundation for the new science of learning.

2. A View of Computer-Supported Collaborative Learning Research and its Lessons

Abstract. This is a review of research on educational technology from a particular historical and theoretical perspective. It focuses on the research field of computer-supported collaborative learning (CSCL) and does so based on the author’s personal experiences in that field. Starting with an overview of the changing role of technology in educational design, it then looks at the shifting function of individual learning as a component of group collaboration, with the central theme of intersubjectivity. The multiple dimensions of philosophical and analytic perspectives that emerged in the past have spawned a variety of methodological and thematic alternatives in current research. Lessons learned from this research point to a number of principles for productive, multi-disciplinary research in the future, as the field of CSCL spreads globally. While it is impossible to predict what collaboration technologies will emerge in the future, it seems clear that to support group cognition effectively, they will have to be designed to address complex social issues of intersubjectivity based on extensive international research efforts. Recommendations are drawn from this perspective for the next generation of designers of educational-collaboration systems.

Introduction

T

he research field of computer-supported collaborative learning (CSCL) is concerned with the design of environments that facilitate collaborative learning. These environments typically provide communication media, explorative virtual worlds, digital workspaces or other computer-based tools. The environments are often designed to be used by small groups of students who are not co-located. The environment design may include curricular materials, scripted tasks for student groups, orchestration roles for teachers and other components of an educational system. To date, most CSCL environments are research prototypes, rather than widely disseminated systems in standard classrooms, although some have been used by school districts around the world. Thus, CSCL is sometimes considered a specialized niche in educational technology or in distance education. However, it can be argued that collaborative learning is a fundamental form of human learning and that CSCL is exploring exciting opportunities for the future of education. For a general introduction to CSCL, see (Stahl 2010a).

CSCL has grown rapidly to incorporate a broad spectrum of approaches. Since the first CSCL conference in 1995, many people have made contributions to the field in strikingly diverse ways. Although most researchers in the past came from Western European or North American universities, people are increasingly coming with other backgrounds and bringing new perspectives with them. In this essay, I would like to provide a conceptual and historical perspective on the CSCL field in order to suggest where I see things heading and to give a sense of what will be needed for future generations of CSCL research.

A rule of thumb in cognitive science is that it takes a decade for a person to become an “expert” in a field—as the CSCL community becomes truly global, it may be even more challenging for CSCL researchers in parts of the world less familiar with the traditions that have become embodied in CSCL. The interdisciplinary field of CSCL is particularly multi-faceted and fast changing, which makes expertise as a CSCL researcher—or perhaps, more realistically, as a leading-edge CSCL research lab—a moving target, requiring a mixture of intellectual backgrounds and skills as well as continuing learning, innovation and growth. The purpose of this essay is to suggest the limitations of superficial, one-dimensional approaches and to provide pointers into the literatures that inform a more nuanced understanding of the nature of research in CSCL and of the major issues confronting the field.

This essay grew out of occasions for which I was asked to provide an overview of CSCL for audiences interested in educational technology. In 2003 I gave the opening keynote of the first conference on e-learning in Germany (Stahl 2003). In discussing “The Future of Computer Support for Learning,” I was especially concerned to avoid a techno-centric approach with the audience of engineers and computer scientists, who tended to focus their concerns on technical issues. More recently, at the 2011 conference on Collaboration Technologies and Systems, I was asked to present an overview of the current situation of CSCL research and again wanted to stress a broad socio-technical perspective (Stahl 2011h). When I was invited to give a seminar for doctoral students in educational technology as part of the CSCL post-conference activities in Guangzhou, Shanghai and Beijing, I extended that overview to formulate a message for students preparing to be the next generation of CSCL researchers (Stahl 2011e). In these talks, I developed the set of points that I think are most important to convey to people who want to design future-generation educational-collaboration systems.

In the following sections, I will try to show some of the major historical and theoretical background for the factors I see as crucial to future significant progress in CSCL:

1. Multidisciplinary. The field of CSCL is by definition multidisciplinary, including concerns from education and computer science, and requiring analysis from human and social sciences. As the other factors emphasize, CSCL research requires the skills from many disciplinary trainings.

2. Multivocality. Increasingly, important CSCL research integrates findings obtained by multiple, quite different analytic approaches. While alternative methods seem based on incommensurable foundations, they often produce complementary insights into the workings of collaborative learning.

3. Design-based research. CSCL is a design science, a basic science and a practical application area. It is focused on the design of computer supports, while it must also investigate fundamental issues of the nature of collaborative learning. At the same time, it strives to produce tools and activities for teachers to introduce into their classrooms. Design-based research is a way of conducting such research, in which technical designs and theoretical insights co-evolve through cycles of educational interventions.

4. Socio-technical. CSCL thrives on inspiration from innovative computer technologies, but it must also be concerned with the human issues and societal contexts of potential usage. CSCL systems must be useful and usable by schools, teachers and students in their everyday circumstances. There are tremendous social pressures to resist educational change and to co-opt it to the point where it loses its intended impact.

5. Leverage technology. CSCL was initially significantly inspired by technical opportunities, including promises of artificial intelligence breakthroughs. The continued development of the field requires that it leverage the opportunities that appear every year in new computer technologies. These technologies usually have to be transformed from their design for corporate and entertainment markets to be educationally effective.

6. International. CSCL began as a primarily Western research field. As it begins to spread globally, it encounters new potentials and new challenges. As, for example, the experiences of Singapore (Looi, So, Toh & Chen 2011c) and Hong Kong (Chan 2011b) suggest, non-Western countries may have more political will to adopt CSCL innovations. However, their researchers may face higher barriers to internalizing the cultural background knowledge surrounding CSCL research traditions. Global partnerships are becoming more important than ever.

Overview of CSCL

Schematic histories of educational technology

To understand the current state of research in educational technology, it is important to place it in its evolving historical context. This involves the histories of education, theory, computer technology, software design and educational applications.

The history of education: This development led to the reconceptualization of learning from the transfer of facts to the ability to communicate understanding. Modern education began with the organization of education around the disciplines of the sciences and liberal arts in the early German universities, followed by the provision of universal public education; the ideals of progressive education (Dewey & Bentley 1949/1991); and an emphasis on creative exploration (Neill 1960). Exploratory learning took many forms, including small-group cooperative learning; project-based learning; problem-based learning (Barrows 1994); and collaborative learning or CSCL. CSCL is a latecomer, emerging from the increasing emphasis on group learning and the potential of computers to connect students and support their group-learning experiences.

The history of theory: One can clearly trace in the history of Western philosophy how the unit of analysis of cognition expanded from the individual mind (Stahl 2013b) (see Figure 1). Despite their differences, the philosophies of idealism, rationalism, empiricism from Socrates through Kant all located cognition in the individual human mind. Recent approaches—behaviorism, cognitive science, situated and distributed cognition—have expanded to larger cognitive units, incorporating artifacts, context, social factors and other people. Philosophically, the watershed event occurred with Hegel’s dialectical and deeply social theory, which provided a dynamic, historical approach for subsequent theories such as those of Marx, Wittgenstein and Heidegger—and their many followers. See (Stahl 2006a) for the theory of group cognition, arising from this historical development.

Figure 1. The history of theory and the transition from the individual unit of analysis to post-cognitive theories. From (Stahl 2013b).

The history of computer technology: Technology spread from isolated machines to social infrastructures. In the era dominated by mainframe computers, both operating systems and the applications they ran were largely custom created for individual corporations or governmental agencies that could afford to run them. With the “personal” desktop computer, generic software applications were developed for mass audiences, primarily driven by the need for productivity tools in business. Computer networking and the Internet fostered groupware, to support communication and cooperation within groups of people. Now we have the small, specialized apps of mobile computing along with social networking media, the cloud, the grid and ubiquitous computing. CSCL emerged with networking—especially the Internet public infrastructure (Jones, Dirckinck-Holmfeld & Lindstrom 2006)—and is now incorporating mobile technology (White 2006).

The history of software design: Design expanded to stress how technology would be enacted, adopted, disseminated and used in practice. Techno-centric design became engrained in developers during the early years, when computational cycles and bits of memory were expensive compared to human resources and training. But that led to costly failures of software that people refused to use (Landauer 1996). Ergonomics and human factors provided an initial step, which had to be expanded to a more thoroughgoing approach of human-centered design, involving consideration of the user experience and user needs throughout the design. For more complex applications and less predictable user populations, design-based research is necessary to test how software is likely to be used, as the software is developed. In general, social informatics has become a central focus, investigating how software is likely to be deployed and what social factors affect this. The result is a concern for socio-technical design, in which the human and institutional settings are central concerns entering into technical design decisions. Nowhere are the social factors more explicit than in CSCL, with its focus on social collaboration within the highly political context of schooling.

The history of educational applications: Support for learning expanded from a focus on individuals acquiring facts to communities building knowledge (Koschmann 1996b). In the 1950s, cooperative learning began to be studied in small groups (Johnson & Johnson 1989). Computer-assisted instruction (e.g., arithmetic drill) developed in the 1960s. This was followed by intelligent tutoring systems (including, for instance, user modeling of students’ algebra misconceptions). Later, attempts were made to support creativity and foster mathematical thinking by teaching programming concepts in environments like Turtle Logo (Papert 1980). Since the mid-1990s, CSCL has been developing (e.g., with the CSILE or Knowledge Forum environment). Although older paradigms still dominate the educational mass market, CSCL technologies represent the latest stage in educational technology.

The roles of technology in CSCL

There is a natural tendency for people to think of opportunities that arise from new technologies in purely technical terms. We have seen this in the preceding historical reviews. People tried to design technologies in terms of technical issues and their solutions failed to be adopted and used because social factors had not been taken into account as central design concerns. I see it all the time still, when a student—particularly one trained in a technical field—has an idea based on some technical possibility and proceeds to design something without investigating the human and social considerations. While it is natural to take such an approach, history has taught us repeatedly that this is a deeply flawed approach; it will meet unforeseen problems and will not succeed no matter how good the idea seems on purely technical grounds. The entire field of social informatics (Grudin 1990; Kling 1999; Orlikowski 1992) attests to this.

Of course, software technology necessarily plays a central role in CSCL research. Researchers need to take software prototypes into classrooms and to conduct laboratory experiments that try out new ideas and get real-world feedback as correctives to their assumptions. Even beyond that, innovative software concepts—at least sketched out in designs, mock-ups and prototypes—are crucial for inspiring researchers, potential funding sources and future users like teachers and students. No one knows yet what future CSCL applications will look like and what kinds of features they will provide for learners or teams of knowledge builders. We need creative visions, programmed as software running on digital devices in order for people to even begin to think about how they might be used.

Technology is undeniably important to CSCL. The whole field is based on the potential of networked computers to bring together learners in ways that were not previously possible. However, simply connecting people is not enough. The medium of connection must be carefully designed, studied, tested, analyzed, refined and re-designed to match educational and social settings and constraints. These requirements cannot be treated as an afterthought after the basic technology is already developed—for instance by adding a pretty user interface. The technology must be selected and designed from the start to meet non-technical requirements. This will probably require the involvement in some fashion of potential users throughout the design and development process. Developers of CSCL technologies must study how actual classroom teachers and their students will enact, adapt, interpret and use proposed applications, along with complementary curricular resources and classroom practices.

Technological advances will certainly continue to inspire CSCL innovations. However, this cannot successfully be done in a predominantly techno-centric manner. Innovation will have to be equally inspired by educational goals and by attempts to improve communication among people through the use of technology. For instance, the technical choice of a text-chat medium versus a discussion forum may depend upon a CSCL educational scenario (e.g., brainstorming among small groups in a classroom period or long-term research within a globally distributed team) and social considerations (e.g., whether the participants can find desirable communication partners and concentrate on intense interaction at a specific time).

The role of individual student learners in CSCL

Another natural tendency is to design for learning by individual students. The usual concept of learning involves an increase in knowledge by an individual mind. This is the traditional conception in educational theory—based on common-sense (or folk-theory) assumptions. Although increased knowledge in general can include bodily capabilities, tacit skills or deeper understanding, within education it is generally taken to mean additional factual knowledge that can be expressed in explicit responses to tests.

CSCL is concerned with promoting collaborative learning—learning that takes place in small groups or classrooms of students. In the 1950s and 1960s, research on “cooperative learning” explored how small groups of students learned together. It investigated small-group dynamics and how to structure educational group processes. However, it did not conceive of learning as a group process, but rather looked at the individual learning that took place in the context of small-group activities (Johnson & Johnson 1989). This is the same approach taken by most small-group research in psychology and sociology, even today. CSCL, in contrast, is interested in collaboration (where students build knowledge together) more than cooperation (where group participants divide up tasks and then share each other’s ideas) (Dillenbourg 1999).

This distinction between cooperation of individuals and collaboration of a group defines a major divide in the theoretical frameworks within the CSCL field. Sfard (1998) expressed this divide as a contrast between the “acquisition metaphor” and the “participation metaphor.” By acquisition metaphor, she meant the view that individual minds acquire knowledge; by the participation metaphor, she meant the view that individuals participate in groups or communities that build knowledge. In the first, the unit of analysis, level of description or subject of agency is the individual student; in the second, it is the group. In her later detailed book on the cognition involved in learning mathematics, Sfard (2008) argued that thinking is fundamentally a communication process, and so it takes place within groups and communities more basically than in individual minds (for a review of this book, see Stahl 2008a).

I have developed the view that the group should be the primary unit of analysis for CSCL research in my theory of group cognition (Stahl 2006a). I have found that this view is difficult for most people to accept and consider because our common-sense assumptions about thinking are deeply ingrained. I have spent years trying to come to terms with this view myself, largely by writing about group cognition and exploring it in my research data (e.g., Stahl 2009c). Most people try to reduce group-cognitive results to “underlying” individual mental constructs. But as we will see later in this essay, the most influential theories for CSCL argue that the individual mind is itself a social product, the result of one’s interactions with parents, friends, colleagues, small groups and communities. Not only are individual mental practices derived from interpersonal and community practices, but there are also group knowledge-building processes that are distinct from and not reducible to individual mental processes (Hutchins 1996).

Of course, a group can only build knowledge with the participation of individuals, who must use their individual powers of understanding and communication. So the cognitive work of individuals, small groups and communities in collaborative learning are inseparable and complexly intertwined (Rogoff 1995). Whereas other fields are primarily concerned with individual learning or with community knowledge building, CSCL must be specifically concerned with supporting the small-group processes and the integration of individual, group and community processes. While particular research projects may have to focus on one of these units of analysis, the CSCL research community as a whole will need to understand all the levels and their interrelationships.

The role of testing and assessment in CSCL

The traditional conception of learning as an increase in the ability of an individual to express knowledge in the form of propositions has lead to the prominence of testing of factual knowledge by individual students. An over-emphasis on testing in schools lessens the motivation of teachers to use collaborative learning approaches and causes students to compete for grades rather than collaborate with their peers for knowledge. It promotes the ideology of individualism (Adorno & Horkheimer 1945) and the culture of competition, preparing young people for the former age of industrial capitalism. But in the global, networked economy of the 21st Century, skills and values of teamwork and collaboration are essential; new forms of assessment are needed that support that (Lee, Chan & van Aalst 2006a).

The ideology of individualism has had implications for educational research. The traditional experimental paradigm involves measuring changes in individual knowledge between a pre-test and a post-test. In many cases, the researcher establishes two or more experimental conditions and then codes events based on a pre-conceived scheme of categories. The statistical differences between the codes of the different conditions are then correlated with the increases in the test scores of students in the corresponding conditions to provide evidence that the difference in the conditions contributed to learning. For instance, if a new educational software application was used in one condition and not in the other, then an increase in the test scores of students who used the software would be taken as validation of the software’s educational value and significant differences in the codes of the conditions might indicate causal factors (Chi 1997).

There are a number of limitations of this research approach, despite its usefulness in certain circumstances. By relying on a given set of coding categories, it limits itself to a preconceived conceptualization (theory) and cannot discover other factors. Furthermore, although it can measure correlations, it does not provide insight into how learning mechanisms take place. Statistical analyses rely on large numbers of data points in order to average over individual differences; particularly for experiments with small groups—where there are learning effects within the groups as well as between individuals—the number of necessary cases is generally unpractical (Cress 2008). Most educational experiments of this type result in no significant findings (Russell 1999).

As discussed below, it is generally more effective to assess educational interventions with innovative software using a mixture of analysis approaches, including case studies that try to understand the meaning making that takes place within small groups. I have used interaction analysis (Jordan & Henderson 1995) techniques to identify some of the processes that groups use to make meaning (Stahl 2006a) and build knowledge together (Stahl 2009c). These studies can provide insight into computer-supported collaborative learning by following the sequential flow of student utterances responding to each other in the logs of their interactions (Stahl 2011b). Because the students are problem solving together, they necessarily express their individual thinking to each other and this is available to analysts in the logs. In addition, the group’s stream of proposals, responses, questions, agreements, etc. is available for analysis as an extended cognitive process. The conversation analysis focuses on the sequential nature of the thinking, which is lost in most statistical coding analyses, where individual utterances are coded and then counted without regard for their sequential-response ordering.

Perhaps the most interesting problem with focusing on the individuals when analyzing or assessing collaborative learning is related to the paradox of “productive failure” (Barron 2003; Kapur & Bielaczyck 2012; Kapur & Kinzer 2009; Pathak, Kim, Jacobson & Zhang 2011; Schwartz 1995). The phenomenon of productive failure is a significant discovery in the CSCL literature. It consists of the finding that small groups that show less learning in the short term sometimes reveal more—and deeper—learning in the long run. It seems that these groups take time to develop a more abstract understanding of the problem they are working on and that this extra group-cognitive effort detracts from their ability to score well on their immediate tasks—relative to groups that just follow standard procedures to get answers without trying to understand the deeper issues. However, the abstract understanding gained by the “failure” groups gives them an advantage when facing challenging tasks in the future. The effect carries over to the individuals in the groups, so that the test scores of the individuals from the groups with the deeper understanding may score poorly on the immediate test, but best on future tests. This seems to provide a strong argument against the validity of traditional testing itself, for testing tends to reflect immediate results more than underlying learning.

The phenomenon of productive failure can be understood more generally in terms of Vygotsky’s theory of the zone of proximal development (Vygotsky 1930/1978). Vygotsky argues that inter-personal learning generally precedes individual learning. One learns initially by interacting with other people and then gradually (often over years) “internalizing” this learning into individual skills. These skills primarily involve the use of language or artifacts to mediate cognition—a use that is generally acquired in a social setting, where cultural artifacts and practices are used. Thus, it should not be surprising that students can accomplish tasks in small groups that they cannot duplicate immediately in individual tests, but that might be essential for their future abilities, which can show up in future tests.

This phenomenon has potential implications for assessment of individual and collaborative learning. Because students must make the thinking visible to each other in collaborative work, an instructor or an analyst with access to the logs of the group interactions can see quite clearly the level of understanding in the group, the contributions of specific individuals and the changes in understanding at both the group and individual levels. For instance, when a teacher has students in a course work on weekly assignments in online chats or discussions and then has them give group presentations to the rest of the class, the learning that is taking place is quite visible to the teacher, and there is no need to subject the students to standard forms of individual testing. Furthermore, the students see each other’s learning and can see how they are doing relative to others. In CSCL this is called “making learning visible” (Stahl 2006a, Chapter 18); it contributes to student meta-learning, or their ability to assess how they are learning themselves, without relying on external sources of feedback from tests and teachers (Lee, Chan & van Aalst 2006b).

The role of supporting intersubjective meaning making

Given the central role of group cognition in CSCL settings, a major goal of educational software should be to support the group processes that foster intersubjective meaning making. It is not sufficient to provide factual knowledge and to motivate individual effort. It is also important to attract students to work together in effective groups:

to provide appropriate communication media for their interaction;

to offer tasks that stimulate interaction and require collaboration;

to provide social awareness of what everyone is doing in the group interaction;

to represent progress on the task in ways that help to conceptualize it;

to display the group approach in a visual joint problem space; and

to document the accomplishments of the groups.

Intersubjectivity is central to the research field of CSCL. Although the concept has been mentioned occasionally in the history of philosophy and in the CSCL literature, it has never been clearly worked out. This should be a task of CSCL research and theory.

What is intersubjectivity? It is not a thing or an individual facility—although it relies on basic human linguistic and mental abilities (Gallese & Lakoff 2005). Intersubjectivity is the ability of people to understand each other. How is it that when one person in a group speaks, types or gestures other people in the group can understand what is meant? We need not speculate whether the meaning is “represented” the same way in each person’s head; that may not be a meaningful question (Wittgenstein 1953). The point is that the people can continue smoothly with their interaction, perhaps adding evidence that they share a joint understanding by their subsequent behavior. Clearly, sharing a language provides an extensive basis for intersubjectivity, because languages are enormous symbolic repositories of culturally transmitted meaning. Relatedly, we are socialized to interact with other people intersubjectively (see Figures 2 and 3).

Figure 2. An infant and adult share a meaningful orientation in the world, mediated by pointing.

Figure 3. God and man share a meaningful gesture (excerpt from Michelangelo, the Sistine Chapel).

As Vygotsky points out in his analysis of an infant gesturing, the establishment of shared meaning provides the basis for our individual understanding of that meaning (Vygotsky 1930/1978, p. 56). Fundamentally, intersubjectivity is grounded in our existing in a shared meaningful world (Heidegger 1927/1996; Stahl, Zhou, Çakir & Sarmiento-Klapper 2011). In addition, it is worked out constantly, as we interact with other people, repair misunderstandings, fill in shared understandings and refine existing partial understandings. Careful analysis of logs of CSCL data can provide detailed analyses of intersubjectivity and its role in computer-supported collaborative learning (Stahl, Zemel & Koschmann 2009). Such analysis, in turn, can suggest ways to improve computer support for intersubjectivity.

Alternative Approaches within CSCL

The theoretical divide

CSCL is a multidisciplinary field. This gives CSCL the enormous advantage that it has applied to the complex problems of designing computer support for collaborative-learning expertise and knowledge from the fields of computer science, education, psychology, communications, artificial intelligence and school-subject domains. However, it also has the consequence that researchers have brought with them to CSCL diverse and seemingly conflicting views of how to conduct science. This has often led to a feeling of “us” and “them” between groups of researchers—sometimes simplistically referred to as a choice between “quantitative” versus “qualitative” approaches to research. This divide is derived from a fundamental dichotomy in the larger scientific world.

Perhaps the most profound and innovative attempt to understand that dichotomy is the work of Habermas:

There are competing theoretical approaches in the social sciences that differ not only in the kinds of problems they address and the research strategies they apply, but in their fundamental principles. They diverge in their choice of categorical frameworks and in how they conceptualize their object domain—that is in how they define what it is they are actually studying. These differences of conceptual strategy express more deeply rooted conflicts: conflicting views of science and cognitive interests. (Habermas 1971/2001, p. 3)

Habermas distinguishes approaches of social sciences in terms of three decisions:

1. Whether “meaning” is admitted as a primitive term.

2. Whether intentional action is conceptualized in the form of purposive-rational action or in the form of communicative action.

3. Whether all social phenomena must be analyzable in the form of statements about the actions of individual subjects.

I would apply the first decision to research in CSCL by distinguishing two general paradigms, which I will refer to as the “objective paradigm” and the “meaningful paradigm.” The objective paradigm has roots in positivist philosophy of natural science and in behaviorist social science, although it includes the critiques of these within cognitive science. The meaningful paradigm has roots in interpretive (verstehende) sociology and anthropology; it includes much of the situated-action critique of cognitivism. Adherents to the objective paradigm try to analyze their data objectively, without subjective interpretation of what the data meant for the subjects. For instance, they rely on objective pre- and post-tests to measure changes in student knowledge and manipulate experimental conditions. Quantitative statistical analyses confirm or disconfirm hypothesized patterns of effects for average subjects. Adherents of the meaningful paradigm try to understand the meaning expressed in their data, often by focusing in detail on the meaning-making processes in a specific case study.

Habermas makes the distinction (in the second decision) between purposive-rational action and communicative action—which he also calls “work” and “interaction” (Habermas 1967/1971). He argues that both are necessary. Purposive-rational action is our primary way of interacting with nature, controlling it to meet our needs by following strategic thinking. It is close to what Suchman (1987) calls “plans” in contrast to “situated action” and to what Dreyfus (1992) attributes to the rationalist tradition in Western philosophy, as opposed to tacit and embodied knowledge. However, communicative action, or interaction, is fundamentally different from work; it is in the social and ethical realm of living with other people. It involves understanding, negotiation and intersubjectivity. Habermas believes that interaction presupposes an ideal form of communication among unconstrained peers—an ideal that is never completely achieved. In practice, work and interaction are always intertwined, and matters of power or manipulation assert work-type strategies into communicative action, imposing systematic distortions. Within CSCL settings, we can see that students blend strategic goal-oriented work on their assigned tasks with social interaction with their peers. This takes place in school contexts filled with interpersonal and power relationships. CSCL research must also involve both purposive-rational action and communicative action, working on technological nature and interacting with students.

Habermas’ third methodological decision relates to the unit of analysis and the ideology of individualism discussed above. Although many social sciences have tried to reduce social phenomena to actions of individuals, there are also holistic, ecological, functionalist and structuralist theories that do not. As we have seen, the history of theory has increasingly moved beyond the individual mind to post-cognitive theories of distributed and situated cognition (Stahl 2013b).

Dimensions of analysis

In addition to the distinction between objective and meaningful stances toward CSCL data and toward the world, there are numerous other dimensions of approaches to analysis. For instance, in the temporal dimension one can look at very brief episodes of interaction or much longer, longitudinal studies—ultimately over a lifetime or even generations, as learning and meaning are embedded in cultural artifacts and preserved.

As already indicated, the unit of analysis is critical; one can study actions and processes of individuals, small groups, classrooms, communities of practice or whole cultures.

From a socio-technical viewpoint, one can focus on technology options, features and approaches or one can consider how a software system will be enacted by its user community, how people will integrate it into their lives and workflow, how usage will be disseminated through communities and how the user community may drive future evolution of the application.

As topics of investigation, one can study different learning issues like motivation, knowledge or efficiency. One can make comparisons based on learner characteristics, such as age, nationality, socio-economic status. Of course, there are the different disciplines of learning—e.g., mathematics, argumentation, science, informal learning. In addition, there are different pedagogical approaches to be supported: instructionist, exploratory, socio-cognitive, socio-cultural, knowledge acquisition or knowledge building. These different research directions may suggest different facilitating technologies: scripting, game-like applications, mobile-device interactions or tabletop interfaces, for instance (Dillenbourg & Evans 2011).

Multi-vocal methods

Although there have been many differences of approach within CSCL research in the past, there seems to be a strong tendency among leading researchers in the present to converge (Spada, Stahl, Miyake & Law 2011). This is not happening by one approach winning out over others, but rather through a growing recognition of the power and even necessity of incorporating multiple approaches in exploring the design of educational applications. This shift has proceeded through a number of steps:

The first step was to reject any a priori commitment to a specific methodological approach—such as the one in which one may have been trained. The widely accepted rule of thumb now is that the approach should be selected based upon the nature of one’s research interests, questions, hypotheses and data.

The second step was to recognize that a research agenda in CSCL will probably go through a sequence of phases and that different approaches are likely to be most productive in different phases. For instance, an informal exploratory approach might be appropriate to a pilot phase in which issues first emerge. Then a comparative statistical test might indicate which factors are most significant. Following that, a micro-analytic case study could probe the processes and mechanisms that are behind the statistical findings. In addition, at any phase mixed methods can be used to triangulate views on the same phenomenon from different theoretical or methodological perspectives. It may then be necessary to iterate the whole sequence of phases multiple times as a software application and its pedagogical scenario are re-designed and refined—so that they co-evolve to refine their effects.

The third step was to discover the complementarity of objective and meaningful analyses. Many researchers who started with one of these approaches realized as they articulated their findings that they needed evidence that could only come through the other approach. Just as people generally need both strategic actions in dealing with nature and communicative actions in interacting with people and just as CSCL as a field needs to address both technical and social issues, so a CSCL research project may need to conduct objective, controlled, statistical analyses as well as careful interpretations of meaning-making processes.

The fourth step was to recognize the power of collaboration across research labs, including globally. By bringing together researchers from different traditions, collaborative research efforts have access to more theoretical viewpoints, methodological approaches, educational technologies and rich data sources. Of course, there are fundamental differences between different approaches and methods cannot be mixed indiscriminately. The issues in moving from an individual method to multivocality have yet to be resolved, even though the trend in that direction seems promising. The research questions that CSCL faces are complex and involve different aspects and components, which may be best analyzed by different methods. An investigation of meaning making in groups may benefit from an objective analysis of individual behaviors and vice versa—without denying the theoretical differences among the approaches.

The multi-vocal approach has been explicitly explored in a series of workshops at CSCL conference from 2007 to 2012. Behind the scenes of these workshops has been an international collaboration of CSCL researchers to explore shared datasets from a multiplicity of analytic and theoretic perspectives, culminating in a large edited volume on multivocality (Suthers, Lund, Rosé & Law 2013). This effort epitomizes the direction in which CSCL research is moving. Other efforts requiring significant multidisciplinary collaborations involving international CSCL research teams include the systemic introduction of CSCL reforms in Hong Kong (Chan 2011b) and Singapore (Looi et al. 2011c). The movement of the CSCL field toward such international collaborations in order to address the complexity of the challenges to significant progress in CSCL has implications for the next generation of work in this field.

Lessons for Future Generations of CSCL Researchers

Lesson 1: Learn collaboratively in multi-disciplinary labs

The consequences of the preceding discussion seem quite clear. While an individual researcher must focus on a specific, well defined project and must rely on his or her background, training and interests, significant contributions to CSCL are likely to continue to come from research collaborations, which span both disciplinary and theoretical boundaries. The illusion that a lone programmer with a bright idea, working in an isolated garage can produce an application that will be significant is an urban myth. Collaborate!

On the other hand, labs interested in educational technology desperately need highly skilled and creative software designers, developers and engineers. Commercial software—even software that claims to be for schools or education—is rarely adequate to meet the needs of creative researchers. Someone has to develop mock-ups and running prototypes to show researchers, teachers and students what is possible and to give them hands-on experience. This takes a technical understanding of the latest software possibilities and the ability to create innovative software. It is important, however, that the software developers understand the perspective of the other researchers and educators and can communicate effectively with them both ways: to understand their ideas and to explain the possibilities and limitations of the technology.

I learned how to collaborate most clearly in the Virtual Math Teams (VMT) Project (Stahl 2009c). This was a collaborative project with the Math Forum, an online resource site for mathematics students and teachers. The project team included math educators (the director of the Math Forum and his staff), an anthropologist, an ethnomethodologist, four research assistants (from four different countries), a series of visiting researchers, software developers and myself. While a number of the researchers had backgrounds in computer science, we had to bring in a series of specialized developers to build, debug, re-design and launch the many versions of our software environment.

Over the years, we had several visiting researchers—mostly from Europe—who each stayed for three to six months. The first was an enthusiast of quantitative data analysis, who developed a multi-dimensional coding scheme for our data and trained us in the objective approach. The next was a statistician, who analyzed our initial coding. Then a dialogical researcher came and exposed us to Bakhtin’s (1986) views. Other researchers helped to refine our software design or conducted studies using our data. We also encouraged international colleagues to run experiments with our software and to analyze our data in a variety of ways. For several years, we conducted weekly data sessions in which the team looked at data excerpts together—line by line—and discussed their meaning from our different viewpoints. Looking at the same data in this shared environment, we learned to see through each other’s eyes.

Lesson 2: Study different approaches to CSCL issues

Learning all the theories, concerns and methods needed to conduct CSCL research is a daunting challenge. It helps to have a solid grounding in computer science, education, psychology, communications, artificial intelligence, philosophy, social science and school disciplines. Not many graduate students start with that, which is why they need to collaborate with others. In addition, newcomers to CSCL have to catch up on some of the classics of the field. To meet this need, I start my courses in CSCL with the following two slim books and three chapters from the CSCL edited volume (Koschmann 1996a). The two books are truly seminal; they define the socio-cultural theory that is central to CSCL; they deserve to be read thoughtfully multiple times. The three chapters define the beginnings of CSCL as a field distinguished from educational technology more broadly conceived.

- Vygotsky (1930/1978) – Vygotsky argues for analysis of psychological and cognitive phenomena in terms of how they develop in individuals through social interaction and how they are mediated by external artifacts and language. Intersubjective learning is primary. The zone of proximal development (the range in which people can accomplish cognitive tasks collaboratively, but not yet individually) shows how group cognition becomes internalized as individual cognition, just as spoken communication becomes self-talk and then internal thought.

- Lave & Wenger (1991) – Learning is conceptualized as legitimate peripheral participation of people within communities of practice, on the model of apprenticeship training. This provides an anthropological basis for viewing learning as a matter of developing social practices in groups and cultures, rather than solely as a mental process of individuals.

- Koschmann (1996b) – This chapter reviews the history of paradigms of educational technology, proposing the stages presented in the beginning of this essay. It provides the historical context for the differences in conceptualizing educational technology.

- Scardamalia & Bereiter (1996) – This pioneering work of CSCL presents the motivation for the CSILE or Knowledge Forum educational-collaboration software, based on the importance of writing within a community. It stresses the educational advantages of public expression, refinement of ideas, scientific community processes of publication, networked sharing of documents, and guidance in scientific reflection. It proposes that classrooms of students engage in networked communication practices analogous to the publication practices of scientific communities, with their conference papers and journals.

- Roschelle (1996) – This is a classic CSCL analysis of dialogical meaning making, shared understanding and external representations in a joint problem space. The author uses conversation analysis and other qualitative methods to articulate the learning of physics in a case study of two students working with a computer simulation of motion and gravity.

Lesson 3: Conduct design-based research

Educational software is not created through a traditional (waterfall) software-development sequence of design, implement, debug, test, disseminate and then research impact. Rather, there is usually an on-going cyclical process of trying something out, seeing how it is used, responding to problems through re-design, testing alternative versions, etc. In other words, the different phases are tightly coupled and the design-implement-test-redesign cycle is repeated as frequently as possible. The software development and the educational research are interdependent. This is how most serious CSCL research is conducted. It is called design-based research (Barab & Squire 2004; Brown 1992; DBR Collective 2003) because the research drives the design and the development work provides both opportunities and motivation for the research. In its richest form, the research process modifies the theory and the analysis methods along with the software applications and the pedagogical practices. Design-based research is particularly appropriate for CSCL because it integrates the software-design process with research into collaborative learning, simultaneously increasing our understanding of how collaborative learning works and how it can be supported effectively.

Again, I look at the VMT Project as an example of design-based research. We constantly modified the software and tried new features—often frustrating groups that were trying to use it. We had teachers try it in classrooms at various levels, from middle school (age 15) to junior college and graduate school. We ran math contests with students from around the world and encouraged other researchers to use it where they could. This supplied us with a continuing flow of feedback on our various versions and interventions. In some cases, informal reports from teachers and remarks from students were enough to guide redesign; in other cases, we studied student interactions in the VMT environment quite intensively—even basing doctoral dissertations on the interactions of a single group. We started with a generic chat app and a simple math problem and gradually evolved a complex collaboration environment for exploring mathematical relationships.

Lesson 4: Engage in socio-technical design

Because CSCL is a meeting place for collaborative learning and computer support, research in this field generally combines some exploration of technological media with an investigation of its use or adoption by students, teachers and/or school systems. Of course, a focused research paper might just report on one aspect of a larger research effort—perhaps a technical feature, a learning achievement or a theoretical conception. However, these findings are likely to emerge from more inclusive research agendas and to be considered within such broader contexts. Most CSCL research should probably not be conceived of as isolated technology innovations, self-contained experiments or well-defined theoretical insights, but as contributions to a larger effort to transform education, using networking technology as a lever.

Lesson 5: Leverage technological advances

CSCL began with the recognition of the potential of computer technology to bring people together in new ways and to support their learning together. So, computational, digital, networked technology will always play a central role in CSCL research. As new techniques, devices and media become available, they will continue to inspire new educational approaches. The popularity of video games (especially multi-user games that require teamwork and learning), mobile computing, tabletop devices, ubiquitous access to information, social networking and future technologies suggest new forms for educational software and new models for collaboration and learning. However, the lesson of the past is that schools are very slow to change and that the promises of past technologies like radio, television and film to transform educational practices did not materialize. To successfully leverage the new technical opportunities will require a deep understanding of existing practices and a careful refining of applications if educational technologies are to enter the classroom effectively without being completely co-opted into the traditional systems of schooling.

Lesson 6: It takes a global village

While educational technology will have to be accepted into one classroom at a time, that acceptance will have to be part of a much larger, well-conceived effort. We still have only relatively vague ideas about what an educational system based on computer-supported collaborative learning would look like. Despite the fact that the world’s major software developers have long recognized the importance of software to support collaboration, they have produced only the most primitive tools for working together—and virtually nothing for learning together. Email and texting have severely limited ability to support serious collaboration, yet that is all that most people use. The mass media’s image and the mass market’s inventory of educational software is at least fifty years behind the times, still oriented toward factual and arithmetic drills and the like. Yet, all that CSCL can offer is a series of research prototypes and proofs of concept. To build a robust knowledge of how to put the ideals of CSCL into practice will take a global effort of researchers, teacher professional development, school reform and political will. The CSCL field has succeeded in spreading the basic ideas and changing attitudes in certain circles. Successful attempts at a small scale can serve as models for larger transformations. It is clear that CSCL has something extremely important for building the future society and that this is increasingly being recognized around the world. It will take a continuing effort by the global CSCL community working together on the technology, pedagogy, research, theory, policy, training and practice to move significantly forward. The past has laid a rich and intricate basis. The present shows hopeful signs (Chan 2011a; Looi, So, Toh & Chen 2011a). The future holds promise for achieving some of the opportunities offered by our technological age.

Conclusion

If you are a student of educational technology or a new researcher planning to be active in the next generation of CSCL research, you may need to focus on a specific project, artifact, intervention or experimental manipulation—but you should also be aware of the multiple dimensions of alternative possibilities and issues. Stay grounded in the specific focus and what you can find in your data, but consider how that data might look with other conceptualizations. Build your argument, but take seriously counter-arguments from other perspectives. Work respectfully with people from different intellectual traditions and invite them to collaborate and bring their approaches to your project. Advances in CSCL will increasingly come from multidisciplinary research labs and from global collaborations.

Part B: Theories of Group Cognition

3. How I View Learning and Thinking in CSCL Groups

This is an invited keynote talk that opened the International Conference on Computers and Education (ICCE 2009) on November 30, 2009, in Hong Kong, China (see video at http://www.youtube.com/user/GerryStahl#p/u/2/h5MpUJnTipM). The intent of the talk was to provide a personal view of the field of computer-supported collaborative learning (CSCL) and to relate it to the Asia-Pacific audience. To do this, it tried to describe—in an informal tone—the approach I am currently taking to analyzing online interaction in small groups. In publishing the talk, I have tried to retain its original tone.

The field of CSCL is particularly interested in the ways small groups can build knowledge together thanks to communication and support from networking technology. I hope that CSCL environments can be designed that make possible and encourage groups to think and learn collaboratively. In my research, my colleagues and I look at logs of student groups chatting and drawing about mathematics in order to see how they build on each other’s utterances to achieve more than they would individually. To answer this important question, we must look carefully at the details of discourse in CSCL groups and develop innovative tools and theories. In this talk, I outline methods and levels of analysis that have resulted in the findings reported in the Virtual Math Teams research cited in the references.

“Learning without thought is labor lost;

thought without learning is perilous.”

--  Confucius  孔丘  Kong Qiu

Views of Learning and Thinking

A

bout 2,600 years ago, Confucius viewed learning and thinking as belonging together.

The learning sciences of the twenty-first century agree. They view learning as involving meaning making by the learners (Stahl, Koschmann & Suthers 2006). Students who just passively accept instruction without thinking about it and coming to understand it in their own way of making sense of things will be wasting everyone’s time. Why? Because they will not be able to use the new knowledge or to explain it. Of course, this construction of meaning takes place over time: someone can learn something one day and make sense of it later, when they try to use it in different circumstances and to explain their use to other people and to themselves. But if they never integrate what they have learned into their own thinking and acting—by applying it where appropriate and talking about it clearly—then they will not have really learned anything important.

What sociologists like Bernstein, as presented in Hasan’s overview (1999), know about social interactions and contribute to our understanding of the significance of group cognition is the way participants internalize the resources that evolve within one interactional context and then recontextualize them in the new and radically different contexts they find themselves in later. In this way, the new knowledge that is created, or the new or enhanced knowledge-building skills that are appropriated, can replicate and spread contagiously. It is the magic that, for instance, makes seemingly inconsequential interactions between mothers and children while cleaning the oven play a key role in a child’s preparation for schooling (Cloran, 1999). It is precisely because of the tremendous impact the results of these interactions can have going forward that the local sacrifice that may occur in terms of efficiency of the interaction can be viewed as a small price to pay when one considers the long-term cost-benefit ratio, the profound impact of one transformational experience of group cognition.

Vygotsky (1930/1978) made an even stronger argument. He showed for the major forms of human psychological functioning that the individual capabilities were derived from experiences of interactions between people:

An inter-personal process is transformed into an intra-personal one. Every function in the child’s cultural development appears twice: first, on the social level and later, on the individual level; first between people (inter-psychological), and then inside the child. This applies equally to voluntary attention, to logical memory, and to the formation of concepts. All the higher functions originate as actual relations between human individuals. (p. 57)

Although all functions of individual cognition are derived from group cognition, the reverse is not true. As Hutchins (1996) demonstrated with his example of the bridge of a large Navy ship, not all group cognition can be internalized by an individual: “The distribution of knowledge described [in the book] is a property of the navigation team, and there are processes that are enabled by that distribution that can never be internalized by a single individual” (p. 284). Whether or not specific skills and knowledge can be mastered by individuals or only by teams, the learning of those skills or knowledge seems to rely heavily and essentially on group cognition. That is why we try to promote and to study group cognition.

What we, as learning scientists, have learned about learning and thinking in recent decades in the West is influenced by what philosophers before us said. For instance, most Western philosophers until the middle of the 1900s thought that knowledge could be expressed by propositions, sentences or explicit statements. If that were true, then the learning of knowledge could, indeed, consist simply of students individually hearing or reading the right sentences and remembering them.

However, Ludwig Wittgenstein’s book, Philosophical Investigations, published in 1953, questioned this view of learning and thinking. It looked at math as a prime example. Mathematical knowledge can be seen as a set of procedures, algorithms or rules. Wittgenstein asked how one can learn to follow a mathematical rule (Wittgenstein 1944/1956, Part VI; 1953, §185-243, esp. §201). For instance, if someone shows you how to count by fours by saying, “4, 8, 12, 16,” how do you know how to go on? Is there a rule for applying the rule of counting by fours? (Such as, “Take the last number and add 4 to it.”) And if so, how do you learn to apply that rule? By another rule? Eventually, you need to know how to do something that is not based on following a propositional rule—like counting and naming numbers and recognizing which numbers are larger. The use of explicit rules must be somehow grounded in other kinds of knowledge. These other kinds include the tacit knowledge of how to behave as a human being in our culture: how to speak, count, ask questions, generalize, put different ideas together, apply knowledge from one situation in another context and so on. And these are the kinds of things that one initially learns socially, in small groups or in child-parent dyads. Wittgenstein’s question brought the logical view of knowledge as explicit propositions into a paradox: if knowledge involves knowing rules, then it must involve knowing how to use rules, which is itself not a rule.

Wittgenstein was an unusual philosopher because he said that problems like this one could not be solved by contemplation, but rather by looking at how people actually do things. He said, “Don’t think, look!” (1953, §66). In studying group cognition, I try to follow Wittgenstein’s advice. I try to view how people actually do things. Rather than telling you what my views or ideas are about learning and thinking in CSCL groups, I want to show you how I view or observe learning and thinking in CSCL groups.

The term “view” in the title of my essay has this double meaning: it means both viewing by looking at something with my eyes and also viewing in the metaphorical sense of thinking about something from a conceptual perspective. The Greek philosopher, Plato, who lived at about the same time as Confucius, made this metaphor popular in Western thought (Plato 340 BCE/1941).

Although Wittgenstein himself did not actually look at empirical examples of how people follow rules in math, we can. By carefully setting up a CSCL session, we can produce data that allows us to view small groups of students learning how to follow math rules and thinking about the math rules. This is what I do to view learning and thinking in CSCL groups. It is the basic approach of the science of group cognition (Stahl 2009d) that I want to describe today.

The work of our research team and other colleagues involves looking closely at some rich examples of student groups learning and thinking about math. We would like to share a brief excerpt from one of these examples with you and talk about how we go about viewing the learning and thinking of this group of students. In particular, how do they construct their group cognition through collaborative meaning-making activities?

In this essay, we will look at the meaning-making work of a group of students, analyzing their language-based interaction at multiple levels:

- the overall event,

- a specific hour-long session of the two-week event,

- a discussion theme that arose,

- a discourse move that triggered that theme,

- a pivotal interchange,

- a single utterance and

- a particular reference in the utterance.

By looking at the linguistic connections, we can see how the syntax, semantics and pragmatics weave a network of meaningful references that accomplishes a set of cognitive achievements.

On the one hand, we can see the linguistic elements of the log and their structure of temporal and hierarchical relationships as accomplishing group cognition by, at each moment, constraining the next utterance as situated in the context of event, session, theme, discourse moves, eliciting adjacency pairs, preceding utterances and network of references. On the other hand, human actors creatively design accountable responses within the constraining situation defined by these contextual elements. That is, among the constraints on the actors is the requirement that their linguistic actions make sense in the on-going discourse and that they reveal their meaning and relevance in their linguistic design.

Although people often design their utterances to convey the impression that they are the result of psychological processes (change of mental state, expression of internal reflections), we can analyze the group cognition in terms of the linguistic effects of the observable words and drawing actions, without making any assumptions about individual mental representations. The individual students are active as linguistic processors—interpreting and designing the utterances—but the larger mathematical and cognitive accomplishments are achieved through the group discourse, which exists in the computer displays, observable by the students and—even years later—by analysts. We can see and make explicit how teams become teams in the ways that they manifest the contingencies and accountabilities of their unique situation, using conventional linguistic structures as resources.

The work of the Virtual Math Teams (VMT) research team—which I directed from 2003-2014—and collaborating researchers involves looking closely at some rich examples of student groups learning and thinking about math. I would like to share a brief excerpt from one of these examples with you and talk about how we go about viewing the learning and thinking in this group of students.

An Example of Learning and Thinking

The event: VMT Spring Fest 2006 Team B

Here, we will be talking about an online event that occurred several years ago. The interaction is preserved in a computer log, which can be replayed by researchers. Three students, probably about 16 years old, were assigned to be Team B and they met with a facilitator in an online chat environment on May 9, 10, 16 and 18, in 2006, for about an hour in the late afternoon each day. The participants were distributed across three time zones in the US. The event was part of the VMT research project. Neither the students nor we know anything more about each other’s personal characteristics or background.

The topic for this event was to explore a pattern of sticks forming a stair-step arrangement of squares (see Figure 1) and then to explore similar patterns chosen by the students themselves.

Session I

1. Draw the pattern for N=4, N=5, and N=6 in the whiteboard. Discuss as a group: How does the graphic pattern grow?

2. Fill in the cells of the table for sticks and squares in rows N=4, N=5, and N=6. Once you agree on these results, post them on the VMT Wiki

3. Can your group see a pattern of growth for the number of sticks and squares? When you are ready, post your ideas about the pattern of growth on the VMT Wiki.

Sessions II and III

1. Discuss the feedback that you received about your previous session.

2. WHAT IF? Mathematicians do not just solve other people's problems—they also explore little worlds of patterns that they define and find interesting. Think about other mathematical problems related to the problem with the sticks. For instance, consider other arrangements of squares in addition to the triangle arrangement (diamond, cross, etc.). What if instead of squares you use other polygons like triangles, hexagons, etc.? Which polygons work well for building patterns like this? How about 3-D figures, like cubes with edges, sides and cubes? What are the different methods (induction, series, recursion, graphing, tables, etc.) you can use to analyze these different patterns?

3. Go to the VMT Wiki and share the most interesting math problems that your group chose to work on.

Figure 1. Topic for VMT Spring Fest 2006.

The VMT online environment consisted primarily of a synchronous chat window and a shared whiteboard. At the end of each session, the students were supposed to post their findings on a wiki, shared with other teams participating in the Spring Fest. Between sessions, the facilitator posted feedback to the students in a textbox on the whiteboard.

The session: Session 3, May 16, 7 pm

Let’s look at an excerpt from the end of the third session. The three students had already solved the original problem of the stair-step pattern of squares. They had also made up their own problem involving three-dimensional pyramids. Now they turned to look at the problem that Team C had described on the wiki after session 2. Team B is looking at an algebraic expression that the other team of students had derived for a diamond pattern of squares. They start to draw the pattern in their whiteboard (see Figure 2) and chat as a team about the problem of this new pattern.

Figure 2. The VMT Replayer showing the VMT online environment.

The theme: “I have an interesting way to look at this problem”

One of the students, Aznx, begins to make a proposal on how to “look” at their problem. First, he announces, “I have an interesting way to look at this problem.” Note that he uses the word “look” in the same double meaning of “view” that was mentioned above. As we will see, he means he has a new way to think about the problem mathematically—and that involves a way of observing a visual image of the problem. The group does its thinking both by typing text or algebraic expressions in the chat window and by simultaneously drawing and viewing diagrams or geometric constructions of the problem in the shared whiteboard (see Çakir, Zemel & Stahl 2009, for an analysis of the coordination by the group of their text, symbols and drawings).

Aznx’ announcement opens an opportunity for the group to discuss a way of looking at the problem. In fact, the group takes up the offer that is implicit in Aznx’ statement and the students spend the next eight minutes trying to each understand it. As it turns out, they will work on this view of the problem for the rest of this session and most of their final session.

A VMT chat session can generally be analyzed as a series of themes or discussion topics. Often, themes come and go, and different themes overlap, with one wrapping up while another gets started. Researchers can identify the boundaries of a theme: when a new theme opens and an old one closes (Zemel, Xhafa & Çakir 2009).

In this case, the group has been talking about how the diamond pattern grows as a geometric figure for a couple of minutes and then they discuss Team C’s algebraic expression for a couple of minutes. As those themes get played out and there is a pause in the chat, Aznx makes a move to open a new theme for the group.

A move: Showing how to view the problem

Aznx’ announcement that he has a perspective to share with the group is a way of introducing a new theme, a “pre-announcement” (Schegloff 2007, pp. 37-44; Terasaki 2004). Conversations often flow by new contributions picking up on something that was already being discussed. Online text chat tends to be more open than face-to-face talking; chat does not follow the strict turn-taking rules of conversation. However, it is still common to do some extra work to change themes even in chat. In a sense, Aznx is asking permission from the group to start a new theme. Quicksilver responds encouragingly right away by saying, “Tell us” (see Figure 3).

line

date

start

post

delay

919

5/16/06

19:35:26

19:35:36

0:00:06

Aznx

I have an interesting way to look at this problem.

920

5/16/06

19:35:41

19:35:42

0:00:03

Quicksilver

Tell us

921

5/16/06

19:35:38

19:35:45

0:00:00

Aznx

Can you see how it fits inside a quare?

922

5/16/06

19:35:45

19:35:45

0:00:07

Bwang

yes

5/16/06

19:35:49

19:35:52

0:00:00

Bwang

[user erased message]

923

5/16/06

19:35:51

19:35:52

0:00:01

Quicksilver

Yes

924

5/16/06

19:35:52

19:35:53

0:00:02

Bwang

oh

925

5/16/06

19:35:55

19:35:55

0:00:06

Bwang

yes

926

5/16/06

19:35:53

19:36:01

0:00:04

Quicksilver

You are sayingthe extra spaces...

927

5/16/06

19:35:58

19:36:05

0:00:06

Aznx

Also, do you see if you add up the missing areas

Figure 3. The move to introduce Aznx’ new way of looking at the group’s problem. (This log for analysis encodes the chat stream and associated awareness messages about when people started typing, along with timing data to reflect the flow of discourse.)

Actually, Aznx already starts typing his proposal before he gets Quicksilver’s response, but it is not posted until afterward. The next step in his proposal is: “Can you see how it fits inside a square?” Here, he structures his contribution as a question, which elicits a response from the other members of the team. Note that he uses the term “see” in his proposal with the same double meaning as the term “look” in his prior announcement. As we shall see (in both senses), the group tries to work out and comprehend Aznx’ proposal both conceptually and visually.

Both Bwang and Quicksilver respond to Aznx’s proposal with “Yes.” However, both modify this response. Bwang starts to type something else, but erases it; then he posts two messages: “oh” and “yes.” This suggests some hesitation in responding to the proposal immediately. Quicksilver follows his initial positive response with, “You are saying the extra spaces …” He is asking for more clarification of the proposal. While Quicksilver is typing his request for clarification, Aznx is typing an expansion of his initial proposal: “Also, do you see if you add up the missing areas …”

The analysis of interaction moves is central to the science of group cognition. This is the level of granularity of many typical group-cognitive actions. Discourse moves are ways in which small online groups get their work done. They often follow conventional patterns—speech genres (Bakhtin 1986) or member methods (Garfinkel 1967)—which makes them much easier for participants to understand. Researchers can also look for these patterns to help them understand what the group is doing.

In this case, a new theme is being opened, one that will provide direction for the rest of this group’s event together. This move is an example of one way in which a group can establish a shared understanding of a diagram or select a joint problem conceptualization (depending on how we take the terms “look” and “see”). Other moves that we often see in VMT logs are, for instance, defining shared references, coordinating problem-solving efforts, planning, deducing, designing, describing, solving, explaining, defining, generalizing, representing, remembering and reflecting as a group.

A pair: Question/response: “Can you see how it fits inside a square?” / “Yes”

In conversation analysis, one typically looks for “adjacency pairs” (Duranti 1998; Sacks 1965/1995; Schegloff 2007). A prototypical adjacency pair is question/answer. Aznx’ offering of a question—“Can you see how it fits inside a square?”—followed by Bwang and Quicksilver’s responses—“yes,” “Yes”—illustrate this structure for the simplest (“preferred”) case: one person poses a yes/no question and the others respond with an affirmative answer.

Response structures are often more complicated than this. Text chat differs from talk in that people can be typing comments at the same time; they do not have to take turns and wait until one person stops talking and relinquishes the floor. They will not miss what the other person is saying, because unlike with talk, the message remains observable for a while. The disadvantage is that one does not observe how people put together their messages, with pauses, restarts, corrections, visual cues, intonations and personal characteristics. While it is possible to wait when you see that someone else is typing a message, people often type simultaneously, so that the two normal parts of an adjacency pair may be separated by other postings. For example, Quicksilver’s question (line 926 in Figure 3) separated Aznx’s continuation of his line 921 posting in line 927, because 926 appeared before 927 although 927 was typed without seeing 926. So in chat we might call these “response pairs” rather than “adjacency pairs.” While they may be less sequentially adjacent than in talk, they are still direct responses of one posting to another.

Because the sequencing in online chat texting is less tightly controlled than in face-to-face talk, response pairs are likely to become entangled in the longer sequences of group moves. This may result in the common problem of “chat confusion” (Fuks, Pimentel & Pereira de Lucena 2006; Herring 1999). It can also complicate the job of the researcher. In particular, it makes the task of automated analysis more complicated. In convoluted chat logs, it is essential to work out the response structure (threading) before trying to determine the meaning making. The meaning making still involves participants interacting through the construction of response pairs, but in chat people have to recreate the ties among these pairs. Realizing this, the group members design their postings to be read in ways that make the response pair or threading structure apparent, as we will see (Zemel & Çakir, 2009).

An utterance: Question: “Can you see how it fits inside a square?”

In his posting—“Can you see how it fits inside a square?”—Aznx is comparing the relatively complicated diamond shape to a simple square. This is a nice strategy for solving the group’s problem. The group can easily compute the number of stick squares that fill a large square area. For instance if there are five little squares across the width of a square area (and therefore five along the height), then there will be five-squared, or 25 little squares in the area. In general, if there are N little squares across the width, there will be N-squared to fill the area. This is a strategy of simplifying the problem to a simple or already known situation—and then perhaps having to account for some differences. So Aznx’ posting seems to be relevant to thinking about the math problem conceptually.

At the same time, Aznx poses his proposal in visual or graphical terms as one of “seeing” how one shape “fits inside” of the other. The group has been looking at diagrams of squares in different patterns, both a drawing by Team C in their wiki posting and Team B’s own drawings in their whiteboard. So Aznx’s proposal suggests visualizing a possible modification to one of the diamond drawings, enclosing it in a square figure (see the blue diamond pattern enclosed in the red square in Figure 4). He is asking the others if they can visualize this also, so that the group can use this to simplify and solve their problem with the diamond.

Figure 4. Blue diamond patterns and red stair-step patterns.

Aznx presents his proposal about re-thinking the problem as a question about visualizing the diagram. The group has been working in the VMT environment, going back and forth between text in the chat and drawings in the whiteboard. They have started with problems presented graphically and have discussed these graphical problems in their text chat. They have shared different ways of viewing the relationships within the drawings and they have gradually developed symbolic algebraic ways of expressing general relationships about patterns in these drawings, working out these symbolic expressions in the chat and then storing them more persistently in the whiteboard.

We have been calling Aznx’ chat posting a “problem-solving math proposal” (Stahl 2006a, chapter 21). However, it is presented in the grammatical form of a question. Aznx did not simply state a proposal like, “I think we should enclose the diamond in a square, calculate the size of the square and then subtract the missing areas.” Rather, he first announced that he had “an interesting way to look at this problem” and then explained his way of looking by asking if the others could “see how it fits inside a square.” Presenting a proposal calls on the others to accept the proposal and to start to work on it. Of course, the others can reject the proposal, ask for clarifications about it, make a counter-proposal or ignore the proposal.

But Aznx’ utterance is not a full proposal that the others must accept or reject. It is another preliminary step. It asks the others if they can visualize something. It puts this to them as a question. If they say yes, then Aznx can proceed to make his proposal—or perhaps the others will see the implications of his interesting way to look at the problem and propose the strategy without Aznx having to advocate it, explain it and defend it. If they say no—that they cannot see how it fits inside a square—then he can explain his view further so they will be better prepared to accept his proposal.

Aznx’ chat posting avoids articulating a complete proposal; by starting the conversation about the visualization, it involves the others in articulating the proposal collaboratively. In fact, in the subsequent discussion, the others do “see” the strategy that is implicit in Aznx’ interesting view of the problem and they do help to articulate the strategy and then pursue it. By designing his proposal as this preliminary question about viewing the problem, Aznx succeeds in directing the group problem solving in a certain direction without his having to fully work out a detailed, explicit proposal. Aznx does not seem to be presenting a solution that he has worked out in his head. Rather, he is presenting his “interesting idea” for an approach to solving the problem so that the group will proceed to use the idea and work as a group to try to solve the problem with this approach.

A reference: “It”

Aznx’ question is ambiguous at a purely syntactic level. It asks the others, “Can you see how it fits inside a square?” To what does the term “it” refer? People use pronouns like “it” rather than lengthy explicit noun phrases when the reference is clear from the context. This situates the utterance in its context—its meaning cannot be gathered from the utterance considered in isolation. Often, “it” will reference something that was recently referred to in a previous contribution that the new utterance is building on. For instance, “it” could refer to something mentioned in Aznx’ previous utterance, “I have an interesting way to look at this problem.” But to say that it refers to “this problem” does not make complete sense. The problem does not fit inside a square.

However, a minute earlier, when the group was discussing Team C’s equations, Aznx said about part of an equation, “The 3n has to do with the growing outer layer of the pattern I think.” He was referencing different aspects of the growth of the diamond pattern, particularly its “outer layer.” Therefore, when he announces that he has an interesting way to view the problem, it is reasonable to assume that his new way of looking may be closely related to the observation that he had just reported about the outer layer of the diamond pattern. Because everyone in the group was following the flow of the discussion, Aznx could refer to the topic of the outer layer of the diamond pattern in the shorthand of the pronoun “it.” When he typed, “Can you see how it fits inside a square?” he could assume that the readers of this posting would understand that he was referring to how some aspect of the diamond pattern can be seen as fitting inside of some square shape.

Although the reference to some aspect of the diamond pattern is relatively clear, the details are not clear about just what aspect of the diamond is to be visualized or focused on visually, where a square is to be constructed, and how the diamond fits inside the square. At this point, only a rather confusing image of a diamond pattern is visible on the whiteboard (see Figure 2). To make sense of “it,” everyone has to follow the flow of discussion and the way in which the math topic is being developed as a “joint problem space,” understood and visualized by the whole group.

Bwang and Quicksilver both respond initially to Aznx’ question with “Yes.” However, as we saw, Bwang indicates some hesitancy in his response and Quicksilver asks for further clarification. Aznx and Quicksilver discuss what they see when they fit a diamond pattern inside a square. Quicksilver notes that the “extra spaces” (colored red in Figure 4) look similar to the stair-step pattern that the team worked on previously. But Aznx goes on to talk about the four squares on the outer areas of the square, confusing Quicksilver. That is, as they each try to work out the details of Aznx’ view, they display that they are not seeing things quite the same way. They have not yet achieved an adequate shared understanding or shared view.

Quicksilver suggests that Aznx show what he means on the whiteboard, so the ambiguity of his proposal can be resolved. Rather than drawing it himself, Aznx asks Bwang to do a drawing, since Bwang said he could see what Aznx was talking about. Bwang has in the past shown himself to be skilled at making drawings on the whiteboard, while Aznx has not tried to draw much.

Bwang draws a very clear diagram on the whiteboard for the diamond pattern when N=2 (see Figure 5). As soon as Bwang completes his drawing, he makes explicit the problem-solving proposal that is implicit in Aznx’ way of viewing the problem or the pattern: “We just have to find the whole square and minus the four corners.” His drawing has made this process very visible. He drew the diamond pattern with white squares and then filled in a large square that the diamond fits into by adding red squares. The red squares fill in symmetrical spaces in the four corners of the diamond pattern. The group can now look at this together in the shared whiteboard, providing a shared view of the matter to the group.

Figure 5. Bwang has drawn the white diamond for N=2 with red squares filling in the corners of an enclosing square. Quicksilver is pointing to a diamond pattern for N=3, also re-drawn lower on the whiteboard.

The group then discusses the view of the diamond pattern fitting into an enclosing square. They eventually realize that some of their observations are only true for the diamond pattern at a certain stage, like N=2.

So Bwang then draws the pattern for N=3. Here it starts to become visible to the group that the red squares in each corner follow the stair-step pattern (see Figure 6).

Figure 6. Bwang expanded his drawing to make the diamond for N=3. Note the red corners are now stair-step patterns.

The group has realized that viewing a graphical image of a mathematical pattern can be very helpful in thinking about the pattern. They treat the whiteboard as a shared, viewable image of aspects of the joint problem space of their collaborative work. Viewing this image and pointing out elements of it ground their chat discourse.

However, the image drawn by Bwang captures just one particular stage in the pattern, one value of N. They then start to look at images for different values of N or different stages in the growing pattern. They count the number of red squares in a corner as N increases and notice that it goes: 0, 1, 3, 6 (see Figure 4). This pattern is familiar to them from their earlier analysis of the stair-step pattern. They call this sequence “triangular numbers,” from Pascal’s triangle, which is often useful in combinatorics math problems. They know that this sequence can be generated by Gauss’ formula for the sum of the consecutive integers from 1 to N: (N+1)N/2. Unfortunately, at that point Bwang has to leave the group. But when they return in session 4, they will quickly put together the simple formula for the enclosing square minus this formula for the number of squares in each of the four corners, to solve their problem.

Viewing the Learning and Thinking

Let us pause now from all these details about the case study of three students in a VMT session and talk about how we view learning and thinking in CSCL groups. We have tried to demonstrate how we view learning and thinking in CSCL groups by viewing with you how a group of three students engaged in collaborative thinking and learning processes within an online environment for drawing and chatting.

We went through several levels of analysis of the group discourse (see Figure 7). We started by mentioning the overall context of the event. This was an online event in which Team B, consisting of three students, met in the Virtual Math Teams environment to discuss patterns of squares formed by sticks. We then focused on the smaller session unit, looking at Team B’s third session, in which they considered a pattern that another group, Group C, had analyzed. Within this session, we identified one of several themes of discussion in that session, namely the one involving Aznx’ “interesting way to look at this problem.”

Event:             VMT Spring Fest 2006, Team B

Session:            session 3, May 16, 7:00-8:00 pm

Theme:            “I have an interesting way to look at this problem”

Move:            Show how to view

Pair:            “Can you see how it fits inside a square?” “Yes”

Utterance:            “Can you see how it fits inside a square?”

Reference:      “it”, diamond pattern

Figure 7. Levels of analysis of online group discourse.

Aznx introduced the theme by initiating a group problem-solving move. Namely, he got the group to view the problem in a certain way, as a diamond enclosed in a square. We saw how the group ended up drawing images in their shared whiteboard of diamond patterns enclosed in squares. Aznx introduced this group move in a subtle way; he did not simply come out and say, “We should analyze this pattern as partially filling an enclosing square.” Rather, he first announced that he had an interesting view, involving the others in his approach to make it a group problem-solving process. Then he asked if the others could view the problem in a certain way. He did this through a question/answer response pair: he asked a question, which elicited a yes-or-no response from the others. By eliciting the response, he oriented the others to looking at the diagram in the whiteboard in a certain way—namely in the way that his question implicitly proposed. A set of lines on the whiteboard are not immediately meaningful—they must be seen (interpreted) as something (Heidegger 1927/1996, §32; Wittgenstein 1953, §II xi).

Aznx’ formulation of his question looks like a simple utterance in question format, but it entails selection from a number of different ways of picturing the relationships among the diamond pattern, the enclosing square and the empty corners. To begin with, one must decide what the reference to “it” is doing.

Indexical references like the pronoun “it” are ubiquitous in online text chat—and unavoidable according to Garfinkel (1967). They require the reader to understand or reconstruct the implicit threading or response structure of the chat. The difficulty of doing this often leads to confusions, which require the participants to spend time clarifying the content and structure of their discussion. For instance, in our example of the move of seeing the diamond in the square, the group had to engage in a couple minutes of chatting and drawing to co-construct a shared understanding of the problem.

Issues of shared understanding can be analyzed as linguistic problems of reference. In other words, in order to view learning and thinking in CSCL groups, we do not try to figure out what is going on in the heads of the students; rather, we try to figure out what is going on in their chat postings and their drawing actions. This is what we call the group’s interaction. In VMT, the interaction of the virtual math team consists of sequences of chat postings and drawing actions.

Our first step in figuring out what is going on in the chat postings and drawing actions is generally to try to analyze the sequencing of these by reconstructing their response structure—what previous action each new action is responding to and what kinds of action it is eliciting, what it is opening up an interaction space for, or what kinds of responses it is making relevant as next postings. Often, this leads to some kind of threading diagram (Çakir, Xhafa & Zhou 2009), uptake graph (Suthers, Dwyer, Medina & Vatrapu 2010b), or interaction model (Wee & Looi 2009). This represents graphically a basic structure of the meaning-making sequencing. Then we try to understand what problem-solving work is being accomplished at each point in the sequence. This involves looking at different levels of granularity, such as the event, session, theme, move, pair, utterance and reference. Understanding the meaning that the group is co-constructing in their interaction generally involves going back and forth through these different levels and integrating partial interpretations from the different levels (Gadamer 1960/1988).

Through this process, we can gradually view the learning and thinking that takes place in the CSCL group. This learning and thinking is not something that takes place primarily in the minds of the individual participants (although the individuals in the group are each continuously using their linguistic skills to understand what is going on and to respond to it with their postings and drawings). Rather, when there is an intense collaborative process taking place in the online environment, the thinking and learning takes place in the visible text and graphical interactions.

According to the theory of group cognition, thinking in a CSCL collaborative interaction does not take place so much the way we usually think of thinking. Thoughts, or cognitive processes, do not take place by neurons connecting and firing in a brain; they take place by text postings and drawings referring to each other and building on each other, in the spirit of the idea of transactivity. We will look more at how this takes place in a minute. Similarly, learning does not take place the way we learned about learning. It is not a change in the amount of knowledge stored in a brain. Rather it is a matter of knowledge artifacts being gradually refined through sequences of text postings and graphical drawings that are interrelated and that explicate each other. The knowledge artifacts may be statements about a problem the group is working on, as viewed from a new perspective that the group has developed. The knowledge artifact might be a drawing like Bwang’s in Figure 6 or an algebraic formula that sums up the group’s analysis of pattern growth.

Unpacking the Group Learning and Thinking

Rather than talking about learning and thinking in the abstract, let us unpack some more how learning and thinking take place in Team B’s interaction—in their text chatting and drawing together. Let’s go back through the hierarchy of levels of analysis in the opposite order to say something about how references, utterances, response pairs, moves, themes, sessions, and events can contribute to learning and thinking in CSCL groups (see Figure 8).

Reference:      network of references, indexical ground, joint problem space

Utterance:            recipient design for reading’s work

Pair:            projection and uptake

Move:            getting the problem-solving work done

Theme:            coherent interactional sequences

Session:            temporal structuring and re-member-ing

Event:             forming groups and co-constructing knowledge objects

Figure 8. Levels of learning and thinking in online group discourse.

Reference: Network of meaning, indexical ground, joint problem space

When one studies logs of virtual math teams, one sees that they spend a lot of time reaching shared understanding about references in their postings. My conference paper later this week reviews an example of this from Team B’s session four, where Aznx, Quicksilver, and Bwang get quite confused about references from the chat to different equations written on the whiteboard (Stahl et al. 2009).

The reason that people devote so much time and energy to resolving confusing references is that the network of references that they build up together plays an extremely important role in their group learning and thinking. In the theory of CSCL, there is considerable emphasis on the ideas of “common ground” (Clark & Brennan 1991) and “joint problem space” (Teasley & Roschelle 1993). A group establishes common ground largely by reaching a shared understanding of how references work in their discourse. As it interacts over time, a group co-constructs a network of references that can become quite complex (Sarmiento & Stahl 2008).

This network of references defines the context or situation in which the group discourse continues to take place. Aznx’ reference to “it” that we looked at contributed to a network of meaning that the group built up continuously through their interaction. This network included images of sticks in various patterns (like diamonds at stage N=2 and N=3), the relationships of the patterns (like a diamond enclosed in a square with stair-step empty corners), concepts referred to by technical terms (like “triangular numbers” or “summation”), and symbols representing mathematical operations (like equations for number of squares in a pattern).

As a group builds up its network of shared references, it can use more shortcut references to point to things without creating confusion. People can use deictic references to point to things in the network, like “this formula,” “the second equation,” or “it.” In linguistic terms, the shared network of references provides a background for referring to things, a so-called indexical ground of deictic reference (Hanks 1992).

In problem-solving terms, the network of references forms a joint problem space, a shared view of the topic that the group is addressing (Sarmiento & Stahl 2008). For Team B, the joint problem space starts with the stair-step pattern and the chart of the number of sticks and squares for each stage of this pattern as presented in the topic description for the event. By the middle of session 3, it includes the diamond pattern and the view of “it” enclosed in a square, forming empty corners. It also includes triangular numbers and their associated formula, as well as several other equations from Team C and from Team B’s own work. The team’s interaction (the text postings and drawings) gradually creates this joint problem space and is situated within it. The work and utterances of the team can only be understood (by the participants and by us as researchers) through an on-going understanding of the joint problem space as a network of meaningful reference.

Utterance: Recipient design for reading’s work

While both the students who participate in the sessions and the researchers who analyze the logs need to understand the network of references, they understand them in very different ways. The students understand how to respond to what is going on the way they might understand how to ride a bike down a hillside. That is, they are not reasoning about it explicitly, rationally, logically, consciously. Rather, they are paying attention to what is going on and responding knowingly and intuitively. Quicksilver has not carried out any kind of analysis of Aznx’ word “it” the way I did; yet he could respond to it with a sophisticated set of questions. He only had a couple of seconds to respond, whereas I could spend hours going back and forth over the log reasoning about explicit interpretations.

People are incredibly skilled at using language without thinking about how they do it. In fact, even researchers are only aware of a small percentage of what people take into account almost instantaneously without being aware of it. We say that Aznx “designs” his announcement and proposal so that it will be read by Bwang and Quicksilver in a way that will lead them to understand in a complex way. They will figure out what “it” is referencing, but also realize some of the ambiguity of the reference. They will also come to think about the strategy for finding the number of squares in the diamond pattern because of this ambiguity. However, Aznx does not design his statement explicitly, through a rational sequence of logical arguments. Rather, as a skilled user of language, he gives voice to a well-designed posting that responds to the current discourse situation. It is somewhat like the way a skilled off-road biker responds to the terrain intuitively as she is speeding down a rough hillside with no time to think about what she is doing—and she somehow designs an optimal path for her journey.

Aznx was successful in designing his question so that it would be read in a certain way within the context of the group’s discussion in their joint problem space. This is what ethnomethodology calls the “accountability” of utterances (Garfinkel 1967). We call this “recipient design.” This simply means that utterances are designed to be understood by their recipients, by the audience for whom they are intended. That is, utterances are designed to meet the expectations of their recipients (Garfinkel 1967). In chat, postings are designed to be read in a certain way by the other chat readers. This is in contrast to utterances in spoken talk, which are designed to be heard, and are therefore given subtle vocal emphasis and timing. Chat postings, on the other hand, can incorporate capitalization, abbreviations, symbols, punctuation, emoticons and special fonts. They can reference previous postings that occurred further back in time because the chat text is persistent, remaining visible or retrievable for longer than speech. In chat, group work takes place as reading; chat postings must be designed to support reading’s work of understanding the posted utterances in their discourse context (Livingston 1995; Zemel & Çakir 2009).

Response pair: Projection and uptake

An important aspect of the design of utterances or postings is how they are designed to fit into what comes before and after them. In general, an utterance performs an uptake or response to something that came before (Suthers, Dwyer, Medina & Vatrapu 2010a). At the same time, it elicits a follow-up, or at least makes relevant certain forms of subsequent utterances by others (Schegloff 2007). Through its uptake and projection, an utterance provides continuity to the discourse—in fact, it thereby creates a temporal structure (Heidegger 1927/1996).

The clearest and simplest example of this is the adjacency pair or response pair, such as a question/answer pair. A question elicits an answer. That is, stating a question projects that an answer will be given in response. It opens a conversational space for an answer. It makes it relevant for the next utterance to be an answer responding to the question. In other words, a question is designed to be read as something that should be responded to with an answer. A question worded like “Can you see how it fits inside a square?” is designed to be answered with a “yes” or a “no.” The question-and-answer pair forms a unity, a small unit of interaction between people. The “yes” response shows that the posting it is responding to was read as a question and creates the pair as a successful question/answer interaction.

One of my first discoveries in studying virtual math teams was that math discourse is largely driven forward by what I called “math proposal response pairs” (Stahl 2006a; 2006b). These have the following structure:

- An individual makes a bid for a proposal to the group suggesting how the group should continue to do its mathematical work.

- Another member of the group accepts (or rejects) the proposal on behalf of the group.

This is the simple, default form of the math-proposal response pair. If the proposal is accepted, then work begins on the proposal, often in the form of a follow-up proposal.

Of course, there are many variations and complications possible. The bid can be ignored or never responded to. In that case, it does not function as an effective proposal; at best it is a “failed proposal.” Before a proposal response is made, there can be other response pairs inserted in the middle of the expected pair—such as a clarification question. It is also possible that someone will propose an amendment to the proposal bid before the original is accepted. Thus, a simple pair can develop a complicated recursive structure of insertions, extensions, repairs, etc.—with each of these being subject to their own insertions, extensions or repairs. Eventually, each of the intervening pairs may get closed with its anticipated response and then the original pair may be completed.

Move: Getting the problem-solving work done

Group problem-solving moves often have the structure of a longer sequence than a simple pair. Such a longer sequence may consist of a complex of response pairs embedded in one another. To identify such a structure, it may be necessary to first conduct a threading analysis to determine what is responding primarily to what. Then, it is often useful to see how this longer response sequence is built up out of simple response pairs (Stahl 2011b).

Together, these intertwining response pairs form a successful move, introducing a new theme for the group. As an example, let’s look again at Aznx’s move in Figure 9. We can see four response pairs there:

line

start

post

919

19:35:26

19:35:36

Aznx

I have an interesting way to look at this problem.

920

19:35:41

19:35:42

Quicksilver

Tell us

921

19:35:38

19:35:45

Aznx

Can you see how it fits inside a quare?

922

19:35:45

19:35:45

Bwang

yes

19:35:49

19:35:52

Bwang

[user erased message]

923

19:35:51

19:35:52

Quicksilver

Yes

924

19:35:52

19:35:53

Bwang

oh

925

19:35:55

19:35:55

Bwang

yes

926

19:35:53

19:36:01

Quicksilver

You are sayingthe extra spaces...

927

19:35:58

19:36:05

Aznx

Also, do you see if you add up the missing areas

Figure 9. The move to introduce Aznx’ new way of looking at the group’s problem.

1. Aznx announces, “I have an interesting way to look at this problem” and Quicksilver responds by asking him to “Tell us.”

2. Aznx asks, “Can you see how it fits inside a square?” and first Bwang responds “yes.” Then Quicksilver responds, “Yes.” Then Bwang responds again, more emphatically, “oh … yes.”

3. Quicksilver asks a clarification question about the proposal implicit in Aznx’ question, “You are saying the extra spaces …[?]”

4. Aznx, in parallel with Quicksilver’s question asks a follow-up question, which contains an implicit further proposal about the group’s work: “Also, do you see if you add up the missing areas […?]”

As the discussion continues, Quicksilver responds to Aznx’ question and the two of them continue to discuss the issues raised in both their questions.

Theme: Coherent interactional sequence

Aznx’ move introduces the theme of the diagonal pattern viewed as enclosed in a square with missing spaces in the four corners. As we have just seen, the move consists of multiple response pairs that drive the work of the group to consider this theme.

As the theme evolves, the group draws and discusses some increasingly elaborate drawings to view the patterns that the theme involves. The group considers different stages of the pattern (N=1, 2, 3, 4) and how the number of missing spaces changes as the diamond pattern grows.

This leads them right to the point where they can formulate an equation to summarize their analysis of the pattern growth. Unfortunately, Bwang has to leave the session and they do not complete this work. During the fourth session two days later, the group picks up this theme and discusses it repeatedly, eventually deriving the equations for number of squares and sticks in the diamond pattern at all stages (Stahl 2011b). This theme is the basis for the equation for number of squares, which simply subtracts the number of missing spaces in the four corners of a square that encloses the diamond pattern.

Session: temporal structuring and re-member-ing

After Bwang left the third session, Aznx and Quicksilver try to review the group’s accomplishments. They become confused about various equations and unsure of their ability to explain what the group has figured out. They end the session with Quicksilver saying, “then let’s pick it up next time when Bwang can explain it.” This ends one session and projects what will happen in a future session.

When the group meets for the fourth session, Aznx and Quicksilver do eventually get Bwang to review the derivation of the equation based on the view of the problem that Aznx introduced in the theme we just considered. The discussion in session four refers back to the group’s work in session three and also to Team C’s work in session two. But it does this in ways that are situated in Team B’s session-four context (Sarmiento-Klapper 2009). The team members and the memories they bring with them from the past are re-constituted in the new situation, made relevant to the current themes, problem space and available resources.

Event: Forming groups and co-constructing knowledge artifacts

At the beginning of session one, the students were not part of a particularly effective group or team. They did not build much on each other’s contributions and were hesitant to make proposals, ask each other to undertake tasks, produce permanent drawings or manipulate mathematical symbols. That all changed dramatically during their four-session event. By the end, they had many graphical, narrative and symbolic representations or expressions related to their mathematical topic. They worked effectively together and solved their problems well. Problem-solving methods that one person introduced were later proposed and used by the other group members.

You may be wondering if each of the students learned mathematics. The interesting thing about looking closely at what really went on in this event is that what we traditionally consider to be the math content actually plays a relatively minor role in the group’s problem solving. Yes, content is brought in: the students talk about triangular numbers and they apply the formula for summing consecutive integers, for instance. Often, this math content is brought in quickly through proposals by individuals. It is then discussed through responses to the proposal that check that everyone understands the math content and agrees on its applicability. However, the bulk of the hard work is not accessing the traditional math content, but selecting, adapting, integrating, visualizing, sharing, explaining, testing, refining, building on and summarizing sequences of group response pairs. These proposals and discussions reference not only math content, but also various related resources that the group has co-constructed.

The learning and thinking of the group takes place through the group’s discourse, as a temporally unfolding multi-level structure of response pairs interwoven into larger sequences of group moves, problem-solving themes, and sessions of events. The group learns about the mathematics of its topic by building and exploring an increasingly rich joint problem space. It thinks about the mathematical relationships and patterns by following sequences of proposals, raising and responding to various kinds of questions, and engaging in other sorts of interactional moves. Some of this gets summarized in persistent knowledge artifacts like drawings, concepts, equations, solution statements and textual arguments. The building of the joint problem space generally requires a lot of work to resolve references and to co-construct a shared network of meaning (Stahl et al. 2011).

The math skills—like following certain procedures to do long division or to transform symbols—are not where the deep learning takes place and real knowledge is involved. Rather, the ability to sustain progressive inquiry through methods of group interaction is the real goal. This ability makes use of the math skills as resources for answering questions and coming up with new proposals.

If you wonder how to view learning and thinking in CSCL groups, follow Wittgenstein’s advice: “Don’t think, look!” My colleagues and I have tried to do this by looking at the work of virtual math teams in the way I have just described. We have been amazed to discover that collaborative learning and group cognition are a lot different than people thought.

CSCL as a New Approach to Computers in Education

Reading is learning, but applying is also learning

and the more important kind of learning at that….

It is often not a matter of first learning and then doing,

but of doing and then learning, for doing is itself learning.

--Chairman Mao  毛泽东 (1936)

Computers in education bring many advantages, even as seen within a traditional view of education:

- They give students and teachers access to all the information on the Web.

- They provide the ability to access lectures anywhere/anytime/on large scales.

- They can support testing, tutoring and scripting of learning processes.

- They offer simulations, educational gaming, virtual reality and artificial intelligence.

But networked computers in education—using CSCL software environments like VMT—also open opportunities for a radically new view of learning and thinking:

- Networking of students can let them get together with others interested in similar topics around the world.

- Effective collaborative-learning experiences help students learn how to work, think and learn in groups. Group work is a new force of production in the world and students need to learn how to produce knowledge in teams.

- CSCL events can give students first-hand, hands-on experience in knowledge building.

- Discussing mathematics in peer groups teaches students how to do math, how to talk about math, how to make math connections, how to learn math and think mathematically.

In this second view of computers in education, book learning of facts and rote procedures has a place, but the more important kind of learning comes through doing. CSCL groups can provide effective learning experiences in which teams of students actually do mathematics by exploring rich problem spaces and discussing them the way that Aznx, Quicksilver and Bwang did.

There are two popular approaches to CSCL theory:

- Collaborative learning can be seen as an extension of traditional individual learning. Individuals possess knowledge that they can state in sentences and can communicate to other individuals. Our commonsense concepts can describe this and we can measure what individuals know at different times. Learning in this traditional view is an increase in individual knowledge

- Collaborative learning can be viewed as being qualitatively different from traditional individual learning, and we need to discover the nature of collaborative learning and its relation to individual learning by exploratory research. We need to re-think our ideas about learning, collaboration, education, computer support, research methodology and cognitive theory (Stahl 2006a). We need to look carefully at data from real CSCL sessions to see what actually takes place there, without imposing our commonsense views.

It should be clear by now that I view learning and thinking in CSCL groups as a mystery to be investigated, not as something well understood to be measured. It is a new form of human existence with great potential. We must observe it to learn how it works. My colleagues and I have begun to do this, as have other researchers in CSCL. I have tried to indicate to you here how you can go about observing learning and thinking in CSCL groups.

It may be easier to understand issues of technology design and of traditional instruction when studying computers in education than to understand this new view of learning and thinking. However, I believe that if we hope to get the most benefit from computers in education and to understand how groups learn and think in CSCL groups, then we will have to closely observe the discourse and interaction in ways similar to what I have presented here.

Notes on Group Cognition

When one studies logs of virtual math teams, one sees that the teams spend a lot of time and effort constructing shared understanding about references in their postings. The reason that teams and other small groups devote so much time and energy to resolving confusing references is that the network of references that they build up together plays an essential role in their group learning and thinking. In the theory of CSCL, there is considerable emphasis on the idea of “common ground” (Clark & Brennan 1991) and “joint problem space” (Teasley & Roschelle 1993). A group establishes common ground largely by reaching a shared understanding of how references work in their discourse. As it interacts over time, a group co-constructs a network of references that can become quite complex.

The “shared understanding” that is built up is akin to the notion of co-orientation, which refers to the mutual orientation of individuals in a group toward an object (knowledge, belief, attitude), and can be traced back to the interactionist social psychology of John Dewey and George Herbert Mead. Psycho-linguistic metaphors of comparing stored mental representations are unnecessary and can be misleading, reducing all knowledge to individual mental possessions. Team members share a world centered on their task; they orient as a group to the objects that populate that world, such as Aznx’ proposals, Bwang’s drawings and Quicksilver’s queries. Because they share a common world—which they co-constitute largely through their discourse, mediated by the larger common social, cultural and historical horizons of their world—they can co-construct a shared understanding.

The shared network of references defines the context or situation in which the group discourse continues to take place (Heidegger 1927/1996, §18). Aznx’ reference to “it” that we looked at contributed to a network of meaning that the group built up continuously through their interaction. This network included images of sticks in various patterns (like diamonds at stage N=2 and N=3), the relationships of the patterns (like a diamond enclosed in a square with stair-step empty corners), concepts referred to by technical terms (like “triangular numbers” or “summation”) and symbols representing mathematical operations (like equations for number of squares in a pattern).

The co-construction of shared understanding by a small group is what I refer to as “group cognition.”

This essay represents a disciplinary perspective from Computer-Supported Collaborative Learning (CSCL), an interdisciplinary field concerned with leveraging technology for education and with analyzing cognitive processes like learning and meaning making in small groups of students (Stahl et al. 2006). Group cognition is a theory developed to support CSCL research by describing how collaborative groups of students could achieve cognitive accomplishments together and how that could benefit the individual learning of the participants (Stahl 2006a).

It may well be that a group of students working together manages to solve problems faster than any of the individual students may have been able to do alone—particularly when the problem is challenging for them. However, the most important benefits of group cognition are the potential for genuinely innovative solutions that go beyond the expertise of any individual in the group. It is the deeper understanding that is achieved through the interaction as part of that creative process—and the lasting impact of that deep understanding that the students take with them when they move on from that interaction—which they may then carry with them as new resources into subsequent group problem-solving scenarios. Group cognition can then be seen as what transforms groups into factories for the creation of new knowledge.

The types of problems that have been the focus of exploration within the group-cognition paradigm have not been routine, well-structured problems where every participant can know exactly what their piece of the puzzle is up-front in such a way that the team can function as a well oiled machine. Many critical group tasks do not fit into well-known and practiced protocols—for example, low-resource circumstances that may occur in disaster situations, where standard solutions are not an option. In acknowledgement of this, the focus within the group-cognition research has been on problems that offer groups the opportunity to explore creatively how those problems can be approached from a variety of perspectives, where the groups are encouraged to explore unique perspectives.

The processes that are the concern of group-cognition research have not primarily been those that are related to efficiency of problem solving. Rather, the focus has been on the pivotal moments where a creative spark or a process of collaborative knowledge building occurs through interaction. Our fascination has been with identifying the conditions under which these moments of inspiration are triggered, with the goal of facilitating this process of group innovation and collaborative knowledge creation.

The field of CSCL has explored what makes group discussions productive for learning under different names, such as transactivity (Berkowitz & Gibbs, 1983; Teasley, 1997; Azmitia & Montgomery, 1993; di Lisi & Golbeck, 1999), uptake (Suthers, 2006), social modes of co-construction (Weinberger & Fischer, 2006), or productive agency (Schwartz, 1998). Despite differences in orientation between the subcommunities where these frameworks have originated, the conversational behaviors that have been identified as valuable are quite similar. Specifically, these different frameworks universally value explicit articulation of reasoning and making connections between instances of articulated reasoning. For example, Schwartz and colleagues (1998) and de Lisi and Golbeck (1999) make very similar arguments for the significance of these behaviors from the Vygotskian and Piagetian theoretical frameworks, respectively. The idea of transactivity as a property of a conversational contribution originates from a Piagetian framework and requires that a contribution contain an explicit reasoning display and encode an acknowledgement of a previous explicit reasoning display. However, note that when Schwartz describes from a Vygotskian framework the kind of mental scaffolding that collaborating peers offer one another, he describes it in terms of one student using words that serve as a starting place for the other student’s reasoning and construction of knowledge. This implies explicit displays of reasoning, so that the reasoning can be known by the partner and then built upon by that partner. Thus, the process is very similar to what we describe for the production of transactive contributions. In both cases, a transactive analysis would say that mental models are articulated, shared, mutually examined and potentially integrated.

Group cognition is a post-cognitive theory. Post-cognitivism is a tradition characterized by situated, non-dualistic, practice-based approaches. Cognitivism—which tends to retain theoretical remnants of the Cartesian dualism of the mental and physical worlds—originally arose through the critique of behaviorism, with the argument that human responses to stimulae in the world are mediated by cognitive activity in the mind of the human agent. This argument was particularly strong in considerations of linguistic behavior (Chomsky 1959). More recently, post-cognitivist theories have argued that cognitive activity can span multiple people (as well as artifacts), such as when knowledge develops through a sequence of utterances by different people and the emergent knowledge cannot be attributed to any one person or assumed to be an expression of any individual’s prior mental representations (e.g., Bereiter 2002, p. 283).

Group-cognition theory explicitly focuses on these inter-personal phenomena and investigates data in which one can observe the development of cognitive achievements in the interactions of small groups of people, often in online collaborative settings, where interactions can be automatically logged. By interaction, we mean the discourse that takes place in the group. Group cognition is fundamentally a linguistic (speech or text) process, rather than a psychological (mental) one. Thus, unlike the theory of transactivity described above, this post-cognitive approach does not assume cognitive constructs such as mental models, internal representations or retrievable stores of personal knowledge. In the online setting of VMT, cognition is analyzed by looking closely at the ways in which meaning is built up through the interplay of text postings, graphical constructions and algebraic formulations (Çakir, Zemel, et al. 2009).

There is a tension between the human sciences and the natural sciences, between understanding team cognition (e.g., with micro-analysis of situated case studies) and explaining it (e.g., modeling, confirming general hypotheses, formulating laws and specifying predictive causal relations). Group cognition in online teams involves both humans and computers—both highly situated collaborative interactions and programmed computer support.

In our research, our colleagues and we look at logs of student groups chatting and drawing about mathematics in order to see if they build on each other’s ideas to achieve more than they would individually. How do they understand each other and build shared language and a joint problem focus? What kinds of problems of understanding do they run into and how do they overcome those? How do they accomplish intersubjective meaning making, interpersonal trains of thought, shared understandings of diagrams, joint problem conceptualizations, common references, coordination of problem-solving efforts; planning, deducing, designing, describing; problem solving, explaining, defining, generalizing, representing, remembering and reflecting as a group? What can we say about the general methods that small groups use to learn and think as groups? How can we support and encourage this better with software support for social awareness, social networking, simulations, visualizations, communication; with intelligent software agents; with pedagogical scaffolds and guidance; with training and mentoring; with access to digital resources; with new theories of learning and thinking? To answer these complex questions, we must look carefully at the details of discourse in CSCL groups and develop innovative tools (both analytic and automated) and theories (of cognition by individuals, small groups and discourse communities).

The field of CSCL is particularly interested in the ways small groups can build knowledge together thanks to communication and support from networking technology. We hope that CSCL environments can be designed that make possible and encourage groups to think and learn collaboratively.

Figure 10. The author delivering keynote.

4. Theories of Group Cognition: Foundations for CSCL and CSCW

Abstract      Both computer-supported collaborative learning (CSCL) and computer-supported cooperative work (CSCW) are centrally concerned with teamwork, learning, problem solving, knowledge building, task accomplishment and other cognitive achievements by small groups of people. There are many theories useful for framing the cognitive work that groups undertake in CSCL and CSCW settings, and they may in principle not be reducible to a single theory. However, they seem foundational for both CSCL and CSCW.

Collaboration research explores questions involving numerous distinct—though interacting—phenomena at multiple levels of description. It is important to conceptualize these group activities as involving individual participants in group processes within larger community contexts. The useful approach may be to clearly distinguish levels such as individual, small-group and community units of analysis, and to differentiate terminology for discussing these different levels.

Theory in general has evolved dramatically over the ages, with a trend to extend the unit of cognition beyond the single idea or even the individual mind. Seminal theoretical works influential within collaboration research suggest a post-cognitive approach to group cognition as a complement to—if not a foundation for—analyzing cognition of individuals or of communities-of-practice. While CSCL and CSCW can both build upon shared theories of cognition, they may derive different implications from those theories as relevant for students versus professionals and may converge in some cases as well.

Theory for CSCL and CSCW

T

here is no one theory of collaboration in learning and working. Research in CSCL and CSCW is guided by and contributes to a diverse collection of theories. Even the word theory means different things to different researchers and plays various distinct roles within collaborative-learning work. The reading of the history of theory presented here is itself reflective of one theoretical stance among many held, implicitly or explicitly, by collaboration researchers.

I originally tried to develop my theory of group cognition (Stahl 2006a) in response to issues of CSCL and CSCW software design. In particular, one of my research studies was an attempt to transform a basic CSCW system (BSCW) into a basic CSCL system (BCSCL or Synergia) (Ch. 7), exploring both mutual compatibilities and differences of emphasis between CSCL and CSCW. My other case studies in that book can be categorized as either CSCL (Ch. 1, 2, 6, 12, 21) or CSCW (Ch. 3, 4, 5) systems. These software development attempts—with their various disappointments—led to my attempts to analyze the interaction and cognition taking place (Ch. 8-13). Then—based on a recognition of the inadequacy of available methods and conceptualizations—to investigations of relevant theory (Ch. 14-21), using data from these studies and later from a project specifically about cognition in online teams (Stahl 2009c).

The nature and uses of theory have changed over time and continue to evolve. The theories most relevant to computer-supported collaborative learning and working—in the view developed in this essay—concern the nature of cognition, specifically cognition in collaborating groups.

Through history, the analysis of cognition has broadened, from a focus on single concepts (e.g., Platonic ideas) or isolated responses to stimulae (behaviorism), to a concern with mental models (cognitivism) and representational artifacts (post-cognitivism). Theories that are more recent encompass cognition distributed across people and tools, situated in contexts, spanning small groups, involved in larger activities and across communities of practice. For collaborative-learning and cooperative-work research, theory must take into account interaction in online environments, knowledge building in small groups and cognition at multiple units of analysis.

A Brief History of Theory

Consider the role of theory in a research field like CSCL or CSCW. These fields are multi-disciplinary by their nature and as a result of their origins (see Stahl et al. 2006, for a history of CSCL from a perspective similar to the one here). Consider the name of CSCL, Computer-supported Collaborative Learning: it combines concerns with computer technology, collaborative social interaction and learning or education—very different sorts of scientific domains. CSCL and CSCW grew out of work in fields like informatics and artificial intelligence, cognitive science and social psychology, the learning sciences and organizational management—domains that are themselves each fundamentally multidisciplinary. Theory in these fields may take the form of predictive mathematical laws, like Shannon’s (1949) mathematical theory of information or Turing’s (1937) theory of computation; of models of memory and cognition; or of conceptions of group interaction and social practice. They may have very different implications for research: favoring either laboratory experiments that establish statistical regularities or engaged case studies that contribute to an understanding of situated behaviors.

In the European tradition, theory begins with the ancient Greeks—especially Socrates, Plato and Aristotle—and continues through the 2,500-year-long discourse of philosophy. In recent times, theory has veered into unexpected directions as it has morphed into sciences based more on empirical research than on intellectual reflection. For instance, the work of Freud, Darwin and Marx replaced traditional philosophic assumptions about fixed natures of minds, organisms and societies with much more dynamic views. Theory always transcended the opinions of common sense—so-called folk theories based on the everyday experience of individuals—to synthesize broader views. However, folk theories have also changed over time as they adopt popularized pieces of past philosophies; thus, a trained ear can hear echoes of previous theories in the assumptions of common-sense perspectives, including within current CSCL and CSCW research literature.

After the dogmatic centuries of the medieval period, philosophy took several significant turns: the rationalism of Descartes, the empiricism of Hume, the Copernican revolution of Kant, the dialectical development of Hegel, the social situating of Marx, the existential grounding of Heidegger and the linguistic turn of Wittgenstein. These all eventually led to important influences on theory in CSCL and CSCW.

In particular, for instance, the field of educational research followed this sequence of philosophic perspectives. Empiricism and positivism in philosophy of science culminated in behaviorism in biology and the human sciences. The central metaphor was that of stimulus provoking response, all objectively observable and unambiguously measurable (as critiqued in Chomsky 1959). The major theoretical move of the generation before ours was to assert the necessity of taking into account cognitive processes in studying human behavior, from Chomsky’s (1969) theories of language based on deep grammar and brain mechanisms to the mental models and internal representations modeled by artificial-intelligence programs. Human-computer interaction, the part of computer science dealing with designing for usage, has gone through a similar sequence of behaviorist and cognitivist theories (see Carroll 2003, for numerous examples). More recently, post-cognitive theories have been influential in CSCL and CSCW, as will be discussed later.

The Unit of Analysis

The history of theory can be tracked in terms of the following issue: At what unit of analysis should one study thought (cognition)? For Plato (340 BCE/1941), in addition to the physical objects in the world, there are concepts that characterize those objects; philosophy is the analysis of such concepts, like goodness, truth, beauty or justice. Descartes (1633/1999) argued that if there is thought, then there must be a mind that thinks it, and that philosophy should analyze both the mental objects of the mind and the material objects to which they refer, as well as the epistemological relation between them. Following Descartes, rationalism focused on the logical nature of mental reasoning, while empiricism focused on the analysis of observable physical objects. Kant (1787/1999) re-centered this discussion by arguing that the mechanisms of human understanding provided the source of the apparent spatio-temporal nature of observed objects and that critical theory’s task was to analyze the mind’s constructivist structuring-categorization efforts. Up to this point in the history of theory, cognition was assumed to be an innate function of the individual human mind.

Hegel (1807/1967) transcended that individualist assumption. He traced the logical/historical development of mind from the most primary instinct of a living organism through stages of intentional-consciousness, self-consciousness and historical-consciousness to the most developed trans-national spirit of the times (Zeitgeist). To analyze cognition henceforth, it is necessary to follow through its biological unfolding and go beyond to the ultimate cultural understanding of a society. Figure 1 identifies Hegel’s approach to theory as forming the dividing line—or watershed—between philosophies or theories oriented on the individual and those oriented to a larger unit of analysis.

Figure 1. Adapted from (Stahl, 2006, p. 289, Fig 14-1).

Philosophy after Hegel can be viewed as forming three mainstreams of thought, following the seminal approaches of Marx (critical social theory), Heidegger (existential phenomenology) and Wittgenstein (linguistic analysis). As taken up within CSCL and CSCW, one can trace how these approaches established expanded units of analysis.

Marx (1867) applauded Hegel’s recognition of the historical self-generation of mankind and analyzed this historical process in terms of the dialectical co-development of the social relations of production and the forces of production. His analysis took the form of historical, political and economic studies of the world-historical processes by which human labor produces and reproduces social institutions. Here, the study of the human mind and its understanding of its objects becomes focused at the epochal unit of analysis of social movements, class conflicts and transformations of economic systems.

Heidegger (1927/1996) radicalized the Hegelian dialectic between man and nature by starting the analysis of man from the unified experience of being-in-the-world. The Cartesian problem of a distinction between an observing mind and an objective world was thereby reversed. Heidegger, instead, had to show how the appearance of isolated minds and an external world could arise through abstraction from the primary experience of being-there, human existence inseparable from the worldly objects that one cares for and that define one’s activity. The primordial unit of analysis of cognition is the involvement of people in their world.

Wittgenstein (1953) focused increasingly on language as it is used to accomplish things in the world through interpersonal communication. He rejected his own early view (Wittgenstein 1921/1974), which reduced a rationalist conception of propositional, logical language to a self-contradictory position. Now, linguistic meaning no longer dwelt in the heads of users or the definitions of the words, but in communicational usage. Echoing the lived world of phenomenology, Wittgenstein acknowledged the role of the human form-of-life. He also conceptualized language as the playing of language games, socially established forms of interaction. The unit of analysis shifted from mental meanings to interpersonal communications in the context of getting something done together.

Kant represented the culmination of the philosophy of mind, in which the human mind is seen as the active constructor of reality out of its confrontation with the objects of nature, which are unknowable except through this imposition of human structuring categories. With Kant—over two hundred years ago—the human mind is still a fixed unit consisting of innate abilities of the individual person, despite how much his philosophy differs from naïve realist folk theories, which accept the world as fundamentally identical with its appearance to the human observer. Hegel overthrew the Kantian view of a fixed nature of mind by showing how the mind has itself been constructed through long sequences of processes. The Hegelian construction of mind can be understood in multiple senses: as the biological development of the brain’s abilities as it grows from newborn to mature adult; as the logical development from simple contrast of being and non-being to the proliferation of all the distinctions of the most sophisticated understanding; or as the historical development from primitive homo sapiens to modern, civilized, technological and cultured person.

Marx, Heidegger and Wittgenstein initiated the main forms of post-Kantian, post-Hegelian philosophy and scientific theory (Stahl 2010d). After Hegel, theory shifted from philosophy to science, to explore the biological, logical and historical processes in more detail and to verify them empirically. Followers of Marx, Heidegger and Wittgenstein adopted approaches to this that can be characterized as social, situated and linguistic. They are all constructivist, following Kant’s insight that the structure of known objects is constructed by the knowing mind. However, they all focus on a unit of analysis broader than the isolated individual mind of Descartes.

Seminal Theories for CSCL and CSCW

The social, situated and linguistic theories of Marx, Heidegger and Wittgenstein entered the discourse of CSCL and CSCW literature with researchers coming from the various scientific traditions that went into forming these research domains, including psychology, education, social science, design studies, computer science and artificial intelligence (e.g., Dourish 2001; Ehn 1988; Floyd 1992; Schön 1983). Although these fields each introduced various theoretical perspectives, we can see the major philosophic influences largely through several seminal texts: Mind in Society (Vygotsky 1930/1978), Situated Learning (Lave & Wenger 1991), Lectures on Conversation  (Sacks 1965/1995) and Understanding Computers and Cognition (Winograd & Flores 1986).

Mind in Society is an edited compilation of Vygotsky’s writings from the early 1930s in post-revolutionary Russia, which has been influential in the West since it appeared in English in 1978. Critiquing the prevailing psychology as practiced by behaviorists, Gestalt psychologists and Piaget, Vygotsky did not try to fit psychology superficially into the dogmatic principles of Soviet Marxism, but rather radically rethought the nature of human psychological capabilities from the developmental approach proposed by Hegel and Marx. He showed how human perception, attention, memory, thought, play and learning (the so-called mental faculties) were products of developmental processes—in terms of both maturation of individuals and the social history of cultures. He proposed a dynamic vision of the human mind in society, as opposed to a fixed and isolated function.

The Hegelian term, mediation, was central for Vygotsky, as it is for CSCL and CSCW. Even in his early years still talking about stimulus and response, he asked how one stimulus could mediate the memory of, attention toward or word retrieval about another stimulus (p. iii). In Hegelian terms, this is a matter of mediating (with the first stimulus) the relation (memory, attention, retrieval) of a subject to an object (the second stimulus). This is fundamental to CSCL and CSCW because there the learning of students or the work of professionals is mediated by technological networking as well as by collaborative interaction.

Another popular term from Vygotsky is the zone of proximal development (pp. 84-91). This is the learning distinction and developmental gap between what individuals can do by themselves (e.g., on pre- and post-tests) and what they can do in collaboration (e.g., situated in a small group). A group of children may be able to achieve cognitive results together that they will not be able to achieve as individuals for a couple more years. This is consistent with Vygotsky’s principle that people develop cognitive abilities first in a social context—supported or mediated by peers, mentors or cognitive aids like representational artifacts—and only later are able to exercise these cognitive abilities as individuals. Vygotsky’s theory, if carried beyond where he had time to develop it, implies that collaborative learning—including in workplaces—provides the foundation upon which all learning is built. Methodologically, it argues against judging the outcomes of collaborative learning by evaluating or assessing individuals outside of their collaborative settings.

Situated Learning went beyond Vygotsky in expanding the unit of analysis for learning at work. For Vygotsky and his followers, analysis must include the mediating artifact (tool or word) and the mentor or group. For Lave and Wenger, the unit of analysis is the even larger community-of-practice. Adopting the theoretical and analytical centrality of social practices in Marx, they focused on learning-at-work as the development of processes and relationships within the communities in which individuals participated. Learning-at-work was viewed on the model of apprenticeship, in which an individual gradually—and primarily tacitly—adopts the practices that are established within the community in which the individual is becoming a member. Within CSCL, this approach can be seen in the idea that one learns mathematics by adopting the (predominantly discursive) practices of mathematicians, such as using mathematical symbolisms, making conjectures about mathematical objects and articulating deductive arguments (Sfard 2008). The CSILE project (Scardamalia & Bereiter 1996), a pioneering CSCL effort, tried to support the communicative practices seen in professional research communities within the learning communities of school classrooms; the unit of analysis for knowledge building mediated by the CSILE discussion software was the discourse of the classroom as a whole. This illustrates a kind of CSCL-at-work in reverse, where learning incorporates work practices.

Lectures on Conversation laid the cornerstone of Conversation Analysis (CA), which studies the linguistic practices of communities. It was based on the ethnomethodological (Garfinkel 1967) perspective, grounded in both Wittgenstein’s linguistic analysis and Heidegger’s (1927/1996) and Husserl’s (1936/1989) phenomenological approach. Like Wittgenstein, CA analyzed language at a unit larger than the isolated word or speech act. CA often focuses on adjacency pairs used in conversation—see (Schegloff 2007) for a systematic presentation based on 40 years of research by the CA community on adjacency-pair structure. An adjacency pair is a sequence of two or three utterances that elicit or respond to each other, such as a question and answer. The significance of the adjacency pair as a unit of analysis is that it includes contributions by multiple people involved in an interaction, and thereby avoids treating speech as an expression of an individual mind. This is analogous to Marx’ (1867) focus on the act of commodity exchange between people as a unit of interaction in contrast to theories that dwell on rational decisions of an individual (Stahl, 2010c). What is important in CA is the mode of interaction carried out by the adjacency pair situated in its on-going, sequential discourse context. This should be contrasted with approaches that code isolated utterances based on assumptions about mental models inside the individual mind of the speaker. A CA analysis explicates how a dyad or small group builds upon and solicits each other’s contributions, thus providing insight into patterns of collaboration. In a sense, the CA unit of analysis is not simply the adjacency pair, which includes multiple speakers, but the linguistic community, which establishes the member methods underlying adjacency-pair practices.

Understanding Computers and Cognition presents a Heideggerian critique of the rationalist foundations of artificial intelligence by a leading AI researcher. The book reviews three theories that endorse contextual analysis: Heidegger’s (1927/1996) situated being-in-the-world, Gadamer’s (1960/1988) historically grounded conception of interpretation and Maturana’s (1987) ecological version of cognition. These theories emphasize the inseparability of the mind from its larger context: human being engaged in the world, interpretation oriented within the horizon of history and the organism bound in a structural coupling with its environment. In contrast, AI software represents mental functions as isolatable units of rational computation, which in principle cannot capture the richness and complexity of situated human cognition and collaboration. The larger, primarily tacit (Polanyi 1966) unit of context cannot be adequately represented in a computer system (Stahl 2010e). Accordingly, the role of computer software should be to support human interaction and collaboration, rather than to replace or fully model human cognition.

The writings of Vygotsky, Lave & Wenger and Sacks further develop the perspectives of Marx, Heidegger and Wittgenstein that view cognition as social, situated and linguistic. Winograd—like other CSCW researchers, including Ehn and Dourish—reviews the foundational post-cognitive theories and considers the implications for computer-supported collaboration. These theories can be—and have been—taken in different directions by researchers when it comes time to follow their implications for research conceptualizations and methods. These directions can perhaps best be seen in terms of alternative theories of individual, small-group and community cognition in collaboration research.

Theories of Individual Cognition

Many research questions within CSCL and CSCW involve individual cognition. Collaboration research is often treated as a sub-discipline of educational or social-psychological research, oriented to the mind of the individual student or worker, within group contexts. Such research can follow traditional scientific research paradigms based on pre-Kantian empiricism (Hume) and/or rationalism (Locke). This research often adopts a constructivist approach, based on the Kantian principle that the student or worker constructs his or her own understanding of reality. Such constructivist theory is cognitivist, in that it involves assumptions about cognitive processes in the mind of the individual underlying the individual’s observed behaviors. For instance, a student’s responses in a test situation are assumed to be reflective of the student’s mental models of some knowledge content, as construed by the student.

Work within CSCL or CSCW certainly acknowledges the importance of the larger social, historical and cultural context. However, it often treats this context as a set of environmental variables that may influence the outcomes of individual student or worker cognition, but are analytically separable from that cognition. In this way, cognition is still treated as a function of an individual mind. This approach may be called socio-cognitive. It acknowledges social influences, but tries to isolate the individual mind as a cognitive unit of analysis by controlling for these external influences.

Followers of Vygotsky, by contrast, are considered socio-cultural. They recognize that cognition is mediated by cultural factors. Yet, they still generally focus on the individual as the unit of analysis. They investigate how individual cognition is affected by cultural mediations, such as representational artifacts or even by collaborative interactions. Vygotsky himself—who was after all a psychologist—generally discussed the individual subject. For instance, his concept of the zone of proximal development measured an individual’s ability when working in a group, not the group’s ability as such. Vygotsky was trying to demonstrate that individual cognition was derivative of social or intersubjective experiences of the individual, and so his focus was on the individual rather than explicitly on the social or intersubjective processes in which the individual was involved.

In this sense, much cognitive research investigates individual cognition in settings of collaboration. In fact, if the research is based on testing of the individual before and after a collaborative interaction and does not actually analyze the intervening (mediating) interaction itself, then it is purely an analysis at the individual unit of analysis, where the collaboration is merely an external intervention measured by presumably independent variables.

If one looks closely at most studies that claim to be about small-group collaboration, one finds that they adopt this kind of methodical focus on the individual within a group setting and treat the group interaction as an external influence on the individual. This is particularly clear in the writings of cooperative learning that preceded CSCL (e.g., Johnson & Johnson 1989). As defined within CSCL (Dillenbourg 1999), in “cooperative” learning students divide up group work and then put the individual contributions together, whereas in “collaborative” learning a group of students does the work together. Similarly on the methodological level, in cooperative learning the analyst distinguishes the contributions to the work and focuses on the learning by the individuals as a result of the cooperative experience, whereas in collaborative learning the analyst may chose to focus on the group processes. The same is true for small-group studies of sociology and social psychology in CSCW: they usually treat the group as a context and analyze the effects on the individual, rather than analyzing the group phenomena and treating the individuals as contributors to the group processes.

A final example of a theory of individual cognition is psycho-linguistic contribution theory (Clark & Brennan 1991). This particular paper is often cited in CSCL and CSCW literature. Although the paper claims to be in the Conversation Analysis tradition, it translates the adjacency-pair structure of grounding shared understanding into the contributions of the individuals. It analyzes the individual contributions as expressions of their mental representations or personal beliefs and treats the resultant shared understanding as a matter of similar mental contents or acceptance of pre-conceived beliefs rather than as a negotiated group product of collaboratively co-constructed meaning making. In a later paper, Clark (1996) tries to unite cognitivism with Conversation Analysis, but he analyzes the situated, engaged interaction as an exchange of signals between rationally calculating minds, who identify deliberate actions based on “knowledge, beliefs and suppositions they believe they share” (p. 12)—i.e., mental constructs of individuals. Interestingly, Clark (1996) concludes in favor of recognizing two independent theories with different units of analysis (the individual or the community, but ironically not the small group): “The study of language use must be both a cognitive and a social science,” he says (p.25).

Theories of Community Cognition

In striking contrast to the insistent focus on the individual as the unit of analysis is the social-science perspective on social processes. Marx provides a good example of this. Where economists of his day analyzed economic phenomena in terms of rational choices of individual producers and consumers, Marx critiqued the ideology of individualism and in its place analyzed sweeping societal transformations such as urbanization, the formation of the proletariat, the rise of the factory system, and the drive of technological innovation. Lave and Wenger (1991) brought this approach to educational theory, showing for instance how an apprenticeship training system reproduces itself as novices are transformed into experts, mentors and masters. Learning is seen as situated or embedded in this process of the production-and-reproduction of structures of socially defined knowledge and power. For Lave and Wenger, the community or community-of-practice is the structure within social organizations (corporations, cultural institutions, etc.) where interaction, task accomplishment, professional exchanges, training and institutional learning take place; it is a prime location for CSCL-at-work.

The theoretical importance of the situation in which learning and work take place is widely acknowledged in CSCL and CSCW. Suchman (1987) demonstrated its centrality for human-computer interaction from an anthropological perspective heavily influenced by both Heidegger—via Dreyfus, (1991)—and Garfinkel (1967), leading to conclusions similar to Winograd’s (Winograd & Flores 1986). Suchman (1987) and Nardi (1996) have helped to establish ethnographic methods—oriented to community phenomena—as relevant to CSCL and CSCW research. Unfortunately, even perspectives like situated cognition can take a reductive turn: Recent commentaries on situated cognition (Robbins & Aydede 2009) and distributed cognition (Adams & Aizawa 2008) frame the issues at the individual level, to the extreme of reducing all cognitive phenomena to neural functions.

Building on Vygotsky and his Russian colleagues, Activity Theory (Engeström 1987; Engeström 1999; Kaptelinin & Nardi 2006) insists on taking an entire activity system as the unit of analysis. In his triangular analysis rubric, Engeström extends Vygotsky’s mediation triple of subject, mediator and object to include mediating dimensions from Marx’s theory: the division of labor, the rules of social relations and the community of productive forces. Like discourse analysis (Gee 1992), activity theory is repeatedly looking at small-group interactions but only seeing the larger, societal issues. For instance, when activity theory addresses the study of teams in the most detail in Chapter 6 of (Engeström 2008), it is mostly concerned with the group’s situation in the larger industrial and historic context; rather than with analyzing how the group interactionally builds knowledge, it paraphrases how the group deals politically with organizational management issues.

There is something of this avoidance of the small group as the scientific focus in other theories popular in CSCL and CSCW as well, for instance even in distributed cognition. In seminal statements of post-cognitivist theory, Hutchins has indeed explicitly pointed to group-cognitive phenomena:

- “Cognitive processes may be distributed across the members of a social group” (Hollan et al. 2000, p. 176).

- “The cognitive properties of groups are produced by interaction between structures internal to individuals and structures external to individuals” (Hutchins 1996, p. 262).

- “The group performing the cognitive task may have cognitive properties that differ from the cognitive properties of any individual” (Hutchins 1996, p. 176).

However, rather than focusing on these group phenomena in detail, he prefers to analyze socio-technical systems and the cognitive role of highly developed artifacts (e.g., airplane cockpits or ship navigation tools). Certainly, these artifacts have encapsulated past cultural knowledge (community cognition), and Hutchins’ discussions of this are insightful. But in focusing on what is really the community level—characteristically for a cultural anthropologist—he does not generally analyze the cognitive meaning making of the group itself (but see his analysis of group or organizational learning in Chapter 8 of Hutchins, 1996, for an exception and an exemplary analysis of CSCL-at-Work).

Even ethnomethodology (Garfinkel 1967; 2006) and conversation analysis (Sacks 1965/1995; Sacks, Schegloff & Jefferson 1974; Schegloff 2007) consider themselves social sciences, versions of sociology or communication studies, but not sciences of the small-group unit of analysis. They aim to analyze social practices, defined across a whole society or linguistic community. This may be a quibble over words, for they do in fact define many important processes at the group unit, although they call them social.

Vygotsky, too, used the term social in an ambiguous way when he said that learning takes place socially first and then later individually. Socially can refer to two people talking as well as to transformations of whole societies. For the sake of distinguishing levels of description or units of analysis in CSCL and CSCW, it seems important to make clear distinctions. Table 1 suggests sets of different terms for referring to phenomena at the individual, small-group and societal levels. The distinction of these three levels has previously been argued for by (Rogoff 1995), (Dillenbourg, Baker, Blaye & O'Malley 1996), (Stahl 2006a) and others. We start with these three levels, which seem particularly central to much CSCL and CSCW work, although other levels might also usefully be distinguished, such as “collective intelligence” at the classroom/shop-floor level or “collective practices” at the school/company level (Guribye 2005; Jones, Dirckinck-Holmfeld & Lindström 2006; Looi, So, Toh & Chen 2011b). Perhaps consistent usage of such terminological distinctions would lend clarity to the discussion of theories in CSCL and CSCW.

Table 1. Terminology for phenomena at the individual, small-group and community levels of description. From (Stahl 2010b, p. 27, Table 2.1 ).

Level of description

Individual

Small group

Community

Role

Person / student

Group participant

Community member

Adjective

Personal

Collaborative

Social

Object of analysis

Mind

Discourse

Culture

Unit of analysis

Mental representation

Utterance response pair

Socio-technical activity system, mediating artifacts

Form of knowledge

Subjective

Intersubjective

Cultural

Form of meaning

Interpretation

Shared understanding, joint meaning making, common ground

Domain vocabulary, artifacts, institutions, norms, rules

Learning activity

Learn

Build knowledge

Science

Ways to accomplish cognitive tasks

Skill, behavior

Discourse, group methods, long sequences

Member methods, social practices

Communication

Thought

Interaction

Membership

Mode of construction

Constructed

Co-constructed

Socially constructed

Context of cognitive task

Personal problem

Joint problem space

Problem domain

Context of activity

Environment

Shared space

Society

Mode of Presence

Embodiment

Co-presence

Contemporary

Referential system

Associations

Indexical field

Cultural world

Form of existence (Heidegger)

Being-there (Dasein)

Being-with (Mitsein), Being-there-together at the shared object

Participation in communities of practice (Volk)

Temporal structure

Subjective experiential internal time

Co-constructed shared temporality

Measurable objective time

Theory of cognition

Constructivist

Post-cognitive

Socio-cultural

Science

Cognitive and educational psychology

Group cognition theory

Sociology, anthropology, linguistics

Tacit knowledge

Background knowledge

Common ground

Culture

Thought

Cognition

Group cognition

Practices

Action

Action

Inter-Action

Social praxis

Theories of Small-Group Cognition

It seems clear that the small-group level should be considered particularly central to CSCL and CSCW theory, because these fields are explicitly concerned with supporting collaboration, knowledge co-construction or group cognition. There are few other domains in which such activities by small groups are in principle such a central concern. We have seen resistance to this focus on the small group, for instance, in the case of activity theory—which could profitably be used to investigate group processes—where Engeström (2008) argued against a focus on small groups because workplace teams tend to come and go quickly, forming changing knots of co-workers around ephemeral tasks.

Engeström’s argument echoes the attitude of Schmidt & Bannon (1992) in their programmatic opening article of the inaugural issue of the CSCW journal. In rejecting the use of the term “group” as a defining concept for CSCW, they reduced the theoretical perspective to one focused on individuals “articulating” (i.e., coordinating) their “distributed individual activities” (p. 15). They made this move despite claiming that their concept of “cooperative work” was congruent with Marx’ (1867) definition of cooperative work as “multiple individuals working together in a conscious way in the same production process.” In 1867, Marx was analyzing in detail the historic shift of the unit of production from the individual to the group, but Schmidt & Bannon insist on still focusing on the individual in 1992. They complain that the units of cooperative workers are not well-formed, clearly defined, persisting groups. But that is beside the point.

The theoretical point is that interacting people accomplish work tasks and associated cognitive tasks (including articulation tasks and power struggles) through group-interaction processes, and that these should be analyzed as such, as local achievements of group interaction, not simply as sums of individual actions and reactions or as effects of external societal forces. In particular, as cooperative work shifts from the manual factory production of Marx’s time to knowledge building and other forms of intellectual production in the information age, group-cognition phenomena call more strongly for analysis at the small-group unit. A small group is not defined ontologically as a certain number of human bodies adjacent to each other for a certain period of clock time, but as a cognitive unit capable of achieving specific tasks of cooperative work and collaborative knowledge building through the interaction among individuals within a larger community context.

There are distinct phenomena and processes at the individual, small-group and community-of-practice levels, and analyses at these different levels of description can reveal different insights. As Grudin (1994) put it in terms of the needs of CSCW,

Computer support has focused on organizations and individuals. Groups are different. Repeated, expensive groupware failures result from not meeting the challenges in design and evaluation that arise from these differences. (p. 93)

There are theoretical, methodological and practical reasons for both CSCL and CSCW to focus on the small-group unit of analysis.

If group phenomena are treated seriously as first-class objects of theory, then one can study how small groups engage in cognitive activities such as: interpersonal trains of thought, shared understandings of diagrams, joint problem conceptualizations, common references, coordination of problem-solving efforts, planning, deducing, designing, describing, problem solving, explaining, defining, generalizing, representing, remembering and reflecting as a group.

In CSCL and CSCW studies of text chat or discussion forums, for instance, analysis can show group-cognitive accomplishments emerging from the network of meaningful references built up by postings, demonstrating how the group’s self-formation and its cognitive accomplishments are enacted in situated interaction. An analytic focus on the group unit of analysis need not imply that groups exist as ontological entities whenever people are observed in proximity or in communication with one another.

The small group is a theoretical construct, not a simple physical observable. Of course, effective groups have to constitute themselves as such and they can change dramatically over time. It is not the more-or-less persistent physical group that is important, but the group processes, which may extend over seconds, days or years. A single momentary exchange of greetings may be a group process of interest, as shown by the early conversation analyses of telephone answering on a help phone line (Hopper, 1992).

A theoretical approach that focuses on small-group interaction is that of dialogicality (Linell 2001; 2009; Mercer 2000; Wegerif 2007). Dialogical theory goes back to Bakhtin (1986), a contemporary of Vygotsky. It stresses the linguistic nature of interaction. It also reiterates the idea that a person’s identity as an individual arises through the confrontation with one’s partners in dialogue—a view that goes back beyond Mead (1934/1962) to Hegel’s (1807/1967) master-slave dialectic (Stahl, 2006, p. 333f). The notion of dialogue partners coming from different perspectives and negotiating from these is an important contribution of dialogic inquiry (Wells 1999). Another key concept is that of a shared dynamic dialogic space, within which knowledge building can take place (Kershner, Mercer, Warwick & Staarman 2010). This is similar to the joint problem space of (Teasley & Roschelle 1993), but now developed in an unambiguously post-cognitive manner.

The idea of an interactional space for interaction within a small group is central to group-cognition theory (Stahl 2006a) as well. The term group cognition was coined to stress the goal of developing a post-cognitive view of cognition as the possible achievement of a small group collaborating so tightly that the process of building knowledge in the group discourse cannot be attributed to any individual or even reduced to a sequence of contributions from individual minds. For instance, the knowledge might emerge through the interaction of linguistic elements, situated within a sequentially unfolding set of constraints. These constraints would be defined by the group task, the membership of the group, and other local or cultural influences, as well as due to the mediation of representational artifacts and media used by the group—within a larger “horizon” of language and history (Gadamer, 1960; Husserl, 1936).

The theory of group cognition absorbs many ideas from the theories discussed above, including that of a shared dynamic dialogical space. Despite some scattered case studies by the authors already mentioned and their colleagues, there is yet not much documentation and analysis of empirical instances of effective group cognition. The analysis of group cognition needs not only specially focused methods to track its occurrence, but even prior to that it needs appropriate collaboration technologies, group methods, pedagogy and guidance to structure and support groups to effectively build knowledge that can be shown to be a group product not reducible to individual mental representations.

The Virtual Math Teams Project (VMT) was launched to generate a data corpus that would allow for the analysis of group cognition. This project and some analyses by a number of researchers are documented in (Stahl 2009c). Group-cognition theory focuses on the sequential team interaction within case studies of small-group collaboration. This takes place within an interaction space or a world in the Heideggerian (1927) sense, which opens up to allow the production of group-cognitive accomplishments. The interaction that takes place within such a world—whether face-to-face or online—is subject to a variety of constraints, as pictured in Figure 2.

Figure 2. A diagram of constraints on sequential small-group interaction. From (Stahl 2010c, p. 256, Figure 1).

Note that Figure 2 is not intended to be a model of objects and processes. Rather it tries to present some of the complex constraints on the discourse through which group cognition might be achieved. Neither the physical individuals nor their group are represented here as such; the dialogical voices (Bakhtin, 1986) of the individuals enter into the sequential team interaction and respond to it. Over time, the sequential team interaction forms the central shared dynamic dialogic space within which the group-cognitive constraints interact. Behind the individual voices that enter into this interaction space are not so much minds containing mental representations, as a fluid background of past experiences and developed resources for action, which surface based on relevance to the interaction. The team discourse is situated in the shared dialogical context generated by the on-going interaction itself; the culture and history associated with the group’s community-of-practice; and the socio-technical environment including the media of communication. The interaction is goal-oriented toward the task—as given externally but as enacted by the group—and mediated by a variety of kinds of artifacts, including codifications of knowledge products previously generated by the group. These artifacts might end up among the team outcomes, in relation to the guiding task. Of course, other constraints and influences are possible as well, coming for instance from the guidance of a teacher or the motivations of a reward system. The point is that one can picture the whole system producing cognitive accomplishments without having to postulate mental representations in individual minds, let alone to reduce the whole system either to rational mental decisions or to regulation by rules of social institutions.

The term constraint in Figure 2 is chosen to be a neutral term, not implicating a notion of mechanistic causality. While it is clear that the traditional conception of causality is inadequate—stemming back to Aristotle and metaphors of physical mechanics from the everyday world—it is less obvious how to think about the working of the constraints upon group cognition. Folk theory adopts a mechanistic worldview, or even an anthropomorphic view of nature combined with a mechanistic view of causality. Observable behavior of people is taken to be the result of rational decision making in the heads of individuals causing the people to behave as a result of the minds acting as the agency for causing words to be produced and limbs to be moved. But the linguistic turn of Wittgenstein (Wittgenstein 1953) and even more so the recent practice turn (Schatzki, Knorr Cetina & Savigny 2001) have veered radically away from such a view.

Latour (1992) seems to be working toward a post-cognitive notion of causality, perhaps relying heavily on Hegel’s notion of mediation. Interestingly, he not only argues against the hegemony of individual minds as agents in the social world, but he also argues against the adequacy of our notion of the social (Latour 2007). History is made neither by rational decisions of individual minds nor by the workings of society. Rather, it is the result of a complex network of mediating actors, including all kinds of artifacts as well as human actors. Thus, Latour seems to be advocating an analytic approach that steers clear of both individual minds and social institutions to focus on a middle ground. He selects the term ‘group’ for this middle ground, precisely because it can be used without implying theoretical preconceptions: “The word ‘group’ is so empty that it sets neither the size nor the content…. This is exactly why I have chosen it” (Latour 2007, p. 29). Figure 2 may illustrate the structure of the kind of network that he would endorse for picking apart and then reassembling instances of group cognition.

Such new conceptualizations of cognition, agency and causality may be particularly appropriate for collaborative learning and cooperative work, especially as they are brought together in CSCL and CSCW. Here, the focus is on interpersonal communication and work practices of groups. CSCL and CSCW should adopt these new perspectives for facilitating computer-mediated discourse focused on improving small-group practices.

A Multiplicity of Theories of Cognition

In general, CSCL and CSCW raise many fundamental questions for traditional theories, oriented as they are to small groups and to online interaction. The accustomed characteristics of the physical world, in which colleagues and interlocutors are embodied and visible to each other, are often missing in these virtual settings, and that brings into question numerous assumptions of folk theories and traditional approaches.

The group itself has no identity as a physical body and has no brain to possess its knowledge; it relies on external memories, which differ essentially from personal memories (Donald 1991). The online world—shared dialogical space—has no location or extension. Group members can come from around the world and do not necessarily share local connections and culture. CSCL and CSCW involve students and workers in qualitatively different social relations of production, modes of being in the world or forms of life; even Marx, Heidegger and Wittgenstein’s foundational philosophies of post-cognitive theory need to be rethought for virtual groups. Concepts of causality, world, knowledge, cognition, intersubjectivity, interaction, temporality and presence (being) need to be re-conceptualized for theories of collaboration.

There are many avenues for developing theories of CSCL and CSCW, as reviewed in this essay. Although there are some similarities among these alternatives—often in terms of their critiques of earlier theories—there are strong differences of position and perspective. This is not necessarily a problem. There is a huge assortment of processes taking place in successful collaborative events: at multiple time scales and involving different aspects of interaction. It is possible to raise innumerable research questions, each requiring possibly different methods of investigation at various levels of analysis. It is likely that CSCL and CSCW require multiple theories, which are not reducible to one grand unifying theory and that even seem incommensurate with each other. This goes essentially beyond the common notion of mixed methods, in which two or more methods of analysis are used to triangulate a single phenomenon from different angles. There are distinct phenomena at different levels of description—and they interact with each other in complex ways in group settings.

CSCL and CSCW study collaboration, from a design perspective. CSCL often involves whole classrooms or schools and widespread educational practices; CSCW often involves large departments or factories and widespread work practices. At the opposite end of the spectrum, much of the actual work comes down to tasks done by individuals. But much of the coordination, decision making, articulation, brainstorming, discovery and knowledge building is accomplished by small groups. Community accomplishments are thereby mediated by small groups, which carry out the necessary activities and involve the individuals. Collaboration involves a tight and complex integration of processes at the individual, small-group and community levels. Computer support for collaboration must provide supports at each level while also supporting the integration of the activities at all levels. To provide insight for this, CSCL and CSCW research must recognize the levels as distinct and conduct analyses at all levels.

Some time ago when I was a CSCL researcher working in a CSCW research lab, I argued that CSCL and CSCW were closely related in terms of their theoretical foundations, but that they differed in terms of the population on which they focused (Stahl 2002a). CSCL is concerned with students who are learning practices at which they are still novices. In the Vygotskian metaphor, they are experiencing group-cognitive processes that they can subsequently internalize. CSCW is concerned with professionals who are refining their skills as experts. They may be engaged in group-cognitive processes which cannot be internalized by an individual, such as navigating a large naval vessel (Hutchins 1996). This essay has argued that even at the level of concrete learning activities the two often separated research fields have much in common.

In CSCL and CSCW, there are many phenomena of interest, and they are largely defined by the theories that conceptualize them. So different theories can be talking about quite different phenomena (although they may unfortunately be calling them by the same name). In order to avoid confusion and arguments about pseudo-problems, we need to be clear about the theories behind research questions, assumptions, methodologies, analysis tools, findings and claims in this research.

This essay has sketched some of the theoretical landscape underlying CSCL and CSCW research. Progress in further developing theories of collaborative learning and working will require careful analysis of case studies of group cognition and experimental results guided by theoretical perspectives that are clearly enunciated.

Part C: Analyzing Group Cognition

5. How a Virtual Math Team Structured its Problem Solving

To develop a science of small-group interaction in groupware, we need a method for analyzing the structure of computer-mediated discourse. Conversation analysis offers an analysis of conversational talk in terms of a fine structure of adjacency pairs and offers some suggestions about longer sequences built on these pairs. This essay presents a case study of students solving a math problem in an online chat environment. It shows that their problem-solving discourse consists of a sequence of exchanges, each built on a base adjacency pair and each contributing a move in the solution process.

Small-Group Information Use

I

nformation, people and technology converge in a practical way in online collaborative problem solving. My colleagues and I have been pursuing a research agenda aimed at investigating how to support online collaborative problem solving. We have focused on the domain of school mathematics—especially beginning algebra and geometry—where students learn formal techniques and tacit practices of solving abstract problems. We find that mechanisms of group problem solving are visible in this context.

Our research—such as that reported in this essay—confirms that there are distinctive processes of information use in problem solving at the small-group unit of analysis. These processes should not be reduced to either the individual psychological level or the larger social community level—despite the fact that groups are physically composed of individuals and that they are embedded in socio-historical contexts. While an approach methodologically focused on the group unit of analysis is in line with current post-cognitive theories it is rarely carried out consistently.

We developed the Virtual Math Teams (VMT) environment and invited students to work in online groups for up to four hour-long sessions. We presented challenging problems for them to explore together and encouraged them to pursue their own questions. The environment was instrumented to capture a complete and accurate record at the group unit of analysis—i.e., all text-chat postings, all drawing actions and all social awareness messages that were displayed to the group. As researchers, we can replay the group interaction and view it as it appeared to the group or browse it in as much detail as needed for analysis.

Because we are pursuing design-based research to improve the VMT environment, we are not oriented toward theoretical hypotheses, statistical generalizations, individual mental representations or socio-cultural influences—except to the extent that they manifest themselves in the group interaction. Rather, we try to understand the situated processes that take place at the group level of description in actual case studies. In particular, we look at the ways in which groups of math students use information and solve problems in our environment so that we can design improved socio-technical supports for their collaborative online problem solving.

We have tried a variety of research approaches in the VMT Project, including coding, statistical comparison, modeling, uptake analysis, conversation analysis, critical ethnography and discourse analysis. In general, we have found the most insightful approach to involve adapting ethnomethodologically inspired conversation analysis (CA) to our context of online text chat by math students.

Structuring Group Cognition at Multiple Levels

A year ago in my opening keynote talk (Stahl 2009a) at the International Conference of Computers in Education (ICCE 2009) in Hong Kong, I claimed that the discourse of group cognition (Stahl 2006a) has a hierarchical structure, typically including the following levels, as illustrated with a particular case study from the VMT Project (Stahl 2009c):

- Group event: E.g., Team B’s participation in the VMT Spring Fest 2006.

- Temporal session: Session 4 of Team B on the afternoon of May 18, 2006.

- Conversational topic: Determining the number of sticks in a diamond pattern (lines 1734 to 1833 of the chat log of Session 4).

- Discourse move: A stage in the sequence of moves to accomplish discussing the conversational topic (e.g., lines 1767-1770—see Logs 1-10 below).

- Adjacency pair: The base interaction involving two or three utterances, which drives a discourse move (lines 1767 and 1769).

- Textual utterance: A text chat posting by an individual participant, which may contribute to an adjacency pair (line 1767).

- Indexical reference: An element of a textual utterance that points to a relevant resource. In VMT, actions and objects in the shared whiteboard are often referenced in the chat. Mathematical content and other resources from the joint problem space and from shared past experience are also brought into the discourse by explicit or implicit reference in a chat posting.

The multi-layered structure corresponds to the multiplicity of constraints imposed on small-group discourse—from the character of the life-world and of culture (which mediate macro-structure) to the semantic, syntactic and pragmatic rules of language (which govern the fine structure of utterances). A theory of group cognition must concern itself primarily with the analysis of mid-level phenomena—such as how small groups accomplish collaborative problem solving and other conversational topics.

The study of mid-level group-cognition phenomena is a realm of analysis that is currently underdeveloped in the research literature. For instance, many CSCL studies focus on coding individual (micro-level) utterances or assessing learning outcomes (macro-level), without analyzing the group processes (mid-level). Similarly, Conversation Analysis (CA) centers on micro-level adjacency pairs while socio-cultural Discourse Analysis is concerned with macro-level identity and power, without characterizing the interaction patterns that build such macro phenomena out of micro-elements. Understanding these mid-level phenomena is crucial to analyzing collaborative learning, for it is this level that largely mediates between the interpretations of individuals and the socio-cultural factors of communities.

The analysis in this essay illustrates the applicability of the notion of a “longer sequence” as vaguely suggested by both Sacks (1965/1995, II p. 354) and Schegloff (2007). A longer sequence consists of a coherent series of shorter sequences built on adjacency pairs. This multi-layered sequential structure will be adapted in this essay from the informal face-to-face talk-in-interaction of CA to the essentially different, but analogous, context of groupware-supported communication and group cognition, such as the text chat of VMT. I will show how a small group of students collaborating online constructed a coherent longer sequence, through which they solved the problem that they had posed for themselves. Methodologically, it is important to note that the definition of the longer sequence—like that of the other levels of structure listed above—is oriented to by the discourse of the students and is not simply a construct of the researcher.

An Analytic Method

Recently, I have been trying to apply our CA approach in a systematic way to the analysis of VMT chat logs. Schegloff’s (2007) book on Sequence Organization in Interaction represents the culmination of decades of CA analysis. As indicated by its subtitle, it provides a useful primer in CA. My goal here is to extend the CA approach based on short sequences of utterances to analyze the larger scale interactions of group problem solving in VMT.

Schegloff’s (2007) presentation highlights the central role of the adjacency pair as the primary unit of sequence construction according to CA. An adjacency pair is composed of two turns by two different people, with an interactional order, such as a question followed by an answer to the question. The simple two-turn pair can be extended with secondary adjacency pairs that precede, are inserted between or follow up on the base pair, potentially recursively. This yields “extensive stretches of talk which nonetheless must be understood as built on the armature of a single adjacency pair, and therefore needing to be understood as extensions of it” (p. 12).

These “extensive stretches of talk” are still focused on a single interaction of meaning making, and not a larger cognitive achievement like problem solving, involving multiple steps. However, both Sacks and Schegloff provide only vague suggestions about the analysis of longer sequences. These suggestions have not been extensively developed within CA. This essay is an attempt to explore them in an online text-chat context.

Schegloff (2007) briefly takes up “larger sequence structures to which adjacency pairs can give rise and of which they may be building blocks … such as sequences of sequences” (p. 12). One way in which a sequence (an extended adjacency pair) may be related to yet separate from a previous, completed adjacency pair “is that it implements a next step or stage in a course of action, for which the just-closed sequence implemented a prior stage” (p. 213). Note the two-way reference, with the second stage having the character of a next, but also the first stage having the character of a prior. This is analogous to the two parts of a simple adjacency pair according to Schegloff:

Adjacency pair organization has (in addition to the backwards import just described) a powerful prospective operation. A first pair part projects a prospective relevance, and not only a retrospective understanding. It makes relevant a limited set of possible second pair parts, and thereby sets some of the terms by which a next turn will be understood—as, for example, being responsive to the constraints of the first pair part or not. (p. 16)

The adjacency pair structure was first discussed extensively by Sacks (1965/1995, II 521-569). In these seminal lectures, he also briefly discussed long sequences. Here, his main point was to state that little is known about the structure of long sequences; that the analytic problem is in principle harder; and that, in particular, it is wrong to assume that an analysis at the level of adjacency pairs will be useful to understanding the co-construction of long sequences:

It turns out that one central problem in building big packages is that the ways the utterances that turn out to compose the package get dealt with as single utterances or pairs of utterances or triplets of utterances, etc., may have almost no bearing on how they’re to be dealt with when an attempt is made to build the larger package. (II p. 354)

The analyses provided by CA come primarily from the study of American adults conducting face-to-face, verbal, informal, social conversation, although some of the early data came from distance conversations by telephone and the field has broadened its sources considerably more recently. However, we must be careful when applying CA methods to online, text-based, learning-related discourse about mathematics by students. Along these lines, Schegloff (2007) warns about his presentation:

Note that this discussion is focused on conversation in particular. Because different organizations of turn-taking can characterize different speech-exchange systems (Sacks et al. 1974, n. 11 729-731), anything that is grounded in turn-taking organization may vary with differences in the turn-taking organization. It is a matter for empirical inquiry, therefore, how the matters taken up in the text are appropriately described in non-conversational settings. (p. 15n)

As we have frequently argued (e.g., Stahl 2006a; 2009c; Stahl et al. 2006), we believe that adapting CA to computer-mediated communication offers the best prospects for analysis of interaction in socio-technical environments like VMT. The preceding review of the topics of adjacency pairs and long sequences indicates that it is an empirical question how well this proposed adaptation might work. We designed and conducted the VMT Project from 2003 to the present in order to produce a corpus of data that could be analyzed in as much detail as needed to determine the structure of group cognition, that is, of collaborative knowledge building through interaction at the group unit of analysis.

In looking at the VMT data corpus, the VMT research team has clearly seen the differences between online text chat and verbal conversation. The system of turn taking so important in CA (Sacks et al. 1974) does not apply in chat. Instead, chat participants engage in reading’s work (Zemel & Çakir 2009), in which “readers connect objects through reading’s work to create a ‘thread of meaning’ from the various postings available for inspection” (p. 274f). The first and second parts of an adjacency pair may no longer be literally temporally adjacent to each other, but they still occur as mutually relevant, anticipatory, and responsive. The task of reading’s work—for both participants and analysts—is to reconstruct the threading of the underlying adjacency-pair response structure (Stahl 2009b).

In CA, adjacency pairs are related to both issues of timing (turn taking) and of sequentiality (response). In chat, they retain their importance solely as sequential, in order to maintain interaction in the absence of turn taking. We have tried to explore the larger sequential structure of problem-solving chat by using the CA notion of openings and closings (Schegloff & Sacks 1973). VMT researchers looked at several math chats from 2004, which used a simple chat tool from AOL. We coded and statistically analyzed the fine-structure threading of adjacency pairs (Çakir, Xhafa, et al. 2009). In addition, we defined long sequences based on when opening and closing adjacency pairs achieved changes in topic (Zemel et al. 2009). These long sequences were graphed to show their roles in constituting the chat sessions, but their internal sequential structures were not investigated.

My colleagues and I have subsequently conducted numerous case studies from the VMT corpus. We have been particularly drawn to the records of Team B and Team C in the VMT Spring Fest 2006. These were particularly rich sessions of online mathematical knowledge building because these teams of students met for over four hours together and engaged in rich explorations of interesting mathematical phenomena. However, partially because of the richness of the interactions, it was often hard for analysts to determine a clear structure to the student interactions. Despite access to everything that the students knew about each other (team members were spread across the US) and about the group interaction, it proved hard to unambiguously specify the group-cognition processes at work (Medina, Suthers & Vatrapu 2009; Stahl 2009b; Stahl et al. 2009).

Therefore, in the following case study, I have selected a segment of Team B’s final session, in which the structure of the interaction seems to be clearer. The interaction is simpler than in earlier segments partially because two of the four people in the chat room leave. Thus, the response structure is more direct and less interrupted. In addition, the students have already been together for over four hours, so they know how to interact in the software environment and with each other. Furthermore, they set themselves a straightforward and well-understood mathematical task. The analysis of this relatively simple segment of VMT interaction can then provide a model for subsequently looking at the more complex data and seeing if it may follow a similar pattern.

The Case Study

Three anonymous students (Aznx, Bwang, Quicksilver) from US high schools met online as Team B of the VMT Spring Fest 2006 contest to compete to be “the most collaborative virtual math team.” They met for four hour-long sessions during a two-week period in May 2006. A facilitator was present in the chat room to help with technical issues, but not to instruct in mathematics.

Figure 1. Screenshot of the VMT environment showing the pattern of horizontal and vertical sticks in the stair-step figure.

In their first session, they solved a given problem, finding a mathematical formula for the growth pattern of the number of squares and the number of sticks making up a stair-step figure. They determined the number of sticks by drawing just the horizontal sticks together and then just the vertical ones (see Figure 1). They noticed that both the horizontals and the verticals formed the same pattern of 1 + 2 + 3 + … + n + n sticks at the nth stage of the growth pattern. They then applied the well-known Gaussian formula for the sum of consecutive integers, added the extra n, and multiplied by 2 to account for both the horizontal and vertical sets of sticks.

In the second session, they explored problems that they came up with themselves, related to the stair-step problem, including 3-D pyramids. Here they ran into problems drawing and analyzing 3-D structures. However, they managed to approach the problem from a number of perspectives, including decomposing the structure into horizontal and vertical sticks.

In the third session, they were attracted to a diamond-shaped variation of the stair-step figure, as explored by Team C in the Spring Fest. They tried to understand how the other team had derived its solution. They counted the number of squares by simplifying the problem through filling in the four corners surrounding the diamond to make a large square; the corners turned out to follow the stair-step pattern from their original problem.

In the fourth session, they discovered that the other team’s formula for the number of sticks was wrong. In the following, we join them an hour and 17 minutes into the fourth session, when one of the students as well as the facilitator had to leave.

Problem-Solving Moves

In this section of the essay, the interaction is analyzed as a sequence of moves in the problem-solving interaction between Bwang and Aznx, the two remaining students. Each move is seen to include a base adjacency pair (in bold face), which provides the central interaction of the move and accomplishes the focal problem-solving activity. The captions of the log excerpts indicate the aim of the move, according to the analysis.

Log 1. Open a Topic

LINE

TIME

AUTHOR

TEXT OF CHAT POSTING

1734

08.17.20

bwang8

i think we are very close to solving the problem here

1735

08.17.35

Quicksilver

Oh great...I have to leave

1736

08.17.39

Aznx

We can solve on that topic.

1737

08.17.42

Quicksilver

Sorry guys

1738

08.17.45

bwang8

oh

1739

08.17.46

Aznx

It shouldn't take much time.

1740

08.17.47

bwang8

ok

1741

08.17.50

Aznx

k, bye Quicksilver

1742

08.17.52

Quicksilver

Just tell me the name of the room

1743

08.17.52

bwang8

bye

1744

08.18.14

Gerry

The new room is in the lobby under Open Rooms

1745

08.18.44

Gerry

It is under The Grid World. It has your names on it

1746

08.18.49

Quicksilver

[leaves the room]

1747

08.19.00

Aznx

Alright found it.

1748

08.19.04

Aznx

Thanks.

In line 1734, Bwang states that the team is close to being able to solve the problem of the number of sticks in the nth stage of the diamond pattern, suggesting that they might stay and finish it. Note that this is the end of the last of the scheduled four sessions for the contest, despite some arrangements underway to allow the team to continue to meet.

Aznx responds in line 1736, indicating—and implicitly endorsing the suggestion—that the team could indeed continue to work on the current topic. This opens the topic for the group.

Quicksilver apologetically stresses that he must leave immediately. He just wants to know the location of the new chat room that the facilitator is setting up for the team to continue its math explorations on a future date. The facilitator supplies this information and everyone says goodbye to Quicksilver. We ignore this other activity in our current analysis, and focus on the problem-solving interactions.

Log 2. Decide to Start

1749

08.19.12

Aznx

I guess we should leave then.

1750

08.19.34

bwang8

well do you want to solve the problem

1751

08.19.36

bwang8

i mean

1752

08.19.39

bwang8

we are close

1753

08.19.48

Aznx

Alright.

1754

08.19.51

bwang8

i don't want to wait til tomorrow

1755

08.19.53

bwang8

ok

Aznx expresses uncertainty about how to proceed now that Quicksilver has gone and the facilitator has arranged things for the future. He questions whether he and Bwang need to go as well. Bwang then reiterates his suggestion that they could stay and finish solving the problem. He argues that it should not take much longer. Bwang directly asks Aznx if he wants to solve the problem now.

Aznx agrees by responding to Bwang’s question in the affirmative. This effects a decision by the pair of students to start working on the problem right away. Bwang continues to argue for starting on the problem now—posting line 1754 just 3 seconds after Aznx’ agreement, probably just sending what he had already typed before reading Aznx’ response. Bwang then acknowledges the response.

Log 3. Pick an Approach

1756

08.19.55

Aznx

How do you want to approach it?

1757

08.20.14

bwang8

1st level have 1*4

1758

08.20.20

Gerry

You can put something on the wiki to summarize what you found today

1759

08.20.29

bwang8

2st level have (1+3)*4

1760

08.20.32

Aznx

bwang you put it.

1761

08.20.35

Aznx

for the wiki

1762

08.20.37

bwang8

ok

1763

08.20.42

Aznx

we actually did quite a lot today

1764

08.20.53

bwang8

3rd level have (1+3+5)*4

1765

08.21.05

bwang8

4th level have (1+3+5+7)*4

1766

08.21.10

Gerry

This is a nice way to solve it

Once a decision has been made to solve the problem, the question of how to approach the problem is raised in line 1756. Bwang immediately lays out his approach in lines 1757, 1759, 1764 and 1765. The approach is the same as they used in the first session: visualize just the vertical or just the horizontal sticks. The two sets follow the same pattern. In fact, the diamond is also symmetric left/right and top/bottom, so the vertical sticks can be divided left/right into two identical sets and the horizontal sticks can be divided top/bottom. This produces four identical sets of sticks (color-coded in Figure 2), each having rows of 1, 3, 5, 7, … sticks, up to (2n-1) for the nth stage of the diamond pattern.

Figure 2. A representation (not from the data) of the diamond figure at stage n=4, color-coding the sticks in four identical (symmetric) sets.

Interspersed with this defining of the approach is a reminder from the facilitator to summarize the team’s work on the Spring Fest wiki for other teams to see, motivating this with a word of encouragement about the team’s work.

Log 4. Identify the Pattern

1767

08.21.12

Aznx

So it's a pattern of +2s?

1768

08.21.15

Aznx

Ah ha!

1769

08.21.15

bwang8

yes

1770

08.21.20

Aznx

There's the pattern!

Aznx has previously been oriented toward finding patterns of growth in the mathematical objects the group has been exploring. Often, someone will create a graphical representation of the object in such a way that it makes the pattern visible. Aznx will then formulate a textual description of the pattern. Then the group will work on a symbolic representation to capture the pattern in a mathematical formula. (See (Çakir, Zemel, et al. 2009) for an analysis of the intertwining of graphical/visual, textual/narrative and symbolic/mathematical modes of interaction within the work of Team C.)

Here, in line 1767, Aznx describes the pattern as involving adding numbers that successively increase by 2. The number of sticks in a given stage of the diamond shape is a sum of numbers that start at 1 and increase successively by 2. When going from one stage to the next, one simply adds another number to this sum that is 2 more than the highest previous one.

Aznx presented his description as a question and Bwang affirmed it at the same time as Aznx posted line 1768. Aznx then emphasized that they had discovered the pattern.

Log 5. Seek the Equation

1771

08.21.39

bwang8

now we have to find a equation that describe that pattern

1772

08.21.49

Aznx

Hold on.

1773

08.21.51

Aznx

I know it.

1774

08.21.57

bwang8

what is it

1775

08.21.58

Aznx

But I'm trying to remember it. =P

1776

08.22.04

Aznx

and explain it as well.

1777

08.22.17

Aznx

try and think of it

1778

08.22.53

Gerry

Maybe Quicksilver can come back here tomorrow or next week to finish it with you

1779

08.23.01

Gerry

I have to go now

1780

08.23.05

Gerry

Bye!

1781

08.23.06

bwang8

ok

1782

08.23.07

bwang8

bye

1783

08.23.23

Gerry

[leaves the room]

1784

08.23.29

bwang8

ok

1785

08.23.32

bwang8

so

1786

08.23.37

bwang8

i think it is this

1787

08.23.53

Aznx

ok

1788

08.23.55

Aznx

i found it

1789

08.24.00

Aznx

n^2

1790

08.24.01

bwang8

(2*n)*n/2

1791

08.24.09

Aznx

or (n/2)^2

Bwang indicates that the next step in their work is to “find an equation that describes the pattern.” Aznx asks Bwang to let him state the equation, implicitly agreeing that this is the next step by trying to produce the equation.

Bwang asks Aznx to state the equation and Aznx expresses difficulty in formulating an adequate and accountable answer. After a half minute of silence with still no formulation from Aznx, the facilitator suggests that Aznx and Bwang might want to wait until a future time when the whole group can work together to finish the problem. The facilitator then says goodbye and leaves the chat room.

After more than a minute since Aznx posted anything, Bwang starts to preface the presentation of his own formulation. Eventually, Aznx joins back in. Simultaneously, Aznx and Bwang post their formulae. For Aznx, it is either n^2 or  (n/2)2. For Bwang, it is 2n(n/2).

Aznx has not given any indication of how he got his proposed formula. Bwang’s formula suggests the use of Gauss’ summation, which the students have used repeatedly in the past. According to this summation of an arithmetic sequence of integers, the result is the sum of the first and last member of the sequence times half the number of members. For a sequence of n members, 1 + 3 + 5 + … + (2n-1), the sum would be [1 + (2n-1)]*(n/2). Adding the 1 and the -1, yields Bwang’s formula, 2n(n/2). Note that the nth odd integer can be represented by (2n-1).

It is likely that Aznx used a similar method, working on his own during his prolonged silence, but got confused about the result when he simplified his expression. As Aznx shows next, Aznx’s first answer is equivalent to Bwang’s answer, once Aznx simplifies it. His second answer is related to part of Bwang’s unsimplified answer.

Log 6. Negotiate the Solution

1792

08.24.14

Aznx

I'm simplifying

1793

08.24.30

Aznx

if u simplify urs

1794

08.24.35

Aznx

its n^2

1795

08.24.59

Aznx

bwang

1796

08.25.01

Aznx

you there?

1797

08.25.03

bwang8

so that's wrong

1798

08.25.07

bwang8

yeah

1799

08.25.08

bwang8

i am here

Aznx simplifies Bwang’s formula: 2n(n/2) = n^2. This is the same as one of Aznx’ proposed formulae. When Bwang does not respond to this posting, Aznx wonders if Bwang is still present online.

Bwang was apparently already typing “so that is wrong” when be received Aznx’ question concerning his presence. This message in effect confirmed that Aznx’ second formula, (n/2)2, is wrong and his first one, which agrees with Bwang’s, is correct.

Log 7. Check Cases

1800

08.25.11

Aznx

so

1801

08.25.13

Aznx

the formula

1802

08.25.22

Aznx

would be 4n^2?

1803

08.25.28

bwang8

let's check

1804

08.25.55

bwang8

Yes

1805

08.26.00

bwang8

it actually is

1806

08.26.02

Aznx

So we got it!

Going along with this, Aznx then multiplies their agreed upon formula by 4 because there were 4 sets of horizontal or vertical sticks, each numbering 1 + 3 + …. Aznx poses his message as a question, soliciting confirmation from Bwang. By offering this next step in the symbolic representation, Aznx demonstrates that he understands where Bwang’s formula came from and he understands the larger strategy of approaching the problem that Bwang had proposed. In other words, Aznx demonstrates a level of mathematical competence and of shared understanding that he did not always display in the previous sessions.

Before being ready to answer whether 4n^2 is actually the correct formula for the number of sticks, Bwang suggests that they first check if the formula works by testing it for a number of values of n and counting the sticks in drawings of diamonds at the corresponding nth stage. A half-minute later, Bwang concludes that the formula does check out. He therefore answers Aznx’ question with confidence, perhaps mixed with a touch of surprise.

Aznx concludes that they got the solution for the number of sticks in the diamond pattern—a problem that Team C had posed for itself, but for which they had derived the wrong formula, without, however, realizing it. Team B had been shocked earlier to discover that the formula they had been struggling to understand from Team C had been wrong; that it did not check out for any values of n.

Log 8. Confirm the Solution

1807

08.26.02

bwang8

omg

1808

08.26.04

Aznx

yay!

1809

08.26.08

bwang8

i think we got it!!!!!!!!!!!!

1810

08.26.12

Aznx

WE DID IT!!!!!!

1811

08.26.12

bwang8

and it is so simple

1812

08.26.14

Aznx

YAY!!!!

1813

08.26.16

Aznx

i know

1814

08.26.17

bwang8

lol

1815

08.26.18

Aznx

lol

Their surprise and excitement is almost uncontrollable. They use every chat technique they know to express their joy. Their postings intertwine like a frenzied dance.

Log 9. Present a Formal Solution

1816

08.26.34

Aznx

So you're putting it in the wiki, right?

1817

08.26.37

bwang8

yes

1818

08.26.41

Aznx

Alright then.

1819

08.26.43

bwang8

ok

1820

08.26.53

Aznx

Give an email to Gery, telling him that we got it. =)

1821

08.26.57

bwang8

ok

1822

08.26.59

Aznx

I meant Gerry

1823

08.27.04

bwang8

are you going to do it

1824

08.27.07

bwang8

or am i

1825

08.27.12

Aznx

You do it.

1826

08.27.14

bwang8

ok

1827

08.27.19

Aznx

Tell him that we both dervied n^2

1828

08.27.29

Aznx

And then we saw that pattern

1829

08.27.37

Aznx

and we got the formula

Once the mathematical exploration is done, it is time to write up a report of one’s findings. Professional mathematicians would do this in the form of a proof. When a group of mathematicians recently conducted an online collaborative analysis of a mathematical problem, it took them longer to write the publishable proof than it did to figure out the approach and solve it (Gowers & Nielsen 2010; Polymath 2010). Bwang posted the narrative shown in Figure 3 to the Spring Fest wiki.

We then move on to understand Team C's formula for summing up the total # of sticks in n-level diamond. We first tried to used the big square and then minus the extra corners, but the corners turns out to be to hard to calculate. Then we tried to simplify Team C's equation to help as find a lead, but we found out that their stick equation is wrong. We then decide to find out a whole new equation and tried to divide the sticks up into vertical and horizontal groups like we did before with all the other problems. The groups can be further divided into 2 equal parts. We found a pattern.

1st level: 1

2nd level: 1+3

3rd level: 1+3+5

4th level: 1+3+5+7

5th level: 1+3+5+7+9

nth level: (2*n)*n/2

We then found out that each of these can be by calculated by (2*n)*n/2 which simplified into n^2. n^2 can then be multiplied by 4 and get the total of sticks in a nth leveled diamond. The final equation is 4(n^2).

Figure 3. Wiki posting by Group B after session 4.

Log 10. Close the Topic

1830

08.27.44

Aznx

when should we meet again?

1831

08.27.49

Aznx

hat's your email?

1832

08.27.52

Aznx

we should keep in touch

1833

08.27.57

bwang8

yeah

Finally, Aznx and Bwang wrap up the conversational topic by exchanging email addresses and agreeing to meet again online with Quicksilver and pursue further mathematical adventures together.

The Sequence of Pairs

Within each of the preceding log excerpts, we have identified a base adjacency pair by means of which the work of a specific move in the problem-solving effort of the small group is interactively accomplished. In most cases, a question is posed and a response is then given to it.

As Schegloff (2007) argues, an adjacency pair is itself a sequence. It embodies a temporal structure, with the first element of the pair projecting the opportunity and expectation of a response in the interactional immediate future. The second element constitutes an uptake of a first element that it implicitly references as in the interactional immediate past (Suthers et al. 2010a). In engaging in the exchange of an adjacency pair, the participants in the interaction effectively co-construct an elementary temporal structure in which future and past are constituted.

In talk-in-interaction, as analyzed by conversation analysis, the immediacy of response is intimately related to the turn-taking structure of vocal conversation (Sacks et al. 1974). As discussed above, the completion of the adjacency pair is often postponed by insertion sequences, such as repairs of misunderstandings or clarification exchanges. The base adjacency pair can also be preceded by introductory exchanges, such as announcements of what is coming, or succeeded by follow-up exchanges or confirmations.

In chat-in-interaction, as seen in the preceding log extracts, adjacency pairs can be delayed by a more complicated response structure, in which multiple participants can be typing simultaneously and postings do not always directly follow the message to which they are responding. Thus, in Log 1, Quicksilver or Gerry can be initiating other topics in the midst of an interaction between Aznx and Bwang. Also, Aznx and Bwang can be typing to each other simultaneously as in Log 6, particularly if there has been an extended period of inactivity. This often makes textual chat harder to follow and to analyze than verbal conversation.

Nevertheless, it is generally possible to identify base adjacency pairs carrying the discourse along. In the previous section, we identified ten pairs. The discourse moves in the log excerpts (each including one of these base adjacency pairs) formed a problem-solving sequence:

- Log 1. Open the topic

- Log 2. Decide to start

- Log 3. Pick an approach

- Log 4. Identify the pattern

- Log 5. Seek the equation

- Log 6. Negotiate the solution

- Log 7. Check cases

- Log 8. Confirm the solution

- Log 9. Present a formal solution

- Log 10. Close the topic

The integrity of each of the ten moves is constructed by the discourse of the participants. Each move contains its single base adjacency pair, which drives the interaction. In addition, there may be several utterances of secondary structural importance, which introduce, interrupt or extend the base pair; there may also be some peripheral utterances by other participants.

The analysis of this essay is an attempt to make explicit the structure of adjacency pairs and a problem-solving longer sequence that is experienced by the participants and is implicit in the formulation of their contributions to the discourse. This is in contrast to analytic approaches that to some degree impose a set of coding categories based on the analyst’s research interests or on an a priori theoretical framework, rather than on the perspective of the participants as evidenced in their discourse.

Lines 1795 and 1796, for instance, show the power for the participants of the adjacency pairings. Here, Aznx has addressed a mathematical proposal to Bwang: “If you simplify yours [expression], it is n^2.” After 24 seconds of inaction, Aznx cannot understand why Bwang has not replied, expressing agreement or disagreement with the first part of the proposal, for which Aznx expects a response. Because it is not a preferred move at this point for Aznx to reprimand Bwang for not responding, Aznx inquires if Bwang has disappeared, perhaps due to a technical software problem, which would not be anyone’s fault. Two seconds later, we see that Bwang was typing a more involved response that implicitly accepted Aznx’ proposal. Bwang then immediately explicitly accepts the proposal in line 1798, allowing Aznx to continue with the start of a new move with line 1802. Here we see Aznx and Bwang clearly orienting to the adjacency-pair structure of their discourse, in terms of their expectations and responses.

Aznx and Bwang co-constructed the longer (ten move) problem-solving sequence by engaging in the successive exchange of adjacency pairs. Sometimes one of the students would initiate the pair, sometimes the other. As soon as they completed one pair, they would start the next. This longer sequence also has a temporal structure. It is grounded in their present situation, trying to find a formula for the number of sticks in the diamond figure. It makes considerable use of resources from their shared (co-experienced) past during the previous four hours of online sessions. It is strongly driven forward into the future by the practices they have learned for engaging in problem solving, culminating teleologically in the presentation of a solution.

The problem-solving sequence analyzed in this essay—covering 100 lines of chat during 10 minutes—is not selected arbitrarily or imposed in accordance with criteria external to the interaction, but is grounded in the discourse as structured by the participants. The excerpted sequence is defined as a coherent conversational topic by the discourse of Aznx and Bwang. They open this topic with their interaction in Log 1 and they close it with the discourse move in Log 10 (Schegloff & Sacks 1973).

This case study provides an unusually clear and simple example of group cognition in a virtual math team. In earlier sessions, the students encountered many difficulties, although they also achieved a variety of successes and learned much about both collaboration and mathematics. At the beginning of their first session, they did not know how to behave together and showed rather poor collaboration skills. Bwang said very little in English, often simply producing drawings or mathematical expressions. Aznx, at the other extreme, tried hard to engage the others, but seemed to have a weak mathematical understanding of what the others were discussing. At various points in the sessions, misunderstandings caused major detours and breakdowns in the group work. Moreover, from an analyst’s perspective the interaction was often almost impossible to parse (Stahl 2009b). By contrast, in the final segment that is here reviewed, the interaction is focused on two participants; they work well together; they seem to follow each other well; and their work goes quite smoothly. The structure of the interaction is also relatively easy to follow.

It seems that Aznx and Bwang have substantially increased their skills in online collaborative mathematics. The level of their excitement—especially in the excerpt of Log 8—shows they are highly motivated. Log 10 indicates that they would like to continue this kind of experience in the future.

Collaborative Mathematical Meaning Making

Shared meaning is co-constructed as the discourse moves (the log excerpts based around adjacency pairs) build on each other to form the longer sequence of the discourse topic. This is a key level of analysis for understanding the workings of group cognition. Because these discourse moves are founded upon adjacency pairs, they essentially involve more than one participant, and therefore lend themselves to being vehicles for cognitive phenomena at the group unit of analysis. Through their sequential positioning and subtle forms of mutual referencing, they contribute to problem solving and other cognitive accomplishments. As an example, we can see how Team B solved their mathematical problem across Logs 5, 6 and 7.

In Log 5, we see that collaborative problem solving of a math topic—like most group meaning making—is an intricate intertwining of individual interpretation and shared meaning (Stahl 2009b).  Bwang (line 1771) states the goal for the dyad of finding an equation to describe the pattern of twos. Aznx immediately announces that he knows the equation (1773) and wants to provide it (1772), to which Bwang acquiesces (1774). However, Aznx has trouble coming up with an equation: remembering it, explaining it, thinking of it or finding it. After a while, Bwang gradually announces that he will provide the equation (1784-1786). Then they both propose equations. Throughout the online session, mathematical proposals originate from the understanding of individual students. In this excerpt, they negotiate about who is to make the proposal, and end up both doing so.

Then it is necessary in Log 6 to decide whose proposal will be adopted by the group as a basis for future work. Interestingly, Aznx reconciles their proposals by algebraically transforming Bwang’s equation to be the same as one of Aznx’ own (1792-1794). This circumvents the possibility that Bwang will reject Aznx’ proposal, which he in fact does (1797). It also establishes a group solution whose meaning (derivation, use, form) is likely to be mutually understood since the solution was proposed by both.

Finally, in Log 7, Aznx takes a further mathematical step, multiplying the n^2 by 4 to account for the 4 symmetrical sets of sticks. However, he presents this final formula in question format (1800-1802), soliciting Bwang’s agreement in order to establish the formula within their joint problem space. Bwang implicitly accepts Aznx’s step and reinterprets the question as requiring a next step of checking the formula for values of n. Bwang presumably checks several values and concludes that the formula works (1804-1805). Aznx summarizes, “So we got it!” Note his use of the pronoun, “we,” attributing the solving to the group.

The formula, 4n^2, is a particularly meaningful expression in this chat, the triumphal culmination of four hours of mathematical exploration. It is a highly meaningful expression for the group, summarizing their analysis of the diamond pattern of sticks at every level of n. The students understand its meaning as a consequence of their participation in the group processes of drawing and discussing together a rich set of related mathematical phenomena. The shared meaning of the math expression is publicly available in the discourse and through its traces in the log; it was co-constructed through the contributions of individuals and is interpreted by those individuals—and later by analysts.

The Structure of Group Cognition

The analysis of the case study in this essay provides a first analysis of the long-sequence-of-moves structure of collaborative mathematical problem solving in a virtual math team. This is a paradigmatic example of group cognition. The small group—here reduced to a dyad—solves a math problem whose solution had until then eluded them (and had escaped Team C as well).

The students accomplish the problem solving by successively completing a sequence of ten moves. Each of the moves seems almost trivial, but each takes place through an interaction that involves both students in its achievement. The moves are commonplace, taken-for-granted practices of mathematical problem solving. They are familiar from individual and classroom problem solving in algebra classrooms. They have also been encountered repeatedly by Team B in their previous four hours of collaborative problem solving (Medina et al. 2009).

Reviewing the sequence of the group’s ten moves presented in this essay, we can follow the mathematical solution process. After opening the topic of the sticks problem (Log 1) and deciding to work on it together (Log 2), the team picked an approach of looking at the number of sticks as being countable with the series (1+3+5+7+…)*4 (Log 3). This series is generated by counting the sticks in a visual representation of the diamond pattern at different values of n (Figure 2). This uses the approach from previous sessions of separating the horizontal and vertical sticks (Figure 1) and then dividing each of those groups into two symmetrical groups (Figure 3). The group then articulates a verbal description of this visual series as being “a pattern of +2s” (Log 4). Both students try to symbolize the pattern of the verbal description as an equation (Log 5) and they come to agreement on the formula as n^2 (Log 6), presumably based on the formula for summing integer series, familiar to them from previous sessions. They then check that their equation works for a number of stages of the diamond pattern (Bwang does this off-line during Log 7). Having solved the mathematical challenge as a group they celebrate the group achievement: “WE DID IT!!!!!!” (Log 8), decide to present their solution publicly (Log 9) and close the discourse topic (Log 10).

It is this sequence of moves that accomplishes the problem solving. The sequence has an inner logic, with each move requiring the previous moves to have already been successfully completed and each move preparing the way for the following ones. Of course, in working on a problem, problem solvers—even professionals (Gowers & Nielsen 2010; Polymath 2010)—often  make mistakes and explore deadends. Team B’s wiki posting (Figure 4) documents that some of this had happened prior to the excerpt analyzed in this essay. Part of what contributes to the unusual clarity of our example is the simplicity of the sequence followed in the final segment.

The common assumption about mathematical problem solving is that information in the form of math facts and manipulations are what are most important. In our analysis of problem solving in a group context, math content and other information is simply, unproblematically included in individual postings. In fact, more often than not, it is implicitly used and understood “between the lines” of the text chat. Of course, this is only possible because the group had already co-constructed a joint problem space (Medina et al. 2009; Sarmiento & Stahl 2008; Teasley & Roschelle 1993) that included this math content as already meaningful for the group. Rather, the important aspects of discourse engaged in collaborative math problem solving are matters of coordination, communication, explanation, decision making and perspective shifting (e.g., moving between visual, verbal and symbolic modes (Çakir, Zemel, et al. 2009)). To some extent, these are interactional moves required by most group activities; to some extent, these are adapted to the nature of mathematical discourse.

In conclusion, the group-cognitive achievement of the solution to the group’s final problem was accomplished by a sequence of moves. Each move was mundane when considered by itself. The moves and their sequencing were common practices of mathematical problem solving. Each move was interactively achieved through the exchange of base adjacency pairs situated in the ongoing discourse. The problem solving was an act of group cognition structured as a sequence of these interactive moves.

While we cannot generalize from the analysis in this essay, it seems that this case study can serve as a perhaps unusually clear and simple model of the structure of group cognition in mathematical problem solving by a virtual math team. It shows the group cognition taking place through the co-construction of a temporal sequence of problem-solving moves. Each move is conducted on the basis of an interactional adjacency pair of chat utterances. While the fine structure adheres to the adjacency-pair system of interactional exchange, the larger problem-solving structure builds on these elements through a sequence defined by the topical moves of mathematical deduction.

More generally, this suggests a multi-layered hierarchical structure to discourse in virtual math teams (Stahl 2009a). Each layer is oriented to by the participant activities:

1. Group event: E.g., Team B’s participation in the VMT Spring Fest 2006. The team meets together and gradually starts to act as a collaborative group.

2. Temporal session: Session 4 of Team B on the afternoon of May 18, 2006. The participants agree when to break up a session, when to meet next, and then show up at the same time.

3. Conversational topic: Determining the number of sticks in a diamond pattern (lines 1734 to 1833 of Session 4). We saw how Bwang and Aznx open the topic and later close it.

4. Discourse move: A stage in the sequence of moves to accomplish discussing the conversational topic (e.g., lines 1767-1770). The team steps through the sequence of moves.

5. Adjacency pair: The base interaction involving two or three utterances, which drives a discourse move (lines 1767 and 1769). Each initial utterance elicits a response.

6. Textual utterance: A text chat posting by an individual participant, which may contribute to an adjacency pair (line 1767). The group members format their separate postings.

7. Indexical reference: An element of a textual utterance that points to a relevant resource. In VMT, actions and objects in the shared whiteboard are often referenced. Mathematical content and other resources from the joint problem space and from shared past experience are also brought into the discourse by explicit or implicit reference in an utterance.

This multi-layered structure corresponds to the multiplicity of constraints imposed on small-group discourse—from the character of the life-world and of culture, which mediate macro-structure, to the semantic, syntactic and pragmatic rules of language, which govern the fine structure of utterances. An understanding of group cognition must concern itself primarily with the analysis of mid-level phenomena—such as how small groups accomplish collaborative problem solving and other conversational topics. This is a realm of analysis that is currently underdeveloped.

The preceding analysis illustrates the applicability of the notion of a long sequence as suggested by both Sacks (1965/1995) and Schegloff (2007). The sequence consists of a coherent series of shorter sequences built on adjacency pairs. This multi-layered sequential structure is adapted from CA to the essentially different, but analogous, context of groupware-supported communication and group cognition. Having seen that this kind of sequential structure exists in the relatively simple case we analyzed, we can now look for longer sequences in the traces of other acts of groupware-mediated group cognition.

6. Interaction Analysis of a Biology Chat

Abstract. This is an analysis of data from initial attempts to combine (a) technology from the Virtual Math Teams (VMT) Project, (b) software helping agents, (c) collaborative small groups and (d) accountable-talk prompting in order to scaffold biology student online chats about videotaped results of a biology experiment. Analysis of the response structure of the chat log of a student group reveals characteristics of their interactions in terms of building collaborative knowledge. In particular, the mediation by the VMT technology, helping agents and accountable-talk training is analyzed to determine their influences in promoting productive learning-oriented interaction. A design-based-research analytic perspective provides suggestions for redesign of the socio-technical approach based on the findings from the interaction analysis. Redesign in response to the analysis results in clear improvement, as seen in analysis of the response structure of a chat log from a second test cycle.

Analyzing Response Structure

T

his essay takes a specific analytic approach, developed within the Virtual Math Teams (VMT) Project (Stahl, 2009). The VMT research team adapted video-based interaction analysis of face-to-face discourse (Jordan & Henderson, 1995) to analyze synchronous text chat by students in their mid-teens as they interact in the online VMT environment, discussing issues raised in school mathematics. We found that, from a structural viewpoint, the most important aspect of discourse is its temporal sequentiality; the field of Conversation Analysis has analyzed this extensively, beginning with (Sacks, 1962/1995) and summarized more recently by (Schegloff, 2007). We adapted such sequential analysis to student chat discourse in the VMT environment at the foundational level of “adjacency pairs” of mutually responsive postings (Stahl, 2006c)—which we take as the unit of interaction—and at the “longer sequence” level (Stahl, 2011)—which we feel is the key level of description for knowledge building in computer-supported collaborative learning (CSCL).

In this essay, I apply the method of analyzing text-chat response structure that we developed in the VMT Project to chat among students discussing a biology experiment conducted in an early version of the environment formerly known as ConcertChat (now VMT). The text chat was integrated with class discussion, a worksheet and videos. In addition, the software was extended with a software agent, which interacted with the students as a chat participant. I ignore most of the larger context of the experiment and focus on what is visible in the chat log. I look at a representative case from each of the first two cycles of experimentation.

In undertaking this essay, I decided to do my own methodological experiment within the biology educational experiment. I wanted to see if sequential analysis could be used effectively as a quick-and-dirty method of evaluation within a design-based-research cycle. Design-based research is a wide-spread approach within educational research for designing technological and pedagogical interventions through iterative cycles of design, prototyping, user trial, analysis and re-design. In the biology experiment, an intervention had been designed for biology classrooms; software agents had been prototyped within a version of the VMT collaboration environment; the intervention was tried in middle-school classrooms; and it was now time to analyze the results. While some experimenters may have been hoping that analysis would show the benefits of agent support or accountable-talk training, my aim was to discover what most needed re-design in the next cycle.

Although design-based research is a much used and discussed approach to educational research, there is no established method for conducting the analysis phase of the iterative cycles. Researchers both friendly to and opposed to Conversation Analysis (CA) have argued that CA sequential analysis is inappropriate in design-based research. Adherents of CA argue that CA cannot be applied to design efforts because it is interested in seeing what emerges of interest from an unguided analysis of the participants’ discourse—which is unlikely to be relevant to a designer’s goal-oriented concerns. On the other hand, researchers from other approaches, such as quantitative coding of discourse, insist that qualitative CA takes too long and is too costly to fit into the workflow and focused research questions of re-design cycles. My experiment was to see if I could conduct a quick sequential analysis that would cheaply and effectively point the way for re-design. That was the practical goal of my methodological experiment.

Theoretically, I was interested in understanding what “really” occurred in the interaction between students and agent. I wanted to “bracket out” the assumptions of the people who set up the biology experiment as well as assumptions about what went on in the heads of the students or the programs of the agent, based on reports from outside the discourse data. As a researcher of group cognition (Stahl, 2006a), I am interested in the effect of the intervention on the group processes, the interaction visible in the chat log. I wanted to see how much I could learn about the group process by viewing the structure resulting from sequential analysis. I wondered what I could fathom of the group knowledge building from micro-analysis of the discourse details, i.e., from how the participants articulated their responses to each other. The goal of accountable-talk training and support is presumably to change certain aspects of the talk by the students, and this is what I wanted to observe directly—not indirectly from statistical verification of hypotheses based on testing responses of individual students outside of the group-interaction context.

Obviously, the behavior of the students will be affected by countless factors, many of which could be studied in theory with various methods and data-collection efforts: the personalities and backgrounds of the students, the programming of the agents, the funding of the schools, the history of American education, prior testing results and future test schedules, etc. But I wanted to see how far I could get in making grounded re-design recommendations by just looking with some care at a small sample of interaction data.

Furthermore, I was only concerned about the group unit of analysis, that is the interactions among group members, not the status of any one individual member. Fortunately, because the group interaction for a period of time during the experiment was mediated by the VMT system, all group interaction among the students and the agent passed through the chat tool and was captured in the chat log exactly as it appeared to the participants. This gave me a complete and reliable log of the group interaction without all the complications and interpretive issues of videotaping and transcribing. As described below, I modified the chat log representation and then constructed a representation of the sequential interaction (Figure 1). Simply looking at this representation allowed me to make some tentative conclusions about the nature of the interaction and to point these conclusions out to others. The conjectures based on this representation guided a careful look at the details of how the specific chat postings involved were designed by their posters, the groups of students.

The problematic aspects of interaction revealed in my quick response analysis of a student chat in the original intervention were taken into account in redesigning the intervention in a second cycle of design-based research a year later. I conducted a similar quick response analysis of a student chat in cycle 2 and was able to see a significant improvement in the behavior of the agent as well as in the discourse of the student group.

Method

1. Following the first classroom intervention, I was supplied with the logs of 16 chats, in spreadsheet format. The chats each lasted about a half an hour and contained the chat postings of three students and an agent. The 16 chats were divided among three conditions. In one condition, the agent prompted students (indirectly) to ask each other to make specific accountable-talk moves. In a second condition, the agent prompted students (directly) to make specific accountable-talk moves. In the final condition, the agent did not make any accountable-talk prompts, but only guided the students through the steps of the assignment (as was also done in the first two conditions).

2. I read through each of the 16 chat logs that I was given and I wrote down a couple sentences of my initial reaction to the quality of the interaction. It struck me that similar patterns of interaction were arising in the 16 logs, and so I decided to analyze one chat in detail to get at key common patterns. I selected log C01 as representative and promising for illustrating the common patterns. This case was from the first condition, in which the agent gave indirect prompts. Clearly, other analyses with different research questions and approaches would want to contrast the different conditions (e.g., Howley, et al. in (Suthers, Lund, Rosé, Teplovs, et al. 2013)), but from my focus on response structure it seemed particularly useful to look closely at one typical example.

3. In order to make the interaction flow visible, I rearranged the spreadsheet to have the postings of each participant in its own column. (The entire log of C01 is reproduced in Appendix A below.) The newer version of VMT produces logs in this format automatically for students, teachers and analysts. We often also have columns for time elapsed since the previous posting and time when a posting was starting to be typed. These figures sometimes help to determine which previous posting a new posting is responding to. In the current log, such detailed reasoning was not generally necessary.

4. I next sketched the response structure of the chat (see Figure 1). I drew an arrow from each posting to the prior posting to which it was responding interactively, for instance to what question is an answer responding? This already gave a visual impression of some aspects of the patterns of responses. These patterns are central to the interactional dynamic of the group.

Figure 1. Sequential response structure of chat C01. Note that only interactions between actors are represented, not instances of a posting by one actor building on his, her or its own previous posting.

5. An important phase of interaction analysis is the exploration of the data, line-by-line, in a data session with other researchers (Jordan & Henderson, 1995). This inherently dialogical or multi-vocal approach can bring in multidisciplinary perspectives and balance one-sided views. A data session can be most effective once some initial analysis has already been undertaken by one of the researchers. After the data session, suggestions have to be synthesized and followed up with further detailed data analysis. There can be multiple cycles of group and individual analysis. The data session for this essay’s analysis included experienced online educators from the Math Forum and two analysts from other chapters (Rosé and Goggins). The session suggested a more complex representation of the response structure, it refined interpretive details and it situated the case study in a deeper understanding of the experimental context. In particular, the data-session discussion proposed the representation of response structure of accountable talk (Resnick, O'Connor & Michaels, 2007) shown in Figure 2, which was used in refining Figure 1.

Figure 2. Sequential response structure of accountable talk.

6.      Once I had a preliminary view of the response structure of the discourse in the chat, I could start to formulate tentative observations about the case study. These observations led to looking at the textual content of the postings. This showed the nature of the group interaction in more detail. The evolving analysis (see next section) also revealed the understandings and reactions of the students to their situation. This highlighted the response of the student group to its given task and to the actions of the agent, to the accountable-talk training and to the software environment.

7.      As I summarized my observations (see discussion section), I felt that they generally applied to the other chats as well. By grouping the problems in relation to different design decisions in the experiment, I was able to propose several general suggestions for future re-design (see conclusions section). Other analysts, taking into account other data, additional knowledge of the constraints on the experiment, and alternative research questions will undoubtedly reach different—hopefully complementary—conclusions. I was interested in seeing what insights an interaction analysis of a single case study could provide for the long-term design-based-research effort. I wanted to do this analysis strictly on the basis of the chat data from a single case study, without being concerned about the many constraints, practicalities, and concerns that influenced the experimental design in all its complexity.

Analysis of the Chat-Response Structure

Figure 2 shows a representation of the response structure of an ideal accountable-talk interaction, as hypothesized by the experimenters. The blue arrows indicate that the agent responds to the students (line 2 and 7) and that the students in turn respond to the agent (lines 3 and 8). There is also a sequence in which the students respond to each other (lines 3, 4, 5, 6). This produces a tight group interaction including the agent and the students. The green arrows indicate that subsequent postings often involve uptake of content from previous postings (e.g., lines 4, 5, 6, 8 by the students). The role of the agent does not involve content, but mediates the student uptake of content by means of accountable-talk prompts (lines 2 and 7, pointed to by the red arrows). Let us see the extent to which the data of actual interaction among students and the agent includes similar patterns of response.

Figure 1 indicates three instances of mediation of accountable talk (red arrows): (i) the response at line 19 to line 16, (ii) the response at line 25 to line 23, and (iii) the response at lines 34 and 35 to line 26. Let us consider each of these in turn.

(i) The agent requests in line 16: “Please discuss what you predict will happen in these two conditions.” Student S034 complies after a lengthy two-and-a-half minutes of silence by asking the group, “what do you think’ds going to happen?” At this point, the agent interjects some information about a third condition and asks the students to move on to discussing that. The timing of this seems questionable if the goal is to encourage extended knowledge-building interaction among the students. Student S041 then ignores the agent’s latest contribution and responds ironically to student S034’s request for a prediction: “the world is going to end in 2012.”

(ii) The agent quickly picks up on S041’s prediction by introducing the indirect prompting for accountable talk in line 23: “S027, now would be a good time to ask S034 to build on what S041 is saying.” This all confuses S034, who states, “im so confused!” But S027 dutifully instructs S034 to explain S041’s remark by building on it and explaining it to S027: “034, would you like to build on to what 041 is saying? and me too!” The first part of this follows the script prompted by the agent, but S027 adds his sympathetic addendum, aligning with S034 by agreeing that he is also confused about what is being asked of them.

(iii) The final mediation is similar to the first. In line 26, the agent requests: “When you are in agreement, write down your predictions and explanations for Conditions A, B and C on your worksheet.” A minute later, after S027 complains again of not knowing what to do, S034 says, “someone predict something.” Student S041 responds again to student 034: “THE WORLD IS GOING TO END IN 2012!”

As the green arrows indicate, almost all uptake of content is associated with these three mediated interactions. Line 8 merely introduces the student, repeating the word “name”: S034 responds to the agent’s “I didn’t get your names yet” with “my name is [S034].” Line 107 responds to line 105’s birthday greeting with “is it ur birthday?” These are not knowledge-building moves, but are social interactions, not directly relevant to accountable talk about curricular content.

There is some evidence that the agent is responding to student postings. The agent’s line 7 succeeds in getting S034 to give his or her name and the agent then responds to that by assigning a role to S034. At line 23, the agent responds to a posting by S041 by asking S027 to ask S034 to build on what S041 said. This is an instance of the indirect mediation. While the timing is appropriate to ask S027 and S034 to discuss a posting by S041, the agent clearly fails to understand the significance of the posting. The agent assumes that S041 has made a prediction about the biology experiment, and not a sarcastic joke. This could have sent the group off on a distracting tangent, but in fact only confused the students about the agent’s behavior and the meaning of the agent’s requests.

If we look at the blue arrows in Figure 1, we see that the only times that the agent responded to the students were in lines 9 and 23. In line 9, the agent started to assign roles that were ignored by the students. In line 23, the agent requested an accountable-talk script to build on a joke.

A look at the high-level visual structure for Figure 1 indicates that the agent dominated the discussion in the early part, but then was ignored for most of the remainder of the chat. Toward the end, there was a significant pattern of interaction among the students, who seemed to be engaged as a group. A closer look at the content of the individual students’ postings suggests that S034 is trying hard to accomplish the class task. S027 seems generally lost. S041 is not interested in the biology and is more oriented to clowning around. There is no apparent correlation of their individual behaviors to the roles assigned to them by the agent.

The period from posting 5 through 18 lasted about four minutes. This period is totally dominated by the agent, which posted over 260 words while the three students responded with a total of 9 words, mostly just stating their names. The agent did not acknowledge their responses or appear to respond to them, except as noted above. Although delivering instructions to the students through the agent may have been motivated by an attempt to establish dialog between the agent and the students, it positioned the agent as an authoritative source of knowledge and commands, while positioning the group of students as a set of largely passive listeners, thus discouraging student discursive agency.

Of course, it made no sense for the agent to ask the students to “build on” to the sarcastic answer in line 22. This response by S041 shows that he/she already did not take the agent seriously. By not interacting with the students in a way that makes sense to them, the agent fails to establish itself as a serious participant in the group discourse. Caught in the middle between human interaction with the other students and obeying the authoritative orders of the agent, S027 follows the agent’s command, but adds his protest against the agent’s leadership in line 25.

S027 and the other students then stop orienting to the agent and the agent is ignored for the next 10 minutes until it again provides an unhelpful indirect prompt for accountable talk at line 69. Instead of responding to the agent prompt, S027 asks who is 34 and says “ooh. hi” when S034 responds. The students go on to work together to fill in the worksheet. One student provides the answers and the others try to figure out how to copy those answers into their own worksheets.

The agent continues to give commands, but they are generally ignored. When in line 69 the agent prompts once more for accountable talk, the students agree that the agent is being an insufferable nuisance. They evaluate the whole supported chat experience by agreeing that “this would be so much easier just in a group,” meaning just sitting together without any computer or agent support and filling in their worksheets. Their only subsequent response to the agent is to celebrate when it leaves.

Discussion: Issues Observed

In the initial experiment, students were placed in small groups of three students and an agent in a chat room. This is a setting that calls for intense text-based interaction. The patterns in Figure 1 are already visually suggestive. The agent does not significantly respond to (i.e., interact with) students. The student responses to the agent are problematic. After trying to be responsive, the students give up and start to engage in their own discussion. The later periods of student interaction show considerable back-and-forth responses as they elicit responses, provide reactions and then acknowledge the responses to each other in various ways. Student responses are tightly situated in the on-going discourse, whereas the agent speaks like an academic textbook, with no sense of contextualization and little apparent attempt at interaction.

The educational experiment is an attempt to support collaborative learning with (a) the VMT software environment, (b) software helping agents (c) a social small-group setting, and (d) accountable-talk prompts. It is a CSCL intervention that aims to scaffold collaborative learning with these forms of computer support and communication structuring.

(a) The first problem is that the lesson design does not succeed in fostering collaboration. The students are each given their own worksheet to fill out and then they are each tested individually. There is no meaningful group task or group goal to be accomplished collaboratively. The questions to be addressed by the students are not open-ended issues to encourage group inquiry and discussion, but questions with instructor-defined correct answers that the students can solve individually. Consequently, there is little evidence of real knowledge building taking place collaboratively. The most that occurs is that a student who knows the correct answer will give it to students who do not know it. Rather than this taking place as accountable talk, it naturally takes place in the form of students copying each other’s answers to fill in their individual forms, without caring much about understanding the science—i.e., a common school process understood by all as cheating rather than collaborating or learning. The VMT environment was designed for shared tasks, with a shared whiteboard provided as a shared external memory that can be even more important for communication and joint work than the text chat (Çakır, Zemel & Stahl, 2009). Rather than this, the experiment uses the whiteboard to display once more a static cartoon of accountable talk, which appears to have been completely ignored by the students. The whiteboard could have contained the worksheet, to be filled out collaboratively by the team. That group artifact could then have been evaluated for the grading, rather than threatening the students with individual quizzes (causing expressions of test phobia). The shared whiteboard (or additional tabs with web browsers or other whiteboards) could also have been used to present data of the biology experiment, rather than having the students have to start up other applications (causing further confusion).

(b) The second problem involves the design of the agent interventions. Primarily, the agent was in effect non-interactive. The agent may have been carefully programmed to intervene in an interactive way, but it does not come off that way in a sequential analysis of the chat—which is more important than the intentions of the programmer. To the students, the agent’s timing did not appear to be effectively coordinated with the student discourse or responses. Inevitably, the agent postings introduced confusion for the students rather than clear structure. They were incredibly verbose—within the chat medium, which is known for its conciseness of expression. It might have made more sense to explain the process in class before breaking into online chat groups. Helping agents should probably not be used to automate teacher-centric instructors, but should get out of the way of student interaction until the students express a need for help. When an agent does intervene, it has to know what is going on well enough to judge what kind of response might be helpful. The agent behavior programmed here was an extreme example of “over scripting” and the opposite of the recommended “SWISH approach” (Dillenbourg, 2002; Dillenbourg & Jermann, 2006).

(c) A third problem involves social identity. Teenage students are mainly learning social skills, despite teacher efforts to have them learn curricular content. So when they are put together to interact in small groups it is essential to them that they know as much as possible about each other. In the VMT Project, we tried to put together students with no prior knowledge of each other so that we researchers could know everything the students knew about each other, so that we could interpret their interaction logs on a par with their understanding of the group interaction. In this biology case study, the students knew each other very well and had well practiced relationships. By assigning the chat participants anonymous identifiers, the experiment interfered with their exercise of these important and motivating social relationships (see chapter by Cress & Kimmerle in (Suthers, Lund, Rosé, Teplovs, et al. 2013)). The students spent much time and attention in overcoming this circumstance (e.g., chat lines 17/18 and 27/28/30), positioning them in opposition to the conditions imposed upon their daily routines by this experimental intervention.

(d) Finally, accountable talk needs to take place at a sophisticated level of discourse. Like all effective discourse, it must be highly situated in the on-going discussion. That is the skill of a teacher who has mastered accountable talk moves, to know just when and how to prompt. A complicated prompt cannot just appear unexpectedly and hope to be helpful in building shared understanding. This poses a major technical challenge for software agents at many levels; it may require many cycles of design-based research to evolve an effective interaction behavior for helping agents that can effectively prompt for accountable talk by students.

Suggestions for Redesign

The biology experiment is cutting-edge research. The components that it brings together each require groundbreaking advances in the knowledge of their domain. It is not a matter of simply applying well-understood techniques.

(a) It took years of research by a large international, interdisciplinary team to develop the integration of collaborative pedagogy, problem wording and interaction technology for the Virtual Math Teams Project in the domain of collaborative online discourse of school mathematics—and there is still much investigation to be done there. Similar explorations will be needed for the domain of online discourse of school biology. A primary issue in guiding student inquiry in small online groups is how to avoid intruding in the important processes of small-group collaboration among the students; the case study just analyzed shows that there is a long way to go in achieving this with the approach tried. Our past research emphasizes how important yet difficult guidance or scaffolding of collaborative knowledge building is to achieve. In the VMT Project, we often had an adult facilitator in the chat room with the group of students. We trained the facilitators to avoid intervening too much in the interaction, mainly answering questions and helping with technology issues. A study of this showed the subtlety of supporting student group agency rather than interfering with it (Charles & Shumar, 2009).

(b) Involving software agents as participants in open-ended collaboration is quite different from the approaches that have been so successful in automated tutors of individual students being trained in well-defined algebra procedures within tightly constrained interfaces. In collaboration with Carolyn Rosè’s research group, we started to explore the interaction of software agents with students in online discussions in the VMT environment with experiments in a mathematics classroom (Stahl et al., 2010). Here we discovered how invasive agents tend to be. Even with “wizard of Oz” experiments in which human researchers played the role of software agents, the presence of the “agents” radically transformed the online interaction. The students oriented their discussion to the agents instead of to each other and to the math problems. Much more experimentation seems necessary to design less invasive agent behaviors, even in theory. In addition, it may be necessary to study successful examples of accountable-talk prompts or interventions by skilled teachers, using the micro-analytic techniques of Conversation Analysis before trying to design software algorithms to replicate such expert behavior. In particular, we need to know how to effectively time interventions and how to adapt the linguistic structure of interventions to the on-going discourse.

(c) Designing effective CSCL interventions and introducing new technologies to scaffold interaction is a complex undertaking. It requires many cycles of iteration. The data analyzed here functions as an initial, pilot iteration. It was probably premature to run multiple conditions and to expect to see effects in subsequent testing of individual students. If anything, the VMT environment, the software agents and the accountable-talk prompts seem to have each done more to interfere with any possibility of collaborative discussion of biology than to promote it.

(d) The theory of accountable talk has intuitive appeal to scientifically well-trained, mature, rational adults, whose thinking is heavily influenced by explicit textual expression. However, theories relevant to CSCL stress the social, situated and linguistic nature of cognition (Stahl, 2012). To introduce accountable-talk moves into the highly situated, socially interactive text-chat interaction of school children will involve much more than providing canned prompts of the form used in the case study. It will require understanding the situated, sequential, social, interactional character of student chat, developing agents that can follow these subtle processes through real-time analysis of cryptic, ironic, juvenile postings and can formulate agent postings that engage in the co-construction of shared understanding. It is even possible that actually accomplishing that would exceed the theoretical possibilities of artificial intelligence to engage in intersubjectivity with humans. But before we can reasonably speculate on that, it seems important to understand the nature of effective knowledge-building discourse and productive accountable-talk prompting. Again, micro-analysis of prototypical examples of such interaction need to be carried out.

The point now is to take the lessons learned back to the drawing board for extensive redesign: (a) First, integrate more aspects of the biology experiment into the collaboration-support software environment by allowing the group to see the diffusion experiment results in a shared view and to embed its inquiry reasoning and its group conclusions in the VMT shared whiteboard. This can make better use of the collaboration tools of the software as a collaborative medium. (b) Second, develop the agents to follow the student discourse and to just intervene when needed. This involves real-time natural language processing of the student postings, which is a complex, subtle and situated skill, which may exceed the current state of the art. (c) Third, encourage collaboration among friends by letting the students know each other’s identities and having them work for a group product, rather than filling in individual worksheets and taking individual tests. This would transform the exercise from one focused on individual learning to collaborative knowledge building. (d) Fourth, figure out how accountable-talk prompts can be contextualized as part of natural verbal interaction. This will involve development of this approach beyond the current conceptualization of the technique.

Methodologically, this stage of research calls for observations of pilot studies in order to guide design in the various aspects of the project. A single case study, looking in detail at the interactions, can provide insight into what group-cognitive processes (Stahl, 2006a) take place empirically—in ways that quantitative comparisons of different conditions generally cannot. This can provide important correctives to what designers assumed would take place based on their best preconceptions. Statistical controlled comparisons and quantitative measures of changes in individual test results at this initial stage would likely produce results that would at best be confusing, but more likely be misleading when interpreted on the basis of researcher preconceptions of what transpires in student interaction. This response analysis from cycle one has tried to provide a quick case study that analyzes the actual interaction (among humans and agents) to reveal processes that are fundamental to human interaction under such conditions and are therefore likely to take place in other cases. It has tried to show how interaction analysis focused on the response structure of interaction can provide insight into group-cognitive processes and can indicate how experimental interventions do or do not support the group interaction. It contributed to guiding the redesign of this design-based research effort at this early stage of educational design.

Cycle Two of Design-Based Research

Due to the practicalities of conducting an experiment in public schools and due to the level of re-design called for by the lessons of the analysis of the first cycle of user testing, it took a year before the next cycle’s user testing could be conducted. In this section, I take a similar approach to seeing what a quick sequential analysis can yield with the data from the second cycle.

1. As described in Dyke, et al. (in (Suthers, Lund, Rosé, Teplovs, et al. 2013)), the new intervention had students working in four conditions. I decided that the revoicing condition would be the most interesting. I wanted to see the effect of the agent prompting students to revoice their chat postings.

2. I read through each of the 5 chat logs in the revoicing condition and I wrote down a couple sentences of my initial reaction to the quality of the interaction. I selected log F01 as the one that seemed to have the richest student interactions. I wanted to see how the agent postings—particularly revoicing prompts—affected the accountable talk of the students.

3. I rearranged the spreadsheet to have the postings of each participant in its own column. (The entire log of F01 is reproduced in Appendix B below.)

4. I next sketched the sequential response structure of the chat (see Figure 3).

Figure 3. Sequential response structure of chat F01.

5. A visual scan of the response structure shows that the tutor (first column) is still very dominant in the discourse. Of 50 postings, now only 10 are by the tutor agent, but most of them are lengthy, whereas many of the student postings are only a word (“yes”, “ok”, or the student’s name). Primarily, most of the student postings are in response—either directly or indirectly to the tutor. However, there are now several brief interactions among the students and even a couple of quite involved interactions (posts 27-33 and 41-50).

6. If we look at the content of the posts, we see that the whole discussion remains closely on-topic, following the agenda of the tutor. The tutor takes a strong instructionist teacher role. The students seem to accept this and respond to it much as they might to a classroom teacher. Although this was not the case in all of the chats, the one analyzed here seems quite successful in terms of student responses to the agent.

7. The student-to-student interaction (stimulated repeatedly by the tutor) progressed well. All the students participated (at least when prompted by the tutor), they discussed each other’s proposals and they all agreed to a group answer after each of the extended interactions. This may have been encouraged by the formulation of the task, which was presented as a group task, to come up with an explanation that everyone agreed with.

8. The focus on accountable talk was reduced to the idea of revoicing—at least in terms of the tutor programming in this chat. The tutor only posted two explicit revoicing moves: postings 39 and 43. In both of these, the tutor proposed an alternative (and more scientifically formal) way of describing a biological process and the student simply said, “yes” to the proposed revoicing. So the agent’s move did not significantly expand the accountable discourse of the students. However, for whatever reason, the students in this group did seem to act in a generally accountable way by including and respecting each other and by describing biological phenomena.

9. Although some of the other groups expressed the kind of confusion about what was going on generally, about the role of the tutor and about the intelligibility of the tutor’s postings that was rampant in the first year, the group in chat F01 did not. They accepted the tutor and responded to its postings as reasonable instructional statements. The timing of the tutor postings was also much improved. Student discussions were not often cut off by the tutor trying to follow a schedule. The tutor even seemed to react to student postings in ways the students could accept.

10. In conclusion, one cycle of re-design was adequate for eliminating the worst problems of agent intrusiveness, at least in the case of this one group, which I selected as most promising based on a skim of the logs. The ultimate goal of the theory of accountable talk is to have groups of students being accountable for their own discourse. It may be that at the level of ninth grade biology most students still need strong instructionist guidance and modeling before they can effectively adopt accountable talk practices in student-centered scientific discourse.

My quick analysis of a sample from the second cycle suggests that the major technical problems were adequately identified by my quick interaction analysis of the first cycle log and that they have been substantially addressed by the extensive re-design effort that it called for. The ground has now been laid for subsequent cycles exploring the complex issues of scaffolding group cognition among young students of science.

Issues for Further Multi-Vocal Analysis

a. Design-based research for designing technology

Too often, research reports are written to give the impression that a well-defined hypothesis was tested and that everything went according to plan, resulting in the reported findings. The widespread popularity of design-based research in educational technology design is a testament to the fact that research in real classrooms rarely simply follows a preconceived experimental plan. Rather, understanding about how to design effective educational technology emerges gradually from iterative attempts to refine prototypes in response to unanticipated issues that only become apparent in messy trials. The initial attempt to promote accountable talk in a biology classroom through the use of conversational agents ran into myriad circumstances that modified the ideal experimental plan. The variety of analyses of the experiments collected in  (Suthers, Lund, Rosé, Teplovs, et al. 2013) reported many of the problems: Dyke, et al. listed some of these. Cress & Kimmerle argued that the experimental situation, as actually implemented, did not support the social aspects of interaction that are so important to the students. The preceding sequential interaction analysis of one group’s chat log from cycle one indicated that the agents were not very “conversational” in the resultant situation. Howley, et al. further investigated the social, linguistic and sequential structure of the chat interactions, both to see how the agents and students positioned each other as knowledge-building partners and to track the temporal unfolding of the chats. These analyses begin to inform the design of the software agents and of the educational intervention generally, suggesting approaches to be tried in cycle two and in subsequent iterations. Other types of analysis can no doubt offer additional suggestions for redesigning features of this multi-dimensional intervention.

b. Scripting of the software agents and situated interaction

Just as the experiment as a whole is situated amid the complex constraints on conducting experiments in typical public school classrooms, so the postings of the agent and students are situated in the unpredictable and subtle constraints of the social and linguistic interaction that unfolds in the chat room. In particular, each posting must make sense as following previous postings. Furthermore, when someone has difficulties making sense of the sequence of postings in this context then there is a need for “repair” processes. The sensitivity of a posting to preceding chat posts motivated my decision to look at the adjacency-pair structure, as a key indicator of the extent to which posts—particularly those of the agent—were meaningfully related to preceding and subsequent posts by students. My analysis revealed that agent posts in cycle one were not adequately situated in this sense. Furthermore, the agent showed no ability (or even inclination) to repair problems of meaning making when they arose.

In a chapter I wrote for a book on scripting (Stahl, 2006b), I cautioned that scripts should be conceptualized as situated resources rather than implementable plans for action. For instance, rather than scripting the agent to instruct the students to watch the video at precisely 8 minutes 15 seconds after the start of the chat, the agent should try to find an appropriate moment roughly 8 or 9 minutes into the chat for doing this, depending on what the students are doing at that point. I cited Suchman’s (1987, p. 181) recommendation that computer support compensate for its limitations by:

1) Extending its access to the actions and circumstances of the user;

2) Clarifying for the user the limits of the computer’s access to the users’ rich interactional resources; and

3) Providing a wider array of alternative resources, particularly to help the users respond to unforeseen breakdowns.

Suchman was talking about the design of help systems for large copying machines. Compared to that, the conversational agents have the significant advantage of having access to all actions in the chat room—they have the same access that the students have to each other’s actions. However, the agents have been programmed to project an anthropomorphic personality, pretending that they have meaning-making and language-understanding capabilities far in excess of what they can actually do. Suchman warned explicitly against doing this because it inevitably confuses the relationships and leads to misunderstandings and frustrations. As Cress & Kimmerly emphasized, a classroom is a highly social setting for the students, and introducing a new social partner with no social skills may not be an effective approach. Finally, the agent is designed to perform multiple roles, scripting the macro-level phases of work as well as the micro-level accountable-talk moves. When the students reject the agent, they are left to their own resources.

c. Sequential interaction analysis of small groups

While the design-based-research approach is often recommended for educational technology, this approach does not generally specify a method for analyzing the results of trials. In the past, I have suggested adapting Conversation Analysis to provide insight into how teachers and students are actually making use of a prototype, rather than quickly counting surface features of interactions or coding utterances based on the designer’s or researcher’s conceptualization of the intervention. Although we have found data sessions based on VMT sessions to provide quite useful design feedback in a matter of hours, many researchers claim that qualitative analysis is too time consuming to give timely feedback. That is why I tried in this essay to see how much insight into central problems of an intervention could be gleaned from a quick adjacency-pair analysis of one typical chat session.

For the data from cycle one, I skimmed through the chats and got a sense of the problematic nature of the sessions, much like the feelings that the authors of the related chapters expressed. I selected a chat session that seemed to have relatively clear examples of the problems. Specifically, I selected a session in the “indirect” condition, which was the condition of greatest interest for the experiment. I then sketched an initial version of Figure 1. Based on the visual appearance of the figure and the content of the connected adjacency pairs of posts, I drafted an initial version of this essay, arguing for the need for changes to the agents and to the intervention in subsequent iterations. During a data session with some of the other chapter authors, refining Figure 1 and our understanding of what took place interactionally in the chat, we agreed on directions for further analysis and experimentation. In this way, the sequential interaction analysis with the graph of adjacency pairs provided a quick sense of where major issues lay, which needed to be addressed in re-design. Thus, it played a role similar to so-called “discount methods” in human-computer interaction, where designers need fast feedback at low cost. Not counting the time to write up the report in this essay, the whole analysis took a matter of a couple of hours. Quantitative analyses took much longer. As it turned out, it was a year before another cycle of trial could take place. So the sequential analysis approach was much quicker than necessary.

d. Accountable talk and off-task student practices

Throughout the history of CSCL, researchers have conducted educational interventions with expectations that the students would engage in knowledge building, inquiry, transactivity, collaborative learning, warranted argumentation and other lofty conceptions of scientific intellectual discourse. These expectations were operationalized so that research assistants could reliably interpret student utterances as falling into different coding categories. Inevitably, few utterances could be coded in the highest categories; a large percentage fell outside the scheme, and they were called “off topic.”

To conclude this essay, I would like to raise the ethnomethodological question: what are the students doing when they are off topic? If they do not do being-a-student by engaging in recognizably accountable talk, how do they do it? Is it due to some personal characteristics of these students that they engage in “cheating” rather than in following the instructions of the agent? Perhaps if we break free of the conceptualizations imposed by the experiment’s world-view, we can understand the off-topic behaviors in a positive light. As Cress & Kimmerle suggest, the teenage students are engaged in social activity with one another. Their social relations support their discussions of curricular topics and their talk in the classroom feeds into their social relations. Any arrangements that interfere with their social relations—such as hiding everyone’s identities—will interfere with the possibility of any kind of interaction and will generate attempts to repair the problem. In addition to the social practices involved in relating with their peers, the students are involved in established classroom practices, oriented largely around earning good grades. While the researchers were looking for accountable talk in the details of interaction, the students were oriented toward completing the individual worksheets and taking tests. Thus, many of the early interactions graphed in Figure 1 involved finding out the identities of the chat room occupants and many of the later interactions involved filling out the worksheets. While the exchange of information needed for the worksheets did not fit the profile of accountable talk, it did contribute to a rather efficient sharing of worksheet answers. Although it did not meet the ideals of collaborative knowledge building or group cognition, the three students did effectively divide the tasks and share the results, in the classic form of cooperation that most people tend to use. If the task had been to fill out a group worksheet, neither the students nor the researchers would have viewed this procedure as cheating. Classrooms are not blank slates waiting for a researcher’s conceptualizations; they are richly structured with established practices that can only be discovered by analysis of interventions in the wild.

Appendix A: Chat Log of Group C01 from Cycle One

time

Tutor

S027

S034

S041

1

11.54.45

2

11.55.35

3

11.55.41

4

11.55.51

5

11.56.01

Hi all! Today, each of you will be both the instructor and the student. Each of you will be responsible for looking for opportunities for a specific discussion strategy to be used in the conversation.

6

11.56.06

Lets get started by introducing ourselves. I am Alex.

7

11.57.04

S034,S041,S027... i didn't get your names yet.

8

11.57.16

my name is [S034]

9

11.57.16

Hi S034 you are the Revoicer. When an explanation or idea is given which would benefit from revoicing, please encourage one of your team mates to do so, for example, by asking them to explain what was said in their own words.

10

11.57.24

S041, you are the Challenger. When you see a statement being made (whether it is right or wrong) which would benefit from being challenged, please encourage one of your team mates to do so, for example, by asking them whether they agree or disagree with what was said, and why.

11

11.57.25

[S041]

12

11.57.31

Ok... So the Explainer is S027. When you see a place where it would be good to get more explanation, you can encourage one of your team mates to do that, for example, by inviting that person to add on to what the other student has already said.

13

11.57.37

You should probably all mark an X opposite your role in the book, so you won't forget.

14

11.57.43

Its nice to meet you all. :) Please make sure you have understood what you are responsible for.

15

11.58.00

In condition A, we placed a glucose solution inside the cell model and immerse it in distilled water. In condition B, we did the opposite.

16

11.58.05

Please discuss what you predict will happen in these two conditions.

17

11.59.29

who is 27

18

12.00.06

[S027]

19

12.00.46

what do you think'ds going to happen?

20

12.01.02

In condition C, we made a modification to condition A. We replace the glucose solution with a starch suspension. In order to detect the presence of starch, we replace the distilled water with Lugol's Iodine solution.

21

12.01.07

You should now move on to discussing what will happen in Condition C and your explanation for this change.

22

12.01.14

the world is going to end in 2012

23

12.01.21

S027, now would be a good time to ask S034 to build on what S041 is saying.

24

12.02.00

im so confused!

25

12.02.08

034, would you like to build on to what 041 is saying? and me too!

26

12.02.32

When you are in agreement, write down your predictions and explanations for Conditions A, B and C on your worksheet.

27

12.02.42

who is 34?

28

12.02.53

[S034]

29

12.03.00

You are now going to watch a video showing the cell in Conditions A, B and C.

30

12.03.04

ooh. hi

31

12.03.05

As you watch the video, write down your observations on your worksheet.

32

12.03.11

Go to the Videos folder on the Desktop, and watch the video which is there.

33

12.03.24

idk what to do.

34

12.03.27

someone predict something

35

12.03.56

THE WORLD IS GOING TO END IN 2012 !

36

12.04.05

oh i get it! what do you think is going to happen when you put the glucose in the distilled water?

37

12.04.22

i think it will turn a different color.

38

12.04.29

[S041]?

39

12.04.45

ok dude what 27 said

40

12.05.04

in your own words

41

12.05.10

?!?!?!

42

12.05.26

you have to be more specific 27

43

12.05.43

ur one to talk!!!

44

12.05.46

the answer to number one : the distilled water will move into the internal envorment of glucose solution

45

12.06.36

kk

46

12.06.45

and the explaination is glucose is denser

47

12.06.57

huh?

48

12.07.05

just write what she[S034] said.

49

12.07.20

for the explination

50

12.07.33

#2 the distilled water will move into the external enviroment

51

12.08.14

watch the video

52

12.08.38

Is everyone back?

53

12.08.45

we have to do all the prediction , thats what NACHO said

54

12.08.48

ok, but where do i click to watch it?

55

12.09.00

Now discuss what you observed in the video about conditions A and B. Compare what you observed to your predictions.

56

12.09.40

nevermind i found it.

57

12.10.01

#3 the water will move into the starch because starch absorbs it

58

12.10.08

S027, now would be a good time to ask S041 to build on what S034 is saying.

59

12.10.33

[S034] are these #'s all of the predictions or explanations?

60

12.11.51

[S034]?

61

12.12.01

yeaaaa

62

12.12.02

You should now move on to discussing what you observed in the video about condition C. Compare what you observed to your predictions. How is this different from what happened in condition A?

63

12.12.25

#3, is that the prediction?

64

12.12.41

Lets spend the rest of the time we have discussing your observations from all the three conditions.

65

12.12.47

Discuss explanations of what you observed in each condition and make sure you fill in your explanations on your worksheets.

66

12.13.00

what is explanation for #2

67

12.13.07

idk.

68

12.13.07

its both, the first part is the prediction the second part is the explanation

69

12.13.13

@S027, can you ask S041 to build on what S034 is saying.

70

12.13.34

i want to shoot myself in the foot -.-

71

12.13.50

ikr.

72

12.14.00

this is so stupid.

73

12.14.11

this would be so much easier just in a group

74

12.14.19

yep yep

75

12.14.23

@S027, can you ask S041 to build on what S034 is saying.

76

12.14.25

you dont have the balls to do it

77

12.14.36

78

12.14.38

Ok, I gotta go.

79

12.14.43

It was nice talking to you all. :-)

80

12.14.55

our tutor left us!!!

81

12.15.10

D:

82

12.15.23

she is a baad tutor!!!

83

12.15.44

im pretty sure its a he

84

12.16.09

really? haah! oh well. idc. he/she/it

85

12.16.31

i ddnt even watch the whole video

86

12.16.49

i watched most of it

87

12.16.58

do you know what happened?

88

12.17.13

i just wrote the observations

89

12.17.42

oh. what was it after 1 hour for condition A?

90

12.19.17

[S041], did you get everything so far?

91

12.19.24

condition a: 620 glucose inside; 635 glucose inside; 630 glucose inside; explanation: the membrane will hold stuff for a certain amount of time

92

12.20.05

condition b: 540 glucose outside; 525 glucose outside; 525 glucose outside; explanation: the glucose was not being absorbed by the water

93

12.20.10

i think .

94

12.21.45

k, i'll fast forward and try to find the last one, when im done writing the condition b.

95

12.21.47

you mean i know !

96

12.22.02

yea, sure i do, whatever. hahahahahha

97

12.22.15

lol

98

12.22.19

lol what is C

99

12.22.27

i dont even think there was one

100

12.22.33

i sd i'd try to find it if i could.

101

12.22.51

there is & ok

102

12.23.04

aaah there is a quiz!!!!!!!!!!!!!

103

12.23.19

im gonna fail!!!!!!!!!!!!!!

104

12.23.57

stfu !

105

12.23.59

HAPPY BIRTHDAAAY [S041] :D :D :D :D :D :D :D :D :D

106

12.24.10

lol thsnks

107

12.24.13

is it ur birthday?

108

12.24.20

[blank line]

109

12.24.23

[blank line]

110

12.24.23

[blank line]

111

12.24.23

[blank line]

112

12.24.23

[blank line]

113

12.24.23

[blank line]

114

12.24.24

...

115

12.24.25

116

12.24.35

bye bye guys.

117

12.25.01

118

12.25.11

119

12.28.34

Appendix B: Chat Log of Group F01 from Cycle Two

No.

Time

Tutor

539CN5

504CN1

550CN5

1

13.57

Hi, my name is Sage. I'll be working with you in this activity. Take a minute to introduce yourselves.

2

14.42

I won't tell you if you're right or wrong, but I'll give feedback as you discuss things as a team.

3

14.49

hi dajha!

4

14.58

Let's look at a setup which is the opposite of the one we saw earlier - the iodine is now inside and the starch is outside.

5

15.24

<< On the whiteboard, you can see a picture of this new setup.

6

15.29

Vinny<3

7

15.38

<< Here's what happened in this setup over time.

8

15.59

Notice how the inside and outside environments changed. Why did these changes happen?

9

16.26

the iodine left the bag

10

16.31

Okay, 504CN1 - why would how we use indicators be relevant?

11

17.39

the iodine was never in the bag the iodine went into the

12

17.48

bag

13

17.51

the weight is decreasing and and the starch water changed colors

14

17.54

Any input from 539CN5?

15

18.59

if you loook at the diogeam it shows the big getting liter because the iodine left the bag

16

19.21

agreed...

17

19.26

agreed...

18

19.32

Okay, team - in a moment we'll move to another set of experiments.

19

19.56

For these experiments, we'll use the same membrane, but with pure water on the outside and a glucose solution on the inside.

20

20.27

Discuss the differences between glucose and starch molecules.

21

20.46

glucose molecules weigh less

22

21.19

We'll use the tube weight as an indicator for water movement.

23

21.23

We'll also use *glucose test strips* to detect glucose in both environments.

24

21.37

<< On the whiteboard, you can see what happened in this setup over time.

25

22.01

The results with glucose are different than what you saw with starch - as a team, discuss and agree upon an explanation for these results.

26

22.50

the weight of the gluclose soltuion increased over the time period..

27

22.59

the water has no suger in it so the strip will stay yellow. the gucose solution has duger so the strip will turn green

28

23.03

i think the glucpse molecules left the bag and went into the water in the process of that water molecules entered the bag

29

23.41

once again......agreed with dajha ... hahaBACKSLASH

30

24.00

always a smart thing to do :)

31

24.41

but also i agree with vicky too ;)

32

24.50

:)

33

25.32

and over time glucose cane out of the bag and got into the water and that is why after 1 hour and 24 hours the strip that was in the water started to change.

34

25.38

Okay, everyone - let's move on to the last experiment.

35

26.00

In this setup we have glucose solution on the outside, pure water on the inside.

36

26.23

What changes do you think you'll see in this condition? Agree on a prediction for what you'll observe over time, *and* an explanation that everyone understands.

37

26.50

the water has not suger so the strip is yellow

38

27.36

and the glucose has suger and the strip is green

39

27.41

So is 539CN5 saying "a glucose test strip changes color when glucose is there?"

40

27.51

yes

41

28.06

i think the gluclose test strip will slowly change to a bright yellow color it is on the weighing scale. also i think the bag of distilled water will weigh more because over time of sitting in the gloclose you will get some glucose molecules in the

42

28.06

i don't think the sugar molecules will get in side the membrane i think they will be to big i think the water molecules will come out of the membrane and go into a concentrated equal enviorment

43

28.13

Would another way to say that be "molecules will move to the area of lower concentration if they can?"

44

28.38

yes

45

29.41

agreed........with dajha AGAIN!.

46

29.51

i agree to

47

30.02

Make sure each of you is clear on your team's prediction and explanation, and write it in your worksheet.

48

30.09

WE ALL AGREE!:)

49

30.45

All right, we're wrapping up. Thanks, team!

50

30.53

just to restate... the sugar molecules will not go into the membrane the distellied water will come out and the test strip will turn yellow

7. Coding Scheme for Sequential Discourse

The coding scheme in Table 1 was developed based on the analysis of adjacency pairs in (Schegloff 2007). It was applied to the log of POW Session 4 of Team B, conducted during VMT SpringFest 2006 (see link to sample at end of this section). This scheme was discussed in (Stahl 2011b; 2011c; 2011d; 2011f). The same log was analyzed in various other VMT publications, including some collected in (Stahl 2009c).

The basic idea is that discourse is built up hierarchically: from (g) various indexical references (e.g., “that”) in (f) textual utterances (e.g., chat postings) contributing to (e) adjacency pairs (e.g., question/answer). Sequences of adjacency pairs (including extensions and recursive embeddings) form (d) discourse moves. The moves contribute to (c) conversational topics (that are opened and closed). Topics are included in larger (b) group events, which make up (a) the entire session (e.g., Session 4 of Team B).

In Table 1, examples of (c) through (f) are included under those headings.

Schegloff’s symbols are listed for use in coding utterances in adjacency pairs. For each symbol, its meaning is given.

The list contains some common FPPs (first pair parts) of adjacency pairs, with their corresponding SPPs (second pair parts).

In Table 2, the four typical Turn Constructional Units are listed, with their common constituent utterances. These are discussed in the next chapter, based on (Zemel & Stahl 2010).

Table 1. Coding Scheme.

VMT 2006

Spring Fest

Team B

Session 4

(c)  Conversa-tional Topic

(d) Discourse Move

(e) Adjacency Pair

(f) Textual Utterance

(g) Indexical Reference

(b) Group Event

(a) Temporal Session

transition

anticipate

announcement

announce; acknowledge; follow up

opening

close

compliment

compliment; acknowledge

Schegloff symbols

meaning of symbol

FPP

SPP

technical

open

explanation

explain; acknowledge; follow up

F, Fbase

first pair part (base FPP)

question / ask

answer

feedback

return to

greeting

greet; return greeting; farewell; return farewell

S, Sbase

second pair part (base SPP)

request

grant

select

introduce new approach

joke

joke; laugh; respond to joke; return laughter

Fpre

pre-sequence FPP

offer *

reject

review

terminate use of approach

proposal

propose; acknowledge; ratify; reject; follow up

Spre

pre-sequence SPP

invite *

accept

wiki

question

question; answer; agree; disagree; follow up

Fins

insert sequence FPP

announce

decline

equation

request

request; acknowledge; accept; reject; follow up

Sins

insert sequence SPP

greet

agree

indexing

suggestion

suggest; acknowledge; ratify; reject; follow up

SCT

sequence closing third

farewell

disagree

compare

directive

direct; acknowledge; receive; reject; follow up; report

Fpost

post sequence FPP

notice *

acknowledge

strategy

evaluation

evaluate; acknowledge; agree; disagree

Spost

post sequence SPP

promise *

contest

wrong

commentary

comment; acknowledge; agree; disagree

+S

preferred SPP

tell *

tease

celebrate

clarification

clarify; acknowledge

PCM

post completion musing

complain *

finess

facilitator

repair

self-correct; question; clarify; acknowledge

propose

comply

follow-up

failed X     escalated X

ni

non-interactive or system message

suggest

perform

closing

+ (continuation)

+ (continue)

+

continuation

request

ratify

construction

direct

follow up

narrative

joke

receive

reflection

laugh

report

compliment

assess

explain

return

clarify

clarify

repair

evaluate

comment

An example of the codes in Table 1 applied to SpringFest 2013, Group B, Session 4 can be found in www.GerryStahl.net/elibrary/science/codes4b.xls . The first part of that session is given below in Table 3. This log includes the excerpts analyzed in Chapter 26 of the Studying VMT book and in (Stahl, G., Zemel, A., & Koschmann, T. (2009). Repairing indexicality in virtual math teams. In the proceedings of the International Conference on Computers and Education (ICCE 2009). Hong Kong, China. Web: http://GerryStahl.net/pub/icce2009.pdf.)

Table 2. Coding Scheme.

TCU: turn constructional unit

Zemel & Stahl NCA 2010 paper & spreadsheet -- actions

PRU

proposal, ratification, follow up

SRU

suggestion, ratification, follow up

RAU

request, acknowledge, follow up

DCR

directive, receipt, follow up, report

An analysis of the codes in Table 2 can be found in the following chapter, adapted from (Zemel & Stahl 2010).

Table 3. Coding of first part of SpringFest 2013, Group B, Session 4.

8. Coordinating Collaborative Action in Online Math Problem-Solving

Alan Zemel, Gerry Stahl

Collaboration is understood as a central theme in CSCL studies. In this paper, collaboration is explored in terms of the ways that interactants in a CSCL setting transition from one activity to a next. Rather than simply initiate a next activity upon completion of a current activity, students using the Virtual Math Teams (VMT) environment initiated next-sequence selection sequences. The way actors transition from a completed activity to a next activity demonstrates their orientation to collaboration. Most frequently, sequence transitions were accomplished in unmarked ways by simply initiating a next sequence. However, there were numerous circumstances in which actors used explicit next-sequence selection procedures. Four such procedures deployed in VMT session are examined in this paper. Three of the four elicited the participation of other actors in the next-sequence selection process, making the choice of a next activity a collaborative matter.

Collaboration and the Coordination of Action

S

ocial interaction is comprised of sequences of actions (Schegloff & Sacks, 1973; Sacks, 1992; Schegloff, 2007). When one interaction sequence is completed, a transition to another sequence becomes relevant. How the transition between sequences is accomplished is both consequential for and a consequence of the way participants conduct themselves within sequences. In face-to-face interactions, turn-taking mechanisms and next-actor selection procedures regulate the manner in which sequential interactions are conducted and the manner in which transitions between sequences of interaction are achieved. Since the affordances of online chat environments are different from those in face-to-face interaction, intra-sequential conduct and transitions between sequences will be different as well.

Collaboration is foundational to CSCL as an essential component in the production of shared knowledge. One way that groups do ‘being collaborative’ involves the way they organize themselves to accomplish learning tasks (Cakir, Zemel & Stahl, 2009; Sarmiento & Stahl, 2008). There are a number of different ways that CSCL construes collaboration. According to Lipponen (1970, p. 73), collaboration can be considered “a special form of interaction or as a process of participation.” Rochelle and Teasley (1995) define collaboration as “a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” (p. 70, emphasis original). Alternatively, “[c]ollaboration can be defined as a process of participating in knowledge communities” (Lipponen, 2002, p. 73, emphasis original). The challenge for CSCL is to specify in detail how such coordination of activity is achieved, how shared conceptions are accomplished, how participation in knowledge communities is done and of what that participation consists. One way to do this is to describe in detail the way actors coordinate and manage the sequence of their activities in CSCL environments. From a CSCL perspective, the problem of initiating a next sequence raises interesting issues about collaboration, decision-making and the way learning’s work is organized and accomplished. When, upon completion of a current sequence, the initiation of a next sequence becomes a decidable matter for the assembled participants in a CSCL setting, the process by which actors participate in the choice of a next sequence is evidence of and informed by their commitment to collaboration.

The transition from one sequence to a next is a concern both to CSCL and to Conversation Analysis (CA). In CA, such transitions have been discussed in terms of the organization of long sequences (Sacks, 1992) or sequences of sequences (Schegloff 2007). While the sequential organization of action has been the principal object of conversation-analytic research, the detailed interactional procedures by which actors initiate next-sequences of action upon the completion of a current-action sequence is an underdeveloped arena of investigation. CSCL environments provide CA analysts with the opportunity to examine the manner by which actors transition from one activity to another (Zemel, 2009). CA provides CSCL with the opportunity to understand the procedural organization of collaboration as a methodical set of shared procedures of interaction to which participants orient in the conduct of their affairs.

In this paper, CA methods are used to identify four procedures by which transitions between sequences of actions are accomplished. One of these procedures is an example of what might be seen as a non-collaborative method of next-sequence selection. Three of these procedures can be considered collaborative in the way they organize the participation of actors in the accomplishment of next-sequence selection. These four examples provide a basis for making certain preliminary observations regarding what is collaborative about collaboration.

Data and Methods

The data consist of time-stamped chat logs and whiteboard displays of math problem-solving sessions among middle-school students. Specific excerpts were taken from the chat sessions of Team B in the VMT Spring Fest 2006. This event featured four teams (Teams A through D) who participated in four consecutive sessions over a two-week period. The chats were sponsored and conducted by the Math Forum of Drexel University as part of its participation in the Virtual Math Teams (VMT) research project, an NSF funded project at Drexel University (Stahl, 2009). Analysis was conducted using complete logs of the chat sessions in conjunction with a software “player” that affords the possibility of reproducing the display of VMT activity as delivered from the servers. The player displays the integrated and coordinated use of chat and whiteboard technologies incorporated into the VMT system. Additionally, this player software permits various “speeds” of playback as well as the ability to step through actions one at a time.

CA is the specific analytical methodology applied to the data. Sacks (1992) and his students developed this analytical approach in the 1960’s and 1970’s. It shares a phenomenological orientation with ethnomethodology, presuming that the analytical task is to identify and describe in detail the shared methods and procedures by which people engage in interaction. CA is principally concerned with sequence in talk-in-interaction. A central assumption of CA is that when people interact, their actions occur as a series of related, orderly and ordered actions. The notion of conditional relevance provides the link between one action and a next action in face-to-face interaction. A series of ordered and related actions, linked by the fact that a first action makes conditionally relevant a second action of a particular sort, are identified as action sequences in CA.

As has been already noted the principal analytical task of CA is to describe the organization of action sequences. Minimally, sequences are pairings of actions where a first action makes conditionally relevant the occurrence of a subsequent action (Hutchby, 2001, p. 66). A great deal of analytical attention has been given to describing various kinds of sequential phenomena, for example question-answer pairs, telephone openings, report-assessment pairs, etc., (Schegloff & Sacks, 1973; Schegloff, 1968; Heritage, 2002; Pomerantz, 1984). Relatively little work has been done to describe how actors transition between sequences. Sacks (1992) and Schegloff (2007) take up discussion of long sequences but this remains a relatively underdeveloped area of investigation in CA studies. One phenomenon we have identified in our data is a set of sequentially organized procedures by which multiple actors explicitly effect a transition between sequences. These procedures form the object of analytical interest for this study.

Transitions between Sequences

A central assumption of CA is that when people interact, their actions occur in the form of a sequence of related, orderly and ordered actions. When a sequence is completed, interactants face the problem of “what to do next.” Consider as an example the following question-answer adjacency-pair sequence:

Table 1: Adjacency-Pair Completion.

Line

Name

Post

dd-mm-yyyy

Time

217

Quicksilver

Did you guys discuss the problem like it said to?

09.05.2006

07.08.16

218

Aznx

Yeah.

09.05.2006

07.08.21

Upon receipt of the response at line 218, the issue for chat participants is to figure out (a) what can follow as a next posting at line 219 following the apparent completion of the question-answer sequence and (b) who will perform that next action? What happens next is a complex matter that depends on the nature of the question, the answer, what has happened in the interaction up to that point, the task at hand, etc.

How interactants elect to proceed may be constrained in various ways that result from and instantiate relationships among interactants, participants’ relationship to the business at hand, institutional affordances, the affordances of the interactional modalities deployed, etc. In this example, the response proffered does not make conditionally relevant any particular kind of next action, making it possible for any participant to potentially initiate a new sequence of some sort.

If we consider an activity to be something like a coherent sequence of action sequences, then there may be certain, more loosely organized constraints on (a) what sequences can be performed as part of an activity and (b) the sequence of those sequences by which the activity is constituted in the first place. According to Schegloff (1990), the coherence of long sequences is a structural feature of the way they are opened, expanded and closed (p. 73). In one study of long sequences in chat interactions, Zemel et al. (2007, p. 407) write: “Among the regularities observed and studied by conversation analysts are the ways that long sequences begin and end. Participants in conversations engage in recognizable boundary-producing activities to which participants orient and by which participants initiate conversations and bring them to a close. These are referred to as openings and closings (Schegloff, 1968; Schegloff & Sacks, 1973). These activities are used to display that some activity in which participants had been engaged is completed or suspended and another is starting. As such, they serve to mark interactional boundaries between long(er) sequences in an ongoing interaction. This permits participants wide ranging opportunities to manage, regulate and build their interaction to become coherent long sequences of recognizable activity.”

There are a number of kinds of sequence selection sequences that can be distinguished by the way they are initiated. In this section, four such next-sequence selection sequences are considered:

- Proposal-ratification-uptake (PRU) sequences,

- Yes-no query request sequences,

- Suggestion-initiated selection sequences, and

- Directive-compliance-report (DCR) sequences.

PRU Next-Sequence Selection Sequences

One particular sequence of actions that marks an interactional boundary between activities construed as long(er) sequences, or sequences of sequences is the proposal-ratification-uptake (PRU) sequence in which a formulation of a next activity is put forward for ratification and uptake. Specifically, interactants in problem-solving chats can select a next sequence to take up by:

1. Proposing a next sequence or activity for others to ratify and take up,

2. Ratifying the proposed next sequence, and

3. Taking up or initiating the proposed next sequence.

PRU sequences are often deployed to do work other than next-sequence selection. For example, in problem-solving work, an actor will propose a possible solution in a way that calls on recipients to ratify its correctness and to accept the proposed solution as the solution endorsed by the collectivity.

Even though PRU sequences are available to accomplish a variety of interactional outcomes, they display an orientation toward recipient participation in the ratification and uptake of the proposed matter. It is this orientation toward recipient participation that distinguishes the PRU sequence as a method for accomplishing next-sequence selection because it may also be a way by which participants can demonstrably display their collaboration. Next-sequence selection is occasioned in various ways by the completion of a prior sequence or activity and by members’ achieved understandings of the ways they are entitled and expected to participate in the ongoing interaction. The proposal-ratification-uptake organization of next-sequence selection sequences is consequential for the way they project how actors in a group are to participate in making decisions regarding the subsequent actions of the group, and when relevant, how actors are to participate in the proposed sequence or activity taken up by the group.

Even though a proposal is put forward, there is no necessary requirement that a proposal always lead to ratification and uptake, even though ratification and uptake are made conditionally relevant by the production of a proposal. A proposal may be rejected or ignored, an alternative proposal may be put forward, etc. Also, ratification may not be done explicitly but may be achieved implicitly through uptake of a proposed next activity.

“Let x y”-initiated PRU Next-Sequence Selection Sequences

One kind of PRU initiation is constructed in the form “Let x y” where x is the subject of the transitive verb to let, and y is a proposed next action. These are frequently produced as “Let’s y” or “Let me y.” Examples of this kind of sequence selection initiation include:

Table 2: “Let x y”-initiated PRU Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

53

Aznx

Let's start this thing.

09.05.2006

06.32.10

Table 3: “Let x y”-initiated PRU Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

393

Quicksilver

Let's go back to original idea: the flat face

10.05.2006

07.21.02

394

Quicksilver

then we can try and get this from that

10.05.2006

07.21.08

Proposal initiators of this sort are constructed with two components. The first component is the transitive verb “let” in an imperative form. The second component is a verb phrase that projects a proposed next action or activity. For example “Let’s” + “start this thing” (Table 2) or “Let’s” + “go back to the original idea” (Table 3). The particular construction of the first component using the verb to let includes a subject portion as in “let me” or “let us” or “let’s.” When presented in the first person plural form, viz. “let’s y,” all recipients, including the actor posting the proposal, are addressed as recipients of the proposal and are thus made accountable for (a) ratification of y as the next activity and (b) the uptake of y. This construction is routinely treated as a way of putting forward the object of the proposal, i.e. the proposal next action, to a set of recipients for their ratification. Furthermore, since it is addressed to the collectivity, it implies that all recipients are included as participants in the uptake of the next matter or the conduct of a subsequent projected action. In Table 3, recipients of the PRU next-sequence selection sequence are identified as persons who would ratify this next activity and take up this activity upon ratification.

A typical example of a PRU sequence used for next-sequence selection is shown below in Table 4.

Table 4: “Let x y”-initiated PRU Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

331

Aznx

So let's brainstorm through some problems that we think are challenging.

10.05.2006

07.09.33

332

Quicksilver

yes...new topic

10.05.2006

07.09.40

333

bwang8

Ok

10.05.2006

07.09.42

334

Quicksilver

3-d figures?

10.05.2006

07.10.20

335

Aznx

I think we should discuss on the different methods.

10.05.2006

07.11.06

336

Aznx

So that we can easily apply our thoughts quickly when seeing a problem.

10.05.2006

07.11.24

337

Quicksilver

Yes....but we must find a question or problem to investigate

10.05.2006

07.11.30

338

Aznx

Yeah.

10.05.2006

07.11.37

339

Aznx

I think we should start off with a conjecture, that we need to prove.

10.05.2006

07.11.50

This Table displays the basic organization of next-sequence selection PRU sequences which displays the following three-part organization:

1. A proposal (line 331),

2. Ratifications by recipients (lines 332 and 333) and

3. Uptake of the proposed activity (lines 334 through 339).

Ratification of a “let x y” proposal may be explicit (as in Table 4) or implicit, as in the following excerpt:

Table 5: Implicit Ratification Example.

Line

Name

Post

dd-mm-yyyy

Time

884

Aznx

Well, let's look at their problem.

16.05.2006

07.29.00

885

bwang8

open browser

16.05.2006

07.29.05

886

bwang8

and click on the link

16.05.2006

07.29.13

Here we see that Aznx proposes that all the participants look at a problem (line 884). Instead of giving an explicit ratification in line 885, Bwang8 implicitly ratifies the proposal by providing instructions for how to accomplish the proposed action.

Ratification of a “let x y” may not always be forthcoming which may cause a proposal to be dropped. In the following example, Aznx proposes that participants solve the formula at line 808. This is not taken up in subsequent postings and Aznx does not recycle his proposal.

Line

Name

Post

dd-mm-yyyy

Time

808

Aznx

Let's sovle it. :P

16.05.2006

07.16.10

809

Quicksilver

Excuse my poor drawings

16.05.2006

07.16.20

810

Gerry

What does the feedback say about the difference this would make?

16.05.2006

07.16.47

811

Aznx

There would be a similar sharing in between the layers.

16.05.2006

07.17.17

812

Aznx

So the number would technically be bigger.

16.05.2006

07.17.26

Table 6: Dropped Proposal Example.

Alternatively, failure to ratify a proposal may cause an interactional escalation to a ratification question, as in the following excerpt:

Table 7: Escalation Example.

Line

Name

Post

dd-mm-yyyy

Time

393

Quicksilver

Let's go back to original idea: the flat face

10.05.2006

07.21.02

394

Quicksilver

then we can try and get this from that

10.05.2006

07.21.08

395

Quicksilver

So we are going back to the flat faced one? Agree?

10.05.2006

07.22.06

396

Aznx

Agree.

10.05.2006

07.22.39

397

bwang8

we can first figure out the bottom level

10.05.2006

07.22.46

In this instance, Quicksilver initiates a PRU sequence with a “let x y” proposal at line 393 and 394. There is no ratification or uptake following the presentation of the proposal. Almost a full minute goes by without a response. This duration gives respondents ample opportunity to ratify or reject the proposal, or proffer an alternative proposal. When no response of any sort is forthcoming, Quicksilver escalates from a “let x y” proposal to a direct request for agreement in the form of a question in line 395. After half a minute, agreement is proffered by Aznx in line 396 and taken up by Bwang8 in line 397.

Query-prefaced “Let x y”-initiated PRU Next-Sequence Selection Sequences

A variation on the “let x y” PRU next-sequence selection sequence involves soliciting a proposal in the first place. In this way, the proposal is presented as a response to a question about what should be a next activity for actors to take up. In the following excerpt, Aznx at line 502 explicitly asks, “So what should we do next?” What follows at line 503 is an abbreviated version of the “let x y” PRU next-sequence selection sequence. Quicksilver responds with “[Let’s] Continue and see if we find any patterns.”

Table 8: Query-prefaced “Let x y”-initiated PRU Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

502

Aznx

So what should we do next?

10.05.2006

07.44.27

503

Quicksilver

Continue and see if we find any patterns

10.05.2006

07.44.42

504

bwang8

i think i got the equation for the middle sticks

10.05.2006

07.44.47

505

Quicksilver

All right...lets see

10.05.2006

07.44.55

506

bwang8

now we know the n by n blocks on the bottom

10.05.2006

07.45.43

507

Aznx

Yeah it seems so.

10.05.2006

07.45.50

508

Quicksilver

yes.

10.05.2006

07.45.57

In this excerpt, Bwang8’s posting at 504 is an uptake, and thus constitutes an implicit ratification of the proposed course of action. An interesting feature of this organization of the next-sequence selection is that by calling for the initiation of a PRU in the first place, the actor making the next-activity query is presumed to be willing to accept a ratified next-activity proposal. By offering a next activity, Quicksilver can be seen to both propose and endorse this next course of action. Bwang8, by taking up the next activity, implicitly endorses the matter. Thus, a next-activity query addressed to a collectivity can be treated as a way by which the questioner can make relevant a PRU sequence without having to explicitly ratify the proposed next activity.

Yes-No Query Requests as Next-Sequence Selection Sequences

An alternative form of next-sequence selection is initiated with a yes/no query. As Koshik (2005), Heritage (2002), Raymond (2003) and others have observed, questions are capable of doing more things than just soliciting information, including making requests. Next-actions can be selected when a participant requests that others (possibly including the requestor as well) perform these actions. As with PRU next-sequence selections, the yes/no query request consists of three parts:

1. A request in the form of a yes/no query,

2. Acknowledgement of the request, followed by

3. Uptake of the requested action.

As is shown in the following example, the yes/no interrogative calls on recipients to act as a collective to comply with the request to expand or extend their collaboration in the production of an answer:

Table 9: Yes-No Query Request Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

63

Aznx

Can we collaborate this answer even more?

09.05.2006

06.34.01

64

Aznx

To make it even simpler?

09.05.2006

06.34.05

65

bwang8

Ok

09.05.2006

06.34.15

66

Aznx

Because I think we can.

09.05.2006

06.34.16

67

bwang8

((1+N)*N/2+N)*2

09.05.2006

06.34.50

In this example, the request is addressed to the collectivity and it is this addressing that makes it relevant for recipients to act in concert to comply with the proffered request. By calling on recipients to act in a collective and concerted manner, Aznx is constituting the projected action as one in which all recipients are expected to participate and, by leaving the organization of the proposed tasks undifferentiated in terms of the specifics of recipient participation, implies a collaborative orientation toward the accomplishment of the proposed task. This is followed at line 65 by Bwang8’s acknowledgement of the request and then his uptake at line 67.

Suggestion-initiated Next-Sequence Selection Sequences

Another procedure for selecting a next sequence is to make a suggestion. This form is very similar to the PRU next-sequence selection procedure and consists of:

1. A suggestion,

2. Ratification of the suggestion, and

3. Uptake of the suggestion as the next sequence.

Suggestion-initiated sequences explicitly foreground the authorship of the suggested next action, making relevant authorship as a consideration for recipients’ ratification and uptake. This is shown in Table 10.

Table 10: Suggestion-initiated Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

339

Aznx

I think we should start off with a conjecture, that we need to prove.

10.05.2006

07.11.50

340

Aznx

Not a hard one, but one that can be challenging.

10.05.2006

07.12.03

341

Quicksilver

Maybe a row of blocks

10.05.2006

07.12.17

342

Quicksilver

Likethis

10.05.2006

07.12.27

In this instance, Aznx suggests starting off “with a conjecture” as a next sequence to take up, in line 339. At line 341, Quicksilver both ratifies and takes up Aznx’s suggestion by offering a conjecture for recipients to consider.

Collectively-produced, Suggestion-initiated Next-Sequence Selection Sequences

In the data, there was one instance of a co-constructed suggestion-initiated next-sequence selection sequence. This is shown below in Table 11:

Table 11: Collectively-produced Suggestion-initiated Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

309

Aznx

So, I think we should focus on discussing on each step more.

10.05.2006

07.03.17

310

Quicksilver

and explain every answer thoroughly

10.05.2006

07.03.30

311

Aznx

Even if the answer was "obvious."

10.05.2006

07.03.40

312

bwang8

Ok

10.05.2006

07.03.48

313

Quicksilver

like i gave a wrong answer, but my explanations didn't come up on the computer because of the lag

10.05.2006

07.03.49

314

Quicksilver

so thats one thing

10.05.2006

07.03.58

Here, both Aznx and Quicksilver, two of three participants in the interaction, co-construct a suggestion in lines 309 through 311. This shared orientation toward the production of a suggestion also implies that both Aznx and Quicksilver ratify the suggestion they are making. Co-constructing a suggestion, or a proposal or request for that matter, serves to provide recipients with a stronger basis for ratification and uptake since the co-constructed suggestion itself displays multiple ratifications. The third participant then ratifies the suggestion in line 312.

Directive-Compliance-Report Sequences as Next-Sequence Selection Procedures

In contrast to PRU next sequence selection sequences, the initiation of a next-sequence selection sequence can take the form of a directive-compliance-report (DCR) sequence in which one actor tells another or a collectivity what they can do as a next activity. In common vernacular terms, this amounts to telling someone else what to do as a next activity. This kind of next-sequence selection sequence seems to be a three-part sequence that consists of:

1. A directive indicating a next sequence to be initiated, addressed to recipients,

2. Compliance, consisting of (a) receipt of the directive followed by (b) performance of the directed next sequence, and

3. A report on completion of the next sequence, when appropriate.

An example is shown in Table 12.

Table 12: DCR Next-Sequence Selection Sequence Example.

Line

Name

Post

dd-mm-yyyy

Time

110

Aznx

bwang you go first

09.05.2006

06.46.14

111

bwang8

Ok

09.05.2006

06.46.18

112

Aznx

tell me when you're done

09.05.2006

06.46.19

113

Gerry

Enter your team name and the values for sticks and squares

09.05.2006

06.46.37

114

Quicksilver

I tried, but it didn't work

09.05.2006

06.46.43

115

Quicksilver

Are we Team B?

09.05.2006

06.46.50

116

Aznx

TEAM B

09.05.2006

06.46.55

117

bwang8

i am done

09.05.2006

06.47.02

At line 110, Aznx directs Bwang8 to be the first in making additions to a wiki page. This posting is concerned with a next matter to perform and who should perform that next matter. In response to this directive, Bwang8 produces an acknowledgement/agreement token at line 111. In line 112, Aznx expands his directive, telling Bwang8 to report on the completion of his task. At line 117, Bwang8 produces a task-completion report. Directives as initiators of next sequence selection sequences are substantially different in their organization from PRU next-sequence-selection sequences, especially with respect to the way participation in the selection procedure is accomplished.

Implications and Discussion

The examples in this paper show that actors routinely initiate next-sequence-selection sequences as ways of selecting a next sequence to perform. By doing so, actors are engaging in demonstrable and concerted actions to elicit the participation of other actors in deciding what sequence to take up next. PRU sequences, yes/no query request sequences, and suggestion sequences call for ratification of the proposal or suggestion from recipients, thus procedurally and formally treating them as accountable for (a) their participation in the selection process and (b) the uptake of the next sequence. By ratifying a proposal or suggestion, participants do not act as individual actors but rather as a collectivity to endorse a possible next sequence. In this way, PRU sequences, yes/no query requests and suggestion sequences serve not only to constitute the assembled participants as a collectivity but also to constitute the collectivity rather than the individual participants as the ‘actor’ in the scene (Lerner, 1993). This contrasts with directive-compliance-report (DCR) sequences that treat individual actors rather than the collectivity as (a) accountable for the selection of a next action and (b) accountable for the performance of that next action. The DCR sequence seems to constitute and instantiate a non-collaborative orientation toward the performance of subsequent action. In one version of this idea, one could argue that with PRU sequences the collectivity decides and takes up a next activity and that with DCR sequences individual actors decide for other actors in the scene.

In the VMT data, we can see that actors organize themselves to transition between sequences in a variety of collaborative ways. A preliminary examination of the Team B data suggests that the collaborative options were far more prevalent than the non-collaborative one:

Table 13: Frequency Count of Transition Method Occurrence

PRU Sequences

Query Sequences

Suggestion Sequences

DCR Sequences

Frequency

38

19

21

24

There were 60 combined occurrences of the collaborative procedures compared to 9 occurrences of the non-collaborative DCR procedure.

This orientation toward collaboration suggests that collaboration may be as important in the conduct of activities as it is in the transition between activities. If a group understands itself to be collaborative, then one would expect collaboration to be achieved as they work within an activity and as they transition between activities. Actors may occasionally display differences in learning competencies within learning activities, which may serve to mask the collaborative nature of their work. Thus examining the transitions between sequences is a perspicuous bit of interactional work for exploring the organization of collaboration in CSCL groups. This paper has described various methods whereby such transitions are accomplished and how, based on their design and achievement, they show an orientation toward collaboration.

References

Çakır, M. P., Zemel, A., & Stahl, G. (2009). The joint organization of interaction within a multimodal CSCL medium. International Journal of Computer-Supported Collaborative Learning, 4(2), 115-149.

Garcia, A. C., & Jacobs, J. B. (1999). The eyes of the beholder: understanding the turn-taking system in quasi-synchronous computer-mediated communication. Research on Language and Social Interaction, 32, 337-367.

Garfinkel, H. (1967). Studies in ethnomethodology. Cambridge: Polity Press.

Heritage, J. (2002). Ad hoc inquiries: Two preferences in the design of routine questions in an open context. In D. W. Maynard, H. Houtkoop-Steenstra, N. C. Schaeffer & J. v. d. Zouwen (Eds.), Standardization and tacit knowledge: Interaction and practice in the survey interview (pp. 313-334). New York: John Wiley & Sons.

Hutchby, I. (2001). Conversation and Technology: From the Telephone to the Internet. Cambridge, UK: Polity Press.

Koshik, I. (2005). Beyond Rhetorical Questions: Assertive questions in everyday interaction. Amsterdam/Philadelphia: John Benjamins Publishing Company.

Lerner, G. H. (1993). Collectivities in action: Establishing the relevance of conjoined participation in conversation. Text, 13(2), 213-245.

Lipponen, L. (2002). Exploring foundations for computer-supported collaborative learning. Paper presented at the CSCL 2002, Boulder, CO.

Luppicini, R. (2007). Review of computer mediated communication research for education. Instructional Science, 35, 141-185.

Maynard, D. W. (1984). Inside plea bargaining: The language of negotiation. New York: Plenum Press.

Pomerantz, A. (1984). Agreeing and disagreeing with assessments: some features of preferred/dispreferred turn shapes. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in Conversation Analysis. Cambridge: Cambridge University Press.

Raymond, G. (2003). Grammar and social organization: Yes/no interrogatives and the structure of responding. American Sociological Review, 68(6), 939-967.

Roschelle, J., & Teasley, S. D. (1994). The construction of shared knowledge in collaborative problem solving. NATO ASI Series F. Computer and Systems Sciences, 128, 69-97.

Sacks, H. (1992). Lectures on conversation, Vols. I & II. Oxford: Blackwell.

Sarmiento, J., & Stahl, G. (2008). Extending the joint problem space: Time and sequence as essential features of knowledge building. Paper presented at the International Conference of the Learning Sciences (ICLS 2008), Utrecht, Netherlands.

Schegloff, E. A. (1968). Sequencing in conversational openings. American Anthropologist, 70, 1075-1095.

Schegloff, E. A. (1990). On the organization of sequences as a source of ‘coherence’ in talk-in-interaction. In B. Dorval (Ed.), Conversational organization and its development (pp. 51-77). Norwood, NJ: Ablex.

Schegloff, E. A. (2007). Sequence Organization in Interaction: A Primer in Conversation Analysis. Cambridge: Cambridge University Press.

Schegloff, E. A., & Sacks, H. (1973). Opening up closings. Semiotica, 7, 289-327.

Schönfeldt, J., & Golato, A. (2003). Repair in chats:  A conversation analytic approach. Research on Language and Social Interaction, 36(3), 241-284.

Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press.

Stahl, G. (2009). Studying virtual math teams. New York, NY: Springer.

Zemel, A., & Çakir, M. P. (2009). Reading’s work in VMT. In G. Stahl (Ed.), Studying virtual math teams (pp. 261-276). New York, NY: Springer.

Zemel, A., Xhafa, F., & Cakir, M. (2007). What’s in the mix? Combining coding and conversation analysis to investigate chat-based problem-solving. Learning and Instruction, 17(4), 405-415.

9. Methodological issues in developing a multi-dimensional coding procedure for small group chat communication

Jan-Willem Strijbos and Gerry Stahl

In CSCL research, collaboration through chat has primarily been studied in dyadic settings. This article discusses three issues that emerged during the development of a multi-dimensional coding procedure for small group chat communication: a) the unit of analysis and unit fragmentation, b) the reconstruction of the response structure and c) determining reliability without overestimation. Threading, i.e., connections between analysis units, proved essential to handle unit fragmentation, to reconstruct the response structure and for reliability of coding. In addition, a risk for reliability overestimation was illustrated. Implications for analysis methodology in CSCL are discussed.

C

oding of communication processes (content analysis) to determine effects of computer-supported collaborative learning (CSCL) has become a common research practice (Barron, 2003; Webb & Mastergeorge, 2003; Fischer & Mandl, 2005). In the past decade, research on CSCL has opened new theoretical, technical and pedagogical avenues of research.

Comparatively less attention has, however, been directed to methodological issues associated with coding (Strijbos, Kirschner, & Martens, 2004).

Early attempts to analyze communication in computer-supported environments focused on counting messages to determine students’ participation and on mean number of words as an indicator for the quality of messages. Later, methods like “thread-length” analysis and “social network analysis” expanded this surface-level repertoire. Now the CSCL research community agrees that surface methods can provide a useful initial orientation, but believes that analysis that is more detailed is needed to understand the underlying mechanisms of group interaction.

Content analysis is widely applied in collaborative learning research (see Barron 2003; Gunawardena, Lowe, & Anderson, 1997; Strijbos, Martens, Prins, & Jochems, 2006; Schellens & Valcke, 2005; Weinberger & Fischer, 2006). Communication is segmented into analysis units (utterances), coded and their frequencies used for comparisons and/ or statistical testing. Increasingly, collaborative-learning studies are moving to a mixed-method strategy (Barron, 2003; Hmelo-Silver, 2003; Strijbos, 2004) and new techniques are being combined with known ones, such as multilevel modeling of content-analysis data (Chiu & Khoo, 2003).

At present, however, the number of studies reporting on the specifics of an analysis method in detail is limited. With respect to content analysis this is highlighted by how many citations still reference Chi (1997), whose article was until recently the most cited article regarding the methodological issues involved. Within the CSCL community an academic discourse is gradually developing on issues such as analysis-scheme construction, comparability and re-use (De Wever, Schellens, Valcke, & Van Keer, 2006), unit of analysis (Strijbos et al., 2006) and specific processes like argumentative knowledge construction (Weinberger & Fischer, 2006)—but many issues remain.

Background

This article reports on an attempt to use coding under circumstances that may be typical in CSCL research, but where coding has not generally been applied. The theory behind our research focuses on group processes and the meaning making that takes place in them. It is elaborated by Stahl (2006a) and Stahl, Koschmann and Suthers (2006). The theory there recommends ethnomethodologically-informed conversation analysis as the most appropriate analysis methodology, but we wanted to try to apply a coding approach as well.

Coding is most frequently used to compare research groups under controlled experimental conditions with well-defined dependent variables; we wanted to use coding to help us explore initial data where we did not yet have explicit hypotheses. Coding is often used in cases of face-to-face talk (e.g., in a classroom) or between communicating dyads; we were interested in online text-based synchronous interaction within small groups of three to five students.

Educational and psychological research using coding generally takes utterances or actions of individuals as the unit of analysis; we wanted to focus on the small group as the unit of agency and identify group processes. In undertaking our inquiry into the use of coding under these circumstances, we strove for both reliability and validity. In this article, we take a close look at reliability and address issues of validity in our discussion.

Our test site, the VMT project, is developing an online service for students to engage in math discourse at a distance. This project takes a design-based research approach (Stahl, 2006b). It started very simply with a well-known technology (AOL IM®) and the established Math Forum Problem of the Week (PoW) service. The PoW service targets students in grades three through twelve. It provides creative, non-routine challenges for volunteers around the globe. The service is divided into four separate branches: algebra, geometry, pre-algebra and math fundamentals. The reported work with the coding scheme was conducted at the end of the first year of the five-year research project.

We wanted to understand what was happening in the chats along a number of dimensions. We wanted insights that would help us to develop the environment and the pedagogical approach. In particular, we were interested in how the students communicated, interacted and collaborated. We were also interested in how they engaged in math problem solving as a group. So we drew upon coding schemes from the research literature that addressed these dimensions while developing the VMT coding scheme.

VMT coding scheme

Multi-dimensional coding schemes are not a novelty in CSCL research, but they are often not explicitly defined. Henri (1992) distinguishes five dimensions: participation, social, interactive, cognitive and metacognitive. Fischer, Bruhn, Gräsel, and Mandl (2002) define two dimensions: the content and function of utterances (speech acts). Finally, Weinberger and Fischer (2006) use four dimensions: participation, epistemic, argument and social. These studies assign a single code to an utterance, or they code multiple dimensions that differ in the unitisation grain size (i.e., message, theme, utterance, sentence, etc.).

The first step in the development of the coding scheme was to determine the unit of analysis; its granularity can affect accuracy of coding (Strijbos et al., 2006). We decided to use the chat line as the unit of analysis mainly because it is defined by the user. It allowed us to avoid segmentation issues based on our (researcher) view. We empirically saw that the chat users tended to only do one thing in a given chat line. Exceptions requiring a separate segmentation procedure were rare and too insubstantial to affect coding. We decided to code the entire log, including automatic system entries. In contrast to other multi-dimensional coding schemes unitisation is the same for all dimensions: a chat line receives either a code or no code in each dimension—this allows for combinations of dimensions and expands the analytical scope.

We decided to separate communicative and problem-solving processes and conceptualized these as independent dimensions. Our initial scheme consisted of:

- the conversational thread (who replies to whom),

- the conversation dimension (based on Beers, Boshuizen, Kirschner, & Gijselaers 2005; Fischer et al., 2002; Hmelo-Silver, 2003),

- the social dimension (based on Renninger & Farra, 2003; Strijbos, Martens, Jochems, & Broers, 2004),

- the problem-solving dimension (based on Jonassen & Kwon, 2001; Polya, 1985),

- the math-move dimension (based on Sfard & McClain, 2003) and

- the support dimension (system entries and moderator utterances).

Then we spent the summer trying to apply these codes to ten chats that we had logged in Spring 2004. Naturally, we wanted our coding to be reliable, so we checked on our inter-rater reliability as we went along. Problems in capturing what was taking place of interest in the chats and in reaching reliability led us to gradually evolve our categories. As the dimensions became more complicated with sub-categories, it became clear that some of them should be split into new dimensions. We ended with the categories in Appendix A, and the additions during calibration trials have been italicized (the math move and support dimension are not discussed in the remainder of this article and therefore not shown).

It turned out that it was important to conduct the coding of the different dimensions in a certain order, and to agree on the coding of one dimension before moving on to consider others. In particular, determining the threading of chat in small groups is fundamental to understanding the interaction. For the participants, confusion about the threading of responses by other participants can be a significant task and source of problems (Fuks, Pimentel, & De Lucena, 2006; O’Neill & Martin, 2003). For researchers, the determination of conversational threading is the first step necessary for analysis (Cakir, Xhafa, Zhou, & Stahl, 2005). Agreement on the threading by the coders establishes a basic interpretation of the interaction. Then, individual utterances can be assigned to codes in a reliable way. In addition, we were interested in the math problem solving. So we also determined the threading of math argumentation, which sometimes diverged from the conversational threading, often by referring further back to previous statements of math resources that were now being made relevant. Determining the problem-solving threading required an understanding of the math being done by the students, and often involved bringing math expertise into the coding process.

In this article, we focus on three issues that emerged in our attempt to apply a coding scheme in preliminary stages of CSCL research:

We tried to use the natural unit of the chat posting as our unit for coding. This rarely led to problems with multiple contents being incorporated in a single posting, but rather with a single expressive act being spread over multiple postings.

The reconstruction of the chat’s response structure was an important step in analyzing a chat. We developed a conversation thread and a problem-solving thread to represent the response structure.

The goal of acceptable reliability drove the evolution of the coding scheme. The calculation of reliability itself had to be adjusted to avoid over-estimation for sparsely coded dimensions.

Unit fragmentation and response structure reconstruction

We started with the calibration of the conversation dimension and combined this with threading in a single analysis step, but quickly discovered that threading actually consisted of two issues namely unit fragmentation and reconstruction of the response structure. “Unit fragmentation” refers to fragmented utterances by a single author spanning multiple chat lines. These fragments make sense only if considered together as a single utterance. Usually, one of these fragments is assigned a conversational code revealing the conversational action of the whole statement, and the remaining fragments are tied to the special fragment by using ‘setup’ and ‘extension’ codes. This reduces double coding. Table 1 provides an example of both codes: line 155 is an extension to 154 and together they are a ‘request’ and line 156 is a setup to line 158 forming a ‘regulation’.

CSCL research on chat technology previously focused on dyadic interaction (e.g., research on argumentation; Andriessen, Baker, & Suthers, 2003), which poses few difficulties to determine who responds to whom. In contrast, the VMT’s small group chat transcripts revealed that the chain of utterances could be problematic.

A discussion forum uses a threaded format that automatically inserts a response to a message as a subordinate object in a tree structure, and in a similar vein, a prefix is added to the subject header of an e-mail reply. Current chat technology has no such indicators identifying the chain of utterances.[1] Moreover, while there is no confusion about the intended recipient in a dyadic setting (the other actor), students in small groups often communicate simultaneously, making it easy to loose track of to whom they should respond. Coding small group conversation in a chat requires the reconstruction of the response structure as illustrated in Table 1.

Table 1. Threading reconstruction (derived from reliability trial R1)

Delay between utterances proved to be important. For example, lines 157 and 158 fully overlap (no delay) and the delay between lines 166 and 167 of 16 seconds reveals that the short utterance of 167 is more likely to be connected to 166 than 164. Our reasoning is that it takes only a few seconds to type and submit this utterance, and if line 167 was intended as a response to line 164 this utterance would have appeared before or simultaneous with line 166.

Connecting utterances to handle unit fragmentation and to reconstruct the response structure is performed simultaneously, and referred to as ‘threading’. The threading is performed separately from the conversational coding, including assignment of extension and setup, because not all spanned utterance connections concern fragmentation. There is one infrequent exception of a spanned utterance in the shape of three fragments coded as ‘explain/critique’ + ‘elaborate’ + ‘extension’, but this emphasizes that coding of extend and setup should be performed separately. In other words, threading only reconstructs connections between the user-defined chat lines that form a) a fragment of a spanned utterance or b) a response to a previous utterance, but the nature of the chat line is decided during coding and not during threading. It also highlights that a coder should be familiar with the codes to ensure that s/he knows which lines should be considered for threading because the conversational code depends on whether or not a thread is assigned.

Calibration trials for the problem-solving dimension revealed a similar need for the reconstruction of a problem-solving thread – to follow the co-construction of ideas and flow of problem-solving acts (e.g., proposing a strategy or performing a solution step) – before the coding of problem solving.

Calibration trials showed that threading is of utmost importance for the analysis of chat-based small-group problem solving and should be assigned before the (conversational) coding. In the next section, we will discuss the reliability for threading and coding of three dimensions in detail, as their calculation presented additional methodological issues – more specifically the risk for reliability overestimation. In line with Strijbos et al. (2006), we address reliability stability by presenting two trials, each covering about 10% of the data.

Reliability of threading, coding and reliability over-estimation

Reliability of threading

Threading is already a deep interpretation of the data and therefore a reliability statistic should be determined. The calculation of ‘threading reconstruction’ reliability proved complicated, because coders can assign a thread indicator to a chat line or not, assign an indicator to the same chat line or to a different chat line. As a result, only a proportion agreement can be computed. We used three coders (first author and two research assistants) and computed two indices for all possible dyads:

for the assignment of a thread or not by both coders (% thread);

for the assignment of the same thread whenever both assigned a thread (% same).

Table 2 presents the results for both reliability trials for each pair of coders. The first trial (R1) consisted of 500 chat lines and the second trial (R2) consisted of 449 chat lines. The top of Table 2 presents the results for the conversational thread and the bottom the results for the problem-solving thread.

Table 2. The proportion agreement indices for the conversational and problem-solving thread by coder pair and reliability trial.

A threshold for the proportion agreement reliability of segmentation does not exist in CSCL research (De Wever et al., 2006; Rourke, Anderson, Garrison, & Archer, 2001), nor in the field of content analysis (Neuendorf, 2002; Riffe, Lacy, & Fico, 1998). Given the various perspectives in the literature, a range of .70 to .80 for proportion agreement can serve as the criterion value. Combined results for the conversational thread reveal that, on average, both coders assign a thread in 80.7% of all cases. Overall, 72.2% of the thread assignments are the same. These combined results show that the reliability of conversational threading is actually quite stable and fits the .70 to .80 range.

The results of both reliability trials reveal for the problem-solving thread that, on average, in 87% of all the instances both coders assigned a thread. Of all threading assignments by either coder 91.5% are the same. These results show that the reliability of problem-solving threading exceeds the .70 to .80 range. It should be noted that the problem-solving thread is often the same as the conversation thread, so the reliability indices are automatically higher. The R2 selection also contained fewer problem-solving utterances than R1, so the problem-solving thread is more similar to the conversational thread and thus reliably higher. Since the reliability of problem-solving threading depends on the number of utterances that actually contain problem-solving content, it will fluctuate between transcripts. Therefore, the first trial should be regarded as a satisfactory lower bound: 77.1% for thread assignment and 89.9% for same-thread assignment.

Reliability of three coding dimensions and reliability overestimation

Given the impact of the conversational and problem-solving threads during the calibration sessions, codes were added or changed, definitions adjusted, prototypical examples added, and rules to handle exceptions established. Nine calibration trials were conducted prior to the reliability trials. We used three coders (first author and two research assistants) and adopted a stratified coding approach for each reliability trial: the coders first individually assigned the conversation threads, followed by a discussion to construct an agreed upon conversational thread, after which each coder independently coded the conversational and social dimension. Next, coders first individually assigned the problem-solving thread before a discussion was held to construct an agreed upon problem-solving thread, followed by assigning the problem-solving codes. Between both reliability trials, minor changes were made in the wording of a definition or adjusting a rule. Mastery of the coding procedure is laborious. Per dimension, it takes about twenty hours of training and discussion with an experienced coder.

In contrast to our initial conceptualization of the dimensions as being independent, we have been thus far unable to avoid ties between some of the conversational codes and the problem-solving dimension. Coding qualitatively different processes, social versus problem-solving, using the same data corpus was problematic – foremost regarding ‘elaborate’, ‘explain’ and ‘critique’ categories. The implications of ties for the validity of the coding scheme should be determined, but this is beyond the scope of the current article.

Calculating the reliability for the conversation, social and problem-solving dimensions proved to be less straightforward than expected. Each chat line receives a conversation code and can have either one or no code for any other dimension, but not all chat lines are eligible to receive a particular code. The social and problem-solving dimensions only apply to a portion of all of the chat lines, and the pool of valid units will fluctuate between different pairs of coders. When not all units are eligible to receive a code, we should decide how we handle units coded by only one coder and the units not coded by either coders in the reliability computation:

include only units coded by both coders (exclude units with missing values)

categorize missing values as ‘no code’ and include this category;

categorize missing values and non-coded units as ‘no code’ and include this category. For possibilities a) and c) we calculated three reliabilities indices as suggested by De Wever et al. (2006): proportion agreement (%), Cohen’s kappa (κ) and Krippendorffs alpha (α) (the latter two correct for chance agreement) for each dimension and pair of coders.

Option b) was only computed for kappa and alpha. To determine whether the reliability is sufficient the .70 to .80 range is mostly used as criterion for proportion agreement.

Perspectives in the literature on a criterion value for kappa differ, but in our opinion these criteria—intermediate, strict and lenient—apply best: below .45 ‘poor’, .45 to .59 ‘fair’, .60 to .74 ‘good’, and .75 and above ‘excellent’ (De Wever et al., 2006; Landis & Koch, 1977; Neuendorf, 2002). We apply the same criteria to alpha. Table 3 shows the reliability results for the conversation, social and problem-solving dimension.

Table 3. Proportion agreement, kappa and alpha by coder for the conversational, social and problem-solving dimension

Although proportion agreement is still often used, it is insufficient to serve as an indicator for reliability because it does not correct for chance agreement, and we report this solely for comparison. Kappa is computed because this is the most widely used statistic that corrects for agreement by chance. However, recent publications revealed that kappa behaves strange, i.e., the kappa for two coders with a radically different distribution of frequencies over categories will be higher than coders with a similar distribution (Artstein & Poesio, 2005; Krippendorff, 2004). Alpha does not suffer from this statistical artifact, so it should be preferred. We retain kappa for comparison because alpha is not widely used in CSCL or educational research. We will first discuss the pair-wise comparisons for the social and problem-solving dimension.

When only those units coded by both coders are included in the computation – kappa_1 and alpha_1 – the reliability is consistently higher than proportion agreement, which is expected because kappa_1 and alpha_1 do not treat all units coded by only one coder as disagreement. It should be noted that alpha affords to ‘include’ missing values in the data matrix, however, units coded by only one coder are ignored in the final computation. So, although it seems that more units are included there is computationally no difference with the case where these units are excluded (Table 3 shows the number of units that ‘appear’ to be used for the computation for alpha_1 but they are in reality the same as for kappa_1).

When the missing values for units that were coded by only one coder are categorized ‘no code’ and this ‘extra’ category is included in the computation – alpha_2 and kappa_2 – reliability drops. This is stronger for the social dimension as compared to the problem-solving dimension, and is caused by the number of missing values; more missing values lead to a stronger downward correction when these are treated as disagreement. Alpha and kappa have similar values, but differ slightly (caused by the different distribution of frequencies over categories).

When the missing values and all units that were not coded by both coders are included and categorised as ‘no code’ – %A, kappaA and alphaA – proportion agreement is consistently higher, alphaA is higher than alpha_2 for the social and problem-solving dimension but is lower than alpha_1 for the social dimension and equal to alpha_1 for the problem-solving dimension. The same pattern is visible for the three kappa indices.

Since proportion agreement does not correct for chance agreement and kappa suffers from a statistical artifact, alpha is preferred. Excluding missing values in the computation neglects a source of disagreement and inflates reliability, so alpha_1 is not adequate. Including all units that were not coded by both coders appears appealing and consistent but treats those units that are conceptually not eligible to receive a code as agreement. So, alphaA also inflates reliability and is not adequate. Including only those units coded by either coder, categorizing missing values as ‘no code’, is the strictest computation. Thus, alpha_2 should be preferred although this statistic is a slight underestimation of the possible ‘eligible’ units – because it ignores the ambiguous units that both coders considered but did not code – but this is favored given the substantial overestimation if missing values are excluded or all non-coded units are included.

The pair-wise comparisons provide insight into the performance of particular coders, but if more than two coders are available, this should be preferred. We had three coders and alpha is suited to compute reliability for more than two coders (although Fleiss kappa can also correct for multiple coders it applies only to nominal data, alpha can also be used for ordinal, interval and ratio data). Again, alpha_2 is preferred over alpha_1 and alphaA for the case of three coders, and appears the best approximation for the reliability for the social and problem-solving dimension.

Considering the reliability statistics for three coders, alpha for the conversation dimension can be considered ‘good’ for both trails, .653 for R1 and .689 for R2. The alpha for the social dimension can be considered ‘fair’ for both trials, .462 for R1 and .480 for R2. The alpha for the problem-solving dimension is ‘poor’ for R1 (.370) and ‘fair’ for R2 (.523).

Discussion

CSCL research using chat technology has focused primarily on dyads. The VMT project investigates chat-based small-group problem solving. During the development of a multi-dimensional coding scheme to analyze interactions in these groups, three new issues emerged that have strong implications for content-analysis methodology and practice in general and chat communication in particular.

The first methodological issue concerns unit fragmentation. We chose the chat line as the unit of analysis because this is defined by the user, but frequently an utterance spanned across several chat lines makes sense only when considered as a whole. Consequently, connections between these units were required prior to coding, and two codes were added to the conversation dimension to mark these fragments (setup and extension).

The second issue concerns the need to reconstruct the response structure. Whereas in a dyadic chat the intended recipient is always the other partner, it is not easy to determine this in a small group. Similarly to fragmentation, the connection between chat lines forming a chain of responses needs to be reconstructed prior to coding of the conversation dimension. Furthermore, the delay between chat line postings proved to be relevant to determining the response structure. Also, a coder must be familiar with the conversational codes. Assignment of both types of connections is performed simultaneously and termed ‘threading.’ This is a deep interpretation of what is going on in the chat. Aggregating all coding divergence would result in very low reliabilities, so agreement on threading prior to coding is necessary.

The third methodological issue concerns reliability calculation. We conducted two trials and computed the reliability for both types of threading. Reliability for the conversation and problem-solving threading could only be expressed as a proportion agreement, but this proved to be sufficiently reliable. Calculation of reliability for the social and problem-solving dimension was problematic: not all chat lines are valid analysis units for these dimensions and can lead to overestimation of their reliability. The extent of overestimation was shown by calculating reliability for the case where a) only units coded by both coders are included (missing values are excluded), b) missing values are categorized as ‘no code’ and included in the computation, and c) missing values and non-coded units are categorized as ‘no code’ and included in the computation. We computed and compared three reliability indices and concluded that excluding missing values and including all non-coded units lead to over-estimation. Including missing values as a ‘no code’ category is the strictest computation and a slight underestimation of the reliability. In our opinion, a slight underestimation should be favored given a substantial overestimation if units with missing values are excluded or all non-coded units are included. If available, the use of more than two coder is preferred, and the valid pool of units should be reported (see for example Hurme & Järvelä, 2005, p. 6).

We included proportion agreement and Cohen’s kappa for comparison, although both statistics are problematic. Overall, coding reliability – Krippendorff’s alpha for three coders – ranged ‘poor’ to ‘good’ in the first trial and ‘fair’ to ‘good’ in the second trail. Nevertheless, reliability is only one aspect of a coding scheme—addressing the extent to which the coding can be reproduced—and it should not be mistaken for validity. We conclude with some reflections on validity. Once we had reliable coding of ten chat logs, we looked for statistical patterns. It turned out that the chats almost fell into two sets depending upon whether the students had seen the math problems in advance of their chats or not. However, there were two anomalous chats that fell into the wrong sets. The use of codes brought this anomaly to our attention, but could not explain it. Using conversation analysis, we could see a difference in interaction patterns that we termed expository versus exploratory (Mercer & Wegerif, 1999; Zemel, Xhafa, & Stahl, 2005). Subsequently, we found that students working in our chat environment developed methods of interacting that were not adequately captured—let alone explained—by codes adopted from the work of researchers investigating other media or from a priori theories of interaction. For instance, we determined that ‘math proposal adjacency pairs’ often play a distinctive driving role in our math chats (Stahl, 2006c). Ethnomethodologically-informed design-based research needs to grasp the methods that participants creatively invent in response to innovative learning situations and technologies; they cannot simply reduce everything to instances of categories of actions generalized from past studies.

Also, we are particularly interested in group cognition (Stahl, 2006a) that takes place at the group unit of analysis, while coding schemes generally focus on the individual. For instance, we look at problem solving by the group as a whole (Stahl, 2006d). Our coding scheme tried to capture group phenomena like proposal bid-and-uptake or interaction question-and-answer by coding these as sequences of individual contributions (e.g., offer followed by response). The format of chat logs and the traditions of coding practice misled us to fragment group interactions into individual contributions. We now want to look at paired interactions and longer sequences as atomic elements of chats.

As the VMT environment evolved and incorporated a shared whiteboard, graphical referencing, math symbols and other functionality, even our multi-dimensional coding of utterances could not capture the increasingly complex and innovative interactions (Stahl, 2006e). To understand the unique behaviors as students adapt to the new environment—custom technology, pedagogical guidance, open-ended math worlds—we need to look closely at the design of unique group interactions, and not simply code them with pre-existing categories, no matter how multi-dimensional and reliable. While general codes can be applied to many of these phenomena, they do not capture what is new, as required for design-based research. Reducing the chat to a sequence of codes that are general enough to be applied reliably can eliminate the content and details that are of particular interest (Stahl, 2002). This is a paradox of reliable and valid coding efforts in exploratory CSCL research.

References

Andriessen, J., Baker, M., & Suthers, D. (Eds.) (2003). Arguing to learn: Confronting cognitions in computer-supported collaborative learning. Dordrecht: Kluwer Academic/ Springer Verlag.

Artstein, R., & Poesio, M. (2005). Kappa^3 = alpha (or beta) (NLE Technical Note 05-1). University of Essex: Natural Language Engineering and Web Applications Group.

Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12, 307- 359.

Beers, P. J., Boshuizen, H. P. A., Kirschner, P. A., & Gijselaers, W. (2005). Computer support for knowledge construction in collaborative learning environments. Computers in Human Behavior, 21, 623-643.

Cakir, M., Xhafa, F., Zhou, N., & Stahl, G. (2005, July). Thread-based analysis of patterns of collaborative interaction in chat. Paper presented at the 12th international conference on artificial intelligence in education, Amsterdam, The Netherlands.

Chi, M. T. H. (1997). Quantifying qualitative analysis of verbal data: A practical guide. The Journal of the Learning Sciences, 6, 271-315.

Chiu, M. M., & Khoo, L. (2003). Rudeness and status effects during group problem solving: Do they bias evaluations and reduce the likelihood of correct solutions? Journal of Educational Psychology, 95, 506-523.

De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46, 6-28.

Fischer, F., Bruhn, J., Gräsel, C., & Mandl, H. (2002). Fostering collaborative knowledge construction with visualization tools. Learning and Instruction, 12, 213-232.

Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning: The role of external representation tools. The Journal of the Learning Sciences, 14, 405-441.

Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17, 397-431.

Fuks, H., Pimentel, M., & De Lucena, C. J. P. (2006). R-U-Typing-2-Me? Evolving a chat tool to increase understanding in learning activities. International Journal of Computer- Supported Collaborative Learning, 1, 117-142.

Henri, F. (1992). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden papers (pp. 117- 136). London: Spinger Verlag.

Hmelo-Silver, C. E. (2003). Analyzing collaborative knowledge construction: Multiple methods for integrated understanding. Computers & Education, 41, 397-420.

Hurme, T. R., & Järvelä, S. (2005). Students’ activity in computer-supported collaborative problem solving in mathematics. International Journal of Computers for Mathematical Learning, 10, 49-73.

Jonassen, D. H., & Kwon, H. I. (2001). Communication patterns in computer mediated and face-to-face group problem solving. Educational Technology Research & Development, 49, 35-51.

Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30, 411-433.

Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.

Mercer, N., & Wegerif, R. (1999). Is “exploratory talk” productive talk? In K. Littleton & P. Light (Eds.), Learning with computers: Analyzing productive interaction (79-101). New York, NY: Routledge.

Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage publications.

O’Neill, J., & Martin, D. (2003). Text chat in action. Paper presented at the ACM Conference on Groupware (GROUP 2003), Sanibel Island, FL.

Polya, G. (1985). How to solve it. Princeton: Princeton University Press.

Renninger, K. A., & Farra, L. (2003). Mentor-participant exchange in the Ask Dr. Math service: Design and implementation considerations. In M. Mardis (Ed.), Digital libraries as complement to K-12 teaching and learning (pp. 159-173). ERIC Monograph Series.

Riffe, D., Lacy, S., & Fico, F. G. (1998). Analyzing media messages: Using quantitative content analysis in research. Mahwah, NJ: Lawrence Erlbaum Associates.

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8-22.

Schellens, T., & Valcke, M. (2005). Collaborative learning in asynchronous discussion groups: What about the impact on cognitive processing? Computers in Human Behavior, 21, 957-975.

Sfard, A., & McClain, K. (2003). Analyzing tools: Perspectives on the role of designed artifacts in mathematics learning: Special issue. Journal of the Learning Sciences, 11(2 & 3).

Stahl, G. (2002). Rediscovering CSCL. In T. Koschmann, R. Hall & N. Miyake (Eds.), CSCL 2: Carrying forward the conversation (pp. 169-181). Hillsdale, NJ: Lawrence Erlbaum Associates.

Stahl, G. (2006a). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press.

Stahl, G. (2006b). Virtual Math Teams project: An overview of VMT. Retrieved January 23, 2007, from http://www.mathforum.org/vmt/researchers/orientation.html

Stahl, G. (2006c). Sustaining group cognition in a math chat environment. Research and Practice in Technology Enhanced Learning (RPTEL), 1(2).

Stahl, G. (2006d). Analyzing and designing the group cognitive experience. International Journal of Cooperative Information Systems (IJCIS), 15, 157-178.

Stahl, G. (2006e). Supporting group cognitions in an online math community: A cognitive tool for small-group text referencing in chat. Journal of Educational Computing Research, 35, 103-122.

Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409-426). Cambridge, UK: Cambridge University Press.

Strijbos, J. W. (2004). The effect of roles on computer-supported collaborative learning. Unpublished doctoral dissertation, Open University of The Netherlands, Heerlen, The Netherlands.

Strijbos, J. W., Kirschner, P. A., & Martens, R. L. (2004). (Eds.) What we know about CSCL: And implementing it in higher education. Boston, MA: Kluwer Academic/Springer Verlag.

Strijbos, J. W., Martens, R. L., Jochems, W. M. G., & Broers, N. J. (2004). The effect of functional roles on perceived group efficiency: Using multilevel modeling and content analysis to investigate computer-supported collaboration in small groups. Small Group Research, 35, 195-229.

Strijbos, J. W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis : What are they talking about ? Computers & Education, 46, 29-48.

Webb, N. M., & Mastergeorge, A. M. (2003). The development of students’ helping behaviour and learning in peer-directed small groups. Cognition & Instruction, 21, 361- 428.

Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46, 71-95.

Zemel, A., Xhafa, F., & Stahl, G. (2005, September). Analyzing the organization of collaborative math problem-solving in online chats using statistics and conversation analysis. Paper presented at CRIWG 2005 conference, Recipe, Brazil.

Appendix A. VMT coding scheme

(Italic signals addition during calibration)

10. A Multi-Dimensional Coding Scheme for Mathematical Collaboration

Gerry Stahl and Jan-Willem Strijbos

This coding scheme in Table 1 was developed in the summer of 2004 as part of the Virtual Math Teams (VMT) Project. Stahl was the PI and Strijbos was a visiting researcher. The scheme was tested by Steve Weimar (Director of the Math Forum) and three PhD students who were Research Assistants on the VMT Project: Johann Sarmiento, Murat Cakir and Ramon Toledo.

The results of this scheme were reported in (Zemel & Stahl 2010), especially (Stahl 2009c). The rules for applying the coding scheme are specified below the table.

Table 1. Math collaboration dimensions coding scheme (version 15)_8-6-04

Dimension

Subcategory

Code

Description

Examples

[Any text between brackets refers to a previous utterance as potentially expressed by another group member to clarify the example]

Conversation

State

S

An utterance in which a statement of any kind is made that is NOT a response to a previous utterance of any kind NOR are they as specific as an offer, request, regulate or repair typing.

“I think I got it”

“I don’t have enough paper”

Offer

O

An utterance in which new conversational content that has not been discussed before is introduced in the conversation and this content is focused on the problem solving process.

“I think we should focus on the altitude”

Request

Rq

An utterance in which a request is made to another group member of any kind. This can be a request for explanation, information, confirmation, disconfirmation. Usually it includes a question mark. If an utterance is framed as a question, but a more specific responding conversational category applies to the content – often the content is a critique or regulate – the utterance is NOT coded as a request, but as critique or regulate. An utterance that consists only of a question mark is still coded as a ‘request’ (? is a chat convention).

“Why do we need the height?”

“So how is it goin?”

“if it’s equilateral it’s a 45-45-90 triangle?”

“Is it not?”

“[Chat convention]?”

Regulate

Rg

An utterance that is focused on regulating the process by expressing information about an individual activity that is being or is going to be performed (e.g. creating a drawing, work with pencil paper), time and/or turn-taking. These can also be utterances that refer to actions that have to be taken according to the script by the participants or utterances that aim to get the attention of other participants. Including any utterance that confirms or disconfirms activity that has been – or is to be peformed – by any group member.

“*scribbeling*”, “searching google”, “let me draw a pic”, “wait a minute”, “I got left behind”, “IM it [the picture] to Powwow chief”, “the guy said something about a link ro sending”, “but look”, “[You drew the triangle] No, I didn’t”

“Is there anyone out there”

“don’t desert me when I need you”

Repair typing

Rt

An utterance in which a typing error in a previous utterance is corrected.

“[We need to use proportions] *proportions*”

“[I got 214.708] 124.708”

Respond

Rp

An utterance that is a response to an utterance by another group member, but it is NOT as specific as the ‘responding’ conversational acts listed below that respond to utterances that contain content regarding the problem solving process.

“I have no idea”

“sorry”

“it’s ok”

Follow

F

An utterance in response to an utterance by another group member, it specifically aims to indicate that one is confirming, disconfirming, giving approval, signaling that one is paying attention to the problem solving – without explicitly agreeing or disagreeing – and/or indicating that one is following/not following someone’s idea or problem solving activity and/or following/not following an utterance that contained a (dis)agreement about the content of the problem solving. If the utterance that they are responding to does NOT contain content regarding the problem solving process, it is coded as a respond.

“ok”

“right”

“I see”

“I’m not following”

“I understand”

“We could do that”

Elaborate

El

An utterance that clearly builds on a previous utterance by the same person and the content is focused on the problem solving. The utterances may be separated by utterance(s) from another group member. The utterances add new content to the conversation building on the content in the utterance that preceded – this can be in the form of a ‘reasoning’ such as ‘If X is A then Y is B” and “This is value 4, so X this will equal Y”, “Before we can compute X we must know Y”. It can also be in the form of subsequent modifications to a mathematical expression. Compared to an extend, an elaborate adds something new to the conversation, whereas an extension is part of split utterance. In addition, an elaboration CAN receive codes in other dimensions.

“[equilateral means all sides are equal] therefore all angles are equal too”

“[oooooh we can squareroot both sides I think (offer)] 3<2n + 3n + 2 (elaborate)” followed by “3<5n +2 (elaborate)”

Extend

Et

An utterance that clearly extends a previous utterance by the same person. The utterances may be separated by utterance(s) from another group member, but is part of an utterance that has been split into smaller utterances. An extend is a part (fragment or clause) of a compound utterance – but it does NOT receive a conversational code of itself. In the case of an extend the conversational code is determined by looking at the whole compound utterance – but the code is assigned to the first part of the compound utterance. Parts of a compound utterance can NOT be coded in any of the other dimensions. If this is the case, it is likely that this part of the compound utterance is an elaborate instead of an extend.

See example 5 in section on extend:

“and I know (offer)”

“that it is (extend)”

“the sides (extend)”

“are between (extend)”

“21 (extend)”

“an 21.5 (extend)”

Setup

Se

An utterance that is similar to an extend but instead of following a previous utterance it precedes the utterance with which it forms a compound utterance. The same rules stated for extend apply for a setup.

“for the 9 [you did it wrong]”

Agree

A

An utterance in which someone explicitly agrees with the content of a previous utterance that contains some content regarding the problem solving process, usually followed by additional information and sometimes a supportive remark.

‘Yes’ as a response to an explain is coded as an ‘agree’.

“Yes, but …”

“I concur”

“Yeah, that’s true”

Disagree

D

An utterance in which someone explicitly disagrees with the content of a previous utterance that contains some content regarding the problem solving process and/or any utterance that starts with – or only contains – ‘No’ (but an utterance does not have to contain a no to be assigned this code). Often the use of ‘no’ is followed by additional information and sometimes a refuting remark. In the case of a double negation the utterance is NOT coded as ‘disagree’, but either as follow or agree depending on how explicit the content of the utterance is.

“No, because …”

“You did it wrong”

“[the 12 triangle is 124.68 cubits] hey its 124.708”

Critique

C

An utterance in which a critique is provided to an utterance by another group member that is focused on the content – and/or process – of problem solving. In contrast to disagree, this type of utterance questions the content of the preceding utterance or provides an alternative without expressing any definite judgement.

“You sure about that, cause ...”

“I don’t think so”

“I don’t think there is a formula, though”

“okay, sorry, you should have said two”

“are you allowed to that”

Explain

E

An utterance in which an explanation is provided to any utterance by another member demanding an explanation   that focuses on the content of the problem solving. In most cases an ‘explain’ is preceded by a ‘request’ and the preceding utterance contains content regarding the problem solving process. An explain can be a reply to a critique that is phrased as a question.

“[Why use Pythagorean theorem?] cause both halves are right triangles”

“[can sum 1 explain 4 me?? PLEASE?] we just multiplied them out”

No Code

Nc

An utterance that can NOT be assigned any conversational codes that are listed above NOR any other code in any of the other dimensions. If the Internet connection (or software) of the facilitator has broken down, lines that seem to be responding but it cannot be judged for a conversational code are also assigned ‘No code’ and no threading is assigned either. A no code is also assigned to entry and exit lines, and scripted facilitation.

“o”

“or something”

“[my iq is in the negatives] [this is off topic] just a little”

Social ref.

Description

Examples

Identity self

Is

An utterance that reflects one's own abilities vis-a-vie the problem at hand. These could be about the strengths or weakness of the self and are often made in the context of some expected norm.

“I have an idea”

“I prevail”

“My IQ is now like in the negatives”

“Now I’m up to speed”

Identity other

Io

An utterance that reflects another group member's abilities vis-a-vie the problem at hand. These could be about the strengths or weakness of the other (in relation to the self) and are often made in the context of some expected norm.

“You are really lost with this problem”

“[Who are you referring to] me or X”

Interest

I

An utterance with a reference to the problem at hand and it demonstrates investment of effort in working on the problem and/or a general expression not specifically focused on the problem at hand that reflects general interest in math. Questions asking for interest are NOT coded as interest; only the actual expression of interest or an answer asking for interest is coded as interest.

“Interesting problem”

“this is easy”

“I like doing geometry”

“this is kinda cool”

Risk-taking (Expressing uncertainty)

Ri

An utterance in which uncertainty or a lack of knowledge is expressed without concern for being wrong or losing face – this includes utterances that express that a group member (self or other) has realized being wrong or having made a mistake. The difference with ‘identity self’ is that the utterance is not focused on their (lack of)   ability to solve the problem.

“[Is there a formula for 60/60/60?] I have no idea”

“[the square root of that is 7.794] oooooppppss …”

Resource

Rs

An utterance in which one is drawing on other sources (e.g., family members, friends, the web, related math principles) to help solve the current problem. This includes utterances in which a link to website – other than a geometric drawing – is posted by a group member.

“My dad is doing the problem, but he does want to give me any hints”

http://somewehere.on.the/web

“search google”

Norms

N

An utterance that refers to general expectations about how to interact such as greeting behavior and netiquette.

“SHOUTING is not always appropriate”

Home

H

An utterance that refers to the [constraints and advantages of the] home environment.

“My mother is calling me”

School

S

An utterance that refers to the [constraints and advantages of the] school environment. This includes references to a (school) library.

“We have not worked on the problem in class yet”

Collaboration group

Cg

Any utterance with a reference to the group (i.e. ‘we’, ‘all group members’, or ‘everybody’, ‘who’). This refers to the collaboration in a broad sense, for example an activity that has been done, or is assumed to be done or will be done by the group.

“We have made some progress so far”

“Let’s get going”

“Lets not do that”

Collaboration individual

Ci

Any utterance with a reference to the self or another member (i.e. ‘screenname’, ‘he’, ‘she’, ’I’, ‘you’). This refers to the collaboration in a broad sense, for example an activity that has been done, or is assumed to be done or will be done by the self or another group member.

“I was doing the problem by my self”

“Now I’m following”

“I see”

Sustain social climate

Ss

An utterance in which an expression is made that reflects on - or aims - to sustain the social climate; or an utterance that is a response to such an utterance.

“lol”, “jk”, “np”, “sorry”, “haha”, “not funny”, “*goes off to sulk*”, “ You have an interesting profile”, “thanks”, “I’m doing a spoof”, “Super”, “it’s ok”, “got lost for a second”, “good”, “good idea”

Greet

Gr

An utterance in which a group member greets the other group members – including a greeting by the facilitator. This in includes saying goodbye.

“What’s up”, “Hi”, “bye”, “c ya later”

Problem S.

Description

Examples

Orientation

O

An utterance in which the original problem – or part of the original problem is restated, parsed in sub problems, reinterpreted or analyzed – without formulating a particular goal or strategy. Referential words can signal the code. In one of the examples on the right, the word ‘it’ refers to the problem statement.

“We are trying to find a triangle whose area is the sum of the areas of those two triangles”

“It never said what the order of the lengths are”

Strategy

(Long term strategy)

S

An utterance in which a specific problem solving strategy is directly suggested or proposed to be followed through by the group. Including representing the problem or a path to follow. Including mentioning a path to a sub goal and/or a strategy that has been used or was applied.

“Let’s calculate the area of the triangles with know side length of 9 and 12”

“draw the altitude”

“you have to make it like a parallelogram”

Tactic

(Short term strategy)

T

An utterance in which a partial component – tactic – of a specific problem solving strategy is suggested to be followed through or referred to with or without revealing the whole strategy and/or any preceding utterances in which the other parts were already suggested. This includes implicit partial strategies to be followed through by the group, including utterances that intend to find out whether a group member should reveal the whole strategy.

“do we know what these equations all equal”

“do they by chance equal zero”

“[what is the order of the lengths] but how are we going to find the correct one?”

“and you can plug it into the quadratic formula”

“[what does that do] we could find a range”

Perform

P

An utterance in which a chosen strategy is – and associated mathematical actions are – performed or executed. This includes modification of an expression or equation to a different form. If an utterance contains a procedure that is performed – but has already been stated in the conversation – it is NOT coded as a ‘perform’ but as a ‘Restate’. An activity – or use of an active verb – can signal a ‘perform’ (e.g. made, found). Whenever a perform is coded a mathematical move has to be assigned.

“20.25 + X = 81”

“I am calculating the height”

“(n^2 +4+4n)<9<(n^2+5n) is possible”

“I made (n^2+5n)+(n^2+4n+4)>9”

“9<2n2 + 9n + 4”

Check

(check in terms of actions; error focused)

Ch

An utterance in which a problem solving strategy and/or a mathematical operation that was performed by another group member is checked or evaluated (usually preceded by a mathematical operation that has been performed). This includes utterances that either confirm or disconfirm the mathematical operation performed and/or the result, BUT there has to be clear connection in the problem solving threading that the utterance to receive the code ‘check’ targets that utterance. If there is any doubt, the code is NOT assigned. An utterance is a ‘Check’ when the check is focused on ‘actions’ or ‘manipulations’ performed by another student. Whenever a check is coded a mathematical move has to be assigned.

“You did it wrong”

“it’s 8.352”

“right”

“exactly”

“where did you get 9<2n2 + 9n + 4”

“3<5n + 2 doesn’t work”

Restate

Re

An utterance in which a single problem solving activity or outcome of that activity is restated – including utterances that aim to (re)establish what they agreed upon. When an utterance is coded as a ‘Restate’ no mathematical move code is given, but a ‘problem solving thread’ is assigned to link back to the utterance in which the content was stated previously in the conversation.

“We know X is Y cubits”

“We agree it is 10.392”

“[we just said that (N+2)^2 is n^2 +4 + 4n] we just multiplied them out”

“(n+2)^2 is in the form of (a+b)^2”

Summarize

Su

An utterance in which the content of a previous problem solving sequence or strategy is summarized and presented as a condensed interpretation of what transpired over a long time period. In a summary the content of multiple utterances is summarized, repeated and/or related in a single utterance. When an utterance is coded as a ‘Summary’ no mathematical move code is given, but a ‘problem solving thread’ is assigned to link back to the utterances in which the summarized content was stated previously in the conversation.

“[(n+2^2] + [n^2 +4n +4] we just said that (n+2)^2 is n^2+4n+4”

Reflect

(check in terms of reasoning)

Rf

An utterance in which is reflected on the chosen strategy or previous utterances in the conversation and/or outcomes of mathematical actions; i.e. surpassing a summary of the problem solving process so far. This includes utterances in which a member attempts to refocus the problem solving – in terms of shifting from a sub strategy to the main strategy of problem solving. Reflect includes utterances in which a member reflects on or checks his/her own or someone else’s problem solving input. An utterance in which a check is performed that focuses on ‘reasoning’ is also coded as ‘Reflect’.

“I wonder if we really needed the areas”

“Okay guys, I don’t think there is a formula to find the height”

“okay, it’s TWO 30/60/90 triangles”

“are you allowed to use that “greater than sign” as an equal sign”

“are you allowed to do that”

“but if it’s to great it won’t work”

Result

R

An utterance in which the results of a ‘perform’ of a solution step is stated. This includes any result from transforming or manipulation a geometric or algebraic expression. A result is the final form and/or outcome of an expression or perform.

“for the 9 triangle it is about 7.79”

“for the 12 it’s 10.39”, “7.794”

“it’s 8.352”

“[(n +2)^2] so I got n^2 + 4n + 4”

“well I said earlier that I just used trial and error and factored it out using the number I had picked and I found that it had to be less than 4”

Math move

Description

A move is defined as the active introduction of information or manipulation of formulas and/or values. In other words, there is something that is actively created and/or added to the problem solving conversation.

Outcomes are coded as part of the specific math move; if only a number is stated use the previous 10 lines to determine the code.

Examples

General

Counting

Co

An utterance in which counting is performed.

“One, two, three … five units”

Numeric computation

Nc

An utterance in which numeric computation is performed or an utterance that contains a numeric approach to an algebraic move (such as permutations or factoring).

“10.39 * 12”

“and that times the base (12)”

“I say there are six possible orders of length”

Geometric expression or link to drawing

Geo

An utterance in which the problem and/or the solution is geometrically expressed or approached using geometrical notation and/or in words, including utterances that refer to a drawing made by the students – this includes spatial references to a drawing or a visual object. This includes the active application or manipulation of formula’s, values and symbolic expression. In other words, something new is created – which includes utterances in the form of reasoning, giving explanations and critiquing.

“We know that length of AB = 9.”

“You can see my drawing at http://somewhere.on.the/globe”

“[After seeing drawing] okay, it’s TWO 30/60/90 triangles”

Algebraic expression

Alg

An utterance in which the problem and/or solution is algebraically expressed or approached with the use of algebraic notation – such as using variables like ‘X’ or notation like ‘r^2’ – and/or in words. Including active application or manipulation of algebraic formula’s, values and symbolic expression. In other words, something new is created – which includes utterances in the form of reasoning, giving explanations and critiquing. Including arithmetic in an algebraic sense such as subtracting form   both sides (transformation) of an equation.

“20.25 + X = 81”

“then subtract 9 form both sides”

Specific

Import new math info

I

An utterance that introduces a new mathematical concept, fact or information in the discussion from either individual knowledge or archive – with or without a reference or relevance to the problem at hand – that could be used by the group in the problem solving process but it’s NOT linked to the problem solving process. The concept, fact or information in the utterance was not provided in the original problem description. Asking for a formula is NOT considered as a mathematical move (“Is there a formula for a 60/60/60 triangle?”). This includes a critique – phrased as question – that introduces new mathematical information.

“We should multiply the height and the edge length to get the area”

“Area of triangle is ½ *b*h”, “proportions?”

“Pi*r^2”

“The Pythagorean theorem can be used on all kinds of triangles”

Import & apply new math info

la

An utterance that introduces a new mathematical concept, fact or information in the discussion imported from either individual knowledge or archive AND it is actively linked to the problem solving process. The concept, fact or information in the utterance was not provided in the original problem description. This includes a critique – phrased as question – that introduces new mathematical information.

“Do you think we could solve for n using the quadratic formula”

“Splitting the triangle gives two right triangles, so we can use the Pythagorean Theorem”

“like the theorem”

Similar problem (instantiation)

Sp

An utterance in which the current problem is matched against a similar problem with a known solution or in a know form or theorem.

“I’ve reduced it to a problem I’ve solved before”

“((N+2)^2 is in the form of (a+b)^2”

Simplified case

Sc

An utterance in which the current problem is transformed in a simplified version of the problem.

NO EXAMPLE YET

Infer pattern

Ip

An utterance in which a pattern or relationship is identified, described or defined by comparing different triangles or the different computational outputs. This includes the use of randomly picked (guessed) numbers to check for a pattern.

“You think it’s the same for all triangles”

Sub problem

Su

An utterance in which the problem is split in sub problems to be worked out and results combined – or whose possible solution or outcome is prerequisite to other operations that must be performed.

“We need the area before we can compute the height”

“then find the area of each triangle”

“has anyone come up with an equation or expression to solve for n”

Test for boundaries

T

An utterance in which different test cases are tried by putting in numbers in the algebraic expression to find boundaries/ extremes: small numbers, with zero, with big numbers, negatives. A test and the way it is performed are both deliberate decisions. This includes the use of randomly picked (guessed) numbers to rule out possibilities.

“We could try to use different values”

Estimate

E

An utterance in which the answer is estimated [based on a previously stated math move]. A degree of uncertainty is expressed, but the answer is partly supported by some sort of reasoning – or activities – in preceding utterances in the problem solving conversation.

“[It’s equilateral so] I think the length is 7”

Trial and error

Te

An utterance in which the answer is guessed by plugging different values into a formula, but there is no support from any reasoning – or activities – in preceding utterances in the problem solving conversation. This includes the use of randomly picked (guessed) numbers to find a starting point for solving the problem, as well as stating the solution without any reasoning or explanation.

“I guess it is 15”

“this is easy, it’s 15”

“well I said earlier that I just used trial and error and factored it out using the number I had picked and I found that it had to be less than 4”

Conduct unit analysis

Cu

An utterance in which the units are compared that a group member expects to use in the answer with the units in the given information and where s/he looks for

conversion factors involving these units.

“Maybe we should look at the relationship between edge length and area of equilateral triangles”

Work backwards

Wb

An utterance in which a predicted solution is stated [and usually followed by actions needed to get there].

“The area of the third triangle equals the other two, so we need to compute their area’s first”

Combinatoric

Cb

An utterance that contains reasoning through permutations and combination, such as factorial expressions.

NO EXAMPLE YET

M. accuracy

Description

Examples

Accurate

Ma

A math move utterance whose content or action is accurate.

“Pythagorean theorem is for right triangles only”

Inaccurate

Mi

A math move utterance whose content or action is inaccurate.

“Circumference equals Pi times r squared”

M. progress

Description

Examples

Toward solution

Pt

A math move utterance that makes the group progress toward the solution.

Depends on problem solution

Away from solution

Pa

A math move utterance that makes the group progress away from the solution.

Depends on problem solution

Sys. support

Description

Entry

E

An automated chat line indicating a person entered the chat

“Anonymous” has entered the chat room

Exit

Ex

An automated chat line indicating a person left the chat

“JayDoubleU” has left the chat room

Technical problem

Tp

An utterance in which a technical problem with computer technology is expressed and/ or a solution to a technical problem with computer technology is provided

“My explorer did not open”

Scripted facilitation

Sf

An utterance posted by the facilitator that is a part of the scripted facilitation guidelines.

“Here’s the URL for the problem you will be working on …”

Unscripted facilitation

Uf

An utterance posted by the facilitator that is NOT a part of the scripted facilitation guidelines,

“We have a new participant who wants to join. Do you mind?”

Drawing facilitation

Df

An utterance posted by the facilitator that specifically refers to the (process of) posting of a drawing.

“If you have a picture you can send it to …”

“To see the picture go <link> somewhere <link/>

Contact facilitator

Cf

An utterance by a group member directed at the facilitator. This can be a question that is asked to the facilitator or response to a question by the facilitator.

“the dood is back!”

ConversationThread

Description

Reply to U  i

U  i

An utterance that is a reply to a previous utterance. Different single utterances can be reply to the same utterance. When assigning the conversational thread code usually no more than 20 preceding lines need to be considered. Extensions are connected to the first part of the extended utterance. An utterance that has a setup is linked to the setup (no forward linking).

Indicate the line number that the utterance directly responds to U21, U 33.

Problem S. Thread

Description

Connect U  i

U  i

An utterance that connects to an earlier utterance seen from a problem solving perspective. Restatements are linked to the original statement, explanations are linked to the utterance requesting the explanation and elaborations are linked to the statement that they elaborate upon, a critique is linked to the utterance that it critiques and a summary is linked to the statements that are summarized (which is an exception of multiple links).

Indicate the line number that the utterance directly responds to U21, U 33.

General Rules and Regulations

Segmentation: The entire log is coded. Unit of analysis is chat log line, as defined by the user in the chat.

Analysis: Combinations of codes in different dimensions can be considered together.

It is allowed to use up to 10 preceding lines to guide the coding and threading decisions, for example the utterances “go there” directly followed by “that’s the pic” – whereas the first can be easily classified as ‘Regulation’ coding the second utterance relies on the preceding one. Considered in this way the second utterance is also assigned the code ‘Regulation’.

Rule 3 does not apply to determining whether conversational content is new in an utterance for which the whole conversation up to that utterance is considered, nor for determining whether an utterance is an extension for which lines that follow the utterance that is considered for coding are used, nor for determining whether an utterance is restating the content of a previous utterance.

In addition to the 10 preceding lines, the time stamps can be a very useful aid to determine the code for an utterance. Especially with respect to the threading, the time stamps can indicate how the utterances form threads. A time stamp can indicate whether an utterance is a reply: for example a long utterance that follows only a few seconds after another utterance is often a reply to an utterance much further back in the conversation.

When assigning codes it is important to focus on the actual written expression and to refrain from any interpretation that transcends the written expression(s).

When assigning codes it is important to pay attention to referential words, such as ‘it’, ‘that’ and ‘this’ – as these can be decisive for assigning a code (see example for orientation).

When assigning codes for any dimension – especially the mathematical moves – if two codes within a single dimension seem to apply it is important to consider the primary function of the (compound) utterance in the context of conversation and/or problem solving and assign the code accordingly.

When coding the following documents are used: original problem statements, the facilitation script, all possible solution paths, any drawing if made by the students, an overview of utterances and line number by group member to make it easier to identify extensions and elaborations.

Stratified coding procedure:

Each utterance is always assigned a conversational code (except for the scripted utterances by the facilitator).

‘No code’ is assigned to any utterance that cannot be coded in ANY dimension.

Each code may receive an additional one code from any of the other dimensions.

While coding, adhere to following format for each utterance:

- Determine the conversational threading;

- Determine a conversational code for the utterance;

- Determine if a social reference code applies;

- Determine if a support system category applies;

- Determine the problem solving threading;

- Determine if a problem solving code applies;

- Determine if a mathematical move code applies;

- Determine the accuracy of the mathematical move;

- Determine the progress made by the mathematical move.

Conversational threading

Determining the conversational thread:

- A conversation thread is assigned to an utterance that is a reply to an earlier utterance or part of a compound utterance in any way;

- Any part of a compound utterance – regardless whether it can stand on its own as a meaningful utterance or whether it depends on other utterance(s) – is linked to the top level part of that compound utterance (i.e. elaborate, extend and setup);

- Any reply to a part of a compound sentence that is not a ‘meaningful utterance’ in itself is connected to the top level part of the compound utterance;

- Questions that focus on the problem solving and ask for an explanation are given a conversational thread (i.e. a reply to an utterance that contains something that is asked to be explained, for example “I don’t understand how you got that”);

- Questions that don’t focus on problem solving are NOT given a conversational thread – but replies to these questions are given a conversational thread;

- When assigning the conversational thread code usually no more than 20 preceding lines need to be considered.

- If an utterance is a reply to a previous utterance that has been coded as a ‘Repair typing’ – or when it is unclear whether the utterance is a reply to the ‘Repair’ or the original utterance – the threading is USUALLY connected to the original utterance (because a repair does not receive any codes from any other dimension to decrease double coding). Unless the utterance corrects – or disagrees – with the repairing act or content of the repair.

- An utterance that replies to an extended utterance is coded as a reply to the top-level part of the extended utterance.

- Occasionally it is not possible to distinguish to which utterance an utterance is a reply. In those cases where an utterance is clearly a ‘respond’ or any of the other specific responding categories, but it can be a reply to two or more utterances, the threading is assigned to the most recent utterance of all possible ones.

- Utterances that contain scripted facilitation receive no conversational thread.

- Utterances that respond to scripted facilitation receive no conversational thread.

- Emoticons are treated as a single utterance and receive no conversational thread.

Conversation dimension

When coding the conversational dimension it is important to keep the other dimensions in mind to determine whether an utterance would receive a code in other dimensions – as it can occasionally affect the conversational code that you assign.

Most utterances during the math collaboration phase will be a response to a previous utterance. The code ‘respond’ is only assigned if none of the specific kinds of responses (follow, agree, disagree, critique, explain) apply.

If an utterance is phrased as a question, it is in general coded as a request. Sometimes a question mark is lacking, and it can be useful to use the preceding lines to determine the code. Exceptions:

Although the use of a question mark may be guiding in assigning a ‘request’, this can be misleading as occasionally utterances are may be phrased as a question, but in fact they can be an ‘offer’ in disguise, such as “We need to calculate the height, right?” In these cases, the utterance is coded as an offer.

If an utterance is framed as a question, but a specific responding conversational category applies to the content – often the content is a critique or regulate – the utterance is NOT coded as a request, but as critique or regulate.

An utterance that consists only of a question mark is still coded as a ‘request’ (? is a chat convention).

An utterance that contains a questions that aims to assess understanding (“Does everybody get it?”) are not coded as ‘Regulate’ because they do aim to get attention for the collaborative process, not inform other about an activity that is performed. This type of questions is still coded as ‘Request’.

Any utterance that contains a ‘No’ is in principle coded as a ‘Disagree’ when it focuses on the math problem solving process. Exceptions:

- Double negation “I don’t think we should not do that”

- Any expression that disconfirms an action that is taken in term of regulation of the collaboration process. Such statements are coded as regulation irrespective whether they contain ‘No’.

When information from the problem statement is linked to new information – or a new way of expressing this information – the utterance is coded as ‘Offer’. For example the problem statement said that triangles had edge lengths of 9 and 12 cubits, so the utterance “AB = 9 or 12” is an ‘Offer’ because the expression AB was NOT used in the original problem statement.

When information from the problem statement is only repeated in an utterance, this utterance is NOT coded as an offer but as ‘State’.

When information is repeated in an utterance that was ‘new content’ in a previous utterance (and this coded as ‘Offer’), this utterance is NOT coded as ‘Offer’ but as ‘State’.

If an utterance is difficult to code with a conversation code – but it can be coded with a code from any of the other dimensions – this utterance MUST be coded with a conversational code (ruling out ‘No code’). Usually this will be a general category such as ‘Respond’ or ‘State’ and occasionally ‘Follow’.

If the Internet connection (or software) of the facilitator has broken down, lines that seem to be responding but that cannot be judged for a conversational code – because of the disrupted recording – are also assigned ‘No code’ and no threading is assigned either.

Especially with respect to ‘Elaborate’ it is important to consider the preceding lines to determine whether an utterance elaborates on the content of a previous utterance by the same person – this can be signaled by reasoning such as ‘If X is … then Y equals ...” and “This value is 4 … so this will equal Y”, “Before we can compute X … we must know Y”.

If the content of an utterance that has been coded as an ‘Offer’ or ‘Elaborate’ is phrased as conclusion or the concluding step of a problem solving sequence, utterances following such an utterance – that contain ‘Yes’ – are coded as agree. If the utterance that contains ‘Yes’ is threaded to a solution step – which is not the final concluding step or utterance – this utterance is coded as a ‘Follow’.

270

KOH

So you draw a line from C to A

Offer

271

AME

Yes

270

Follow

272

KIL

Yes

270

Follow

273

KOH

Which is the radius that you wanna find

270

Elaborate

274

AME

Yes

373

Agree

275

KIL

Yes

373

Agree

276

ROB

Yes

373

Agree

Example 0. No code and no conversational thread because of multiple thread possibilities (Powwow 2, Gerry’s group).

If a ‘Repair typing’ is coded, it is not assigned any code in any other dimension to decrease double coding (i.e. the original ‘uncorrected’ utterance is assigned codes in the other dimensions – a repair is not similar to repeating an utterance), but the ‘Threading’ is assigned.

In general, a ‘State’ will not be a reply to another utterance. If an utterance is a reply to another utterance (and thus should also receive a threading indication), it should be coded as ‘Respond’ or a more specific conversational category.

If an Internet link is inserted in the conversation, the utterance is coded as a ‘State’ or a ‘Respond’, depending on the conversational thread.

Any automated support system entries are coded as ‘no code’ in the conversational dimension but they are still assigned the support code.

If any of participants posts any – or a part – of the facilitator guidelines (spoof?), this utterance is assigned ‘no code’.

Unscripted facilitation is part of the conversation and thus it must be assigned a conversational code.

Scripted facilitation is not part of the conversation and is assigned ‘no code’.

No code is also assigned to utterances that seem to be a response but the utterance has multiple possible threads and its content is insubstantial enough on its own to be considered a ‘State’. In addition, no conversational threading has been assigned.

277

AV

I just ignored the other number

State

278

SE

My IQ right now is like in the negatives

State

279

AV

I had 10.39 instead of 10.392

Extend

U 277

280

AV

My IQ is 206 … not joking … but that is off topic

Respond

U 278

281

AV

So now we add the two areas

State

Adding areas is part of problem statement

282

SE

Just a little

No code

Example 1. No code and no conversational thread because of multiple thread possibilities (Powwow 2, Gerry’s group).

If several utterances by the same group member can be seen as the responding utterance (respond or a more specific category), the first utterance is ALWAYS coded as a responding utterance. The code for the utterances that follow depend on whether they have a clear connection in the sense of a response.

300

SE

ok

Respond

U296

301

SE

sorry

State

Sustain

No clear connection

302

SE

I was trying to do the problem by my self

Regulate

Collaboration individual

303

SE

Kinda got left behind

Regulate

304

SE

lol

State

Sustain

Example 2.  Example of multiple consecutive utterances where the first is considered as the responding utterance (Powwow 2, Gerry’s group).

Elaborate, extend and setup

Elaborate, extend and setup are all specific utterances that are connected to the preceding or the following utterance(s).

An elaborate builds on a previous utterance by the same person and the content is focused on the problem solving.

162

EEF

i made (n^2+5n) + (n^2+4n+4)>9

Offer

Perform

Algebraic

---

---

---

---

---

---

166

AZN

then subtracted 9 from both sides

Elaborate

Perform

Algebraic

---

---

----

---

---

---

173

AZN

i made (n^2+5n) + (n^2+4n+4)>9. Then subtracted 9 from both sides.

State

Restate

No math because of ‘restate’

Example 3. Elaborate (Powwow 9)

293

AV

So basically we know it’s going to be split into two 30-60-90 triangles again

Offer

Geometric

294

AV

because it’s equilateral

Elaborate

Geometric

Example 4. Elaborate (Powwow 2, Gerry’s group)

An extend is a part (fragment or clause) of a compound utterance – but it does NOT receive a conversational code of itself. In the case of an extend the conversational code is determined by looking at the whole compound utterance - but the code is assigned to the first part of the compound utterance. Parts of a compound utterance can NOT be coded in any of the other dimensions.

An extend or continuation can be signaled by chat convention such as the use of a colon (:) or (…) to indicate that the author will continue ‘talking’ but is now typing a longer statement.

An extend is ONLY assigned if BOTH the current utterance and the preceding one can be seen as a single conversational utterance. If both parts can be assigned a different conversational code, an ‘extend’ code is NOT assigned. ‘Extend’ serves to decrease double coding that may occur as result of split up statements.

If an utterance can be assigned a more specific conversation code than extension based on the content of the utterance, it is NOT assigned ‘Extend’ (line 260). In this example it is also illustrated that the ‘Offer’ code is assigned to the top level part of a compound utterance.

253

PP

We agree  it is 10.39

Agree

AV

---

260

PP

and that times the base [12]

Offer

Multiplying 10.39 and 12 was already stated, but 124.704 was not stated before.

AV

---

266

PP

equals 124.704

Extend

Example 5. Extension (Powwow 2, Gerry’s group)

319

PP

I’m doing trial and error

Explain

320

PP

and I know

Offer

Result

Geometric

321

PP

that it is

Extend

322

PP

the sides

Extend

323

PP

are between

Extend

AV

---

326

PP

21

Extend

AV

---

328

PP

And 21.5

Extend

Example 6. No extend for line 320 because of new content in the extended statement, extend for all parts of the split utterance that starts with line 320 (Powwow 2, Gerry’s group).

Complicated structures of elaborate and extend can occur. An elaborate is always coded according to the other dimensions. If the elaborate is the top level part of an extend, the extend is considered in the context of whole extended elaborate.

134

AZN

We just said that (n+2)^2 = N^2+4n+4

Explain

Restate

No math because of ‘restate’

---

---

---

---

---

---

137

AZN

So we basically just factored out

Elaborate

Restate

Same

---

---

----

---

---

---

139

AZN

the two expressions that had parenthesis

Extend

Example 7. No code for the second extend part because it is part of the rational for assigning the problem solving code for the first part (Powwow 9).

A setup is an utterance that is similar to an extend but instead of following a previous utterance it precedes the utterance with which it forms a compound utterance. The same rules stated for extend apply for a setup.

198

PP

for the 9

this utterance is setting up

Setup

199

PP

you did it wrong

the next utterance

Disagree

Example 8. Setting up from (Powwow 2, Gerry’s group)

Although considering combined statements – using connectives like ‘for’ and ‘so’ might seems attractive, this actually introduces the problem of what is connected

180

PP

this is easy

Can be considered as single utterance.

State

181

PP

For the 12 triangle

This utterance is a setup

Setup

182

PP

144=36+x

For this offer

Offer

Example 9. No setting up (Powow 2, Gerry’s group)

In example 9 “144= 36+x” is new content in the conversation and the preceding lines indicates to which triangle this refers. Thus line 181 is an essential setup for line 182.

Social referencing

If an utterance can be coded as ‘Identity self/other’ or ‘Context home/school’, for example in “I could not do this problem in class”, the utterance is coded as ‘Identity self’.

If an utterance can be coded as ‘Resourceful’ or ‘Context home/school’, for example in “My dad does not want to help me”, the utterance is coded as ‘Resourceful’.

If an utterance can be coded as ‘Identity self’ or ‘Identity other’, for example in “Einstein over here was confusing me”, the utterance is coded as ‘Identity self’.

If an utterance can be coded as ‘Risk taking’ or Identity self’, for example in “I think I got (this does not sound right) that n has to be more than -5”, the utterance is coded as ‘Risk taking’.

If an utterance can be coded as ‘School’ or ‘Collaboration individual’, the utterance is coded as ‘School’.

If an utterance can be coded as ‘Risk taking’ or ‘Collaboration individual’, the utterance is coded as ‘Risk taking’.

If an utterance can be coded as ‘Interest’ or ‘Collaboration individual/group’, the utterance is coded as ‘Interest’.

If an utterance can be coded as ‘Identity self’ or ‘Collaboration individual’, the utterance is coded as ‘Identity self’.

In general the following hierarchy applies:

- ‘Identity self/ other’ precede over ‘Collaboration individual/ group’

- ‘Interest’ and ‘Risk taking’ precede over ‘Collaboration individual/ group’

- ‘Resource’, ‘Home’, ‘School’, precede over ‘Collaboration individual/ group’

Any unscripted facilitation questions that are asked by the facilitator – in which the whole group is addressed – and/or answers by students to unscripted facilitation questions are not coded as ‘Collaboration individual’ or ‘Collaboration group’, because the facilitator is not part or the group.

Problem solving threading

Determining the problem solving thread:

Any utterance that has been assigned an ‘elaborate’, ‘extend’, ‘setup’, critique’, ‘explain’, ‘agree’, ‘disagree’ and ‘follow’ as a conversational coded is assigned a problem solving thread. Extend and setup codes are only coded in the problem solving thread if they are tied to an utterance with problem solving content. The problem solving thread of these utterances is similar to the conversational thread previously assigned;

An offer that builds on the content of a previous utterance in the problem solving process, is connected to that utterance. Offers can be connected to offers. If the content of the statement is not connected to any content that has been discussed before, the utterance is a top level offer and receives no problem solving thread;

Any part of a compound utterance is connected to the top level part of that compound utterance (i.e. elaborate, extend and setup);

Any connection to a part of a compound sentence that is not a ‘meaningful utterance’ in itself is connected to the top level part of the compound utterance.

Questions that focus on the problem solving that ask for an explanation or give a critique are given a problem solving thread (i.e. to the utterance that contains something that is asked to be explained). Answers to such question are also given a problem solving thread;

Questions and answers that don’t focus on problem solving are NOT given a problem solving thread;

Any utterance that restates an earlier utterance gets a problem solving thread;

Any utterance that evaluates the content of an earlier utterance is assigned a problem solving thread;

Any utterance that restates several earlier utterances is assigned a problem solving thread to each utterance – stated previously – that is included. In the case of extensions these connections are assigned to the top-level statement;

Any utterance in which problem solving content is restated, is connected to the utterance in which the content was mentioned for the first time in the problem solving process (always connect back to the original utterance);

When assigning the problem solving thread code it is possible that more than 200 preceding lines need to be considered.

Occasionally it is not possible to distinguish to which utterance an utterance is a reply – in the sense of the problem solving thread – because two or more utterances are possible. In those cases, threading is assigned to the most recent utterance of all possible ones.

Responses by students to a problem-solving question asked by the facilitator are NOT given a problem solving thread.

C-thread

P-tread

59

AME

What did you say BV was?

58

43

60

FIR

2.27

59

59, 43

61

MCP

With the numbers given, BV would be

59

59

62

MCP

yeah

60

60

63

AME

I think thats wrong

60

60

64

FIR

how so?

63

63

65

AME

I know whats wrong with the pic

66

MCP

base would be twice that

61

61

67

FIR

what

65

68

MCP

4.54 ish

66

66

69

AME

The diagnol is not 4.6

67

67

70

FIR

exactly

69

69

71

MCP

Otherwise, the red lines and the base are almost an equilateral triangle

69

69

72

AME

I think this requires trig

70

69

Example 9.5. Difference in C-thread, P-thread and multiple threads (Powow 10)

Problem solving

An utterance is only coded as a strategy if that strategy is explicitly verbalized in an utterance (example 10).

45

AV

I think we should start with the formula for the area of a triangle

Explicit

Strategy

Example 10. Explicit strategy (Powwow 2, Gerry’s group)

A tactic is an utterance in which a part of a specific problem solving strategy is suggested or referred to without revealing the whole strategy and/or any other preceding utterances in which the other parts were already suggested. Including partial strategies in which to be followed through by the group (example 11) and implicit strategies (example 12 and 13).

77

AME

I remembered the property- The sum of the 2 smallest sides is more than the largest one.

Tactic

----

Example 11. Tactic (Powwow 9)

247

AV

so now for the new triangle we have: 194.79 = 1/2bh

Tactic

----

Example 12. Implicit strategy (Powwow 2, Gerry’s group)

101

LIF

Do we know what these equations all equal

Tactic

----

102

LIF

Do they by chance equal zero

Tactic

Example 13. Tactic and implicit strategy (Powwow 9)

If an utterance has been coded as a restate, it will not receive a math code to decrease double coding.

154

AME

But it never says which order the lengths of the segments are

State

Restate

155/6

---

------

----

----

157

AME

we have to find out

Elaboration

Tactic

Example 14. Elaboration, Restate and Tactic (Powwow 9)

If an utterance contains a procedure that is performed – but has already been stated in the conversation – it is NOT coded as a perform but as a ‘Restate’.

Whenever a ‘perform’ or ‘check’ is coded, a mathematical move has to be assigned to the same utterance.

Mathematical moves

Definition of a mathematical ‘move’ is critical, i.e. the active introduction of information or manipulation of formulas’ and/or values. In other words, there is something that is actively created and/or added to problem solving conversation. The lines in example 15 are coded as mathematical moves, whereas the lines in example 16 are NOT coded as moves because they do not involve active creation, transformation or manipulation).

77

AME

I remembered the property- The sum of the 2 smallest sides is more than the largest one.

Tactic

Geometric

78

---

------

----

----

79

AZN

I thought of factoring (N+2)^2 and n(n+5)

Strategy

Algebraic

Example 15. Mathematical moves (Powwow 9)

101

LIF

Do we know what these equations all equal

Tactic

----

102

LIF

Do they by chance equal zero

Tactic

Example 16. No mathematical moves (Powwow 9)

Asking for a formula is NOT considered as a mathematical move (“Is there a formula for a 60/60/60 triangle?”).

In most cases a ‘perform’ or ‘check’ is assigned a mathematical move code.

If a result is part of an extended performance of consecutive steps the result receives the same mathematical code as the preceding performing utterance(s).

In case there are several possible math moves that can be assigned use the following hierarchy:

- ‘Geometric expression’ precedes over ‘Algebraic expression’

- ‘Geometric expression’ precedes over ‘Numeric computation’. This includes utterances of which the object of the computation is a geometrical concept, such as the perimeter.

- ‘Algebraic expression’ precedes over ‘Numeric computation’.

- If an utterance contains the use of geometric notation (BV, AB), ‘Geometric expression’ is coded over ‘Algebraic expression’ (even if the statement can be seen as an algebraic notation)

- If an utterance contains the use of variables (x, a, b), the utterance is coded as ‘Algebraic expression’.

- Utterances that have been coded as ‘Reflect’, ‘Check’ and ‘Results can be considered as mathematical moves.

Acknowledgements

The authors express thanks to Murat Cakir, Nan Zhou, Ramon Toledo and Johann Sarmiento for their hard work in developing, testing and using the coding system in the reliability trials. The reported research was funded by NSF grants 0325447 and 0333493.

References

Adams, F., & Aizawa, K. (2008). The bounds of cognition. Malden, MA: Blackwell.

Adorno, T. W., & Horkheimer, M. (1945). The dialectic of enlightenment (J. Cumming, Trans.). New York, NY: Continuum.

Bakhtin, M. (1986). Speech genres and other late essays (V. McGee, Trans.). Austin, TX: University of Texas Press.

Barab, S., & Squire, K. (2004). Introduction: Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences. 13(1), 1-14.

Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. Journal of The Learning Sciences. 9(4), 403-436.

Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences. 12(3), 307-359.

Barrows, H. (1994). Practice-based learning: Problem-based learning applied to medical education. Springfield, IL: SIU School of Medicine.

Bereiter, C. (2002). Education and mind in the knowledge age. Hillsdale, NJ: Lawrence Erlbaum Associates.

Bourdieu, P. (1972/1995). Outline of a theory of practice (R. Nice, Trans.). Cambridge, UK: Cambridge University Press.

Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences. 2(2), 141-178.

Çakir, M. P., Xhafa, F., & Zhou, N. (2009). Thread-based analysis of patterns in VMT. In G. Stahl (Ed.), Studying virtual math teams. (ch. 20, pp. 359-371). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/20.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_20.

Çakir, M. P., Zemel, A., & Stahl, G. (2009). The joint organization of interaction within a multimodal CSCL medium. International Journal of Computer-Supported Collaborative Learning. 4(2), 115-149. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-009-9061-0.

Carroll, J. (Ed.). (2003). HCI models, theories and frameworks: Toward a multidisciplinary science. San Francisco, CA: Morgan Kaufmann Publishers.

Chan, C. (2011a). CSCL theory-research-practice synergy: The Hong Kong experience of implementing knowledge building in classrooms. International Journal of Computer-Supported Collaborative Learning. 6(2).

Chan, C. K. K. (2011b). Bridging research and practice: Implementing and sustaining knowledge building in Hong Kong classrooms. International Journal of Computer-Supported Collaborative Learning. 6(2), 147-186. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-011-9121-0.

Chi, M. T. H. (1997). Quantifying qualitative analysis of verbal data: A practical guide. Journal of the Learning Sciences,. 6(3), 271-315.

Chomsky, N. (1959). Review of verbal behavior, by B. F. Skinner. Language. 35(1), 26-57.

Chomsky, N. (1969). Aspects of a theory of syntax. Cambridge, MA: MIT Press.

Clark, H. (1996). Using language. Cambridge, UK: Cambridge University Press.

Clark, H., & Brennan, S. (1991). Grounding in communication. In L. Resnick, J. Levine & S. Teasley (Eds.), Perspectives on socially-shared cognition. (pp. 127-149). Washington, DC: APA.

Clinton, H. (1996). It takes a village: And other lessons children teach us. New York, NY: Simon & Shuster.

Cohen, E. G., Lotan, R. A., Abram, P. L., Scarloss, B. A., & Schultz, S. E. (2002). Can groups learn? Teachers College Record. 104(6), 1045-1068.

Cress, U. (2008). The need for considering multilevel analysis in CSCL research—an appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning. 3(1), 69-84. Doi: http://dx.doi.org/10.1007/s11412-007-9032-2.

DBR Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher. 32(1), 5-8.

Descartes, R. (1633/1999). Discourse on method and meditations on first philosophy. New York, NY: Hackett.

Dewey, J., & Bentley, A. (1949/1991). Knowing and the known. In J. A. Boydston (Ed.), John dewey: The later works, 1925-1953. (Vol. 16). Carbondale, IL: SIU Press.

Dillenbourg, P. (1999). What do you mean by "collaborative learning"? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches. (pp. 1-16). Amsterdam, NL: Pergamon, Elsevier Science.

Dillenbourg, P., Baker, M., Blaye, A., & O'Malley, C. (1996). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in humans and machines: Towards an interdisciplinary learning science. (pp. 189-211). Oxford, UK: Elsevier. Web: http://tecfa.unige.ch/tecfa/publicat/dil-papers-2/Dil.7.1.10.pdf

Dillenbourg, P., & Evans, M. (2011). Interactive tabletops in education. International Journal of Computer-Supported Collaborative Learning. 1-24. Web: http://dx.doi.org/10.1007/s11412-011-9127-7. Doi: citeulike-article-id:9720046.

Donald, M. (1991). Origins of the modern mind: Three stages in the evolution of culture and cognition. Cambridge, MA: Harvard University Press.

Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA: MIT Press.

Dreyfus, H. (1972). What computers cannot do. New York, NY: Harper and Row.

Dreyfus, H. (1991). Being-in-the-world: A commentary on Heidegger's Being and time, division I. Cambridge, MA: MIT Press.

Dreyfus, H. (1992). What computers still can't do: A critique of artificial reason. Cambridge, MA: MIT Press.

Duranti, A. (1998). Linguistic anthropology. Cambridge, UK: Cambridge University Press.

Ehn, P. (1988). Work-oriented design of computer artifacts. Stockholm, Sweden: Arbetslivscentrum.

Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki, Finland: Orienta-Kosultit Oy.

Engeström, Y. (1999). Activity theory and individual and social transformation. In Y. Engeström, R. Miettinen & R.-L. Punamäki (Eds.), Perspectives on activity theory. (pp. 19-38). Cambridge, UK: Cambridge University Press.

Engeström, Y. (2008). From teams to knots. Cambridge, UK: Cambridge University Press.

Floyd, C. (1992). Software development and reality construction. In C. Floyd, H. Zuellinghoven, R. Budde & R. Keil-Slawik (Eds.), Software development and reality construction. (pp. 86-100). Berlin, Germany: Springer Verlag.

Fuks, H., Pimentel, M., & Pereira de Lucena, C. (2006). R-u-typing-2-me? Evolving a chat tool to increase understanding in learning activities. International Journal of Computer-Supported Collaborative Learning. 1(1), 117-142. Doi: http://dx.doi.org/10.1007/s11412-006-6845-3.

Gadamer, H.-G. (1960/1988). Truth and method. New York, NY: Crossroads.

Gallese, V., & Lakoff, G. (2005). The brain's concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology. 21(3-4), 455-479.

Garfinkel, H. (1967). Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.

Garfinkel, H. (2006). Seeing sociologically: The routine grounds of social action. Boulder, CO: Paradigm Publishers.

Gee, J. P. (1992). The social mind: Language, ideology, and social practice. New York, NY: Bergin & Garvey.

Geertz, C. (1973). The interpretation of cultures. New York, NY: Basic Books.

Giddens, A. (1984a). The constitution of society. Outline of the theory of structuration. Berkeley, CA: U of California Press.

Giddens, A. (1984b). Elements of the theory of structuration. In The constitution of society. (pp. 1-40): U of California Press.

Gowers, T., & Nielsen, M. (2010). Massively collaborative mathematics. Nature. 461(15), 879-881.

Grudin, J. (1990). Why CSCW applications fail: Problems in design and evaluation. In the proceedings of the CSCW '90.  Los Angeles, CA. Proceedings pp. 85-93.: ACM.

Guribye, F. (2005). Infrastructures for learning: Ethnographic inquiries into the social and technical conditions of education and training. Unpublished Dissertation, Ph.D., Department of Information Science and Media Studies, University of Bergen. Bergen, Norway.

Habermas, J. (1967/1971). Labor and interaction: Remarks on Hegel's Jena philosophy of mind. In Theory and practice. (pp. 142-169). Boston, MA: Beacon Press.

Habermas, J. (1971/2001). On the pragmatics of interaction: Preliminary studies in the theory of communicative action (B. Fultner, Trans.). Cambridge, MA: MIT Press.

Hanks, W. (1992). The indexical ground of deictic reference. In A. Duranti & C. Goodwin (Eds.), Rethinking context: Language as an interactive phenomenon. (pp. 43-76). Cambridge, UK: Cambridge University Press.

Harré, R., & Moghaddam, F. (2003). Introduction: The self and others in traditional psychology and in positioning theory. In R. Harré & F. Moghaddam (Eds.), The self and others. (pp. 1- 11). Westport, CT: Praeger.

Hegel, G. W. F. (1807/1967). Phenomenology of spirit (J. B. Baillie, Trans.). New York, NY: Harper & Row.

Heidegger, M. (1927/1996). Being and time: A translation of Sein und Zeit (J. Stambaugh, Trans.). Albany, NY: SUNY Press.

Herring, S. (1999). Interactional coherence in cmc. Journal of Computer Mediated Communication. 4(4) Web: http://jcmc.indiana.edu/vol4/issue4/herring.html.

Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: Toward a new foundation of human-computer interaction research. ACM Transactions on Computer-Human Interaction. 7(2), 174-196.

Husserl, E. (1917/1991). On the phenomenology of internal time (1893-1917) (J. Brough, Trans.). Dodrecht, NL: Kluwer.

Husserl, E. (1936/1989). The origin of geometry (D. Carr, Trans.). In J. Derrida (Ed.), Edmund Husserl's origin of geometry: An introduction. (pp. 157-180). Lincoln, NE: University of Nebraska Press.

Hutchins, E. (1996). Cognition in the wild. Cambridge, MA: MIT Press.

Johnson, D. W., & Johnson, R. T. (1989). Cooperation and competition: Theory and research. Edina, MN: Interaction Book Company.

Jones, C., Dirckinck-Holmfeld, L., & Lindstrom, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer-Supported Collaborative Learning. 1(1), 35-56. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-006-6841-7.

Jones, C., Dirckinck-Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer-Supported Collaborative Learning. 1(1), 35-56. Doi: http://dx.doi.org/10.1007/s11412-006-6841-7.

Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the Learning Sciences. 4(1), 39-103. Web: http://lrs.ed.uiuc.edu/students/c-merkel/document4.HTM.

Kant, I. (1787/1999). Critique of pure reason. Cambridge, UK: Cambridge University Press.

Kaptelinin, V., & Nardi, B. A. (2006). Acting with technology: Activity theory and interaction design. Cambridge, Mass.: MIT Press. Acting with technology. 333 p. pages.

Kapur, M., & Bielaczyck, K. (2012). Designing for productive failure. Journal of the Learning Sciences. 21(1), 45-83.

Kapur, M., & Kinzer, C. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning. 4(1), 21-46. Doi: http://dx.doi.org/10.1007/s11412-008-9059-z.

Kershner, R., Mercer, N., Warwick, P., & Staarman, J. K. (2010). Can the interactive whiteboard support young children’s collaborative communication and thinking in classroom science activities? International Journal of Computer-Supported Collaborative Learning. 5(4).

Kling, R. (1999). What is social informatics and why does it matter? D-Lib Magazine. 5(1) Web: http://www.dlib.org/dlib/january99/kling/01kling.html.

Koschmann, T. (Ed.). (1996a). CSCL: Theory and practice of an emerging paradigm. Hillsdale, NJ: Lawrence Erlbaum Associates.

Koschmann, T. (1996b). Paradigm shifts and instructional technology. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm. (pp. 1-23). Mahwah, NJ: Lawrence Erlbaum.

Koschmann, T., & Stahl, G. (2009). Symposium: Examining practices of computer-mediated learning. Presented at the international conference on Computer Support for Collaborative Learning (CSCL 2009), Rhodes, Greece. Web: http://GerryStahl.net/pub/cscl2009koschmann.pdf.

Landauer, T. K. (1996). The trouble with computers: Usefulness, usability, and productivity. Cambridge, MA: MIT Press.

Latour, B. (1992). Where are the missing masses? The sociology of a few mundane artifacts. In W. E. Bijker & J. Law (Eds.), Shaping technology/building society. (pp. 225-227). Cambridge, MA: MIT Press.

Latour, B. (2007). Reassembling the social: An introduction to actor-network-theory. Cambridge, UK: Cambridge University Press.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

LeBaron, C., & Streeck, J. (2000). Gesture, knowledge and the world. In D. McNeill (Ed.), Language and gesture. (pp. 118-138). Cambridge, UK: Cambridge University Press.

Lee, E., Chan, C., & van Aalst, J. (2006a). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning. 1(1), 57-87. Doi: http://dx.doi.org/10.1007/s11412-006-6844-4.

Lee, E. Y. C., Chan, C. K. K., & van Aalst, J. (2006b). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning. 1(1), 57-87. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-006-6844-4.

Lerner, G. (1993). Collectivities in action: Establishing the relevance of conjoined participation in conversation. Text. 13(2), 213-245.

Linell, P. (2001). Approaching dialogue: Talk, interaction and contexts in dialogical perspectives. New York, NY: Benjamins.

Linell, P. (2009). Rethinking language, mind, and world dialogically: Interactional and contextual theories of human sense-making. Charlotte, NC: Information Age Publishing.

Livingston, E. (1987). Making sense of ethnomethodology. London, UK: Routledge & Kegan Paul.

Livingston, E. (1995). An anthropology of reading. Bloomington: IN: Indiana University Press.

Looi, C.-K., So, H.-j., Toh, Y., & Chen, W. (2011a). CSCL in classrooms: The Singapore experience of synergizing policy, practice and research. International Journal of Computer-Supported Collaborative Learning. 6(1), 9-38. Doi: http://dx.doi.org/10.1007/s11412-010-9102-8.

Looi, C.-K., So, H.-j., Toh, Y., & Chen, W. (2011b). CSCL in classrooms: The Singapore experience of synergizing policy, practice and research. International Journal of Computer-Supported Collaborative Learning.

Looi, C.-K., So, H.-J., Toh, Y., & Chen, W. (2011c). The Singapore experience: Synergy of national policy, classroom practice and design research. International Journal of Computer-Supported Collaborative Learning. 1-29. Web: http://dx.doi.org/10.1007/s11412-010-9102-8. Doi: citeulike-article-id:8764996.

Marx , K. (1867). Das Kapital: Kritik der politischen Oekonomie (Vol. I). Hamburg, Germany: Otto Meisner.

Marx, K. (1867/1976). Capital (B. Fowkes, Trans. Vol. I). New York, NY: Vintage.

Maturana, H. R., & Varela, F. J. (1987). The tree of knowledge: The biological roots of human understanding. Boston, MA: Shambhala.

Maxwell, J. (2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher. 33(2), 3-11. Web: http://www.aera.net/pubs/er/pdf/vol33_02/2026-02_pp03-11.pdf.

Mead, G. H. (1934/1962). Mind, self and society. Chicago, IL: University of Chicago Press.

Medina, R., Suthers, D. D., & Vatrapu, R. (2009). Representational practices in VMT. In G. Stahl (Ed.), Studying virtual math teams. (ch. 10, pp. 185-205). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/10.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_10.

Mercer, N. (2000). Words and minds. How we use language to think together: Routledge.

Mercer, N., & Wegerif, R. (1999). Is "exploratory talk" productive talk? In K. Littleton & P. Light (Eds.), Learning with computers: Analyzing productive interaction. (pp. 79-101). New York, NY: Routledge.

Merleau-Ponty, M. (1945/2002). The phenomenology of perception (C. Smith, Trans. 2 ed.). New York, NY: Routledge.

Mühlpfordt, M., & Stahl, G. (2007). The integration of synchronous communication across dual interaction spaces. In the proceedings of the International Conference of CSCL: Of mice, minds, and society (CSCL 2007). C. Chinn, G. Erkens & S. Puntambekar. New Brunswick, NJ.  Web: http://GerryStahl.net/vmtwiki/martin.pdf.

Nardi, B. (Ed.). (1996). Context and consciousness: Activity theory and human-computer interaction. Cambridge, MA: MIT Press.

Neill, A. S. (1960). Summerhill: A radical approach to child rearing: Hart Publishing Company.

Orlikowski, W. (1992). Learning from notes: Organizational issues in groupware implementation. In the proceedings of the CSCW '92.  Toronto, Canada. Proceedings pp. 362-369. ACM.

Papert, S. (1980). Mindstorms: Children, computers and powerful ideas. New York, NY: Basic Books.

Pathak, S. A., Kim, B., Jacobson, M. J., & Zhang, B. (2011). Learning the physics of electricity: A qualitative analysis of collaborative processes involved in productive failure. International Journal of Computer-Supported Collaborative Learning. 6(1).

Piaget, J. (1985). The equilibrium of cognitive structures: The central problem of intellectual development: Chicago University Press.

Plato. (340 BCE/1941). The republic (F. Cornford, Trans.). London, UK: Oxford University Press.

Polanyi, M. (1962). Personal knowledge. London, UK: Routledge & Kegan Paul.

Polanyi, M. (1966). The tacit dimension. Garden City, NY: Doubleday.

Polymath, D. H. J. (2010). A new proof of the density hales-jewett theorem. arXiv. Web: http://arxiv.org/abs/0910.3926v2.

Robbins, P., & Aydede, M. (Eds.). (2009). The Cambridge handbook of situated cognition. Cambridge, UK: Cambridge University Press.

Rogoff, B. (1995). Sociocultural activity on three planes. In B. Rogoff, J. Wertsch, P. del Rio & A. Alvarez (Eds.), Sociocultural studies of mind. (pp. 139-164). Cambridge, UK: Cambridge University Press

Roschelle, J. (1996). Learning by collaborating: Convergent conceptual change. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm. (pp. 209-248). Hillsdale, NJ: Lawrence Erlbaum Associates.

Russell, T. (Ed.). (1999). The no significant difference phenomenon. Mindspring Press. Web: http://cuda.teleeducation.nb.ca/nosignificantdifference/.

Sacks, H. (1965/1995). Lectures on conversation. Oxford, UK: Blackwell.

Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking for conversation. Language. 50(4), 696-735. Web: http://www.jstor.org.

Sarmiento, J. (2007). Bridging: Interactional mechanisms used by online groups to sustain knowledge building over time. In the proceedings of the international conference on Computer-Supported Collaborative Learning (CSCL '07).  New Brunswick, NJ.  Web: http://GerryStahl.net/vmtwiki/johann.pdf.

Sarmiento, J., & Stahl, G. (2008). Extending the joint problem space: Time and sequence as essential features of knowledge building [nominated for best paper of the conference]. In the proceedings of the International Conference of the Learning Sciences (ICLS 2008).  Utrecht, Netherlands.  Web: http://GerryStahl.net/pub/icls2008johann.pdf.

Sarmiento-Klapper, J. W. (2009). Bridging mechanisms in team-based online problem solving: Continuity in building collaborative knowledge. Unpublished Dissertation, Ph.D., College of Information Science and Technology, Drexel University. Philadelphia, PA, USA.

Sawyer, R. K. (2003). Group creativity: Music, theater, collaboration. Mahwah, NJ: Lawrence Erlbaum.

Scardamalia, M., & Bereiter, C. (1996). Computer support for knowledge-building communities. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm. (pp. 249-268). Hillsdale, NJ: Lawrence Erlbaum Associates.

Schatzki, T. R., Knorr Cetina, K., & Savigny, E. v. (Eds.). (2001). The practice turn in contemporary theory. New York, NY: Routledge.

Schegloff, E., & Sacks, H. (1973). Opening up closings. Semiotica. 8, 289-327.

Schegloff, E. A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge, UK: Cambridge University Press.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books.

Schwartz, D. (1995). The emergence of abstract representations in dyad problem solving. Journal of the Learning Sciences. 4(3), 321-354.

Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher. 27(2), 4-13.

Sfard, A. (2008). Thinking as communicating: Human development, the growth of discourses and mathematizing. Cambridge, UK: Cambridge University Press.

Shannon, C., & Weaver, W. (1949). The mathematical theory of communication. Chicago, Il: University of Illinois Press.

Spada, H., Stahl, G., Miyake, N., & Law, N. (Eds.). (2011). Connecting computer-supported collaborative learning to policy and practice: CSCL 2011 conference proceedings. (Vol. I, II, III). Lulu: ISLS. Web: http://GerryStahl.net/pub/cscl2011proceedingsI.pdf, http://GerryStahl.net/pub/cscl2011proceedingsII.pdf, http://GerryStahl.net/pub/cscl2011proceedingsIII.pdf.

Stahl, G. (2002a). Groupware goes to school. In J. H. J. Pino (Ed.), Groupware: Design, implementation and use: Proceedings of the 8th international workshop on groupware (CRIWG '02). (pp. 7-24). La Serena, Chile: Springer Lecture Notes in Computer Science 2440. Web: http://GerryStahl.net/cscl/papers/ch11.pdf.

Stahl, G. (2002b). Rediscovering CSCL. In T. Koschmann, R. Hall & N. Miyake (Eds.), CSCL 2: Carrying forward the conversation. (pp. 169-181). Hillsdale, NJ: Lawrence Erlbaum Associates. Web: http://GerryStahl.net/cscl/papers/ch01.pdf.

Stahl, G. (2003). Keynote: The future of computer support for learning: An American/German delfic vision. In the proceedings of the First Conference on e-Learning of the German Computer Science Society (DeLFI 2003). A. Bode, J. Desel, S. Rathmeyer & M. Wessner. Munich, Germany. Proceedings pp. 13-16.  Web: http://GerryStahl.net/publications/presentations/delfi.

Stahl, G. (2004a). Building collaborative knowing: Elements of a social theory of CSCL. In J.-W. Strijbos, P. Kirschner & R. Martens (Eds.), What we know about CSCL: And implementing it in higher education. (pp. 53-86). Boston, MA: Kluwer Academic Publishers. Web: http://GerryStahl.net/cscl/papers/ch16.pdf.

Stahl, G. (2004b). Thinking at the group unit of analysis. In the proceedings of the CSCL SIG Symposium of the European Union Kaleidoscope Network of Excellence (KAL '04).  Lausanne, Switzerland.  Web: http://GerryStahl.net/pub/kal2004.pdf & http://GerryStahl.net/pub/kal2004ppt.pdf.

Stahl, G. (2005). Group cognition: The collaborative locus of agency in CSCL. In the proceedings of the international conference on Computer Support for Collaborative Learning (CSCL '05). T. Koschmann, D. Suthers & T.-W. Chan. Taipei, Taiwan. Proceedings pp. 632-640. Lawrence Erlbaum Associates. Web: http://GerryStahl.net/pub/cscl2005.pdf & http://GerryStahl.net/pub/cscl2005ppt.pdf.

Stahl, G. (2006a). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press. Acting with technology series. 510 pages. Web: http://GerryStahl.net/elibrary/gc. Doi: http://mitpress.mit.edu/books/group-cognition.

Stahl, G. (2006b). Sustaining group cognition in a math chat environment. Research and Practice in Technology Enhanced Learning (RPTEL). 1(2), 85-113. Web: http://GerryStahl.net/pub/rptel.pdf.

Stahl, G. (2007). Meaning making in CSCL: Conditions and preconditions for cognitive processes by groups. In the proceedings of the international conference on Computer-Supported Collaborative Learning (CSCL '07). C. Chinn, G. Erkens & S. Puntambekar. New Brunswick, NJ. ISLS. Web: http://GerryStahl.net/pub/cscl07.pdf.

Stahl, G. (2008a). Book review: Exploring thinking as communicating in CSCL. International Journal of Computer-Supported Collaborative Learning. 3(3), 361-368. Web: http://GerryStahl.net/pub/Sfardreview.pdf. Doi: http://dx.doi.org/10.1007/s11412-008-9046-4.

Stahl, G. (2008b). Integrating a wiki into support for group cognition. In the proceedings of the International Conference of the Learning Sciences (ICLS 2008).  Utrecht, Netherlands.  Web: http://GerryStahl.net/pub/icls2008wiki.pdf.

Stahl, G. (2009a). Keynote: How I view learning and thinking in CSCL groups. In the proceedings of the International Conference on Computers and Education (ICCE 2009).  Hong Kong, China.  Web: http://GerryStahl.net/pub/icce2009keynote.pdf, http://youtu.be/h5MpUJnTipM, http://GerryStahl.net/pub/icce2009.ppt.pdf.

Stahl, G. (2009b). Meaning making in VMT. In G. Stahl (Ed.), Studying virtual math teams. (ch. 26, pp. 505-527). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/26.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_26.

Stahl, G. (2009c). Studying virtual math teams. New York, NY: Springer. Computer-supported collaborative learning series #11. 626 pages. Web: http://GerryStahl.net/elibrary/svmt. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3.

Stahl, G. (2009d). Toward a science of group cognition. In G. Stahl (Ed.), Studying virtual math teams. (ch. 28, pp. 555-579). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/28.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_28.

Stahl, G. (2010a). Global introduction to CSCL. Philadelphia, PA: Gerry Stahl at Lulu. Gerry Stahl's assembled texts. 196 pages. Web: http://GerryStahl.net/elibrary/global.

Stahl, G. (2010b). Group cognition as a foundation for the new science of learning. In M. S. Khine & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education. (pp. 23-44). New York, NY: Springer. Web: http://GerryStahl.net/pub/scienceoflearning.pdf.

Stahl, G. (2010c). Guiding group cognition in CSCL. International Journal of Computer-Supported Collaborative Learning. 5(3), 255-258. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-010-9091-7.

Stahl, G. (2010d). Marx and Heidegger. Philadelphia. PA: Gerry Stahl at Lulu. Gerry Stahl's assembled texts. 217 pages. Web: http://GerryStahl.net/elibrary/marx.

Stahl, G. (2010e). Tacit and explicit understanding in computer support. Philadelphia, PA: Gerry Stahl at Lulu. Gerry Stahl's assembled texts. 374 + xii pages. Web: http://GerryStahl.net/elibrary/tacit.

Stahl, G. (2011a). Coding scheme for sequential discourse. Web: http://GerryStahl.net/pub/coded4b.pdf.

Stahl, G. (2011b). How a virtual math team structured its problem solving. In the proceedings of the Connecting computer-supported collaborative learning to policy and practice: CSCL 2011 conference proceedings. N. Miyake, H. Spada & G. Stahl. Lulu: ISLS. Proceedings pp. 256-263.  Web: http://GerryStahl.net/pub/cscl2011stahl.pdf, http://GerryStahl.net/pub/cscl2011stahl.ppt.pdf, http://youtu.be/0Dg02YQCQIE.

Stahl, G. (2011c). How I view learning and thinking in CSCL groups. Research and Practice in Technology Enhanced Learning (RPTEL). 6(3), 137-159. Web: http://GerryStahl.net/pub/rptel2011.pdf, http://youtu.be/h5MpUJnTipM.

Stahl, G. (2011d). How to study group cognition. In S. Puntambekar, G. Erkens & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methodologies, approaches and issues. (ch. 5, pp. 107-130). New York, NY: Springer. Web: http://GerryStahl.net/pub/analyzinginteractions.pdf.

Stahl, G. (2011e). Seminar: Past, present and future of CSCL. Presented at the International Summer School of Doctoral Candidates at the Beijing China Normal University, Beijing, China. Web: http://GerryStahl.net/pub/cscl2011seminar.pdf, http://GerryStahl.net/pub/cscl2011seminar.ppt.pdf.

Stahl, G. (2011f). The structure of collaborative problem solving in a virtual math team. In the proceedings of the iConference 2011.  Seattle, WA.  Web: http://GerryStahl.net/pub/iconf2011.pdf.

Stahl, G. (2011g). Theories of cognition in CSCW. In the proceedings of the European Computer-Supported Cooperative Work.  Aarhus, Denmark.  Web: http://GerryStahl.net/pub/ecscw2011.pdf.

Stahl, G. (2011h). A view of computer-supported collaborative learning research today. In the proceedings of the Collaboration Technologies and Systems (CTS 2011). W. Smari. Philadelphia, PA.  Web: http://GerryStahl.net/pub/cts2011.pdf.

Stahl, G. (2012a). Theorien des CSCL [theories in CSCL -- in German] (J. Haake & M. Muehlpfordt, Trans.). In J. Haake, G. Schwabe & M. Wessner (Eds.), CSCL-kompendium 2.0. (pp. 16-30). Frankfurt, Germany: Oldenburg. Web: http://GerryStahl.net/pub/theorien.pdf, http://GerryStahl.net/pub/theories.pdf, .

Stahl, G. (2012b). A view of computer-supported collaborative learning research and its lessons for future-generation collaboration systems. Web: http://GerryStahl.net/pub/fgcs.pdf.

Stahl, G. (2013a). Interaction analysis of a biology chat. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs & N. Law (Eds.), Productive multivocality in the analysis of group interactions. (Vol. #14 Springer CSCL Book Series, ch. 28, pp. 511-540). New York, NY: Springer. Web: http://GerryStahl.net/pub/multivocal.pdf.

Stahl, G. (2013b). Theories of cognition in collaborative learning. In C. Hmelo-Silver, A. O'Donnell, C. Chan & C. Chinn (Eds.), International handbook of collaborative learning. (ch. 4, pp. 74-90). New York, NY: Taylor & Francis. Web: http://GerryStahl.net/pub/clhandbook.pdf.

Stahl, G. (2013c). Theories of collaborative cognition: Foundations for CSCL and CSCW together. In S. Goggins & I. Jahnke (Eds.), CSCL@work. (Vol. #13 Springer CSCL Book Series, ch. 3, pp. 43-64). New York, NY: Springer. Web: http://GerryStahl.net/pub/collabcognition.pdf.

Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences. (pp. 409-426). Cambridge, UK: Cambridge University Press. Web: http://GerryStahl.net/elibrary/global.

Stahl, G., & Strijbos, J. W. (2004). A multi-dimensional coding scheme for mathematical collaboration. Web: http://GerryStahl.net/pub/multi_dim_codes.pdf.

Stahl, G., Zemel, A., & Koschmann, T. (2009). Repairing indexicality in virtual math teams. In the proceedings of the International Conference on Computers and Education (ICCE 2009).  Hong Kong, China.  Web: http://GerryStahl.net/pub/icce2009.pdf.

Stahl, G., Zhou, N., Çakir, M. P., & Sarmiento-Klapper, J. W. (2011). Seeing what we mean: Co-experiencing a shared virtual world. In the proceedings of the Connecting computer-supported collaborative learning to policy and practice: CSCL 2011 conference proceedings.  Lulu: ISLS. Proceedings pp. 534-541.  Web: http://GerryStahl.net/pub/cscl2011.pdf, http://GerryStahl.net/pub/cscl2011.ppt.pdf, http://youtu.be/HC6eLNNIvCk.

Strijbos, J. W., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small group chat communication. Learning & Instruction. Special issue on measurement challenges in collaborative learning research. 17(4), 394-404. Web: http://GerryStahl.net/vmtwiki/jw.pdf.

Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge, UK: Cambridge University Press.

Suchman, L. A. (2007). Human-machine reconfigurations: Plans and situated actions (2nd ed.). Cambridge, UK: Cambridge University Press.

Suthers, D., Dwyer, N., Medina, R., & Vatrapu, R. (2010a). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning. 5(1), 5-42. Doi: http://dx.doi.org/10.1007/s11412-009-9081-9.

Suthers, D., Lund, K., Rosé, C. P., & Law, N. (2013). Productive multivocality. Cambridge, MA: MIT Press.

Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010b). A framework for conceptualizing, representing and analyzing distributed interaction International Journal of Computer-Supported Collaborative Learning. 5(1), 5-44.

Suthers, D. D., Lund, K., Rosé, C. P., Teplovs, C., & Law, N. (Eds.). (2013). Productive multivocality in the analysis of group interactions. New York, NY: Springer. CSCL book series. Doi: http://dx.doi.org/10.1007/978-1-4614-8960-3.

Teasley, S. D., & Roschelle, J. (1993). Constructing a joint problem space: The computer as a tool for sharing knowledge. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools. (pp. 229-258). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Terasaki, A. K. (2004). Pre-announcement sequences in conversation. In G. Lerner (Ed.), Conversation analysis: Studies from the first generation. (pp. 171-224). Philadelphia, PA: John Benjamins.

Turing, A. M. (1937). On computable numbers, with an application to the entscheidungsproblem. Proceedings of the London Mathematical Society. 2(1), 230.

Vygotsky, L. (1930/1978). Mind in society. Cambridge, MA: Harvard University Press.

Wee, J. D., & Looi, C.-K. (2009). A model for analyzing math knowledge building in VMT. In G. Stahl (Ed.), Studying virtual math teams. (ch. 25, pp. 475-497). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/25.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_25.

Wegerif, R. (2007). Dialogic, education and technology: Expanding the space of learning. New York, NY: Kluwer-Springer.

Wegner, D. (1986). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior. (pp. 185-208). New York, NY: Springer Verlag.

Wells, G. (1999). Dialogic inquiry: Towards a socio-cultural practice and theory  of education. Cambridge, UK: Cambridge University Press.

White, T. (2006). Code talk: Student discourse and participation with networked handhelds. International Journal of Computer-Supported Collaborative Learning. 1(3), 359-382. Web: http://ijcscl.org/?go=contents. Doi: http://dx.doi.org/10.1007/s11412-006-9658-5.

Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation of design. Reading, MA: Addison-Wesley.

Wittgenstein, L. (1921/1974). Tractatus logico philosophicus. London, UK: Routledge.

Wittgenstein, L. (1944/1956). Remarks on the foundations of mathematics. Cambridge, MA: MIT Press.

Wittgenstein, L. (1953). Philosophical investigations. New York, NY: Macmillan.

Zemel, A., & Çakir, M. P. (2007). Reading's work: The mechanisms of online chat as social interaction. In the proceedings of the National Communication Association Convention.  Chicago, IL.  Web: http://GerryStahl.net/vmtwiki/alan2.pdf.

Zemel, A., & Çakir, M. P. (2009). Reading’s work in VMT. In G. Stahl (Ed.), Studying virtual math teams. (ch. 14, pp. 261-276). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/14.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_14.

Zemel, A., & Stahl, G. (2010). Coordinating collaborative action in online math problem-solving. In the proceedings of the National Communication Association’s 96th Annual Convention (NCA 2010).  San Francisco, CA.  Web: http://GerryStahl.net/pub/nca2010.pdf.

Zemel, A., Xhafa, F., & Çakir, M. P. (2009). Combining coding and conversation analysis of VMT chats. In G. Stahl (Ed.), Studying virtual math teams. (ch. 23, pp. 421-450). New York, NY: Springer. Web: http://GerryStahl.net/vmt/book/23.pdf. Doi: http://dx.doi.org/10.1007/978-1-4419-0228-3_23.

Zhou, N., Zemel, A., & Stahl, G. (2008). Questioning and responding in online small groups engaged in collaborative math problem solving. In the proceedings of the International Conference of the Learning Sciences (ICLS 2008).  Utrecht, Netherlands.  Web: http://GerryStahl.net/pub/icls2008nan.pdf.

Notes

[←1]

The VMT environment—based on ConcertChat—actually includes a threading option. However, it is never used in practice. The problem may be that it makes it harder to know when a new chat message is posted, which is generally the most important thing to be aware of.

Table of Contents

Contents

Introduction

References

The Need for a New Science of Group Cognition

The Construct of Group Cognition

The Group Unit of Description

A Model of the New Science

The Nature of the New Science

Group cognition in a virtual math team (research question)

Non-laboratory experimental design (validity)

Data collection at the group level of description (unit of analysis)

Instrumentation and data formats (objectivity)

Collaborative data sessions (reliability)

Describing group practices (generalizability)

The Foundational Role of Group Cognition

Introduction

Overview of CSCL

Schematic histories of educational technology

The roles of technology in CSCL

The role of individual student learners in CSCL

The role of testing and assessment in CSCL

The role of supporting intersubjective meaning making

Alternative Approaches within CSCL

The theoretical divide

Dimensions of analysis

Multi-vocal methods

Lessons for Future Generations of CSCL Researchers

Lesson 1: Learn collaboratively in multi-disciplinary labs

Lesson 2: Study different approaches to CSCL issues

Lesson 3: Conduct design-based research

Lesson 4: Engage in socio-technical design

Lesson 5: Leverage technological advances

Lesson 6: It takes a global village

Conclusion

Views of Learning and Thinking

An Example of Learning and Thinking

The event: VMT Spring Fest 2006 Team B

The session: Session 3, May 16, 7 pm

The theme: “I have an interesting way to look at this problem”

A move: Showing how to view the problem

A pair: Question/response: “Can you see how it fits inside a square?” / “Yes”

An utterance: Question: “Can you see how it fits inside a square?”

A reference: “It”

Viewing the Learning and Thinking

Unpacking the Group Learning and Thinking

Reference: Network of meaning, indexical ground, joint problem space

Utterance: Recipient design for reading’s work

Response pair: Projection and uptake

Move: Getting the problem-solving work done

Theme: Coherent interactional sequence

Session: temporal structuring and re-member-ing

Event: Forming groups and co-constructing knowledge artifacts

CSCL as a New Approach to Computers in Education

Notes on Group Cognition

Theory for CSCL and CSCW

A Brief History of Theory

The Unit of Analysis

Seminal Theories for CSCL and CSCW

Theories of Individual Cognition

Theories of Community Cognition

Theories of Small-Group Cognition

A Multiplicity of Theories of Cognition

Small-Group Information Use

Structuring Group Cognition at Multiple Levels

An Analytic Method

The Case Study

Problem-Solving Moves

The Sequence of Pairs

Collaborative Mathematical Meaning Making

The Structure of Group Cognition

Analyzing Response Structure

Method

Analysis of the Chat-Response Structure

Discussion: Issues Observed

Suggestions for Redesign

Cycle Two of Design-Based Research

Issues for Further Multi-Vocal Analysis

a. Design-based research for designing technology

b. Scripting of the software agents and situated interaction

c. Sequential interaction analysis of small groups

d. Accountable talk and off-task student practices

Appendix A: Chat Log of Group C01 from Cycle One

Appendix B: Chat Log of Group F01 from Cycle Two

Collaboration and the Coordination of Action

Data and Methods

Transitions between Sequences

PRU Next-Sequence Selection Sequences

“Let x y”-initiated PRU Next-Sequence Selection Sequences

Query-prefaced “Let x y”-initiated PRU Next-Sequence Selection Sequences

Yes-No Query Requests as Next-Sequence Selection Sequences

Suggestion-initiated Next-Sequence Selection Sequences

Collectively-produced, Suggestion-initiated Next-Sequence Selection Sequences

Directive-Compliance-Report Sequences as Next-Sequence Selection Procedures

Implications and Discussion

References

Background

VMT coding scheme

Unit fragmentation and response structure reconstruction

Reliability of threading, coding and reliability over-estimation

Reliability of threading

Reliability of three coding dimensions and reliability overestimation

Discussion

References

Appendix A. VMT coding scheme

(Untitled)

General Rules and Regulations

Stratified coding procedure:

Conversational threading

Conversation dimension

Elaborate, extend and setup

Social referencing

Problem solving threading

Problem solving

Mathematical moves

Acknowledgements

References

Table of Contents

Contents

Introduction

References

The Need for a New Science of Group Cognition

The Construct of Group Cognition

The Group Unit of Description

A Model of the New Science

The Nature of the New Science

Group cognition in a virtual math team (research question)

Non-laboratory experimental design (validity)

Data collection at the group level of description (unit of analysis)

Instrumentation and data formats (objectivity)

Collaborative data sessions (reliability)

Describing group practices (generalizability)

The Foundational Role of Group Cognition

Introduction

Overview of CSCL

Schematic histories of educational technology

The roles of technology in CSCL

The role of individual student learners in CSCL

The role of testing and assessment in CSCL

The role of supporting intersubjective meaning making

Alternative Approaches within CSCL

The theoretical divide

Dimensions of analysis

Multi-vocal methods

Lessons for Future Generations of CSCL Researchers

Lesson 1: Learn collaboratively in multi-disciplinary labs

Lesson 2: Study different approaches to CSCL issues

Lesson 3: Conduct design-based research

Lesson 4: Engage in socio-technical design

Lesson 5: Leverage technological advances

Lesson 6: It takes a global village

Conclusion

Views of Learning and Thinking

An Example of Learning and Thinking

The event: VMT Spring Fest 2006 Team B

The session: Session 3, May 16, 7 pm

The theme: “I have an interesting way to look at this problem”

A move: Showing how to view the problem

A pair: Question/response: “Can you see how it fits inside a square?” / “Yes”

An utterance: Question: “Can you see how it fits inside a square?”

A reference: “It”

Viewing the Learning and Thinking

Unpacking the Group Learning and Thinking

Reference: Network of meaning, indexical ground, joint problem space

Utterance: Recipient design for reading’s work

Response pair: Projection and uptake

Move: Getting the problem-solving work done

Theme: Coherent interactional sequence

Session: temporal structuring and re-member-ing

Event: Forming groups and co-constructing knowledge artifacts

CSCL as a New Approach to Computers in Education

Notes on Group Cognition

Theory for CSCL and CSCW

A Brief History of Theory

The Unit of Analysis

Seminal Theories for CSCL and CSCW

Theories of Individual Cognition

Theories of Community Cognition

Theories of Small-Group Cognition

A Multiplicity of Theories of Cognition

Small-Group Information Use

Structuring Group Cognition at Multiple Levels

An Analytic Method

The Case Study

Problem-Solving Moves

The Sequence of Pairs

Collaborative Mathematical Meaning Making

The Structure of Group Cognition

Analyzing Response Structure

Method

Analysis of the Chat-Response Structure

Discussion: Issues Observed

Suggestions for Redesign

Cycle Two of Design-Based Research

Issues for Further Multi-Vocal Analysis

a. Design-based research for designing technology

b. Scripting of the software agents and situated interaction

c. Sequential interaction analysis of small groups

d. Accountable talk and off-task student practices

Appendix A: Chat Log of Group C01 from Cycle One

Appendix B: Chat Log of Group F01 from Cycle Two

Collaboration and the Coordination of Action

Data and Methods

Transitions between Sequences

PRU Next-Sequence Selection Sequences

“Let x y”-initiated PRU Next-Sequence Selection Sequences

Query-prefaced “Let x y”-initiated PRU Next-Sequence Selection Sequences

Yes-No Query Requests as Next-Sequence Selection Sequences

Suggestion-initiated Next-Sequence Selection Sequences

Collectively-produced, Suggestion-initiated Next-Sequence Selection Sequences

Directive-Compliance-Report Sequences as Next-Sequence Selection Procedures

Implications and Discussion

References

Background

VMT coding scheme

Unit fragmentation and response structure reconstruction

Reliability of threading, coding and reliability over-estimation

Reliability of threading

Reliability of three coding dimensions and reliability overestimation

Discussion

References

Appendix A. VMT coding scheme

(Untitled)

General Rules and Regulations

Stratified coding procedure:

Conversational threading

Conversation dimension

Elaborate, extend and setup

Social referencing

Problem solving threading

Problem solving

Mathematical moves

Acknowledgements

References