Northwestern University, Learning Sciences Department — Center for Learning Technologies in Urban Schools

Innovative learning environments designed by the research community often fail to scale up beyond controlled research and development sites. We argue that one important reason is that designers do not account for teacher learning about innovations. Often, innovative learning environments require significant changes in teaching practice, and those changes in turn require teacher learning. While workshops, conferences and other professional development interventions begin to address the need for teacher learning, they rarely provide the kind of on-going support for learning that teachers need.

What is needed is an environment that sustains teacher learning through the enactment process (Ball & Cohen, 1996). The goal of the Living Curriculum Project (LC) is to build case-based support for teacher learning directly into project-based science (PBS) curricula. The idea is to embed the experience of teachers teaching the curriculum into curricular artifacts so that users can draw on that experience to inform their teaching. Using cases to support learning has a number of advantages. Cases provide learners with analogs to reason with (Gentner, 1988; Kolodner, 1993), afford comparisons, represent complexity (Spiro, Coulson, Feltovich, & Anderson, 1988), ground abstract theory in concrete reality, and — when delivered digitally — provide point-of-need support.

The Living Curriculum is an on-line case-based multimedia performance support system for teachers learning to use project-based science (PBS) curricula. It gives teachers access to examples of the curriculum in use, student work, and to instructional strategies. The goal is to help teachers learn PBS teaching as they teach PBS units. The LC takes the experience of teachers who have taught particular PBS units in the past, represents it in cases, and indexes those cases to the underlying curriculum.

While past efforts have sought to support teaching practice with case material (Kraijcik et al., 1996; Lampert & Loewenberg-Ball, 1998), the LC responds to three challenges not addressed by previous designs:

This paper describes research conducted to design the Living Curriculum to meet these goals. We begin with a description of the LC prototype. Then we describe design research conducted for the prototype. That research is reported on two trajectories. First we describe focus group research used to design the indices to LC case content. Then we describe case studies used to document and interpret teacher practice around a particular PBS curricula — the ReNUE project (1). The case study research helped us to identify what content to include in the LC. Finally, we report iterative design and user testing as well as some long-range design goals.

Figure One is an example of the LC interface. It has four sections: the curriculum itself and three indices to the case content. In the curriculum frame (right) users can browse through the ReNUE curriculum. Using the three indices (left) users can access case content relevant to the curriculum. The Curriculum Zoomer (top left) indexes case content according to tasks allowing users to access only the content that is helpful to support those tasks (e.g., assessment). The Ask a Question section (middle left) indexes answers to questions that teachers typically raise during particular parts of the ReNUE curriculum. Finally, the teacher case section (bottom left) indexes case content as narrative descriptions of the curriculum in use.

All three indices are context dependent. For that reason the organization of the curriculum itself is important. The ReNUE curriculum is organized at three levels. At the top-level users can read an overview of the project including descriptions of the four phases of the project. Phases are thematically related collections of activities. For example, the ReNUE project has a Modeling Phase that includes six activities through which students build aggregate models of an environmental problem. Finally, activities are lesson plans in ReNUE. As users navigate through the curriculum, the case content behind the three indices changes. For example, at the top level of the ReNUE clicking on "Assessment" in the Curriculum Zoomer accesses content relevant to assessing ReNUE on the whole. Clicking on "Assessment" in the modeling phase of ReNUE accesses content relevant to assessing student models. Finally clicking on "Assessment" from a particular modeling activity accesses case content relevant to assessing that part of the modeling task.

Figure One: The Living Curriculum Prototype

The design of the Living Curriculum as described above emerged from a course of design research that included focus groups (employing a concept mapping methodology) to develop user models that informed the indices employed in the LC as well as case studies used to determine what content to include in the system. The focus groups helped us to learn how teachers expect to: (1) interact with curricular materials, (2) identify problems that teachers encounter when they teach PBS units, (3) determine what instructional or classroom management solutions seem to address those problems, and (4) identify resources that seem helpful to support PBS teaching practice. Those results were used to design the interface and indices described above. To develop content for the Living Curriculum we conducted case studies of teachers using ReNUE. Those case studies helped us learn what challenges students and teachers face doing ReNUE, how teachers implement ReNUE (i.e., defined the space between the written and the enacted curriculum, and provided us with access to the thinking process of teachers. The case results both defined what content to include in the LC and provided classroom video to populate the system. The next two sections of the paper report on the focus group and case study results respectively.

To organize content in the LC we needed a user model that helped us to understand how teachers organize their work. Since we specifically wanted to support learning about the ReNUE project, we wanted our model to explicitly address the concerns, questions, and issues that teachers face when teaching ReNUE. To develop a user model we convened a focus group of ReNUE teachers.

We used a qualitative version of Trochim’s (1989) concept-mapping methodology to run the focus group. In concept mapping focus groups brainstorm a list of items in response to some focus question, individuals sort those items in into categories, and software performs a cluster analysis on the sorts to produce a concept map. In our adaptation a group of teachers who had taught the ReNUE project brainstormed a list of questions, concerns or problems that they encountered. Next the individual participants sorted the items into piles and labeled each pile. Then, they each organized their categories into a concept map (i.e., model) of resources for the ReNUE curriculum. The result was a ReNUE curriculum model from each of the participating teachers.

Once each teacher had created a model of her own, we asked her to share the model with the group in a brief presentation. After the presentations we asked the group to generate one model using the categories that they had each created. This process helped us to determine how the various categories that the teachers formed individually relate to one another (i.e., when is the conceptual content the same, but the label different).

Three middle school science teachers who had just taught ReNUE participated in the concept mapping session. Two participants had less than three years teaching experience. The third participant was an experienced (27 years) teacher with five years of PBS teaching experience.

While the only rule for category formations was that no item could be in a category by itself, there was remarkable similarity between the sorts of the three participants. There were four categories that all three participants created (student resources, parent resources, assessment, and grouping), three categories that two of the three participants used (mentoring, extensions and curriculum), and only one unique category (administrator resources).

Student resources include project milestone documents that list student deliverables, writing samples that help students prepare reports, and student web-pages that list research sources. Parent resources are tips and ideas for communicating with parents about student work in PBS classrooms (e.g., newsletters home about science class). Assessment relates to need to find alternate means for determining grades in PBS classrooms (e.g., rubrics used to assess student artifacts). Finally, Grouping provides tips and advice relevant to managing the work of students on group projects.

The next three categories were included in the models of two of the three teachers. Mentoring is a project specific category with information that helps teachers to manage the relationship between students and email mentors. Extensions address the connection between projects and other academic disciplines. In particular it addresses the need to support literacy tasks in science (e.g., writing papers and reading science tasks). Finally Curriculum concerns the fit between PBS units and standard science curricula in terms of curriculum coverage.

Finally one teacher raised the need for "Administrator Resources" that address the need to help administrators learn to support the work of teachers in PBS classroom.

Once the teachers sorted the brainstormed items into categories, we asked them each to organize their categories as if they were designing a model (concept map) to index an on-line curricular resource. Three patterns of organization emerged: (1) there was an emphasis on resources, (2) there were task directed supports, and (3) there was a need for temporal organization of content.

Resources: Teachers specified a need to address resources to four different user groups: teachers, students, administrators and parents. While some of the resources included in each section would be the same, they would be targeted differently. For example, both teachers and students need resources that help them with grouping, but teachers need help managing groups and students need help working with one another.

The teachers also agreed that users need the capacity to add resources to the system. For example, one teacher suggested that students should be able to add links to web pages that were helpful to their research for the ReNUE project making them available to students in other schools.

Task supports: All three teachers included a task-based scheme to organize resources. Here teaching tasks like assessment, grouping, mentoring, extensions, and curriculum (i.e., standards) were indexed together. When teachers presented their maps to the group they argued that the goal of task based indexing is to allow users to go directly to information relevant to their particular concerns. For example, a user preparing to assess students’ work does not want to sift through the curriculum to find assessment strategies, rather the user wants a simple index that can be used to browse only that part of the curriculum relevant to assessment.

Temporal organization: Each of the teachers’ models conveyed some notion of project sequence. One teacher in particular organized her resources in sequence, even where that meant that some categories were indexed in three separate parts of her model. For example, assessment appeared in the Before ReNUE category so that teachers could plan, again in the During ReNUE category so that teachers could have access to the rubrics, and again in After ReNUE category so that teachers could collate assessments of individual project artifacts into a grade.

As the final task of the focus group, the teachers were asked to draw on their three individual models to create one group model. That task raised a tension between task and temporal organization. The group began by adopting the temporal organization. But, as work proceeded it became clear that the many of the task-based categories are repeated multiple points. To resolve this tension, the group agreed to adopt "a time line that can link back to any of these activities." The notion here was that a timeline organized around the lessons in ReNUE could index the resources that are relevant to any particular activity. Thus, temporal organization took precedence over task-based organization in the final model.

Three key design decisions emerged from these focus group data. First, the idea to provide distinct resources for teachers, students, parents and administrators resulted in the construction of multiple interfaces to the LC. Second, the identification of task-based and temporal organization of content resulted in the use of multiple indices to the teacher component of the LC. Third, the idea to add generativity tools emerged.

Based on the distinction between parent, student, teacher and administrator resources we designed the LC with two interfaces: one targeting teachers and one targeting administrators. Since the parent and student resources that the focus group suggested were really resources to help teachers to communicate with parents and provide resources to students, those resources are included in the teacher interface. Due to space limitations, this paper reports only on the design of the teacher interface.

As described in the "Living Curriculum Prototype" section above, the teacher LC includes three indices. The Curriculum Zoomer follows from the finding that teachers organize content around tasks. The Teacher Case index follows from the finding that teachers also organized content temporally. The Ask a Question index did not follow from the focus group data but from the work of our colleagues on question based indexing (Ferguson, Bareiss, Birnbaum, & Osgood, 1992).

As shown in figure one the Curriculum Zoomer indexes tasks like mentoring, assessment, student resources, community connections (parents) and planning from the focus groups. It also indexes "Problems & Opportunities." The Problems & Opportunities category responds to user-tests in which teachers asked for some resource that quickly addresses known difficulties that students encounter when working on ReNUE.

The Teacher Case follows from the focus group result that teachers seek temporal organization of resources. From any point in the curriculum the user can access a teacher case that presents a narrative describing how the case teacher did the lesson in question. Included here are tips that the teacher provides, resources that she used, reflections on teaching and student work.

The ASK a Question section indexes the most frequently asked question at any point in the curriculum. Those questions have been identified from the focus group brainstorm items, traces of email discussions around the ReNUE project (Lento, Dharkar, & Gomez, 1999), and other teacher feedback.

Finally, following from the focus teachers’ idea to organize resources into a timeline, all three of the LC indices are context sensitive. The content accessed by the indices depends upon the ReNUE page that the browser is currently pointed to. Thus, the curriculum is the central organizational component of the LC forming a timeline from which the user can access case content using any one of the three indices.

While focus groups helped us to build indices to LC content, the content itself was derived from studies of the ReNUE curriculum in use. Having cases that represent the experience of teachers provides an illustration of the curriculum as, grounds learning about curriculum in concrete reality and daily classroom practice and provides a space for users to compare and contrast cases. Moreover, grounding professional development in examinations of practice has been shown to improve practice (Richardson, 1998). This section describes the collection, analysis and representation of case content.

Daily videotaped observations of the ReNUE project were conducted in two teachers’ classrooms. Both teachers were experienced (over 25 years). One had taught PBS units for five years, the other was new to PBS teaching. Observations attended to teaching practice as it unfolded in the classroom. In particular the observation focused on tasks that constitute classroom practice, the management techniques employed in the classroom, the instructional strategies used by the teachers, and the interactions between the teacher and her students. Using these foci one or two clips were selected from each day’s teaching. The clips were to serve as examples of ReNUE teaching practice. We did not use the categories from the focus group data to select examples because this more general framework allowed us to attend to the teaching as well as the interactions between teachers and students. A focus on teaching is important because the goal is to help users learn about teaching. Content identified through this more general framework was later analyzed and indexed according to the focus group categories.

On a weekly basis, classroom video clips were shared with the case teacher in reflective interviews. The purpose of those interviews was to get the teacher’s reactions to and reflections on what happened in the classroom. It is important to document the teacher’s thinking because the goal of the LC is help teachers to learn what other teachers do to enact PBS. Reflections provide an opportunity for enacting teachers to provide rationale for the teaching decisions that they made, to provide context about the teaching episode, to evaluate the progress of the project, to identify problems, and suggest improvements.

The reflective interviews were themselves videotaped. As was the case with the classroom video, the video was later edited into short video clips that represented the teacher’s thinking. Those clips were indexed with the classroom video clips that motivated the reflection. Thus, the final record includes a library of edited classroom video clips accompanied by teacher reflections.

There were three kinds of results from the case studies: (1) Project Revisions, (2) Illustrations of Teaching and Learning, and (3) Diagnostic Toolkits. Project Revisions take classroom and instructional strategies employed by the case teachers as the basis for curricular changes. While curricular design was not an explicit goal of this research, it became clear that often times curriculum revisions are the best representations of what we learned from the cases. Illustrations of Teaching and Learning come closest to the original goal of this research. Illustrations make visible teaching strategies or curriculum structures that effectively support student learning so that LC users can capitalize on those strategies and structures. Finally, the cases have been used to design Diagnostic Toolkits that index student challenges doing particular tasks to responsive instructional strategies. While space limitations do not permit a full examination of case content the following is a brief summary of what was learned from the cases (for a full description of results see Shrader, In Progress).

Project Revisions: A number of project revisions emerged from the cases. For example, the modeling task in ReNUE was revised between Judy and Lou’s case. Judy’s students built scale models to represent what they learned but focused on aesthetics rather than on science content. So, the curriculum was revised to incorporate the use of Model-It.

Two types of illustrations were identified. Illustrations of Learning Around Curricular Structures document learning opportunities created curricular structures. For example, analysis of students’ use of prompts embedded in Model-It reveals that prompts cause students to stop work, ask questions, seek help, and revise their models. Thus, prompts are a focal point of student learning. Such analyses have high practical value for LC users as a means of helping them capitalize on the use of prompts. Illustrations of Learning around Teaching Strategies have a similar goal: to make visible the teaching strategies employed by case teachers and demonstrate their relationship to student learning. For example, Judy employs a deliberate questioning strategy every time she joins a group. The purpose of that strategy is to bring her up to date on what the students know so that she can determine whether students need help and, if so, what kind of help to provide. The case analysis endeavors to make that strategy visible to and; therefore, sharable with LC users.

Finally, much of the case analysis goes to the development of Diagnostic Toolkits that respond to challenges that students face around two major task structures in ReNUE: Research and Modeling. Analysis of the case materials reveals a set of systematic bottlenecks that students encounter as they conduct research on the one hand and complete modeling tasks on the other. For example, when students are completing research tasks they encounter four consistent challenges: (1) narrowing their focus to one question; (2) locating relevant sources; (3) reading sources; and, (4) summarizing what they learned. Much like (Minstrell, 1992) work on facets Diagnostic Toolkits are intended to help LC users to identify these challenges when they arise in their own classrooms and to provide them with teaching strategies that respond to each challenge.

Based on these results the following design decisions emerged:

1. Where teacher cases could be used to inform curricular improvement, we have done so. The revised activities in turn are illustrated by the classroom examples that motivated the redesign.
2. The narratives for teacher cases are constructed to: (a) highlight the challenges that students face when doing particular tasks in ReNUE and to provide examples of the teaching that responds to those challenges, (2) illustrate the affordance of particular curricular structures (e.g., Model-It prompts) to provide opportunities for learning and emphasize the need for teachers to capitalize on those opportunities, and (3) illustrate the affordance of particular instructional strategies to support student learning.
3. A category was added to the Curriculum Zoomer to index the Diagnostic Toolkits to the ReNUE curriculum. The new category (called Problems & Opportunities) indexes the categories of the Diagnostic Toolkits to the Research and Modeling Phases of ReNUE. Here teachers can zoom in to learn what student difficulties to expect and examine suggested strategies for responding to those.

At its core the LC includes examples that illustrate students’ conceptual challenges and the corresponding instructional strategies that teachers have used to address those challenges. Figure Two shows such an example. In the figure there is a video clip selected to illustrate students’ work defining objects for their Model-It models. Below the video clip text unpacks the student challenges and teaching depicted.

Figure Two: Living Curriculum Content

The question from a performance support perspective is how to help users locate examples like this when they are important to their work in the classroom. We have already discussed the three indices. Here we discuss how each of those indices might be used to access a clip like the one in figure two.

While the curriculum browser is pointed to the modeling phase of ReNUE a user might click on "Problems & Opportunities" from the Curriculum Zoomer. This would produce a list of student difficulties with modeling one of which would be "Deciding what to include in a model." By selecting that option the user would get a page like the one in figure two. Alternatively, with the browser pointed to a modeling activity in ReNUE, the user might click on one of the teacher cases. This would produce a page that describes how the teacher taught that lesson. Figure four shows such a page (relevant to a lesson in which students prepare to use Model-It by building a paper prototype of their model). It is organized into headings that show how the teacher did the lesson step-by-step. Each heading has a brief text that describes that step and a link to a video like the one in Figure Two. Finally, to access content using the Ask a Question index, the user simply selects a question from those listed on any given page of the ReNUE curriculum.

Figure Three: Modeling — Teacher Case

User testing of the LC prototype revealed that representations like one in Figures Three are ineffective. First, they are too text dependent and do not effectively communicate the organization of the underlying content. Second, the language used in the headings is not clear to teachers.

Current work on the LC focuses on experimenting with alternate representations of these pages. One example of that work is shown in Figure Four. Here blocks are used to chuck content into episodes. In this case there are two episodes. The first is a "Classroom Instruction" episode. The second is a "Group Work" episode. Within episodes icons are used to communicate types of examples. There are three example types shown: "Classwork", "Group work", and "Teacher Reflection." Teacher reflections are also indented to show their relationship to the clip that precedes them. Finally, the text from each heading is hidden until the user mouses over the heading (as shown by the grayed heading). Our hope is that this representation is easier to skim, more effective at communicating the point of each clip, and helps users get more easily to relevant content.

Figure Four: Teacher Case — Alternate Representation

Work on the LC proceeds on three fronts. We continue iterative design and user testing of LC indices and representations of content. Our goal is to complete a version for use in schools during the 1999-2000 academic year. We are also working to design tools that support the creation and indexing of new content. The work required to create the current prototype is not scalable to support the development of content for multiple LC units. We need tools that allow for the collection, analysis, representation, and indexing of content. Finally, we are working on the development of generative tools. Generative tools would allow end users to: (1) make notes on contents in the LC, edit the content of the LC to create more personalized versions, ground on-line discussions with their peers in content from the LC, and add content of their own to the system.

We have begun work on simple generativity tools that allow users to make a note about a particular page in the LC. Once the note is created any time the user links to system displays the note. While the note feature currently works using cookies, we are working on a plug in tool to support note capacity. The next step is to link the note to the larger database so that notes can be shared with a user community on email or discussion lists. Once the note feature is fully functional, we will begin work on editable web pages. Here the idea is to allow the end user to make personal versions of web pages by editing the content of any page and saving those changes on the server where the will be served again the next time the user browses that page. For this functionality we look to Phelps & Wilensky (1996) work on multi-valiant documents.

We have described the need for performance support that helps teachers learn to teach PBS units as they use PBS units. The Living Curriculum is an attempt to provide that kind of support. We began design work by drawing on teachers to inform both the system indices and the underlying content. The idea is to use teachers’ knowledge to create indices that they can use to access content relevant to concerns that arise while they are teaching the unit. We believe that such indexing is important because the system must respond to the articulated needs of teachers to be useful. The goal is to deliver helpful content to the user on the user’s terms. This is a system constraint relevant to performance support. To be useful performance support should be responsive to rather than prescriptive of use.

Our goal is to test the LC at scale during the 1999-2000 academic year to determine whether or not the system provides teachers with the kind of support they need to learn about PBS practice. On the basis of that feedback the system will be redesigned and we will begin designing tools to support the creation of additional LC units as well as tools that support generativity.

In addition to the authors Daniel Edelson, James Spillane, Dale Reed, Richard Halverson, Rachel Kern, Colette Marine, & Joseph Gray contributed to this work.

This research is supported by RePP grant (#9720423) from the National Science Foundation.

(1) ReNUE is an environmental science project designed by the Center for Learning Technologies in Urban Schools in which students identify an environmental problem, research that problem using Internet and library resources, build a model of the problem, propose a solution, and defend their model and solution to their class (Shrader et al., 1999).

(2) Model-It is systems modeling software designed to scaffolds students through the modeling process.

Ferguson, W., Bareiss, R., Birnbaum, L., & Osgood, R. (1992). ASK systems: An approach to the realization of story-based teachers. The Journal of the Learning Sciences, 2, 95-134.

Gentner, D. (1988). Analogical inference and analogical access. In A. Prieditis (Ed.), Analogica . Los Altos, CA: Morgan Kaufmann.

Kolodner, J. L. (1993). Case-Based Reasoning. San Mateo, CA: Morgan Kaufmann Publishers, Inc.

Kraijcik, J., Soloway, E., Blumenfeld, P., Marx, R., Ladewsky, B., Bos, N., & Hayes, P. (1996). The Casebook of Project Practices: An example of an interactive multimedia system for professional development. The Journal of Computers in Mathematics and Science Teaching, 15(1/2), 119-135.

Lampert, M., & Loewenberg-Ball, D. (1998). Teaching, Multimedia and Mathematics: Investigations of Real Practice. New York: Teachers' College Press.

Lento, E. M., Dharkar, A. A., & Gomez, L. M. (1999). Its the practice: When and why teachers talk on listserves. Paper presented at the American Educational Association Annual Meeting, Montreal, Quebec, Canada.

Minstrell, J. (1992). Facets of students’ knowledge: A practical view from the classroom. Paper presented at the Theoretical and instructional implications of various students’ conceptions in science.

Phelps, T. A., & Wilensky, R. (1996, March 20-23, 1996). Toward Active, Extensible, Networked Docuements: Multivalent Architecture and Applications. Paper presented at the Proceedings of the ACM Digital Libraries '96, Bethesda, MD.

Richardson, V. (1998). Teached Education and the Construction of Meaning. In G. Griffin (Ed.), Teacher Eucation for a New Century: Emerging Perspectives, promising practices, and future possibilities (NSSE Yearbook) . Chicago: University of Chicago Press.

Shrader. (In Progress). Design Research for the Living Curriculum. Unpublished Dissertation, Northwestern University, Evanston, IL.

Shrader, G., Williams, K., Lachance-Whitcomb, J., Finn, L. E., Walker, L., & Gomez, G. M. (1999, March 11-12, 1999). Work in the Work Cirlce: Collaborative design to improve teaching practice. Paper presented at the Spencer Foundation Conference on Collaborative Research for Practice, New Orleans.

Spiro, R. J., Coulson, R. L., Feltovich, R. J., & Anderson, D. K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In V. Patel (Ed.), Tenth Annual Conference of the Cognitive Science Society (pp. 375-383). Mahwah, NJ: Lawrence Erlbaum.