Chris.hoadley.name

PROCEEDINGS OF THE
IADIS INTERNATIONAL CONFERENCE
e-Society 2003
LISBON, PORTUGAL
JUNE 3-6, 2003
International Association for Development of the Information Society
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re- use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Permission for use must always be obtained from IADIS Press. Associate Editors: Luís Rodrigues, and Patrícia Barbosa LEARNING, DESIGN, AND TECHNOLOGY: THE
CREATION OF A DESIGN STUDIO FOR EDUCATIONAL
INNOVATION
Assistant Professor, College of Education, and School of Information Sciences and Technology The Pennsylvania State University Director, Learning Design and Technology Program Stanford University School of Education ABSTRACT
This paper describes the creation of a design studio course for students in an innovative graduate degree program in the School of Education at Stanford University. While design studios are common in other fields, they are unusual in education. The rationale for this course is presented, our experiences running the course for four years are summarized, and implications for academic training in the information society are discussed. In particular, we highlight the overlooked role of design training in traditionally research-oriented fields. KEYWORDS
Design-based research, educational technology, curriculum, design studios. 1. INTRODUCTION
The evolution towards an information society has driven changes in all aspects of work, home, and school. In this paper, we describe an unusual attempt in post-graduate education to prepare educationists who can effectively study and apply information technologies in educational settings. First we describe how information technology advances created an unmet need for experts in technology for learning, and how a new graduate program was created at Stanford’s School of Education to meet these needs. Second, we discuss how tensions between research-oriented and design-oriented needs of the learning technology field led to a new design studio course, and how this course combined research and design training. Third, we reflect on our experiences running the course and how information technology challenges us to combine research and design in higher education more generally. 2. THE IMPETUS: TECHNOLOGY AND ITS IMPACT ON
EDUCATIONAL PRACTICE AND EDUCATION DEPAR TMENTS
While educational applications of information technology have existed from the very beginnings of digital computing, education departments at universities have been in some ways slow to absorb technological preparation into the curriculum. Of course, in the earliest days of computing and personal computing, the LEARNING, DESIGN, AND TECHNOLOGY: THE CREATION OF A DESIGN STUDIO FOR EDUCATIONAL technical expertise required to provide technology meant that most educational technologies were driven partly or primarily by technologists and computer scientists (for instance, the LOGO programming language for children was developed at MIT by researchers in computer science and artificial intelligence). Three traditions of educational technology arose in university education departments. The first developed studies of online learning tools as an extension of other forms of media, applying many of the same research techniques that had been applied to filmstrips or movies before the advent of computing in schools, generally with an instructivis t philosophy (Reiser, 2001). The second strand highlighted the role of computers from a policy and society perspective (Cuban, 1996; Means et al., 1993). The third resulted from inventive designs of computers for education, often led by individuals with a background in computing or in a computing-intensive field like math or science (Molnar, 1997) and linked to research in cognition. Although arguably the most inventive, this third strand often failed to have wide impact because the innovations rarely made it into the mainstream. By the 1990s, there was pent-up demand for workers who could create these inventive applications of technology for education outside the “hothouse” of specialized research projects with government funding. In Silicon Valley, many educational technology companies were growing (Lucas Learning, Computer Curriculum Corporation, Sunburst Software, the Learning Company) and other companies were turning their attention to the education market (most notably Apple Computer beginning in the 1980s). Higher education generally failed to provide these workers who were cross-trained in technology development and design, modern theories of learning, and the pragmatics of school classrooms and education policy. It was in this context that Stanford founded the LDT (Learning, Design, and Technology) program. 3. THE CONTEXT: THE LDT MASTER’S PROGRAM AT STANFORD
The Learning Design & Technology (LDT) program at Stanford is a 12-month master’s program in the School of Education where students come together as a community of practice around becoming “learning designers.” The program provides students with an intensive year of study. Three main areas encompass the student experience: (1) taking courses in education, computer science, sketching, prototyping, product design and business, (2) working as interns in local learning design agencies, and (3) designing a final master’s project of their own choosing that addresses a significant learning problem with technology and serves as the culmination of all their theoretical and practical experiences. The program is designed for persons who aim to develop new and better ways to use information technology and new media for learning. LDT graduates continue on to professional work and study in a variety of roles in K-12 schools, school technology support agencies, colleges and universities, nonprofit organizations, and the corporate world. The goals of the program are to train students to be proficient in: • Applying educational theory and cutting-edge research about learning • Exploring the possibilities of emerging information technologies for learning • Working individually and in teams to design creative, effective solutions to learning problems. The program attracts students and applicants from a variety of backgrounds, including education backgrounds such as teachers, community technology center (computer clubhouse) directors, and museum exhibit designers; students from technology backgrounds such as information architects, product managers, programmers, and technology managers; and a variety of others with interests in education and technology including photographers, community developers, scientists, journalists, and book editors. Approximately 15-20 students form each cohort. The program was conceived in the mid -1990s by several faculty members in the Stanford University School of Education, including Decker Walker, Jim Greeno, Mike Kamil, Brigid Barron, and others. The 1997-1998 school year was the first offering of the LDT Master’s Degree as a pilot test. In Fall 1998, we (the authors) both joined the LDT program; Hoadley as a consulting assistant professor on loan from a nearby nonprofit research institution, and Kim as a program assistant while a doctoral student in psychological studies in education at Stanford. Hoadley was entrusted with teaching an (as yet ill-defined) portion of the core seminar sequence for the LDT program, while Kim was responsible for coordinating academic matters and serving as a teaching assistant for the core seminar sequence. IADIS International Conference e-Society 2003 4. THE ISSUE: RESEARCH AND DESIGN
The relationship between research and design in the LDT program is the central story of this paper. Below, we describe how research and design are often understood in academic education departments (in the US, at least) and examine several models from disciplines other than education of how scientific research and design can be linked. 4.1 The schism between research and design
In most fields, design and science are seen as separate activities. Science involves objective testing of models of reality against data, while design is trying to change reality to achieve a desired (possibly underspecified) effect. Traditionally, design is seen as a consumer of science. For instance, a physicist may uncover fundamental laws of mechanics, and the mechanical engineer (designer) then determines how to apply those fundamental laws to a particular problem. In this view, science is “pure research” where design is either “applied research” or not research at all. Indeed, many would argue that these two should not be mixed; if the scientist has the intentionality, the agenda of a designer, he or she may be more susceptible to biasing the results of the study. However, in the social sciences (Flyvbjerg, 2001) and in applications of technology to human problems (Carroll & Rosson, 1992), the division between design and science is less clear. Scientists must produce the phenomenon they wish to study, which often involves significant design (Hoadley, 2002), and designers often must reduce ambiguous (but important) aspects of the design problem through the design process, and often collect data in the traditions of science (Kelley & Littman, 2001). In education, researchers have explicitly identified the need to combine research and design in a deeper way (Lagemann, 2002; Robinson, 1998). One problem is the difficulty of applying theoretical models of learning to problems of teaching (Lagemann, 2002), while another is the degree to which findings are context -dependent and not easily generalized into universal laws like Newton’s Laws in mechanics (Design-Based Research Collective, 2003). Theoreticians have recognized that design can actually form a basis for the principled empirical investigation that is the hallmark of science (Design-Based Research Collective, 2003; Dewey, 1929). Despite the recognized need and attempts over the last 100 years, blending research and design has largely not happened in education (Lagemann, 2000). Academic education departments do often comprise both research (often under labels like educational psychology) and design (under labels such as instructional design or curriculum and instruction). Usually, researchers enjoy high status while designers are treated as practitioners and have less status (Lagemann, 2000). 4.2 Challenge: Applying a research tradition to technology design
At Stanford, the College of Education has a world-wide reputation for excellence in research. In terms of non-technology educational design (such as curricular or instructional design) the School has generally not only provided for training of designers but has done so in the context of research on this training (e.g.Hammerness & Darling-Hammond, 2001). However, the preparation of technology designers was, in 1997-98, a new endeavor. The LDT Program in its pilot year had two primary means of accomplishing this goal. First, the students were placed in internships in area learning design companies or agencies. Second, the students took one course in human-computer interaction which was offered by a combination of the Stanford Computer Science department and the Symbolic Systems interdepartmental program. Yet, while these opportunities provided valuable experiences in technology design, they were largely disconnected from the rest of the curriculum, including the learning and educational context issues covered in the core seminar. In trying to address this in the core seminar, we examined several models of how technology design could be blended with research, including human-computer interaction (HCI), engineering science, and education. 4.2.1 Models from HCI
Perhaps the closest model for blending research and design was the HCI model. In the HCI community, the design of new technological systems goes hand in hand with data collection on those systems, and design LEARNING, DESIGN, AND TECHNOLOGY: THE CREATION OF A DESIGN STUDIO FOR EDUCATIONAL decisions are justified on the basis of prior research in psychology and other social sciences (Card, Newell, & Moran, 1983; Carroll & Rosson, 1992; Norman & Draper, 1986). Indeed, HCI has pushed the forefront of psychology in several areas, most notably via the exploration of psychological models of learning with intelligent tutoring systems (J. Anderson, Boyle, & Reiser, 1985). In HCI, students learn to identify relevant psychological or social science models, to build and refine systems, and to collect data on the interaction of the designed systems with users, comparing the intended vs. actual outcomes. Interestingly, HCI is an example of a domain where the scholarly (research) and practitioner communities overlap. The ACM CHI conference attracts both practitioners and researchers, with both academic -style research papers and demonstrations or “interactive experiences” and design cases that are more practice oriented. 4.2.2 Models from engineering science
In engineering science, design is often viewed as a process of optimization; both theory-building and design of technologies with interesting properties are creative activities sanctioned by the field. Students in this field learn a great deal of theory and analytical skill, and they also learn the pragmatics of making things happen in labs (for instance, how to anneal a particular type of material sample, or how to work with the practical limitations of laboratory measurement equipment.) One common way students are introduced to the relationship between design and research is to be introduced to research-friendly design processes or expose to studies of productive design practices in the domain (Finger & Dixon, 1989a, 1989b). 4.2.3 Models from education
In education, two trends bear examination: how designers already do research, and how researchers already do design. First is research in traditional educational design: instructional design and curricular design. While these two areas differ, in both research is part of the tradition usually through evaluative work. Often, summative evaluations of designs are the Ph.D. theses in these areas. In addition instructional design is well known for a heavy focus on design processes that uncover important elements of the problem, such as needs analysis. Thus designers do, to some extent, already conduct research. A second trend related to the link between design and research is the role played by designers and practitioners in educational research traditions. For instance, the action research tradition (Aguinis, 1993; Masters, 1995) has documented ways in which researchers can productively and empirically explore educational “agendas” in a manner consistent with the scientific method. The recent culmination of these trends has played an important role in the recent prominence of the area of educational research called the Learning Sciences. Interestingly, the learning sciences have been heavily involved in the educational uses of technology as well as interdisciplinary design and research. One methodological approach taken in the learning sciences is the design experiment, in which a design of an educational intervention is researched continuously through its design and deployment (Brown, 1992; Collins, 1992). More recently, this has been more accurately renamed design-based research methods in education (Design-Based Research Collective, 2003; Hoadley, 2002). Students in the learning sciences often experience long apprenticeships in which they learn both design and research skills by working with large, long-running research and development projects. 5. THE DESIGN STUDIO COURSE
In response to the survey of different models of combining design and research, Hoadley undertook to create a course that would train students not only in research and theory but also in application of these to problems of design. The course, titled: “Learning, Design, and Technology: Design Methods” is described in further detail below and was taught and refined over the course of the next four years with help and feedback from Kim and Decker Walker, the professor in charge of the LDT Master’s Program. For a summary of the course activities, see Table 1, or the online syllabus from 1998 at: http://www.ciltkn.org/protected/syllabi/syllabi/hoadley-f98.pdf IADIS International Conference e-Society 2003 Table 1. Major educational activities in LDT: Design Methods 2001-2002 used to support design activities and also to support reflection on design processes personal histories and design processes; students designed websites to represent expert design process to interested others designer” sites with all readings instruction on learning design methods (Small group or individual) technology supported learning sciences Design method articles on models of design processes from other disciplines and education 5.1 Studio models
Borrowing from architectural education, the course was created as a design studio, in which designing was the students’ primary activity. Looking at examples, critiquing and receiving critique, and discussion are other ways students in studios learn. This course drew on all of these methods. The course involved two or three major design projects each year, culminating in a large final project that many students used as a springboard or pilot test for their master’s project. A great deal of class time was devoted to sharing and helping with these final projects. One attraction of the studio model is that it dovetails with other mo dern instructional techniques. In particular, problem-based learning (Boud & Feletti, 1991) suggests students should engage in authentic, open-ended, realistic problems as a context for learning ideas and skills. This provides a context for students to integrate ideas across topics as they apply to a particular problem, and offers some of the complexity they are likely to confront outside of educational settings. However, design studios, though rare, are not unheard of in education, nor do they necessarily help integrate research and design. LEARNING, DESIGN, AND TECHNOLOGY: THE CREATION OF A DESIGN STUDIO FOR EDUCATIONAL 5.2 The role of examples
One means of addressing both design and research skills was to rely heavily on examples. Cognitive research suggests that case-based learning environments do support learning through case-based reasoning (Schank & Cleary, 1995) and these are widely used in professional education such as medical education, business and legal education, and, to some extent, teacher professional development. Over half of the course readings provided examples of work in the learning sciences that combined research with the design of a particular technological innovation. While predominantly math and science examples were included in the course reader, students also brought their own examples from other domains in a weekly “show and tell” time. In many ways, this is similar to professional participation in HCI, in that research and design are both valued as scholarly contributions and formed part of the basis of the course. The students also considered examples of designers and design processes, discussed more below. 5.3 The role of design process
In this course, design process was brought to the fore as an integrator of design and research. The learning sciences examples in the readings contained more information than typical educational research papers on design processes leading up to the interventions described, and were augmented with personal stories either by the instructor or guest lecturers on the evolution of these research projects and their designed technology over time. In addition, a number of readings from engineering, HCI, and education presented explicit models of design such as participatory design, reflective pra ctitioner design, user- and learner-centered design, informant design, and so forth. (For full citations see the course syllabus.) In the first course project, students interviewed designers about their design processes and often uncovered embedded research activities as well. Students explicitly reflected on their own design processes in a number of ways, including in-class exercises using different design processes, by writing in their design journals, and by writing a design process reflection paper at the end of the course. In addition, the whole-class project (added in the final iteration of the course) provided students with an opportunity to synthesize these readings and the interviews with a framework provided by Hoadley. Exposure to, and reflection on, so many design processes that were involved with research allowed students to see a variety of ways research can be embedded in the design process and vice versa, what Hoadley has termed elsewhere research-based design and design-based research (Hoadley, 2002). This process-oriented approach is similar to the process-oriented approach used in engineering sciences and also in instructional design. 5.4 The role of learning theory and traditional educational research
This course built on the students’ earlier experiences in the prior semester, where the core seminar focused on learning theories. In the studio course, students were always responsible to justify their work in terms of learning outcomes. This generally took two forms: either the students could justify their work with elaborated learning theory and design rationale, or they could justify their work with empirical data on their own designs. This type of justification was particularly emphasized for the final project proposal and final report, but also played out in class discussions, critiques of readings, and their master’s project proposals (due during or shortly after the course). Students were also required to conduct at least one empirical investigation of their final projects before turning them in. These empirical investigations were sometimes informal [in the spirit of “Guerrilla HCI,” \Neilsen, 1994 #791] and took several forms. One common form was to perform a summative evaluation of the design near the end of the project. Another form was to perform a formative evaluation of the technology design early in the project. A third form, which harkens to design experiments, was to conduct a quick study to answer a theoretical question that arose during the design process, and without which the students might not know how to proceed. One interesting development was that we had to change some elements of the course to more forcefully emphasize learning outcomes rather than just usability outcomes—because many of the readings were drawn from an HCI tradition and most were concurrently taking an HCI class, students sometimes neglected to examine the learning outcomes, rather than usability, associated with their interventions. IADIS International Conference e-Society 2003 5.5 Socialization into the field
One additional way to help students to link research and design was to help socialize them into the small field of learning designers who combined technology design with research in education. In particular, when students interviewed learning designers, they were encouraged to find exemplary designers, most of whom had some research background and many of whom were primarily researchers and only secondarily learning designers. The Silicon Valley location and the proximity to the University of California at Berkeley as well as the researchers in the School of Education made finding suitable interviewees much easier. Interviewees were invited to final project presentations so other students would have a chance to meet them. In addition, over the course of the semester students developed shared criteria for their design projects, which has been demonstrated to facilitate domain learning (Cuthbert & Hoadley, 1998; Fredericksen & White, 1997). A backgrounded, but equally important, form of socialization occurred when students discussed the prospects for their master’s projects related to the course project with their academic advisers, who emphasized the research side at least as much as the design side. We made heavy use of this aspect of the course to encourage students to think about learning models and not just the usability issues previously mentioned. “Where’s the learning?” became a question posed frequently by both of us, and internalized by the students. 5.6 The CPC model and design ontologies
When learning disparate ideas, students run the risk of developing so-called “knowledge in pieces,” or inconsistent, memorization-based understandings (diSessa, 1988). Hoadley provided two models he developed to help students integrate their understandings in the course. The first, an ontology of design methods (Hoadley, in preparation) provided a framework for understanding different ideas about optimal design processes and to compare them. This framework was used by the students in thinking through the whole-class project: a website on design methods that synthesized readings on design methods and the interview websites. The second framework to help the students, the CPC Model (Hoadley & Kirby, submitted), was used to structure the examples around three major advantages technology can provide in learning environments: advantages in content (multimedia or dynamic representations, simulations, and microworlds; extended information access), advantages in process (task-support, scaffolding, reductions in cognitive load, or productivity enhancing tools), and advantages in context (technology-afforded changes in the social context or milieu of learning, collaborations that cross the boundaries of the learning setting, changes to the learners perceptions of self or others that shift their learning context). These three advantages loosely mapped to the three major theories of learning they were taught: transmission models, information processing/cognitive psychological models, and socio-cultural models of learning. (For an overview of the latter two, see J. R. Anderson, Greeno, Reder, & Simon, 2000). The multiple applications of the CPC Model (Is it a design framework, an analytic framework, or a theoretical framework?) proved to be helpful in getting students to think through how theory motivated their designs, and how designing impacted the kinds of scientific questions could be asked about technology and people in learning environments. 6. OUTCOMES: SUCCESSES AND CHALLENGES
While the course was not perfect, we found great success in using the design studio course to help students integrate their research and design skills and knowledge. Like any instructors, we refined the course over time to improve it on the basis of our experiences, even if those experiences were not culled from formal studies. What we offer below are some informal observations on successes and challenges of the course. 6.1 Where LDT students went
Students from the LDT program have been competitive applicants in the technology market (especially during the dot com boom years in Silicon Valley). Many have found applied, design-oriented jobs in industry using their design skills and their education backgrounds. During the early years of the program, students mainly took jobs in educational software and Internet-related companies such as Ninth House, Achieva, LEARNING, DESIGN, AND TECHNOLOGY: THE CREATION OF A DESIGN STUDIO FOR EDUCATIONAL Yahoo!, LeapFrog, iPlanet, Cisco and Classroom Connect. In addition, some graduates are finding jobs in research and consulting in the educational research sector. Each year some students continue in their studies and pursue research-oriented doctorates in educational technology. A small number of graduates also head to K-12 education and find jobs as directors of technology for a school or district. This is notable because it confirms our belief that these students are capable of both research and design, and it also is striking in how students work in each of the three traditional strands of educational technology represented in academic departments, as well as the fourth area of applied educational design outside of research contexts. The graduates, as a whole, have had a remarkable impact on bringing innovative, constructivist learning technologies to fruition outside the academic “hothouse” of funded research projects. We view this range of successful outcomes as a hallmark for the success of our synergistic approach. Like the Bauhaus School in Germany which attempted to unite the more abstract fine arts community with the more practice-based designers in the skilled trades but whose graduates made advances in both areas (Whitford, 1984), our students have gone on to success that seems to surpass that of graduates of either instructional design programs or more traditional education degrees. In one iteration of the course, one group of students who developed a final project on a structured discussion interface for learning won an HCI competition held on campus, surpassing a few hundred other students, many of whom were majoring in HCI. 6.2 Reactions to the course
One measure of the success of the course is the degree to which its innovations were adopted in other settings. For example, the design journal activity was first initiated earlier by other faculty to the core seminar in the quarter prior to the design studio. Student interest was sustained as students continued keeping design journals even after the studio course was over, even continuing the practice on into their professional lives. Finally, an intergenerational tradition began in which alumni of the program would come back to share their experiences and their design journals with students currently in the program. Another measure of the course’s success is the degree to which it helped students choose design-oriented courses from programs outside of the School of Education. During the quarter prior to and the quarter of the design studio, few students took design-oriented courses as electives. But two courses offered in the quarters following the the design studio (one on designing persuasive technologies and one on product design) grew so popular with the program’s students that one even came to have a special section offered especially for education students. In general students responded favorably to the course and to the program. From 1998 until 2002 the program’s reputation and reach increased as evidenced by greater numbers of applicants from an increasingly international pool. While we do not have a formal student of the course, students have commented on several strengths of the approach that indicate design and research are both combined in their professional lives. One student (spontaneously!) wrote back a year after graduating that the course “was one of the high points of a program that I thoroughly enjoyed,” and that “The readings [the instructor] assigned and the projects… were integral to my growth as a learning designer. I use those lessons to inform my design decisions on an almost daily basis.” This particular student was employed at an e-learning company that produces online courses for secondary schools in the US. 7. CONCLUSION
We have attempted to present one model of how graduate education in learning and technology can embrace the overlap between research and design and use each to strengthen the other. However, we believe the issue of bringing design, theories of human thinking and learning, and social science research skills to bear on problems facing the information society are of importance in all sectors, not just for learning. Much as the business schools have grown their MIS programs, other departments specializing in other facets of human experience will need this type of program so that we may intelligently invent our technological future, and not ignore the lessons and tools disciplines outside technology can bring to bear. The development of human capacity of this sort will become increasingly necessary as the problems we wish to solve become less and less amenable to “canned” (fixed) solutions. IADIS International Conference e-Society 2003 This will not be easy, for two reasons. First, as noted earlier, designers and researchers may not respect each other, and the boundaries between their work are powerfully drawn. Few disciplines (HCI is an exception) support scholars-practitioner dialogue in the same venues. Secondly, and more subtly, we will have to develop new epistemologies for both research and design that help us know how we know in these fields. Already, there are groups confronting this very issue; the issue of granting Ph.D. degrees in design has brought up these same ideas, confronting boundaries and developing new epistemologies in the context of graduate education. Two conferences have been held on the subject, in Ohio in 1998, and in France in 2000 (see: http://www.jiscmail.ac.uk/files/PHD-DESIGN/ohio.htm, and http://www.jiscmail.ac.uk/files/PHD-DESIGN/france.htm for details). We will have to apply these newfound epistemologies and uncover new ways of training and developing capacity. Just as the Bauhaus created a new category of work through the invention of a new way of educating people with theoretical, practical, and interdisciplinary knowledge, we may find that the education of designer-researchers can help catapult us to new ways to apply technology in society. ACKNOWLEDGEMENTS
We gratefully acknowledge the current and past LDT faculty and students, and the guest lecturers and interviewees who made the course possible. Special thanks to Decker Walker for contributing significantly to the refinement of the design studio course over the years, and to Hunter Gelbach for his help with the course in 2000-2001. Thanks to Brian Smith for calling attention to the parallels between designer-researchers in education and the artist-craftspeople at the Bauhaus. REFERENCES
Aguinis, H. (1993). Action research and scientific method: Presumed discrepancies and actual similarities. Journal of Applied Behavioral Science, 29(4), 416-431. Anderson, J., Boyle, C., & Reiser, B. (1985). Intelligent Tutoring Systems. Science, 228, pp. 456-462. Anderson, J. R., Greeno, J. G., Reder, L. M., & Simon, H. A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 29(4), 11-13. Boud, D., & Feletti, G. (1991). The Challenge of problem-based learning. London: Kogan Page. Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. Journal of the Learning Sciences, 2(2), 141-178. Card, S. K., Newell, A., & Moran, T. P. (1983). The psychology of human-computer interaction. Hillsdale, N.J.: L. Carroll, J. M., & Rosson, M. B. (1992). Getting around the task-artifact cycle: how to make claims and design by scenario. Transactions on Information Systems, 10(2), 181-212. Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O'Shea (Eds.), New directions in educational technology. New York: Springer-Verlag. Cuban, L. (1996, Oct. 9). Techno-Reformers and Classroom Teachers. Education Week, 16. Cuthbert, A., & Hoadley, C. (1998). Using KIE to help students develop shared criteria for house designs, Annual Meeting of the American Educational Research Association. San Diego, California: American Educational Research Association. Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Dewey, J. (1929). The sources of a science of education. New York: H. Liveright. diSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. B. Pufall (Eds.), Constructivism in the computer age. The Jean Piaget symposium series. (pp. 49-70): Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. Finger, S., & Dixon, J. R. (1989a). A Review of Research in Mechanical Engineering Design, Part I: Descriptive, Prescriptive, and Computer-Based Models of Design Processes. Research in Engineering Design, 1(1), 51-68. Finger, S., & Dixon, J. R. (1989b). A Review of Research in Mechanical Engineering Design, Part II: Representation, Analysis, and Design for the Life Cycle. Research in Engineering Design, 1(2), 121-137. Flyvbjerg, B. (2001). Making social science matter: why social inquiry fails and how it can succeed again (S. Sampson, Trans.). New York: Cambridge University Press. LEARNING, DESIGN, AND TECHNOLOGY: THE CREATION OF A DESIGN STUDIO FOR EDUCATIONAL Fredericksen, J. R., & White, B. Y. (1997). Cognitive facilitation: a method for promoting reflective collaboration. Paper presented at the Computer Support for Collaborative Learning 97, Toronto. Hammerness, K., & Darling-Hammond, L. (2001). Meeting old challenges and new demands: The Stanford Teacher Education Program in 2001. Paper presented at the Annual meeting of the American Association of Colleges of Teacher Education, Dallas, TX. Hoadley, C. (2002). Creating context: Design-based research in creating and understanding CSCL. In G. Stahl (Ed.), Computer Support for Collaborative Learning 2002 (pp. 453-462). Mahwah, NJ: Lawrence Erlbaum Associates. Hoadley, C. (in preparation). Describing design: Moving beyond methods and principles to richer forms of design knowledge.Unpublished manuscript. Hoadley, C., & Kirby, J. A. (submitted). The content, process, context framework: Supporting design of technology enhanced collaborative learning environments.Unpublished manuscript. Kelley, T., & Littman, J. (2001). The art of innovation : lessons in creativity from IDEO, America's leading design firm (1st ed.). New York: Currency/Doubleday. Lagemann, E. C. (2000). An Elusive Science: The Troubling History of Education Research. Chicago: University of Lagemann, E. C. (2002). Usable knowledge in education: A memorandum for the Spencer Foundation Board of Directors (Memorandum). Chicago, IL: Spencer Foundation. Masters, J. (1995). The history of action research. In I. Hughes (Ed.), Action research electronic review. Sydney: Means, B., Blando, J., Olson, K., Middleton, T., Morocco, C. C., Remz, A. R., et al. (1993). Using technology to support education reform. Washington, DC: U. S. Department of Education. Molnar, A. R. (1997). Computers in education: a brief history. T.H.E. Journal, 24, 63-68. Neilsen, J. (1994). Guerrilla HCI: Using discount usability engineering to penetrate the intimidation barrier. In R. G. Bias & D. J. Mayhew (Eds.), Cost-justifying usability. Boston: Academic Press. Norman, D. A., & Draper, S. W. (1986). User centered system design. Hillsdale, NJ: L. Earlbaum Associates. Reiser, R. A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development, 49(2), 57-67. Robinson, V. (1998). Methodology and the research-practice gap. Educational Researcher, 27(1), 17-26. Schank, R. C., & Cleary, C. (1995). Engines for Education. Mahwah, NJ: Lawrence Erlbaum Associates. Whitford, F. (1984). Bauhaus. London: Thames and Hudson.

Source: http://chris.hoadley.name/papers/Hoadley-Kim-2003.pdf