FREE ELECTRONIC LIBRARY - Abstract, dissertation, book

Pages:   || 2 | 3 |

«Abstract: This paper reports on results of a randomized controlled trial evaluating the impact of an online tutoring system on Grade 6 students’ ...»

-- [ Page 1 ] --

Randomized Controlled Trial (RCT) Evaluation of a

Tutoring System for Algebra Readiness

Carole Shandy Steven A. Weiling Christopher

Beal Hauk Schneider Li Harrison

University of Arizona WestEd

Abstract: This paper reports on results of a randomized controlled trial evaluating the impact of

an online tutoring system on Grade 6 students’ mathematics learning. Starting with 71 teachers randomly assigned to use or not use the AnimalWatch program to replace 15% of mathematics class lessons, the study ended with 58 teachers’ classes (35 Treatment, 23 Control). Data from over 1200 students are included in the analysis (795 Treatement, 496 Control). Small but significant effects were found. Treatment group gains were larger than Control group gains on a major standardized test and on project-based topic quizzes (Cohen’s d=0.3). Exploration of fidelity of implementation indicated several factors at work and that the minimal effective implementation of this type was in classes where students completed at least 8 of 14 lessons spaced across several months. Software use records indicated that 75% of student-item interactions resulted in students persisting to a correct answer.

Objectives The study goal was to evaluate the impact of an online tutoring system on Grade 6 students’ mathematical proficiency relative to business-as-usual classroom instruction.

In particular, the research question driving the work: Do Grade 6 students who use the program AnimalWatch as replacement for about 15% of mathematics instructional time show greater learning gains than those who do not use AnimalWatch? The project responds to the clear need to improve the mathematics proficiency in the United States.

In international comparisons, U.S. students score in the average range overall, and much lower than students from other nations that are comparable in terms of economic development (Gonzales, Williams, Jocelyn, Roey, Kastberg, & Brenwald, 2008;

Organization for Economic Cooperation and Development [OECD], 2010). Although there is some evidence that mathematics scores have improved over the last decade the growth remains slow, and in many states there has been no improvement at all. There is special concern about algebra as a gatekeeper course for subsequent success, pointing to the need to ensure that middle school students are prepared to succeed in algebra with thorough proficiency in basic computation, fractions, ratios and proportions and other algebra readiness topics (Stein, Kaufman, Sherman & Hillen, 2011; Wimberly & Noeth, 2005). Thus, educators, researchers and policy-makers are searching for approaches that can help to improve students’ learning outcomes in math.

One possibility for addressing the identified need is the use of tutoring software for mathematics learning (Kulik, 2003). Such software is designed with the goal of providing individualized instruction to the learner via the use of features such as integrated explanations, videos and other multimedia resources to help them master concepts. In contrast, standard paper-based worksheets and end-of-chapter practice sets do not provide interactive, customized learning options. The opportunity to direct their own learning while using software may also provide a motivational advantage for some students. For example, struggling learners can choose to look at a multimedia lesson or explanation on the computer in private, whereas they might be quite reluctant to do so in the “live” classroom. Software also provides integrated tracking (e.g., problems completed, number completed correctly and so on) that can inform both students and teachers about progress in real time. These features are argued to lead to more effective learning (Bloom, 1984; Cohen, Kulik, & Kulik, 1982; Graesser, Conley & Olney, 2011;

vanLehn, 2011; Woolf, 2009).

Theoretical framework A considerable body of work suggests that the ideal learning context is one-on ­one instruction with an experienced human tutor (Bloom, 1984). Human tutors present problems that help diagnose the student’s sources of difficulty, choose problems within the student’s Zone of Proximal Development, scaffold the student to a successful solution, and then attribute the success to the student’s effort and enhanced understanding (Brown, Ellery, & Campione, 1994; Lepper, Woolverton, Mumme, & Gurtner, 1993). Human tutors balance the goal of providing challenge to ensure cognitive conflict, resolution and improved understanding, with the goal of sustaining the learner’s motivation and persistence (Merrill, Reiser, Ranney, & Trafton, 1992).

Unfortunately, it is very difficult for math teachers to provide much individual help in large classes. Many Grade 6 math teachers have four to five classes a day with 25-30 students or more per class. It is physically impossible for even the most dedicated and skilled teacher to diagnose and assist every student. Nor is it necessary when technology ­based systems can perform some of these functions, providing students with individualized practice opportunities and, in turn, providing teachers with real-time reports of students’ progress.

AnimalWatch is an intelligent tutoring system designed to give individualized instruction by selecting problems that will build a Grade 6 student’s proficiency (Beal & Arroyo, 2002). Like other intelligent tutoring systems, AnimalWatch uses student models (representations of what the student is estimated to understand, in relation to the target domain: here a network of 30 math topics mapped to the California Grade 6 curriculum frameworks) to guide problem selection. Though not the topic of the current report, we note that the student’s actions while working on problems (e.g., answers, clicks to view multimedia help resources) can be researched to update our understanding of the nature and importance of individual tracking of a model for each student.

Although tutoring software is designed to include features intended to promote learning, research results in the K-12 classroom have been inconsistent. A recent meta-analysis of a wide range of computer-based instruction concluded that the use of technology was generally associated with improvements to student achievement (Tamin, Bernard, Borokhovski. Abrami & Schmid, 2011). VanLehn (2011) reviewed a battery of tutoring systems that adapt instruction in ways similar to what a human tutor would do and concluded that such software was as effective as human tutoring. Several studies suggest that there may be benefits to the use of tutoring software specifically for mathematics learning. Arroyo, Woolf, Royer, Tai and English (2010) reported that students who used a tutoring system for geometry along with modules for basic skills practice showed significant improvements in mathematics problem solving. Barrow, Markman and Rouse (2009) concluded that a computer-assisted instructional program for pre-algebra had a significant benefit for students, in comparison to business-as-usual   2   instruction. Beal, Arroyo, Cohen and Woolf (2010) found that a tutoring system for middle school mathematics was comparable to small-group tutoring delivered by a mathematics teacher and also showed benefits in a quasi-experimental design study with a business-as-usual comparison condition. Ritter, Kulikowich, Lei, McGuire and Morgan (2007) reported positive results associated with the Cognitive Tutor, an instructional system for high school algebra that has been widely used in classrooms.

Although results from these studies are encouraging, conclusion are limited because they have typically involved fairly brief interventions, and the use of studyspecific mathematics tests as outcome measures. In contrast, research using more rigorous experimental methodologies and more standardized outcome measures has not demonstrated a consistent benefit associated with tutoring systems. For example, a review of four widely-used mathematics tutoring systems for mathematics conducted by the What Works Clearinghouse concluded that there was no overall evidence that the software was associated with better student mathematics test performance (Campuzano, Dynarski, Agodini & Rall, 2009). Thus, there is a need for more systematic research on the potential benefits of tutoring systems to support mathematics instruction, and how teacher implementation may influence the effectiveness of tutoring systems. The study reported here was designed to address this need through a rigorous evaluation of the impact of one tutoring system for algebra readiness on students’ mathematical proficiency.

Methods Experimental design and participants The study was a randomized controlled trial (RCT). Teachers were recruited and signed participation consent forms before being randomly assigned to either the Treatment or Control condition. Treatment teachers participated in a two-hour online professional development module designed to introduce them to the software and its features, and then integrated it into their instruction throughout the school year. Software use replaced approximately 15% of instructional time, usually through weekly visits to a computer lab or use of laptop carts in the classroom. Control teachers continued with business-asusual instruction. The student sample was consenting Grade 6 students. Though begun with 71 teachers (36 Treatment group, 35 Control), due to student movement and teacher attrition, we ended the year with 1,010 in 35 Treatment classes and 636 students in 23 Control classes across 40 districts. Their districts identified about 20% of students as English Learners. In most districts, a letter sent home to caregivers notified them of study activities and offered an opt-out form to abstain from their child’s scores being included in analysis; in one district, active caregiver consent was obtained for all participants.

Tutoring system The AnimalWatch software evaluated in this study focused on word problem solving, generally considered a central component of mathematics proficiency (Koedinger & Nathan, 2004). As recommended by the National Council of Teachers of Mathematics (NCTM, 2000) and Common Core State Standards (CCSS, 2010) AnimalWatch integrates mathematics with meaningful content. In this case, with authentic environmental science material, focusing on narratives about tracking and monitoring the status of endangered and threatened species (hence the AnimalWatch system’s name). By connecting mathematics problem solving with science, the student encounters   3   many examples of mathematics in real-world problem contexts, at a point in the curriculum when many students begin to complain that mathematics is disconnected from their lives. The topic of environmental science is engaging to many pre-teens, both boys and girls, and aligns with the science frameworks for middle school (Grade 6 focuses on Earth Science, and Grade 7 on Life Science) as well.

Word problems. AnimalWatch includes approximately 1400 word problems organized into 20 learning objectives. Fourteen of these learning objectives were considered “core” for the study. Each learning objective centers on one topic in sixth grade mathematics such as operations with integers, fractions, or rates. Within each learning objects are narratives about endangered and threatened species, including the Giant Panda, Przewalski Wild Horse (Takhi), Right Whale, California Condor, Snow Leopard, White Shark, Great Cats, and Snakes of Asia. Each word problem includes an introduction with authentic background information, a graphic (image, figure or table) and a question derived from the introduction. Scientific terms in the word problems that may not be familiar to students are linked to an integrated glossary. Students enter their answer, including units, into an answer box and receive immediate feedback. For example, in one problem, a white shark is described as swimming from the ocean surface down to a depth of 30 meters below the surface, and then swimming up to a depth of 16 meters. The numbers in the problem are consistent with current scientific knowledge about shark behavior. The student is asked to compute the total distance traveled. Students must understand the concept of the number line, the analogy of the ocean’s surface as the zero point, and the concept of elevation (defined in the integrated glossary) to solve the problem. The problem is linked to the state’s mathematics standards for Number Sense (working with integers) and Mathematical Reasoning (deciding how to approach the problem, evaluating the answer for reasonableness in the context of the original situation). If the student makes one or two problem-solving errors, text feedback about accuracy is presented, followed by an operations hint (e.g., “No, that’s not quite right.” “Are you sure you’re subtracting?”). A third error elicits a recommendation to view the associated help resources. When the student clicks on the “help” icon, a menu window appears, showing the options available for that topic, including text explanations (e.g., how to find the least common denominator), worked examples, interactive solutions, and video lessons. Students’ requests to view help resources are automatically tracked, and these process data will be extracted for later analyses. A fourth error elicits the correct answer, which the student is required to enter before moving on to a new problem. Students also have the option of deciding a question is “too hard” and the answer is shown, but the question marked as incorrect.

Basic skills practice. Royer et al.’s (1999) work indicates that improved fluency with simple computation and mathematics facts is linked to improved problem solving performance on achievement test items. Thus, AnimalWatch includes skill builders that provide students with opportunities to build fluency through practice with multiplication facts, recognizing decimal-fraction-percent equivalents (e.g., identifying that 1/8 = 0.125 = 12.5%), rounding and estimation, one-variable equations (e.g., finding the value of x when 2x = 10), finding the least common denominator of two unlike fractions, and operating with negative integers. Each skill builder set presents the student with a series of 10 mathematical statements, one at a time (e.g., 35 + (-15) = 20) which the student identifies as either true or false; the immediate feedback is a green check if correct and   4   red x if not. Skill builders can be repeated multiple times with new items. The skill builders are motivating for students because they can achieve near-perfect scores with repetition, and repetition in turn helps to builds fluency with the concepts and operations, indicated by decreases in time required to solve the items.

Pages:   || 2 | 3 |

Similar works:

«Healing Lessons: Urban High School Teachers Learning to Teach Black Youth with Post-Traumatic Stress Disorder By Dawn Ligaya Custodio Williams Ferreira A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Education in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Jabari Mahiri, Chair Professor Na’ilah Suad Na’sir Professor Malo André Hutson Spring 2012 Healing Lessons: Urban High...»

«1 DAVID MONTGOMERY 434 Clinton Street, Apartment 1 · Brooklyn, NY · 11231 day (212) 998-5869 · evening (718) 923-0573 email: dm635@nyu.edu EDUCATION Ph.D. in Educational Theatre, New York University, New York, NY Nov., 2006 Concentration: Drama Education Dissertation: Living an Arts Partnership: The Experience of Three Middle School Teachers in a Drama Residency Dissertation Committee: Professors Philip Taylor (Chair), Margot Ely, & Elizabeth Quintero M.A. in Educational Theatre, New York...»

«{2015} OUTDOOR LEARNING project Thank you to our funders, team members and community partners for their valuable contributions to this pilot project and being part of making special wild places in Vancouver. FUNDERS Vancouver Foundation: Greenest City Community Grant program, Vancouver Board of Parks and Recreation, CityStudio, SFU Semester in Dialogue CITYSTUDIO TEAM {leads on the grant and the development and roll-out of the project} Project Supervisors: Janet Moore & Duane Elverum,...»

«The Ideology of U.S. Spanish in Foreign and Heritage Language Curricula: Insights from Textbooks and Instructor Focus Groups Item type text; Electronic Dissertation Authors Al Masaeed, Katharine Burns Publisher The University of Arizona. Rights Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is...»

«1 Wolfgang Butzkamm From drill to discourse. Exploiting the combinatorial power of language. The lecture is an attempt to revitalise pattern practice methodology with concrete examples taken from various classrooms. Observations of how learners acquire languages naturally without pedagogical interventions provide a theoretical foundation. In the first part, I argue that double comprehension (understanding both functions and forms) is the most important single factor in language acquisition. I...»

«УДК 371.13:002 ББК 74.580.22 С 28 А.С. Сейтбатталова Начальник отдела международных проектов и программ департамента международного сотрудничества Павлодарского государственного университета им. С. Торайгырова; Email: mailto:SeitbattalovaAS@yandex.ru ПЕДАГОГИЧЕСКАЯ СИСТЕМА...»

«Designing and Developing a Quality Online Course Faculty Reference Guide Garrett College Office of Distance Learning Teaching Online: The Tutorial to Designing and Developing a Quality Course Office of Distance Learning Fall 2010 301.387.3155 dlearn@garrettcollege.edu What do you need to complete this Tutorial?1. You need to have completed your Basic Blackboard training or have extensive experience in using Blackboard. 2. You need to be enrolled (as instructor) in an online Blackboard course...»

«MODELING PROTEINS, MAKING SCIENTISTS: AN ETHNOGRAPHY OF PEDAGOGY AND VISUAL CULTURES IN CONTEMPORARY STRUCTURAL BIOLOGY by Natasha Myers B.Sc., Honours Biology, McGill University, 1997 M.E.S., Masters in Environmental Studies, York University, 2001 Submitted to the Program in Science, Technology and Society in Partial Fulfillment of the Requirements for the Degree of Doctor in Philosophy in History, Anthropology, and Science, Technology, and Society at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY...»

«Jl. of Technology and Teacher Education (2006) 14(3), 531-541 The Impact of Online Teaching on Faculty Load: Computing the Ideal Class Size for Online Courses LAWRENCE A. TOMEI Robert Morris University Pittsburg, PA USA tomei@rmu.edu This study examined the impact of substituting didactic instruction, face-to-face advisement, and conventional evaluation with distance-based delivery of content, electronic counseling, and online assessment. It analyzed the impact of distance learning demands on...»

«MERLOT Journal of Online Learning and Teaching Vol. 10, No. 3, September 2014 The Importance of Student-Instructor Connections in Graduate Level Online Courses Sheila A. Joyner Assistant Professor Department of Educational Leadership Sam Houston State University saj014@shsu.edu Matthew B. Fuller Assistant Professor Department of Educational Leadership Sam Houston State University mbf005@shsu.edu Peggy C. Holzweiss Assistant Professor Department of Educational Leadership Sam Houston State...»

«1 SANTANDER, SPAIN EDUCATION INTERNSHIPS AND SERVICE LEARNING PROGRAM INFORMATION Table of Contents: Overview: page 2 Dates: page 2 Requirements: page 2 Map of Spain page 2 About Santander: page 3 About the school: page 4 How much does it cost: page 5-7 a. Tuition b. Travel c. Accommodation d. Passports and visas e. Travel insurance f. Ground transportation Other important information: page 7 a. Study abroad advisors b. Spanish language c. Assignments Scholarships: page 8 The application...»

«The development of Scotland’s Curriculum for Excellence: Amnesia and Déjà Vu Mark Priestley*a and Walter Humesb a University of Stirling, bUniversity of the West of Scotland Scotland’s new Curriculum for Excellence (CfE) has been widely acknowledged as the most significant educational development in a generation, with the potential to transform learning and teaching in Scottish schools. In common with recent developments elsewhere, CfE seeks to re-engage teachers with processes of...»

<<  HOME   |    CONTACTS
2016 www.abstract.xlibx.info - Free e-library - Abstract, dissertation, book

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.