WORKING PAPER PROGRAMS FOR MIDDLE SCHOOL MATH: An Inventory of Existing Technology Andrew Saultz 1W ichigan State University Policy ‘\1ml_\‘515 fur ( 7211M n‘ma lxduczltn m PACE h {[131}, /\\‘\\'\\;cdp( )licflnmJ my; Programs for Middle School Math: An Inventory of Existing Technology Andrew Saultz University Distinguished Fellow, Educational Policy Program Michigan State University February 2012 INSTITUTE OF GOVERNMENTAL STUDlES LIBRARY FEB 16 Ni? UNlVERSlTY OF CALlFORNlA Policy Analysis for California Education (PACE) is an independent, non—partisan research center based at Stanford University, University of California, Berkeley and University of Southern California. The Views expressed in this working paper are those of the author, and do not necessarily reflect the Views of PACE or its funders. Any errors of fact or interpretation that remain are the responsibility of the author and not of PACE. Executive Summary This report provides information regarding existing technology programs that are used to teach middle school mathematics. The goal is to provide an overview of options currently available, and to organize the information for policymakers. This inventory of programs is subdivided into three categories: all inclusive programs, targeted programs, and apps. All-inclusive programs are designed to provide instructional, practice, and evaluation materials. Targeted programs differ because they attempt to provide only one of these three component parts. In most cases, targeted programs focus on providing additional practice and/or evaluation of students. Many of the targeted programs work to assist students who have struggled in the traditional learning environment. The final category of programs, apps, is focused on providing students with an engaging experience to practice their mathematical skill. However, these stand-alone programs are not usually associated with a specific curriculum. This inventory shows that a wide variety of programs use technology to teach middle school mathematics. However, research on these programs is limited. While a few of the programs have participated in randomly controlled trials (RCTs), the vast majority has not been evaluated for effectiveness. As a result, we encourage researchers and policymakers to explore this gap in the research. What is clear is that a number of options currently exist for policymakers and educators looking to incorporate technology into the teaching of middle school mathematics. The following should not be considered an all-inclusive list of every mathematical program available, as this is a rapidly evolving industry. Nevertheless, this report is intended to provide a broad inventory of existing programs. Introduction The purpose of this report is to provide an overview of the existing technology programs and curricular materials for teaching middle school mathematics. The implementation of the Common Core Standards provides an opportunity to explore the possibilities in which technologies and online resources can be utilized throughout California’s education system. Further, the budget crisis has pushed policymakers, practitioners, and researchers to explore all options. This fiscal challenge, combined with an ever- growinglpressure to produce student achievement results, has created enormous pressure to identify and implement effective programs and materials. Our goal is to outline options for teaching mathematics in middle school in California, with the hope of helping policymakers better understand these issues. I Math curriculum has been a contentious issue in the State of California for many years. While textbooks, pedagogy, curricula, and the teacher credentialing process have all been heavily debated, there is also an understanding of the vital importance of learning mathematics effectively in middle school. One specific area of political controversy has been the issue of whether or not to require algebra to all eighth graders or not. The previous State Board of Education chose to implement a system that allows students to take algebra in the 8th or 9th grade. Mastering algebra is vital for students in two significant ways. 2 First, algebra helps students move from concrete to abstract thinking. This intellectual leap is vital in many courses throughout high school, and beyond. Until algebra, most students have only studied concrete ideas. Secondly, learning algebra provides the necessary tools and knowledge for students to succeed in a number of courses that are required to graduate high school in the state of California. Students need algebra to successfully master many science courses including physics and chemistry. As a result, middle school mathematics is vitally important to the long-term success of students throughout California. Policymakers need to work to craft a coherent, effective plan that utilizes the best learning materials. This inventory is designed to help guide that work. California has both the California State Standards (CSS) and the California Common Core Standards (CCCS). The relationship between these two standards exemplifies the political difficulties of educational policy for middle school mathematics. Requiring all students to complete Algebra I in the eighth grade is one significant area of contention. The State Board of Education (SBOE) did not pick a side in this debate, and has adopted both sets of standards: the CCSS (which presents Algebra I as a sequence that culminates in 9th grade) and the CSS, which require students to complete Algebra I in the 8th grade. Local districts and educators are left to figure out which set of standards to follow. A 21 member Academic Content Standards Commission (ACSC), established by Senate Bill X5 1 in January 2010, developed the California Common Core Standards. A substantial amount of the CCCS overlaps with the California State Standards. Major differences in the math portion include the organization of the standards themselves, options available to eighth grade students, and shifting some of the skills to new grade levels. The goal of the CCCS was to develop standards that were research- based, and more closely aligned to international benchmarks. The most recent projected timeline for implementation of the math portion of the CCCS is an adoption of new frameworks in January 2012, adoption of new instructional materials in August of 2014, and new instructional materials available for schools in December of 2014. This schedule is tentative. Background Information Mathematics education in the United States has long been a debated topic. Part of the controversy stems from US students performing consistency at the bottom end of the range of industrialized nations on international math assessments (Cogan et al 2001; Schmidt et al, 1997). In particular, the Third International Mathematics and Science Study (TJMSS) provides significant empirical evidence that students in the United States are scoring well below similar students in other nations. These data have been analyzed and largely used to argue that substantial changes need to occur in the teaching, curriculum, and course offerings of the middle school level (Schmidt, et al, 1997; Cogan, et al, 2001). Additional research has explored the use of technology in education. In some cases, positive effects of using technology to improve student achievement have been found (Reid—Griffin & Carter, 2004). Another possible benefit of using technology in the classroom is that it may lead to better student engagement. Since most technology programs are highly individualized, students are able to target the exact concept or area where they need help. Other research has found that students are more willing to 3 make mistakes when working individually on computers. Recent programs have expanded to also include a substantial data collection and analysis component as well. Teachers, or parents, may use this information to better help individual students, or identify areas where groups of students may need additional help. Clearly, there are many benefits to using technology in the classroom. Current Online options- An Inventory As one might imagine, current technologies and online options vary substantially. Part of this variance is due to the relative ease with which these materials can be developed. In fact, some existing math programs utilize open source software, or other free online materials. The goal of this inventory is to provide a broad overview of existing materials. This is not an all-inclusive list; however, we outline major programs as well a representation of programs that took differing approaches. Please see the annexed table for a comprehensive list of current options to teach mathematics. Some of these options do not directly address teaching algebra or teaching middle school students, but focus more generally on mathematics in middle school. To better organize the programs that we view as viable options, we have organized the current programs into three sections. The first grouping is a list of programs that seek to be "all inclusive" math programs. These programs attempt to teach, they let students practice, and they evaluate student knowledge. The second grouping includes programs attempting to help struggling students, or students who do not understand a particular subject(s) in the traditional setting. We will refer to this grouping as "targeted" programs. Finally, the third grouping is programs designed to allow students to practice their skills. We will refer to these programs as "apps." All-inclusive programs Many current technology and online sources contain components that teach students new ideas, let them practice concepts, track their progress, and even tests their mastery. Some of these programs also sort data for educators and parents. These programs vary in their depth, breadth, costs and effectiveness. Districts have also used these programs to supplement traditional education. Other districts have moved into a more formalized structure where one of these programs essentially encompasses the math program. The I Can Learn program works to provide a complete educational experience. One distinguishing feature of this program is that it has proven positive statistical results through randomly controlled trials (RCTs). The two main studies of 3541 and 2400 students respectively, have been peer reviewed by the US Department of Education's What Works Clearinghouse (Kirby, 2004; Kirby, 2006). Both reviews concluded that the studies met the RCT standards and had positive effects. A number of other reviews by the What Works Clearinghouse have not met the WWC evidence standards or have met the standards with reservations. The online program includes a warm up section, a lesson presentation with detailed lectures, an [Can Learn tutor, and quizzes. The tutor program prompts students to answer questions throughout the lesson, and includes a progression to help them proceed. If a student misses a question, the tutoring program offers hints, and then connects the student back to the part in the lesson that addresses the concept. The quizzes are graded instantly and the students receive feedback about 4 their performance and what questions they missed. The benefits of this program are its very detailed materials, its tutor component, and its data system. One possible drawback of this system is its costs. The Kahn Academy is another all—inclusive program. This program, founded by Sal Kahn in 2004, is a series of 10-20 minute YouTube lectures. Each lecture is delivered by Sal, who has three degrees from MIT. The only requirement to be able to access the Kahn Academy materials is a Google or Facebook account (both programs are free). These lectures target specific concepts for students to learn. The program also has a "knowledge map" where students complete math exercises to earn rewards called badges. The knowledge map begins with basic elementary math concepts (addition 1) and moves through a series of calculus lessons. To advance, the students must successfully answers ten questions on a given topic in a row. If a student struggles, or decides they do not completely understand a given concept, they can link back to the appropriate lecture. The Kahn Academy also includes a coach login, where a teacher and/or parent may be connected to a student account. This allows a teacher to track topics a student has covered, how long they spent on that topic, the lectures they watched, and their proficiency. The benefit of this program is that it is very user friendly, it is free, and it tracks data well. Possible drawbacks include that it doesn't follow a prescribed curriculum and it requires the student to be driven. The Kahn Academy is currently piloting Kahn Academy 2.0 in Los Altos. Evidence from the pilot program is expected around January 2012. The Ignite Math Program is another all-inclusive program. This program uses Ignite coaches to work alongside classroom teachers. The project based approached works to develop concrete goals for each classroom. The program provides multimedia videos and interactivities with offline print material, and works to develop lessons to connect with multiple intelligences. Ignite Math has lessons that are divided into units and topics, student course books, and teacher guides. Advantages of this program seem to be its comprehensive approach, its diverse pedagogical approach, and its sustainability. Possible drawbacks include cost and a data system that is not as developed as other programs. Future School Math is a program with a twenty-five year history rooted in student tutoring. The program uses a "smart diagnostic tool" to analyze various student assessments. The program places a heavy emphasis on individualized education, and develops a plan for each student. Formative assessments are used in coordination with video tutorials, study plans, and student monitoring. While this program was initially a program that focused on struggling students, it has expanded in recent years. Advantages of this program are the formative assessments and individual customization. Possible disadvantages are that it is expensive, and that the activities do not seem as engaging. The Sketchpad Lesson Link is a program that aligns to the California standards and textbooks. The program uses a problem-based approach to curriculum, and provides teachers with student materials and teacher resources. Face-to-face workshops focus on student work and classroom videos to allow teachers and teacher leaders an opportunity to assess their work. This is combined with on-site and virtual coaching that provides hands-on support to improve instructional practices. Finally, the program also provides moderated online courses to help teachers learn and implement the Common Core Standards for Mathematics. Clearly, the primary benefit of this program is its focused and detailed 5 vary in cost from $50 to $300. This set of apps clearly engages students, but has not been formally studied to evaluate effectiveness, and does not directly tie to any specific curriculum. Possible Options and Problems The rapid advance of technology has allowed many programs to enter the market quickly. The benefit of this pace has been a massive expansion of the number of programs that are working to address improving mathematics education. However, this massive proliferation has led to wide variation in quality. To create an app for an iPod, for example, requires one to have knowledge of the technology. One problem is that this knowledge does not necessarily equate to knowledge of pedagogy, teaching and learning, or even what is best for students. A second major problem with the rapid expansion of technological programs for teaching mathematics is a lag in the research on these programs. While a few of these programs have worked hard to test their programs with empirical research, the vast majority has not been tested. A curricular spine may be an option that California policymakers contemplate for the future of middle school mathematics education. This would provide a structure of a course without necessarily prescribing a specific way of obtaining results. This would provide flexibility for the incredible diversity between and within schools throughout the state. Teachers and/or schools would be allowed to adopt the various component programs to complete the curricular spine that they deemed best for their constituents. This option may also be attractive due to the variance in the quality and accessibility of technology between districts. This is one example of an option that would not force policymakers to choose one exclusive approach. There are other policy options that may help integrate technology into the classroom more effectively. Some scholars have suggested reforming the contiguous counties rule that restricts student choice (lzumi, et al, 2010). For virtual charter schools, this is a significant obstacle. Izumi also recommends allowing multiple authorizers for charter schools, allowing teacher reciprocity, and attaching funding to each child as a way of providing additional policy supports for technological innovation (lzumi, et al, 2010). These options should be considered as policy looks to integrate more technology into schools. Looming Questions There are a number of key issues that need to be addressed as policymakers consider the future of mathematical education in California. One important question to ask and address is: what is the current technological capacity of schools throughout California? As policymakers consider the future role of many of these programs, the status of the infrastructure in schools will be very important. This is especially true considering the current fiscal restraints at the state and local levels. A thorough assessment of hardware, broadband quality, and software within California schools would be useful for policymakers moving forward. Another important question is: what is the current teacher capacity for integrating technology into the classroom? With the increasing pace of technological reform and progress, it is important to consider possible challenges to implementation. One possible area of difficulty is that teachers may or may not be able to integrate these technologies effectively. Information should be gathered on the current teacher workforce, its knowledge of instructional technologies, and its ability to use these tools to help identify next steps. Policymakers should consider providing professional development opportunities for teachers and a realistic timeline to phasing new materials into the classroom to help lessen possible implementation problems. Long term, policymakers should consider adjusting teacher training to better prepare them to integrate technologies. Next Steps The goal of this report is to provide a broad inventory of existing programs that use technology to teach middle school mathematics. While this information will assist policymakers in the future, it is not meant to inform policy exclusively. Gathering a group of experts within the field of teaching mathematics will be an important step to prioritize and evaluate these programs. Additionally, a group of technology experts may help policymakers consider specific details regarding infrastructure and capacity when implementing these programs. Conclusion It is clear that California mathematics education at the middle school level can improve. Student performance on state, national, and international math assessments has fallen short of public expectations. A policy window has been created with the implementation of the Common Core Standards and the current financial crisis. Policymakers should look to explore new instructional materials to improve student learning. There are a plethora of programs that attempt to improve mathematics education for middle school students. These programs vary substantially in their approach, design, and effectiveness. The three categories of programs that emerged through this research were the all-inclusive programs, targeted programs, and apps. All-inclusive programs strive to provide instruction, problem sets for students to practice their skills, and assessments that track student performance. Most targeted programs were designed for sub-populations of students who struggled in the traditional learning environment. These programs usually work to instruct in a different way, or allow students to practice their skills at their own pace. Apps are the newest category of programs, and allow students to practice their skills in creative and exciting ways. These programs do not address instruction or assessment, and usually require students to have access to expensive hardware. There is substantial variation within each major category. The California budget situation is not going to improve anytime soon. As a result, policymakers should work to ensure to maximize the effectiveness of every dollar spent on education. The use of technology 9 to teach middle school mathematics may improve student learning in a cost-effective manner. Careful analysis of the various program options is an important next step. All three categories of programs provide important possibilities to enhance student learning in the future. 10 Jiji Math/ Targeted 19 states, A spatial/ $7500 unlimited The initial emphasis on Teacher cannot track Mind Research Institute 1000 schools, temporal approach, Jiji is a license, or $3000 a the visual is better for student progress. mindresearch.net 13,000 penguin who works year for 60 student ELL students. Mostly K-5 games teachers, through a number of license, and $200 a Is entertaining. with only a little based in Santa problems that increase in year for the renewal Research shows algebra preparedness. Ana, CA difficulty. Students begin fee. increase in test scores. ‘ the program visually and It is research-based. are slowly introduced to language and symbols. Khan Academy/ All Piloted in Students work through a A completely free Student tracking for This program is not khanacademy.org inclusive. schools in Los progression in what is service available teachers. ties to any specific Altos, CA, but called a map of knowledge, online via YouTube. Lectures are brief and standards. has not beginning With basic informative. It has not been expanded addition and moving Students can control extensively explored formally into many schools. through calculus. The site has over 2100 lectures (about 25% are math related). Each topic has a 10-12 minute video followed by exercises to test student understanding. if students respond correctly 10 times in a row, they can move up to the next topic. If a student is struggling with a topic, there are links to the appropriate lecture. Teachers can monitor their students' progress (how much time they took, percentage of correct answers, which lectures they viewed). Videos are also available in Spanish. the pace. at scale. 11 E2020/ Targeted Used as a Students logon to an online Unknown. 0 Lessons are presented 0 Currently only used education2020.com credit recovery system that has lectures, in a professional for credit recovery. ‘ program practice problems, and manner. 0 This program has not nationwide. exams. Schools or teachers 0 There is quality been used as a base can set a paSSing rate for tracking for student curriculum before. each exam, and can require performance. students to watch films. It 0 There is the ability to is a WGll organized site. logon any time (whilst still being tracked). Agile Mind/ Apps USEd It mostly covers Visual Unknown. 0 An entertaining site. 0 The program appears agilemind.com sporadically math. Students logon to o Excellent tracking to be more like a throughout play online games and mechanisms. game than an actual the country, complete exercises to learn lesson. especially in math. There is online 0 Lessons are not as the South. tracking of students. The closely tied to learning Originated in program shows both objectives. Austin, TX. individual and group mistakes. Assessment and Targeted Throughout This is an artificially Unknown. 0 Spanish and English 0 This program does not Learning Knowledge the country. intelligent assessment and instruction available. appear to be tied Spaces (ALEKS)/ learning system With 0 It avoids MC. closely to any aleks.com adaptive questioning that Reassesses students curriculum. quickly determines a frequently. 0 There are less teacher student’s knowledge. The tools than in other program reassesses itself programs. throughout to make sure the student is in the correct learning place. It also has knowledge of space theory and avoids MC. Education Program for Targeted Mostly in CA, This program is for K-7; it Roughly $500 per 0 Challenging 0 Minimum test scores Gifted Youth (EPGY)/ but it is also sets an individualized pace; course. curriculum. for students to use. epgyschools.stanf0rd.edu expanding. it reports data to teachers 0 Quality data reporting, 0 This program is and assisting students; it was created initially for expensive. 0 Designed as an 12 hlgh achievers. advanced course. IXL/ Targeted All over the The website is designed to $199 per year per This program is It was designed for ixl.com country. help students practice classroom. standards based. struggling students. math (K-8) with real-time Teacher views are real- it has tests for reports to teachers. It time reports. students but it does includes state standards Can click on any skill not seem to have and proficiency analysis. It (255 for 8th grade) and many instructional is also recommended for it will take you to the materials. struggling students. exercises. Ignite! Math/ All- Sporadically This site features Unknown. Lessons are divided We are not sure if igniteforschools.com inclusive used multimedia videos and into units and topics. progress is tracked to throughout interactive content with Student course book the teacher or not. the country, structured offline print provided, It has not yet been though it materials. The program Teacher guides used extensively. appears to be works to teach multiple provided. a new intelligences and provides program. formal assessments and quizzes alongside informal assessments in print materials. Grades 4-8. Explore Learning/ Targeted Used This program is research Unknown. Gizmos seem advanced The program seems explorelearing.com throughout CA based and is designed to and interesting. more supplemental as and the supplement the curriculum. An engaging program it is not designed as a country. It uses ”gizmos" to teach with a very creative stand-alone program. (like electronic presentation. manipulatives. Future School All- Used This program has been in $35 per month for 5 Data tracking appears Activities do not seem Mathematics/ inclusive nationwide use over 25 years. It student accounts and to be extensive. as engaging. and provides tutoring for one parent account, Individualized It is expensive. futureschools.com internationally. students; a Smart Diagnostic Tool for analyzing student assessment; formative assessment; video tutorials; study plans; and student or, $95 per student, peryean education plan for each student. Study plans. It has a CA curriculum already entered with CA courses. Lectures are dry. 13 monitoring. Sketchpad Lesson Link/ All- Used This site provides $1199 for a small 0 It is tied to CA it doesn’t seem as keyonline.keypress.com inclusive. nationwide. introductory information school (less than 500 standards. innovative. with concept or skill students); $1899 for a o It is very well it has less data- development. It provides a medium school (500- organized. tracking. math overview for grades 1000 students); and 0 provides group or It is expensive, 3—12, with activity notes $2499 for a large individual plans. and student worksheets school (1000-2500) . it is a problem-based that is aligned to CA students. curriculum. standards and textbooks. The Geometer’s Targeted CA, This visualization tool A school license with o It has a strong visual Not a full curriculum. Sketchpad/ nationwide. provides introductory 50 computers is $1000 component. It is supplemental. dynamicgeometry.com information alongside per year. 0 individualized pacing. concept or skill development for grades 3— 12. Tinkerplots: Dynamic Targeted CA A data exploration website $700 for a school 0 Strong data analysis Restricted to working Data Exploration / for grades 4-7 only. license with 50 program. with data sets. keypress.com Students can insert "9W computers. 0 Students learn about Expensive for limited data or explore existing data sets, graphs, scope. data sets; build graphs, text charts, etc. and photos. Brightstorm online Targeted Nationwide Over 2000 very detailed Most of the lectures o Lectures are very There is nothing to Videos for Math Exam lectures from algebra to are free but test technically detailed. prepare for algebra. Prep/ calculus. Test preparation preparation materials 0 Much information. This program may be Brightstorm.com for PSAT, SAT, and ACT. must be purchased for . The site is well too advanced for an unlisted sum 0f organized. middle school money. students. Lectures may be too dry. It does not provide teacher resources or tracking tools. | Can Learn Program] All- New Orleans, This program presents a $300 per year for an There are quality lectures. icanieanfiom inclusive. but has warm-up section, lesson online professional Materials and sections are 14 developm teachers grades 6-8. presentations with lecturers, l CAN Learn tutors and quizzes that are graded upon completion. expanded. * This information was collected in the summer of 2011 and may not be up-to-date. 15 Works Cited Logan, L.S.; Schmidt, W.H.; Wiley, D.E. (2001). Who takes what math and in which track? Using TIMSS to characterize US. students' eighth-grade mathematics learning opportunities. Educational Evaluation and Policy Analysis, Vol. 23, No.4, pp. 323-34l. Conklin, H.G. (2009). Purposes, practices, and sites: a comparative case of two pathways into middle school teaching. American Educational Research Journal, vol. 46, No.2, pp. 463-500. lzumi, L.T., Murray, V.E, Murray, J.D., Stacey, E.B, Chaney, R.S., Randolph, ID. (2010). Short—circuited: the challenges facing the online learning revolution in California. Pacific Research Institute, San Francisco, CA. Kirby, P. C. (2004). I CAN Learn® in Orleans Parish Public Schools: Effects on LEAP 8th grade math achievement, 2003-2004. New Orleans, LA: Ed-Cet, Inc. Kirby, P. C. (2006). I CAN Learn® in Orleans Parish Public Schools: Effects on LEAP 8th grade math achievement, 2003-2004. New Orleans, LA: Ed-Cet, Inc. National Center for Educational Statistics, Pursuing Excellence: A Study of US. Eighth Grade Mathematics and Science Teaching, Learning, Curriculum and Achievement in International Context, Content 1996; Available at:http://www.ed.gov/NCES/TIMSS/index.html. National Council of Teachers of Mathematics, Curriculum and Evaluation Standards for School Mathematics. Reston,V A, 1989. Reid-Griffin, A. & Carter, G. (2004). Technology as a tool: applying an instructional model to teach middle school students to use technology as a mediator of learning. Journal of Science and Technology, Vol. 13, No.4, pp. 495-504. Schmidt, W.H.; McKnight, CC; and Raizen, SA. (1997). A splintered vision: an investigation of US. science and mathematics education. Academic Press, Boston, MA. 16 U. C BERKELEY LIBRARIES \IIIIIIIIIIII C1197003‘IE