How this Unit was Developed

Unit Conceptualization

GSLC's unit goals were to develop an evolution unit that integrates heredity, evolution, argumentation, and opportunities for data analysis. Providing teachers with educative, "just-in-time" support materials became a central frame for the unit. Conceptualizing and outlining the unit occurred through researching the Framework and NGSS, multi-day consultations with a science advisory board comprised of university biology faculty, researching the science and science education literature, organizing and sequencing disciplinary core ideas to build a conceptual storyline, and identifying phenomena that best fit with the unit's goals. The unit underwent three rounds of development and testing: Phase 1 - Natural Selection module, Phase 2 - unit pilot, Phase 3 - unit field test.

The phases of development and testing are outlined below:

Phase 1: Natural Selection Module

The team initially developed one module on a central topic in evolution, Natural Selection, which eventually became the fourth module in the unit. The module included science content, opportunities for data collection and analysis, and activities that built skill in argumentation. The data were from published scientific studies. The argumentation activities formed the basis of the unit-wide argumentation scaffold that builds students' capacity to produce written arguments from evidence.

The module underwent two rounds of local classroom testing in 2013, involving GSLC researcher observations and student pre/posttesting. The first classroom enactment identified areas where the materials required clarification, and the second identified how the materials and draft assessment items could better align to the module's learning goals. A third enactment occurred at an out-of-state school and provided written and verbal feedback from a teacher who was not involved in developing the materials or teacher guides. All three rounds informed module revisions. The pre/post assessment measures were developed in parallel by an external partner, Project 2061 (assessment measures and additional instrumentation used in the research are described in more detail below).

In spring 2014 the module was pilot tested nationally with 461 students in 20 biology classes, taught by seven teachers. The teachers were chosen to represent a broad array of teaching settings and student demographics. Data analysis showed statistically significant learning gains from pretest to posttest. Survey data revealed that the lessons were easy to enact, contained appropriate math, language, and science content, and that teachers would use the materials again. We used feedback from teachers and students to inform revisions and refinements of the Natural Selection material. Together, the assessment and survey data suggested that our approach showed promise. We learned some lessons along the way that we could apply to a larger unit.

Phase 2: Evolution Unit Development and Pilot testing

Beginning in 2014, the GSLC developed the storyline for the remaining four modules using a plan outlined in the initial conceptualization phase. The team identified appropriate phenomena and relevant published scientific data for the unit, drafted the unit-wide argumentation scaffold based on the claim-evidence-reasoning framework, drafted paper and multimedia lessons and activities, and designed embedded formative assessments.

Lessons and activities from two new modules were tested in one local teacher's classrooms, paying particular attention to the argumentation scaffold. GSLC researcher observations, teacher interview data, and student informal interview data provided feedback for lesson revisions.

With its growing understanding and confidence in the evolving unit's conceptual flow, classroom utility, and effectiveness for learning, the GSLC team completed drafts of lessons within each of the unit's five modules. To establish degree of alignment to the NGSS, an external reviewer, AAAS Project 2061, conducted an alignment evaluation using the Educators Evaluating the Quality of Instructional Products (EQuIP) rubric (Achieve Inc., 2016). Based on the analysis, revisions included clarifying and making more explicit the unit's alignment to various components of the NGSS.

To test the entire unit's classroom utility, usability, and effectiveness for student learning, GSLC conducted a two-phase unit pilot test. In the first phase, GLSC invited 20 biology teachers from 41 states representing diverse teaching contexts (geographic, ethnic/linguistic, student achievement, block vs. daily instruction schedule) to inform further revisions during a summer 3.5-day in-person institute (2016). In the second phase, these 20 teachers implemented the unit in their 9th and 10th grade introductory biology classrooms (2016-2017 school year). Teachers provided extensive feedback via daily teacher logs, end-of-implementation written surveys, and through conversations with researcher-observers. Based on this feedback, the lessons were revised and new lessons were added. Ten teachers participated in the fall and ten in the spring, which allowed for testing of activities that were revised based on fall teachers' feedback.

Results from 944 students who completed both pre- and posttests showed statistically significant student learning gains. Survey results indicated that all teachers planned to use all or parts of the unit in the future, and that the unit built students’ conceptual understanding of evolution.

Substantial revisions to the unit were made based on data from the pilot test, including:

• streamlining lessons in order to reduce the overall length of the unit from ten weeks to eight;
• developing additional online activities to eliminate content gaps and support student learning;
• creating videos that are closely aligned to module content to replace ones from other sources that were not as well-aligned;
• adding explicit connections to Crosscutting Concepts and Science Practices to individual activities on the teacher websites
• adding clearer enduring understanding goals at the end of each module;
• creating less time-consuming formative assessments for each module;
• reformatting copy masters to save paper;
• clarifying lesson procedures, content, and questions that students and teachers found confusing; and
• refining online and paper-based activities to eliminate distracting elements and to address learning goals more efficiently.

In addition, new teacher support materials were developed to help teachers enact the unit, including creating short ("just-in-time") teacher training videos for each module and for activities that are more complicated to enact. The module training videos provide a justification for and "road map" of concepts and practices contained in the module's lesson sequence. More complicated activities to enact or prepare, such as the unit's several modeling exercises, benefit from a visual preview of the materials in use.

Phase 3: Evolution Unit Field Testing (randomized controlled trial)

The final step of curriculum testing included a randomized controlled trial (RCT) during the 2017-2018 school year. The RCT was designed to compare pre/post learning gains made by students whose teachers were randomly assigned to either the treatment (GSLC unit) or control (business as usual) condition. Participants in both conditions received an introductory webinar on the research procedures. For the treatment condition, participants received a 1-hour orientation to the unit but otherwise were required to use the unit in sequence, without modification, or the addition of external materials.

Forty-four teachers (21 in treatment and 23 in control), representing 23 states, and recruited from the GSLC email list, participated. This efficacy trial enables us to explore new questions about the mediating factors that might influence teacher and student outcomes. We will pay particular attention to the fidelity of curriculum implementation, and the differences in instruction between the treatment and control conditions. Results from the randomized controlled trial field test will be reported in summer 2018.

Development of Assessments and Surveys

Pre/post assessment items for each phase of module and unit testing were developed and piloted nationally by an external partner, AAAS Project 2061, and revised according to established assessment procedures (Herrmann-Abell & DeBoer, 2014; DeBoer, et al., 2008). Iniitally, GSLC and Project 2061 collaboratively drafted a set of shared learning goals to be used to guide the development of both the curriculum materials and the assessment measures. Assessment tasks were aligned with these learning goals and associated dimensions of the Framework and NGSS and incorporate published scientific data but do not use the same phenomena as the lessons. To maintain the rigor of the research throughout the process of testing, neither partner saw the other's work; i.e., the GSLC curriculum developers did not see the assessment items and Project 2061 did not see the unit.

AAAS created items targeting shared biochemistry, common ancestry, heredity, natural selection, speciation, and the practices of data analysis and argumentation. Items were pilot tested with 4,588 middle and high school students throughout the United States during the fall of 2015 and spring of 2016. Four test forms were created for online pre/post-testing students during the fall 2016 and spring 2017 pilot test of the curriculum. Each test contained a total of 25 items including seven linking items. Data were analyzed using the Rasch model through the software package WINSTEPS (Linacre, 2016) to obtain measures of item difficulty and student performance level.

Open-ended instruments including logs, teacher surveys, and student surveys were developed at each stage of unit testing by the GSLC's internal and external evaluator. Instruments were vetted and revised iteratively based on feedback from pilot test and efficacy trial teachers and GSLC curriculum development and research staff.

We regularly conduct workshops for practitioners and present at research conference. Look for us at annual conferences for NSTA (National Science Teachers Association; national as well as some regional conferences), NABT (National Association of Biology Teachers), USTA (Utah Science Teachers Association), NARST (National Association for Research in Science Teaching), ASTE (Association for Science Teacher Education), and AERA (American Educational Research Association).

We are preparing manuscripts for publication for both researcher and practitioner audiences on various aspects of the development process, testing results, and descriptions of the unit. Links to published articles will be available here.