2013

Evaluation of Science4Us: Perceptions and Impacts

\

About McREL Mid-continent Research for Education and Learning (McREL) is a nonprofit education and

Mid-continent Research for Education and Learning (McREL) is a nonprofit education and research organization based in Denver, Colorado. For more than 40 years, McREL has been helping educators use research to improve student achievement. As a national leader in research and evaluation, school improvement, standards-based education, balanced leadership, professional development, policy development, and scenario planning, our highly respected experts provide services to educators in 50 states and 18 foreign countries. McREL’s client list includes federal, regional, and state agencies; school districts; institutions of higher education; foundations; private organizations; and international entities. Learn more at www.mcrel.org.

The following individuals contributed to this report. Sheila A. Arens, Ph.D.

Contact McREL 4601 DTC Blvd., Ste. 500 Denver, CO 80237 Web site: www.mcrel.org • E-mail: info@mcrel.org P: 303.337.0990 • F: 303.337.3005

Cynthia Long, M.Ed.

Contents

Executive Summary ... 4 Overview and Methodology ... 8 Introduction ... 9 Evaluating Science4Us ... 9 Instruments ... 12 Sample ... 12 Teacher Sample ... 12 Demographic / Background Characteristics ... 12 Approach to Teaching Science ... 14 Student Sample ... 17 Data Analysis ... 18 Findings ... 18 Use of 5E instructional model ... 18 Engage ... 18 Explore ... 19 Explain ... 19 Elaborate ... 19 Evaluate ... 20 Progression of learning across the module, States of Matter ... 20 Recommendations for sequencing the learning: ... 22 Clear learning goals ... 22 Formative Assessment ... 23 Motivation ... 23 Inquiry opportunities ... 24 Teacher support ... 24 Alignment to NGSS ... 25 Summary ... 28 Teacher Outcomes ... 31 Perceptions of Quality, Utility and Relevance ... 32 Student Outcomes ... 35 Observations ... 35 Think-Alouds & Student Assessments ... 39 Student Motivation & Self-Efficacy in Science ... 43 Student Content Knowledge ... 44 Summary ... 47

Tables and Figures

Table 1: Evaluation Questions, Data Collection Methods, and Sources of Data ... 11

Table 2: Gender of Participating Teachers ... 13

Table 3: Grade Levels and Teaching Experience ... 13

Table 4: Highest Education Level of Participating Teachers ... 13

Table 5: Recent Professional Development in Science ... 14

Table 6: Perceived Level of Qualification for Different Subject Areas ... 15

Table 7: Average Number of Days Per Week and Minutes Per Week Spent Instructing in

Subject Areas ... 15

Table 8: Number of Minutes of Science Homework Assigned in a Typical Week ... 16

Table 9: Pretest Scores for Teacher Self-Efficacy in Science and Perceptions of Student

Engagement ... 16

Table 10: Content Assessment (pretest / covariate) ... 16

Table 11: Student Demographic Characteristics ... 17

Table 12: Student Pretest Scores--Computer Comfort, Appreciation of Science, Confidence

in Science ... 17

Table 13: Student Pretest Scores: Science Content Knowledge ... 17

Table 14: Next Generation Science Standards Matter and its Interactions ... 26

Table 15: Integration of Best Practices for K-2 Learning ... 29

Table 16: Teacher Self-Efficacy in Science ... 34

Table 17: Teachers' Perceptions of Student Engagement ... 35

Table 18: Focus of Observed Lessons ... 36

Table 19: Use of the 5E Instructional Model ... 36

Table 20: Science4Us Activities and Strategies ... 37

Table 21: Observed Instructional Strategies ... 39

Table 22: Student Perceptions of Science4Us (% and n in each category) ... 43

Table 23: Pretest and Posttest Student Motivation in Science ... 44

Table 24: Pretest and Posttest Student Self-Efficacy in Science ... 44

Table 25: Student Content Assessment Descriptive Data, All Students ... 46

Table 26: Student Content Assessment Descriptive Data, 1st and 2nd Graders ... 46

Figure 1: Assessment of S4U Module ... 21

Figure 2: Primary Teacher Activities During Observations ... 38

Figure 3: Primary Student Activities During Observations ... 38

Figure 4: Overall Student Content Knowledge Scores By Grade, By Time ... 45

Executive Summary

To date, very little rigorous research has been conducted on early science education interventions, despite the importance of engaging students in the early grades to interest and motivate them to persist in science (Tai, Liu, Maltese, & Fan, 2007). Science4Us (S4U) aims to improve science instruction and student outcomes in science, thus preparing students for future careers in STEM by (1) providing K-2 students with opportunities to control their own learning, communicate and challenge ideas and claims, and revise their ideas if needed in an interactive and innovative environment (CSMEE, NRC, 2000); and (2) providing teachers with embedded, aligned, and on-demand professional development (PD) that increases their capacity to address misconceptions and prior knowledge, deepen their science content knowledge, and provide them with sound pedagogy for science instruction.

In 2013, Science4Us hired McREL to conduct a small-scale evaluation study. Evaluators examined the physical science-States of Matter module with the goal of providing formative feedback on the quality, relevance, and utility of the S4U intervention, and providing insight into how S4U implementation relates to student outcomes such as science motivation and science achievement. Evaluators were guided by four overarching questions:

• Does S4U reflect sound teaching practices? • What are teacher reported benefits of S4U? • What is the experience of student users of S4U? • What are the student outcomes (science, knowledge, motivation and self-efficacy in science) associated with S4U?

Teachers were invited to participate in the study in early spring 2013. Teachers were not randomly assigned to implement S4U or conduct business as usual; as such, the study design is quasi-experimental. Seven teachers, assigned to the “Early” implementers group were asked to implement the S4U States of Matter module in late spring 2013 with their 156 students. Four “Late” group comparison teachers were asked to implement business-as- usual science teaching in late spring 2013 with their 116 students.

A McREL science content expert examined the States of Matter module to determine the extent to which S4U reflects sound teaching practices. In particular, the McREL evaluator was assessing whether the 5E instructional model was reflected throughout the module, whether a progression-sequence across a lesson (clear storyline) was evident, whether there were clear learning goals, the extent to which opportunities for formative assessment were apparent in the module, whether the module was motivating/engaging for students, whether the module contained inquiry opportunities for students, the extent to which the module provided support for teachers (including instructional strategies, content-related support, and technology support), and the extent to which the module was aligned with the

Next Generation Science Standards. This analysis identified a number of strengths as well as several areas of potential improvement.

Some of the strengths of the program include that S4U accommodates the uneven and episodic ways that early elementary students tend to learn, as students can engage at their own pace—repeating activities as necessary to ensure understanding. Moreover, students can engage in learning in multiple ways because of the program’s use of sounds, words, and pictures. The models and animations facilitate student development of clearer mental images, an aspect of the intervention that is imperative as science concepts increase in complexity. The content of S4U is integrated with mathematics and literacy. Evaluators’ suggestions for improvement largely centered on the arrangement of activities to ensure a strong content progression and advice around how to progress through the activities. Other suggestions related to facilitating more seamless implementation such as allowing teachers to print all activities and improving the teacher guide, allowing teachers to “fast forward / rewind” on all content presentations, and adding more opportunities for cooperative group work.

To understand the perceived benefits of S4U from a teacher standpoint, researchers asked teachers to elaborate on their experiences implementing the intervention. Teachers were asked to log daily activities and to complete a survey about their experiences. Across all eight sessions in the States of Matter module, participating Early group teachers sometimes used the Teacher Guide and sometimes used the offline materials. On the post survey, teachers indicated that the Teacher Guide was helpful for understanding the content, the layout of the lesson, and the pacing of the lesson; however, teachers indicated a need for improved layout of the teaching sequence, as well as an easier-to-read format for the Teacher Guide. Teachers inconsistently used offline materials with their class—this was reported on both the logs and the survey, and was consistent with the sampled classroom observations. Teachers appreciated that offline materials provided information on how to differentiate and supplement; the also appreciated the content of the offline materials. Noted improvements to offline materials included making a black line master available. Teachers reported making less use of the How-to-Videos, with the majority indicating that they never or only rarely consulted these across their two-week implementation of Science4Us. Implementation among Early group teachers varied; research conducted over a longer period of implementation should include minimum implementation guidelines to enable analysis of outcomes by achieved relative strength of implementation.

Researchers also examined teacher feedback to determine the extent to which Early group teachers were pleased with the student materials. Teachers expressed satisfaction with the quality, utility, and relevance related to most aspects of the student materials (including, for instance, the developmental appropriateness of Science4Us, the support for emergent literacy embedded in the sessions, the open-ended nature of the program, the stimulation of student interest, and the encouragement of active student involvement). A few aspects of the student materials were rated less positively (more than 25% of the respondents indicating “Strongly Disagree” or “Strongly Agree”). For instance, several teachers disagreed that the

States of Matter module focused on content their students needed to learn and several indicated that the module was not aligned to the relevant science standards for their students. Because the implementation period was abbreviated, it is possible that a longer period of implementation would provide all K-2 teachers with content aligned to the particular science standards being addressed. That said, all Early group teacher respondents indicated that students learned something from the States of Matter module (29% of teachers indicated their students had learned “a little” whereas the remaining 72% of teachers indicated their students had learned “some” or “a lot”).

Not surprisingly, across both the Early and Late groups, teachers’ perceptions of student engagement were high at both pre- and posttest, regardless of whether students were in the S4U group or the business as usual group. Observers noted that the majority of students were engaged in the lesson activities during the classroom observations of Early group students.

Forty-three percent of teachers indicated that using the Science4Us States of Matter module resulted in changes in their instructional practice. However, the use of S4U during the two- week implementation period did not appear to result in changes to teachers’ self-efficacy in science compared to their counterparts in the Late group (scores increased from pretest to posttest for teachers, regardless of group but differences between Early and Late group teachers at posttest were not statistically significant).

Student outcomes

Researchers used observations, think aloud protocols, student interviews, and student survey feedback to learn about the experience of student users of S4U.

In classroom observations, the focus of the observed lesson was described as mostly working on science concepts, but working on some facts/vocabulary. Observers also looked for evidence of the 5E model in these classrooms; evidence of the 5E instructional model was mostly observed in during use of online materials, and—during the observed class period—explanation and elaboration activities were observed more frequently than the other components of the 5E model (though all 5E components were observed). In other words, teachers were utilizing the intended 5E components of S4U. Observers also looked for evidence of teachers engaging in activities or using strategies indicative of Science4Us implementation. Observers noted instances of teachers providing explanation for the states of matter, lessons that included Best Practices from Teacher Guide, lessons that included Misconceptions from the Teacher Guide, and the use of literacy skills and strategies. Teacher use of Science4Us was generally limited to the online materials and teachers made minimal or no use of the Science4Us suggested discussion questions or extension activities during the observed classes. During these observations, the most common teacher activity was supporting the Science4Us work of the whole class and the most commonly observed student activity was participating in a whole class online activity.

Researchers used think-alouds as a means of gathering information about student experiences with and perceptions of Science4Us. Students were asked to navigate through various Science4Us features and encouraged to verbally explain what they were doing as they worked through the different tasks. Researchers found that many students appeared unsure of how to access different Science4Us features. This might be explained by the instructional models employed by the teachers of the students who participated in thinkalouds: although one teacher created opportunities for students to individually engage with Science4Us in a computer lab, two teachers exclusively used the program in a whole class setting. Once students entered the two target activities (“Three States” and “Take a Note”), they were able to successfully complete each activity. Although student feedback about each activity was positive, six students described a preference for activities in which there is a correct answer.

Researchers also used an assessment to collect data on student beliefs about their competence in science, enjoyment in learning about science and engaging in science activities, and ease of learning science. Students’ agreement with positive statements (“I like science”) ranged from 70 percent to 100 percent of items; on average, students agreed with 90 percent of statements. Students in the Early group were asked to reflect on Science4Us. Over 80% of students reported positive experiences with Science4Us, felt that it was beneficial to their learning of science, and felt that the program helped them like science more.

Finally, researchers collected data on student appreciation of science (student motivation) and confidence in science (self-efficacy) prior to the start of the study and after the conclusion of implementation. There were no significant differences between Early and Late group students on the motivation in science measure. However, researchers found a statistically significant difference between Early and Late group students on their selfefficacy in science, with Late group students scoring higher at posttest than their Early group peers. In other words, students who did not use Science4Us appeared to have improved their self-efficacy in science compared to the group of students who used S4U. This finding may be explained by the fact that students in the Science4Us Early group were exposed to more unfamiliar—and perhaps more challenging—science concepts, which resulted in decreased confidence to engage in science. However, in terms of content knowledge in science, researchers found that all students increased in their content knowledge over time, but that Early group students significantly outperformed their Late group counterparts. Student self-efficacy in science may have diminished in the Science4Us group, but their content assessment scores improved relative to the comparison group. To test the robustness of this latter finding, researchers examined these data without the Kindergarten class. Again, researchers found a significant difference between the treatment groups, with Early group 1st and 2nd grade students outperforming their Late group 1st and 2nd grade counterparts.

Overview and Methodology

To date, very little rigorous research has been conducted on early science education interventions, despite the importance of engaging students in the early grades. Research suggests that the majority of scientists and graduate students pursuing degrees in science developed their interest in the field prior to middle school (Maltese & Tai, 2010), so early exposure to science at the middle and younger grades is important in attracting students into science and engineering (Tai, Liu, Maltese, & Fan, 2007). Science4Us (S4U) aims to improve science instruction and student outcomes in science, thus preparing students for future careers in STEM by (1) providing K-2 students with opportunities to control their own learning, communicate and challenge ideas and claims, and revise their ideas if needed in an interactive and innovative environment (CSMEE, NRC, 2000); and (2) providing teachers with embedded, aligned, and on-demand professional development (PD) that increases their capacity to address misconceptions and prior knowledge, deepen their science content knowledge, and provide them with sound pedagogy for science instruction.

This evaluation specifically focused on the physical science-States of Matter module of the comprehensive web-based digital science curriculum of S4U for K-2 teachers and their students. The ultimate goal of the proposed evaluation was to provide formative feedback on the quality, relevance, and utility of the S4U intervention, and to provide insight into how using S4U relates to student outcomes such as science motivation and science achievement. Four overarching questions were used to guide the evaluation and reflect key evaluation outcomes:

• Does S4U reflect sound teaching practices? • What are teacher reported benefits of S4U? • What is the experience of student users of S4U? • What are the student outcomes (science, knowledge, motivation and self- efficacy in science) associated with S4U?

These questions were used to guide research efforts over the project period and to examine the extent to which S4U successfully accomplishes its intended outcomes, with a focus on continuous quality improvement.

Treatment teachers were asked to implement the S4U States of Matter module in April 2013 to May 2013. They will receive ongoing access to the complete S4U program through the 2013-14 school year. Comparison teachers were asked to implement business-as-usual science teaching in May 2013. They will receive access to the complete S4U program at the conclusion of the intervention and through the 2013-14 school year. As a quasi-experimental study, teachers self-selected into the treatment and comparison conditions; however, should the need arise, researchers will attempt to balance the ratio of treatment and comparison teachers.

The evaluation activities and findings described herein occurred between April 2013 and June 2013.

Introduction

In 2007, the National Research Council called on policy-makers and education leaders to provide—at all grade levels—teachers who are adequately prepared in science content and processes, structures that provide adequate time for science instruction, and necessary resources for science instruction. Nevertheless, six years later, across the United States many students reach the third grade having received little to no instruction in science. Teachers are largely underprepared to teach science at the early grades and necessary structures and resources for teaching science well are not provided.

It is not unusual to read reports that young children receive an average of 30 minutes per day in science (down from 45 minutes per day prior to NCLB; Center on Education Policy, 2008) or that instructional time for science in 2008 was at its lowest average since 1988, hovering around 2.3 hours per week— compared with English language arts’ average of 12 hours per week and mathematics’ average of 6 hours per week (Blank,

Elementary- aged students are more likely than their older counterparts to be curious about the natural world—unfortunately, just as interest in science peaks, minimal science opportunities are provided.

arts’ average of 12 hours per week and mathematics’ average of 6 hours per week (Blank, 2012). Research indicates that states with a low average for science instructional hours per week also have low science scale scores on assessments such as the National Assessment of Educational Progress (Blank, 2012).

Moreover, the quality of science instruction at the elementary grades is typically low. Many early elementary teachers are underprepared to teach science, instead supplementing their standard curriculum with disconnected science-like “fun” activities that lack coherence and do not provide meaningful opportunities for student learning to occur. Indeed, novice teachers and teachers who lack confidence about their science knowledge sometimes avoid science instruction or only engage students in science that represents a low risk for loss of classroom control (Davis & Smithey, 2009). In other words, science does not get taught well, or sometimes does not get taught at all, because of a lack of teacher preparation and capacity coupled with a sharp focus on other subject areas.

Evaluating Science4Us

To date, very little rigorous research has been conducted on early science education interventions, despite the importance of engaging students in the early grades. Science4Us (S4U) aims to improve science instruction and student outcomes in science, thus preparing students for future careers in STEM by (1) providing K-2 students with opportunities to control their own learning, communicate and challenge ideas and claims, and revise their ideas if needed in an interactive and innovative environment (CSMEE, NRC, 2000); and (2) providing teachers with embedded, aligned, and on-demand professional development

Children are naturally intrigued and motivated to learn about the world around them; early exposure to science is important to attract students to science and engineering. However, as the demand for good elementary science teaching often does not get the same attention as mathematics and literacy, teaching science at the elementary grades is not often prioritized.

(PD) that increases their capacity to address misconceptions and prior knowledge, deepen their science content knowledge, and provide them with sound pedagogy for science instruction.

To examine processes and outcomes associated with S4U, McREL proposed a small-scale quasi-experimental evaluation study that would enable an understanding of the critical ingredients of the S4U intervention and identify promise for reaching intended outcomes associated with teacher practice and student outcomes. Although the evaluation would be limited by a small sample, the work was intended to examine the following:

• The physical science components of the S4U intervention for grades K, 1, and 2 against best practices for teaching science to students in these grade levels.

• The quality and feasibility of intervention implementation.

• The quality and feasibility of intervention implementation. • The perceived quality, utility, and relevance of student materials and embedded

• The perceived quality, utility, and relevance of student materials and embedded teacher professional development. • Teacher science knowledge, pedagogy aligned to the physical science modules, and

• Teacher science knowledge, pedagogy aligned to the physical science modules, and self-efficacy and confidence in science and science teaching.

• Student perceptions of S4U.

• Student perceptions of S4U.

• Student motivation and self-efficacy in science.

• Student motivation and self-efficacy in science.

• Changes in student understanding related to physical science.

• Changes in student understanding related to physical science.

• What are teacher reported benefits of S4U? • What is the experience of student users

• Does S4U reflect sound teaching practices? • What are teacher reported benefits of

module/program. Answers will also provide data on how implementation relates to teacher and student outcomes. And, given

• What is the experience of student users of S4U?

• What are the student outcomes

• What are the student outcomes (science knowledge, motivation and self- efficacy in science) associated with S4U?

provide data on how implementation relates to teacher and student outcomes. And, given the dearth of information in the field regarding early science programs, the answers to the evaluation questions may also inform the field on best practices for teaching science in early elementary school.

Table 1 provides an overview of the evaluation questions, data collection methods and sources of data.

Table 1: Evaluation Questions, Data Collection Methods, and Sources of Data

This report provides details on the data collection instruments, the sample (teachers and students), data analysis, and key findings. The findings section of the report is organized around the evaluation questions.

Instruments

McREL researchers created or identified data collection instruments to enable the collection of formative and summative (outcome) data. These included surveys and implementation logs for teachers, assessments of student understanding of physical science concepts, surveys of student motivation and self-efficacy in science, classroom observation protocols (for use in a sample of classrooms), and interviews / Think-Alouds for students (for use with a sample of students).

The McREL research team recruited twelve teachers from four schools in Colorado to participate in the study (initial recruitment flyers are provided in Appendix A). The teacher st nd sample consisted of one Kindergarten teacher, four 1 grade teachers, and seven 2 grade teachers.

Prior to the study, teachers were asked to complete a background survey to gather demographic details, information about teachers’ professional background, perceived selfefficacy in science and details about pedagogy (see Appendix C for a copy of this survey).

Demographic / Background Characteristics As shown in Table 2, all participating teachers were female. All participating teachers

Teacher Sample

Table 2: Gender of Participating Teachers

Teachers were also asked to indicate their current grade level, the average number of years they taught at their current grade level, and the average number of years of K-2 teaching experience prior to the current school year. As shown in Table 3, the majority of teachers in nd the Early group taught 2 grade during the 2012-2013 school year and teachers in this group had more experience than their Late group counterparts.

Table 3: Grade Levels and Teaching Experience

Participating teachers were also asked to share details on their education background, including their highest level of education (Table 4), the area of their degree(s), and their exposure to college-level science courses and science-specific professional development (see Table 5).

Table 4: Highest Education Level of Participating Teachers Early Group Teachers Late Group Teachers

Participating teachers varied in the year in which they last took formal course for college credit in science. All seven Early group teachers responded to this question. On average, these teachers had last taken a formal college course in science 23.7 years prior. Only three Late group teachers responded to this question. On average, these teachers had last taken a formal college course in science 10 years prior. Regardless of treatment group, participating teachers had—on average—more recently taken a course on the teaching of

participating teachers had—on average—more recently taken a course on the teaching of science (Early group teachers average = 9.6 years; Late group teacher average = 9 years). However, fewer teachers responded to this question, so it may be that some participating teachers have never enrolled in a course on the teaching of science. Researchers also asked participating teachers to indicate the total amount of time they had received professional development in science in the last twelve months and in the last three years.

Table 5: Recent Professional Development in Science

Approach to Teaching Science

To gain a sense of the confidence with which participating teachers approach science teaching, teachers were asked to describe how qualified they perceived themselves to be to teach different sciences (life, earth/space, physical) as well as other subjects such as mathematics, reading/language arts, and social studies. As shown in Table 6, across both Early and Late group teachers, confidence is generally higher for non-science subject areas.

Table 6: Perceived Level of Qualification for Different Subject Areas

Teachers were also asked to indicate the number of days per week and approximate minutes per day they teach different subject areas (mathematics, science, social studies, reading/language arts). Table 7 provides these data. As shown, both Early and Late group teachers reported spending fewer days per week and minutes per day providing instruction in Science and Social Studies. This is consistent with national trends. Moreover, this suggests that students in Late group classrooms were likely receiving some science instruction throughout the course of the study.

Table 7: Average Number of Days Per Week and Minutes Per Week Spent Instructing

Table 7: Average Number of Days Per Week and Minutes Per Week Spent Instructing in Subject Areas Early Group Late Group Teachers

The majority of teachers indicated that they assign between 0-30 minutes of science nd homework in a typical week. Only one Early group teacher (2 grade) indicated she assigned more (61-90 minutes). Table 8 provides these data.

(2^{\mathrm{n d}}

Table 8: Number of Minutes of Science Homework Assigned in a Typical Week Early Late

Textbooks

Two teachers (one Early group and one Late group) indicated what textbook / curriculum they use most often for teaching science. Both indicated they use FOSS. The one teacher (Early group) who provided feedback on the FOSS curriculum indicated it was “Very Good.”

Teachers were asked to complete additional items to allow researchers to assess their selfefficacy in science, their perceptions of student engagement, and their science content knowledge. Table 9 provides data collected at pretest. Table 10 provides results from the 1 teacher science content assessment, which is later used as a covariate in student analyses.

Table 9: Pretest Scores for Teacher Self-Efficacy in Science and Perceptions of Student Engagement

Table 10: Content Assessment (pretest / covariate) Average Score –

1 Because teachers took the science content assessment at varied times during the two week implementation and because the intent of the States of Matter module was not to increase teachers’ science content knowledge per se, these data were only used as a covariate in student analyses. 16

Student Sample

• What are the characteristics of students in the study schools, and what are the characteristics of students participating?

Prior to the start of the study, researchers asked teachers to administer two assessments to the students. The first of these contained demographic items as well as items to assess student appreciation of and confidence in science. The second was a science content assessment that covered general physical science. Table 11 provides demographic characteristics of the student sample.

Table 11: Student Demographic Characteristics Early

• Data were missing from 18 students.

Table 12 and Table 13 provide pre-assessment scores. There were no significant differences between Early and Late group students on any pretests.

Table 13: Student Pretest Scores: Science Content Knowledge Average Score –

Data Analysis

McREL staff employed both formative and summative evaluation strategies to examine the evaluation questions. Quantitative data analysis primarily consisted of calculating frequencies and using descriptive statistics as well as running analyses of covariance (ANCOVA) using pretest student and teacher scores as covariates (as appropriate).

Findings

Findings are presented in the following sections, organized by evaluation question.

Does S4U reflect sound teaching practices?

To address the question about whether S4U reflects sound teaching practices—including ancillary questions regarding integration of pedagogy (5Es) and content toward the end of creating a logical, sequential progression; alignment with best practices in using technology for teaching science in younger grade levels; and comparison of S4U with best teaching practices, McREL staff compared the physical science components of the S4U intervention for grades K, 1, and 2 against best practices for teaching science to students in these grade levels. More specifically, McREL examined relevant literature on pedagogical approaches in science and used this literature to create a set of rubrics (by grade band) against which S4U was compared (to the extent possible, McREL staff also compared the S4U curriculum against state standards in science).

Use of 5E instructional model

The 5E instructional model provides a framework that can be used at different levels (activity, lesson, unit) and helps students build on their conceptual understanding through a series of experiences, from accessing students’ prior knowledge to assessing student understanding. The “Es”, in order, are engage, explore, explain, elaborate, evaluate. The 5E instructional model, developed by BSCS, “is grounded in sound educational theory, has a growing base of research to support its effectiveness, and has had a significant impact on science education.” A summary table of the 5E instructional model is provided in Appendix

D. An analysis of how each of the 5Es is well-reflected—and not as well-reflected—in the S4U physical science modules is provided below. Science4Us uses the 5E instructional model as intended for the engage, explore, explain, and evaluate; additional effort on elaborate may be warranted (as discussed below). Engage In the Engage session, Science4Us uses an interactive notebook and a story to access prior knowledge and engage students. Students interact with the information by circling objects, watching an animation, and drawing pictures. These types of interactive engagement— coupled with questions—are intended to engage students. The interactivity also allows teachers to assess what students know and provide students the opportunity to make connections to their own experiences, knowledge and skills related to the lesson so that they can begin to build on their understanding. The offline session is aimed at gathering prior knowledge through a KWL chart.

Explore The Explore session gives students the opportunity to choose an object to drop in a tank of water and make a prediction whether the object will sink or float. Students begin to explore the properties of solids and liquids. This activity is fun and interactive—and it gives students practice in making predictions and may lead students to asking more questions or to consider their own pre- or mis-conceptions. However, the direct link between understanding states of matter and properties of matter is not clear in this session. This might be a good opportunity to bring in density in an age-appropriate manner as a property of matter. An extension might include dropping ice in the water (same matter, but different behavior). This session did not seem to clearly align to the learning goal. The offline sessions included exploring solids and liquids and were differentiated for grade level. These offline sessions, Pour It, were very appropriate for this phase of the lesson and appropriate for each grade level.

Explain The Explain session presents graphics, animations, examples, and explanations of solids, liquids, and gasses. This session also discusses particles and their nature at each state. This is a solid example of an explain phase. The session provides the opportunity for students to “show what they know” by identifying solids and liquids from pictures. The offline session is differentiated at each grade level and has students create a concept map with terms and phrases they have learned. This would be considered a solid explain session; however, including pictures with the words at Kindergarten and possibly first grade would make it more appropriate at those grade levels.

Elaborate The engage, explore, explain, and evaluate sessions are clear and mostly follow the aim of each “E”. However, although the Elaborate sessions provide additional information and experiences for students, they are random in their order and intent. The goal of each elaborate activity is not obvious to the user. That their titles are different than the “E” they represent does not clarify their intent. It might be helpful, for example, to cluster the elaborate sessions associated with an explicit literacy connection.

The sessions that clearly integrate the content of the module with literacy skills include Alphabetize (including a useful link to the glossary), SillyBulls, and Take a Note. Making this connection evident would benefit the program. (note: all activities integrate literacy, but sometimes to a lesser degree).

The session that most aligns with the goal of an elaborate is Three States. This session extends and applies student understanding by digging deeper into the nature of particles and integrating and applying this understanding in connections to literature through poetry and the arts through actions.

Jo Jo Spilled It and Aroma Maze both explore characteristics of a state of matter and are more “game like.” Investigate, Contain It, and Sink or Float examine at a deeper level properties of matter. The Investigate offline session includes differentiated experiments comparing masses of matter, which also examines properties at a deeper level. Recommendations for the Investigate online session include using a graduated cylinder for measuring and pouring 50mL into the beaker. This is a more accurate representation of the type of tool used in science for measuring volume. Could there be an opportunity for students to read the ruler instead of clicking the end of the liquid thus generating the reading on the data table? This might provide students with a way to arrive at deeper understandings. In addition, the numbers generated are identical. Could numbers be generated that are more reflective of multiple trials in an offline investigation? Can the phrase, The evidence supported or did not supported your prediction to introduce this language be used? The use of the term “thicker” might advance misconceptions; one suggestion would be to introduce “viscous” through the use of adjectives (in this case, “gooey”).

Evaluate

The Evaluate activity is an online multiple-choice assessment where students click on the appropriate answer. This evaluate not only assesses student conceptual understanding, but also provides feedback after a student completes the assessment and provides the correct answer for those that the student missed. Providing a more detailed explanation or a review of the concepts missed would benefit the program. The offline assessments are paper/pencil differentiated assessments that provide good evaluation of student understanding. Analysis and alignment of missed questions with the concepts that might need further instruction would increase meaning of the evaluation.

Progression of learning across the module, States of Matter

Best practices for children in the early grades include balancing children’s need for focused instruction about a specific subject area or concept with children’s need to build in what they already know and to make connects between concepts and domains of learning—that is, to experience an integrated curriculum (Copple, 2009). Developing and implementing a curriculum that creates a coherent progression allows students (and teachers) to access prior knowledge about the specific content. They can build on this understanding and deepen or add to it. As they progress through a lesson, if the learning opportunities (activities, lab experiences, readings, etc.) are aligned to build a story, students learning will progress. Telling a “content story” helps student make important connections. “…in fact, the brains of children in this age span are looking for meaningful connections when presented with new information (Bransford, Brown, & Cocking 2003).

The Science4Us module, States of Matter, is comprised of many learning opportunities which follow the BSCS 5E Instructional model (Bybee et al, 2006). This framework is developed as a learning cycle in which student prior knowledge is accessed (engage), students engage in an inquiry (explore), students and teacher interact to deepen content knowledge (explain), students extend their knowledge or apply it to a new situation (elaborate), and students demonstrate their understanding (evaluate)—all focused on a big idea or main learning goal. Science4Us develops their content story through the 5E model with moderate success.

Some of the activities do not seem to align well. However, potential connections are there and just need to be made overtly for students to see the connections. Figure 1 provides an overview.

| Prior knowledge

Figure 1: Assessment of S4U Module

Recommendations for sequencing the learning:

The following recommendations for sequencing the learning in the States of Matter module are provided for consideration.

• Leave Engage as is.

• For Explore, make more explicit connections between the sink or float activity with the states of matter, or include an alternative explore where students are investigating similarities and differences among solids, liquids, and gasses. Also, add Take a Note as an additional explore. • Leave Explain as is.

• Leave Explain as is.

• For Explain/Elaborate, add Fact Lab and Does Not Belong.

• For Explain/Elaborate—Three States.

• For Elaborate, add a Science and Literacy session that includes Silly Bulls and Alphabetize.

• For Elaborate, add an adventure session that includes all of the “games”: Jo Jo Spilled It; Aroma Maze; Contain It; Match It; Sink or Float (or move both the original explore and Sink or Float to a module that focuses more on properties (observing matter) • Leave Evaluate as is.

• Leave Evaluate as is.

An additional recommendation that would enhance a coherent progression through the States of Matter module is to clearly define the order in which the activities should be explored. The 5E instructional model is intentional in the order, and this order supports the building of understanding and making clear connections among ideas. Students (and teachers) will not gain as much benefit from this framework if they randomly progress through the module in any order. It is not clear in the student view that the activities should proceed in order. A recommendation would be to have a “next activity arrow” at the end of each activity.

Clear learning goals Setting clear learning goals helps guide teacher instruction. These goals should also be

Setting clear learning goals helps guide teacher instruction. These goals should also be shared with students so that they know what direction they are headed in their learning. Primary grade children are eager for and need explanations; expository information; direct instruction about a new concept, word, or event; and opportunities to practice new skills (Copple, 2009). The destination, or learning goal, needs to be clear so that students can participate in activities, skills development, direct instruction, and other learning experiences that lead them to learning a new concept. Research suggests that when children have the opportunity to study or focus on a specific new concept in some depth and then apply what they have learned, they make gains in every domain—from language to science to emotional development (AAAS 2008; Hyson 2008; Spada & Lightbown 2008). Identifying and sharing with students clear learning goals provides focus which allows students to build on conceptual understanding toward a goal. This learning goal should be a continuous “reference point” throughout a lesson.

Science4Us includes a core concept for the module States of Matter. It also provides objectives for each session within the module (engage, explore, etc.). The core concept and individual objectives are clear. However, the objectives are written about what students will do and not necessarily about what they will learn. For example, in the Explain the objective states: Students connect their prior knowledge and experiences with the three states of matter to formal explanations and assess their understanding of new material. This indicates a pedagogical process and not the concepts and/or skills students learn. Throughout each individual session and the entire module, learning goals are inferred, can be identified, and are reviewed throughout. However, they are not explicitly stated in the teacher materials or shared with students to provide focus and direction for the learning.

Formative Assessment

Assessing student understanding is meaningful when teachers use the information gained to help students progress in their learning. In formative assessment, students are active participants with their teachers, sharing learning goals and understanding how their learning is progressing, what next steps they need to take, and how to take them (Heritage, 2007). Research indicates that students benefit most when teachers use assessment to understand the extent to which students are learning and to make corresponding changes in their instruction (Black & Wiliam, 1998ab). When a curriculum provides embedded opportunities for formative assessment to occur, and teachers are supported in how to use these embedded assessments, both students and teachers can work together to help students develop a deeper understanding of the content. According to research, formative assessment practice has powerful effects on student learning and motivation (see Black & Wiliam, 1998b).

Science4Us provides embedded formative assessment opportunities in both the online and offline activities. The program has multiple types of formative assessments including questions that students answer online with resulting feedback, both written and spoken. Interactives, such as graphing in Take a Note , Match It, and Aroma Maze that asks students to perform a task and provides feedback. There are open ended questions throughout that help generate discussion among students. The teachers’ materials provide question suggestions along with possible misconceptions to look for. Show What You Know in the Explain and Fact Lab also provide opportunities for students to self-assess and for teachers to check on student understanding. Students can repeat activities until they reach full understanding and teachers can provide additional support and move forward based on the multiple formative assessments embedded in this module.

Motivation

One of the most important goals for students in the K-2 age group is developing an enthusiasm for learning through motivating and engaging experiences. If a desire to learn is created, students are more likely to persevere. Many opportunities must be created for students to experience success because they can easily become frustrated and discouraged (Copple, 2009). In addition to creating opportunities for students to be successful, experiences where students are activity engaged in their own learning increases motivation to learn. “Learning where children are passive (at the expense of engaged, direct experience learning) tend to yield rote memorization rather than real gains in concept development, problem-solving abilities, complex thinking skills, and real-world application of new knowledge” (NEGP 1997).

If learning applies directly to students and taps into their interests, and if their teacher encourages them to persist and shows they care, students feel more secure and are more likely motivated to learn (Hyson 2008). Based on research, motivation stems from relevance, opportunities for success, and active learning.

Science4Us has components in the States of Matter module that are motivating and engaging to students. Science4Us has colorful graphics, interesting and fun videos and songs, animations, and interactive opportunities. The varied presentation of concepts meets students where they are as digital natives and provides opportunities for students to make choices and explore at their own pace. These characters singing songs and telling stories hook students. The interactive activities keep students engaged in the learning actively instead of passively. The varied activities and multiple ways concepts are presented allows all learners to be successful in the way they learn best. Because students can repeat activities as many times as they’d like, there is also increased opportunity for success.

Inquiry opportunities

The Next Generation Science Standards convey that “in all inquiry-based approaches to science teaching, our expectation is that students will themselves engage in the practices and not merely learn about them secondhand.” Research shows that when students actively explore and investigate scientific phenomena, they are more engaged and successful in learning scientific content and skills. Through direct experiences, students not only develop an understanding of scientific practices, but also learn about the nature of science. Primary grade children continue to need lots of hands-on, experiential learning (AAAS 2008). When presented with a new concept, primary grade children need physical actions or direct experiences to help them grasp the idea, much as adults need vivid examples and illustrations to grasp unfamiliar concepts (Pica, 2004).

In Science4Us, inquiry opportunities are offered in online interactives such as the Investigate where students choose which liquids to race, predict which will travel the farthest, and “run” the investigation with multiple trials and collecting data. Students also have mini-inquiries such as Does it Sink or Float, making predictions and “dropping” objects in a tank of water. Offline experiences include students exploring solids and liquids. These investigations are differentiated for grades K-2, less guided an more depth added as students progress through the grades.

The questions presented online, offline, and in the teacher guide are inquiry-based. These questions encourage discussion among students and often generate more questions and can even promote further investigations.

Teacher support

Teacher support is provided in a written/printable format, in online videos, and throughout Science4Us through the use of icons. This gives teachers the opportunity to get support “just in time” , as they need it and in a way that works for them. Not only do the students get the opportunity to learn in different ways, so do the teachers. The embedded professional development within this program provides support to guide teachers through the facilitation of a good experience for their students. However, there is a lot of information within the program that takes time to access and learn. A recommendation would be to support teachers in successful implementation with fidelity by offering a professional development workshop that gives them the opportunity to:

• experience the online and offline activities
• to find and use the teacher explain and the teacher guide
• to learn about and internalize best practices (instructional strategies, formative assessment, eliciting prior knowledge and identifying misconceptions)
• to learn about and practice using the 5 E instructional model
• to develop and share clear learning goals
• to identify, understand, and implement a coherent progression of learning (sequence)
• to learn about and practice inquiry in the classroom
• to learn science content
• to practice and streamline using technology in the classroom
• to facilitate a blended-learning environment successfully All of these features are within the Science4Us program. It is difficult for teachers to learn this information independently. Professional development support would help teachers implement Science4Us so that students (and teachers) would get the most out of this well- developed program , and ultimately, tap into the curiosity and desire to learn , the natural scientists, that are K-2 students . The goal being to move them forward in their life-long learning.

Alignment to NGSS

The information in Table 14 is from the Next Generation Science Standards released in April, 2013 (Achieve, 2013). In the K-2 grade band, properties of matter is suggested at second grade. Some of the content presented in the States of Matter module is seen in the fifth grade standards. However, in the module, students begin to understand that matter does have properties that can be observed. This is a foundational understanding that is needed as students progress in their learning. For the purpose of this study, only the States of Matter module was reviewed. However, preliminary review of the other Physical Science Modules suggest that temperature influences changes in matter, that there are different types of matter, and that properties of matter can be observed.

Table 14: Next Generation Science Standards Matter and its Interactions 2-PS1 Matter and its Interactions

2-PS1 Matter and its Interactions *The performance expectations marked with an asterisk integrate traditional science content with

*The performance expectations marked with an asterisk integrate traditional science content with engineering through a Practice or Disciplinary Core Idea. The section entitled “Disciplinary Core Ideas” is reproduced verbatim from A Framework for K-12 Science Education: Practices, Cross- Cutting Concepts, and Core Ideas. Integrated and reprinted with permission from the National Academy of Sciences.

2-PS1 Matter and its Interactions

2-PS1 Matter and its Interactions Students who demonstrate understanding can:

Students who demonstrate understanding can:

2-PS1-1. Plan and conduct an investigation to describe and classify different kinds of materials by their observable properties. [Clarification Statement: Observations could include
color, texture, hardness, and flexibility. Patterns could include the similar properties that different
materials share.]

2-PS1-2. Analyze data obtained from testing different materials to determine which materials

2-PS1-2. Analyze data obtained from testing different materials to determine which materials have the properties that are best suited for an intended purpose.* [Clarification Statement:
Examples of properties could include, strength, flexibility, hardness, texture, and absorbency.] [Assessment Boundary: Assessment of quantitative measurements is limited to length.]

2-PS1-3. Make observations to construct an evidence-based account of how an object made of a small set of pieces can be disassembled and made into a new object. [Clarification
Statement: Examples of pieces could include blocks, building bricks, or other assorted small
objects.]

2-PS1-4. Construct an argument with evidence that some changes caused by heating or

2-PS1-4. Construct an argument with evidence that some changes caused by heating or cooling can be reversed and some cannot. [Clarification Statement: Examples of reversible
changes could include materials such as water and butter at different temperatures. Examples of
irreversible changes could include cooking an egg, freezing a plant leaf, and heating paper.]

The performance expectations above were developed using the following elements from the NRC document A Framework for K-12 Science Education:

Science and Engineering Disciplinary Core Ideas Crosscutting Concepts

Science and Engineering Practices Planning and Carrying Out Investigations Planning and carrying out

Analyzing data in K–2 builds on prior experiences and progresses to collecting, recording, and sharing observations.

Analyze data from tests of an object or tool to determine if it works as intended. (2-PS1-2)

Planning and carrying out investigations to answer questions or test solutions to problems in K–2 builds on prior experiences and progresses to simple investigations, based on fair tests, which provide data to support explanations or design solutions.

Plan and conduct an

Analyzing and Interpreting Data Analyzing data in K–2 builds on

Plan and conduct an investigation collaboratively to produce data to serve as the basis for evidence to answer a question. (2-PS1-1)

Disciplinary Core Ideas PS1.A: Structure and Properties of Matter Different kinds of matter

Different kinds of matter exist and many of them can be either solid or liquid, depending on temperature. Matter can be described and classified by its observable properties. (2-PS1-1)

Different properties are suited to different purposes. (2-PS1-2),(2-PS1-3)

A great variety of objects can be built up from a small set of pieces. (2-PS1-3)

Crosscutting Concepts Patterns Patterns in the natural and

Patterns in the natural and human designed world can be observed. (2-PS1-1)

Events have causes that generate observable patterns. (2-PS1-4)

Simple tests can be designed to gather evidence to support or refute student ideas about causes. (2-PS1-2)

Energy and Matter Objects may break into

Objects may break into smaller pieces and be put together into larger pieces, or change shapes. (2-PS1-3)

| Constructing Explanations and Designing Solutions Constructing explanations and designing solutions in K-2 builds on prior experiences and progresses to the use of evidence and ideas in constructing evidence-based accounts of natural phenomena and designing solutions.

Constructing Explanations and Designing Solutions Constructing explanations and designing solutions in K–2 builds on prior experiences and progresses to the use of evidence and ideas in constructing evidence-based accounts of natural phenomena and designing solutions.

Make observations (firsthand

Make observations (firsthand or from media) to construct an evidence-based account for natural phenomena. (2-PS1-3)

Every human-made product is designed by applying some knowledge of the natural world and is built using materials derived from the natural world. (2-PS1-2)

Engaging in Argument from Evidence Engaging in argument from

Engaging in argument from evidence in K–2 builds on prior experiences and progresses to comparing ideas and representations about the natural and designed world(s).

Construct an argument with evidence to support a claim. (2- PS1-4)

The Science and Engineering Practices in NGSS are:

1. Asking questions (for science) and defining problems (for engineering)
2. Developing and using models
3. Planning and carrying out investigations
4. Analyzing and interpreting data
5. Using mathematics and computational thinking
6. Constructing explanations (for science) and designing solutions (for engineering)
7. Engaging in argument from evidence
8. Obtaining, evaluating, and communicating information Science4Us has integrated these practices into the States of Matter module. Science4Us provides tasks for students to do and asks questions that tap into students’ increasing ability to solve problems and think about things in different ways. Presenting science in a more integrated approach increases students’ understanding because their world is integrated. Moreover, Science4Us integrates literacy and mathematics throughout the module. Models are used to demonstrate different states of matter. Students plan investigations—especially in the offline activities—with more student choice across K-2. Questions and explanations are modeled and encouraged and students are asked to draw, write, and discuss what they have observed and learned. These practices are important for students to understand the nature of science throughout their lives and Science4Us capitalized on the opportunity to engage students in these practices.

Summary A McREL science content expert examined the States of Matter module to determine the extent to which the 5E instructional model was reflected throughout the module, whether a progression-sequence across a lesson (clear storyline) was evident, whether there were clear learning goals, the extent to which opportunities for formative assessment were apparent in the module, whether the module was motivating/engaging for students, whether the module contained inquiry opportunities for students, the extent to which the module provided support for teachers (including instructional strategies, content-related support, and technology support), and the extent to which the module was aligned with the Next Generation Science Standards. The following table (Table 15) shows the rubric used to assess each of these characteristics and serves as a quick overview of the findings of this content analysis. Following this summary of findings, researchers have provided a set of highlights and suggestions for improving the States of Matter module.

Table 15: Integration of Best Practices for K-2 Learning Characteristic Not Low Moderate High

• K-2 students learn in uneven and episodic ways (Copple, 2009). In Science4US, students are able to learn at their own pace and repeat activities until they understand the concepts. • The program is fun for students (our student survey data also support this claim) and

The following highlights and suggestions are provided for consideration.

• The program is fun for students (our student survey data also support this claim) and easy to use.

• The science content is accurate.

• The modules are well thought-out with teacher and student support, varied activities, multiple learning opportunities, engaging content, and high expectations for conceptual understanding.
• The content is integrated, where appropriate, with mathematics and literacy.
• The interactive notebooks allow students to demonstrate their understanding through words and pictures.
• Having both sound, words, and pictures provides children with multiple ways of engaging in their learning (for instance, scrolling over a picture produces the name of that picture).
• Learning scientific vocabulary through multiple activities (alphabetize, models, interactives) can facilitate student understanding as it becomes more contextualized.
• The amount of content presented in each activity was deemed appropriate (not too much information at one time). Suggestions
• Some of the activities could be rearranged so that a strong content progression is supported.
• Provide clearer direction for progressing through the activities and a streamlined naming scheme (for clarity: i.e. Engage: What is matter?; Elaborate: Three States, etc.)
• Offer the ability to print all related documents (teacher explain, teacher guide, and offline activities) in one pdf.
• Some activities cannot be stopped (for example, at the end of a class) without having to start over at the beginning. It might be beneficial to have a “fast forward / rewind” on all content presentations (if not on the first viewing, then definitely on the second). It can be very cumbersome to find specifics within presentations/activities .
• Provide specific suggestions within the online activities for engaging in the offline activities (breaking up the computer activities with hands-on/inquiry investigations). (this suggestion was corroborated during the observations in which teachers paid less attention to the online activities.
• Add more opportunities for cooperative group work.
• Consider adding a comprehensive teacher guide or making the materials that would comprise it easy for teachers to access (i.e., an online resource with all Science4Us teacher materials [e.g., teacher explain, teacher guide activities for all grade levels]). This would allow teachers to more easily search for information and make connections between different modules/activities. This suggestion was corroborated by teacher survey data.

Teacher Outcomes

What are teacher reported benefits of S4U?

• How do teachers use / implement S4U in their classroom? How do they support the online experience?

Teacher Implementation (Logs) Sessions Œ - “

Reviewing Teacher Guide

All teachers providing feedback on the teacher implementation log indicated that they reviewed the Teacher Guide. On average, teachers indicated they spent 15 minutes reviewing the Teacher Guide. The estimated time reviewing ranged from a low of three minutes to a high of 45 minutes. Using Offline Materials Approximately 38% of teachers providing feedback on the teacher implementation log indicted that they used offline materials with their class. The average amount of time teachers used the offline materials was 19 minutes, ranging from a low of 10 minutes to a high of 45 minutes. Teachers indicated the following uses of offline materials:

• We used our science notebooks to do the solids and liquids activity as well as writing about ways we used water

• Drew in our journals how we use water.

• We did our own experiments / observations wand made a T chart in our science journals.

• Looked a little too complicated for my K kids. If we had more time, I would do it with our 3 rd grade buddies.

• Good to put knowledge on paper.

• This experiment looks awesome, but we may do it next week when we get through all the school deadlines and assessments required this week as the school year comes to an end.

• Questions, discussion.

Using Online Materials

All teachers providing feedback on the teacher implementation log indicated that they used the online materials with their class. The average amount of time teachers spent using the online materials was 13 minutes, with a low of ten minutes to a high of 45 minutes.

Teachers indicated the following uses of the online materials:

• We used the pictures of the liquids and solids and then made predictions about whether the items would sink or float as a whole class.

• The kids enjoyed playing with the float/sink-we made it into a game.

• I took them into the computer lab today and they worked through their side of the program independently.

• Whole class watched video / did activities.

• Journaled/drew about what sinks and what floats.

• Good visuals, will probably have them watch it again tomorrow.

• Silly Bulls. Hard to do whole group--kids lose interest because they were not all 100% involved.

• We liked this one, did it during snack time.

• Liquid thickness and how far it travels down an incline in five seconds. Led to a great discussion!

• This activity reinforces the data we collect and graphs we make in class--AWESOME!

• Online "assessment" of what was learned.

• Notebook pages.

Comments / Suggestions / Challenges

The following comments / suggestions / challenges were provided:

• Loved the interactive nature of the online material.

• 1st and 2nd session may be a little short. Either that or I didn't teach it to its fullest.

• Don't think there was enough direction for the kids or myself. I just didn't feel like I knew that they were heading in the right direction. Would love more training.

• Great explanation of matter and the real-life scenario of cleaning a fish tank, especially since we have fish in our class!

• On the part that we choose the liquid/gas/solid online activity, do both a real photo and a microscope image together to reinforce concepts.

• I wish I had more time to work on these activities with the kids. We are trying to fit in too much right now, but they are still learning and discovering lots! :)

• Really good for the kids to practice using the mouse / fine motor skills.

• It was really cool to see how much the kids learned over this unit. Post assessment: There weren't many lessons on temperature, but that was assessed. Did I miss some lessons?

• I liked all the sessions. Some of the activities were a bit easy--especially circle the … Perhaps make choices more challenging. Additional teacher comments from sticky notes and margin notes on implementation logs:

• States of matter is not a K standard, but great lessons. I look forward to using this program next year to help supplement by science Kindergarten standards! Thanks for the opportunity!

• Sorry with the end of school I did not get to write on each log. We really enjoyed the program. I peeked at Habitats Living vs. Non-Living. Loved it!

Perceptions of Quality, Utility and Relevance

Teachers in the Early group were asked—at post study—to provide feedback about their I loved it, the kids loved it. Sometimes experience with Science4Us—both in terms science takes a lot of extra preparation of the student materials and in terms of the gathering materials, and this is a good way to go with the videos. They still get a teacher materials and resources. Findings good visual and experience to explore related to questions about quality, utility, and with experiments. relevance are provided in Appendix E. The following provides highlights from this feedback.

• Do teachers perceive the S4U student materials to be of high quality, utility and relevance? Researchers examined teacher feedback to determine the extent to which teachers were pleased with the student materials. Teachers expressed satisfaction with the quality, utility, and relevance related to most aspects of the student materials (including, for instance, the developmental appropriateness of Science4Us, the support for emergent literacy embedded in the sessions, the open-ended nature of the program, the stimulation of student interest, and the encouragement of active student involvement).

Some aspects of the student materials were rated less positively (more than 25% of the respondents indicating “Strongly Disagree” or “Strongly Agree”). For instance, several teachers disagreed that the States of Matter module focused on content their students needed to learn and several indicated that the module was not aligned to the relevant science standards for their students.

• Do teachers perceive the S4U embedded professional development is of high quality, utility and relevance?

Researchers also examined the extent to which teachers used the professional development and their perceptions of it. All participating teachers reported that—across all eight sessions in the States of Matter module—they at

in the States of Matter module—they at least sometimes used the teacher guide and at least sometimes used the offline materials. Teachers indicated that the Teacher Guide was helpful for understanding the content, the layout of the lesson, and the pacing of the lesson. Teachers offered that they would appreciate improved layout of the teaching sequence, as

I really enjoyed using it and was surprised at how easy it was to use and how much the kids learned.

Teachers offered that they would appreciate improved layout of the teaching sequence, as well as an easier-to-read format for the Teacher Guide, and the option for a paper copy of the Teacher Guide. In terms of the offline materials, teachers indicated that they appreciated the information on how to differentiate and supplement as well as the content; teachers indicated several improvements including the availability of a blackline master. Teachers reported making less use of the How-to-Videos, with the majority indicating that they never or only rarely consulted these across their two-week implementation of Science4Us. Teachers who provided an explanation for why they did not use the videos indicated that they did not need to because the program was self-explanatory or—more frequently—that they did not have time.

Physical science knowledge

Examining changes in teachers’ physical science knowledge under the constraints of an abbreviated implementation of a module not intended to increase teachers’ science content knowledge seemed unadvisable. Researchers report descriptive statistics on teachers’ content knowledge scores at pretest; these data were used as a covariate in student analyses.

Pedagogy

When asked whether their use of the Science4Us States of Matter module resulted in changes in their instructional practice, 43% indicated it had. One teacher who provided an explanation for how it had changed her practice noted that “It offered new ideas of what engaged students

Some students understand the three states of matter. They did not know what matter was before this.

how it had changed her practice noted that “It offered new ideas of what engaged students and excited them towards learning.” In addition, all respondents indicated that students learned something from the States of Matter module (29% of teachers indicated their students had learned “a little” whereas the remaining 72% of teachers indicated their students had learned “some” or “a lot.”

Self-efficacy in Science

Average self-efficacy scores for both the Early group and Late group teachers increased from pretest to posttest. Differences between Early and Late group teachers at posttest was not statistically significant.

Table 16: Teacher Self-Efficacy in Science Average

• Do teachers perceive that the S4U student materials enhance student achievement and engagement in science?

Researchers examined teachers’ perceptions of student engagement via

and items specifically related to student engagement.

post-study survey about the intervention, and items specifically related to student engagement.

As shown in Table 17, prior to the start of the study, teachers’ perceptions of their students’ levels of engagement were high. In addition, perceptions of engagement post-study reflected no variability (all items comprising the average score were at the top of the range). Because of the lack of variability, analysis of covariance was not possible. A repeated measures analysis revealed no interactions between group (Early or Late) and perceptions of student engagement, nor did it reveal any main effects for student engagement.

Table 17: Teachers' Perceptions of Student Engagement

Student Outcomes

What is the experience of student users of Science4Us?

Researchers used observations, think aloud protocols, student interviews, and student survey feedback to learn about the experience of student users of Science4Us. The following sections provide findings based on each of these data sources.

Observations

Researchers observed use of Science4Us in three second grade classrooms at two schools in May 2013. The three lessons ranged from 33 to 48 minutes in length; researchers were present for the duration of the lesson. Two researchers independently observed and rated one lesson together, discussing and coming to agreement on ratings. This allowed for improved consistency in use of the observation protocol. One researcher rated the remaining two lessons.

In two of the classrooms, teachers projected Science4Us on a screen and called on individual students to interact with the computer or called on the whole class for input. Students in another class were situated in the computer lab, where they listed to the Science4Us module with headphones and worked independently, at their own pace as the teacher walked around to provide support. Table 18 provides an overall assessment of the focus of the observed lessons.

Table 18: Focus of Observed Lessons Based on time spent, the focus of observed lessons is best described as…

…almost entirely working on mathematics/science concepts

Relative to the 5E Instructional Model, observers looked for evidence of Engagement (accessing prior knowledge, engagement in a new concept through the use of short activities that promote curiosity, making connections between past and present learning experiences), Exploration (using activities to identify current concepts [i.e.,
misconceptions] / processes /skills and facilitating conceptual change [e.g., labs that use
prior knowledge to generate/explore new ideas or design / conduct an investigation]), Explanation (focusing students’ attention on a particular aspect of their engagement / exploration to demonstrate their conceptual understanding/skills or the direct introduction of a concept, process, or skill), Elaboration (developing students’ deeper and broader understanding through new experiences), and Evaluation (encouraging students to assess their understanding and abilities and providing opportunities for teachers to evaluate student progress toward achieving the educational objectives). The numbers in Table 19 indicate the number of classrooms in which the strategy was observed. Specific evidence for the use of these activities or strategies is also provided in the table.

Table 19: Use of the 5E Instructional Model

Evidence: Evidence of the 5E instructional model was most frequently observed in teacher use of the S4U online

Evidence of the 5E instructional model was most frequently observed in teacher use of the S4U online materials. For example, one teacher projected the online S4U engage activity, calling individual students to the computer to draw a picture around matter that is changing and things that can make matter change. All students in the class were encouraged to call out “yes” if they agreed with or “no” if they disagreed with the items their classmates circled. This teacher then moved sequentially through the online exploration, explanation, and elaboration activities with the entire class, calling on students to interact with the activity whenever possible. Although there was some explanation of key concepts during this time, the teacher moved through activities quickly with little opportunity for discussion. Similarly, in another classroom, the class worked through activities of interest to the students as determined by a show of hands. The class worked on engagement, explanation, and elaboration (e.g., Silly Bulls) activities. In the third class, students worked on S4U activities independently in the computer lab. Although had a list of target activities to complete, they were allowed to work on these target activities (exploration, explanation, elaboration, and evaluation) in any order.

The one exception to observing the 5E instructional model through use of S4U was an engagement activity in which a teacher weighed a piece of clay and students were asked to predict whether it would have a different weight if molded into a different shape. One student molded the clay and the teacher then weighed the clay a second time.

Related to Science4Us, observers looked for evidence of teachers engaging in activities or using strategies indicative of Science4Us implementation (for instance, teachers explaining states of matter or using prior knowledge strategies or self-assessment strategies). Table 20 provides data from observations and supporting evidence related to the Science4Us activities and strategies.

Table 20: Science4Us Activities and Strategies

Evidence: As described in Table 20, teacher use of Science4Us was generally limited to the online materials.

Evidence: As described in Table 20, teacher use of Science4Us was generally limited to the online materials. Teachers made minimal or no use of Science4Us suggested discussion questions or extension activities. Two teachers facilitated conversations about the states of matter; however one conversation was framed as a brief review of prior knowledge before beginning a unit on changes in matter. Two lessons included “best practices” from the Science4Us Teacher Guide. These included reviewing, summarizing, and previewing important content and concepts and using everyday examples to help students answer questions and compare the different states of matter (for example, in one classroom the teacher made connections to a visit from a weather person). Similarly, two lessons included a discussion of misconceptions identified in the Science4Us Teacher Guide. One teacher reminded students that when a chemical change—such as a banana ripening—happens, it cannot be undone, and two teachers demonstrated how changing the shape of an object (e.g., a ball of clay, paint in a can) does not lead to changes in the object’s mass.

As described in Table 20, teacher use of Science4Us was generally limited to the online materials. Teachers made minimal or no use of Science4Us suggested discussion questions or extension activities. Two teachers facilitated conversations about the states of matter; however one conversation was framed as a brief review of prior knowledge before beginning a unit on changes in matter. Two lessons included “best practices” from the Science4Us Teacher Guide. These included reviewing, summarizing, and previewing important content and concepts and using everyday examples to help students answer questions and compare the different states of matter (for example, in one classroom the teacher made connections to a visit from a weather person). Similarly, two lessons included a discussion of misconceptions identified in the Science4Us Teacher Guide. One teacher reminded students that when a chemical change—such as a banana ripening—happens, it cannot be undone, and two teachers demonstrated how changing the shape of an object (e.g., a ball of clay, paint in a

Content extensions described in the Science4Us Teacher Guide were not observed, nor were prior knowledge or self-assessment strategies (aside from those embedded in the online materials), use of the glossary, student reports, or supporting/additional materials. Although the Science Notebook was used passively during all observations (material was automatically saved to the Notebook), the teacher and students did not use the “view” or “log” features to enter the notebook and access or create new Notebook content.

Type of Classroom Activities and Student Engagement

Type of Classroom Activities and Student Engagement While observing each of the classes, the researcher scanned the room every five minutes to establish whether 80% or more students were engaged in the lesson activity. During the May site visits, 80% or more of the students were engaged in the lesson activity in 74% of the classroom scans. Also at each scan, the researcher identified the primary type of teacher and student activity. The most commonly observed teacher activity was supporting the S4U work of the whole class and the most commonly observed student activity was participating in a whole class online activity (see Figure 2 and Figure 3).

Primary Teacher Activities

Figure 2: Primary Teacher Activities During Observations

Figure 3: Primary Student Activities During Observations

Finally, observers noted the percent of time different instructional arrangements were used. Table 21 provides these data.

Table 21: Observed Instructional Strategies

Instructional Percent

arrangement Whole class 65% Pairs/small groups-- Individuals 35%

Think-Alouds & Student Assessments

Researchers asked teachers to select students for participation in a think-aloud about Science4Us and an assessment of their perceived competence in and enjoyment of science (the Puppet Interview Scales of Competence in and Enjoyment of Science [PISCES]). Think- alouds and the PISCES assessment were administered preceding or following the classroom observations. Teachers were asked to select students who were high-performing and low- performing in science or mathematics to participate; no additional guidance was given. Eight students participated the think aloud and the PISCES (three girls and five boys; between two to four students per class). Think-alouds lasted between 11 and 22 minutes, averaging 15 minutes per think-aloud. PISCES assessments lasted between 7 and 13 minutes, averaging 10 minutes per assessment.

Think-alouds Researchers designed the think-alouds as a means of gathering information about student experiences with and perceptions of Science4Us. Students were asked to navigate through various Science4Us features and encouraged to verbally explain what they were doing as they worked through the different tasks. Appendix A contains the protocol researchers used for the think-alouds.

Glossary Students were first asked to demonstrate how they would use the Science4Us glossary to learn what the word “convert” means. Three students explicitly stated that they had not used the glossary feature before (e.g., I haven’t used a glossary before” and “We haven’t done this before”) and six students needed prompting to locate and open the glossary application. Three students entered the glossary by clicking on the letters below the glossary. One of these students had first tried to enter the glossary by clicking on the content area tabs above and experienced challenges (either because the student was holding the cursor slightly off-center or because the glossary had not fully loaded). With prompting, this student instead entered the glossary using the letters below the glossary. One student was not successful in entering the glossary. This student wondered about how to enter the glossary (e.g., “[Do I] use the labels”, “[Are the words] in alphabetical order”) before removing his/her hand from the mouse and stating, “I don’t’ see how.”

Two students entered the glossary but were not able to find the target word:

• One student used the Physical Science tab to enter the physical science portion of the glossary; however, once in the glossary the student did not see—or use the audio feature to find— the word “convert” on the page. The student did not navigate from page to page to search for the target word (which was on the page); rather, after looking at the page for a few seconds the student said, “It’s not here.” One student explained the process of using a glossary as “look[ing] at all the letters

• One student explained the process of using a glossary as “look[ing] at all the letters and see[ing] if you can find the letters in the word.” This student began looking for “convert” in the K Glossary. Although there were no words in the K Glossary, the student needed prompting to look for “convert” in the C Glossary. Upon entering the C Glossary, the student did not see—or use the audio feature to find— the word “convert” on the page. The student attempted to sound out a few words on the page and then said, “I can’t find it.”

Once in the glossary, five students successfully located the target word. However, three of these students needed spelling clues to find the word (a researcher asked one student how the word is spelled, told another student the word starts with the letter “c”, and told a third student the word starts with the letters “c-o-n”). Two students navigated through the glossary and found the target word independently. The five students who found the word “convert” in the glossary were next asked, “What does ‘convert’ mean?” Four students read the definition out loud; one of the four added, “I convert a sandwich with a knife.” One student clicked on the audio symbol for the definition, explaining that he/she prefers to listen to definitions.

Science Notebook

Students were then asked to use the science notebook to write or draw about ways they use water. Two students had previously used the science notebook to write/draw. Although these two students quickly located the notebook icon, one was confused as to whether to select “log” or “view” to accomplish the task at hand. One of these students drew a picture of drinking water and another drew a picture of a man by a drinking fountain.

The six students who had not used the notebook before needed help to enter the science

The six students who had not used the notebook before needed help to enter the science notebook; none of these students were able to enter the notebook independently. Once in the notebook, they all drew pictures of ways in which they use water. For example, two students drew a person drinking water. Other pictures included a water balloon fight, a glass of water with a straw, and water in solid, liquid, and gas phases (an ice cube, a glass of water, and steam from an iron). The most frequently used drawing tool was the pencil (n=8). Students also used the drawing tools to change the color of the pencil line (n=8), the paint can to fill in shapes (n=2), and the eraser (n=1; the student only used the narrowest line size). One student appeared confused when a tool did not work as expected; this student was asked “do you have the right tool selected” and with this prompt self-corrected and continued drawing. Another student accidentally colored the entire notebook blue with the paint can. This student tried to remove the blue color using several tools before asking, “How do you get this off?” Once prompted to select “undo” and shown the location of this feature, the student removed the blue and continued drawing. One student also used the notebook to write (type) a sentence about water. This student typed “water can be a solid or a liquid or a gas” and used the writing tools to increase the font size.

“Three States” Students were next asked to demonstrate use of the activity “Three States.” Five students readily found and entered the activity (although one needed to be prompted to enter the activity after finding it and another needed to be reminded of the name of the target activity). Two students navigated through the activities, but were unable to find and enter “Three States” without support (one could not find the activity and another accidentally entered a different activity).

In most cases, students listened to the introductory song before beginning the activity, and one changed the display to full screen mode. When choosing verbs (action words) to describe liquids and adjectives (describing words) to describe solids, students were prompted to explain aloud what they were doing in the activity. Responses to this prompt included “stream of consciousness” narratives, such as “it’s hard not soft, but it can be rubbery” and “splash, it can fall on the ground and makes a splashy noise.”

Students were asked two follow-up questions about this activity. First, they were asked to describe things they liked about the activity. Responses included:

• “That you really had to think about it to find out to get the ideas of solids and liquids.”
• “[That you get to] say what it’s like.”
• “Finding the different kinds of words. That I get to learn about states of matter.”
• “[That you get to] make decisions about what to do. I like videos and how they talk about liquids.”
• “When you play with it you learn a lot and play games and do tests. I also like the songs.”
• “You got to choose your own words and save it in a T-chart so you could see it another time.”
• “I don’t know. You learn lots of stuff.”
• “I learned about [the] three states and what they fit in [and] how they would move.” Next, students were asked to describe things they disliked about “Three States.” Seven students did not provide constructive feedback, responding “nothing,” “I pretty much liked everything,” and “I don’t know.” One student disliked “that you really had to read.”

“Take a Note” The final activity students were asked to use was “Take a Note.” Four students readily found and entered the activity, two needed prompting to scroll down and then found it independently, one entered a few other activities before finding the target activity, and one needed prompting to find the activity. Again, students listened to the introductory song before beginning the activity, and two changed the display to full screen mode. When prompted to explain aloud what they were doing in the activity, students generally explained their rationale for placing substances in a particular state of matter. For example, one student said, “juice so I will put it in liquid; ice is a solid; air a gas.” Another was quiet during the activity, but afterwards stated, “I used two things I could use in case I needed to know how many people in my class…different types of graphs and I took notes about it.”

All students placed all substances in the correct state of matter. One student had trouble initially placing dish soap in the liquid bar, but was able to place it on the liquid bar on the second attempt. The majority of students quickly placed substances, but others appeared to use the particle viewer for support in determining whether the substance was a solid, liquid, or gas. Students completed the tally chart on the first attempt; they generally created all three charts before checking to see if their answers were correct. .

Students were asked three follow-up questions about this activity. First, they were asked to describe things they liked about the activity. Responses included:

• “[I liked that] you had to, you can make tally marks and drag. I liked adding it up.”
• “[I liked] doing the graphs and listening to what the professor had to say and the frogs had to say and everything.”
• “[I liked] that you did tally marks and bar graphs. I like tall y marks and bar graphs.”
• “[I liked] identifying what they are, reading, moving things, [and] pictures.”
• “[I liked] looking at particles [and] making the bar go up in the graph.”
• “[I liked] that you got to know if it was a solid, liquid, or gas.”
• “[I liked that you] look up and know where [the] tallies are, [and that they] can change and [that I’m] learning more about graphs.”
• “[[I liked that it] tells you what you learned, graphs, graph of what they were, graph that you would make it to tell you what you learned.”
  Next, they were asked to describe things they disliked about the activity. Six students responded that they couldn’t think of anything they disliked. One student reported finding the bars confusing and one indicated a preference for checkmarks instead of tally marks because “it’s easier.”
  
  Finally, students were asked whether they like it when there is a “right” answer. Six students agreed that they do and two indicated they do not have a preference. Student responses to this question are presented below.
  \
• “Yes, so you know if you are right or wrong.”\
• “Yes, I know I got the right answer. I feel happy and proud of myself.”\
• “Uh huh. Because if you know, you know you are smart by getting it right.”\
• “Yes, makes me feel good.”\
• “I don’t’ care.”\
• “Yes, because when there’s a right answer you feel you make the right answer.”\
• “It doesn’t’ matter.”\
• “Yes, then you know if you are right or wrong and need to learn more.”
  \

Summary: Think Alouds\

During the think-alouds, many students appeared unsure of how to access different Science4Us features. For example, when asked to use the glossary three students were unable to find the target word and three needed spelling cues to find the target word. Similarly, six students needed support to locate and enter the science notebook, and the majority of students used only the pencil tool and color palette to accomplish the task at hand. This might be explained by the instructional models employed by the teachers of the students who participated in think-alouds: although one teacher created opportunities for
students to individually engage with Science4Us in a computer lab, two teachers exclusively
used the program in a whole class setting. Once students entered the two target activities
(“Three States” and “Take a Note”), they were able to successfully complete each activity.
Although student feedback about each activity was positive, six students described a
preference for activities in which there is a correct answer.

PISCES Assessment

Researchers used the PISCES assessment to collect data on student beliefs about their
competence in science, enjoyment in learning about science and engaging in science
activities, and ease of learning science. In this assessment, students were asked to select a
puppet that is most like them from a set of ethnically diverse puppets, and then asked to
respond to twenty statements by telling the researcher which puppet’s thinking about these
statements is most like their own. Appendix A contains the PISCES protocol. Students’
agreement with PISCES statements ranged from 70 percent to 100 percent of items; on
average, students agreed with 90 percent of statements.

Student Surveys

Researchers asked students in the Early group for feedback on Science4Us. Specifically,
students were asked whether (1) Science4Us helps them understand science, whether they
learn a lot of new things when they used Science4Us, (3) whether they like using
Science4Us, and (4) whether Science4Us makes them like science more. Students used
“smiley” face ratings for each of these four items. As shown in Table 22, over 80% of
students reported positive experiences with Science4Us, felt that it was beneficial to their
learning of science, and felt that the program helped them like science more. Very few
students expressed unhappiness with Science4Us; however, one student who rated the
Science4Us items using the “sad” face indicated that he had not yet had a turn to use
Science4Us on his own.

What are the student outcomes (science, knowledge, motivation and self-efficacy in science)
associated with S4U?

| | 笑脸 | 不笑脸 | 悲伤表情 |
| --- | --- | --- | --- |
| Science4Us helps me understand science. | 81.5%(106) | 13.8%(18) | 4.6%(6) |
| I learn a lot of new things when I use Science4Us. | 84.6%(110) | 12.3%(16) | 3.1%(4) |
| I like using Science4Us. | 83.1%(108) | 11.5%(15) | 5.4%(7) |
| Science4Us makes me like science more. | 80.0%(104) | 13.8%(18) | 6.2%(8) |

Student Motivation & Self-Efficacy in Science
As previously described, researchers collected student outcome data on appreciation of

As previously described, researchers collected student outcome data on appreciation of
science (student motivation) and confidence in science (self-efficacy). Both measures were
43 administered twice—each was administered prior to the start of the study and each was
administered after the conclusion of Science4Us implementation of the States of Matter
module for Early group classrooms and at the end of two weeks’ time for Late group
classrooms. Researchers used pretest scores for both the motivation and self-efficacy
analyses as a covariate. There were no significant differences between Early and Late group
students on the motivation in science measure (see Table 23), indicating that the average
motivation scores for Early and Late group students were statistically equal.

Table 23: Pretest and Posttest Student Motivation in Science
ADJUSTED

| | Average Score-Appreciation of Science(pretest)(standard deviation) | Average Score-Appreciation of Science(posttest)(standard deviation) | ADJUSTED Average Score-Appreciation of Science(posttest) | Difference(ANCOVA with pretest as covariate) |
| --- | --- | --- | --- | --- |
| Early Group Students | 10.11 | 10.56 | 10.58 | ns |
| (3.56) | (3.46) | (3.47) | | |
| n=146 | n=138 | n=129 | | |
| Late Group Students | 10.83 | 11.48 | 11.54 | ns |
| (3.19) | (3.48) | (3.51) | | |
| n=108 | n=81 | n=78 | | |

Table 24 provides data on students’ self-efficacy in science. Again, researchers analyzed
data using the pretest self-efficacy in science score as a covariate. Researchers found a
statistically significant difference between Early and Late group students on their selfefficacy in science, with Late group students scoring higher at posttest than their Early
group peers [F (2, 202) = 4.77, p = .030]. Table 24 provides descriptive data for the Analysis
of Covariance.
\

• p < .05
  
  Table 24: Pretest and Posttest Student Self-Efficacy in Science
  ADJUSTED
  
  Student Content Knowledge
  
  | | Average Score-Confidence in Science(pretest)(standard deviation) | Average Score-Confidence in Science(posttest)(standard deviation) | ADJUSTEDAverageScore-ConfidenceinScience(posttest)(standard deviation) | Difference(ANCOVAwith pretestas covariate) |
  | --- | --- | --- | --- | --- |
  | Early GroupStudents | 8.28 | 7.41 | 7.43 | Significant* |
  | (2.67) | (2.30) | (2.29) | | |
  | n=146 | n=136 | n=127 | | |
  | Late GroupStudents | 8.01 | 7.74 | 7.74 | Significant* |
  | (2.27) | (2.14) | (2.15) | | |
  | n=108 | n=81 | n=78 | | |
  
  For the outcome question regarding changes in student content knowledge (Does
  participation in Science4Us physical science modules lead to increased understanding of
  physical science concepts?), researchers first examined overall student content knowledge by time of assessment (pretest versus posttest) by grade level, regardless of whether
  students were in the Early or the Late group. As expected—and, as shown in Figure 4,
  regardless of treatment group, students at all grade levels increased in their content
  knowledge over time.
  
  Figure 4: Overall Student Content Knowledge Scores By Grade, By Time
  
  In addition, significant increases in scores on the content knowledge assessment were
  evident for both the Early group and Late group students (perhaps an indication of
  maturation or repeated testing—but perhaps an indication that all of the students are
  learning some aspects of the physical properties content being addressed by the
  2
  intervention materials). As previously noted, teachers indicated that they are teaching
  some amount of science each day across both the Early and Late groups.
  
  To examine whether students who participated in the intervention (Early group students)
  outperformed their non-participating peers (Late group students), researchers used pretest
  average science content assessment scores as a covariate. Researchers found a significant
  difference between the treatment groups, with Early group students outperforming their
  Late group counterparts [F (2,208) = 3.90, p = .050]. Table 25 provides descriptive data for
  the Analysis of Covariance.
  
  2
  These analyses are available upon request.
  \

• • *
    
    Table 25: Student Content Assessment Descriptive Data, All Students
    Average Score ADJUSTED
    
    | | Average Score-Content Assessment in Science(pretest)(standard deviation) | Average Score-Content Assessment in Science(pretest)(standard deviation) | ADJUSTEDAverage Score-Content Assessment in Science(posttest)(standard deviation) | Difference(ANCOVA w/pretest as covariate) |
    | --- | --- | --- | --- | --- |
    | Early GroupStudents | 6.25 | 7.01 | 7.05 | Significant* |
    | (1.89) | (1.78) | (1.77) | | |
    | n=148 | n=138 | n=130 | | |
    | Late GroupStudents | 6.04 | 6.57 | 6.57 | Significant* |
    | (1.78) | (1.86) | (1.86) | | |
    | n=113 | n=81 | n=81 | | |
    
    ^{\star}p<.05
    
    Researchers examined these data without the Kindergarten class using an Analysis of
    Covariance with science content pretest used as a covariate. Findings were robust to the
    inclusion of the Kindergarten class; researchers found a significant difference between the
    st nd
    treatment groups, with Early group 1 and 2 grade students outperforming their Late
    st nd
    group 1 and 2 grade counterparts [F (2,197) = 4.46, p = .036]. Table 26 provides
    descriptive data for this Analysis of Covariance.
    
    1^{\mathrm{s t}}
    
    Z^{\mathrm{n d}}
    
    1^{\mathrm{s t}}
    
    2^{\mathrm{n d}}
    
    Table 26: Student Content Assessment Descriptive Data, 1st and 2nd Graders
    Average Score – ADJUSTED
    
    | | Average Score-Content Assessment in Science(pretest)(standard deviation) | Average Score-Content Assessment in Science(pretest)(standard deviation) | ADJUSTEDAverage Score-Content Assessment in Science(posttest)(standard deviation) | Difference(ANCOVA with pretest as covariate, using all students) |
    | --- | --- | --- | --- | --- |
    | Early GroupStudents(1stand2ndonly) | 6.44(1.78)n=133 | 7.13(1.69)n=127 | 7.17(1.67)n=119 | Significant* |
    | Late GroupStudents(1stand2ndonly) | 6.04(1.78)n=113 | 6.57(1.86)n=81 | 6.57(1.86)n=81 | |
    | | | | | |
    
    (1^{\mathrm t}}^{\mathrm t}
    
    2^{\mathrm{n d}}
    
    \bar{\mathbf{1}}^{\mathrm{s t}}
    
    ^{\star}p<.05
    \
• • *
    \

Summary\

Research on early elementary science programs and their associated outcomes is limited— and research on early elementary science programs that integrate technology is extremely limited. Although this study was conducted under constraints (time and sample size), it reflects a starting point for rigorous research aimed at examining the extent to which science interventions created specifically the younger generation of digital native students impact achievement and motivation outcomes.

A McREL content expert assessed the extent to which the S4U States of Matter module reflected sound teaching practices. Although several recommendations were made, the expert appreciated how S4U accommodates the uneven and episodic ways that early elementary students tend to learn and is integrated with mathematics and literacy. Moreover, the expert found the S4U intervention to be engaging and motivating for students (an analytic finding that is supported by teacher and student feedback). Suggestions for improvement of S4U largely centered on the arrangement of activities to ensure a strong content progression and making materials available offline.

Researchers also examined Early group teacher feedback on their perceptions of the student materials. Teachers expressed satisfaction with the quality, utility, and relevance related to most aspects of the student materials (including, for instance, the developmental appropriateness of S4U, the support for emergent literacy embedded in the sessions, the open-ended nature of the program, the stimulation of student interest, and the encouragement of active student involvement). However, some teachers were unsure that the particular module under study was focused on content their students needed to learn. A longer period of implementation would provide all K-2 teachers with content aligned to what they are expected to teach.

Forty-three percent of teachers indicated that using the S4U States of Matter module resulted in changes in their instructional practice. However, the use of S4U during the two-week implementation period did not appear to result in changes to teachers’ self-efficacy in science compared to their counterparts in the Late group. Teacher self-efficacy in science is one’s confidence in his/her ability in science teaching. Although researchers did not find statistically significant differences at posttest between the two groups, supplemental analyses support that self-efficacy mediates student outcomes. Researchers need to study these changes over a longer period of time to determine if one year of S4U implementation and participation in teacher professional development components can contribute to enhancements in teacher self-efficacy. Likewise, through implementation of the intervention and use of the embedded professional development, teacher knowledge is expected to increase. We did not examine changes over time in teacher knowledge for this study; future studies should examine the extent to which participation leads to changes in teacher understanding of science.

Early elementary-aged students enjoy science. We therefore expected to see increases in students' in Early group teachers’ classrooms in motivation and self-efficacy in science, the latter of which is indicator that students believe they are good at and can do science. Self- efficacy in science scores for all students declined over time. In spite of this trend, researchers found that students with the teachers who did not use S4U materials actually scored better than students in the S4U classrooms on the measure of student self-efficacy in science. It is possible that the shortened implementation served to heighten students' awareness of what they don't know—that being exposed to science being taught in a vastly different way than that to which they were accustomed served to diminish their belief that they could do the work of science. So, in spite of actually doing better on a proximal science measure (discussed below), students failed to recognize they were actually gaining knowledge. A more protracted study, using a randomized design, would provide data to address questions about whether student self-efficacy in science is enhanced by S4U.

In terms of content knowledge in science, researchers found that all students increased in their content knowledge over time, but that Early group students significantly outperformed their Late group counterparts. Findings on the student proximal measure are promising. In an abbreviated implementation, researchers found an effect [d=.28] for student academic achievement in the group receiving instruction in S4U. Although the science lessons taught between the two groups likely differed, we know that students in both groups were receiving instruction in science. Further studies are needed—particularly ones that employ student outcome measures that more widely assess knowledge in a variety of age- and grade-span appropriate content in science.

Researchers acknowledge that the small sample size at the school- and teacher-level limits the statistical power of the current study. And, because the study design was quasi- experimental, concerns about internal validity threats such as selection bias and test-retest effects are warranted. However, this small-scale study provides preliminary support for an early elementary science intervention—an area not widely researched—and it provides support for the promise of Science4Us to effect meaningful impacts on student and teacher outcomes.
\

• • *
    \

References\

Achieve, Inc. (2013). Next Generation Science Standards. Achieve on behalf of twenty-six states and partners that collaborated on the NGSS.

Archer, L., DeWitt, J., Osborne, J., Dillon, J., Willis, B., & Wong, B. (2010). “Doing” science versus “being” a scientist: Examining 10/11-year-old schoolchildren’s constructions of science through the lens of identity. Science Education, 94 (4), 617-639. Banilower, E. R. (2005). A study of the predictive validity of the LSC Classroom Observation Protocol. Report to the National Science Foundation. Retrieved from: http://www.horizon-research.com/LSC/news/cop_validity_2005.pdf Bencze, J. L. (2010). Promoting student-led science and technology projects in elementary teacher education: Entry into core pedagogical practices through technological design. International Journal of Technology and Design Education, (20)1, 43-62. Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.

Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139–144.

Bleicher, R. E. (2006). Nurturing confidence in preservice elementary science teachers. Journal of Science Teacher Education, 17(2), 165-187. Bleicher, R. E. & Lindgren, J. (2005). Success in science learning and preservice science teaching self-efficacy. Journal of Science Teacher Education, 16, 205-225. Blumberg, F. C., Rosenthal, S. F., & Randall, J. D. (2008). Impasse-driven learning in the context of video games. Computers in Human Behavior, 24(4), 1530-1541.

Bransford, J., Brown, A., & Cocking, R. (2000). How People Learn: Brain, Mind, and Experience & School. Washington, DC: National Academy Press.

Broussard, S. C. & Garrison, M. E. B. (2004). The relationship between classroom motivation and academic achievement in elementary-school-aged children. Family and Consumer Sciences Research Journal, 33, 106-120.

Bybee, R.W., Taylor, J.A., Gardner, A., van Scotter, P., Powell, J.C., Westbrook, A., Landes,

N. 2006. The BSCS 5E Instructional Model: Origins, Effectiveness, and Applications. Colorado Springs, CO.
Center for Science, Mathematics, and Engineering Education, National Research Council. (2000). Inquiry and the National Science Education Standards: a guide for teaching and learning. Washington, DC: National Academy Press.

Chiong, C., & Shuler, C. (2010). Learning: Is there an app for that? Investigations of young children’s usage and learning with mobile devices and apps. New York: The Joan Ganz Cooney Center at Sesame Workshop.

Clark, T., Englert, K., Frazee, D., Shebby, S., & Randel, B. (2009). Assessment: A McREL report prepared for Stupski Foundation's Learning System. Denver, CO: Mid- continent Research for Education and Learning. - See more at:
\

• • *
    
    http://www.mcrel.org/products-and-services/products/product- listing/01_99/product-81#sthash.tvFSemqm.dpuf
    
    Copple, C., & Bredekamp, S. (Eds.). (2009). Developmentally appropriate practice in early childhood programs serving children from birth through age 8 (3rd ed.). Washington, DC: National Association for the Education of Young Children.
    
    Dean, C. B., Stone, B., Hubbell, E., & Pitler, H. (2012). Classroom instruction that works: Research-based strategies for increasing student achievement (2nd ed.). Alexandria, VA: ASCD. Davis, E. A., & Smithey, J. Beginning teachers moving toward effective elementary science teaching. Science Education, 93, 745-770. Edey, J., & Gomez, S. (n.d.) Action Research Summary Copernicus Project SSI Year 4: K-6 Literacy through Science. Retrieved from http://www.copernicusproject.ucr.edu/ActionResearchSummary/K6/Use%20of%20S cience%20Notebooks.pdf
    
    Fulp, S. L. (2002). The 2000 national survey of science and mathematics education: Status of elementary school science teaching. Chapel Hill, NC: Horizon Research, Inc.
    
    Garris, R., Ahlers, R., and Driskell, J. (2002). Games, Motivation, and Learning: A Research and Practice Model Simulation & Gaming, 33, 441-467.
    
    Heritage, M. (2007). Formative assessment: What do teachers need to know and do? Phi Delta Kappa International.
    
    Hsu, C. Y, Tsai, C.C., & Liang, J.C. (2011). Facilitating preschoolers’ scientific knowledge construction via computer games regarding light and shadow: The effect of the prediction-observation-explanation (POE) strategy. Journal of Science Education and Technology, 20, 482-493
    
    Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661-667. doi: 10.1111/j.0956-7976.2004.00737.x
    
    Klopfer, E., Osterweil, S., & Salen, K. (2009). Moving learning games forward. Boston: The Education Arcade, Massachusetts Institute of Technology.
    
    Kirriemuir, J. and McFarlane, A. (2004) Literature review in games and learning. Futurelab report. Bristol. Futurelab.
    
    Lepper, M. R., & Henderlong, J. (2000). Turning "play" into "work" and "work" into "play": 25 years of research on intrinsic versus extrinsic motivation. In C. Sansone & J. M. Harackiewicz (Eds.), Intrinsic and Extrinsic Motivation: The Search for Optimal Motivation and Performance (pp. 257-307). San Diego: Academic Press.
    
    Mantzicopoulos, P., Patrick, H., & Samarapungavan, A. (2008). Young children's motivational beliefs about learning science. Early Childhood Research Quarterly, 23, 378-394.
    
    Mayo, M.J. (2009). Video games: A route to large-scale STEM education? Science, 323. Available: http://www.sciencemag.org/cgi/content/full/323/5910/79 [accessed April 5, 2010].
    \
• • *
    
    McMurrer, J. (2007). Choices, changes, and challenges: Curriculum and instruction in the NCLB era. Washington, DC: Center on Education Policy.
    
    McMurrer, J. (2008). Instructional time in elementary schools: A closer look at the changes for specific subjects. Washington DC: Center on Education Policy.
    
    Murphy, C. & Beggs, J. (2003). Children’s perceptions of school science. School Science Review, 84(308), 109-116.
    
    National Research Council. America's Lab Report: Investigations in High School Science. Washington, DC: The National Academies Press, 2006.
    
    National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Committee on Science Learning, Kindergarten Through Eighth Grade. Richard A. Duschl, Heidi A. Schweingruber, and Andrew W. Shouse, Editors. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
    
    National Research Council. (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Committee on a Conceptual Framework for New K-12 Science Education Standards. Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
    
    National Research Council (NRC). (2006). America’s Lab Report: Investigations in High School Science. Committee on High School Science Laboratories: Role and Vision, S. R. Singer, M. L. Hilton, and H. A. Schweingruber, Editors. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
    
    National Research Council (NRC). (1999). How people learn: Brain, mind, experience, and school. J. D. Bransford, A. L. Brown and R. R. Cocking (Eds). Washington, DC: National Academy Press.
    
    Olson, C. K. (2010). Children’s motivations for video game play in the context of normal development. Review of General Psychology, 14(2), 180-187.
    
    Olson, S., & Labov, J. (2009). Nurturing and sustaining effective programs in science education for grades K-8. Washington, D.C.: The National Academies Press.
    
    Owen, S., & Moyle, K. (2008). Students' voices: Learning with technologies: Australian Information and Communications Technology in Education Committee (AICTEC).
    
    Patrick, B., Hisley, J., & Kempler, T. (2000). What’s everybody so excited about? The effects of teacher enthusiasm on student intrinsic motivation and vitality. Journal of Experimental Education, 68, 217-236.
    
    Patrick, H., & Yoon, C. (2004). Early adolescents’ motivation during science investigation. Journal of Educational Research, 97, 319-328.
    
    Prensky, M. (2001, October). Digital natives, digital immigrants. On the Horizon, 9(5). Retrieved from http://www.marcprensky.com Prensky, M. (2006). Listen to the natives. Educational Leadership, 63(4).
    \
• • *
    
    Quellmalz, E.S., Timms, M.J., and Schneider, S.A. (2009). Assessment of student learning in science simulations and games. Paper commissioned for the National Research Council Workshop on Gaming and Simulations, October 6-7, Washington, DC. Available: http://www7.nationalacademies.org/bose/Schneider_Gaming_CommissionedPaper. pdf [accessed March 23, 2010].
    
    Roth, K.J. (under review). Linking elementary science professional development to teacher and student learning: The importance of the science content storyline. Manuscript submitted to the Journal of Research in Science Teaching.
    
    Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. Journal of Educational Research, 95, 323-333.
    
    Slavin R. E., Lake, E., Hanley, P., & Thurston, P. (2012), Effective programs for elementary science: A best-evidence synthesis. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education. Squire, K. & Patterson, N. (2009). Games and simulations in informal science education. Paper commissioned for the National Research Council Workshop on Gaming and Simulations, October 6-7, Washington, DC. Retrieved from http://www7.nationalacademies.org/bose/Gaming_Sims_Commissioned_Papers.htm l Tweed, A. (2009). Designing effective science instruction: What works in science classrooms. Arlington, VA: National Science Teachers Association.
    
    Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower, E. R., & Heck, D., J. (2003). Looking inside the classroom: A study of K-12 mathematics and science education in the United States. Chapel Hill, NC: Horizon Research, Inc.
    
    Zembaul-Saul, C. (2009). Learning to teach elementary school science as argument. Science Education, 93, 687-719.
    \
• • *
    \

Appendices\

\

• • *
    \

Appendix A: Recruitment Materials\

Sent via email to local science teachers network, March 8, 2013
\

Are you a K–2 innovative teacher who wants to know how to incorporate science in\

\

your classroom?\

\

Are you interested in receiving high-engagement interactives, computer simulations,\

\

and online learning tools for early elementary students in science FOR FREE?\

McREL and Science4Us are teaming up to find K-2 teachers who are interested in using an interactive, online learning platform with their students and contributing to a study about that innovative tool.

Science4Us has developed a standards-based, interactive, and engaging science program and accompanying online professional development program for elementary school teachers.

We are currently looking for K-2 teachers who are willing to pilot test one Physical Science module with their students. The module can be completed in as little time as two weeks (assuming four days per week, 30 minutes a day). You would receive training from the publisher in how to implement and use the module with your students and you would be able to choose from several Physical Science modules. To help with our research, we would ask you to provide feedback and allow us to administer a science and an engagement assessment to your students (with permission from their guardians). And, to compensate your for your participation in the study, Science4Us will provide you full, free access to the program for the 2013-2014 school year. To learn more about the program, go to http://www.science4us.com/learn-more/.

We have a limited number of slots available for interested K-2 teachers. If you’d like to apply to be considered, please contact Cyndi Long at McREL ( clong@mcrel.org) by Monday, April 1, 2013.

To learn more about the program, go to http://www.science4us.com/learn-more/. To learn more about McREL, go to http://www.mcrel.org.

We look forward to hearing from you! Feel free to share this invitation with K-2 teachers who might be interested.
\

• • *
    \

Appendix B: Consent Forms\

\

TEACHER CONSENT (Note: Comparison group consent will not include the items marked\

“DELETED FOR COMPARISON TEACHERS”—all other aspect of the consent form will be i d e n t i c a l)

DATE
\

Dear Colleague:\

You have been asked to take part in a study aimed to provide information about student and teacher outcomes associated with use of the Science4Us (S4U) States of Matter.

Participation in this study means consenting to the following activities:
\

• Administer / allow for the administration of two student surveys and science assessments (approximately 15 minutes per survey/assessment).\
• Complete two teacher surveys (approximately 30 minutes per survey).\
• Attend a professional development webinar on how to use Science4Us (approximately 60 minutes).[NOTE: DELETE FOR COMPARISON TEACHERS]\
• Implement the Science4Us States of Matter module as recommended by the program developer (approximately 4 hours total; 30 minutes per day for four days per week over two weeks). [NOTE: DELETE FOR COMPARISON TEACHERS]\
• Complete an implementation log detailing use of the Science4us States of Matter module (approximately 5 minutes per log) [NOTE: DELETE FOR COMPARISON TEACHERS]\
• Allow researchers to observe a science lesson, if your class is selected and facilitate the collection of student data using a think-aloud protocol with a small sample of students (30 minutes per observation and 15 minutes per think-aloud/interview)\
• Facilitate collection of classroom attendance records, demographic information, and language proficiency information for participating students.
  Researchers will collect data from you prior to, during, and following implementation of the Scienc4Us modules to determine levels of implementation and your perceptions of the benefits of Science4Us. Researchers will also collect data from your students to determine knowledge, motivation, and self-efficacy outcomes in science that can be associated with the Science4Us intervention. We will ask for assistance administering pretests to students prior to your use of the Science4Us module and posttests to your students following your use of the Science4Us module. If your classroom is selected, during the implementation of Science4Us, researchers will also ask to observe your classroom implementation of Science4Us and ask to interview a sample of your students about their experiences and whether participation in the intervention increases their motivation in science.
  
  A direct benefit of study participation is access to S4U in the 2013-14 school year. Moreover, your participation in the study will contribute to an understanding of use of S4U on teacher and student outcomes and the improvement of future versions of the program. There are no known risks related to your participation in this study. Your participation is completely voluntary. You may choose to withdraw from the study at any time. Should you choose to withdraw, you will still have access to Science4Us in the 2013-14 school year.
  \

• • *
    
    The information gathered from the activities listed above will be kept strictly confidential. Your name will not be used in any study reports. Instead, comments will be summarized. We may directly quote what is said in a report, but we will not use the name of the person making the comment. Data files will be kept in a safe place during the study and destroyed after the end of the study.
    
    Should you have any questions about this study or your rights as a participant, you may Sheila Arens, Senior Director at Mid-continent Research for Education and Learning (McREL), at 303-632-5625 or sarens@mcrel.org.
    
    I have read (or had someone read) this form and understood the descriptions of the study. I have asked for and received a satisfactory explanation of any language that I did not fully understand.
    
    I agree to participate in this study, and I understand that I may withdraw my consent at any time. I have received a copy of this consent form.
    \

NAME (Please Print)\

_____ I give consent to participate in this study.

_____ I do NOT give consent participate in this study.
\

SIGNATURE : ____________________________________________________\

Please return this agreement to: Dr. Sheila A. Arens, Senior Director McREL 4601 DTC Blvd, Suite 500 Denver, CO 80237 Fax: (303) 337-3005 email: sarens@mcrel.org
\

• • *
    \

McREL’s Study of Science4Us:\

\

Memorandum of Understanding between School (Principal) and\

McREL
\

Study Description\

McREL is inviting K-2 teachers from your school to participate in a brief study of Science4Us (S4U). S4U is a comprehensive web-based digital science curriculum for early elementary students. S4U is a standards-based, interactive, and engaging science program with accompanying online professional development.

The ultimate goal of the proposed evaluation will be to provide formative feedback on the quality, relevance, and utility of the S4U intervention, and to provide insight into how using S4U relates to student outcomes such as science motivation and science achievement. Four overarching questions organize the research and reflect key outcomes for this study:
\

• Does S4U reflect sound teaching practices?\
• What are teacher reported benefits of S4U?\
• What is the experience of student users of S4U?\
• What are the student outcomes (science, knowledge, motivation and self-efficacy in science) associated with S4U?
  To address these questions, we are inviting K-2 teachers from your school to participate in a two-week study of one of the S4U modules.
  
  The study sample will include K-2 teachers and their students. Teachers will be able to choose whether they receive training and use the materials in spring 2013 (the “early” group) or during next year (the “late” group). McREL will try to make sure your teachers’ preferences can be honored; however, we will need to have about half of the teachers in our entire study delay the start of using the materials. Teachers in the early group—those who start this spring—will receive a short training on how to use S4U (approximately 60 minutes, delivered via webinar) as well as access to the online S4U materials. Their students will also receive access to the materials. These teachers will be asked to:
  \
• Administer / allow for the administration of two student surveys and science assessments (approximately 10 minutes per survey/assessment).\
• Complete two teacher surveys (approximately 30 minutes per survey).\
• Attend a professional development webinar on how to use Science4Us (approximately 60 minutes).\
• Implement the Science4Us States of Matter module as recommended by the program developer (approximately 4 hours total) during spring 2013.\
• Complete an implementation log detailing use of the Science4us States of Matter module (approximately 5 minutes per log; 8 logs).\
• Allow researchers to observe a science lesson, if their class is selected and facilitate the collection of student data using a think-aloud protocol with a small sample of students (30 minutes per observation and 15 minutes per think-aloud/interview),\
• Facilitate collection of classroom attendance records, demographic information, and language proficiency information for participating students.
  \

• • *
    
    During spring 2013, researchers will collect data from your teachers prior to, during, and following implementation of the Scienc4Us modules to determine levels of implementation and their perceptions of the benefits of Science4Us. Researchers will also collect data from those teachers’ students to determine knowledge, motivation, and self-efficacy outcomes in science that can be associated with the Science4Us intervention. We will ask teachers assistance in administering pretests to their students prior to their use of the Science4Us module and posttests to their students following their use of the Science4Us module. If their classroom is selected, during the implementation of Science4Us, researchers will also ask to observe their classroom implementation of Science4Us and ask to interview a sample of their students about their experiences and whether participation in the intervention increases their motivation in science.
    
    During spring 2013 K-2 teachers in the late group will engage in their science curriculum “business as usual” using any science programs or activities and engaging in any professional development activities that do not involve S4U. During the school year following study completion (2013-2014), teachers in the late group will receive identical materials and trainings in S4U. In spring 2013, these teachers will be asked to:
    \

• Administer / allow for the administration of two student surveys and science assessments (approximately 10 minutes per survey/assessment).\
• Complete two teacher surveys (approximately 30 minutes per survey).\
• Facilitate collection of classroom attendance records, demographic information, and language proficiency information for participating students.
  Students in participating classrooms of both early and late group teachers will complete two student surveys and science assessments (approximately 10 minutes per survey / assessment) in spring 2013. McREL will select a sample of classrooms of early group teachers for observations and for student “think-alouds.” McREL will secure parental permission to collect data from students.
  \

Study Responsibilities\

\

McREL will provide S4U materials and professional development to teachers in the early group\

during April of 2013. This includes a professional development webinar on how to use Science4Us (approximately 60 minutes) and access to all Science4Us materials for their classroom. For this study, teachers in the early group are only being asked to implement one module—the States of Matter module during spring 2013. For early group teachers, during spring 2013, McREL will monitor teacher use of S4U and the process of data collection during the course of the study. McREL will also provide professional development to teachers in the form of an initial 60 minute training (delivered via webinar); early group teachers will have access to additional training through professional development embedded in the online S4U materials.

McREL will provide each late group teacher with the same S4U materials and trainings during the school year following study completion (2013-2014) to balance the resources received by all teachers in the study.
\

• • *
    
    McREL will work with participating teachers in both the early and late groups to facilitate the administration of the surveys and assessments in spring, 2013. Responses to this data collection will be used only for statistical purposes. The reports prepared for the study will summarize findings across the sample and will not associate responses with a specific school, teacher, or individual. We will not provide information that identifies you or your school to anyone outside the study team, except as required by law. McREL will manage the distribution and return of study instruments, analyze the data, and report findings. McREL will assign all districts, schools and individual participants ID numbers and strip all identifying information from the data. No identifying information will be included in reports on this study. Participation in this study is voluntary. Teachers may withdraw from this study at any time without penalty.
    \

The school agrees to support interested K-2 teachers in participating in this study. All K-2\

grade teachers in the early group will use the Science4Us program, attend the training session, and complete all data collection activities. Teachers in the late group will continue their usual instructional practice, complete data collection activities, and refrain from using Science4Us during the study year. Teachers in the late group will receive identical Science4Us materials and training during the school year after study completion (2013-2014).

\\\_________________________________________________________________________

I understand and agree to the information in the Memorandum of Understanding. I have received a copy of this form for my files.

School Name:
\

School Representative’s Name:\

\

Title: Phone:\

\

E-m a i l :\

\

School Representative’s Signature: Date:\

\

Principal Researcher’s Signature: Date:\

Please return this agreement to: Dr. Sheila A. Arens, Senior Director McREL 4601 DTC Blvd, Suite 500 Denver, CO 80237 Fax: (303) 337-3005 email: sarens@mcrel.org
\

• • *
    \

PARENT PASSIVE CONSENT August 23, 2013\

\

Dear Families of Students:\

This spring, your child’s teacher is working with a company named McREL to learn about an engaging, online program called Science4Us. The purpose of this study is to find out more about:
\

• Teacher and student experiences when they use the Science4Us program; and
  \
• Whether Science4Us helps teachers and students learn about science.
  We ask your permission for your child to be part of the study. This means your child will participate in the following activities:
  \
• Complete two surveys about their feelings about learning science (approximately 10 minutes per survey)
  \
• Complete two science tests (approximately 15 minutes per test)
  Your child will not be pulled out of class to participate in this study—the tests would be administered before your child’s teacher begins the program and then shortly after your child’s teacher completes the program.
  
  
  Your child may also be asked to help us by participating in an additional interview during which we will ask them to tell us how they use different pieces of Science4Us and what they are thinking about the program as they use it. This will require approximately 15 minutes. If your child is selected for this part of the study, researchers will ask to work with your child in a separate part of the classroom.
  
  With this information, McREL will learn how students feel about Science4Us and whether Science4Us helps students learn.
  
  Your child’s answers on their surveys and tests will be kept secret. No student will be named in any report about the study. All students in the study will be given an ID number instead of using their name. No personal information (name, birthday, etc.) will be shown. Any personal information we have will be locked up in a file and will not be given to anyone.
  
  Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across classrooms and will not link information to a specific teacher or student. We will not provide information that identifies you, your child, or your child’s school to anyone outside the study team, except as required by law (for instance, in cases of child abuse).
  \

• • *
    
    Your child’s participation in this study is voluntary. If you do not wish your child to be part of it, please fill out the form on the next page. Your child may still use the Science4Us materials, but we will not survey or test your child. If you wish to take your child out of the study at any time, you may.
    
    If you have any questions about the study or about your child’s part in it, please call [insert teacher’s name and phone number]. You may also call or email me. I can be reached at McREL at 303-632-5625 or by email at sarens@mcrel.org.
    
    Sincerely,
    \

Sheila Arens\

\

• • *
    \

RETURN THIS FORM ONLY IF YOU DO NOT WANT YOUR CHILD TO PARTICIPATE IN THE\

\

STUDY OF Science4Us\

\

If you DO NOT wish to give your permission please:\

\

1) Write your child’s school name and his or her name in the blanks below.\

2) Check (‘X”) in the box under it. 3) Sign your name and write the date.
\

4) Return this form to your child’s teacher by .\

\

Thank you.\

\

School Name: ____________________________________\

\

Child’s name: ____________________________________\

\

□ My child does NOT have my permission to participate in the study of Science4Us.\

Pa r e n t ’ s S i g n a t u r e :
\

D a t e :\

\

• • *
    \

Appendix C: Instruments\

\

• • *
    
    S4U Classroom Observation Protocol - Early Group
    
    | Observation Setting | |
    | --- | --- |
    | School: | Date: |
    | Teacher: | Observer: |
    | Time(Start): | Time(End): |
    | Grade: | Class Size: |
    
    Session and lesson objective:
    
    Description of use of Science4Us:
    
    Provide a quick sketch of how students are seated:
    
    Based on time spent, the focus of this lesson is best described as: (check one)
    
    Almost entirely working on the development of facts/vocabulary
    
    Mostly working on the development of facts/vocabulary, but working on some science concepts
    
    Mostly working on the development of facts/vocabulary, but working on some science concepts
    
    About equally working on facts/vocabulary and working on mathematics/science concepts
    
    Describe any major interruptions (e.g., fire drill, assembly, shortened class period)/ unusual circumstances:
    
    Almost entirely working on mathematics/science concepts
    \
• • *
    
    Instructions: Mark (X) whether the following activities/strategies were observed (O) during the classroom observation. In the Evidence/Comments column, note specific examples as well as the frequency and/or extent to which the activity/strategy was observed.
    
    | BSCS 5E Instructional Model | O | Evidence/Comments |
    | --- | --- | --- |
    | Engagement | | Accessing prior knowledge, engagement in a new concept through the use of short activities that promote curiosity, making connections between past and present learning experiences. |
    | Exploration | | Activities in which current concepts (i.e., misconceptions)/processes/skills are identified and conceptual change is facilitated (e.g., labs that use prior knowledge to generate/explore new ideas or design/conduct an investigation). |
    | Explanation | | Focusing students' attention on a particular aspect of their engagement/exploration to demonstrate their conceptual understanding/skills or the direct introduction of a concept, process, or skill. |
    | Elaboration | | Through new experiences, the students develop deeper and broader understanding, more information, and skills. |
    | Evaluation | | Encourages students to assess their understanding and abilities and provides opportunities for teachers to evaluate student progress toward achieving the educational objectives. |
    | Science4Us | | |
    | Teacher provided explanation the states of matter | | Teacher explains solid, liquid, and gas and their properties/characteristics. |
    | Lesson includes Best Practices from Teacher Guide | | |
    | Lesson includes Misconceptions from Teacher | | |
    
    $^1$ http://sharepoint.snoqualmie.k12.wa.us/mshs/ramseyerd/Science%20Inquiry%2011%2020112012/What%20is%20Inquirv%20Sciencе%20(long%20version).pdf
    \
• • *
    
    | Guide | | |
    | --- | --- | --- |
    | Lesson includes Content Extension from Teacher Guide | | |
    | Use of prior knowledge strategies | | Kwl, Knowing, Woodworking, Learning, KLEW, Knitting, Learning, Evidence, Wondering, KwL, Q & R Map, T chart, Boltule and Afar, Prediction Work, Word Tops, List and Stem, Allocation Guides, Prms, Cons, What I Will |
    | Use of literacy skills strategies | | FLUENCY (Practice and Review), MAIN (IDEA AND SUPPLEMENT DETAILS) (Main Idea & Word Notch, Main Idea & Picture Match), CAUSE AND EFFECT (What Happened, Match it, T Chart, Signal Words, Find It, It... Them Statements, Phone Cards), WRITING SKILLS (Shape Form, SV Picto Poem, Cladron Tools, Narrative Writing in Nonfiction, Shorts), COMPARE AND CONTRAST (H Map, Varn Diagram, Similaries and Differences), Categorize and Classify (Name Mr. Fue., Sort the Groups, One of These Things Is NOT Like the Others, Categorizing Objects T-chart, ABC Grid), VOCABULARY (1 See Chapters, Word Study, Drawing the Words, Vocabulary Index Cards, Close Activity: Vocabulary, Andrages, Vocabulary Draw It, Recorded Vocabulary Words, Teenage Vocabulary, Root Words and Suffixes, Gutter Sorting, Vocabulary Categories, Adjective and Question Cards, What Am I?) |
    | Use of self-assessment strategies | | Short form Lab Write-up; Cooperative Learning Reflection; How Do I Do All...t, narrative Lab Write-up; ABC Recess Reflection Prompts |
    | Use of Science Notebook | | |
    | Use of Glossary | | |
    | Use of Student Reports | | |
    | Use of supporting/additional (i.e., not Science(Us) materials | | |
    \
• • *
    
    Instructions: In 5-minute time intervals, mark (X) the primary activity in which the teacher is engaged and in which the majority of students (approximately 80%) are engaged. An example is provided
    
    | Teacher Activity | Time | | | | | |
    | --- | --- | --- | --- | --- | --- | --- |
    | 8:45 | | | | | | |
    | Lecturing/ explaining to the whole class | | | | | | |
    | Modeling/demonstrating to the whole class | X | | | | | |
    | Supporting the Science4Us work of the whole class | | | | | | |
    | Supporting the work of the whole class (not-Science4Us) (describe): | | | | | | |
    | Supporting the Science4Us work of a small group of students | | | | | | |
    | Supporting the work of a small group of students (not-Science4Us) (describe): | | | | | | |
    | Supporting the Science4Us work of individual students | | | | | | |
    | Supporting the work of individual students (not-Science4Us) (describe): | | | | | | |
    | Assessing students/evaluating student performance | | | | | | |
    | Correcting off-task student behavior | | | | | | |
    | Science4Us logistics (e.g., facilitating computer access) | | | | | | |
    | Other teacher activity (e.g., talking off topic, observing class, working at own desk/computer) (describe): | | | | | | |
    | Student Activity | Time | | | | | |
    | 8:45 | | | | | | |
    | Listening to the teacher | | | | | | |
    | Participating in a whole class offline activity/discussion | X | | | | | |
    | Participating in a whole class online activity | | | | | | |
    | Participating in a small group offline activity/discussion | | | | | | |
    | Participating in a small group online activity | | | | | | |
    | Participating in an individual offline activity/discussion | | | | | | |
    | Participating in an individual online activity | | | | | | |
    | Science4Us logistics (e.g., logging on to computers) | | | | | | |
    | Off-task student behavior | | | | | | |
    | Other student activity (describe) | | | | | | |
    | Percentage of Ss on-task:At this point in time, at least 80% of the students in the class appear actively engaged in the classroom activity (Y/N)?Total # of students in class ___ 80% =___ | | | | | | |
    
    Considering only the instructional time of the lesson, approximately what percent of this time was spent in each of the following arrangements (should sum to 100%)?
    
    Whole class ____ %
    
    Pairs/small groups ____ %
    
    Individuals ____ %
    \
• • *
    
    Instructions: Use the space below for recording what happens in the classroom chronologically during the observation session. Be sure to include both what the teacher is doing and what students are doing as well as information about where this lesson fits in with the overall unit, the focus of the lesson, and the roles of any other adults in the classroom. Also describe transitions from one activity or location to another. Use the following key to distinguish activities: T = Teacher activity; S = Student activity; TX = Transition
    
    | Time | Activities |
    | --- | --- |
    | | |
    \
• • *
    
    Teacher Implementation Log
    
    Date: ____
    \

1. Did you review the Teacher Guide?
   
   Yes
   
   No
   
   Amount of time reviewing Teacher Guide (0-60 minutes): ____
   \
2. Did you use the offline materials with your class?
   
   Yes No
   
   Amount of time using the offline materials (0-60 minutes): ____
   
   Description of use of offline materials:
   \
3. Did you use the online materials with your class?
   
   Session: $ \textcircled{1} $
   
   Yes
   
   Amount of time using the online materials (0-60 minutes): _____
   \
4. Comments/Suggestions /Challenges:
   
   Description of use of online materials:
   
   Teacher Instructions?
   \

• • *
    
    S4U Think Aloud Protocol
    
    Date: ____ School: ____ Grade level: ____
    
    Evaluator: ____ Start time: ____ End time: ____
    \

1. Show me how you use Science4Us to learn what “convert” means [ be sure to explain
   what you are doing as you show me] .
   
   a. What does “convert” mean?
   \
2. Show me how you used/ use the notebook to write or draw about ways you use water
   [ be sure to explain what you are doing as you show me] .
   \
3. Show me how you use the activity “Three States (Explore)” [ be sure to explain what you
   are doing as you show me] .
   
   
   a. What do you like about this activity?
   
   b. What do you dislike about this activity?
   \
4. Show me how you use the activity “Take a Note (Investigate)” [ be sure to explain what
   you are doing as you show me] .
   
   a. What do you like about this activity?
   
   b. What do you dislike about this activity?
   
   c. Do you like it when there is a “right” answer to an activity?
   \

• • *
    \

Science4Us: Teacher Survey\

Welcome Dear Educator, Thank you for taking the time to complete the Science4US (S4U) survey for Mid-continent Research for Education and Learning (McREL). Your participation in completing this survey is voluntary. You may choose to stop completing the survey at any time. If you have any questions about the survey, please call or email McREL Senior Director, Sheila Arens, Ph.D. She can be reached at 303-632-5625 or by email at sarens@mcrel.org. Thank you for your time!
\

Background Information\

Please answer the following questions.
\

1. What is your name?\
2. What is your gender: Female Male\
3. Are you: American Indian or Alaskan Native Asian Black or African-American Hispanic or Latino Native Hawaiian or Other Pacific Islander
   \

• • *
    
    White Mixed Race (please specify)
    \

4. What grade(s) do you teach? (Select all that apply) K 1 2
   \
5. At what school do you teach?
   \
6. Do you have any of the following degrees?
   Yes No Bachelors Masters Doctorate
   \
7. Please indicate the subject(s) for each of your degrees. (Select all that apply.)
   Bachelors Masters Doctorate Elementary Education Biology/Life Science Chemistry Earth/Space Science Physics Science Education (any science discipline) Other subject If you selected "other" above, please specify.
   
   \

• • *
    \

8. In what year did you last take a formal course for college credit in:
   
   Science:
   
   The Teaching of Science:
   \
9. What is the total amount of time you have spent on professional development in science or the
   teaching of science in the last 12 months? And in the last 3 years? (Include attendance at professional
   meetings, workshops, and conferences, but do not include formal courses for which you received
   college credit or time you spent providing professional development for other teachers.)
   
   | | None | Less than6 hours | 6-15hours | 16-35hours | More than35hours |
   | --- | --- | --- | --- | --- | --- |
   | In the last12months | | | | | |
   | In the last3 years | | | | | |
   
   Background Information (continued)
   \
10. Many teachers feel better qualified to teach some subject areas than others. How well qualified do
    you feel to teach each of the following subjects at the grade level(s) you teach, whether or not they
    are currently included in your curriculum?
    
    | | Not Well Qualified | Adequately Qualified | Very Well Qualified |
    | --- | --- | --- | --- |
    | Life Science | | | |
    | Earth/Space Science | | | |
    | Physical Science | | | |
    | Mathematics | | | |
    | Reading/Language Arts | | | |
    | Social Studies | | | |
    \

• • *
    
    | | Days Per Week | Approximate Minutes Per Day |
    | --- | --- | --- |
    | Mathematics | | |
    | Science | | |
    | Social Studies | | |
    | Reading/Language Arts | | |
    \

13. How many years have taught at the K-12 level prior to this school year?
    \
14. How many years have you taught at your current grade level(s) prior to this school year?
    
    
    Teaching Practices and Pedagogy
    \
15. Think about your goals for students when teaching science. How much emphasis does each of
    the following student objectives receive?
    
    | | None | Minimal Emphasis | Moderate Emphasis | Heavy Emphasis |
    | --- | --- | --- | --- | --- |
    | Increase students' interest in science | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn basic science concepts | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn important terms and facts of science | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn science process/inquiry skills | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Prepare for further study in science | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn to evaluate arguments based on scientific evidence | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn how to communicate ideas in science effectively | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    | Learn about the applications of | $\Box$ | $\Box$ | $\Box$ | $\Box$ |
    \

• • *
    
    science in business and industry
    
    | Learn about the relationship between science, technology, and society | | | | | |
    | --- | --- | --- | --- | --- | --- |
    | Learn about the history and nature of science | | | | | |
    | Prepare for standardized tests | 16. About how often do you do each of the following in your science instruction? | | | | |
    | | Never | Rarely (e.g., a few times a year) | Sometimes (e.g., once or twice a month) | Often (e.g., once or twice a week) | All or Almost All Science Lessons |
    | Introduce content through formal presentations | | | | | |
    | Pose open-ended questions | | | | | |
    | Engage the whole class in discussions | | | | | |
    | Require students to supply evidence to support their claims | | | | | |
    | Ask students to explain concepts to one another | | | | | |
    | Ask students to consider alternative explanations | | | | | |
    | Allow students to work at their own pace | | | | | |
    | Help students see connections between science and other disciplines | | | | | |
    | Assign science homework | | | | | |
    | Read and comment on the reflections students have written | | | | | |
    \

17. In your typical class how often do students take part in the following types of activities? (if you teach multiple classes/grade levels, think about your average emphasis)
    \

• • *
    
    | | Never | Rarely (e.g., a few times a year) | Sometimes (e.g., once or twice a month) | Often (e.g., once or twice a week) | All or Almost All Science Lessons |
    | --- | --- | --- | --- | --- | --- |
    | Listen and take notes during presentation by teacher | | | | | |
    | Watch a science demonstration | | | | | |
    | Work in groups | | | | | |
    | Read from a science textbook in class | | | | | |
    | Read other (non-textbook) science-related materials in class | | | | | |
    | Do hands-on/laboratory science activities or investigations | | | | | |
    | Follow specific instructions in an activity or investigation | | | | | |
    | Design or implement their own investigation | | | | | |
    | Participate in field work | | | | | |
    | Answer textbook or worksheet questions | | | | | |
    | Record, represent, and/or analyze data | | | | | |
    | Write reflections (e.g., in a journal) | | | | | |
    | Prepare written science reports | | | | | |
    | Make formal presentations to the rest of the class | | | | | |
    | Work on extended science investigations or projects (a week or more in duration) | | | | | |
    
    Use computers as a tool (e.g., spreadsheets, data analysis)
    \
• • *
    
    | Use mathematics as a tool in problem-solving | | | | | |
    | --- | --- | --- | --- | --- | --- |
    | Take field trips | | | | | |
    | Watch audiovisual presentations(e.g.,videotapes,CD-ROMs,videodiscs,television programs,films,or filmstrips) | | | | | |
    
    Teaching Practice and Pedagogy (continued)
    \

18. About how often do students in your typical science class use computers to:
    
    | | Never | Rarely(e.g.,a few times a year) | Sometimes(e.g.,once or twice a month) | Often(e.g.,once or twice a week) | All or Almost All Science Lessons |
    | --- | --- | --- | --- | --- | --- |
    | Do drill and practice | | | | | |
    | Demonstrate scientific principles | | | | | |
    | Play science learning games | | | | | |
    | Do laboratory simulations | | | | | |
    | Collect data using sensors or probes | | | | | |
    | Retrieve or exchange data | | | | | |
    | Solve problems using simulations | | | | | |
    | Take a test or quiz | | | | | |
    \
19. How much science homework do you assign to your students in a typical week?
    \ 0-30 minutes
    \ 31-60 minutes
    \ 61-90 minutes
    \ 91-119 minutes
    \

• • *
    
    \ 2-3 hours
    \ More than 3 hours
    \

20. Are you using one or more commercially published textbooks or programs for teaching science to
    this class?
    \ Yes
    \ No
    
    If yes, please indicate the title, author, publisher, and publication year of the one textbook/program
    used most often by students in this class.
    
    | | Textbook/program used most often |
    | --- | --- |
    | Title | |
    | First Author | |
    | Publisher | |
    | Publication Year | |
    | Edition | |
    
    How would you rate the overall quality of this textbook/program?
    \

• • *
    \

Self-efficacy in Science\

21. Please indicate the degree to which you agree or disagree with each statement.

| | Strongly Disagree | Disagree | Agree | Strongly Agree | Uncertain |
| --- | --- | --- | --- | --- | --- |
| When a student does better than usual in science, it is often because the teacher exerted a little extra effort | | | | | |
| I am continually finding better ways to teach science | | | | | |
| Even when I try very hard, I don't teach science as well as I do most subjects | | | | | |
| When the science grades of students improve, it is most often due to their teacher having found a more effective teaching approach | | | | | |
| I know the steps necessary to teach science concepts effectively | | | | | |
| I am not very effective in monitoring science experiments | | | | | |
| If students are underachieving in science, it is most likely due to ineffective science teaching | | | | | |
| I generally teach science ineffectively | | | | | |
| The inadequacy of a student's science background can be overcome by good teaching | | | | | |
| The low science achievement of some students cannot generally be blamed on their teachers | | | | | |
\

• • *
    
    When a low achieving child progresses in science, it is usually due to extra attention given by the teacher
    
    I understand science concepts well enough to be effective in teaching elementary science
    
    Increased effort in science teaching produces little change in some students' science achievement
    
    The teacher is generally responsible for the achievement of students in science
    
    Students' achievement in science is directly related to their teacher's effectiveness in science teaching
    
    If parents comment that their child is showing more interest in science at school, it is probably due to the performance of the child's teacher
    
    I find it difficult to explain to students why science experiments work
    
    I am typically able to answer students' science questions
    
    I wonder if I have the necessary skills to teach science
    
    Effectiveness in science teaching has little influence on the achievement of students with low motivation
    
    Given a choice, I would not invite the principal to evaluate my science teaching
    
    | When a student has difficulty understanding a science concept, I am usually at a loss as to how to help the student understand it better | | | | | |
    | --- | --- | --- | --- | --- | --- |
    | When teaching science, I usually welcome student questions | | | | | |
    | I don't know what to do to turn students on to science | | | | | |
    | Even teachers with good science teaching abilities cannot help some kids learn science | | | | | |
    | Perceptions of Student Engagement your level of agreement with the following statements. | | 22. Think about your students during science classes in the last two weeks. In general, please indicate | | | |
    | | Strongly Disagree | Disagree | Neither Agree nor Disagree | Strongly Agree | Don't Know |
    | Students exhibit body postures that indicate they are paying attention | | | | | |
    | All students are focused on the learning activity with minimum disruptions | | | | | |
    | Students express thoughtful ideas, reflective answers, and questions relevant or appropriate to learning | | | | | |
    | Students exhibit confidence and can initiate and complete a task with limited coaching and can work in a group | | | | | |
    
    Students exhibit interest and enthusiasm in science
    \

Thank You!\

Thank you for taking our survey. Your response is very important to us.
\

• • *
    \

Science4Us Physical Science Content Assessment-Teacher\

\

Page One\

Welcome! Dear Educator, Thank you for taking the time to complete the Science4US (S4U) survey for Mid-continent Research for Education and Learning (McREL). Your participation in completing this survey is voluntary. You may choose to stop completing the survey at any time. If you have any questions about the survey, please call or email McREL Senior Director, Sheila Arens, Ph.D. She can be reached at 303-632-5625 or by email at sarens@mcrel.org. Thank you for your time!

Please answer the following questions to the best of your ability.
\

1. What is your name?\
2. Which two properties of a candle will stay about the same after the candle is melted? shape and physical state temperature and hardness color and mass thickness and texture\
3. Which of the following best describes a hypothesis? It ensures that successful results will be obtained from an experiment. It must be accepted as true by the scientific community.
   \

• • *
    
    \ It is a testable proposal that may lead to experimentation.
    
    \ It must originate from academic science and scientists.
    \

4. Although scientific knowledge is durable and robust, it is also...
   
   \ highly subjective.
   
   \ recognized as static.
   
   \ open to change.
   
   \ based purely on experimentation.
   
   Use the data table below to answer the following question.
   
   | Substance | Phase | Mass | Volume |
   | --- | --- | --- | --- |
   | 1 | liquid | 6g | 8cm3 |
   | 2 | liquid | 9g | 9cm3 |
   | 3 | liquid | 20g | 5cm3 |
   | 4 | Solid | 5g | 4cm3 |
   | 5 | Solid | 8g | 9cm3 |
   
   9mathrm c c m^{3}
   
   5mathrm c c m^{3}
   
   \ Substance 4 will float in Substance 1.
   
   \ Substance 3, in solid form, will float in the liquid form of Substance 3.
   
   9mathrm c c m^{3}
   
   \ Substance 5 will float in Substance 2.
   \

• • *
    \

6. When a physical change takes place, no new substances are produced. In which of the following situations is a physical change taking place? A piece of metal on a car begins to rust. A banana in a bowl ripens and turns yellow. A piece of paper burns in a candle flame. Ice on the surface of a lake begins to melt.\
7. Matter that expands when heated, diffuses rapidly, and is highly compressible is considered to be
   a... gas. solid. liquid. plasma.\
8. When a compound in the liquid state vaporizes and becomes a gas, which of the following characteristics of the compound remains the same? distance between the molecules chemical composition of the substance kinetic energy of the molecules specific heat capacity of the substance
   \

New Page\

Identify which microscopic view of states of matter is described in questions 7-12. solid liquid gas plasma
\

• • *
    \

9. Particles vibrate but general do not move around a lot. solid liquid gas plasma\
10. Particles are well separated with no regular arrangement. solid liquid gas plasma\
11. Particles vibrate and slide past each other. solid liquid gas plasma\
12. Particles move freely at high speeds. solid liquid gas plasma\
13. Particles are close together but with no regular arrangement.
    \

• • *
    
    solid liquid gas plasma
    \

New Page\

14. Scientific ideas are developed through reasoning. Reasoning is based on all of the following except... observable facts. inferences from data. evidence determined from data. opinions of scientists.
15. Why does ice float in a liquid? The shape of the ice cube has greater surface area. When freezing, the ice cube traps air and has less water. Ice is less dense than liquid water. The liquid forces particle movement which pushes ice up.
16. All of the following ideas support good practices in science except... developing and using models. constructing explanations based on evidence. analyzing and interpreting data. using opinions to engage in argument that supports scientific evidence.
17. Which of the following is an example of a mixture?
\

• • *
    
    sand nitrogen sugar water
    \

18. When a gas turns into a liquid, the process is called... condensation. evaporation. deposition. sublimation.\
19. Mixtures consist of particles that are... homogeneous. heterogeneous. homologous. none of the above.\
20. You are given a liquid mixture to identify as a colloid, solution, or suspension. When you shine a light from a master pointer you can see the beam spread out in the mixture. You wait several hours and find that the particles do not fall to the bottom of the beaker. The mixture is most likely a... solution. colloid. suspension. solid.\
21. Engaging in scientific practice includes... a linear, step-by-step process to obtain irrefutable results.
    \

• • *
    
    investigating by experimenting but does not include models.
    
    observing, asking questions, making claims, and planning investigations.
    
    limiting the collection of evidence to specific disciplines of study.
    \

Thank You!\

Thank you for taking our survey. Your response is very important to us.
\

• • *
    \

Kids and Science\

\

Name: _______________\

\

1. Are you a boy or a girl? (circle one)\

\

boy girl\

\

2. Do you use a computer at home? (cirlce one)\

\

yes no\

\

EXAMPLE\

\

I like vanilla ice cream.\

\

• • *
    \

3. I like using a computer.\

\

4. I am good at using a computer.\

\

5. Using a computer helps me do better in science.\

\

6. I think scientists have interesting jobs.\

\

7. I would like to be a scientist.\

\

• • *
    \

8. Science is one of my favorite subjects.\

\

9. Science is fun.\

\

10. Science makes me think.\

\

11. I enjoy learning science.\

\

12. Science is very important outside of school.\

\

• • *
    \

13. I usually do well in science.\

\

14. I am good at science.\

\

15. Science is hard for me.\

\

16. I don’t do well in science.\

\

17. It is hard for me to work on science with other\

\

students.\

\

• • *
    \

Administration Instructions for Science Motivation and Interest Survey\

\

(administer second)\

Script(read aloud): Now you are going to complete a booklet about how much you like different things in science.

Take a look at Page 1. Put your name at the top of the page where it says “Name:___”
\

Under this, you will see two questions. The first question asks if\

you are a boy or a girl. Circle the picture of the boy if you are a
\

boy. Circle the picture of the girl if you are a girl. The second\

\

question asks whether you use a computer at home. If you use a\

computer at home, cirlce the answer yes. If you do not use a computer at home, circle the answer no.
\

For all of the other questions in this booklet, you’ll circle the face\

\

that matches how you feel. For instance, if the question says “I\

\

like vanilla ice cream” and you do like ice cream, you would circle\

\

the face that is smiling. If you only kind of like vanilla ice cream\

\

(you would eat it, but you’d rather have chocolate!), you would\

\

circle the fact in the middle to show that you kind of like vanilla ice\

\

cream. And, if you really do not like vanilla ice cream, you would\

\

cirlce the frowning face. Do you have any questions? Okay, turn to page two.\

\

(Read each question to the students while they make their selections by circling the face)\

\

• • *
    \

Show what you know about: Science!\

\

Name: _______________\

\

Practice\

\

Butter going soft and runny on a hot day is an example of...\

\

B. Evaporation\

\

A. Freezing\

\

C. Melting\

\

• • *
    \

Physical Science Content Assessment-Student\

\

When you make soap bubbles,\

1. what is inside the bubbles?
\

A. Air\

\

B. Soap\

\

C. Water\

\

D. Nothing\

\

• • *
    \

Physical Science Content Assessment-Student\

\

Circle the picture that shows\

\

2. what it will look like if you put\

\

a solid in a box.\

\

A. B. C.\

\

• • *
    \

Physical Science Content Assessment-Student\

\

How do particles move when\

3. temperatures get colder?
\

A. Fast\

\

B. They don’t move at all\

\

C. Slow\

\

D. They run\

\

• • *
    \

Physical Science Content Assessment-Student\

\

Which state of matter is\

4. usually measured by a scale?
\

A. Solid\

\

B. Liquid\

\

C. Gas\

\

D. A book\

\

• • *
    \

Physical Science Content Assessment-Student\

\

When should you wear a hat and\

\

5.scarf? Circle the thermometer that\

\

goes with wearing a hat and scarf.\

\

• • *
    
    Physical Science Content Assessment-Student
    \
    This graph shows the number of some pets in a
    6.
    pet store. Which of the following data tables has
    the number of pets shown in the graph?
    \
    | cats | dogs |
    | --- | --- |
    | 3 | 6 |
    
    | turtles | dogs |
    | --- | --- |
    | 3 | 5 |
    \
• • *
    \

Physical Science Content Assessment-Student\

\

Explain or draw a picture of\

7. what will happen:
\

• =
  \

• • *
    \

Physical Science Content Assessment-Student A teacher buys a cookie. The cookie\

\

weighs 2 ounces. The cookie breaks into\

8. three pieces. How much does the broken cookie weigh?
\

A. 0 oz. B. 1 oz. C. 2 oz. D. 3 oz.\

\

• • *
    \

Physical Science Content Assessment-Student\

\

Circle the objects that will float\

9. in water.
\

• • *
    \

Physical Science Content Assessment-Student\

\

Circle picture(s) of matter.\

10.
\

• • *
    
    Physical Science Content Assessment-Student
    \

Draw a picture of what you\

11.
\

would be doing if you were a scientist.\

\

• • *
    \

Physical Science Content Assessment-Student\

\

Circle each picture below that\

12.
\

shows how you could do science.\

\

Investigate Ask questions\

\

Observe with your senses\

\

Use Tools\

\

Draw pictures or write things down\

\

Share and explain your ideas\

\

• • *
    \

Administration Instructions for Science Content Knowledge Assessment (administer first)\

\

Script(read aloud): Today you will look at some pictures and answer some\

\

questions about science. Whether the questions seem easy or hard, it is\

\

important that you do your best. If you have a question, raise your hand. It\

\

is important that you do not call out your questions or answers.\

\

I will read each question out loud. Follow along as I read the question and\

choose the best answer. First, put your name on the top of your booklet, where it says “Name:___”
\

On that same page, you will see a practice question. Let’s try this one\

\

together. The question says: “Circle the best answer. Butter going soft and\

\

runny on a hot day is an example of…” Under this question are three\

\

choices. Is butter going soft and runny on a hot day “A” Freezing? Is that\

\

the correct answer? Is it “B” Evaporation (see the clouds, raindrops, and\

water?)? Or is it “C” Melting—like what might happen to a snowman on a sunny day?
\

Circle the picture or the letter with the picture that you think is the right\

\

anwer to the question. [When approximately 95% of students are finished,\

\

say] Good work! The correct answer is C, Melting. Did you all get that one?\

Do you have any questions before we start? [take a moment to answer all student questions]?
\

Before we begin, remember to listen to me read each question before you\

\

answer. You can follow along as I read. Ok, let’s begin. Open your booklet to\

\

as I read. [Do NOT provide the answers to the remaining questions]\

\

• • *
    
    Appendix D: Summary of the BSCS 5E Instruction Model
    Phase Summary
    
    | Phase | Summary |
    | --- | --- |
    | Engagement | The teacher or a curriculum task accesses the learners' prior knowledge and helps them become engaged in a new concept through the use of short activities that promote curiosity and elicit prior knowledge. The activity should make connections between past and present learning experiences, expose prior conceptions, and organize students' thinking toward the learning outcomes of current activities. |
    | Exploration | Exploration experiences provide students with a common base of activities within which current concepts (i.e., misconceptions), processes, and skills are identified and conceptual change is facilitated. Learners may complete lab activities that help them use prior knowledge to generate new ideas, explore questions and possibilities, and design and conduct a preliminary investigation. |
    | Explanation | The explanation phase focuses students' attention on a particular aspect of their engagement and exploration experiences and provides opportunities to demonstrate their conceptual understanding, process skills, or behaviors. This phase also provides opportunities for teachers to directly introduce a concept, process, or skill. Learners explain their understanding of the concept. An explanation from the teacher or the curriculum may guide them toward a deeper understanding, which is a critical part of this phase. |
    | Elaboration | Teachers challenge and extend students' conceptual understanding and skills. Through new experiences, the students develop deeper and broader understanding, more information, and adequate skills. Students apply their understanding of the concept by conducting additional activities. |
    | Evaluation | The evaluation phase encourages students to assess their understanding and abilities and provides opportunities for teachers to evaluate student progress toward achieving the educational objectives |
    \
• • *
    
    Appendix E: Early Group Teacher Feedback about Science4Us
    Please respond to the following statements about Science4Us by indicating the extent to
    which you agree or disagree.
    
    | | Strongly Disagree | Disagree | Agree | Strongly Agree | Don't Know |
    | --- | --- | --- | --- | --- | --- |
    | The content of Science4Us is developmentally appropriate for my students. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 3(42.9%) | 0(0.0%) |
    | The content of Science4Us supports emergent literacy concepts. | 0(0.0%) | 0(0.0%) | 6(85.7%) | 1(14.3%) | 0(0.0%) |
    | The Science4Us program is open-ended and allows students to explore. | 0(0.0%) | 1(14.3%) | 5(71.4%) | 1(14.3%) | 0(0.0%) |
    | The States of Matter module includes content my students need to learn. | 0(0.0%) | 2(28.6%) | 3(42.9%) | 2(28.6%) | 0(0.0%) |
    | The States of Matter module includes skills my students need to learn. | 0(0.0%) | 1(14.3%) | 3(42.9%) | 3(42.9%) | 0(0.0%) |
    | The States of Matter module is aligned to the relevant science standards for my students. | 0(0.0%) | 4(57.1%) | 2(28.6%) | 1(14.3%) | 0(0.0%) |
    | The Science4Us program is effective in communicating target science knowledge and skills. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 4(57.1%) | 0(0.0%) |
    | The program provides problem solving opportunities. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 3(42.9%) | 0(0.0%) |
    | The content is free of gender, ethnic, and/or racial bias. | 0(0.0%) | 0(0.0%) | 0(0.0%) | 7(100.0%) | 0(0.0%) |
    | The Science4Us program provides feedback that is effective and non-threatening. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 3(42.9%) | 0(0.0%) |
    | The Science4Us program moves from level to level at appropriate intervals. | 0(0.0%) | 0(0.0%) | 4(66.7%) | 2(33.3%) | 0(0.0%) |
    | The Science4Us program stimulates student interest. | 0(0.0%) | 0(0.0%) | 2(28.6%) | 5(71.4%) | 0(0.0%) |
    | The Science4Us program encourages active student involvement. | 0(0.0%) | 0(0.0%) | 1(14.3%) | 6(85.7%) | 0(0.0%) |
    | The rate and level of difficulty of Science4Us can be adjusted for individual students. | 0(0.0%) | 0(0.0%) | 5(71.4%) | 1(14.3%) | 1(14.3%) |
    | The display/interface of Science4Us is effective. | 0(0.0%) | 0(0.0%) | 5(71.4%) | 2(28.6%) | 0(0.0%) |
    \
• • *
    
    | Students can exit/quit the Science4Us program easily, quickly,and safely. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 4(57.1%) | 0(0.0%) |
    | --- | --- | --- | --- | --- | --- |
    | The Science4Us program provides students with help that is clear and effective. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 3(42.9%) | 0(0.0%) |
    | The Science4Us program can be used independently by my students. | 0(0.0%) | 1(14.3%) | 4(57.1%) | 1(14.3%) | 1(14.3%) |
    | The Science4Us program operates quickly. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 3(42.9%) | 0(0.0%) |
    | I would recommend the Science4Us States of Matter module to a colleague. | 0(0.0%) | 0(0.0%) | 2(28.6%) | 5(71.4%) | 0(0.0%) |
    
    Think about your planning and implementation of each of the eight sessions in the States of
    Matter module. In how many sessions did you consult the...
    
    | | Never(0 sessions) | Rarely(1-2 sessions) | Sometimes(3-4 sessions) | Often(5-6 sessions) | All or Almost All(7-8 sessions) |
    | --- | --- | --- | --- | --- | --- |
    | Teacher | 0 | 0 | 4 | 2 | 1 |
    | Guide? | (0.0%) | (0.0%) | (57.1%) | (28.6%) | (14.3%) |
    
    | What is the most important thing you learned from the Teacher Guide? |
    | --- |
    | Background/prior knowledge |
    | Content |
    | Content - this is a unit that I hadn't taught and so this was a good refresher |
    | Gave me some background knowledge before introducing the lesson to the kids |
    | Helped me prepare for the lesson for content and time needed. |
    | Layout of the lesson |
    | Materials needed, inquiry questions and pacing. |
    \
• • *
    
    Please explain how the Teacher Guide can be improved:
    
    Wouldn't mind having a paper copy included with the program, that way you can have the
    program open while you are reading the lesson
    
    Think about your planning and implementation of each of the eight sessions in the States of
    Matter module. In how many sessions did you consult the...
    
    | | Never(0 sessions) | Rarely(1-2 sessions) | Sometimes(3-4 sessions) | Often(5-6 sessions) | All or Almost All(7-8 sessions) |
    | --- | --- | --- | --- | --- | --- |
    | How-to | 2 | 4 | 1 | 0 | 0 |
    | Videos? | (28.6%) | (57.1%) | (14.3%) | (0.0%) | (0.0%) |
    
    Please explain why you never/rarely used the How-to Videos:
    
    | Please explain why you never/rarely used the How-to Videos: |
    | --- |
    | Didn't have the time. |
    | Program was very self-explanatory, didn't need to |
    | Time |
    | Time was a factor. |
    | Lack of time on my part |
    | Didn't have time as this was not given to us until the last 2 weeks of school and there just wasn't time with everything else that needed to get done to sit and watch a video |
    
    Didn't have the time.
    
    Program was very self-explanatory, didn't need to
    
    Didn't have time as this was not given to us until the last 2 weeks of school and there just wasn't
    time with everything else that needed to get done to sit and watch a video
    
    What is the most important thing you learned from the How-to Videos?
    
    Just that--how to teach it or navigate
    
    Think about your planning and implementation of each of the eight sessions in the States of
    Matter module. In how many sessions did you consult the...
    
    | | Never(0 sessions) | Rarely(1-2 sessions) | Sometimes(3-4 sessions) | Often(5-6 sessions) | All or Almost All(7-8 sessions) |
    | --- | --- | --- | --- | --- | --- |
    | Offline materials(such as PDFs)? | 2(28.6%) | 0(0.0%) | 4(57.1%) | 1(14.3%) | 0(0.0%) |
    \
• • *
    
    I enjoyed the experiments, although we changed one.
    
    Content
    
    Other activities that my students could use
    
    | Please explain how the offline materials (such as PDFs) can be improved: |
    | --- |
    | I really liked them, not all entirely leveled for kindergarten |
    | Instead of one idea offer several that would teach the same principle. |
    | n/a |
    | There wasn't a blackline master for the web |
    
    There wasn't a blackline master for the web
    
    | | Strongly Disagree | Disagree | Agree | Strongly Agree | Don't Know |
    | --- | --- | --- | --- | --- | --- |
    | The Teacher Guide materials gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 5(71.4%) | 2(28.6%) | 0(0.0%) |
    | The How-to Videos gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 1(14.3%) | 2(28.6%) |
    | The PDFs and Short Videos gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 3(50.0%) | 2(33.3%) | 1(16.7%) |
    | The embedded professional development (Teacher Guide, How-to Videos, PDFs and Short Videos) provided comprehensive information about the States of Matter module. | 0(0.0%) | 0(0.0%) | 2(28.6%) | 3(42.9%) | 2(28.6%) |
    | The embedded professional development (Teacher Guide, How-to Videos, PDFs and Short Videos) met my individual learning needs. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 2(28.6%) | 2(28.6%) |
    | The embedded professional development (Teacher Guide, How-to Videos, PDFs and Short Videos) provided useful information and strategies for teaching the States of Matter module. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 2(28.6%) | 2(28.6%) |
    \
• • *
    
    | The States of Matter module increased my capacity to teach students about the three states of matter. | 0(0.0%) | 0(0.0%) | 1(14.3%) | 6(85.7%) | 0(0.0%) |
    | --- | --- | --- | --- | --- | --- |
    | I felt comfortable teaching the States of Matter module. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 4(57.1%) | 0(0.0%) |
    
    Think about your planning and implementation of each of the eight sessions in the States of
    Matter module. In how many sessions did you consult the following embedded professional
    development materials?
    
    | | Never(0 sessions) | Rarely(1-2 sessions) | Sometimes(3-4 sessions) | Often(5-6 sessions) | All or Almost All(7-8 sessions) |
    | --- | --- | --- | --- | --- | --- |
    | Teacher | 0 | 3 | 4 | 0 | 0 |
    | Guide | (0.0%) | (42.9%) | (57.1%) | (0.0%) | (0.0%) |
    | How-to Videos | 2(33.3%) | 3(50.0%) | 1(16.7%) | 0(0.0%) | 0(0.0%) |
    | PDFs and Short Videos | 2(33.3%) | 2(33.3%) | 1(16.7%) | 0(0.0%) | 1(16.7%) |
    
    Please respond to the following statements about Science4Us by indicating the extent to
    which you agree or disagree.
    
    | | Strongly Disagree | Disagree | Agree | Strongly Agree | Don't Know |
    | --- | --- | --- | --- | --- | --- |
    | The Teacher Guide materials gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 5(71.4%) | 2(28.6%) | 0(0.0%) |
    | The How-to Videos gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 1(14.3%) | 3(42.9%) |
    | The PDFs and Short Videos gave me a better understanding of how to facilitate learning in the States of Matter module. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 1(14.3%) | 2(28.6%) |
    | The embedded professional development (Teacher Guide, How-to Videos, PDFs and Short Videos) provided comprehensive information about the States of Matter module. | 0(0.0%) | 0(0.0%) | 4(57.1%) | 2(28.6%) | 1(14.3%) |
    | The embedded professional development (Teacher Guide, How-to Videos, PDFs and Short Videos) met my individual learning needs. | 0(0.0%) | 0(0.0%) | 3(42.9%) | 3(42.9%) | 1(14.3%) |
    | The embedded professional development (Teacher Guide, How-to | 0(0.0%) | 0(0.0%) | 3(42.9%) | 3(42.9%) | 1(14.3%) |
    \
• • *
    
    Videos, PDFs and Short Videos) provided
    useful information and strategies for
    teaching the States of Matter module.
    
    | The States of Matter module increased my capacity to teach students about the three states of matter. | 0(0.0%) | 0(0.0%) | 2(28.6%) | 5(71.4%) | 0(0.0%) |
    | --- | --- | --- | --- | --- | --- |
    | I felt comfortable teaching the States of Matter module. | 0(0.0%) | 0(0.0%) | 2(28.6%) | 5(71.4%) | 0(0.0%) |
    
    Did your use of the Science4Us States of Matter module result in any changes in your
    instructional practices?
    \
    Open-Text Response Breakdown for "Yes (please explain)"
    
    As long as I had the module I would love to use it.
    
    It offered new ideas of what engaged students and excited them towards learning.
    
    More class discussions using visuals (experiments during the module)
    
    As a result of using the Science4Us States of Matter module, how much did your students
    learn about the states of matter?
    \ \
• • *
    
    Please explain your rating. Give specific examples of student behaviors or other indicators of student learning. Student conversations and interactions The students would yell at the screen the correct answers. They got it. Some students understand the three states of matter. They did not know what matter was before this. First, this is not a 2nd grade standard so most everyone had very little knowledge of matter. I feel in the short amount of time we had the students learned quite a bit! I could see that the word "matter" and its context explained several questions that the students had from earlier in the year. Wished I would have had more time to teach it fully. In the Lesson #8 (assessment of the unit), I was so surprised at how well they did, and how well they were able to write about their learning in their journals. Students were able to give specific details on the three states of matter. We also had another speaker come in to talk about weather. She asked about the three states of matter and how the molecules look in all. She was pleasantly surprised that the kids could give details.
    
    What barriers exist to using Science4Us in your instruction?
    \

Open-Text Response Breakdown for "Lack of training (describe additional needs):"\

Might have been helpful to meet face to face with facilitators. I'm more of a hands on learner and need to ask clarifying questions.
\

Open-Text Response Breakdown for "Other (please specify):"\

Not sure what science standards will end up being next year. Time!
\

• • *
    \

Open-Text Response Breakdown for "Other (please specify):"\

I would love to use this with my entire class, on individual computers, but lack the resources to do it successfully The timing of this was unfortunate. I believe it could have been more fully taught if I'd had it earlier in the year

What else do you want to share about your experiences using the Science4Us States of Matter module? Loved the program-wish I would have had more time to use it/interact with it. I loved it, the kids loved it. Sometimes science takes a lot of extra preparation gathering materials, and this is a good way to go with the videos. They still get a good visual and experience to explore with experiments. I really enjoyed using it and was surprised at how easy it was to use and how much the kids learned. I really enjoyed the variety of concepts the program offered and the colorful, creative presentation. The kids were really excited about it and several commented that they got onto it at home. Very exciting especially considering the time of the year.
\