| Principal Investigator: | Deborah Lowe Vandell |
| Researchers: | Valerie Hall, Pilar O’Cadiz, Andrea Karsh, Rahila Simzar |
| Funding: | The Samueli, Noyce, and Bechtel Foundations |
Description | Key Findings | Report
Description:
The Power of Discovery: STEM2 Initiative is a multi-site intervention that seeks to promote young people’s engagement, interest, and understanding of science, technology, engineering and mathematics (STEM) subject matter by providing staff professional development, online resources, and curricula to 625 afterschool program sites in California. UC Irvine was serving as the external evaluator of this project.
Website: http://powerofdiscovery.org. See link to “Evaluation”.
Key Findings:
Key Findings [2014-15 Evaluation]:
Findings from the final Power of Discovery evaluation indicate that the Initiative was successful in providing opportunities for staff to engage in meaningful STEM Professional Development and to meet frequently with colleagues about STEM programming at their sites. In formal tests of the PoD Logic Model conducted in both 2013-14 and 2015, STEM Professional Development was related to staff beliefs and confidence in the STEM area, which predicted higher quality program experiences. Program quality program experiences then predicted relative gains in student-level academic and social outcomes.
Key Findings [2013-14 Evaluation]:
Findings from the 2013-14 study of the Power of Discovery Initiative suggest that:
- The Power of Discovery provided training activities and collaborative opportunities that supported the professional development of a diverse workforce.
- Staff development in the STEM area and collaboration with classroom teachers can serve as effective supports for the development of afterschool staff beliefs about the value of STEM learning and their perceived efficacy in implementing STEM activities.
- As posited in the logic model, these staff beliefs were found to be linked to the improvements in student engagement and interest, suggesting that staff beliefs may be a mechanism or strategy for impacting program activities.
- Staff beliefs and program activities were related to gains in student self-reports of math efficacy and to other student outcomes.
- STEM activities that are engaging and challenging can foster the development of students’ interest and motivation in science and mathematics, including future careers in these areas.
- Suggestive correlations between observations of the quality of STEM activities and staff and student beliefs were also found.
Taken together, these findings suggest that the PoD was effective during its first full year of implementation.
Key Findings [Pilot Study 2011-2012]:
The background of program staff reflected the diversity of the students and communities served. About 70% of staff were Hispanic/Latino or Caucasian, about 16% were African American, and about 14% were Asian American, Pacific Islander, Filipino, or Other. 95% of program staff had attended some college, and 45% had completed a 2-year or 4-year college degree. Over 90% of staff had some experience in other afterschool programs (other than current position), and over 75% had other experience in school settings.
From the pre-participation Staff Survey to the post-participation Staff Survey, the amount of time per week that staff reported spending on STEM activities increased. Staff reports also showed an increase, from pre to post, in the frequency of staff meetings about STEM issues, in the frequency of interactions with classroom teachers about STEM topics taught in the classroom, and in the frequency of STEM-related events that staff held for parents.
Program staff completed and submitted documentation of 990 STEM activities. The majority of activities (67%) focused on just one STEM area; about 16% of the activities focused on three or four STEM areas. 80% of the activities were reported by staff to be either “very challenging” for the students (22%) or “somewhat challenging” (58%), and students were either “very engaged” (41%) or “mostly engaged” (35%). Overall, staff reported that the activities went well: 35% went “very well” and 42% went “mostly well.”
Students in grades 3-12 completed 2,298 surveys, reporting on their own skill development, positive behavior, and program experiences. Over 60% of students reported good or excellent quality experiences with staff and activities at their program, and over 50% of student reported good or excellent experiences with peers. Analyses of associations between program experiences and outcome scores were all significant: students who reported higher quality of experiences with staff, activities, and peers, also had lower scores in misconduct and higher scores in math efficacy, science efficacy, work habits, and social competencies (p < .01). Regression analyses, which controlled for gender and pre-participation scores, were also positive and significant. Greater quality of student-reported experiences with staff and program activities was significantly related to higher scores in math efficacy, science interest, science efficacy, science career, and work habits (p < .01), a more positive view of the future (p <.05), and lower scores in misconduct (p < .01). Greater quality of student-reported experiences with peers was significantly related to higher scores in all student outcomes except misconduct (p < .01).
Analyses of the effects of staff beliefs and efficacy on student outcomes were also positive and significant. Stronger staff beliefs about the importance and benefit of STEM activities for students was significantly related to higher post-participation outcome scores in science efficacy, science interest, and work habits. Greater amount of time spent on STEM activities was significantly related to higher scores in math efficacy and greater student-reported likelihood of a future career in science. A greater sense of staff efficacy in implementing STEM activities was significantly related to a greater sense of student efficacy in science.