After the planning committee developed a theory of action, we needed to think about what changes we could make that would bring about improvement, and move us toward our aim of increasing the number of students who are successful in advanced high school STEM courses.
We chose to focus on one of the primary drivers—improving instruction in advanced STEM education. From there, we narrowed our focus even further to supporting students’ literacy skills as a key support to improving instruction.
KSTF Senior Fellow Heather Haines (also a member of the planning committee) volunteered her Advanced Placement (AP) Chemistry classes for the pilot study. Heather wants to develop her students’ literacy skills, specifically by building their comprehension of scientific texts and improving their ability to demonstrate understanding by citing from a text to make an argument. These skills are crucial for students to master in order to do well on the AP Chemistry exam, and Heather felt that her students needed more support in this area.
Each week Heather scaffolds the students’ reading and writing skills by using discussions, different writing prompts, and/or examples. The committee collects data on the strategies that Heather uses and scores the students’ understanding and skills using a rubric.
Early in the semester, Heather’s goal was to make sure that her students could appropriately cite from a piece of text about chemistry. She created a series of prompts that she gave to her students twice a week, to complete at the beginning of class. (This is the prompt Heather used during the first week of this project, and this is the rubric we used to score question 3.)
Heather kept track of how many students were in class on the day of the prompt and any questions or confusions that came up, and how the discussion went after the prompt on a cover sheet. She then scanned and uploaded the cover sheet with all student responses. Other members of the planning committee scored the question of interest during the week and kept track of the number of students that scored a 3 or 4. Here are the results of the first few weeks:
Later in the semester, once Heather felt that her students could cite text, she revised her prompts to help us measure how well they could use a given quote from text as evidence to support a claim and reasoning (this is an example of a prompt used later in the semester). Of course, the rubric had to evolve to reflect that we were now measuring something different.
This pilot project is still a work in progress, but it has helped us learn a lot about implementing small changes that might lead to our overall aim of increasing the number of students who successfully complete STEM courses. We also learned a lot about designing a Plan-Do-Study-Act (PDSA) cycle, which is a key piece of improvement science, including how we might measure whether or not something is an improvement, and the importance of tracking data over time. We also thought about how we might measure student success throughout the year prior to capstone grades, like AP scores, that only happen once per year.
The work the planning committee did with Heather is an example of the type of work that teams might be engaged in. Ultimately, we want teams to share what they learn with each other and start testing improvements in other sites—but that will take us at least a year to get to that point, maybe longer.