A 3-Step Process for Evaluating Costly Investments

Over the past two and a half years, teachers, schools, and districts have done everything possible to support students academically, socially, and emotionally. As most schools return to a sense of normalcy this year, it provides an opportunity to reevaluate the investments made during the pandemic. Many districts may have spent money on resources with an annual fee or other upkeep costs, and now is the perfect time to evaluate whether those technologies are still supporting what our students need to learn. I have a three-step process that schools can use to find what is working.

1. Check Fidelity

As districts decide whether solutions are working, it’s important to analyze whether they are being implemented with fidelity. For example, if an intervention needs 50 minutes per week of engagement to move the needle on achievement and a class is using it for only 35 minutes per week, then we are wasting those 35 minutes.

This failure in planning tends to lead to teachers questioning the effectiveness of the product without the product even having a chance. I equate this to having strep throat, being prescribed antibiotics, and then taking only half of the dosage every day. Nobody would do that, but we do it with high-priced and time-intensive tools designed to support student learning.

From the administrative point of view, it is vital to check on whether or not teachers and students are actually using the product. One simple step to get started is to list all of the solutions and interventions you are deploying and then simply create a spreadsheet with the number of minutes each is intended to be used per week. Compare this with both usage data and the number of minutes available in the schedule. Most schools that do this find it would be literally impossible to use all their tools with fidelity.

Once a fidelity check is complete, it becomes imperative to see how the tools or solutions being used all interact with each other. In our district, we refer to this as “integration.” We attempt to answer whether each tool complements the other, duplicates efforts, or even pulls in a different direction.

2. Less Can Be More

If a district finds that it is impossible for all of the products to be used, it’s time to make some difficult decisions. Too often, we add multiple solutions at the same time. Even if we find success, it is impossible to determine which support is moving the needle or if it truly is the synergy of everything working together. Most schools threw the entire kitchen sink at the problems that the pandemic created, so we must be discerning at this moment.

This is where qualitative feedback from teachers, students, and parents is extremely important. Schools must work together at all levels to ensure that the supports and interventions being used meet the needs of both the students and the teachers. In many cases, less can be more.

An activity I encourage people to do in this scenario is what I titled “Initiative Purge.” In a faculty meeting, simply have staff list all initiatives or programs that are currently being used in their building on sticky notes, and have them place the notes in categories such as academic solutions, social and emotional solutions, behavioral solutions, locally created initiatives, etc., on butcher paper hung on the wall.

No matter how you proceed and with what categories, there will likely be overlap. That is OK; use the terms and the categories that are most relevant to your unique context. Then, start to have a real discussion with the people who are using these systems to see what is being used and what is working.

Once the purge is complete (this can be literal as described above or a less regimented approach), it becomes vital to measure effectiveness. In essence, the first two steps are done to ensure that the rollout of each new initiative or solution has been done appropriately. Now we must determine whether or not the tool itself is working.

3. Does Reality Match the Sales Pitch?

Many districts go to great lengths to ensure that they are going to make a quality purchase. They identify a need, meet with multiple service providers, create a rationale for the expenditure, and present this information to all stakeholders.

Unfortunately, the scrutiny typically ends there, and districts fail to analyze whether a purchase is the right fit. Many purchases turn out to be wrong—the key is not to stay wrong. This takes humility as a leader and courage as a teacher. For a teacher to voice concern that a product, resource, or support is not meeting expectations takes courage. That courage is exactly what schools need to ensure that they are making wise investments with their limited resources and, more important, to ensure that they are not inadvertently burdening their teachers with resources that do not help them move their students forward.

In our district, we are going through a “solution divorce” right now as we move on from a product that does not serve us anymore. To be clear, it was our stakeholders (administration, teachers, students), not me as the superintendent, who noticed this first and led the change. We have a keen focus on this, so while we did not need to use an initiative purge to get to this point, it certainly could have facilitated this, and we would have ended up at the same place.