5 Steps to Evaluate Intervention Programs

If you are reading this, your school likely has an intervention program. It may be called Response to Intervention (RTI) or multitiered system of support (MTSS), but some type of programmatic approach to intervention is likely in place. The education community has spent billions of dollars to provide this increased level of support for kids. The issue, however, is that it has not worked, and we don’t talk enough about evaluating intervention programs. 

In 2008, John Hattie’s research showed that intervention had a statistically significant impact (effect size) upon student achievement. This became the justification for how intervention programs are implemented. However, research has found that, despite the ubiquitous usage of RTI, not only has it been ineffective, but it has actually had an inverse effect in some schools. 

This is our challenge: We must figure out how to do RTI/MTSS well.

Does intervention work… or not?

The answer is yes (to both). Intervention systems work when they are supported by great Tier 1 instruction, embedded as research-based tools, implemented with fidelity, and when the most qualified educators are working with our neediest students. 

If those four qualifiers are not met, then RTI or MTSS is likely costing your school or district an exorbitant amount of money and time while not having a significant impact on the performance of your students. There are five steps an administrator can take to examine the effectiveness of these programs.

5 steps to evaluate intervention programs

Step one: Determine what success looks like. These days, most schools just do RTI or MTSS because it is “what we have always done.” However, as with anything being done in schools, the return on investment (ROI) of both time and money should be calculated. So, while there are some universally accepted answers to what success should look like, each school system needs to make this decision and start measuring against it. 

As you consider this, remember that intervention is designed to close gaps, not just cause a year of growth. A great Tier 1 curriculum with solid instruction should create a year of growth for each student without intervention. Intervention is designed to help students not at grade level catch up to their peers and local or national norms. This means that a student should ideally move from (for example) the 18th percentile to the 30th percentile, not just move consistently throughout the year and remain in the 18th percentile. 

Step two: Confirm that Tier 1 is solid. I was asked once how I would fix RTI in a school that didn’t have enough resources to provide interventions to every student who needed it. The answer is simple. If that is the issue, the interventions are not the problem: The core curriculum and instruction are. 

A school cannot intervene its way out of low-rigor or low-quality core instruction. To be clear, I think most teachers are teaching their tails off. I do believe, however, that we (on aggregate) do not demand enough from our students. Increased rigor leads to increased engagement, which leads to increased performance. It is simple, just not easy. 

Step three: Evaluate the tools you are using. RTI and MTSS are based on interventions. Not all interventions are created equal. 

There are great (free) resources on the internet to help you understand whether or not the interventions being used can help produce the desired results you established in step one. Too often, intervention is simply homework help and not remediation of deficient skills. This throws off the purpose of the entire program. 

When selecting programs, entities such as the Florida Center for Reading Research (FCRR) or the What Works Clearinghouse (WWC) help school leaders determine the validity of the intervention being used. For instance, the WWC maintains a dashboard where administrators can check the research base and the effectiveness of interventions. 

In addition, it is vital to measure what is actually working for your students. The data from each intervention group should be measured in a disaggregated fashion and compared with a static benchmark. For example, if students given Intervention A move from an average percentile rank of 23 to 31 after its use, and students given Intervention B simply maintain their percentile rank of 23, then those two data points mean vastly different things. Aggregating the data of students in both intervention groups would show an overall increase from the 23rd percentile to the 27th. It is going to appear to be a success. And overall, it is. But the truth is that only one of the two interventions is actually achieving success.

Step four: Run a fidelity check. It takes time for interventions to be successful once you implement research-based tools. In far too many schools I work with, I find research-based interventions being used less intensely than necessary to make gains.

If a company sells you a product and says it only works if students are exposed for 30 minutes a day and you provide 30 minutes every other day, then do not be surprised when it does not work. I liken this to taking a prescription medication. Penicillin works, but only if you take it in the prescribed dosage from your doctor. Remember, as intervention tiers continue to increase (from Tier 1 to Tier 2 to Tier 3), the intensity of the interventions does as well. In Tier 2 and Tier 3, it becomes even more vital to ensure that each student is getting the full benefit of receiving the intervention by ensuring that they are receiving the full time. 

Step five: Realign staffing to help students with the greatest needs. As a school leader, I can say with confidence that my school could not run without our instructional aides and paraprofessionals. However, it makes no sense to take our students with complicated learning needs out of their classroom, which is generally led by a trained professional, and place them with support staff in order to remediate their deficiencies.

I encourage everyone to do a quick test of their system to see which adult is with their neediest students the majority of the time. When intervention systems work, it is when the most highly trained professional is working with the neediest children. 

Imagine scenarios where certified teachers are helping kids with the needed interventions while instructional aides and paraprofessionals are working to support the learning of students who are already achieving at grade level after the certified teacher has completed direct instruction. Simply, the kids with the greatest needs deserve the greatest level of support.