Looking at Edtech Through an Equity Lens

Two years ago, I went from buyer of education technology to builder of it. I was formerly the redesign administrator at a district personalized learning school, and I oversaw the implementation of many edtech products. I left to build a technology platform that helps students set goals, reflect on learning strategies, and focus on the process of learning rather than just the products of it.

One issue bubbled up consistently at my school, and now it’s something that I’m actively working to design against: a concern that edtech products can impact our most struggling students in a way that leaves them further behind.

This unintended consequence can be masked in technology-rich environments, since students often appear busy and can fly under the radar for a while. Unless we’re explicitly looking, these equity implications aren’t always easy to spot.

First, Do No Harm

My last school implemented a comprehensive personalized learning platform in which students worked through online lessons at their own pace with little adult support. The goal was for teachers to spend more time working with students on projects and applications of content. We went through weeks of professional development, transformed our instructional model, and implemented with 250 sixth graders.

We saw our average scores improve on summative and benchmark assessments. However, on deeper analysis, we found that students who were on or above grade level were soaring, but those who were behind—more than half our student population—were actually doing significantly worse. The platform and our implementation were not meeting their needs, and we were unintentionally widening the equity gap.

The difference was so large that we pulled out of the program after a year to go back to the drawing board. We needed to find a design that gave all students a chance to soar.

Design Matters

I’ve had similar experiences with content applications, discovering design flaws that disproportionately affect certain populations of students. Our math team, for example, selected a popular math app known for its rigor and alignment to standards. In practice, however, we discovered deep design flaws that impacted struggling students.

For one, the software used very complex language. A word problem might say, “Imagine two adjacent households and the amount of energy they expend,” instead of “Two neighbors get their electricity bill.” We served large populations of English learners and students who were behind in reading, and these language choices immediately shut them down.

Another example was how the software provided feedback. Students would do 10-question assignments, but they didn’t find out how they were doing until they completed all 10. So a student would work really hard for an hour and then get a 1/10—a frustrating experience that led some students to disengage and give up. The program was not designed for students who were behind and needed help as they worked.

I worry that these types of issues might be pegged to student behavior or motivation; in reality, the product design is at fault.

All Products Aren’t Necessarily Good for All Students

I also wonder if even the most widely used products are closing or widening equity gaps. Digital portfolio apps can be beautiful, but do they just create a way for deeply engaged parents to become more engaged, and not really move the needle for others? If teachers stop making parent calls because they’re using communication apps now, what happens to families that don’t have a smartphone or computer at home?

Further, many commonly used edtech products don’t have rigorous studies demonstrating efficacy. I’ve seen many published results that strongly suggest correlation equals causation, often without a control group. Students may improve their reading level when they use a certain reading tool, but they might have improved by the same amount with traditional instruction. Or perhaps the improvement is the result of a high quality teacher. Well-designed studies are critical to unpack these issues. It’s also incredibly rare to see impact results disaggregated by different student groups, making equity concerns difficult to uncover.

What Can Schools Do?

Here are a few specific actions schools and teachers can take to ensure that their edtech choices are serving all students:

• Ask questions of edtech companies, particularly around impact and their experience with different student populations. Apply a critical eye to studies posted on websites, looking for the problems discussed above.
• Ask yourself critical questions about how and why something works: Is this tool truly scaffolding learning experiences, or is it a worksheet in a digital format?
• Sign in as a student and go through all the core elements of a tool. Put yourself in the shoes of one of your higher performing students and one of your lower performing students. Where are the challenges? How can you solve them?
• Follow your gut educator sense. If you’re worried that a tool is going to help some students at the expense of others, brainstorm differentiated scaffolds you can use to support students. If that’s not enough, consider other tools in the same space.
• Pilot with a diverse set of students and pay attention to differences in how they are using, enjoying, and experiencing results from a product. Even short pilots (two to four weeks) can provide a lot of information.
• Break down results data by relevant subgroups of students. Look specifically for unintentional widening of equity gaps.

These observations are not intended to push teachers to stop using tools they like—the goal is to make sure we’re not accidentally leaving a set of students behind.