Demystifying Ed-tech Algorithms

We have just put up a full schedule for the Digital Pedagogy Lab 2016 Institute and are excited to announce special workshops with Jeremy Dean, Zach Whalen, Jess Reingold, Jeffrey McClurken, Autumm Caines, Maha Bali, Chris Friend, and Kris Shaffer. In this post, Kris talks about the two workshops he’ll be leading in Fredericksburg. There are still a few spots left at the Institute, so we hope you’ll join us in August!

Every assignment is an algorithm. Every curriculum is an algorithm. As long as there is a logic and a goal ― explicit or implicit ― there is an algorithm.

Consider an elementary-school arithmetic worksheet. They typically don’t present problems in an obvious order: 1+1, 1+2, 1+3, … Why? It is too easy for students to “hack” the assignment ― to discover the algorithm behind it and use the algorithm to get the answers, rather than perform the mental exercise the assignment is meant to encourage. A random ordering of questions ― a different algorithm ― removes that hackability, making the intended mental exercise a more likely process for students to engage in the assignment. A different random ordering of questions for each student ― yet another algorithm ― reduces the likelihood of rote copying, but also diminishes collaborative potential. And increases grading time if the assignment is paper-based, leaving the teacher less time for creative pedagogical planning or interaction with individual students.

All of these assignment structures have an underlying logic and a set of goals behind them. What do we want students to gain from the assignment? What do we want to assess in the assignment? And how? With a paper-based assignment, even if we didn’t create the assignment ourselves, we can see the algorithm at work. Though it may not be stated outright, there is no black box hiding the assignment’s structure and logic. But what about a digital assignment, or a standardized test, that is created by a computer according to an algorithm we neither know nor can see at work? When the algorithm ― and its goals ― hide behind proprietary trade secrets, how can we know, let alone critique, the pedagogy at work in our own courses?

This is a problem. But not one without a solution.

The world of ed-tech algorithms may seem like a black box, and in a way, it is. But they have no monopoly on data science. Ed-tech companies use the same statistical and machine-learning tools as other researchers, and if we know those tools, what they’re good at, and to what ends they are typically used, we can “reverse-engineer” those black boxes. We can figure out the algorithms most likely in use, as well as their implicit goals. With that understanding in hand, we can critique those tools and make informed decisions and policies.

To this end, I will be leading two workshops at Digital Pedagogy Lab’s 2016 Institute. The first one ― Coding for Teachers ― is a hands-on introduction to the kinds of things educators can do with code. No previous programming experience is necessary, and the emphasis will be on how to do big things with just a little training. The second workshop ― Demystifying Ed-tech Algorithms ― takes a critical perspective. We’ll explore the kinds of algorithms and machine-learning tools available for ed-tech developers, and then walk through a series of assignments ― both traditional and digital ― to attempt to “reverse-engineer” them: what is the underlying logic, the implicit pedagogical goals, the covert ideological values?

A wholesale rejection of algorithms in education is like a wholesale rejection of technology. It is impractical, uncritical, and risks throwing the baby out with the bathwater. However, with a bit of information and practice, we can discover ― and critique ― the algorithms underlying our own practices, make informed decisions about digital tools in our courses, and offer nuanced suggestions to administrators and policy makers.

I hope to see you there!

Kris Shaffer

Among with Digital Pedagogy Lab

Kris Shaffer is a data scientist and Technical Director, Web Intelligence for Yonder.

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.