This piece originally appeared in the Boston Globe on September 23, 2026.
Teddy Svoronos is a senior lecturer of public policy at the Harvard Kennedy School. His book “Course Corrections: Teaching (and Learning) What Matters in the Age of AI” is forthcoming.
I teach statistics to students of public policy. My goal is to get them to understand, use, and interpret statistics and ask for empirical evidence in order to create good policy. But now I realize that I need to rethink how I do that.
Let’s say a policymaker wants to assess whether a government-funded job training program is actually helping people find work. Answering that correctly requires the technical skills I have spent most of my career teaching: how to take a group of people in the program, carefully select a comparable group of people with similar attributes such as income, demographics, and education level, and compare their outcomes.
Getting the math right is important (and harder than it sounds!), but it’s only one step. Even if you correctly do the statistical matching that I just described, the two groups may still be different in important ways, like their motivation to find a new job. You also have to choose an outcome and time frame that makes sense. Should we see how much money they make a year after the job training program? Is one year enough to measure the impact or too long? Or there may be important details about that particular job training program that would make it fail in a different district or with a different group of people.
It’s a messy process, rife with hard math, judgment calls, and an intimate knowledge of the people whose lives you’re trying to improve. These are the skills I train students to practice.
A few weeks ago, I asked the latest version of Claude to complete one of my assignments. I recommend every professor in every discipline do this, because the results were illuminating in ways I didn’t expect.
To start with, Claude completed the assignment perfectly. Of course, this doesn’t mean students shouldn’t learn the skills themselves; after all, children learn arithmetic even though we have calculators, because they develop “number sense” in the process. But what Claude did next gave me real pause.
After measuring the effect of a job training program as I asked, it went much further. With a little nudge, it investigated assumptions (is the effect we measured just the product of quirks in the data?), tried alternative methods (do other statistical methods do a better job?), and wrote a full report about the shortcomings of its analysis (what real-world conditions would these methods not hold up against?). In short, it had stopped doing homework and started doing real work — the kind of work that I hope my best students will do in their careers.
This work blew me away, but there was a problem: Claude was making choices that I would not have made, and I was not training my students on how to weigh these choices. I had spent so much time teaching students the mechanics of the method itself (which Claude obviously knew), that I had failed to help them develop the judgment required to make sense of these AI outputs.
For example, Claude spent 20 minutes re-creating its analysis in a way that appeared reliable and definitive. But upon checking its work, I found that much of the time it was trying out statistical models that were very similar to the original one it used, making the elaborate report that it wrote much less convincing than it might seem. This may sound like a minor detail, but it is exactly the kind of choice that can mislead a policymaker into thinking they’ve had a real impact when it was, in reality, statistical noise.
This experience forced me to confront a hard truth as a teacher of statistics. In short, now I need to spend less time on fundamental procedures that I used to consider essential, and more on the higher-level intuition that has taken me years to build. To add to that complexity, I have to help my students learn how to use AI effectively, even though doing so could undermine the very judgment I aim to develop.
In June, I convened a group of statistics faculty from a variety of institutions to wrestle with a question that Ji Son from Cal State Los Angeles put succinctly: When AI can produce the work, what kinds of humans do we need?
At least for now, the answer appears to be that humans are still very important in statistics: to understand what questions to ask, to think through the implications of different findings, and to translate the results of imperfect analyses to the real world. But when my colleagues and I were students, learning these skills required us to do things that today’s analysts simply do not need to do. We had to fuss over code that wouldn’t run, fix messy datasets, calculate and recalculate quantities — steps that are now trivially easy for AI models. So how can we teach the higher-level judgment that today feels like the only thing that matters?
The group of us disagreed on the best way to do it. Some believed strongly that the repetitive, frustrating work of failed analyses was the only way to learn what a good analysis looks like and that learning to code was still crucial. Others believed that the skills to critique statistical approaches are distinct enough from conducting an analysis that they should be taught explicitly and that our students should instead learn how to create AI systems where teams of AI agents critique one another’s code.
But regardless of where we found consensus and where we did not, consider how distinct these conversations are from today’s discourse about AI in higher ed. Whether or not students are cheating on their homework is unrelated to whether their homework is building the right mindsets to begin with. To do right by our students, we need to interrogate not only how we teach but what we teach.
If you are a teacher in higher ed, you may find yourself at one of two extremes as you read this.
One is the sense that you shouldn’t change anything about how you teach. After all, the foundations of our respective disciplines are centuries or millennia old, and the mindsets that we help our students to build will outlast the proclamations of tech CEOs. My own view is that some courses absolutely should stay the same, depending on their goals. If the explicit objective of a course is for students to be able to appreciate the act of reading poetry, I don’t think they need to learn how to read an AI’s analysis of a poem. Similarly, despite being relatively tech-forward in my approach to teaching, I still maintain a strict no-tech policy in the classroom when it’s time to focus on thinking through problems.
But if you believe that part of the reason you teach mindsets and skills is so that your students can use them in the real world, you are setting them up to fail when they enter careers where facility with AI is expected. A student recently pointed out this risk to me in my own teaching. I use AI quite a bit in my course, but I do it by giving students access to guardrailed chatbots that have been designed to facilitate learning and not give away answers. Although my students are generally appreciative of this, one expressed concern that they weren’t actually learning how to use commercial AI tools “on the job,” like producing hallucination-free reports or orchestrating complex analyses that used to require a team of analysts.
The other end of the spectrum is to decide that the single most important thing is to teach students how to use AI effectively. If AI can do something fluently, it is not worth teaching students how to do it. We no longer calculate with abacuses, type on typewriters, or write in assembly code, and clinging to how professors learned to do things reflects a lack of imagination about where things are going.
I worry that I may be too precious about the approaches that were important to my own growth, but there’s a frustrating Catch-22 at play. There is emerging evidence that using these autonomous AI tools while building new skills can stifle the development of the very judgment required to use them effectively, just as watching someone drive doesn’t do much to help you yourself learn how to drive. (Recent evidence of this comes not from an anti-AI organization but from Anthropic itself.) This means that when and how AI is introduced into assignments can prevent, or even undo, the development of essential mindsets. Once again, such decisions need to be made carefully, and depend largely on who you are teaching, when you are teaching them, and for what ends.
All of these considerations are overwhelming, particularly alongside concerns about cheating that won’t just go away. But these are exactly the questions that I believe will make the case for higher education’s relevance in a changing world. Our roles as educators and mentors demand that we embrace these questions not as a side issue but as the work itself.