Is AI Grading Effective?

Last year, my daughter took the state-mandated test required of all students in her grade, but, instead of a team of teachers and other professionals grading it, it was done by AI grading. 

I don’t love this. 

I’ve worked in education long enough and have worked with AI long enough to know that AI does some things really well, but it simply cannot do other things (yet). 

My initial concern, of course, is that novel, creative writing will be penalized and rote, average writing will be rewarded. It’s not a great outcome, and it will generally prompt teachers to teach and reward average writing. 

But there’s another concern. Why is my kid providing content to help tech companies refine large language models for their generative AI? Just this year, Reddit sold access to its posts to an AI company for $60M.

“AI companies like OpenAI have so far been training their large language models by scraping data from the web without seeking permission from either websites or users.” 

Is this happening at the state level, with students required by the state to provide such content? It is, to some extent, though the financial details are unclear. Now, Reddit has about 73M active daily users, and even the states with the largest public school systems (California with about 6M, Texas with 5.8M, and Florida with about 3M) provide significantly less content. 

Still, one wonders about the advisability of machine learning as a way to evaluate student writing. Is it producing learning gains or stifling them? Is there labor involved? If so, how is it being compensated? Are there copyright issues?

“Students typically will own the copyright to works created as a requirement of their coursework, degree, or certificate program.” A school, “however, retains the right to use student works for pedagogical, scholarly, and administrative purposes.” 

What does this mean in the context of AI grading papers?

In short, parents are right to be skeptical of this practice, and it’s just more evidence to suggest that, while AI certainly has an important role to play in education, it’s still in the early stages of adoption, and we’re not entirely clear as to the ramifications of its use – not just educationally, but also legally, socially, and societally.