There was a post just a few hours ago on the frontpage asking not to use AI for writing [0]. I copied the content and pasted it on multiple "AI detection" tools. It scored from 0% and to up to 80%. This is not gonna cut it. As someone who used LLMs to "improve" my writing, after a while, no matter the prompt, you will find the exact same patterns. "Here's the kicker" or "here is the most disturbing part" those expressions and many more come up no matter how your engineer the prompt. But here's the kicker, real people also use these expressions, just at a lesser rate.
Detection is not what is going to solve the problem. We need to go back and reevaluate why we are asking students to write in the first place. And how we can still achieve the goal of teaching even when these modern tools are one click away.
I see what you did there.
I think we'll still need ways to detect copy-pasted zero-shot content that's generated by LLMs, for the same reasons that teachers needed ways to detect plagiarism. Kids, students, and interns [1] "cheat" for various different reasons [2], and we want to be able to detect lazy infractions early enough so that we can correct their behavior.
This leads to three outcomes:
1. Those that never really meant to cheat will learn how to do things properly.
2. Those that cheated out of laziness will begrudgingly need to weigh their options, at which point doing things properly may be less effort.
3. Those that meant to cheat will have to invest (much) more effort, and run the risk of being kicked-out if they're rediscovered.
[1] But also employees, employers, government officials, etc.
[2] There could be some relatively benign reasons. For example, they could: not know how to quote/reference others properly; think it's OK because "everyone does it" or they don't care about subjects that involve writing; do it "just this once" out of procrastination; and similar.
The whole argument is "A written response to an answer is no longer a valid form of testing for knowledge"
We don't need better detection. We need better ways to measure one's grasp of a concept. When calculators were integrated into education, the focus shifted from working the problem out, to using the correct formulas and using the calculator effectively. Sure, elementary classes will force you to 'show your work', but that's to build the foundation to build on, I believe.
We don't need to detect plagiarism if we're asking students verbal answers, for example
Yes, and it's not a sound argument, IMO.
"Grasping concepts" is not the only learning goal in schools or universities. Many classes - including within STEM programmes - want to teach students about writing, argumentation, researching, critical analysis, dealing with feedback, etc.
Oral exams can be more stressful, depending on the student. They also don't check for the student's writing or researching ability. They can be gamed with rhetorical skills. Grading of oral exams tends to be more opaque. And so on.
Then there's the issues I explained above, where you don't want to inadvertently reward cheating. Even if you don't care about the cheaters, you should try your best to detect and reward real effort. Otherwise, it'd be stupid not to cheat and use the class for free credits, at which point, from an educational POV, it's a useless class.
So, all in all, there are still very good reasons for doing take-home written responses and essays, and good reasons for wanting to detect cheating or plagiarism.
One major missing piece in using AIs is self-expression. The idea of writing is to express your own ideas, to put yourself on the page; someone writing for you, AI or biological, can't do that. There are far too many nuances and subtleties.
I suspect many students write to pass the class, and AI can do that. Perhaps the problem is the incentives to write that way.