These things are moving so quickly, but I teach a 2nd year combinatorics course, and about 3 months ago I tried th latest chatGPT and Deepseek -- they could answer very standard questions, but were wrong for more advanced questions, but often in quite subtle ways. I actually set a piece of homework "marking" chatGPT, which went well and students seemed to enjoy!
Super good idea!!
Luc Julia (one of the main Siri's creators) describe a very similar exercice in this interview [0](It's in french, although the au translation isn't too bad)
The gist of it, is that he describes this exercice he does with his students, where they ask chatgpt about Victor Hugo's biography, and then proceed to spot the errors made by Chatgtp.
This setup is simple, but there are very interesting mechanisms in place. The student get to learn about challenging facts, do fact checking, cross reference, etc. While also asserting the reference figure of the teacher, with the knowledge to take down chat gpt.
Well done :)
Edit: adding link
[0] https://youtube.com/shorts/SlyUvvbzRPc?si=2Fv-KIgls-uxr_3z
this is amazing strategy
forgot the link :)
Arf seems I'm one of those :).. thanks for the heads up!
That’s a great idea to both teach the subject and AI skepticism.
> I actually set a piece of homework "marking" chatGPT, which went well and students seemed to enjoy!
This. This should be done everywhere. It is the best way to let students see first hand that LLM output is useful, but can be (and often is) wrong.
If people really understands that, everything will be better.
Very clever and approachable, and I've been unintentionally giving myself that exercise for awhile now. Who knows how long it will remain viable, though.
When you say the latest chatGPT, do you mean o3?
Whatever was best on a paid account 3 months ago. I was quite disappointed to be honest, based on what I had been hearing.
I think by default ChatGPT will choose 4o for you. So unless you actually chose o3 you haven’t used the best model.
that's a cool assignment!