I think they should just have to properly explain how AI tends to make things up when it doesn’t know, and that it’s good for coming up with ideas or suggesting directions for research but that you shouldn’t rely on it, because currently their advertising makes you think you can rely on it

The “AI can make mistakes” kind of disclaimers they hide in the corner don’t really cut it

It's not even in the corner, it's completely hidden by default. You have to click on "show more" to expand the AI barf to see it.