People who are already using LLMs to drive Clojure REPLs say it works well (see #ai-driven-development on the Clojure Slack). I don't think forgetting which cells have been run is a problem with the described set up.