Yeah, a bunch of people started going on witch hunts against the people where were obviously using AI (eg ~30 second solve times) and also anyone who had "AI" anywhere in their bio.
IMO the levels it got to was wildly out of proportion, even if these people were cheating (say what you will about AI, but if the rules say not to use it and you do: you're a cheater) but maybe I would have felt differently if the timezones meant I could take part, rather than waking up to drama.
Just to not feed the witch hunt further, note that human 30 second solve times can be entirely possible for the easiest puzzles, with enough experience, practice, and a bit of risk-taking; see e.g. https://adventofcode.com/2021/leaderboard/day/1 part 1. But the 4 second solution times we saw last year are not, no matter how you look at it.
But yes, you would have people openly share repositories for automatically ingesting the puzzle text, solving the puzzles, and submitting the results the moment the puzzles opened, leading to inhuman solution times. So, despite the rules stating that you can't do that, the result is that whenever the puzzles were easy enough for an LLM to solve them with high probability – and most of them are – the leaderboards would be overrun with such solutions.
In 2023, the LLMs would still struggle enough that the overall leaderboard (taking all 25 ⋅ 2 puzzles into account) would still be dominated by ordinary people (many of them recording their solutions), in 2024 that was no longer the case. Personally I would go from being able to top 100 regularly, to almost never being able to. I'm going to miss the thrill, and think it's a bit saddening that we can't have nice things, but also ultimately think that getting rid of it is the best option.
Yeah, a bunch of people started going on witch hunts against the people where were obviously using AI (eg ~30 second solve times) and also anyone who had "AI" anywhere in their bio.
IMO the levels it got to was wildly out of proportion, even if these people were cheating (say what you will about AI, but if the rules say not to use it and you do: you're a cheater) but maybe I would have felt differently if the timezones meant I could take part, rather than waking up to drama.
Just to not feed the witch hunt further, note that human 30 second solve times can be entirely possible for the easiest puzzles, with enough experience, practice, and a bit of risk-taking; see e.g. https://adventofcode.com/2021/leaderboard/day/1 part 1. But the 4 second solution times we saw last year are not, no matter how you look at it.
sounds like the competition had challenges even before AI or any rules about AI or not.
I never followed this challenge aside from doing a few 10 years ago in college.
If speed to problem completion matters though, then question release time will be brutal for some.
There's some good discussion here: https://old.reddit.com/r/adventofcode/comments/1h9cub8/discu...
But yes, you would have people openly share repositories for automatically ingesting the puzzle text, solving the puzzles, and submitting the results the moment the puzzles opened, leading to inhuman solution times. So, despite the rules stating that you can't do that, the result is that whenever the puzzles were easy enough for an LLM to solve them with high probability – and most of them are – the leaderboards would be overrun with such solutions.
In 2023, the LLMs would still struggle enough that the overall leaderboard (taking all 25 ⋅ 2 puzzles into account) would still be dominated by ordinary people (many of them recording their solutions), in 2024 that was no longer the case. Personally I would go from being able to top 100 regularly, to almost never being able to. I'm going to miss the thrill, and think it's a bit saddening that we can't have nice things, but also ultimately think that getting rid of it is the best option.
[dead]