Hey Andrej, thanks for the effort. It's highly appreciated!
In your video regarding your own use of LLMs[1] you mention https://arena.ai/leaderboard and https://labs.scale.com/leaderboard to keep track of the status quo. That video is from February 2025 and https://artificialanalysis.ai has gained quite a lot of popularity but also earned criticism repeatedly due to updating the benchmarks right before a new model release. Would you still recommend the arena and scale leaderboards? And if you don't mind, what's your opinion wrt artificialanalysis?
No i don't recommend those anymore. Currently I recommend Epoch's ECI. AA was kind of bad but recent changes are trending a little bit better. I think benchmarks are in a really bad state right now otherwise, I basically don't trust them, good related reading https://epoch.ai/data/benchmark-reviews-documentation/includ...