Anyone has an insight into how much money labs are putting into benchmarks?
Just Arg-AGI-3 is quoted above 20K USD and footnote says average of 5 runs (!!). Likely just a drop in the bucket to the training budget but still..
Anyone has an insight into how much money labs are putting into benchmarks?
Just Arg-AGI-3 is quoted above 20K USD and footnote says average of 5 runs (!!). Likely just a drop in the bucket to the training budget but still..
20k is small potatoes for the marketing impact.