> Training included trillions of tokens of Cursor data which capture a wide-range of user interactions with codebases and software tools.
This -- training on work done on hard, real-world tasks -- seems to be how most frontier models are making capability gains these days. In fact people make decent money doing that for data companies like Mercor. However it's also striking that Cursor managed to gather so much of such data.
Turns out Cursor will train on everything you do unless you opt-out, even if you're already paying for it with cash! Are that many people really not opting out?
This is why it seems like a significant concern to me: It's very clear that typical, run-of-the-mill coding has been completely commoditized, so the primary value remaining is either in novel use-cases and applications, or novel technical solutions to hard problems.
Presumably the value for novel use-cases could be captured by building a business around it via the usual moats (distribution, relationships, network effects, first mover advantage, etc.) so the code and techniques do not matter as much.
However novel technical solutions, which are already hard to monetize without building a whole damn business around it, could at least be capitalized on by simply being able to claim credit for it. I'd at least like the option of being "paid in exposure" if I'm not getting paid in cash. But having them "leaked" unwittingly via the training corpus to whosoever happens to prompt the model with the same problem removes even that option.
I know people have been calling out this risk forever, and I don't use any tool that I can't opt-out of training completely, but the scale at which this is happening -- on an ongoing basis, mind you, after training on the data of the whole world, and that too after paying for the product -- is surprising. I'm bullish on the technology but we really should be way more careful handing these AI companies even more of our intellectual crown jewels.
Unless you're working on private data, is it such a problem to let Cursor train on your data?
I personally work mostly on small, unoriginal projects. I don't really mind letting Cursor keep the data for training if it leads to a better product for me.
I often do reverse engineering work.
I find the idea of all the AI tools I use keeping all the (mangled, decompiled, and not even mine in the first place) code I point it at, and then using it in training to be hilarious.
And if it results in the next generation of AIs having more suspicious knowledge of proprietary software internals and better reverse engineering capabilities, then all the better.
It's not a problem for you maybe, but if you run a business providing some niche specific software in some area, having the model suck up all your code and learn from it is pretty harmful because your competitors and customers will be able to vibe code your product after the next model release. You're training a competitor to your business. People are very worried about this which is why companies like Claude don't train on enterprise data, and for lots of companies this exact situation is a deal breaker.
It's the same as any service that makes you opt out of sucking up your data. It's a bad default, and it's not obvious to the average that it's even happening.