I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.
Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. In much of Europe, getting bureaucracy out of the way and encouraging real investment would go a long way. People leave because they can make more money and they want to be surrounded by the best people. People would trade some of that off to stick around their home countries, however if you go to California and talk to folks from e.g. NL or DE working on this stuff, they have a lot to say about innovation and working culture back home.
This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
The EU was build on the principles of collaboration, to overcome the nationalistic impulses. Free trade and free movement of people, no need for everyone to replicate what everyone else is doing. Preventing individual nation states from favoring and supporting their home grown firms over other EU firms is a central legal principle.
But this only works if it is reciprocal. And ideally if the partners are roughly on the same level. When you force developing nations in trade deals to not protect local firms, you are also preventing them from moving up the value chain and locking them into the position of "raw material providers".
When trading with China we know that China has absolutely no qualms supporting its strategic industries with a truck load of subsidies. And it is preventing foreign firms from investing and selling on the domestic market.
For the longest time the US was considered a safe partner though. Sure, plenty of disagreements in the details, but in principle someone whom you can rely on. That idea has been decisevly dismantled over the last 10ish years. The US unilaterally cut of the EMails of EU courts. The US has unilaterally decided that EU partners cannot use Fable/Mythos.
The only reasonable reaction is to make sure that the EU can maintain and create its own critical infrastructure.
Let's not overstate the case. The EU was built to keep France and Germany from getting into another war, and it has been successful at that. All the other "principles" are just window dressing.
Isn't it up to Europeans to define the purpose of the EU? What's your claim to having the right to define it?
I'm not defining anything. I'm just explaining why the key decision makers actually created the EU. The statements made for public consumption shouldn't be taken too seriously.
You're vastly underestimating the economic importance of EU integration.
Is that why the EU economic growth rate is so stagnant?
This "vaguely nationalistic world view around tech" is a direct consequence of the US government ... The EU was build on the principles of collaboration ... But this only works if it is reciprocal.
This sounds great but doesn't really make any sense. What skeptics are saying is there should be a pan-EU effort to build frontier models, rather than one-off toy models built by each country as a box-ticking exercise.
"We built the EU as a powerful supranational organization, so logically the EU's response to a great power challenge in the realm that may well define the 21st century is gonna be to shatter its efforts into 27 useless pieces, because Trump bad" is just absolutely ridiculous and will not lead to anything good. Be the change you want to see, etc.
Look I agree with all you said. The EU should do this but the EU is also structurally bad at large concerted efforts exactly because it is a collaboration mechanism, not a super state.
Whatever topic comes up on this board, if the EU is mentioned people go a bit cray-cray.
If an individual country trains a language model, it's not ambitious enough. If we try to do one for the whole EU they will say it takes too long (you need to get all countries on board, you see). If the EU announces it by executive decision it's a dictatorship and government driven economic intervention, if a EU company does it they'll say it's not good enough.
You can't win with these people. In my opinion, you shouldn't even try to convince them.
>This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
And in response NL should weaponize ASML for example. Then both sides would naturally back down. Specialization is the most efficient way of developing tech civilization.
Whereis everybody building their own mediocre versions would be repeating Russia in its attempt to make its own national messenger - a lot of government money sunk, yet people are still using Telegram/etc.
Of NL weaponizes ASML they will just take their IP and leave NL and then they have nothing. They have already pushed a lot of things out of NL and should invest in leveraging the position without weaponizing it.
If ASML is just IP then everyone would have already copied it. They have an extremely deep and high tech supply chain that is nearly impossible to copy.
ASML wiki for those like me who were confused https://en.wikipedia.org/wiki/ASML
(despite the name, i am not dutch)
I am quite interested to hear how you would pronounce your name, dike-ster-house (assuming you live in the US)?
> dike-ster-house
yep, basically. 3rd one down https://www.howtopronounce.com/dijksterhuis
the story behind it: i was looking for a name to write music under. and i was reading academic papers about subliminal messaging in music and stuff. i came across a paper where they seemed to be absolutely ripping into this "dijksterhuis" guy's previous paper. felt bad for him. felt like he was an underdog. so i ran with that name.
> (despite the name, i am not dutch)
Some close or far ancestor was though. :'-)
[dead]
Why would it have to be mediocre?
Both Netherlands and Sweden produce highly competent researchers. Per capita on par with any other location on the planet, including California.
This will be good for Europe as a whole and also the planet.
>and also the planet.
What a glorious day for Canada and, therefore, the world.
I don’t think it’s good that the USA has so much influence due to their dominance in information technology.
It was only nominally democratic, now it’s a total shit show.
>total shit show
Are there people legitimately brain-rotted such that they believe stuff like this?
Outside of the constant flow of "hey guys, look at the latest dumb thing Trump did" coming from entertainment outfits posing as news, what are we even talking about? Life goes on as normal for the Americans. You are way too online.
Yes, the current administration has been in power for less than two years so obviously the “day to day life” hasn’t changed much for the average American (except everything has gotten more expensive).
The US is no longer looked at as a superpower, it just surrendered to a regional power and will pay $300B USD in tribute.
But for Trump and his cronies life certainly has improved.
But the damage is done and we are witnessing the decline and destruction of the worlds greatest Empire of all time.
Nitpick: it’s 6 years.
The Republican Party looked very different during Trumps first term
The us stopped being looked at as a superpower as a brain rotted Alzheimer’s patients president rose to power on the campaign “don’t let Trump win” and then did little productive except stoke wars and pardon his son and his friends from crimes.
I’m assuming you’re talking about Biden, and no, not really. Still considered a superpower.
Up until the second Trump presidency, most of us outside the USA had basically the same opinion as we had before. Pretty messed up politics , but strong economy and strong military.
The first minor dip was with Trumps first presidency. But shit went downhill fast with his second, unfettered reign when MAGA took total control over the Republican Party.
Now it is 110% clear that the American Dream is dead and buried, except for the well connected wealthy, and your military was state of the art, two decades ago.
The 21st century belongs to China and Europe.
Producing local researchers isn’t enough. If you want a competitive model you need to bring the best researchers from all over the world. That means lots of investment that hasn’t been common in Europe.
But it's a start
Sure, but there is still a more dense talent pool with SV companies, and training frontier models requires massive amounts of capital. Are Netherlands and Sweden prepared to invest 10s of billions?
My impression is the talent pool for core model development in SV isn't huge and is already grabbed by the usual suspects. Everyone else sloshing in the vague broader AIsphere are application bros whose value in the supply chain is minimal.
Honestly I have a strong suspicion that “the talent pool” will shift towards Europe and China and I believe it’s already happening.
And yes, I do believe Europe will invest in this technology.
You missed the part where they got 13 million EUR , 3 years ago! and dont have a release yet....Definitely doing it at European pace...they will have release two when AGI is around...
[dead]
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
It’s a response to the actually-nationalist practice of the United States. I can understand why it might feel different from California, but things are a bit scary over here right now.
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
It's a direct response to the MAGA / America First attitude of the American electorate.
Collectively Europe has talent and money to support it. Whole point of sovreignity around it is to have inhouse the talent and capacity to have essential resources even if it can be have for cheap from other nations/alliances.
That is why people lament at idea of being completely dependent on Russian gas, US tech or Chinese manufacturing.
The EU should absolutely be running frontier labs producing frontier models.
The fact they aren’t points to the moribund nature of things in the EU tech space. Both China and America are doing this although with very different approaches.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment.
As for nationalism, like it or not, this is a govt sponsored effort, and governments and universities are funded by the public of specific nations
this exactly. its not nationalistic at all to support your local development. this project has different goals than people perceive and their comments are all missing the plank ;)
[dead]
I don't see it as a nationalist world view around tech in Europe. There is certainly a push for "sovereignty", but that mostly means sovereign from the US. But European countries are actively working together on this.
> Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
Yeah, why is Anthropic making their own mediocre models, if they could just pool their resources with OpenAI, that makes no sense.
Hopefully you understand that there's an obvious selection bias you encounter when talking to folks from NL or DE in California. Ofc the people that moved will think there's a good reason for them moving. But people who live here have good reasons for staying here.
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
Guess which country blocked access to a SOTA model based on national security bullshit.
Okay fair point
> pooling resources
Your argument would suggest the EU developing a European model would be a better direction. A heavy-weight competitor would help advance the field after all.
> getting bureaucracy out of the way and encouraging real investment
I don't think this is really about regulation - it's about network effects. The only way to compete with strong network-effects is to create your own.
> This vaguely-nationalist world view around tech
Nationalism breeds nationalism and it is the fundamental reason European states feel the need to build their own expertise. Can you imagine if your country was subject to the whims of an aspiring dictator?
Users don't need to build wheels. Wheel builders need to first learn to build some mediocre wheels, then they can build better ones.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
if something’s strategically important for safety it’s wise to have competency in it.
Tech sovereignty is a response to American imperialism. It has not gone unnoticed that the White House has summoned the tech bros and weaponised tech.
Surely this just uses state funds to train a pipeline of people to be immediately poached by frontier labs?
Great, now they will get expertise and then get hired by openai and move
There is something to be said for this "most cheapest" approach, there is also something to be said for making models that are entirely ethically sourced:
1. Free of controversy like unlicensed training materials
2. Free of exploitative rlfh loops by people in low-wages countries
3. The leasons learned (and published) from going through the entire training process on "European" hardware: "AI factories" (the term for Slurm HPC/HTC systems with lots of heavy GPU nodes, heavily subsidized by our government [0])
1 and 2 are strong counter-LLM arguments at the moment, and hold back some groups of potential users. Another is energy/water use, so going for maximum green energy would be a nice boon as well. 3 is something I consider to be highly useful for our European identity and "way of the ninja" (for you Naruto fans out there).
[0 https://hpc-portal.eu/funding-opportunities]
one would also hope there'll be less pressure to "make line go up", i.e. not having to do attention-engineering via deliberate sycophancy to trap individuals into using it more and more and more and more and more and more.
but in general, yes, as someone who is vehemently anti-ai GPT-NL has piqued my interest specifically because of the ethical protections / measures they're talking about. question is whether they stick to it.
If open frontier models start closing up and states start more export controls on AI services and hardware, it might be good to ensure the supply chain is there to reproduce the SotA, or even a couple generations behind it.
Sounds like calling those model "open source" did its wicked job. You can not take an open weight model and build a next-generation model using that as a foundation. Once those companies decide it's no longer in their interest to release new open weights everything you've created this way becomes a pile of rapidly deprecating legacy.
OP probably means using the existing open weights as a base for further homegrown development and research, not that the homegrown models are always updated based on whatever US or China are doing in the moment
TNO doesn't serve nor represent global interest and hence does not care about global progress. It exists to enhance knowledge on things within Dutch society primarily with some ripple effect outward to EU because they have interests within the EU.
Its purpose is not to become some kind of OpenAI or global foundation offering services/tools on that scale.
There is a lot of critisism on this project, not invalid, but mostly based in lack of understanding of what the goals are of the organization as well as the people building the thing.
The people building it, are well aware of how it will be less capable than other LLMs on a general reasoning aspect, not only due to having actually purchased _all_ licensed data that has been used as inputs. Not being a multi billion dollar corporation, this means having very little data and should be an obvious signal to observers that it has not the goal to outdo other models.
In my opinion (personal) its a project that has a learning and demonstration value that is not 'look how well our model performs against others', but still offers value.
Do we know for sure how much national corpus of knowledge (like dutch) goes into these "global" models and how that affects "localized" model biases? What's wrong with specialized models?
Same reasons why every country, or close allies, build their own tanks or space program. You want to keep some level of capability within your control. Compared to weapon programs, AI research is very cheap.
And what happens once the "solid baselines" become unavailable for a reason or the other?
You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
> You keep building on the last available version?
yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull.
Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.
> But good to have researchers and professors "in-house".
I'm not in this field, but I think we already have them. Probably the main difference is that we have most or all of them in academia and next to none insode private companies. But we do have them, and they could start working for private companies if the market moves in that direction in the EU as well
Unfortunately in this game first principles requires massive resources, not "some". Building in-house on top of existing open weights is a good way to bootstrap this process, especially since there's nothing inherently magical or particularly expertise-heavy when it comes to weights themselves
Seems like you don’t understand.
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
It seems like you don't understand. For fine tuning, it's cheaper to fine tune an existing model. For massive changes, it's better to retrain from scratch. Otherwise, model will UNLEARN a lot first, and then you will train about twice longer to the same result.
https://en.wikipedia.org/wiki/Catastrophic_interference
Today, you keep seeing "sovereign" LMs that are subject to the sovereignty of some human-led state. Tomorrow, the "sovereign" LMs will be called that for a completely different reason.
Their legality is very questionable given all the likely copyright infringement going on, and a state can't really ignore that.
States are the things which can ignore that, and I'm pretty sure US and China already do. No state is going to respect copyright if they think its future is at stake, and apparently even Netherlands thinks the future is at stake.
(Of course states can ignore copyright in a legally polite manner, such as asserting that training on all published material in the National Library is fair game)
Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
[1] https://www.taiwannews.com.tw/news/6245677
[2] https://www.theguardian.com/technology/2024/apr/16/techscape...
The Chinese models are almost certainly taught to comply with "Chinese values" in the RLHF step, not from filtering the training data. There may be a few things which are too radioactive to be allowed even in the training material - but that's more likely to be things like child abuse images for a visual model, things non-Chinese values also have an issue with.
I'm pretty sure no county taking a stab at making their own model for sovereignty purposes will let "proper licensing" stand in their way.
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best
Current frontier models (closed and open) are already really good at small languages too. I use them in Finnish sometimes, and the language is immaculate. They underestand even somewhat obscure dialects. Multilinguality seems to be a mostly solved problem.
This already exists https://eurollm.io/
How do people not know about it and keep making stuff from scratch?
I did not know about EuroLLM. I had a look to the paper (https://arxiv.org/abs/2602.05879) describing it:
Specifically, we discard documents shorter than 200 characters (Xue et al., 2021a), and any page containing the phrase “lorem ipsum,” the word “javascript,” or curly brackets (Raffel et al., 2023)....
It is quite surprising/funny to see all documents with javascript removed.
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
One example is not the same as "most LLMs". My experience is the same with most LLMs. Especially the smaller ones are English oriented (probably makes sense given the size constraints).
It really doesn't matter if the model sucks and doesn't perform well. Given the funding amount and their lofty ambitions, it seems very unlikely they will be able to pull it off properly.
Yeah China and US models have baises but so will any model. The biases do not get in the way of the product though. You don't open those models just to ask for what happened in Taianaman square or if Taiwan is a state. You dont ask ChatGPT to generate CASM. But they are very good at the tasks you actually expect from a LLM. If you fail at that, nobody will use your model no matter how "ethically sourced" a colonizer-based entity like Europe made it.
> no matter how "ethically sourced" a colonizer-based entity like Europe made it
The attempt is laughable, buy every country should at least try to keep up with frontier technology, even if they fail massively or are massively underfunded.
On the other hand, it's arguably wasteful for an incompetent govt to do something like this, since the money will almost certainly not be well spent. It will just go to people good with MS Word. That's the likely failure mode for such NL innovation projects. The actual solution is a culture shift, but that is much harder if not impossible to pull off and requires decades. But we (NL people and govt) should work towards that. Most likely all these govt led innovation attempts are a sad waste of tax money.
The culture shift that has generated this is the same one that causes the other story on HN this morning about xAIs gas generators being a national security issue. Ie one towards corruption graft and the public ill.
I don’t want Europe to model itself on the US, whatever the economic gain. Hopefully we are large enough to find a third way between China and the US.
There is something north of 8% OCR error rates.. that will hurt model quality!
Uh, some would say it's easy to determine what input went into the training for kimi and qwen.. since they were caught stealing it from American labs. Some cultural cliches may never change.
It's well-known that all commercial models are based on stolen content. That doesn't mean there is no filtering/censoring, just that the censoring likely depends on where it's happening…
> It's well-known that all commercial models are based on stolen content.
Does that mean that Chinese models are the "Robin Hood"s of the AI era?
> since they were caught stealing it from American labs. Some cultural cliches may never change.
Has a formal lawsuit been brought to bear? Given, Anthropic & OpenAI are being dragged through courts for copyright violation (or stealing, as you'd call it, if the companies involved were culturally Chinese) by newspapers, publishing houses etc; one'd think they'd pass on some of that medicine to Alibaba, which does have business entities registered in the US.
> since they were caught stealing it from American labs
...and "good guys" the American labs were caught stealing from authors all over the world[1].
[1]: www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material
.... Anthropic began buying books in bulk, tearing off the bindings and scanning each page before feeding the digitized versions into its AI model, according to court documents.
Wow. This image of Anthropic employees ripping books apart to use them to train models is a powerful one, seems like an inflection point in the history of information.
>> Some cultural cliches may never change.
Let’s just gloss over the monstrous amount of copyrighted and pirated material the American labs trained on. China bad. American good. Some cultural cliches never change.
How about, both China and the US bad, Europe at least somewhat decent because we lack the financial incentives to behave like utter arseholes?