I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
There's an absolutely massive cultural and behavioural bias in those models. Models will suggest things like "go to the hospital" for things that require GP appointments, "just drive three hours" while it's faster to go places by train, and so on. They will do it in anglicised Dutch (compound words split, English-like grammar structures) that's perfectly understandable, but the cultural bias is there if you know to look for it.
Furthermore, the expertise in designing and training these models is valuable as well. The existing models are good as a starting point in terms of learning from previous mistakes, but we should not just let a handful of American and Chinese people keep the knowledge and expertise.
One problem with this particular project, though, is that copyright has been enforced for Dutch LLM training before, and the AI industry cannot exist without massive scale piracy, the likes of which has never been seen before. A lot of Dutch training material exists in pirated books that AI companies in countries that do not care about copyright have access to, but are exempted from the training set here. The impact of enforcing copyright on an AI model will be quite interesting to see.
It is not about the country but the language. Most llms have poor or no support for Dutch.
Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages.
Lots of bias towards English sentence structure, idioms, etiquette, etc.
There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Can you provide some examples of these use cases?
Support bots and question answering with access to sensitive pii?
Yes, but what's the point of a support bot that writes good Dutch when it can't follow instructions, doesn't understand the questions or can't solve problems? I might be wrong, but I don't think atm these models have the cognitive ability to perform any task in a satisfactory manner.
As for accessing pii, I imagine the value here is in the fact they're local, which has nothing to do with the "sovereignty" of these models. If anything, a model is more likely to be tricked by a malicious prompt the farther it is from the sota.
I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
To be fair. There is a security concern angle: even open-source models could be trained as sleeper agents that act adversarially (for example, adding backdoors) when used in specific national companies in specific settings. This is very difficult to detect or void, so if you want to be sure 100% that this isn't the case, you have to train your own model from scratch.
An LLM is an encoding of a culture, a way of viewing the world.
They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.
In many ways, they are ideology made code.
If we leave building them to the US and China, only their way of seeing things will be digitized.
I don't like the idea of that.
Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
Chinese models aren't censored as much as you think, you can download the model and run it somewhere else and they will happily tell you about Tiananmen Square. Or heck, ask DeepSeek via Openrouter, it will do the same.
The censorship works kind of like with Fabel, it kicks in before the model responds.
Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?
Maybe the Dutch really really want an LLM that tells them the truth as straight as possible no matter how harsh - that might be tricky
Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?
I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?
Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.
But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.
I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.
They could apply the Polder Model of consensus decision making with a mixture of experts.
https://en.wikipedia.org/wiki/Polder_model
Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:
"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."
https://www.tomshardware.com/tech-industry/artificial-intell...
Calling a bunch of LLMs a "council" is just rebranding well known ensemble methods with shrill marketing hype, nothing original or out of the ordinary. Mixture of experts and every other idea in his stack has a literature older than PewDiePie's career.
Yet another attention craving influencer who shilled crypto scams during the crypto bubble and is now marketing "AI councils" during the AI bubble.
He's not a serious or honest person, AI is just what he pivoted to after crypto. That's not innovation; it's attaching trendy branding to ideas that were already old when Marvin Minsky wrote The Society of Mind in 1986, three years before PewDiePie was zero years old in 1989.
The only thing PewDiePie's brought to the table is cleverly optimized YouTube thumbnails designed to attract clicks. The architecture is decades old; only his marketing and shilling is state of the art.
Lighten up dude.
This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
Oh, it's all fine with cultural context here -- we don't even dub English language movies here because we are that cheap
[flagged]
>Countries should want control over _where_ the compute is happening
Yeah but Europe doesn't build any computer hardware, and EU Green eco-communists and NIMBVYs don't want to have data centers built in their backyard, so the only way left for EU consultancies to milk taxpayer money for the AI bubble, is shipping a sovereign AI model for each country/language.
Watch out US tech sector, we're coming for you. Feel our wrath.
Have you heard of ASML? NXP?
Ignorant comment
Please don't move the goalposts. What computer parts does ASML or NXP make?
ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically.
And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric parts like ASICS, FPGAs, CPUs or GPUs.
So not only are you the ignorant one here, but you also have the audacity to insult others with so much confidence.
@dwa3592 below. Firstly, why are you moving the goalposts in bad faith again just to stir an argument? What does that have to do with my original comment?
And secondly, there's other lithography machines out there, not just ASML.
And thirdly, the IP Nvidia, AMD, etc develop to etch on silicone via ASML machines makes them more valuable than ASML.
Fourthly, repeating my "let that sink in" phrase is just childish and low-IQ trolling, unworthy of this platform.
Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
That was a big strategic mistake. In the US case it was borne of the mistaken belief that we shared values and were partners.
But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting, but I think the success of Ukraine is one reason to be optimistic that current adversity might actually leave us better off in the long run.
Corrupt countries with broken legal systems tend not to fare that well in the longer run.
>Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
NO, it's hosed because it's not competitive and slept on the wheel at several key digital economical revolutions plus sleeping at the wheel at preventing obvious geopolitical issues (gas dependence to Russia, losing auto industry to China, losing semiconductor industry, losing SW industry, etc).
You can't be an economic leader if you keep losing on all fronts and only be a leader at how much welfare your spending.
> the mistaken belief we shared values and were partners.
We do share. US and a lot of latin america is mostly European immigrants and European culture, making our cultures are much more similar than the african and middle eastern ones the EU has been importing and adopting. Where we differ is that US still has free speech and isn't devolving into a stasi police state that arrests people for Tweets that the political establishment find uncomfortable.
> might actually leave us better off in the long run.
How? EU's economy has been pretty much stagnant since 2019 when you account for inflation loss.
> But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting,
How? Where is Europe's Nvidia and AMD? Where is Europe's TSMC? ASML can't feed an entire continent.
>>ASML only makes the lithography machines
Woah! only lithography machines???? it is literally impossible to make any device capable of running anything close to AI without ASML. Let that sink in.
Funnily ASML owes its current success partly to US funded research (straight from Wikipedia):
> Two years later, it joined a consortium, which included Intel and two other U.S. chipmakers, in order to exploit fundamental research conducted by the US Department of Energy. Because the Cooperative Research and Development Agreement (CRADA) it operates under is funded by the US government, licensing must be approved by Congress.[12]
Why are you acting childish and petty? I said EU hasn't got AI compute manufacturing(aka no equivalent IP to Nvidia and AMD and no equivalent to TSMC or Samsung fabs), not that it doesn't have lithography machines manufacturing.
Surely you understand that while you can have the latter, you can also lack the former.
In a recent podcast, it was summarized as:
ASM (International) makes machines that add material to a silicon wafer (deposition).
ASML makes machines that remove material from said wafers (lithography, etching)
(I was a bit surprised that's not combined in 1 machine. But let's move on)
Then Besi makes machines to stack / interconnect / package those ICs into a package. I'm assuming pick & place machines are other companies' turf.
The above are all Dutch companies, operating a pretty important section of the tech stack.
Iirc there were (& probably still are) some IC fabs in Europe, but mostly older nodes (like useful for microcontrollers used by car manufacturers. Wikipedia has a list). So for SOTA smartphone SoCs it's off to Taiwan (TSMC), South Korea (Samsung) or China (who makes everything, including smartphones & the chips going in there).
So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check. Money? This is a rich continent.
What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
But now the important thing: several of such initiatives are starting to bear fruit, and b) confidence that EU can do such things, is growing.
As for bureaucracy / red tape... sigh... (won't be fixed any time soon)
>In a recent podcast, it was summarized as: ....
Yes, all true, and all things I didn't disagree with because I wasn't talking about that.
The point I was talking about you only addressed to some extent in a line below that:
>What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
Yeah exactly, the EU doesn't have computer manufacturing capabilities (just like I said 5 layers up) and it never will because it doesn't invest and also doesn't attract investors to invest.
>So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check.
No they're not. We don't have the skilled workers for that. Nobody in EU knows how to design Nvidia and AMD level GPUs and Altera and Xilinx levels of FPGAs that power AI datacenters. Nobody in EU knows how to make competitive 2nm fabs, otherwise EU fabs would have already bought ASML EUV machines and updated their ancient processes to the highly more profitable nodes instead of being stuck making cheap legacy nodes for cars and white goods. Those old nodes are still important to have to an extent, but ask yourself, would you rather sell a die for 10k a pop or sell 1000 dies for 10 cents a pop? Would you rather make more money or less money?
> Money? This is a rich continent.
Money is meaningless if you're not using it right. China is pushing to beat the EU and they have less money than the EU.
The World's Most Important Machine ;)
https://www.youtube.com/watch?v=MiUHjLxm3V0
Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.
Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.
The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.
Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?
>ASML only makes the shovel making machines
>>Yeah but Europe doesn't build any computer hardware,
Well, then this is will be a good start.
EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections.
So of course, semiconductor manufacturing sovereignty is very low on their priority list.
How many in that list of things is the US also doing?
The US is a single country. Russia is not on the US' doorstep. The US has its own oil. The US prints the world reserve currency.
Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens.
Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well.
The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners.
US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
That's what bureaucrats are supposed to be doing BTW.
Anyone who uses quotation marks around far right has clearly stated their allegiance.
It is no wonder then that such a person would do their best to poo-poo the worlds most successful peace project and the bastion of rule of law.
And yet, the ESMC Dresden fab is getting built. And now there's a Chips Act 2.0.