Right now, there are approximately five languages that are presumed to be acceptable for core applications in the base system: C, C++, Shell (which probably means specifically bash), Perl, and Python. The most recent language to be added to that list is Python, about 20 years ago. That's not to say that everybody likes those languages (indeed, there's quite a few commenters here who I think would be surprised to learn that not only is C++ on this list, but that it's been on it for at least 25 years).
There's other languages that are considered acceptable, even desirable, languages to write applications in (e.g., Java, PHP, Go), but Rust is really the first language to compete sufficiently close to C's competence for people to contemplate adding it to the base-system-languages list. I'd say only Go has ever come close to approaching that threshold, but I've never seen it contemplated for something like systemd.
Interestingly, I wonder if the debates over the addition of C++, Python, and Perl to the base system language set were this acrimonious.
> Interestingly, I wonder if the debates over the addition of C++, Python, and Perl to the base system language set were this acrimonious.
I think any projects that are run by people that see themselves as "X-people" (like Python-people, Perl-people) always have a bit "ick" reaction to new languages being added to projects they might see as part of a language's community.
So say you're a C++ developer, contributed to APT over the years, see all of it linked to the C++ community which you are part of too, and someone wants to start migrating parts of it to Rust/$NewLang. I think it might sometimes affect more for these people than just the code, might even be "attacking" (strong word perhaps) their sense of identity, for better or worse.
I was (and am still) a C++ person. But I somehow became also a Rust person. I am sure some people felt attacked, but imho Rust has most of the characteristics that made C++ a good language and that's why I "embraced" Rust rather than felt attacked.
Yeah rust is not C++ but it really compliments C++ well. Each has their niche and tbh I see them converging into the two kings of the systems space rather than actually competing with each other.
If anyone sees that horrible mess of hacks around pre-STL C++'s lacks of namespace in combination with latest C++ features as part of the C++ community I'd be very surprised :D
If APT were a hardcore C++ project surely we'd have like adopted namespaces everywhere by now.
> I think any projects that are run by people that see themselves as "X-people" (like Python-people, Perl-people) always have a bit "ick" reaction to new languages being added to projects they might see as part of a language's community.
I would say that Pythonistas are quite accustomed to "(other) languages being added" to the Python ecosystem. After all, NumPy relies on Fortran, as well as C.
Asserting that kind of "ownership" over code seems rather distasteful to me. Maybe there would be less acrimony if developers got paid for it somehow.
Yeah, maybe Python was a bad example, didn't really think specifically about the languages I made as examples, they were just examples with the hope of making it easier to understand what exactly I was talking about.
Some communities indeed are better at embracing multiple languages, Python, JavaScript and Java/JVM comes to mind, where it isn't uncommon to call out to other languages.
> So say you're a C++ developer, contributed to APT over the years, see all of it linked to the C++ community which you are part of too, and someone wants to start migrating parts of it to Rust/$NewLang. I think it might sometimes affect more for these people than just the code, might even be "attacking" (strong word perhaps) their sense of identity, for better or worse.
How is language relevant here? If someone just rewrote it in the same language instead of a different one, do you feel the reaction would be significantly better?
This is really a Rust specific thing, since so many C++ projects were 1-on-1 translated to Rust only changing the licence.
Rust has been the tool of choice for stealing GPL3 open source projects where some people have spent all their free time on at some point in their life.
Rewriting software is not "stealing" it. That is an absurd framing, and one that seems akin to the arguments employed against FOSS software in the first place back in the 1980s and 1990s.
In your view, was writing a BIOS re-implementation from scratch "stealing" from IBM? Are all of the vaguely Unix-compatible operating systems "stealing" from Unix (ATT/Bell)? Why is the "free time" of the original developer more sacrosanct than the "free time" of the re-implementer?
I don't think they mean stealing in a legal/ownership sense, but stealing in the "I can no longer say this is my project" (control/intellectual/credit) sense.
> Why is the "free time" of the original developer more sacrosanct than the "free time" of the re-implementer?
This has nothing to do with free time. It has to do with the fact that the former actually went through a ton of pain to get the project from its original conception and past bugs & design flaws into its current useful state, and the latter is merely translating the solution right in front of them. And not only is the latter going to deprive the former of the ability to even say "this is my project" (despite them having spent years or decades on its design etc.), but they're also probably going to be able to take the whole thing in a different direction, effectively leaving the original author behind.
Whether you feel this is a good or a bad thing for society aside, it should be obvious why this could be genuinely upsetting in some situations.
Sure, it's upsetting. IBM was upset about BIOS being cloned, Bell got upset about Unix being cloned, Oracle got upset about the JDK being cloned, etc. Many of them were upset enough to engage in decade-long lawsuits. But we'd ultimately be in a much worse place if "avoiding making original creators upset" was the primary factor in development, over things like compatibility, bugs, performance, security etc.
I happen to like living in a world where Pipewire can just be a drop in replacement for PulseAudio that can implement the same APIs and do everything that PulseAudio does but better. Or where I can play my games on Linux because Valve re-implemented DirectX and all the Windows platform libraries. Or where there are half a dozen "vim" clones to pick from if I want to. If there are broad, tangible benefits to rewriting a piece of software, then it ought to be rewritten, hurt feelings nonwithstanding.
I don't really understand the argument that the original author gets deprived of anything by a piece of free software being rewritten, certainly not credit. Obviously stealing the name would be a dick move and maybe open you up to legal action but that's not really what happens 99.99% of the time.
> Sure, it's upsetting. IBM was upset about BIOS being cloned, Bell got upset about Unix being cloned, Oracle got upset about the JDK being cloned, etc.
This is not like that though; moving from a pro-user license to a pro-business license is the reason for being upset, not just losing control over the product.
With the move, any future improvements run the very real risk of being extended then closed off by tech companies.
It's simply hubris for an entire community of developers to look at an existing working product that got popular due to the goal of being pro-user, then say to themselves "we can replace it with this new thing I created, but in order for my new thing to gain traction it must be immediately favourable to big business to close off".
If you view it in that light, then, yeah, you can understand why the upset people are upset, even if you don't agree with them.
> Many of them were upset enough to engage in decade-long lawsuits. But we'd ultimately be in a much worse place if "avoiding making original creators upset" was the primary factor in development, over things like compatibility, bugs, performance, security etc.
Original creators did frequently get upset, but the difference between "some individual forked the code or cloned the product" is very different to "an entire community celebrating a gradual but persistent and constant effort by the same community to move the ecosystem away from a pro-user license".
I hope this gives you some insight into why people are upset, even if you don't agree with them. Most of them aren't articulating it like I did.
[EDIT: I'm thinking of this like an Overton-Window equivalent, shifting from pro-user to pro-business. It seems like an accurate analogy: some people don't want this shift to happen while others are actively pushing for it. This is obviously going to result in conflict.]
You're conflating two different things. The purpose of rewriting it is not to change the license. The purpose of rewriting it is to provide a modernized alternative with some benefits from the choice of language -- and the choice of license is incidental.
Kinda like how the purpose of creating LLVM was to create a more extensible compiler framework, not to create a permissively licensed compiler. As it happens the license of LLVM has not really led to the proprietary hellscape that some people suggested it would, and in any case the technical benefits vastly outweigh the drawbacks. Companies like Apple that do have proprietary compilers based on LLVM are difficult to describe as "freeloaders" in practice because they contribute so much back to the project.
> The purpose of rewriting it is not to change the license.
I didn't say it was.
> The purpose of rewriting it is to provide a modernized alternative with some benefits from the choice of language
I did not contend that either.
> and the choice of license is incidental.
This I disagree with - the license choice is not incidental; it is foundational to gain popularity in a hurry, to gain widespread adoption at corporates.
The rewriter's intentions is to gain popularity over the incumbent software; using a pro-user license does not gain popularity for all those pro-business use-cases.
The license switch from pro-user to pro-business is not incidental. It's deliberate, and it's too achieve the stated goal of replacing the existing software.
This is one place where I feel that the ends do not justify the means.
The difference is in the cases you mention these are massive companies with plenty of money simply trying to protect their revenue streams, whereas here we're talking about open source maintainers that are often doing this for the sake of their love for the project and the users, and getting very little other than recognition or ability to influence the project in return. It's not like you're talking about making a billionaire make a million less than he otherwise would.
Moreover, often these folks have rare expertise in those subjects, and/or rare willingness to work on them as open side projects. If you tell them this is just something they have to deal with, I wouldn't be surprised if you also remove some of the incentives folks have to keep contributing to OSS voluntarily in the first place. Very little money, and massive risk of losing control, recognition, etc. even if you "succeed"... how many people are still going to bother? And then you get shocked, shocked that OSS is losing to proprietary software and dying.
The primary benefit of FOSS over proprietary software is, in fact, the ability and willingness to change things and open up changes to a wider pool of developers. Fiefdoms are actively hostile to open source principles. Obviously it's good to treat people with respect but you shouldn't let worse code win out just because of ego alone. Sometimes software gets forked or rewritten and sometimes the fork/rewrite wins out, that's evolution and competition and it keeps the ecosystem healthy.
> Shell (which probably means specifically bash)
Debian has ongoing efforts to make many shell scripts (like postinst Scripts in packages etc.) non-bash-specific.
A minimal Debian installation doesn't contain bash, but rather dash, which doesn't support bash extensions.
> A minimal Debian installation doesn't contain bash, but rather dash, which doesn't support bash extensions.
Please don't make up wrong facts that would be trivial to check first.
All minimal Debian installations include bash as it is an essential package. Where essential is used in the sense of https://www.debian.org/doc/debian-policy/ch-binary.html#esse...
Whether with a base install via the installer, or debootstrap, I've never seen bash missing.
For clarity, 'sh' is what is softlinked to dash. Not bash.
> there are approximately five languages that are presumed to be acceptable for core applications in the base system: [...] Python
I don't know if you've tried to get someone else's Python running recently, but it has devolved into a disaster effectively requiring containers to accurately replicate the exact environment it was written in.
Core system applications should be binaries that run with absolutely minimal dependencies outside of default system-wide libraries. Heck, I would go as far as to say applications in the critical path to repairing a system (like apt) should be statically linked since we no longer live in a storage constrained world.
> I don't know if you've tried to get someone else's Python running recently, but it has devolved into a disaster effectively requiring containers to accurately replicate the exact environment it was written in.
Please show me a project where you believe you "effectively require containers" just to run the code, and I will do my best to refute that.
> since we no longer live in a storage constrained world.
I think you do care about the storage use if you're complaining about containers.
And I definitely care, on principle. It adds up.
For reasons I can only assume have to do with poorly configured CI, pip gets downloaded billions of times annually (https://pypistats.org/packages/pip), and I assume those files get unpacked and copied all the time since there would be no good reason to use uv to install pip. That's dozens of petabytes of disk I/O.
> Please show me a project where you believe you "effectively require containers" just to run the code
I guess GP meant "containers" broadly, including things like pipx, venv, or uv. Those are, effectively, required since PEP 668:
https://stackoverflow.com/questions/75608323/how-do-i-solve-...
> "containers" broadly, including things like pipx, venv, or uv.
This statement makes no sense. First off, those are three separate tools, which do entirely different things.
The sort of "container" you seem to have in mind is a virtual environment. The standard library `venv` module provides the base-line support to create them. But there is really hardly anything to them. The required components are literally a symlink to Python, a brief folder hierarchy, and a five-or-so-line config file. Pipx and uv are (among other things) managers for these environments (which manage them for different use cases; pipx is essentially an end-user tool).
Virtual environments are nowhere near a proper "container" in terms of either complexity or overhead. There are people out there effectively simulating a whole new OS installation (and more) just to run some code (granted this is often important for security reasons, since some of the code running might not be fully trusted). A virtual environment is... just a place to install dependencies (and they do after all have to go somewhere), and a scheme for selecting which of the dependencies on local storage should be visible to the current process (and for allowing the process to find them).
> This statement makes no sense. First off, those are three separate tools, which do entirely different things.
They are all various attempts at solving the same fundamental problem, which I broadly referred to as containerization (dependency isolation between applications). I avoided using the term "virtual environment" because I was not referring to venv exclusively.
and storage is a big deal for debian! It is basically the other player in embedded Linux other than rolling your own where a 4GB or smaller emmc is pretty common.
this is not normally a problem for a distro where the environment is well known
It is such a non problem it forced them to hack a „fuck you this Python is owned by the distribution not you“ message into pip requiring you to agree to „breaking your system“ to use it.
Of all the languages, python in the base system has been an unmitigated garbage fire.
> it forced them to hack a
It was not their action, nor is it hacked, nor is the message contained within pip.
The system works by pip voluntarily recognizing a marker file, the meaning of which was defined by https://peps.python.org/pep-0668/ — which was the joint effort of people representing multiple Linux distros, pip, and Python itself. (Many other tools ignore the system Python environment entirely, as mine will by default.)
Further, none of this causes containers to be necessary for installing ordinary projects.
Further, it is not a problem unique to Python. The distro simply can't package all the Python software out there available for download; it's completely fair that people who use the Python-native packaging system should be expected not to interfere with a system package manager that doesn't understand that system. Especially when the distro wants to create its tools in Python.
You only notice it with Python because distros aren't coming with JavaScript, Ruby etc. pre-installed in order to support the system.
Well the essential system Python should be in /usr/sbin and read-only (insofar Python allows that with its __pycache__ spam).
The fact that users have to keep up with multiple PEPs, error messages, --single-version-externally-managed, --break-system-packages, config files everywhere, stealth packages in .local and uv to paper over all of this shows that Python packaging is completely broken.
> the essential system Python should be in /usr/sbin
There's still quite a bit you can do with the "system Python". Mine includes NumPy, bindings for GTK, QT5 and QT6, Freetype, PIL....
> insofar Python allows that with its __pycache__ spam
This is, to my understanding, precisely why the standard library is pre-compiled during installation (when the process already has sudo rights, and can therefore create the `__pycache__` folders in those locations). This leverages the standard library `compileall` module — from the Makefile:
> The fact that users have to keep up with multiple PEPs, error messages, --single-version-externally-managed, --break-system-packages, config files everywhere, stealth packages in .local and uv to paper over all of this shows that Python packaging is completely broken.Please do not spread FUD.
They don't have to do any of that. All they have to do is make a virtual environment, which can have any name, and the creation of which is explicitly supported by the standard library. Further, reading the PEPs is completely irrelevant to end users. They only describe the motivation for changes like --break-system-packages. Developers may care about PEPs, but they can get a better summary of the necessary information from https://packaging.python.org ; and none of the problems there have anything to do with Linux system Python environments. The config files that developers care about are at the project root.
Today, on any Debian system, you can install an up-to-date user-level copy of yt-dlp (for example) like so, among many other options:
You only have to know how one of many options works, in order to get a working system.> All they have to do is make a virtual environment
Okay so to create a five line script I have to make a virtual environment. Then I have to activate and deactivate it whenever using it. And I have to remember to update the dependenceis regularly. For my five line script.
Seems to me the companies managing mloc-codebases pushed their tradeoffs on everyone else.
You, too: please do not spread FUD.
> Okay so to create a five line script... For my five line script.
I can guarantee that your "five line script" simply does not have the mess of dependencies you imagine it to have. I've had projects run thousands of lines using nothing but the standard library before.
> Then I have to activate and deactivate it whenever using it.
No, you do not. Activation scripts exist as an optional convenience because the original author of the third-party `virtualenv` liked that design. They just manipulate some environment variables, and normally the only relevant one is PATH. Which is to say, "activation" works by putting the environment's path to binaries at the front of the list. You can equally well just give the path to them explicitly. Or symlink them from somewhere more convenient for you (like pipx already does for you automatically).
> And I have to remember to update the dependenceis regularly.
No, you do not in general. No more so than for any other software.
Programs do not stop working because of the time elapsed since they were written. They stop working because the world around them changes. For many projects this is not a real concern. (Did you know there is tons of software out there that doesn't require an Internet connection to run? So it is automatically invulnerable to web sites changing their APIs, for example.) You don't have to remember to keep on top of that; when it stops working, you check if an update resolves the problem.
If your concern is with getting security updates (for free, applying to libraries you also got for free, all purely on the basis of the good will of others) for your dependencies, that is ultimately a consequence of your choice to have those dependencies. That's the same in every language that offers a "package ecosystem".
This also, er, has nothing to do with virtual environments.
> Seems to me the companies managing mloc-codebases pushed their tradeoffs on everyone else.
Not at all. They are the ones running into the biggest problems. They are the ones who have created, or leveraged, massive automation systems for containers, virtualization etc. — and probably some of it is grossly unnecessary, but they aren't putting in the time to think about the problem clearly.
And now we have a world where pip gets downloaded from PyPI literally billions of times a year.
Thank you! Exactly what I wanted to explain.
Yet, if I write a dockerfile, and need to use perl, system perl is fine.
If I need a python script, I have to arrange for all the RUN lines to live inside a virtual environment inside the container.
People are too harsh on this. It's not hard to install a version manager and set your primary python to that. Which is just good hygiene.
My understanding of the reasoning is that python-based system packages having dependencies managed through pip/whatever present a system stability risk. So they chose this more conservative route, as is their MO.
Honestly if there is one distribution to expect those kinds of shennanigans on it would be Debian. I don't know how anybody chooses to use that distro without adding a bunch of APT sources and a language version manager.
yes because then you're starting to use non-distro python packages. If you want to do that, use a virtualenv, there is no safe other way (even if there was no python in the base system) .
Yes, the distro people are strong believers in virtual environments as best practice - for you, not them.
There's a good reason for this. The average user has no idea and doesn't care what language some random distro-packaged program is written in. They want to be able to run ubxtool or gdal_calc or virt-manager or whatever without setting up a virtual environment. Python developers on the other hand should be adept at such things, should they choose to use a non-distro packaged version of something.
The tricky part is when "users" start using pip to install something because someone told them to.
This should become the official error message!
Can you expand? I'm honestly curious how it is not a problem in Pythons case
That asks GP to prove a negative. What problems have you encountered, and why do you believe them to be common?
It really, really isn't. I wish this would stop being repeated so much.
Is X11/KDE considered part of the "base system"? If yes, then:
... isn't so surprising.The Debian base system is much, much smaller. I'm surprised that people consider Python to be part of it. However, APT depends on Perl and the C++ run-time library, so those two languages have been part of the base system for a very long time.
> I'm surprised that people consider Python to be part of it. However, APT depends on Perl
Pardon?
Which apt is that?
? file `which apt`
/usr/bin/apt: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=157631f2617f73dee730273c7c598fd4d17b7284, for GNU/Linux 3.2.0, stripped
Debian doesn't ship files in /usr/local.
You can of course add your own "apt" binary in /usr/local/bin/apt which can be written in any language you like, say COBOL, Java, Common Lisp or Python.
Ah, of course. I guess this wrapper is provided by Mint.
I am curious: What does the wrapper do? Can you share the source code here?
It appears to re-organize various apt-related commands and sub-commands (including ones provided by additional distro-specific scripts) so that they're all available as sub-commands of `apt`. I don't do much micro-management so I haven't really thought about what the "proper" underlying commands are, except when I've had to look up specific `dpkg` invocations. It's about 200 lines.
Sorry, I meant that something in the package management stack depends on Perl (not APT specifically).
It looks like in trixie, it's libpam-modules that pulls in debconf, which is written in Perl. And libpam-modules is required by util-linux, which is Essential: yes.
I wonder how much of that Perl support is simply regex and other parsing utilities.
I’ve notice a lot of that in base OS systems
Its a curiosity more than anything though
A substantial fraction of apt is written in perl. It's actually still pretty core to debian, even if applications are using it less.
There is no Perl code in APT itself, nor in dpkg on the installing side; Perl is used by dpkg-dev, i.e. when building packages.
Ah, my apologies. I must admit I am not entirely clear on where the boundaries are between the different tools and other elements of the debian packaging process.
I'm sure the Debian base system doesn't include GUI stuff at all.