> I'm not convinced. The core OS kernel code, sure; but more than half of the Linux kernel is drivers ...
I agree that more than half of the Linux kernel is drivers. In fact, in my analysis of Red Hat 7.1 at https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html I found that 57% of the Linux kernel was in the drivers subdirectory. Obviously that varies over time. So I completely agree with that part!
> and those are not just applications but separate applications; a change in an Intel wifi driver is unlikely to interact in any way with the nvme driver.
They aren't really separate applications, though. More importantly, that's not the primary effort (and cost) driver for this effort estimation model. I used the COCOMO model, an effort estimation model that was publicly available and widely used at the time. I then applied its rules to the Linux kernel of that time, as best I could. I discussed every parameter I set, and why, here: https://dwheeler.com/essays/linux-kernel-cost.html
Here are some of the reasons why kernel code takes more effort, per the model:
* It inherits some of the general challenges of writing embedded software. There's very little in the way of a safety net (this is the safety net), and it's closer to the bare metal. You're writing in C (or later in Rust), not a language that shields you from many challenges. In COCOMO parlence this code is "semidetached".
* RELY: Required software reliability: High (1.15). The Linux kernel developers care a lot about reliability, which takes longer.
* CPLX: Product complexity: Extra high (1.65). "The kernel must perform multiple resource handling with dynamically changing priorities: multiple processes/tasks running on potentially multiple processors, with multiple kinds of memory, accessing peripherals which also have various dynamic priorities. The kernel must deal with device timing-dependent coding, and with highly coupled dynamic data structures (some of whose structure is imposed by hardware). In addition, it implements routines for interrupt servicing and masking, as well as multi-processor threading and load balancing. And yes, that includes drivers; drivers must handle threading and other challenges that "normal" code doesn't, because the driver code is where those complexities are handled."
* TIME: Execution time constraint: High (1.11). "Although it doesn’t need to stay at less than 70% resource use, performance is an important design criteria, and much effort has been spent on measuring and improving performance."
* VIRT: Virtual machine volatility: High (1.15). "The most common processor (x86) doesn’t change that quickly, though new releases by Intel and AMD do need to be taken into account [but] the other components of underlying machines (such as motherboards, peripheral and bus interfaces, etc.) change on a weekly basis. Often the documentation is unavailable, and when available, it’s sometimes wrong (which from a developer’s point of view looks like a volatile interface, since it keeps changing). The Linux kernel developers spend a vast amount of time identifying hardware limitations/problems and working around them. What’s worse, there’s a variety of different hardware, and new ones keep arriving... the interface of the underlying machine is actually quite volatile."
Not every factor makes things worse. The people analyzing it (ACAP) and developing the code (PCAP) are unusually capable, with high experience in the programming language (LEXP) and modern development practices (MODP). But these only partly compensate for the fundamental challenge of writing highly performant kernel code.
Anyway, that's my rationale. I did this back in 2004, so I was necessarily using data and models available at the time. Most importantly, I think its key point was absolutely correct: it would have cost far more than $50,000 USD to re-develop the Linux kernel of that time.
They aren't really separate applications, though. More importantly, that's not the primary effort (and cost) driver for this effort estimation model. I used the COCOMO model, an effort estimation model that was publicly available and widely used at the time. I then applied its rules to the Linux kernel of that time, as best I could.
COCOMO says that the cost scales superlinearly with the number of SLoC; my point is that drivers scale linearly because they're effectively separate projects. (In fact, to the extent that they're not separate projects, the cost in fact scales sublinearly since there's a bunch of code being copied and pasted when new drivers are written.) Your tool has this "--multiproject" concept; a better estimate would have identified which parts of the tree (primarily drivers) were functionally separate from the rest of the kernel.
I think its key point was absolutely correct: it would have cost far more than $50,000 USD to re-develop the Linux kernel of that time.
Oh, absolutely. I'm not disputing the conclusion; just the claim that an OS kernel is inherently more complex than another application of the same size.
> COCOMO says that the cost scales superlinearly with the number of SLoC; my point is that drivers scale linearly because they're effectively separate projects. (In fact, to the extent that they're not separate projects, the cost in fact scales sublinearly since there's a bunch of code being copied and pasted when new drivers are written.) Your tool has this "--multiproject" concept; a better estimate would have identified which parts of the tree (primarily drivers) were functionally separate from the rest of the kernel.
If all drivers were essentially completely independent projects that never interacted with each others, then yes, I'd agree that multiproject would be a better model. And if each was mainly a copy-and-paste of another, then it'd definitely be sublinear.
However, I have a very different expectation. In the Linux kernel, there's a strong pressure to try to create common interfaces that different drivers support. There's also pressure to create common lower-level functions that everyone can call. This reduces total code and reduces long-term maintenance, but ends up creating more interlinkages because the drivers are NOT really isolated separate projects at all. Maybe the drivers start somewhat independent, though I'm skeptical of even that, but I think that's not at all where they stay.
Let's get specific. The Linux kernel groups similar hardware into distinct subsystems. This includes networking, input subsystem (keyboards/mice/etc.), ALSA (sound), V4L2 (webcams/video capture cards), and GPIO.
Each subsystem defines a unified programming interface using standard data structures and callback functions. Drivers are typically split into a core framework that handles general logic and low-level portions. Driver developers plug into the existing framework for common features like power management and buffering. These interfaces often change as new drivers are created that require changes to the interface.
Because in practice there's a lot of interaction among drivers, I would not expect that, after years of driver development, the different drivers would really be independent. Indeed, many have asked the Linux kernel developers to make it easy to have completely separate and isolated drivers, but the developers have resisted because they believe it's important to have the drivers integrated into the kernel to support that kind of constant collaboration between drivers.
It'd be cool to see an analysis to settle the question. Future research for someone else I suppose :-).