I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
> I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
Nobody can agree on what those common things are in a general purpose langusge though. It works for something like Go, because it is largely used for servers and command line tools from what I can tell.
For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.
And even if did put some of those things in std, that is stable forever. That is how C++ ended up with an unusably slow regex implementation in the standard library. And it isn't fixable. Everyone just ignores it and uses a third party library like PCRE, RE2, etc instead (depending on their specific needs). I would not expect web or GUI things in the standard library for this reason, those standards change too frequently for that to be stable forever.
And even something like Python, which is known for data science and machine learning, doesn't have core libraries for those activities in the standard library. Numpy, Pandas, Scipy, Pytorch etc are all third party projects.
This is spot on. I am extremely uncomfortable with the large number of transitive dependencies that end up in pretty much any non-trivial rust application. No amount of memory safety will save us if a tiny, ubiquitous library that nobody scrutinizes because it’s nested eight layers down the dependency graph gets compromised.
I think Go apps tend to have better dependency hygiene because the language has a better standard library, which results in better culture around dependencies.
I also think it’s frustrating that both cargo and npm totally ignore the decades of prior art from Linux distributions that have figured out good ways to improve dependency management. We’d be in better shape if there was a community curated subset of known-good dependencies that are release-managed together that the broader ecosystem could build on, sort of like Ubuntu having “main” and “universe”.
I learned programming from C# which has an excellent standard library. Just recently started using Go which seems to have a good standard library. Been using python for years, can't say I ever had a complaint about the standard library.
Rust meanwhile seems to be following some "no standard library" philosophy.
I feel like there's an opportunity out there to become like an amoeba: fund and build a third party Rust standard library, absorb absolutely everyone who is so grateful to have a library, and then get so big absorb Rust itself.
I think one thing that has made Go have better dependency hygiene is not merely having "batteries included" in the standard lib, but the early focus on having production-ready implementations of a lot of stuff in the standard lib, as opposed to minimum viable implementations. I used to get a long way with one or two dependencies that would barely fan out at all. It's been a while since I worked on a Go project, so I'm not sure if it's still that way.
I think Go ends up with fewer dependencies mostly because there is more friction in finding them. With rust (and npm) you just "cargo search xxx", then "cargo add xxx". Go makes you web search and poke around github looking for something. It's not onerous or anything, but just that extra step slows things down just a bit. C projects tend to have the least because it's even more annoying to add them and you have to create build commands that deal with slightly different distributions (and learn makefile+pkg-config or cmake, or autoconf).
Certainly having a larger stdlib helps Go projects keep their deps down, but I don't think it's the real reason.
Actually I think this is more of a culture thing. Or maybe "is also" a culture thing.
One of the core tenets of early Go was the maxim "a little copying is better than a little dependency".
Probably because of this stance, they didn't even HAVE a dependency-management solution for years
I strongly agree the fewer dependencies the better, on average.
Perhaps, but I think it depends on where the Go devs are coming from. In my experience the lack of proper dependency management in older versions of Go didn't really lessen dependencies, it just made teams have to deal with annoying $GOSRC issues. But then I worked at a place where the devs used PHP (w/composer) and node before Go was introduced.
Personally I came from a C background so I tended to use deps more sparingly.
In the end it's a bit of a balancing act—lots of dependencies is a larger opportunity for these kind of supply chain attacks to affect you. Copying stuff into your repo protects you from that, but it also makes it way more likely that you'll miss security fixes, unless you're actively looking for them. Re-implementing what you need is also viable but it slows down the dev process.
I agree. Java has the same ability to quickly add deps, and in some case they can be sprawling, but it's still perfectly possible to develop complex apps without a lot of deps coming in.
Culture has a lot to do with it.
> Certainly having a larger stdlib helps Go projects keep their deps down, but I don't think it's the real reason
I disagree: having a big stdlib that encompasses a huge chunk of common functionality (logging, a plethora of network server/client protocols, etc) means that for basic needs, the question of 3rd party libs never arise, whereas for other language, the question is not if you need a dep, but which one. Making gorilla easier to find won't undo the cultural reluctance of adding a 3rd party dep when using the (very pragmatic, IMO) stdlib server is adequate to the task.
Go has an official package search at https://pkg.go.dev/
There’s no poking, unless you are into that sort of thing
Every Rust crate is heavily scrutinized these days using LLMs (which also caught this issue). The days when no one looked at dependencies are gone.
If those transitive dependencies are baked into the language runtime, how is anything materially different? You just shift the vector around.
Because the standard library of a programming language has a lot more eyes on it than one of the many small dependencies does.
You are just shifting where the eyes are, the amount of eyeballs (developer time) is the same. Unless the proposal is to increase the amount of (other) developers time.
The proposal ends up amounting to either shifting stuff around, or asking to other people to put in more effort? Into a project that usually doesn't pay (programming languages and runtimes)
Which by the way, asking other people to do stuff for free IS the attack vector, if you are the kind of dev or company that gets hit by these, your ultimate root cause is that your business strategy is using code without paying for it. No such thing as a free lunch, you will pay for it, among other things, with an increased cybersecurity risk.
>There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
The big reason is money & time. Maybe less so today if you are happy with an AI generated stdlib, but I don't think it's fair to compare Rust (built by Mozilla, who's primary product is an open source web browser) to Swift/AppKit (built by Apple, 4.6T marketcap at the time of writing).
When Go was released it had an amazing stdlib, but that was because Google was funding it.
There's probably to truth to this, but I would weight the design intention behind the language just as or more heavily than available resources.
Swift was intended from the start to not only replace Objective-C and all of its use cases, but also take on new use cases and bring a number of features from other languages that were newer and had different dominant paradigms.
Certainly having a juggernaut like Apple behind it has been instrumental in its success in achieving those goals. That said, a community-lead project with similar aims could probably achieve those aims as well, given enough time.
I think it's just rare for enthusiast-led projects to set out to do such things. My theory on why things tend to go that way is that the types of people to start programming language projects tend to heavily lean nuts-and-bolts and theory with a purist/idealist sort of mentality that's more concerned ideological purity than practical usability.
So to sum this all up, a community-run practical, multi-purpose, batteries-included language likely needs to at least partially designed and helmed by product engineer types so it doesn't end up anemic and stuck in an ideological rut.
Python is batteries included, but all the batteries have corroded.
Look at C++'s long in the tooth STL.
You don't want to marry a language to fast aging libraries you have to support for eternity. Better libraries always naturally emerge.
Rust's decision here is fine.
The only thing I'd like to see is the ability to programmatically limit transitive dependency count or depth in Cargo. I'd also like crates to specify whether they limit their own deps and whether they have panick-y behavior or not.
I think maybe the right answer is to have a standardized, curated group of libraries pegged at some sort of LTS release that only backports security fixes. However, someone would need to pay for creating and maintaining this -- and then you wonder where the money would come from?
That or potentially fast tracking merging popular libraries into the standard library. Or at least concepts from popular libraries. Basically all the "most downloaded" crates on the Crates.io front page should be candidates for merging into the standard library.
https://crates.io/
Good ideas are pulled into the standard library.
https://crates.io/crates/once_cell
https://doc.rust-lang.org/std/cell/struct.OnceCell.html
Not everything should be eligible for this treatment. Certainly not an HTTP library or something without extremely widespread applicability.
Nearly all of my projects used once_cell, so it's good to see that pulled in. I wouldn't want to see anyhow, thiserror, anything opinionated, or anything domain specific in the std. That's a weight you have to bear forever.
Remember how long Python 2 -> 3 took, and look at the ancient and awful stuff in Python 3's standard library. Rust is not the right language for this.
someone tried doing this for rust, but the reaction was that it was vibe-coded/low quality.
https://github.com/rust-stdx/stdx
https://news.ycombinator.com/item?id=48571266
Note that there are baby versions of this that are uncontentious, for example
https://blessed.rs/crates
this is missing the LTS release. but it is a curated group of libraries that are relatively uncontentious to recommend.
The fallacy is assuming that standard != unchangeable.
One can very simply....make breaking changes. Yes it will require some work to update but that's already the case when you upgrade library versions.
The dependency hell doesn't help anyone here, the downside is much worse - lots of bloat, overgeneric libraries and awful supplychain security.
Rust releases a new version every 6 weeks and the latest version is the only supported version. This is only tenable when the amount of breaking changes is tiny and each instance's impact is analyzed and cost-benefit tradeoff is checked and mitigations are put in where possible.
Some features, e.g. the never type[0], are held back years due to all the work that's needed to minimize the breakage.
[0] https://youtu.be/3jM4cnEVrLc?t=271
I’m a big fan of how C# handles this. The standard library is versioned like any other library. Your program is explicit about the major version of the standard library you’re pulling in. And all major versions of the standard library continue to work.
For rust, this would mean something like adding std = 1.0.6 to your cargo.toml. The advantage is it means the standard library can deprecate and replace features. Upgrading std is an explicit step by the developer. Annoying, but in the age of LLMs it should be pretty easy.
The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.
> The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.
C++ went through that around C++11, where some types had to have ABI breaks (on some implementations, such as that of GCC's libstdc++). It has been a while, but as I remember it std::string was affected since it went from COW to SSO. Map might have been affected too (don't quite remember). It was a mess.
Rust can link multiple semver versions of the same dependency, but you can't pass data structures between those versions. For transitive dependencies that usually isn't a big deal if your direct dependencies use them internally rather than exposing the types in their own APIs. But std contain vocabulary types that everyone uses (Option, Result, String, etc) so that would really not work.
Can we pick and choose the good without the bad? If we agree stability is a net-negative we could just say there will be a list of blessed trustworthy libraries with extra scrutiny and bureaucracy (make no mistake this is completely necessary to what people are asking for) to make sure they don't go rogue. But without a guarantee they will stick around and get updates forever.
Rust has a sizable and well featured standard library at this point. I think it would be absurd to claim that the base language is "near-unusable" if you are using it for system programming, which is its intended use case.
Huge standard libraries make sense for Java, Go, Apple platform etc. because they are developed by giant enterprises that can manage the overhead. Thinner modular systems are a more natural fit for open source development.
True. And the rust standard library is pretty good. But I really wish we had a good, fast, small futures executor in std. And accompanying async variants of File and Socket and so on.
Async rust is a jungle of weird compatibility questions. People treat async runtimes like sports teams. (I know I do). I wish it were more like Nodejs where async is just built in.
It’s being used for a wide variety of other purposes however, and so perhaps it’s time to adapt to that reality.
While overhead is a real concern, I think it’s often blown out of proportion. Once a language achieves a certain baseline of stability and isn’t in constant flux and the standard library matures, changes become infrequent and maintenance load is low. The work is heavily front-loaded.
Not at all true, look at the large parts of Python's standard library that are effectively unusable or irrelevant. And they even allow some breaking changes over time (unlike Rust). This is especially true when it comes to anything related to internet protocols or file formats, but there are other modules that have far better replacements on PyPi too (re vs regex for example).
This was actually the main thing that put me off of Rust. I get the argument for a small std lib. I just also don’t agree it’s worth it. Go seems to handle having a batteries included standard lib just fine.
People confuse what they want.
They do not want a big stdlib. The downsides are real (the stdlib cannot make breaking changes), and there are no upsides (except, maybe, for faster compilation, since std comes precompiled).
They want more official crates (e.g. `regex` and `libc` are official crates, maintained by the Rust project). And the Rust project does not oppose to that, it just doesn't have the funding.
> there are no upsides (except, maybe, for faster compilation
Another big reason to put something in std is providing types for cross-crate compatibility. If I want to pass a String from one crate to another, I’m glad that string is defined in std so there’s an obvious type we can both use in our APIs.
C - for example - does not have this luxury. Everyone makes their own string type, and C APIs all need to translate strings at the api boundary. It’s super annoying.
It would be nice if we had a standard way in rust to declare a type as serialisable. Right now lots of crates have a serde feature flag to do this with serde. But (a) they need to put this behind a feature flag, since it adds a dependency on serde. And (b) this doesn’t work with other serialisation libraries.
The other big one is futures. There’s still no futures executor in std. Async crates have to decide if they want to tie themselves to a single executor (like Tokio) or be compatible. There is no async stream api in std. No async file api. And so on. It’s a compatibility nightmare.
That is true, but this does not require a huge stdlib. And Rust is partially open to that (even adding async traits for IO in std is something they'll likely do in the future), but because std can't make breaking changes, you need to be very careful and slow.
> The other big one is futures. There’s still no futures executor in std. Async crates have to decide if they want to tie themselves to a single executor (like Tokio) or be compatible. There is no async stream api in std. No async file api. And so on. It’s a compatibility nightmare.
We need useful traits to abstract over async runtimes. But there are a lot of design points for runtimes so this can't ever go into std. There is embassy, for embedded systems. There is monio and glommio built around io-uring. Then there is smol intended to be simple and lightweight. And then there is the kitchensink of tokio.
Clearly embassy is the one we want in the standard library. /s (It is the only one I personally would have a use for. But no, I don't think any of them belong in std. Traits to abstract over them? Yes, absolutely.)
But even your other example of strings: there are many design points for strings too, that might be better fits for specific use cases: string builder (with a capacity, what String in rust is), copy on write, small string optimisation (like compact_str), even interned strings. So your example of a vocabulary type is somewhat flawed, a better example would be Option or Result. Even anyhow and thiserror build on top of those vocabulary types.
Who said a standard library can't make breaking changes? That used to be a norm some time ago.
There are upsides, your program is smaller, better security because a random kid can't pwn your deps, quality and interoperability. What's not to love?
> Who said a standard library can't make breaking changes?
The std making a breaking change is the language making a breaking change. Most mainstream languages guarantee stability, certainly Rust.
> your program is smaller
How so? It doesn't matter if the code is in std or a crate.
> better security because a random kid can't pwn your deps, quality and interoperability
That's exactly what I said: you don't need bigger std, you just need more official crates.
>Most mainstream languages guarantee stability, certainly Rust.
That's their mistake, right there... When you're upgrading software, you presumably want a better version. You can't have something better without changing it. It's a logical contradiction.
>It doesn't matter if the code is in std or a crate. It very much does because you don't need 95% of the random stuff in the crates. Even for something like rand, you need an xorshift and that's roughly it. 25 lines of code, sorted. You can even write it as a copypaste "dependency" without much fuss in practically any language. That, versus importing a whole rand library with lots of different algorithms, deps on crypto, tests, OS random-based seeding, customisability, etc.
>you just need more official crates
I guess that's one solution but it would probably just be a better idea to split std in two SLAs, one is guaranteed for core stuff i.e. the status quo, one is YMMV.
> You can't have something better without changing it. It's a logical contradiction.
Sorry, that's a strawman. Of course you cannot have a better version without changing things. You definitely can have a better version without changing public API.
And people upgrade their toolchain not because they want a different API. They mostly want bugs fixed, support, and sometimes features. Breaking the API has a huge cost and minor benefit.
> It very much does because you don't need 95% of the random stuff in the crates.
But why does it matter if all that stuff comes in std or in an different crate? As I said security doesn't play a role here. You can't even avoid compile-time cost because the std can only come precompiled because it only changes with the toolchain, which is another downside we want to avoid.
> I guess that's one solution but it would probably just be a better idea to split std in two SLAs, one is guaranteed for core stuff i.e. the status quo, one is YMMV.
And again, how is that different from what I say, other than calling "official crates" as "std"?
> Who said a standard library can't make breaking changes?
I don't want my language creating work for me. "Batteries included" is a feature of higher level languages. Rust is going to live for a very long time, probably beyond all of our lifetimes, and it needs to endure fads without picking up baggage and liabilities.
Rust could have a mandate where certain crates are "blessed" as recommendations. This could come with additional review, oversight, support, documentation, etc.
The Cargo.toml spec should have the ability to limit direct and transitive dependency count and depth. (As well as other things, like "nopanic" annotations/guarantees.)
The Crates.io repo should grow namespaces so we don't typo squat popular packages. It's much harder to miss organization names changing than package names changing.
On the other hand, there are some bad Go standard libraries that are frozen in time.
Yeah, but I will take a not great library that works everywhere the compiler does, than be at the whims of which platforms are supported by 3rd party libraries.
I can use most of the clunky Python, Java, .NET and if it must be, Go, standard libraries, than hunting down for dependencies with platform tier support and such.
Which is how you get the mess that is the standard library of C++. Where regex is an unusably slow joke and everyone uses third party libraries for that instead.
And tons of parts of the standard library (and language) being cordoned off as "legacy, don't use for new development". Of course figuring out what you shouldn't use in C++ can be hard too. It isn't well documented (or universally agreed upon), and it takes having it as a full time job to be able to keep up with these days.
No, I much prefer the way Rust is doing it.
Add more and more to a standard library, and you're going to start losing your "works everywhere".
Naturally there is a balance, still I think Java, .NET, Python, Go, Smalltalk, manage quite well, while having subsets to slim down when needed for specific deployment scenarios like embedded devices.
Platform support is a non-issue for every library being discussed. This is an array construction macro, it has nothing to do with what operating system you're running on.
Just because some trees have a specific trait, it doesn't mean the whole forest is the same.
which, as painful as it may be, is ok. Better to have a safe functional stdlib library than a exposed external crate (which then asks the question; what's the replacement...)
Those aren't the only two choices.
We also have the classic example of PHP with numerous not safe stdlib ways to use mysql.
To me the answer is still to vendor your dependencies and don't be on the bleeding edge of updates unless you're willing to invest the time into validating them.
> We also have the classic example of PHP with numerous not safe stdlib ways to use mysql.
I think it goes without saying that emulating PHP is rarely a good decision.
Don't forget C++ where large parts of the standard library are unusable (regex is slow and unfixable) or soft deprecated (dont use iostreams for formatting, use std::format, etc).
No, I prefer what rust is doing. It suits a system programming language. Which is what Rust is.
Which isn't actually a problem. You can ignore the bad standard library and use something different.
> I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
> There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
I wholeheartedly agree. Most standard libraries out there (when I say most I easily mean of 99% languages that I know) have pitifully tiny implementations. They at most provide for string parsing, io, a bare networking layer (that's at best a wrapper around the POSIX syscalls), threading and that's it. If you want to do anything more complex, you either have to roll it yourself or use an assortment of the "blessed libs" for any given language, that may or may not work properly, and may or may not even work well with each other. I got pissed off by this at some point so I decided to go ahead and roll my own standard lib that would actually have everything I need to.. actually develop apps. Now I know most people probably aren't going to be using it (mainly because its functional API isn't what most C++ devs want) but for me it's actually been brilliant.
I don't see the problem with boost? Isn't that a perfect example of your well supported, community embraced option? I certainly feel much safer pulling something in from boost via the official debian repos than I do pulling a random package with npm or cargo or etc.
Primarily I think the underlying concern has to do with the pathway for authoring code. When you have contributors whose submissions are gated with a rigid third party process and where that third party is the one responsible for curating the code (as opposed to the author also being the curator as well as the publisher) then you have the possibility to catch a lot of wrongdoing before it succeeds.
> I don't see the problem with boost?
There are many issues that I've seen taken up with Boost, but my personal peeve is how it can make building projects that incorporate it a pain, both because of its sheer size but also because it can sometimes be difficult to appease with its own dependencies.
that seems unrelated to the security problem to me
> I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
> There’s no reason why this can’t be replicated elsewhere.
It is replicated elsewhere and is not limited to Apple; I can do the same using a tech stack of perhaps 2x techs and very limited (manually added) deps. I'm thinking Lazarus, which comes with most things I'd need for most apps.
The Odin programming language does this! It has also decided not to provide a package manager.
Go took the same approach and ended up having to implement a halfarsed one when everyone started implementing their own.
Has Go not had the most secure ecosystem?
What is your critique of their approach? Is it not the case that `go get` is the only one which doesn't even provide a way for the person downloading to run the downloaded code until it is actually executed by the consuming codebase? That seems pretty sound by comparison.
I guess you're saying, "they had to" meaning they should've seen the need and provided it? I'll say this to that (imagined) take; plenty of useful software was made without it, and they got the job done when they knew the absolute most about what a good solution would need. I totally understand being annoyed at the fact that this is the story of every evolution in Go, but I genuinely think they're picking good implementations when they decide on them.
> What is your critique of their approach?
It’s half arsed, brittle and far from user friendly.
> Is it not the case that `go get` is the only one which doesn't even provide a way for the person downloading to run the downloaded code until it is actually executed by the consuming codebase?
If you’ve added the package to your imported then odds are your next step is going to build it. Thus negating any benefit.
I think the real issue is malicious packages entering package ecosystems. Whether your package manage executes on downloads or not is moot because you’ve still got untrusted code sat in your project imports, just waiting to be accidentally executed.
> I guess you're saying, "they had to" meaning they should've seen the need and provided it? I'll say this to that (imagined) take; plenty of useful software was made without it, and they got the job done when they knew the absolute most about what a good solution would need. I totally understand being annoyed at the fact that this is the story of every evolution in Go.
I think you’re being too charitable here. I think Russ Cox just didn’t want a package manager because C doesn’t have one. But ended up relenting after everyone nagged the Go team for years afterwards.
And really what they built was the bare minimum to manage package version pinning and updates. But it has none of the visibility that central package repositories have. So how do you know if a Go package has been compromised when the only source of truth is the compromised origin?
Ultimately I think devs need to think about their dependencies and decide which ones get pinned and a serious review before pulling. If the thing has got binaries, it gets a serious review. If it does anything with cryptography, it gets a serious review... etc.
I don't think automatic updating is a good idea at all. It's just the honor-system, and trust is a security flaw.
Thanks for the thorough reply. I personally haven't experienced brittleness/unfriendliness, but I have only written around 10k lines of Go. Not exactly a power user, but I like to think I understand it.
> If you’ve added the package to your imported then odds are your next step is going to build it. Thus negating any benefit.
I actually think this is the benefit! I don't need to go to a website to see what changed, I can just have a look at the code. The best dependencies have a changelog. Ideally I can look at a diff.
> I think you’re being too charitable here. I think Russ Cox just didn’t want a package manager because C doesn’t have one. But ended up relenting after everyone nagged the Go team for years afterwards.
That's entirely possible.
> So how do you know if a Go package has been compromised when the only source of truth is the compromised origin?
Fair point, and probably worse in this future we're in now that NIST is drowning in CVEs and has turned away from some share of them.
Spoiler alert! JavaScript has no language provided package manager.
If Odin gets moderately successful someone will probably reinvent it.
NPM came along in 2010, Javascript was huge before that.
It was only when people wanted a common approach to shipping both in browser and "native" that this glitch happened. I believe a standard library and "batteries" would have abated it entirely or resulted in a slightly less-bad situation. I do think Cargo is a slightly less-bad situation in many ways, and that it can be done better.
> It was only when people wanted a common approach to shipping both in browser and "native" that this glitch happened.
Nah. Npm was invented because node had its node_modules directory. But it was tricky to find and download modules you wanted to use. Until npm, you had to add libraries to node_modules by hand. And check them in to git or something. And keep them up to date somehow. Npm added a searchable index and a tool to automatically install all your modules. Npm was only bundled alongside Nodejs many years later.
Bundling was separate. I can’t remember if browserify predated npm or not. But it was a wild idea at the time to make node modules build for the browser too. Browserify - and later webpack and friends - work with or without npm.
Sure. Maybe threshold is bit higher than moderately. But unless your language tries to sabotage itself by making code artifacts uncomposable (a la C/C++ where best way to compose libraries is through shell commands) some package manager will be inevitable.
Batteries also don't help if dependencies don't replace them. Arrayref functionality has been part of Rust std lib for a while now.
>I think we should be taking a more “batteries included” approach
https://www.youtube.com/watch?v=GZOuz-SG7-g
Funny how you skipped "I should build my own batteries" and when straight to "increasing and centralizing the duties of your main gratis 'vendor'".
I don't like the "bring your own" approach because unless one happens to be an absolute tour de force 10x engineer unstoppable god of a programmer (which most of us, myself included, are not), whatever you build is never going to be as well-rounded, fleshed out, and complete as something built by a larger organization, especially when it comes to UI libraries (which are monstrous projects if done right, e.g. meeting accessibility requirements).
I'm happy to contribute to a larger effort but anything I can build on my own is going to be a thin, flimsy happy meal toy compared to something with the backing of a company or well organized FOSS project.
The counterpoint is that whatever std invents is probably not going to be as good as what moviated people in the community make.
The rand crate is my favorite example of this. They have a whole bunch of different rngs. All rated by quality and performance. Some are csrngs and some aren’t. It’s a delight. “Batteries included” languages don’t come close.
Then there’s serde - which uses a clever technique with traits to allow compile time specialisation and optimisation of binary and json serialisers. Most people had no idea that was even possible in rust until someone in the community made it.
Maybe std should pick up the best crates from the community and bring them in house? This might be a good idea. But also maybe not. Since serde came out, several other crates have found ways to crush its performance numbers. Often by 2-5x if memory serves. What a relief we didn’t immortalise the slow version of serde, right?
I’m playing devils advocate here. I think std should be a bit bigger too. But there are real tradeoffs in doing so.
There's also the time factor. Something you build on your own will not have decades of development and polish behind it.
>https://docs.rs/arrayref/latest/arrayref/
>This package contains just four macros, which enable the creation of array references to portions of arrays or slices (or things that can be sliced).
The package in the OP is definitely not in the category of decades of development, it's closer to a leftpadism.
I agree that building your own is hard and something that can be done by exceptional engineers, but software is famously a winner-take-all industry, even if the optimal strategy for lower percentile programmers and median programmers to import generic modules, as long as building your own results in the best product (which it does, whether importing frontend or backend modules, if you hand code something, it's going to be the optimal strategy for building a winner product. Even programmers that are median or below average probably have an incentive to aim for being exceptional, because the best value of median and below average programmers is not building median or low value software, but having a shot at building exceptional software.
Also the challenge of building a generic module is much larger than building your own. A generic module needs to have flexibility for many different options and integrations, when you build your own you build just what you need and tightly integrate it with the product
> Also the challenge of building a generic module is much larger than building your own. A generic module needs to have flexibility for many different options and integrations, when you build your own you build just what you need and tightly integrate it with the product
True but I believe overestimated. Usually rather than building what's needed, what gets built is what is thought to be needed (often a substantially smaller subset), and then over time you end up building a markedly poor version of a generic module.
I think that even for core modules like an http server or client, but that's not the case and we don't need to get into that debate.
let's look at the actual package from OP
>https://docs.rs/arrayref/latest/arrayref/
>This package contains just four macros, which enable the creation of array references to portions of arrays or slices (or things that can be sliced).
I'm no rust programmer, but that doesn't sound like something that moves the needle. I talked about the ratio between value to risk being a relevant decision parameter, so getting infected by an http framework would be defensible, getting infected by adding this to a project to me is a PIP, and getting infected by installing leftpad or a custom cursor plugin in an IDE would be fireable.
with LLMs it's becoming more common to just vibe up anything you need that might be missing. even if there is an available package you can pull in. the most secure option as well