Creating from scratch also creates hidden debt, it's just moved onto yourself. Especially when working with dates and timezones.
Creating from scratch also creates hidden debt, it's just moved onto yourself. Especially when working with dates and timezones.
I cannot imagine having the spare time to invest in building date/time foundations and maintaining them through changes to DST timing and country/time zone changes.
The only crazier idea I can think of is implementing character encoding conversions myself.
Seriously.
Nobody needs a package for "left-pad", which is the most infamous example.
But there are a lot of areas where it's not a good use of your time to reinvent the wheel. There's no moral virtue in writing everything yourself. And popular packages have gone through a more bug-finding and bug-fixing than your personal code probably ever will.
A library that goes "poof" when you need to upgrade it is also a hidden debt
There is also a distinction to be made between "technical" and "political" dependencies. Technical dependencies usually track some spec or common consensus, and political dependencies are e.g. timedate-library. Political dependencies are almost like business logic, because they have to track changing political decision made all over the world (or be local to some country).
Timedates are hard, and units may require even harder historical/present "political" tracking as how they are defined, and I would never want to maintain this kind of dependency: https://github.com/ryantenney/gnu-units/blob/master/units.da...
And what comes to timedate problems, I try to keep it simple if the project allows: store and operate with UTC timestamps everywhere and only temporarily convert to to local time (DSTs applied, if such) when displaying it in user-facing UI. This functionality/understanding can be locked into own 20-line microlibrary-dependency, which forces its responsible person to understand country's timezone and when e.g. DST changes and where/how/who decides DST changes and what APIs is used to get UTC time (and of course, its dependencies, e.g. NTP, and its dependencies. e.g. unperturbed ground-state hyperfine transition frequency of the caesium-133 atom, which result is then combined with the Pope Gregory XIII's Gregorian calendar, which is a type of solar calendar mixed with religious event for fixing the time, which which is then finally corrected by rewinding/forwarding "the clock" because its too bright or dark for the politicians).
A service goes poof, a library either slowly deteriorates or breaks just as easily as a self-written one (if an underlying platform breaks it for some reason). The self-written one is maintained by 1 person, the other is used by 100+ people who could jump in a collaborate on its fixing.
I would still rather using a library for dates, a million times so.
> a library either slowly deteriorates or breaks just as easily as a self-written one
Yes, I agree with this
> The self-written one is maintained by 1 person, the other is used by 100+ people who could jump in a collaborate on its fixing.
Libraries that have 100 people collaborating on it are very few
Most likely you'll have to vendor it and fix whatever issues you have.
Even worse when it's a dependency of a dependency and you also use it, so, let's say a dependency of yours bumps the version but this breaks your code. (Not sure this breaks only in python or in js as well, but it's possibly that it does)
Honest question: why is it so common for software developers to not upgrade their dependencies on regular intervals?
I can’t for the life of me figure out why. If you update everything incrementally you bypass the upgrade version problem when you’re so far behind that so much has changed that it becomes an overwhelming burden.
I think frozen dependencies are a big anti pattern, and places where I work that regularly update their deps tended to have better software practices generally
Because time spent updating dependencies won't contribute to adding new features.
Besides, any update risks breaking stuff. Not freezing dependencies isn't an option, because that means any commit can cause breakage in a completely unrelated part of the codebase, in a way which can be extremely confusing to debug. And you don't really want to install the very newest versions either, better wait a week or two for someone else to run into bugs and release a .1 version.
The sweet spot is somewhere in the middle: update often enough to avoid updates becoming a massive burden, stick with fixed versions between updates. I reckon it's best to just schedule some dedicated time for it every month or two.
It typically takes me maybe an hour to update my dependencies? I run type checks and e2e tests as part of that to have a relatively high degree of confidence that nothing has been broken. Also splitting the change into multiple steps (i.e. do minor/patch upgrades first, check nothing's broken, run a major upgrade, check nothing's broken, etc) means it's fairly to see where something is causing problems and needs to be handled more carefully.
I do this typically every couple of weeks, and it takes up almost no time at all in comparison to time spent on other work. Someone needs to review the eventual PR created, but that's also typically fairly easy. NPM makes this all very easy to do. In Python I've used tools like PDM or uv to handle dependencies similarly.
Yes, in python only with modern tools this is somewhat feasible
And you still have upgrades that break interfaces and such
That’s why we do it on an interval and not when it’s first available. Avoids bleeding edge bugs but prevents the overwhelming factor of being years out of date. It typically gets handed out at the end of the month or there about.
Being judicious in selecting dependencies goes a long way too. Not always easy but certainly worth the time
I'm glad to hear that all libraries you use honor semver religiously. For the rest of us stuck here on planet Earth it's rolling the dice what, exactly, is going to happen going from 1.7.12 to 1.7.13 for any random dep. The only way to find out is to try it and then unfuck things when it spits out some transitive error message because it turns out that package-A and package-F hate each other but only on Thursdays with a full moon
They don’t all follow semver perfectly or in some cases at all, but we have an interval when we do upgrades.
Our test suite is comprehensive and will catch most breakages automatically. The key to success is robust testing, as it cuts the manual footprint significantly.
This does mean we are quite judicious with selecting dependencies.
It isn’t all that complicated when everyone is following best practices most of the time I have found[0]
It still leavings me wondering in a lot of cases
[0]: perhaps this is the real heart of the issue is best practices are systematically ignored. I’ve worked at places like that and it’s no wonder they grind through folks
I mean this seriously: congratulations on working someplace that has such rigorous requirements that you are able to write a comprehensive test suite that doesn't get completely rewritten every month or so. I guess because I enjoy working in startups, the cost I have to pay is that "requirements" are spelt "fever dream" and the "acceptance criteria" field in Jira is often just "." (because it's a required field, donchano)
A member of staff who goes “poof” as the only one who understands your wrapper library that has a critical bug is a more common kind of hidden debt.