This will get worse with new generations. They grow up on tables and phones where file system is a completely foreign concept. You need an app for everything.
I encountered juniors straight out of school who don’t know what tar is, or rsync, or what a symlink is.
It’s all learnable and everyone starts somewhere. But you’d think natural curiosity would kick in and they’d have picked up some of this on their own by the time they have a job.
I didn't even know about tar or rsync until I started to use Linux, and I don't really see how else one get to the point where you need to know about them. And even symlinks are still in my mind as 'shortcuts but [...]' (even though the other way around is probably more accurate). Even as a dev, it's very possible to go through life without touching Linux.
If you've built any open source software ever, it's pretty hard not to know of tar, even if you don't know exactly what it is. I think it's reasonable to expect anyone interested in computers would be familiar with common compression tools, let alone a CS major.
> I think it's reasonable to expect anyone interested in computers would be familiar with common compression tools
So 7zip and Winrar?
I joke, but only halfway. If you're only normal Windows user, you'd never hear of anything else (unless you want to go back to Winzip, which does still exist, but I've never heard of anyone using it any more).
If they were Windows game developers, they wouldn’t have to touch any of that. I guess it depends on where their interests lie and what platform they developed on.
But here’s one I heard literally two days ago: we counted three engineers (out of many) who knew that physical memory was not actually a giant flat space of contiguous addresses, and that there were multiple layers of address-mapping and region-joining glue logic between a program and the hardware, including in os libraries, and even inside the hardware.
Maybe knowing such information is archaic or useless for most engineers. But the good ones (or at least a certain flavor of the good ones) ask questions that lead them there.
How could it be? For example, each memory stick doesn't know beforehand what other memory sticks it will be used with, so the "physical" addressing of the memory on each stick has to be independent of the others, i.e. a local address, that gets mapped to a virtual one.
But that's not relevant? What would be relevant if, depending on where your bits are stored, acces time is significantly slower. If, regardless of where my data goes, my access time is constant then I do not care as a dev?
That's the abstraction I'm working with when coding. Which is necessary because in most cases this should be an implementation detail.
Really not caring about something also means not making a positive and false claim about it. What I said was relevant to that claim. That claim in turn was, according to your heuristics, not relevant (for daily coding).
This guy is right. Physical memory isn't flat. Every system with more than a few cores is NUMA. There are caches and cache lines. Memory channel interleaving. Ranks and banks and rows. Each DRAM chip keeps the last selected row and can access addresses within that row more quickly even if other accesses to other chips occur in the middle, unless a refresh cycle occurs.
It's all a layer below even OS programming. It's configured at the BIOS level and then performed in hardware. But that's the point, isn't it? Virtual memory is below the application programmer, too, but here we're chastising him for not understanding it. If we do that, shouldn't we equally chastise people for not understanding physical memory? Or speculative execution? Or head seeking and servo tracks? Or Ethernet line coding?
Even NUMA is a "flat space of contiguous addresses". There's no bank switching or whatever. Yes, the access times are non-uniform, but that's orthogonal. The point of my comment to the OP is that if you're going to be super-pedantic about this stuff, you need to get your terminology right.
Are we assuming x64? Many ARM systems have gaps in their physical address space. x64 can too but not always. For example the Gameboy Advance used the top 8 bits of the address to identify the peripheral that is accessed.
I wrote x86 assembly back when we were switching from real mode to protected mode, and I still have a feeling that I would not have been able to answer whatever question you asked there in a way that would satisfy you.
I also could no longer tell you how to balance a binary tree or implement Quicksort.
You seem to be under the impression that any part of the average contemporary identity-and-behavioral-development stack encourages asking questions. Babysitting/teach-to-the-test schools don't, authoritarian parents and community authorities and corporate overseers don't, consensus-manufacturing politicians and media don't. (Heck, even our national soccer team doesn't.)(That was a joke.)
Worse, people who find it within themselves to ask questions anyway frequently face silence or a smackdown. The Internet's Own Boy remains no longer with us.
For what it’s worth, I recently retired from a 30 year career as a developer and while I’m aware of all three of those things, I’ve never used any of them in or out of work.
And I’ve hosted my own web server these last 20 years.
I've got a friend, I'll call him Bob, mostly because Bob is nowhere near his real name and I'm about to say some not very flattering things about him. He is my friend and not my job colleague or anything, so there's certain objectivity to this, in fact I appreciate Bob and I wish nothing but the best for him.
Bob has been programming for a bit more than a decade. He is, barring the people I know from other countries, the best paid programmer I know, and I know well over a dozen.
Bob has no interest whatsoever in any programming language that is not what he has been using his whole career, nor does he have any interest or curiosity in dipping his toes in other related fields, web development, tinkering with arduinos, home servers... Nothing. Bob has not built a computer, ever, doesn't know how to do it, nor what each part in that object does, beyond the hard drive because he did plug an extra one once or twice.
More anecdotally I played some Factorio and Satisfactory with Bob, now I know these are not excellent representations, but I expected a degree of order, planning and foresight, I was very much surprised when facing the reality of none of that being present, and I very much did not expect to see the same thing the few times I've looked at his code, but I did see that same thing.
I promise you Bob doesn't make up for all of this in social skills.
Now, is Bob a good engineer? I really, really don't think so. Is he a curious person? A bit, not much, I get the feel he just ended in CS with no particular interest for it, but I'll say it again: He is the best paid programmer I know.
Is that frightening? Well yeah, in a way. It's also endearing in a "Well damn if this guy can do it then surely I can too" sort of way. Money is not everything of course, but it's as good of a proxy as any.
> Considering many people doesn't know what a file is, keeping the spirit alive is important, I think.
Files aren’t sacred. It’s actually troubling that many technical people never consider alternatives. Most of the worlds data today is probably not stored in filesystems, rather in databases and object stores that use custom storage backends.
I didn't make such claim, and there's no need to comeback with such force. A file is a good abstraction of what a "casual" user works with. E-mails, documents, images, videos. These are great applications for files. Also, while some people hate UNIX's "everything is a file" approach with a passion, it's a great way to allow people to explore what's behind the wallpaper they see every day.
You know what, these big databases and object stores are living on bog standard but scalable filesystems most of the time. Ceph, ZFS, Lustre, etc., and sharded/stored as files. There are some very high performance and bespoke systems out there [0], yet they also provide POSIX compliant filesystem views outside, not because they are sacred, but because it works.
Filesystems provide great utilities for making abstractions other than files, by using files as universal containers you can play with. Again, while they are not sacred, they are an important building block, and they're here to stay. Creating a custom storage backend, throwing away all filesystem stuff is a fool's errand, since filesystems handle much more than providing a tree and some attributes on that file/folder hierarchy. Since they are the first level on top of physical storage devices, they also take care of the device underneath them (e.g. TRIM, FFFS (Flash Friendly File System), etc.).
It's a great irony that E-Mails are already stored as human readable databases on disks.
I’m just pushing back on the notional ignorance of non-technical users for not understanding one abstraction among many.
And I’m complaining about technical and non-technical users who don’t understand why object stores (and web servers, and ftp servers, and archives) aren’t file systems just because they can hold files.
And don’t get me started on documents. Document object models can be so much more when they’re not just treated as a sequence of bytes.
After the nth time of my Android Chrome app silently corrupting and auto-deleting my browsing history (all of it; it's not a bug, it's the intended behavior!), would love to hear about a file alternative that's modular enough to resist that kind of catastrophic failure and interoperable enough to inspect/repair if anything ever happens.
how would any alternative design prevent intended behavior?
But the most obvious alternative to a filesystem would be formatting a whole hard drive as an sqlite database. Obviously it would be a radical rethink in OS design.
There are also "single-level stores" from the last millennium - designs where there is no separation between volatile and nonvolatile storage. All memory in these systems is treated as nonvolatile. A Word document, for example, would be something like a suspended Word process. A directory is a process that only manages pointers to other files and directories. Obviously processes must be extremely lightweight in such a system. KeyKOS is an example of this and you can read papers about it and its Unix emulation layer. This is one of the many things humanity explored before settling on the hierarchical filesystem as the base layer of storage.
>how would any alternative design prevent intended behavior?
If you're asking how something other than the sqlite database Chrome profile data is stored in would prevent the intended behavior of deleting corrupted files silently, obviously the two issues are orthogonal. But a file-based system would mean a corrupted file (the others untouched) rather than a corrupted database (everything is gone). I'm horrified at the idea of replacing my resilient, inherently modular set of files with a brittle monolithic database.
> But the most obvious alternative to a filesystem would be formatting a whole hard drive as an sqlite database.
At the end of the day, you'll be chasing pointers inside that SQLite database, where you store tables, indexes and such. Interestingly, this is how a filesystem works. Tables, indexes, redirections and fields. Very much like a database. EXT4 is a table of redirections, nothing fancy [0].
I agree that files aren't sacred, they are just another abstraction at the end of the day, but what the other user (I think) is referring to is the trend in which, as smartphone use becomes ubiquitous, users lose sight of the technical side of things.
It's not that the average user is ignorant of the many ways in which data can be stored and retrieved. It's that they are becoming ignorant of such abstractions existing altogether. It's hard to start thinking about how images are stored if all the user knows is "they are in the gallery".
It's sad that we need this new concept of "IndieWeb", as the whole Internet evolved into a monstrosity hosted and guardrailed by a handful of megacorporations. Hosting files became a privilege, when it should've been a (human) right all along.
edit: The tech to host yourself is obviously still there, but the _mindset_ changed to cloud only.
We are a long removed from when putting something in your public_html folder would instantly give you a personal web site. Not that I am saying people should still hand write html or anything but it was a very quick and natural way to participate back in the day. (A unix account usually gave you automatic p2p chat too using finger to see if your buddy was online and talk/ytalk to talk to them.) This was little less than magic at the time even when your friend was sitting on the terminal right next to you.
Considering many people doesn't know what a file is, keeping the spirit alive is important, I think.
This will get worse with new generations. They grow up on tables and phones where file system is a completely foreign concept. You need an app for everything.
It's not like the preceding generations were any different, until they had to do work on a PC.
I encountered juniors straight out of school who don’t know what tar is, or rsync, or what a symlink is.
It’s all learnable and everyone starts somewhere. But you’d think natural curiosity would kick in and they’d have picked up some of this on their own by the time they have a job.
I didn't even know about tar or rsync until I started to use Linux, and I don't really see how else one get to the point where you need to know about them. And even symlinks are still in my mind as 'shortcuts but [...]' (even though the other way around is probably more accurate). Even as a dev, it's very possible to go through life without touching Linux.
If you've built any open source software ever, it's pretty hard not to know of tar, even if you don't know exactly what it is. I think it's reasonable to expect anyone interested in computers would be familiar with common compression tools, let alone a CS major.
> I think it's reasonable to expect anyone interested in computers would be familiar with common compression tools
So 7zip and Winrar?
I joke, but only halfway. If you're only normal Windows user, you'd never hear of anything else (unless you want to go back to Winzip, which does still exist, but I've never heard of anyone using it any more).
If they were Windows game developers, they wouldn’t have to touch any of that. I guess it depends on where their interests lie and what platform they developed on.
Ok, sure, windows.
But here’s one I heard literally two days ago: we counted three engineers (out of many) who knew that physical memory was not actually a giant flat space of contiguous addresses, and that there were multiple layers of address-mapping and region-joining glue logic between a program and the hardware, including in os libraries, and even inside the hardware.
Maybe knowing such information is archaic or useless for most engineers. But the good ones (or at least a certain flavor of the good ones) ask questions that lead them there.
But physical memory is a giant flat space of contiguous addresses. Do you mean virtual memory?
How could it be? For example, each memory stick doesn't know beforehand what other memory sticks it will be used with, so the "physical" addressing of the memory on each stick has to be independent of the others, i.e. a local address, that gets mapped to a virtual one.
But that's not relevant? What would be relevant if, depending on where your bits are stored, acces time is significantly slower. If, regardless of where my data goes, my access time is constant then I do not care as a dev?
That's the abstraction I'm working with when coding. Which is necessary because in most cases this should be an implementation detail.
Really not caring about something also means not making a positive and false claim about it. What I said was relevant to that claim. That claim in turn was, according to your heuristics, not relevant (for daily coding).
This guy is right. Physical memory isn't flat. Every system with more than a few cores is NUMA. There are caches and cache lines. Memory channel interleaving. Ranks and banks and rows. Each DRAM chip keeps the last selected row and can access addresses within that row more quickly even if other accesses to other chips occur in the middle, unless a refresh cycle occurs.
It's all a layer below even OS programming. It's configured at the BIOS level and then performed in hardware. But that's the point, isn't it? Virtual memory is below the application programmer, too, but here we're chastising him for not understanding it. If we do that, shouldn't we equally chastise people for not understanding physical memory? Or speculative execution? Or head seeking and servo tracks? Or Ethernet line coding?
Even NUMA is a "flat space of contiguous addresses". There's no bank switching or whatever. Yes, the access times are non-uniform, but that's orthogonal. The point of my comment to the OP is that if you're going to be super-pedantic about this stuff, you need to get your terminology right.
Are we assuming x64? Many ARM systems have gaps in their physical address space. x64 can too but not always. For example the Gameboy Advance used the top 8 bits of the address to identify the peripheral that is accessed.
I almost questioned the same thing but I think we both know that's what he meant.
I wrote x86 assembly back when we were switching from real mode to protected mode, and I still have a feeling that I would not have been able to answer whatever question you asked there in a way that would satisfy you.
I also could no longer tell you how to balance a binary tree or implement Quicksort.
You seem to be under the impression that any part of the average contemporary identity-and-behavioral-development stack encourages asking questions. Babysitting/teach-to-the-test schools don't, authoritarian parents and community authorities and corporate overseers don't, consensus-manufacturing politicians and media don't. (Heck, even our national soccer team doesn't.)(That was a joke.)
Worse, people who find it within themselves to ask questions anyway frequently face silence or a smackdown. The Internet's Own Boy remains no longer with us.
For what it’s worth, I recently retired from a 30 year career as a developer and while I’m aware of all three of those things, I’ve never used any of them in or out of work.
And I’ve hosted my own web server these last 20 years.
Windows web server?
If all you do is write code for Windows, why would you need to know what any of that is?
Because Steam... /s
The lack of basic computer knowledge I’ve seen from developers is terrifying.
It's almost inspiring how far someone can get without understanding really basic stuff about how an operating system operates.
> inspiring
You spelled "frightening" wrong.
It's a complex feeling, really.
I've got a friend, I'll call him Bob, mostly because Bob is nowhere near his real name and I'm about to say some not very flattering things about him. He is my friend and not my job colleague or anything, so there's certain objectivity to this, in fact I appreciate Bob and I wish nothing but the best for him.
Bob has been programming for a bit more than a decade. He is, barring the people I know from other countries, the best paid programmer I know, and I know well over a dozen.
Bob has no interest whatsoever in any programming language that is not what he has been using his whole career, nor does he have any interest or curiosity in dipping his toes in other related fields, web development, tinkering with arduinos, home servers... Nothing. Bob has not built a computer, ever, doesn't know how to do it, nor what each part in that object does, beyond the hard drive because he did plug an extra one once or twice.
More anecdotally I played some Factorio and Satisfactory with Bob, now I know these are not excellent representations, but I expected a degree of order, planning and foresight, I was very much surprised when facing the reality of none of that being present, and I very much did not expect to see the same thing the few times I've looked at his code, but I did see that same thing.
I promise you Bob doesn't make up for all of this in social skills.
Now, is Bob a good engineer? I really, really don't think so. Is he a curious person? A bit, not much, I get the feel he just ended in CS with no particular interest for it, but I'll say it again: He is the best paid programmer I know.
Is that frightening? Well yeah, in a way. It's also endearing in a "Well damn if this guy can do it then surely I can too" sort of way. Money is not everything of course, but it's as good of a proxy as any.
> Considering many people doesn't know what a file is, keeping the spirit alive is important, I think.
Files aren’t sacred. It’s actually troubling that many technical people never consider alternatives. Most of the worlds data today is probably not stored in filesystems, rather in databases and object stores that use custom storage backends.
I didn't make such claim, and there's no need to comeback with such force. A file is a good abstraction of what a "casual" user works with. E-mails, documents, images, videos. These are great applications for files. Also, while some people hate UNIX's "everything is a file" approach with a passion, it's a great way to allow people to explore what's behind the wallpaper they see every day.
You know what, these big databases and object stores are living on bog standard but scalable filesystems most of the time. Ceph, ZFS, Lustre, etc., and sharded/stored as files. There are some very high performance and bespoke systems out there [0], yet they also provide POSIX compliant filesystem views outside, not because they are sacred, but because it works.
Filesystems provide great utilities for making abstractions other than files, by using files as universal containers you can play with. Again, while they are not sacred, they are an important building block, and they're here to stay. Creating a custom storage backend, throwing away all filesystem stuff is a fool's errand, since filesystems handle much more than providing a tree and some attributes on that file/folder hierarchy. Since they are the first level on top of physical storage devices, they also take care of the device underneath them (e.g. TRIM, FFFS (Flash Friendly File System), etc.).
It's a great irony that E-Mails are already stored as human readable databases on disks.
[0]: https://docs.weka.io/4.3/weka-system-overview/filesystems
I’m just pushing back on the notional ignorance of non-technical users for not understanding one abstraction among many.
And I’m complaining about technical and non-technical users who don’t understand why object stores (and web servers, and ftp servers, and archives) aren’t file systems just because they can hold files.
And don’t get me started on documents. Document object models can be so much more when they’re not just treated as a sequence of bytes.
After the nth time of my Android Chrome app silently corrupting and auto-deleting my browsing history (all of it; it's not a bug, it's the intended behavior!), would love to hear about a file alternative that's modular enough to resist that kind of catastrophic failure and interoperable enough to inspect/repair if anything ever happens.
how would any alternative design prevent intended behavior?
But the most obvious alternative to a filesystem would be formatting a whole hard drive as an sqlite database. Obviously it would be a radical rethink in OS design.
There are also "single-level stores" from the last millennium - designs where there is no separation between volatile and nonvolatile storage. All memory in these systems is treated as nonvolatile. A Word document, for example, would be something like a suspended Word process. A directory is a process that only manages pointers to other files and directories. Obviously processes must be extremely lightweight in such a system. KeyKOS is an example of this and you can read papers about it and its Unix emulation layer. This is one of the many things humanity explored before settling on the hierarchical filesystem as the base layer of storage.
>how would any alternative design prevent intended behavior?
If you're asking how something other than the sqlite database Chrome profile data is stored in would prevent the intended behavior of deleting corrupted files silently, obviously the two issues are orthogonal. But a file-based system would mean a corrupted file (the others untouched) rather than a corrupted database (everything is gone). I'm horrified at the idea of replacing my resilient, inherently modular set of files with a brittle monolithic database.
> But the most obvious alternative to a filesystem would be formatting a whole hard drive as an sqlite database.
At the end of the day, you'll be chasing pointers inside that SQLite database, where you store tables, indexes and such. Interestingly, this is how a filesystem works. Tables, indexes, redirections and fields. Very much like a database. EXT4 is a table of redirections, nothing fancy [0].
[0]: https://blogs.oracle.com/linux/understanding-ext4-disk-layou...
Yes, believe it or not, all memory gets populated with data structures to be useful. That doesn't mean all data structures are equivalent.
I agree that files aren't sacred, they are just another abstraction at the end of the day, but what the other user (I think) is referring to is the trend in which, as smartphone use becomes ubiquitous, users lose sight of the technical side of things.
It's not that the average user is ignorant of the many ways in which data can be stored and retrieved. It's that they are becoming ignorant of such abstractions existing altogether. It's hard to start thinking about how images are stored if all the user knows is "they are in the gallery".
The article was a lot of words just to say that.
It's sad that we need this new concept of "IndieWeb", as the whole Internet evolved into a monstrosity hosted and guardrailed by a handful of megacorporations. Hosting files became a privilege, when it should've been a (human) right all along.
edit: The tech to host yourself is obviously still there, but the _mindset_ changed to cloud only.
> edit: The tech to host yourself is obviously still there, but the _mindset_ changed to cloud only.
Sadly with CGNAT, port blocks, hosting any server being a terms of service violation...
Non-cloud shared hosting costs $2 per month
We are a long removed from when putting something in your public_html folder would instantly give you a personal web site. Not that I am saying people should still hand write html or anything but it was a very quick and natural way to participate back in the day. (A unix account usually gave you automatic p2p chat too using finger to see if your buddy was online and talk/ytalk to talk to them.) This was little less than magic at the time even when your friend was sitting on the terminal right next to you.
Oh, wait until the kids learn about hardware.