I can’t think of many occasions where I’ve needed SSH access to a box (ie they’re not just a fleet of ephemeral systems) but I’ve also not had the permission to install an alternative shell on it. I’m not saying it never happens. But it’s not as common a problem as it was 20 years ago.
Unfortunately though, it’s not generally that hard to switch between different shells. I hear the argument you made a lot from other people too and yet we all acknowledge you should be able to code in Bash and a “proper” programming language (Python, Go, JavaScript, C, Rust, whatever). In fact I’ve learned probably 2 dozen languages in my career and it’s rare when I need to pause and remind myself how to do x in y.
Or to put it another way: you don’t say “I’m not going to use $IDE on my dev machine because I’m stuck with vi on the remote server”.
But if people don’t feel comfortable working in two different shells and that is the dealbreaker for using murex, or any other alternative shell, then I’m not going to hold that against them. Everyone has their own preference when it comes to workflows and productivity tools.
Oh, you mean bash specifically. Then yeah, that's reasonable enough. Although I agree with the other commenter that sh/bash have the advantage of ubiquity.
None of those have the #1 best thing bash is good at: bash is already installed, nushell and fish are not.
Powershell is, as I understand it, not available on Linux but is omnipresent on Windows, so hopefully the windows folks can use it like they would bash.
PowerShell works on Windows because of WMI, to a first approximation.
Because Windows is tied together through APIs and databases and not text files (unlike Linux), everything is text is not as useful as something that can talk objects. And a lot of Windows is object oriented, from the window message dispatching system through COM and down to NT's object manager.
Ruby replaced all my shell needs. Almost 25 years ago.
I even have a shell written in ruby (it handles both
bash-like behaviour as well as ruby code as-is); admittedly
it is not quite perfect for everything, but I improve on
it steadily. And it works on Windows too, which was one
reason I wrote it in the first place (need to have it work
via cmd.exe as-is).
Never looked back to shell. It is too awful to use.
That’s not correct because POSIX passes what is ostensibly an array of strings.
Windows, on the other hand, only passes one string. So it’s up to the application to choose how to handle whitespace, quotation marks, and other nuances with parsing parameters.
Variable expansion in Bash is lazy. But there’s no reason why variables cannot be tokenised so that strings with spaces aren’t treated as multiple parameters. And in fact that’s exactly how some other shells work, such as the one I maintain.
While unix handles splitting into an array of strings, it still leaves parsing those strings up to the application. One of the bigger problems is the ambiguity caused by filenames starting with `-`. Some applications support the `--` marker to end such parsing, but it's inconsistently implemented and the caller has to actively remember to use it.
A typed array (e.g. by having a required single byte marker at the start of each string), or even nested structures similar to s-expressions would have avoided this.
Funny enough, I’ve actually spent a long time trying to figure out how to solve this problem in murex but it always comes back to the same problem: anything smart I implement into the shell is immediately lost the moment I call fork().
I even considered writing a wrapper around some commands to send -- regardless of whether the user includes it or not. But that’s error prone too because
1. it’s a GNUism so you can’t guarantee compatibility with any tools that don’t call GNUs flag parsing library (which is particularly problematic outside of Linux)
2. If there a file called “-f” (for example), how do you know if the user intended that to be called as a file reference or a command flag? You then need to bake in a bunch of additional syntax sugar to make that explicit. Which results in a pretty awful user experience
3. The maintenance overhead for this would be astronomical
If UNIX were designed from day one to have that parameter type passed then this would be a very simple problem to solve. But the PDP machine UNIX was originally designed to run on wouldn’t have been powerful enough for that anyway. So once again we are limited by an architecture designed to run on mid-range systems of the 1970s.
Array of arguments is vastly superior and more secure than every program/runtime inventing a slightly different way of splitting a command string into an array of arguments. No debate. A real problem is the related birth defect in ssh2.
Well, the only other way I can think of that it could be done is the Windows way, whereby you pass the unparsed command line, spaces and all, as a string to the new process. And while this is arguably the cleaner interface, in practice it has meant even worse quote handling, since how -- or even whether -- double quotes are parsed now depends on the probably undocumented process startup code chosen by the program's compiler vendor.
Want to quote a command line that may already contain double quotes, in order to pass it as an argument to some other program? No, you don't. It isn't right to want that.
The other other way would be more structured. Arguments are not just an array, they also contain `--switch` and key/value pairs (e.g. `--key value` or `--key=value` depending on who you ask). One problem with the flat array approach is there's no way to distinguish a literal value starting with `-` from a switch, which means there needs to be some way to workaround that (and users have to remember the workaround; how often do people remember to use `rm -- "$FILENAME"`).
In practice almost every application does still need to do its own parameter parsing; a flat array is not enough.
> hope some typescript-like "typed shell" becomes mainstream someday
The trouble with trying to invent a new, more robust shell language is you basically just end up re-inventing any number of scripting languages (eg Perl, Python, awk, ...), so you might as well use one of those.
Most regular programming languages aren’t well suited for shells because they have a verbose syntax due to their readability goals. But with a shell, the vast majority of times you’re typing in stuff that you have no intention of reading back ever again.
I’ve done a fair amount of research here and I actually think we do need a new programming language for the shell (and then I created one).
Excellent article, thanks for the link. I do agree with the premise. But every time I write some Bash code and think there should be a better way to do this, I ask myself why I don't just learn Perl. I dunno. Feels like inventing another solution would just hit the old "there are now 14 standards" problem where it would solve some problems but introduce others (see: PowerShell).
For the things shell is good at (running other commands, pipes, and shuffling files), I have yet to find anything even close to as good.
I agree that shells are uniquely good for that but there are plenty of better shells than Bash.
Such as Fish, nushell, Elvish, or the project I help maintain, “murex”
None that I can rely on being available wherever I see a terminal. It's bash or sh as far as I'm concerned.
Fortunately, 99.9% of the time I’m on my computer, where I can install whatever I want.
Then you ssh into an instance and have to remember how bash/sh work
I can’t think of many occasions where I’ve needed SSH access to a box (ie they’re not just a fleet of ephemeral systems) but I’ve also not had the permission to install an alternative shell on it. I’m not saying it never happens. But it’s not as common a problem as it was 20 years ago.
Unfortunately though, it’s not generally that hard to switch between different shells. I hear the argument you made a lot from other people too and yet we all acknowledge you should be able to code in Bash and a “proper” programming language (Python, Go, JavaScript, C, Rust, whatever). In fact I’ve learned probably 2 dozen languages in my career and it’s rare when I need to pause and remind myself how to do x in y.
Or to put it another way: you don’t say “I’m not going to use $IDE on my dev machine because I’m stuck with vi on the remote server”.
But if people don’t feel comfortable working in two different shells and that is the dealbreaker for using murex, or any other alternative shell, then I’m not going to hold that against them. Everyone has their own preference when it comes to workflows and productivity tools.
At what point does an alternate shell get baked into the image?
A different shell??? Nushell, fish, powershell, etc.
Oh, you mean bash specifically. Then yeah, that's reasonable enough. Although I agree with the other commenter that sh/bash have the advantage of ubiquity.
None of those have the #1 best thing bash is good at: bash is already installed, nushell and fish are not.
Powershell is, as I understand it, not available on Linux but is omnipresent on Windows, so hopefully the windows folks can use it like they would bash.
No, powershell runs on Linux, it just sucks because it wants objects and everything on *nix speaks streams of text.
PowerShell works on Windows because of WMI, to a first approximation.
Because Windows is tied together through APIs and databases and not text files (unlike Linux), everything is text is not as useful as something that can talk objects. And a lot of Windows is object oriented, from the window message dispatching system through COM and down to NT's object manager.
I did.
Ruby replaced all my shell needs. Almost 25 years ago. I even have a shell written in ruby (it handles both bash-like behaviour as well as ruby code as-is); admittedly it is not quite perfect for everything, but I improve on it steadily. And it works on Windows too, which was one reason I wrote it in the first place (need to have it work via cmd.exe as-is).
Never looked back to shell. It is too awful to use.
Unfortunately half of its badness isn't isn't the shell itself, but the convention of how parameters are passed to processes on Unix systems.
That’s not correct because POSIX passes what is ostensibly an array of strings.
Windows, on the other hand, only passes one string. So it’s up to the application to choose how to handle whitespace, quotation marks, and other nuances with parsing parameters.
Variable expansion in Bash is lazy. But there’s no reason why variables cannot be tokenised so that strings with spaces aren’t treated as multiple parameters. And in fact that’s exactly how some other shells work, such as the one I maintain.
While unix handles splitting into an array of strings, it still leaves parsing those strings up to the application. One of the bigger problems is the ambiguity caused by filenames starting with `-`. Some applications support the `--` marker to end such parsing, but it's inconsistently implemented and the caller has to actively remember to use it.
A typed array (e.g. by having a required single byte marker at the start of each string), or even nested structures similar to s-expressions would have avoided this.
Funny enough, I’ve actually spent a long time trying to figure out how to solve this problem in murex but it always comes back to the same problem: anything smart I implement into the shell is immediately lost the moment I call fork().
I even considered writing a wrapper around some commands to send -- regardless of whether the user includes it or not. But that’s error prone too because
1. it’s a GNUism so you can’t guarantee compatibility with any tools that don’t call GNUs flag parsing library (which is particularly problematic outside of Linux)
2. If there a file called “-f” (for example), how do you know if the user intended that to be called as a file reference or a command flag? You then need to bake in a bunch of additional syntax sugar to make that explicit. Which results in a pretty awful user experience
3. The maintenance overhead for this would be astronomical
If UNIX were designed from day one to have that parameter type passed then this would be a very simple problem to solve. But the PDP machine UNIX was originally designed to run on wouldn’t have been powerful enough for that anyway. So once again we are limited by an architecture designed to run on mid-range systems of the 1970s.
Array of arguments is vastly superior and more secure than every program/runtime inventing a slightly different way of splitting a command string into an array of arguments. No debate. A real problem is the related birth defect in ssh2.
Well, the only other way I can think of that it could be done is the Windows way, whereby you pass the unparsed command line, spaces and all, as a string to the new process. And while this is arguably the cleaner interface, in practice it has meant even worse quote handling, since how -- or even whether -- double quotes are parsed now depends on the probably undocumented process startup code chosen by the program's compiler vendor.
Want to quote a command line that may already contain double quotes, in order to pass it as an argument to some other program? No, you don't. It isn't right to want that.
The other other way would be more structured. Arguments are not just an array, they also contain `--switch` and key/value pairs (e.g. `--key value` or `--key=value` depending on who you ask). One problem with the flat array approach is there's no way to distinguish a literal value starting with `-` from a switch, which means there needs to be some way to workaround that (and users have to remember the workaround; how often do people remember to use `rm -- "$FILENAME"`).
In practice almost every application does still need to do its own parameter parsing; a flat array is not enough.
yeah but... that convention is so much of a security/bug headache
sometimes, its footguns seem worse than javascript...
hope some typescript-like "typed shell" becomes mainstream someday
> hope some typescript-like "typed shell" becomes mainstream someday
The trouble with trying to invent a new, more robust shell language is you basically just end up re-inventing any number of scripting languages (eg Perl, Python, awk, ...), so you might as well use one of those.
Most regular programming languages aren’t well suited for shells because they have a verbose syntax due to their readability goals. But with a shell, the vast majority of times you’re typing in stuff that you have no intention of reading back ever again.
I’ve done a fair amount of research here and I actually think we do need a new programming language for the shell (and then I created one).
I wrote a blog about this problem: https://murex.rocks/blog/split_personalities.html#conclusion
Excellent article, thanks for the link. I do agree with the premise. But every time I write some Bash code and think there should be a better way to do this, I ask myself why I don't just learn Perl. I dunno. Feels like inventing another solution would just hit the old "there are now 14 standards" problem where it would solve some problems but introduce others (see: PowerShell).
I don’t disagree with you per se. But if the new shell solves enough annoying problems then I think it has merit in existing.
> hope some typescript-like "typed shell" becomes mainstream someday
Might want to check out nushell (https://www.nushell.sh/)
PowerShell
The widwit answer: "Dont do X" (and nothing more)
The enlightened answer "You should use A, B or C for these reasons"
Even though bash is installed on many systems, I try to encourage people to use better designed shells that have less of these footguns :
Fish (https://fishshell.com/): No implicit word splitting : spaces in variables won't unexpectedly become separate arguments.
Zsh (I use this: https://ohmyz.sh/): Arrays start at 1 by default, but crucially, unquoted variables don't implicitly split into multiple arguments.
Nushell (https://www.nushell.sh/): Passes structured tables and records between commands : avoids fragile parsing of text with awk/grep.