It's easy to blame the military and the higher-ups, after all they made this choice. But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.
> A Palantir spokesperson told Bloomberg the company “is not responsible for the underlying data nor identifying intelligence deficiencies” and that there is no evidence its software was at fault.
If you work for Palantir, you are writing software that kills people. Software that, in this case, killed 123 children. What salary is worth that?
I encourage others who work in the public sector, whether directly or through a contractor, to read and internalize Don't Get Distracted by Caleb Hearth [0]. I think about it nearly every day.
>It's easy to blame the military and the higher-ups
My brother in Cthulu, they're the ones making the life or death decision that killed people, who else should be to blame?
>But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.
If you wanna go the historical tech-tree scapegoat route then none of this would have happened if DARPA wouldn't have invented the internet, or if Thomas Edison wouldn't have invented the lightbulb, or if the fish haven't ventured onto land and evolve into us land mammal apes who invented sharpened sticks as murder objects.
Look, if Palantir had any redeeming qualities then I might agree with you. Unlike electricity which runs our computers, homes, cars, etc. What does Palantir do? It spies on people and helps governments to create "people we don't like" lists. In the absolute best case it's a "minority report" engine, but it's still just there to try and stop pre-crime (not crime).
This isn't the case where we are talking about a company doing anything socially good. It's simply there to help authoritarian governments.
Palantir only exists because government wants and funds these types of tools. If it Palantir wouldn't exist then another dozen companies with different names would exist that would create the same Torment Nexus for the government due to inexhaustible demand for such tools.
>It's simply there to help authoritarian governments.
The fact that you're so close yet so far away to figuring out who's to blame still boggles my mind.
Are you implying that the demand for these evil services absolves the suppliers from any moral responsibility? Simply because “if we don’t do this somebody else will”.
It’s an evil service and no matter who requests it or supplies it, doing so is an evil act.
>Are you implying that the demand for these evil services absolves the suppliers from any moral responsibility?
No, you are implying that. I'm saying that evil are (primarily) those who use it, not (just) those who make it. You can try to make whatever you want, if nobody will buy it to fund it, then you won't have money to make it, so your evil thign won't exist. Palantir only exists because governments are evil and willing to fund it to buy it and use it. If you want likes of Palantir to not exist then you need to hold your government accountable to not spend money on evil things.
Are gun makers evil and responsible whenever someone gets killed by a murderer, or are those who pull the trigger responsible for who they kill?
> Are gun makers evil and responsible whenever someone gets killed by a murderer, or are those who pull the trigger responsible?
Yes. There's a reason Remington decided to settle for $73M dollars with the sandyhook parents even though they didn't "pull the trigger" [1]. That's not a number you pick because you think you'll win the case.
[1] https://abcnews.com/US/sandy-hook-families-settle-remington-...
>There's a reason Remington decided to settle for $73M dollars with the sandyhook parents
It's only because of how the legal system works in the US(for better and for worse), with profit-seeking ambulance chasing lawyers, and where cases are decided on the emotions and societal moral panics of the people in the jury and the negative image such cases can cause to a brand, not if someone or a company is objectively right or wrong in the eyes of the law.
Meaning it's cheaper to pay people off even if they're legally wrong, than try to prove you are right but then deal with the bad publicity of the emotionally charged mob with pitchforks and torches who blames you for their kids dying and looks bad in the media.
Keep in mind parents of that time also blamed violent videogames like Doom for it, so it's not like they were right about who was to blame, they were just looking for retribution from anyone they could, aided by unscrupulous ambulance chasing lawyers looking to cash in on the drama that was feeding on the public's emotions amplified by the media also cashing in.
So highly mediatized and politically divisive/charged legal cases, are not (always) great examples of objectivity and righteousness of the US legal system.
I absolutely knew you’d bring up guns and gun making but it’s not a valid comparison. If the gun companies were also involved in the actual shooting of people part that would work.
Palantir were in no way obliged or coerced into making an evil thing, they just decided being evil for money was ok.
its clearly a democratically approved legal act though. If its as evil as you suggest, the better approach would be to convince your peers that its evil and thus should be made illegal
Again, absolving the company that built it and runs it. They weren’t coerced into doing so and must share the blame for the evil that was created.
I know, right? I'm not even sure if this is the biggest installation on Earth. If it's possible for us, it's possible for any number of private military contractors. I bet there's a similar capability in the hands of every advanced county on Earth. Anyway, shaddup, the focusing crystal's not calibrating right, and tomorrow I have present the targeting parameters to Dr. Evil.
Thank you for sharing the talk.
> I encourage others who work in the public sector, whether directly or through a contractor, to read and internalize Don't Get Distracted by Caleb Hearth.
I know a guy who works for well-known hardware company. He isn't a software dev, but close enough, works with Verilog on hardware design and verification. One day he tells me how he is working on video processing module and he doesn't know if that module will be part of surveillance system, missile guidance or nanny cam. It's possible that even middle management doesn't know more than him, and very likely the module will find uses in multiple products. He is a small cogs in a large machine, maybe his work will save a baby's life, maybe it will kill a person, he cannot know.
Well, at least there are definitely jobs in hardware and software where you can in fact be 100% sure what you build won't kill another person. Surveillance, Missile guidance, and nanny cams though? What, does he work for Honeywell or something?
The govt is investing in the military whether you like it or not, there is a set amount of GDP going into the sector regardless of your moral choice.
Would you rather not have someone make good software that does not misidentify schools at sites? Or should we outsource this to worse engineers who don't care about morals and don't find this metric meaningful?
That's a reasonable argument, but I don't think I could make the decision to work on software that's designed to kill people. I consider myself a pretty good software developer, and yes, I could probably do a better job building software to help kill people than many others, but I don't think "if you don't do it, someone less capable will, and more innocent people will get killed" would sway me. I just don't want to be involved in that, and I'm lucky that I don't have to be.
Well you made a personal choice to stick your head in the sand to not have to deal with the hard emotions, and that's fine. But to then pass judgement on people who don't sounds a bit unfair given this.
Of course it could have happened. You don't even need targeting for this to happen. You can drop bombs at random and kill whoever is unlucky enough to be in the wrong place. The US has a long history of doing it (Japan, Vietnam, ...).
So yeah, the military and whoever decided that a war was a good idea are to blame.
People working at Palantir are as much responsible for the killing of these 123 children than seatbelt manufacturers are responsible for the people who died because their product failed. That is, because they didn't do their job correctly, people died, definitely worth feeling bad about. However, should people stop working in seatbelt manufacturing because of it? Absolutely not, we need seatbelts, because without them, things would be worse.
Palantir, or whoever does target identification, is like the seatbelt of the reckless drivers of the militaries, who, for the lack of precise information, would just carpet bomb the place. It is not a hypothetical, it is how the US has worked before they had precision ammunition.
This is essentially a trolley problem. The war is the trolley, people who work for Palantir are the ones pulling levers. And if there is no one to pull the lever, the trolley is gong on a track with a lot of people on it.
If I write healthcare software, and a doctor orders the wrong drug that kills the patient, am I responsible?
It depends. Did you write a bug that caused the wrong drug to get ordered? Did you design or implement an algorithm that recommends drugs, and the algorithm was a bad one? Did you design or implement a confusing UI flow that led to the doctor ordering the wrong drug?
If so, I'd say yes, you share some of the responsibility. Legally, I'm sure you'd be fine, but if that were me, I'd feel responsible.
The doctor of course is responsible too; they're the one who didn't double-check that it was an appropriate drug. But multiple people can share responsibility for a bad thing.
Legally, probably not...?
Whether you may/should feel responsible depends on how the software works, I guess. Did it automatically recommend some drug, somewhat confidently, with gray text on gray background saying "AI sometimes makes mistakes"?
So I think we should behave responsibly when making software, even more now with AI
Let's not forget the shareholders who invested in this company knowing what they do. That includes some of my colleagues. I told them that whoever created and named this company is clearly a psycho.