That sounds so expensive it's hard to see it making money. You'd processing a 2fps video stream for each customer. That's a huge amount of data.
And all that is for the chance to occasionally detect that someone's seen an ad in the background of a stream? Do any platforms even let a streamer broadcast an NFL game like the example given?
I used to work for an OTT DSP adtech company i.e. a company that bid on TV ad spots in real time. The bidding platform was handling millions of requests per second, and we were one of the smaller fish in the sea. This system is very real. Your tv is watching what you’re watching. I built the attribution pipeline, which is what this is. If you go buy a product from one of these ads, this is how they track (attribute) it. Not to be alarmist butttt you have zero privacy.
The TV thing isn't a new story, this was public. Everyone should have known about it and no one cared. (I could inset a boilerplate rant about Snowden here)
Those datacenters are not being built so that you can talk to ChatGPT all day, they are being built to generate and optimize ads. People who were not previously very suggestible are going to be. People who are suggestible will have their agency sold off to the highest bidder.
Avoid owning a TV? Your friends will. Maybe you can not have a FB/IG/WhatsApp account, only use cash, not have a mobile phone, but Meta (or Google, or Apple) can still detect your face in the background of photos/videos and know where you shop, travel and when.
This is really interesting. Can you expand on this? What are OTT and DSP in this context?
Do you have a sense for what data is tracked and how it's used? Or if this sort of system is blind in certain cases? (eg: I hook up an N64 to the a/v ports -- will I get retro game ads on the TV?)
OTT = over the top = ads that aren't shown on cable ("linear") DSP = demand-side platform = real-time bidding on ad space on behalf of advertisers
What data is tracked? Don't think we can see what's plugged into the TV if it's not connected to the internet but besides that... all of it... If we have your TV we know where you live. We know what you're watching (hopefully our customers' ads!). We know all the devices that connect to your home network. We know where those devices go when you leave the house. We know you were driving down this stretch of road when you saw that ad on that billboard or on the side of that truck ("out-of-home" advertising). We know if you saw that ad and then bought something ("conversion" + "attribution"). We know what apps you have downloaded. Did you know Candy Crush is spying on you, too? Did you know Grindr sells your IP address? We likely know your age and your race and how much your home cost and where you went to college and how many kids you have ("segmentation"). Privacy laws have gotten in the way a little bit, but not much - it's less "we can't get this data anymore" and more "here's the hoop(s) we now have to jump through but we still get it".
I don't want to freak anyone out. In my time in adtech I never felt like anyone was using this data for anything besides "Please buy more coca-cola..." but you never know. Privacy _can_ exist it's just insanely hard because there's so much money hell-bent on tracking you down.
So, you helped with this... Why?
> you have zero privacy
Is this data linked to me personally in some way (e.g. though an account) or is it anonymous data?
They can definitely work out who you are from your IP address. (or get close enough that the advertisers don't care) Not too many people are putting a VPN on their router and using throwaway accounts for their smart TVs. This might be difficult anyhow if your log into major services such as Amazon, etc, who will know who you are.
I'm not saying this is impossible to avoid, but it ends up being a LOT of work when the alternative is just not connecting the TV to the internet and using a laptop / Apple TV / etc. instead.
Personally identifiable. Most smart TVs force a login to connect to the Internet or even use at all.
>Not to be alarmist butttt you have zero privacy.
Hence why I will never connect my TV to the internet
I understand the perils of a capitalist system but whyyy would you agree to build this
The perils of the capitalist system man. For what’s its worth, I left adtech many moons ago specifically because it is a horrifyingly depressing industry and very very not fun to talk about at parties.
I'm glad you got out, but given your vantage point what would you say to those who feel pressured to do these types of jobs? Would you say more "it isn't worth it" or "if you have to... but get out as fast as possible" or something else?
Money pays the bills. It’s probably not deeply rooted.
Forgive me, but I'd actually like to hear vrosas's response or someone else with a similar background. I appreciate you trying to answer my question and help try to make me informed, but I don't want to hear speculation, especially the rather obvious ones. That's not helping, it just adds more noise to the conversation and discourages a response by them. We all know money pays the bills, no one needs to hear that. But hey, if that's what they say, then you'll be proven right. So let's wait and find out. I really do want to understand their mentality. I hope you do too because how else do we break the cycle?
My man’s not wrong. Adtech has some seriously cool engineering problems and scale. It’s its own form of high frequency trading mixed with everyone you’d imagine from a modern day Mad Men. Plus tons and tons and tons of money.
But that's different from money pays by bills. And I think it's also important to recognize that there can be some fun in this due to the challenge.
I started on another side of engineering and I get that. Building rockets is exciting and fun. But while you build those things it's easy to forget you're building something much more destructive.
It’s not different. I take jobs to pay bills and in the selection process pick opportunities that offer career growth and interesting problems. Money pays the bills.
I've talked to a lot of engineers building DRM technology, and most of them are just a combination of swept up in the fun of the challenge, and also deeply bought into the idea of protecting intellectual property. I would say probably 90% don't see any philosophical issues with what they're building at all. If you can convince them of that, quite a few of them would probably try to get out, but it's quite an uphill battle. I forget who said the quote and the exact words, but something along the lines of it's very difficult to disabuse somebody of a belief when their livelihood depends on believing it.
As someone who was in an industry that I later discovered was doing things I wasn't personally ethically okay with, I would advise them to do similar to me. Start looking for a new gig and just get out as soon as you can.
Unfortunately as an individual there just isn't much you can do. There will always be someone willing to do the job that you aren't willing to do. Just get out and find something you can sleep at night doing
This is incredibly close-minded of you. It’s important creative work can be compensated for. This debate was tired in 2002. You should at least understand the other side instead of treating it with the moral simplicity that’d apply equally to Nazis.
Heh, I love that you accuse me of being closed-minded on this, while simultaneously claiming that nothing relevant has happened since 2002 because you're "tired" of the debate.
I'd also recommend you actually find out about somebody's background before making such a giant assumption about how much they know about the issue.
And if you don't think DRM has been used to abuse legitimate users, you yourself have a significant amount of understanding the other side to do. A great start might be just reviewing some of the anecdotes in this comment section or any of the other many threads that continually happen on HN. It turns out that DRM technology has actually changed a bit in the last 24 years, and it's not just about "protecting" IP
If I read a full biography of you, does it change anything about my perception that you're close-minded for saying working on DRM is obviously 100% morally wrong and it's beyond you to understand why anyone would?
It makes its creator the money they can spend buying the products they see in TV ads.
If someone is going to get paid to build it anyway, I might as well be the one getting paid for it.
This attitude is the reason “someone is going to get paid”.
If you see a unattended laptop in a coffeeshop, do you steal it because “someone will steal it, so it might as well be me”?
Why stop here? We can also blame the people, who implemented such features on the TVs, the people who worked at companies, who used data acquired by these devices for advertisement, the people who worked on the mentioned ads for such devices and the people who bought products from companies, that spend money on such marketing techniques.
At this point you might as well blame the average guy for global warming...
The average guy is exactly the person responsible for global warming. The evil of the world is just the meta accumulation of the average person following their mirco incentives.
Where I'm from, it probably would not be stolen by anyone.
Where do you draw the line?
Ready to do anything for money as long as it seems legal-ish or your ass is covered by hierarchy?
If something should not be done: make it illegal. Trying to have a gentlemen's agreement not to do something seems like a futile position.
Having you own morale and ethics is far from futile. Each individual should be able to question the law and object taking part in something they don't agree, as long as it doesn't break the law.
Killing someone is legal in certain countries for different reasons (I'm not talking about war). Not sure I would like to get involved in that business, for instance if I don't agree on how and why people are sentenced to death in my country.
Some people are built with low ethics. Sure, if it's not made illegal, they'll always find someone to do it. Looks like in that case it might be illegal, as TV makers are sued.
Yeah, there are reasons why "someone is going to do it anyway" is a classic example of an ethically unsound argument.
It isn't ethically unsound. It's a commons/coordination problem. What is the optimal strategy in infinite-round prisoners dilemma with randomized opponents? The randomization effectively makes it an infinite series of one-round prisoners dilemma. So the best strategy is always to defect.
The only way you can change this is very high social trust, and all of society condemning anyone who ever defects.
If morality never factors into your own decisions, you don't get to be upset when it doesn't factor into other peoples'. In other words, society just sucks when everyone thinks this way, even if it true that resolving it is hard.
This is called a “replacement excuse”. It’s a hallmark of nihilists and utilitarians, but I tend to prefer the more prosaic group noun, “jerks”.
This is an intellectually and morally deficient position to take. There is no moral principle in any system anywhere in the history of the universe that requires me to bind myself to a contract that nobody else is bound to.
We can all agree, as a society, "hey, no individual person will graze more than ten cows on the commons," and that's fine. And if we all agree and someone breaks their vow, then that is immoral. "Society just sucks when everyone thinks this way" indeed.
But if nobody ever agreed to it, and you're out there grazing all you're cattle, and Ezekiel is out there grazing all his cattle, and Josiah is out there grazing all his cattle, there is no reasonable ethical principle you could propose that would prevent me from grazing all my cattle too.
> There is no moral principle in any system anywhere in the history of the universe that requires me to bind myself to a contract that nobody else is bound to.
Is there not? I don't feel this makes sense to me, as the conclusion seems to be "if everyone (or perhaps a large amount of people) do it, then it's not immoral". My immediate thought goes to moral systems that universalise an action, such that if everyone did it and it makes the world worse, then it's something that you should not do. That would be an example of a system that goes counter to what you say. Since morals are personal, you can still have that conclusion even if other people do not subscribe to the same set of moral beliefs that you have. Something can be immoral to you, and you will refuse to do it even if everyone else does.
> But if nobody ever agreed to it [...] there is no reasonable ethical principle you could propose that would prevent me from grazing all my cattle too.
Why not? I don't quite understand your conclusion. Why could the conclusion not be "I feel what everyone else is doing is wrong, and I will not do it myself"? Is it because it puts you at a disadvantage, and you believe that is unfair? Perhaps this is the "reasonable" aspect?
Your confusion is understandable. The way the terms "moral" and "ethical" are thrown around is sloppy in most vernacular. Generally, ethics refers to system-wide morality. E.g., I may feel that personal morality compels me to offer lower rates to clients, even though a higher rate may be acceptable under legal ethics. I tried to make that distinction clear in my post ("moral principle in any system") but perhaps I didn't do a good enough job.
The original poster was not referring to individual moral feelings, but to formal ethical systems subject to systematized logical thinking: "classic example of an ethically unsound argument."
There is no religious tradition, no system of ethics, no school of thought in moral philosophy, that is consistent with that position. The closest you might come is Aristotelian virtue ethics. But it would be a really strained reading that would result in the position that opting out of commons mismanagement is required. Aristotle specifically said that being a fool is not a virtue. If anything, a virtue ethics lens would compel someone to try to establish formal community rules to prevent the tragedy of the commons.
I think this argument would justify slavery: no one (white people) has decided that holding others as slaves is bad, therefore I can hold slaves.
But let me entertain it for a moment: prior to knowing, e.g., that plastics or CO2 are bad for the environment, how should one know that they are bad for the environment. Fred, the first person to realize this would run around saying "hey guys, this is bad".
And here is where I think it gets interesting: the folks making all the $ producing the CO2 and plastics are highly motivated to say "sorry Fred, your science is wrong". So when it finally turns out that Fred was right, were the plastics/CO2 companies morally wrong in hindsight?
You are arguing that morality is entirely socially determined. This may be partially true, but IMO, only economically. If I must choose between hurting someone else and dying, I do not think there is a categorically moral choice there. (Though Mengzi/ Mencius would say that you should prefer death -- see fish and the bear's paw in 告子上). So, to the extent that your life or life-preserving business (i.e. source of food/housing) demands hurting others (producing plastics, CO2), then perhaps it is moral to do so. But to the extent that your desire for fancy cars and first class plane tickets demands producing CO2...well (ibid.).
The issue is that the people who benefit economically are highly incentivized to object to any new moral reckoning (i.e. tracking people is bad; privacy is good; selling drugs is bad; building casinos is bad). To the extent that we care about morality (and we seem to), those folks benefitting from these actions can effectively lobby against moral change with propaganda. And this is, in fact, exactly what happens politically. Politics is, after all, an attempt to produce a kind of morality. It may depend on whom you follow, but my view would be that politics should be an approach to utilitarian management of resources, in service of the people. But others might say we need to be concerned for the well-being of animals. And still others, would say that we must be concerned with the well-being of capital, or even AIs! In any case, large corporations effectively lobby against any moral reckoning against their activities and thus avoid regulation.
The problem with your "socially determined morality" (though admittedly, I increasingly struggle to see a practical way around this) is that, though in some ways true (since society is economics and therefore impacts one's capacity to live) is that you end up in a world in which everyone can exploit everyone else maximally. There is no inherent truth in what the crowd believes (though again, crowd beliefs do affect short-term and even intermediate-term economics, especially in a hyper-connected world). The fact that most white people in the 1700s believed that it was not wrong to enslave black people does not make that right. The fact that many people believed tulips were worth millions of dollars does not make it true in the long run.
Are we running up against truth vs practicality? I think so. It may be impractical to enforce morality, but that doesn't make Google moral.
Overall, your arguments are compatible with a kind of nihilism: there is no universal morality; I can adopt whatever morality is most suitable to my ends.
I make one final point: how should slavery and plastics be handled? It takes a truly unfeeling sort of human to enslave another human being. It is hard to imagine that none of these people felt that something was wrong. Though google is not enslaving people nor are its actions tantamount to Nazism, there is plenty of recent writing about the rise of technofascism. The EAs would certainly sacrifice the "few" of today's people for the nebulous "many" of the future over which they will rule. But they have constructed a narrative in which the future's many need protection. There are moral philosophies (e.g. utilitarianism) that would justify this. And this is partially because we have insufficient knowledge of the future, and also because the technologies of today make highly variable the possible futures of tomorrow.
I propose instead that---especially in this era of extreme individual power (i.e. the capacity to be "loud" -- see below)---a different kind of morality is useful: the wielding of power is bad. As your power grows, so to does the responsibility to consider its impact on others and to more aggressively judge every action one takes under the Veil of Ignorance. Any time we affect the lives of others around us, we are at greater risk of violating this morality. See eg., Tools for Conviviality or Silence is a Commons (https://news.ycombinator.com/item?id=44609969). Google and the tech companies are being extremely loud, and you'd have to be an idiot to see that it's not harmful. If your mental contortions allow you to say "harm is moral because the majority don't object," well, that looks like nihilism and certainly doesn't get us anywhere "good". But my "good" cannot be measured, and your good is GDP, so I suppose I will lose.
It is definitely ethically unsound and it is definitely a common example even related to Nazis. Similar to "just following orders". Which I'll remind everyone, will not save you in a court of law[0]...
You are abdicating your own moral responsibility on the assumption of a deterministic reality.
The literal textbook version of this ethical issue, one you'll find in literally any intro to ethics class is
Sometimes a variant will be introduced with a direct acknowledgement of like donating 10% of your earnings to charity to "offset" your misgivings (ᶜᵒᵘᵍʰ ᴱᶠᶠᵉᶜᵗᶦᵛᵉ ᴬˡᵗʳᵘᶦˢᵐ ᶜᵒᵘᵍʰ).But either way, it is you abdicating your personal responsibility and making the assumption that the job will be done regardless. But think about the logic here. If people do not think like you then the employer must then start offering higher wages in order to entice others. As there is some function describing people's individual moral lines and their desire for money. Even if the employer must pay more you are then helping deter that behavior because you are making it harder to implement. Alternatively the other person that does the job might not be as good at the job as you, making the damage done less than had you done the job. It's not hard to see that often this will result in the job not even existing as truthfully these immoral jobs are scraping the bottom of the barrel. Even if you are making the assumption that the job will be done it would be more naive to assume the job is done to the same quality. (But kudos on you for the lack of ego and thinking you aren't better than other devs)
[0] https://en.wikipedia.org/wiki/Superior_orders
Most of those convicted at the Nuremberg trials eventually had their sentences commuted and only served a fraction of their time. Only a few were convicted and executed. Justice rarely prevails.
[dead]
Objectively incorrect. There is no reasonable argument that it's ethically unsound. The fact that you immediately Godwin'd should have been your first clue.
Considering you're a military lawyer I'm absolutely certain you've heard this example before and its connection to Nazi Germany. I'm not dating anyone is a Nazi for making that argument, but it is a classic example when pointing to how Ordinary Men can do atrocities. And no, I didn't make a grammatical mistake there.
> will not save you in a court of law
Not in the USA. LEO or ICE - or even some judges misuse and never are punished. Qualified immunity.
Moral is different story. Too many people in HN work in Google or Apple. That by itself if immoral.
Some doesn't change the law.
You're right to push back in case I intended something different. But I'll state this clearly: those LEO, ICE agents, and judges are committing crimes.
But the fact that not all criminals are punished or prosecuted does not change the laws either.
What I'm concerned about is people becoming disenfranchised and apathetic. Dismissing the laws we have that does punish LEO, ICE agents, and judges for breaking the laws. To take a defeatist attitude. Especially in this more difficult time where that power is being abused more than ever. But a big reason it is being able to be abused is because a growing apathetic attitude by people. By people giving up.
So I don't know about you and your positions. I don't know if you're apathetic or invested. All I know is a random comment from a random person. It isn't much to go on. But I hope you aren't and I hope you don't spread apathy, intentionally or not.
Care to articulate them?
If you want a consequentialist answer:
If, for ethical reasons, fewer people were willing to take these jobs, then either salaries would have to rise or the work would be done less effectively.
If salaries rise, the business becomes more expensive and harder to scale. If effectiveness drops, the systems are less capable of extracting/using people’s data.
Either way, refusing these jobs imposes real friction on the surveillance model.
If you want a deontological answer:
You have a responsibility not to participate in unethical behavior, even if someone else would.
The fact that it can be used to "justify" almost anything. It obviously doesn't work as a defense in the court, and neither does it work as a justification for doing legal but unethical things.
Soooo.... Why did you build it for them? You didn't have to further enable it. Despise people who just drop this kind of thing without any hint of repentance or contrition.
Would love to know what are the best things we can do to prevent this sort of tracking in general. PiHole? Don't re-use emails? On a scale of 1 to fucked are we cooked?
I don't think they mean that kinda streamer - the idea is the roku tv can tell you're watching an ad even if it's on amazon prime, apple tv, youtube, twitch, wherever, and associate the ad watching with your roku account to potentially sell that data somehow?
That way they aren't cut out of the loop by you using a different service to watch something and still have a 'cut'.
It'd make sense if they're using streamer in a different sense than I'm used to. I see that's at the bottom of the definitions Google will produce.
Yeah I think they mean "user of a streaming service" here, which would more conventionally be user or watcher or so on.
The actual screenshot isn’t sent, some hash is generated from the screenshot and compared against a library of known screenshots of ads/shows/etc for similarity.
Not super tough to pull off. I was experimenting with FAISS a while back and indexed screenshots of the entire Seinfeld series. I was able take an input screenshot (or Seinfeld meme, etc) and pinpoint the specific episode and approx timestamp it was from.
> The actual screenshot isn’t sent, some hash is generated from the screenshot and compared against a library of known screenshots of ads/shows/etc for similarity.
this is most likely the case, although there's nothing stopping them from uploading the original 4K screengrab in cases where there's no match to something in their database which would allow them to manually ID the content and add a hash or just scrape it for whatever info they can add to your dossier.
I thought that similar inputs do not give similar hashes..but apparently that is cryptographic hashing. Locality-Sensitive Hashing methods (e.g. Perceptual hashing[1]) makes similar inputs have similar hashes.
[1] https://en.wikipedia.org/wiki/Perceptual_hashing
Ah, bingo, yes!
I should have been more specific in my comment. Perceptual hashing allowsfor higher similarity scores between similar looking images.
Lots of cool techniques to experiment with. Highly recommend playing around if you’re interested.
I immediately did a little exploration for potential utility in neuroimaging analyses...not that anything was immediately obvious to me, but I love learning about things like this.
I assume these systems are calculating an on device perceptual hash. So not that much data needs get flown back to the mothership.
That's the thing about scaling; you offload the work to the "client" (the TV in this case) and make it do the work, it need not send back more than a simple identifier or string in an API call (of course they'll send more), so they get to use a little bit of your electricity and your TVs processing power to collect data on you and make money, with relatively little required from them, other than some infra to handle the requests, which they would have had anyway to collect the telemetry that makes them money.
Client side processing like this is legitimate and an excellent way to scale, it just hits a little different when it's being used for something that isn't serving you, the user.
source: backend developer
Confirming how many people actually seen the ad is worth big bucks. No one wants to pay for ads they cannot confirm and publisher can make up impressions - if you can catch publisher making up numbers you might get a huge discount or loads of money back.
Not necessarily, it can be done on-device, the screenshot hashed, and the results deduplicated and accumulated over time, then compressed and sent off in a neat package. It'd still be a huge amount of data when you add it all up, but not too different from the volume that e.g. web analytics produces.
Then server-side the hash is matched to a program or ad and the data accumulated and reduced even further before ending up in someone's analytics dashboard.
Are there video "thumbprints" like exists for audio (used by soundhound/etc) - IE a compressed set of features that can reliably be linked in unique content? I would expect that is possible and a lot faster lookup for 2 frames a second. If this is the case, the "your device is taking a snapshot every 30 seconds" sounds a lot worse (not defending it - it's still something I hope can be legislated away - something can be bad and still exaggerated by media)
There are perceptual hashing algorithms for images/video/audio (dsp and ML based) that could work for that.
Given that the TV is trying to match one digital frame against another digital frame, you could probably get decent results even with something super naive like downsampling to a very low resolution, quantizing the color palette, then looking for a pixel for pixel match.
All this could be done long before any sort of TV-specific image processing, so the only source of "noise" I can think of would be from the various encodings offered by the streaming service (e.g. different resolutions and bitrates). With the right choice of downsample resolution and color quantization I have to imagine you could get acceptable results.
That's basically what phash does
I've been led to believe those video thumbprints exist, but I know the hash of the perceived audio is often all that is needed for a match of what is currently being presented (movie, commercial advert, music-as-music-not-background, ...).
This is why a lot of series uploaded to YouTube will be sped up, slowed down, or have their audio’s pitch changed; if the uploader doesn’t do this, it gets recognized by YouTube as infringing content.
You only need to grab a few pixels or regions of the screen to fingerprint it. They know what the stream is and can process it once centrally if needed.
Is this what these sort of companies are doing?
In a word yes. Here is a starting point.
https://arxiv.org/abs/2409.06203
Attribution is very painful and advertisers will pay lots of money to close that loop.
Is it? I don’t think you need particularly high fidelity to fingerprint ads/programs.
it's hashed on the tv then they compare hashes in aggregate
[dead]