Dystopian Surveillance Is Becoming a Reality
dallincrump.com181 points by speckx 18 hours ago
181 points by speckx 18 hours ago
Many geeks are the embodiment of the "don't trust the government" / "don't trust corporations" counterculture, but when equity grants entered the equation, we architected much of the modern techno-surveillance state. Flock is an YC company. Probably a quarter of the people who frequent this site work at Meta or Google.
We're always eager to say how much we dislike it, but there's so little self-reflection in our community.
Tech companies have been monetizing our privacy for a long time.
Google's ex-CEO Eric Schmidt in 2009:
"If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place. If you really need that kind of privacy, the reality is that search engines—including Google—do retain this information for some time and it's important, for example, that we are all subject in the United States to the Patriot Act and it is possible that all that information could be made available to the authorities."
25% of hacker news readership works at meta or google? You mean 25% of meta and google reads Hn, no?
A regular person who happened to be smart (and possibly hardworking) has to choose between $100k regular software engineering job at a company with low social impact or $400k at a cream-of-the-crop company, which will further add on the specialist's value by the virtue of having this company's name on their CV. This is a life-changing amount of money which compounds over the years, where a person can genuinely start thinking about early retirement, especially if they don't like the corporate culture.
Why should society allow individuals have to make such difficult choices between being principled or rich? It's the government's job to control. And I don't mean salary control, but to understand why there are such differences and work to mitigate them.
Having principles is about making choices that are difficult, not about making choices that are easy. "I didn't really want to build a panopticon, why didn't the government stop me" is not a particularly sympathetic position to take (even if true).
I think you are simplifying it a tad bit. Having principles means having principles. On principle, I don't allow some types of devices in my household. That decision is not difficult, but it is simple. The decision parent describes is not difficult in the sense an individual may easily arrive of what is 'right', but it is absolutely not simple to disentangle oneself.. especially once you do have some year history to back you. Parent was pointing to that. Your dismissal is easy, but it simply shows you never had to choose.
"Many geeks are the embodiment of the "don't trust the government" / "don't trust corporations" counterculture, but when equity grants entered the equation, we architected much of the modern techno-surveillance state."
Is that really "embodiment"
Sounds like the opposite
People in the practice of abusing blind trust (the people generally are never seen; they fear being publicly named)
Did everyone read yesterday's Brian Cantrill anecdote
Corporations like Google and Meta, and most other so-called "tech" companies, _assume_ trust
Getting trust for free, never earned. "Don't trust corporations". Except for "Big Tech". WTF
"Don't trust the government" is laughable when the person works for a goverment contractor
No such thing as integrity in SillyCon Valley
> Probably a quarter of the people who frequent this site work at Meta or Google.
No, lol
That would make up between 0.3% and 0.5% of the global tech workforce, if you added all of their employees together.
HN is not a forum for the global IT workforce. It's a portal attached to a VC firm, populated to a large extent by people in the SF Bay Area who work in big tech or various VC-funded startups.
I've gotten some traffic from HN before and if I recall correctly, about 30% of it was from California, followed by Washington (Seattle) and then NY.
> HN is not a forum for the global IT workforce.
What things are intended to be and what they become are often vastly different things.
There's huge amount of cross-pollination on HN, even from non-IT industry participants.
i mean do you want the honest answer? i’m not a white american guy so i can say it.
importing hordes of foreign tech workers or hiring asians broadly whose culture is very much the opposite of “don’t tread on me” led to much of this.
might not be comfortable to say but most of our tech companies are led by people who aren’t “Americana” (cultural)
and broadly speaking, there are very few people like me (i’m asian) that value the American way of independence at all cost and most are just strivers and consumerist
so the goal for most of them is to make 300k base and 300k stock per year. fuck the consequences.
the days of making tech for the love of the game are long gone.
Shame that people are down voting you, there's an uncomfortable truth to it I don't think people want to face.
Culture is defined by people not borders- simply crossing an arbitrary line doesn't change someone's beliefs and values.
The immigration boom especially on the West coast has vastly diluted assimilation into American culture- coming from an immigrant, who married into another family of immigrants- too many people move here and stick to their own cultural enclave, building a version of their home country in miniature instead of trying to integrate.
the problem isn't the workers. plenty of "white american guys" with a "don't tread on me" culture share this goal. scapegoating "hordes of foreign tech workers or hiring asians broadly" ignores that the SV culture rewards this.
> so the goal for most of them is to make 300k base and 300k stock per year. fuck the consequences.
"What could be more American than that?" — Wadsworth, Clue (1985)
What could be more American than a foreign born mafia engaging in human trafficking?
The fascinating part is that fuck the consequences was until more recently kinda penalized, but advent of tech overlords changed that calculus ( they even have their own saying to capture that 0.
yes and it wasn’t always that way. most of these trends of the “god like” founder are maybe from the past 30 years.
and it reeks of extractive personalities that took over
> I don’t like the idea that anything I say in a public setting or in a conversation with others could end up in a transcript summary on someone’s personal device.
Pretty minor if it's just that individual. But for certain this means there is a transcript on Apple's servers that can be subpoenaed by law enforcement, or sooner or later, searchable without a warrant.
It certainly will _not_ be just for that individual.
Believing in that would be like believing web tracking is only ever used to "enhance your browsing experience".
No it’s bad even for an individual why does someone I speak to need to ever have a transcript of our chat? I understand in a professional setting but if we’re just having a drink its weird
One intriguing use I encountered was a doctor who automatically transcribed the conversation during an appointment. This has HIPAA concerns, of course, and I think (hope, at least) that this was in the context of medical software that is designed to be HIPAA compliant. But it was an interesting way for a doctor to try and avoid missing something in conversation.
I agree with you that it's weird outside of professional settings.
I try to record many specialist professionals (after asking first).
Experts know what they are saying, and they often rapidly download dense technical information. I first noticed the problem after asking a friend for advice on handling a psych issue with a friend's child.
Otherwise experts talk and I listen and I understand, but I can easily forget nuance and detail.
I also learnt to record medical specialists talking to my mum, so I could replay the details to my dad.
Why would dad trust my recall? And will he trust facts if I've repeated them as a non-specialist? And perhaps dad picks up different aspects due to his own experiences.
Because having a chat with you was special and I want to remember it but my aging fallible human brain is going to forget it, so I'd like to be able to remember it and hold on to and cherish that moment we shared.
"I want to replace the human experience with SQL queries" in a nutshell
if chats are so important to you, write it down, with your hand. journaling is fun and great and there's evidence that it's good for your mental health because it allows you intentional time for introspection and insight-building
You're definitely going to forget it if you don't have to make any effort to remember it.
Then write a note, stop recording me.
If you can't be bothered, it wasn't that meaningful or important.
Yeah, even with Meta glasses like those things record everything that the user sees, but the flock camera situation is really concerning their cameras are everywhere and I think they can also hear what people are saying
There was a time when I was very, “taking a photograph in public of a publicly visible thing is legal and should remain so.”
Now I’m kinda feeling like this Meta glasses shit should be straight-up illegal. These really aren’t for people doing art or documenting civil rights violations or anything.
I realize it makes me a kind of hypocrite, but it’s all so creepy now.
They would have to encrypt it in such a way where even if Law Enforcement went to apple with a subpoena, they would not be able to hand over that data at a technical level.
The better way IMHO is use local models so data never actually leaves the device.
We really need to figure out how to encrypt everything on device with keys controlled by the owner of the device.
Entirely technically feasible, and has been for years and years..
..but impossible to have unless you control the operating system that is distributed with phones that normal people will use.
"We"?
Yeah, we, the software developers.
Any recordings should be kept in something akin to password manager.
Heck, all of my data in any app should be kept in something like that.
Problem is, that's a hardware problem. Try asking Apple nicely to open their ecosystem; the appropriate expected response is, bring your own phone.
That being said, I do actually expect that this stuff will (generally*) stay on-device. Apple's oddly good about that.
The legal implications of these devices are just baffling. I see Meta glasses being sold where I live. The law states that you cannot photograph of film people without their consent, even in the street where there are other people around ...
Now it will also be voice recording one has to be aware off ?
Is that the AI future we want or need?
> The law states that you cannot photograph of film people without their consent, even in the street
What country do you live in? This is almost certainty not true in the US
Likely germany, we have very strict recording laws. It’s actually stricter on audio recording, due to Germany’s history: you’re not allowed to record people without their consent if they could expect privacy, which includes outside when for example 2 persons are talking to each others privately. That makes the meta glasses pretty much illegal to use in most situations, though I don’t believe that specific situation has been tested in court.
Note how it’s about recording, not sharing.
(It’s way more nuanced and complicated in practice of course)
From that same link: "Two-party consent laws apply to private conversations where participants have a reasonable expectation of privacy."
You have no reasonable expectation of privacy when you are in public or can be clearly heard from public. If you are "in the street" no one needs to get your permission to record you or what you say.
The solution is legal and political, not technical.
We need all of the above, because it's too easy for the political winds to change and modify the laws to enable mass surveillance with a flip of a switch. It needs to be both illegal and technically hard to surveil people without their knowledge.
The solution is all of the above and more.
Every big problem needs to be addressed from multiple angles, whatever you can do, you can contribute to the solution.
As a Brit I hope Americans understand just how amazing the first amendment is. Here in the UK (and Europe) surveillance can be used to criminalise us for things we say in private. At least in the US this kind of thing is much harder for the government to do.
I don't know about UK, but everything in the U.S constitution's first amendment is most definitely also in Estonia. I would pretty confidently say also most of EU, probably. They are pretty basic things.
Cell phones are so useful that I'm not surprised that people are willing to bug themselves and their homes with them, but wearables offer so much less while being just as invasive. I suspect that people will take the bait anyway. I don't see very many "glassholes" running around though, and there was a lot of pushback against "friend" https://en.wikipedia.org/wiki/Friend_(product)
The average person doesn't like being recorded without their consent. That is why they've come up with epithets like "glasshole" for people using wearables where it's very in-your-face and obvious that you're being recorded. It's also why we see as much widespread vandalization against Flock cameras.
The problem is: it's not obvious to most people your cell phone (or TV, or car, etc.) would start recording you without your consent or knowledge. It's only just starting to become common knowledge that your phone's location data is being captured and shared with everyone.
It's interesting to see how people behave so differently when it's covert. When they can't see the person behind the camera it's as if becomes less real to them. I've seen videos of people screaming at someone for filming or taking photos in public while standing directly under surveillance cameras.
Even after being told about the cameras around them or the fact that every Tesla that drives past them is also recording they're still convinced the threat is the one person holding a camera. It just never occurs to them that the street photographer is not only less likely to use what they capture against them, but is also far less capable of causing harm even if they wanted to.
At this point, and I assure you I am not being contrarian -- you can check my post history, I would say just enjoy the ride man.
You can always assume permanent surveillance. Electromagnetic radiation can be read from a distance so any device is completely visible.
You’re obviously not a radio guy. Attenuation and interference are real. Many devices are adequately shielded from all but the most determined and skilled eavesdroppers. And not all equipment leaks useful information in its emissions.
If you’re really paranoid, you can build yourself a small Faraday room. It’s not that difficult. It’s also sometimes possible to buy used telco equipment sheds that come pre-shielded, they are popular with radio nerds.
> I don’t like the idea that anything I say in a public setting or in a conversation with others could end up in a transcript summary on someone’s personal device.
If it's only that though, how is it different from someone just writing down what you said?
Just like running your own mail server doesn't help when nearly everyone you talk to uses GMail, simply not using these devices yourself is likely not to make much difference.
We just have to make it the cool thing to run your own mail server. B-)
I think it's cool, and completely necessary. No need to be compatible with old email, really.
"Audio Intelligence features are private by design, combining hardware, software, and services to protect user privacy in a way that only Apple can. These features do not create or store audio recordings, and raw audio used for processing is completely inaccessible to the operating systems, apps, the user, or Apple. By design, Audio Intelligence features do not identify and attribute speakers, to protect the privacy of both the user and those around them. Siri Recap produces a brief, high-level summary of conversations — not a transcript — and is designed to exclude sensitive information like financial information or government-assigned identifiers. Live Rewind text snippets and Siri Recap summaries are end-to-end encrypted in the Siri app with iCloud syncing, and are not accessible even by Apple. Read more about how Audio Intelligence protects privacy:" https://news.ycombinator.com/item?id=49632685
The whole point of the debate is that "those around them" didn't agree to send their so-called "de-identified" data to Apple. In fact, "those around them" had no chance to agree/disagree at all.
Notice how they focus on “audio recordings” specifically, meaning other types of recording (like transcripts used to generate summaries) are fair game.
"We know our product is an unlawful violation of federal wiretap law since it will make zero-party-consent recordings, not that we care about violating the law at Apple because we know we can out spend anyone who would litigate against us, but we like to keep up appearances."
I've experimented with things like this, e.g., the Narrative Clip. I think the concept is quite novel and interesting, but the privacy implications are just far too gross for anything outside of an experiment. Of all companies, I find it appalling that it's Apple implementing this one.
I don't even know how this is supposed to work e.g., in states with call recording legislation, let alone any new legislation. It feels utterly appalling that every Apple watch could theoretically be recording all surroundings now.
I watch "The First 48" religiously and its amazing how they go directly to Flock data in every new episode.
there is no "becoming", it has been a fact for long time, and expect it to get a lot worse.
"The primary antagonist for freedom of thought in the beginning of our struggle was the Universal Catholic Church, an institution directed at the control of thought in the European world, based around weekly surveillance of the conduct and thoughts of every human being, based around the censorship of all reading material and in the end based upon the ability to predict and to punish unorthodox thought.
The tools available for thought control in early modern Europe were poor even by 20th century standards, but they worked. And for hundreds of years, the struggle primarily centered around that increasingly important first mass manufactured article in Western culture: the book. Whether you could print them, possess them, traffic in them, read them, teach from them without the permission or control of an entity empowered to punish thought.
By the end of the 17th century censorship of written material in Europe had begun to break down first in the Netherlands then in the UK then afterwards in waves throughout the European world and the book became an article of subversive commerce and began eating away at the control of thought.
By the late 18th century that struggle for the freedom of reading had begun to attack the substance of Christianity itself and the European world trembled on the brink of the first great revolution of the mind. It spoke of “liberté, égalité, fraternité”, but actually it meant freedom to think differently." -- https://archive.org/details/EbenMoglenRepublica2012
1960: "I have a great idea! lets have every person in the country carry a radio tracking beacon!" "That'll never fly!" 2012: "I can has TWO iphones??"
"Becoming"? Lol it's been a reality for a long time. It's just ramping up now.
I really recommend Limes Inferior written by Andrzej Zajdel. I hope it is possible to read it in English. It describes a dystopian state that is currently coming to us.
>enabled could potentially be recorded without their knowledge or consent.
The audio is not recorded.
A realtime transcript—even abbreviated—is a recording.
Legally they are not the same thing.
Depends on the jurisdiction. No matter how you slice it's it's an interception, it may be a transmission (to the iPhone if nowhere else), and there is an argument to be made that the text extracted is a recording.
It is in no way settled law especially in places with 2 party consent laws like Florida or California. Nevermind Oregon. Or the Europe. A cursory search suggests France's laws have some teeth to them.
It also doesn't address the inadvertent listening problem.
It may not be settled case law, but I'm going to trust Apple's lawyers on this one.
The core issue with mass, high detail survellance is 1) the chilling effect and 2) the construction of guilt effect
When every single thing you do is put under a microscope you will be found guilty of nearly anthing. It will be possible to construct a believable number of crimes that you commited from circumstantial evidence and/or the pile of minor offenses citizens commit every day.
The american system would defacto flip into "you are guilty, we just need to find out of what." and this becomes a way to control and witchhunt a population into silence and complaince.
The whole modern world is dystopian garbage.
nuh uh, reality is distopian
we have entropy to fight, so the margins have always been small,and with narso/nialists running things it gets nasty while they seek scapegoats for the contradictions they have created
[dead]
[dead]
[dead]
Surveillance isnt a problem until people use it for personal gain
The Fourth Amendment to the US Constitution would seem to disagree.
Fourth Amendment of the US Constitution protects only U.S. citizens inside U.S.
NSA is allowed to collect any information about non-U.S. citizens outside U.S. and there are some limitations to collection of information about U.S. citizens outside U.S.
https://en.wikipedia.org/wiki/Foreign_Intelligence_Surveilla...
I wasn't replying to a question of jurisdiction. It was a statement that surveillance is fine until there's personal gain involved. That is not consistent with the text or intent of the amendment. Other countries also have rights protected against unreasonable searches and seizures. Some even explicitly protect privacy.
The US Constitution applies to people in the US, by the way, not just citizens.
> Surveillance isnt a problem until people use it for personal gain
It absolutely can be a problem when used for organizational or ideological gain - e.g., when used by state, religion, business, or political group to spread propaganda, suppress opposition, unjustly favor an in-group, etc.
The classic case of using mass surveillance to help a burgeoning theocratic or fascist state is a pretty standard example.
Yes but here is the catch. Surveillance comes with the incentives to us it for the personal gain. Or to phrase this differently: knowledge is power and thus the people who are already in power have good motivation to accumulate as much knowledge as possible in order to stay in power.
It is very easy to take the philosophical stance where every concept can be totally detached from the incentives it creates and the social dynamics it is part of. It is easy to take that stance, but it is very hard to use that stance to explain reality, because reality plays by more complex rules.
Please, always consider the status quo and the incentives involved instead of stating the surveillance equivalent of "weapons don't kill, people do".
> I don’t like the idea that anything I say in a public setting...could end up in a transcript summary on someone’s personal device.
My friend, if there's a reporter standing behind you, it could end up on the nightly news. And it's been that way forever. Certainly, a plethora of terrifyingly broad data collection greatly exacerbates the problem, but it's not a new problem.
> bla bla bla...Amazon Echo...listening in on us without our knowledge or consent. I've lost track of how many times I've experienced or heard others...discussing a particular topic...and then as soon as they get on Facebook...they see posts and ads featuring the very topic they were discussing.
There are lots of real problems with surveillance tech, but the stuff about smart speakers specifically is still a dumb conspiracy theory. People are CONSTANTLY testing the hardware in these things and watching network activity. They really are not listening to you, and blindly repeating this claim greatly harms your credibility.
> bla bla bla...George Orwell envisioned in his dystopian novel 1984
Okay, new rule, these articles need to pick at least one other dystopian novel. There are so many. What's wrong with Fahrenheigh 451, where the government had everyone look out their doors in a mass manhunt? That's a good one! Use it to talk about Ring cameras or something.
>My friend, if there's a reporter standing behind you, it could end up on the nightly news. And it's been that way forever.
Can't tell if this is meant to be ironic or not. Are you seriously comparing a future consumer product to be potentially sold in the hundreds of millions with TV news reporters?
That's the most HN spin I've seen this year
> My friend, if there's a reporter standing behind you, it could end up on the nightly news
It's a matter of scale. Now everybody with a device is a reporter.
It’s been 20 years since first iphone, this is nothing new. And somehow people are not recording everyone all the time.
What is new is that now people will be recording everyone all the time, as a default.
That’s simply not possible with the announced Apple products. Afaict, Apple will let people save *transcripts* of last 15s and save *summaries* of transcripts.
Live streamers make a habit of doing this and are widely hated for it.
Exactly, they are *hated* for it. Afaik, I haven’t met anyone live streaming in my boring real life.
Re smart speakers / Google Assistant etc I think there have even been some lawsuits about inadvertent recording. I think they get it right most of the time, so my impression has been the lawsuits are about the mistakes.
That all said, as a technologist in the past I could say "of course they aren't recording you and data mining your conversations" because that's obviously an expensive and complicated proposition when much cheaper forms of data mining exist. But with AI advancements I think that calculus could change - the systems that could do this are already being built for other reasons; the main reason to not do it is cost but that's being eroded.
Performative nitpicking is tiresome, why equivocate? Either you agree with the thrust of the argument and are looking to strengthen it, or you disagree. Which is it?
> Certainly, a plethora of terrifyingly broad data collection greatly exacerbates the problem, but it's not a new problem.
I argue that this is a false equivalence. When the scale of something changes by an order of magnitude (or multiples thereof), it’s no-longer the same problem/circumstance.
> the stuff about smart speakers specifically is still a dumb conspiracy theory
We know for certain that they do listen to us even when they shouldn't and send that data https://www.techradar.com/news/your-smart-speaker-could-be-a...
We also know that there have been multiple instances of employees and low paid contractors listening to people having sex and talking about very personal things.
I don't think concerns over smart speakers spying on us qualifies as a "conspiracy theory". People looking for a conspiracy theory are better off digging into how this happened https://interestingengineering.com/innovation/amazon-alexa-c...
If our devices were never listening to us in secret things like audio beacons wouldn't have been invented https://arstechnica.com/tech-policy/2015/11/beware-of-ads-th...
Every single time any article about how terrible things are / how its getting worse, someone always pipes in about how "this has always been a problem" - which is a completely useless observation that in practice only has the effect of normalizing the terribleness.