This article reminds me of this excellent tongue-in-cheek piece of writing by Jonathan Zeller in McSweeney's:
Calm Down—Your Phone Isn’t Listening to Your Conversations. It’s Just Tracking Everything You Type, Every App You Use, Every Website You Visit, and Everywhere You Go in the Physical World
There is so much time spent “debunking” audio recordings being shared with various entities it makes me more suspicious.
Just like Facebook’s “we never sell your data (we just stalk you and sell ads using your data)”. I’m sure there’s a similar weasel excuse… “we never listen to your audio (but we do analyze it to improve quality assurance)”
It’s similar with the TSA facial recognition photos. “We delete your photo immediately” but what they don’t say is that they don’t delete the biometrics from that photo.
> There is so much time spent “debunking” audio recordings being shared
Not really. 99% of the time it's someone claiming that it happens.
And it's always an anecdote, never clear proof that it happened. Let alone that it happened because of the audio and not web activity. And that the conversation was actually the cause for the ad and not the other way around.
Is it technically possible? Sure. But if so many people are so certain that it definitely happens, why didn't dozens of people already prove it with a fresh Google/Apple account and phone?
Not saying this is true, but the amount of time and effort put into saying "no one is listening to you" could be attributed to the novel 1984, where the government is actively listening to its citizens. Enough people could associate the novel with government surveillance that it's what people interpret as the most likely surveillance happening - and enough people don't understand tech that it's lost on them that a) the tech to actively listen to millions of people constantly doesn't exist at the appropriate level to be effective b) there are significantly more and far more effective ways to monitor people with current tech than via microphone. It's truly unfortunate people don't understand tech to realize what's actually possible and what is actively happening vs what they imagine could be happening
We don't "listen" to your audio, the microphone does, and your phone transcribes it to text on your device. You cannot listen to text. Therefore we don't listen to your phone audio.
There is a small list of reasons why it needs to be "debunked:"
1. Your phone is gathering data that you don't realize that it gathers.
One of the biggest examples of this is real-time location data that is brokered by cellular carriers and sold as aggregated marketing data. You don't have to give your apps permission to do anything like that because your cellular carrier can get that data regardless of your phone's OS.
2. Your phone is gathering data that you gave it permission to gather, perhaps gathering it in a way you didn't think it would do.
For example, let's say you give an app permission to read your entire photo library so that you can upload a photo. But since you gave it that permission on the OS level, it might be uploading more images than you explicitly select. Another example used to be clipboard data before the OSes asked permission for use of the clipboard. One last example is text that you enter but do not submit.
Another big aspect of this is that people don't realize how these ad networks work in real time. It's not a slow thing for an advertising company to learn something about you and react accordingly, it can happen in a few short seconds.
2. The average person doesn't have any comprehension of how easy it is for data science practices to uncover information about you based on metadata that seems benign or that you don't know exists.
Most people don't understand how your behavior in an app can be used to tell the company things you like and dislike. The TikTok algorithm is a great example, it can tell what you like just by extremely subtle inputs, how you swipe, how long you watch the video. A lot of people don't realize how many things about them aren't particularly unique and how many preferences can be tied to a really specific persona that you fall into.
A real world example of all of this put together is that I was spending a lot of time browsing appliances because I just bought one, and I went to physically visit a friend. We were talking about my new appliance, and later they got ads for that specific appliance. So, the person's reaction would naturally be "it was listening to us!!" but in reality, it is more likely that our cellular carrier or carriers knew we were physically in the same place and reported that piece of information to some kind of data broker. Consider how there are a limited amount of cellular carriers, that location data may not have needed to even exit the cellular carrier to sell this data to someone. I.e., if we both have the same cellular carrier , our company already has that information and it isn't selling it to another company, it's perhaps just telling a data broker that Person A and Person B interact with each other.
Just note that I'm not claiming this is exactly how it all works as I'm not in that industry, but the general ideas here apply. The general takeaway is that literally recording audio with a microphone just isn't necessary to derive hyper-specific things about people.
I can just say that I knew an entrepreneur in early post Y2K who developed apps to track music played in clubs in SF for folks like ASCAP, BMI, and SESAC. They gave out "free" phones (these were the small expensive candybars and nice flip/slideups) to the influencers of the day. They compressed the audio for orthogonality, and had a huge number of hashes to match. If they got more than a few consecutive matching hashes at a location that wasn't paying royalties, they got an enforcement call.
So the idea that it takes a huge amount of computing resources, battery life, permissions, or bandwidth to do matching of keywords is hilarious. That's what "siri", "hey google", "alexa" etc are all doing 24 hours a day. Just add another hundred and report them once an hour. You don't need low latency. It's just another tool in the bag!
Of course the cat food example is bad, because if they weren't looking for that you wouldn't get a response. Who would be willing to pay big for clicks on cat food. Now bariatric surgery? DUI? HELOC? Those pay.
Reminds me of something that a Telco exec once said in jest - “A bank can track which hotel you stayed at last night, the Telco knows who you slept with”
The article omits a real, serious source of microphone data though: your smart TV. I know beyond a shadow of a doubt that my TV (a Toshiba Fire TV, although I’m sure many do it) is listening to every conversation I have within earshot, even when I am not using the voice remote, and selling it to ad networks.
And of course it is also doing screen recognition (the kind of stuff OP article mentions), but that is not what I’m talking about. I’m talking about microphone data picking up live conversation from people in the room.
Privacy-seeking users have physically removed microphones from phones. This should also be possible with laptops and televisions.
If Toshiba Fire TV is related to Amazon Fire TV, then it may include Alexa for voice recognition, which could be optionally disabled. In theory, Alexa is only activated after on-device recognition of the configured wake word.
Way back then I exposed massive data collection from Twitter by Google which made it possible to plot locations at which you used Twitter in Google Maps by simply putting your Twitter handle into the search field. Somehow they knew about these locations even when you opted out of sharing location data with Twitter (I checked) -- so this was only possible by Twitter privately providing this information to Google.
This "experiment" has since then been shut down, but exposing this and many other other forms of activism permanently has cost me my Twitter account, to the point that asking to reinstate it several times because I was permanently suspended for no valid reason led to X Support directly rerouting every attempt to appeal this decision into the digital trash can.
This one used data shared by the user (opt-in on sharing geolocation in the app or browser), which then is publically exposed through the API (like this feature says it would).
Mine doesn't give a shit, geolocation was shared even when turned off by the user in Twitter.
Do note that at first it was assumed just Chrome was involved, but then people started to message me that they also saw it when using the apps, Firefox, Safari and other browsers aswell.
By the way: somewhat later we (thanks to a group effort) figured out it wasn't "just" Chrome as mentioned, and this basically led to the strong assumption there was some serious data sharing involved.
And yes that screenshot from this person is 100% real; my pins for example were sprinkled all across Brighton in the UK near places with Wifi access (I recently went on a city trip there at the time), and my home town in the Netherlands.
Tweets were geolocated, with a 'see tweets near me' page until about 14 years ago, so it's entirely feasible that at least some of that infrastructure has survived the feature being removed.
Doesn't every site route every support request for every reason into the digital trash can? You're supposed to just make a new account, using as many mechanisms as possible to make sure the site can't link it to your old account.
A few years ago I tried to create a separate digital footprint from scratch (just an experiment out of boredom when my isp offered a second number for free). I used an ultra cheap never before used android phone and set it up outside my home.
Google went nuts. All sorts of captchas, security checks and attempts to link me to other information popping up on every step. Eventually it wouldn’t let me use the phone unless I provided a credit card number.
At the time I am typing this, the title on the page is:
""Your phone isn’t secretly listening to you, but the truth is more disturbing""
Which is presently also the title on this post.
Then as I read it becomes clear that it is merely focusing on Facebook.
However the confusion that may stem from
"Your phone isn’t secretly listening to you"
The blog post never attempts to establish that
your phone is not listening to you, just that some
companies may not be going it.
The truth is that your phone may well be listening to you .
There is plenty of malware / spywear that uses exploits
to achieve it.
Like the NSO group¹.
Tools to do so can be bouught on the malware market from other sources
as well and we must assume that Mossad, NSA, and other major intellitence
agencies have tools that exceed what you can buy on the open market.
You phone may aboslutely be listening to you.
but probably it is not.
In aggregate, your phone is not listening to you, but if you are of great interest to a powerful adversary, it very well might be. But at that point, I would wager that's one of the smaller things on your plate.
If you can’t trust the software, why would you trust the software? Am I supposed to rely on the hope that an attacker can take over some part of the OS, but not the one rendering a tiny blob in the status bar?
yeah, I liked the simplicity of having things on my tv, but I gave up and got an apple tv box. I was getting way too many "I was just talking about that!" ads on some of the "free" services i was watching old tv shows and movies on. I'm a pretty frugal guy for the most part but buying a separate box that doesn't sell everything you do and say to advertisers is worth it.
> "Apps were automatically taking screenshots of themselves and sending them to third parties. In one case, the app took video of the screen activity and sent that information to a third party.”
> Out of over 17,000 Android apps examined, more than 9,000 had potential permissions to take screenshots. And a number of apps were found to actively be doing so, taking screenshots and sending them to third-party sources.
Which permission is that, and how do you detect which apps are doing that and stop them?
There is a permission to record the screen. It requires user consent and there's an icon in the status bar while it's being used. It's impossible to use this covertly.
What I believe the article is speaking about, is an app taking screenshots of its own windows. This is obviously possible and obviously requires no permissions whatsoever. Just make a screen-sized bitmap and do
I followed the links to the study they referenced, and it says:
> Unlike the camera and audio APIs, the APIs for taking screenshots and recording video of the screen are not protected by any permission
However they also talk about doing static analysis on 9,100 out of the 17,260 apps, to determine (amongst other things) “whether media APIs are actually referenced in the app’s code”.
They then talk about doing a dynamic analysis to see which apps actually call the APIs (rather than just link to a library that might call it, but the app never calls that function the library).
The soundbite is bad, it shouldn’t say “had potential permissions to take screenshots”, it should just say “had the potential to take screenshots”
I doubt there's a specific "ability to send surreptitious screen shots to developer" permission. It must be a combination of permissions: one for making network connections, another for capturing the screen without making it obvious to the user, etc.
For apps that want to send their own screens to third parties, there's no permission needed or possible. The app is drawing the content to the screen. It knows what the content is.
When it's a developer tool we call it RUM or real user monitoring. It's super useful for solving bugs, but obviously the potential for abuse or user hostile activity is super high.
... and is this permission to take screenshots of anything else you are doing on your phone at any time, or is it permission to take screenshots while you have that app open?
People seem to ignore the cost and accuracy aspects of a phone listening to you 24/7. At least with today’s constraints, it is highly unlikely to be happening.
First, the cost to transcribe audio is not free. It is computationally expensive. Any ad network or at scale service would not be able to afford it, especially in orgs where they are concerned about unit economics.
Secondly, the accuracy would be horrible. Most of the time, your phone is in your pocket and would pick up almost nothing. More over, it’s not like you are talking about anything of value to advertisers in most cases. Google is a money printing machine because people search with an intent to buy. The SNR of normal conversation is much much much lower. That makes the unit economics of doing this gets much worse.
Third, it would be pretty hard to not notice this was happening. Your phone would get hot, your battery would deplete very quickly, and you’d be using a lot of data. Moreover on iOS you could see the mic is being used and the OS would likely kill the app if it was using too many resources in the background.
So until we find an example of this actually happening, it’s not worth worrying about.
For all of these reasons, audio snooping is much more likely to be something done by wired, stationary devices that maybe have a decent amount of RAM + a fair bit of usually-idle processing capacity (to run the transcription model locally and just push the resulting text), and which are expected to draw a decent amount of power and use the Internet at vaguely-arbitrary times.
These are all points that were brought up in the article as to why voice recording is less useful than all of the other tracking mechanisms advertisers have available
While I think that audio recording is not a thing, your economic argument is not complete.
What if only the audio of "high value" targets is recorded. Meaning people who buy a lot of stuff. So it might be worthwhile to only record their sounds. Which will explain why random testing (usually with new/clean phones) is never successful in detecting a recording event.
I think this is a genuine concern for prominent people. Like if you are Mark Zuckerberg, there is material interest in a bad actor installing malware on his laptop. But for a random person where you get low value data that may or may not let you better target some low value ads? That is much harder to justify. Would have to reevaluate as things change and the cost of compute goes down.
Television, not phone, but YouTube sure intrigued me at minimum yesterday. First, it revealed pretty clearly that even with history turned off, it will use the history of other accounts accessed from the same IP to serve recommendations anyway. Without history, it turns off the home page recommendations, but when I ran a search, it showed me completely unrelated videos from a rock climbing channel my wife had watched on another account. I have never watched any rock climbing content on this account.
The second incident was the "listening to you thing," though. Not on the phone, but on a smart television. Exterminator was there to do the quarterly spray of my house and I was showing him scars from when I fell off a skateboard trying to bomb a hill I couldn't handle late last year, talking about what happened, and not five minutes later I turn on the television, open YouTube, and the very first recommendation on my wife's account is a video of a guy falling off his longboard at 50 MPH. Not like it's some kind of secret that we both skate and I watch a lot of downhill videos on this account, but I have never once specifically searched for, watched, or even been recommended a video of a crash, until they decide to do so five minutes after I was talking about it in front of that television.
If what you're talking about is the source of the ad, why did you see the ad yourself? Were you shouting about ear wax removal at your phone?
There are millions of ways the adware running on your phones could've correlated your profile and spread the "infection" to your friend. Basic location access being the most important one, but sharing an IP address (your friends' WiFi?), being near the same Bluetooth beacons, having the same stored SSIDs, or mere coincidence that your friend saw the same ad targeting a wide demographic are much more probable than "my phone is listening 24/7".
She wouldn't because she has much better things to do in life. Matter of fact, its an ad you would never look at, just because you don't even have a need for it.
Readit News
Calm Down—Your Phone Isn’t Listening to Your Conversations. It’s Just Tracking Everything You Type, Every App You Use, Every Website You Visit, and Everywhere You Go in the Physical World
https://www.mcsweeneys.net/articles/calm-down-your-phone-isn...
Just like Facebook’s “we never sell your data (we just stalk you and sell ads using your data)”. I’m sure there’s a similar weasel excuse… “we never listen to your audio (but we do analyze it to improve quality assurance)”
Not really. 99% of the time it's someone claiming that it happens.
And it's always an anecdote, never clear proof that it happened. Let alone that it happened because of the audio and not web activity. And that the conversation was actually the cause for the ad and not the other way around.
Is it technically possible? Sure. But if so many people are so certain that it definitely happens, why didn't dozens of people already prove it with a fresh Google/Apple account and phone?
1. Your phone is gathering data that you don't realize that it gathers.
One of the biggest examples of this is real-time location data that is brokered by cellular carriers and sold as aggregated marketing data. You don't have to give your apps permission to do anything like that because your cellular carrier can get that data regardless of your phone's OS.
2. Your phone is gathering data that you gave it permission to gather, perhaps gathering it in a way you didn't think it would do.
For example, let's say you give an app permission to read your entire photo library so that you can upload a photo. But since you gave it that permission on the OS level, it might be uploading more images than you explicitly select. Another example used to be clipboard data before the OSes asked permission for use of the clipboard. One last example is text that you enter but do not submit.
Another big aspect of this is that people don't realize how these ad networks work in real time. It's not a slow thing for an advertising company to learn something about you and react accordingly, it can happen in a few short seconds.
2. The average person doesn't have any comprehension of how easy it is for data science practices to uncover information about you based on metadata that seems benign or that you don't know exists.
Most people don't understand how your behavior in an app can be used to tell the company things you like and dislike. The TikTok algorithm is a great example, it can tell what you like just by extremely subtle inputs, how you swipe, how long you watch the video. A lot of people don't realize how many things about them aren't particularly unique and how many preferences can be tied to a really specific persona that you fall into.
A real world example of all of this put together is that I was spending a lot of time browsing appliances because I just bought one, and I went to physically visit a friend. We were talking about my new appliance, and later they got ads for that specific appliance. So, the person's reaction would naturally be "it was listening to us!!" but in reality, it is more likely that our cellular carrier or carriers knew we were physically in the same place and reported that piece of information to some kind of data broker. Consider how there are a limited amount of cellular carriers, that location data may not have needed to even exit the cellular carrier to sell this data to someone. I.e., if we both have the same cellular carrier , our company already has that information and it isn't selling it to another company, it's perhaps just telling a data broker that Person A and Person B interact with each other.
Just note that I'm not claiming this is exactly how it all works as I'm not in that industry, but the general ideas here apply. The general takeaway is that literally recording audio with a microphone just isn't necessary to derive hyper-specific things about people.
So the idea that it takes a huge amount of computing resources, battery life, permissions, or bandwidth to do matching of keywords is hilarious. That's what "siri", "hey google", "alexa" etc are all doing 24 hours a day. Just add another hundred and report them once an hour. You don't need low latency. It's just another tool in the bag!
Of course the cat food example is bad, because if they weren't looking for that you wouldn't get a response. Who would be willing to pay big for clicks on cat food. Now bariatric surgery? DUI? HELOC? Those pay.
And of course it is also doing screen recognition (the kind of stuff OP article mentions), but that is not what I’m talking about. I’m talking about microphone data picking up live conversation from people in the room.
If your smart toaster, light bulb, or fridge was listening to you, would anyone even notice? Does anyone examine these devices in depth?
It's like that old Soviet Russia joke, except it's not a joke.
If Toshiba Fire TV is related to Amazon Fire TV, then it may include Alexa for voice recognition, which could be optionally disabled. In theory, Alexa is only activated after on-device recognition of the configured wake word.
This "experiment" has since then been shut down, but exposing this and many other other forms of activism permanently has cost me my Twitter account, to the point that asking to reinstate it several times because I was permanently suspended for no valid reason led to X Support directly rerouting every attempt to appeal this decision into the digital trash can.
Let's say nothing surprises me anymore.
This one used data shared by the user (opt-in on sharing geolocation in the app or browser), which then is publically exposed through the API (like this feature says it would).
Mine doesn't give a shit, geolocation was shared even when turned off by the user in Twitter.
Do note that at first it was assumed just Chrome was involved, but then people started to message me that they also saw it when using the apps, Firefox, Safari and other browsers aswell.
Here is a remnant from someone who replied at the time:
https://xcancel.com/kpcuk/status/601451439215353857
By the way: somewhat later we (thanks to a group effort) figured out it wasn't "just" Chrome as mentioned, and this basically led to the strong assumption there was some serious data sharing involved.
And yes that screenshot from this person is 100% real; my pins for example were sprinkled all across Brighton in the UK near places with Wifi access (I recently went on a city trip there at the time), and my home town in the Netherlands.
A few years ago I tried to create a separate digital footprint from scratch (just an experiment out of boredom when my isp offered a second number for free). I used an ultra cheap never before used android phone and set it up outside my home.
Google went nuts. All sorts of captchas, security checks and attempts to link me to other information popping up on every step. Eventually it wouldn’t let me use the phone unless I provided a credit card number.
""Your phone isn’t secretly listening to you, but the truth is more disturbing""
Which is presently also the title on this post.
Then as I read it becomes clear that it is merely focusing on Facebook.
However the confusion that may stem from "Your phone isn’t secretly listening to you"
The blog post never attempts to establish that your phone is not listening to you, just that some companies may not be going it.
The truth is that your phone may well be listening to you . There is plenty of malware / spywear that uses exploits to achieve it.
Like the NSO group¹.
Tools to do so can be bouught on the malware market from other sources as well and we must assume that Mossad, NSA, and other major intellitence agencies have tools that exceed what you can buy on the open market.
You phone may aboslutely be listening to you. but probably it is not.
¹
https://www.bloomberg.com/news/features/2023-01-24/nso-group...https://www.britannica.com/topic/Pegasus-spywarehttps://citizenlab.ca/2016/08/million-dollar-dissident-iphon...
https://newatlas.com/computers/smartphone-listening-conversa...
https://www.bloomberg.com/news/features/2023-01-24/nso-group...
[0] https://dl.acm.org/doi/10.1145/3646547.3689013
> Out of over 17,000 Android apps examined, more than 9,000 had potential permissions to take screenshots. And a number of apps were found to actively be doing so, taking screenshots and sending them to third-party sources.
Which permission is that, and how do you detect which apps are doing that and stop them?
What I believe the article is speaking about, is an app taking screenshots of its own windows. This is obviously possible and obviously requires no permissions whatsoever. Just make a screen-sized bitmap and do
It does sound believable that third-party advertising/marketing/tracking SDKs, which many apps are chock full of, could be doing this.*Unless there's a zero-day that allows it.
> Unlike the camera and audio APIs, the APIs for taking screenshots and recording video of the screen are not protected by any permission
However they also talk about doing static analysis on 9,100 out of the 17,260 apps, to determine (amongst other things) “whether media APIs are actually referenced in the app’s code”.
They then talk about doing a dynamic analysis to see which apps actually call the APIs (rather than just link to a library that might call it, but the app never calls that function the library).
The soundbite is bad, it shouldn’t say “had potential permissions to take screenshots”, it should just say “had the potential to take screenshots”
First, the cost to transcribe audio is not free. It is computationally expensive. Any ad network or at scale service would not be able to afford it, especially in orgs where they are concerned about unit economics.
Secondly, the accuracy would be horrible. Most of the time, your phone is in your pocket and would pick up almost nothing. More over, it’s not like you are talking about anything of value to advertisers in most cases. Google is a money printing machine because people search with an intent to buy. The SNR of normal conversation is much much much lower. That makes the unit economics of doing this gets much worse.
Third, it would be pretty hard to not notice this was happening. Your phone would get hot, your battery would deplete very quickly, and you’d be using a lot of data. Moreover on iOS you could see the mic is being used and the OS would likely kill the app if it was using too many resources in the background.
So until we find an example of this actually happening, it’s not worth worrying about.
Like a smart TV, for example.
Second thing I do is block the TV access to internet after I do one firmware update.
Building a word cloud would be trivial and with minimal battery impact
What if only the audio of "high value" targets is recorded. Meaning people who buy a lot of stuff. So it might be worthwhile to only record their sounds. Which will explain why random testing (usually with new/clean phones) is never successful in detecting a recording event.
Dead Comment
The second incident was the "listening to you thing," though. Not on the phone, but on a smart television. Exterminator was there to do the quarterly spray of my house and I was showing him scars from when I fell off a skateboard trying to bomb a hill I couldn't handle late last year, talking about what happened, and not five minutes later I turn on the television, open YouTube, and the very first recommendation on my wife's account is a video of a guy falling off his longboard at 50 MPH. Not like it's some kind of secret that we both skate and I watch a lot of downhill videos on this account, but I have never once specifically searched for, watched, or even been recommended a video of a crash, until they decide to do so five minutes after I was talking about it in front of that television.
Here’s a simple experiment I ran and still works.
Back in the day there was a truly ghastly add for ear wax removal that showed up on YouTube in the UK.
In an experiment, and prank, I told two of my close friends about this, and how this horrid advert would kill my appetite when it came up.
And then I made it a point to repeat “ear wax removal” loudly several times.
Sure enough. A day later my dear friend messaged me with something on the lines of “I hate you”
Their phones were Android and iOS. I believe it was the Android user suffered.
There are millions of ways the adware running on your phones could've correlated your profile and spread the "infection" to your friend. Basic location access being the most important one, but sharing an IP address (your friends' WiFi?), being near the same Bluetooth beacons, having the same stored SSIDs, or mere coincidence that your friend saw the same ad targeting a wide demographic are much more probable than "my phone is listening 24/7".
Do note, this was tested in a park, so no shared WiFi, no Bluetooth beacons/devices. Also, this ad doesn’t/didn’t show up for others, ever.
Can you not see all the biases and fallacies in your own comment?
Likely you all ignored it in week 1 of the 4 week campaign and by week 4 you’d seen it so many times it stuck in your head.