> The two hops, the two companies, are already acting in partnership, so what is there technically in the relay setup to stop the two companies from getting together—either voluntarily or at the secret command of some government—to compare notes, as it were, and connect the dots?
The OHTTP scheme does not _technically_ prevent this.
It increases the number parties need to cooperate to extract this information, hoping it would be caught somewhere in the pipeline.
and if a government say already forced ISPs to collect metadata about who connects to whom and when, I imagine they don't even need to bother getting data from the relay hosts
It's cool how neural networks, even convulutional ones, are one of the few applications that you can compute through homomorphic encryption without hitting a mountain of noise/bootstrapping costs. Minimal depth hurrhah!
I don't think Apple is doing this. They compute the embedding on device in the clear, and then just do the nearest-neighbor part in HE (which does lots of dot products but no neural network).
There are people doing NNs in HE, but most implementations do indeed require bootstrapping, and for that reason they usually use CKKS.
I was going to make my usual comment of FHE being nice in theory but too slow in practice, and then the article points out that there’s now SHE (somewhat homomorphic encryption). I wasn’t aware that the security guarantees of FHE could be relaxed without sacrificing them. That’s pretty cool.
Is there any concrete info about noise budgets? It seems like that’s the critical concern, and I’d like to understand at what point precisely the security breaks down if you have too little (or too much?) noise.
SHE vs FHE has nothing to do with security. Instead, it’s about how many operations (eg homomorphic multiplications and additions) can be performed before the correctness of the scheme fails due to too much noise accumulating in the ciphertext. Indeed all FHE schemes are also SHE schemes.
What typically makes FHE expensive computationally is a “bootstrapping” step for removing the noise that accumulated after X operations and threatening correctness. After bootstrapping you can do another X operations. Rinse and repeat until you finish the computation to you want to perform.
Im not an expert on this, but my understanding is "noise" is less a security breakdown and more the entire system breaksdown. That is where the "somewhat" comes in, unlike "full" where the system can do (expensive) things to get rid of noise, in somewhat the noise just accumulates until the system stops working. (Definitely talking out of my hat here)
Interesting! Are there any security concerns with SHE? If not, it sounds like all of the benefits of FHE with none of the downsides, other than the noise overwhelming the system. If that’s true, and SHE can run at least somewhat inexpensively, then this could be big. I was once super hyped about FHE till I realized it couldn’t be practical, and this has my old excitement stirring again.
it's incredibly algorithm-dependent. if you look into the thesis that originates the 'bootstrapping' technique to transform SHE algorithms into FHE, they determine the noise limit of their specific algorithm in section 7.3 and then investigate expanding the noise limit in 8 and 10.
Not sure whether they use FHE or not (the literature I’m looking at simply says “homomorphic”), but we use ElectionGuard at our company to help verify elections for our customers. So there are definitely some practical uses.
> One example of how we’re using this implementation in iOS 18, is the new Live Caller ID Lookup feature, which provides caller ID and spam blocking services. Live Caller ID Lookup uses homomorphic encryption to send an encrypted query to a server that can provide information about a phone number without the server knowing the specific phone number in the request.
There is another reason which I dislike this which is that now Apple has reason for "encrypted" data to be sent randomly or at least every time you take a picture. If in the future they silently change the photos app (a real risk that I have really emphasized in the past) they can now silently pass along a hash of the photo and noone would be the wiser.
If an iPhone was not sending any traffic whatsoever to the mothership, at least it would ring alarm bells if it suddenly started doing so.
Isn't this the same argument that they can change any part of the underlying OS and compromise the security by exfiltrating secret data? Why specific to this Photos feature.
No. GP means that if the app was not already phoning home then seeing it phone home would ring alarm bells, but if the app is always phoning home if you use it at all then you can't see "phoning home" as an alarm -- you either accept it or abandon it.
Whereas if the app never phoned home and then upon upgrade it started to then you could decide to kill it and stop using the app / phone.
Of course, realistically <.00001% of users would even check for unexpected phone home, or abandon the platform over any of this. So in a way you're right.
Any app can do this really, just can’t update the entitlements and a few other things. I would think it unlawful for Apple’s own apps to have access to functionality/apis that others don’t…
I love homomorphic encryption, but why can't they just do a neural search locally?
- iOS Photos -> Vectors
- Search Query "Dog photos" -> Vectors
- Result (Cosine Similarity): Look some dog photos!
iPhones have plenty of local storage and compute power for doing this kind of thing when the phone is idle. And cosine similarity can work quickly at runtime.
> This seems like a lot of data the client is getting anyway. I don’t blame you for questioning if the server is actually needed. The thing is, the stored vectors that are compared against are by far the biggest storage user. Each vector can easily be multiple kilobytes. The paper discusses a database of 35 million entries divided across 8500 clusters.
Because the blog post needs some sort of concrete example to explain, but all concrete examples of fully-homeomorphic encryption are generally done better locally at the moment due to the extreme costs of FHE.
I didn't like their license because it's BSD-3-Clause-Clear but then they state:
"Zama’s libraries are free to use under the BSD 3-Clause Clear license only for development, research, prototyping, and experimentation purposes. However, for any commercial use of Zama's open source code, companies must purchase Zama’s commercial patent license"
So Its not free, you need to pay for patent license, and they don't disclose how much.
I recommend OpenFHE as an alternative Free open source solution.
I know its C++ and not Rust, but no patent license and it can do the same thing the blog post wants to do, it even has more features like proxy-reencryption that I think Concrete can't do.
> If a company open sources their code under BSD3-clear, they can sell additional licenses to use the patents included in the open source code. In this case, it still isn’t the software that is sold, but rather the usage rights of the patented intellectual property it contains.
Concrete from Zama is a completely different algorithm to the one used in this product; the former uses the CKKS algorithm, while the latter is the BFV algorithm.
Zama has already hit competitors with (French) patent charges. Apple's HE implementation is in Swift and uses BFV, which is very different HE from anything Zama does and doesn't use their source.
This would be even more exciting if there were some way to guarantee your phone, the servers, etc. are running untampered implementations, and that the proxies aren't colluding with Apple.
If someone or something can tamper with your phone, then nobody needs to collude with proxies or Apple. They can just ask your phone to send them exactly what they want, without all the homomorphic encryption dance.
The idea that Apple is going to use this feature to spy on you, completely misses the fact that they own the entire OS on your phone, and are quite capable of directly spying on you via your phone if they wanted to.
Upgrades have to be possible. What you want probably is attestation that you're running a generally available version that other users run too as opposed to one specially made for you, but since a version could be made for all those subject to surveillance this wouldn't be enough either.
I'm not sure there's a way out of this that doesn't involve open source and repeatable builds (and watch out for Reflections on Trusting Trust).
Readit News
The OHTTP scheme does not _technically_ prevent this. It increases the number parties need to cooperate to extract this information, hoping it would be caught somewhere in the pipeline.
There are people doing NNs in HE, but most implementations do indeed require bootstrapping, and for that reason they usually use CKKS.
Is there any concrete info about noise budgets? It seems like that’s the critical concern, and I’d like to understand at what point precisely the security breaks down if you have too little (or too much?) noise.
What typically makes FHE expensive computationally is a “bootstrapping” step for removing the noise that accumulated after X operations and threatening correctness. After bootstrapping you can do another X operations. Rinse and repeat until you finish the computation to you want to perform.
(written in 2009) http://crypto.stanford.edu/craig/craig-thesis.pdf
some newer FHE don't encounter a noise limit or don't use the bootstrapping technique.
Privacy by design is always nice to see.
Let's supouse Apple is evil (or they recive an order from a judge) and they want to know who is calling 5555-1234
1) Add a new empty "spam" numbers encrypted database to the server (so there are now two encrypted databases in the system)
2) Add the encrited version of 5555-1234 to it.
3) When someone checks, reply the correct answer from the real database and also check in the second one and send the reply to the police.
If an iPhone was not sending any traffic whatsoever to the mothership, at least it would ring alarm bells if it suddenly started doing so.
Whereas if the app never phoned home and then upon upgrade it started to then you could decide to kill it and stop using the app / phone.
Of course, realistically <.00001% of users would even check for unexpected phone home, or abandon the platform over any of this. So in a way you're right.
Deleted Comment
- iOS Photos -> Vectors
- Search Query "Dog photos" -> Vectors
- Result (Cosine Similarity): Look some dog photos!
iPhones have plenty of local storage and compute power for doing this kind of thing when the phone is idle. And cosine similarity can work quickly at runtime.
Deleted Comment
> This seems like a lot of data the client is getting anyway. I don’t blame you for questioning if the server is actually needed. The thing is, the stored vectors that are compared against are by far the biggest storage user. Each vector can easily be multiple kilobytes. The paper discusses a database of 35 million entries divided across 8500 clusters.
[1]: https://github.com/fguzman82/CLIP-Finder2 [2]: https://github.com/apple/ml-mobileclip
I didn't like their license because it's BSD-3-Clause-Clear but then they state:
"Zama’s libraries are free to use under the BSD 3-Clause Clear license only for development, research, prototyping, and experimentation purposes. However, for any commercial use of Zama's open source code, companies must purchase Zama’s commercial patent license"
So Its not free, you need to pay for patent license, and they don't disclose how much.
I recommend OpenFHE as an alternative Free open source solution. I know its C++ and not Rust, but no patent license and it can do the same thing the blog post wants to do, it even has more features like proxy-reencryption that I think Concrete can't do.
It's like saying: "FREE* candy! (Free to look at, eating is $6.95 / pound)"
> If a company open sources their code under BSD3-clear, they can sell additional licenses to use the patents included in the open source code. In this case, it still isn’t the software that is sold, but rather the usage rights of the patented intellectual property it contains.
Every day I like the Apache licence more.
Just like 3+1 is not 3.
Deleted Comment
Copied from openfhe.org
Or maybe thats not what you meant...
The idea that Apple is going to use this feature to spy on you, completely misses the fact that they own the entire OS on your phone, and are quite capable of directly spying on you via your phone if they wanted to.
I'm not sure there's a way out of this that doesn't involve open source and repeatable builds (and watch out for Reflections on Trusting Trust).