Love the idea. However, there is no guarantee that the no-upload will not change. Which begs a question: wouldn't it be great if browsers provided a guaranteed "offline only" mode? Since they don't, I personally would prefer downloading this app as a standalone electron (or similar) app, or a docker container where I could disable Internet access.
It's a great idea indeed. But the paranoid _like myself?_ would probably not recommend it as it runs on...well...a browser. I'm not at ease with the chance that a code change someday would / could allow a copy of my file to be uploaded anywhere.
As for having the browsers run in "offline mode" (proxy equal 127.0.0 1?)...neat but I would vote against it. My fridge just needs to keep stuff cold, my toaster should serve bread, my Internet browser should not be an app.
For the attention mechanism, there isn't much difference between
And I think you could implement an algorithm for this in RASP (a language for manually programming transformers) roughly like this:1. The first layer uses attention to the "Original:" and "Modified:" tokens to determine whether the current token is in the original or modified parts.
2. The second layer has one head attend equally to all original tokens, which averages their values, and another head attends equally to all modified tokens, averaging them as well. The averages are combined by computing their difference.
3. The third layer attends to tokens that are similar to this difference, which would be the ones in the {removed part}/{added part}.
The only ordering-dependent part is whether you compute the difference as original_average - modified_average or the other way around.
If a model can detect additions but not removal, that would show that it is capable of learning this or a similar algorithm in principle, but wasn't trained on enough removal-style data to develop the necessary circuitry.