I think this will be great for extensions. Currently the only solid choice is to use the dead simple storage.local which only allows to retrieve things by ID.
There's one problem though, this new API is for the web, so the nature of this storage is temporary - obviously the user must be able to clear it when clearing site data and this what makes it currently an unviable solution for a persistent extensions data storage. https://bugs.chromium.org/p/chromium/issues/detail?id=138321...
Agreed, people want permission based file system access and so I don't see why a site can't prompt a user to store a file in a user-specified location. If they clear their site settings then giving permission to the same file should retain their data. I don't want to have to worry about losing something important or having my browser act as a file system.
> I don't see why a site can't prompt a user to store a file in a user-specified location.
This is already possible with the file system access API. Clearing site data has no impact on these files. But you do have to re-grant permission every time you reload the page.
I’m getting Flash flashbacks. Flash apps had filesystem access in a designated area, with some amount of user control, that is not easily wiped from the browser; Flash games used that for saves and data. Years after the demise of Flash, the web platform is still catching up.
It's different though, extensions are like local apps and deserve persistent storage, while you are talking about how Flash was being used by remote sources. Also my proposed approach is that this data would be wiped when the extension is removed from the browser - which is what happens for storage.local
Both APIs are part of the same standard. If you just want to store things for your site/webapp you want the origin private filesystem and you can just go for it. If you want to do something with one of the user's files it requires interaction from the user to select the folder/file since it'd be a security nightmare otherwise. You can also do both e.g. store a bunch of backend stuff for your photo editing webapp in the origin private filesystem but prompt the user to select their images folder so they can start editing them.
> the user must be able to clear it when clearing site data and this what makes it currently an unviable solution
What are you suggesting here? That extensions should have some special storage, that can't be (easily) deleted? Noooooooo thank you. Firefox has "forget about this site" and similar tools, which I use near daily. Don't fuck with my ability to do housekeeping.
yeah because full access to the filesystem is only solution.
asking for permission to a longer storage.local exemption for the user? noooo let's not even think about giving user control of things. let's the European union pass post-fact legislation after file system access is abused by adnetworks. then they get the blame of being annoying, not the browser vendor who sell ads.
let's stop being naive. there are a million ways to fix that bug with better UX and respect for user control, but this is a scape goat to have that feature.
Does anyone have hands-on experience with how well different browsers support this? This blog post and caniuse both say Chrome support is currently somewhat limited (though being improved); caniuse also mentions that Safari (both desktop and iOS) only support the Origin Private File System part of the File System API, and that Firefox doesn't support it all. I'm curious how all that actually shakes out in practice.
The Origin Private File System is coming to all browsers, Chrome, Safari, and Firefox. It's a universally agreed-on Web standard: https://fs.spec.whatwg.org/.
Just because the standard exists doesn't mean every browser has already implemented 100% of it, hence why I was asking if anyone's had any experience actually trying to use it.
EDIT: I saw from your other comments on this post that Firefox has implemented at least the origin private file system part in nightly builds; that's useful info that I was looking for.
What happens is that one browser (usually Chrome) implements a useful new feature. Then applications are released that use that feature. Then the other browser vendors start hearing from their users that they suck because they don't support the nice new feature and hence can't run the nice new applications. And eventually they add support.
> What happens is that one browser (usually Chrome) implements a useful new feature.
What happens is Chrome releases a Chrome-only non-standard (at a rate of 400 APIs per year). It cobbles together a barely legible spec [1], and then uses its multiple propaganda channels like web.dev to present them as actual completed standards that other browsers are just slow to implement.
Even if other browsers only learn about this a few days before Chrome ships, or have multiple objections.
There isn't an "official" NPM package or ES6 model yet, but I believe they are working on it. It's also very much designed for the browser currently, but again I believe they are intending to support server WASM environments too eventually.
> There isn't an "official" NPM package or ES6 model yet
(The sqlite team's JS/WASM Guy here...)
An ES6 module build was added shortly after the 3.40 release.
NPM/node.js are nowhere on our radar. Our build is structured such that people who want to plug it in the resulting JS file to whatever their favorite toolchain is are welcomed to do so, but we have neither the ambition nor the bandwidth to support every build/bundling platform out there, especially ones none of us otherwise use.
> It's also very much designed for the browser currently, but again I believe they are intending to support server WASM environments too eventually.
The JS code is targeted solely at browsers and there are no plans on changing that in the foreseeable future.
The WASM build itself, we are working to provide server-side support for. We have a branch which builds under wasi-sdk, but we cannot create a JS binding for that build until/unless we find some substitute for Emscripten's transparent translation of POSIX I/O APIs.
> but we have neither the ambition nor the bandwidth to support every build/bundling platform out there, especially ones none of us otherwise use.
For what it's worth (though it seems like the decisions is done), there's no need to support every build/bundling platform - the NPM registry is rather 'tooling agnostic' and publishing there would make it available to a lot more people with a lot less effort.
Hey, I am the maintainer of sql.js. I was in contact with them, and I do not think they are working on a npm module, but I offered them to work on the existing "official" sql.js module.
For example, it should be possible to build websites for downloading YouTube videos that does not use server side code execution or bandwidth. These were popular at some point but disappeared after Java applets stopped being supported by most browsers.
And make every website part of a botnet? I sure hope not.
WebSockets are already a problem because many web devs don't know that other origins can connect to them and potentially fetch data they're not supposed to (even ignoring the fact you can construct these without a browser), unleashing raw sockets to the web platform would be hell.
WebRTC and WebSockets provide more than enough already. If you need even more, your browser just isn't the place for this type of code IMO.
mmap would make WebAssembly a lot better, unironically. Even if it was only supported for read-only pages. A lot of cpu time and IO are wasted doing reads into temporary buffers that then get copied into the wasm heap.
The custom header requirement will make this version of sqlite unusable for many usecases. Can SQLite fallback to using slower persistant APIs if neither Cross-Origin-Opener-Policy or Origin-Embedder-Policy are set? If not, does the SQLite project intend to provide a version that would?
Devs who don't need the high performance but want wider compatibility will probably continue to use the "unofficial" versions.
Also, last I checked, you could definitely load web assembly in a page that has a File:/// origin.
My use case is an entirely offline webapp that I'm building as a side project, partly just to see what is possible. I've tried indexeddb, pouchdb, absurd-sql, and various others, and have ended up back at localstorage, which is plain but low on edge cases.
The other big pain point I am hacking around is cross device sync; I'm leaning towards simple chrome and firefox extensions that'll slurp data into your profile in order to sync, which'll work for cross computer sync for my needs. However, chrome on android doesn't support extensions, and there's no other good way that I've found to sync between a user's chrome profile and android, so that is still a hole in the story. Some kind of webrtc based thing is imaginable, but it's clunky.
This is definitely better, yeah. The blog post[1] on absurd-sql notes that it's a hacky solution that would be improved with file system access; it references the old Storage Foundation proposal [2], which has since evolved into the current File System Access API proposal(s).
Readit News
There's one problem though, this new API is for the web, so the nature of this storage is temporary - obviously the user must be able to clear it when clearing site data and this what makes it currently an unviable solution for a persistent extensions data storage. https://bugs.chromium.org/p/chromium/issues/detail?id=138321...
This is already possible with the file system access API. Clearing site data has no impact on these files. But you do have to re-grant permission every time you reload the page.
https://developer.mozilla.org/en-US/docs/Web/API/File_System...
What are you suggesting here? That extensions should have some special storage, that can't be (easily) deleted? Noooooooo thank you. Firefox has "forget about this site" and similar tools, which I use near daily. Don't fuck with my ability to do housekeeping.
asking for permission to a longer storage.local exemption for the user? noooo let's not even think about giving user control of things. let's the European union pass post-fact legislation after file system access is abused by adnetworks. then they get the blame of being annoying, not the browser vendor who sell ads.
let's stop being naive. there are a million ways to fix that bug with better UX and respect for user control, but this is a scape goat to have that feature.
EDIT: I saw from your other comments on this post that Firefox has implemented at least the origin private file system part in nightly builds; that's useful info that I was looking for.
What happens is Chrome releases a Chrome-only non-standard (at a rate of 400 APIs per year). It cobbles together a barely legible spec [1], and then uses its multiple propaganda channels like web.dev to present them as actual completed standards that other browsers are just slow to implement.
Even if other browsers only learn about this a few days before Chrome ships, or have multiple objections.
[1] Example, File System API https://wicg.github.io/file-system-access/ "not a W3C Standard nor is it on the W3C Standards Track."
https://developer.mozilla.org/en-US/docs/Web/API/FileSystemF...
Discussion from back when the SQLite project announced this last year: https://news.ycombinator.com/item?id=33374402
There isn't an "official" NPM package or ES6 model yet, but I believe they are working on it. It's also very much designed for the browser currently, but again I believe they are intending to support server WASM environments too eventually.
(The sqlite team's JS/WASM Guy here...)
An ES6 module build was added shortly after the 3.40 release.
NPM/node.js are nowhere on our radar. Our build is structured such that people who want to plug it in the resulting JS file to whatever their favorite toolchain is are welcomed to do so, but we have neither the ambition nor the bandwidth to support every build/bundling platform out there, especially ones none of us otherwise use.
> It's also very much designed for the browser currently, but again I believe they are intending to support server WASM environments too eventually.
The JS code is targeted solely at browsers and there are no plans on changing that in the foreseeable future.
The WASM build itself, we are working to provide server-side support for. We have a branch which builds under wasi-sdk, but we cannot create a JS binding for that build until/unless we find some substitute for Emscripten's transparent translation of POSIX I/O APIs.
For what it's worth (though it seems like the decisions is done), there's no need to support every build/bundling platform - the NPM registry is rather 'tooling agnostic' and publishing there would make it available to a lot more people with a lot less effort.
Deleted Comment
Deleted Comment
https://developer.mozilla.org/en-US/docs/Web/API/File_System...
What's next, mmap() for javascript?
For example, it should be possible to build websites for downloading YouTube videos that does not use server side code execution or bandwidth. These were popular at some point but disappeared after Java applets stopped being supported by most browsers.
WebSockets are already a problem because many web devs don't know that other origins can connect to them and potentially fetch data they're not supposed to (even ignoring the fact you can construct these without a browser), unleashing raw sockets to the web platform would be hell.
WebRTC and WebSockets provide more than enough already. If you need even more, your browser just isn't the place for this type of code IMO.
Devs who don't need the high performance but want wider compatibility will probably continue to use the "unofficial" versions.
Also, last I checked, you could definitely load web assembly in a page that has a File:/// origin.
My use case is an entirely offline webapp that I'm building as a side project, partly just to see what is possible. I've tried indexeddb, pouchdb, absurd-sql, and various others, and have ended up back at localstorage, which is plain but low on edge cases.
The other big pain point I am hacking around is cross device sync; I'm leaning towards simple chrome and firefox extensions that'll slurp data into your profile in order to sync, which'll work for cross computer sync for my needs. However, chrome on android doesn't support extensions, and there's no other good way that I've found to sync between a user's chrome profile and android, so that is still a hole in the story. Some kind of webrtc based thing is imaginable, but it's clunky.
https://bugzilla.mozilla.org/show_bug.cgi?id=1789116
[1] https://github.com/jlongster/absurd-sql
https://vlcn.io/docs/guide-persistence#persistence-options
[1] https://jlongster.com/future-sql-web
[2] https://developer.chrome.com/docs/web-platform/storage-found...