I've been playing with a toy app that dabbles in the Cal/CardDAV space, and it blows my mind that for all the power latest generation languages have, the thing I keep coming back to is PHP-based Sabre/DAV. That's not to say PHP isn't modern now, but instead a reflection of my surprise that there doesn't appear to be any other library out there that does as good or nearly as good a job at DAV as that one, and that one is pretty darn old.
On a different point, I don't think the author's point about having to "also" inspect the headers is a fair critique of DAV - HTTP headers are to indicate a certain portion of the request/response, and the body a different one. I wish it was simpler, but I think it's an acceptable round peg in a round hole use of the tools.
Author here, I'd be more inclined to agree about the headers if they were consistent. For instance, why is only Allow and DAV part of the header (and all of their bizarre options) and not things like supported report set or privileges? It would be better to have all of this in the body somehow, especially Depth.
I wrote a standalone CardDAV server ages ago and the biggest frustration for me was just how buggy the clients were. At some point I stopped self-hosting and moved on.
I've been self-hosting one of the CalDAV+CardDAV servers based on Sabre for few years now and that thing is solid. Multiple clients - iPhone, Android, Thunderbird. Using it with very few users though.
Mounting WebDAV -- if you are in a situation, where you have to do it (e.g. own^W^W^Wnextcloud) is such an adventure. Everything - mac, win, linux - supports WebDAV. You mount and it works! Then you notice HOW it works: files are downloaded in full before program can access them, some operations are super slow, some fail or time out, plaintext credentials in mysterious places...
I heard DeltaV is very advanced, and Subversion supported it. I'm afraid to ask.
I'm using the nextcloud app on my android, and for my Linux systems I mount WebDAV using rclone, with VFS cache mode set to FULL.
This way I can:
1. Have the file structure etc synced to local without downloading the files
2. Have it fetch files automatically when I try to read them. Also supports range requests, so if I want to play a video, it sort of streams it, no need to wait for download.
3. If a file has been accessed locally, it's going to be cached for a while, so even if I'm offline, I can still access the cached version without having to verify that it's the latest. If I'm online, then it will verify if it's the latest version.
Overall, this has worked great for me, but it did take me a while before I set it up correctly. Now I have a cache of files I use, and the rest of the stuff that I just keep there for backup or hogging purposes doesn't take disk space and stays in the cloud until I sync it.
Sine you are mounting and not syncing the files, what happens when you edit a file offline? And what if on another offline device the file is also edited?
Windows officialy removed support for WebDAV. It still works, but nothing is guaranteed. It has stupid limitation on file size of 10MB, it can be lifted to 2GB (max signed 32 bit number) in Registry, but it is still not very much in modern world (I wanted to share my medial library via WebDAV and failed due to this limitation). It lose credentials on regular basis, errors are too vague («Wrong credentials» means both mistyped password AND expired server certificate), etc.
You might wanna look into OpenCloud (formerly known as nextcloud-go) [1]. I still use Nextcloud for the uploading of files and the calendar (though I may switch the latter), but I now sync the dir with Immich. Performance-wise a relief. I also swapped Airsonic Advanced (Java) with Navidrome (Go). Same story.
Its certainly not the optimal design, but it exists in pretty much all cars, so we use it because it's there, and because of it's universal presence, its also hard to replace.
The sad part is, in a world that is increasingly mobile first, and computing devices move in and out of network coverage, HTTP based protocols actually handle frequent disconnects/reconnects much better than something like SMB.
For my personal backup needs, running from my phone, WebDAV is king. S3 would probably be better, protocol wise, but i can't have that in a simple "wrapper" that simply exposes existing files, and WebDAV works perfectly fine for LAN anyway.
Subversion works ok over webdav, it has done it for decades.
Mounting a directory through nfs, smb or ssh and files are downloaded in full before program access them. What you mean?
Listing a directory or accessing file properties, like size for example do not need full download.
I am confused, what do you mean? What OS forces you to download whole file over NFS or SMB before serving read()? Even SFTP does support reading and writing at an offset.
Actually - I believe - within Windows 11 - the "WebClient" service is now deprecated (which is what - IIRC, actually implements the WebDAV client protocol so that it works with Windows File Explorer, drive mappings, etc.)...
Played around with WebDAV alot... a long time ago... (Exchange Webstore/Webstorage System, STS/SharePoint early editions)...
I built a go caldav server and client for my task management app (http://calmtasks.com) and had a similar experience, which surprised me. Go generally has at least one good, working, and well documented implementation for all standard protocols.
Apple calendar supports caldav but in a way not specified in the spec. I basically had to send requests and responses to figure out how it works. I would be willing to open source my server and client (alot of which was built using/on top of existing libraries) if there is interest.
When working on pimsync[1] and the underlying WebDAV/CalDAV/CardDAV implementation in libdav, I wrote "live tests" early on. These are integration tests, which use real servers (radicale, xandikos, nextcloud, cyrus, etc). They do things like "create an event, update the event, fetch it, validate it was updated". Some test handle exotic encoding edge cases, or trying to modify something this a bogus "If-Match" header. All these tests were extremely useful to validate the actual behaviour, in great deal because the RFCs are pretty complex and easy to misinterpret. For anyone working on the field, I strong suggest having extensive and easy to execute integration tests with multiple servers (or clients).
All servers have quirks, so each test is marked as "fails on xandikos" or "fails on nextcloud". There's a single test which fails on all the test servers (related to encoding). Trying to figure out why this test failed drove me absolute crazy, until I finally understood that all implementations were broken in the same subtle way. Even excluding that particular test, all server fail at least one other test. So each server is broken in some subtle way. Typically edge-cases, of course.
By far, however, the worst offender is Apple's implementation. It seems that their CalDAV server has a sort of "eventual consistency" model: you can create a calendar, and then query the list of calendars… and the response indicates that the calendar doesn't exist! It usually takes a few seconds for calendars to show up, but this makes automated testing an absolute nightmare.
I once implemented a WebDAV server in PHP. The standard isn't that bad and clients are more or less following the standard. It's still horrible how they are doing that. I saw behaviors when opening a single file like:
- does / exists?
- does /path/to exists?
- does /path/to/file exists?
- create a new file /path/to/file.lock
- does /path/to/file.lock exist?
- does / exist?
- does /path/to/file exists?
- lock /path/to/file
- get content of /path/to/file
- unlock /path/to/file
- does /path/to/file.lock exist?
- remove /path/to/file.lock
(if not exactly like that it was at least very close, that was either Finder on OS X or Explorer on Windows).
Without some good caching mechanism it's hard to handle all of the load when you get multiple users.
Also the overwrite option was never used. You'd expect a client to copy a file, get and error if the target exists, ask user if it's ok, send same copy with overwrite flag set to true. In reality clients are doing all steps manually and delete the target before copying.
It was satisfying seeing it work at the end, but you really need to test all the clients in addition to just implementing the standard.
Articles like this shitting on WebDAV really rubs me the wrong way as I've seen first hand discussion that goes like: "internet say WebDAV is hell, what's the better alternative? S3 or course!" And now every cloud provider instead of providing a webdav interface provide an S3 one and it's worse by every possible way, you can't rename a file / folder because S3 does not support that, you can't support a classic username / password authentication mode but are force to use an uggly access_key_id and secret_access_key, can't bash your way around with a simple curl command to do anything because generating the signature requires a proper programming language and you have to trust Amazon to do the right thing instead of going through the RFC process except they've already shown a few months ago their complete lack of care for any s3 compliant server by introducing a breaking change that literally broke the entire ecosystem of "S3 compliant" implementations overnight and without any prior warning.
I hope WebDAV had a better reputation, it carries the original promise of s3 of being actually simple but S3 won the war with evangelism. I would much have preferred a world where new version of the webdav protocol are made to address the quirks exactly like what happened with protocols like http, oauth, ...
I once implemented JavaScript's new async-for in plain Objective-C for a WebDAV app that I wrote for a client, about 15 years ago. I was so much smarter back then than I am now. Does this happen to everyone? You just go downhill? Anyway I'm sure there were complex edge cases of WebDAV that I missed, but it worked really well in all my tests, and my client never complained about it.
For myself I don't think I was smarter before, I just paid less attention to what I was doing. I didn't know about all the edge cases. I hadn't built it before so I massively underestimated how much work it would be to get done. This makes it much easier to start.
What I did before with ignorance, I now do with experience. For projects which support it, I write tests first. Find the edge cases and figure out what I'm going to skip. I will know the scope of my project before I start it.
With solid tests in place, my productivity and confidence soars. And the implementation doesn't result in as many bugfixes than they didn't in the past.
This kind of improvement is hard to notice. You're looking at the end result of your previous work and your memory of working on it will be incomplete. Instead you're looking at what it would take for you to implement it now.
On top of all of this, do you have more responsibilities or think through your actions more than you did before? This sucks time and mental bandwidth. You have less opportunity to use your intelligence.
I had the same feeling before about a story I wrote. The stars aligned for me to write something truly excellent. For years I thought that it would be my best work. I've never been so relieved to hate something. I will always be proud of it but I no longer think it's the best I can do.
> Ah, looks like it was somewhat superseded by RFC 4918, but we’re not going to tell you which parts! How about those extension RFCs? There’s only 7 of them…
This is a major complaint I have with RFCs.
If you want to know the current standard for a protocol or format you often have to look at multiple RFCs. Some of them partially replace parts of a previous RFC, but it isn't entirely clear which parts. And the old RFCs don't link to the new ones.
There are no less than 11 RFCs for HTTP (including versions 2 and 3)
I really wish IETF published living standards that combined all relevant RFCs together in a single source of truth.
Is this true anymore? AFAIK, I've seen "Updated by" (rfc2119), "Obsoleted by" (rfc3501), but that might changed afterwards https://stackoverflow.com/a/39714048
Readit News
On a different point, I don't think the author's point about having to "also" inspect the headers is a fair critique of DAV - HTTP headers are to indicate a certain portion of the request/response, and the body a different one. I wish it was simpler, but I think it's an acceptable round peg in a round hole use of the tools.
I heard DeltaV is very advanced, and Subversion supported it. I'm afraid to ask.
Overall, this has worked great for me, but it did take me a while before I set it up correctly. Now I have a cache of files I use, and the rest of the stuff that I just keep there for backup or hogging purposes doesn't take disk space and stays in the cloud until I sync it.
https://www.thehacker.recipes/ad/movement/mitm-and-coerced-a...
own^H^H^Hnextcloud
or
own^Wnextcloud
You might wanna look into OpenCloud (formerly known as nextcloud-go) [1]. I still use Nextcloud for the uploading of files and the calendar (though I may switch the latter), but I now sync the dir with Immich. Performance-wise a relief. I also swapped Airsonic Advanced (Java) with Navidrome (Go). Same story.
[1] https://github.com/opencloud-eu/opencloud
Do you use this for anything other than photos and videos?
Its certainly not the optimal design, but it exists in pretty much all cars, so we use it because it's there, and because of it's universal presence, its also hard to replace.
The sad part is, in a world that is increasingly mobile first, and computing devices move in and out of network coverage, HTTP based protocols actually handle frequent disconnects/reconnects much better than something like SMB.
For my personal backup needs, running from my phone, WebDAV is king. S3 would probably be better, protocol wise, but i can't have that in a simple "wrapper" that simply exposes existing files, and WebDAV works perfectly fine for LAN anyway.
Mounting a directory through nfs, smb or ssh and files are downloaded in full before program access them. What you mean? Listing a directory or accessing file properties, like size for example do not need full download.
Thank you!!!!
Played around with WebDAV alot... a long time ago... (Exchange Webstore/Webstorage System, STS/SharePoint early editions)...
Apple calendar supports caldav but in a way not specified in the spec. I basically had to send requests and responses to figure out how it works. I would be willing to open source my server and client (alot of which was built using/on top of existing libraries) if there is interest.
Also, would be nice to add some screenshots of the web UI.
Looks like a nice little app!
All servers have quirks, so each test is marked as "fails on xandikos" or "fails on nextcloud". There's a single test which fails on all the test servers (related to encoding). Trying to figure out why this test failed drove me absolute crazy, until I finally understood that all implementations were broken in the same subtle way. Even excluding that particular test, all server fail at least one other test. So each server is broken in some subtle way. Typically edge-cases, of course.
By far, however, the worst offender is Apple's implementation. It seems that their CalDAV server has a sort of "eventual consistency" model: you can create a calendar, and then query the list of calendars… and the response indicates that the calendar doesn't exist! It usually takes a few seconds for calendars to show up, but this makes automated testing an absolute nightmare.
[1]: https://pimsync.whynothugo.nl/
Also the overwrite option was never used. You'd expect a client to copy a file, get and error if the target exists, ask user if it's ok, send same copy with overwrite flag set to true. In reality clients are doing all steps manually and delete the target before copying.
It was satisfying seeing it work at the end, but you really need to test all the clients in addition to just implementing the standard.
I hope WebDAV had a better reputation, it carries the original promise of s3 of being actually simple but S3 won the war with evangelism. I would much have preferred a world where new version of the webdav protocol are made to address the quirks exactly like what happened with protocols like http, oauth, ...
What I did before with ignorance, I now do with experience. For projects which support it, I write tests first. Find the edge cases and figure out what I'm going to skip. I will know the scope of my project before I start it.
With solid tests in place, my productivity and confidence soars. And the implementation doesn't result in as many bugfixes than they didn't in the past.
This kind of improvement is hard to notice. You're looking at the end result of your previous work and your memory of working on it will be incomplete. Instead you're looking at what it would take for you to implement it now.
On top of all of this, do you have more responsibilities or think through your actions more than you did before? This sucks time and mental bandwidth. You have less opportunity to use your intelligence.
I had the same feeling before about a story I wrote. The stars aligned for me to write something truly excellent. For years I thought that it would be my best work. I've never been so relieved to hate something. I will always be proud of it but I no longer think it's the best I can do.
This is a major complaint I have with RFCs.
If you want to know the current standard for a protocol or format you often have to look at multiple RFCs. Some of them partially replace parts of a previous RFC, but it isn't entirely clear which parts. And the old RFCs don't link to the new ones.
There are no less than 11 RFCs for HTTP (including versions 2 and 3)
I really wish IETF published living standards that combined all relevant RFCs together in a single source of truth.
Deleted Comment