Readit News logoReadit News
Posted by u/acheong08 3 years ago
Show HN: Open-source obsidian.md sync server
https://github.com/acheong08/obsidian-sync

Hello HN,

I'm a recent high school graduate and can't afford $8 per month for the official sync service, so I tried my hand at replicating the server.

It's still missing a few features, such as file recovery and history, but the basic sync is working.

To the creators of Obsidian.md: I'm probably violating the TOS, and I'm sorry. I'll take down the repository if asked. It's not ready for production and is highly inefficient; Not competition, so I hope you'll be lenient.

kepano · 3 years ago
(Obsidian CEO here)

Impressive! It's fun to see the diversity of ways people sync/backup their Obsidian files. The nice thing about storing all your notes on your device is that it makes it possible to move and edit your Markdown files in many different ways. That diversity of solutions is what makes the ecosystem of Markdown tools resilient over the long term.

There are already a handful of tools that allow you to sync your notes for free, including Git, Syncthing, and some other options more specialized for Obsidian (see community plugins).

Obsidian is a small company, we're not VC backed (100% user-supported), so the Sync pricing helps us stay in business and keep the lights on. We also have a 40% educational discount on all our services[1] so you could be paying $4.80 instead of $8 :)

Reverse engineering things is a fun technical challenge, and also helps us find potential holes in our system. The main problems I see with your solution: 1. it could easily break in a future update to the app, 2. "Obsidian Sync" is a trademark, so you should consider renaming the repo otherwise it can be confused for an official tool — that would be my only request

[1]: https://help.obsidian.md/Licenses+and+payment/Education+and+...

Lord_Zero · 3 years ago
I don't see why you cant let people download and self-host the sync service if they want. Most people would probably pay to use yours but many people would want to self host as opposed to using fragile and feature-incomplete community supported plugins... people seem to replicate the sync server anyway, why not give them a better experience and live up to the "your notes" promise that is all over the website.
PurpleRamen · 2 years ago
There is little benefit, but big harm in releasing the server. They would be responsible for supporting users in using the server, which could be little or big additional work. Releasing the server would also demand additional work for the release itself, like writing documentation, and fixing some errors. And it could also lower the little income they already have, and at least send the message that there are fewer reasons to financially support them and the development of Obsidian.

On the other side, the alternatives are working well enough, so there is not much reason to release the server.

iamdamian · 3 years ago
I discovered Obsidian this year, and it's not an understatement to say that it helped restore my faith in local-first computing and humane tools. Obsidian has helped me reclaim my thought process and my writing, and I'm much happier using it.

Like others have mentioned, your approach and ethos are a large part of why I decided to go all in on Obisidian, and that ethos is reflected in Sync. With Sync, everything is end-to-end encrypted and can be unsynced at any time without having to "export" anything or jump through hoops. Configuration is file-based and can be selectively synced, including themes. Beyond that, I can rest easy knowing that if Sync is ever down, I have all of my files right there.

When I downloaded Obsidian for the first time, I thought it was a shame that I didn't need to pay for it. So I was more than happy to find Sync and pay for the service to help what I see as a good company.

Kudos, and please keep up the good work.

dansalvato · 3 years ago
I'm a brand new user and thought all this time that Obsidian Sync was 100% cloud-only, so I spent a bunch of time trying to spin up a custom sync solution. This comment inspired me to look more carefully at it, and now I'm a paid user!

I only wish that https://obsidian.md/sync made it more obvious that you can sync a local vault to the cloud. Emphasis on "You still own your data, no vendor lock-in" is a huge sell for users like me who are typically forced to choose between seamlessness vs. owning our data. I'm delighted to learn that Obsidian offers both.

ilyin · 3 years ago
Time and again I keep being impressed with your selfless commitment to building and sharing things in the most humane and long-term way. A beautiful example to set, thanks.
LorenDB · 3 years ago
> Time and again I keep being impressed with your selfless commitment to building and sharing things in the most humane and long-term way.

And yet they haven't open sourced the app. That would be truly humane and long-term. As it is, it's just another proprietary app that, if it dies, can't be continued by the community.

I don't think monetary issues should be a concern with open sourcing the app - companies like Bitwarden, Gitlab, and New Vector (the company behind Matrix) are doing just fine with their open source products.

intellectronica · 3 years ago
This attitude is in no small part the reason I'm happy to pay those $8 every month - support a great team building and continuously improving a package so useful it has become an integral part of my life and, as a bonus, get a slick and reliable sync service on top.
guiambros · 3 years ago
Same. Love Obsidian, and pay for Sync mostly to contribute to the fantastic work they do.
throwaway5959 · 3 years ago
Came here to say this. Keep up the great work!
bachmeier · 3 years ago
I'll join in with the others and say a few good words. It's unusual in 2023 for a company to focus on delivering a good customer experience while respecting the preferences of the customer. Imagine a company that builds a business around providing value to the customer.

Renewed for my third year of Sync yesterday. While I could use other tools to sync (I mostly have markdown files in my vaults, so there's not a lot of data moving around), the quality of the sync service, and the ease of setting it up, makes me renew without thinking about it. And it's not easy getting me to pay for anything these days due to subscription fatigue, scam fatigue, and price hike fatigue.

qntmfred · 3 years ago
I've been using obsidian since last year, and I love it. After previously haven fallen in and then out of love with notion, obsidian has been just what the doctor ordered.

So I was excited to see a obsidian HN post, and really didn't expect to see the CEO as the top post. Kudos to you.

I've been a sync customer more or less since I started using obsidian, mostly because I use the same vault on my phone plus a couple PCs.

But honestly, the sync has been disappointing. When I restructure folders or rename stuff, half the time the next time I open on a different device, it deletes shit. I can usually figure it out and recover from the sync log, that's a big trustbreaker.

kepano · 3 years ago
The rename issue is already solved and in the process of being rolled out across all the clients. Our original implementation of renaming for Sync was naive and slow. This was especially noticeable if you renamed a folder with lots of files. It would first delete the remote file on the Sync server and re-upload the local file to the new path. If you did this with a whole folder it meant waiting for all the remote files be deleted and then reuploaded. It's safe, just slow. Now we're tracking renames which is much better in every way :)
lucw · 3 years ago
Super happy Obsidian Sync user here. I migrated over from Notion, here are the areas where Obsidian blows Notion out of the water:

- Insanely low input lag especially on mobile. Obsidian is FAST. When i'm out and about checking my shopping list, I have a tolerance for maybe 250ms of lag. That's it. You need to show me my data FAST. Notion is a basically a webapp which simply doesn't compete, asks you to re-login constantly. YOU DO NOT ASK A MOBILE USER to RELOGIN.

- Read-only mode: accidental taps instead of scrolls on mobile are a HUGE issue on Notion, especially when you're out and you're in a stressful situation and you need your notes fast. You can not afford an accidental edit. And you simply don't have to focus to carefully touch your smartphone to distinguish swipes from touches. Notion developers just don't get this. With Obsidian, when i'm ouside, i'm most likely in Read-only mode. You can still check checkboxes when in read-only mode, and that's perfect for a shopping list workflow.

- Standard format (markdown). I had tried for some time to automate my Notion backups and it just wasn't possible. With Obsidian, I have no issue using proprietary software because the backing store is Markdown. It integrates nicely with my storage practices. My obsidian files are backed up hourly to my rsync.net account which has 48 hourly snapshots. If I accidentally overwrite something 16 hours ago, I can recover it.

As long as Obsidian meets those 3 checkboxes for me, I'm happy to pay $10/month, $20/month, $30/month, whatever.

yaysyu · 3 years ago
Small nit, Notion has been rewritten into a native app on Android and iOS in the last year or so and the performance is significantly better.

Haven’t had the logging out issue either on iOS.

belthesar · 3 years ago
A response like this was exactly the reminder I needed to upgrade my Catalyst license. My opsec can be a little on the paranoid side, so while I believe y'all have the best of intentions and would build sync with the expectation that my data is safe, I would prefer to not use your sync service. But I use Obsidian so regularly that it's definitely worth supporting good work done by folks that align with me. Keep up the wonderful work, both as a software company and as a beacon for how to be a good steward in an increasingly cynical industry.
kepano · 3 years ago
For reference, below is our blog post on how you can verify the Sync encryption. Most people don't have the time or skills to set up their own sync service, but if you do setting up a DIY solution can be a fun project.

https://obsidian.md/blog/verify-obsidian-sync-encryption/

jpgvm · 3 years ago
I had never heard of your service until now but your reply has probably landed you $8/month.

Bootstrapped startups deserve my money and commendable leadership should be rewarded.

ler_ · 3 years ago
I don't usually comment but just wanted to say that I absolutely love Obsidian. I have been using it every day for about 2 years now and my life would not be the same without it. It's amazing. I also gladly pay for Sync, it's more than worth it.
spondyl · 3 years ago
This sync server project is pretty handy for a headless daemon I had been meaning to cobble together in the background.

The sort of thing that could say; pull down markdown files inside a CI pipeline from the sync server which is probably better handled using Git for sync but it seemed like a fun challenge.

I've got most of a working implementation from trawling through minified Obsidian code with the debugger, which of course is now a little moot given the OP's project basically captures everything needed.

Anyway, one core problem I ran into was how to decrypt file blobs. I was dreading the amount of time I would have to waste trawling through all the bits required and then they just... tweeted it out: https://obsidian.md/blog/verify-obsidian-sync-encryption/

I'm sure it would be possible to reverse engineer with enough time of course but it's nice to see some "unofficially official" user script type stuff. It gives the perception (and I'm sure it's accurate) that they dogfood and tinker with their own stuff as much as anyone else in the community.

EDIT: Ah, I missed that Kepano had already shared that blog post below already but it's still a neat writeup

txnf · 3 years ago
I just want to say how much I love your product. So many of the best things never make it financially, but I really really hope you all make it. I'm 100% willing to support obsidian financially, just need to know where and how to do that. I don't want an account or a sync thing, I just wanna give you all money for the great product.
dabeeeenster · 3 years ago
What a wonderful reply. Thank you and kudos!
acheong08 · 3 years ago
Thank you. I’ll be renaming the repository. (Looks like a situation similar to vaultwarden)
andymac4182 · 3 years ago
I have been looking at building a self hosted implementation of this service because of compliance reasons.

The $$$ of the service isn't the issue but making sure there is a good compliance option and self hosted to ensure that data doesn't leave boundaries and is stored securely on company servers.

If there was an option to have a paid self hosted option I would happily look at paying for that.

kepano · 3 years ago
Down the road we might make something like that for the commercial use case — makes sense.
ashishb · 3 years ago
I love Obsidian over all the other tools that I have tried. https://ashishb.net/all/why-i-prefer-obsidian-for-taking-not...
antback · 3 years ago
Like everyone else, I would like to express my absolute love for your product. Awesome response and I hope you can work at Obsidian and be happy for many years to come.
iepathos · 3 years ago
Great product, thank you. Really appreciate the way you handled the trademark issue here with a tactful request. Definitely earning points with open source community
darkteflon · 3 years ago
Long-term paying user of Obsidian Sync here. It’s fantastic - more reliable than any other sync service I’ve tried (haven’t tried the one subject of this post) that has to interface with the iOS ecosystem.

Really like the fact that Obsidian itself is free and you charge for native sync (and publishing, which I also use) to support ongoing development of this fantastic editor and ecosystem. Thanks.

martin82 · 3 years ago
What a class act of a reply! I applaud this!
fire · 3 years ago
Honestly I just want a sync solution I can host on my NAS that the mobile app can still access, which unfortunately isn't something the app natively supports, and more or less all of the current community solutions come with one caveat or another ( like the otherwise lovely git solution having issues on iOS clients, for example )
lucw · 3 years ago
You can't host the sync server on your NAS, but you can definitely synchronize your notes to your NAS, which means you still have them available in case your desktop goes down and your smartphone breaks or is lost or something.
nsteel · 3 years ago
Does Remotely Save not work on iOS? If it does, you can host a WebDAV server on your NAS.
entilzha · 3 years ago
While you’re here, a killer feature for me would be the ability to privately host obsidian sites (similar to publish). Even if it required subscribing to publish to download a tarball of the site (that isn’t public), it could still be worth it. My use case is sharing obsidian notes with non-users (eg coworkers) in a private way.
kepano · 3 years ago
There are a few options for this already. A good one just came out a few days ago called Quartz: https://github.com/jackyzha0/quartz
vGPU · 3 years ago
I was previously using a third party sync solution, but guess who just got a new subscription?
hyperific · 3 years ago
Just gotta say I absolutely love Obsidian. I've been using it for about 6 months now and I'm continuously blown away at how versatile it is. Canvases were such an incredible addition. The plugins from the community are top notch.
mind-blight · 3 years ago
I love your reply here, and your product . Have you considered trying to implement a collaborative document editing tool? I would definitely pray for shared vaults that allow multiple users to modify files
jrgaston · 3 years ago
Obsidian is a great product, thank you! And the optional Sync service works beautifully to keep my files synched between Windows, iPhone, and iPad.
j_0 · 3 years ago
Thank you kepano. I love this response and will continue to support Obsidian while cheering on this project that taps into your platform.
menthe · 3 years ago
Your promise did not last very long.
shepherdjerred · 3 years ago
Just wanted to say thank you for not attacking/threatening the creator!
moeffju · 3 years ago
I just bought a yearly sync subscription due to this comment. Kudos!
stereoradonc · 3 years ago
Give him sync for free! Or atleast let me sponsor the kid.
kepano · 3 years ago
I don't want to reduce his motivation to learn how to reverse engineer things! I learned a lot from those kinds of projects. However, we do have a way to gift someone a subscription[1]

@acheong08 if you want a Sync subscription I'll be happy to cover it.

[1]: https://help.obsidian.md/Licenses+and+payment/Gifting

poglet · 3 years ago
Would love to get Dropbox support on iOS
rogeliodh · 3 years ago
Please consider regional prices. US$8 monthly is high for in-develop countries
kepano · 3 years ago
This is very hard for us to do because we want to remain a small team — it will introduce a lot of complexity for payments/localization

Dead Comment

surge · 3 years ago
You're prob not violating ToS since you're not doing it for commercial use, and you're actually reinventing the wheel as a couple of options for that are already available as community plugins. I've been using Remote-save for a while with a almost free private S3 bucket and encryption to sync between my computers and phone for a while.

https://github.com/remotely-save/remotely-save

It supports more than just S3 (as is listed in the README), its just what I use.

That said, what you made is pretty cool. I guess you're trying to replace the API/backend similar to Vaultwarden does with BitWarden paid service.

For me, its not that I can't afford $8 a month, its just that $8 is to me fairly steep for basic file storage and sync I get for free from some services. $12 a year would be a fair price to me. I get some of it goes to support Obsidian development, but still seems steep, $2-3 a month would be something I'd subscribe for.

nsteel · 3 years ago
Remotely save is excellent. I use it with WebDAV to work between my computers, phone and tablet.
echelon · 3 years ago
If Obsidian stops acheong08, I'm done with Obsidian.

I want a durable way to edit, sync, and preserve my notes, not just a pretty interface.

I'm happy to pay Obsidian for sync, but if they PM this to only one distribution channel and subsequently enshittify it, I don't want it. Stopping acheong08 would be a bad signal forewarning what's to come.

rs_rs_rs_rs_rs · 3 years ago
Dude, just pay the $8, no reason to cluch your pearls like that.
solarkraft · 3 years ago
Syncthing actually works pretty well when you have a tool that automatically resolves sync conflicts: https://discuss.logseq.com/t/automatically-merge-syncthing-c...
freedomben · 3 years ago
I'm using Syncthing to keep in sync between a desktop and two Android phones over tailscale, and it works pretty good, but I have to occasionally open the Syncthing app on Android to "bump" it and get it to sync. It's in a git repo (which syncthing handles great even though there are some older posts online telling you not to do that), and on the desktop I can review and "sanity check" stuff and then commit it to ensure it is safe. Overall a good solution.
UltimateEdge · 3 years ago
Why not exclude the .git directory from being synced, if you're not going to be committing on your mobile devices? This is what I do.
ephbit · 3 years ago
> .. I have to occasionally open the Syncthing app on Android to "bump" it and get it to sync.

Almost the same with me.

Turning WIFI off and on again is easier on my Android, it triggers syncthing to sync. But that might also require you to have set the run conditions of syncthing accordingly.

MikusR · 3 years ago
The great thing about syncthing is that it has public relays.
karpour · 3 years ago
I'm using syncthing with Obsidian but of course i have regular conflicts which i just deal with manually. Thanks for pointing me to this!
runjake · 3 years ago
FWIW, I get sync conflicts with Obsidian Sync pretty regularly. They're always pretty minor and not worth complaining about, but they happen nonetheless, and it may not be entirely SyncThing's fault.
dmje · 3 years ago
FWIW - the sync service provided by Obsidian is really worth paying for. Dropbox / Github just aren't as slick and integrated. The mobile app doesn't work without sync (not without some really nasty workarounds), the option to "view sync version history", the fact that it's rock solid... all worth paying for IMO.
input_sh · 3 years ago
Yeah I can think of 30 different ways to make the sync work, but I can't be bothered to, Sync works really well, and I feel like financially supporting the Obsidian team in some way.

$8/month is a bit steep, but I snatched $5/month early bird pricing and I consider that to be a lot more acceptable of a price.

Edit: worth pointing out there's a 40% discount for non-profits and students: https://help.obsidian.md/Licenses+and+payment/Education+and+...

Sytten · 3 years ago
It kind of boggles my mind when people say 8$/month is steep. I get that we don't all live in rich countries, but still. If you are using a tool everyday this is really not much. I think we have been conditioned to expect software to be free. Maybe because there is nothing tangible?

It's also hard to do micro payments on the internet ironically, on that 8$ you can easily pay almost a dollar of fees. If that wasnt the case I expect we would have a much different internet...

xmprt · 3 years ago
I haven't tried Dropbox or Github or Syncthing like others have suggested but having tried Obsidian Sync I'm not as impressed as everyone else seems to be. Maybe I need to set my bar lower but especially on my phone, background sync doesn't work well and it usually takes a few seconds to load and a few more seconds to sync the data which feels like ages considering all the data required to load it is already on my phone and the diff is a few paragraphs at most.
BrandoElFollito · 3 years ago
> The mobile app doesn't work without sync (not without some really nasty workarounds)

You simply synchronize the folder as with the desktop apps. Works right of the box without any workarounds.

m3adow · 3 years ago
Yeah, not sure what this guy is talking about. I'm syncing my notes with my Nextcloud using the FolderSync App on Android which I used for other stuff anyways. Works like a charm, never had any problems.

Obsidian is great, but I'm not paying over 90 bucks a year for a simple file syncing service I can build in less than an hour.

Maybe the publish feature is worth it, but I'm content with the ObsidianPublisher workflow I set up for now.

ytechie · 3 years ago
It works great for me just using iCloud Drive.
iamdamian · 3 years ago
I tried iCloud Drive and didn't mind it. I ended up moving to Sync, though, because 1) it's end-to-end encrypted and 2) I can sync any directory, not just subdirectories of iCloud Drive.
acheong08 · 3 years ago
> view sync version history

Actively working on that. It’s a feature I’ll need since I mess up *very often*

bosie · 3 years ago
just a friendly reminder as it bit me often: the version history is gone/gets wiped any time you rename a file/folder.
rodja_ · 3 years ago
> I'm a recent high school graduate and can't afford $8 per month for the official sync service

What happened to the $47000 you got from Apple [1]?

[1]: https://medium.com/bugbountywriteup/how-i-earned-47000-usd-a...

Arch485 · 3 years ago
That's about 1 year of tuition in NA; OP is probably saving it for that
freedomben · 3 years ago
Dang, if money isn't plentiful, there are way cheaper options in NA than that. A good school can definitely help you network with future successful people and to some orgs the name on the diploma can help open some doors, but the degree is less needed nowadays than it's ever been.
vGPU · 3 years ago
The OP stated at the bottom that he was saving it for tuition.
MH15 · 3 years ago
I've been using the main Obsidian git extension, https://github.com/denolehov/obsidian-git. Took some work to set it up ergnonomically but it works great now. I enabled auto-commit and push on save, and auto-pull when you start the editor. No merge conflicts yet between two machines.

Should note I use Obsidian for a journal, so it's pretty much append-only.

microflash · 3 years ago
Obsidian Git has been working great for me too. I use different configuration directories for each device to avoid configuration conflicts. It works very seamlessly once set up.
gandalfthepink · 3 years ago
OP This is not addressed to you but to other folks in the comments.

The two obsidian itself is very open and it's free for use. Those making demands to reduce price or make the project open source are just thankless. How do you plan to make the business sustainable? Someone has to pay developer salaries, right?

You want open source? Use MarkText, Zettlr. The company is generous enough to offer the main product for free.

t3chguy · 3 years ago
Whilst the reply from the CEO seems sound, their team promptly closed the path the plugin used to make self-hosted full-featured Sync & Publish work. "For Security" - ignoring the handful of other gaping security holes as less important I guess?

Anyway, I decided to fork and make it work with v1.4.5. Enjoy https://github.com/acheong08/obi-sync/issues/19#issuecomment...