I have started doing something completely different than using bookmarks. I set up yacy[1] on a personal, internal server at my home, which I can access from all my devices, since they are always on my wireguard vpn.
Yacy is actually a distributed search engine, but I run in 'Robinson mode' as a private peer, to keep it isolated, as I just want a personal search of only sites I have indexed.
Anytime I come across something of interest, I index it with yacy, using a a depth of 0 (since I only want to index that one page, not the whole site). This way, I can just go to my search site, and search for something, and anything related that I've indexed before pops up. I found this works way better than trying to manage bookmarks with descriptions and tags.
Also, yacy will keep a cache of the content which is great if the site ever goes offline or changes.
If I need to browse, I can go use yacy's admin tools to see all the urls I have indexed.
I have been using this for several months and I am using this way more than I ever used my bookmarks.
This is great, and is something I've wanted for a while. I use pinboard which is supposed to have similar capabilities (click 'search full text', 'search mine' after turning on and paying for 'archiving'), but I've never been totally confident in it (pages would change, and the cached version was updated to a 404 page), and ended up letting my archiving subscription lapse.
I think google used to offer something that did this as well as search all your local files, but I think that went the way of all gThings.
This is Mac only and I have no affiliation other than I like this developer but your request reminded me that he just launched this app: https://andadinosaur.com/launch-history-book
I have wished for a while that browser would store the entire page of any bookmark you save automatically, and put a decent search engine on it. I wrote a script once to do it for my bookmarks, and it didn't even take that much space on my hard drive.
Your system could be a Firefox addon, kinda like what scrapbook used to be, but automatic. Even with a note system, and storing metadata, Zotero style, but without the need for the dual setup.
How mozzilla is not pursuing that kind of innovation in Firefox itself is beyond me. Instead they continously try to ape the Chrome UI and collect telemetry without permission so that they can give you "exciting" "new" features like colorways.
It would also be nice to be able to search through my aggregated browsing history on every device I use.
Maybe I should open a feature request to Google/Fracebook to provide an API hook for that, since they probably already have all that information anyway.
This is insane! Thanks for recommendation! Was able to set it up relatively quickly and now have a personal search of all the things that I find interesting, this is insane!
How does the process of bookmarking goes then? meaning, how hard is it to add something to crawl with depth0 on a day to day. Can it be done with a bookmarklet?
me too, when I first used delicious I was hoping that individual curators could provide a more in-depth "meta search" engine! Pocket is not bad, it provides some interesting links to new stuff on the web.
I really liked this setup. The only point of friction for me was adding the links to the index via the crawler everytime. So I created a Firefox Extenstion to do it directly from the address-bar.
Thanks for this recommendation! Do you only index things with a depth of 0? I just set Yacy up a couple days after seeing this post, and didn't realize what that meant at first.
Now I have 10s of 1000s of pages indexed after importing my bookmarks/history, and I'm wondering if it'd be more useful to _only_ index the pages I've visited/bookmarked, or if it'd also be good to crawl those sites further.
I guess one distinction would be whether or not I thought I could use Yacy as a full-time replacement for google/ddg. It'd be nice if I could index "everything", but then have a toggle to search only my bookmarks/history or something similar to that.
That sounds really cool. How difficult is this to set up?
How confident are you that this will keep your bookmarks safe? Losing indexed data might not be such a big concern for a search engine that is intended to re-index everything continuously.
I defnitely felt betrayed when I found out that Firefox automatically deletes old history :/ I guess they think that most users need to trash their profile to fix some obscure bug before that anyway...
Yes, it does automatically keep a cache from when it indexed the site. I have it set to not automatically recrawl sites, so the cache is from when I added the site.
I'd love to hear more about how you set all this up, particularly, do you have some sort of extension or bookmarklet that submits the current page to yacy's crawler?
Nice! I’ve been working on and off on a similar idea (searchable index of link contents) as a cli app eventually web web frontend. It’s on Python so packaging has been an issue.
I have tried many different solutions the last two decades, but none of of them really stuck or became useful over time. I kinda gave up and as a last ditch effort started to do the simplest thing I could think of: ctrl+D to add bookmarks in Firefox, jotting down a few keywords on each entry. No folders, no structure, just a flat list and some keywords.
A few months in I noticed how powerful this simple system was. When talking with someone else about a tool, github-repo or article I had seen but did no remember the name or title of, finding it back was suddenly a breeze. Since I keep my desktop and mobile bookmarks in sync, it it just a matter of typing in a keword in the address bar in firefox and it shows up instantly!
On desktop, you can limit the search to bookmarks only by starting with a *, which is helpful to avoid browser history etc.
I have really low bar for adding a bookmark now as the mental overhead is so low and it is done notime. It has become the second brain I always wanted :)
I also do this, but I don't like my browser cluttered up with thousands of accumulated bookmarks, many of which I don't return to. So I export them every month, delete the embedded favicons, and then wipe. If I want to find something I just grep for it.
I use Firefox bookmarks too and sync so new devices are easy to setup. The "Bookmark search plus 2" add on allows you to search folder names and shows which folder a bookmark is stored in.
Reasons:
1. Archives - those tutorials and guides stay when the original pages go 404
2. API - I use the api to automatically post my bookmarks to my blog
3. Full-text search: this is very very useful when needed
4. Social Discovery: Search that niche website / app on Pinboard. It shows lots of other people who found that same thing as interesting. We can then follow them and subscribe to their favourites as RSS feed.
I’ve been a Pinboard customer since 2010 and I subscribed to the archival service several years. But archival seems to have stopped on my account. I think I emailed once but never received a reply (which I’ve heard is common). I love the philosophy of Pinboard and I also like Maciej. That said I recently decided to roll my own bookmarks tool with Wayback Machine archival capability.
Archival hasn't been working for months for me and nobody answers the mails (I've tried three times over the last year or so) so as much as I like pinboard.in at the moment I am looking for something else
Same. Having imported my delicious bookmarks dating back to 2005 or so, I have a fairly large set of links that I try to tag consistently. I don't actually read a ton of them, but being able to full-text search or filter by combining tags makes it really useful for digging up things I barely remember coming across.
I love Pinboard. It has all the features I'd expect from a bookmarking service, but nothing superfluous. There's no upsell. There's no advertisement or JavaScript bloat.
Part of the reason for Pinboard's success is the lack of VC pressure for growth. I'm happy to keep paying for Pinboard indefinitely.
I use pinboard as well. Early user of del.icio.us, I exported it all to pinboard and paid a one-time lifetime fee. Too many old links are dead, but that's the nature of the web, and I hope waybackmachine can help with some of them (I never paid for the full-text-archive feature of pinboard, it would have been a good idea but it's too late now). Sometimes it definitely helps me find some old highlights that still lurk in a shiny way in my mind.
Pinboard is phenomenal. I used to keep all my links in Simplenote but Pinboard is far superior for a number of the reasons listed here already. I may only search through it for something once a week but I find I tag things much more thoroughly in Pinboard than anything else I've used.
I just became a Pinboard customer a few months ago!
I picked Pinboard because the UI is simple but functional. No 30mb blob of JavaScript. It pairs well with todo.txt… now I just need a simple Dropbox-based notes app to complete the trio.
I also continue to use Pinboard, for much the same reasons. Since 2010! I don't use the social features but it's nice to have a tool that's been constant and reliable for over a decade.
My incredibly unsophisticated, but surprisingly effective approach, is to share by email with myself (e.g. mail to myname+bookmark@mydomain.com).
Mail rules can then file them, I can add any relevant notes or hashtags to the mail body at the time I share the link, and the chronological ordering is helpful. Imap search is usually 'good enough' to turn up a half-remembered link or article.
I have been meaning to add an imap script to complement this with something like a simplepage archive, but have never got round to it.
Scenario, if I save a link from HackerNews, I like to save the submission of it as well. I wonder if I send a HackerNews link to forlater.email; it can parse the article and remind me I got it from HackerNews.
I dropped pinboard.in recently. The interface hasn't had improvements in years, the extensions are all third party, and the API if you wanted to build your own is pretty limiting. The mobile interface is pretty poor too.
I'm now moved over the Raindrop.io[1], which is another solo-developer outfit, but has had a lot of work put into it. It does all the same stuff Pinboard does (including page archiving but beside the social and public directory things... which nobody uses), but has a bunch of additional features. It has a much more complete API, a well maintained extension, and mobile apps! Definitely worth giving a go.
I've been paying for Raindrop for a few years and its organisation capabilities are really good: tags, collections, folders, search, etc. All in a quite polished UI!
Agreed it's much better service, I've been really enjoying it.
One feature I recently learned was Highlights [1]
You can select a passage of text on the page, then when bookmarked, it'll save the selected text. Allows for multiple highlights. And then visiting the page in the future those texts clips will then be highlighted again.
I use Raindrop.io and have it hooked up to NewsBlur and ArchiveBox as secondary backups [1].
This way whenever something is bookmarked it's saved in Newsblur and published to Dropbox, which ArchiveBox picks up every hour and saves a local copy and to archive.org.
Pinboard is still quite active. If you need proof just go to /recent which is a live firehose and interesting to see what people are bookmarking. I use Pinboard and regularly export my bookmarks incase their servers are hacked/wiped/corrupted.
I pay to keep the servers running, not so I can have something new and shiny every month. If it somehow quits doing what pinboard does, then I'll look at alternatives.
That actually reminds me I haven't done an export in a looong time and I should.
I was using Magnolia before Pinboard and it went down permanently. Fortunately, at the time, I was doing link blog posts once or twice a week so I was able to recover most of my links with a bit of work.
I use Zotero for this now. I have a bunch of sub-collections (e.g. technical, interesting, fitness, etc.) and when I see a webpage I like I use the plug-in to save to Zotero. Better than a bookmark because it also saves a snapshot of the webpage, and, I can easily cite it if I'm writing a document.
Same. I don't even use the citation features of Zotero, it's purely a bookmark manager for me. I can choose whether to save the page with or without a snapshot, use both folders and tags for organization, add notes if I want to, and on supported sites (like Github), get an automatic bookmark summary too.
The interface took a bit of getting used to, but I learned some of the shortcuts, installed Zutilo [1], and ultimately just accepted the fact that I'll have to use the mouse for some things, as everything else about the program makes it worth it.
Do you or anyone else have thoughts on if Zotero would be too much for someone who doesn't need to write papers or cite documents? A large component of my day-to-day work is doing a lot of research and managing it for the duration of the project.
Readit News
Yacy is actually a distributed search engine, but I run in 'Robinson mode' as a private peer, to keep it isolated, as I just want a personal search of only sites I have indexed.
Anytime I come across something of interest, I index it with yacy, using a a depth of 0 (since I only want to index that one page, not the whole site). This way, I can just go to my search site, and search for something, and anything related that I've indexed before pops up. I found this works way better than trying to manage bookmarks with descriptions and tags.
Also, yacy will keep a cache of the content which is great if the site ever goes offline or changes.
If I need to browse, I can go use yacy's admin tools to see all the urls I have indexed.
I have been using this for several months and I am using this way more than I ever used my bookmarks.
[1] https://yacy.net/
I think google used to offer something that did this as well as search all your local files, but I think that went the way of all gThings.
https://www.browserparrot.com/
https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#Import-l...
You create some rules for topics you want to index and it'll go out and crawl them. Searching through it is a global hotkey away.
I have wished for a while that browser would store the entire page of any bookmark you save automatically, and put a decent search engine on it. I wrote a script once to do it for my bookmarks, and it didn't even take that much space on my hard drive.
Your system could be a Firefox addon, kinda like what scrapbook used to be, but automatic. Even with a note system, and storing metadata, Zotero style, but without the need for the dual setup.
Not what you're going for -- you don't have a list of specifically opted-in 'bookmarks' to browse.
but I have often wanted "wait, what was that site involving X I was looking at maybe last week?"
Maybe I should open a feature request to Google/Fracebook to provide an API hook for that, since they probably already have all that information anyway.
Deleted Comment
The offline caching sounds awesome.
Thanks for sharing
If someone is interested, you can download it from here: https://addons.mozilla.org/en-US/firefox/addon/yacy-it/
Limitation: It currently supports only YaCy running on localhost and unprotected. You might have to configure CORS as outlined here -> https://github.com/tecoholic/yacy-it#configuring-yacy
Now I have 10s of 1000s of pages indexed after importing my bookmarks/history, and I'm wondering if it'd be more useful to _only_ index the pages I've visited/bookmarked, or if it'd also be good to crawl those sites further.
I guess one distinction would be whether or not I thought I could use Yacy as a full-time replacement for google/ddg. It'd be nice if I could index "everything", but then have a toggle to search only my bookmarks/history or something similar to that.
Edit : looks like the docker config allow to mount a arbitrary folder , that folder can be shared. I don’t need it to be concurrent proof.
Again, thanks this look nice.
How confident are you that this will keep your bookmarks safe? Losing indexed data might not be such a big concern for a search engine that is intended to re-index everything continuously.
I defnitely felt betrayed when I found out that Firefox automatically deletes old history :/ I guess they think that most users need to trash their profile to fix some obscure bug before that anyway...
A few months in I noticed how powerful this simple system was. When talking with someone else about a tool, github-repo or article I had seen but did no remember the name or title of, finding it back was suddenly a breeze. Since I keep my desktop and mobile bookmarks in sync, it it just a matter of typing in a keword in the address bar in firefox and it shows up instantly!
On desktop, you can limit the search to bookmarks only by starting with a *, which is helpful to avoid browser history etc.
I have really low bar for adding a bookmark now as the mental overhead is so low and it is done notime. It has become the second brain I always wanted :)
It's useful for grouping a set of pages that you use together, but only now and then.
A bit primitive, but it works for me.
https://github.com/blakewatson/bookmarks
https://github.com/klenwell/pinprick
I find it a good way to keep in touch with past bookmarks and do some light maintenance.
I have over 30K bookmarks and add multiple hundreds a month.
Part of the reason for Pinboard's success is the lack of VC pressure for growth. I'm happy to keep paying for Pinboard indefinitely.
I picked Pinboard because the UI is simple but functional. No 30mb blob of JavaScript. It pairs well with todo.txt… now I just need a simple Dropbox-based notes app to complete the trio.
Deleted Comment
Mail rules can then file them, I can add any relevant notes or hashtags to the mail body at the time I share the link, and the chronological ordering is helpful. Imap search is usually 'good enough' to turn up a half-remembered link or article.
I have been meaning to add an imap script to complement this with something like a simplepage archive, but have never got round to it.
edit: feature/request
Scenario, if I save a link from HackerNews, I like to save the submission of it as well. I wonder if I send a HackerNews link to forlater.email; it can parse the article and remind me I got it from HackerNews.
For example, I'd submit "https://news.ycombinator.com/item?id=31852384" and in the email, it'd contain:
1. HackerNews link (so i can review the comments as well)
2. Original article link and content
food for thought, but love your product idea
I'm now moved over the Raindrop.io[1], which is another solo-developer outfit, but has had a lot of work put into it. It does all the same stuff Pinboard does (including page archiving but beside the social and public directory things... which nobody uses), but has a bunch of additional features. It has a much more complete API, a well maintained extension, and mobile apps! Definitely worth giving a go.
[1]: https://raindrop.io/
One feature I recently learned was Highlights [1]
You can select a passage of text on the page, then when bookmarked, it'll save the selected text. Allows for multiple highlights. And then visiting the page in the future those texts clips will then be highlighted again.
[1] https://help.raindrop.io/highlights/
This way whenever something is bookmarked it's saved in Newsblur and published to Dropbox, which ArchiveBox picks up every hour and saves a local copy and to archive.org.
1. https://www.ecliptik.com/bookmarking-with-raindrop/
I pay to keep the servers running, not so I can have something new and shiny every month. If it somehow quits doing what pinboard does, then I'll look at alternatives.
Deleted Comment
I was using Magnolia before Pinboard and it went down permanently. Fortunately, at the time, I was doing link blog posts once or twice a week so I was able to recover most of my links with a bit of work.
https://www.zotero.org/
The interface took a bit of getting used to, but I learned some of the shortcuts, installed Zutilo [1], and ultimately just accepted the fact that I'll have to use the mouse for some things, as everything else about the program makes it worth it.
[1] https://github.com/wshanks/Zutilo