I've landed on Postgres/ClickHouse/NATS since together they handle nearly any conceivable workload managing relational, columnar, messaging/streaming very well. It is also not painful at all to use as it is lightweight and fast/easy to spin up in a simple docker compose. Postgres is of course the core and you don't always need all three but compliment each other very well imo. This has been my "go to" for a while.
I’m like here’s a giant thing of ice cold booze have fun.
> What does an AI crawler do different from a search engine (indexing) crawler?
Many people don't want the extra bot traffic hitting their site that comes from AI, especially when AI chat & AI overviews in Google provide such a small amount of traffic in return and that traffic pretty much always has horrendous conversion rates (personally seen across multiple industries).
You will probably tend to filter out candidates with other options and be left by candidates that are more desperate and willing to invest the time.
Except that auth might not be a core part of your insurance or tax app, and you'd rather spend your energy on the part of "agility" that has to do with the core parts of your app.
I couldn't believe my eyes. I was working in my own project beside this team with the list, so thankfully I was left out of the whole disaster.
A guy I knew wasn't that lucky. I saw how he suffered from this harmful list. Then I told him a story about the Danish film director Lars von Trier I recently had heard. von Trier was going to be chosen to appear in a "canon" list of important Danish artists that the goverment was responsible for. He then made a short film where he took the Danish flag (red with a white cross) and cut out the white lines and stitched it together again, forming a red communist flag. von Trier was immediately made persona non grata and removed from the "canon".
Later that day my friend approached the bugs caused/fixed list, cut out his own line, taped it together and put it on the wall again. I never forget how a PL came in the room later, stood and gazed at the list for a long time before he realized what had happened. "Did you do this?" he asked my friend. "Yes", he answered. "Why?", said the PL. "I don't want to be part of that list", he answered. The next day the list was gone.
A dear memory of successful subversion.