Apparently, the regular search crawler does it, but the ai thingie doesn't.
Can confirm. My website is flooded with AI bots despite attempts to block crawlers to certain parts of it.
Readit Newslucgagan commented on Claude can now search the web anthropic.com/news/web-se... · Posted by u/meetpateltech
lucgagan commented on Lm.rs: Minimal CPU LLM inference in Rust with no dependency github.com/samuel-vitorin... · Posted by u/littlestymaar
Correct me if I am wrong, but these implementations are all CPU bound?, i.e. if I have a good GPU, I should look for alternatives.
lucgagan commented on ScrapeGraphAI: Web scraping using LLM and direct graph logic scrapegraph-doc.onrender.... · Posted by u/ulrischa
There’s also llm-scraper in TypeScript
Something similar I worked on in the past https://github.com/lucgagan/auto-playwright/
lucgagan commented on Show HN: VideoGist – Useful YouTube video summaries videogist.co/... · Posted by u/nliang86
Very cool. I've been using a similar approach to summarize videos on my website https://ray.run/videos/112-playwright-installation-2022 Doesn't cost much and getting great feedback about it.
Is there a way to try this before I buy it?
I am building something that would probably benefit from this, but with that price tag (solo indy dev) that's going to be a big ask! might be worth it, just no way of knowing without trying it first
> Is copying parts of README enough to call for attribution to the original project?
> Is copying error message enough to call for attribution of the original project?
> Where is the line?
If you and them just had the same idea, and both created the same project, without knowing of the other one's wor, then it'd be no-harm-no-foul. If you saw what they were doing, and thought you could do it better, and did your own thing, that'd also be fine; being inspired by someone else doesn't make you a copy.
But copying actual chunks of their work, like the phrasing in the README or in error messages, would count as outright copying them, and thus, credit would be nice.
There's no concrete line in the sand, but I think it's clear you took more than just their general idea, here.
Thanks for the response. I paused for a day responding to everyone to think through all the feedback I've received, and shared an update https://github.com/lucgagan/auto-playwright/issues/15
If thier project is an inspiration and you did not credit them then in my opinion that is a bit of an ethical problem. The other thing that comes to mind for me is that if you were inspired by their project why to contribute to it rather than create a copycat?
Theirs is a commercial project with MIT client. I made an open-source version of their project. (Their backend is closed source. I have no knowledge of it.)