> Perhaps you’re a user of LLMs. I get it, they’re neat tools. They’re useful for certain kinds of learning. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate
Am I the only one using LLMs as if they were a search engine? So before LLMs I was searching on Google things like "pros cons mysql mongodb". I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed). Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references". So, I get a quick overview of what to keep an eye on, and the references are there to avoid relying on hallucination.
It's true that sometimes I go ahead and say "give me a data schema for storing photos metadata in postgres. I wanna keep X in a different table, though" (or something like that). But I do that because I know very well what the output should look like (I just don't wanna spend time typing it, and sometimes I forget the actual type that I should use (int vs integer?)).
The few times I've used LLMs as question answering engines for anything moderately technical, they've given subtly-but-in-important-ways incorrect information such that taking them at face value would've likely lost me hours or days of pursuing something unworkable, even when I ask for references. Whether or not the "references" actually contain the information I'm asking for or merely something tangentially related has been rather hit or miss too.
The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
Absolutely. And I’m finding the same with “agent” coding tools. With the ever increasing hype around Cursor I tried to give it a go this week. The first 5 minutes were impressive, when I sent a small trial ballon for a simple change.
But when asking for a full feature, I lost a full day trying to get it to stop chasing its tail. I’m still in the “pro” free trial period so it was using a frontier model.
This was for a Phoenix / Elixir project; which I realize is not as robustly in the training data as other languages and frameworks, but it was supposedly consuming the documentation, other reference code I’d linked in, and I’d connected the Tidewave MCP.
Regardless, in the morning with fresh eyes and a fresh cup of coffee, I reverted all the cursor changes and implemented the code myself in a couple hours.
Yes, you have to be very careful when querying LLM's, you have to assume that they are giving you sort of the average answer to a question. I find them very good at sort of telling me how people commonly solve a problem. I'm lucky, in that the space I've been working has had a lot of good forums training data, and the average solution tends to be on the more correct side. But you still have to validate nearly everything it tells you. It's also funny to watch the tokenization "fails". When you ask about things like register names, and you can see it choose nonexisting tokens. Atmel libraries have a lot of things like this in them
And the output will be almost correct code, but instead of an answer being:
PORT_PA17A_EIC_EXTINT1
you'll get:
PORT_PA17A_EIC_EXTINT_NUM
and you can tell that it diverged trying to use similar tokens, and since _ follows EXTINT sometimes, it's a "valid" token to try, and now that it's EXTINT_ now NUM is the most likely thing to follow.
That said, it's massively sped up the project I'm working on, especially since Microchip effectively shut down the forums that chatgpt was trained on.
>The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
This is basically the only thing I use it for. It's great at it, especially given that Google is so terrible these days that a search describing what you're trying to recall gets nothing. Especially if it involves a phrase heavily associated with other things.
For example "What episode of <X show> did <Y thing> happen?" In the past, Google would usually pull it up (often from reddit discussion), but now it just shows me tons of generic results about the show.
I agree. Use with caution. One of my personal pet peeves with LLM answers is their propensity to give authoritative or definite answers, when in fact they are best guesses, and sometimes pure fantasy.
> Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references".
In near future, companies will probably be able to pay lots of money to have their products come up better in the comparison. LLMs are smart enough to make the result seem "organic" -- all verifiable information will be true and supported by references, it will only be about proper framing and emphasis, etc.
I'm very grateful that we have a lot of players training LLM's, including several that are published as open models and open weights.
I fully expect LLM results to start including ads, but because of the competition I hope/believe the incentives are much better than they are for, say Google's search monopoly.
It could potentially be more insidious though.
We'll probably start sending prompts to multiple models and comparing the results with lower-power local models.
This is already the case, SEO content, sponsored comparison sites, influencer marketing, it's all about subtle framing. LLMs just supercharge the problem by making it easier and cheaper to scale.
The real issue isn't that LLMs lie, it's that they emphasize certain truths over others, shaping perception without saying anything factually incorrect. That makes them harder to detect than traditional ads or SEO spam.
Open-source LLMs and transparency in prompt+context will help a bit, but long-term, we probably need something like reputation scores for LLM output, tied to models, data sources, or even the prompt authors.
This already happens unintentionally, e.g. Wikipedia loops, where bad info on Wikipedia gets repeated elsewhere, and then the Wikipedia article gets updated to cite that source.
When LLM-generated content is pervasive everywhere, and the training data for LLMs is coming from the prior output of LLMs, we're going to be in for some fun. Validation and curation of information are soon going to be more important than they've ever been.
But I don't think there'll be too much intentional manipulation of LLMs, given how decentralized LLMs already are. It's going to be difficult enough getting consistency with valid info -- manipulating the entire ecosystem with deliberately contrived info is going to be very challenging.
I use it the same way. The feeling is that I'm back in ~2010 when Googling stuff felt like a superpower. I could find anything back then.
Of course, it didn't last long, and trying to Google now is an exercise in pain and frustration. Lots of people have complained about the various things Google and marketers have done to get there, idk, I just don't like how it works now.
Top LLMs feel amazingly good at rapidly surfacing info online, and as I go through the references they're usually pretty good. I guess the same forces as before will apply, and there will be some window of opportunity before it all goes away again.
I wonder when LMMs and services like chatgpt become as bloated as search engines are today, with their own equivalent of SEO/SEM tools and other unwanted stuff distracting and disturbing accuracy, even if one finally stops hallucinating.
If you do a web search and find a random blog post full of spelling errors and surrounded by ads, you're not going to trust that at the same level as a Stack Overflow post with a hundred upvotes, or an article with a long comment thread on HN.
But an LLM digests everything, and then spits out information with the same level of detail, same terminology, and same presentation regardless of where it came from. It strips away a lot of the contextual metadata we use to weigh credibility and trust.
Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
There will be a race between the attempts monetize online LLM services like this and the development of consumer owned hardware that can enable local LLMs with sufficient power to deliver the same service but ad free.
Combined with RAG a self hosted LLM will definitely be able to deliver a more impartial and therefore better solution.
I do this. But the killer usecase for me is writing all boilerplate and implementing some half-working stuff keeps my attention on the issue which makes me able to complete more complex things.
A recent example is when I implemented a (Kubernetes) CSI driver that makes /nix available in a container so you can run an empty image and skip a lot of infra to manage.
I talked to it a bit and eventually it wrote a Nix derivation that runs the CSI codegen for Python and packages it so I could import it. Then I asked it to implement the gRPC interface it had generated and managed to get a "Hello World' when mounting this volume (just an empty dir). I also asked it to generate the YAML for the StorageClass, CSIDriver, Deployment and DaemonSet.
So LLM left me with a CSI driver that does nothing in Python (rather than Go which is what everything Kubernetes is implemented in) that I could then rewrite to run a Nix build and copy storepaths into a folder that's mounted into the container.
Sure implementing a gRPC interface might not be the hardest thing in hindsight, but I've never done it before and it's now a fully functional(ish) implementation of what i described.
It even managed to switch gRPC implementations because the Python one was funky with protoc versions in Nix(Python bundles the grpc codegen it's so stupid) so i asked it to do the same thing for grpclib instead which worked.
i’ve stopped writing “real” code for the most part, i just bang out some pseudo code like:
read all files in directory ending in .tmpl
render these as go templates
if any with kind: deployment
add annotation blah: bar
publish to local kubeapi using sa account foo
and tell it to translate it to x lang.
so i control the logic, it handles the syntax.
asking it to solve problems for you never seems to really work, but it remembers syntax and if i need some kinda reader interface over another or whatever.
can’t help me with code reviews tho, so i spent most of my time reading code instead of remembering syntax. i’m ok with it.
> Am I the only one using LLMs as if they were a search engine?
Nope, you're not alone as I also do this. I'm also not using any AI IDE's (yet).
It's funny, I actually recently failed a live technical interview where I was using my LLM of choice to answer Google-like queries instead of an IDE like cursor. The interviewer told me that he had actually never seen anyone use AI for coding like that before. Up to that point, I assumed that most coders were mainly using AI as a search engine and not necessarily using AI IDE's yet. Are we really that rare?
Not at all, I've been doing this with ChatGPT and Claude for a long time. I only recently (last couple weeks) started playing around with Claude Code on command line (not in an IDE). I didn't like Cursor very much. YMMV
In the past you looking around documentation, SO answers etc would have hopefully helped you learn more about the tools and develop skills required to independently analyze the pros and cons. If you ask an LLM (or a search engine or a colleague) and take their words as the ground truth then you won’t develop the skills. Worst, sooner or later no one will have enough knowledge nor analytic skill to form an opinion on any sufficiently deep subject and they will all be dependent on corporate chatbots to spoon feed them information that may or may not be biased in a way against your interest. Now imagine if ChatGPT tells you to use azure and Gemini tells you to use GCP…
This, and I use them for code review and rapidly generating prototypes that I heavily edit. Almost none of the LLM code survives usually. You could ask "Why dont you just write it yourself then?" but sometimes getting started with the skeleton of a working project is the most difficult part of it.
It is nice when it works, but sometimes I run into trouble where I don't know a right word to put in the prompt to get the answer I'm looking for. I've recently been playing around with Raku and had a really cryptic type signature error and
Claude was of absolutely no help because it didn't know about the interaction of Signature literals and 'Slurpy' sigils in method parameters. Only when I learned about it and included the word Slurpy in my prompt would it actually regurgitate the information I was looking for but at that point I already knew it.
This plus easier search of poorly/not-at-all documented APIs is like 80% of my usage too. Besides that, a lot of “here’s my design for xyz system, am I a stupid idiot for using this architecture?”.
Yup this is where 90% of the productivity benefits come from for me. Instead of needing to spend an hour scouring documentation I can ask an LLM and have an answer in 5 minutes
Best metaphor I have found to how I use them is as "a hunting dog".
They can get into small crevasses and the foliage and whatnot and they don't mind getting wet. They can fluster rabbits out. And are somewhat smart. But you still have to make the kill and you have to lead the dog, not the other way around.
"Copilot" is a great marketing name, but a bit deceiving.
> I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed).
This learning time that you welcomed is what you will now miss out on. The LLM gives you an answer, you don't know how good it is, you use it, and soon enough, if running into a similar issue, you will need to ask the LLM again since you were missing out on all that learning the first time which would have enabled you to internalize all the concepts.
It's like the regex engine example from the article. An LLM can create such a thing for you. You can read through it, it might even work, but the learning from this is orders of magnitudes less than what you get if you build this yourself.
I think it depends. LLMs can link references of where they took the content they throw at you. You can go and read such references. I like what LLMs provide, and at the same time I don't wanna blindly follow them, so I always allocate time for learning whether it's with LLMs or not.
I recently ended up having a multiple hour long conversation with a friend of mine about the potential impact of LLMs and similar tools, which I then made into a blog post: https://blog.kronis.dev/blog/ai-artisans-and-brainrot
Basically, if you use agentic tools (not just to look things up or get surface level answers, but write your code for you), then it's quite likely that your brain no longer does as much heavy lifting as it would have before, nor does anything you do to solve the issues you're working with have much staying power.
Over time, this will decrease our cognitive capabilities in regards to specific things, very much how the same has largely happened with language syntax knowledge thanks to various IDEs and language servers, auto-complete and so on, you no longer need to memorize as much, so you don't.
While outsourcing some of that doesn't seem too bad, it's hard to tell what will happen to our critical thinking skills in the long term, since we're not just getting rid of some technical implementation details, but making the tool do a lot of thinking for us. All while screwing over those who don't use these tools, because they have to deal with the full friction of solving the problems.
For really basic overviews of different technologies I've found YouTube videos to be useful.
For code examples I've always found GitHub code search to be super useful.
I haven't really found a use case for LLMs where it's any faster than the research tools already available on the internet.
A good example is writing AWS CDK code, which is a pain in the ass. But there are so many examples to pull from on GitHub code search that are being used in real projects that I've found this method to be faster than prompting an LLM that may or may not be correct.
I just used an LLM to sleuth out a concurrency issue earlier - I could see what was happening in the debugger, I just couldn’t really see why - asked the LLM to talk me through lifecycle stuff, and boom, first thing it brought up was apparently the answer. Thank you, glorified fuzzy search! Much quicker than pouring through the docs.
I got lucky sure, and sometimes I don’t get so lucky - but it works often enough to have become a part of my regular approach at this point. Why is the thing broke? Maybe the robot knows!
Even when using them to code, I use them as a search engine. Rather than telling them to implement feature X, I clone a repo which has a similar feature and say:
"explore the repo I've cloned at /src/foo and explain how it achieves barFeature. Now look a the project at /src/baz and tell me why it would be difficult to use foo's approach in baz"
I rarely have it do anything novel, just translate ideas from existing projects into mine. Novel work is for me to enjoy coding directly.
I learned about window functions in SQL using an LLM. I hadn't written SQL in over a decade and never ran across them. It explained how they work and the trade-offs. It was great!
My main use for LLMs is to ask it to give me some boilerplate configuration for some library/tool I am not very familiar with and then look up the docs for the options it spits out. Like:
"give me the terraform configuration for an AWS ECS cluster with 2 services that can talk to each other, with one of them available publicly and the other one private"
Occasionally to give some small self-contained algorithms, for example:
Give me a way to format the difference between dates as human-readable text. For example: 1 day, 16 hours.
> Am I the only one using LLMs as if they were a search engine?
I quite like the search-first LLMs like Gemini and Copilot for this reason. They give you links which you can use to verify the output and seem to be less prone to directing you to SEO spam than Google Search/Bing.
Although I do partly think that search engines today are delivering poor performance compared to what they delivered historically so LLMs are benefiting from that.
It's kind of scary how good it is. I haven't completely switched over, but I think a lot of that is just not wanting to admit that this is the new paradigm and the implications. The ability to not only find what you're looking for faster, but have it tailored to your specific context? It's hard to go back from that.
> Am I the only one using LLMs as if they were a search engine?
Google agree with you and now at the top there is a AI generated answer, clearly labeled as AI generated, and it cites the sources. I was trying to escape AI, but I have to recognize the implementation by Google is quite good.
My most common use is to treat it like one of those recipe clippers that grabs a recipe out of an overly long blog post. Last weekend I punched '28 years after credits?' into Kagi and got back only what I wanted. Finally I can have a recipe clipper for any topic. I'm happy with it.
i'm currently using it in large part as a question asking tool. feed it a bunch of documents, then ask it to ask me questions to figure out whats been missed -- stuff that i probably know, or could find out, or look for more documents that have the answer, but arent clear from the documents i had handy.
this afternoon i spent some time feeding context, with the idea of making a QnA context for maybe 50 different website routes, so i can have it ask me similar questions per route, so i can have a hell of a lot of pretty accurate documentation for an intern to pick up a couple to work on, or get some LLM to take a try at filling out the details.
im feeding it knowledge, with the output being documents, rather than me getting knowledge from it.
I primarily use it this way or as a rubber duck. I pretty much use it as a replacement for going on a Google adventure. When I use it for code, it's mostly as a glorified auto-complete or write some boilerplate I can't be bothered to type.
That's basically how I use it though I cannot wait for Gemini to be a click away in my browser with the current screen/page(s)/tab(s) embedded so I can ask it stuff about the current long article/documentation page. We're becoming very, very lazy.
I tell to myself to use them as "teachers", not "interns", i.e. ask them questions to guide my process or look for the sources of knowledge needed to understand or do something, instead of asking them to get things done (except tedious, simple tasks).
Make sure to spend some time asking them questions on topics you already know a lot about. (I like to ask the AI about a game I developed called Neptune's Pride.)
A year ago the AI would just make up some completely random stuff. The current crop do a very good job, but still not 100% correct.
They are wrong enough that I would be wary of using them to "teach" me topics I don't know about.
Nope, I do this too (most of the time.) I don’t like working on code I don’t understand. I have started to ask it to use a client API I’ve written to figure out how clients would work with the stuff I write though. It’s great.
I think one of the major issues I (and others) have with LLM's as search is they are heavily biased towards outdated data. The other day I was trying to have it get me the most updated versions of X, X, X from an old project. I tried several providers and all of them without exception gave me an "upgrade" to an older version than was in the project for at least 1 item(by years). I'm sure they are choosing LTS releases over the new ones because that is most popular overall in a dataset. However no matter how "hard" I prompt they keep giving me those old ones over "latest"
My LLM use case preference is exactly that - a google replacement. I've modified my prompts to ask for links to direct source material, thereby allowing me to go deeper. I find this in no way different then a search engine.
The downside however, is that at least 50% of the links cited no longer exists. This points to a general issue with LLMs, temporal knowledge. I tend to at least go to archive.org on a per-link basis when so inclined.. But clearly that's not ideal.
Does anyone have a better solution, short of just asking the LLM to remove the extra step, and link to the archive.org cache for the ingestion date (which frankly, I also don't have faith in being accurate).
It's a suggestion engine. An autocomplete (I am not saying GPT is just autocomplete of course it is more advanced) but end if the day you need to verify everything it does right now. May not be true in the future though.
Search finds you sources then you can decide if you trust them. AI generates code and you have no idea if what it generated was perfect, slop or almost perfect but bad in some major way.
Nope! I use llms like I used to use Google and StackOverflow. Really great to bounce ideas off of, brainstorm, yes save time too ... But the at the end of the day, what will separate programmers in the future is those that bother to put an ounce of effort into things, read, and learn. Versus those who want the easy button and answer given to them. They're going to be the next generation of people who leak your passwords, social security numbers, PII, and create massive global security incidents.
To be clear. AI doesn't kill people. People kill people. In this case, lazy people. Welcome to Idiocracy my friends.
One of the best things that I’ve done for myself, career-wise, came during a 6-month sabbatical between jobs. I had a bunch of projects that I wanted to do, but I kept finding myself stymied by scope creep - when you don’t have any constraints, it’s hard to decide how big a project should be, and the projects would grow to these massive unfinishable beasts.
I decided to constrain myself to one week per project - whatever I could get done in a week was what I’d do for this project. The experience of going from zero to something useable in either a new language or a new framework or a new field in a week was enormously confidence-building - learning that I was actually pretty good at this whole programming thing and breaking through a bunch of walls I’d run into previously in my professional career gave me an incredible amount of trust in myself to be able to take on new challenges that really helped me when I was looking for a new job. At the same time, it reminded me what I loved about programming - in a week, I was able to build something that solved a problem I had or that someone I knew had, and I got to do so in a creative and intellectually challenging fashion.
If ever you find yourself in a position to take a couple months off between jobs, skip the leetcoding or whatever else and just build some toy projects - you’ll be amazed how much you already know.
I would just add that gen AI can actually be a fantastic aid when you're building some toy projects just for yourself.
I am primarily a backend engineer, though I am capable on the front end, just not particularly fast. I am especially slow at the voodoo that is CSS. In the past, I would have been turned off from doing some end-to-end personal projects because I know I would have been bogged down just battling CSS for hours and hours (it's particularly problematic that I'm slow at CSS but also a stickler for how I want things to look). With AI tools I can literally just say "make it pretty". It gets me about 85% of the way there, but importantly it gets me to the point where it puts in all the styling framework and then I just have to fix bugs/edit/tweak it, which goes very quickly.
So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
On the one hand, the machines can spin up these bridges over unfamiliar code chasms.
But on the other hand, you (at least I) have no investment in the code. No familiarity with it. No "skin in the game" when I can tell the AI to just redo it.
One way around that is by using the "Chat bots as search engine", and then, rather than cut and pasting stuff into your project, type it in by hand. Even if you're just straight up copying it, by typing it in you have a bit more connection to it. When I do that I can better see the unfamiliar patterns rise out of the code. It's much better for me than just reading it.
Also gives me opportunity to tweak it as I like, which can break things, which can teach me things.
Mind, I've been advocating this "type it, don't paste it" philosophy for years and years, for any code snippet found in a book or tutorial or wherever. It's just now with the bots, the code is more tuned to our exact use case, and the incentive for moving it over is less.
But I find grabbing a blob of code, pasting it into my project, and getting errors so that I can tell the bot whats wrong is not particularly satisfying.
Watching Claude figure out its own errors even less so. (It's very cool to watch, just not very joyful to me outside of "wow, that's cool".)
On the other hand, I have no problem pasting a snippet back with a "what exactly is this doing here" to get better explanations of what I'm seeing.
The joy of personal projects is they can look like shit! You'd also be surprised how much you can learn by just screwing around with something for a day. A big part of the whole exercise for me was getting out of my comfort zone and learning that I can learn. I'm primarily backend as well, and came up through the ops side (as opposed to formal CS), which always led to a lot of imposter syndrome, so a lot of the benefit came from just putting myself in an uncomfortable position and just staying there long enough to work my way through it.
That said, my goal for this was to wear the hair shirt (and also this was before GenAI was really a Thing) - if you just want to make something neat, absolutely vibe your way there.
> So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
I'm the opposite of you (my strength is in voodoo) but my experience is the same. Whereas before I'd spend 2-3 weekends getting bogged down on elements that aren't my strengths and never complete, now I can easily whip up a mostly functioning web app (just for me) in just a few hours.
This is how 100% of non-technical executives view software engineers, really. And probably explains a lot of the hysteria (unfounded as I may believe it to be) around LLMs eating the industry.
A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?
If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Within a job, what matters is the _team learning_, and it's roughly the same idea. At least one person in the team should be able to explain anything the team does. Trusting it to an AI puts the whole team in a precarious situation, even if it is something as simple as a CSS layout.
I ended up migrating to fixing things in the libraries I use because they piss me off.
Bad onboarding docs, broken or bent SDLC, major performance issues. I spent all day yesterday fixing perf issues in a library that competes with one I loathe for trying to complect an entire problem domain.
You probably have the same thing I do. Teamwork is a very different motivation than working on something either never intended for others or with a long MVP cycle that makes it too easy to get bogged down with side quests. "It would be way less tedious to deal with X if I did Y first. But now Y is blocked so I do nothing."
In my younger years, classic ASP + SQL was easy, and I had IIS + SQL Server set up on a server at my house. And I could write HTML/CSS/JavaScript.
Deployment was... run update scripts, push ASP files via FTP.
Now I know there are newer ways to do things, and I'd love to set up CI/CD in my personal life, but every time I want to do a "toy project" I get hung up researching and thinking about my own personal software development life cycle.
What you choose for hosting and deployment for your toy projects?
I've got a VPS and domain I've had for a long time - for most of these, docker compose + the VPS did the job just fine. I was in the same boat - I'd fallen behind on a lot of the ops stack, and project setup wound up being a tar pit for a couple projects before I got frustrated and just said "Fuck it" and did it the dumb way, which wound up being the correct approach.
I found the trick with the personal projects was to pick the thing I was trying to learn with that project and focus on that - if I was trying to learn React, don't also try to learn kubernetes or lambdas or anything else. You really have to be comfortable with your project kind of sucking for all this to work.
So I'd say if you want to learn proper modern deploy + CI/CD + etc, take a project where you already know the rest of the stack (hell, make it Hello World if you have to) and use that. Keep things simple and in scope, get something stupid working, and then iterate to make it better.
For most of my toy projects, it's a systemd service file and rsync + restart command for deploys. Might have to SSH in and run an install command occasionally.
You can get a VPS in any cloud provider and treat it as your server, just copy files to it. It depends on the scope of your projects, though, if you need a lot of resources it can get pretty pricey for just toy projects.
I pay $12 for a server on Digital Ocean, and I host multiple servers behind an Nginx. But I am primarily a frontend/Desktop developer, so they are simple.
I am in such a phase as I write this. What should have been a week-long project has already gone through three and not finished. This is a simple multi-platform app I am building in Flutter.
One challenge I face is to be able to build something useful within a week. In two projects, I could make wirhin two days each.
For the Flutter one mentioned above, it took me two-three days just to get the tooling/SDKs to work and get me to a hello-world program working on Windows. Then it took about a week to program with all features I needed.
And then, damned again -- Took me a couple days to get it to build APK (random errors; the issue in the build tooling is still open). And then a week of craziness to sort out Android permissions. I still have the latter unresolved.
It took me 8 months of active shopping to find a job w/ 25 years of experience. Unless you are really good at interviewing with an interesting resume, I'd suggest caution until the job market recovers.
I'm single, rent controlled, and was working at a FAANG-ish company.
The fear was real, for sure, but it honestly made a huge, huge difference for both my mental health and my employability afterwards. I also got to spend some time figuring out what I actually wanted to be doing, so I had a much clearer picture of where my skills and interests aligned after, which helped the search as well.
For what it's worth, the two tricks I found for leet-code interviews were 1) do the simple stupid thing first - get something _working_, and then make it good and 2) explain what you're doing and what you're thinking about to your interviewers. I'm still not Great at the leet-code style, but you can get a lot of the way there with some basic knowledge about algorithmic design and complexity and data structures. In general, I've found finishing the problem and being able to talk coherently about it trumps finding the ideal solution.
(I also code like a psychopath - I'm constantly rewriting what I'm writing while I'm writing it, so a) slowing down and b) telling interviewers what I'm doing and why made that look less chaotic.)
I'm also not the person you want to hire if you're looking for deep algorithmic knowledge, though - I've got a long career of delivering high impact systems and solutions, but if you want someone to rewrite the index parsing algorithm in your mature database software to minimize lookup speed, you've got the wrong guy.
Toy Software is a lot like working on your bike/car/boat/whatever.
Working on your bike is fun. Working on the bike you need to ride to work tomorrow is stressful.
I miss writing toy software. It’s fun. But inevitably I want to use said software and that’s where the trouble starts. You find all the bugs and none of the time to fix them.
I recently wrote my own invoicing application. I got wrapped up in the joy of adding the features I wanted. Many of these features come at a premium monthly fee for a big named product.
I needed to get an invoice out in a timely fashion. As much as I wanted to use my app, I found certain kinks I needed to work out (with styling, adding addresses, etc.) -- This was where I realized what you have articulated.
At some point, it becomes better to prioritize the "fun" in working on my bike, and the "usefulness" of the daily driver bike.
In doing so, perhaps the fun/usefulness of each converges over time.
I used to self-host my email. I don't any more, for exactly this reason. And of course this applies to every other "just do it yourself!" enthusiast appeal. I actually want to delegate it to someone who does it full time.
Strange example, because self-hosted email has been pretty much setup-and-forget for me. I’ve been hosting my own email for several domains on a $5 VPS for over a decade and it’s largely just hummed along, without needing babysitting. Very rarely, some ISP’s spamblocker gets a little too squirrelly and I have a brief deliverability issue, but those are rare and easy to resolve.
I also have more "working software" interests than i have time and energy. Many of them have a ton of mundane boring details i've done a thousand times before. I'm actually a bit excited that i can work on what i want to work on.
With that said it's a challenge and a chore on its own to curate what "AI" writes. So it's definitely not all roses.
Disclaimer: I'm still early in the experimentation phase. We'll see if i hold these opinions in a few months.
I actually only like working on software that has a real use, at least to myself. The coolest is when I spin something up fast at work and people start relying on it.
Same goes for my car, it's satisfying doing inexpensive fixes to keep it running. Wouldn't want some mid-2000s BMW project that's stuck in the garage half the time.
I dunno. Working on a bike you want to use to ride to work could be stressful if it is your only bike, you need it to get to work, and your manager will be really mad if you don’t show up to work one day. If there was any slip available it wouldn’t be so bad.
Actually, I really like this analogy because
1) We can’t really work on non-trivial fixes for our cars because they are too complex
2) We’ve totally structured our living situations around these over-complex beasts
3) Sometimes they do just break down at random, and it becomes a mini-crisis for everyone involved, which can’t really be resolved without missing a day of work to go to a professional
4) The bike is in exactly the same situation, except you can actually fix most bike problems yourself with a little kit
And yet, it is seen as a serious adult thing to have a car, instead of a bicycle.
Some thoughts:
Outside of, like, customer service, most places are throughput-oriented and it doesn’t even really matter all that much if somebody misses a day.
If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs, and able to fix our bikes. Maybe there’d be a bus route as well.
As someone who's stuck to owning a single bike for over a decade (tho I will probably change this in the coming year) for both athletic rides and commuting, I'll say the "it is seen as a serious adult thing to have a car" can sometimes make having a single bike worse.
If my bike flats and I'm out of tubes and I have a busy day planned, I have to take it to a local bike shop but those generally close pretty early. If I miss the window to take it to an LBS then I'm SOL. With a car there's generally a developed enough ecosystem that I can get a roadside assistance type service to put a spare on the car until I can actually get my car into a shop.
This is a good analogy for SaaS vs using a tinkerer tool. A tinkerer tool often can be customized exactly to your needs. I can work on my bike at 10 PM in my garage after dinner as long as I have the parts. But sometimes your tinkerer tool is going to leave you working all night or you'll be SOL. But SaaS is generally a full-time job for other folks. Sure you pay money, but there's fewer situations you're SOL.
(Personally when my bike is inop and I need to head to work I use transit.)
> 1) We can’t really work on non-trivial fixes for our cars because they are too complex
Learning how to fix my own cars is something I’ll always be grateful for. They’re not that complex. Most of it is just bolts and a diagram reference.
Sure, if you crack a piston cylinder you just need a new engine, that isn’t really fixable.
Brakes/rotors/calipers, it is almost criminal what you pay a shop for in lieu of doing it yourself. Spark plugs are super easy. Even things like taking the intake manifold off a direct-inject car to replace the fuel injectors isn’t terribly hard.
What has made cars complex is all the electronics that tie into it. The mechanics of an ICE car are mostly about the same.
> If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs
And then reality hits. You're married and your partner's job is on the other side of the city. You just bought and house and now you want to switch jobs but you'd be limited to only jobs in biking distance. eTc....
I bike to work (was 4 miles, now 2), but it's unrealistic to expect people to be in bicycle range of their jobs.
I really just write software for myself only. The time difference (and fun level) between writing software for yourself and writing production quality software is huge.
Feature doesn't work in Safari? Who cares, I don't use Safari anyway. No need to fix.
Perfect example for me is that I use beancount for tracking my finances. It broke one day because I tried upgrading some dependencies from v2 to v3. I tried messing around with the imports and some function names but it didn't help and after wasting 30m I decided to give up on the upgrade.
2 weeks later when I wasn't busy trying to get it to work, I gave it another shot. This time went so much more smoothly and now everything works (fingers crossed since I haven't actually tried out ALL the functionality).
I did this for years to learn computer graphics. I spent so many weekends and evenings building weird things that never made me a single dollar, but all the knowledge I gained eventually led to the job of my dreams. Here are some of the weird things I built:
I quite enjoy the spirit of the article, and I believe joy in programming has become even more important in the AI agent coding age we're leaving.
However, am I the only one finding those time estimates way too short?
I'm not the fastest programmer on Earth, but I'm not the slowest either, and I think most of those projects would take me a lot more time than those estimates, specially if I'd be working only 2-3 hours per day.
I feel most of those projects would take significant time researching and learning about the subjects, before even starting to code.
Example: recently I replaced my Pelican blog by my own hacky static site generator written in Odin, working 2-3h per day, it took me 2 weeks -- and this is a simpler project than many on that list.
Your project is vastly more specified than a toy project, because it has a real customer (you) who will expect it to work after it ships. That expectation is what separates toys from real tools.
I bet you could write a word processor in an hour. It might only add or delete one character at a time (no selections). It might persist only to a hard-coded "output.txt" filename. You might have to kill the process because there's no quit operation. But that would suffice for a toy word processor.
Fair point, my example was indeed "shipped to production" and may not compare a throwaway static generator toy project.
I still think those estimates are off, because I think many of those projects would need significant research and learning time, possibly more than actually coding -- not to mention time spent troubleshooting when something goes wrong.
For me, one of the enjoyable aspects of toy projects is that there is no schedule to meet. Which means that I can take my time. And yes, that means I take much more time than the article implies. As an examine, I've been polishing my PEG-based Turing complete language (which features both AOT-to-C and a fully featured repl) since COVID was a daily aspect of life.
When I use dynamically typed languages, it's not necessarily the lack of types that make me write code quicker, it's the fact that I can call anything from anywhere with a complete disregard for whether I should.
When I'm working in statically typed languages, esp. those with explicit error handling (Go, Rust, Haskell, Odin, Zig etc), I find myself spending a lot of time thinking about edge cases, bugs etc because I am forced to do so and find myself thinking in terms of engineering rather than exploring.
I mean, all of those depend on how much was imported as third party dependency, how much was not addressed at all, and what was actually solved manually.
They're all doable if you focus on the core problems. He's got a github with many of the examples up, it's instructive on how to keep things small.
And I think it's important that (based on the code) he had a solid understanding of the problem space before starting to write code. You don't write that tight when you explore a new space.
I'm surprised by the number of bad takes on LLMs in this thread.
LLMs spoon-feed you with information about how things are implemented. You are not supposed to know how everything works when you start these projects. You're supposed to try your best, inevitably fail, then research the topic and understand where you went wrong, then adjust your approach. If you know how everything works and just follow the tutorial, you won't know what makes other methods fail, and by proxy what makes the one you chose work.
Write a language parser with a regex. Find out that it can't parse recursive statements. You've now learnt that regex can only parse a specific subset of syntaxes. Try to workaround this by pattern-matching the most nested statement first. Find out that it blows up performance. You now know more about time complexity and know what to watch out for when you write a real parser.
Write a non-optimizing compiler from scratch. Find out that you can't make do with unsound optimizations because you can't keep track of what optimizations are applied where. Find out that implementing sound optimizations is hard because you need to track use-def chains. Then you'll understand why SSA is used. Find out that code motion is a mess. Learn about sea of nodes. Merge every optimization pass into one because you're unable to order passes right. Learn how e-graphs solve this.
Write a layout engine. Get stuck on being unable to define what a "width" is. Workaround this with min/max/natural widths, introduce binary search, etc. Learn how this stuff works in practice (this is something I haven't personally done yet).
They say we learn from mistakes. Please don't let the smart (or "smart", depending on how you look at it) machine stop you from making them. It's not a teacher and it doesn't know how to educate.
A lot of people say if you don’t use LLMs then you will fall behind. I’m starting to think that not using them will be a significant advantage in the long run.
I think LLMs improve productivity in the present at a significant cost for the future. It's like cutting an R&D department. You might be able to utilize existing approaches better, but you won't make progress, and I think people are way too overconfident in believing everything important has already been developed.
I guess the counterargument here would be that LLMs could improve research as well by optimizing menial tasks. It's kind of similar to how computing has enabled brute-force proofs in math. But I think the fact that students are still required to prove theorems on paper and that problems with brute-force solutions are still studied analytically should show that tools like computers or LLMs are not at all a replacement for the typical research process.
IMO we are going to see a large class of people who have cognitive deficits brought on by AI tool usage.
I've been wondering lately about how to distinguish between tools that enhance your cognitive ability, and tools that degrade it. Jobs called a computer a "bicycle for the mind," and it seems like LLMs are an easy-chair for the mind. I'm not sure a priori how to distinguish between the two classes of tools though. Maybe there is no other tool like an LLM.
i think theres both. the LLM is an incredible tool you should be able to use well, but its a complement to your other knowledge and tools, not a replacement. if you dont add the LLM to your toolset, youre not going to be building at the same scale as people who are, and if you dont have the backing knowledge, your LLM outputs are gonna be junk because you wont be able to point it in the right direction soon enough in the context window
But your brain was the clipboard. That simple process of transcription was something that you couldn't avoid learning from even if you wanted to. You'd notice the connections between the commands you typed and things that happened when you ran the program even if you weren't trying to.
Things would start to click, and then you'd have those moments of curiosity about how the program might behave differently if you adjusted one particular line of code or changed a parameter, and you'd try it, which would usually provoke the next moment of curiosity.
This was how many of us learned how to write code in the first place. Pasting the output from an LLM into your source tree bypasses that process entirely -- it's not the same thing at all.
One thing I've learned about building toy software projects or even just small personal software (stuff that's actually useful) is to avoid implementing flexible configuration engines. I used to fall into this trap where I thought I should make my software to be configuration-centric so that it would be usable for others in case I decided to make my code FOSS. The problem with that is making a configuration engine is a lot less efficient than just writing the code to do the damn thing. I've almost never shared any code outside of small libraries that were very low in complexity, and I wasted a lot of time writing code for other hypothetical people when I should have just been writing it for myself.
It took me a solid decade to accept these lessons. I cannot build a product that even remotely competes with things like Visual Studio and .NET tooling. How many hours would it take a solo developer to recreate a debugger experience that anyone else could be bothered to look at for more than 5 seconds?
The hardest part of building something configurable is the UI/UX. The moment you step outside of the host language, your tooling support goes to zero. Sure, you can pick other popular languages as the language of configuration (SQL, python, Lua, etc.) and piggyback off those ecosystems. But, you still have to build a way for a "non programmer" to visit all of the patchwork sites of "configuration" in a coherent way. Telling the business people to go edit a folder full of .py and .sql files will probably not be received well. Wrapping all of this with a fancy web interface doesn't make the problem less severe either (it makes it substantially worse).
If you have a way to build/deploy your product in <60 seconds from source, worrying about elaborate configuration and customization architecture seems extremely idiotic to me now. The best path I've found is something approximating a common library and a per-customer executable that consumes it and implements the expected contracts. Each customer's project lives in a subfolder on the same monorepo. The power of the hierarchical filesystem is tragically understated throughout our industry.
This is also a great way to stand out among other candidates in the resume-review stage of a hiring process.
The number one question I have as a hiring manager is always "can this person actually write code". A GitHub profile with 2-3 projects with multiple commits to each one answers that question really well. I won't filter out people who don't have that - lots of great engineers have never published code before - but the people who do have that are much more likely to get picked for the phone screen.
The converse is actually a negative signal for me. If you list a GitHub profile on your resume, but that GitHub profile is a ghost town consisting exclusively of "test repo please ignore" and "freshman-year assignments I was required to fork from professor's repo," that tells me you don't have an above-average understanding of why you add something to your resume.
If you don't have an impressive GitHub profile, that's fine. To your point - the absence of it is not a penalty, and the presence of a good one is a very positive signal. But the presence of one that turns out to be a dud is a negative.
I've been using vhs ( https://github.com/charmbracelet/vhs ) to script gif creation- really showcases CLI workflows if you're demoing multiple commands.
This is great! Asciinema is well-made and a lot of fun, but it sadly doesn't show the recordings in motion right there in the markdown/README. Looks like vhs neatly solves that problem, appreciate the share.
Readit News
Am I the only one using LLMs as if they were a search engine? So before LLMs I was searching on Google things like "pros cons mysql mongodb". I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed). Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references". So, I get a quick overview of what to keep an eye on, and the references are there to avoid relying on hallucination.
It's true that sometimes I go ahead and say "give me a data schema for storing photos metadata in postgres. I wanna keep X in a different table, though" (or something like that). But I do that because I know very well what the output should look like (I just don't wanna spend time typing it, and sometimes I forget the actual type that I should use (int vs integer?)).
The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
But when asking for a full feature, I lost a full day trying to get it to stop chasing its tail. I’m still in the “pro” free trial period so it was using a frontier model.
This was for a Phoenix / Elixir project; which I realize is not as robustly in the training data as other languages and frameworks, but it was supposedly consuming the documentation, other reference code I’d linked in, and I’d connected the Tidewave MCP.
Regardless, in the morning with fresh eyes and a fresh cup of coffee, I reverted all the cursor changes and implemented the code myself in a couple hours.
#define PA17_EIC_LINE PIN_PA17A_EIC_EXTINT_NUM #define PA17_EIC_BIT PORT_PA17A_EIC_EXTINT1 #define PA17_PMUX_INDEX 8 //pa17 17/2 #define PA17_PMUX_TYPE MUX_PA17A_EIC_EXTINT1
And the output will be almost correct code, but instead of an answer being:
PORT_PA17A_EIC_EXTINT1
you'll get:
PORT_PA17A_EIC_EXTINT_NUM
and you can tell that it diverged trying to use similar tokens, and since _ follows EXTINT sometimes, it's a "valid" token to try, and now that it's EXTINT_ now NUM is the most likely thing to follow.
That said, it's massively sped up the project I'm working on, especially since Microchip effectively shut down the forums that chatgpt was trained on.
This is basically the only thing I use it for. It's great at it, especially given that Google is so terrible these days that a search describing what you're trying to recall gets nothing. Especially if it involves a phrase heavily associated with other things.
For example "What episode of <X show> did <Y thing> happen?" In the past, Google would usually pull it up (often from reddit discussion), but now it just shows me tons of generic results about the show.
You have to "gut check" the answers and know when to go deeper.
A lot of answers are low stakes, and it's OK to be a little wrong if it helps go in the right direction.
Google AI helpfully showed me this awesome CRD that created exactly what I wanted... sadly, there is no such CRD in reality
Dead Comment
In near future, companies will probably be able to pay lots of money to have their products come up better in the comparison. LLMs are smart enough to make the result seem "organic" -- all verifiable information will be true and supported by references, it will only be about proper framing and emphasis, etc.
I fully expect LLM results to start including ads, but because of the competition I hope/believe the incentives are much better than they are for, say Google's search monopoly.
It could potentially be more insidious though.
We'll probably start sending prompts to multiple models and comparing the results with lower-power local models.
I would never describe the output I've seen from LLMs as "organic".
The real issue isn't that LLMs lie, it's that they emphasize certain truths over others, shaping perception without saying anything factually incorrect. That makes them harder to detect than traditional ads or SEO spam.
Open-source LLMs and transparency in prompt+context will help a bit, but long-term, we probably need something like reputation scores for LLM output, tied to models, data sources, or even the prompt authors.
When LLM-generated content is pervasive everywhere, and the training data for LLMs is coming from the prior output of LLMs, we're going to be in for some fun. Validation and curation of information are soon going to be more important than they've ever been.
But I don't think there'll be too much intentional manipulation of LLMs, given how decentralized LLMs already are. It's going to be difficult enough getting consistency with valid info -- manipulating the entire ecosystem with deliberately contrived info is going to be very challenging.
Of course, it didn't last long, and trying to Google now is an exercise in pain and frustration. Lots of people have complained about the various things Google and marketers have done to get there, idk, I just don't like how it works now.
Top LLMs feel amazingly good at rapidly surfacing info online, and as I go through the references they're usually pretty good. I guess the same forces as before will apply, and there will be some window of opportunity before it all goes away again.
But an LLM digests everything, and then spits out information with the same level of detail, same terminology, and same presentation regardless of where it came from. It strips away a lot of the contextual metadata we use to weigh credibility and trust.
Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
Combined with RAG a self hosted LLM will definitely be able to deliver a more impartial and therefore better solution.
A recent example is when I implemented a (Kubernetes) CSI driver that makes /nix available in a container so you can run an empty image and skip a lot of infra to manage.
I talked to it a bit and eventually it wrote a Nix derivation that runs the CSI codegen for Python and packages it so I could import it. Then I asked it to implement the gRPC interface it had generated and managed to get a "Hello World' when mounting this volume (just an empty dir). I also asked it to generate the YAML for the StorageClass, CSIDriver, Deployment and DaemonSet.
So LLM left me with a CSI driver that does nothing in Python (rather than Go which is what everything Kubernetes is implemented in) that I could then rewrite to run a Nix build and copy storepaths into a folder that's mounted into the container.
Sure implementing a gRPC interface might not be the hardest thing in hindsight, but I've never done it before and it's now a fully functional(ish) implementation of what i described.
It even managed to switch gRPC implementations because the Python one was funky with protoc versions in Nix(Python bundles the grpc codegen it's so stupid) so i asked it to do the same thing for grpclib instead which worked.
Maybe my brain has been permanently altered by hacking Lisp.
so i control the logic, it handles the syntax.
asking it to solve problems for you never seems to really work, but it remembers syntax and if i need some kinda reader interface over another or whatever.
can’t help me with code reviews tho, so i spent most of my time reading code instead of remembering syntax. i’m ok with it.
Nope, you're not alone as I also do this. I'm also not using any AI IDE's (yet).
It's funny, I actually recently failed a live technical interview where I was using my LLM of choice to answer Google-like queries instead of an IDE like cursor. The interviewer told me that he had actually never seen anyone use AI for coding like that before. Up to that point, I assumed that most coders were mainly using AI as a search engine and not necessarily using AI IDE's yet. Are we really that rare?
They can get into small crevasses and the foliage and whatnot and they don't mind getting wet. They can fluster rabbits out. And are somewhat smart. But you still have to make the kill and you have to lead the dog, not the other way around.
"Copilot" is a great marketing name, but a bit deceiving.
This learning time that you welcomed is what you will now miss out on. The LLM gives you an answer, you don't know how good it is, you use it, and soon enough, if running into a similar issue, you will need to ask the LLM again since you were missing out on all that learning the first time which would have enabled you to internalize all the concepts.
It's like the regex engine example from the article. An LLM can create such a thing for you. You can read through it, it might even work, but the learning from this is orders of magnitudes less than what you get if you build this yourself.
Basically, if you use agentic tools (not just to look things up or get surface level answers, but write your code for you), then it's quite likely that your brain no longer does as much heavy lifting as it would have before, nor does anything you do to solve the issues you're working with have much staying power.
Over time, this will decrease our cognitive capabilities in regards to specific things, very much how the same has largely happened with language syntax knowledge thanks to various IDEs and language servers, auto-complete and so on, you no longer need to memorize as much, so you don't.
While outsourcing some of that doesn't seem too bad, it's hard to tell what will happen to our critical thinking skills in the long term, since we're not just getting rid of some technical implementation details, but making the tool do a lot of thinking for us. All while screwing over those who don't use these tools, because they have to deal with the full friction of solving the problems.
For code examples I've always found GitHub code search to be super useful.
I haven't really found a use case for LLMs where it's any faster than the research tools already available on the internet.
A good example is writing AWS CDK code, which is a pain in the ass. But there are so many examples to pull from on GitHub code search that are being used in real projects that I've found this method to be faster than prompting an LLM that may or may not be correct.
I got lucky sure, and sometimes I don’t get so lucky - but it works often enough to have become a part of my regular approach at this point. Why is the thing broke? Maybe the robot knows!
"explore the repo I've cloned at /src/foo and explain how it achieves barFeature. Now look a the project at /src/baz and tell me why it would be difficult to use foo's approach in baz"
I rarely have it do anything novel, just translate ideas from existing projects into mine. Novel work is for me to enjoy coding directly.
"give me the terraform configuration for an AWS ECS cluster with 2 services that can talk to each other, with one of them available publicly and the other one private"
Occasionally to give some small self-contained algorithms, for example:
Give me a way to format the difference between dates as human-readable text. For example: 1 day, 16 hours.
I quite like the search-first LLMs like Gemini and Copilot for this reason. They give you links which you can use to verify the output and seem to be less prone to directing you to SEO spam than Google Search/Bing.
Although I do partly think that search engines today are delivering poor performance compared to what they delivered historically so LLMs are benefiting from that.
Google agree with you and now at the top there is a AI generated answer, clearly labeled as AI generated, and it cites the sources. I was trying to escape AI, but I have to recognize the implementation by Google is quite good.
this afternoon i spent some time feeding context, with the idea of making a QnA context for maybe 50 different website routes, so i can have it ask me similar questions per route, so i can have a hell of a lot of pretty accurate documentation for an intern to pick up a couple to work on, or get some LLM to take a try at filling out the details.
im feeding it knowledge, with the output being documents, rather than me getting knowledge from it.
A year ago the AI would just make up some completely random stuff. The current crop do a very good job, but still not 100% correct.
They are wrong enough that I would be wary of using them to "teach" me topics I don't know about.
That's basically how I use them.
Do you often find the need to search for APIs that do not exist but wish they did, and what the code using these imaginary APIs might look like?
Ask it to generate a docker compose file and it will put a "Version:XX" at the top all the time even though its been deprecated for years.
In go it loves to suggest using io.util even though its also been deprecated for years. I even have it as a rule to "never use io.util" in cursor.
The downside however, is that at least 50% of the links cited no longer exists. This points to a general issue with LLMs, temporal knowledge. I tend to at least go to archive.org on a per-link basis when so inclined.. But clearly that's not ideal.
Does anyone have a better solution, short of just asking the LLM to remove the extra step, and link to the archive.org cache for the ingestion date (which frankly, I also don't have faith in being accurate).
It pulls in info from the docs, forums, code samples, shows me how things are connected, and may even suggest why something is good or bad.
Plenty of opportunity to look up docs I didn’t even know about yet.
Search finds you sources then you can decide if you trust them. AI generates code and you have no idea if what it generated was perfect, slop or almost perfect but bad in some major way.
To be clear. AI doesn't kill people. People kill people. In this case, lazy people. Welcome to Idiocracy my friends.
I decided to constrain myself to one week per project - whatever I could get done in a week was what I’d do for this project. The experience of going from zero to something useable in either a new language or a new framework or a new field in a week was enormously confidence-building - learning that I was actually pretty good at this whole programming thing and breaking through a bunch of walls I’d run into previously in my professional career gave me an incredible amount of trust in myself to be able to take on new challenges that really helped me when I was looking for a new job. At the same time, it reminded me what I loved about programming - in a week, I was able to build something that solved a problem I had or that someone I knew had, and I got to do so in a creative and intellectually challenging fashion.
If ever you find yourself in a position to take a couple months off between jobs, skip the leetcoding or whatever else and just build some toy projects - you’ll be amazed how much you already know.
I am primarily a backend engineer, though I am capable on the front end, just not particularly fast. I am especially slow at the voodoo that is CSS. In the past, I would have been turned off from doing some end-to-end personal projects because I know I would have been bogged down just battling CSS for hours and hours (it's particularly problematic that I'm slow at CSS but also a stickler for how I want things to look). With AI tools I can literally just say "make it pretty". It gets me about 85% of the way there, but importantly it gets me to the point where it puts in all the styling framework and then I just have to fix bugs/edit/tweak it, which goes very quickly.
So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
On the one hand, the machines can spin up these bridges over unfamiliar code chasms.
But on the other hand, you (at least I) have no investment in the code. No familiarity with it. No "skin in the game" when I can tell the AI to just redo it.
One way around that is by using the "Chat bots as search engine", and then, rather than cut and pasting stuff into your project, type it in by hand. Even if you're just straight up copying it, by typing it in you have a bit more connection to it. When I do that I can better see the unfamiliar patterns rise out of the code. It's much better for me than just reading it.
Also gives me opportunity to tweak it as I like, which can break things, which can teach me things.
Mind, I've been advocating this "type it, don't paste it" philosophy for years and years, for any code snippet found in a book or tutorial or wherever. It's just now with the bots, the code is more tuned to our exact use case, and the incentive for moving it over is less.
But I find grabbing a blob of code, pasting it into my project, and getting errors so that I can tell the bot whats wrong is not particularly satisfying.
Watching Claude figure out its own errors even less so. (It's very cool to watch, just not very joyful to me outside of "wow, that's cool".)
On the other hand, I have no problem pasting a snippet back with a "what exactly is this doing here" to get better explanations of what I'm seeing.
That said, my goal for this was to wear the hair shirt (and also this was before GenAI was really a Thing) - if you just want to make something neat, absolutely vibe your way there.
I'm the opposite of you (my strength is in voodoo) but my experience is the same. Whereas before I'd spend 2-3 weekends getting bogged down on elements that aren't my strengths and never complete, now I can easily whip up a mostly functioning web app (just for me) in just a few hours.
If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Within a job, what matters is the _team learning_, and it's roughly the same idea. At least one person in the team should be able to explain anything the team does. Trusting it to an AI puts the whole team in a precarious situation, even if it is something as simple as a CSS layout.
Bad onboarding docs, broken or bent SDLC, major performance issues. I spent all day yesterday fixing perf issues in a library that competes with one I loathe for trying to complect an entire problem domain.
You probably have the same thing I do. Teamwork is a very different motivation than working on something either never intended for others or with a long MVP cycle that makes it too easy to get bogged down with side quests. "It would be way less tedious to deal with X if I did Y first. But now Y is blocked so I do nothing."
Versus 'someone is waiting for this.'
Deployment was... run update scripts, push ASP files via FTP.
Now I know there are newer ways to do things, and I'd love to set up CI/CD in my personal life, but every time I want to do a "toy project" I get hung up researching and thinking about my own personal software development life cycle.
What you choose for hosting and deployment for your toy projects?
I found the trick with the personal projects was to pick the thing I was trying to learn with that project and focus on that - if I was trying to learn React, don't also try to learn kubernetes or lambdas or anything else. You really have to be comfortable with your project kind of sucking for all this to work.
So I'd say if you want to learn proper modern deploy + CI/CD + etc, take a project where you already know the rest of the stack (hell, make it Hello World if you have to) and use that. Keep things simple and in scope, get something stupid working, and then iterate to make it better.
I pay $12 for a server on Digital Ocean, and I host multiple servers behind an Nginx. But I am primarily a frontend/Desktop developer, so they are simple.
One challenge I face is to be able to build something useful within a week. In two projects, I could make wirhin two days each.
For the Flutter one mentioned above, it took me two-three days just to get the tooling/SDKs to work and get me to a hello-world program working on Windows. Then it took about a week to program with all features I needed.
And then, damned again -- Took me a couple days to get it to build APK (random errors; the issue in the build tooling is still open). And then a week of craziness to sort out Android permissions. I still have the latter unresolved.
I would love the opportunity to take ~6 months off, but I feel scared that if I just quit, it might take me a longer time to find a job again.
The fear was real, for sure, but it honestly made a huge, huge difference for both my mental health and my employability afterwards. I also got to spend some time figuring out what I actually wanted to be doing, so I had a much clearer picture of where my skills and interests aligned after, which helped the search as well.
(I also code like a psychopath - I'm constantly rewriting what I'm writing while I'm writing it, so a) slowing down and b) telling interviewers what I'm doing and why made that look less chaotic.)
I'm also not the person you want to hire if you're looking for deep algorithmic knowledge, though - I've got a long career of delivering high impact systems and solutions, but if you want someone to rewrite the index parsing algorithm in your mature database software to minimize lookup speed, you've got the wrong guy.
Working on your bike is fun. Working on the bike you need to ride to work tomorrow is stressful.
I miss writing toy software. It’s fun. But inevitably I want to use said software and that’s where the trouble starts. You find all the bugs and none of the time to fix them.
I needed to get an invoice out in a timely fashion. As much as I wanted to use my app, I found certain kinks I needed to work out (with styling, adding addresses, etc.) -- This was where I realized what you have articulated.
At some point, it becomes better to prioritize the "fun" in working on my bike, and the "usefulness" of the daily driver bike.
In doing so, perhaps the fun/usefulness of each converges over time.
I'm paraphrasing but -
"Do the hard work that you need to do, but always give yourself an hour a day for the bullshit 'i don't have time for this but I want to do it' stuff.
It'll keep you happy, which will keep you productive through the slog... and those little touches will stand out and make SOME people happy"
For the unfamiliar (the man, not the talk): https://en.wikipedia.org/wiki/Andy_Schatz
With that said it's a challenge and a chore on its own to curate what "AI" writes. So it's definitely not all roses.
Disclaimer: I'm still early in the experimentation phase. We'll see if i hold these opinions in a few months.
Same goes for my car, it's satisfying doing inexpensive fixes to keep it running. Wouldn't want some mid-2000s BMW project that's stuck in the garage half the time.
Actually, I really like this analogy because
1) We can’t really work on non-trivial fixes for our cars because they are too complex
2) We’ve totally structured our living situations around these over-complex beasts
3) Sometimes they do just break down at random, and it becomes a mini-crisis for everyone involved, which can’t really be resolved without missing a day of work to go to a professional
4) The bike is in exactly the same situation, except you can actually fix most bike problems yourself with a little kit
And yet, it is seen as a serious adult thing to have a car, instead of a bicycle.
Some thoughts:
Outside of, like, customer service, most places are throughput-oriented and it doesn’t even really matter all that much if somebody misses a day.
If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs, and able to fix our bikes. Maybe there’d be a bus route as well.
If my bike flats and I'm out of tubes and I have a busy day planned, I have to take it to a local bike shop but those generally close pretty early. If I miss the window to take it to an LBS then I'm SOL. With a car there's generally a developed enough ecosystem that I can get a roadside assistance type service to put a spare on the car until I can actually get my car into a shop.
This is a good analogy for SaaS vs using a tinkerer tool. A tinkerer tool often can be customized exactly to your needs. I can work on my bike at 10 PM in my garage after dinner as long as I have the parts. But sometimes your tinkerer tool is going to leave you working all night or you'll be SOL. But SaaS is generally a full-time job for other folks. Sure you pay money, but there's fewer situations you're SOL.
(Personally when my bike is inop and I need to head to work I use transit.)
Learning how to fix my own cars is something I’ll always be grateful for. They’re not that complex. Most of it is just bolts and a diagram reference.
Sure, if you crack a piston cylinder you just need a new engine, that isn’t really fixable.
Brakes/rotors/calipers, it is almost criminal what you pay a shop for in lieu of doing it yourself. Spark plugs are super easy. Even things like taking the intake manifold off a direct-inject car to replace the fuel injectors isn’t terribly hard.
What has made cars complex is all the electronics that tie into it. The mechanics of an ICE car are mostly about the same.
And then reality hits. You're married and your partner's job is on the other side of the city. You just bought and house and now you want to switch jobs but you'd be limited to only jobs in biking distance. eTc....
I bike to work (was 4 miles, now 2), but it's unrealistic to expect people to be in bicycle range of their jobs.
Feature doesn't work in Safari? Who cares, I don't use Safari anyway. No need to fix.
2 weeks later when I wasn't busy trying to get it to work, I gave it another shot. This time went so much more smoothly and now everything works (fingers crossed since I haven't actually tried out ALL the functionality).
How about working on your bike that other people also work on daily? That's work.
- Tiny ray-tracer: https://github.com/diegomacario/Super-Sunshine - 2D rigid-body simulator: https://github.com/diegomacario/Dyna-Kinematics - Character animation system: https://diegomacario.github.io/Animation-Experiments/public/... - Animation curve visualizer: https://diegomacario.github.io/Animation-Magic/public/index.... - Motion capture playback in the web: https://diegomacario.github.io/Hands-In-The-Web/public/index...
That's a mountain of code that served one purpose: to gain knowledge.
However, am I the only one finding those time estimates way too short?
I'm not the fastest programmer on Earth, but I'm not the slowest either, and I think most of those projects would take me a lot more time than those estimates, specially if I'd be working only 2-3 hours per day.
I feel most of those projects would take significant time researching and learning about the subjects, before even starting to code.
Example: recently I replaced my Pelican blog by my own hacky static site generator written in Odin, working 2-3h per day, it took me 2 weeks -- and this is a simpler project than many on that list.
I bet you could write a word processor in an hour. It might only add or delete one character at a time (no selections). It might persist only to a hard-coded "output.txt" filename. You might have to kill the process because there's no quit operation. But that would suffice for a toy word processor.
I still think those estimates are off, because I think many of those projects would need significant research and learning time, possibly more than actually coding -- not to mention time spent troubleshooting when something goes wrong.
When I use dynamically typed languages, it's not necessarily the lack of types that make me write code quicker, it's the fact that I can call anything from anywhere with a complete disregard for whether I should.
When I'm working in statically typed languages, esp. those with explicit error handling (Go, Rust, Haskell, Odin, Zig etc), I find myself spending a lot of time thinking about edge cases, bugs etc because I am forced to do so and find myself thinking in terms of engineering rather than exploring.
They're all doable if you focus on the core problems. He's got a github with many of the examples up, it's instructive on how to keep things small.
And I think it's important that (based on the code) he had a solid understanding of the problem space before starting to write code. You don't write that tight when you explore a new space.
Edit: https://github.com/ssloy?tab=repositories for the shortcut link :)
I mean regex is incredibly powerful. 5 days?!?!
LLMs spoon-feed you with information about how things are implemented. You are not supposed to know how everything works when you start these projects. You're supposed to try your best, inevitably fail, then research the topic and understand where you went wrong, then adjust your approach. If you know how everything works and just follow the tutorial, you won't know what makes other methods fail, and by proxy what makes the one you chose work.
Write a language parser with a regex. Find out that it can't parse recursive statements. You've now learnt that regex can only parse a specific subset of syntaxes. Try to workaround this by pattern-matching the most nested statement first. Find out that it blows up performance. You now know more about time complexity and know what to watch out for when you write a real parser.
Write a non-optimizing compiler from scratch. Find out that you can't make do with unsound optimizations because you can't keep track of what optimizations are applied where. Find out that implementing sound optimizations is hard because you need to track use-def chains. Then you'll understand why SSA is used. Find out that code motion is a mess. Learn about sea of nodes. Merge every optimization pass into one because you're unable to order passes right. Learn how e-graphs solve this.
Write a layout engine. Get stuck on being unable to define what a "width" is. Workaround this with min/max/natural widths, introduce binary search, etc. Learn how this stuff works in practice (this is something I haven't personally done yet).
They say we learn from mistakes. Please don't let the smart (or "smart", depending on how you look at it) machine stop you from making them. It's not a teacher and it doesn't know how to educate.
I guess the counterargument here would be that LLMs could improve research as well by optimizing menial tasks. It's kind of similar to how computing has enabled brute-force proofs in math. But I think the fact that students are still required to prove theorems on paper and that problems with brute-force solutions are still studied analytically should show that tools like computers or LLMs are not at all a replacement for the typical research process.
I've been wondering lately about how to distinguish between tools that enhance your cognitive ability, and tools that degrade it. Jobs called a computer a "bicycle for the mind," and it seems like LLMs are an easy-chair for the mind. I'm not sure a priori how to distinguish between the two classes of tools though. Maybe there is no other tool like an LLM.
Things would start to click, and then you'd have those moments of curiosity about how the program might behave differently if you adjusted one particular line of code or changed a parameter, and you'd try it, which would usually provoke the next moment of curiosity.
This was how many of us learned how to write code in the first place. Pasting the output from an LLM into your source tree bypasses that process entirely -- it's not the same thing at all.
http://mikehadlow.blogspot.com/2012/05/configuration-complex...
It took me a solid decade to accept these lessons. I cannot build a product that even remotely competes with things like Visual Studio and .NET tooling. How many hours would it take a solo developer to recreate a debugger experience that anyone else could be bothered to look at for more than 5 seconds?
The hardest part of building something configurable is the UI/UX. The moment you step outside of the host language, your tooling support goes to zero. Sure, you can pick other popular languages as the language of configuration (SQL, python, Lua, etc.) and piggyback off those ecosystems. But, you still have to build a way for a "non programmer" to visit all of the patchwork sites of "configuration" in a coherent way. Telling the business people to go edit a folder full of .py and .sql files will probably not be received well. Wrapping all of this with a fancy web interface doesn't make the problem less severe either (it makes it substantially worse).
If you have a way to build/deploy your product in <60 seconds from source, worrying about elaborate configuration and customization architecture seems extremely idiotic to me now. The best path I've found is something approximating a common library and a per-customer executable that consumes it and implements the expected contracts. Each customer's project lives in a subfolder on the same monorepo. The power of the hierarchical filesystem is tragically understated throughout our industry.
Dead Comment
The tip I can share is: publish them on GitHub.
You'll find out some of them are useful to a lot of people.
I have multiple toy repos with 20+ stars and one with 200+ stars.
Just put one screenshot and simple instructions to run on README.md, no need to write a manual.
As simple as `npm install && npm run start` is enough instruction for most.
The screenshot is important tho.
The number one question I have as a hiring manager is always "can this person actually write code". A GitHub profile with 2-3 projects with multiple commits to each one answers that question really well. I won't filter out people who don't have that - lots of great engineers have never published code before - but the people who do have that are much more likely to get picked for the phone screen.
If you don't have an impressive GitHub profile, that's fine. To your point - the absence of it is not a penalty, and the presence of a good one is a very positive signal. But the presence of one that turns out to be a dud is a negative.
Examples: https://github.com/bbkane