This article is just gatekeeping the word "engineer" for no reason. When I was developing an agent recently I picked a model (thus fixing most of the "uncertainties" mentioned) and started to construct the system prompt, from simpler to more complex, sometimes back to simpler, discovering that to this particular model the name choice means a lot, observing how often I was or wasn't getting desired results, etc etc. I think it would be challenging to find a definition of the word "engineer" that wouldn't include this process, even if it, obviously, wasn't as complex as some other engineering projects.
Words have meaning. I'm not a prescriptivist but when you have an army of formerly PM's, Designers, etc all at once exclaiming "I'm an engineer too!" that should signal that maybe the thing they're doing isn't really engineering.
Do you count making a square space splash page 'engineering'? Tools improving to the point that the barrier of entry plummets is great. That doesn't mean you're now engaging in the same fundamental task that happened before things got easier/
Words have meaning, but we can still argue over the meaning of words. For example, I believe that engineering necessarily involves calculus, and if you’re not doing calculus ever in your job, you’re more of a “technician” or “specialist” than an engineer. I have a degree in chemical engineering, and have been titled “chemical engineer”, “controls engineer”, “electrical engineer”, and “field engineer”. Currently my title is “software engineer”.
But I believe the last time I did any engineering was when my title was “Intern” - none of my jobs since then have required actual rigorous engineering and could have been done equally well or better by someone without an engineering degree.
I currently believe “software developer” would be a more appropriate title for me.
> an army of formerly PM's, Designers, etc all at once exclaiming "I'm an engineer too!" that should signal that maybe the thing they're doing isn't really engineering
Not really disagreeing with your other points but this specific one is an ad hominem. Anyone can engineer, so the only signal to look at is what they are actually doing.
It otherwise implies that PMs, designers, etc. can’t be competent engineers, which I do not believe of the competent PMs, designers, etc. I’ve worked with.
If the thing standing in your way to engineering a great product was syntax, and a bunch of gate-kept and inconsistent linux cli tooling, then its fine to call it engineering.
Terms have meaning, and words can refer to different terms.
People quibbling about what's an engineer or not is about as helpful as the stupid hotdog sandwich thing, it's arbitrary an engineer is not a determinate thing.
AFAIK the reason why the word engineer has some specific clout people get touchy about is because in normal engineering fields becoming a licensed engineer is kind of a big deal for them so they get really particular about it. I only ever refer to myself as an engineer to bother people who get cunty about it. Get over yourself, why do you care if a designer calls themself an engineer? Are you worried it'll make it hard to find other true engineers so you will have a harder time finding civil engineers to talk about how calling apis is basically the same as building bridges?
Well, according to ABET Engineering is: “The profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind”.
By that definition, developing and training an LLM model could be argued to be a form of engineering… but exercising that model definitely isn’t. An analogy to mechanical engineering would be the difference between designing a CNC machine vs programming the machine to carve out a widget.
You've described what sounds a lot like experimentation: observing how a black box behaves on different inputs. Conducting experiments is closely related to engineering — I don't think anyone would claim different. But engineering is not limited to experimentation alone.
And anyone running a few ETL flows is a Data Engineer.
The gatekeeper holding closed the barn door on "engineer" got trampled long ago. Near as I can tell, it happened in the 90's, around the time lots of comp sci grads started hitting the profession, at a time when lots of programmers already in the field had come over from electrical engineering.
All sorts of human activities involve processes like this. Art, for instance.
Engineering is about precision it's not about this fuzzy sort of "iterate til it works" approach. Sure, iterations is involved, but knowing the precise conditions and bounds under which a system functions in a specific way is what engineering is actually about. I would not want a person building a bridge to use your process of iteration and guess work.
Yes, under this idea a lot of software engineering isn't actual engineering.
Under this idea, most engineering isn't engineering. Some of the oldest engineering is ad hoc battlefield engineering to construct siege engines and building tunnels to deliver explosives. That frequently only needed to work one time.
With bridges, you only need that high confidence because there are high costs and risks. Also, the stakeholders are usually governments, who require very predictable results. All that effort is worth it because the artifact will be useful for a long time for a lot of people.
It might be okay if some widget only lasts for 6 months. So, you empirically shave off material until it's as cheap as possible while failing at an acceptable rate.
The cost of shipping is low for software, so the risk profile is even more different. This can be shifted for high stakes software and I think there are some social issues there, but many things are shaped more like Facebook than aircraft control systems. They can fall over and no one's going to actually die.
I think the core of engineering is in evaluating these tradeoffs and figuring out where you can expend effort most efficiently.
This article is just gatekeeping the word "engineer" for no reason.
For a while in the 80's it was a pop culture joke that everyone was an "engineer."
Trashman: Sanitation engineer
Housewife: Domestic engineer
and so on.
The modern tech industry comes along and missed the part where the rest of society isn't impressed by slapping "engineer" at the end of your title unless you have a specific licensee or degree.
Honestly if we call software "engineering" in the first place, without any kind of certification/liability, I don't see the problem with applying it to everything else.
Indeed. Engineer is a silly word to gatekeep. Its original literal meaning is just someone who builds engines (in the original, broader sense of “engines” which basically means “machines”). The appropriation of the term to mean formalized categories with certifications etc. came hundreds of years later.
Prompting is more art than engineering. But understanding the basics of LLMs, which are engineered systems, helps you get better results. I think of it like being a good interviewer: you’re not controlling the conversation, but you know how to guide it. A good prompter, like a good interviewer, knows how to ask the kind of question that gets a meaningful answer. I think of LLMs like a giant library and my task is to steer it to the right sections for a good answer.
You're view of it is correct for agents but for my purposes, which are mostly brainstorming, researching and asa thinking tool, I have GPT 4o memory systems turned on. This allows it to be more personable and allows crafting of answers more attuned to my meaning. System and chat history memory obviously introduce global state which is not exactly a tunable hyperparameter for more consistent results. You should probably turn it off for agents so your view is accurate.
My definition of science is something like "a methodical approach to reach some truth" whereas engineering is something like "a methodical approach to reach some functionality". And I think some version of that is pretty universally accepted.
So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.
"Done properly" anything can be science/engineering. Just make a process, give it a KPI on how well it's performing, and you have an engineering. Movie making done properly is where audience is engaged with every act, you test screen it, see audience reaction, tweak, have a specific set of evals, and then you have something scoring high on the metrics. Part of that happens today, but you would not call that science. It's not what people mean when they call something as science or engineering. Your's is too broad of a definition to mean anything specific.
I have no problem taking issue with using the term "engineering" for this, but this reads like OP does not believe that you can get better results based on the way you structure your prompt and what information you include. That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs.
For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.
There are plenty of engineering disciplines that need to have statistical approaches to solving problems. Prompt engineering, should be approached by creating benchmark datasets and defining measures of efficacy and reliability.
More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present.
Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.
If a founder hires an engineer who develops a product, the founder is not magically an engineer. He hired the work out.
This isn’t complicated but people struggle to understand it for some weird reason.
If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer.
I could go on and on. Using an LLM to write a biology paper does not make that person a biologist.
Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet.
It wild that so many struggle with this basic ass concept.
Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt.
We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly.
But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
Prompting "ChatGPT" is very different than prompting LLMs. For the latter, whether you want to use the E word or not, every one of the unknowns the author cites either don't exist or are greatly reduced when you're using LLMs to construct something purpose-built.
Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.
Readit News
Do you count making a square space splash page 'engineering'? Tools improving to the point that the barrier of entry plummets is great. That doesn't mean you're now engaging in the same fundamental task that happened before things got easier/
But I believe the last time I did any engineering was when my title was “Intern” - none of my jobs since then have required actual rigorous engineering and could have been done equally well or better by someone without an engineering degree.
I currently believe “software developer” would be a more appropriate title for me.
Not really disagreeing with your other points but this specific one is an ad hominem. Anyone can engineer, so the only signal to look at is what they are actually doing.
It otherwise implies that PMs, designers, etc. can’t be competent engineers, which I do not believe of the competent PMs, designers, etc. I’ve worked with.
I dunno, do you read bytecode?
AFAIK the reason why the word engineer has some specific clout people get touchy about is because in normal engineering fields becoming a licensed engineer is kind of a big deal for them so they get really particular about it. I only ever refer to myself as an engineer to bother people who get cunty about it. Get over yourself, why do you care if a designer calls themself an engineer? Are you worried it'll make it hard to find other true engineers so you will have a harder time finding civil engineers to talk about how calling apis is basically the same as building bridges?
Dead Comment
By that definition, developing and training an LLM model could be argued to be a form of engineering… but exercising that model definitely isn’t. An analogy to mechanical engineering would be the difference between designing a CNC machine vs programming the machine to carve out a widget.
What's the analogous threshold to say you're engineering?
There was value in the word Engineer connoting professionalism and accountability. Checkers and draftsmen were checkers and draftsmen, not engineers.
Then, anyone who knew the difference between a patch cable and a crossover cable became a network engineer.
The gatekeeper holding closed the barn door on "engineer" got trampled long ago. Near as I can tell, it happened in the 90's, around the time lots of comp sci grads started hitting the profession, at a time when lots of programmers already in the field had come over from electrical engineering.
Engineering is about precision it's not about this fuzzy sort of "iterate til it works" approach. Sure, iterations is involved, but knowing the precise conditions and bounds under which a system functions in a specific way is what engineering is actually about. I would not want a person building a bridge to use your process of iteration and guess work.
Yes, under this idea a lot of software engineering isn't actual engineering.
With bridges, you only need that high confidence because there are high costs and risks. Also, the stakeholders are usually governments, who require very predictable results. All that effort is worth it because the artifact will be useful for a long time for a lot of people.
It might be okay if some widget only lasts for 6 months. So, you empirically shave off material until it's as cheap as possible while failing at an acceptable rate.
The cost of shipping is low for software, so the risk profile is even more different. This can be shifted for high stakes software and I think there are some social issues there, but many things are shaped more like Facebook than aircraft control systems. They can fall over and no one's going to actually die.
I think the core of engineering is in evaluating these tradeoffs and figuring out where you can expend effort most efficiently.
For a while in the 80's it was a pop culture joke that everyone was an "engineer."
Trashman: Sanitation engineer
Housewife: Domestic engineer
and so on.
The modern tech industry comes along and missed the part where the rest of society isn't impressed by slapping "engineer" at the end of your title unless you have a specific licensee or degree.
That's an interesting way to put it. indeed.-
And by “done properly,” i mean done in a regimented way with evals to verify that a wide range of inputs produce the desired outputs.
Prompting is much closer to discovering the properties of an already existing system than building something using engineering methods.
So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.
For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.
And here you're assuming I'm (the author) not doing that.
Just the fact that the US wakes up and the available compute goes down affects model output significantly more than any magical prompting.
I've literally had the same prompt on the same code produce exactly two different results (proper magic and complete broken hallucination).
I've had one-line "do it"s one-shot complex problems, and I've had detailed precise instructions completely ignored producing horrendous code (and vice versa).
Until you have a way to measure your "should be obviously", it's nothing but wishful thinking.
More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present.
Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.
This isn’t complicated but people struggle to understand it for some weird reason.
If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer.
I could go on and on. Using an LLM to write a biology paper does not make that person a biologist.
Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet.
It wild that so many struggle with this basic ass concept.
Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt.
We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly.
But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.