> Our diagnosis is that individual developers do not pay for tools.
I know this first hand, building a developer tool startup and failing to reach any level of revenue. In the end, the tech was bought out by a larger company to recover a fraction of our VC investment.
The challenge is that when you're building software for developers, they already know how it must work.
It's like trying to sell magic tricks to magicians. Sell magic to regular people, and you'll see some significant revenue.
I've used Kite before. It was ok. But I am a SWE. It's entirely possible that Kite would have seen major adoption if the push was towards non-technical folks trying to get their feet wet in software. Eg: Data scientists or business.
The reason why BI tools sell so well at the moment is that you have tons of C-level execs that like the appeal of a business-optimizing tool requiring little to none of any actual software development.
Let that be a lesson to everyone. You can't blow away developers. They're just too damn ~~smart~~ well-informed.
Edit: Another anecdote: A buddy of mine built a bespoke OCR and document indexing/search tool. He has ~60 paying clients (almost exclusively law-firms and banks) that primarily work with printed pages on paper. No Saas. No free tier. The client data resides on an on-premise Windows box, avoiding issues with sensitive data in the cloud etc.
He's a solo dev with support contracts and nets something like $1000/month from each client.
For your average lawyer/paralegal, the ability to locate and reference a single page from thousands of pages in under a second is magic. So they pay for it wholeheartedly.
I’m a web developer. My company pays for JetBrains IntelliJ for me. And I love it. But, if I had to pay for it out of my own pocket, I’d use VS Code instead. I’ve used both and IntelliJ is superior to VS Code, but not to such an extent that I would pay my own money for it. But I’m more than happy to have my company buy it for me.
JetBrains IDE's is the most notable exception of dev tools I personally pay for, insane value and productivity makes it a no-brainer purchase given its instant ROI from time saved. Life's too short to not maximize your productivity for a few $'s.
I have been paying for Resharper (and now Rider and DataGrip) for over fifteen years. Some companies where I worked paid for it other didn't but it didn't matter. It improves the quality of my code and I believe it gives me a competitive edge. It also helps keeping me sane as I work with a lot of legacy code.
I see it like a construction worker that uses his own tools instead of the broken down ones of the site. It makes good business sense that I can work faster and better when I use better tools. I also pay for other tools such as dbForge and SmartSvn/Git but Jetbrains' tools have been the longest running ones (I hope this does not sound like I am a fanboy or something).
This is a product that I actually _would_ pay for as an individual. It's reasonably priced and worth the increase in efficiency and better experience. Plus their pricing is fair and flexible.
Call me biased because I hate java. Oddly if a person hates Java, they should love intelliJ. I hated PyCharm because of its Java-centric heritage. I am a CLI guy. The closest I've come to loving an IDE is VSCode.
You may want to reconsider, or rather, consider the value of your own time.
If you value your time just even a little bit, consider how many of those tools are multipliers.
Obviously not 'JIRA' for a single dev, but in many cases JetBrains is worth it.
Would you wear Basketabll sneakers out for a jog? Well you could, but if you're going to run buy a pair of running shoes. Probably once a year. Costs about the same as Jetbrains for a year - as a very crude analogy.
I am in charge of the Software Team and do have financial support from the company e.g. if I deem a tool necessary it will be purchased – no, or little questions asked. It was insanly hard for met to justify, untill I saw the stuff marketing buys. Best decision to get the tool for our team
As a C dev I pay for CLion out of my own pocket. Partially because it's a great quality tool, and partially because there's not enough good quality tools for working in C on the market so I like to support the ones that do
I sat through scores of interviews and pairing sessions with developers back when Sublime was a thing and the vast majority (>90%) of devs would rather exit out of that pop-up asking for support then pay the measly $30 or whatever regardless of their massive incomes and increased productivity that Sublime brought them.
We developers are no more altruistic than anyone else regardless of the lies we fed ourselves in the early days of FOSS, internet, bitcoin, etc.
It's worth mentioning that a ST4 license is $99 USD (and ST3 used to be $80).
Still a relative drop in the bucket for how powerful ST is and how much value its users derive from it, but there _is_ a bit of sticker shock when comparing it to most other software.
I say this as someone who, as a broke college student, got very good at hitting esc every 10 times I saved (which is how often it asks). I eventually switched to VSCode, and the rest is history.
> the vast majority (>90%) of devs would rather exit out of that pop-up
Converting just 5% of users to paying customers is considered pretty good for shareware/freemium, so just because you saw a lot of people using the trial does not mean it is unsustainable. I have no idea how much the developer of Sublime Text makes, but considering that they have been around since 2008 I would assume it's definitely sustainable.
I'm a contractor so my case is a bit different, I can't expect a client to pay for every tool I want to use. But I gladly pay for Sublime Text and Sublime Merge because it makes my work more enjoyable and effective, and this is also good for whoever is paying me.
I pay for tools independently if they are affordable and I can use them commercially (even at work, even if my employer does not pay for it). I pay for JetBrains yearly and am at the lowest renew cost as a result, so theres no incentive for me to stop paying them yearly. I also saw that you can get Visual Studio Pro for $45 a month, which is really decent considering you get a professional grade IDE all to yourself.
The other thing is they have to be tools I want to use. I am an outlier I am sure. I hear often "let your employer pay for it" but they don't always necessarily pay for the tools I need to use. Having my own JetBrains license grants me strong freedom.
They do pay, but they can not pay same amount as some corporation can. If you target individuals you need to find good pricing model for them. The best model - at least for me, that always attract me is pay for a year of subscription to updates. After 1y passes you can pay at lower rate to continue receiving updates for new features and bug fixes or you can continue to use the latest version before your support has expired. I'm hooked instantly to this since it brings me value without constant commitment. One year I may decide to extend one tool, the other I pay extension for other tool.
Jetbrains' pricing model is also good, they reduce price each year (until 3rd), so you get rewarded for having a long term subscription. If you break commitment you get the regular pricing and you start over.
I remember trying Kite, but I removed it once I saw the pricing. It was more expensive than Jetbrains IDEs (which are less than 2$ a month for individuals when you pay in a bundle - 149$ at the time) which bring much more value for the money. For me it didn't make sense to pay 20$ for just incremental improvement (if even that) over Jetbrains Intelisense.
I (=as the employee of the company) often don't want to bother to pay, because a business case, then getting the invoice right (if that is even possible and the seller allows paying by PO), then getting it paid in time, is often not worth it. Often free alternatives exist, which makes it even more a no-brainer. Never mind that as a whole we are an enormous company but software dev is not core business so really we are a small part of a whole, yet some sellers only want to sell the most expensive enterprise tier.
Sublime is too expensive. $99 is too much. They should make a special christmas offer %90 off and many (of those who will never pay otherwise) will pay.
> towards non-technical folks trying to get their feet wet in software. Eg: Data scientists or business.
A bit tangential to the original post, but where does this belief that data scientists are non-technical folks? I am a data scientist myself, and in my view it's way more technical than most software development. Albeit I wouldn't still call neither data scientists nor software engineers "smarter" than the average.
Sure, if you want to train your bread and butter text classifier it just takes 10 lines of boilerplate code. But you don't need an AI-assisted tool for that - you just go to hugging face, copy paste those 10 lines, done, it's certainly faster than getting some AI-assisted code editor work for you.
For everything that is a bit more complicated, you need endless adjustments to your code, and it's quite unlikely more than a handful of people before you ever wrote the same code. It is, indeed, a somewhat painful and slow process (because just "testing" your code often takes minutes, if not hours, so finding out bugs becomes annoying). And a somewhat simple, AI-based, error highlight tool might be useful to weed out the most stupid ones and save some time.
But I will never trust something like copilot (or Kite, I guess, which I never tried) to write my code for me, as the challenging parts of the work involve long-term connection between different pieces of code (data loader, loss function, model function) that are written independently but must "cooperate" in a very non-trivial way. It is not at all uncommon that I make hours-long screen sharing calls with a colleague, discussing non-trivial mathematical computations, only to end up changing one or two lines of code that don't have an immediate link with the problem we are trying to solve.
This kind of things are notoriously hard for AI to grasp, so they can't do any decent job in writing that for me. Add on top that a lot of the code you find freely online is just ridiculously bad or broken, and you might only get unusable models generated by AI engines trained on those.
So, what kind of work are you referring to when talking about "data scientists or business" in your comment?
I found that "being technical" means different things to different people.
In the software world people seem to be referred to as technical when they write software systems not as much as singular scripts.
Data science is definitely technical but a lot of code work tends to happen in Jupyter notebooks or something similar. The main challenge is in understanding the ML/AI algorithms, the possible choices for your analyses that actually make sense for the problem, ... .
Besides that, due to the AI hype, there are so many people in data science who don't know much about coding or software engineering. Therefore, helping these people might be profitable (or not).
I work at a firm with many data scientists (I am one of them -- though my title has wavered back and forth between data scientist and ML engineer). Whether or not data scientists are "technical" and in what sense could be a difficult question to answer.
I can't speak very broadly, but at least for my company most data scientists are not doing the kind of work you describe. There certainly are some folks constructing and training complex machine learning models, but I think the majority work on the level of more basic statistical models and rules of thumb, where a project's final output might be a dashboard or presentation. Arguably some might refer to this as data analysis rather than data science, but none of these terms are particularly well defined.
That's not to say they aren't technical in some sense. All of them can and do code to one degree or another (with perhaps the exception of a small number of people who've been in the industry for decades), though not all of them do so with high proficiency or attention to software engineering best practices. That also goes for some of the engineers where I work, admittedly.
All in all, the broad level of technical aptitude has grown over the past few years. But not everyone with the title of data scientist is a machine learning specialist, nor are they necessarily skilled at software engineering.
Edit: As for Copilot, I found it worse than useless. It miserably failed every test I threw at it, from machine learning to (especially) Spark data pipelines, only redeeming itself with a string handling problem -- for which the solution was still entirely wrong but at least interestingly wrong. I frankly don't see how anyone pays for it, though perhaps it's better for projects with a ton of boilerplate.
>> Our diagnosis is that individual developers do not pay for tools.
> I know this first hand, building a developer tool startup and failing to reach any level of revenue.
Just because your startup failed doesn't mean an entire category is unsustainable.
I've been living from sales of a developer tool for the last 10 years, and there are plenty of other paid developer tools out there that show developers absolutely do pay for developer tools.
Now, maybe some of the startups have unrealistic expectations. A Python documentation reader probably wont turn into a billion dollar revenue company no matter how smart it is.
But I'm pretty sure there is a market for dev tools. Maybe the market is smaller, or harder to crack than you thought, but saying "there is no way" isn't going to help anyone.
Could you talk a bit more about your success? What does it do, what pricing model do you use, how long did it take you to acquire customers, do you feel it's worth it? Asking for a friend, of course.
> Our diagnosis is that individual developers do not pay for tools
I believe this is incorrect. I pay for many tools, but I would not pay for Kite. The problem is not that developers don't pay for tools, but that Kite, or AI-assisted code, does not address a pain point. It's a slight improvement, but I don't feel pain when I need to write code without it.
That's different from something like CI tools that I pay for. When I need to wait long for CI to finish I get annoyed. That's when I pay.
> It's like trying to sell magic tricks to magicians. Sell magic to regular people, and you'll see some significant revenue.
Just FYI, magic tricks are basically only sold to magicians. There's a thriving market of magic shops, especially online, where magicians go to buy new tricks (in the form of books/videos), new "gimmicks", etc.
I'd wager that a significant portion of all magic that is done is actually by magicians, for magicians, and partly in order to sell magic tricks.
You used magicians selling magic tricks to other magician's earlier, then said "smart" later.
I think it's just domain awareness, and the "they're smart" trope needs to be dismantled.
I think it plays into the technocracy problems we have now. We can solve it, we need more tech. More more more. People think we can solve social/political problems with tech - insidious.
I pay for tools, even though I'm kinda broke and would prefer to save a few hundred a year - my IDE delivers a lot of value and the support is excellent (thanks Jetbrains). I installed Kite briefly but it seemed so resource hungry I switched it off soon after without ever really trying to use it. That's not a judgement on Kite, I just didn't have the time or resources to spend at the time I encountered it.
I'm sorry it hasn't worked out for them, but they get my respect for this unusually frank self-assessment, real humility, and following through on the fine words with the actions of sharing their tools they built. They achieved a lot and I hope their future endeavors are wildly successful.
The challenge is that when you're building software for developers, they already know how it must work.
Do you mean that what you built didn't worked as it should? I don't understand, I've paid multiple times for tools that I find useful, even if they weren't perfect.
This misconception has been promoted by companies with an interest in promoting their platforms, using the expeditive procedure of subsidizing (often inferior) tools, with the collateral effect of making impossible for tools vendors to compete.
But by no means it's a law of physics. Make something programmers want. It's weird how little have the tools improved in twenty years.
What’s the best book/material on how to build a business like your friend’s? Discovering the niche, and expanding the client base are the main questions.
Be it as it may, everyone I know who tried Copilot trial is now paying for it. While my company expenses it, I started paying with my own money before that.
>> Our diagnosis is that individual developers do not pay for tools.
That was a really shocking insight for me. We do not own our means of production. And suffer all the textbook consequences that follow.
Maybe unions could help with that. Imagine using union-funded licenses, compute, storage etc. to experiment with your side projects, build your prototypes without risk of losing IP to your current employer.
I've paid for jetbrains multiple times, even though it's quite expensive for someone making money in a a third world currency. I pay for copilot.
However, I would never pay for stuff that I can get free. I could talk to my company to buy it, but I would settle for something close and free if it come to that.
People in 3rd countries use pirate copies / cracks very often. In my country (Moldova) that's absolutely OK having everything pirated even in government structures. AFAIK all JetBrains products are available cracked.
I am an individual developer and I pay for my PHPStorm and GitHub CoPilot - it saves me so much time, I could never imagine. This software is completely worthy to be paid for, even considering "free" copies are available.
OCR and document scanning companies is a big deal. I have a family member that used to do sales in this area and indeed companies with a lot of records and paper are paying big to get that digitized.
Maybe I am older, having started in the late 90's. But I think developer tools nowadays are so cheap compared to my salary that I pay for them without thinking twice.
I clearly remember paying around $80 USD for a WholeTomato [1] license back in 2003 for C# when I first came out of University into my first job. And that was a glorified auto-completer.
Software has turned really cheap. The downside of is that it is almost a commodity, but the development effort has not really decreased correspondingly.
If my employee would rather not pay for a tool, why would I spend my own money to help them save money?
The blog post was quite clear on the ”the tech doesn’t work” part, which seems like a more likely reason for their demise. Selling developer tools is hard, but selling non-functional tools is exponentially harder
It is surprising to me to see that you don't view yourself as a programmer. Maybe you're not a professional software developer, but writing code for fun sure sounds programmery to me.
"Our diagnosis is that individual developers do not pay for tools. Their manager might, but engineering managers only want to pay for discrete new capabilities, i.e. making their developers 18% faster when writing code did not resonate strongly enough."
I never used Kite, but I've tried Github Copilot twice, and found it marginal at best (and distracting at worst - which is why I turned it off both times). If Kite was similar, the reason I'm not paying is that coder AIs are not providing any value.
Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.
My experience with copilot has been very different. It easily pays for itself, and getting my employer (seed stage startup) to spring for it for the entire team was an easy sell.
Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.
The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.
My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
> Just the time saved writing tests alone pays for it.
This, so much. My code since using Copilot is easily ten times better tested than it was before, and I wasn't especially lazy when it comes to testing.
Given 1-2 hand-written unit tests, Copilot can start filling in test bodies that correctly test what's described in the function name. When I can't think of any more edge cases, I'll go prompt it with one more @Test annotation (or equivalent in another language) and it will frequently come up with edge cases that I didn't even think of and write a test that tests that edge case.
(One great part about this use case for those who are a little antsy about the copyright question is that you can be pretty darn confident that you're not running a risk of accidental copyright violation. I write the actual business logic by hand, which means copilot is generating tests that only interact with an API that I wrote.)
Testimonials of this form are near worthless to a company. Maybe it's true for you. Statistically, it's highly likely to be misleading.
People overestimate their willingness to pay for something for a number of reasons, but one of the biggest is that they incorrectly visualise what the choice to pay or not looks like. They often imagine a moment of abstract choice after which everything remains exactly the same but some small amount of money magically vanishes from their bank account. In reality, paying for something is a tedious inconvenience, and not paying for it more often takes the form of never getting round to putting your card details in than consciously deciding "this isn't worth it".
It can be taken to questionable extremes, but there's truth in the idea that the only real evidence as to what customers will do is what they actually do, not what they say they will do. I don't know if their interpretation is correct, but it sounds like Kite at least has evidence of the former sort.
^ This, I may not use copilot as much on production code, but the testing code it produces makes it easily worth it from a time saved and coverage perspective.
> Since we literally build all of this our B.S. detection meter is really high.
Oh no. The only thing that's high is our conviction that our BS meter is high. We fall prey to, come up with and promote as much BS as the next person.
I pay for copilot. It saves me a modest number of minutes of time per week. That's worth a small fee.
And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
> [we] use copilot basically a line or three at a time as a smart autocomplete.
I think this is the best way to think of CoPilot. GitHub is selling it like its going to write all your code for you, but in reality it is just next-generation auto-complete.
That's not a bad thing. In some ways I'd argue its actually better. GitHub needs to change its marketing because even most developers seem to think that its out there to take away our jobs. Its not and can not. But it provides the smartest auto-complete you've ever seen and that can be useful, especially when wading through mundane parts of your codebase.
It's really just perfect for remembering obscure things and can easily be prompted to generate the boilerplate. If you surround it with your style you will see it try to use the same techniques, however if you work on large code bases it gets annoying when it starts copying the bad habits you are trying to get rid of. In those cases it's actually kind of good for bringing to your attention that the building next door is still on fire.
I feel the problem with discussions about Copilot is that they consist of roughly two groups of people talking past each other. The first group believes that Copilot should be able to write code for whatever you tell it to write code for. The second group thinks Copilot is a fairly overpowered autocomplete.
The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way.
The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways.
I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components!
Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy.
Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
I'm not a lawyer, but my understanding is that even small snippets of licensed code can be problematic. I don't know exactly what the cutoff is, but when I tried Copilot it often suggested to auto complete snippets of code that were long enough that if I was intentionally coping them from a licensed codebase, I would handle the license. I'm not talking about whole giant functions, but small functions or large chunks of a function.
It is true that more commonly it suggested at most a few lines of obvious code which could really only be written the way it suggested, but a number of people in the comments on this article mentioned using Copilot to come up with test cases, so I think people are actually using it to suggest larger snippets of code.
Loads of devs I’ve spoken to, from junior to principal level, absolutely love Github Copilot though. Don’t know who is paying it for them, nor if Kite was significantly worse, but I think that at least Copilot has a brilliant future ahead of it.
I _love_ Copilot but the only reason I use it is because I qualify under their open source developers program, I just can't justify paying $10/month for it.
"Our diagnosis is that individual developers do not pay for tools."
I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions.
The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
The sheer volume of subscription services I've signed up for as the CTO for a startup is mind-boggling. $8 here, $19 there, $49 for something important, $99 for something essential.
Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools.
1) I need to justify these expenses each for what value they bring, some things are nice to have but bring so little value on paper.
2) You can't just enable tools for some people, there's huge overlap and that overlap gets greater
I get that people need to be paid, but these things very quickly add up. I'm paying about 7-13% of peoples salaries already in these subscriptions, and I feel like a total dick for saying no to people or trying to consolidate these.
I’d recommend anybody thinking about building a devtool to read Neil Davidson’s “Don’t roll the dice”. It’s a pretty old book, but Neil has also made it available for free now and the general lessons still hold true today.
Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
When I tried kite over a year ago I was relatively unimpressed with it. Even though it ran as a plug-in to jet brains IDE it required a windows installed service and two separate executables running in the background (kite.exe, kited.exe), and that stuff continue to run after exiting my IDE which was unacceptable for me.
Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
I'd be curious to hear about services/tools developers do pay for. The diagnosis that developers do not pay for tools seems off to me.
A few tools that I put on the company card when I worked at a Big Tech Co as an IC:
* DataGrip (Jet brains)
* Colab Pro (Google)
* Postman Pro
These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was.
For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
Yeah I called BS when I read that line too. I pay for plenty of dev tools. Similar list to you.
* JetBrains (PyCharm professional, DataGrip, and Goland) ~$250/yr
* Lucidchart (Diagramming) ~100 /yr
* Paw (HTTP Client) ~$50 /yr
* Docker Pro ~$60 /yr
I think there's probably more, but I'm not at my work laptop to look, but those are the big ones. Those are only individual subscriptions. There's also huge costs when associated with things like Gitlab Premium ($20 - $100 /user/month), CI/CD, Code coverage tools, security scanners, etc. Companies pay A LOT for development tools.
If Kite thinks that the problem why no one will pay $9/mo for their service is because developers or their company's are cheap, they need to re-assess. The reason they couldn't sell their service is because it wasn't providing enough value to justify it. But companies are paying hundreds of dollars a month per developer in most cases for various tools. The extra $9 for Kite isn't the dealbreaker if there was enough value from it.
I think “company card” is the differentiating point here. I’m not sure how many IC devs have access to that. Which makes me think you don’t fall into the group that they’ve defined as IC.
On the contrary, I find Github Copilot extremely helpful and saving me heaps of time. Yes, it's not writing the logic instead of me, but it acts like the best companion I could have in most of the cases.
I paid for intellij - damned near the entire architecture team where I worked had a copy, and the company sure as shit wasn't the one paying for it.
(I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.)
Developers will pay for software, if the value proposition is there.
> in part because they were too slow distancing themselves from Russia
I'd cut them some slack here. They had to get their team out of there first—with the way Putin is running things, they sure as hell couldn't announce they were leaving Russia until everyone who was going to follow them was out of there.
On the day of the invasion they tweeted a statement condemning the attack, and within two weeks announced they were leaving Russia.
> Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.
Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
With Copilot it's important to configure your editor so that it only generates completions when you ask for them. Then it can be very useful at times, especially when you're doing something you know is routine but you don't recall off-hand how to do (e.g. opening and writing to a file in append mode in a language you only use occasionally) Having it suggesting stuff every time you hit enter quickly gets annoying.
The tool just has to be very well integrated and easy to use. That's why copilot is seeing adoption, because Microsoft owns VSCode and has built a very simple integration of Copilot into VSCode.
That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
Copilot had 400k paying customers within it's first month [1]. I'm not a fan of mass generalization about large cohorts of people. Will everyone use a tool? Of course not. You just need dedicated early adopters that see the value add.
Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter.
I tried Kite and didn't even keep using the free trial or whatever the free tier was at the time. I don't think the issue is that developers won't pay for tools. Companies like Jetbrains are incredibly successful selling tools to developers. The issue for Kite is that Kite wasn't worth paying for.
Paying for copilot :) at least in go it’s great to write tests and sometimes some smaller functions :) totally worth paying for it, even from your own pocket if the company wouldn’t allow expensing it
If the company wouldn't pay for it then better think twice because you could get in hot water with legal. That's not a tool one's job or even company's business is worth risking over.
Copilot has a ton of still unresolved legal and compliance issues (copyright violation problems, sending proprietary code to Microsoft as you are writing it, etc.) and most larger businesses won't touch it with a 10 foot pole for that reason. There is even a class action lawsuit against Microsoft over Copilot already.
I pay out of pocket for a JetBrains license because it makes me a LOT more productive. I don't spend money on a lot of dev tools, but if it saves me non-trivial time, it's a no-brainer.
Yea.. i happily pay for several JetBrains tools and i'd love to pay for even more. I've got several problems that i don't want to spend time solving myself.
Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
That's why I tried it twice. I've been hearing people say they liked it. But I haven't found it very helpful, and often distracting, so I ended up turning it off. I'll probably try again next year when the models are improved to see if I feel any differently then.
Tabnine, in emacs, with lsp mode gives you single line completions and predicts and fills out word by word in a very effective manner. It's an actual timesaver and worth the money (in emacs). The vscode experience is more problematic, but that's vscode's completions guis fault, not the tabnine server's.
I disagree, Github copilot makes me happy, helps a lot with guessing patterns in my own project base code, I just write the good function name and he guess what I want to do.
“Our 500k developers would not pay to use it. Our diagnosis is that individual developers do not pay for tools.”
I don’t like depending on something I could lose in a month or tethers me to the internet. I consider that more a service than a tool. I’d prefer to just buy something once that just works, but that business model might be dead too since people will pirate things that aren't tethered to some serverside component.
I guess what I’m saying is that I want to buy tools, but people are only renting. Personally I’m largely holding out hope this becomes someone’s open source passion project and I can truly own my tools.
Software developers are also more likely than most to be "free software people". I for one am excited to see Kite go open source; if it's truly open, including the underlying recommendation models and algorithms, I will be happy to use it and set up a monthly donation for whoever wants to keep working on it.
Exactly. Non-free software is always paying for a service. If you don't get the source (and the ability to use it) you don't get the software. The source code is the software... the binary is an merely way to access a small part of it.
> If it's truly open, including the underlying recommendation models and algorithms, I will be happy to use it and set up a monthly donation for whoever wants to keep working on it.
Knowing examples such as Hudson CI & co, that probably makes it "no one", at a statistical scale.
I think it’s a combination of a few behaviors: Developers have this “if I can do it myself albeit in 10 times more time I won’t pay for the service even if it doesn’t make any financial sense whatsoever” and “This is cool but requires investment of my time while not providing way out of they start to suck or disappear” and “this is clever but what I need is help with the boring bits” mentality.
The stuff most developers are comfortable paying for is things like hosting, tools that do something the developers find very boring or have no domain overlap and don’t have viable free alternative.
“Why would I pay 9.99 if I can set up a free alternative in a few days and host it myself for 4.99? If I can’t host it myself I don’t trust you anyway”
I like what the other has said in under another comment — selling software to software engineers is like selling magic to magician. You are a magician, a very unfriendly use case to prioritize. It is reasonable for you to want what you want, but it would be suicide for the business to prioritize acquiring you as a customer.
Condolences to the Kite team. But, congratulations, too - you have some of the highest value engineering experience in the world. I'm sure you'll land somewhere great; try and take some time off if you can afford it!
Mulling over business models, and noticing the 'devs won't pay' narrative in the blog post, it's interesting to see the existing business models in AI; basically they seem to be:
* API-driven cloud calls (this is a way to get high value out of your existing cluster if you're AWS, MS, etc.)
* Platform play + possible eventual lock-in: OpenAI/Microsoft
* Subscription service for very specific needs (Grammarly, writing support)
I wonder if engineers would pay $9.99/month (or even $49.99/month) for a 'grammar checker for PRs' - essentially: "Avoid embarrassing bugs before you commit". That is, I wonder if Kite could have been successfully sold as the third tier - sub service for something very specific.
I guess if it's a good idea, someone could pull the Kite repos and launch it -- but my guess is there may be a market in there.
Devs almost always lack any kind of purchase authority. Any tool that appeals to devs needs to appeal to their management more, either showing some kind of cost savings over existing tools, increased dev productivity, or the new fangled "dev experience" where this tool, by shear awesomeness, will let devs put aside the low salary, process hell, and keep them employed.
I understand that's Kite's perspective (and yours -- "purchasing won't pay for this"), but devs are not paid meager salaries in general, and definitely might care about their code quality when it's put out in 'public' whether that be internal repos, or github.
Payscale estimates average engineering salary as having between $3,000 and $7,000 a month more in disposable income over writers -- and I would guess almost every professional writer pays for grammarly.
But, I agree that this is a new concept, and just spitballing -- right now, these sorts of linters and code formatting tools are mostly open source, so it would be some product marketing work to see if the market would actually pay.
If you don't have a massive platform (ie if you aren't Amazon or MS) the vertical end to end solution is much more easy to sell - ie sell the benefit directly.
"we were 10+ years too early to market, i.e. the tech is not ready yet."
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
Sounds like you know why people didn't pay for it. If it truly did make people as productive as you claim, it would have sold like hot cross buns on a cold day.
Exactly, they started a decade earlier and got outdone despite the massive head start. Then again maybe without the big data they had no hope of succeeding, but they should've mentioned that specifically instead of giving a bunch of contradicting statements.
If they're claiming 18%, it was probably more like 5% to 10% and it's really hard to sell something that's 5% better (especially when the alternative is free/ do nothing).
There are different types of market timing. They're right about the tech not being ready, but others are because you found the right "moment." Wordle was a success because it got traction in a covid winter. The exact same experience was just as viable 10 years earlier, but people weren't looking for entertainment in the same way.
Kite messed up privacy expectations one too many times by uploading everything in my home folder without consent. They were repeatedly shamed for this on HN and every time it seemed like they didn't understand why people were mad about consensual analytics.
The thesis that helping developers write code has value is flat wrong. We spend so much more time reading, reviewing, designing, arguing/bitching about code than we do writing it. Orders of magnitude more.
Any developer tooling company must understand this basic fact.
Indeed! Software development is not about writing syntax, but about "knowing what to write and where to put it".
Having a tool that rapidly creates setters, getters, or even common algorithms in function/method bodies is neat, crucial even. But also a problem that has mostly been solved for decades now.
The actual difficulty, where software devs spend (or should spend?) most time is indeed in what you say "reading, reviewing, designing, arguing". Where I'd like to add that the "arguing/bitching" is crucial if done with the right people (stakeholders, business, etc: creating a domain -or ubiquitous- language).
No AI can help me with that. And the current AIs make that worse. Rather than learning and applying ubiquitous language, rather than evolving a clean, maintainable architecture, it blurps a generic(ish) blurp of code. That often has no place where it was suggested, is inconsistent, breaks encapsulation or coupling and so on. If you blindly accept all the suggestions, the code often becomes worse fast; but you do write a lot of lines of code quickly. Whoever cares about that, though?
I've tried Kite once, and wasn't really impressed. For example, back when I tried it, it wouldn't offer any kind of autocompletion within a string. Even vim's built-in autocomplete tries to complete words for you there, based on other words you've used before.
Kite did sometimes offer some good suggestions in regular code, but it tried really hard to understand your code, and went belly-up when it didn't.
At that time, I tried some other ML-based autocompletion tool which wasn't specific to python, and which usually worked much better, except that it used far too much memory and caused regular crashes.
Maybe they improved kite since I tried it, or maybe "individuals don't pay for dev tools" isn't the whole story. Or maybe both.
Anyway, kudos for both trying and for open-sourcing the code at the end!
Readit News
I know this first hand, building a developer tool startup and failing to reach any level of revenue. In the end, the tech was bought out by a larger company to recover a fraction of our VC investment.
The challenge is that when you're building software for developers, they already know how it must work.
It's like trying to sell magic tricks to magicians. Sell magic to regular people, and you'll see some significant revenue.
I've used Kite before. It was ok. But I am a SWE. It's entirely possible that Kite would have seen major adoption if the push was towards non-technical folks trying to get their feet wet in software. Eg: Data scientists or business.
The reason why BI tools sell so well at the moment is that you have tons of C-level execs that like the appeal of a business-optimizing tool requiring little to none of any actual software development.
Let that be a lesson to everyone. You can't blow away developers. They're just too damn ~~smart~~ well-informed.
Edit: Another anecdote: A buddy of mine built a bespoke OCR and document indexing/search tool. He has ~60 paying clients (almost exclusively law-firms and banks) that primarily work with printed pages on paper. No Saas. No free tier. The client data resides on an on-premise Windows box, avoiding issues with sensitive data in the cloud etc.
He's a solo dev with support contracts and nets something like $1000/month from each client.
For your average lawyer/paralegal, the ability to locate and reference a single page from thousands of pages in under a second is magic. So they pay for it wholeheartedly.
I see it like a construction worker that uses his own tools instead of the broken down ones of the site. It makes good business sense that I can work faster and better when I use better tools. I also pay for other tools such as dbForge and SmartSvn/Git but Jetbrains' tools have been the longest running ones (I hope this does not sound like I am a fanboy or something).
When GitHub Copilot decided they wanted to charge $10 a month after the beta was over I noped out.
For what it does, it sure as shit isn't worth paying more than my yearly subscription to IntelliJ...
If you value your time just even a little bit, consider how many of those tools are multipliers.
Obviously not 'JIRA' for a single dev, but in many cases JetBrains is worth it.
Would you wear Basketabll sneakers out for a jog? Well you could, but if you're going to run buy a pair of running shoes. Probably once a year. Costs about the same as Jetbrains for a year - as a very crude analogy.
At least in some cases.
Would be silly for corpo to not buy it for developers
Sublime Text.
I sat through scores of interviews and pairing sessions with developers back when Sublime was a thing and the vast majority (>90%) of devs would rather exit out of that pop-up asking for support then pay the measly $30 or whatever regardless of their massive incomes and increased productivity that Sublime brought them.
We developers are no more altruistic than anyone else regardless of the lies we fed ourselves in the early days of FOSS, internet, bitcoin, etc.
:(
Still a relative drop in the bucket for how powerful ST is and how much value its users derive from it, but there _is_ a bit of sticker shock when comparing it to most other software.
I say this as someone who, as a broke college student, got very good at hitting esc every 10 times I saved (which is how often it asks). I eventually switched to VSCode, and the rest is history.
Converting just 5% of users to paying customers is considered pretty good for shareware/freemium, so just because you saw a lot of people using the trial does not mean it is unsustainable. I have no idea how much the developer of Sublime Text makes, but considering that they have been around since 2008 I would assume it's definitely sustainable.
Software developers very simply would rather build their own half assed solution to a problem rather than pay for a half assed solution.
Offer quality, we'll pay.
The other thing is they have to be tools I want to use. I am an outlier I am sure. I hear often "let your employer pay for it" but they don't always necessarily pay for the tools I need to use. Having my own JetBrains license grants me strong freedom.
Dead Comment
Jetbrains' pricing model is also good, they reduce price each year (until 3rd), so you get rewarded for having a long term subscription. If you break commitment you get the regular pricing and you start over.
I remember trying Kite, but I removed it once I saw the pricing. It was more expensive than Jetbrains IDEs (which are less than 2$ a month for individuals when you pay in a bundle - 149$ at the time) which bring much more value for the money. For me it didn't make sense to pay 20$ for just incremental improvement (if even that) over Jetbrains Intelisense.
Hijacking the quote...
I can't count the number of times I see well-paid developers using the Sublime Text trial.
A bit tangential to the original post, but where does this belief that data scientists are non-technical folks? I am a data scientist myself, and in my view it's way more technical than most software development. Albeit I wouldn't still call neither data scientists nor software engineers "smarter" than the average.
Sure, if you want to train your bread and butter text classifier it just takes 10 lines of boilerplate code. But you don't need an AI-assisted tool for that - you just go to hugging face, copy paste those 10 lines, done, it's certainly faster than getting some AI-assisted code editor work for you.
For everything that is a bit more complicated, you need endless adjustments to your code, and it's quite unlikely more than a handful of people before you ever wrote the same code. It is, indeed, a somewhat painful and slow process (because just "testing" your code often takes minutes, if not hours, so finding out bugs becomes annoying). And a somewhat simple, AI-based, error highlight tool might be useful to weed out the most stupid ones and save some time.
But I will never trust something like copilot (or Kite, I guess, which I never tried) to write my code for me, as the challenging parts of the work involve long-term connection between different pieces of code (data loader, loss function, model function) that are written independently but must "cooperate" in a very non-trivial way. It is not at all uncommon that I make hours-long screen sharing calls with a colleague, discussing non-trivial mathematical computations, only to end up changing one or two lines of code that don't have an immediate link with the problem we are trying to solve.
This kind of things are notoriously hard for AI to grasp, so they can't do any decent job in writing that for me. Add on top that a lot of the code you find freely online is just ridiculously bad or broken, and you might only get unusable models generated by AI engines trained on those.
So, what kind of work are you referring to when talking about "data scientists or business" in your comment?
In the software world people seem to be referred to as technical when they write software systems not as much as singular scripts.
Data science is definitely technical but a lot of code work tends to happen in Jupyter notebooks or something similar. The main challenge is in understanding the ML/AI algorithms, the possible choices for your analyses that actually make sense for the problem, ... .
Besides that, due to the AI hype, there are so many people in data science who don't know much about coding or software engineering. Therefore, helping these people might be profitable (or not).
I can't speak very broadly, but at least for my company most data scientists are not doing the kind of work you describe. There certainly are some folks constructing and training complex machine learning models, but I think the majority work on the level of more basic statistical models and rules of thumb, where a project's final output might be a dashboard or presentation. Arguably some might refer to this as data analysis rather than data science, but none of these terms are particularly well defined.
That's not to say they aren't technical in some sense. All of them can and do code to one degree or another (with perhaps the exception of a small number of people who've been in the industry for decades), though not all of them do so with high proficiency or attention to software engineering best practices. That also goes for some of the engineers where I work, admittedly.
All in all, the broad level of technical aptitude has grown over the past few years. But not everyone with the title of data scientist is a machine learning specialist, nor are they necessarily skilled at software engineering.
Edit: As for Copilot, I found it worse than useless. It miserably failed every test I threw at it, from machine learning to (especially) Spark data pipelines, only redeeming itself with a string handling problem -- for which the solution was still entirely wrong but at least interestingly wrong. I frankly don't see how anyone pays for it, though perhaps it's better for projects with a ton of boilerplate.
> I know this first hand, building a developer tool startup and failing to reach any level of revenue.
Just because your startup failed doesn't mean an entire category is unsustainable.
I've been living from sales of a developer tool for the last 10 years, and there are plenty of other paid developer tools out there that show developers absolutely do pay for developer tools.
Now, maybe some of the startups have unrealistic expectations. A Python documentation reader probably wont turn into a billion dollar revenue company no matter how smart it is.
But I'm pretty sure there is a market for dev tools. Maybe the market is smaller, or harder to crack than you thought, but saying "there is no way" isn't going to help anyone.
I believe this is incorrect. I pay for many tools, but I would not pay for Kite. The problem is not that developers don't pay for tools, but that Kite, or AI-assisted code, does not address a pain point. It's a slight improvement, but I don't feel pain when I need to write code without it.
That's different from something like CI tools that I pay for. When I need to wait long for CI to finish I get annoyed. That's when I pay.
I tried Kite a few years ago but didn't feel like I was getting much value out of it and never payed for it.
In contrast, I started paying for Github Copilot as soon as it was no longer free.
¯\_(ツ)_/¯
Just FYI, magic tricks are basically only sold to magicians. There's a thriving market of magic shops, especially online, where magicians go to buy new tricks (in the form of books/videos), new "gimmicks", etc.
I'd wager that a significant portion of all magic that is done is actually by magicians, for magicians, and partly in order to sell magic tricks.
I think it's just domain awareness, and the "they're smart" trope needs to be dismantled.
I think it plays into the technocracy problems we have now. We can solve it, we need more tech. More more more. People think we can solve social/political problems with tech - insidious.
I'm sorry it hasn't worked out for them, but they get my respect for this unusually frank self-assessment, real humility, and following through on the fine words with the actions of sharing their tools they built. They achieved a lot and I hope their future endeavors are wildly successful.
Do you mean that what you built didn't worked as it should? I don't understand, I've paid multiple times for tools that I find useful, even if they weren't perfect.
This misconception has been promoted by companies with an interest in promoting their platforms, using the expeditive procedure of subsidizing (often inferior) tools, with the collateral effect of making impossible for tools vendors to compete.
But by no means it's a law of physics. Make something programmers want. It's weird how little have the tools improved in twenty years.
That was a really shocking insight for me. We do not own our means of production. And suffer all the textbook consequences that follow.
Maybe unions could help with that. Imagine using union-funded licenses, compute, storage etc. to experiment with your side projects, build your prototypes without risk of losing IP to your current employer.
However, I would never pay for stuff that I can get free. I could talk to my company to buy it, but I would settle for something close and free if it come to that.
I am an individual developer and I pay for my PHPStorm and GitHub CoPilot - it saves me so much time, I could never imagine. This software is completely worthy to be paid for, even considering "free" copies are available.
Software has turned really cheap. The downside of is that it is almost a commodity, but the development effort has not really decreased correspondingly.
[1] https://web.archive.org/web/20030618043241/http://www.wholet...
Deleted Comment
The blog post was quite clear on the ”the tech doesn’t work” part, which seems like a more likely reason for their demise. Selling developer tools is hard, but selling non-functional tools is exponentially harder
Deleted Comment
I never used Kite, but I've tried Github Copilot twice, and found it marginal at best (and distracting at worst - which is why I turned it off both times). If Kite was similar, the reason I'm not paying is that coder AIs are not providing any value.
Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.
Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.
The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.
My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
This, so much. My code since using Copilot is easily ten times better tested than it was before, and I wasn't especially lazy when it comes to testing.
Given 1-2 hand-written unit tests, Copilot can start filling in test bodies that correctly test what's described in the function name. When I can't think of any more edge cases, I'll go prompt it with one more @Test annotation (or equivalent in another language) and it will frequently come up with edge cases that I didn't even think of and write a test that tests that edge case.
(One great part about this use case for those who are a little antsy about the copyright question is that you can be pretty darn confident that you're not running a risk of accidental copyright violation. I write the actual business logic by hand, which means copilot is generating tests that only interact with an API that I wrote.)
Testimonials of this form are near worthless to a company. Maybe it's true for you. Statistically, it's highly likely to be misleading.
People overestimate their willingness to pay for something for a number of reasons, but one of the biggest is that they incorrectly visualise what the choice to pay or not looks like. They often imagine a moment of abstract choice after which everything remains exactly the same but some small amount of money magically vanishes from their bank account. In reality, paying for something is a tedious inconvenience, and not paying for it more often takes the form of never getting round to putting your card details in than consciously deciding "this isn't worth it".
It can be taken to questionable extremes, but there's truth in the idea that the only real evidence as to what customers will do is what they actually do, not what they say they will do. I don't know if their interpretation is correct, but it sounds like Kite at least has evidence of the former sort.
(Don't have much experience with it)
Since we literally build all of this our B.S. detection meter is really high.
Kite thought it can go after the up and coming new developers by doing slightly shady things.
However, developers also have an incredible allergy to such tactics and it forever taints your brand.
So overall, developers do pay for tools, just not useless ones with shady growth tactics.
I briefly tried Kite a few years ago. I didn't notice anything shady although maybe I just didn't stick around long enough.
What shady tactics are you referring to?
Developers easily fool themselves thinking they’ll save $9 p/m by building something from scratch in 3 weeks.
Oh no. The only thing that's high is our conviction that our BS meter is high. We fall prey to, come up with and promote as much BS as the next person.
And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
I think this is the best way to think of CoPilot. GitHub is selling it like its going to write all your code for you, but in reality it is just next-generation auto-complete.
That's not a bad thing. In some ways I'd argue its actually better. GitHub needs to change its marketing because even most developers seem to think that its out there to take away our jobs. Its not and can not. But it provides the smartest auto-complete you've ever seen and that can be useful, especially when wading through mundane parts of your codebase.
The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way.
The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways.
I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components!
Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy.
Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
It is true that more commonly it suggested at most a few lines of obvious code which could really only be written the way it suggested, but a number of people in the comments on this article mentioned using Copilot to come up with test cases, so I think people are actually using it to suggest larger snippets of code.
"GitHub Copilot blocks your ability to learn." Is a common refrain.
I don't see ANY industry-wide consensus on whether GitHub Copilot truly helps developers right now.
The only scenario I can get anyone to agree on is generating templates. Aka, JSON or CSS files that you then edit.
[1] https://youtu.be/cdiD-9MMpb0?t=8723
Others have already pointed out the case as a reply.
Dead Comment
I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions.
The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
The sheer volume of subscription services I've signed up for as the CTO for a startup is mind-boggling. $8 here, $19 there, $49 for something important, $99 for something essential.
Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools.
Gitlab, Confluence, Jira, Asana, 1Password, co-pilot, codepen, sentry, jetbrains, gitlab plugins for jetbrains, Visual Studio, Docker Desktop, Perforce, Slack, etc;etc;etc;etc
Then there's things like Spacelift ($250!)!
The most frustrating thing is that:
1) I need to justify these expenses each for what value they bring, some things are nice to have but bring so little value on paper.
2) You can't just enable tools for some people, there's huge overlap and that overlap gets greater
I get that people need to be paid, but these things very quickly add up. I'm paying about 7-13% of peoples salaries already in these subscriptions, and I feel like a total dick for saying no to people or trying to consolidate these.
Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
Sounds like everyone has dodged a bullet!
A few tools that I put on the company card when I worked at a Big Tech Co as an IC:
* DataGrip (Jet brains)
* Colab Pro (Google)
* Postman Pro
These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was.
For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
* JetBrains (PyCharm professional, DataGrip, and Goland) ~$250/yr
* Lucidchart (Diagramming) ~100 /yr
* Paw (HTTP Client) ~$50 /yr
* Docker Pro ~$60 /yr
I think there's probably more, but I'm not at my work laptop to look, but those are the big ones. Those are only individual subscriptions. There's also huge costs when associated with things like Gitlab Premium ($20 - $100 /user/month), CI/CD, Code coverage tools, security scanners, etc. Companies pay A LOT for development tools.
If Kite thinks that the problem why no one will pay $9/mo for their service is because developers or their company's are cheap, they need to re-assess. The reason they couldn't sell their service is because it wasn't providing enough value to justify it. But companies are paying hundreds of dollars a month per developer in most cases for various tools. The extra $9 for Kite isn't the dealbreaker if there was enough value from it.
Maybe the product is poor. Maybe people didn’t believe the claim. But nobody said, “nah I don’t want my devs to be 18% faster.”
(I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.)
Developers will pay for software, if the value proposition is there.
I'd cut them some slack here. They had to get their team out of there first—with the way Putin is running things, they sure as hell couldn't announce they were leaving Russia until everyone who was going to follow them was out of there.
On the day of the invasion they tweeted a statement condemning the attack, and within two weeks announced they were leaving Russia.
https://twitter.com/jetbrains/status/1496786254494670851?lan...
https://blog.jetbrains.com/blog/2022/03/11/jetbrains-stateme...
Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
https://github.com/community/community/discussions/7553#disc...
https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_w...
That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter.
[1] https://www.ciodive.com/news/github-copilot-microsoft-softwa...
Copilot has a ton of still unresolved legal and compliance issues (copyright violation problems, sending proprietary code to Microsoft as you are writing it, etc.) and most larger businesses won't touch it with a 10 foot pole for that reason. There is even a class action lawsuit against Microsoft over Copilot already.
Indeed, I payed for a debugger because MSVC is pretty terrible
Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
I don’t like depending on something I could lose in a month or tethers me to the internet. I consider that more a service than a tool. I’d prefer to just buy something once that just works, but that business model might be dead too since people will pirate things that aren't tethered to some serverside component.
I guess what I’m saying is that I want to buy tools, but people are only renting. Personally I’m largely holding out hope this becomes someone’s open source passion project and I can truly own my tools.
Knowing examples such as Hudson CI & co, that probably makes it "no one", at a statistical scale.
The stuff most developers are comfortable paying for is things like hosting, tools that do something the developers find very boring or have no domain overlap and don’t have viable free alternative.
“Why would I pay 9.99 if I can set up a free alternative in a few days and host it myself for 4.99? If I can’t host it myself I don’t trust you anyway”
Mulling over business models, and noticing the 'devs won't pay' narrative in the blog post, it's interesting to see the existing business models in AI; basically they seem to be:
* API-driven cloud calls (this is a way to get high value out of your existing cluster if you're AWS, MS, etc.)
* Platform play + possible eventual lock-in: OpenAI/Microsoft
* Subscription service for very specific needs (Grammarly, writing support)
I wonder if engineers would pay $9.99/month (or even $49.99/month) for a 'grammar checker for PRs' - essentially: "Avoid embarrassing bugs before you commit". That is, I wonder if Kite could have been successfully sold as the third tier - sub service for something very specific.
I guess if it's a good idea, someone could pull the Kite repos and launch it -- but my guess is there may be a market in there.
Payscale estimates average engineering salary as having between $3,000 and $7,000 a month more in disposable income over writers -- and I would guess almost every professional writer pays for grammarly.
But, I agree that this is a new concept, and just spitballing -- right now, these sorts of linters and code formatting tools are mostly open source, so it would be some product marketing work to see if the market would actually pay.
"we were 10+ years too early to market, i.e. the tech is not ready yet."
"Our diagnosis is that individual developers do not pay for tools."
"We built the most-advanced AI for helping developers, but it fell short of the 10× improvement required to break through because today’s state of the art for ML on code is not good enough."
Sounds like you know why people didn't pay for it. If it truly did make people as productive as you claim, it would have sold like hot cross buns on a cold day.
> making their developers 18% faster
If they're claiming 18%, it was probably more like 5% to 10% and it's really hard to sell something that's 5% better (especially when the alternative is free/ do nothing).
Deleted Comment
"just" the code you were working on, surely?
Any developer tooling company must understand this basic fact.
Having a tool that rapidly creates setters, getters, or even common algorithms in function/method bodies is neat, crucial even. But also a problem that has mostly been solved for decades now.
The actual difficulty, where software devs spend (or should spend?) most time is indeed in what you say "reading, reviewing, designing, arguing". Where I'd like to add that the "arguing/bitching" is crucial if done with the right people (stakeholders, business, etc: creating a domain -or ubiquitous- language).
No AI can help me with that. And the current AIs make that worse. Rather than learning and applying ubiquitous language, rather than evolving a clean, maintainable architecture, it blurps a generic(ish) blurp of code. That often has no place where it was suggested, is inconsistent, breaks encapsulation or coupling and so on. If you blindly accept all the suggestions, the code often becomes worse fast; but you do write a lot of lines of code quickly. Whoever cares about that, though?
Kite did sometimes offer some good suggestions in regular code, but it tried really hard to understand your code, and went belly-up when it didn't.
At that time, I tried some other ML-based autocompletion tool which wasn't specific to python, and which usually worked much better, except that it used far too much memory and caused regular crashes.
Maybe they improved kite since I tried it, or maybe "individuals don't pay for dev tools" isn't the whole story. Or maybe both.
Anyway, kudos for both trying and for open-sourcing the code at the end!