It is a little more than semantic search. Their value prop is curation of trusted medical sources and network effects--selling directly to doctors.
I believe frontier labs have no option but to go into verticals (because models are getting commoditized and capability overhang is real and hard to overcome at scale), however, they can only go into so many verticals.
OpenEvidence does use an LLM. It's ChatGPT for doctors/medical research, tuned to give references in respected journals etc. Hospitals (at least US, Canada, UK) allow and even encourage (US) staff to use it as a quick lookup, the way they'd otherwise Google for a dosage say, it just does that better.
(My wife's a hospital doctor & author and introduced me to it; other family in other countries.)
See this. I use OpenEvidence. It has access to full text from some of the major medical journals. But generalist models seem to outperform it. Not sure what is going on there.
btw - OpenEvidence is also the name that competitive debaters used for their giant archive of policy debate, LD debate, and (small amounts of) PF debate evidence. That project has been going on for decades now.
We turned that into a proper, ready-for-use-in-AI dataset and contributed it to the mainstream AI community under the name OpenDebateEvidence. Presented at NeurIPS 2024 Dataset and Benchmark track.
Much of the scientific medical literature is behind paywalls. They have tapped into that datasource (whereas ChatGPT doesn't have access to that data). I suspect that were the medical journals to make a deal with OpenAI to open up the access to their articles/data etc, that open evidence would rely on the existing customers and stickiness of the product, but in that circumstance, they'd be pretty screwed.
They’re one of the two big names in legal data - Thomson Reuters Westlaw and RELX LexisNexis. They’re not just search engines for law, but also hubs for information about how laws are being applied with articles from their in house lawyers (PSLs, professional support lawyers - most big law firms have them as well to perform much the same function) that summarise current case law so that lawyers don’t have to read through all the judgements themselves.
If AI tooling starts to seriously chip away at those foundations then it puts a large chunk of their business at risk.
TR will not disappear. But their value to the market was "data + interface to said data" and that value prop is quickly eroding to "just the data".
You can be a huge, profitable data-only company... but it's likely going to be smaller than a data+interface company. And so, shareholder value will follow accordingly.
The assumption is that Claude has access to a stream of fresh, currated data. Building that would be a different focus for Anthropic. Plus Thomson Reuters could build an integration. Not totally convinced that is a major threat yet.
So the value of a skill file is that it tells the model how to format its response for use within the software environment surrounding the model.
With programming, it's mostly about how to tell it to use some API.
But all the model can do is reply some text, and the actual work needs to be done by the software(the agent harness) which needs to parse the model response and translate it into actual work.
My point is there is no magic: the model just reads the skill file and then uses that as a template for a textual response, which is then parsed and processed by traditional software.
Yes, the model will read it and it will influence its response, but without some extensive software harness around the model to give it data for context and and so on: totaly useless.
Why? Because garbage in is garbage out.
So telling the model to review a contract and pay attention to "Whether indemnification is mutual or unilateral" will result is some response from the model, but without additional data it will be at the same level as what you can get from a google search.
The effect on established companies is exactly zero.
Now, having an in-house skills and proprietary software around the model to integrate it into your system, that would be valuable indeed, but not something an AI lab can replicate without building the whole company from scratch.
This is correct. AI is a huge boon for open source, bespoke code, and end-user programming. It's death for business models that depend on proprietary code and products bloated with features only 5% of users use.
> Could this lead to more software products, more competition, and more software engineers employed at more companies?
No, it will just lead to the end of the Basic CRUD+forms software engineer, as nobody will pay anyone just for doing that.
The world is relatively satisfied with "software products". Software - mostly LLM authored - will be just an enabler for other solutions in the real world.
There are no pure CRUD engineers unless you are looking at freelance websites or fiver. Every tiny project becomes a behemoth of spaghetti code in the real world due to changing requirements.
> The world is relatively satisfied with "software products".
you can delete all websites except Tiktok, Youtube and PH, and 90% of the internet users wouldnt even notice something is wrong on the internet. We dont even need LLMs, if we can learn to live without terrible products.
I think so too. But because of code quality issues and LLMs not handling the hard edge cases my guess is most of those startups will be unable to scale in any way. Will be interesting to watch.
Not if they don't have access to capital. Lacking that, they won't be building much of anything. And if there a lot of people seeking capital, it gets much harder to secure.
Capital also won't be rewarded to people who don't have privileged/proprietary access to a market or non-public data or methods. Just being a good engineer with Claude Code isn't enough.
I think companies will need to step up their game and build more competitive products with more features, less buggy and faster than what people can build
If it turns out that AI isn't much more productive, it could also turn out that people still believe it is, and therefore don't value software companies.
If that happens, some software companies will struggle to find funding and collapse, and people who might consider starting a software company will do something else, too.
Ultimately that could mean less competition for the same pot of money.
I left software about 10 years ago for this reason. I saw engineers being undervalued, management barriers to productivity and higher compensation possibilities for non-tech functions.
How do you feel about this in retrospect? Those observations sound heavily firm-dependent, but I would be interested in learning which non-tech functions offer higher compensation possibilities
Can this really be a kind of herding stampede behavior over Cowork? It’s been out several days now and just all the sudden today, all the traders suddenly got it into their little herd animal heads that everyone should rush to the exists… after that equally sketchy silver and gold rug pull type action last week?
Markets are not as efficient as the textbooks would have you believe. Investors typically rely on a fairly small set of analysts for market news and views. It might take those guys a while to think about stuff, write a note etc. The deepseek crash last year lagged by several days as well.
I'm out of the loop, but I thought there were sophisticated automated trading algorithms where people pay to install microwave antennas so they can have 1ms lower latency. And I thought those systems are hooked up to run sentiment analysis on the news. Maybe the news is late?
I am with you. I think that it is more likely to be related to Japanese carry trade unwind starting to worry the banks, while continuing to drive the “AI disrupt everything” narrative via mainstream news.
I might be not across the detail, but to me the legal plugin seems like it’s mostly adding some skills (prompts) that are fairly basic that any technically minded people could do, and is not enough of an improvement for completely non technical people to use.
It could also be that we have been in an economy-wide speculative bubble for a couple of years. Whispers of an AI bubble were a way to self-soothe and avoid the fact that we are in an everything bubble.
Readit News
How could it possibly keep up with LLM based search?
I believe frontier labs have no option but to go into verticals (because models are getting commoditized and capability overhang is real and hard to overcome at scale), however, they can only go into so many verticals.
Interesting. Why wouldn't an LLM based search provide the same thing? Just ask it to "use only trusted sources".
(My wife's a hospital doctor & author and introduced me to it; other family in other countries.)
See this. I use OpenEvidence. It has access to full text from some of the major medical journals. But generalist models seem to outperform it. Not sure what is going on there.
We turned that into a proper, ready-for-use-in-AI dataset and contributed it to the mainstream AI community under the name OpenDebateEvidence. Presented at NeurIPS 2024 Dataset and Benchmark track.
https://neurips.cc/virtual/2024/poster/97854
https://huggingface.co/datasets/Yusuf5/OpenCaselist
For example, only 7% of pharmaceutical research is publicly accessible without paying. See https://pmc.ncbi.nlm.nih.gov/articles/PMC7048123/
Edit: seems like it is ~10M USD.
If AI tooling starts to seriously chip away at those foundations then it puts a large chunk of their business at risk.
You can be a huge, profitable data-only company... but it's likely going to be smaller than a data+interface company. And so, shareholder value will follow accordingly.
The assumption is that Claude has access to a stream of fresh, currated data. Building that would be a different focus for Anthropic. Plus Thomson Reuters could build an integration. Not totally convinced that is a major threat yet.
I'm like: oh that's it, a bunch of skills files?
So the value of a skill file is that it tells the model how to format its response for use within the software environment surrounding the model.
With programming, it's mostly about how to tell it to use some API.
But all the model can do is reply some text, and the actual work needs to be done by the software(the agent harness) which needs to parse the model response and translate it into actual work.
My point is there is no magic: the model just reads the skill file and then uses that as a template for a textual response, which is then parsed and processed by traditional software.
So in terms of legal skills, a stand-alone skill like the contract review skill at https://github.com/anthropics/knowledge-work-plugins/blob/ma... is basically useless.
Yes, the model will read it and it will influence its response, but without some extensive software harness around the model to give it data for context and and so on: totaly useless.
Why? Because garbage in is garbage out.
So telling the model to review a contract and pay attention to "Whether indemnification is mutual or unilateral" will result is some response from the model, but without additional data it will be at the same level as what you can get from a google search.
The effect on established companies is exactly zero.
Now, having an in-house skills and proprietary software around the model to integrate it into your system, that would be valuable indeed, but not something an AI lab can replicate without building the whole company from scratch.
No, it will just lead to the end of the Basic CRUD+forms software engineer, as nobody will pay anyone just for doing that.
The world is relatively satisfied with "software products". Software - mostly LLM authored - will be just an enabler for other solutions in the real world.
> The world is relatively satisfied with "software products".
you can delete all websites except Tiktok, Youtube and PH, and 90% of the internet users wouldnt even notice something is wrong on the internet. We dont even need LLMs, if we can learn to live without terrible products.
Capital also won't be rewarded to people who don't have privileged/proprietary access to a market or non-public data or methods. Just being a good engineer with Claude Code isn't enough.
If that happens, some software companies will struggle to find funding and collapse, and people who might consider starting a software company will do something else, too.
Ultimately that could mean less competition for the same pot of money.
I wonder.
Deleted Comment
Something seems quite off. Am I the only one?
I might be not across the detail, but to me the legal plugin seems like it’s mostly adding some skills (prompts) that are fairly basic that any technically minded people could do, and is not enough of an improvement for completely non technical people to use.