I was one of the first hires on the Cyc project when it started at MCC and was at first responsible for the decision to abandon the Interlisp-D implementation and replace it with one I wrote on Symbolics machines.
Yes, back then one person could write the code base, which has long since grown and been ported off those machines. The KB is what matters anyway. I built it so different people could work on the kb simultaneously, which was unusual in those days, even though cloud computing was ubiquitous at PARC (where Doug had been working, and I had too).
Neurosymbolic approaches are pretty important and there’s good work going on in that area. I was back in that field myself until I got dragged away to work on the climate. But I’m not sure that manually curated KBs will make much of a difference beyond bootstrapping.
I was one of the first hires on the Cyc project when it started at MCC and was at first responsible for the decision to abandon the Interlisp-D implementation and replace it with one I wrote on Symbolics machines.
Yes, back then one person could write the code base
A coworker of mine who used to work at Symbolics told me that this was endemic with Lisp development back in the day. Some customers would think there was a team of 300 doing the OS software at Symbolics. It was just 10 programmers.
No not at all. We’re talking early-mid 1980s so people in the research community (at least at the leading institutions) were by then pretty used to what’s called cloud computing these days. In fact the term “cloud” for independent resources you could call upon without knowing the underlying architecture came from the original Internet papers (talking originally about routing, and then the DNS) in the late 70s
So for example the mail or file or other services at PARC just lived in the network; you did the equivalent of an anycast to check your mail or look for a file. These had standardized APIs so it didn’t matter if you were running Smalltalk, Interlisp-D, or Cedar/Mesa you just had a local window into a general computing space, just as you do today.
Most was on the LAN, of course, as the ARPANET was pretty slow. But when we switched to TCP/IP the LAN/WAN boundaries became transparent and instead of manually bouncing through different machines I could casually check my mail at MIT from my desk at PARC.
Lispms were slightly less flexible in this regard back then, but then again Ethernet started at PARC. But even in the late 70s it wasn’t weird to have part of your computation run on a remote machine you weren’t logged into interactively.
The Unix guys at Berkeley eventually caught up with this (just look at the original sockets interface, very un-unixy) but they didn’t quite get it: I always laughed when I saw a sun machine running sendmail rather than trusting the network to do the right thing on its behalf. By the time Sun was founded that felt paleolithic to me.
Because I didn’t start computing until the late 70s I pretty much missed the whole removable media thing and was pretty much always network connected.
I was born in late USSR and my father is software engineer. We had several books that were not available for "general public" (they were intended for libraries of science institutions). One of the book was, as I understand now, abridged translation of papers from some "Western" AI conference.
And there were description if EURISCO (with claims that it not only "win some game" but also that it "invented new structure of NAND-gate in silicon, used by industry now") and other expert systems.
One of the mentioned expert systems (without technical details) said was 2 times better in diagnose cancer than best human diagnostician of some university hospital.
And after that... Silence.
I always wonder, why did this expert system were not deployed in all USA hospitals, for example? If it is so good?
Now we have LLMs, but they are LANGUAGE models, not WORLD models. They predict distribution of possible next words. Same with images — pixels, not world concepts.
Looks like such systems are good for generating marketing texts, but can not be used as diagnosticians by definition.
Why did all these (slice of) world model approaches dead? Except Cyc, I think. Why we have good text generators and image generators but not diagnosticians 40 years later? What happens?..
I started my career in 1985, building expert systems on Symbolics Lisp machines in KEE and ART.
Expert systems were so massively oversold... and it's not at all clear that any of the "super fantastic expert" systems ever did what was claimed of them.
We definitely found out that they were, in practice, extremely difficult to build and make do anything reasonable.
The original paper on Eurisko, for instance, mentioned how the author (and founder of Cyc!) Douglas Lenat, during a run, went ahead and just hand-inserted some knowledge/results of inferences (it's been a long while since I read the paper, sorry), asserting, "Well, it would have figured these things out eventually!"
Later on, he wrote a paper titled, "Why AM and Eurisko appear to work" [0].
Depends on your definition of "super fantastic expert" systems.
I was one of the developers/knowledge engineers of the SpinPro™ Ultracentrifugation Expert System at Beckman Instruments, Inc. This was released in 1986, developed over about 2 years. This ran on an IBM PC (DOS)! This was a technical success, but not a commercial one. (The sales force was unfamiliar with promoting a software product, and which had little impact on their commissions vs. selling multi-thousand dollar equipment.)
https://pubs.acs.org/doi/abs/10.1021/bk-1986-0306.ch023 (behind ACS paywall)
Our second Expert System was PepPro™, which designed procedures for the chemical synthesis of peptides (essentially very small proteins). This was completed and to be released in 1989, but Beckman discontinued their peptide synthesis instrument product line just two months before. This system was able to integrate end-user knowledge with the built-in domain knowledge. PepPro was recognized in the first AAAI Conference on Innovative Applications of Artificial Intelligence in 1989.
https://www.aaai.org/Papers/IAAI/1989/IAAI89-010.pdf
Both of these were developed in Interlisp-D on Xerox 1108/1186 workstations, using an in-house expert system development environment, and deployed in Gold Hills Common Lisp for the PC.
One of the first things software engineers learn is that people are bad at manually building models/programming.
The language and image models weren't built by people but by observing an obscene amount people going about their daily lives of producing text and images.
> Cyc was used by the Cleveland Clinic for answering ad hoc questions from medical researchers; it reduced the time from as long as a month of manual back-and-forth between medical and database experts, to less than an hour.
I've read similar things about image models from 12 years ago beating the pants off most radiologists. I think the difference is that most writers, illustrators, musicians, drivers, etc. eke out a marginal living, while radiologists have enough reserves to fight back. The "move fast and break things" crowd in silicon valley isn't going to undertake that fight while there's still so much low-hanging fruit, ripe for the harvest.
I would love to see a Cyc 2.0 modeled in the age of LLMs. I think it could be very powerful, especially to help deal with hallucinations. I would love to see a causality engine built with LLMs and Cyc. I wrote some notes on it before ChatGPT came out: https://blog.jtoy.net/understanding-cyc-the-ai-database/
I used to volunteer inputting data into Cyc back in the day. And I get massive déjà vu with current LLM's. I remember that the system ended up with an obsession with HVAC systems lol.
When I got to go to Cycorp in the late 80's for training, I had some really interesting talks with the people there. They got funding from a lot of sources, and of course each source needed their own knowledge encoded. One person mentioned that they had a fairly large bit of the knowledge base filled with content about military vehicles.
I worked on Cyc as a visiting student for a couple of summers; built some visualization tools to help people navigate around the complex graph. But I never was quite sold on the project, some tangential learnings here: https://hyperphor.com/ammdi/alpha-ontologist
>if they could not come to a consensus, would have to take it before the Master, Doug Lenat, who would think for a bit, maybe draw some diagrams on a whiteboard, and come up with the Right Representation
So looks like Cyc did have to fall back on a neural net after all (Lenat's).
Has Cyc been forgotten? Maybe it's unknown to tech startup hucksters who haven't studied AI in any real way but it's a well known project among both academic and informed industry folks.
There is MindAptive who have something about symbolics as a kind of machine language interface that I think went the other way as in trying to do everything under the sun but its the last time I came across anything reminding me of Cyc
from 2015-2019 i was working on a bot company (myra labs) where we were directly inspired by cyc to create knowledge graphs and integrate into LSTMs.
the frames, slots and values integrated were learned via a RNN for specific applications.
we even created a library for it called keyframe (modeling it after having the programmer specify the bot action states and have the model figure out the dialog in a structured way) - similar to how keyframes in animation work.
it would be interesting to resurrect that in the age of LLMs!
The Cyc project proposed the idea of software "assistants" : formally represented knowledge based on a shared ontology, reasoning systems that can draw on that knowledge, handle tasks and anticipate the need to perform them.[1]
The lead author on [1] is Kathy Panton who has no publications after that and zero internet presence as far as i can tell.
Back in the mid 1990s Cyc was giving away their Symbolics machines and I waffled on spending the $1500 in shipping to get them to me in Denver. In retrospect I should have, of course!
Readit News
I was one of the first hires on the Cyc project when it started at MCC and was at first responsible for the decision to abandon the Interlisp-D implementation and replace it with one I wrote on Symbolics machines.
Yes, back then one person could write the code base, which has long since grown and been ported off those machines. The KB is what matters anyway. I built it so different people could work on the kb simultaneously, which was unusual in those days, even though cloud computing was ubiquitous at PARC (where Doug had been working, and I had too).
Neurosymbolic approaches are pretty important and there’s good work going on in that area. I was back in that field myself until I got dragged away to work on the climate. But I’m not sure that manually curated KBs will make much of a difference beyond bootstrapping.
Yes, back then one person could write the code base
A coworker of mine who used to work at Symbolics told me that this was endemic with Lisp development back in the day. Some customers would think there was a team of 300 doing the OS software at Symbolics. It was just 10 programmers.
I don't want to rob you of your literary freedom, but that threw me off. Mainframes were meant, yes?
No not at all. We’re talking early-mid 1980s so people in the research community (at least at the leading institutions) were by then pretty used to what’s called cloud computing these days. In fact the term “cloud” for independent resources you could call upon without knowing the underlying architecture came from the original Internet papers (talking originally about routing, and then the DNS) in the late 70s
So for example the mail or file or other services at PARC just lived in the network; you did the equivalent of an anycast to check your mail or look for a file. These had standardized APIs so it didn’t matter if you were running Smalltalk, Interlisp-D, or Cedar/Mesa you just had a local window into a general computing space, just as you do today.
Most was on the LAN, of course, as the ARPANET was pretty slow. But when we switched to TCP/IP the LAN/WAN boundaries became transparent and instead of manually bouncing through different machines I could casually check my mail at MIT from my desk at PARC.
Lispms were slightly less flexible in this regard back then, but then again Ethernet started at PARC. But even in the late 70s it wasn’t weird to have part of your computation run on a remote machine you weren’t logged into interactively.
The Unix guys at Berkeley eventually caught up with this (just look at the original sockets interface, very un-unixy) but they didn’t quite get it: I always laughed when I saw a sun machine running sendmail rather than trusting the network to do the right thing on its behalf. By the time Sun was founded that felt paleolithic to me.
Because I didn’t start computing until the late 70s I pretty much missed the whole removable media thing and was pretty much always network connected.
Also see: https://www.youtube.com/watch?v=cMMiaCtOzV0
I had learned about "AI" in the 80's. The promise was that with lisp and expert systems and prolog and more.
the article said cyc was reading the newspaper every day.
I thought, wow, any day now computers will leap forward. The japanese 5th generation computing will be left in the dust. :)
And there were description if EURISCO (with claims that it not only "win some game" but also that it "invented new structure of NAND-gate in silicon, used by industry now") and other expert systems.
One of the mentioned expert systems (without technical details) said was 2 times better in diagnose cancer than best human diagnostician of some university hospital.
And after that... Silence.
I always wonder, why did this expert system were not deployed in all USA hospitals, for example? If it is so good?
Now we have LLMs, but they are LANGUAGE models, not WORLD models. They predict distribution of possible next words. Same with images — pixels, not world concepts.
Looks like such systems are good for generating marketing texts, but can not be used as diagnosticians by definition.
Why did all these (slice of) world model approaches dead? Except Cyc, I think. Why we have good text generators and image generators but not diagnosticians 40 years later? What happens?..
Expert systems were so massively oversold... and it's not at all clear that any of the "super fantastic expert" systems ever did what was claimed of them.
We definitely found out that they were, in practice, extremely difficult to build and make do anything reasonable.
The original paper on Eurisko, for instance, mentioned how the author (and founder of Cyc!) Douglas Lenat, during a run, went ahead and just hand-inserted some knowledge/results of inferences (it's been a long while since I read the paper, sorry), asserting, "Well, it would have figured these things out eventually!"
Later on, he wrote a paper titled, "Why AM and Eurisko appear to work" [0].
0: https://aaai.org/papers/00236-aaai83-059-why-am-and-eurisko-...
I was one of the developers/knowledge engineers of the SpinPro™ Ultracentrifugation Expert System at Beckman Instruments, Inc. This was released in 1986, developed over about 2 years. This ran on an IBM PC (DOS)! This was a technical success, but not a commercial one. (The sales force was unfamiliar with promoting a software product, and which had little impact on their commissions vs. selling multi-thousand dollar equipment.) https://pubs.acs.org/doi/abs/10.1021/bk-1986-0306.ch023 (behind ACS paywall)
Our second Expert System was PepPro™, which designed procedures for the chemical synthesis of peptides (essentially very small proteins). This was completed and to be released in 1989, but Beckman discontinued their peptide synthesis instrument product line just two months before. This system was able to integrate end-user knowledge with the built-in domain knowledge. PepPro was recognized in the first AAAI Conference on Innovative Applications of Artificial Intelligence in 1989. https://www.aaai.org/Papers/IAAI/1989/IAAI89-010.pdf
Both of these were developed in Interlisp-D on Xerox 1108/1186 workstations, using an in-house expert system development environment, and deployed in Gold Hills Common Lisp for the PC.
The language and image models weren't built by people but by observing an obscene amount people going about their daily lives of producing text and images.
https://news.ycombinator.com/item?id=40070667
That's not true
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/
>Why did all these (slice of) world model approaches dead?
Because they don't work
So looks like Cyc did have to fall back on a neural net after all (Lenat's).
Bonus points if that is combined with modern differentiable methods and SAT/SMT, i.e. neurosymbolic AI.
https://mindaptiv.com/intro-to-wantware/
I think the issue in this area is mostly to convince and sell to bureaucratic institutions.
Deleted Comment
the frames, slots and values integrated were learned via a RNN for specific applications.
we even created a library for it called keyframe (modeling it after having the programmer specify the bot action states and have the model figure out the dialog in a structured way) - similar to how keyframes in animation work.
it would be interesting to resurrect that in the age of LLMs!
The lead author on [1] is Kathy Panton who has no publications after that and zero internet presence as far as i can tell.
[1] Common Sense Reasoning – From Cyc to Intelligent Assistant https://iral.cs.umbc.edu/Pubs/FromCycToIntelligentAssistant-...