Readit NewsAnd I am one of the best customers of these 3 physical shops, in my town.
So sure, I don't buy the latest trends based on ads. I investigate a lot to buy GREAT stuff. Sometimes the shopkeeper has headaches to find the obscure stuff I discovered online that NOBODY knows it exists.
Am I an exception?
I don't know but those services are great to maintain a freedom of choice.
The question is are there techniques we can adopt from bio neural nets that can enhance the training speed and efficiency of synthetic neural nets?
The results seem to indicate that the limiting factor is rather the hardware current-state AI is running on than the algorithms.
I think this may have something to do with the professionalisation of politics, or the existence of career politicians. If you want to climb up the ladder in politics, working on short-term goals is probably the best way to do this. Infrastructure projects are high-risk, low-reward. Infrastructure projects may take a long time, may be reversed/aborted by the next government, may piss off potential voters, may require to fight off NIMBYs, or aren't noticed due to the preparedness paradox.
Not quite as old, or at the scale of the Dujiangyan system, but still evidence that the “Western” culture did once build for long term. Less ancient, but more indicative, are the European cathedrals built by multiple generations over a century.
This is almost a cliche in reporting on China that seems to reflect a serious blindpsot in western media and/or business attitudes.
You can find plenty of articles complaining about "overcapacity" of battery factories in China even as they double in capacity and output each year.
Chinese electricity generation went from 4,000 TWh (the same as the US) in 2010 to double that in 2020. The US was basically the same after 10 years.
So a 100% "oversupply" in 2010 would be a zero percent oversupply within a decade given China's growth.
Most telling to me is that decarbonisation and electrification of transport and heating has long been known to require a doubling(!) of electricity production for developed nations (and a similar increase in developing nations where it gets hidden by other growth).
Apparently the US simply never had a plan to achieve that, and amazingly it still isn't part of the conversation around AI power. Instead they're just claiming the best parts of the existing power systems and passing the costs onto local consumers.
I wonder if this is more of a cultural thing, meaning Western cultures being more aligned to short-term gains instead of long-term gains. I mean, look at the Dujiangyan irrigation system that was build 2500 years ago and is still maintained until today. This isn't something the Western world would even consider.
But one has to imagine that seeing so many huge datacenters go up and not being able to do training runs etc. is motivating a lot of researchers to try things that are really different. At least I hope so.
It seems pretty short sighted that the funding numbers for memristor startups (for example) are so low so far.
Anyway, assuming that within the next several years more radically different AI hardware and AI architecture paradigms pay off in efficiency gains, the current situation will change. Fully human level AI will be commoditized, and training will be well within the reach of small companies.
I think we should anticipate this given the strong level of need to increase efficiency dramatically, the number of existing research programs, the amount of investment in AI overall, and the history of computation that shows numerous dramatic paradigm shifts.
So anyway "the rest of us" I think should be banding together and making much larger bets on proving and scaling radical new AI hardware paradigms.
Because the alternatives lack the breakthroughs that give them an edge against current-state AI and don't generate the hype like transformers or diffusion models. You have stuff like neuromorphic hardware that is hardly accessible and in its infancy, e.g. SpiNNaker. You have disciplines like Computational Neuroscience that try to model the brain and come up with novel models and algorithms for learning, which, however, are computational expensive or just perform worse than conventional deep learning models and may benefit from neuromorphic hardware. But again, access is difficult to such hardware.
* except if you are developing complicated algorithms or do numeric stuff. However, I believe that the majority of developers will never be in such a situation.