> I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
I think their move makes a lot of sense. Frontier models can probably not advance much further with the datasets we have currently available. Most of the text that goes in is online chatter, images and some scientific texts. That's great for chatbots, knowledge retrieval and programming. But with that database genuine discovery is hard to do. I think for the next step in intelligence the models need to have access to much more data: Data from physics, chemistry, biology experiments - and so on. And they need the data in much higher fidelity than you can currently access. If we just feed AI with all the knowledge we've acquired it's much harder for it to become smarter than us - it basically needs it's own eyes, ears, nose and so on.
That’s because a significant portion of the work and spending going on at frontier labs is generating new, more curated data, in select domains like software engineering and now biology[1].
I think that was his argument as well. It looks like there is no more data to find, but then it becomes important to find more data - lo and behold we can make more.
Sounds like the oil scare from 90' - we thought we were gonna run out of oil. But as oil gets more expensive it pays to dig further down to find the stuff that didn't make sense to dig up before.
I think there are multiple separate events here, timed to coincide in order to minimise disruption. It's hard to be sure whether they share overlapping causes, but I'd guess that there's at least an element of that.
Ed Zitron has predicted 16 of the last 0 AI bubble bursts...
But more seriously, this has to do with DeepMind falling way behind the frontier in intelligence and cost per task. There's basically no reason to use a Gemini model today.
Public warning: please don't trade on Ed's idiotic analyses. Or if you do, look at his track record so far, and apply the Kelley Criterion to your bet sizing.
"Based on estimates of their burn rate and historic analyses, I hypothesize that OpenAI will collapse in the next 12-24 months unless it raises more funding than in the history of the valley and creates an entirely new form of AI."
And they did raise new funding as he predicted...Since July 2024, OpenAI has raised or secured roughly $170 billion in new capital. Numbers never see before. So yes its the one, and remarkably right so far.
Notice how he didn't say "I predict they will raise new funding", he predicted collapse, with the implicit retort that there's no way they will raise more funding than has ever been done before and invent a whole new AI. He was mocking the supposed things that would need to happen. He strongly assumed collapse.
2 years these charlatans have had their "imminent collapse" predictions repeatedly wrong, meanwhile all valuations, AGI progress in unsolved problems, and frontier leadership have been repeatedly vindicated that we are beginning ASI / the singularity.
Each generation of model is more expensive to train and run than the last.
More efficient hardware? That means throwing out billions of dollars worth of existing hardware and buying billions more in hardware. The cheapest hardware is the stuff they already have because spending $70B in hardware to halve a $2-4B electricity bill just doesn’t make financial sense (and that’s if doubling hardware efficiency can happen before they go under).
If they cut costs by using smaller models, they are then racing to the bottom vs China which seems hard to do (especially with China’s cheap solar energy).
As it stands, the leaked financials prove him correct. Given their losses (not counting restructuring) they are going to need enough funding to buy any of the bottom 300-400 of the F500 companies outright just to keep the lights on for the next year.
To say that’s an outrageous amount isn’t overstating in the slightest.
Given they are already being forced to drop inference prices in an attempt to compete with Chinese providers, I don’t know if billionaire investors will be willing to hand out that much money this time around if they can’t generate enough value to break even despite the hype.
Basically, he had a server that did code indexing running on his workstation. If he didn't login and run a command to refresh his prod credentials every day, the server would lose its ability to talk to prod (as it should) and fail. Much of the company depended on that service. Eventually it was moved to prod.
"Once, in early 2002, when the index servers went down, Jeff Dean answered user queries manually for two hours. Evals showed a quality improvement of 5 points."
> I am sure VCs are fighting to invest and they will get 4B investment immediately with this team
This is probably the only time some prominent VCs would be active during August, only for Jeff Dean & Co's company.
I can see them now frantically negotiating, responding and firing off emails right now during their holidays to get an allocation in Jeff's new company.
Would love to hear the pitch: We never had the lead in AI and now definitely lost it, despite the gazillions dollars and engineering resources of Google, but now will be different?
The "lead" in AI, even with google's TPU advantage, may not make economic sense for a company. Can you make back the training investment on a competitive frontier model, before everyone switches to a newer frontier model within a year?
> Announcing Discovery Loop!
> I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
I’ve been reading this sentiment on HN since GPT4o, yet models got better and better
[1] https://www.synbiobeta.com/read/anthropic-is-hiring-biologis...
Sounds like the oil scare from 90' - we thought we were gonna run out of oil. But as oil gets more expensive it pays to dig further down to find the stuff that didn't make sense to dig up before.
If Demis Hassabis got a Nobel prize for being a Project Manager, is Zitron up for the Nobel on Economy for excellence in economic forecast?
But more seriously, this has to do with DeepMind falling way behind the frontier in intelligence and cost per task. There's basically no reason to use a Gemini model today.
"Based on estimates of their burn rate and historic analyses, I hypothesize that OpenAI will collapse in the next 12-24 months unless it raises more funding than in the history of the valley and creates an entirely new form of AI."
-July 29, 2024
https://x.com/edzitron/status/1817955630784917548
2 years these charlatans have had their "imminent collapse" predictions repeatedly wrong, meanwhile all valuations, AGI progress in unsolved problems, and frontier leadership have been repeatedly vindicated that we are beginning ASI / the singularity.
Open weights will drop costs, but distribution matters. OpenAI has that.
Costs will come down. Deep entrenchment will not.
Each generation of model is more expensive to train and run than the last.
More efficient hardware? That means throwing out billions of dollars worth of existing hardware and buying billions more in hardware. The cheapest hardware is the stuff they already have because spending $70B in hardware to halve a $2-4B electricity bill just doesn’t make financial sense (and that’s if doubling hardware efficiency can happen before they go under).
If they cut costs by using smaller models, they are then racing to the bottom vs China which seems hard to do (especially with China’s cheap solar energy).
To say that’s an outrageous amount isn’t overstating in the slightest.
Given they are already being forced to drop inference prices in an attempt to compete with Chinese providers, I don’t know if billionaire investors will be willing to hand out that much money this time around if they can’t generate enough value to break even despite the hype.
https://github.com/LRitzdorf/TheJeffDeanFacts
Uh oh...
ahahah golden.
Will be following their journey
I am sure VCs are fighting to invest and they will get 4B investment immediately with this team
This is probably the only time some prominent VCs would be active during August, only for Jeff Dean & Co's company.
I can see them now frantically negotiating, responding and firing off emails right now during their holidays to get an allocation in Jeff's new company.