Another trend I believe was a key contributing factor is big tech's adoption and widespread distribution of LLMs. Think about a scenario where OpenAI doesn't launch ChatGPT and Microsoft keeps their Bing chatbot under wraps. In that setting, many developers might feel they have to fix the "hallucination" issue before releasing anything.
Sure, Microsoft, OpenAI, and later Google faced criticism for users' early encounters with LLMs going off-track [0], but thanks to that, and only a year after ChatGPT's launch, even the average user is aware that hallucinations can occur in AI responses. This widespread understanding helps LLM builders and incumbents deploy LLM-powered apps faster and with less scrutiny.
Totally! I do think ChatGPT (via OpenAI and Microsoft) deserves a lot of credit for lighting the fuse here. Without the viral launch of ChatGPT, we probably don't see the massive surge in demand/hype/investment, and without that, a lot of the last year's worth of developments probably don't happen in public. The Anthropic founding team also probably deserves a mention as without the outside pressure from Claude, it's unclear if OpenAI would have moved forward with publicly launching ChatGPT.
When I started researching this post, I knew that ChatGPT had sparked the current boom. But I wasn't sure where all of the dry powder had come from, which is what I was wanted to get at with the 3 trends I mentioned.
Thank you for this summary. For me, the crucial part is, "But as it turns out, efficiently learning the relationships between pieces of data is useful for many, many domains beyond translation." As the dust settles, for me, it becomes increasingly evident that the quality of data and the ability to accurately establish these relationships are paramount for the next phase.
In addition to the importance of data quality, it is worth delving into the significance of semantic mapping in this context. Semantic mapping plays a pivotal role in enabling AI systems to not only understand data relationships but also to derive meaningful insights and context from diverse datasets.
I think you're right, Sharif - right now the best models are a product of advantages in data, research talent, and compute. Over time, data becomes far more valuable out of those three.
Yours is a great explanation of the technological advancements that made it possible to develop increasingly impressive models across the spectrum. That's the "supply" side of things.
But I still feel like ChatGPT was the true inflection point in terms of helping AI make the jump into mass consciousness. The demand side.
Prior to that, I personally was already experimenting with GPT-3 all the way back in 2021 and finding its ability to write fiction, poems, etc. on demand to be really impressive. (Even though it lagged well behind GPT-4.) But when I showed it to friends and family, I still got a "nice parlour trick, but what's the point" level of reaction. ChatGPT changed all that and truly kicked off the current hype cycle.
So we're now at a point where both the "supply" and "demand" are on the rise at just the right time: The "demand" / hype side makes it lucrative and worthwhile for startups and incumbents to want to develop AI products further to begin with, while the "supply" side makes it possible for ever-smaller companies to create models and products of their own.
Like you, I don't know where we are on the S-curve, but one thing's certain: It's not a boring time to be alive.
Fair point. ChatGPT deserves a lot of credit for igniting the hype that we've been living through for the past year. But ChatGPT and GPT-4s capabilities didn't provide a satisfying explanation (to me) as to why Midjourney had gotten SO good in the past year alone, or how ElevenLabs was able to create voices that are near-indistinguishable from real humans.
It was too much of a coincidence that so many modalities of AI became so sophisticated, practically at the same time. So I started digging.
I like your perspective on matching the supply and demand curves though - without the demand, we probably would have seen fewer advancements in the 12 months or so.
> It's not a boring time to be alive.
While I am grateful for this, there is also a part me that is quite tired of living through history in real-time. If I never heard the word "unprecedented" again, that would be okay.
Yup, it's clear from your analysis that the underlying models (data+training) and hardware are the main enablers, but I'm sure the goldrush has been greatly accelerated by AI going mainstream and seeping into public consciousness the way it did in late December 2022.
Another trend I believe was a key contributing factor is big tech's adoption and widespread distribution of LLMs. Think about a scenario where OpenAI doesn't launch ChatGPT and Microsoft keeps their Bing chatbot under wraps. In that setting, many developers might feel they have to fix the "hallucination" issue before releasing anything.
Sure, Microsoft, OpenAI, and later Google faced criticism for users' early encounters with LLMs going off-track [0], but thanks to that, and only a year after ChatGPT's launch, even the average user is aware that hallucinations can occur in AI responses. This widespread understanding helps LLM builders and incumbents deploy LLM-powered apps faster and with less scrutiny.
[0] https://fortune.com/2023/02/21/bing-microsoft-sydney-chatgpt-openai-controversy-toxic-a-i-risk + https://www.npr.org/2023/02/09/1155650909/google-chatbot--error-bard-shares
Totally! I do think ChatGPT (via OpenAI and Microsoft) deserves a lot of credit for lighting the fuse here. Without the viral launch of ChatGPT, we probably don't see the massive surge in demand/hype/investment, and without that, a lot of the last year's worth of developments probably don't happen in public. The Anthropic founding team also probably deserves a mention as without the outside pressure from Claude, it's unclear if OpenAI would have moved forward with publicly launching ChatGPT.
When I started researching this post, I knew that ChatGPT had sparked the current boom. But I wasn't sure where all of the dry powder had come from, which is what I was wanted to get at with the 3 trends I mentioned.
Thank you for this summary. For me, the crucial part is, "But as it turns out, efficiently learning the relationships between pieces of data is useful for many, many domains beyond translation." As the dust settles, for me, it becomes increasingly evident that the quality of data and the ability to accurately establish these relationships are paramount for the next phase.
In addition to the importance of data quality, it is worth delving into the significance of semantic mapping in this context. Semantic mapping plays a pivotal role in enabling AI systems to not only understand data relationships but also to derive meaningful insights and context from diverse datasets.
I think you're right, Sharif - right now the best models are a product of advantages in data, research talent, and compute. Over time, data becomes far more valuable out of those three.
This is a solid and thorough take!
Yours is a great explanation of the technological advancements that made it possible to develop increasingly impressive models across the spectrum. That's the "supply" side of things.
But I still feel like ChatGPT was the true inflection point in terms of helping AI make the jump into mass consciousness. The demand side.
Prior to that, I personally was already experimenting with GPT-3 all the way back in 2021 and finding its ability to write fiction, poems, etc. on demand to be really impressive. (Even though it lagged well behind GPT-4.) But when I showed it to friends and family, I still got a "nice parlour trick, but what's the point" level of reaction. ChatGPT changed all that and truly kicked off the current hype cycle.
So we're now at a point where both the "supply" and "demand" are on the rise at just the right time: The "demand" / hype side makes it lucrative and worthwhile for startups and incumbents to want to develop AI products further to begin with, while the "supply" side makes it possible for ever-smaller companies to create models and products of their own.
Like you, I don't know where we are on the S-curve, but one thing's certain: It's not a boring time to be alive.
Fair point. ChatGPT deserves a lot of credit for igniting the hype that we've been living through for the past year. But ChatGPT and GPT-4s capabilities didn't provide a satisfying explanation (to me) as to why Midjourney had gotten SO good in the past year alone, or how ElevenLabs was able to create voices that are near-indistinguishable from real humans.
It was too much of a coincidence that so many modalities of AI became so sophisticated, practically at the same time. So I started digging.
I like your perspective on matching the supply and demand curves though - without the demand, we probably would have seen fewer advancements in the 12 months or so.
> It's not a boring time to be alive.
While I am grateful for this, there is also a part me that is quite tired of living through history in real-time. If I never heard the word "unprecedented" again, that would be okay.
Yup, it's clear from your analysis that the underlying models (data+training) and hardware are the main enablers, but I'm sure the goldrush has been greatly accelerated by AI going mainstream and seeping into public consciousness the way it did in late December 2022.
It's unprecedented, really.