I think about Moore's Law and that particular roadmap all the time. I wonder if there will be a similar roadmap for AI that makes itself evident over the next few years, kind of a self-fulfilling prophecy for generative AI.
Moore's Law is such an interesting concept when applied to generative AI. We've scaled transformer architectures way faster than Moore's Law ever did, but that's because we've thrown more money behind more GPUs. The key question is can we find a transistor-equivalent for AI that brings the cost or speed of inference down just as quickly.
I think about Moore's Law and that particular roadmap all the time. I wonder if there will be a similar roadmap for AI that makes itself evident over the next few years, kind of a self-fulfilling prophecy for generative AI.
Moore's Law is such an interesting concept when applied to generative AI. We've scaled transformer architectures way faster than Moore's Law ever did, but that's because we've thrown more money behind more GPUs. The key question is can we find a transistor-equivalent for AI that brings the cost or speed of inference down just as quickly.
The Uncanny Valley theory can be used to explain the general development progression in the AI field!