The release of DeepSeek's R1 model is significant, combined with the transparent research and the fact that it is under the permissive MIT license, allowing commercial use. This shifts power dynamics.
There are two messages:
1. First towards OpenAI: We can achieve the same results with at least one order of magnitude less resources.
2. Second towards Meta: We can totally out-opensource you.
How are companies who have invested vastly more resources, yet do not seem to possess any significant competitive advantage, supposed to receive a return on their initial investment? It appears highly unlikely.
The $500 billion in infrastructure could be perceived as a bailout to the investors of OpenAI, who are frankly in a tight spot.
Yeah unfortunately chinese cos don’t release much on their RLHF/safety process, so we don’t know if they’re actively pruning answers related to Taiwan. But we saw the same thing from DeepSeek V3 when it came out last month, so i’m inclined to believe the anecdata.
I have to correct this. Just seeing a test of the self-hosted version and it refused to talk about Tiananmen Square, and also gave a canned response about Taiwan’s status as an independent country.
I’ve been digging into this a bit more - it seems like it depends on whether you’re using R1 or R1-Zero. Zero hasn’t had any RLHF, meaning no feedback on not talking about Taiwan. It’s less stable/performant but also less censored.
Excellent summary, Charlie. I've made the observation to Daniel that we've entered a paradigm where a big company like OpenAI can spend big bucks and build a new model, and then someone else can come along and make that model run efficiently. We kind of need both approaches right now!
True, I ought to have mentioned that this type of thing is much easier to do once you know it can be done in the first place. But I also wonder how much longer the big labs can sustain their burn rate if their profits get undercut almost immediately. The closest parallel that comes to mind is the pharma industry, where leading companies develop new drugs in the US and are copied elsewhere. But those companies get exclusivity over the new drugs here, and use it to recoup their research costs (and more) before generics can hit the market. There isn’t really any analogous system here- anyone can use a fast follower open weight model as soon as it’s available.
Good point. "Move fast, break everything" will probably end up being the actual mantra. The big companies want to blast things out right now, while the money/power advantage is very high... we'll see a lot of great stuff from the leaders this year, I think, and even if DeepSeek doesn't destroy future benchmarks (they still could!), there's a huge benefit to doing this type of thing.
I'm thinking: most of the world hasn't even touched anything generative yet, from this new AI paradigm we're in. One way to close that massive gap is to make the tools much cheaper, and another way is to bundle them w/existing products, as Apple is doing (and as Microsoft once did, which led to an explosion in internet use).
I'm not really saying anything you and Dan aren't already noticing, but I want to underscore how important even models that are catching up actually are.
The release of DeepSeek's R1 model is significant, combined with the transparent research and the fact that it is under the permissive MIT license, allowing commercial use. This shifts power dynamics.
There are two messages:
1. First towards OpenAI: We can achieve the same results with at least one order of magnitude less resources.
2. Second towards Meta: We can totally out-opensource you.
How are companies who have invested vastly more resources, yet do not seem to possess any significant competitive advantage, supposed to receive a return on their initial investment? It appears highly unlikely.
The $500 billion in infrastructure could be perceived as a bailout to the investors of OpenAI, who are frankly in a tight spot.
According to a colleague the model is “talking about Taiwan, no problem”
Yeah unfortunately chinese cos don’t release much on their RLHF/safety process, so we don’t know if they’re actively pruning answers related to Taiwan. But we saw the same thing from DeepSeek V3 when it came out last month, so i’m inclined to believe the anecdata.
I have to correct this. Just seeing a test of the self-hosted version and it refused to talk about Tiananmen Square, and also gave a canned response about Taiwan’s status as an independent country.
I’ve been digging into this a bit more - it seems like it depends on whether you’re using R1 or R1-Zero. Zero hasn’t had any RLHF, meaning no feedback on not talking about Taiwan. It’s less stable/performant but also less censored.
Excellent summary, Charlie. I've made the observation to Daniel that we've entered a paradigm where a big company like OpenAI can spend big bucks and build a new model, and then someone else can come along and make that model run efficiently. We kind of need both approaches right now!
True, I ought to have mentioned that this type of thing is much easier to do once you know it can be done in the first place. But I also wonder how much longer the big labs can sustain their burn rate if their profits get undercut almost immediately. The closest parallel that comes to mind is the pharma industry, where leading companies develop new drugs in the US and are copied elsewhere. But those companies get exclusivity over the new drugs here, and use it to recoup their research costs (and more) before generics can hit the market. There isn’t really any analogous system here- anyone can use a fast follower open weight model as soon as it’s available.
Good point. "Move fast, break everything" will probably end up being the actual mantra. The big companies want to blast things out right now, while the money/power advantage is very high... we'll see a lot of great stuff from the leaders this year, I think, and even if DeepSeek doesn't destroy future benchmarks (they still could!), there's a huge benefit to doing this type of thing.
I'm thinking: most of the world hasn't even touched anything generative yet, from this new AI paradigm we're in. One way to close that massive gap is to make the tools much cheaper, and another way is to bundle them w/existing products, as Apple is doing (and as Microsoft once did, which led to an explosion in internet use).
I'm not really saying anything you and Dan aren't already noticing, but I want to underscore how important even models that are catching up actually are.
They are making really open open ai
They make it in China (Hanzghou is amazing place btw)
Fantastic