On Monday, Anthropic released Claude 3, a family of models that hit new highs on various benchmarks, add multi-modal capabilities, and are a real competitor to GPT-4.
I was first a bit shocked by how much cheaper Gemini appeared to be (even taking your "rough estimate" note into consideration), but then it hit me that the price was for the Pro model which is comparable to GPT-3.5, so that fits.
I'm still bummed that Claude hasn't made it to Denmark after so many months. And, unlike many other services, using a VPN isn't an option since they verify your account using a local phone number. Claude's been highlighted as one of the best LLMs for PDF summarization, etc., so I can't wait to check it out when I finally can.
I also just watched Matt Wolfe's anecdotal testing today:
In his testing, he actually found both Sonet and Opus to be better at one-off code than GPT-4 Turbo. Then again, it was just one test. But GPT-4 Turbo seemed better at many reasoning tasks.
Thanks for the video! I'm going to be playing around with Claude via the API to start with, but if that goes well I might sign up for a Claude Pro subscription. I didn't mention this in the piece, but the use case that's really impressed me is using Opus inside of Perplexity - I'm often putting together research summaries, and GPT-4's writing style is *so* formulaic at this point. Having Claude write a natural-sounding summary was so refreshing.
Oh man, that alone sounds like a breath of fresh air. I hate that GPT-4 has to be prompted out of sounding like a generic robot. Can't wait to check out Claude whenever it gets to DK!
Nice write-up, Charlie! It's easy to see why this is a big deal.
I think Claude will push Gemini and GPT4 to release additional improvements much sooner than otherwise. The arms race is ON!
Excellent summary!
I was first a bit shocked by how much cheaper Gemini appeared to be (even taking your "rough estimate" note into consideration), but then it hit me that the price was for the Pro model which is comparable to GPT-3.5, so that fits.
I'm still bummed that Claude hasn't made it to Denmark after so many months. And, unlike many other services, using a VPN isn't an option since they verify your account using a local phone number. Claude's been highlighted as one of the best LLMs for PDF summarization, etc., so I can't wait to check it out when I finally can.
I also just watched Matt Wolfe's anecdotal testing today:
https://www.youtube.com/watch?v=jnUhpLAuaBA
(Ignore the clickbait-y thumbnail and title)
In his testing, he actually found both Sonet and Opus to be better at one-off code than GPT-4 Turbo. Then again, it was just one test. But GPT-4 Turbo seemed better at many reasoning tasks.
Thanks for the video! I'm going to be playing around with Claude via the API to start with, but if that goes well I might sign up for a Claude Pro subscription. I didn't mention this in the piece, but the use case that's really impressed me is using Opus inside of Perplexity - I'm often putting together research summaries, and GPT-4's writing style is *so* formulaic at this point. Having Claude write a natural-sounding summary was so refreshing.
Oh man, that alone sounds like a breath of fresh air. I hate that GPT-4 has to be prompted out of sounding like a generic robot. Can't wait to check out Claude whenever it gets to DK!
Super excited to try Claude, big fan of Dario
This AI search engine is getting access to Claude-3 Opus with 1 million token context. I have prerelease access, and it is a pretty amazing product!!!
Sign up for the waitlist if you are interested:
https://rubiks.ai/