On Monday, Anthropic released Claude 3, a family of models that hit new highs on various benchmarks, add multi-modal capabilities, and are a real competitor to GPT-4.
I was first a bit shocked by how much cheaper Gemini appeared to be (even taking your "rough estimate" note into consideration), but then it hit me that the price was for the Pro model which is comparable to GPT-3.5, so that fits.
I'm still bummed that Claude hasn't made it to Denmark after so many months. And, unlike many other services, using a VPN isn't an option since they verify your account using a local phone number. Claude's been highlighted as one of the best LLMs for PDF summarization, etc., so I can't wait to check it out when I finally can.
I also just watched Matt Wolfe's anecdotal testing today:
In his testing, he actually found both Sonet and Opus to be better at one-off code than GPT-4 Turbo. Then again, it was just one test. But GPT-4 Turbo seemed better at many reasoning tasks.
Nice write-up, Charlie! It's easy to see why this is a big deal.
I think Claude will push Gemini and GPT4 to release additional improvements much sooner than otherwise. The arms race is ON!
Excellent summary!
I was first a bit shocked by how much cheaper Gemini appeared to be (even taking your "rough estimate" note into consideration), but then it hit me that the price was for the Pro model which is comparable to GPT-3.5, so that fits.
I'm still bummed that Claude hasn't made it to Denmark after so many months. And, unlike many other services, using a VPN isn't an option since they verify your account using a local phone number. Claude's been highlighted as one of the best LLMs for PDF summarization, etc., so I can't wait to check it out when I finally can.
I also just watched Matt Wolfe's anecdotal testing today:
https://www.youtube.com/watch?v=jnUhpLAuaBA
(Ignore the clickbait-y thumbnail and title)
In his testing, he actually found both Sonet and Opus to be better at one-off code than GPT-4 Turbo. Then again, it was just one test. But GPT-4 Turbo seemed better at many reasoning tasks.
Super excited to try Claude, big fan of Dario
This AI search engine is getting access to Claude-3 Opus with 1 million token context. I have prerelease access, and it is a pretty amazing product!!!
Sign up for the waitlist if you are interested:
https://rubiks.ai/