Artificial Ignorance

Artificial Ignorance

Share this post

Artificial Ignorance
Artificial Ignorance
AI Roundup 077: A good week for open-source
Copy link
Facebook
Email
Notes
More

AI Roundup 077: A good week for open-source

July 26, 2024.

Charlie Guo's avatar
Charlie Guo
Jul 26, 2024
∙ Paid
9

Share this post

Artificial Ignorance
Artificial Ignorance
AI Roundup 077: A good week for open-source
Copy link
Facebook
Email
Notes
More
2
Share
Artwork created with Midjourney.

A good week for open-source

Meta released Llama 3.1 405B, the first open model competitive with GPT-4. A day later, Mistral released Mistral Large 2, which had similar performance despite having three times fewer parameters.

Between the lines:

  • In addition to the upgraded performance, the new model family also sports a 128K context window and multilingual support, with multimodal inputs coming soon.

  • The release is a win for open-source AI, especially since Llama allows developers to use its outputs to train other LLMs.

  • But there are still some reasons to be skeptical: the Mistral model is only "open" for non-commercial use, and Llama requires certain naming conventions for downstream models.

Looking a gift llama in the mouth

Looking a gift llama in the mouth

Charlie Guo
·
July 24, 2024
Read full story

Elsewhere in frontier models:

  • Stability AI unveiled Stable Video 4D, a model that generates videos from eight new perspectives based on video input.

  • Google Research detailed NeuralGCM, a model that can rapidly, efficiently, and accurately simulate Earth’s atmosphere.

  • And OpenAI published Rule-Based Rewards, a method for automating model fine-tuning to ensure safe behavior.

​

Source: Google DeepMind

AI IMO

Google DeepMind has solved International Math Olympiad problems at a "silver medal level" thanks to two new models, AlphaProof and AlphaGeometry 2.

Why it matters:

  • "Silver medal level" means better than roughly 75% of competitors - a very impressive bar, though the AI did get more time than a normal human.

  • AlphaProof uses Lean, a programming language that can create verifiable proofs. It generates ideas for solutions and then only uses ones that are possible in the proof language.

  • There was also meaningful improvement over the first AlphaGeometry, which used a synthetic training approach to solving geometric problems.

Elsewhere in the FAANG free-for-all:

  • Microsoft unveiled a Bing generative search that shows LLM-generated answers with sources available to a small subset of users.

  • Google updated its Gemini chatbot with Gemini 1.5 Flash, featuring a 32K-token context window in its free tier and links to related content.

  • Meta plans to roll out its AI assistant to the Quest in August, in experimental mode for US and Canadian users.

  • OpenAI teased SearchGPT, a Perplexity/Google competitor that can organize links and summarize findings (though it is currently only available via a waitlist).

​

Artwork created with Midjourney.

People are worried about training data

Since ChatGPT's launch, many have protested the use of public Internet content to train models. Now, we have a measurement of the Internet's response: 28% of the web's "most actively maintained, critical sources" block AI scrapers.

The bottom line:

  • Even if you believe it's okay to scrape YouTube videos or other data without consent, this is a tragedy of the commons playing out in real-time.

  • That's not to mention that AI companies can be outright disrespectful with how often they're hitting servers to grab content that's fair game.

  • The logical outcome seems to be a balkanization of the Internet; walled gardens that are only searchable by companies that have paid licensing fees.

Elsewhere in training data:

  • A new study found that AI models collapse when trained on recursively generated data.

  • Meta received an EU request to pause future AI model training on data in the EU.

  • And academic publisher Taylor & Francis sold authors' research to Microsoft AI without informing them or allowing opt-outs.

Elsewhere in AI anxiety:

  • Anthropic came out against California's AI safety bill but suggested several amendments that could change its position.

  • SAG-AFTRA announces a strike against major video game companies, including EA, over AI protections in contract negotiations.

  • Meta's Oversight Board wants to refine the company's policies around AI-generated explicit images.

  • And AOC's Deepfake AI Porn Bill unanimously passes the Senate.

Artificial Ignorance is reader-supported. If you found this interesting or insightful, consider becoming a free or paid subscriber.

Things happen

Airtable launches Cobuilder, an AI tool that generates apps from text prompts. EA used AI to create 3D avatars of 11,000 college football players for its game. VCs poured billions into generative AI startups in H1 2024. Malaysia's Johor state is attracting tech giants for data center construction. California Senator Scott Wiener discusses his AI safety bill. The grimy residue of the AI bubble. Google reportedly wants to put Gemini in smart glasses. Sam Altman calls for a US-led global coalition to ensure a democratic vision for AI. Kamala Harris takes a leading White House role on AI. Scientists hunt for clues about China's supercomputing progress. Nvidia is reportedly working on a new AI chip for China. Some Activision Blizzard artists were allegedly forced to use AI in their work. Japan lures AI companies with copyright laws that alarm creators. Taylor & Francis sold access to authors' research to Microsoft without informing them. AI paid for by ads – the GPT-4o mini inflection point. Big tech wants to make AI cost nothing. AI method rapidly speeds predictions of materials' thermal properties. Japanese chain uses AI to gauge staff smiles. 77% of employees report AI has increased workloads. $1T rout hits Nasdaq 100 over AI jitters. OpenAI removes AI safety leader Mądry. Americans admit they can't tell what's real anymore due to AI scams. TSMC CEO predicts AI chip shortage through 2025. The AI job interviewer will see you now. AI is quietly being used to pick your pocket.

Last week’s roundup

AI Roundup 076: Grand theft audio

AI Roundup 076: Grand theft audio

Charlie Guo
·
July 19, 2024
Read full story

Fundraising ($1.51B raised by 25 companies)

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Charlie Guo
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More