TAI #140: xAI’s Grok-3 steps up the AI Arms Race
Also, OpenAI’s GPT-4.5 and 5 Roadmap and Anthropic’s upcoming Hybrid Reasoning Model
What happened this week in AI by Louie
The big model releases just keep coming and this week xAI made its long anticipated move with Grok-3. OpenAI’s roadmap for GPT-4.5 and GPT-5 is also now public. OpenAI is set to roll out GPT-4.5 as its “last non-chain-of-thought model” within weeks while GPT-5 is now due within months with a focus on unifying its o-series and GPT-series into a system that can dynamically decide when to engage deeper reasoning. Meanwhile, Anthropic is preparing to launch its next major model within weeks, also promising a “hybrid” system that can toggle between fast responses and complex, multi-step reasoning. These releases (together with Deepseek r1 and Gemini Flash-Thinking) all complete the trend towards reinforcement learning based reasoning models dominating R&A focus and SoTA performance at all leading AI labs.
Elon Musk’s xAI unveiled Grok-3 on Tuesday, claiming (controversially) the “Smartest AI on Earth” after the model was trained on “more than 10x” the compute power of its predecessor Grok-2. xAI has caught up to the LLM leaders in impressive time after its founding in just March 2023. To train the model xAI built the largest single training cluster in the world at rapid speed with 100,000 NVIDIA H100 GPUs. The “Colossus” cluster has since also added another 100,000 H200 GPUs.
Several different models were announced today with a non-reasoning foundation model, a “mini” version and two distinct reasoning modes: “Think,” which visibly traces its thought process, and “Big Brain,” a more computationally expensive mode for solving harder problems (we think this is analogous to o1-pro which likely uses parallel model instances for multiple attempts at an answer behind the scenes). xAI is also rolling out “Deep Search,” which it describes as a next-generation search engine built on Grok’s architecture.
Grok-3 achieved a top-ranking score of 1400 in the LMSYS Chatbot Arena, outperforming competing models like OpenAI’s o-series, DeepSeek-V3, and Google’s Gemini variants. On other benchmarks Grok-3 model outperformed other non-reasoning models (for example 75 GPQA score cs Gemini-Pro 2.0 at 65 and 4o at 50. The Grok-3 reasoning modes increased scores further (85 on GPQA) and on GPQA at least it outperforms all other publicly available reasoning models (eg o3-mini 80 and Deepseek r1 71). It does not however beat the unreleased o3 (87.7%).
In terms of access, Grok-3 is available via the X Premium Plus subscription, now hiked to $40/month after two consecutive price increases. xAI is also introducing a new “SuperGrok” plan, which promises priority access to advanced features. Musk has also teased that Grok will soon support synthesized voice interactions, taking a page from OpenAI’s ChatGPT Advanced Voice Mode.
Why should you care?
Grok-3 looks to be an extremely strong model with performance perhaps somewhere between OpenAI’s o1 and o3 models. The Grok-3 API should also be available within weeks which will allow us to start building with it and for more third party testing to firmly establish where it stands and what it brings to the LLM Developer toolkit. This release also pressures OpenAI to reclaim its leadership in public models with the release of GPT-4.5 or o3, together with pressure for new models from Anthropic and Deepmind - all of which is only going to accelerate progress even faster. The AI capital race is also heating with reports xai is looking to raise another $10bn after raising $12bn last year. OpenAI meanwhile is still attempting to pivot from its charity structure and raise $40bn. It is getting increasingly difficult even for huge tech companies to compete in this race! Fortunately we don’t need... All these incredible models and huge amounts of investment and GPUs are going towards models we can all use and build upon. And even better, many AI labs are also still releasing strong open source models too which gives us even more flexibility.
— Louie Peters — Towards AI Co-founder and CEO
Hottest News
OpenAI CEO Sam Altman detailed plans for GPT 4.5 and GPT 5 AI models in a roadmap published on X. The company plans to ship GPT 4.5, which will be OpenAI’s “last non-chain-of-thought model.” Following GPT 4.5, “a top goal for us is to unify o-series models and GPT-series models by creating systems that can use all our tools, know when to think for a long time or not, and generally be useful for a very wide range of tasks,” Altman says.
AI startup Anthropic is gearing up to release its next major AI model, according to a report from The Information. The report describes Anthropic’s upcoming model as a “hybrid” that can switch between “deep reasoning” and fast responses. The company will reportedly introduce a “sliding scale” alongside the model to allow developers to control costs.
OpenAI’s board of directors has “unanimously” rejected billionaire Elon Musk’s offer to buy the nonprofit that effectively governs OpenAI. The New York Times reported that OpenAI also sent a letter to Musk’s lawyer, Marc Toberoff, saying that the bid was “not in the best interests of OpenAI’s mission.”
Perplexity announced Deep Research, which “excels at a range of expert-level tasks—from finance and marketing to product research.” Like Google and OpenAI, Perplexity’s Deep Research aims for more in-depth answers with real citations for more professional use cases than a consumer chatbot would provide. Perplexity’s Deep Research is available for free — non-subscribers get an unspecified but limited number of queries per day, while paying subscribers get unlimited queries.
Google’s Gemini AI chatbot can now tailor answers based on the contents of previous conversations. Gemini can summarize a previous conversation you’ve had with it or recall information you shared in another conversation thread. The recall feature is rolling out to English-speaking subscribers of Google’s $20-a-month AI chatbot subscription, Google One AI Premium.
Elon Musk's xAI is discussing to raise $10 billion, which could value the AI startup at about $75 billion, Bloomberg News reported, citing a person familiar with the matter. The report said that existing investors, including Sequoia Capital, Andreessen Horowitz, and Valor Equity Partners, are in talks to participate in the transaction.
Five 5-minute reads/videos to keep you learning
This article walks you through using Gemini Flash 2.0 for PDF parsing. Gemini 2.0 achieves near-perfect OCR accuracy while remaining cost-effective, handling up to 6,000 pages per dollar. It excels in document chunking for RAG pipelines, although struggles with bounding box detection persist.
In this article, the author uses AI coding assistance to validate an email address. The string must follow the format of a valid email, and a function will tell me whether it is a valid email address or not. The goal is to test whether AI coding assistants can really help with complex tasks.
This article provides a clear and beginner-friendly explanation of transformer models. It breaks down complex concepts by walking through a small-scale implementation of a transformer, helping readers understand the core mechanisms behind attention, token embeddings, and model training.
The internal workings of deep learning models are very complex and challenging to understand. Hence, they are sometimes also referred to as “black boxes.” This article helps you better understand the inner functionalities of the Deep learning models.
This study aims to determine when a single ReAct agent becomes overloaded with instructions and tools and subsequently experiences a performance drop. It uses one of the simplest agentic architectures to examine how it performs as more instructions are given.
Repositories & Tools
Browser Use helps you automate browser interactions using AI agents.
Mark it down is a Python tool for converting files and office documents to Markdown.
Scira is an AI-powered search engine that helps you find information online.
Top Papers of The Week
This paper presents a novel language model architecture that scales test-time computation by reasoning in latent space. It uses a recurrent block to unroll computations, contrasting with models that add more tokens. Without specialized training data, it handles small context windows and complex reasoning.
Researchers explore Test-Time Scaling (TTS) to enhance LLMs during inference. They find that a 1B model, with compute-optimal TTS, can outperform a 405B model on complex tasks. TTS strategies depend on policy models, Process Reward Models, and problem difficulty, suggesting smaller models can excel with the right computational approach.
This paper proposes a post-training framework called Reinforcement Learning via Self-Play (RLSP). RLSP involves three steps: supervised fine-tuning with human or synthetic demonstrations of the reasoning process, using an exploration reward signal to encourage diverse and efficient reasoning behaviors, and RL training with an outcome verifier to ensure correctness while preventing reward hacking. Our key innovation is to decouple exploration and correctness signals during PPO training, carefully balancing them to improve performance and efficiency.
As language model abilities grow, evaluating and supervising them becomes challenging. Automating these tasks with other models, termed "AI Oversight," shows that models favor those similar to them. This similarity risk arises from correlated failures as model capabilities increase. Addressing and correcting model similarity becomes crucial in AI oversight to mitigate these risks.
Reinforcement learning enhances large language models for complex reasoning tasks. Comparing models, o3 outperforms specialized pipelines without hand-crafted strategies, winning a gold medal at the 2024 IOI. Further, o3 achieves a Codeforces rating comparable to elite humans, establishing its strength without domain-specific techniques and highlighting the efficacy of scaling general-purpose reinforcement learning.
Quick Links
1. Publishers sue Cohere over alleged copyright infringement. A consortium of 14 publishers, including Condé Nast, The Atlantic, and Forbes, have accused Cohere of using at least 4,000 copyrighted works to train its AI models and display large portions of articles — and even entire articles — for users, harming the publishers’ referral traffic.
2. OpenAI released a significantly expanded version of its Model Spec, which defines how its AI models should behave. The new 63-page specification outlines guidelines for how AI models should handle everything from controversial topics to user customization. It is free for anyone to use or modify.
3. Nous Research unveiled Hermes 3, its new reasoning model. It is described as an “LLM that unifies reasoning and intuitive language model capabilities” and allows the user to switch at will between longer reasoning processes and shorter, faster, less computationally demanding responses.
Who’s Hiring in AI
Machine Learning Engineer @Apple (Sunnyvale, California, USA)
Summer 2025 Internship - Data Researcher (EMEA) @Dow Jones (Spain/Remote)
Software Engineer - Lakeflow PhD Candidates @Databricks (Mountain View, CA, USA)
Fullstack Engineer - AI Feature Innovation @Contentful (Berlin, Germany)
Agentic AI Engineer @VML MAP (Copenhagen, Denmark)
Software Engineer - AI & ML @Lakeside Software (US/Remote)
Interested in sharing a job opportunity here? Contact sponsors@towardsai.net.
Think a friend would enjoy this too? Share the newsletter and let them join the conversation.
very useful board, thanks for sharing so I can apply it to