The Mind of AI Decoded 🧠, AI's Impact on Jobs 💼, US Chip Wars ⚠️
PLUS: Open Source vs. Big Tech ⚔️, Google's Titan Transformer Upgrade 💪, and Faster Coding Models 💻
👋 This week in AI
Each week, I wade through the fire hose of AI news and distil it into what you actually need to know. Let me know what you think by replying to this email.
🎵 Don’t feel like reading? That’s okay, listen to two AI podcast hosts talk about it instead.
📰 Latest news
Peering Into the Black Box: Understanding the Mind of AI
Anthropic's research offers a fascinating glimpse into the complex inner workings of large language models (LLMs).
Their work reveals how millions of concepts are represented within Claude Sonnet, one of their deployed models.
These representations aren't single points but rather distributed patterns of activation across the model's "neurons". For example, the concept of the "Golden Gate Bridge" isn't stored in one location but is instead a complex pattern across many neurons.
This pattern activates not only when the model encounters the phrase "Golden Gate Bridge" but also related concepts like San Francisco landmarks, or even visual representations of the bridge.
Researchers can manipulate these activation patterns, influencing the model's output in predictable ways. This reveals how meaning is encoded and processed within these complex systems.
“Importantly, we can also manipulate these features, artificially amplifying or suppressing them to see how Claude's responses change.
For example, amplifying the "Golden Gate Bridge" feature gave Claude an identity crisis even Hitchcock couldn’t have imagined: when asked "what is your physical form?", Claude’s usual kind of answer – "I have no physical form, I am an AI model" – changed to something much odder: "I am the Golden Gate Bridge… my physical form is the iconic bridge itself…".
Altering the feature had made Claude effectively obsessed with the bridge, bringing it up in answer to almost any query—even in situations where it wasn’t at all relevant.”
Why it Matters
This ability to identify and manipulate conceptual representations is a major step towards understanding how LLMs function.
It provides valuable insights into how these models learn, reason, and generate text.
By exploring these inner mechanisms, researchers can potentially improve model performance, address biases, and enhance their ability to perform complex tasks.
ChatGPT Scheduled Tasks: A Step Towards AI Assistance
OpenAI's new "Tasks" feature is a notable evolution in chatbot functionality.
Previously confined to real-time interactions, ChatGPT can now schedule actions and reminders, much like traditional digital assistants.
This feature, currently available to Plus, Team, and Pro subscribers ($20 and $200 monthly tiers), allows users to set one-time or recurring tasks, receive proactive suggestions, and manage them through chat threads or a dedicated web interface.
Furthermore, it hints at the next milestone in OpenAI’s roadmap: Agents. With sophisticated AI agents, ChatGPT will take actions like information retrieval, data analysis, and web navigation.
Why it Matters
The introduction of Tasks indicates a maturing of chatbot user interfaces and features.
It moves beyond simple conversational exchanges toward a more proactive and helpful AI assistant model. This shift aligns with OpenAI's broader strategy of monetising advanced AI capabilities through premium features.
For users, Tasks offers increased utility and convenience, streamlining everyday tasks and potentially integrating with more powerful AI tools in the future. While still in beta, the development signals a trend towards more integrated and functional AI assistants, blurring the lines between conversational AI and traditional productivity tools.
New US AI Chip Export Rules Reshape Global Tech Landscape
The Biden administration has announced new regulations for AI chip exports, creating a tiered system that prioritises US leadership in AI and manages the global flow of this technology.
Eighteen allied nations, including the UK, Canada, and Japan, receive unrestricted access. Other countries face quotas, with allowances ranging from 1,700 GPUs for small orders to 320,000 GPUs over two years for companies with "National Verified End User" status.
General access allows for 50,000 GPUs per country, potentially doubling to 100,000 for countries aligning with US tech policies. "Universal Verified End Users" in allied nations can use up to 7% of their AI compute globally but must keep 75% within the US or allied territories.
Why it Matters
These regulations are going to reshape the global AI landscape.
While designed to bolster US AI leadership and manage the spread of advanced AI capabilities, they have drawn criticism from industry players like Nvidia, who fear stifled innovation and market distortion.
The tiered access system creates a complex web of incentives and restrictions, influencing where AI development flourishes and how global tech companies operate.
The rules also have geopolitical implications, impacting relations with various countries and potentially influencing international collaborations in AI research and development.
The long-term effects of this framework will unfold over the coming years as the regulations are implemented and adapted to the evolving technological and political landscape.
📝 Read the White House press release
Google’s Titan: You Have My Attention
Transformer-based models excel at processing information within a fixed window, but struggle to "remember" information from earlier stages during inference (generating the output).
This limits their ability to handle long sequences and build upon previous knowledge. Google's Titan architecture addresses this by incorporating a long-term memory module alongside the traditional attention mechanism.
Unlike typical Transformers that primarily rely on pre-training to encode knowledge, Titans can actively learn and retain information during inference.
This enables them to process significantly larger context windows, exceeding 2 million tokens, with improved accuracy in complex tasks.
Tests across language modelling, reasoning, genomics, and time-series analysis demonstrate superior performance compared to both Transformers and recurrent models.
Why it Matters
Titans' ability to "remember" during inference is a shift in how AI models process and retain information and another signal that we’re moving from pre-training to inference as a focus.
This enhanced memory capacity could lead to more contextually aware language understanding, more accurate predictions by incorporating past trends in time-series analysis, and faster discoveries in fields like genomics by analysing longer sequences.
The improved scalability and accuracy will unlock the ability to tackle complex, long-range dependencies in data, opening new possibilities across various applications.
📁 Check out an unofficial implementaion
Codestral 25.01: Faster Coding, Without the Frontier Model Overhead
Mistral AI's Codestral 25.01 delivers faster code generation, roughly twice as fast as its previous version, while maintaining leading performance in key coding tasks.
It achieves this efficiency despite being smaller than many frontier models, making it a more accessible and cost-effective option.
Developers can access it through various IDEs, local deployments, or cloud platforms.
Why it Matters
Codestral 25.01 offers the speed and performance benefits of larger, more expensive frontier models, but with reduced overhead. This makes it an attractive choice for developers seeking a balance between power and practicality.
The increased speed translates to greater productivity, while its broad availability ensures easy integration into existing workflows.
This combination of performance, accessibility, and efficiency positions Codestral 25.01 as a leading model for developers looking to accelerate their coding process.
📁 Check out the model on Mistral's website
Designing the Future: How AI is Reshaping the Design Industry and the Workforce
The World Economic Forum's (WEF) 2025 Future of Jobs report reveals a shifting landscape for the design industry in the age of AI.
While graphic design is predicted to be the 11th fastest-declining job, UI/UX design is expected to be the 8th fastest growing. This contrast highlights the evolving needs of a digitally driven world, where user experience takes centre stage.
Despite AI's growing capabilities, creative thinking and design skills remain highly valued, with employers rating AI's capacity to replicate these skills as low.
The report, based on a survey of employers representing over 14 million workers, notes an almost eightfold increase in AI investment since late 2022, with 86% of employers anticipating AI-driven business transformation by 2030.
Why it Matters
This report underscores the importance of adapting to the changing demands of the job market. While some design roles face decline due to AI, others are experiencing growth, highlighting the need for reskilling and upskilling.
The enduring value of human creativity and design thinking presents opportunities for professionals to focus on areas where AI is less proficient.
The broader trend of AI adoption necessitates workforce adaptation, with employers planning to reskill existing employees, hire new talent with AI skills, and transition workers from AI-disrupted roles to other positions.
Sky-T1: Levelling the AI Playing Field
UC Berkeley's Sky-T1-32B-Preview demonstrates that open-source AI models are rapidly catching up to closed-source counterparts, often with only months of lag time.
This open-source reasoning model achieves performance comparable to o1-preview on popular benchmarks, scoring 82.4% on Math500 and 86.3% on LiveCodeBench-Easy. Remarkably, it was trained for under $450.
Critically, the team has open-sourced all resources, including training data, code, model weights, and infrastructure.
Why it Matters
Sky-T1 highlights the accelerating pace of open-source AI development. Building a model rivaling closed-source performance at a fraction of the presumed cost demonstrates increasing accessibility and democratisation within the field.
The open-source nature fosters community involvement, enabling researchers and developers to build upon this work, driving further innovation and potentially accelerating the progress of reasoning models.
OpenAI's Blueprint for AI in America: A Framework for American Dominance
OpenAI's "AI in America" proposes a framework for fostering US leadership in AI while ensuring responsible development and deployment.
The blueprint advocates for balanced regulations, prioritising child safety, content provenance, and user preferences.
Highlighting the $175 billion in global funds seeking AI investment, OpenAI stresses the need for US infrastructure development to capture these funds and drive economic growth.
The company, with 300 million users and 3 million developers, champions government-industry collaboration for responsible AI practices.
Why it Matters
OpenAI's framework aims to shape the future of AI in the US. Its adoption could spur investment in domestic AI infrastructure, creating jobs and boosting the economy.
The focus on responsible AI aims to build public trust and broaden access to AI's benefits. By balancing innovation with safeguards, OpenAI's blueprint seeks to guide the long-term growth and beneficial application of AI in America.
Missed the last one?
Superintelligence Soon? 🤯 Nvidia Breaks Moore's Law 🌌 AI Phishing 50% Success Rate 🎣
👋 Welcome to this week in AI.