System 1, System 2 and The Multi-Agents Future

February 1, 2025

In cognitive psychology, System 1 and System 2 are two distinct modes of thinking first proposed by psychologists Keith Stanovich and Richard West, and later popularized by Nobel laureate Daniel Kahneman in his book "Thinking, Fast and Slow."

think-machine

System 1 and System 2 Modes of Human Thinking

System 1 represents our fast, automatic, and intuitive thinking process. It:

Operates automatically and quickly
Requires little or no effort
Has no sense of voluntary control
Is emotional, stereotypic, and unconscious

Examples of System 1 thinking include:

Detecting that one object is closer than another
Completing "2+2=?"
Reading words on a billboard
Driving a car on an empty road
Understanding simple sentences

System 2 represents our slower, more analytical, and conscious thinking process. It:

Requires attention and effort
Is logical and calculating
Involves complex computations
Is deliberate and conscious
Can follow rules and compare objects

Examples of System 2 thinking include:

Solving complex mathematical problems
Focusing on a particular voice in a crowded room
Filling out a tax form
Checking the validity of a complex logical argument
Parallel parking in a tight space

The interplay between these two systems shapes how we make decisions, form judgments, and interact with the world around us. While System 1 is incredibly efficient and usually serves us well, it can also lead to cognitive biases and systematic errors. System 2, though more reliable for complex decisions, requires significant mental energy and can't always be engaged due to its demanding nature.

System 1 and System 2 LLMs

Just as humans have two distinct thinking modes, we can draw parallels to different types of Large Language Models (LLMs). Traditional LLMs like GPT-3.5 often exhibit System 1-like characteristics: they're fast, intuitive, and can generate responses quickly based on pattern recognition. However, newer models like ChatGPT o1-preview and DeepSeek-R1 are being developed to incorporate more System 2-like reasoning capabilities.

These reasoning-enhanced LLMs demonstrate several System 2-like characteristics:

Chain of Thought Reasoning
- Explicitly shows step-by-step reasoning process
- Breaks down complex problems into logical chains
- Makes intermediate thoughts visible and verifiable
- Uses self-talk and scratchpad techniques
- Can backtrack and correct reasoning errors
Inference-Time Computing
- Performs actual computations during inference
- Uses external tools and calculators when needed
- Executes code to verify solutions
- Dynamically adapts reasoning based on intermediate results
- Can handle novel scenarios not seen in training
Step-by-Step Problem Solving
- Break down complex problems into smaller, manageable parts
- Show explicit reasoning chains
- Verify intermediate results before reaching conclusions

For example, when given a complex mathematical word problem, these reasoning LLMs might:

First parse the problem statement
Identify key variables and relationships
Create a solution strategy
Execute calculations step-by-step
Verify the answer against the original constraints

These reasoning LLMs is fundamentally searching through an encoded world model built during training, rather than just pattern matching. This allows it to make novel connections and inferences by traversing its learned representations in structured ways. Whether this constitutes true System 2 thinking remains an open question, but these advances suggest we may be gradually approaching System 2-like capabilities, even if we're not quite there yet. Also note that there are other proposals for combining System-1 LLM with dedicated built "System 2 LLM", which is a process-based verifier based on supervision.

Obviously System 2 like behavor comes with a cost, the computation required for the reasoning model is much higher:

Multi-Agents Future = Coordinated System 1 + System 2 LLMs

Let's consider building a social media chat bot as an example of how System 1 and System 2 LLMs can work together effectively:

The System 2 component handles high-level planning and decision-making. It develops conversation strategies based on user profiles and context, while planning multi-turn conversation flows. This component evaluates potential risks and consequences, maintains long-term conversation coherence, decides when to switch conversation topics, and monitors for sensitive topics or potential violations.

The System 1 component handles immediate responses and actions. It generates natural, fluent responses and handles routine social interactions. This component processes and responds to simple queries while maintaining consistent tone and style. It excels at generating engaging content quickly and handling real-time language translation tasks.

Here's how they might coordinate in a conversation:

The process begins with Initial Planning, where System 2 analyzes the user profile and conversation history, sets conversation goals and constraints, and identifies appropriate topics and tone. Next comes the Conversation Flow phase, where System 1 takes charge of generating immediate responses, maintaining natural dialogue rhythm, and handling basic follow-up questions.

Throughout the conversation, Strategic Checkpoints occur where System 2 periodically reviews the conversation direction, adjusts strategy based on user engagement, and identifies when deeper reasoning is needed. Finally, Course Correction involves both systems working together - System 2 identifies potential issues while System 1 executes the corrective responses, with both coordinating to maintain conversation flow.

In fact, this two layer structure is exactly what the popular agent - AIXBT uses!

This System 1 + System 2 architecture can be generalized to many AI applications:

Software Development Assistant

System 2 (Architect):
- Plans overall code structure and architecture
- Evaluates security implications and edge cases
- Makes high-level design decisions
- Reviews code for potential issues
System 1 (Coder):
- Generates actual code implementations
- Handles routine coding tasks
- Provides quick documentation
- Suggests immediate fixes

Research Assistant

System 2 (Research Director):
- Develops research strategy
- Identifies key questions and hypotheses
- Evaluates source credibility
- Synthesizes complex findings
System 1 (Research Assistant):
- Rapidly scans and summarizes papers
- Extracts relevant quotes and data
- Formats citations and references
- Generates draft sections

Financial Advisor

System 2 (Strategic Planner):
- Analyzes long-term market trends
- Develops investment strategies
- Assesses risk profiles
- Plans portfolio rebalancing
System 1 (Market Monitor):
- Tracks real-time market movements
- Executes routine trades
- Generates market updates
- Alerts on immediate opportunities

Creative Assistant

System 2 (Creative Director):
- Develops overall creative vision
- Plans narrative structure
- Ensures thematic consistency
- Evaluates artistic choices
System 1 (Creator):
- Generates content variations
- Handles style and formatting
- Produces quick drafts
- Makes immediate artistic adjustments

This architecture's power lies in its ability to combine deep strategic thinking with rapid tactical execution, making it applicable across diverse domains where both careful planning and quick responses are needed.

Final Thoughts

As I am very curious and excited about the emerging multi-agent world, there are several remaining open questions:

Fundamentally, the bandwidth of communication between agents could be much higher than communication between humans. This could make agent coordination more efficient. How would this improved efficiency enhance the effectiveness of agents?
Following the first question, it's possible that the boundary between multi-agents and single agents is quite blurred, in the sense that a single LLM could create internal structures that combine System 1 and System 2 thinking in more interesting ways.

Let me end this in a "less wrong" way: the goal of this article is not to build a map that perfectly matches the territory in detail, but rather to provoke thinking about how human and AI agents work and evolve. While I believe System 1 and System 2 will almost surely be a leaky abstraction, like any other abstraction or map, it might shed some light on our multi-agent future.

Shumo Chu