1.1 What is an Agent?

When exploring any complex concept, it's best to start with a concise definition. In the field of artificial intelligence, an agent is defined as any entity that can perceive its Environment through Sensors, and autonomously take Actions through Actuators to achieve specific goals.

This definition contains four fundamental elements of an agent's existence. The environment is the external world in which the agent operates. For an autonomous vehicle, the environment is the dynamically changing road traffic; for a trading algorithm, the environment is the ever-changing financial market. The agent is not isolated from the environment—it continuously perceives the environmental state through its sensors. Cameras, microphones, radar, or data streams returned by various Application Programming Interfaces (APIs) are all extensions of its perceptual capabilities.

After acquiring information, the agent needs to take actions to influence the environment, changing its state through actuators. Actuators can be physical devices (such as robotic arms or steering wheels) or virtual tools (such as executing code or calling a service).

However, what truly endows an agent with "intelligence" is its Autonomy. An agent is not merely a program that passively responds to external stimuli or strictly executes preset instructions; it can make independent decisions based on its perceptions and internal state to achieve its design goals. This closed loop from perception to action forms the foundation of all agent behavior, as shown in Figure 1.1.

1.1.1 Agents from a Traditional Perspective

Before the current wave of Large Language Models (LLMs), pioneers in artificial intelligence had already spent decades exploring and building the concept of "agents." These paradigms, which we now call "traditional agents," are not a single static concept but have undergone a clear evolutionary path from simple to complex, from passive reaction to active learning.

The starting point of this evolution is the structurally simplest Simple Reflex Agent. Their decision-making core consists of "condition-action" rules explicitly designed by engineers, as shown in Figure 1.2. A classic automatic thermostat works this way: if the sensor perceives that the room temperature is higher than the set value, it activates the cooling system.

This type of agent relies entirely on current perceptual input and has no memory or predictive capability. It's like a digitized instinct—reliable and efficient, but therefore unable to handle complex tasks that require understanding context. Its limitations raise a key question: What should an agent do if the current state of the environment is insufficient as the sole basis for decision-making?

Figure 1.2 Decision logic diagram of a simple reflex agent

To answer this question, researchers introduced the concept of "state" and developed Model-Based Reflex Agents. This type of agent has an internal World Model used to track and understand aspects of the environment that cannot be directly perceived. It attempts to answer: "What is the world like now?" For example, an autonomous vehicle driving through a tunnel, even if its camera temporarily cannot perceive the vehicle ahead, its internal model will still maintain a judgment about that vehicle's existence, speed, and estimated position. This internal model gives the agent a primitive form of "memory," making its decisions no longer solely dependent on instantaneous perception but based on a more coherent and complete understanding of the world state.

However, merely understanding the world is not enough—an agent needs clear goals. This led to the development of Goal-Based Agents. Unlike the previous two types, their behavior is no longer passively reacting to the environment but actively and proactively selecting actions that can lead to a specific future state. The question this type of agent needs to answer is: "What should I do to achieve my goal?" A classic example is a GPS navigation system: your goal is to reach the office, and the agent will plan an optimal route using search algorithms (such as A*) based on map data (world model). The core capability of this type of agent is reflected in its consideration and planning for the future.

Going further, real-world goals are often not singular. We not only want to reach the office but also want the shortest time, the most fuel-efficient route, and to avoid congestion. When multiple goals need to be balanced, Utility-Based Agents emerge. They assign a utility value to every possible world state, representing the level of satisfaction. The agent's core goal is no longer simply to achieve a specific state but to maximize expected utility. It needs to answer a more complex question: "Which behavior will bring me the most satisfactory result?" This architecture allows the agent to learn to balance conflicting goals, making its decisions closer to human rational choice.

So far, the agents we've discussed, although increasingly complex in functionality, still rely on the prior knowledge of human designers for their core decision-making logic, whether rules, models, or utility functions. What if an agent could learn autonomously through interaction with the environment without relying on presets?

This is the core idea of Learning Agents, and Reinforcement Learning (RL) is the most representative path to realizing this idea. A learning agent contains a performance element (the various types of agents we discussed earlier) and a learning element. The learning element continuously modifies the performance element's decision-making strategy by observing the results of the performance element's actions in the environment.

Imagine an AI learning to play chess. It might start by making random moves, but when it finally wins a game, the system gives it a positive reward. Through extensive self-play, the learning element gradually discovers which moves are more likely to lead to ultimate victory. AlphaGo Zero is a milestone achievement of this philosophy. In the complex game of Go, through reinforcement learning, it discovered many effective strategies that surpass existing human knowledge.

From simple thermostats to cars with internal models, to navigation that can plan routes, to decision-makers who know how to weigh pros and cons, and finally to learners who can self-evolve through experience. This evolutionary path demonstrates the development trajectory that traditional artificial intelligence has undergone in building machine intelligence. They have laid a solid and necessary foundation for our understanding of more cutting-edge agent paradigms today.

1.1.2 New Paradigm Driven by Large Language Models

The emergence of large language models represented by GPT (Generative Pre-trained Transformer) is significantly changing the construction methods and capability boundaries of agents. LLM agents driven by large language models have fundamentally different core decision-making mechanisms from traditional agents, thus endowing them with a series of entirely new characteristics.

This transformation can be clearly seen from the comparison of the two in multiple dimensions such as core engine, knowledge source, and interaction method, as shown in Table 1.1. In short, the capabilities of traditional agents stem from engineers' explicit programming and knowledge construction, and their behavior patterns are deterministic and bounded; while LLM agents, through pre-training on massive data, have acquired implicit world models and powerful emergent capabilities, enabling them to handle complex tasks in a more flexible and general way.

Table 1.1 Core comparison between traditional agents and LLM-driven agents

This difference enables LLM agents to directly process high-level, ambiguous, and context-rich natural language instructions. Let's use an "intelligent travel assistant" as an example to illustrate.

Before the emergence of LLM agents, planning a trip typically meant users needed to manually switch between multiple dedicated applications (such as weather, maps, booking websites), with the user themselves playing the role of information integration and decision-making. An LLM agent, however, can integrate this process. When receiving an ambiguous instruction like "plan a trip to Xiamen," its working method reflects the following points:

• Planning and Reasoning: The agent first decomposes this high-level goal into a series of logical subtasks, for example: [Confirm travel preferences] -> [Query destination information] -> [Draft itinerary] -> [Book tickets and accommodation]. This is an internal, model-driven planning process.
• Tool Use: When executing the plan, the agent identifies information gaps and proactively calls external tools to fill them. For example, it will call a weather query interface to get real-time weather, and based on the information "rain is forecast," it will tend to recommend indoor activities in subsequent planning.
• Dynamic Adjustment: During the interaction, the agent treats user feedback (such as "this hotel exceeds the budget") as new constraints and adjusts subsequent actions accordingly, re-searching and recommending options that meet the new requirements. The entire process of "check weather → adjust itinerary → book hotel" demonstrates its ability to dynamically modify its behavior based on context.

In summary, we are shifting from developing specialized automation tools to building systems that can autonomously solve problems. The core is no longer writing code but guiding a general "brain" to plan, act, and learn.

1.1.3 Types of Agents

Following the review of agent evolution above, this section will classify agents from three complementary dimensions.

(1) Classification Based on Internal Decision Architecture

The first classification dimension is based on the complexity of the agent's internal decision architecture. This perspective was systematically proposed in "Artificial Intelligence: A Modern Approach"^[1]. As described in Section 1.1.1, the evolutionary path of traditional agents itself constitutes the most classic classification ladder, covering from simple reactive agents to model-based agents that introduce internal models, and then to more forward-looking goal-based and utility-based agents. Additionally, learning capability is a meta-capability that can be endowed to all the above types, enabling them to self-improve through experience.

(2) Classification Based on Time and Reactivity

In addition to the complexity of internal architecture, agents can also be classified from the time dimension of decision-making processing. This perspective focuses on whether an agent acts immediately after receiving information or acts after deliberate planning. This reveals a core trade-off in agent design: the balance between Reactivity, which pursues speed, and Deliberation, which pursues optimal solutions, as shown in Figure 1.3.

Figure 1.3 Relationship between agent decision time and quality

• Reactive Agents

This type of agent makes nearly instantaneous responses to environmental stimuli with extremely low decision latency. They typically follow a direct mapping from perception to action, with no or minimal future planning. The simple reactive and model-based agents mentioned above belong to this category.

Their core advantage lies in fast speed and low computational overhead, which is crucial in dynamic environments requiring rapid decision-making. For example, a vehicle's airbag system must react within milliseconds of a collision—any delay could lead to serious consequences; similarly, high-frequency trading robots must rely on reactive decision-making to capture fleeting market opportunities. However, the cost of this speed is "short-sightedness." Due to lack of long-term planning, reactive agents easily fall into local optima and struggle to complete complex tasks requiring multi-step coordination.

• Deliberative Agents

In contrast to reactive agents, deliberative (or planning) agents engage in complex thinking and planning before acting. They do not immediately react to perceptions but first use their internal world model to systematically explore various future possibilities, evaluate the consequences of different action sequences, in hopes of finding an optimal path to achieve goals. Goal-based and utility-based agents are typical deliberative agents.

Their decision-making process can be likened to a chess player. They don't just look at the immediate move but anticipate possible opponent responses and plan out subsequent moves, even dozens of moves ahead. This deliberative capability enables them to handle complex tasks requiring long-term vision, such as formulating a business plan or planning a long-distance trip. Their advantage lies in the strategic nature and foresight of their decisions. However, the flip side of this advantage is high time and computational costs. In rapidly changing environments, when a deliberative agent is still deep in thought, the best moment to act may have long passed.

• Hybrid Agents

Complex tasks in the real world often require both immediate reactions and long-term planning. For example, the intelligent travel assistant we mentioned earlier needs to adjust recommendations based on user's immediate feedback (such as "this hotel is too expensive") (reactivity), while also being able to plan a complete multi-day travel itinerary (deliberation). Therefore, hybrid agents emerged, aiming to combine the advantages of both and achieve a balance between reaction and planning.

A classic hybrid architecture is hierarchical design: the lower layer is a fast reactive module that handles emergencies and basic actions; the upper layer is a deliberative planning module responsible for formulating long-term goals. Modern LLM agents demonstrate a more flexible hybrid mode. They typically operate in a "think-act-observe" loop, cleverly integrating both modes:

• Reasoning: In the "thinking" phase, the LLM analyzes the current situation and plans the next reasonable action. This is a deliberative process.
• Acting & Observing: In the "acting" and "observing" phases, the agent interacts with external tools or the environment and immediately receives feedback. This is a reactive process.

Through this approach, the agent decomposes a grand task requiring long-term planning into a series of "planning-reaction" micro-loops. This enables it to flexibly respond to immediate environmental changes while ultimately completing complex long-term goals through coherent steps.

(3) Classification Based on Knowledge Representation

This is a more fundamental classification dimension that explores what form the knowledge used by agents for decision-making exists in their "minds." This question is at the core of a debate that has lasted more than half a century in the field of artificial intelligence and has shaped two distinctly different AI cultures.

• Sub-symbolic AI

Sub-symbolism, or connectionism, holds that knowledge does not take the form of explicit rules. Instead, it is implicitly distributed across a complex network of neurons as statistical patterns learned from massive amounts of data. Neural networks and deep learning are its representative approaches.

As shown on the left side of Figure 1.4, sub-symbolic AI is like a babbling child. It does not learn to recognize cats from rules such as "cats have four legs, are furry, and meow." Instead, after seeing thousands of cat images, its neural network learns the visual patterns associated with the concept of a cat. This approach is powerful in pattern recognition and robust to noisy data. It can readily process unstructured data such as images and sounds, tasks that are often difficult for systems that rely on explicit rules.

However, this powerful intuitive capability also comes with opacity. Sub-symbolic systems are typically viewed as a Black Box. It can identify a cat in a picture with amazing accuracy, but if you ask it "why do you think this is a cat?", it likely cannot provide a logically sound explanation. Additionally, it performs poorly on pure logical reasoning tasks and sometimes produces hallucinations that seem reasonable but are factually incorrect.

• Symbolic AI

In contrast to sub-symbolic AI, symbolism, often called traditional artificial intelligence, holds that intelligence stems from logical operations on symbols. These symbols are human-readable entities such as words and concepts, and the operations follow strict logical rules, as shown in the middle of Figure 1.4. This is like a meticulous librarian organizing world knowledge into clear rule bases and knowledge graphs.

Its main advantage lies in transparency and interpretability. Since reasoning steps are explicit, its decision-making process can be fully traced, which is crucial in high-risk fields such as finance and healthcare. However, its "Achilles' heel" lies in fragility: it relies on a complete rule system, but in the real world full of ambiguity and exceptions, any new situation not covered can lead to system failure, which is the so-called "knowledge acquisition bottleneck."

• Neuro-Symbolic AI

For a long time, the two camps of sub-symbolism and symbolism developed like two parallel lines. To overcome the limitations of the above two paradigms, a "grand reconciliation" idea began to emerge, which is neuro-symbolic AI, also called neuro-symbolic hybrid. Its goal is to merge the advantages of both paradigms, creating a hybrid agent that can both learn from data like neural networks and perform logical reasoning like symbolic systems. In other words, it attempts to combine the pattern recognition strengths of sub-symbolic AI with the logical reasoning strengths of symbolic AI. Nobel Prize-winning economist Daniel Kahneman's dual-system theory proposed in his book "Thinking, Fast and Slow" provides an excellent analogy for understanding neuro-symbolism^[2]. Following the left-to-right order in Figure 1.4, the correspondence can be summarized as follows:

• System 1 is a fast, intuitive, parallel thinking mode, similar to the powerful pattern recognition capability of sub-symbolic AI.

• System 2 is slow, methodical, logic-based deliberative thinking, just like the reasoning process of symbolic AI.

• Neuro-symbolic AI combines the pattern recognition of System 1 with the logical reasoning of System 2, enabling the two to work together.

  

  Figure 1.4 Knowledge representation paradigms of sub-symbolism, symbolism, and neuro-symbolic hybrid

Human intelligence stems from the collaborative work of these two systems. Similarly, a truly robust AI also needs to combine the strengths of both. Large language model-driven agents are an excellent practical example of neuro-symbolism: their core is a large neural network that provides pattern recognition and language generation capabilities; during operation, they also generate a series of structured intermediate steps, such as thoughts, plans, or API calls, which are explicit, operable symbols. In this way, they combine neural pattern recognition with symbolic logical reasoning.