What Actually Happens Inside an AI Agent?

“AI Agent” has become one of those terms that can mean almost anything.

So let’s make it simple.

At its core, an agent is a system that can decide what to do next based on a goal and the information available to it.

The basic loop

A simple agent can be represented as:

        Goal
          |
          v
       Reason
          |
          v
     Choose Action
          |
          v
     Execute Tool
          |
          v
       Observe
          |
          +----------+
                     |
                     v
                  Reason

The loop continues until the agent believes the task is complete.

Why tools matter

An LLM by itself mostly produces tokens.

An agent becomes much more useful when it can interact with the outside world.

For example:

LLM
 |
 +--> Search customer
 |
 +--> Query database
 |
 +--> Call API
 |
 +--> Create ticket
 |
 +--> Send notification

The model decides what action should happen.

The application executes the action.

That distinction is extremely important.

The agent is not the LLM

A useful mental model is:

                 Agent
                   |
       +-----------+-----------+
       |           |           |
       v           v           v
      LLM        Tools       State
       |           |           |
       +-----------+-----------+
                   |
                   v
              Orchestrator

The LLM is one component.

The surrounding system provides:

• Tool execution
• State management
• Permissions
• Memory
• Retries
• Observability
• Guardrails
• Workflow control

This is where agent engineering starts becoming an architecture problem rather than just a prompting problem.

And that’s where things get interesting.