Do We Really Need an LLM for Every Decision?

There is a strange assumption creeping into AI application development:

If we’re building an AI product, every decision should involve an LLM.

I don’t think that’s good architecture.

LLMs are powerful

LLMs are particularly useful when a problem involves:

• Natural language
• Ambiguous intent
• Unstructured information
• Reasoning over context
• Summarization
• Classification
• Generating human-friendly responses

But that doesn’t mean they should control everything.

Consider a payment workflow

Suppose we need to check whether a payment is above ₹100,000.

We don’t need:

User
 ↓
LLM
 ↓
"Is this payment above ₹100,000?"
 ↓
Yes

We can simply write:

if payment.amount > 100000:
    require_additional_review()

It is deterministic.

It’s easy to test.

It’s cheap.

And we know exactly why the decision happened.

Where the LLM belongs

Now consider:

“The customer says they don’t recognize this payment and believes someone else may have used their account.”

That’s different.

We may need to understand the customer’s intent, extract relevant information, and determine which workflow should handle the request.

That’s where an LLM can provide value.

Good AI architecture

A production AI system will often look more like:

                 User
                   |
                   v
            LLM / Intent Layer
                   |
          +--------+--------+
          |                 |
          v                 v
     Business Rules      AI Workflow
          |                 |
          +--------+--------+
                   |
                   v
                Result

The LLM doesn’t need to own every decision.

It needs to own the decisions where language and reasoning actually add value.

That’s an important architectural distinction.