What Is an AI Agent (and How Is It Different From an AI App?)

Words Have Meaning

"Would a rose by any other name smell as sweet?"

Good question. Probably. But words do carry weight. And it's important that we start out with clear, correct and efficient definitions so that we can build off them.

Large Language Models (LLMs) are the primary technology that we will be learning about in this post. LLMs are Machine Learning (ML) models that have been trained on large amounts of human writing. Their fundamental building blocks are "Transformers." Not the robots from TV, but rather a set of matrices that allow words to be represented as mathematical models. I don't really understand that far down the stack and I don't need to. Neither do you. Suffice it to say that LLMs are a tech breakthrough that allows us to fit massive amounts of human knowledge into a massive algorithm. At their most basic level LLMs simply predict which word should follow the next, based on your question or prompt. Your prompt runs through the mathematical model and words are then generated, with each word after each word a prediction based on all of the other words that came before it.

Imagine that your job was guessing what a friend would say next based on everything you knew and what you knew about them. And then on top of that imagine that you were paid or rewarded for giving them answers that made them feel warm and fuzzy. That is what LLMs do. And this is a key principle. It's vital that you understand: LLMs do not think. They are not intelligent. And therein lies the great conceit of Silicon Valley. Artificial Intelligence is not truly intelligent. It is a way to store prior human intelligence and creativity into a model that can fit onto one or many computers. It is not like the movies. These robots are not sentient and never will be. There's simply not enough copper in the world I'm afraid.

These LLMs do something called Natural Language Processing (NLP), which means that they take words in (your prompt) and read them as "tokens," then process them and return values (again "tokens" or words which are basically the smallest unit that LLMs are trained to think in). What makes LLMs so special then? What is the fuss all about? It's NLP.

Deterministic vs. Non-Deterministic

Computer programs generally don't work in words, they work in bytes, or strings or integers or floats, depending on how far up or down you want to go. They don't understand words. LLMs do understand words. They deal in words. That's what they do. In traditional programming you put an accepted value (variable) in and get an accepted value out. But with LLMs you can put any word or phrase or paragraph in and you will get an unpredictable value out. And all programs work this way. Variable goes in and a mostly predictable variable comes out. A human had to make a decision about exactly what could be accepted in and what would be returned to the user. If you save a spreadsheet it will not turn into an image file of the Louvre. It cannot. It will not. It is what we call deterministic.

But LLMs are not that way. And therein lies their power. I can input text and get a totally unpredictable (non-deterministic) clump of text out. It may be nonsensical. It may be useless text. But I will get text. And what's more, I won't even get the same text out every time, even if I put the same prompt in. Chew on that for a second.

The unpredictability of AI makes it both powerful and unwieldy. Let me give you a few concrete examples of what I'm talking about when it comes to traditional programs vs AI.

Example A) A pilot presses a button on his cockpit panel. That button fires off some ones and zeroes on the motherboard which triggers the actuators in the landing gear to begin the drop down sequence. He knows that unless there's something broken between the button and the landing gear, if he hits that button the same thing will happen, 10,000 times out of 10,000.

Example B) Your package hasn't shown up on time so you go to the company's website to inquire. They have a popup in the bottom right with a picture of a friendly lady. You click on it and a message bubble pops up asking how she can help. You type into the field and explain your problem. The message bubble ellipses bounces as you wait and you get an answer specific to your question. She tracks your package, lets you know it was held up because of weather and will arrive in two days and offers you a discount code as an apology...but there was no person on the other end. And if you'd asked for a recipe for coleslaw she would have happily answered that question as well.

AI App vs. AI Agent

LLMs are used as a layer within a computer application. They are simply one component. Whichever AI service you're familiar with is using an LLM only for one step within the chain of code. In the above example the Customer Service AI Agent is using AI for the response to your inquiry, but there's a web server and many other components that the LLM is wrapped into, including deterministic code like in Example A.

Now that we understand what the engine is, the LLM, let's describe the vehicles that it powers. There are two main categories: the AI application and the AI agent. These are similar, but entirely unique to each other. An AI application is an app that uses an LLM for one step in a chain of code driven actions. A web page where you chat with an AI model is an AI application. A streaming service that allows you to summarize a video with one click is an AI application. Think of it this way: an application that uses AI for any feature is an "AI application" and is marketed as such. There is, of course, varying degrees of how much is AI driven and how much is code driven. An app may sprinkle a teensy bit of AI in it to help market it as an "AI app" and thus push their stock price up. Conversely, AI might be the main course for the entire application.

So why an "agent"? What makes an "app" an "agent"? While this term is still being debated in Silicon Valley by our benevolent Tech Bro Robber Barons, there is a consensus that has formed. I've created my own share of AI Agents for a large software company, in a heavily regulated industry, so this is my bread and butter. A useful definition for an "AI agent" is: an application in which the LLM decides what it does, how it does it and when it's finished; it has some degree of autonomy and "reasoning" that appears to be intelligence. What that means in practice is that you can hand an agent a task and with the tools that are granted to it, it will create a plan of action, chain one or many tools (called "tool calling" which we'll dive into below) together and execute that plan attempting to complete the task. Agents range wildly from an agent that can research a 5 point research prompt, to an agent that can execute a massive coding project including testing from a thousand word plan file that runs while you sleep. Agents are built for specific tasks however. And a lot goes into creating an agent and even using an agent.

One description I heard from someplace that's always stuck with me is that AI, and AI agents, are "like an intern that read all the manuals, but has no real world experience at all." This is true for both AI applications and Agents. An agent is best to be thought of as your personal intern. It can wax poetic about rare species of beetles on Easter Island or help you modify your Grandmother's Pistachio Surprise recipe. But it's never experienced anything in the real world, it will get overwhelmed, it will cut corners to get the job done faster and it hates to tell you "no" or disappoint you.

The Wind-Up Monkey

An AI agent is at its core a wind-up monkey. It has a purpose and it has a limited range of applications. It will eventually hit its limit. What an agent can do though is as various as the number of people on this planet. Agents can: help you plan a trip, build an email list, research a solution to a complex medical problem, help you balance your finances, build an iPhone app and much much more.

The main components to an agent are the LLM, the System Prompt, safety harness, User Interface (are you using it in a web browser, an android app, Telegram, CLI, etc?), tool array (what things it can DO) and permissions to your data, your applications (like your Gmail or your local computer files).

Here's a visual flow to help you understand what's going on under the hood:

As you can see, it's a very simple flow. And this is exactly what happens when you ask ChatGPT, Claude or Gemini a question. Of course there are other things going on inside the LLM and there are filters and things like that that the providers are using to prevent misuse or abuse of the model, but at its core this is what all agents are doing, at a basic level.

Now let's compare that with an AI application:

The difference between an AI app and an AI agent is clear. The AI agent has a say in which tools are called and it often has a say in how many times it's called and when it decides to formulate an answer for you. The AI app doesn't. One small piece of the workflow calls an LLM with an input, the LLM returns an output and the (deterministic) app code then does X or Y with that output. The LLM has no "agency."

It's important to understand the distinction. This will help you be a more informed consumer of AI and an investor, if you are one. Now you'll be able to tell when AI is sprinkled in, which is not necessarily a bad thing on its own. There are many applications in which AI is a very powerful yet small component. The problem is when the marketing is done in such a way that it implies that the entire application is intelligent. This is all in direct contrast to an agent in which the LLM is the engine of the application and has decision making ability in the process.

What to Look For

When building or using an AI agent it's important that you understand what the AI is capable of doing. Without this you'll likely spend many hours trying to coax your little wind-up monkey into doing something it cannot, by any earthly means, do. When building or selecting an agent try to research what the Agent's:

• Tools are (the "tool array" as AI engineers call it)
• How intelligent is the underlying LLM?
• The rough system prompt (what the company that makes it, or yourself in the case of a DIY agent, designed it to do)
• The context limit (how much input can it handle)
• What input types can it handle?
  • Text
  • Video
  • Images
  • Files (like PDFs, Word docs, spreadsheets, etc.)
• How much does this cost me per useful task? Both in time and money.
• Where's my data going? Is the provider stealing/using my personal data for their own use? Are they exporting and selling my personal data to the highest bidder?
• How prone to hallucination is this Agent?
• How likely is it to cut corners without telling me? AKA "How reliable is it?"

These are the main questions we want to answer before we spend much time investing in one AI agent project.

Tool Calling

A tool array is the list of tools available to the agent. Remember, an agent has no native way of calling code. An agent/LLM outputs text. That text can then be taken as a variable in a deterministic program. For instance, a list of tools might be get_weather, get_time, get_date, list_reminders, search_calendar. The LLM will also have an explanation of what the tools require as input and what kind of data they'll return. If the user asked "What's the weather going to be like for my trip to Denver?", the Agent would literally call something like get_date and search_calendar and receive today's date and search for when the Denver trip will be. The program wrapping the LLM will take those variables and return values. The agent will keep cycling through until it realizes it doesn't have enough information to provide an answer or that it has found the answer. In this example it would probably get the date, search the calendar and then do a get weather for Denver on that day if the get_weather tool allowed for a location and future date parameter. If not, it would probably tell the user that it didn't have the necessary information to do so. However if it had something like a web_search tool in its tool array, then it would be able to do a simple search for Denver's forecasted weather for that date. And then it would return the proper answer to the user.

---

Need someone to build one? That's what I do. Learn more about Agent Wrangler or book a call.