Game AI without machine learning

Every enemy that breaks off an attack to reload, every guard that stops patrolling to investigate a noise, runs on code you could write in an afternoon. Finite state machines, behavior trees, and utility AI account for nearly all shipped game AI, in budget AAA combat encounters and game jam boss fights alike. None of it touches a model or a training run.

Say “game AI” to a player and they picture something adaptive. Say it to the person who shipped the enemy, and they usually mean a state machine that swaps between three animations based on distance to the player. The three techniques below cover almost every behavior you will actually need, and picking the right one up front saves you from writing a behavior tree for a wall turret or a finite state machine for a boss with nine interlocking abilities.

What is a finite state machine?

A finite state machine, FSM, is a small set of named states, like idle, patrol, chase, and attack, where exactly one state is active at a time. Each state defines what the character does while it holds that state, and a set of transitions defines which conditions move it into a different state. An enemy in CHASE checks each frame whether the player is in attack range; if so, it moves to ATTACK.

FSMs are the right tool when a character has a handful of clearly distinct modes and the current mode is all the logic needs to remember. A patrolling guard, a dialogue tree, a menu screen: all of these are state machines whether you call them that or not.

They break down through state explosion. Add a “stunned” state to an enemy that already has five states, and every one of those five needs its own transition into stunned and usually its own transition back out. Past a dozen or so states, the transition table stops fitting in your head, and bugs show up as “the enemy got stuck in ATTACK after being stunned during ATTACK” because one transition edge was never written.

state = IDLE

function update():
    switch state:
        case IDLE:
            if player_seen(): state = CHASE
        case CHASE:
            if player_in_attack_range(): state = ATTACK
            elif not player_seen(): state = IDLE
        case ATTACK:
            if not player_in_attack_range(): state = CHASE
            else: perform_attack()

What is a behavior tree?

A behavior tree is a tree of nodes evaluated from the root every tick. Composite nodes control the flow: a sequence runs its children in order and stops at the first failure, a selector runs its children in order and stops at the first success. Leaf nodes are the actual conditions and actions, like “health below 20%” or “run to cover,” and because selectors express priority, a tree can check flee before attack without a hand-written transition for every combination.

Behavior trees are the right tool once a character has more than a few overlapping behaviors that need priority ordering: flee beats attack, attack beats patrol, and a designer should be able to see that ordering by looking at the tree. Subtrees are reusable too, so a “search area” branch built for one enemy type can be dropped into another enemy’s tree unchanged.

They break down through complexity creep. Nothing stops a tree from growing to forty nodes with decorators and cooldowns wired in at every layer, all reading and writing the same shared blackboard. At that size, the tree gives back exactly the readability problem it was supposed to solve: it takes real effort to trace why a given leaf fired on a given tick, and the fix for one bug, a decorator that suppresses an action for two seconds, has side effects on branches nobody was looking at.

Selector "root"
  Sequence "flee if low health"
    Condition: health < 20%
    Action: run_to_cover()
  Sequence "attack if in range"
    Condition: player_in_attack_range()
    Action: attack_player()
  Action: patrol()

What is utility AI?

Utility AI scores every candidate action against the current world state and picks the highest scorer each tick, instead of following a fixed branch. Each action carries one or more consideration curves, small functions that map an input like “distance to player” or “ammo remaining” to a score between 0 and 1. Multiply the curves together and an action with a single disqualifying factor, zero ammo for “reload,” drops to zero regardless of how good its other inputs look.

Utility AI is the right tool when a character has many considerations pulling in different directions and no clean priority order exists. Squad members deciding whether to push or fall back, weighing ally positions and health, are a good fit: a behavior tree would need a growing pile of special-case branches to approximate what a handful of curves handle directly.

It breaks down into tuning hell. Curves interact nonlinearly, so raising the weight on “aggression” to fix one encounter can make an agent suicidal in a different one, and there is rarely a single number you can point to as the cause. Utility AI is close to unusable without a debug overlay that shows the score breakdown per action per frame; teams that skip building that tool end up tuning by feel and re-breaking old fights with every new one.

actions = [attack, reload, retreat, take_cover]

function choose_action(world_state):
    best_action = none
    best_score = 0
    for action in actions:
        score = 1.0
        for curve in action.considerations:
            score = score * curve.evaluate(world_state)
        if score > best_score:
            best_action = action
            best_score = score
    return best_action

What should I start with?

Going deeper

Hierarchical FSMs fix the “stunned” transition problem by nesting states inside a parent state. Instead of wiring a transition from every leaf state into STUNNED, you wire one transition out of the whole “grounded” superstate, and every child state inherits it. Most FSM implementations that survive past a prototype end up hierarchical whether or not the code calls itself that.

Decorators wrap a single child node and modify its result: invert a condition, or add a cooldown before it can fire again. Services, the term Unreal’s Behavior Tree editor uses, run on a timer independent of tree evaluation and write to the blackboard, so an expensive check like “find nearest cover point” runs every half second instead of every tick. Both exist to keep the tree shallow instead of encoding more of that logic as nodes.

Utility curve design is its own small discipline. Linear curves are rare in practice; logistic and quadratic curves let a consideration stay near zero until a threshold, then ramp up fast, which matches how people actually reason about thresholds like “low health” better than a straight line does. Multiplying curves together, rather than averaging them, is what makes a single disqualifying factor able to veto an action outright, and it is usually the detail missing when a utility system picks actions that feel wrong despite every individual curve looking reasonable in isolation.

Learning-based approaches, reinforcement learning and behavior cloning mainly, earn their cost when the behavior space is too large to hand-author: continuous physical control, or a strategy opponent that needs to adapt because a fixed heuristic gets solved and exploited within a week of release. That describes a small fraction of shipped games. The AI Game Dev Org community covers that ground in Discord as projects run into it; there’s no dedicated article for it here yet.

Stuck between two of these three for a specific enemy? Describe it in the AI Game Dev Org Discord and someone will tell you which one you actually need.

Sources

• Game Programming Patterns: State
• The Behavior Tree Starter Kit (Game AI Pro)
• Building a Better Centaur: AI at Massive Scale (GDC Vault)