Sensors and Perception: How an NPC Sees and Hears

An enemy that turns its head, sees the player peek around a corner, and shouts "there he is!" looks like it understands the world. It understands nothing. Underneath it there are a handful of geometric checks running every frame: an angle, a distance, a ray that hits a wall. The feeling that the NPC perceives is, almost always, the sum of cheap tricks well orchestrated.

This article is about those tricks. How to build a vision cone that doesn't fire false positives, how a guard "hears" a gunshot without actually simulating acoustics, and how to make it remember what it has seen instead of forgetting it the moment the player crouches behind a crate.

1. AI doesn't see, it simulates seeing

The first thing worth internalizing is that an NPC's perception is a query system, not a physical simulation. Consoles and computers don't cast photons, nor is the NPC's retina modeled. Instead, they ask concrete questions: is the target inside the field of view? is something covering it? how far away is it? Each question is cheap on its own, and the order in which they are asked matters for performance.

Perception also has to be tolerant. A player who peeks half a pixel around a corner shouldn't trigger the highest alert instantly, just as a real guard doesn't detect someone in the gloom immediately. That's why it can be split into two concepts that often get mixed up:

• Detection: answers "can I perceive this stimulus right now?"
• Knowledge: refers to "what do I know about the target, even if I can't see it right now?"

The first is instantaneous and stateless. The second is memory, and it's what gives the impression of intelligence.

2. The vision cone

The "Vision Cone" is a concept that combines two checks: a distance limit and an angle limit. Distance is trivial (a sqrMagnitude compared against the radius squared, avoiding the square root). The angle is where people overcomplicate things.

The intuitive way is to use Vector3.Angle, which returns degrees. It works, but it computes an arccosine internally. The cheap way is to compare dot products: if you normalize the direction toward the target and compare it with the NPC's look direction, the dot product gives you the cosine of the angle between the two. By precomputing the cosine of the half view angle once, from there on you only need to compare numbers.

1public class VisionCone : MonoBehaviour
2{
3    [SerializeField] private float viewRadius = 12f;
4    [SerializeField, Range(0, 360)] private float viewAngle = 90f;
5
6    private float _cosHalfAngle;
7
8    private void Awake()
9    {
10        // Precompute the cosine of the half angle only once.
11        _cosHalfAngle = Mathf.Cos(viewAngle * 0.5f * Mathf.Deg2Rad);
12    }
13
14    public bool IsInViewCone(Vector3 target)
15    {
16        Vector3 toTarget = target - transform.position;
17
18        // 1) Distance filter without a square root.
19        if (toTarget.sqrMagnitude > viewRadius * viewRadius)
20            return false;
21
22        // 2) Angle filter with a dot product.
23        Vector3 dir = toTarget.normalized;
24        float dot = Vector3.Dot(transform.forward, dir);
25        return dot >= _cosHalfAngle;
26    }
27}

Order matters: first discard by distance, which is the cheapest, and only if the target is in range do you compute the angle. This "filter by increasing cost" pattern repeats across the whole perception system. Each expensive check only runs if the cheap ones have already passed.

3. Line of sight: the raycast

Being inside the cone doesn't mean seeing. Between the NPC and the target there might be a wall, a column, or a container. For that, you cast a ray from the NPC's "eyes" toward the target and check what it hits first. If the first thing it hits is the target, there is line of sight. If it hits geometry first, the target is hidden.

The key here is to filter well what can block the ray. The raycast must ignore the NPC itself and consider only obstacles (and optionally the player). In Unity this is solved with a well-configured LayerMask; in Unreal, with collision channels and a list of ignored actors. In both cases the goal is the same: prevent the ray from hitting the enemy's own collider or irrelevant triggers.

1[SerializeField] private Transform eyes;      // ray origin point
2[SerializeField] private LayerMask obstacleMask;
3
4public bool HasLineOfSight(Vector3 target)
5{
6    Vector3 origin = eyes.position;
7    Vector3 dir = target - origin;
8    float distance = dir.magnitude;
9
10    // If the ray hits no obstacle, the view is clear.
11    return !Physics.Raycast(origin, dir.normalized, distance, obstacleMask);
12}

One raycast per enemy per frame is acceptable. The problem shows up with dozens or hundreds of NPCs. There it pays to stagger the checks: not every enemy needs to review line of sight every frame. Spreading them out over time (for example, each NPC checks every 3 or 4 frames, offset from each other) reduces the cost spike without the player noticing the delay of a few milliseconds.

4. Hearing the world: sound events

Sight is directional; hearing is not. An NPC doesn't need to look toward a gunshot to hear it. Instead of simulating wave propagation, you can use an event model: when something makes noise (a footstep, a gunshot, a door), it emits a "sound event" with a position, an intensity, and a radius. Any NPC within that radius receives the event.

The elegant thing about this model is that it decouples the emitter from the receiver. The player who fires knows nothing about the enemies; they just publish an event. The NPCs subscribe and decide what to do. A ScriptableObject as an event channel works very well for this in Unity; in Unreal, a UObject with a multicast delegate (or the AIPerception system itself with AISense_Hearing) plays the same role.

1// --- Emitter ---
2public struct SoundEvent
3{
4    public Vector3 position;
5    public float radius;      // how far it reaches
6    public float intensity;   // 0..1, how loud it "shouts"
7}
8
9public class Weapon : MonoBehaviour
10{
11    [SerializeField] private SoundChannel soundChannel;
12
13    private void Fire()
14    {
15        // ... firing logic ...
16        soundChannel.Raise(new SoundEvent
17        {
18            position = transform.position,
19            radius = 20f,
20            intensity = 1f
21        });
22    }
23}
24
25// --- Receiver ---
26public class Hearing : MonoBehaviour
27{
28    [SerializeField] private SoundChannel soundChannel;
29
30    private void OnEnable()  => soundChannel.OnSound += HandleSound;
31    private void OnDisable() => soundChannel.OnSound -= HandleSound;
32
33    private void HandleSound(SoundEvent e)
34    {
35        float sqrDist = (e.position - transform.position).sqrMagnitude;
36        if (sqrDist > e.radius * e.radius) return;
37
38        // Perceived intensity decays with distance.
39        float dist = Mathf.Sqrt(sqrDist);
40        float perceived = e.intensity * (1f - dist / e.radius);
41
42        // Only reacts if it exceeds the attention threshold.
43        if (perceived > 0.15f)
44            InvestigatePosition(e.position, perceived);
45    }
46}

Keep in mind that sound gives no certainty, it gives an approximate direction. The NPC doesn't know what produced the noise, only where. That's why the correct reaction to a sound is not "attack", but "investigate": move toward the point of origin and increase vigilance. It's sight, afterward, that confirms or rules out the threat. This division between "hear to suspect" and "see to confirm" is what makes stealth feel fair.

5. Memory: remembering what was perceived

This is where an AI stops looking like a sensor and starts looking like a character. Without memory, as soon as the player breaks line of sight the NPC forgets them and goes back to patrolling as if nothing happened. With memory, the guard remembers where it last saw the player, goes there, and searches for a while before giving up.

The minimal model is a "last known position" (last known position) plus a confidence level that decays over time. While the NPC perceives the target, confidence rises and the known point is updated. When it loses sight of the target, confidence starts dropping little by little. Only when it reaches zero does the NPC "truly forget" and return to its routine.

1public class PerceptionMemory
2{
3    public Vector3 LastKnownPosition { get; private set; }
4    public float Confidence { get; private set; } // 0..1
5
6    private const float DecayPerSecond = 0.2f;
7
8    // Called every frame it perceives something of the target.
9    public void Reinforce(Vector3 position, float strength)
10    {
11        LastKnownPosition = position;
12        Confidence = Mathf.Min(1f, Confidence + strength);
13    }
14
15    // Called every frame it perceives NOTHING.
16    public void Decay(float deltaTime)
17    {
18        Confidence = Mathf.Max(0f, Confidence - DecayPerSecond * deltaTime);
19    }
20
21    public bool HasTarget => Confidence > 0f;
22}

The gradual decay is what gives the human behavior. A guard that has just lost sight of the target (high confidence) heads straight and decisively to its last known position. A guard that hasn't seen the player in a while (low confidence) searches more loosely and gives up sooner. Tuning the decay rate per enemy type (a guard dog forgets fast, a surveillance drone barely forgets) gives variety without writing different AIs.

6. Alert states

Confidence is a continuous number, but the NPC's behavior is usually expressed in discrete states: unaware, suspicious, alerted. Mapping the number to states with thresholds prevents the enemy from changing behavior every frame, and makes it easier to hook animations, music, and decisions to each level.

The trap is using a single threshold to go up and down. If the cut between "suspicious" and "alerted" is at 0.5, a target hovering around that value will make the NPC flicker between both states. The solution is hysteresis: require more to enter a state than to leave it. You enter alert at 0.7, but don't leave it until dropping below 0.4. That dead band eliminates the flicker.

1public enum AlertState { Unaware, Suspicious, Alerted }
2
3public AlertState Evaluate(AlertState current, float confidence)
4{
5    switch (current)
6    {
7        case AlertState.Unaware:
8            if (confidence > 0.3f) return AlertState.Suspicious;
9            break;
10
11        case AlertState.Suspicious:
12            if (confidence > 0.7f) return AlertState.Alerted;   // going up costs more
13            if (confidence < 0.1f) return AlertState.Unaware;
14            break;
15
16        case AlertState.Alerted:
17            if (confidence < 0.4f) return AlertState.Suspicious; // going down costs less
18            break;
19    }
20    return current;
21}

Each state triggers a different behavior: in unaware the NPC patrols, in suspicious it investigates the last known point and looks around, in alerted it chases and attacks. By hooking the state machine to confidence (and this to the sensors and memory), the whole chain stays connected: I see or hear something, confidence rises, I change state, I change behavior.

7. Putting it all together

The pieces on their own are simple. Convincing perception is born from orchestrating them in the right order, every frame. A central PerceptionSystem collects the stimuli from the sensors, updates the memory, and exposes an alert state that the rest of the AI (the behavior tree, the state machine) can query.

1public class PerceptionSystem : MonoBehaviour
2{
3    [SerializeField] private VisionCone vision;
4    [SerializeField] private Transform target;   // the player
5
6    private readonly PerceptionMemory _memory = new();
7    private AlertState _state = AlertState.Unaware;
8
9    public AlertState State => _state;
10    public Vector3 LastKnownPosition => _memory.LastKnownPosition;
11
12    private void Update()
13    {
14        bool sensed = false;
15
16        // Sight: cone + line of sight (increasing-cost filters).
17        if (vision.IsInViewCone(target.position) &&
18            vision.HasLineOfSight(target.position))
19        {
20            _memory.Reinforce(target.position, strength: 1.5f * Time.deltaTime);
21            sensed = true;
22        }
23
24        // Hearing resets the memory via events (see section 4),
25        // no need to query it here every frame.
26
27        if (!sensed)
28            _memory.Decay(Time.deltaTime);
29
30        _state = Evaluate(_state, _memory.Confidence);
31    }
32}

From here on, the rest of the AI doesn't touch sensors or raycasts: it only reads State and LastKnownPosition. That separation is what keeps the system manageable as it grows. You can add a new sensor (smell, touch sensor, corpse detection) without touching the behavior tree, because they all feed the same memory and the same alert state.

Conclusions

An NPC's perception is neither magic nor an expensive simulation. It's a cone, a ray, a sound radius, and a decaying number, connected with judgment. No single piece is hard on its own; what convinces the player is orchestrating them in the right order and letting them feed back into each other.

• AI doesn't see, it queries. Perception is a set of cheap questions (angle, distance, occlusion), not a physical simulation.
• Filter by increasing cost. Discard first with the cheap stuff (distance with sqrMagnitude), and reserve the expensive stuff (the raycast) for when the rest has already passed.
• Separate detection from knowledge. A boolean canSeePlayer is the number one cause of flickering AIs; memory is what gives the impression of intelligence.
• Hearing is suspecting, seeing is confirming. Sound gives direction, not certainty: the correct reaction to a noise is to investigate, not to attack.
• Decaying confidence is the character. A last known position plus a number that drops over time turns a switch into a guard that hesitates, searches, and gives up.
• Use hysteresis in the states. Requiring more to enter alert than to leave it eliminates the flicker between behaviors.
• Centralize and decouple. A PerceptionSystem that exposes State and LastKnownPosition lets you add new sensors without touching the behavior tree.

When those four pieces work together, the player swears the enemy has seen them, heard them, and remembers them. And in a sense, it's true.