Applied Behavior Analysis for Game and UX Designers

by ... posted on
GAMEDEV, PSYCH

Earlier this year, I decided to go “back to school” and study Psychology in an academic setting. Balancing full-time work with coursework is demanding but fortunately, the classes are online and asynchronous so it’s feasible with a little planning and effort.

My first class of the Spring semester was Applied Behavior Analysis (ABA) and I’ve found its principles are applicable (no pun intended) in the realm of both Game Design and UX Design.

ABA is a form of therapy that involves understanding and changing behavior through observable, measurable principles. It’s often used to help young children with development disorders learn new skills. But beyond its clinical use, the ABA framework offers powerful tools for shaping behavior in any system including video games!

What is Applied Behavior Analysis?

As I progressed through the class, I started to see game mechanics, player loops and First-Time User Experiences in a new light. Many of the concepts we studied such as Reinforcement, Generalization and Extinction, showed up constantly in game systems I’ve built or played. The scientific rigor of ABA offers a new language for understanding why players behave the way they do and how to design systems that support, shape, or redirect that behavior. So without further ado, here are a few key ABA concepts and how they show up in games!

Reinforcement

Reinforcement is everywhere in games and I mean everywhere. In ABA, reinforcement is the strengthening of a behavior by delivering a consequence after it occurs. There are two types - positive and negative; the former involves adding(+) a desired stimulus, the latter removing(-) an aversive stimulus.

In games, reward systems and positive feedback loops are forms of reinforcements. Various sensations that we often call “juice” are also reinforcement. Player actions are reinforced by sound effects, particle effects, character animations and more.

Punishment

Punishment is when a behavior is weakened by delivering a consequence. Like reinforcement, there is positive punishment and negative punishment; the former introduces(+) an averse stimulus while the latter removes(-) a reinforcing stimulus. Note that positive and negative does not mean “good” or “bad” but more like Mathematics, it’s addition and subtraction. Regardless of “polarity,” the goal of both types of punishment is to reduce the likelihood of the behavior recurring.

In games, Punishment includes failure states, removal of user input or introduction of time-outs. Players who prematurely quit multiplayer battles may be punished by increasing matchmaking time. Roguelikes thrive on punishment by removing all items and progress on failure which in turn, discourages reckless behaviors. Cooldowns kick in to discourage spamming emote inputs that might irritate other players.

Influential Factors of Reinforcement and Punishment

The effectiveness of both Reinforcement and Punishment is influenced by Immediacy, Contingency, Motivating Operations, Magnitude, and Individual Differences.

  • Immediacy links a behavior to a stimulus based on time. If there is a delay, the link is weakened and so is the effectiveness of the reinforcement or punishment. In games, immediacy significantly helps with clarity. When an experience progress bar fills up right after defeating an enemy, it is clear what action helped improve the player character and will reinforce similar actions.

  • Contingency is the consistency of the stimulus occurring every time the behavior occurs. Without contingency, behavior modification will not occur as the stimulus will seem randomly applied. A player must be punished with a Game Over each and every time they fall in a pit in a platformer. If they only “died” sometimes when they fell in a pit, it wouldn’t be clear what behavior should be avoided. Fortunately, contingency in games is easy to implement due to their systemic nature.

  • Motivating Operations are preceding events that change the effectiveness of the reinforcer or punisher which in turn affects the promotion or reduction of the behavior. Recently, games often use Daily Login bonuses which grant various bonuses or currency to improve play sessions and encourage daily play behavior. Various collectables in a survival game can become more enticing to collect if the environment is tuned correctly. For example, if the player dies from starvation, collecting food becomes important and so hunger is a stronger reinforcer.

  • Magnitude means that the intensity of the reinforcement or punishment influences how effective it can be. Oftentimes, rewards and punishments are proportional to player behavior. Health points are often removed based on the magnitude of mistakes. For example, slow moves might be more damaging since they should be easier to dodge. Or defeating multiple enemies in a row is rewarded with more points or bombastic acknowledgment from an in-game announcer.

  • Individual Differences means that everyone assigns different values to stimuli and that what might reinforce and punish one person might not equally affect another. Basically, don’t assume everyone is the same! Leveling up might motivate one type of player in an MMORPG but another player might not care about their character as much as social interactivity with other players.

Extinction

Extinction is the removal of a reinforcer to decrease a behavior over time. In order to use extinction procedures, the specific reinforcer must be identified and eliminated.

Farming weaker enemies in order to defeat the final boss is often not possible in RPGs because extinction is implemented by awarding less experience or granting less and less loot. This encourages the player to move to new areas and explore the rest of the game. Extinction is often used to nudge players into using newly introduced mechanics by reducing the effectiveness of older ones.

Generalization

Generalization is when a behavior that occurs in one context successfully occurs in another similar context. Typically, the former context is the training environment while the latter one is a non-training scenario. And if you thought of in-game tutorials, you would be right! Tutorials are attempts at generalization as skills learned in the tutorial are used to conquer progressively difficult challenges in the levels that follow it.

Discriminative Stimuli (SDs)

These are the antecedent stimuli that are present when a behavior can be reinforced. Think of these as cues. Glowing collectables, telegraphing attack danger zones, NPC quest indicators, subtle vibrations when near a hidden item, all of these are examples of SDs. They tell the player that a desirable interaction is available and they should go do it.

Prompting

Prompting is when Discriminative Stimuli is presented before the desired behavior has occurred in order to assist or guide the subject to perform said desired behavior. Prompting provides an opportunity for reinforcement. Tutorial pop-ups and hint systems are examples of prompting. Think of it as the provision of the cue. It’s particularly important for good UX design as prompting increases the probability of correct responses and prevents frustration. It also aids learning through guided practice.

It’s important to note that prompts go beyond teaching new skills. Prompts can involve appropriate timing and usage of existing skills. For example, a player may already know how to jump to clear obstacles in a platformer but might need prompting to learn that jump can be used to stomp enemies or break overhead blocks.

Fading

Fading is the gradual removal of prompts as the learner becomes more independent. The idea behind fading is the player can perform a behavior without any assistance. Typically you would start fading once the individual shows progress. Many games show an input guide in the HUD and once a player has performed enough actions, it is literally faded away as a form of ABA fading. Fading can beyond this into level design with steady removal of environmental hints in platformers or puzzle games. If prompting is implemented, fading must be used to ensure long term learning and independence.

An interesting form of fading is gradual progression across different prompt types. There are actually different types of prompts such as visual, verbal and modeling prompts. Different prompts can have different levels of intrusiveness and this is important to consider. For example, some games might stop player input and show an NPC Ghost or a video on how to perform new moves. This would be a modeling prompt and to fade this, we might transition to a verbal prompt where an NPC might simply announce what to do. This could be further faded into an in-world contextual icon in the UI.

Shaping

Shaping is the reinforcement of successive approximations of the desired behavior. Basically, a complex behavior is broken down into simpler or easier ones and these are reinforced gradually. A key aspect of shaping is differential reinforcement where closer approximations of the behavior are rewarded while mistakes are not. Shaping is immensely important for building momentum and confidence; it’s the reason why difficulty curves exist. Initial levels often focus on teaching the game’s rules and idiosyncrasies; eventually the player graduates into harder levels that introduce more challenges and newer concepts to apply their skills towards.

Token Economies

In ABA, a token economy is a behavior modification system where tokens are used to reinforce desirable behaviors by assigning value to them using backup reinforcers. In other words, exchangeable tokens are used to encourage behaviors. Also, undesirable behaviors can be discouraged by removal of these tokens, adversely changing the exchange rate or reducing the availability or value of backup reinforcers.

Token economies are utilized in so many games. Think of activity punch cards, free gacha pulls, Achievements or Trophies, materials for item crafting, in-game currency or experience points and you pretty much know what a token economy in a game looks like.

An important aspect of token economies that I rarely see in games is the fading of the tokens. Designers must fade tokens or generalization may fail to occur. A good time to fade tokens is when players are consistently earning a huge amount of tokens. When this happens (assuming there’s no cheating or token counterfeiting), it means the player’s target behavior is consistently reinforced by the tokens. When this happens, it’s a good idea to fade the tokens in lieu of the game’s natural reinforcers. When a game does not fade its tokens, it might mean it lacks natural reinforcers which is a whole different problem, one that I would personally categorize as catastrophic. In other words, I believe your game should not be the token economy itself; the token economy should support the game and not be the game!

Conditioned Stimuli (CS) and Conditioned Responses (CR)

To understand these and how they relate to games, we must first know about Unconditioned Stimulus (US) which is a stimulus that automatically triggers a response without prior learning. Dust in the nose is a US that causes sneezing, which is an Unconditioned Response (UR). We don’t learn to sneeze; we just do it.

A Conditioned Stimuli (CS) is when a neutral stimulus is paired with a US to invoke a response identical (or similar) to the UR. This response is learned so it is a Conditioned Response (CR). The classic example of this is Pavlov’s bell (CS) which made dogs salivate (CR) without the presence of food (US).

This technique is also called Respondent Conditioning and repeated pairing of US and CS can be used to affect player behavior and invoke emotional responses. For example, sudden, loud noises (US) tend to startle us and we might jump or gasp (UR). By pairing these loud sound effects with enemy appearances or attacks, they can now become Conditioned Stimuli. With enough repetition, it might be possible for a player to feel a fear response (CR) to the enemy even without the loud noises (US). A similar example works for Invincibility music which might hype someone up at the gym while they’re working out.

Desensitization

Desensitization is the process of increasing a specific behavior response by gradual, repeated exposure to a stimulus. It’s often used to treat fears but it’s possible to use in the world of gaming too. Think of intense multiplayer matches with other human players; many successful titles usually have you playing with easy bots first and slowly reduce the bots and introduce more human players. This is a form of desensitization that is called In Vivo Desensitization because we’re working with a gradual exposure to the actual difficult situation. The other form is called Systematic Desensitization which deals with imagining or simulating the scenario while maintaining a relaxed state. The intention is to replace the fear response with the relaxation response. Video tutorials or streamer gameplay is an example of this; you’re not playing with humans but you’re seeing how a game could go down and this can help increase your confidence when it’s time to play against others.

Telemetry and Behavior Modification Procedures

In order to change behavior, one must first observe and record them. Many games are run on computers that are connected to the internet so this is actually possible. Using telemetry, we can collect player behavior data and measure their dimensions to draw different conclusions.

There are four dimensions of behavior: Frequency, Duration, Intensity, and Latency. A handy mnemonic can be FIDL, said aloud as fiddle, where each letter represents the starting letter of each of the four dimensions.

  • Frequency can be defined as the rate at which the behavior occurs. It is measured by tracking the number of instances of the behavior that happened over a period of time.

  • Duration is how long a specific instance of the behavior lasted. It is the time between when a behavior started and when it stopped.

  • Intensity is the measure of physical force or magnitude of the instance of the behavior. Intensity requires some sort of scale where values can be rated and relativity can be understood.

  • Latency can be defined as the time between an event and the start of the behavior. It is helpful to think of it as the “lag” between what stimulates the behavior to the actual occurrence of the instance of the behavior.

The most important takeaway here is these dimensions are measurable aspects of behaviors. Because they can be tracked, they can be studied and used for creation and assessment of behavior modification procedures.

In games, we add telemetry hooks so we can record various actions in the game: how long game sessions are, what level players might quit at, which tools were used the most, etc. When we update the game, we can collect the data again and see how effective our changes may have been.

Conclusion

I believe ABA can be used to make better games. We can use Prompting, Fading and Shaping to build adaptive user interfaces that let all sorts of players enjoy the game and improves its accessibility. By understanding Reinforcement and Punishment, we can build better game loops and balanced mechanics. Extinction can gently push gamers to explore new gameplay systems. When it comes to Live Ops, Motivating Operations and Token Economies are instrumental in keeping a game engaging over a long period of time. The science of ABA can help us understand gamer behavior that we innately know but can’t explain easily.