How Video Games Are Helping Us Understand AI
Maybe we’re not the good guys.
I run. I jump. I crouch. I move blocks. I do all of these things and more in games. When regaling others of our actions we become a synthesis of player and character. It’s an interesting boundary to play with as a developer, that moment we come to embody another figure with new goals, emotions, and abilities.
Across the video game medium, there are countless perspectives for players to embody that enable brand new contexts for understanding new experiences. Games allow us to become gun-slinging treasure hunters, cute Italian plumbers saving princesses, or even just regular old humans with different life experiences to our own. Some of the most thought-provoking titles, however, take that perspective and twist it to deliver a mirror on our time playing. The Talos Principle, for example, wraps its puzzles in a meditation on what it means to be sentient. BioShock asks players what it means to have agency within the medium of games itself.
So it is with Observation by development studio No Code, a game that asks the player questions about their own agency within the virtual space. Set in the near future inside an international space station, the player takes control of the station’s artificial intelligence system, S.A.M. After an incident has rebooted S.A.M’s systems, the player needs to help surviving crew member Emma Fisher figure out what happened to the station.
Writer and director Jon McKellan explains that their inspirations for the game came from all over, “everything from contemporary sci-fi movies such as Arrival and Interstellar, to real-life accounts of life on the ISS, and even works from H.P Lovecraft are a big influence.” The game proudly wears these inspirations on its sleeves and yet No Code have nevertheless been able to carve out a space where they have attempted something truly different with Observation.
Something interesting happens when we pick up a controller. A virtual world has been created, seemingly from nothing. And we extend our intentions and actions into that world through buttons and analogue sticks. For so many who count games as a frequent pastime, it’s a seamless connection. We have come to embody so many beloved characters from Link to Lara Croft that their stories can become warm and familiar. But there’s something about the jarring experiences, those that shine a light on the very skin we wear as we walk through the halls of Rapture in BioShock, through the ruins of Dubai in Spec Ops: The Line. Those that question who we are in this virtual world and the actions we take.
Observation takes the opportunity to do this by having us embody the space station AI of S.A.M. Asked simple diagnostic questions and given simple instructions to fix technological problems, the player in this role is primarily looking after the wellbeing of the human characters. Putting the player in the role of an AI when there are other characters with seemingly more agency in the situation stands in contrast to the idea of the player as an all-powerful being to instigate change. The position of protagonist is not open, but side character has a few vacancies.
What results is this hybrid being of player and S.A.M trying to understand what’s going on together. This relationship between S.A.M and the player was important for No Code. “The moment S.A.M becomes ‘conscious’ is when the player takes over,” Jon tells us. “The player is providing that consciousness, they are providing the human flaws, the trust (or lack of), and the curiosity that normally would be absent from the AI.” The combination of the player and the character of S.A.M creates a unique perspective with the player contributing their own moral compass to the events on board the station.
S.A.M’s own identity is illuminated when we delve into the inspirations behind the technology of the station: “When we started researching the technology used on the ISS, we quickly discovered that it spans its 20-year life span. Some technology from its original setup is still reliable and still used and it’s 20 years old. They have a mix of old DV style cameras and brand new 4K cameras, because everything you want to put up there, costs a lot of money. If it works, you don’t want to replace it.” McKellan explains.
This Frankenstein-like approach to technology aboard the ISS inspired No Code to create an environment which constantly brings up new challenges for the player. As the station’s AI, the player must monitor different cameras and connect to other pieces of technology, bringing them online with various point-and-click puzzles. Through such an eclectic mix of technologies, the player is challenged to understand the simulated processes a piece of AI goes through.
Later on in the story, Emma and S.A.M discover that the station has been moved to orbiting around Saturn with a strange, hexagonal storm brewing below. More is afoot than the machinations of a malfunctioning AI. An alien presence is aboard the ship and has been communicating with S.A.M and guiding their… hand. This presence manifests as an ominous, floating hexagon (the very best of shapes, if Terry Cavanagh is to be believed).
As this force lures Emma and S.A.M to the surface of the planet, its intentions are discovered in the final scene: to combine human and machine into one entity. Where previously in the game there would have been a divide between how organic matter and synthetic systems interact, there is only harmony now: horrific, Lovecraftian harmony, but that’s beside the point.
The final scene is complex in terms of what it tries to achieve with the union of human and machine. The blockade that we might feel when trying to interact through S.A.M is purely through our own limitations as humans through the controller. The paradox is that this is communicated through a non-interactive cutscene, serving to jettison the agency of the player. This suggests that the player was never embodying the AI at all; they were the guiding hand of the alien force. Using a narrative device to shepherd the player through certain story beats, the player-character as alien force uses any means necessary to deliver its quarry to the surface of Saturn.
We might consider AI as pre-programmed, cold machines, inherently functional and unemotional. Jon and the team used the technological mess of the space station to create an untrustworthy source in S.A.M.: “S.A.M. is an unreliable narrator, and we see it all through his/its eyes, and he’s not himself. To emphasize that point, we weighed closer to the ‘imperfect technology’ side of the spectrum.” The player is required to make up for these gaps in S.A.M.’s knowledge and become part of the AI themselves.
Observation crafts a story about what it means to be AI that also helps us understand our own limitations as humans. Embodying alien perspectives is something that researcher Alan Hook has been investigating through inter-species design. “My current research explores different methods for designing for nonhuman animals,” he explains, “I’m exploring how designing as a process can help us empathise with other species.” To begin designing games for other animals, Alan stars by questioning every assumption he might make as a designer. And this line of questioning is the sort of design approach that helps us understand what it means to be AI.
Horses have a larger field of view and cats see at a higher frequency. When designers take consideration of the subject into account it reflects back upon the humanistic society we live in. Alan explains: “By designing for other species we open up a lot of the assumptions we make in game design, and we start from scratch. To work with other species opens up the process, and allows us fresh eyes as designers.”
Play is an empathetic activity, but Observation goes further. We are forced to confront the real limitations of what it means to be human. We can readily imagine any computer system extracting information and delivering it at lightning speed, but we, as an AI, are limited by our human processing speed. We can only look through one camera at once instead of multiple as some omnipotent network.
The game shines a light on what it means to be an AI by pointing out our own inadequacies. That we play through the game to get to the point of assimilation is crucial, argues Alan: “Play is really important as it can be a process of becoming with other species and allows us to work across species boundaries.”
Good sci-fi challenges our understanding of the human condition by placing it in alien circumstances. Observation makes terrific use of the video game format by placing the player within the trope of the untrustworthy AI that we see in so much of science fiction. But it goes further and twists what we think of as robotic actions to argue that, in this virtual world, we are the aliens.
Great sci-fi understands the form to lean on its strengths and deliver a unique perspective. Observation does so many interesting things that it’s difficult to nail them down. BioShock was so groundbreaking at the time because it hit the player over the head with their own lack of agency. Observation builds on the foundations of so many great titles over the last 12 years. It shifts the ground beneath our feet as we struggle to understand who we are in the virtual world.