The Letter - a double empathy game

thumbnail for this post

The Letter is:

A dialogue-puzzle RPG that combines strategic card-based encounters with meaningful narrative choices, allowing players to act as mediators in complex communication scenarios.

Its underlying system implementation is an LLM-assisted language deconstruction and reconstruction mechanism that we have dubbed the “universal translator and transformer”.

The theme of the game is: facing the double empathy problem in a complex world.

Design challenge:

Improve the real-time universal translator and transformer prototype for individuals and organizations to deconstruct texts into neutral core meanings and reconstruct them in various culturally and emotionally diverse representations with enhanced understanding, reduced polarization, and improved communication across different perspectives.

HBO-i domains: Software Engineering, User Interaction. IT-Areas: Game Design, Artificial Intelligence, Media Design, Smart Mobile. And combinations.

Current Gameplay Concept

The Letter is a dialogue-puzzle RPG where players become linguistic diplomats mediating through campaigns filled with encounters that challenge their ability to interpret and convey critical information across NPCs. Players uncover and relay core intents, resolving communication barriers (double empathy problem) by interpreting language through a set of nuanced modalities.

Core Loop

In The Letter, players navigate two primary encounter types that challenge their linguistic intuition:

Deconstruction Encounters: Players encounter an NPC struggling to express their needs effectively. Using a card-based deconstruction system, the player removes unnecessary modalities to uncover the message’s core meaning.

Reconstruction Encounters: With the core meaning established, players tailor the message for a second NPC using submodality cards collected in their catalogue, constructing a message that aligns with the recipient’s communication style and needs.

Each modality and submodality is represented by a card, which players accumulate in a catalogue. Players can apply these cards in deconstruction and reconstruction phases, tailoring messages to NPCs and building their mediation skills.

Targeted Experience

The Letter provides a DISCOVERY-driven experience, with FANTASY and SUBMISSION elements to immerse players in a linguistically complex world.

Discovery: Explore the elasticity of how people communicate. Uncover new submodalities, adding to the player’s catalogue, and reveal deeper communication layers.

Fantasy: Players embody a linguistic diplomat, immersing themselves in resolving conflicts through language.

Submission: The card-based system offers strategic choices within an accessible interface, allowing players to explore language meaningfully as a rote activity without heavy cognitive demands or time constraints.

In game aesthetics, it draws inspiration from games such as Strange Horticulture.

The Assignment

Problem

In today’s interconnected world, communication is not just about sharing information but about bridging gaps in understanding, especially across diverse perspectives. One societal challenge that the universal translator and transformer game prototype seeks to address is the double empathy problem, a concept that highlights how misunderstandings often arise from differences in lived experiences and communication styles between individuals or groups.

The double empathy problem suggests that mutual misunderstandings occur when two parties, especially those from different neurotypes, cultures, or social groups, struggle to comprehend each other’s perspectives because their communication frameworks are inherently different.

The Universal Translator aims to mitigate these misunderstandings by deconstructing texts into neutral core meanings and reconstructing them in forms that resonate with diverse perspectives. However, achieving this requires not just technical innovation but also exploring how these tools are applied in real-life scenarios to build empathy and foster meaningful dialogue.

We have both a playable game prototype and a technical system that can take a text and peel off layer by layer different cultural nuances, world views, perspectives and emotional sentiments until only a core neutral meaningful text remains. The system uses both the power of Large Language Models (LLMs) to deconstruct the text, as the mathematical power behind text embeddings, which allows us to transform the text while keeping its meaning as much the same.

From the core neutral meaningful nugget, we can again build up, layer by layer, different text representations that convey the same meaning but in completely different representations.

We can use such a universal translator or transformer of meaning, to investigate how people from different perspectives or with seemingly completely different opinions can find common ground in what is essential. Or, how how to explore how you can repackage your message to be accepted or understood by a person with a different cultural, social, or ideological background.

This is a very relevant topic in a world where information is twisted into false narratives and information is represented in ways to increase polarization between people instead of bridging the gap.

Assignment

Depending on the background of the student and the state of the existing prototypes, we see a mix of the following opportunities:

  • improving AI implementation of the the text de/re-construction mechanism
  • designing, implementing, and testing encounters and a campaign for this game
  • further improving the interface and supporting game mechanics
  • port the game prototype from Unity3D to Godot
  • create a sandbox version of the game (explore your own text input)

Guidance

The DPBTSE (The Dead Philosophers Brainstorm To Solve Everything) is part of the research group Interaction Design (IXD) of Fontys ICT. The DPBTSE is a combination of a thinktank and a creative lab for the research group interested in working with the combination of applied:

  • Sciences,
  • Arts (and design),
  • Creativity and Philosophy, to create innovative interactive concepts, products, services and experiments therefore.

The project is open ended, meaning we will adapt and respond to interesting intermediate results more than that we have a set target. We need adventurous students that are willing to get engaged with the researchers, leading to a potentially very rewarding internship.

Your contacts are:

  • Olaf Janssen, PhD in computational physics, experience in mobile app development, web development and design, AI and LLM, game design and development.
  • John van Litsenburg, teacher and developer of UXD / IXD strategy, concepts, product and services development and 2d / 3d design and of societal impact of artificial intelligence. Work and worked for Media design, Smart Mobile, Artificial intelligence.

Feel free to contact with any questions, input or ideas about the internship assignment.