Universal Translator and Transformer

thumbnail for this post

Design challenge:

Improve the real-time universal text translator and transformer for individuals and organizations to deconstruct texts into neutral core meanings and reconstruct them in various culturally and emotionally diverse representations with enhanced understanding, reduced polarization, and improved communication across different perspectives.

The challenge can be approached from different perspectives and can be adjusted to the IT Area of interest of the student with an interest in:

HBO-i domains: Software Engineering, User Interaction
IT-Areas: AI engineering, web development (front-end/back-end), UX design

Context

The research group Interaction Design (IXD) is interested in the shape, matter, and power of our relationship with information. In particular, we see how humans struggle in the face of information overload:

  • Quantity of information (too much or not enough),
  • Quality of information (information density, ease of use of info),
  • Complexity or Simplicity of information (too much or too little).

This struggle is not new (Kierkegaard, 1840), but the technologies surrounding us are. Do those new technologies and the products built around them help us deal with information overload or do they only make things worse?

We (hereby) pose strategic design experiments for ideas, concepts, products, services, and environments with regards to new ways of information moderation, transformation, curation, and publishing by means of (generative) AI.

Part of the experiments is to critically look at the (potential) positive and negative impact of such technologies and ideas on democracy, healthcare, education, journalism, arts and cultural exchange, museums, publishing, built environments, and crossovers.

This critical view is essential because it is dubious whether new services such as generative AI are really making us more creative, more productive, smarter, happier, work better as a society.

Currently, we are pursuing projects that deal with:

  • enticing intellectual and creative pursuits
  • moderating info-overload instead of contributing
  • mediating people using virtual representatives

The Assignment

Problem

We have a proof-of-concept system that can take a text and peel off layer-by-layer different cultural nuances, world views, perspectives and emotional sentiments until only a core neutral meaningful text remains. The system uses the power of Large Language Models (LLMs) to deconstruct the text, as the mathematical power behind text embeddings, which allows us to transform the text while keeping its meaning as much the same.

From the core neutral meaningful nugget, we can again build up, layer by layer, different text representations that convey the same meaning but in completely different representations.

We can use such a universal translator or transformer of meaning, to investigate how people from different perspectives or with seemingly completely different opinions can find common ground in what is essential. Or, how how to explore how you can repackage your message to be accepted or understood by a person with a different cultural, social, or ideological background.

This is a very relevant topic in a world where information is twisted into false narratives and information is represented in ways to increase polarization between people instead of bridging the gap.

Assignment

Depending on the background of the student, the AI-assisted system behind the universal translator tool can be improved or the student can work on a novel interface design for such a real-time universal translator and transformer.

Several challenges to work on:

  • improving the text deconstruction mechanism (see appendix for current layers)

  • turning the proof-of-concept into a professional context of vector and graph databases

  • a system to apply the translator into mediating different opinions

  • creating an appealing and intuitive interface to transform the text real-time

Research opportunities

  • How to set up the vector and graph databases for storing and analysing the huge text transformations

  • Investigate techniques to use the (mathematical) relations of text embeddings to bridge the gap between people with different perspectives

  • Exploring and prototyping new prompting techniques to improve the text de- and re-construction generations using cloud or local LLMs

  • Study user interaction patterns with synthesized content and their contributions to the refinement process.

  • As with all projects, we invite you to think critically of the possible impact of anything you create.

Expected outcome

Depending on the experience and interest of the student, we envision:

  • a more professionally designed platform for the universal translator prototype

  • a novel and intuitive interface design for the universal translator

  • prototypes or experiments to apply the tool to counter hostility and polarization between people, while maintaining a healthy discourse between different opinions

Guidance

The DPBTSE (The Dead Philosophers Brainstorm To Solve Everything) is part of the research group Interaction Design (IXD) of Fontys ICT. The DPBTSE is a combination of a thinktank and a creative lab for the research group interested in working with the combination of applied:

  • Sciences,
  • Arts (and design),
  • Creativity and Philosophy, to create innovative interactive concepts, products, services and experiments.

The project is open ended, meaning we will adapt and respond to interesting intermediate results more than setting a clear target. We need adventurous students that are willing to get engaged with the available researchers, leading to a potentially very rewarding internship.

Your contacts are:

  • Olaf Janssen, PhD in computational physics, experience in mobile app development, web develoment and design, AI and LLMs, game design and development.

  • John van Litsenburg, teacher and developer of UXD / IXD strategy, concepts, product and services development and 2d / 3d design and of societal impact of artificial intelligence. Work and worked for Media design, Smart Mobile, Artificial intelligence.

Feel free to contact with any question about the internship assignment.

Appendix

Current language deconstruction stack, or modality stack (subject to change):

Core 

  1. Nugget Core Meaning: The fundamental, context-independent meaning of the text. This is the most abstract representation of an idea or fact, stripped of any emotional, cultural, or stylistic influences. 

  2. Lexical Semantics: The dictionary meanings of words and phrases and their relationships (e.g., synonyms, antonyms). This layer adds the first level of specificity to the core meaning. 

  3. Syntactic Structures: Grammar and sentence structure that organize words and phrases into coherent statements. This modality includes understanding parts of speech and their arrangements. 

These first three layers are so elementary linked to the medium text that together we can represent this as text and preserve the most neutral specific core meaning. We could also represent it in a knowledge graph. For this project, we consider layer 3 as the lowest level of our stack. 

Neutral Linguistic Intent and Quality 

  1. Pragmatic Context: The intended use of language in situational contexts, including speech acts (e.g., requests, offers, commands) and implicatures, which require an understanding of the speaker’s intentions and the conversational context. 

  2. Referential Context: The specific entities, locations, times, and real-world references mentioned in the text. This layer anchors abstract meanings to concrete instances. 

  3. Discourse Coherence: The logical flow and connectivity of ideas across sentences and paragraphs, ensuring that the text forms a coherent whole rather than disconnected fragments. 

The following three layers add a neutral context to the core message placing it in a particular application context and whether the message has a coherent application at all. 

Social Code 

  1. Sentiment and Emotional Tone: The emotional layer of the text, which includes sentiments (positive, negative, neutral) and more specific emotional states (joy, anger, sadness). 

  2. Cultural References and Symbolism: Implicit and explicit references that require cultural knowledge to decode, including idioms, proverbs, cultural symbols, and allusions. 

  3. Sociolinguistic Variations: Variations in language use influenced by social factors, including dialects, sociolects, and registers. This modality reflects the social identity and status of the speaker or writer. 

The next three levels add meaning that resonate in a specific social and cultural context. 

Personal identity 

  1. Stylistic Features: The choice of words, tone, and rhetorical devices that reflect the author’s personal style, genre conventions, or the text’s intended effect on the reader. 

  2. Intertextuality: References to and influences from other texts, which require knowledge of those texts to understand fully. 

  3. Philosophical and Ideological Stances: The underlying beliefs, worldviews, and ideologies that shape the perspective from which the text is written. 

  4. Temporal and Spatial Contexts: The historical time, geographical place, and cultural setting in which the text was produced and is interpreted. 

  5. Psychological and Personality Traits: Indications of the psychological state or personality traits of the speaker or writer, as inferred from language use patterns.